You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: src/ServiceContracts/WebContract.php
+6Lines changed: 6 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -213,6 +213,7 @@ public function search(
213
213
* @param int $maxDepth Maximum link depth from the starting URL (0 = only the starting page)
214
214
* @param int $maxPages Maximum number of pages to crawl. Hard cap: 500.
215
215
* @param \ContextDev\Web\WebWebCrawlMdParams\Pdf|PdfShape1 $pdf PDF parsing controls. Use start/end to limit text extraction and OCR to an inclusive 1-based page range.
216
+
* @param bool $settleAnimations When true, waits briefly for CSS and transition animations to settle before extracting each crawled page. Defaults to false. This adds a bit of latency in exchange for more stable output on animated pages.
216
217
* @param bool $shortenBase64Images Truncate base64-encoded image data in the Markdown output
217
218
* @param int $stopAfterMs Soft time budget for the crawl in milliseconds. After each scrape, the crawler checks the elapsed time and, if exceeded, returns the pages collected so far instead of continuing. Min: 10000 (10s). Max: 110000 (110s). Default: 80000 (80s).
218
219
* @param int $timeoutMs Optional timeout in milliseconds for the request. If the request takes longer than this value, it will be aborted with a 408 status code. Maximum allowed value is 300000ms (5 minutes).
* @param list<string> $includeSelectors CSS selectors. When provided, only matching subtrees (and their descendants) are kept and everything else is dropped. When omitted, the entire document is kept. Examples: "article.main", "#content", "[role=main]".
259
261
* @param int $maxAgeMs Return a cached result if a prior scrape for the same parameters exists and is younger than this many milliseconds. Defaults to 1 day (86400000 ms) when omitted. Max is 30 days (2592000000 ms). Set to 0 to always scrape fresh.
260
262
* @param \ContextDev\Web\WebWebScrapeHTMLParams\Pdf|PdfShape2 $pdf PDF parsing controls. Use start/end to limit text extraction and OCR to an inclusive 1-based page range.
263
+
* @param bool $settleAnimations When true, waits briefly for CSS and transition animations to settle before extracting HTML. Defaults to false. This adds a bit of latency in exchange for more stable output on animated pages.
261
264
* @param int $timeoutMs Optional timeout in milliseconds for the request. If the request takes longer than this value, it will be aborted with a 408 status code. Maximum allowed value is 300000ms (5 minutes).
262
265
* @param bool $useMainContentOnly when true, return only the page's main content in the HTML response, excluding headers, footers, sidebars, and navigation when detectable
263
266
* @param int $waitForMs Optional browser wait time in milliseconds after initial page load. Min: 0. Max: 30000 (30 seconds).
@@ -276,6 +279,7 @@ public function webScrapeHTML(
@@ -318,6 +322,7 @@ public function webScrapeImages(
318
322
* @param list<string> $includeSelectors CSS selectors. When provided, only matching HTML subtrees (and their descendants) are kept before conversion to Markdown. When omitted, the entire document is kept. Examples: "article.main", "#content", "[role=main]".
319
323
* @param int $maxAgeMs Return a cached result if a prior scrape for the same parameters exists and is younger than this many milliseconds. Defaults to 1 day (86400000 ms) when omitted. Max is 30 days (2592000000 ms). Set to 0 to always scrape fresh.
320
324
* @param \ContextDev\Web\WebWebScrapeMdParams\Pdf|PdfShape3 $pdf PDF parsing controls. Use start/end to limit text extraction and OCR to an inclusive 1-based page range.
325
+
* @param bool $settleAnimations When true, waits briefly for CSS and transition animations to settle before converting to Markdown. Defaults to false. This adds a bit of latency in exchange for more stable output on animated pages.
321
326
* @param bool $shortenBase64Images Shorten base64-encoded image data in the Markdown output
322
327
* @param int $timeoutMs Optional timeout in milliseconds for the request. If the request takes longer than this value, it will be aborted with a 408 status code. Maximum allowed value is 300000ms (5 minutes).
323
328
* @param bool $useMainContentOnly Extract only the main content of the page, excluding headers, footers, sidebars, and navigation
Copy file name to clipboardExpand all lines: src/Services/WebService.php
+9Lines changed: 9 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -349,6 +349,7 @@ public function search(
349
349
* @param int $maxDepth Maximum link depth from the starting URL (0 = only the starting page)
350
350
* @param int $maxPages Maximum number of pages to crawl. Hard cap: 500.
351
351
* @param \ContextDev\Web\WebWebCrawlMdParams\Pdf|PdfShape1 $pdf PDF parsing controls. Use start/end to limit text extraction and OCR to an inclusive 1-based page range.
352
+
* @param bool $settleAnimations When true, waits briefly for CSS and transition animations to settle before extracting each crawled page. Defaults to false. This adds a bit of latency in exchange for more stable output on animated pages.
352
353
* @param bool $shortenBase64Images Truncate base64-encoded image data in the Markdown output
353
354
* @param int $stopAfterMs Soft time budget for the crawl in milliseconds. After each scrape, the crawler checks the elapsed time and, if exceeded, returns the pages collected so far instead of continuing. Min: 10000 (10s). Max: 110000 (110s). Default: 80000 (80s).
354
355
* @param int $timeoutMs Optional timeout in milliseconds for the request. If the request takes longer than this value, it will be aborted with a 408 status code. Maximum allowed value is 300000ms (5 minutes).
* @param list<string> $includeSelectors CSS selectors. When provided, only matching subtrees (and their descendants) are kept and everything else is dropped. When omitted, the entire document is kept. Examples: "article.main", "#content", "[role=main]".
425
428
* @param int $maxAgeMs Return a cached result if a prior scrape for the same parameters exists and is younger than this many milliseconds. Defaults to 1 day (86400000 ms) when omitted. Max is 30 days (2592000000 ms). Set to 0 to always scrape fresh.
426
429
* @param \ContextDev\Web\WebWebScrapeHTMLParams\Pdf|PdfShape2 $pdf PDF parsing controls. Use start/end to limit text extraction and OCR to an inclusive 1-based page range.
430
+
* @param bool $settleAnimations When true, waits briefly for CSS and transition animations to settle before extracting HTML. Defaults to false. This adds a bit of latency in exchange for more stable output on animated pages.
427
431
* @param int $timeoutMs Optional timeout in milliseconds for the request. If the request takes longer than this value, it will be aborted with a 408 status code. Maximum allowed value is 300000ms (5 minutes).
428
432
* @param bool $useMainContentOnly when true, return only the page's main content in the HTML response, excluding headers, footers, sidebars, and navigation when detectable
429
433
* @param int $waitForMs Optional browser wait time in milliseconds after initial page load. Min: 0. Max: 30000 (30 seconds).
@@ -442,6 +446,7 @@ public function webScrapeHTML(
@@ -457,6 +462,7 @@ public function webScrapeHTML(
457
462
'includeSelectors' => $includeSelectors,
458
463
'maxAgeMs' => $maxAgeMs,
459
464
'pdf' => $pdf,
465
+
'settleAnimations' => $settleAnimations,
460
466
'timeoutMs' => $timeoutMs,
461
467
'useMainContentOnly' => $useMainContentOnly,
462
468
'waitForMs' => $waitForMs,
@@ -525,6 +531,7 @@ public function webScrapeImages(
525
531
* @param list<string> $includeSelectors CSS selectors. When provided, only matching HTML subtrees (and their descendants) are kept before conversion to Markdown. When omitted, the entire document is kept. Examples: "article.main", "#content", "[role=main]".
526
532
* @param int $maxAgeMs Return a cached result if a prior scrape for the same parameters exists and is younger than this many milliseconds. Defaults to 1 day (86400000 ms) when omitted. Max is 30 days (2592000000 ms). Set to 0 to always scrape fresh.
527
533
* @param \ContextDev\Web\WebWebScrapeMdParams\Pdf|PdfShape3 $pdf PDF parsing controls. Use start/end to limit text extraction and OCR to an inclusive 1-based page range.
534
+
* @param bool $settleAnimations When true, waits briefly for CSS and transition animations to settle before converting to Markdown. Defaults to false. This adds a bit of latency in exchange for more stable output on animated pages.
528
535
* @param bool $shortenBase64Images Shorten base64-encoded image data in the Markdown output
529
536
* @param int $timeoutMs Optional timeout in milliseconds for the request. If the request takes longer than this value, it will be aborted with a 408 status code. Maximum allowed value is 300000ms (5 minutes).
530
537
* @param bool $useMainContentOnly Extract only the main content of the page, excluding headers, footers, sidebars, and navigation
Copy file name to clipboardExpand all lines: src/Web/WebWebCrawlMdParams.php
+20Lines changed: 20 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -32,6 +32,7 @@
32
32
* maxDepth?: int|null,
33
33
* maxPages?: int|null,
34
34
* pdf?: null|Pdf|PdfShape,
35
+
* settleAnimations?: bool|null,
35
36
* shortenBase64Images?: bool|null,
36
37
* stopAfterMs?: int|null,
37
38
* timeoutMs?: int|null,
@@ -124,6 +125,12 @@ final class WebWebCrawlMdParams implements BaseModel
124
125
#[Optional]
125
126
public ?Pdf$pdf;
126
127
128
+
/**
129
+
* When true, waits briefly for CSS and transition animations to settle before extracting each crawled page. Defaults to false. This adds a bit of latency in exchange for more stable output on animated pages.
130
+
*/
131
+
#[Optional]
132
+
public ?bool$settleAnimations;
133
+
127
134
/**
128
135
* Truncate base64-encoded image data in the Markdown output.
@@ -374,6 +383,17 @@ public function withPdf(Pdf|array $pdf): self
374
383
return$self;
375
384
}
376
385
386
+
/**
387
+
* When true, waits briefly for CSS and transition animations to settle before extracting each crawled page. Defaults to false. This adds a bit of latency in exchange for more stable output on animated pages.
Copy file name to clipboardExpand all lines: src/Web/WebWebScrapeHTMLParams.php
+20Lines changed: 20 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -28,6 +28,7 @@
28
28
* includeSelectors?: list<string>|null,
29
29
* maxAgeMs?: int|null,
30
30
* pdf?: null|Pdf|PdfShape,
31
+
* settleAnimations?: bool|null,
31
32
* timeoutMs?: int|null,
32
33
* useMainContentOnly?: bool|null,
33
34
* waitForMs?: int|null,
@@ -95,6 +96,12 @@ final class WebWebScrapeHTMLParams implements BaseModel
95
96
#[Optional]
96
97
public ?Pdf$pdf;
97
98
99
+
/**
100
+
* When true, waits briefly for CSS and transition animations to settle before extracting HTML. Defaults to false. This adds a bit of latency in exchange for more stable output on animated pages.
101
+
*/
102
+
#[Optional]
103
+
public ?bool$settleAnimations;
104
+
98
105
/**
99
106
* Optional timeout in milliseconds for the request. If the request takes longer than this value, it will be aborted with a 408 status code. Maximum allowed value is 300000ms (5 minutes).
@@ -272,6 +281,17 @@ public function withPdf(Pdf|array $pdf): self
272
281
return$self;
273
282
}
274
283
284
+
/**
285
+
* When true, waits briefly for CSS and transition animations to settle before extracting HTML. Defaults to false. This adds a bit of latency in exchange for more stable output on animated pages.
* Optional timeout in milliseconds for the request. If the request takes longer than this value, it will be aborted with a 408 status code. Maximum allowed value is 300000ms (5 minutes).
Copy file name to clipboardExpand all lines: src/Web/WebWebScrapeMdParams.php
+20Lines changed: 20 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -30,6 +30,7 @@
30
30
* includeSelectors?: list<string>|null,
31
31
* maxAgeMs?: int|null,
32
32
* pdf?: null|Pdf|PdfShape,
33
+
* settleAnimations?: bool|null,
33
34
* shortenBase64Images?: bool|null,
34
35
* timeoutMs?: int|null,
35
36
* useMainContentOnly?: bool|null,
@@ -110,6 +111,12 @@ final class WebWebScrapeMdParams implements BaseModel
110
111
#[Optional]
111
112
public ?Pdf$pdf;
112
113
114
+
/**
115
+
* When true, waits briefly for CSS and transition animations to settle before converting to Markdown. Defaults to false. This adds a bit of latency in exchange for more stable output on animated pages.
116
+
*/
117
+
#[Optional]
118
+
public ?bool$settleAnimations;
119
+
113
120
/**
114
121
* Shorten base64-encoded image data in the Markdown output.
@@ -321,6 +330,17 @@ public function withPdf(Pdf|array $pdf): self
321
330
return$self;
322
331
}
323
332
333
+
/**
334
+
* When true, waits briefly for CSS and transition animations to settle before converting to Markdown. Defaults to false. This adds a bit of latency in exchange for more stable output on animated pages.
0 commit comments