Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 7 additions & 11 deletions src/content/docs/ruby-gem/how-to/handling-dynamic-content.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,7 @@ request:
max_requests: 6
browserless:
preload:
wait_for_network_idle:
timeout_ms: 5000
wait_after_ms: 5000
channel:
url: https://example.com/app
selectors:
Expand Down Expand Up @@ -49,15 +48,14 @@ The `browserless` strategy is necessary when:
For dynamic sites, rendering once is often not enough. Use `request.browserless.preload` to wait, click, or scroll before the
HTML snapshot is taken.

### Wait for JavaScript Requests
### Wait Before Capturing Dynamic Content

```yaml
strategy: browserless
request:
browserless:
preload:
wait_for_network_idle:
timeout_ms: 4000
wait_after_ms: 4000
```

### Click "Load More" Buttons
Expand All @@ -67,12 +65,11 @@ strategy: browserless
request:
browserless:
preload:
wait_after_ms: 3000
click_selectors:
- selector: ".load-more"
max_clicks: 3
delay_ms: 250
wait_for_network_idle:
timeout_ms: 3000
wait_after_ms: 250
```

### Scroll Infinite Lists
Expand All @@ -84,9 +81,8 @@ request:
preload:
scroll_down:
iterations: 5
delay_ms: 200
wait_for_network_idle:
timeout_ms: 2500
wait_after_ms: 200
wait_after_ms: 2500
```

These preload steps can be combined in a single config when a site needs several interactions before all items appear.
Expand Down
16 changes: 5 additions & 11 deletions src/content/docs/ruby-gem/reference/strategy.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -69,8 +69,7 @@ request:
max_requests: 6
browserless:
preload:
wait_for_network_idle:
timeout_ms: 5000
wait_after_ms: 5000
channel:
url: "https://example.com/app"
selectors:
Expand All @@ -93,22 +92,17 @@ strategy: browserless
request:
browserless:
preload:
wait_for_network_idle:
timeout_ms: 5000
wait_after_ms: 5000
click_selectors:
- selector: ".load-more"
max_clicks: 3
delay_ms: 250
wait_for_network_idle:
timeout_ms: 4000
wait_after_ms: 250
scroll_down:
iterations: 5
delay_ms: 200
wait_for_network_idle:
timeout_ms: 3000
wait_after_ms: 200
```

- `wait_for_network_idle`: pauses before and after preload steps
- `wait_after_ms`: inserts a fixed wait before or after preload steps
- `click_selectors`: clicks matching elements until they disappear or `max_clicks` is reached
- `scroll_down`: scrolls until the page height stops growing or `iterations` is reached

Expand Down
Loading