fix: fall back to fetch when prerender returns 4xx - #276
Conversation
page.content() treats WAF/challenge HTML as success and cancelled a parallel 2xx fetch. Co-authored-by: Cursor <cursoragent@cursor.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
Included review availability: Your plan includes up to 2 reviews per rolling hour; 1 remains after this review. 📝 WalkthroughWalkthroughThe prerender flow now checks error responses before canceling the fetch request. It returns a successful fetch response when available and preserves the prerender response when the fetch also fails. Tests cover both outcomes. ChangesPrerender fallback handling
Estimated code review effort: 2 (Simple) | ~10 minutes Merge Risk: ⚪ Minimal · up to This localized change makes prerendered 4xx responses fall back to a successful fetch while preserving the prerender result when both responses fail; no actionable merge-blocking risk remains after normal checks and review. 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
* fix: keep fetch when shadow-DOM prerender retry is 4xx #276 only cancelled a parallel fetch when prerender itself was usable. The auto-mode shadow-DOM upgrade still treated WAF/challenge HTML from page.content() as success and replaced a working 2xx fetch. Co-authored-by: kikohumanbeatbox <kikohumanbeatbox@gmail.com> * test: share shadow-DOM HTML fixture Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: kikohumanbeatbox <kikohumanbeatbox@gmail.com> Co-authored-by: Kiko Beats <josefrancisco.verdu@gmail.com>
#276 returns the parallel fetch (mode: fetch, 2xx) when page.content() is a 4xx challenge. The #277 upgrade guard only checked status, so that later fetch replaced the already-good first snapshot — including a 200 soft-block or interstitial body. Only accept a successful prerender result for the auto-mode upgrade. Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Bug and impact
prerender()startsgotand Chrome in parallel, then treats anypage.content()as success and cancels fetch. A CloudFront/WAF 403 document is still a successfulpage.content(), so a parallel 200 fetch is dropped. That is howhttps://www.un.org/en/about-uskept the challenge page whileprerender=falsereturned the real HTML.Root cause
Chrome success is defined as “got a document”, not “got a usable status”. Fetch is only used when Chrome throws.
Fix
If prerender
statusCode >= 400, wait for fetch. Use fetch when it is a 2xx/3xx. Otherwise keep the prerender document (both blocked, or fetch rejected).Validation
prerender 4xx falls back to a successful fetch— mocked Chrome 403 + local 200 →mode: fetch,statusCode: 200prerender 4xx is kept when fetch is also unsuccessful— mocked Chrome 403 + local 403 → keep prerender 403pnpm exec ava test/mode.jspassedMade with Cursor
Summary by CodeRabbit
Bug Fixes
Tests