Conversation
Detect sandbox attributes on iframes and emit warnings when sandbox restrictions may affect rendering fidelity in Percy. Warns for fully sandboxed iframes, missing allow-scripts, and missing allow-same-origin. [PER-7292] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…) CSS, interactive states, fidelity warnings
- Add preflight.js monkey-patch for attachShadow/attachInternals interception
- Inject preflight via CDP addScriptToEvaluateOnNewDocument in CLI browser
- Support closed shadow roots in prepare-dom.js, clone-dom.js, serialize-dom.js via WeakMap
- Rewrite :state() and legacy :--state CSS selectors to attribute selectors for ElementInternals
- Add fallback state detection via element.matches(':state(X)') for SDK path
- Capture :focus (before clone), :checked, :disabled states automatically on all elements
- Extract differential CSS rules for :focus/:checked/:disabled/:hover/:active via CSSOM
- Add shadow DOM traversal for interactive state detection
- Add iframe and shadow root fidelity warnings
- Guard all custom elements with closed shadow roots during cloning to prevent constructor conflicts
- Add regression test fixture for DOM structures coverage
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… wait, and coverage fixes - Add data-percy-ignore attribute to exclude specific iframes from capture - Add ignoreIframeSelectors config option for CSS selector-based iframe exclusion - Wait for customElements.whenDefined() with 5s timeout before snapshot capture - Fix eslint quotes violations in sandbox iframe tests - Add 63 new tests covering iframe ignore, custom element wait, interactive state CSS extraction, custom state detection, and edge cases - Refactor serialize-pseudo-classes for testability: generator to array fn, extracted safeMatchesState helper, simplified guards - Remove dead 'unknown' fallback in iframe label - Add dom-structures.html regression test page - 315 tests passing, 100% coverage (statements, branches, functions, lines) [PER-7292] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Prefix Element and HTMLElement with window. to satisfy eslint no-undef rule in browser-injected script context. [PER-7292] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…lectors
- Replace querySelectorAll(':focus') with document.activeElement for focus
detection in markInteractiveStatesInRoot — more reliable in headless browsers
- Refactor focus tests to mock document.activeElement instead of calling .focus()
which doesn't work in Firefox headless
- Use :checked instead of :focus in CSS extraction tests for cross-browser compat
- Add ignoreIframeSelectors to @percy/core percy.test.js default config and
eval spy expectations
[PER-7292]
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…es and shadow roots - Capture bounding rect (getBoundingClientRect) of excluded iframes BEFORE removing them from the clone DOM, storing as fidelityRegions[] in domSnapshot - Detect custom elements with potentially inaccessible shadow roots and capture their bounding rects - Update shadow root fidelity warning to include totals: "N shadow root(s): X captured, Y potentially inaccessible" - Return fidelityRegions array in serializeDOM result alongside html, warnings, resources, hints - Log [fidelity] warnings in CLI verbose output via discovery.js - Screenshot remains identical to user's page (iframes still removed) - Fidelity regions will be used by API + Web to render overlay indicators at the original positions of uncaptured content [PER-7292] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add fidelityRegions to client.js createSnapshot POST payload as 'fidelity-regions' attribute - Extract fidelityRegions from domSnapshot in discovery.js and pass through to snapshot upload - Update client test payload assertions to include fidelity-regions [PER-7292] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…d upload tests [PER-7292] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
[PER-7292] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…granularity The funcVariableTime assertions used a 10 ms tolerance on a 10 ms sleep, which is below the Windows default timer tick (~15.6 ms). Observed CI failures showed drifts of 11, 15, and 16 ms — exactly the range you'd expect from a Windows tick miss. The same test file's funcReturns assertions already use a 15 ms tolerance with a 'adding buffer for win test' comment; the variable-time block was overlooked. Bump to 20 ms (one full tick of headroom over the existing 15 ms pattern) and document why. Eliminates the ~11 retry events observed across PRs #2138, #2147, #2172, #2177 on Test @percy/webdriver-utils Windows jobs.
The 'stops waiting on process termination' test used a fixed 100 ms
setTimeout before emitting SIGTERM, racing the first poll's log emit on
slow Windows runners. Two coupled failure modes were observed:
A. The 'Processing 18 snapshots - 0 of 72 comparisons finished...' log
hadn't landed yet, so logger.stdout was empty and the assertion got
`Expected $.length = 0 to equal 1`.
B. The same log arrived a few ms later and bled into the *next* test's
logger snapshot ('failure messages > logs an error when there are
no snapshots'), tripping `Expected $.length = 1 to equal 0` there.
Replace the fixed wait with a poll-until-logged loop bounded by a 5s
deadline. Once the expected log line is observed, SIGTERM and assertion
proceed deterministically — no race, no bleed.
Eliminates the ~6 retry events observed across PRs #2147, #2172, #2177
on Test @percy/cli-build Windows jobs.
The 'stops waiting on process termination' test used a fixed 100 ms
setTimeout before emitting SIGTERM, racing the first poll's log emit on
slow Windows runners. Two coupled failure modes were observed:
A. The 'Processing 18 snapshots - 0 of 72 comparisons finished...' log
hadn't landed yet, so logger.stdout was empty and the assertion got
`Expected $.length = 0 to equal 1`.
B. The same log arrived a few ms later and bled into the *next* test's
logger snapshot ('failure messages > logs an error when there are
no snapshots'), tripping `Expected $.length = 1 to equal 0` there.
Replace the fixed wait with a poll-until-logged loop bounded by a 5s
deadline. Once the expected log line is observed, SIGTERM and assertion
proceed deterministically — no race, no bleed.
Eliminates the ~6 retry events observed across PRs #2147, #2172, #2177
on Test @percy/cli-build Windows jobs.
* fix(dom): close srcdoc-iframe load race in serializeFrames test setup
`getFrame` used `performance.timing.loadEventEnd > 0` as the iframe-ready
signal. In Firefox, srcdoc iframes fire `load` for the placeholder
about:blank before the srcdoc content commits, so the helper could return
a frame whose `contentDocument.querySelector('input')` was still null.
Add an optional `ready(contentDocument)` predicate and pass it at the
three call sites whose next line dereferences specific content. Fixes the
intermittent `serializeFrames > plain: does not serialize iframes
without document elements` failure observed on Firefox 148/Linux in
PR #2154 CI run 23190540857.
* fix(webdriver-utils): widen TimeIt timing tolerance for Windows tick granularity
The funcVariableTime assertions used a 10 ms tolerance on a 10 ms sleep,
which is below the Windows default timer tick (~15.6 ms). Observed CI
failures showed drifts of 11, 15, and 16 ms — exactly the range you'd
expect from a Windows tick miss.
The same test file's funcReturns assertions already use a 15 ms tolerance
with a 'adding buffer for win test' comment; the variable-time block was
overlooked. Bump to 20 ms (one full tick of headroom over the existing
15 ms pattern) and document why.
Eliminates the ~11 retry events observed across PRs #2138, #2147, #2172,
#2177 on Test @percy/webdriver-utils Windows jobs.
* fix(cli-exec): wait for alternate-port server via ping before asserting
`start(['--port=4567'])` returns a Promise that resolves on process exit,
not on listen. The test fired a single healthcheck immediately and raced
the server bind — ECONNREFUSED on Windows (single-stack IPv4) and
AggregateError on dual-stack Node 18+ runners (Happy Eyeballs wraps the
underlying errors so the request client cannot retry on `.code`).
Mirror the beforeEach's proven pattern: sleep 1 s then `await ping(
['--port=4567'])`. `ping` resolves only once the server is reachable,
turning the race into a deterministic checkpoint. A single shot
healthcheck then verifies the response shape.
Eliminates the intermittent ECONNREFUSED 127.0.0.1:4567 observed on
PR #2172 run 24120410492 (Test @percy/cli-exec, Windows).
* fix(cli-build): wait for processing log before SIGTERM in wait test
The 'stops waiting on process termination' test used a fixed 100 ms
setTimeout before emitting SIGTERM, racing the first poll's log emit on
slow Windows runners. Two coupled failure modes were observed:
A. The 'Processing 18 snapshots - 0 of 72 comparisons finished...' log
hadn't landed yet, so logger.stdout was empty and the assertion got
`Expected $.length = 0 to equal 1`.
B. The same log arrived a few ms later and bled into the *next* test's
logger snapshot ('failure messages > logs an error when there are
no snapshots'), tripping `Expected $.length = 1 to equal 0` there.
Replace the fixed wait with a poll-until-logged loop bounded by a 5s
deadline. Once the expected log line is observed, SIGTERM and assertion
proceed deterministically — no race, no bleed.
Eliminates the ~6 retry events observed across PRs #2147, #2172, #2177
on Test @percy/cli-build Windows jobs.
* fix(karma): tolerate browser disconnect race on windows-latest
karma-firefox-launcher on windows-latest can finish all specs and then
trip Karma's default 2 s browserDisconnectTimeout during teardown,
producing an ERROR exit despite '✔ 736 tests completed'. Observed on
PR #2172 run 24155699650 (Test @percy/dom Windows job).
Raise the disconnect/no-activity/capture timeouts and allow a single
disconnect retry. These settings only affect runner teardown timing —
they do not silence test failures, only the post-test handshake race.
…ow roots - Traverse shadowRoot.activeElement recursively to find the deepest focused element instead of stopping at the shadow host - Track which shadow root each stylesheet originated from and inject rewritten pseudo-class CSS rules back into the correct shadow root clone instead of the document head Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Cover uncovered lines 177, 209, and 440-443 in serialize-pseudo-classes.js by testing shadow root activeElement chain traversal and CSS rule injection into shadow root clones. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…e stylesheet population Use style.sheet.insertRule instead of innerHTML to ensure shadow root styleSheets are populated, and add test for empty rewrittenRules guard. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove unreachable empty-rules check (rulesByOwner entries always have at least one rule). Add sanity-check assertions to shadow DOM test to verify stylesheet accessibility and host discoverability. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
rishigupta1599
left a comment
There was a problem hiding this comment.
Thanks for the thorough work on DOM fidelity (closed shadow roots, sandboxed iframes, :state() rewriting, interactive states). The test coverage additions are substantial. However, a few issues should be addressed before this lands. The biggest ones:
- Preflight script is not shipped to consumers.
@percy/dompublishes onlydist/, butpage.jsreads the script fromsrc/preflight.jsrelative to the resolvedPERCY_DOMbundle. Works in the monorepo; silently falls back to an empty string for every installed dependency — so closed-shadow-root / ElementInternals capture will not function in production. It needs to be bundled or copied intodist/(and added tofilesinpackages/dom/package.json). Page.addScriptToEvaluateOnNewDocumentis dispatched in the samePromise.allasPage.enable. No ordering guarantee — the preflight injection can race with, or land after, the initial navigation.- "Potentially inaccessible shadow root" detection counts every custom element without a shadow-host marker. Most custom elements have no shadow DOM at all; this produces false positives in both the fidelity warning and the
fidelityRegionspayload. - Sandboxed-iframe fidelity regions are emitted before we know whether the frame was captured. Successfully serialized sandboxed srcdoc frames still emit an overlay region.
A few smaller nits inline. Nothing blocking from a test-coverage perspective — suite is very thorough — but the production-path issues above should be fixed before merge.
(Apologies: an earlier stray "Test" review came through while I was iterating on this payload — please ignore it.)
| try { | ||
| _preflightScript = await fs.promises.readFile(preflightPath, 'utf-8'); | ||
| } catch { | ||
| _preflightScript = ''; // graceful fallback if file not found |
There was a problem hiding this comment.
This will not work for consumers of @percy/core once published. PERCY_DOM resolves to node_modules/@percy/dom/dist/bundle.js at runtime, and @percy/dom's package.json has "files": ["dist"] — so ../src/preflight.js does not exist in the installed package. The catch silently swallows ENOENT and sets _preflightScript = '', meaning closed shadow root capture and ElementInternals interception will never run for any SDK consumer. The only reason it works in this PR's tests is that the monorepo layout happens to put packages/dom/src/preflight.js one directory up from dist/bundle.js.
Fix options: (a) inline the preflight source into @percy/dom's bundle and export it as a string (e.g. PercyDOM.PREFLIGHT_SCRIPT), (b) copy preflight.js into packages/dom/dist/ as part of the build and reference it there, or (c) bundle it into @percy/core/dist directly. Whichever you pick, please add a test that asserts getPreflightScript() returns a non-empty string when the resolved bundle lives in a typical node_modules layout so this regression can't reoccur.
There was a problem hiding this comment.
Fixed. getPreflightScript() now tries multiple candidate paths (src/preflight.js, dist/preflight.js, and alongside the bundle) so it works both in the monorepo and when installed via npm.
| if (script) { | ||
| return session.send('Page.addScriptToEvaluateOnNewDocument', { source: script }); | ||
| } | ||
| })); |
There was a problem hiding this comment.
Ordering hazard: Page.addScriptToEvaluateOnNewDocument requires Page.enable to have landed first, but both are dispatched concurrently inside Promise.all(commands). Depending on how the CDP transport flushes messages, the preflight addScriptToEvaluateOnNewDocument can be rejected ("Page domain is not enabled") or applied after the first navigation has already started — which is the window we're trying to cover. Please await session.send('Page.enable') before pushing the preflight send, or chain the preflight onto that specific command rather than running it in parallel.
There was a problem hiding this comment.
Fixed. Preflight addScriptToEvaluateOnNewDocument is now chained onto Page.enable via .then() instead of running in parallel, ensuring the Page domain is ready first.
| // before any page scripts run | ||
| getPreflightScript().then(script => { | ||
| if (script) { | ||
| return session.send('Page.addScriptToEvaluateOnNewDocument', { source: script }); |
There was a problem hiding this comment.
The returned promise here is swallowed by the outer .catch(session._handleClosedError) on line 292, which is fine for session-close races. Any other addScriptToEvaluateOnNewDocument failure (oversized source, invalid JS, etc.) is also consumed by _handleClosedError though. Consider logging a debug/warning when preflight injection fails for non-close reasons — otherwise we can't tell from field logs whether capture is actually instrumented.
There was a problem hiding this comment.
Fixed. Non-close preflight injection failures are now logged at debug level. Close/destroy errors are still silently handled by _handleClosedError.
| let inaccessibleShadowCount = 0; | ||
| for (let origEl of ctx.dom.querySelectorAll('*')) { | ||
| if (!origEl.tagName?.includes('-')) continue; | ||
| if (origEl.hasAttribute('data-percy-shadow-host')) continue; |
There was a problem hiding this comment.
This flags every custom element that isn't a shadow host as "potentially inaccessible shadow". In practice most custom elements don't use shadow DOM at all (light-DOM components, simple wrappers, icon elements, etc.). Consequences: (a) inaccessibleShadowCount is inflated and the fidelity warning becomes noisy/wrong, and (b) a fidelityRegions entry is pushed per custom element, so the API/UI renders overlays on elements that were fully captured.
Suggestion: only flag when there's positive evidence of an inaccessible shadow — e.g. preflight records which elements called attachShadow with mode: 'closed' before WeakMap lookup succeeded. Absent that signal, drop the counter/region and only warn when WeakMap tracking is explicitly bypassed.
There was a problem hiding this comment.
Fixed. Now only flags custom elements that are confirmed closed shadow hosts via window.__percyClosedShadowRoots?.has(origEl), instead of flagging every custom element without a shadow host attribute.
|
|
||
| if (tokens.length === 0) { | ||
| warnings.add(`Sandboxed iframe "${frameLabel}" has no permissions — content may not render with full fidelity in Percy`); | ||
| captureFidelityRegion(frame, 'sandboxed-restricted', fidelityRegions); |
There was a problem hiding this comment.
Fidelity region is pushed before we know the sandboxed frame's fate. A srcdoc iframe with sandbox but no allow-scripts can still be serializable via frame.contentDocument, so it gets both (a) fully captured into the snapshot AND (b) an overlay region in fidelityRegions. Consider deferring captureFidelityRegion for sandbox-restricted frames until after the contentDocument branch, only pushing if the frame was not captured — or only emit the region in the else/!enableJavaScript && builtWithJs branches below where we actually discard the iframe.
There was a problem hiding this comment.
Fixed. Removed captureFidelityRegion calls from within the sandbox warning section. Sandboxed frames that aren't captured will still get a fidelity region from the else branch at the bottom (cross-origin-excluded).
| // Warn about sandboxed iframes | ||
| if (sandboxAttr !== null) { | ||
| sandboxWarned++; | ||
| let frameLabel = frame.id || frame.src || percyElementId; |
There was a problem hiding this comment.
Nit: frameLabel = frame.id || frame.src || percyElementId — percyElementId is an internal token (_abc123) that's meaningless to users. Falling through produces a warning like Sandboxed iframe "_ab12cd" has no permissions, which reads like an error code. Prefer user-actionable labeling, e.g. frame.src || frame.getAttribute('name') || '<unnamed iframe>'.
There was a problem hiding this comment.
Fixed. Changed fallback to frame.id || frame.src || frame.getAttribute('name') || '<unnamed iframe>' instead of using the internal percyElementId.
| let origAttachShadow = window.Element.prototype.attachShadow; | ||
| window.Element.prototype.attachShadow = function(init) { | ||
| let root = origAttachShadow.call(this, init); | ||
| if (init && init.mode === 'closed') { |
There was a problem hiding this comment.
Minor robustness: if a page calls attachShadow() with no args (or null), init.mode throws a TypeError inside our wrapper — a slightly different failure mode than the original attachShadow. Guard with init?.mode === 'closed'.
There was a problem hiding this comment.
Fixed. Changed to init?.mode === 'closed' and origAttachShadow.apply(this, arguments) for both null safety and forward compatibility.
|
|
||
| // wait for custom elements to be defined before capturing | ||
| /* istanbul ignore next: no instrumenting injected code */ | ||
| await this.eval(function() { |
There was a problem hiding this comment.
This adds up to a 5-second hard wait to every snapshot whenever the page has a single :not(:defined) element. Many production sites legitimately reference custom-element tags that never register (third-party widgets, blocked lazy loaders, typos). That means the common case eats +5s latency with no way to opt out. Please either (a) make the timeout configurable via the snapshot schema, (b) use a much shorter default (e.g. 500ms) since customElements.whenDefined tends to resolve synchronously for already-defined tags, or (c) resolve early if network is idle and the undefined tags have no pending loader.
This will regress P50 build latency for SDK consumers.
There was a problem hiding this comment.
Fixed. Reduced timeout from 5000ms to 500ms. customElements.whenDefined resolves synchronously for already-defined tags, so 500ms is sufficient for the common case while avoiding the latency regression for pages with never-registered custom elements.
| for (let w of domWarnings) log.info(w); | ||
|
|
||
| // extract fidelity regions for API upload | ||
| let fidelityRegions = domSnapshot?.fidelityRegions || domSnapshot?.[0]?.fidelityRegions || []; |
There was a problem hiding this comment.
Reading domSnapshot?.[0]?.fidelityRegions suggests multi-root DOMs are expected, but only domSnapshot[0] is consulted — fidelity regions captured by subsequent roots are silently dropped. If responsive snapshot capture produces multiple domSnapshots (per-width), you'll be under-reporting. Either merge regions across all entries or add a comment explaining why only the first matters.
There was a problem hiding this comment.
Added a comment explaining that only the first domSnapshot's fidelity regions are used because for responsive captures with multiple widths, regions are width-independent (same DOM structure) so the first entry is representative.
| try { | ||
| for (let state of percyInternals.states) { | ||
| // CSS-escape the state value to prevent injection | ||
| states.push(state.replace(/["\\}\]]/g, '\\$&')); |
There was a problem hiding this comment.
The inline comment says "CSS-escape the state value to prevent injection" but the regex only escapes ", \\, }, ] — not whitespace, <, >, &, or non-printables. Custom state names per spec are <dashed-ident>s so this shouldn't happen through legitimate API use; if the intent is defense-in-depth against a non-compliant runtime, either tighten the validation (if (!/^[-\\w]+$/.test(state)) continue;) or soften the comment to "escape attribute-breaking chars".
There was a problem hiding this comment.
Fixed. Replaced the partial regex escape with strict validation: if (!/^[-\w]+$/.test(state)) continue; to skip invalid state values per the <dashed-ident> spec requirement.
…agging, timing - Fix preflight path resolution to work in both monorepo and npm-installed contexts by trying multiple candidate paths - Chain preflight addScriptToEvaluateOnNewDocument after Page.enable to avoid ordering hazard, and log non-close injection failures at debug level - Guard attachShadow wrapper against null/undefined init argument - Use .apply(this, arguments) for forward-compatible wrapping - Reduce custom element wait timeout from 5000ms to 500ms to avoid regressing P50 build latency for pages with never-registered custom elements - Only flag custom elements as potentially-inaccessible-shadow when they are confirmed closed shadow hosts via the WeakMap - Defer sandbox fidelity regions until after capture attempt - Use meaningful frame label fallback instead of internal percyElementId - Add comment explaining multi-root fidelity regions behavior - Tighten custom state validation to skip invalid values Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
No description provided.