Repository navigation
feat(ui): make the failure pages easier to read and act on - #788
Merged
Merged
Conversation
…fails Filing and the tracker sync could silently do the wrong thing: - A create from the cluster page and from one of its executions used two dedupe keys, so the same failure could be filed twice. Filings are now keyed on the cluster (a filing recorded against an execution still answers), and recorded against it. - After the link to a Piwi-filed issue was removed, filing again answered with the old key, said "created" and linked nothing. A done filing now answers only while its link is on the cluster; otherwise it is retired and a new issue is filed. A skipped filing no longer blocks the cluster. - A Jira timeout or 5xx was reported as a failure although the outbox retries it. It now comes back as queued, and the cluster and execution pages show "Filing queued" with a retry instead of a fresh Create. - The "ticket is Done, mark resolved?" state hid a regression and showed while the cluster still failed. The regression now comes first, and the reconcile is offered only once the newest finished run no longer fails; otherwise the state notes the Done ticket. - resolve-on-close and reopen-on-ticket-reopen acted on the ticket's state at every sweep, undoing a reopen or a manual resolve. They now act on a move, never resolve a regressed cluster, and a created link records its status category so the first sync can see a move. - The create modal's error rendered below the preview, out of view; it now shows at the top, and an already-filed issue is named as such. Link errors were shown as a green "Copied to clipboard" toast and overwrote the clipboard; they are error toasts now, a bare key gets a hint, and adding a link confirms it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…ages The tracker's actions and links were spread over the state line, a chip, the Details popover and the menus, and differed between the cluster page and the execution page. They now sit in the same places on both: - The situation block has an Issue line: "Tracked in PROJ-123, To Do in Jira." with the key as a link, "No issue yet." with Create issue and Link an issue, or the queued filing. A Done issue on a failure that goes on (it failed in the latest finished run) says so and offers a new issue. - One rule names the cluster's known issue on every surface: its most recently linked tracker issue, sent by the server with the cluster and with the execution. - The page's More menu offers Open PROJ-123, Create issue or Filing queued · Retry, and Link an issue, which links an issue to the cluster. - A link chip's key and title are the link; its remove button is labelled and shows on focus and on touch screens, and removing an issue Piwi filed asks first. - The execution's Details lists its links by what they belong to (this execution, this test, the run) and edits them there. - The Situation leaves the ticket to the Issue line. A Done ticket on a failure that stopped leads the Next line to "Mark the cluster resolved — PROJ-123 is Done", and the State line leaves out a reconcile button the Next line already offers. - The create modal names the cluster, says when the project has no default Jira project (with a link to set one), and explains a disabled share link. The issue title is the cluster's name from either page. - Search (Ctrl K and the search MCP tool) finds a cluster by its name, signature, #id or the key of a linked issue. - The cluster's Activity lists what Piwi wrote to its issue: the filing, its comments and moves, queued or failed. - Copy summary and Copy failure name the ticket. - The Details popover scrolls within the viewport. Linking an issue by hand now records its status category, as filing one does, so resolve-on-close and reopen-on-ticket-reopen act on its first move. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
The timeline colored every step over 500 ms orange and every one over 2 s red, so a test of quick steps showed a column of orange, while the 28.4 s request the Most likely line names stayed grey and read "28400 ms". A green check on every passed step, a "slowest" badge, category badges and a green dot on almost every tab added more color that said nothing. - One rule says when a duration stands out: at least 1 s, and a third of the test or more (shared/duration-standout.ts, unit-tested). It takes an input object and returns the reason, so a usual time from past runs can join it. Steps and requests both follow it. - A duration is one TimelineDuration: the number through DurationValue (28.4s, never 28400 ms), its share of the test and a bar on the test's clock, requests included. Only a duration that stands out takes the one warning tone of app/utils/duration-tone.ts, says why on hover and carries data-standout. The failure keeps the status palette: the failing row's tint, title and bar, the failed marks and the failure line on the axis come from STATUS_PALETTE, which gains a row tint per outcome. - The Category column, the per-row category badges and the "slowest" badge are gone. The per-category rollup stays as one meta text line, and an event row names its kind (console warning, backend error) before its text. - A passed step's mark is a muted check; the failure's cross is the only colored mark in the column. - A step's parameters are closed by default, the failing step's too, and open from the step's title (aria-expanded); StepParamsDisclosure becomes StepParams, the list alone. - A bar the "Around the failure" window cuts ends in an arrow at the cut edge, and its tooltip says it started before or runs on past the window. - Evidence tabs drop the green "has data" dot and show their count as meta text. The open tab (SELECTED_TAB_CLASS) and the window switch, now one segmented control with aria-pressed, are neutral, so green stays the primary action's. The switch shares the type chips' row with the mark key, which leaves out the Calls band the axis already labels: all five lanes of execution 37 fit on the first screen at 1280x800. Docs (features/evidence.md, within its word budget), the Evidence help, the AGENTS rules for durations and selected states, the step-params scene and new timeline-durations scenes follow. E2E: test-run-case-page and performance-ui check the standout rule, the switch and the parameters instead of the badge; timeline-durations covers a request longer than the test and the cut arrow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
The duration bars in the timeline table were placed with `item.at / testMs`, but `at` counts from the timeline's origin, the earliest timestamp anywhere in the execution. One console entry or backend log stamped before startedAt (a server whose clock runs behind) pushed every bar right, and with 10 s of skew every step of a 4.4 s test became a sliver at the end of its track. - buildFailureTimeline returns `testStart`, the test's own start on the timeline's clock (0 when nothing predates startedAt). The REST endpoint and the demo share it through getFailureTimeline; the endpoint's OpenAPI description names it. - The bar geometry moves to pure helpers in app/utils/timeline-rows.ts: testTrack (from the test start, as long as the test) and barPlace (clamped, never narrower than 1.5%), unit-tested with evidence 3 s and 10 s before the start and a request that outlasts the test. The no-axis waterfall uses barPlace too. - windowCuts and windowCutText replace the card's inline check: a bar the window cuts on both sides says "Started before and runs on past this window". - A step title's aria-controls is set only while its parameter list is open, so it never names an id missing from the DOM. The title carries data-testid="step-params-toggle", which the step-params scene clicks, and test-run-case-page checks aria-controls closed and open. - SEGMENTED_SELECTED_CLASS joins SELECTED_TAB_CLASS in app/utils/index.ts; the window switch, the failing step's screen view, the locator drawer's reading switch and the project's failure segments use it, and the AGENTS rule names both constants (a tab strip, a segmented control or view switch). - The AGENTS duration rule covers the execution timeline, and says the Network tab and the trace's request list still color by fixed thresholds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
A findability test of the Issue line left four gaps, closed here on the
cluster page and on the execution page alike:
- A filing Jira refused for good (its create-issue action failed or was
skipped, and nothing newer replaced it) is named on the Issue line in one
meta sentence quoting Jira's reason, wherever filing is offered ("The last
filing failed: Jira answered ..."). So File a new issue and Create issue
are no longer a blind retry. The server sends it with the cluster and with
the execution's failureCluster (issueFilingFailure), read by
clusterIssueFilings, which also gives the queued filings in the same two
queries; the demo reads it through the same shared handlers.
- The issue key gets a tooltip with the issue's summary and says it opens in
Jira; the known issue the server sends now carries that summary (title).
- "The issue is Done, but the failure goes on." is body text on the sentence
that names the issue, not a muted line under it.
- The dialog opened from File a new issue is titled so and says a new issue
is filed next to the Done one, which stays linked. Link an issue on a
tracked cluster says the current issue stays linked, and that a Jira issue
(or issue on a connected tracker) linked now takes its place, since the
cluster's issue is its newest such link, as clusterKnownIssues decides.
The issue-tracking page and the issue help topics say the same; the page
stays within its word budget.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…he cluster's links The Details popover opened upward once the Issue line pushed the facts line down. At 1280x800 on a failing execution it reached the top of the screen, covered the navbar's Share and Export, and cut its last links group with no sign that more was below. Its Links section also left out the cluster's links, so the ticket the Issue line names was not there. - The popover opens below the Details button and scrolls inside when it is taller than the room there, with a "More below" fade while content is hidden; it opens above only when the button sits less than 200 px from the bottom of the screen (a phone), and never over the navbar. - From md up its groups sit in two columns, so it is about half as tall. - Links ends with a read-only "Cluster #N" group listing the cluster's links, its issue among them, with a link to the cluster page that edits them. The server sends those links with the execution's failureCluster through the shared handler, so the demo shows them too. The navbar's More actions menu opened on every click made once the page had hydrated (/failure-clusters/10 and /test-run-cases/5, dev server); a click before hydration ends does nothing, as for any server-rendered control, so the menu is left as it is. The execution-details-links scene captures the popover open at 1280x800, scrolled to its links. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…ed filing at once Review follow-ups on the Details popover and the Issue line. - The execution's Details open below the button when the content fits there, or when there is at least as much room below as above, and above otherwise. The 200 px rule kept it below on a failing execution at 1280x800 and hid the Cluster #2 group under "More below"; it now opens above there and shows every group. - The popover's top limit is the top of the panel body, read when it opens, so it stays under the navbar when the public demo's banner pushes the navbar down. On a phone it stays within the screen width. - Links spans both columns from md up, so link chips are as wide as elsewhere. - "Edit on the cluster page" opens /failure-clusters/N#links, where the cluster's Details open on load, aligned to their button. - When Jira refuses a filing, the Create issue dialog stays open with the reason and the page behind it refreshes (a new `failed` event), so the Issue line names the refusal without a reload. - The refusal sentence shows only where filing is offered, as documented, and says how long ago the filing failed ("The last filing failed 4 hours ago: Jira answered ..."). IssueFilingFailure drops its unused status. - The two screenshot scenes that track cluster 1 share one prepare helper. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
app:measure printed numbers but held the failure pages to nothing, and it could not say whether the Next step's button sits near what it copies. The budgets now live in a pure module, scripts/lib/detail-page-budgets.mjs, shared by the script and a unit test. At 1280x800: at most 15 text styles in the execution page's situation block and 12 in the cluster page's, at most 25 controls above the fold with the navbar included, at most one solid primary button above the fold, and a Next step that copies a code change (copy-git-apply, copy-patch) shows that change in the block. copy-recipe and copy-flake-command are reported without a verdict. The report prints a verdict per route and a summary of breaches; --check exits 1 on a breach, --port picks the port of the server the script boots. The page measure gains the Next step (kind, action, how far below its button the visible [data-copies] content sits), the controls above the fold split into navbar, block and below, the Most likely grade (replacing the clue strengths, which read nothing since the grade moved to StoryLine), the solid primaries, and the positions of Most likely, Issue, the occurrences trend and Activity (the cluster-history hook no longer exists). distinctTextStyles skips SVG <title>/<desc> text and visually hidden text: the empty sparkline bars' titles counted as a 13th style on cluster #2. The script waits for the page's API requests before reading, since the cluster's What changed line fills after hydration and two runs read different pages otherwise, and it reports a route whose record answers 404 instead of measuring the page's error state. Hooks, attributes only: data-next-kind and data-next-action on NextStepLine, data-shot="most-likely" on StoryLine, data-copies on DiagnosisResult's PatchBlock (copy-git-apply), LocatorHealingPanel's suggested edit (copy-patch) and ReproduceSection (copy-recipe). The seeded #37 E2E pins data-next-action. A unit test reads shared/next-step.ts and fails when a copy-... primary is missing from NEXT_STEP_COPIES. Measured on a copy of the dev database, 1280x800, before -> after: controls above the fold unchanged on every route (#37 38, #13 38, #87 38, #768 28, #748 21, cluster #10 33, #2 29, #5 30, #1 30); text styles unchanged on the executions, one fewer on each cluster (#2 13 -> 12, #10/#5/#1 12 -> 11); Next content: #37 and #13 not on the page, #87 1 782 px below, cluster #2 2 265 px, #1 2 447 px, #5 not on the page, #10 mark-resolved (copies nothing). The rule in apps/application/AGENTS.md, the root command table and the run-app skill (default routes, budgets, the ?tab=diagnosis and ?tab=history routes that no longer exist, #768 and #748) follow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…atabase Without --url, app:measure booted a server on the local dev database. That database carries whatever earlier work left in it (simulated Jira links such as CHK-101 and CHK-7, triage changes), so the Issue line, the Next step and the control counts depended on the machine and the verdicts were not reproducible. dev-server.mjs gains seedThrowawayDb(dir): it empties the directory, regenerates public/demo/seed.sql when it is missing or no longer hashes to seed.version.json (written to the throwaway directory first and copied, so the tracked version file is left alone), then runs app:seed:dev with PIWI_DATABASE_PATH and PIWI_STORAGE_PATH pointing inside it. startServer takes an optional env for the server, and skips seeding the dev database when that env names its own; take-feature-screenshots is unchanged. Progress messages go to stderr so a --json report stays parseable. Without --url the measure seeds .data/measure/ and boots its server on it; the report says so, and with --url it says it measured the server's own data and that a connected tracker's state is not covered by the budgets. Two runs without --url (1280x800, default routes) give identical objects for every route. Controls above the fold: executions 37, 13, 87, 768 and 748 at 38, 38, 37, 28 and 21; clusters 10, 2, 5 and 1 at 32, 29, 29 and 26. The content a Next step copies sits 2 453 px below its button on cluster 10 and 2 109 px on cluster 2, and is not on the page of executions 37 and 13. The same script with --url on a copy of the dev database reads execution 87 at 38 and clusters 10, 5 and 1 at 33, 30 and 30, with Mark resolved as cluster 10's Next. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
ARCHITECTURE.md still described the execution page as diagnosis-first, with a Diagnosis tab, a right rail of cards and ?tab= aliases. The page reads top to bottom in one column now, and its component table listed cards that are gone. The paragraph now walks both failure pages: SituationBlock and the order of its ROWS (with the component behind each row and the bug report's use of the same block), the facts line, where the rows are built (the shared handlers, shared/situation.ts, shared/cluster-state.ts, computeNextStep) and how the Next action reaches the page (useNextStepActions), then what sits under the block on each page and the section locator both provide. The component table describes test-case/ and cluster/ as they are and gains integrations/ (the Issue line and the tracker dialogs), bug-reports/, gaps/, notifications/ and reports/. A "Failure pages" entry joins the shared building blocks. The execution-history scene opens /test-run-cases/229 without ?tab=history, describes the History block, and no longer waits for chart geometry: the block is a strip of squares with no SVG, so the scene timed out on its chart wait. Four report scenes capture the first screen of a pass on retry (execution 768) and a test that did not run (execution 748) at 1280x800 and 390x800, so every route app:measure reads by default has its picture. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
app:measure booted its server on 3050, the port app:screens uses, and never checked it was free. nuxt dev then bound another port quietly, the health check passed against the server already on 3050, the run measured that server's code and data, and the shutdown wait failed after 30 s. The throwaway server also inherited the shell and .env (nuxt dev fills only the keys still undefined): a Postgres URL routed it away from the seeded SQLite file, an AI provider turned execution #748's Next from copy-recipe into diagnose, and Jira credentials connected a tracker, while the report still said it measured a freshly seeded database with no tracker. dev-server.mjs gains portInUse(port), a TCP probe of localhost. startServer refuses a port something listens on, and waitForPortFree waits on the same probe rather than on a failing health request. seedThrowawayDb returns the settings the server must not take from the machine: no database URL, local storage, authentication off, no AI provider or key, no Jira credentials, the default locale and time zone. app:measure boots on 3060 by default, checks the port before seeding, and the report names no AI provider alongside no issue tracker. The run-app skill says so and gains the port pitfall. Checked: with a listener on 3060 the measure stops at once ("port 3060 is already in use; stop that server or pass --port"), and startServer refuses the same way for app:screens. With a .env setting a Postgres URL, an AI provider, Jira credentials and fr-FR, the measure reads the same numbers as before on every default route (executions 37, 13, 87, 768 and 748 at 38, 38, 37, 28 and 21 controls; #748's Next copy-recipe); the previous script with only the AI provider in .env read #748 at 20 controls with diagnose as Next. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
The skill said --expand unfolds every collapsed section, the More ways to fix sections included. It clicks the collapsible cards' role="button" toggles only: More ways to fix uses plain buttons and is an accordion, which opens the section the Next step points at and holds one section open at a time. An agent following the skill believed it had seen every section. The line now names the evidence cards as what --expand unfolds, and says to click another More ways to fix section in a scene's run to see it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
The drift guard checked that every copy-... primary of the next-step policy has a class in NEXT_STEP_COPIES, but nothing checked that a component carries each budgeted action in a data-copies attribute. A dropped or mistyped carrier made every distance for that action read "not on the page", which looks like a page regression rather than a missing hook. The test reads app/components/**/*.vue and requires each budgeted action (copy-git-apply, copy-patch) among the tokens of some data-copies value. Renaming the DiagnosisResult carrier to copy-git-aply fails it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…rywhere
Piwi's own diagnosis stores its patch validation at the top of the details,
an agent's diagnosis inside its suggested fix. Each reader picked its own
place. The fix plan read only the suggested fix and otherwise re-validated
the patch against an empty file map, so every Piwi-made fix plan said
"unchecked". The diagnosis card and the diagnosis history read only the top
level, so an agent's patch carried no badge. The AI usage tally and the
next-step policy read both, in opposite orders.
storedPatchValidation (shared/patch.ts) is now the one reader: the suggested
fix's validation, else the top-level one, when it is an object with a string
status. The order is the one the next-step policy already used, so no next
step changes. The fix plan, getClusterPatchFacts, the AI usage tally, the
diagnosis card and the diagnosis history all use it, and the fix plan
re-validates structurally only when nothing is stored.
Visible change: get_fix_plan, GET /api/failure-clusters/:id/fix-plan and the
fix plan's "Copy as Markdown" now report the stored verdict ("Applies
cleanly" on the seeded clusters) instead of "Unverified", agreeing with the
next step that proposes the patch. The issue body built from the fix plan does
not print the validation, so filed issues are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…k steps denying the clues
The failure pages said "unassigned" next to the code owner ("cluster #1
(open, unassigned). Owner @Checkout-Team."), which reads as a contradiction:
an assignee and a code owner are two different things. The situation's
cluster clause now names an assignee only when one is set ("(open, assigned
to Avery)", "(open; fixed once before, the fix did not hold)"), and the
cluster state's failing sentence ends after the run ("Still failing — last
seen 8 hours ago in run #1.", where it ended ", open, unassigned."). The
reports and analytics keep their "Unassigned" category.
The two fallback next steps claimed that nothing explained the failure,
while the Most likely line above them often showed a strong clue: the policy
never sees the clues. They now say what the step does. "Diagnose with AI"
gives "A diagnosis reads the clues, the evidence and the recent commits
together and proposes a fix." and "Reproduce locally" gives "Run it on the
recorded commit and browser to watch it fail."
The execution route added a CODEOWNERS owner to the verdict after
getTestRunCase had built the situation sentence, so the sentence never
named that owner. getTestRunCase now takes an optional resolveOwner, called
for a failing test with no piwi:owner annotation before the verdict and the
situation are built; the verdict accepts an owner with its source. The route
passes a resolver around resolveOwners (still caught, so a CODEOWNERS read
never blocks the page). The demo, MCP and the export pass none in this
commit.
getTestRunCase also reads the cluster's diagnosis with scope 'cluster', as
the cluster page does, so an execution-scoped diagnosis on the same cluster
is never shown as the cluster's.
MCP output change: get_fix_plan and explain_failure return these two next
steps with the new title and why in nextStep, and their situation text no
longer says "unassigned" in the cluster clause.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
readLocatorTarget lower-cased the accessible name it read from the failing
locator so it could match the console and the ARIA snapshot without regard
to case, and the sentences printed that lower-cased value: the Most likely
line of the seeded checkout failure read 'The button "pay" stayed
disabled', under a headline that says getByRole('button', { name: 'Pay' }).
The target now keeps the name as the test wrote it beside a lower-cased
needle. The story, the blocked element's clue and the console clue print the
name; the ARIA match uses the needle and the console match lower-cases both
sides, so matching stays case-insensitive.
The docs already write "Pay"; the tour-execution and gather-evidence images
show the line and are recaptured with the other failure-page illustrations.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
… on both pages Two rules decided what a Done ticket means, each in two forms. The cluster page compared run positions to tell whether the failure went on and offered "Mark the cluster resolved — KEY is Done" from its state; the execution page compared run start times and never offered it, so on the dev database (CHK-7 Done on cluster #10) the cluster proposed marking it resolved while its latest execution proposed applying the patch. The cluster page's Issue line also read "the failure goes on" from any state that was not ticket-done, so a snoozed cluster whose failure had stopped said "The issue is Done, but the failure goes on." shared/cluster-state.ts now holds both rules: - failureGoesOn(lastSeen, latestFinished): the cluster was last seen in the latest finished run or a later one, or no run has finished. Runs are compared by start time, then by id when two started in the same second (run start times are stored to the second, and a failing run followed at once by a passing one otherwise read as "goes on" on the execution page). - ticketReconcileKey(cluster, { failureGoesOn, now }): the Done ticket's key when the cluster is open, not snoozed, its fix did not regress and the failure stopped. computeClusterState takes failureGoesOn instead of the latest finished run id and offers the reconcile through ticketReconcileKey. getFailureCluster computes failureGoesOn from the runs it already loads (now ordered by start time then id), returns it, and passes the helper's key to the next step. getTestRunCase uses the same two helpers and passes the key to its next step, so the execution page offers "Mark the cluster resolved — KEY is Done" as the cluster page does. The cluster page hands cluster.failureGoesOn to its Issue line. MCP output change: explain_failure returns nextStep.kind 'mark-resolved' for an execution whose cluster's Done ticket no longer fails, and get_fix_plan's nextStep follows the same failureGoesOn rule. features/issue-tracking.md already describes this behavior. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…turn
The execution page's Situation sentence names the owner the repository's
CODEOWNERS gives a failing test with no piwi:owner annotation ("... Owner
@payments."), through getTestRunCase's resolveOwner. explain_failure and
get_fix_plan built their situation from getTestRunCase with no resolver, so
for such a test the agent read a sentence without the owner the page shows,
and get_fix_plan returned that sentence next to an ownership block that
names the CODEOWNERS owner.
codeownersOwnerResolver (server/utils/scm/ownership.ts) is now the one
resolver, returning the CODEOWNERS owner of one spec file, a failed read
resolving to no owner. The execution route, explain_failure and get_fix_plan pass it, so
the situation text both tools return names the same owner as the page.
explain_failure resolves the execution's project itself to build it.
MCP output change: explain_failure.situation and get_fix_plan.situation end
with "Owner @team." when only CODEOWNERS names one.
The docs say the cluster and execution pages (not the inbox) fall back to
CODEOWNERS; the comment on the cluster handler's owner says how each page
resolves it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…e ticket stopped failing The Issue line's header comment said only the cluster page offers to resolve the cluster once its Done ticket's failure stopped. Both failure pages do, through ticketReconcileKey: the execution page proposes "Mark the cluster resolved — KEY is Done" as the cluster page does. Comment only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
The toolbox renders a section's body only while it is open, so an action that drives a panel inside a folded section found no panel and did nothing. On the execution page that was Copy patch, Copy locator, Pick from snapshot and All alternatives once Locator fix was folded; on the cluster page, More actions > Show context and Copy prompt and the next step's Re-diagnose while Diagnosis was folded (cluster #2 opens on Locator fix, so its Diagnosis is folded by default). - Toolbox exposes openSection and scrollToSection as promises that resolve once the section's body is mounted. - useNextStepActions waits for that before every locator panel call: the copies open the section without scrolling, Pick from snapshot and All alternatives scroll to it. With no panel on the page, an error toast says so instead of a dead button. Read the diagnosis, Diagnose and Reproduce await the same promise. - The cluster page opens Diagnosis before Show context and Re-diagnose, with the same toast when the panel is missing. Copy prompt copies from the cluster's context endpoint through useCopyAiPrompt, the path the next step's Copy AI prompt already takes, so it needs no panel and leaves the sections as they are; DiagnosisPanel drops its duplicate copy. In E2E, Copy prompt with Diagnosis folded shows the toast and fills the clipboard; on a seeded replace-locator execution with a line edit (#87 or #533, read from the API), Pick from snapshot opens the picker and Copy patch copies the healing edit's diff with Locator fix folded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
… step checked The cluster's next step decides on replace-locator from the healing of its latest occurrence (latestTestRunsCaseId), but the page's Locator fix section, its hoisted healing fetch and the LocatorHealingPanel read the most-affected test's newest execution (affectedTestCases[0]). The two differ on every seeded cluster (#1: 2 against 37, #2: 3 against 87, #5: 497 against 586), and on #5 the cited source line differs (15 against 7), so the step and the section could reason on two failures. - locatorCaseId, hasLocatorPanel, the hoisted fetch and the panel read latestExecId, the execution computeNextStep checks. - On both failure pages the healing fetch is awaited (not lazy, and immediate) when the next step is replace-locator, so the Locator fix section is in the page's first render after a client-side navigation rather than added once the fetch lands. The execution page fetches it for that step even when the verdict alone would not. The server render already waited for an immediate fetch, lazy or not. In E2E, on seeded clusters #2 and #5 (skipped elsewhere, and unless the step is replace-locator), a client-side navigation from the project's cluster list with a slowed healing answer renders the section at once, fetches only the latest occurrence's healing and shows its recommended locator. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
Both failure pages carry More actions in the navbar with the same issue labels, but in two orders: the cluster page listed the ticket, then Quarantine all affected tests, Show context, Copy prompt, Re-run in CI, Copy retry command and Copy summary; the execution page listed Copy retry command and Quarantine first, then the ticket and Copy failure. Someone moving between the two pages had to read each menu again. Both menus are now grouped arrays (the dropdown draws a separator between groups), in one order: - the ticket: Open KEY, Create issue or Filing queued · Retry, then Link an issue; - the test actions: Quarantine this test or Release from quarantine, Quarantine all affected tests, Re-run in CI; - the copies: Copy failure or Copy summary, Copy retry command, and on the cluster page Show context and Copy prompt; - Refresh, alone. Each page shows only the items it has; an empty group is dropped. SituationBlock keeps its #actions slot, which the bug report page uses. In E2E, the execution page's menu test also checks that order and that Refresh closes the menu in its own group; a new cluster-page test does the same on the seeded two-test cluster, with Copy summary present. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
Both failure pages fetch the execution's locator healing to decide whether the Locator fix section has anything to show, and the panel inside that section fetched it again under the same key. Outside hydration Nuxt's default getCachedData returns nothing, so every mount of the panel sent a second request and rendered nothing until it landed: after a client-side navigation the section's row was in the first render but its body came about 1.5 s later on a slow answer, and on the execution page each fold and unfold of the section asked for the healing again. - LocatorHealingPanel takes the healing as a prop and keeps no fetch of its own; both pages pass the answer they already hold. The panel renders it as soon as it is there, and keeps it on screen while a refresh runs. - A confirmed snapshot pick refreshes the page's fetch by its key (locator-healing-<id>), the key the page uses; the panel refreshed a URL that matched no key, so a pick never updated the list. In E2E, the cluster test counts the healing requests in an array and expects exactly one, for the latest occurrence, with the panel's body ([data-shot=alternative-locators]) present at first render; the execution test expects no healing request while the Locator fix section is folded and opened again. Both fail on the previous panel (no body at first render; four extra requests). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
… made The server proposes replace-locator whenever the healing carries a recommendation, and a rename found in the run's own diff returns its locators in fromDiffRename alone. The panel counted that list, but the two pages' gates for the Locator fix section did not, so such a failure had no section. The next step's actions open that section first, so all four of them (Copy patch, Copy locator, Pick from snapshot, All alternatives) ended in the "Locator fix not available" toast. hasHealingAlternatives(healing), in #shared/locator-healing, is the one rule, used by the panel and by both pages' section gates. It asks for a source other than none and at least one ranked list from any rung of the lookup (the diff, the failing page, a prior pass, the ARIA snapshot). Unit tests cover each list on its own, a diff rename alone, and the empty cases (no answer, source none, empty lists). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…g one Toolbox.openSection set the open key to whatever it was given, even a section the toolbox does not show. A next-step action whose section is missing (Copy patch with no Locator fix section, as on a project that declines fixtures) folded the section the reader had open, such as Diagnosis, opened nothing, then showed its error toast. - openSection returns at once for a key outside the sections it renders; scrollToSection then finds no section to scroll to, and the callers fall through to their panel-missing toast. - useNextStepActions documents scrollToSection and openSection by the actions that use each. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…on folded On the cluster page, More actions › Show context and the next step's Re-diagnose act on the Diagnosis panel, which the toolbox mounts only while its section is open; both go through the page's onDiagnosisPanel, which opens the section first. Only Copy prompt was covered, and it no longer touches the panel. The new test runs in the AI diagnosis block (mock provider configured, the cluster diagnosed, so its next step is Follow the diagnosis with Re-diagnose inline). With Diagnosis folded, Show context opens the section and the "Context sent to AI" dialog; folded again, Re-diagnose opens the section and asks for a new diagnosis, a request the test aborts so the stored diagnosis stays for the tests that follow. It fails when onDiagnosisPanel does not open the section. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
The execution timeline's table header carries a visually hidden "Kind" label (sr-only, so absolutely positioned). Its header cell was not positioned, so the label's containing block was the dashboard panel and it sat about 890 px down the fixed dashboard group, outside the panel's own scroller. At 1280x800 that gave the fixed group about 90 px of overflow, and any scrollIntoView on the page (Pick from snapshot opening the Locator fix section, a clue revealing its evidence) scrolled the whole dashboard up: the navbar went off screen and a blank band stayed at the bottom, even after the picker closed. The header cell is relative, so the label stays inside the table and its scroller clips it; the fixed group no longer overflows on the execution or cluster pages. In E2E, after Pick from snapshot and Cancel on the seeded replace-locator execution, the navbar's More actions button is still at or below the top of the viewport (it was at -46 px). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
The run page loads its rows after hydration, so the issue key chip on the failing execution's row can appear a few seconds after navigation on a local dev server, which compiles each page on its first visit. The check now waits for hydration and gives the row the 15 s other run-page checks use. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…s in cluster #10's diagnosis Cluster #1's suggested fix added page.waitForLoadState('networkidle'), a wait Playwright advises against, in a product that recommends better waits. The story's own evidence says why the click times out: the Pay button stays disabled until POST /api/checkout/quote answers, which takes 28.4 s on failing runs. The fix now registers a wait on that response before the fills, awaits it before the click and marks the flow test.slow(). derivePatch takes several edits and groups them into hunks the way git diff does, so the patch is a two-hunk diff that validatePatch reports as applying at its stated lines. Diagnosis 1 (summary, root cause, hypotheses, evidence, investigation steps, prevention tips) is written around the slow quote, and its earlier version says the network capture superseded it. The demo AI's timeout-interaction template recommends waiting on the request or the web-first condition the element depends on instead of networkidle. Cluster #10's error says "Expected 26 rows, found 51" while its diagnosis said 50 instead of 25. The summary and the first sentence of the root cause now quote 51 and 26 and say the header row accounts for the difference. The spec and the error are unchanged, so the fingerprint, the cluster title and the demo examples hold. demo-seed-consistency checks that the checkout patch has several hunks and applies at its stated lines, that no story patch, seeded diagnosis or demo AI template recommends networkidle, and that a stored diagnosis of a count assertion quotes both counts of its error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…est that did not run The reporter, the blob import and the run simulator store every attempt as its own execution, but the seed collapsed the Web Dashboard's flaky test into one passing row, so the red 1/2 attempt on #738 and #768 had no execution to link to. FLAKY_CASES gives that test the error its failed attempt reports (toHaveAttribute on locator('html'), ending in a tests/admin/settings.spec.ts frame), and an rng-free block after the dedicated flake blocks splits every remaining retry pass with splitFlakyExecution. The new rows take the next ids: 829 is the failed attempt of #738 and 830 the one of #768. Every existing id is unchanged, and the stored flaky root cause of the test stays 'other'. A test the run never reached carried four planned 0 ms steps, a start time and a worker, which the reporter never sends: materializeUnrunTests emits no steps, startedAt or workerIndex, and no attempts. The max-failures rows and the serial cascade's blocked rows now store none of these (the cascade keeps its one didnotrun attempt with no start, as onTestEnd reports it), so the execution page shows no empty Evidence card on #748. The slowest-step reduce and the worker-lane re-pack skip rows without steps or a start time. demo-seed-consistency checks that every retry pass has its failed attempt stored as its own execution with an error, and that a test that did not run carries no steps, start time, worker or step events. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
Copy apply command is a heredoc for bash, zsh or Git Bash, so on the locator step with an edit a PowerShell user had no way to apply the edit from the Next row, although the docs name Download .patch as the portable route. The step's secondary actions are Copy locator, Download .patch, Pick from snapshot and All alternatives: Copy locator stays inline and Download .patch opens the overflow menu, so the row shows no new control. The download saves the change the row shows (the edit's diff), named after the cluster (piwi-fix-cluster-N, or piwi-fix without one). A locator step without an edit offers no download. Docs: fix-plans.md says Copy locator copies the recommended locator and is the primary when there is no line to rewrite, since it is also offered next to Copy apply command; the portable route holds for both code-change steps. Tests: the policy's secondary actions with and without an edit; E2E, the seeded locator step with an edit (#87) downloads the edit's diff from the menu. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
… demo seed The two failure pages build their lines from the same shared handlers, but nothing checked that the lines agree once they meet real data. This test regenerates the demo seed into its own temporary directory, loads it into a temporary libSQL database and builds, for every failure cluster, every problem execution (failed, timed out, did not run) and every cluster's latest occurrence, the lines each page shows: Most likely through pickMostLikely (gated on the verdict on the execution page), the Next step and its source line through nextStepSourceLine, the cluster's State and the execution's Situation. It does so with and without an AI provider. Six rules, each on kinds and sources rather than sentences: - a verified fix reads the same in State and Next: a State that offers Mark resolved sits beside a Next that marks it resolved, and a Next that applies the diagnosed patch to a verified fix sits beside an unconfirmed fix with no Mark resolved of its own; - a Next from the diagnosis quotes its summary when Most likely shows something else; - a cluster and its latest occurrence's execution lead with the same Most likely; - no Next says nothing explains the failure beside a strong or medium story or clue; - no Situation or State sentence says "unassigned"; - a cluster-level Next (mark resolved, replace the locator, apply the patch, follow the diagnosis, see what changed) is the same step on the latest occurrence's execution. The seed's cluster #10 reads "verified" because the seed stores no at-fix patch check, so the test also rebuilds every diagnosis-verified cluster with its patch recorded as still applying at the fix's commit, which exercises the unconfirmed-fix pairing through the real handlers. A last test names the seeded pages each rule must reach (cluster #10, #37 and cluster #1, clusters #1, #2 and #10), so the file cannot pass without checking anything. Every rule was seen failing against a deliberately broken builder. The did-not-run rule joins with the step that opens the run. The current code and seed break none of the rules. The test runs in about 20 s under app:test:unit. apps/application/AGENTS.md names it under Demo data requirements. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…lines coherence test The rule that no Next says nothing explains the failure beside a strong or medium Most likely is about the fallback steps, Diagnose with AI and Reproduce locally. Its coverage pin named execution #37, whose Next applies the patch in both AI modes, so a seed with no fallback step beside a strong or medium clue would still pass. The pin now names execution #682 (a medium clue, Next Diagnose with AI, and Reproduce locally without a provider), and the test also asserts that both fallback steps are among the pages the rule checks. The execution pages built for the cluster rules now come from the cluster handler's own latest occurrence, with the page's fallback to the most-affected test's latest execution, instead of a hand-copied SQL query. A cluster whose latest occurrence has no page built is a violation of the same-Most-likely and same-step rules, never a silent skip. The source line is built with the facts the pages pass: the patch and its validation from the cluster's fix plan (buildFixPlan) and, when the step replaces a locator, the execution's locator healing (getLocatorHealing), so a failure prints the line the page shows. The State kinds that offer Mark resolved are a set typed against ClusterState['kind'], so a renamed kind fails the typecheck. The test header lists the six rules, and apps/application/AGENTS.md shortens to one sentence that points to it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
The next-step paragraph grew with the change shown in the Next row and the fix checked at its commit, which took the page to 1,506 words. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
Contributor
Performancemain fbc9ddd → #788 5d2783b · PostgreSQL 16.15 · large dataset (2,160 runs, 474,922 executions) · authentication on No regression · 0 improvements · 217 unchanged Pages
API calls (29)
SQL per request, where it changed (4)Test run › Tests — full load in the browser — 70 → 70 statements
Requests the server made to itself while rendering (head): Test run › Changes — full load in the browser — 81 → 81 statements
Requests the server made to itself while rendering (head): Test run › Timeline — full load in the browser — 70 → 70 statements
Requests the server made to itself while rendering (head): GET /api/test-runs/1500/failure-groups — 15 → 15 statements
How this is measuredEach build ran as its own production server ( A time is a regression when the median grows past both floors and the Mann–Whitney test gives p < 0.01 (which takes at least six samples a side); a count (SQL statements, bytes, requests) when it grows past both floors. Floors: server render 10 % and 15 ms, sql per render 10 % and 3, db pages read per render 25 % and 1000, html size 10 % and 20 kB, largest paint 10 % and 50 ms, full load 10 % and 75 ms, sql per full load 10 % and 5, api calls 25 % and 3, main-thread blocking 25 % and 50 ms, dom nodes 15 % and 1000, response time 15 % and 5 ms, sql 10 % and 2, response size 10 % and 20 kB. SQL counts come from Machine: 4 × AMD EPYC 7763 64-Core Processor, Node v24.21.0. Suite: |
Collaborator
Author
|
Lint, typecheck and unit tests failed on 5a154b4 in No fix exists yet. Proposed patch, outside this PR's scope: // packages/editor/tests/lsp.test.ts, 'reports its context'
const status = await waitFor(async () => {
const s = (await client.sendRequest('piwi/status')) as StatusResult;
return s.contexts[0]?.reachedFiles ? s : undefined;
});Re-running the failed job once. Generated by Claude Code |
The Attempts tab and the timeline each spelled out when one duration is much slower than another. shared/duration-standout.ts now holds the rule once: SLOWER_FACTOR (2) and SLOWER_MIN_DELTA_MS (1 s), with isMuchSlower(ms, thanMs), and stepMatchKey(step), the label plus params key a step is matched on from one execution to another (attempt-diff's private params signature). attempt-diff imports them: a step is much slower by isMuchSlower, a request keeps its absolute floor (REQUEST_SLOW_MS, 1 s in all) with the shared factor. Its results do not change, attempt-diff.test.ts passes untouched. The usual duration of a step, which reads the same rule, comes with the timeline next. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…eline
A step or a request only stood out when it took a third of the test, so a
click that usually takes 600 ms and took 2.5 s in a long test stayed grey. The
timeline now carries each one's usual duration, and the slower-than rule of the
Attempts tab colors it.
getUsualDurations (shared/handlers/test-cases.ts) reads the test's last five
passing executions on the same browser, in runs eligible as a baseline, the
execution itself left out, then their requests: two bounded queries, called by
getFailureTimeline after the pure build, so the REST route and the demo share
it. buildUsualDurations and attachUsualDurations (shared/duration-standout.ts)
take the median per step (stepMatchKey, falling back on the label when the
params change from run to run) and per request route, kept when at least two
executions ran it, and set TimelineItem.usual.
durationStandout gains usualMs and the `usual` reason: at least 1 s, twice the
usual time and 1 s more; a third of the test still wins as `share`. A row that
stands out by its usual time says it under the duration ("usually 600ms"), the
hover names it too. On the seeded executions 37, 13 and the clusters' latest,
the rule colors no row beyond the share rule's (the 28 s Pay click and the
28.4 s request on #37); the usual durations cost about 1.5 ms per /timeline
call on the seeded SQLite database.
Docs: the OpenAPI description of /timeline, the case.evidence help (the exact
rule), the Timeline bullet of features/evidence.md (the page diff's `+3 -1 ~2`
summary cut to stay within the 1,500-word budget, now 1,498), the AGENTS.md
duration rule. New scene timeline-usual-duration; E2E in
timeline-durations.spec.ts (the API sends usual 600 and 300, the click and the
request stand out by `usual` and say their usual time).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
The Network tab and the trace's request list colored a request by fixed thresholds (orange from 500 ms, red from 1 s), so the 28.4 s quote request of execution 37 read in the warning tone on the Timeline and in failure red on the Network tab, and every sub-second request turned orange. useExecutionTimeline (app/composables) reads /timeline under one key per execution (execution-timeline-<id>, dedupe 'defer', getCachedData reuseWithinRender). FailureTimelineCard reads it there, and EvidenceTabs holds it for its Timeline and Network tabs, so a tab switch never fetches it again; it loads only for a failed execution, when one of those two tabs is open. The EvidenceTabs change is kept to that call, the per-request usual map and two props on TestCaseNetworkRequests. TestCaseNetworkRequests applies durationStandout to each request (the test's duration and the request's usual time from the timeline, by index) with the duration-tone classes, data-standout and the reason on hover, and shows "usually 300ms" beside a request that stands out against its usual time. TraceNetworkList, which has no index in common with the timeline, applies the share of the test; its bars take the warning tone when they stand out, the failed palette when they failed, neutral otherwise. The hard-coded reds of both views (failed span, failure text, error log counts, row accent, failing window) come from STATUS_PALETTE and the status-failed token. TestCasePerformancePanel keeps the standard Web Vitals bands through one ratingClass helper: poor in text-error, needs improvement in the warning tone, good uncolored (no brand green in the evidence card). AGENTS.md's duration rule now covers the Network tab and the trace's list, and the case.evidence help says Network follows it. New scenes network-tab-durations(-mobile) on execution 37. E2E: the Network tab colors the 9 s quote and the usual-time report request, not the quick ones; on the execution page fixture neither quick request is colored, no second /timeline request on a tab switch, no hydration mismatch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
A test that failed and then passed on retry opened on a green "Passed"
chip on a purple edge, with no headline, a "Passed on retry" badge and no
next step; the server built no verdict for it because the passing row has
no error, so its `why` was null and its computed step was "Reproduce
locally" (or "Diagnose with AI").
Server: getTestRunCase loads the last failed attempt of the same run, test
and browser (one select, only for a retry pass) and builds the verdict from
its error and steps. FailureVerdict.attempt names that attempt and the
execution that holds it; the verdict carries no cluster, which stays on
the failed attempt's page. The policy reads `why` = passed-on-retry from
the status even when the failed attempt is not stored, so a retry pass
always reaches the Flake Lab rows or "Compare the failing attempt with the
passing one". The since line of a retry pass drops the "Passed on retry"
lead (the status chip says it) and "first failed in this run", leads with
"Newly flaky" only when the test newly passes on retry, and says "failed
on attempt 1, passed on attempt 2", with "attempt 1" a new `attempt` part
linked to the failed execution; commit, author and owner stay. MCP
explain_failure falls back to the verdict's headline, so an agent reads
the failed attempt's error too.
Page: StatusChip takes the execution's retries and reads "Passed on retry"
in the flaky color and icon, and every chip is in sentence case ("Didn't
run", "Timed out") rather than CSS capitalize. The header drops its
"Passed on retry" badge. A retry pass shows the failed attempt's headline
with the meta line under it (the attempt clause and the latest line), the
Next row, and its evidence on Attempts; it shows no Most likely, no
toolbox, and fetches neither /clues nor locator healing.
Coherence test: every retry pass of the seed enters the loop, and its
Next must be compare-attempts or a Flake Lab step, with no Most likely
(#768 and #21 are named as reached).
Docs: evidence.md says a retry pass leads with its failed attempt's error
and opens on Attempts, and cuts two sentences to stay inside its budget;
first-failure.md, concepts.md, flaky-tests.md (same word count) and the
regression-or-flaky recipe describe the chip and the attempt clause. The
help topic of the block names the attempt clause. Report scene
execution-retry-pass-flake-lab (#21); retry-pass-clarity(-mobile) are the
before and after of #768.
Tests: failure-verdict (attempt), situation (the retry-pass since line),
new execution-states (libsql: verdict, attempt clause, compare-attempts,
a retry pass without its failed attempt), status-palette
(statusChipLabel), mcp-situation-owner (explain_failure on a retry pass),
E2E execution-states.spec.ts and the seeded #768 and #21 in
test-run-case-page.spec.ts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…a step that opens the run A test that did not run opened with no Most likely, a "Did not run" card under the block whose cause badge repeated its sentence (text-primary links, grey prose), and a next step of "Reproduce locally" or "Diagnose with AI" when no blocker was known: #748, cut by its run's failure budget, offered to reproduce a test that never started. Server: shared/did-not-run.ts builds one sentence per reason (describeDidNotRun): the hook or the serial-group failure that blocked it, with the blocking test as a part linked to its execution; the run's failure budget with its failed count; the global timeout; an interruption; no reason recorded. Run cutoffs carry the note "Reported by Playwright", a failed beforeAll says what it does to its group. getTestRunCase returns it as `didNotRun` (null for an execution that ran), and MCP explain_failure adds its text. Next: a new row 1b, kind `open-run` with no secondary actions, for a test that did not run with no blocker. A max-failures cutoff with one failed execution in the run opens it ("Open the failure that stopped the run"; one select of at most two rows, only then); with several, or an earlier failure of its group with no blocker found, "Open the run's failures" (/test-runs/:id?tab=failure-groups); a global timeout "See what made the run slow" (the run's Timeline tab); an interruption or no reason "Open the run". Its why, and the open-blocker why, say what to do and never restate the reason Most likely now gives. useNextStepActions navigates `open-run` to the run and its tab. Page: DidNotRunLine renders the sentence in the Most likely row (body style, the blocker a sentence link, the note in the meta style, no badge or icon). DidNotRunCard is deleted; its blocked-tests branch had no caller, the toolbox's Blocked section lists them. The seed and the reporter send no steps for an unrun test, so the evidence card already stays away on #748; EvidenceTabs keeps its rule. adr/ui-simplification.md is left as the record of its time. Coherence test: its did-not-run rule is on: every did-not-run execution gives its reason and its Next is open-blocker or open-run (#748 and #7 are named as reached). Docs: evidence.md (one sentence, and a shorter Attempts lead), the Most likely and Next sections of first-failure.md, "open the run" in the policy of fix-plans.md, which also drops a sentence of the plan's loop to come back within its 1,500-word budget; the situation help names the did-not-run reason. Report scene execution-did-not-run-blocked (#7); did-not-run-clarity(-mobile) are the before and after of #748. Tests: did-not-run (each reason, the hooks, the blocker part), next-step (open-run by reason, one against several failures, the blocker row wins, whys that do not restate the reason), execution-states (libsql: the failed count, the blocker link, the run's one failure), fix-sections, E2E execution-states.spec.ts (a run cut at its failure budget: Most likely, no evidence card, Next opens the run; the blocked test links its blocker; 375 px) and the seeded #748 in test-run-case-page.spec.ts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…ce across its retries A test that did not run because its run reached its maximum number of failures opens the run's one failure when there is one. That case counted the run's failed rows, and each attempt is its own row: a test that failed on two attempts, or a test whose first attempt failed before it passed on retry, gave two rows. With retries on, Most likely read "(1 failed)" from the run's distinct-test counter while Next read "Open the run's failures". getTestRunCase now reads the run's failed rows and its retry rows (the only rows that can be or replace a final failed attempt), keeps one final attempt per test and browser with finalAttempts, and opens the one left when it is a failure. A test that failed on every retry opens its last attempt, and a test that passed on retry is not counted. The query runs only while the run's failed counter is at most one, so the sentence's count and Next agree. Tests: execution-states adds a run with one test failing on retries 0 and 1 and a flaky test whose first attempt failed: Next is open-execution on the last attempt. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…explain_failure explain_failure gives a test that passed on retry its failed attempt's headline, while its error is empty and its steps are the passing attempt's: an agent read a failure with nothing failing under it and no way to open the attempt that failed without listing the run. The tool adds `failedAttempt`, the executionId and retry index of the attempt the headline comes from (the verdict's attempt), so an agent calls explain_failure on it in turn. The field is additive and absent on any other execution; the tool description in shared/mcp-tools.ts names it. Tests: mcp-situation-owner checks failedAttempt on the retry pass and its absence on the failed attempt. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
… per row The Situation help on an execution page was one 146-word paragraph whose first sentence held four nested parentheticals. It is a lead sentence and a list, as the inline-help rule asks of a hint longer than two sentences: Headline (with the line under it, and the failed attempt of a test that passed on retry), Most likely (the rule that picks it, or why a test did not run), Next and Cluster. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…Timeline The evidence page said a passing execution other than a retry pass shows "identity and facts only", which reads as if it had no evidence card; it has one, open on Timeline. The sentence names it again, and two phrasings elsewhere on the page are shorter so the page stays at 1,497 words. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…ers neutral The selected row of Affected tests, the opened attempt on the Attempts tab and the current item in the history strip and the facts line's attempts were drawn in the primary color, so green marked a selection as loudly as the page's one primary action. - TestRow marks the active row with SELECTED_ROW_CLASS (a raised surface in a neutral ring); the cluster's bulk bar sits on bg-elevated, and Quarantine is a neutral outline button. - AttemptsCard: the opened attempt takes SELECTED_ROW_CLASS with aria-current, "this one" is meta text, "open", "View in" and the flake-suspect link are neutral links, and the only-on chips use the status palette. - ExecutionFactsLine and HistoryStrip ring the current item with CURRENT_ITEM_RING_CLASS (ring-inverted, offset by the page background so it reads on any status color in both themes); the history's "Last passed in" link keeps its sentence's color. - AGENTS.md names the two new constants in the one-accent rule. - New report scenes cluster-affected-tests-selected(-dark); the cluster layout E2E checks the clicked row is the one pressed row. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
The evidence card and the affected tests now draw failures in the status palette, standing-out durations in the one warning tone and selections in neutral classes. Two tests keep them that way. - tests/unit/evidence-accent.test.ts scans the evidence card's and the affected tests' components (EvidenceTabs, FailureTimelineCard and its parts, FailingStepSnapshot, AttemptsCard, ClusterAffectedTests, TestRow) for a raw red, orange or green Tailwind shade and for a primary class outside hover: and focus-visible:. Run against the files before the timeline work and before the neutral selection commit, it fails on each of them. - tests/failure-page-accent.spec.ts renders a failing execution (Timeline, Network and Console tabs) and a two-test cluster (a selected row and the bulk bar), swaps the primary color on the root and fails on any element whose color, background, border, ring, fill or stroke follows it; it also allows at most one solid primary button on each page. - A card's header icon keeps SectionCard's color through one constant, CARD_ICON_CLASS, and carries data-card-icon so the E2E leaves it out. - AGENTS.md cites both tests in the one-accent rule and the timeline duration rule. The Network tab's TestCaseNetworkRequests and TraceNetworkList still color a request by fixed thresholds, so the scan does not list them yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
The Most likely line names a request and a console entry, but nothing in the evidence card pointed at them: the Timeline row of the 28.4 s request on execution 37 read like any other, every tab looked alike, and a citation chip opened its tab scrolled so the tab strip sat above the view, with no row marked. - pickMostLikely returns the line's citations: every citation of the story's clues, or the lone clue's; none for a diagnosis, whose citations name sections only. useEvidenceHint hands them to the evidence card. - citedItemIds (app/utils/timeline-rows.ts) matches them to the timeline items, a request, a console entry, a dialog or a step by index, a request's backend logs to the request, the failing steps to the failed step. Those rows read "Most likely" in meta text and carry data-cited, in the table and the phone cards. - The tabs holding a cited section carry a small neutral dot, titled "The Most likely line cites this tab" (Screen, Network and Console on execution 37). - The section locator takes the citation's index: StoryLine, CluesCard and a click on a timeline request or console row pass it. EvidenceTabs keeps its tab strip at the top of the view and brings the cited row under it; a cited request or console entry is ringed for two seconds (CITED_ROW_CLASS, a neutral ring), and the Network tab shows it whatever filter was open. - CluesCard's citation chips lose the down-arrow icon that read as Download. - evidence.md says a citation opens and marks its evidence and that Most likely marks the rows and tabs it cites (two cuts keep the page at 1,497 words); the Evidence help and ARCHITECTURE.md say the same. - Unit tests for citedItemIds and the line's citations; the timeline-durations E2E checks the cited request's row and tab, and that its Network citation opens the tab with the strip in view and the request ringed. New report scene timeline-cited-row (execution 37: the marks, then the Console and Network citations). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…ary color The Network tab drew its open filter (All, Failed, With logs) and its Captured / Full trace switch in Nuxt UI's default primary color, and its "Learn more" link sat in the primary green inside a sentence. Both are a selection or a link in prose, never the page's primary action, so the failure page accent E2E flags them on the evidence card. - Both UTabs take color="neutral". - The link keeps the sentence's color (text-inherit with SENTENCE_LINK_CLASS). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
The passed-on-retry, did-not-run and cited-row changes each kept features/evidence.md under 1,500 words on their own; together they reached 1,506. The intro drops "where it came from" and the situation block's parts are listed without articles. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…y classes The Network tab, the trace's request list and the Performance panel now color a duration through durationStandout and the one warning tone, so the accent scan covers TestCaseNetworkRequests, TraceNetworkList and TestCasePerformancePanel too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
The server-side processing time was emerald, so a 28.4 s server time read as good next to the same request's 28.4 s duration in the warning tone. It is a duration like the others and takes no color of its own; the duration rule colors the request's total when it stands out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…he headline does The Attempts tab listed "Attempt 1" then "Retry 1", while the line under the headline and the facts line say "failed on attempt 1, passed on attempt 2" for the same executions. Every row now reads "Attempt N", counted from 1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…st likely" A cited row read "Most likely" on its own, the same words as the situation block's row label: a reader could take the request for the cause itself, and a page query for the label matched the rows too. The rows now read "Cited by Most likely", in the same meta text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
The passing-execution test looked for a "Fix" heading, which the toolbox no
longer carries under any condition ("More ways to fix"), so the check could not
fail. It now counts [data-shot="fix"]. The spec's comments name the card as the
page does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
…e pages A pass over the docs-and-scenes checklist of the failure-pages plan: - case.evidence help lists every tab (Attempts and Locators included), says which tab the card opens on (a retry pass on Attempts) and what the cluster's subject line names; case.timeline says a step opens its parameters from its title; fix.toolbox and first-failure say the execution links to the cluster's fix plan, which is Markdown on the cluster page only. - evidence.md: the usual-time rule needs 1 s more as well as twice the time. - ai-diagnosis.md: an execution shows the cluster diagnosis's summary and patch. - mass-failure.md: the section the Next line points at follows the block as its own card. - Comments name "More ways to fix" as the page does, the toolbox orders its sections, LocatorHealingPanel says when it folds, AttemptsCard numbers every attempt, and cluster-state.ts describes the line as it is. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
Recaptured every docs scene on a freshly seeded database with the clock frozen (app:screens:docs, the server started with an AI provider so the diagnosis shows its configured state), and kept the images whose subject changed on this branch: - the execution page (gather-evidence, tour-execution) and its Locators tab; - the cluster page's situation block (cluster-issue-line) and diagnosis (ai-diagnosis, tour-ai-diagnosis); - locator healing's recommended fix, the Create issue dialog and its required fields; - the run's Changes tab and the failure-clusters grouping, whose seeded timeouts now last their 30 s; - the four other tour-*.png, recaptured with the two above so the README tour reads as one set. The other docs images differed only by timestamps or by the server's environment and are left as they were. The blog post's alt texts for gather-evidence.png and ai-diagnosis.png describe what those images show. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
The execution page and the failure cluster page are where a failure gets understood and fixed. A UI/UX review found them complete but hard to land on: lines that contradict each other, a main button that copies a change the reader never sees, no answer to "is it still failing?", everything at the same weight, a timeline that colors almost every step, and tracker controls with no fixed place. This branch implements the whole review (priorities 1 to 10).
Situation block, both pages
Issue tracker
Evidence and timeline
SELECTED_ROW_CLASS,CURRENT_ITEM_RING_CLASS, …), held bytests/unit/evidence-accent.test.tsandtests/failure-page-accent.spec.ts.Demo data and tooling
networkidle, retry passes store their failed attempt, did-not-run rows store no steps;tests/unit/failure-lines-coherence.test.tschecks every seeded failure's lines agree.app:measure --checkholds the failure pages to budgets on a freshly seeded database.Merges
main(0.50.0, rule-based filing from #787): automatic filing goes through the per-cluster dedupe and Done rule, and its writes show in the Issue line and Activity.Budgets (
app:measure --check, 1280×800)14 budget breaches at the start, 5 now. The five left are all over the 25-control budget, and in each one the situation block itself stays at 7 to 17 controls: the excess is what sits below the block and now enters the first screen because the block got shorter, the evidence card's tab strip (8 to 10 tabs) on #37, #87, #768 and cluster #10, and the history strip's 15 links on #748. Text styles, Next content distance and the one solid primary button pass everywhere.
How was it tested?
app:typecheck,app:lint,app:format, commitlint,app:screens:check,app:generate:demo+app:check:demo:runtime.app:measuretables for each milestone; an independent review of the riskier lots, with its findings fixed.Checklist
type(scope): subject)apps/docs/)🤖 Generated with Claude Code
https://claude.ai/code/session_01DN1TP3gsavgzTbFgfPDUkb