test: PR checks on a broken PR, then remediated (do not merge) - #1949
Draft
marcleblanc2 wants to merge 14 commits into
Draft
test: PR checks on a broken PR, then remediated (do not merge)#1949marcleblanc2 wants to merge 14 commits into
marcleblanc2 wants to merge 14 commits into
Conversation
dev/check-redirects.mjs checks every entry in src/data/redirects.ts: source shadows a page, source has a #fragment, duplicate source, /docs prefix, chained redirect, missing destination page or heading. The workflow compares against the merge base, so only redirects a PR breaks are reported, grouped by problem with the fix explained under each heading, and posts one suggested change per fixable entry the PR added (deleted again once the finding is gone). Not part of `npm run check`: main has hundreds of pre-existing findings. Squash of the check-redirects branch rebased onto main; the check-links commits it carried are already on main. Amp-Thread-ID: https://ampcode.com/threads/T-01a08fee-74b4-76dc-aaf9-d1245d68fdc9 Co-authored-by: Amp <amp@ampcode.com>
…ady names Amp-Thread-ID: https://ampcode.com/threads/T-01a08fee-74b4-76dc-aaf9-d1245d68fdc9 Co-authored-by: Amp <amp@ampcode.com>
…h (from the check-links PR) Amp-Thread-ID: https://ampcode.com/threads/T-01a08fee-74b4-76dc-aaf9-d1245d68fdc9 Co-authored-by: Amp <amp@ampcode.com>
… fact per line - Review comments: one suggested change per finding with a fix, no review body. Each starts with a marker so the workflow can delete suggestions for findings that are fixed and skip ones already posted. - Summary comment and review comments list line, link, problem, and fix on their own lines. - Absolute links to this site get their own section instead of Outbound. - Case-mismatch findings now carry a fix. - Wording: 'links on this site', 'these other pages', drop docs.sourcegraph.com; reproduce command matches package.json. Amp-Thread-ID: https://ampcode.com/threads/T-01a08fee-74b4-76dc-aaf9-d1245d68fdc9 Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a08fee-74b4-76dc-aaf9-d1245d68fdc9 Co-authored-by: Amp <amp@ampcode.com>
…roblem Amp-Thread-ID: https://ampcode.com/threads/T-01a08fee-74b4-76dc-aaf9-d1245d68fdc9 Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a08fee-74b4-76dc-aaf9-d1245d68fdc9 Co-authored-by: Amp <amp@ampcode.com>
… update comments whose text changed Amp-Thread-ID: https://ampcode.com/threads/T-01a08fee-74b4-76dc-aaf9-d1245d68fdc9 Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-01a08fee-74b4-76dc-aaf9-d1245d68fdc9
…ving the old wording Amp-Thread-ID: https://ampcode.com/threads/T-01a08fee-74b4-76dc-aaf9-d1245d68fdc9 Co-authored-by: Amp <amp@ampcode.com>
Vercel shows build logs only to members of its team, so contributors saw a red X and a login wall. On vercel.deployment.error this posts the tail of the build log on the PR; on the next successful build the same comment is updated to say the failure is fixed. Fork PRs are skipped so the project-scoped Vercel token is never used on their behalf.
… (do not merge) Check links: three absolute self-links (one to a moved page) and a dead external link on one line; a missing page, missing heading, and wrong-case path on the next; and the "Symbol search" heading renamed to break the inbound anchor link from search-based-code-navigation.mdx. Spell check: nine misspellings on one line, five of them block-list words. Check redirects: one broken entry per category (shadowed page, #fragment source, /docs prefix, duplicate source, chain, missing page, missing heading). Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-01a08fee-74b4-76dc-aaf9-d1245d68fdc9
Amp-Thread-ID: https://ampcode.com/threads/T-01a08fee-74b4-76dc-aaf9-d1245d68fdc9 Co-authored-by: Amp <amp@ampcode.com>
Restore the deleted page, the shadowed redirect page, and both renamed headings; point the dead links at real targets; fix the one misspelling CSpell guessed wrong (across, not arcos); drop the unsorted allow-list entry. Amp-Thread-ID: https://ampcode.com/threads/T-01a08fee-74b4-76dc-aaf9-d1245d68fdc9 Co-authored-by: Amp <amp@ampcode.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
Direct preview links to pages changed in this PR: |
marcleblanc2
force-pushed
the
marc/test-pr-checks-remediated-2
branch
from
September 11, 2026 17:44
406a779 to
f74ea26
Compare
Contributor
✅ The redirects an earlier revision of this PR broke are fixed |
Contributor
✅ The broken links an earlier revision of this PR introduced are fixed |
This comment has been minimized.
This comment has been minimized.
Contributor
Author
✅ The Vercel build that failed on an earlier revision of this PR passes |
marcleblanc2
force-pushed
the
marc/test-pr-checks-remediated-2
branch
from
September 11, 2026 17:45
f74ea26 to
406a779
Compare
This was referenced Sep 11, 2026
Merged
marcleblanc2
added a commit
that referenced
this pull request
Sep 11, 2026
…er line in reports (#1916) Follow-ups from testing the PR checks on #1913 / #1914. - Review comments: one suggested change per finding (not one per line), no review body. Each comment starts with an HTML marker; the workflow deletes suggestions whose finding is gone (or that GitHub could no longer place, `line: null`) and skips ones already posted, so resolved suggestions disappear like the spell check's do. - Summary comment and review comments put line, link, problem, and fix each on their own line. - Absolute links to this site get their own **Absolute links** section instead of being lumped into Outbound. - Case-mismatch findings now come with a fix (`/Code-Search/queries` → `/code-search/queries`). - Wording: "Write links on this site as relative paths", "fix the inbound links on these other pages", dropped `https://docs.sourcegraph.com/…`; reproduce command matches `package.json` (`pnpm check links …`). Tested locally against the `test-pr-checks-broken` branch with the CI recipe (baseline from `origin/main`, `--diff`, `--review`); build-mode run (`node dev/check-links.mjs`) still clean. Trade-off: when several fixes sit on one line, applying one suggestion outdates the others until the next run re-posts them, because GitHub will not batch overlapping suggestions. <!-- pr-stack-merge-order --> ## Merge order for the PR-check stack Trial-merged onto `main` in this order with no conflicts: 1. #1946 Vercel build log comment — independent; first so the other PRs' Vercel failures get a readable log 2. #1916 check-links report format — adds `dev/sync-review-comments.sh`, which #1935 calls 3. #1935 redirect check — needs #1916 merged first 4. #1947 spell check comment updates — independent 5. #1944 check-links, generated-docs sync PR — conflicts with #1916 on `dev/check-links.mjs`; rebase after #1916 merges Squash-merge each, then rebase the next onto `main`. #1948 (broken) and #1949 (fixed) are the example PRs that exercise every check; never merge, close them once the stack has landed. --------- Co-authored-by: Amp <amp@ampcode.com>
marcleblanc2
added a commit
that referenced
this pull request
Sep 12, 2026
## Problem - When a PR's Vercel build fails, the Vercel bot comment and the `Vercel` check link to the Vercel inspector, which only Vercel team members can open - Contributors who aren't Vercel team members see a red X and a login wall, with no hint what broke - Example: #1913 ## Solution - On `vercel.deployment.error` (the same `repository_dispatch` channel `preview-links.yml` uses), comment the tail of the build log on the PR Vercel built the deployment for (`meta.githubPrId`), like the links / redirects / CSpell checks do. Each PR gets its own deployment, so two PRs at the same commit each get their own comment - On the next `vercel.deployment.success`, update that comment to ✅ on every open PR at the commit that has one. The success path only reads PR comments; it does not touch Vercel - Lines that look like credentials (JWTs, `vcp_` / `ghp_` / `github_pat_` / `AKIA…` tokens, `Bearer …`, `*TOKEN=` / `*SECRET=` / `*KEY=` assignments) are redacted before the log is written, since the comment and artifact are public and the build gets `VERCEL_OIDC_TOKEN`, `VERCEL_DEPLOYMENT_KEY` and `VERCEL_ENV_ENC_KEY`. The code fence is sized longer than any backtick run in the log, so a log line cannot close it and inject Markdown ## Token - `VERCEL_TOKEN` repo secret is a [project-scoped token](https://vercel.com/docs/accounts/access-tokens) for `sourcegraph-docs` only, expiring 2026-12-10. Vercel has no read-only scope; project scope is the narrowest it offers, and it can read and write everything in that one project - Enumerated with read-only calls: - Allowed: project settings, environment variables (`/env`, including `?decrypt=true`; the project has none), deployments list, deployment metadata, build log events, domains, custom environments, project list (returns only this project) - Denied: team, team members, user, log drains, webhooks, Edge Config, other projects, minting tokens - Writes within the project (env vars, deployments, domains, settings) are allowed per Vercel's docs; not exercised - `fetch-log` asks GitHub for an open PR from this repository at the commit before it contacts Vercel, so a dispatch for a fork PR or a stale commit never uses the token. `repository_dispatch` runs the workflow and script from `main`, so a PR can't change the code the token is handed to - Anyone with write access can read any repo secret by pushing a workflow; this token limits what that buys them to one Vercel project ## Slack - The Vercel Slack app already posts ":red_circle: … failed to deploy … `<short sha>` | sourcegraph-docs" to `#alerts-vercel-doc-site`. On a failure, a last step finds that post (looking back 30 minutes, then polling for up to 5 more since Vercel and this workflow are triggered by the same event) and uploads the full redacted log into its thread, linking the PR comment. `continue-on-error`, so a Slack problem can't hide the PR comment - Needs the `SLACK_BOT_TOKEN` repo secret and `SLACK_CHANNEL_ID` repo variable; skips quietly without them. The bot is the app in `dev/slack-app-vercel-build-report.json` (`channels:history` to find the post, `files:write` to reply); it must be `/invite`d to the channel - Not yet run end to end; the app and secret are still to be created ## Tested - Run locally against the two example PRs, which is how the comments there got posted (from my account, since GitHub neither delivers `repository_dispatch` nor resolves `workflow_dispatch` for a workflow that isn't on `main` yet): - #1948: ❌ [build log comment](#1948 (comment)) - #1949: ❌ posted on a broken revision, then updated to ✅ [after the fixed revision built](#1949 (comment)) - A PR with no failed build exits with `has no failed build to resolve` - That test found a bug: with two open PRs at the same head SHA, only the first PR the `commits/{sha}/pulls` API listed got the comment. Vercel records the PR a deployment was built for (`meta.githubPrId`), so `fetch-log` now reads it and the comment lands on that PR only. The success path stays off Vercel and keeps the commit lookup, since it only updates comments that already exist - Re-ran `fetch-log` after the reorder: #1948's head → `pull_request=1948`; a commit with no open PR stops at `No open PR with head …` with `VERCEL_TOKEN=invalid`, proving Vercel was not contacted. Redaction and fence sizing checked against JWT, `vcp_`, `ghp_`, `AKIA…`, `Bearer`, `KEY: value` lines and a log line of six backticks - After merge, re-run on a PR by hand: `gh workflow run vercel-build-report.yml -f id=dpl_... -f state=error -f sha=<pr head sha>` <details><summary>Dry-run output (abridged)</summary> > ### ❌ The Vercel build failed for this PR > > Vercel only shows build logs to members of its team, so here is the end of the log. > Run `npm run build` locally to reproduce. > > **Build log** > ``` > Running build in Cleveland, USA (East) – cle1 > ... > ❌ Found 2 dead link(s) in 1 file(s): > 📄 docs/code-search/features.mdx > Line 154: /code-search/no-such-page > ... > ❌ Failed checks: links, filenames > ELIFECYCLE Command failed with exit code 1. > Error: Command "pnpm run build" exited with 1 > ``` </details> ## Amp thread - [Vercel build failure report](https://ampcode.com/threads/T-01a09014-dfa8-740c-95b4-9e28c43cae51) <!-- pr-stack-merge-order --> ## Merge order for the PR-check stack Trial-merged onto `main` in this order with no conflicts: 1. #1946 Vercel build log comment — independent; first so the other PRs' Vercel failures get a readable log 2. #1916 check-links report format — adds `dev/sync-review-comments.sh`, which #1935 calls 3. #1935 redirect check — needs #1916 merged first 4. #1947 spell check comment updates — independent 5. #1944 check-links, generated-docs sync PR — conflicts with #1916 on `dev/check-links.mjs`; rebase after #1916 merges Squash-merge each, then rebase the next onto `main`. #1948 (broken) and #1949 (fixed) are the example PRs that exercise every check; never merge, close them once the stack has landed. --------- Co-authored-by: Amp <amp@ampcode.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Same commit as #1920, which stays broken for comparison. Here, a second commit applies exactly the fixes the checks suggested (check-links, check-redirects, and spell check review suggestions, including CSpell's wrong guess
arcos), and a third fixes what the checks could only report: the dead external link, missing page/heading links, the inbound anchor link, the deleted page and the page shadowed by a redirect,across, and the unsorted allow-list entry.Built on
mainplus the check PRs not merged yet: #1880, #1916, #1919.Replaces #1914. Do not merge.
Merge order for the PR-check stack
Trial-merged onto
mainin this order with no conflicts:dev/sync-review-comments.sh, which ci/redirects: Add Redirect check for PRs #1935 callsdev/check-links.mjs; rebase after check-links: one suggestion per fix, synced with findings; one fact per line in reports #1916 mergesSquash-merge each, then rebase the next onto
main.#1948 (broken) and #1949 (fixed) are the example PRs that exercise every check; never merge, close them once the stack has landed.