Skip to content

Automated the Marketplace release behind a tag push and guarded what the VSIX ships - #38

Merged
dzykovic merged 3 commits into
mainfrom
release/automation
Sep 4, 2026
Merged

dzykovic merged 3 commits into
mainfrom
release/automation

Conversation

@dzykovic

@dzykovic dzykovic commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Risk & Size

Risk: High
Size: Large

What & Why

Releasing was a manual Marketplace upload: build locally, eyeball npx vsce ls, drag a VSIX
into the manage portal. This makes a vX.Y.Z tag push the whole release — guard, verify,
contents check, package, attest, GitHub Release, then the Marketplace — so the artifact users
install is always the one CI built from the tagged commit, and every gate runs before the
first publish step because Marketplace versions are write-once.

The pipeline is a port of the one proven in ainova-systems/markdown-review-comments, adapted
to this repo's esbuild build and 7-file package.

No spec accompanies this: it changes release plumbing, not extension behaviour — no new FR-0xx,
nothing in Features.md or Architecture.md to keep in step. That matches the precedent set by
26952e9, the comparable tooling change.

Changes

  • .github/workflows/release.yml — the pipeline, on a v* tag push or a manual dispatch
    with a tag. Marketplace and Open VSX steps are skipped when their secret is absent, so the run
    still produces a verified, attested GitHub Release and says in its job summary that the upload
    is manual. secrets is unavailable to an if: expression, so a step resolves presence into an
    output the publish steps gate on; each credential reaches only the step that needs it, never
    job-level env — which would expose it to the lifecycle scripts npm ci runs.
  • scripts/check-package-contents.mjs — asserts the VSIX ships exactly the expected seven
    files. It caught a real leak on its first run: scripts/ itself was on its way into the
    package, and .vscodeignore now excludes it. vsce honours .vscodeignore only, and this
    repo hides RELEASE_PLAN.md, .sandbox/, .tmp/ and the spec drafts through
    .git/info/exclude, which vsce does not read at all.
  • scripts/changelog-section.mjs — extracts one ## [x.y.z] section for the release notes,
    failing the release rather than publishing empty notes.
  • .github/workflows/ci.yml — adds the build step and the contents check, plus permissions
    and concurrency; actions move to current majors; the CI artifact is named -ci so it cannot
    be mistaken for a release build. No Node matrix — verify is tsc --noEmit plus an esbuild
    bundle and neither result can differ per Node version; the extension runs on the Node VS Code
    embeds anyway.
  • .github/workflows/codeql.yml, .github/dependabot.yml — static analysis on the sources
    that build sbx invocations and handle credentials, and monthly dependency/action updates with
    the one runtime dependency (yaml) in its own group.
  • docs/RELEASING.md — the runbook. Behaviour only: which secret gates which registry and
    what happens without it. Deliberately silent on how a token is minted or scoped.
  • intelligence/skills/vscode-publish-marketplace/SKILL.md — rewritten. It documented a
    human upload gate and asserted that no PAT is provisioned by design; both are now false and
    would have actively misled a future session. dev-project-profile.md › Releases updated to
    match (tagger: maintainer still holds — a person pushes the tag, CI does the rest).

Committed in two parts: e27ff8e is the intelligence CLI schema alignment (0.11.0 → 0.11.6) that
running the sync required, kept separate from the release work in 1725e4d.

How to Verify

  1. npm ci && npm run verify — green.
  2. npm run build && node scripts/check-package-contents.mjs — VSIX contents verified: 7 files, exactly as expected.
  3. node scripts/changelog-section.mjs 0.5.0 — prints the 0.5.0 notes;
    node scripts/changelog-section.mjs 9.9.9 exits 1.
  4. CI on this PR runs the same gates, including the contents check.

The pipeline itself is exercised for real by the first v0.6.0 tag. Do not dry-run it against
v0.5.0: the GitHub Release step would refresh the existing asset happily, but vsce publish
fails on a version already in the gallery and the run goes red for nothing.

VSCE_PAT is not set on this repository yet — until it is, a tag push produces the GitHub
Release and reports the Marketplace step as skipped.

…the VSIX ships

Pushing a `vX.Y.Z` tag now runs the full pipeline: the tag, `package.json` and
`CHANGELOG.md` must agree, `npm run verify` must be green, the VSIX must contain
exactly the expected files, and the package is attested before anything is
published. Only then does it create the GitHub Release and publish to the
Marketplace.

Both registry steps degrade gracefully. `secrets` is not available to an `if:`
expression, so a step resolves presence into an output the publish steps gate on,
and each credential is handed only to the step that needs it — never job-level
env, which would expose it to the lifecycle scripts `npm ci` runs. Without
`VSCE_PAT` the run still produces a verified, attested GitHub Release and says in
its job summary that the upload is manual.

`scripts/check-package-contents.mjs` asserts the VSIX ships exactly seven files.
It earned its place immediately: `vsce` honours `.vscodeignore` only, so the new
`scripts/` directory was on its way into the package until the check caught it.
This repository hides several paths through `.git/info/exclude`, which `vsce`
does not read at all, so the guard matters more here than the file count suggests.

CI gains the same contents check, plus `permissions` and `concurrency`, and its
actions move to current majors. No Node matrix: `verify` is a typecheck plus an
esbuild bundle, and neither result can differ per Node version.

`vscode-publish-marketplace` documented a manual upload gate and claimed no PAT
was provisioned by design; both are now false, so the skill and the profile's
release note describe the automated path instead.
Copilot AI lite review requested due to automatic review settings September 4, 2026 18:07
@github-advanced-security

Copy link
Copy Markdown

You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool.

What Enabling Code Scanning Means:

  • The 'Security' tab will display more code scanning analysis results (e.g., for the default branch).
  • Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results.
  • You will be able to see the analysis results for the pull request's branch on this overview once the scans have completed and the checks have passed.

For more information about GitHub Code Scanning, check out the documentation.

@dzykovic dzykovic added the ai:ready-to-merge CI green, threads answered, mergeable - awaiting the owner's accept label Sep 4, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A few fixable issues (shelling out for npx in the contents-check script, contradictory script comments, and redundant double-build in CI) should be addressed before merging.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Automates the VS Code Marketplace release process by moving publishing behind a vX.Y.Z tag-triggered GitHub Actions workflow, adding packaging/contents guards and release-note extraction scripts so the shipped VSIX is consistently built and verified in CI.

Changes:

  • Added a tag-driven release workflow that builds, verifies, checks VSIX contents, packages, attests provenance, creates/updates a GitHub Release, and conditionally publishes to registries based on secrets.
  • Added repo scripts to (1) enforce exact VSIX contents and (2) extract versioned release notes from CHANGELOG.md (failing the release if missing/empty).
  • Updated runbooks/intelligence docs and CI plumbing to match the new automated release policy, plus added CodeQL and Dependabot configuration.
File summaries
File Description
scripts/check-package-contents.mjs New script to assert VSIX ships exactly the expected file set.
scripts/changelog-section.mjs New script to extract a single version section from CHANGELOG for release notes.
.github/workflows/release.yml New tag-triggered release pipeline (verify, contents check, package, attest, GitHub Release, conditional publishes).
.github/workflows/ci.yml CI updated to run verify plus packaging/contents checks and upload a clearly named CI VSIX artifact.
.github/workflows/codeql.yml Adds CodeQL scanning for TypeScript security-and-quality queries.
.github/dependabot.yml Adds monthly Dependabot updates with grouped runtime/dev dependencies and actions.
.vscodeignore Excludes scripts/** to prevent packaging helper scripts into the VSIX.
docs/RELEASING.md New release runbook documenting tag-driven release flow and secret-gated publishing behavior.
CONTRIBUTING.md Documents new package-contents check and updated release process.
AGENTS.md Updates published skill descriptions and release policy snippet to reflect automated pipeline.
intelligence/skills/vscode-publish-marketplace/SKILL.md Rewrites skill to align with tag-driven automation and new pre-tag gates.
intelligence/rules/dev-project-profile.md Updates release policy description to reflect automated publishing behind maintainer tag push.
intelligence.yaml Bumps intelligence schema_version to 0.11.6.
intelligence.lock Updates intelligence engine/package resolution to 0.11.6.
Review details
  • Files reviewed: 13/14 changed files
  • Comments generated: 3
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread scripts/check-package-contents.mjs
Comment thread .github/workflows/ci.yml Outdated
Comment thread scripts/check-package-contents.mjs Outdated
…comments

`npm run verify` is `tsc --noEmit && node esbuild.js`, so it already produces the
bundle the contents check needs — the separate build step re-bundled for nothing
and left CI doing something the release workflow does not. The runbook and the
publish skill described the same redundant pair, so they move with it.

The comment about filtering `vsce ls` output contradicted the one above it: the
file correctly says `vsce ls` does not run `vscode:prepublish`, then claimed to
be stripping that script's output. It filters non-path noise; say so.

Recorded why `shell: true` is deliberate: on Windows `npx` is `npx.cmd`, which
Node will not resolve without a shell — spawning it with explicit args fails
ENOENT — and the command is a literal with nothing interpolated into it.
@dzykovic
dzykovic merged commit eb16d90 into main Sep 4, 2026
3 checks passed
@dzykovic
dzykovic deleted the release/automation branch September 4, 2026 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai:ready-to-merge CI green, threads answered, mergeable - awaiting the owner's accept

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants