test(cli): add release smoke tests#2861
Open
sjinks wants to merge 1 commit into
Open
Conversation
Contributor
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a credential-free “release smoke test” harness intended to exercise VIP CLI’s highest-risk argument parsing / dispatch paths against built dist/bin/* entrypoints, plus docs and npm script wiring to run it before release.
Changes:
- Added
helpers/release-smoke-test.jsscript to execute a set of safe--help-based smoke cases againstdist/bincommands. - Added Jest coverage for the helper’s validation/execution/redaction helpers.
- Documented the new smoke test flow in
docs/TESTING.mdanddocs/RELEASING.md, and exposed it vianpm run smoke:release.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| package.json | Adds smoke:release npm script entry. |
| helpers/release-smoke-test.js | Implements the release smoke test runner, case validation, and failure logging/redaction. |
| tests/helpers/release-smoke-test.js | Adds Jest tests for validation/execution behavior and credential redaction. |
| docs/TESTING.md | Documents running the release smoke tests before release. |
| docs/RELEASING.md | Adds release checklist step to run the smoke tests. |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
sjinks
added a commit
that referenced
this pull request
May 29, 2026
… feedback ## Purpose and Context Addresses PR #2861 review feedback and adversarial review findings on the release smoke test helper. Two primary goals: (1) Eliminate code duplication that caused SonarCloud to fail the quality gate at 3.5% (threshold ≤ 3%). (2) Add assertions that verify the security-critical properties of the spawnSync call site — previously no test confirmed shell:false, nodeExec, or the credential-exclusion env allowlist. ## Key Changes - __tests__/helpers/release-smoke-test.js: - Fix misleading jest.mock comments ("before requiring the helper" was a carry-over from CommonJS; jest.mock is hoisted before imports) - Add beforeEach in safe-flag validation to share common mock setup across six acceptance tests; remove inline duplication - Add beforeEach in binary-file-system validation to share existsSync and lstatSync defaults; remove inline duplication from several tests - Add assertion: spawnSync called with process.execPath, shell:false, and env containing NODE_ENV=test and DO_NOT_TRACK=1 - Add assertion: WPVIP_DEPLOY_TOKEN and NPM_TOKEN are not forwarded to child processes even if set in the host environment - Add tests: exitCode validation rejects strings and non-integers - helpers/release-smoke-test.js: - Add exitCode type and integer validation in validateExpectationFields - Add comment explaining HOME/USERPROFILE/APPDATA forwarding rationale and the auth short-circuit dependency for stricter CI isolation - Add comment documenting the credential adjacency edge-case limitation in formatCommandForLogging - docs/RELEASING.md: - Add explicit npm run build prerequisite to the smoke test checklist ## Testing and Validation 87 Jest tests pass (0 failures). ESLint and tsc pass.
## Purpose and Context Release preparation needs a credential-free smoke check for CLI parser and dispatch paths that are easy to regress but expensive to catch during a manual release. The helper exercises built `dist/bin` commands using safe non-executing forms so release operators can verify high-risk command routing before publishing. ## Key Changes - Add a release smoke test helper for `logs`, `slowlogs`, `wp`, and `dev-env shell` parser and dispatch paths. - Add validation for safe arguments, built binary paths, required child binaries, expected output fields, and symlink-safe filesystem checks. - Run smoke cases with `shell: false`, an explicit environment allowlist, isolated temporary credential paths, timeout handling, and credential redaction in failure output. - Add Jest coverage for validation failures, spawn behavior, credential redaction, cleanup handling, and platform-portable path assertions. - Document the release smoke workflow and add the `smoke:release` npm script. ## Impact and Considerations The smoke runner targets built release artifacts and requires `npm run build` before `npm run smoke:release`. It avoids API, Docker, and Lando side effects by using `--help` or equivalent non-executing argument forms. Child processes do not inherit host home/profile credential paths by default; unexpected credential lookups land in a temporary scope that is removed after each case. ## Testing and Validation Added focused unit coverage for the release smoke helper. During PR validation, the focused helper Jest suite passed, ESLint passed, TypeScript type-checking passed, scoped diff whitespace checks passed, and SonarCloud reported the Quality Gate passed with 0.0% duplication on new code. Refs PLTFRM-2356
sjinks
force-pushed
the
pltfrm-2356-add-release-smoke-script-for-high-risk-commands-logs
branch
from
bd82b73 to
2689c35
Compare
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds a release smoke test script for high-risk VIP-CLI parser and dispatch paths. The script runs credential-free checks against built
dist/bincommands forlogs,slowlogs,wp, anddev-env shell, with safeguards for safe process execution, filtered environment variables, symlink rejection, and credential redaction in failure output.Related tracking: PLTFRM-2356.
Changelog Description
Added
Pull request checklist
New release checklist
Steps to Test
npm run build.npm run smoke:release.npx jest __tests__/helpers/release-smoke-test.js --no-coverage.