Skip to content

Strengthen PR test trust baseline#16

Draft
Jason-hub-star wants to merge 49 commits into
masterfrom
codex/test-trust-baseline
Draft

Strengthen PR test trust baseline#16
Jason-hub-star wants to merge 49 commits into
masterfrom
codex/test-trust-baseline

Conversation

@Jason-hub-star

@Jason-hub-star Jason-hub-star commented Jun 2, 2026

Copy link
Copy Markdown
Owner

Summary

  • stabilize the FlightLog until-date robustness test and status smart-wait path, and guard public docs so the resolved FlightLog date/time boundary regression is not re-advertised as active flaky work
  • require regression issue links when a bug-fix PR cannot include reproduction coverage immediately
  • guard PR .NET CI invariants so the public gate keeps its pull_request trigger, three-OS matrix, fail-fast: false, and no continue-on-error
  • guard PR .NET CI against path/event narrowing so paths, paths-ignore, branches-ignore, types, or pull_request_target cannot silently shrink the "all PRs" gate
  • guard CI gate ordering so Shared/Core/Cli/Mcp tests must run before publish, pack, or published/installed CLI smoke steps
  • preserve running Unity project path case policy in both editor discovery and process matching: Windows collapses case-only paths after normalization, while Linux/macOS keep case-distinct project paths separate
  • preserve slash/backslash process matching through normalized paths without weakening Unix case sensitivity
  • add Plugin IPC pipe-name helper guardrails so Shared Constants.GetPipeName() and Unity Plugin PipeNameHelper keep the same path normalization, Windows case policy, SHA-256 hash, prefix, and 16-hex suffix rules
  • clarify the flaky-test template so resolved regressions are not advertised as active flaky examples
  • add command/catalog/CLI/MCP/plugin guardrails, including Plugin shared copy drift checks for WellKnownCommands, wire DTO fields, StatusCode, Exec parser sentinels, duplicate CLI/Plugin command registrations, catalog-local CLI exceptions, and catalog WellKnown command MCP reachability exceptions
  • add MCP black-box schema parity guards so unityctl_schema must return the complete CommandCatalog command set, category-filtered calls must return only matching catalog commands, and command lookups must return the catalog definition for both canonical command names and CLI aliases
  • harden PR/release workflows so Shared/Core/Cli/Mcp tests gate packaging and publishing
  • expand published and installed CLI smoke coverage for dotnet tool install, tools --json, schema, doctor, check, and workflow verify
  • strengthen workflow verify CLI smoke so published and installed tools must report the requested artifactsDirectory and preserve the validate / projectValidate step evidence
  • guard Unity Integration evidence handling so license preflight writes license-preflight.txt / planned-smoke.txt before GameCI and actions/upload-artifact runs with if: always() after preflight
  • keep Unity live validation manual/nightly only, with a guardrail against reintroducing tag-push triggers that could block release tags on Unity secrets
  • require the Unity live smoke doctor --json step to return a successful CommandResponse, and guard against reintroducing doctor.json || true
  • add platform/readiness regressions for pipe path normalization, slash/backslash/case policy handling, Unity discovery, headless/batch locks, IPC timeout guidance, dirty scene policy, and parser edges
  • update README/docs trust signals to 863 PR .NET tests and document the Unity Integration split
  • preserve Unity Integration preflight evidence with both license-preflight.txt and planned-smoke.txt when UNITY_LICENSE / UNITY_SERIAL secrets are missing
  • document the exact Unity live-gate rerun path after repository Actions secrets are configured

Local verification

  • dotnet test tests/Unityctl.Shared.Tests -c Release /p:UseSharedCompilation=false --filter PublishedCliSmoke_CoversReadmeEntryPoints - 1 passed
  • local dotnet run --project src/Unityctl.Cli --no-build -c Release -- workflow verify ... --artifacts-dir <tmp>/artifacts --json - exit 1 as expected without live Unity, but preserved artifactsDirectory and the validate / projectValidate step evidence
  • dotnet test tests/Unityctl.Shared.Tests -c Release /p:UseSharedCompilation=false - 107 passed
  • dotnet test tests/Unityctl.Core.Tests -c Release /p:UseSharedCompilation=false - 152 passed
  • dotnet test tests/Unityctl.Cli.Tests -c Release /p:UseSharedCompilation=false - 579 passed
  • dotnet test tests/Unityctl.Mcp.Tests -c Release /p:UseSharedCompilation=false - 25 passed
  • git diff --check

Remote verification

  • CI - dotnet run 26807365824 passed on Ubuntu, macOS, and Windows for head d515e19.
  • GitGuardian security check passed for head d515e19.

Unity Reality Check

  • Unity live blocker tracking issue: Configure Unity Integration Actions secret #17.
  • Manual CI - Unity Integration run 26803038419 was triggered on head a4dea73.
  • Live Unity validation is still blocked because neither UNITY_LICENSE nor UNITY_SERIAL is configured; gh secret list --repo Jason-hub-star/unityctl currently shows only NUGET_API_KEY.
  • Both matrix jobs uploaded preflight artifacts successfully:
    • unityctl-live-2021.3.11f1/license-preflight.txt
    • unityctl-live-2021.3.11f1/planned-smoke.txt
    • unityctl-live-6000.0.64f1/license-preflight.txt
    • unityctl-live-6000.0.64f1/planned-smoke.txt
  • Downloaded artifacts from run 26803038419 confirm planned-smoke.txt preserves intended coverage: init, doctor, check, scene hierarchy, player-settings set/get readback, and workflow verify projectValidate.
  • The latest workflow guard keeps live validation on workflow_dispatch and schedule only, and requires doctor.json to be a successful response after the license gate is satisfied.
  • After adding UNITY_LICENSE or UNITY_SERIAL, rerun with gh workflow run ci-unity.yml --ref codex/test-trust-baseline, watch with gh run watch <run-id> --exit-status, and download artifacts with gh run download <run-id> --dir <artifact-dir>.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant