Skip to content

test: matrix scenarios - source route clean install and base-demo loop (Bash and Zsh) #2561

Open

Description

@codeforester

Goal

Automate the source-checkout half of the manual macOS validation checklist for both shells.

Background

The checklist defines the Source Checkout Install Path and the Base Dogfood Smoke Test with explicit acceptance conditions, but they are run by hand. Real shell profile updates and activation cannot run in ordinary CI because they mutate user-owned state; the matrix accounts exist for exactly that.

Next-release planning review of 2026-10-14/15 against main at 7faf6dfb. Tracking: #2555.

Prerequisites: #2559, #2560.

Scope

  • Scenario: bootstrap --source at the ref under test, basectl setup --profile dev, update-profile, fresh login shell, version, check, doctor, logs, projects list.
  • Scenario: base-demo setup, check, doctor, test, demo -- --non-interactive, activate and clean exit, and basectl test base.
  • Map each checklist step to a named scenario step and assert the documented acceptance conditions.

Acceptance Criteria

  • Both source cells pass on the v1.10.5 tag; the baseline result is recorded.
  • Both source cells pass on main at the time of merge.
  • A failing step names the owner category (Base bug, base-demo, environment) in the result.
  • Every checklist step in the source path maps to a scenario step.

Validation

# on the dev Mac, as admin:
<matrix runner> --cells source-bash,source-zsh --scenario source-clean-install

Attach the result JSON to the PR.

Non-Goals

  • No brew route (separate issue).
  • No new product behavior; failures become separate bug issues.

Project Fields

  • Status: Backlog
  • Priority: P1
  • Area: Setup
  • Initiative: Contract Hardening
  • Size: M
  • Milestone: v1.11.0
  • Category: ci

Agent Assignment

Assignee: codeforester. Blocked on #2559 and #2560. Then implementation-ready on the dev Mac.

Metadata

Metadata

Assignees

Labels

ciContinuous integration, tests, automation, or release workflows

Type

No type

Projects

  • Status
    Backlog

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions