[metalwarrior665]

Check #notif-testing-repo-for-github-actions channel to see live test of this branch.

AI Summary

This PR refactors the test reporting and notification system by splitting the monolithic report-tests command into two focused commands: test (which runs vitest and produces a report) and notify (which sends notifications for both test results and releases).

Key Changes

• New test command (bin/run-tests.ts): Runs the repository's vitest suite on specified builds or all Actors, producing a structured TestReport that replaces the previous JSON parsing approach. Handles vitest binary resolution from the repository's node_modules and manages parallelism settings.

• New notify command (bin/notify.ts): Unified notification system that handles both test reports and release announcements. Replaces the separate report-tests and notify-release commands with a single entry point that reads different input types via stdin.

• Refactored test-report.ts: Converted from a reporting script into a data structure and conversion module. Defines TestReport, TestCaseReport, and TestRun interfaces that serve as the contract between test and notify. Includes toTestReport() to convert vitest's JSON output to the standardized format.

• Updated stdin-input.ts: Added parsers for test builds (parseTestBuilds) and unified notify input (parseNotifyInput), replacing the previous parseReleaseChanges. Validates that flags match the input type (e.g., --changelog-only only works with releases).

• Simplified slack.ts: Removed release-specific notification logic; now only provides low-level Slack client utilities (getSlackClient, sendSlackMessage, sendingLabel).

• Updated CLI entry point (bin/main.ts): Registered new test and notify commands with their respective options. Removed report-tests and notify-release commands.

• Updated workflows: Modified GitHub Actions workflows to use the new test and notify commands instead of the old report-tests and notify-release.

Notable Implementation Details

• The test command resolves vitest from the repository's node_modules to ensure tests use the locked version, not the bundled CLI's version.
• notify creates notification targets (currently Slack only) only when a message may be sent, so broken tokens are caught early but unnecessary token checks are avoided.
• The TestReport format is versioned (v1) to allow future evolution while maintaining backward compatibility.
• Test failures are reported as a structured report rather than directly posting to Slack, enabling multiple notification targets and better testability.
• Input validation ensures flags are only used with compatible input types, preventing silent misconfigurations.

https://claude.ai/code/session_0149HKZkpP35zMLDjVKKf6Cm

[metalwarrior665]

Merged both Slack tokens to this one. I already added SLACK_BOT_TOKEN to our orgs

[metalwarrior665]

Keeping the channel fallback now before teams add the workflow inputs. We remove the fallback before releasing this to community.

[metalwarrior665]

This file should be 1:1 with current code, just with few small UX improvements

[metalwarrior665]

Commit messages could spam a page of text before

[metalwarrior665]

Commits now contain links to them

[metalwarrior665]

No more 2 channels, #delivery-public-actors is dead

[JuanGalilea]

bad name, on the workflow I thought it meant test all actors (which made no sense since you passed a blob) so i went looking for this definition.
Why not --latest or --default-build

[JuanGalilea]

docs link would be useful here.
Also mention the ones set by default (default and json)

[JuanGalilea]

being this positional and defined first, does it even collide with --reporter and its fake array?

[JuanGalilea]

nit: ternary for "enum" means we'll have to restructure the whole thing if we ever get a different use case (like test summaries).
Did a snippet of how it could look on a modification-safe way:

const misfits: string[] = [];
let stdinInputDescriptor: string;
switch (input.kind) {
    case 'tests':
        stdinInputDescriptor = 'a test report (the output of test)';
        if (changelogOnly) misfits.push('--changelog-only');
        if (repoUrl) misfits.push('--repo-url');
        break;
    case 'release':
        stdinInputDescriptor = 'a release (the output of get-affected-actors)';
        if (on !== 'failure') misfits.push('--on');
        if (title) misfits.push('--title');
        if (jobUrl) misfits.push('--job-url');
        break;
    default:
        throw unreachable(input);
}
if (misfits.length > 0) {
    throw new Error(`${misfits.join(', ')} can't be used with ${stdinInputDescriptor} on stdin`);
}

Probably overkill tho

[JuanGalilea]

did a nit comment about input.kind and its possible modification on assertFlagsFitInput.
Looking at the file as a whole, this seems like a place for strategy pattern (instead of that switch) since we're bound to have split behavior for all different types of it

[JuanGalilea]

branding our outputs (as in with a kind prop) would make our lives easier. (not to be solved here)

[JuanGalilea]

Checking for it being an object at least helps with the has. An unknown is pretty much useless

Suggested change

	const json = parseStdinJson(raw, z.unknown(), { expected, example });
	const json = parseStdinJson(raw, z.looseObject({}), { expected, example });