ci: gate on relay interop's real exit code, rename job to relay-conformance - #171
Merged
Conversation
…rmance Replaces `output=$(relay interop ... 2>&1) || true` + a brittle grep of stdout with a plain `relay interop --strict --protocol RCP target/release/rust-rcp` invocation whose real exit code gates the step directly -- matching go-RCP's existing clean pattern for the same check. The old form silently tolerated both a genuinely broken run and any future change to relay interop's output format, since `|| true` swallowed the real exit code before the grep ever ran. Also renames the job from `relay-interop`/"RELAY interop" to `relay-conformance`/"RELAY conformance": this step only checks self-consistency against RELAY's own embedded golden vector, not a third-party reference implementation, so "interop" overstates what it actually verifies. Confirmed nothing else in this workflow (or, per GitHub's branch-protection API, this repo's required-status-checks list) references the job by its old name, so the rename is safe. Both fixes verified locally against the exact relay v2.0.4 build this job installs (matching the pinned version): a real rust-rcp binary gets RESULT: PASS / exit 0, and mutation-testing against a deliberately broken binary confirms RESULT: FAIL / exit 1 -- the previous `|| true` form would have swallowed that failure. SoundMatt/RELAY THEME-K (#125) / INTEROP-04 (#149). Signed-off-by: Matt <47545907+SoundMatt@users.noreply.github.com>
…ting relay-conformance job CI failed immediately on push: the previous commit renamed this job to relay-conformance, colliding with an already-existing job of the same id (RELAY spec v2.0 unit conformance). Duplicate YAML mapping keys are technically valid per the YAML spec (last one wins) so python's yaml.safe_load silently accepted it and my earlier check didn't catch the collision -- GitHub Actions' own schema validation correctly rejected the workflow file outright. Renamed to relay-cli-conformance / "RELAY CLI conformance" instead, distinguishing it from the existing relay-conformance job. Verified with a proper duplicate-key detector (not yaml.safe_load) that no other job id collides. Signed-off-by: Matt <47545907+SoundMatt@users.noreply.github.com>
SoundMatt
added a commit
to SoundMatt/rust-LIN
that referenced
this pull request
Aug 21, 2026
…tdout (#43) Replaces `interop_out=$(relay interop ... 2>&1) || true` + a grep of stdout for ERROR/FAIL with a plain `relay interop --strict --protocol LIN ./target/release/rust-lin` invocation whose real exit code gates the step directly. The old form silently tolerated both a genuinely broken run and any future change to relay interop's output format, since `|| true` swallowed the real exit code before the grep ever ran. --protocol LIN restricts the check to rust-lin's own protocol and --strict fails on a missing convert rather than treating it as an innocuous skip -- matching the pattern already fixed in rust-RCP (SoundMatt/rust-RCP#171) and go-RCP/go-LIN's existing clean form. The surrounding job is already correctly named 'conformance' (not 'interop'), so no rename needed here, unlike rust-RCP's fix. Verified locally against the exact relay v2.0.4 build this job installs: a real rust-lin binary produces RESULT: PASS / exit 0; mutation-tested against a deliberately broken binary, confirmed RESULT: FAIL / exit 1 (the previous || true form would have swallowed this). Also confirmed no job-id collisions exist in this workflow (5 unique job ids, none duplicated) before pushing, after the job-collision bug found while fixing rust-RCP's identical pattern. SoundMatt/RELAY THEME-K (#125) / INTEROP-04 (#149). Signed-off-by: Matt <47545907+SoundMatt@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the exit-code-swallowing half of SoundMatt/RELAY's INTEROP-04
(#149), against the pattern defined in
docs/INTEROP-ARCHITECTURE.md(THEME-K, SoundMatt/RELAY#125).
What changed
output=$(relay interop ... 2>&1) || true+ a brittle grepof stdout with a plain
relay interop --strict --protocol RCP target/release/rust-rcpinvocation whose real exit code gates thestep — matching go-RCP's existing clean pattern for the identical
check. The old form silently tolerated both a genuinely broken run
and any future change to
relay interop's output format, since|| trueswallowed the real exit code before the grep ever ran.relay-interop/"RELAY interop" →relay-conformance/"RELAY conformance": this step only checksself-consistency against RELAY's own embedded golden vector, not a
third-party reference implementation, so "interop" overstated what
it verifies. Confirmed via GitHub's branch-protection API that this
repo's required-status-checks list doesn't reference the job by its
old name, so the rename is safe.
Verification
cargo build --release --locked/cargo test --release --lockedclean (pure CI YAML change, no Rust source touched)
relay v2.0.4buildthis job installs: a real
rust-rcpbinary producesRESULT: PASS/ exit 0; mutation-tested against a deliberately broken binary,
confirmed
RESULT: FAIL/ exit 1 (the previous|| trueform wouldhave swallowed this)