eval: add discoverable DeltaWire v7 pair boundary #1
Workflow file for this run
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: DeltaWire v7 no-live validation | |
| on: | |
| push: | |
| branches: [eval/deltawire-v7] | |
| paths: | |
| - "labs/20-deltawire/**" | |
| - ".github/workflows/deltawire-v7.yml" | |
| pull_request: | |
| paths: | |
| - "labs/20-deltawire/**" | |
| - ".github/workflows/deltawire-v7.yml" | |
| permissions: | |
| contents: read | |
| jobs: | |
| validate-v7: | |
| runs-on: ubuntu-latest | |
| steps: | |
| - uses: actions/checkout@v7 | |
| with: | |
| fetch-depth: 0 | |
| - uses: actions/setup-go@v6 | |
| with: | |
| go-version-file: labs/20-deltawire/go.mod | |
| cache-dependency-path: labs/20-deltawire/go.sum | |
| - uses: actions/setup-python@v6 | |
| with: | |
| python-version: "3.12" | |
| - uses: astral-sh/setup-uv@v7 | |
| - name: Build and stage the frozen Linux binary | |
| run: | | |
| cd labs/20-deltawire | |
| mkdir -p eval/.generated eval/pairs/range-large-v7-r1/environment/.generated | |
| GOWORK=off CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -o eval/.generated/deltawire-linux-amd64 ./cmd/deltawire | |
| cp eval/.generated/deltawire-linux-amd64 eval/pairs/range-large-v7-r1/environment/.generated/deltawire | |
| test "$(sha256sum eval/.generated/deltawire-linux-amd64 | cut -d' ' -f1)" = e5198d15000e093a2a28e57ad4a093dde8f66bcab21462549ddf9114064c25f4 | |
| - name: Validate skill examples using the actual Linux binary | |
| run: | | |
| python3 labs/20-deltawire/eval/scripts/v7/skill_conformance.py \ | |
| --skill labs/20-deltawire/eval/pairs/range-large-v7-r1/skill/deltawire \ | |
| --binary labs/20-deltawire/eval/.generated/deltawire-linux-amd64 \ | |
| --report /tmp/skill-conformance.json | |
| if test -f labs/20-deltawire/eval/results/preflight-v7/skill-conformance.json; then | |
| cmp /tmp/skill-conformance.json labs/20-deltawire/eval/results/preflight-v7/skill-conformance.json | |
| fi | |
| - name: No-live v7 validation | |
| run: bash labs/20-deltawire/eval/scripts/v7/validate.sh | |
| - name: Prove workflow has no model invocation | |
| run: | | |
| ! grep -E 'runner.py .*pair-v7|harbor run|GEMINI_API_KEY' .github/workflows/deltawire-v7.yml |