fix(review): bind verified evidence in scripts/ci/opencode_review_normalize_output.py#563
fix(review): bind verified evidence in scripts/ci/opencode_review_normalize_output.py#563seonghobae wants to merge 8 commits into
Conversation
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head4ff50eca109bb1410275e024c92fef348f278147. -
Head SHA:
4ff50eca109bb1410275e024c92fef348f278147 -
Workflow run: 29404144238
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["CI script: opencode_review_normalize_output.py"]
S1 --> I1["review and security gate shell path"]
I1 --> R1["Review risk: CI script: opencode_review_normalize_output.py"]
R1 --> V1["bash -n plus Strix self-test"]
Evidence --> S2["Test (2 files)"]
S2 --> I2["regression suite"]
I2 --> R2["Review risk: Test (2 files)"]
R2 --> V2["targeted test run"]
OpenCode Review Overview
Pull request overviewOpenCode cannot approve yet because required coverage evidence did not pass. Review outcome1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
Coverage evidenceCoverage Decision
Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script (2 files)"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script (2 files)"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test: test_opencode_model_pool_runner.py"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test: test_opencode_model_pool_runner.py"]
R3 --> V3["targeted test run"]
|
seonghobae
changed the title
seonghobae
changed the title
681447e to
0042e90
Compare
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head0042e9079990d5478294a369200101d88a2df79b. -
Head SHA:
0042e9079990d5478294a369200101d88a2df79b -
Workflow run: 29538997259
-
Workflow attempt: 1
Coverage evidence
Coverage evidence job did not run or did not publish coverage evidence.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["CI script: opencode_review_normalize_output.py"]
S1 --> I1["review and security gate shell path"]
I1 --> R1["Review risk: CI script: opencode_review_normalize_output.py"]
R1 --> V1["bash -n plus Strix self-test"]
Evidence --> S2["Test: test_opencode_model_pool_runner.py"]
S2 --> I2["regression suite"]
I2 --> R2["Review risk: Test: test_opencode_model_pool_runner.py"]
R2 --> V2["targeted test run"]
0042e90 to
099f538
Compare
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head099f538b7503288325077595a11eb1a7d8f016b4. -
Head SHA:
099f538b7503288325077595a11eb1a7d8f016b4 -
Workflow run: 29553130324
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 3
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["CI script: opencode_review_normalize_output.py"]
S1 --> I1["review and security gate shell path"]
I1 --> R1["Review risk: CI script: opencode_review_normalize_output.py"]
R1 --> V1["bash -n plus Strix self-test"]
Evidence --> S2["Test: test_opencode_model_pool_runner.py"]
S2 --> I2["regression suite"]
I2 --> R2["Review risk: Test: test_opencode_model_pool_runner.py"]
R2 --> V2["targeted test run"]
099f538 to
196c5a4
Compare
There was a problem hiding this comment.
Pull request overview
OpenCode could not approve from deterministic current-head evidence because GitHub Checks have failed.
Findings
1. HIGH Current-head GitHub Checks - Fix failed required checks before approval
- Problem: Failed same-head checks remain for
196c5a4c76dcbde28a374c55e975f9b45c74fc5a. - Root cause: The model-unavailable evidence fallback is allowed only when peer GitHub Checks are complete and clean.
- Fix: Read and fix the failed check logs below, then rerun the current-head checks.
- Regression test: Keep the model-unavailable fallback gated on an empty failed-check rollup.
Failed checks:
- scan-pr-queue check run: cancelled (https://github.com/ContextualWisdomLab/.github/actions/runs/29706612581/job/88244598475)
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["CI script: opencode_review_normalize_output.py"]
S1 --> I1["review and security gate shell path"]
I1 --> R1["Review risk: CI script: opencode_review_normalize_output.py"]
R1 --> V1["bash -n plus Strix self-test"]
Evidence --> S2["Test: test_opencode_model_pool_runner.py"]
S2 --> I2["regression suite"]
I2 --> R2["Review risk: Test: test_opencode_model_pool_runner.py"]
R2 --> V2["targeted test run"]
Superseded by current head 3b2255b, which adds the regression-tested REST fallback filter for the cancelled scheduler check cited by this old-head review.
There was a problem hiding this comment.
Pull request overview
OpenCode cannot approve yet because required coverage evidence did not pass.
Review outcome
1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence
-
Problem: The required coverage-evidence job result was
failure, so OpenCode cannot establish approval sufficiency for this head. -
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
-
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports
successwith required evidence or explicit no-source not-applicable evidence. -
Regression test: Keep the approval branch checking
needs.coverage-evidence.result == successbefore posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present. -
Result: REQUEST_CHANGES
-
Reason: coverage-evidence result was
failure, so required test/docstring evidence was not proven for current head55219f0b317a63a5effab6e19e38fc5e2ab639cf. -
Head SHA:
55219f0b317a63a5effab6e19e38fc5e2ab639cf -
Workflow run: 29728911432
-
Workflow attempt: 1
Coverage evidence
Coverage Decision
- Result: FAIL
- Test evidence: not proven passing
- Docstring evidence: not proven passing when configured
- Failure count: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script (2 files)"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script (2 files)"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test: test_opencode_model_pool_runner.py"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test: test_opencode_model_pool_runner.py"]
R3 --> V3["targeted test run"]
중복 구현 감사 결과current HEAD
#560의 current-head OpenCode review 실행 29745643580이 현재 진행 중입니다. 기능 유실을 막기 위해 #560이 병합될 때까지 이 PR은 열어 두고, 병합 직후 중복 PR로 정리하겠습니다. 오래된 |
1 participant
자동 리뷰 증거 계약
Purpose
This pull request repairs three fail-closed review-process edge cases observed on the protected base:
pathand positivelinewith one matchingsource-line-sha256receipt while omitting only the redundantpath:linetext in free-form evidence.evidencewhile copying a different valid changed-file location into the redundant structured fields.scan-pr-queuescheduler replacement even though the typed check-rollup path already ignores that scheduler cancellation.The normalizer now restores the missing redundant citation, or rebinds the structured location only when the evidence contains exactly one distinct current-head changed-file citation whose recomputed sealed line digest matches the single receipt. The REST fallback ignores only cancelled
scan-pr-queue; real failures, timeouts, action-required states, startup failures, and unrelated cancellations remain blockers.Fail-closed boundaries
Current HEAD evidence
90f4c5171b6d6e4906b792100547ffef30b2cc5a.github/workflows/opencode-review.ymlscripts/ci/opencode_review_normalize_output.pyscripts/ci/test_strix_quick_gate.shtests/test_opencode_model_pool_runner.pypython3 -m pytest -q: 639 passedruff check .: passedgit diff --check: passedReview run
29728911432correctly failed closed when the first version left two new guard branches uncovered. This head adds direct missing-receipt and invalid-line regressions; the exact sandbox command now reports 639 passed and 100% coverage. Earlier exact-head review run29712237568reached GPT-4.1 successfully but rejected its otherwise structured response because the evidence citation and redundant structured location differed. Other pool attempts ended in timeout, quota, context, authentication, or provider errors. This change addresses only that safely provable binding drift and keeps all ambiguous cases closed.Verified current-head source traces
scripts/ci/opencode_review_normalize_output.py:686observed comparison of a cited line's recomputed digest with the single receipt.source-line-sha256=749d101c3c3a59a42cc50783eddd72e1d5fef3a9f510feaa47091dee5d660ab6scripts/ci/opencode_review_normalize_output.py:688observed the exactly-one-distinct-location guard.source-line-sha256=56d7c2581137da6e2720fa3bd9d5e6e070f3944a4a7481e758cd6b1f516aa5cbscripts/ci/opencode_review_normalize_output.py:750observed structured-field rebinding through the receipt-verified locator.source-line-sha256=26801fb3ba1c564f5e5feec60c2e504e48068cc16aeacc09829960432ffa9c98tests/test_opencode_model_pool_runner.py:192observed the unique cited line replacing the drifted structured line.source-line-sha256=dd19cfb3e654e798e96f1f770cbd0b8a4c5c18dc58e88553a3614c16834e8e01tests/test_opencode_model_pool_runner.py:220observed identical multi-location evidence remaining unmodified and rejected.source-line-sha256=4088b2e8c04ae361be179c67b641283833fb799062d70ae8e7631c674617ccaa.github/workflows/opencode-review.yml:6121observed the narrowly scoped workflow-less scheduler cancellation filter.source-line-sha256=6c45a4ae7b52a024b440370824958af8c43445b1d5f610efbab63025810a6037scripts/ci/test_strix_quick_gate.sh:1017observed the matching static workflow contract.source-line-sha256=4050a1b3da1248f6e5f40262a6c81a241f3b491deb50976eaecf9dbc1c467eefExpected review behavior
The protected-base normalizer may repair only redundant binding drift that is independently and uniquely proven by the current-head changed-file manifest, sealed source tree, exact line bytes, and one source-line receipt. It must continue to reject every missing, unsafe, ambiguous, mismatched, circular, or unobserved case. The approval gate should ignore only the superseded scheduler cancellation and continue to surface any real failed peer check.