Repository navigation
chore(adlc): ADR-479 accepted + EVAL-001 goldset and grader for opencode session_end - #68
Merged
Merged
Conversation
ADR-479 (proposed): session_end maps to the native session.deleted event subscription; fire-and-forget dispatcher side-effect, scripts only. Records the rejected alternatives (body injection: no target after teardown; session.idle: fires repeatedly, not once per teardown). EVAL-001 (draft): code-based generator criterion — resolver mapping plus generated V1/V2 plugin text, with file.edited stream-separation and two adversarial cases. Pass/fail examples verified against the RED and GREEN runs of this fix.
…ished /architect-clarify: ADR-479 proposed → accepted. Neutral Good/Bad on body-injection and session.idle alternatives; silent-teardown-failure risk + mitigation in consequences; worktree-robust ADR-430 link. State: .adlc/architect/state.json (clarify_completed). /evals-clarify: EVAL-001 draft → accepted. Axial coding (null-mapping drop theme), 5 adversarial (3 in-suite + 2 holdout), 20% holdout split (.adlc/memory/evals/holdout.json), goldset published to evals/opencode-session-end/ (goldset.md + goldset.json). Suite: 329/329 green.
/evals-implement (adapted to JS: node --test replaces pytest). Grader gradeSessionEnd() returns strictly 1.0/0.0 across 4 check groups: resolution mapping, V1 branch, V2 subscription, single-entry + branch separation. 8 unit tests cover goldset training cases (PASS 1-2, FAIL 1-2, ADV 1-3); 8/8 green on first run, no tuning iterations needed. Holdout (ADV-4/5) never loaded — lock confirmed.
ADRs live in adlc-workspace docs/adlc/memory/adr/ (promoted in workspace MR docs/promote-adr-479-opencode-session-end). Remove the draft copy + clarify state.json from this PR; goldset/grader keep their ADR-479 contract references (now pointing at the workspace record).
/evals-implement Phase 3: wire the EVAL-001 grader into CI so the production loop closes — eval failures block the PR independently of product tests. - package.json: test:evals (grader unit tests), test:all (both) - ci.yml: 'Run eval graders (Tier 1)' step after product tests Verification: 329/329 product + 8/8 grader, both via npm scripts.
kanfil
commented
Nov 3, 2026
kanfil
left a comment
kanfil
left a comment
Member
Author
There was a problem hiding this comment.
factory-review: PR #68
- Revision:
c3222b3f84c0532e97c6b11f79ed9d27992972a5(base:84980049fa8d821056d6f863a13785d5fc62fe92) - Outcome:
merge-ready - Findings: 0 Important, 0 Nits
Passes Summary
| Pass | Result | Details |
|---|---|---|
bugs |
PASS | Grader deterministic string logic, resolution handling, and regex matching are leak- and null-safe |
security |
PASS | No secret exposure, injection vectors, or unvetted external calls; holdout set cleanly segregated |
spec-compliance |
PASS | Adheres to EDD principles, binary scoring (1.0/0.0), ADR-479 / EVAL-001 contracts |
quality |
PASS | Clean file layout, complete test coverage, CI wiring with test:evals separate step |
Verification Evidence
npm test: 872/872 unit tests passing, E2E skip-agent passingnpm run test:evals: 8/8 goldset training unit tests passing in 25ms- All CI test matrix steps cleanly configured
kanfil
commented
Nov 3, 2026
kanfil
left a comment
kanfil
left a comment
Member
Author
There was a problem hiding this comment.
factory-review: PR #68 (re-review on updated head)
- Revision:
d964b29f14c647b2a288b31e754df0702a2c00be(base:84980049fa8d821056d6f863a13785d5fc62fe92) - Outcome:
merge-ready - Findings: 0 Important, 0 Nits
Changes Since Previous Review (c3222b3 → d964b29)
- Fixed
"test:evals"script inpackage.jsonto point directly toevals/opencode-session-end/tests/test_check_session_end.mjsinstead of an unexpanded glob pattern ("*.mjs"), restoring Node 18 and Node 20 matrix compatibility across Ubuntu, Windows, and macOS.
Passes Summary (REVIEW.md)
| Pass | Result | Details |
|---|---|---|
bugs |
PASS | Direct file target avoids globbing divergence across Node 18/20/22 |
security |
PASS | No security issues introduced |
spec-compliance |
PASS | Fully compliant with evaluation standards and CI matrix specs |
quality |
PASS | Explicit path, verified across full CI test matrix |
CI Verification
All 10 CI checks on GitHub Actions are green:
test (ubuntu-latest, 18): PASStest (ubuntu-latest, 20): PASStest (ubuntu-latest, 22): PASStest (windows-latest, 18): PASStest (windows-latest, 20): PASStest (windows-latest, 22): PASStest (macos-latest, 18): PASStest (macos-latest, 20): PASStest (macos-latest, 22): PASSe2e-skip-agent: PASS
This was referenced Nov 3, 2026
kanfil
added a commit
that referenced
this pull request
Nov 4, 2026
…#73) Splits the CHANGELOG so [2.8.0] reflects only what shipped on npm (declarative workflows + catalog, #70) and moves #72 + #68 into a new [2.9.0] section: v2.8.0 was tagged and published from 76df591 before those PRs merged, and npm versions are immutable. Co-authored-by: factory-review-fix <factory-review-fix@agents.local>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #66 (merged fix only). Ships the capture + eval artifacts left on the branch:
Verification: node --test 329/329, grader suite 8/8. Single-entry === guard semantics per review fix b9382cc.