Repository navigation
feat(skills): add diagnostic traces and guarded coding experiments - #130
Merged
Merged
Conversation
Capture scrubbed action/observation artifacts independently of the bounded reviewer digest. Link optional native review traces to learned proposals without treating provider output as task verification. Evaluate exact pending skill bytes against installed baselines in fresh fixture workspaces. Freeze suite and interpreter fingerprints, run a separate verifier, preserve every replay success and consume final tests once. Retain immutable candidates and advisory reports for human review. Check operator-supplied answer markers, absolute size and growth before execution. Bind attached results to proposal and baseline hashes; changed or damaged evidence cannot appear current. Keep model execution and automatic promotion out of this initial workflow.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
LORE’s bounded review digest drops successful tool observations, and pending skills have no reproducible task-level comparison with their installed baseline.
Add opt-in local diagnostic traces for Claude and Codex, then an explicit human-terminal evaluation workflow for pending coding skills. Freeze candidate/baseline bytes, fixture hashes, runner/verifier inputs and the interpreter fingerprint. Require training improvement, validation non-regression and preservation of every successful replay case before consuming final tests once.
Retain immutable candidates and advisory reports. Check copied operator-supplied answer markers, skill size and growth before execution; show stale or damaged evidence during proposal review. Exact-byte human approval remains the installation authority.
The included deterministic coding fixture tests the evaluator contract. It does not demonstrate model improvement. Suites are trusted local scripts, with clean environment, deadlines and output limits; this initial workflow supplies neither a sandbox nor a model adapter.
Validation: final committed-tree rerun passed all 213 native Rust tests, native launcher checks and all 41 Python test files. Linux native, macOS compile and Python 3.12 CI are green. New tests cover successful observations, trace-to-proposal linkage, replay regression, runner failure, timeouts, copied answers, duplicate split fixtures, one-use final tests and stale/tampered evidence.