Skip to content

session-end: passing test edits are recorded as failed outcomes #4

Description

@LeonEvo1103

Summary

hooks/session-end.js records ordinary test-code additions such as assert.equal(...) as a failed evolution outcome, even when the assertion is valid and the tests pass.

Verified on main at 3048fb73910a0a8944f74fb2e70abd5752a60b2f.

Reproduction

  1. Create a temporary Git repository with a committed test file.

  2. Add this line to the tracked file without committing it:

    assert.equal(1 + 1, 2);
  3. Run hooks/session-end.js with COPILOT_WORKSPACE_DIR pointing to the temporary repository and MEMORY_GRAPH_PATH pointing to a temporary JSONL file.

  4. Inspect the emitted message and memory entry.

Actual result

Signals: [test_failure]
{
  "signals": ["test_failure"],
  "outcome": {
    "status": "failed",
    "score": 0.3
  }
}

Expected result

Adding test syntax must not be treated as evidence that the session failed. Without explicit test execution evidence, the hook should record a neutral/unknown result or avoid inferring failure.

Cause

  • hooks/_signals.js:58-64 treats assert. and expect( as test_failure keywords.
  • The code-line filter at hooks/_signals.js:69-102 does not recognize Git diff lines prefixed with +, so added test code reaches keyword matching.
  • hooks/session-end.js:335-342 maps any test_failure signal directly to failed / 0.3.

Impact

Common Node/Jest test additions systematically pollute evolution memory with false failed outcomes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions