Skip to content

Self-healing edits: intercept edit calls, auto-test, revert on failure #1

@alderpath

Description

@alderpath

Problem

When the LLM applies an edit and tests fail, it spirals into a verbose debug loop: read → edit → test → fail → re-read → re-edit → test → pass. This costs 3-5 turns and thousands of output tokens.

Proposed Solution

Intercept edit tool calls in gate.js, apply the change through the daemon, run cargo test automatically, and only keep the edit if tests pass. If tests fail, revert immediately and return a compressed error message. The LLM never sees the failure spiral.

Implementation Status (built, needs hardening)

Working:

  • crates/reliary-agent/src/heal.rsheal_edit() applies content, runs cargo test, reverts on failure
  • crates/reliary-agent/src/daemon.rsapply-edit endpoint (reads new content from temp file, delegates to heal)
  • pi/gate.js — intercepts edit tool calls via tool_call hook, applies old→new replacement, writes to temp file, calls daemon

Gaps before production:

  • Multi-edit: only processes edits[0], silently drops remaining edits
  • Bash sed interception: LLM prefers sed -i ~60% of the time, bypasses hook
  • Workdir heuristic: hard-codes process.cwd(), should walk up for project root markers (Cargo.toml, pyproject.toml, package.json)
  • Language detection: hard-codes cargo test --quiet, no Python/JS/Go/Make support
  • Timeout: 30s test limit → fall through to native edit on slow suites
  • Circuit breaker: 3 failed heal attempts → disable for rest of session

Verification

Last confirmed working at commit 02d3a49. Test: LLM fixes zone.rs bug, daemon applies + tests + returns "OK: tests pass".

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions