Skip to content

pool: a chat /task never reaches the Model Pool judge #1527

Description

@AbirAbbas

What happened

The Model Pool judge scores tasks when they land (poolJudgeHook in cmd/codeaf/poolrecord.go, wired as taskLanded in cmd/codeaf/chatv3.go). A task started with the chat's ordinary /task never reaches that hook, so its landing is never judged. This was found on 2026-09-25 while fixing the judge's keyless-boot bug in #1519: a hand check of that fix could not reach the judge from /task at all.

Replication

Deterministic: a cmd/codeaf test that lands a /task through the chat's own task road, with a stub worker and the pool in record mode, then checks the pool's judged/ directory, or pool/judge-last.json. Nothing is written today. Compare with the road that does call taskLanded.

By hand (real model, about $0.01):

  1. Run /task on a trivial change with CODEAF_MODEL_POOL recording.
  2. After it lands, codeaf pool show lists no new judged landing.

The fix

Every road that lands a task, /task included, reports its landing to the judge hook once.

Acceptance

  • e2e: after a /task lands, codeaf pool show lists its judged entry.
  • Unit: every landing road goes through the one hook. A structural test over the landing sites would do.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:sessionThe engine — turns, tasks, the toolbelt, checkpointsbugSomething the code does that it should not

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions