Skip to content

fix: acknowledge journaled coordinator messages after the injected-text cache expires - #26

Merged
SergioTCG merged 4 commits into
mainfrom
fix/coordinator-message-acknowledgement
Oct 7, 2026
Merged

SergioTCG merged 4 commits into
mainfrom
fix/coordinator-message-acknowledgement

Conversation

@SergioTCG

Copy link
Copy Markdown
Owner

Summary

A coordinator message whose prompt hook arrived after the 120-second in-memory injected-text record, or after a daemon restart, failed the delegated task with:

A local terminal prompt replaced the delegated worker turn.

The handler had already authenticated the prompt against the persisted journal. It still sent it down the local-input path. This PR settles an authenticated current-generation message as bridge input. It also adds behavioural regression tests that drive the actual UserPromptSubmit handler.

Incident

A live Codex worker:

  • The coordinator message was delivered at T.
  • Its exact <sab-team-message …> prompt hook arrived at T + 123.331 s. Codex queues input submitted during a running turn and fires UserPromptSubmit when it actually submits it.
  • The task failed 79 ms later.
  • The prompt's digest matched the persisted journal (teamProviderTurn.promptHash) for the exact task, session, channel and generation. It was not manual terminal input.

Root cause

Step Code Result
1 providerPromptTurnMarker() recognizes <sab-team-message>
2 providerPromptAcknowledgesTask() authenticates the exact generation, bound turn and journal digest, so acknowledgedTurn is set and the turn is activated
3 the lifecycle branch acknowledgedTurn && teamTaskId && … teamTaskId comes from taskMarker(), which knows only <sab-team-task>, so it is null and the acknowledged branch is skipped
4 fallback unacknowledgedPromptTeamEffect() inside the 120 s cache injected is true and nothing happens; after expiry or a restart it returns fail_task
5 mirroring the same prompt was also posted to Slack as 💬 You (terminal)

A sibling defect: the journal digest was computed over the raw hook prompt. Claude Code wraps pasted input in <pasted_content id="…">, so Claude task and message prompts could never match the journal once the cache expired or the daemon restarted.

taskMarker() keeps its semantics. Its other caller, withoutDelegatedTaskPrompt(), must purge only queued task-start envelopes and not coordinator messages.

Commits

  1. refactor: move the handler into daemon/prompt-submit.mjs, verbatim. daemon.mjs cannot be imported in tests, so createPromptSubmitHandler() now holds the exact former body and the daemon delegates to it with its own effects. Supporting moves, each byte-identical in effect:

    • createInjectedTextCache() (with an injectable clock);
    • the state-taking lifecycle helpers;
    • isSystemPrompt;
    • coordinatorTaskMessagePrompt().

    The handler contract tests now read the new module, and the full suite passes (566).

  2. test: the regression suite on its own. 10 of its 14 tests fail here on purpose, reproducing the incident through the actual handler.

  3. fix:

    • An acknowledged current-generation coordinator message is settled as bridge input and never reaches the local-input fallback.
    • Journal digests use the unwrapped prompt.
    • A prompt that is acknowledged, or whose exact digest is journaled (journaledTeamProviderPrompt(), provenance only), is not mirrored as typing.

Preserved guarantees

  • Task, session, channel, native-turn and generation checks are unchanged. Acknowledgement still requires the exact current or pending generation, the bound turn, and the journal digest or injected record.
  • A tag alone is never trusted: altered, misaddressed or tag-only text is still local input.
  • Stale same-task generations are ignored, so they neither fail the task nor authorize anything.
  • Terminal tasks are never resumed.
  • Delayed callbacks cannot discard a newer staged generation.
  • Genuine local terminal input still fails the delegated turn and revokes a coordinator's team turn.
  • The cache timeout is unchanged. Recognition comes from persisted evidence.

Regression tests (test/prompt-submit.test.mjs)

These drive createPromptSubmitHandler(), the code the daemon runs. They use:

  • real team state built with teams.mjs;
  • the real injected-text cache on a fake clock;
  • the delivery path's own journal functions and envelope builders;
  • a JSON round-trip of state plus an empty cache to stand in for a restart.

Slack, pollers, audits and persistence are recorded instead of performed.

Requirement Test Before fix After fix
1. Valid current message after >120 s codex: / claude: … hooked after the 120 s cache … ✖ fails the task ✔
2. Valid message after restart, empty cache codex: / claude: … recognized after a daemon restart ✖ fails the task ✔
3. Task-start acknowledgement still works (incl. after restart) codex: / claude: the original task-start acknowledgement … ✖ mirrored (Codex) / fails (Claude) ✔
4. Stale same-task generation neither fails nor authorizes a stale same-task generation … ✖ mirrored as typing ✔
5. Altered / untrusted / wrong task, session, channel, generation altered or misaddressed …, … replayed on another session or a rebound channel ✔ ✔
6. Genuine local input ends delegated authority genuine local terminal input … ✔ ✔
7. Delivery/ack/completion races cannot corrupt newer work a hook that beats its transport callback …, a late older-generation hook cannot discard a newer in-flight …, a declared completion survives …, a late hook for a terminated task … 1 ✔, 3 ✖ ✔

Mutations. Each was applied on its own against the suite, then the files were restored byte-identical:

Mutation Result
Drop the acknowledged-message branch 7 fail
Authenticate the raw wrapped prompt 3 fail
Mirror journaled bridge input again 10 fail
Trust the tag without the digest 1 fails
Journal lookup ignores the digest 2 fail
Treat every acknowledgement as a message (breaks task start) 14 fail
Cache-timeout-only "fix" (unfixed handler, 1 h TTL) 8 fail, including every post-restart case

Validation

  • Focused regression suite: 14 pass, 0 fail.
  • Full npm test: 581 pass, 0 fail.
  • npm run check, node --check (daemon/channel/scripts), Pi extension --list-models and shellcheck all pass. Every named import in daemon.mjs resolves (319 checked).
  • ⚠️ npm run audit fails on a newly published advisory, GHSA-6qxp-vccf-f47h (@modelcontextprotocol/sdk ≤ 1.30.1, OAuth client). This PR changes no dependencies; a separate lockfile PR fixes it.

Deployment notes

  • Daemon restart needed: yes. Only daemon modules change; hooks, the CLI and the MCP channel are unchanged. There is no state migration and no new persisted field.
  • Running sessions survive the restart. The fix recognizes messages that were delivered before the restart.
  • Coordinate the restart with the operator. Other teams may be active, and a dispatching claim is never retried across a restart.
  • Merging deploys within 6 h through the self-updater.
  • Failed tasks are not resumed, released or recreated automatically.

🤖 Generated with Claude Code

https://claude.ai/code/session_011Xs8AVBVrTGfpzUMiCQp2R

npm audit now reports a high-severity advisory for @modelcontextprotocol/sdk
1.12.0–1.30.1: the OAuth client could send credentials to an authorization
server chosen by the MCP server. CI's production-dependency audit therefore
fails on every run, main included.

Pin the lockfile to 1.31.0, the first patched release; package.json's
^1.30.0 range is unchanged. Upstream, 1.30.1 and 1.31.0 change only the
OAuth client and the HTTP transport's body limits. SAB uses only the stdio
Server, so the vulnerable path was never loaded.

Validated:
- npm audit reports 0 vulnerabilities, npm test passes (564), and npm run
  check passes.
- An end-to-end smoke test ran the real channel/server.mjs on 1.31.0
  against a local fake daemon. It covered the MCP handshake with both
  claude/channel capabilities, an SSE message delivered as
  notifications/claude/channel, and a permission request relayed to the
  daemon.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011Xs8AVBVrTGfpzUMiCQp2R
sergejberisaj and others added 3 commits October 7, 2026 18:06
The daemon's UserPromptSubmit branch decides whether a native prompt is the
bridge's own input (a delegated task, a coordinator message, a Slack
injection) or local input, and applies that to the team lifecycle. It could
only be tested through source-regex contracts, because daemon.mjs cannot be
imported without starting a live bridge.

Its body now lives verbatim in daemon/prompt-submit.mjs as
createPromptSubmitHandler(). The daemon creates it with its own persistence,
poller, Slack and audit effects and delegates to it. Behaviour is unchanged.

Supporting pure moves, each byte-identical in effect:
- the injected-text cache becomes createInjectedTextCache() in util.mjs,
  with an injectable clock;
- currentTeamTaskProviderTurn and teamTaskTurnOwnsCurrentLifecycle delegate
  to state-taking functions in team-provider-turn.mjs;
- isSystemPrompt moves to providers.mjs;
- the coordinator message envelope is built by coordinatorTaskMessagePrompt()
  in teams.mjs.

The handler contract tests now read the new module; their assertions are
unchanged. Full suite: 566 pass.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011Xs8AVBVrTGfpzUMiCQp2R
…handler

Behavioral regression tests for the actual UserPromptSubmit handler. They
use real team state, the real injected-text cache on a controllable clock,
and the same journal functions as the delivery path. Only Slack, pollers,
audits and persistence are recorded instead of performed.

This commit deliberately leaves 10 of the 14 tests failing. They reproduce
the incident:
- A journaled Codex coordinator message hooked 123.3 s after delivery, past
  the 120 s cache, fails the running task with "A local terminal prompt
  replaced the delegated worker turn".
- The same happens after a daemon restart with an empty cache.
- Claude's <pasted_content>-wrapped task and message prompts can never match
  the journal.
- The bridge's own journaled prompts are mirrored to Slack as terminal
  typing.

The four passing tests pin behaviour that must not change: untrusted or
misaddressed input, genuine local input, replay on another session or a
rebound channel, and a hook that beats its transport callback.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011Xs8AVBVrTGfpzUMiCQp2R
…xt cache expires

A coordinator message delivered while a worker is mid-turn may be queued by
the provider and submitted later. Codex does this, and fires
UserPromptSubmit only at submission. In a live team the exact message
prompt arrived 123.3 s after delivery, beyond the 120 s in-memory
injected-text record.

The prompt hook authenticated it correctly: the marker, generation, bound
turn and journaled digest all matched. The lifecycle branch, however, keyed
on taskMarker(), which recognizes only <sab-team-task>. So the acknowledged
message fell through to the local-input fallback, which failed the active
task with "A local terminal prompt replaced the delegated worker turn". The
same happened after a daemon restart, and the prompt was also mirrored to
Slack as terminal typing.

Fix:
- An acknowledged current-generation coordinator message is settled as
  bridge input. It never reaches the local-input fallback.
- Mirroring skips any prompt that is acknowledged or whose exact digest is
  in the provider-turn journal (current, pending or history).
  journaledTeamProviderPrompt() proves provenance only and authorizes
  nothing.
- Journal digests are computed on the unwrapped prompt. Claude Code wraps
  pasted input in <pasted_content>, so Claude task and message prompts could
  never match the journal after the cache expired or a restart.
- taskMarker() keeps its semantics: withoutDelegatedTaskPrompt() relies on
  it to purge only task-start envelopes.

Task, session, channel, native-turn and generation checks are unchanged. A
tag alone is still untrusted; stale generations are ignored; terminal tasks
are not resumed; genuine local input still fails the delegated turn.

Tests: the reproduction commit's 14 handler tests all pass now, where 10
failed before. Seven mutations turn them red, including a cache-timeout-only
"fix" and trusting a tag without its digest. Full suite: 581 pass.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011Xs8AVBVrTGfpzUMiCQp2R
@SergioTCG
SergioTCG force-pushed the fix/coordinator-message-acknowledgement branch from 942a8a3 to 0c6702c Compare October 7, 2026 16:06
@SergioTCG
SergioTCG merged commit 1f049f9 into main Oct 7, 2026
3 checks passed
@SergioTCG
SergioTCG deleted the fix/coordinator-message-acknowledgement branch October 25, 2026 21:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants