Skip to content

Give the procedure watcher a separate staleness window for Held - #793

Merged
xmap merged 1 commit into
mainfrom
worktree-agent-a411f10d98a420bf3
Sep 9, 2026
Merged

Give the procedure watcher a separate staleness window for Held#793
xmap merged 1 commit into
mainfrom
worktree-agent-a411f10d98a420bf3

Conversation

@xmap

@xmap xmap commented Sep 9, 2026

Copy link
Copy Markdown
Owner

The defect

ProcedureWatcher flags a stalled conduct by writing Decision(context=ProcedureProgress, choice=Stall). It watched Running and Held against one threshold, procedure_watcher_stale_after_seconds, defaulting to an hour.

That works for Running, which also gets a second chance: if the status timestamp looks stale, the watcher re-checks the latest activity recorded_at and stays quiet if the procedure is still logging steps.

Held gets no such second chance, and structurally cannot, because a held conduct accepts no activity entries. So staleness came purely from last_status_changed_at, and any hold past an hour raised a stall alarm. An operator pausing a conduct overnight for a bakeout, or to wait on beam or on a collaborator, produced a false alarm. Because the alarm is a Decision, it is a durable cross-BC record, not a log line.

The fix

A separate procedure_watcher_held_stale_after_seconds, defaulting to a week, selected per item by status. Running behaviour is untouched, second chance included.

The default follows the sibling precedent rather than inventing one: campaign_watcher_stale_after_seconds is already a week and is already documented as "how long a campaign may sit Held." The new setting carries its own @field_validator matching the others in that file.

The module docstring now says why the two statuses need different clocks, so the next reader does not have to rediscover it.

Verification

  • New regression test test_tick_does_not_flag_held_overnight_pause proves a 12 hour hold stays quiet
  • The pre-existing test_tick_flags_stale_held_without_folding_activity still proves a genuinely stalled Held does flag, with an explicit short window so it does not need to simulate a week
  • Two new settings-validator tests
  • The fix was checked by mutation: with the config field kept but the watcher logic reverted, the new test fails, then passes again with the logic restored
  • 25 watcher tests, 1,584 tests/unit/operation, 37,444 tests/architecture, ruff and pyright clean

Tradeoff worth stating

A week-long window trades a false positive for a slower true positive: a conduct parked by the machine rather than by an operator now sits unnoticed for longer. Both live in one bucket today, so the trade is forced. Labelling why a hold happened would dissolve it and let each origin carry its own window; that is a separate design question and deliberately not in this PR.

🤖 Generated with Claude Code

Running and Held shared one hour-long threshold
(procedure_watcher_stale_after_seconds), and Running alone got a
second chance via read_procedure_activity_recency before flagging.
A Held conduct accepts no activity, so it has no such second chance
and structurally cannot have one: any deliberate operator pause past
an hour (a bakeout, waiting on beam, waiting on a collaborator) wrote
a real cross-BC Decision(context=ProcedureProgress, choice=Stall).

Add procedure_watcher_held_stale_after_seconds (default 604800.0, a
week) and select the threshold per item's status in _watch_tick.
This mirrors campaign_watcher_stale_after_seconds, which already
documents a week as how long a campaign may sit Held. No new
ProcedureStatus member, hold_origin discriminator, or event field is
needed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  apps/api/src/cora/api
  _procedure_watcher.py
  apps/api/src/cora/infrastructure
  config.py
Project Total  

This report was generated by python-coverage-comment-action

@xmap
xmap merged commit 1221b01 into main Sep 9, 2026
19 checks passed
@xmap
xmap deleted the worktree-agent-a411f10d98a420bf3 branch September 9, 2026 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant