Govern a base with one run at a time, so concurrent runs stop deciding the same cluster repeatedly - #916
Merged
Conversation
…iding the same cluster ten times Every document's extraction enqueues a govern job for its base, and enqueue_unless_queued only skips a job that is still queued; with 64 workers each new job starts at once. On the typed-graph bench nine govern runs overlapped on one base, all read the same queue heads, and 1346 review pairs received 8888 keep decisions (one pair ten decisions in 38 seconds from ten runs); a third of the run's model tokens went there, and the duplicate-key errors on agent_decisions_open were the concurrent proposals colliding. govern() now takes a per-base session-level advisory lock (try only, as the vector index build does). A run that does not get it exits and enqueues one govern for a minute later, deduplicated against queued jobs only, so pairs that arrive after the running job read its last queue head are still picked up. The queue and cluster reads skip rows the running job has marked adjudicating. close_review_auto reports how many rows it closed, and a keep or already-merged outcome records no decision when the row was closed by someone else. Store test: the second lock attempt on a base fails until the first is released; another base is another lock. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Wayland Yang <wayland0916@gmail.com>
This was referenced Sep 25, 2026
WaylandYang
added a commit
that referenced
this pull request
Sep 25, 2026
…ith tokens by phase (#917) Two groups on dev at 961c3c0 with gemini-3.5-flash, judged and with errata: precision 91.0% / 88.5% before errata and 96.0% / 96.5% after; same-sentence gold recall 11.3% / 13.2% before errata and 18.6% / 22.4% after, with the errata retractions mostly wrong on this binary (72/98, 164/196), which is what #906's two-vote retraction addresses; 158k and 183k tokens per document. The token table by phase shows where they went: the alignment window carried the rule proposals' per-item property tables, the two-vote phrase alignment, and the governance agent deciding the same pairs from up to ten concurrent runs (9481 decisions on 1421 pairs), fixed in #916. The bench judge now fails fast when it would read a sealed key from llm_settings instead of sending the ciphertext and collecting 401s. Signed-off-by: Wayland Yang <wayland0916@gmail.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
WaylandYang
added a commit
that referenced
this pull request
Sep 25, 2026
…tokens by phase (#918) Same corpus, model and judge as the pre-#906 baseline, on dev with #906 and #916. Judged precision 81.0% / 79.0% before errata and 83.0% / 80.5% after (the loss against the baseline is misworded facts from minimal reasoning); same-sentence gold recall 14.2% / 14.3% before errata and 17.8% / 17.1% after, with errata now adding without retracting wrongly (2 and 1 retractions, none judged stated); 31k and 34k tokens per document, a fifth of the baseline and still five times 0044's ceiling, seven tenths of it in alignment and rule proposals. 47 and 48 minutes a group against three to five hours. Signed-off-by: Wayland Yang <wayland0916@gmail.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found while attributing the typed-graph bench's tokens (run on dev before #906). A third of one run's model tokens, about 5.3M of 16M, went to the governance agent, and not because there was that much to decide: 1346 review pairs received 8888
keepdecisions, one pair ten decisions within 38 seconds from ten different runs. The duplicate-key errors onagent_decisions_openin the server log were the concurrent proposals colliding.Cause. Every document's extraction enqueues a
governjob for its base, andenqueue_unless_queuedonly skips a job that is still queued; with 64 workers each new job starts at once. Nine govern runs overlapped on one base, all read the same queue heads (the lock only marksstage = adjudicating, which the queue did not exclude), all asked the model about the same clusters, andclose_review_auto'sWHERE status = 'pending'let only the first close the row while every run recorded a decision.Fix.
govern()takes a per-base session-level advisory lock, try only, the way the vector index build does. A run that does not get it exits and enqueues onegovernfor a minute later (enqueue_unless_queued_after, deduplicated against queued jobs only, so the running job does not suppress it), which covers pairs that arrive after the running job read its last queue head.queue()andcluster_of()skip rows markedadjudicating.close_review_autoreturns how many rows it closed; a keep or already-merged outcome records no decision when the row was already closed by someone else.#906 switched governance off in the bench base, which hides this from the bench but not from a real base with a running agent.
Verified on a fresh database: new store test
a_base_is_governed_by_one_run(second lock attempt fails until the first releases; another base is another lock), the server's governance and adjudication tests, workspace clippy.🤖 Generated with Claude Code