chat fails loud per-request + expert_observe harness (final rescue from #2056) - #2252
Open
joelteply wants to merge 4 commits into
Open
chat fails loud per-request + expert_observe harness (final rescue from #2056)#2252joelteply wants to merge 4 commits into
joelteply wants to merge 4 commits into
Conversation
…nic (#201) `ChatModule::executor()` did `.cloned().expect(...)` — a hard panic if a `chat/poll`, `chat/send`, or `persist_posted` landed before `start_server` called `install_executor_on_all` (a boot race). Panicking there SIGABRTs the whole core and takes every other module down with it, for a per-request contract violation that only concerns that one request. Convert `executor()` to `Result<Arc<CommandExecutor>, String>` returning the SAME loud, contract-naming message, and `?`-propagate it in the 3 callers (all already `Result<_, String>`). Faithful to [[no-fallbacks-ever]] — still loud, still names `install_executor_on_all`, no silent default — while satisfying #26 (faculties degrade, never panic): a command that races boot fails loudly to its caller instead of crashing the process. Regression test: a pre-install `poll()` returns the loud error naming the contract instead of panicking. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo
…uting (#230/#229) Runs a GGUF MoE through the core/llama FFI with LiveExpertObserver attached (the existing cb_eval → ffn_moe_topk seam), generates N tokens to drive real routing, then dumps the model-intrinsic affinity: hot/cold expert distribution + co-occurrence + prefetch candidates. That affinity is the INPUT to expert prefetch (#227), grid placement (#180), compaction, and distillation (#233). Uses the in-process FFI (not the live llama-server lane) because affinity is model-intrinsic — valid data, zero risk to live serving. First run (Qwen3-Coder-30B-A3B, 96 tokens, Metal): 43,640 activations, 6116/6144 expert-slots fired (~99.5%), hottest expert only 0.20% (~12x uniform), 6092 colder share 95.8%. FINDING: for an 8/128 (6.25%-active) MoE, activation over a generation is BROAD, not tiny-hot — so the paging win is the tier ladder + affinity placement, not a small resident set. Prefetch predictor returned 0 candidates over 96 tokens (needs more data). K3 (1.8% active) should be far more concentrated — same harness will quantify it when weights land. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo
…et-size + Jaccard (#230) Evolves the MoE glass-box harness from the pooled first cut to the measurement that actually decides the paging architecture (BigMama's three methodological guardrails): diverse multi-domain corpus, PER-DOMAIN concentration (top-K% activation share vs the uniform null), cross-domain hot-set Jaccard, shared-base-vs-own-only activation MASS split, and the working-set-size curve (experts resident for 50/80/90/95% of a domain's mass). Prefill-dominant sampling on realistic input, not a degenerating greedy loop. Ran live on Qwen3-Coder-30B-A3B (Metal): pooled top-10% = 18.6% (mild — the smear), but per-domain = 25–38% with near-disjoint hot sets (code↔prose Jaccard 0.05) and a tiny 38-expert universal core — i.e. paging is domain-working-set SWAPPING, not frequency tiering. This is the #180 evidence; the harness is the reusable probe for any MoE (incl. K3 at weight-drop). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo
Rebase fallout from the fail-loud change itself: executor() became Result<Arc<CommandExecutor>, String> (that IS the 'no process panic' fix), and every call site needs `?`. The chat/poll anchor lookup at mod.rs:155 was the one that did not get converted, so it called execute_json on a Result. One character. Caught by cargo check, not by review. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Q4NU4VNiELPQfBpCacDZGc
joelteply
force-pushed
the
fix/loud-failures-and-expert-observe
branch
from
August 15, 2026 03:25
0c92e8c to
881c68c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fifth and final rescue from #2056. With this, every unique unit of that 56-commit branch is on canary in reviewable pieces.
What lands
8d77db049ChatModule executor fails loud per-request, not process panic (Shell Timeout Mitigation Strategies - Architectural Patterns for Command Resilience #201).executor()becomesResult<Arc<CommandExecutor>, String>with a message naming the actual cause ("CommandExecutor not installed — start_server must call install_executor_on_all"), so a missing executor is a failed request instead of a downed core.b357ab112+c7032fd8dexpert_observe — glass-box LIVE MoE expert routing: per-domain concentration, working-set size, Jaccard overlap.9d4509bce(airc room-route error echo) was dropped as already upstream — cherry-pick reported it empty, confirming canary has it.One fix on top
The fail-loud commit converts
executor()to fallible but missed its own call site atchat/mod.rs:155— thechat/pollanchor lookup calledexecute_jsonon aResult. One?. Caught bycargo check, not by review, which is the argument for compiling every rescued unit rather than trusting that a commit which passed CI once still applies.Verification
cargo check -p continuum-core --lib --testsclean.#2056 is now fully rescued
#2056, #2053, and #2133 are superseded by these five and can be closed — their unique content is landed, their remaining commits target APIs canary has already replaced.