Skip to content

Semantic activation has no shipped operator journey; model 'loading' state persists 16+ min past load_deadline_ms with no typed failure; day-old eval client gets connection reset #753

Description

@ScriptedAlchemy

Context

Attempted the first real-operator-profile verification of semantic retrieval at full scale (project /fast/projects/tracedecay, ~4,790 files / ~226k chunks, model JinaEmbeddingsV2BaseCode installed and verified on disk). Three distinct gaps surfaced; filing together because they gate the same journey.

1. No shipped surface can activate semantic retrieval

The operator profile has semantic.runtime.v1 → active_profile: null (registry default). Activation requires a native evaluated-profile publication (semantic_evaluate_and_publish, profile hybrid-conservative) followed by a CAS configuration_set of active_profile — but the only clients of that invocation are the dev binaries tracedecay-search-eval-direct / tracedecay-search-eval, which are not shipped (~/.local/bin/tracedecay has no subcommand for it, no MCP tool exposes it, 246-tool catalog checked). A real operator cannot reach semantic-ready through any product journey; strict_semantic search truthfully reports unavailable forever. (The invocation itself is additionally broken by #752.)

2. loading lifecycle state ran 16+ minutes with load_deadline_ms=30000, no typed transition

The effective config pins resources.load_deadline_ms: 30000. During the one observed projection attempt (02:41–02:57 UTC, quiet host: load1 ≈ 20), tracedecay tool runtime reported semantic_runtime.state = "loading" continuously for ≥16 minutes while daemon RSS climbed from 11.4 GB to 18.5+ GB, until a kernel OOM kill ended the process. Either the load deadline is not enforced on this path, or the surfaced lifecycle state was stale — both are progress-reporting truthfulness violations (the operator cannot distinguish a hung load from a working one, and Indexing{completed_units,total_units} was never reached so no progress was ever visible).

Also note: after the aborted attempt, subsequent sealed generations (04:13 fresh|ready, 04:27, 05:10) never re-triggered projection within the same daemon lifetime — whether that is the publish-failure memo backoff or the deferred-activation retry being single-shot, the runtime stayed silently at installed with no typed record of why nothing was scheduled.

3. One-day protocol drift breaks the eval client with a raw connection reset

tracedecay-search-eval-direct built from branch HEAD at 03:00 (Aug 30) against the operator daemon built Aug 29 20:30 (087e6674286e) fails evaluate_and_publish with io error: Connection reset by peer (os error 104) — a transport reset, not a typed UnsupportedRevision. The Aug 29 vintage of the same binary connects fine and receives typed responses. Version skew within one day of the same branch should yield a typed refusal, not a reset.

Environment notes (for reproduction honesty)

Measurements were attempted under heavy peer load: kernel global OOM killed tracedecay processes 23 times over the night (daemon oom_score_adj=200 makes it the preferred victim; unit drop-ins set MemoryMax=infinity), and the shared checkout's continuous worktree edits kept the code index in near-permanent full-rebuild (~15–20 min per cycle; fresh ready windows lasted 1–4 min; search returns typed generation_unverified/generation_rebuilding throughout, which is correct and truthful). Items 1–3 are code defects independent of that environment.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions