feat(runtime): archive tool results through durable ledger refs

[likun666661]

Summary

Refs #4071. Writer cutover after #4972's bounded ledger reader.

• New production Tool Result archives publish rewriteVersion 2 placeholders with explicit ledger identity and a versioned maka://archive-ledger/v1/... resource reference. No new archive payload is written to ArtifactStore or context-offload.sqlite.
• Preflight verifies the exact persisted source projection, predecessor, serialized bytes, hash and reader budget. The Host rechecks under Session admission before committing the existing projection transition; rejected preparation or commit leaves the original projection intact.
• Replay and ArchiveRead dispatch by reference kind. Legacy v1 Artifact readers remain, but the Host's Artifact archive writer is removed. Missing evidence does not activate a fallback writer.
• Conversation copy regenerates source digests, hashes, lengths, event and predecessor identities against the copied projection, including changed image refs. Copies do not require a nonexistent ledger-archive Artifact.
• Remap retained ledger resource addresses in copied conversation text and ArchiveRead call refs; preserve resourceRef on archived-result read models.
• Use the frozen v1 body encoding on both producer and reader paths.

Review Follow-up

Rebased onto main 5a1514c1d; current head db8e5296b, compatibility epoch 132. Four package builds and 84 focused archive/copy/evidence/Host/model-composition/UDS tests passed after rebase. Earlier head IDs and counts below are historical validation records.

Latest integration head: 25d205f8f, merged main through e60e85d77; epoch 131. Four affected package builds passed; 81 archive/copy/evidence/Host/model-composition/two-client tests passed; final full storage 1192 passed, 8 skipped, 0 failures. Root lint/format pass. Earlier full runtime/Host counts below belong to the preceding integration baseline, not this newest main merge. GitHub reports MERGEABLE and CI is running.

Addressed all three P2 findings in 69d6b1881, after merging main in 975a7b827:

• Evidence budget applies to selected identity/projection fields and transitions, not unrelated raw Tool Result bytes. The regression archives a persisted MCP-shaped large raw image with a small media projection and reads it after reopen. SQLite still parses the addressed source JSON; no claim of constant SQLite parsing cost is made.
• Copy rewrites typed ArchiveRead result/projection identities and rebuilds dependent archive transitions in dependency order.
• Valid text checkpoints remain: covered digests, remapped summary refs and predecessor identities are rebuilt only after event rewrites. No blanket checkpoint deletion.
• A new main tool-wire golden changed only because this PR updates the two ArchiveRead descriptions. Restoring the old descriptions reproduces the old hash; the updated golden passes the entire model-composition file.

Follow-up validation: 51 focused tests passed; 30 model-composition tests passed; full storage 1198 passed / 8 skipped; full runtime 3316 passed / 13 skipped / the same 2 Responses reasoning failures. Full Host run: 1785 passed / 12 skipped / 6 existing peer-stream cancellations, plus the tool golden mismatch subsequently fixed and verified by the 30-test model-composition rerun. Builds, affected typechecks, root lint/format and diff checks passed. No full Host rerun after the golden-only update, Desktop E2E or cross-platform benchmarks. The results below also retain the original pre-review validation for context.

Verification

• Core/storage/runtime/runtime-host builds passed; runtime, runtime-host and storage typechecks passed.
• Core full suite: 820 passed.
• Storage full suite rerun: 1198 passed, 8 skipped. One initial concurrent context-writer test failure passed alone and on the full rerun.
• Final focused archive/prune/copy/capability/Host archive/two-client UDS revision suites: 93 passed.
• Production composition/recovery run: 42 passed, 1 Host connection failure; that single test (“49 KiB idle Message”) passed on isolated rerun.
• Real SQLite evidence close/reopen, no new Artifact count, mixed legacy/ledger reads, cross-Session refusal, stale preparation refusal, transition failure, resource decoding and copied-image hash reconstruction covered.
• Negative control: retaining the source hash during image-ref copy makes the new regression fail; reverted before final verification.
• Root lint, format, ASF headers and diff checks passed.

Full suites are not all green locally:

• Runtime: 3309 passed, 13 skipped, 2 existing Responses reasoning failures (Alibaba streamed reasoning replay; pinned SDK official summary events).
• Runtime Host: 1780 passed, 12 skipped, 6 cancelled in the unchanged resumable-peer-stream suite (event loop exits with a pending promise).

Not run: Desktop E2E/full repo build, live-provider archive exercise, Linux/Windows runs, cross-platform latency/RSS benchmarks. The UDS revision suite is existing integration coverage; the new image-hash copy regression runs through real ledger stores, not a full UI journey.

Compatibility and Copy Behavior

Host compatibility epoch advances 131 → 132. Once v2 refs are written, rolling back the writer must retain the v2 reader. The epoch gate does not make older binaries safe to reopen those durable refs.

When copying a conversation containing ledger archives, complete known reference rewrites first, rebuild dependent transitions, then rebuild supported valid text checkpoints from the final copied events and summary. Preserve checkpoint lineage and unaffected summaries; existing exclusions for unsupported/provider-native or invalid checkpoints remain.

New writes require durable modelProjection evidence and fit within the reader's 64-transition/2-MiB evidence budget with commit headroom. Unsupported or oversized evidence remains unpruned. No large-object streaming support is introduced. Ledger placeholders are capped before commit; nested re-archiving of an existing archive placeholder is refused.

No existing RuntimeEvent or archive file is rewritten/deleted. Legacy migration, reachability-based reclamation, full ArtifactStore retirement and #4027 cold-start acceptance remain separate work.

AI use

• No generative tool made a substantive contribution
• Generative tooling made a substantive contribution

Tool(s) and scope: Codex implemented the reference/Host/copy changes, authored regression tests, ran local checks and prepared this PR. Commit includes Generated-by: Codex.

Checklist

• Tests cover the change and fail without it
• Lint, format, typecheck and the affected suites pass locally (focused/core/storage pass; full-suite exceptions above)

Does this PR entail a change in behavior?

• Yes — new archives reference ledger projections instead of publishing duplicate payloads.
• No

[Astro-Han]

Thanks for working on the ledger-backed writer cutover. Reviewed ef9daf23f03146ae0b7c739479288fbfbc51c946 with four independent review lanes and a primary cross-check. Reusing the durable event/transition authority and removing the Artifact payload writer is the right direction. I found three P2 issues: supported MCP image results can become unarchivable, copied ArchiveRead results retain source references, and valid compaction checkpoints are discarded indiscriminately. Details are inline.

This is an AI-assisted review. A focused exact-head probe using real SQLite evidence reads and Host archive services passed (1/1): a 2.13 MB MCP-shaped image event is rejected as too_large, the writer declines, and the projection remains unchanged. This is not an end-to-end connector/model test; the copy findings were independently checked in source. No full-suite run or human acceptance is claimed. The current head also has merge conflicts and only an effort-label check, so the resolved head needs validation.

中文

感谢推进 ledger 归档写入切换。四路独立审查后由主审交叉核对，复用持久事件与 transition 权威、移除 Artifact 重复正文写入的方向合理。确认三个 P2：支持的 MCP 图片结果可能无法归档，复制后的 ArchiveRead 返回结果仍保留源引用，以及无差别丢弃有效压缩 checkpoint，具体见行内评论。

本轮为 AI 辅助审查。使用真实 SQLite evidence reader 与 Host archive services 的 exact-head 定向验证通过（1/1）：2.13 MB 的 MCP 图片形状事件被判 too_large，writer 拒绝，原投影保持不变。这不是 connector/model 端到端测试；复制问题已沿源码独立核对。未运行全套测试，也不代表人工验收。当前 head 仍有合并冲突，仅有 effort label 检查，解冲后的版本需要验证。

[Astro-Han]

Thanks again for this work. Following the inline review of ef9daf23f03146ae0b7c739479288fbfbc51c946, I suggest addressing the three findings as two coherent contracts:

1. Archive eligibility follows the effective projection. The archive represents the exact projection being replaced. A supported, small projection should not become unarchivable because unrelated raw data makes its containing RuntimeEvent large. Keep evidence reads bounded, but align the storage read with the identity, projection and transition evidence needed for reconstruction. Raising the whole-row limit alone leaves the same coupling in place.

2. Conversation copy preserves reference closure and supported replay state. References to copied resources must resolve to the corresponding target-Session facts, including references returned by ArchiveRead. Supported, valid text checkpoints should retain their replay role. Keeping raw history is necessary, but does not by itself preserve that role. Complete the typed reference rewrites and rebuild dependent identities/digests/checkpoints from the final content in dependency order; avoid blanket checkpoint deletion. Existing exclusions for unsupported/provider-native checkpoints can remain.

These fit the existing storage and copy owners; they do not call for a new framework, a parallel authority, or generic recursive rewriting of opaque tool output. The ledger-backed direction remains sound.

A small set of consumer-facing regressions would establish the contracts:

• Large raw MCP result + small media projection → archive → reopen → read.
• ArchiveRead → copy → read using the reference returned in the copied ArchiveRead result.
• Copy a conversation containing a ledger archive and a valid unaffected text checkpoint → replay still uses that checkpoint.

This is architectural synthesis of the AI-assisted review above, not an additional severity finding or a claim of human acceptance.

中文

感谢推进这项工作。建议把三条行内问题按两个契约统一修订：

1. 归档资格由有效投影决定。 归档保存的是被替换投影的精确语义；不能因为同一 RuntimeEvent 中无关原始数据很大，就拒绝一个合法的小投影。读取仍应有界，但应围绕重建所需的身份、投影和 transition 证据组织。单纯调大全行预算仍保留了同一耦合。
2. 会话复制保持引用闭合和支持的重放状态。 对已复制资源的引用，包括 ArchiveRead 返回结果中的引用，都应指向目标会话对应事实；支持且有效的文本 checkpoint 应保留其重放作用。原始历史还在只是必要条件。应按依赖顺序完成类型明确的引用重写，再基于最终内容重建相关身份、摘要与 checkpoint，避免无差别删除。既有不支持/provider-native checkpoint 的排除规则可以保留。

修订应落在现有 storage 和 copy 权威处，不需要新框架、并行实现，也不需要递归修改任意 opaque 工具正文。ledger 方向本身合理。

用三条面向实际消费者的回归建立契约即可：大 raw MCP 结果加小媒体投影的归档与重启读取；ArchiveRead 后复制并使用复制结果中的地址继续读取；同时含 archive 和不受影响有效文本 checkpoint 的复制及重放。

这是对前述 AI 辅助评审的架构归纳，不是新增定级意见，也不代表人工验收。

[Astro-Han]

Thanks for the focused follow-up. Rechecked 25d205f8 against the three earlier P2 findings and #4071's ledger-canonical writer/copy contract. All three earlier findings are addressed; I found no additional P0–P2 issue in this follow-up.

• Evidence eligibility now follows the persisted projection and reconstruction fields, without counting unrelated raw Tool Result bytes. The query remains Session/target scoped and bounded; this does not claim constant SQLite parsing cost.
• Copy rewrites the known ArchiveRead result envelope and durable projection, rebuilding dependent transitions from the final copied projection. It preserves reference closure without recursively changing opaque tool content.
• Supported valid text checkpoints are rebuilt after reference rewriting, including coverage digests, summary references and predecessor lineage. The blanket deletion is gone.

The solution stays at the existing storage/copy owners. Host admission still rechecks before the existing transition commit, and the production Artifact archive payload writer is removed. I found no new durable authority or unjustified parallel implementation. The dependency-order work is necessary for an ArchiveRead result that is itself archived; the new regressions cover distinct consumer obligations.

Verification: 4 Host archive cases and 27 conversation-copy cases passed in a temporary exact-head source-loader probe using existing third-party dependencies and local generated model metadata. This is not a fresh workspace build. Three negative controls restored only the old evidence reader or copy implementation: the large-raw-image case failed at evidence admission, and the two copy cases failed on the source URI and missing checkpoint respectively. No production files were changed.

Current-head CI is green, but GitHub reports CONFLICTING; local merge-tree confirms the conflict is in the compatibility epoch. Please reconcile that boundary against current main and validate the resulting integration head. The PR body's current MERGEABLE/CI-running sentence is also stale. These integration steps remain before merge; this review does not claim they are complete.

AI-assisted follow-up with an independent reviewer and primary-agent source/probe reconciliation.

中文

感谢这轮针对性修订。已按 25d205f 对照三个旧 P2 和 #4071 的 ledger-canonical 写入/复制契约复核：三个旧问题均已修复，本轮未发现新增 P0–P2。

• 证据预算只计算持久投影及重建所需字段，不再让无关 raw Tool Result 决定归档资格；仍按 Session/目标有界读取，不声称 SQLite 解析成本与原始行大小无关。
• 复制重写已知 ArchiveRead 结果及 durable projection，并根据最终投影按依赖顺序重建 transition，保持引用闭合，不递归修改任意工具正文。
• 地址重写完成后重建有效文本 checkpoint 的 coverage、摘要引用和前驱，不再无差别删除。

修复仍在现有 storage/copy 权威处，Host admission 仍在既有 transition 提交前复核，生产 Artifact 归档正文 writer 已移除。没有发现新持久权威或不必要的并行实现；被再次归档的 ArchiveRead 结果确实需要依赖顺序处理，回归保护的是不同消费者契约。

临时 exact-head 源码加载验证通过 Host 4 项、copy 27 项；复用了已有第三方依赖和本地生成的模型元数据，不是重新构建 workspace。三个反向验证仅恢复旧 evidence reader 或 copy 实现，分别在大 raw 图片准入、源 URI 残留、checkpoint 缺失处失败。未修改生产文件。

当前 CI 绿色，但 GitHub 为 CONFLICTING，merge-tree 确认冲突位于 compatibility epoch。请对照当前 main 收敛该边界并验证最终集成 head，同时更新正文中过时的 MERGEABLE/CI-running 状态。合并前仍需完成这些集成步骤。此次经独立 reviewer 与主审源码/探针复核。

[Astro-Han]

Thanks for addressing the feedback. Approving the reviewed head 25d205f8: all three earlier P2 findings are resolved, and the follow-up found no additional P0–P2 issues. Verification details are in the preceding review.

Please rebase onto current main, reconcile the compatibility epoch, and validate the resulting integration head before merging. Current-head CI is green; this approval does not imply the outstanding merge conflict has been resolved.

AI-assisted review with independent reviewer and primary-agent verification.

中文

感谢修复。批准已复核的 25d205f8：三个旧 P2 均已解决，本轮未发现新增 P0–P2，验证详情见上一条评审。

请自行 rebase 到当前 main，收敛 compatibility epoch，并在合并前验证最终集成版本。当前 head 的 CI 绿色；本次批准不代表尚存的合并冲突已解决。

此次为 AI 辅助评审，经独立 reviewer 与主审复核。