Skip to content

ENG-664: expose compaction result for host persistence + skip low-yield re-summarization#249

Draft
ea-rus wants to merge 4 commits into
jorge/eng-648-reduce-anton-cowork-server-token-usage-promptskillsfrom
feat/eng-664-conversations-resend-the-full-history
Draft

ENG-664: expose compaction result for host persistence + skip low-yield re-summarization#249
ea-rus wants to merge 4 commits into
jorge/eng-648-reduce-anton-cowork-server-token-usage-promptskillsfrom
feat/eng-664-conversations-resend-the-full-history

Conversation

@ea-rus

@ea-rus ea-rus commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds ChatSession.last_compaction — a read-only result of the most recent
    history compaction this session ran ({"summary": str, "covered_through": int}
    or None). summary is the exact compacted message content (including the
    _COMPACTED_MARKER prefix) so a host can seed it back verbatim next turn and
    have _summarize_history recognize + extend it in place. covered_through
    is how many of the session's seed (initial_history) messages are now
    folded into it, letting a host map that count onto its own message list to
    find the cutoff.
  • _summarize_history now skips the LLM round-trip when there's little new
    material to fold in (excluding any already-carried-forward prior summary
    from the measurement) — avoids paying for a summarization call that would
    barely extend an already-large summary.
  • No change to the compaction algorithm itself, and no new required
    constructor parameters — existing hosts are unaffected.

This is the anton-side half of ENG-664 (cowork-server persists last_compaction
across turns instead of resending full history every turn). The once-per-turn
compaction guard (_compacted_this_turn) already existed and needed no change.

Test plan

  • New tests/test_history_compaction.py: last_compaction is None
    before any compaction, populated correctly after one, and the
    low-yield-skip guard (including the "prior summary doesn't count as
    new material" case).

Fixes https://linear.app/mindsdb/issue/ENG-664/conversations-resend-the-full-history-every-turn-and-throw-away-the

…omptskills' into feat/eng-664-conversations-resend-the-full-history
@Dav1dF

Dav1dF commented Jul 17, 2026

Copy link
Copy Markdown

Read this alongside the prompt-caching work (the gateway's cache alignment from mindsdb/mindshub_inference — ENG-601/ENG-586) to check for a conflict, and wanted to note the opposite on the record: this design is exactly what prefix caching needs.

  • Replaying the persisted summary verbatim keeps the seed byte-stable between compactions, and the tail is append-only — so a cached prefix survives across turns.
  • The low-yield re-summarization skip is also the cache-friendly behavior: every re-summarization rewrites the front of history and resets the cached prefix, so avoiding marginal ones saves twice.

One expectation to write down somewhere (here is fine): when cache alignment turns on in prod, cache-hit dips will correlate with compaction events. That is inherent to any history trimming, and last_compaction / the compaction event give us the attribution — no change needed in this PR, just flagging so nobody reads the dips as a caching bug later.

Linear: ENG-664; caching context on ENG-586.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants