ENG-664: expose compaction result for host persistence + skip low-yield re-summarization#249
Draft
ea-rus wants to merge 4 commits into
Conversation
5 tasks
…omptskills' into feat/eng-664-conversations-resend-the-full-history
|
Read this alongside the prompt-caching work (the gateway's cache alignment from mindsdb/mindshub_inference — ENG-601/ENG-586) to check for a conflict, and wanted to note the opposite on the record: this design is exactly what prefix caching needs.
One expectation to write down somewhere (here is fine): when cache alignment turns on in prod, cache-hit dips will correlate with compaction events. That is inherent to any history trimming, and |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ChatSession.last_compaction— a read-only result of the most recenthistory compaction this session ran (
{"summary": str, "covered_through": int}or
None).summaryis the exact compacted message content (including the_COMPACTED_MARKERprefix) so a host can seed it back verbatim next turn andhave
_summarize_historyrecognize + extend it in place.covered_throughis how many of the session's seed (
initial_history) messages are nowfolded into it, letting a host map that count onto its own message list to
find the cutoff.
_summarize_historynow skips the LLM round-trip when there's little newmaterial to fold in (excluding any already-carried-forward prior summary
from the measurement) — avoids paying for a summarization call that would
barely extend an already-large summary.
constructor parameters — existing hosts are unaffected.
This is the anton-side half of ENG-664 (cowork-server persists
last_compactionacross turns instead of resending full history every turn). The once-per-turn
compaction guard (
_compacted_this_turn) already existed and needed no change.Test plan
tests/test_history_compaction.py:last_compactionisNonebefore any compaction, populated correctly after one, and the
low-yield-skip guard (including the "prior summary doesn't count as
new material" case).
Fixes https://linear.app/mindsdb/issue/ENG-664/conversations-resend-the-full-history-every-turn-and-throw-away-the