Skip to content

feat(attachments): hold a turn's inline attachments to the Memory event quota (PR-6) - #1143

Merged
philmerrell merged 1 commit into
developfrom
feature/document-offload-pr6
Sep 17, 2026
Merged

philmerrell merged 1 commit into
developfrom
feature/document-offload-pr6

Conversation

@philmerrell

Copy link
Copy Markdown
Contributor

PR-6 of docs/specs/document-context-offload.md §4E / §5 row 6. Branched from develop; independent of the #1137#1138#1139#1140 stack (spec §5: "PR-6 is unrelated to both and can go whenever").

Why

The stored unit is the message, not the file. A turn's inline attachments are persisted as one AgentCore Memory event; anything over the SDK's ~72 KB conversational limit goes out as a base64 blob (4/3 inflation) bounded by the 10 MB event quota, so ~7.5 MB of raw attachment bytes per turn is the break point. Past it create_message re-raises SessionException — a hole in history, strictly worse than the per-file oversized note.

The existing guards do not cover this. The per-file gate is 4 MB, so two legal files already exceed it. The 5-file cap lives only in the SPA. The call-aligned prod measurement (validation doc, Claim 7) puts the rate at ~1.3–1.4% of attachment turns, several with only 3–4 files. Three code findings from the same pass: the backend max_files_per_message setting was dead, the file_upload_ids resolver silently truncated at 5, and the direct files path had no count or aggregate cap at all.

What

  • _apply_inline_byte_budget (inference_api/chat/routes.py): after per-file partitioning, the inline set (documents and images — they are in the same event) is held to INLINE_ATTACHMENTS_MAX_TOTAL_BYTES. Default 7.5 MB = 10 MB × 3/4, derivation in the constant's comment in shared/files/models.py. Policy is first-fit in attachment order: earlier attachments win, and a later smaller file that still fits rides along rather than being punished for a large neighbour. Both output lists keep attachment order so the marker and note are deterministic (they land in the cacheable prefix). Trimmed files join the existing oversized-note path, never the exception.
  • Guidance wording: the aggregate case gets its own sentence because the remedy differs — per-file is "try a smaller file", aggregate is "send them in a follow-up message".
  • _apply_message_file_cap: the server side of the SPA's MAX_FILES_PER_MESSAGE, applied across both files and file_upload_ids before the S3 fetch (so a client cannot fan out unbounded reads by sending IDs). Direct files beyond the cap are named in the note; IDs beyond it are counted. FILE_UPLOAD_MAX_FILES_PER_MESSAGE is now read from one shared constant; the app_api FileService attribute points at it instead of holding a second, unread copy. The resolver's max_files accepts None and logs when it truncates instead of doing so silently.
  • Observability: one content-free EMF metric AttachmentTurnOverQuota (Bytes, the requested inline total) in AgentCoreStack/Compaction with capBytes / inlineFileCount / droppedFileCount as properties, gated by the same observability kill switch as the compaction records, plus a count-only warning line. The spec's rate came from a proxy; this is the direct count.
  • Kill switch: ATTACHMENT_TURN_GUARD_ENABLED=false restores the prior behaviour exactly (resolver [:5], no aggregate budget). Default on, house style. 0 on either tuning knob disables that cap alone.
  • Docs: the env table in FILE_UPLOAD_IMPLEMENTATION.md gains the three variables.

What this does not change

  • The SPA's 5-file cap is untouched.
  • The per-file INLINE_DOCUMENT_MAX_BYTES gate, the tabular and presentation diversions, and the [Attached files: …] marker semantics are unchanged; over-budget files are excluded from the marker the same way oversized ones already were.
  • No CDK change: like INLINE_DOCUMENT_MAX_BYTES, the new variables are env-backed with defaults and not plumbed through the inference-api environment construct.
  • No new packages; no dependency pins touched.
  • The spec file itself is not edited here — its §5/§8 re-cut is on the feat(documents): document_read tool + document-context analytics (offload PR-1) #1137 branch and would conflict.

Tests

tests/apis/inference_api/test_attachment_turn_guard.py: budget under / at / over the cap, first-fit ordering, images counting, single over-budget file moved not raised, budget disabled; a parametrised "nothing reaches SessionException" property — every kept set base64-encodes under the 10 MB event quota, and every input lands in exactly one bucket; the partition → budget chain on three files that each pass the per-file gate; the count cap on direct files, upload IDs, mixed, and disabled; every new guidance sentence including the named / counted / singular forms and that the existing notes are byte-identical with the new defaults; marker exclusion; the kill switch with the empty-string case; the metric's namespace, unit, properties, silence with observability off, and never-raises. test_files.py gains the resolver max_files=None case. The pre-existing carve-out and recovery tests pass unchanged.

Full backend suite locally: 8793 passed, 3 skipped, 23 failed, 10 errors — all 33 non-passing are in test_sheet_preview.py and test_sheet_preview_route.py, which need a real openpyxl that this worktree's venv lacks (environment-only, same on develop; CI has the package).

🤖 Generated with Claude Code

…nt quota (PR-6)

The stored unit is the message, not the file: a turn's inline attachments
are persisted as one AgentCore Memory event, base64-inflated 4/3 into a
blob payload and bounded by the 10 MB event quota, so ~7.5 MB of raw bytes
per turn is the break point. Past it create_message re-raises
SessionException — a hole in history. The per-file 4 MB gate and the SPA's
5-file cap do not protect: ~1.3–1.4% of prod attachment turns exceed it,
several with only 3–4 files (document-context-offload-validation.md, Claim 7).

- `_apply_inline_byte_budget`: first-fit in attachment order against
  INLINE_ATTACHMENTS_MAX_TOTAL_BYTES (default 7.5 MB, derivation in the
  constant's comment); images count. Trimmed files join the existing
  oversized-note path with their own wording ("send in a follow-up").
- `_apply_message_file_cap`: the server side of the SPA's
  MAX_FILES_PER_MESSAGE, applied to both `files` and `file_upload_ids`
  before the S3 fetch. The resolver's silent `[:5]` becomes a note to the
  user; the app_api's dead `max_files_per_message` now reads the same
  shared constant.
- One content-free EMF metric, AttachmentTurnOverQuota (Bytes) in
  AgentCoreStack/Compaction, plus a count-only log line.
- ATTACHMENT_TURN_GUARD_ENABLED=false restores the prior behaviour.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@philmerrell
philmerrell merged commit b577d29 into develop Sep 17, 2026
6 checks passed
philmerrell added a commit that referenced this pull request Sep 17, 2026
Resolves the document-context offload stack (#1137-#1141, #1143) against
the feedback work. Every conflict was both sides adding adjacent members
to the same model, flag module, allowlist or interface, so the resolution
keeps both, except FEEDBACK_ROW_PROJECTION's turn-class comment: PR-1
widened that projection to the full document-context field set, which
already carries hasDocuments and documentDigests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant