Skip to content

feat(feedback): content-free thumbs on assistant messages, joined to cost rows (offload PR-7) - #1142

Merged
philmerrell merged 5 commits into
developfrom
feature/message-feedback-pr7
Sep 17, 2026
Merged

philmerrell merged 5 commits into
developfrom
feature/message-feedback-pr7

Conversation

@philmerrell

@philmerrell philmerrell commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Document-context offload PR-7 (docs/specs/document-context-offload.md §5 row 7, §6.1 "the outcome signal — not buildable today"; compaction thresholds §7.2 "outcome signal joined to compaction"). Independent of the PR-1…6 stack; branches from develop.

Also partially builds docs/specs/response-feedback.md (§5 storage decision verbatim, §6 reason buckets as codes, §10's signal discriminator, §11 PR-1's capture half). That spec's §13 records what landed and what did not: retry-with-correction (its PR-1 "consequence"), implicit signals, eval sampling and the author surfaces remain. Free text is deliberately not stored — see §13 for why.

Why

The cost work has no quality signal joined to its cost data: the compaction stack and the offload analytics can say what a turn cost and what bounded its prefix, but not whether the user thought the answer was any good. MessageMetadata carried only a # feedback: placeholder comment. This adds the surface, keyed so the join is a one-key lookup on the rows #1137 writes.

What

  • F# row family (apis/shared/sessions/feedback.py, schema documented with C#/D# in metadata.py): PK=USER#{user}, SK=F#{sessionId}#{messageId}, GSI_PK=SESSION#{id}, GSI_SK=F#{messageId}. messageId is the same 0-based index the C# cost row carries for the assistant message. The row holds value (+1/−1), updatedAt and an optional reason code from a closed enum — the six buckets of response-feedback §6 (wrong / instructions / length / tool_failed / outdated / other) — plus signal: "explicit" so the spec's implicit-signal phase shares the family without a backfill (every reader filters on it) — content-free by construction: the request model is a Literal and the storage layer raises on anything else. Deterministic SK ⇒ a second click replaces; one thumb per (user, message). 365-day TTL like C#.
  • Session rollups thumbsUp / thumbsDown ADDed on the S# row by delta (replace and delete adjust both), only while COST_DIAGNOSTICS_ENABLED — absent reads "not tracked", never 0.
  • Routes PUT / DELETE /sessions/{id}/messages/{messageId}/feedback on app_api via get_current_user_from_session (cookie-aware; Bearer-only would 401-loop the SPA). Ownership through the session row; another user's session is 404. RESPONSE_FEEDBACK_ENABLED (the spec's name) is the kill switch (default on; =false ⇒ routes 404 and the read merge is skipped).
  • Read merge: GET /sessions/{id}/messages merges the thumb onto metadata.feedback, so a reload restores the pressed state.
  • Admin profile GET /admin/costs/sessions/{id}/profile gains feedback: {up, down, byTurnClass: {full, digestOnly, retrieved, none: {up, down}}, unjoined} and dataCoverage.feedback. The join reads the turn class from the C# row's hasDocuments / documentDigests / documentReads; those fields arrive with feat(documents): document_read tool + document-context analytics (offload PR-1) #1137 (offload PR-1). Rows written before it carry none, so byTurnClass is null and the UI says "turn class not tracked" rather than bucketing everything under none. Precedence full > retrieved > digestOnly (a retrieving call still holds the digest). unjoined counts thumbs whose message has no cost row. Rows are authoritative; the session rollups are the fallback when no rows exist.
  • Content policy: FEEDBACK_ROW_PROJECTION added to ALL_PROJECTIONS (the existing walk covers it); SESSION_ROW_PROJECTION gains thumbsUp/thumbsDown; CALL_ROW_PROJECTION gains the three turn-class fields (feat(documents): document_read tool + document-context analytics (offload PR-1) #1137 widens it to the full document-context set — expect a trivial adjacent-line conflict whichever merges second).
  • SPA: thumbs pair beside Copy in the message actions (Angular 21 signals, ng-icon, tooltips on the icon-only buttons), keyed on the run's last message (its msg-{sessionId}-{index} id is the cost row's messageId). Optimistic with rollback; a thumbs-down reveals the six reason chips; a second click on the pressed thumb withdraws. A kill-switch 404 hides the pair. Admin anatomy page gets a Feedback tile: down-thumb rate headline, up · down · <rate> of n per turn class underneath, "not tracked" otherwise.

What this does not change

  • No CDK: the row keys on the existing PK/SK and SessionLookupIndex; the flag is default-on with unset ⇒ on, so nothing needs threading.
  • No anatomy-row column, no fleet-level dashboard query (the spec marks the dashboard column optional; the session rollups make that a one-table query when wanted).
  • The offload spec file: feat(documents): document_read tool + document-context analytics (offload PR-1) #1137 rewrites §5/§6.1/§8, so the row-7 "built" note belongs there after it merges. response-feedback.md IS updated here (status + §13).
  • Nothing in the inference path or the cost writer; _store_message_metadata_cloud is untouched.

Tests

  • Storage (moto): key shape beside the cost row, replace-not-append with rollup deltas, delete and decrement, rollups absent while diagnostics off, other user ⇒ not owned, free text / bad value refused, preview sessions echo without persisting, per-user vs admin query, implicit-signal rows never read as thumbs, get_all_message_metadata merge and its flag-off path, model round trip.
  • Routes: 200 with body, 422 for free text / 0 / 5 / non-int / negative index, 204 delete, 404 other user, 404 kill switch, 503 no table.
  • Content policy: existing model + projection walks pass; new reader test seeds a stray displayText on an F# row and asserts it never leaves storage, and that messageId joins the C# row.
  • Profile join: rows with the turn-class fields bucket correctly (incl. retrieved > digestOnly); rows lacking them count up/down with byTurnClass=None; no rows ⇒ rollup fallback with honest coverage; reader failure never breaks the profile. Drill-down route test updated for the new coverage key.
  • SPA (run from the worktree against a per-package link of the main install with docx-preview/pptx-preview stubbed, since the main install lacks them): message-actions spec (+7: pair renders with tooltips and no text input, hidden for non-server ids and when unavailable, +1 on the run's last message, pressed click withdraws, down reveals chips and a pick re-sends with the code), new message-feedback.service spec (8), session-profile.util spec (+3), plus the neighbouring anatomy-page / assistant-message / message-list / session.service specs — 151 passed across those files. ng build (AOT) clean.
  • Full backend suite on the final tree: 8805 passed, 3 skipped. (The first run had 1 failure — the drill-down route test pinning the exact dataCoverage key set — fixed in the second commit.) Not exercised live in the browser.

🤖 Generated with Claude Code

philmerrell and others added 3 commits September 16, 2026 16:37
…ost rows (offload PR-7)

A thumbs up/down on assistant messages, persisted as an F# row on the
sessions-metadata table keyed (sessionId, messageId) beside the C# cost row,
so the admin session profile joins the outcome signal to the call's turn
class in one key lookup. Value ±1, timestamp and an optional reason code
from a closed enum — never free text, by the request model and the storage
layer both.

- apis.shared.sessions.feedback: F#{sessionId}#{messageId} rows (documented
  with the C#/D# schemas in metadata.py); a second click replaces; session
  rollups thumbsUp/thumbsDown ADDed while COST_DIAGNOSTICS_ENABLED.
- GET /sessions/{id}/messages merges the thumb onto metadata.feedback so a
  reload restores the pressed state.
- PUT/DELETE /sessions/{id}/messages/{messageId}/feedback via
  get_current_user_from_session; MESSAGE_FEEDBACK_ENABLED kill switch (404).
- Admin profile: feedback {up, down, byTurnClass, unjoined} + dataCoverage.
  feedback; byTurnClass is null ("not tracked") until the C# rows carry
  hasDocuments / documentDigests / documentReads (#1137).
- content_policy: FEEDBACK_ROW_PROJECTION walked by the existing test;
  session projection gains thumbsUp/thumbsDown; call projection gains the
  three turn-class fields.
- SPA: thumbs pair with reason chips in the message actions, optimistic with
  rollback; Feedback tile with down-thumb rate by turn class on the anatomy
  page.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ason buckets, explicit/implicit signal discriminator

RESPONSE_FEEDBACK_ENABLED per spec §5; reason codes are the spec's §6
buckets (still a closed enum — free text stays out by decision, recorded in
the spec's new §13); rows carry signal="explicit" and every thumb reader
filters on it so the §10 implicit phase needs no backfill and never sums in.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
philmerrell and others added 2 commits September 16, 2026 18:50
Resolves the document-context offload stack (#1137-#1141, #1143) against
the feedback work. Every conflict was both sides adding adjacent members
to the same model, flag module, allowlist or interface, so the resolution
keeps both, except FEEDBACK_ROW_PROJECTION's turn-class comment: PR-1
widened that projection to the full document-context field set, which
already carries hasDocuments and documentDigests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The merge resolution appended develop's `documents` member after the
interface's closing brace, landing it on FeedbackProfile and breaking the
AOT typecheck.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@philmerrell
philmerrell merged commit 69eeabb into develop Sep 17, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant