feat(feedback): content-free thumbs on assistant messages, joined to cost rows (offload PR-7) - #1142
Merged
Merged
Conversation
…ost rows (offload PR-7)
A thumbs up/down on assistant messages, persisted as an F# row on the
sessions-metadata table keyed (sessionId, messageId) beside the C# cost row,
so the admin session profile joins the outcome signal to the call's turn
class in one key lookup. Value ±1, timestamp and an optional reason code
from a closed enum — never free text, by the request model and the storage
layer both.
- apis.shared.sessions.feedback: F#{sessionId}#{messageId} rows (documented
with the C#/D# schemas in metadata.py); a second click replaces; session
rollups thumbsUp/thumbsDown ADDed while COST_DIAGNOSTICS_ENABLED.
- GET /sessions/{id}/messages merges the thumb onto metadata.feedback so a
reload restores the pressed state.
- PUT/DELETE /sessions/{id}/messages/{messageId}/feedback via
get_current_user_from_session; MESSAGE_FEEDBACK_ENABLED kill switch (404).
- Admin profile: feedback {up, down, byTurnClass, unjoined} + dataCoverage.
feedback; byTurnClass is null ("not tracked") until the C# rows carry
hasDocuments / documentDigests / documentReads (#1137).
- content_policy: FEEDBACK_ROW_PROJECTION walked by the existing test;
session projection gains thumbsUp/thumbsDown; call projection gains the
three turn-class fields.
- SPA: thumbs pair with reason chips in the message actions, optimistic with
rollback; Feedback tile with down-thumb rate by turn class on the anatomy
page.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ason buckets, explicit/implicit signal discriminator RESPONSE_FEEDBACK_ENABLED per spec §5; reason codes are the spec's §6 buckets (still a closed enum — free text stays out by decision, recorded in the spec's new §13); rows carry signal="explicit" and every thumb reader filters on it so the §10 implicit phase needs no backfill and never sums in. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Resolves the document-context offload stack (#1137-#1141, #1143) against the feedback work. Every conflict was both sides adding adjacent members to the same model, flag module, allowlist or interface, so the resolution keeps both, except FEEDBACK_ROW_PROJECTION's turn-class comment: PR-1 widened that projection to the full document-context field set, which already carries hasDocuments and documentDigests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The merge resolution appended develop's `documents` member after the interface's closing brace, landing it on FeedbackProfile and breaking the AOT typecheck. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Document-context offload PR-7 (
docs/specs/document-context-offload.md§5 row 7, §6.1 "the outcome signal — not buildable today"; compaction thresholds §7.2 "outcome signal joined to compaction"). Independent of the PR-1…6 stack; branches from develop.Also partially builds
docs/specs/response-feedback.md(§5 storage decision verbatim, §6 reason buckets as codes, §10'ssignaldiscriminator, §11 PR-1's capture half). That spec's §13 records what landed and what did not: retry-with-correction (its PR-1 "consequence"), implicit signals, eval sampling and the author surfaces remain. Free text is deliberately not stored — see §13 for why.Why
The cost work has no quality signal joined to its cost data: the compaction stack and the offload analytics can say what a turn cost and what bounded its prefix, but not whether the user thought the answer was any good.
MessageMetadatacarried only a# feedback:placeholder comment. This adds the surface, keyed so the join is a one-key lookup on the rows #1137 writes.What
F#row family (apis/shared/sessions/feedback.py, schema documented withC#/D#inmetadata.py):PK=USER#{user},SK=F#{sessionId}#{messageId},GSI_PK=SESSION#{id},GSI_SK=F#{messageId}.messageIdis the same 0-based index theC#cost row carries for the assistant message. The row holdsvalue(+1/−1),updatedAtand an optionalreasoncode from a closed enum — the six buckets of response-feedback §6 (wrong/instructions/length/tool_failed/outdated/other) — plussignal: "explicit"so the spec's implicit-signal phase shares the family without a backfill (every reader filters on it) — content-free by construction: the request model is aLiteraland the storage layer raises on anything else. Deterministic SK ⇒ a second click replaces; one thumb per (user, message). 365-day TTL likeC#.thumbsUp/thumbsDownADDed on theS#row by delta (replace and delete adjust both), only whileCOST_DIAGNOSTICS_ENABLED— absent reads "not tracked", never 0.PUT/DELETE /sessions/{id}/messages/{messageId}/feedbackon app_api viaget_current_user_from_session(cookie-aware; Bearer-only would 401-loop the SPA). Ownership through the session row; another user's session is 404.RESPONSE_FEEDBACK_ENABLED(the spec's name) is the kill switch (default on;=false⇒ routes 404 and the read merge is skipped).GET /sessions/{id}/messagesmerges the thumb ontometadata.feedback, so a reload restores the pressed state.GET /admin/costs/sessions/{id}/profilegainsfeedback: {up, down, byTurnClass: {full, digestOnly, retrieved, none: {up, down}}, unjoined}anddataCoverage.feedback. The join reads the turn class from theC#row'shasDocuments/documentDigests/documentReads; those fields arrive with feat(documents): document_read tool + document-context analytics (offload PR-1) #1137 (offload PR-1). Rows written before it carry none, sobyTurnClassisnulland the UI says "turn class not tracked" rather than bucketing everything undernone. Precedence full > retrieved > digestOnly (a retrieving call still holds the digest).unjoinedcounts thumbs whose message has no cost row. Rows are authoritative; the session rollups are the fallback when no rows exist.FEEDBACK_ROW_PROJECTIONadded toALL_PROJECTIONS(the existing walk covers it);SESSION_ROW_PROJECTIONgainsthumbsUp/thumbsDown;CALL_ROW_PROJECTIONgains the three turn-class fields (feat(documents): document_read tool + document-context analytics (offload PR-1) #1137 widens it to the full document-context set — expect a trivial adjacent-line conflict whichever merges second).msg-{sessionId}-{index}id is the cost row'smessageId). Optimistic with rollback; a thumbs-down reveals the six reason chips; a second click on the pressed thumb withdraws. A kill-switch 404 hides the pair. Admin anatomy page gets a Feedback tile: down-thumb rate headline,up · down · <rate> of n per turn classunderneath, "not tracked" otherwise.What this does not change
SessionLookupIndex; the flag is default-on with unset ⇒ on, so nothing needs threading.response-feedback.mdIS updated here (status + §13)._store_message_metadata_cloudis untouched.Tests
get_all_message_metadatamerge and its flag-off path, model round trip.displayTexton anF#row and asserts it never leaves storage, and thatmessageIdjoins theC#row.byTurnClass=None; no rows ⇒ rollup fallback with honest coverage; reader failure never breaks the profile. Drill-down route test updated for the new coverage key.docx-preview/pptx-previewstubbed, since the main install lacks them):message-actionsspec (+7: pair renders with tooltips and no text input, hidden for non-server ids and when unavailable, +1 on the run's last message, pressed click withdraws, down reveals chips and a pick re-sends with the code), newmessage-feedback.servicespec (8),session-profile.utilspec (+3), plus the neighbouring anatomy-page / assistant-message / message-list / session.service specs — 151 passed across those files.ng build(AOT) clean.dataCoveragekey set — fixed in the second commit.) Not exercised live in the browser.🤖 Generated with Claude Code