fix: Anthropic usage frame spec follow-ups (always-zero scalars, no 2… - #1474
Open
hachall wants to merge 1 commit into
Open
fix: Anthropic usage frame spec follow-ups (always-zero scalars, no 2…#1474hachall wants to merge 1 commit into
hachall wants to merge 1 commit into
Conversation
…4h tier) Three follow-ups from the billing-fix audit, none changing token counts: - cache_read_input_tokens / cache_creation_input_tokens are now plain counts that always serialize (0 when caching is unused), matching the live Anthropic API instead of the SDKs' looser Optional typing - the cache_creation object no longer emits an ephemeral_24h_input_tokens field: 24h writes are disabled at the server config level, so an always-zero tier on the customer frame would imply a product we have turned off (Anthropic's spec defines exactly the 5m and 1h fields) - the streaming module's 'input_tokens is non-standard on message_delta' comment predates the current Messages schema, which carries input and cache fields on message_delta.usage; the delta now always emits the cache scalars alongside them
Contributor
There was a problem hiding this comment.
Pull request overview
Aligns dwctl’s Anthropic Messages “usage” framing with Anthropic’s live API behavior so downstream billing/analytics see consistent cache fields and the customer-facing schema doesn’t imply disabled products.
Changes:
- Always serialize
cache_read_input_tokens/cache_creation_input_tokensas scalar counts (0 when unused) across blocking + streaming Anthropic responses. - Stop emitting the
ephemeral_24h_input_tokenstier in Anthropiccache_creationbreakdowns. - Update/extend Anthropic translation tests to assert the new serialization behavior and the absence of the 24h tier.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| dwctl/src/request_logging/serializers.rs | Updates Anthropic prompt token reconstruction to use always-present cache scalars. |
| dwctl/src/inference/translation/anthropic/streaming.rs | Emits cache scalars unconditionally in message_delta.usage (0 when unused). |
| dwctl/src/inference/translation/anthropic/response.rs | Makes anthropic_usage return scalar cache counts (0 default) and drops Optional cache scalars. |
| dwctl/src/inference/translation/anthropic/model.rs | Changes Anthropic Usage cache scalar fields from Option<u64> to u64 (defaulting to 0) and removes 24h tier from CacheCreation. |
| dwctl/src/inference/translation/anthropic/mod.rs | Updates/adds tests covering zero cache scalars and “never emit 24h tier”. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+221
to
+222
| /// Always serialized (0 when caching is unused), matching the live | ||
| /// Anthropic API rather than the SDKs' looser Optional typing. |
Comment on lines
137
to
+141
| /// serializer relies on exactly that sum to recover `prompt_tokens` (total input); | ||
| /// subtracting only one bucket here would make it double-bill the other. Returns | ||
| /// `(input, output, cache_read, cache_creation)`; the cache values are `None` | ||
| /// when zero/absent so they serialise out. Shared by the blocking and streaming | ||
| /// paths. | ||
| pub(super) fn anthropic_usage(usage: &Value) -> (u64, u64, Option<u64>, Option<u64>, Option<super::model::CacheCreation>) { | ||
| /// `(input, output, cache_read, cache_creation)`; the cache scalars are plain | ||
| /// counts (0 when unused) and always serialize, matching the live Anthropic API. | ||
| /// Shared by the blocking and streaming paths. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…4h tier)
Three follow-ups from the billing-fix audit, none changing token counts: