feat: add exact Fast usage and tier-aware cost tracking

.agent-maintainer/dashboard-source-baseline.json

@@ -13,12 +13,12 @@
     "frontend/dashboard/src/api/client.ts": {
       "group": "source",
       "nonblank": 765,
-      "physical": 826
+      "physical": 825
     },
     "frontend/dashboard/src/api/types.ts": {
       "group": "source",
-      "nonblank": 424,
-      "physical": 450
+      "nonblank": 422,
+      "physical": 448
     },
     "frontend/dashboard/src/app/zh-Hans/advancedPages.ts": {
       "group": "source",

CHANGELOG.md

@@ -2,6 +2,24 @@
 
 ## Unreleased
 
+- Ingest aggregate `response.completed` telemetry from local
+  `codex-completions*.jsonl` exporter files and conservatively reconcile exact
+  Fast/Standard service-tier evidence to canonical calls without retaining raw
+  response bodies or arbitrary OTLP attributes.
+- Show exact service tier separately from the existing Fast proxy in Calls,
+  details, and CSV exports; older or unmatched history remains Unknown.
+- Apply documented model-family Fast multipliers to confirmed Codex credit
+  estimates with source URL/date/confidence and local overrides while leaving
+  standard-credit allowance calibration unchanged.
+- Cache Standard, Batch, Flex, and Priority API pricing together, select the
+  observed tier per call, and expose Standard/Priority cost scenarios plus an
+  explicit local billing basis without applying ChatGPT Fast multipliers to API
+  USD estimates.
+- Preserve exact response tiers in dashboard labels and CSV contracts, clear
+  OTel staging on confirmed database reset, and make incremental source cursors
+  descriptor-safe across concurrent file rotation. Verify a bounded content
+  anchor before resuming so same-inode rewrites cannot silently skip telemetry.
+
 ## 0.20.0 - 2026-07-16
 
 - Reinvent the Threads tab around inline row expansion, progressively loading and virtualizing every call in the selected thread while preserving explicit investigator actions, deep links, responsive layouts, retry recovery, and aggregate-first privacy boundaries.

docs/architecture.md

@@ -17,6 +17,15 @@ Shareable outputs remain aggregate-first and must omit indexed/raw content unles
 ## Boundaries
 
 - `parser.py` converts local JSONL events into aggregate `UsageEvent` records. It also attaches metadata-only call-origin categories, diagnostic facts from `diagnostic_facts.py`, archived-session flags, conservative thread keys, source cursors, and parser diagnostics.
+- `parser/otel.py`, `store/otel_ingest.py`, and `store/otel_reconciliation.py`
+  provide aggregate-only completion enrichment from local
+  `~/.codex-usage-tracker/otel/codex-completions*.jsonl` files. The importer
+  retains only approved completion identifiers, token counters, model/effort,
+  application version, and service-tier metadata; response bodies and arbitrary
+  OTLP attributes are never persisted. Reconciliation requires one canonical
+  call group with the same conversation id and all four token counters. Model
+  and effort narrow a match when both sides provide them, but timestamps are not
+  identity evidence.
 - `store/content_index.py` owns normalized local content-index population and cleanup. It may persist bounded local snippets, tool-call metadata, command roots/labels, file path hashes/basenames, parser adapter metadata, source provenance, parse warnings, and FTS5 search rows. Investigation/report builders may persist bounded run summaries in `investigation_runs`. These surfaces must not feed raw/indexed content into default CSV, dashboard HTML, support bundle, or aggregate report payloads.
 - `call_origin.py` owns the pure call-origin classifier and migrated-row fallback. It must not open source JSONL files; source-log reads belong in refresh/indexing or explicit context loading.
 - `schema.py` owns persisted SQLite columns and migrations. Add columns or tables there before changing refresh, export, or MCP behavior.
@@ -62,6 +71,10 @@ Shareable outputs remain aggregate-first and must omit indexed/raw content unles
     totals through `canonical_usage_events`. New physical-row investigation
     surfaces must be bounded, explicitly labeled, and tested against an
     original/copy/new-call fixture.
+15. Treat service tier as exact only when a completion explicitly reports it, or
+    when Codex `0.143.0` or newer omits it under the documented Standard
+    protocol. Older or unmatched calls remain Unknown. Latency and reasoning
+    effort are useful throughput signals, not proof of Fast usage.
 
 Normal aggregate responses use a 64-entry, 256-KiB-per-entry process cache.
 Allowance history and diagnostics use a separate four-entry, 8-MiB-per-entry

docs/cli-json-schemas.md

@@ -1466,4 +1466,38 @@ Most setup and file-writing commands accept `--json` and return a schema-specifi
 - `init-thresholds --json`, `init-projects --json`
 - `support-bundle --json`
 
+Refresh and rebuild payloads add bounded OTel counters under
+`parser_diagnostics`: `otel_files_scanned`, `otel_imported`,
+`otel_duplicates`, `otel_matched`, `otel_pending`, `otel_ambiguous`, and
+`otel_conflicts` (zero-valued counters may be omitted from the immediate JSON
+payload). Stored refresh metadata keeps the same keys with zero defaults so
+localhost status is stable. These surfaces never expose OTel source paths,
+semantic fingerprints, response bodies, arbitrary attributes, or staging ids.
+
+Aggregate call rows may add nullable `service_tier`, `fast`,
+`service_tier_source`, and `service_tier_confidence` fields. `service_tier`
+preserves the exact normalized response value such as `priority`, `default`,
+`standard`, `flex`, or `batch`; `fast` is the derived speed classification.
+
+Cost-annotated rows may add `standard_cost_usd`, `priority_cost_usd`,
+`pricing_service_tier`, `billing_basis`, and
+`cost_semantics="api_token_estimate"`. `estimated_cost_usd` remains the
+API-token-equivalent estimate selected from the observed tier when pricing-v2
+contains that table. `billing_basis` is `unknown`, `chatgpt_credits`, or
+`api_tokens` and describes applicability rather than changing the observed tier.
+
+Credit-annotated rows may also add `standard_usage_credits`,
+`fast_usage_credits`, `usage_credit_multiplier`,
+`usage_credit_multiplier_source`, `usage_credit_multiplier_source_url`,
+`usage_credit_multiplier_fetched_at`, and
+`usage_credit_multiplier_confidence`. These are ChatGPT-equivalent credit
+scenarios and are never used as generic multipliers for API USD. `fast: null`
+means Unknown; clients must not replace it with the separate throughput proxy.
+CSV export includes the exact tier, both scenario families, multiplier
+provenance, and the separately named proxy candidate.
+
+`rebuild-index` retains aggregate OTel staging for reconciliation. Confirmed
+`reset-db --yes` clears both `otel_completion_events` and
+`otel_completion_sources` along with the other tracker-owned rows.
+
 `context` already returns JSON because it is an explicit on-demand context request. Treat `codex-usage-tracker-context-v1` output as sensitive local context even though it is redacted and size-limited by default. `max_entries=0` requests all matching entries and `max_chars=0` removes the character cap for that explicit request. Tool output and compacted replacement history are omitted unless explicitly requested. Compaction entries may include metadata such as `replacement_history_available`, `replacement_entry_count`, and `replacement_history_included`; replacement text appears only when `include_compaction_history` is true for that local request. Evidence responses include `action_timing`, derived from timestamps in the same selected-turn source scan, plus per-entry `action_timing` fields such as `since_turn_start_ms`, `since_previous_entry_ms`, and `reported_duration_ms` when available. MCP returns `codex-usage-tracker-context-disabled-v1` when raw context loading has not been explicitly enabled with `CODEX_USAGE_TRACKER_ALLOW_RAW_CONTEXT=1`.

docs/dashboard-guide.md

@@ -117,6 +117,11 @@ Use `Calls` view when you want to inspect individual model calls.
 - Time values are shown in your browser's local date/time format while sorting and time filtering still use the logged timestamp.
 - Calls include `Duration`, derived from turn start for a new turn or the previous same-turn call end for continuations, and `Prev gap`, the elapsed time since the previous call in the resolved thread.
 - Calls view token columns separate total tokens, cached input, uncached input, and output so the accounting can be scanned without expanding a row.
+- The `Service Tier` column preserves the exact response value with distinct
+  labels for Priority/Fast, Fast, Default/Standard, Standard, Flex, Batch, and
+  other bounded explicit tiers. It shows `Unknown` when completion telemetry is
+  absent. The detail view keeps the older throughput heuristic labeled as a
+  separate Fast proxy candidate rather than historical proof.
 - Source pucks are call-level estimates derived from local event metadata. `User` means the token-count segment included a user message, `Codex` means it followed tool output, compaction, or agent-continuation metadata, and `Unknown` means the source event metadata was unavailable or ambiguous.
 - Click a column header like `Time`, `Thread`, `Tokens`, `Cost`, or `Cache` to sort. Use the sort menu for `Highest Codex credits`. Click the same header again to reverse the direction.
 - Hover a row to scan a compact aggregate preview in `Call Details`; click a Calls row to open the dedicated call investigator.
@@ -136,6 +141,16 @@ Useful interpretation notes:
 - `Cached input` and `Uncached input` are split so cache behavior is visible without storing transcript text.
 - A cost with `*` means the pricing row is marked as a best-guess estimate.
 - Codex credits are estimated from aggregate input, cached-input, and output token counters. Direct model matches use the bundled OpenAI Codex rate-card snapshot; inferred labels are marked estimated, and local credit-rate overrides are marked user-provided.
+- Confirmed Fast rows apply the source-stamped model-family multiplier to
+  ChatGPT-equivalent Codex credits. API-equivalent USD estimates instead select
+  the exact API tier table; the cost detail identifies whether the configured
+  billing basis is API tokens, ChatGPT credits, or Unknown without claiming
+  actual spend when the basis is unknown.
+- CSV exports include exact tier/provenance, the separate Fast proxy,
+  Standard/Priority API cost scenarios, Standard/Fast credit scenarios,
+  `billing_basis`, and credit multiplier source URL/date/confidence. Missing
+  exact tier evidence remains blank/Unknown instead of being reconstructed from
+  duration or reasoning effort.
 - `Usage observed` is not a live account query. It uses the latest local Codex `token_count.rate_limits` snapshot when present; otherwise configure `~/.codex-usage-tracker/allowance.json` with values copied from Codex Settings > Usage, the Codex Usage dashboard, or `/status` when you want current remaining allowance context.
 
 ## Threads View

docs/database-schema.md

@@ -32,6 +32,39 @@ identity, parser coverage, replacement, and refresh revision. Changing or removi
 a source causes its owned physical rows/materializations to be replaced in a
 transaction rather than accumulated blindly.
 
+## OTel Service-Tier Enrichment (Schemas 30–31)
+
+Schema 30 adds nullable `service_tier`, `fast`, `service_tier_source`, and
+`service_tier_confidence` columns to `usage_events`. Null means the tracker does
+not have exact tier evidence; it is not interpreted as Standard.
+
+`otel_completion_sources` records device/inode identity, size, the last complete
+byte and line cursor, a bounded SHA-256 resume anchor, and an update timestamp
+for each local `codex-completions*.jsonl` file. The default directory is the
+`otel` sibling of the selected database (`~/.codex-usage-tracker/otel` for the
+default database), so alternate databases do not ingest another tracker
+instance's telemetry.
+`otel_completion_events` stores one semantic
+fingerprint plus aggregate matching fields, the exact normalized response tier,
+derived Fast classification, tier provenance, a
+bounded match status (`pending`, `matched`, `ambiguous`, `conflict`, or
+`invalid`), and the matched aggregate record id when available.
+
+Append refresh resumes after the last complete JSONL line, retries a partial
+trailing line, and restarts a cursor after rotation, truncation, or a mismatched
+resume anchor. Schema 31 adds the anchor; a legacy cursor without one is safely
+reread once and then anchored. Cursor identity, size, offsets, and anchor bytes
+come from the open descriptor so a path rotation or same-inode replacement
+cannot persist an offset against different content. Rebuild keeps
+the aggregate OTel staging rows, resets their match pointers, and reconciles
+them against the rebuilt canonical calls. A match requires conversation id plus
+input, cached-input, output, and reasoning-output counters to resolve to exactly
+one canonical group. Existing contradictory tier values are preserved and the
+completion is marked as a conflict.
+
+`reset-db --yes` is intentionally different from rebuild: it deletes both OTel
+staging tables and their source cursors along with the other tracker-owned rows.
+
 ## Allowance Intelligence Materializations
 
 `allowance_observations` stores normalized structured weekly and 5-hour snapshots,
@@ -68,7 +101,7 @@ Identical keys reuse a completed snapshot or the same in-flight job.
 
 ## Privacy And Rebuilds
 
-Allowance materializations and logical identity contain aggregate metadata only.
+Allowance materializations, OTel completion staging, and logical identity contain aggregate metadata only.
 They do not contain prompts, assistant text, tool output, or raw JSONL content.
 Rebuilds can recreate them from physical aggregate rows and source provenance.
 The local content index is a separate opt-in investigation layer and is not used

docs/pricing-and-credits.md

@@ -14,7 +14,7 @@ Enable optional cost estimates:
 codex-usage-tracker update-pricing
 ```
 
-This fetches OpenAI text-token pricing from `https://developers.openai.com/api/docs/pricing.md`, parses the selected tier, and writes a source-stamped local cache to `~/.codex-usage-tracker/pricing.json`. The default tier is `standard`; other supported tiers are `batch`, `flex`, and `priority`.
+This fetches OpenAI text-token pricing from `https://developers.openai.com/api/docs/pricing.md` once, parses the `standard`, `batch`, `flex`, and `priority` tables, and writes a source-stamped pricing-v2 cache to `~/.codex-usage-tracker/pricing.json`. The top-level `models` object remains the selected-tier compatibility projection, while `api_service_tiers` retains every published table so mixed historical calls can be priced by their observed response tier.
 
 The updater supports both the older four-value pricing rows and the newer five-value rows used by GPT-5.6 for input, cached input, cache writes, and output. GPT-5.6 Sol, Terra, and Luna are loaded from the published table, and the documented `gpt-5.6` alias resolves to `gpt-5.6-sol`. Current Codex logs do not expose a separate cache-write token counter, so cost estimates use the logged uncached input, cached input, and output counters without adding an explicit cache-write charge.
 
@@ -39,6 +39,20 @@ codex-usage-tracker init-pricing
 
 Edit `~/.codex-usage-tracker/pricing.json` with USD-per-million-token rates for local overrides or models that are not present in the OpenAI pricing table. Normal reports never contact the network; only `update-pricing` refreshes the local pricing cache.
 
+### Billing Basis And Tier Scenarios
+
+OTel `service_tier=priority` proves the response tier, but it does not prove whether the call used ChatGPT/Codex credits or an API key. Set the top-level local `billing_basis` only when you know the billing path:
+
+```json
+{
+  "billing_basis": "unknown"
+}
+```
+
+Supported values are `unknown`, `chatgpt_credits`, and `api_tokens`. `update-pricing` preserves this local value. Invalid values make the pricing config invalid instead of silently guessing.
+
+`estimated_cost_usd` is always an API-token-equivalent estimate. With pricing-v2, it selects the exact observed API table for `priority`, `default`/`standard`, `flex`, or `batch`; it never applies a generic ChatGPT Fast multiplier. Each annotated row also exposes `standard_cost_usd`, `priority_cost_usd`, `pricing_service_tier`, `billing_basis`, and `cost_semantics` so an unknown billing path can be reported as bounded scenarios rather than false actual spend. Pricing-v1 files remain readable and keep their existing single-table behavior.
+
 ## Codex Credits
 
 `Codex Credits` is a calculated usage number, not a dashboard-only unit. The tracker uses Codex's logged aggregate token counters and the bundled OpenAI Codex rate-card snapshot to estimate credits consumed by local Codex calls.
@@ -62,6 +76,34 @@ codex-usage-tracker update-rate-card
 
 The local snapshot is written to `~/.codex-usage-tracker/rate-card.json`. Each bundled rate and alias includes source URL, fetched date, tier, confidence, and alias rationale where applicable. Use `--source-file` only when you have a reviewed replacement JSON snapshot you want the tracker to validate and use.
 
+### Confirmed Fast Usage
+
+When a call has exact OTel evidence that `fast=1`, the tracker first computes
+`standard_usage_credits`, then applies the documented model-family Fast
+multiplier to produce `usage_credits`:
+
+- GPT-5.6 family: `2.5x`
+- GPT-5.5 family: `2.5x`
+- GPT-5.4 family: `2.0x`
+
+The row also exposes `fast_usage_credits`, `usage_credit_multiplier`, and
+source URL/date/confidence fields so the adjustment is auditable independently
+from OTel tier evidence. A confirmed
+Fast call whose model has no documented multiplier stays at `1.0x` and is
+marked `no_documented_fast_multiplier`; the tracker does not guess. Standard
+and Unknown-tier calls also stay at `1.0x`.
+
+The bundled `fast_multipliers` rate-card entries are source-stamped. A local
+`allowance.json` may override a model-family entry with `multiplier`,
+`source_name`, `source_url`, and `fetched_at`; valid local entries are marked
+`user_override`, while malformed entries do not replace the bundled value.
+
+This adjustment applies only to Codex/ChatGPT usage-credit estimates. It never
+changes API USD estimates, which use the observed API service-tier table and
+aggregate token counters. Allowance-drain calibration uses the
+standard-credit baseline so a newly observed Fast label does not redefine the
+historical credit-to-percentage relationship.
+
 ## Usage Observed
 
 `Usage observed` is different from `Codex Credits`. The tracker cannot currently read your logged-in ChatGPT plan, live remaining credits, reset windows, or usage from other agentic surfaces automatically.
@@ -90,6 +132,13 @@ Configure the usage component:
 - Codex upstream log formats can change, and parser compatibility may require tracker updates before new event shapes are fully understood.
 - Pricing and rate-card sources can change outside this project. Refresh or pin local files when reports need a known source snapshot.
 - Local Codex logs may not include usage from other ChatGPT agentic surfaces that share the same allowance.
+- Service tier is exact only when a local completion explicitly reports it or
+  Codex `0.143.0` or newer establishes Standard through omission. Older or
+  unmatched history remains Unknown; latency and reasoning effort cannot prove
+  Fast usage.
+- `priority` can describe ChatGPT Fast mode or API Priority processing. The
+  retained telemetry does not identify authentication or billing, so leave
+  `billing_basis` as `unknown` unless you have independent evidence.
 - Live account allowance cannot be read automatically by this local tracker, and the dashboard does not infer live remaining allowance from the logged-in account plan.
 - Observed local-log snapshots may be stale until Codex records another model call, and may omit other agentic surfaces that share the same allowance.
 - Pricing can change after a report is generated. Use `pin-pricing` when you need reproducible historical cost estimates.