Problem
Two correctness gaps in the analytics savings-snapshot collector (internal/analytics/collector.go) surfaced during CodeRabbit review of PR #1049. Both are "heavy lift" changes deferred from that PR (which made them safe/observable) into this follow-up so they can be designed properly.
1. Fixed 100k history fetch cap silently undercounts
Collect calls GetAllPurchaseHistory(ctx, purchaseHistoryFetchLimit) with a hardcoded purchaseHistoryFetchLimit = 100000. Once purchase_history grows past that size the source set is silently truncated, so snapshots undercount with no error signal. Older-but-still-active 1y/3y commitments are exactly the rows most likely to fall off.
Interim mitigation in PR #1049: the collector now logs a loud WARNING when the result fills the cap, so the undercount is observable. The proper fix is an active-only store query (filter by status/expiry server-side) or a paginated/cursor scan, which requires a new method on the config store interface.
2. Persisted units are schedule-dependent
Collect converts monthly values to hourly rates (EstimatedSavings / HoursPerMonth, UpfrontCost / (Term*HoursPerYear), MonthlyCost / HoursPerMonth), but the query layer later does SUM(total_savings), SUM(total_commitment), SUM(total_usage) as if they were additive monthly totals. With the default daily schedule a $720/month purchase contributes only ~$30 to that month's series, and changing the schedule changes the numbers.
Fix options (design decision needed): store schedule-invariant monthly amounts (amortize upfront by term in months), OR weight by elapsed time during aggregation. Either touches the materialized-view definitions in migration 000067 and the integration suite, so it was not changed blindly under the PR #1049 timeline. No consumer is currently misled: the Trends frontend is deferred (#1048).
Evidence
CodeRabbit comments on PR #1049:
internal/analytics/collector.go:25 / 92-95 - "Remove the fixed 100k history cap or page this read."
internal/analytics/collector.go:146,153 - "The persisted units don't match the query-layer aggregations."
Files
internal/analytics/collector.go
internal/analytics/postgres_analytics.go (query-layer SUM/AVG semantics)
internal/database/postgres/migrations/000067_* (materialized-view math, if the monthly-amounts approach is chosen)
Acceptance
- Active commitments are never silently dropped (active-only or paginated read).
- The monthly trends series is schedule-invariant: changing the collection cadence does not change the reported monthly totals.
- Regression tests cover both: a >cap dataset, and identical totals under two different collection cadences.
Problem
Two correctness gaps in the analytics savings-snapshot collector (
internal/analytics/collector.go) surfaced during CodeRabbit review of PR #1049. Both are "heavy lift" changes deferred from that PR (which made them safe/observable) into this follow-up so they can be designed properly.1. Fixed 100k history fetch cap silently undercounts
CollectcallsGetAllPurchaseHistory(ctx, purchaseHistoryFetchLimit)with a hardcodedpurchaseHistoryFetchLimit = 100000. Oncepurchase_historygrows past that size the source set is silently truncated, so snapshots undercount with no error signal. Older-but-still-active 1y/3y commitments are exactly the rows most likely to fall off.Interim mitigation in PR #1049: the collector now logs a loud WARNING when the result fills the cap, so the undercount is observable. The proper fix is an active-only store query (filter by status/expiry server-side) or a paginated/cursor scan, which requires a new method on the config store interface.
2. Persisted units are schedule-dependent
Collectconverts monthly values to hourly rates (EstimatedSavings / HoursPerMonth,UpfrontCost / (Term*HoursPerYear),MonthlyCost / HoursPerMonth), but the query layer later doesSUM(total_savings),SUM(total_commitment),SUM(total_usage)as if they were additive monthly totals. With the default daily schedule a$720/monthpurchase contributes only ~$30to that month's series, and changing the schedule changes the numbers.Fix options (design decision needed): store schedule-invariant monthly amounts (amortize upfront by term in months), OR weight by elapsed time during aggregation. Either touches the materialized-view definitions in migration
000067and the integration suite, so it was not changed blindly under the PR #1049 timeline. No consumer is currently misled: the Trends frontend is deferred (#1048).Evidence
CodeRabbit comments on PR #1049:
internal/analytics/collector.go:25/92-95- "Remove the fixed 100k history cap or page this read."internal/analytics/collector.go:146,153- "The persisted units don't match the query-layer aggregations."Files
internal/analytics/collector.gointernal/analytics/postgres_analytics.go(query-layer SUM/AVG semantics)internal/database/postgres/migrations/000067_*(materialized-view math, if the monthly-amounts approach is chosen)Acceptance