Repository navigation
fix(db): widen purchase_history.account_id to VARCHAR(255) - #1677
Conversation
purchase_history.account_id has been VARCHAR(20) since 000001, which fits an AWS account ID (12 digits) and nothing else. An Azure subscription ID is a 36-character GUID and a GCP project ID runs to 30 characters, so SavePurchaseHistory's INSERT is rejected with SQLSTATE 22001 after the commitment has already been purchased and billed. The error is not swallowed: savePurchaseHistory returns it and the caller stamps a history_write_failed audit-gap marker on the execution (#621). But the purchase_history row itself is lost, so the billed commitment is invisible in the History view, absent from GetActivePurchaseHistory (and therefore undercounted in analytics), and unseen by the grace-period and suppression logic. The identical widening was already applied to the sibling column: 000067 widened savings_snapshots.account_id for exactly this reason and 000074 repaired it on partially-migrated databases. purchase_history was missed. - up: probe-guarded ALTER so it is correct on a fresh database, an already-deployed one, and on re-run under auto-heal. Resolves the table via 'purchase_history'::regclass so the probe follows search_path exactly as the ALTER does, and re-reads the catalog afterwards so the migration cannot be recorded as applied without having widened the column. - down: narrows only from VARCHAR(255), and refuses with a named error rather than truncating when any account_id exceeds 20 characters. Documents the CUDLY_FORCE_MIGRATION_VERSION=95 recovery for the dirty state a refusal leaves behind. Rows already lost to 22001 were never inserted and cannot be recovered by this migration; affected executions are identifiable via the history_write_failed marker on purchase_executions.error. Regression test replicates the real failing scenario with SavePurchaseHistory's own INSERT column set and a 36-char Azure GUID / 30-char GCP project ID: both are rejected with 22001 at version 92 and round-trip untruncated after 000095. Also covers the lossless rollback path and up-migration idempotency. Closes #1603
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 56 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (5)
Comment |
The shared insert helper hardcoded provider='azure' with Azure's service, region and resource_type, so the GCP case inserted an Azure-shaped row and only the account_id differed. In a test whose entire point is that Azure and GCP identifiers overflow VARCHAR(20), a GCP case carrying 'westeurope'/'Standard_D4s_v3' misrepresents the scenario it claims to cover. Group the identifier with its provider/service/region/resource_type in a commitmentRow fixture and pass those through as bind parameters, so each case inserts a row its own provider would actually produce. Adds an AWS fixture for the lossless-rollback test, which previously used a bare 12-digit literal. Test-only; no migration or production code changes.
TestMigrations_AutoHealDirty rolls back one migration and asserted the resulting version equals headVersion-1. That encodes an invariant this repository does not hold: migration numbers are not contiguous, because renumbering to dodge collisions with in-flight PRs leaves gaps. The set already skips 61-62, 68-69, 82, 84-85. Those gaps were harmless only because none of them sat immediately below head, so headVersion-1 happened to be a real migration. Adding 000095 on top of main's 000093 puts a gap directly below head for the first time, and golang-migrate's Steps(-1) lands on 93 -- the next version that actually exists -- so the assertion failed with expected 94, actual 93. Derive the expected version from the .up.sql filenames instead. This is the root fix rather than a renumber: the next migration to land above a gap would have hit the same assertion, and renumbering 000095 to 000094 would have collided with the in-flight #1523. Test-only; no migration or production code changes.
|
@coderabbitai full review |
✅ Action performedFull review finished. Your included review limit is currently reached under our Fair Usage Limits Policy. Your recent PR review activity is in the 95th percentile or higher among CodeRabbit users, so adaptive limits apply. This review may still proceed through usage-based billing if eligible. Your next included review will be available in 56 minutes. |
Adversarial review record (merged)Independent reviewer, distinct from the author. Recording it here because the merge rested on this, not on the CI checks — this PR's Verified against a real Postgres testcontainer, not by reading the diff. The regression test genuinely bites. Pre-migration schema rejected both realistic values live: Migration numbering: A latent CI trap was found and fixed: Down migration fails loudly rather than truncating. Scanner/projection parity confirmed — Two non-blocking notes: Sibling issues filed: #1676, #1679. |
Closes #1603
The defect
purchase_history.account_idhas beenVARCHAR(20)since migration000001. That fits an AWS account ID (12 digits) and nothing else:The value reaching
SavePurchaseHistoryiscloud_accounts.external_id, which isVARCHAR(255)at its source (000011), so nothing upstream constrains it. The INSERT is rejected withSQLSTATE 22001after the commitment has already been purchased and billed.The identical widening was already applied to the sibling column and skipped here:
000067widenedsavings_snapshots.account_idtoVARCHAR(255)citing this exact reason, and000074repaired it on partially-migrated databases.purchase_historywas missed.Is the failure swallowed?
No — and this PR does not change that.
savePurchaseHistoryreturns the error (internal/purchase/execution.go:661) andrecordHistoryAuditGapstamps ahistory_write_failedmarker on the execution rather than flipping it tofailed, which is deliberate per #621 (flipping it would tempt a re-approval of an already-fired purchase). So there is no second silent-failure defect to fix here. The loss is thepurchase_historyrow itself: the billed commitment is invisible in the History view, absent fromGetActivePurchaseHistory(so analytics undercount committed spend), and unseen by the grace-period and suppression logic.Already-lost rows
Not recoverable by this migration — they were never inserted. Affected executions are identifiable:
The marker carries the provider-side commitment ID. The double-sided wildcard is deliberate:
recordHistoryAuditGapappends viaappendErrNote, so the marker is not necessarily at the start of the field. No backfill is attempted here; tracked separately in LeanerCloud/cloud-commitments-platform#146.Migration
Number 000095, checked against
origin/main(tops out at000093) and every open PR branch (000094is claimed by #1523).Up — probe-guarded so it is correct on a fresh database, an already-deployed one, and on re-run under auto-heal (
IF NOT EXISTScannot repair a wrong column type). Two deliberate divergences from000067:'purchase_history'::regclass, which followssearch_pathexactly as the bareALTER TABLE purchase_historydoes.buildMigrateDSNappends no connection options (RDS Proxy), sosearch_pathis the role default; a probe resolving differently from the ALTER could silently no-op while golang-migrate recorded the migration as applied — the p0 would persist under a green deploy. It also re-reads the catalog after the ALTER and raises if the column is still too narrow, and raises if the column is absent entirely.atttypmod = -1(unboundedvarchar) as already-wide.000067'scharacter_maximum_length IS NULLbranch would have narrowed such a column.Down — narrows only from
VARCHAR(255)(the exact state the up produces; guarding on "anything wider than 20" would impose a state000095never created). It refuses with a named error rather than truncating when anyaccount_idexceeds 20 characters, and documents theCUDLY_FORCE_MIGRATION_VERSION=95recovery for the dirty state a refusal leaves behind (force to current, since the up's effects are present).Dependent objects: none. No view, materialized view, FK or partition references
purchase_history, so no drop/recreate dance (contrast000067, which had to drop three views). Widening a varchar is binary-coercible, so Postgres skips the table rewrite and leavesidx_purchase_history_account_timestampin place.Go side: no change needed.
scanPurchaseHistoryRowscansaccount_idinto a plainstring(NOT NULLin000001, never relaxed since), and all eightpurchase_historySELECT projections stay in lockstep. Verified rather than assumed.Other identifier columns
Swept every
VARCHARcolumn in the migrations holding an account/subscription/project/tenant identifier. Only three areVARCHAR(20):purchase_history.account_idsavings_snapshots.account_id000067ri_exchange_history.account_idCHECK (account_id ~ '^\d{12}$')— explicitly AWS-12-digit by construction, not a defectEverything else is
VARCHAR(255), orVARCHAR(36)for Azure GUID columns incloud_accounts(exact fit).Verification
A green suite is not evidence, so:
Pre-fix failure, verbatim (migrated to version 92, then the real INSERT column set):
Both round-trip untruncated after
000095.Falsification — with the up migration neutralized to
SELECT 1;, the post-migration assertions fail:So the test is not vacuous — it could not have stayed green with the bug present.
The test uses
SavePurchaseHistory's own INSERT column set, not a narrower proxy. It pins to version 92 viaMigrateToVersionrather than migrating to HEAD (so the next migration PR's CI is unaffected), and the two rollback tests useRollbackMigrations(1)so they reverse exactly000095even once000094lands.Also covered: the lossless rollback path (narrowing actually happens with only AWS IDs present) and up-migration idempotency (re-executing the file is a no-op) — without these, the
ALTERbranch after the down's guard and the header's auto-heal claim would ship untested.Gates
go test -tags integration ./internal/database/postgres/migrations/— full package, exit 0go test ./internal/database/postgres/... ./internal/config/...— 703 passedgolangci-lintat the CI-pinned v2.10.1, with and without--build-tags=integration—0 issues, exit 0go build ./...,go vet,gocyclo -over 10— clean (no non-test Go code added)--no-verify