Skip to content

fix(db): widen purchase_history.account_id to VARCHAR(255) - #1677

Merged
cristim merged 3 commits into
mainfrom
fix/1603-account-id-width
Aug 3, 2026
Merged

cristim merged 3 commits into
mainfrom
fix/1603-account-id-width

Conversation

@cristim

@cristim cristim commented Jul 29, 2026 •

Copy link
Copy Markdown
Member

Closes #1603

The defect

purchase_history.account_id has been VARCHAR(20) since migration 000001. That fits an AWS account ID (12 digits) and nothing else:

  • an Azure subscription ID is a 36-character GUID
  • a GCP project ID is 6 to 30 characters

The value reaching SavePurchaseHistory is cloud_accounts.external_id, which is VARCHAR(255) at its source (000011), so nothing upstream constrains it. The INSERT is rejected with SQLSTATE 22001 after the commitment has already been purchased and billed.

The identical widening was already applied to the sibling column and skipped here: 000067 widened savings_snapshots.account_id to VARCHAR(255) citing this exact reason, and 000074 repaired it on partially-migrated databases. purchase_history was missed.

Is the failure swallowed?

No — and this PR does not change that. savePurchaseHistory returns the error (internal/purchase/execution.go:661) and recordHistoryAuditGap stamps a history_write_failed marker on the execution rather than flipping it to failed, which is deliberate per #621 (flipping it would tempt a re-approval of an already-fired purchase). So there is no second silent-failure defect to fix here. The loss is the purchase_history row itself: the billed commitment is invisible in the History view, absent from GetActivePurchaseHistory (so analytics undercount committed spend), and unseen by the grace-period and suppression logic.

Already-lost rows

Not recoverable by this migration — they were never inserted. Affected executions are identifiable:

SELECT execution_id, error FROM purchase_executions
WHERE error LIKE '%history_write_failed%';

The marker carries the provider-side commitment ID. The double-sided wildcard is deliberate: recordHistoryAuditGap appends via appendErrNote, so the marker is not necessarily at the start of the field. No backfill is attempted here; tracked separately in LeanerCloud/cloud-commitments-platform#146.

Migration

Number 000095, checked against origin/main (tops out at 000093) and every open PR branch (000094 is claimed by #1523).

Up — probe-guarded so it is correct on a fresh database, an already-deployed one, and on re-run under auto-heal (IF NOT EXISTS cannot repair a wrong column type). Two deliberate divergences from 000067:

  • Resolves the table via 'purchase_history'::regclass, which follows search_path exactly as the bare ALTER TABLE purchase_history does. buildMigrateDSN appends no connection options (RDS Proxy), so search_path is the role default; a probe resolving differently from the ALTER could silently no-op while golang-migrate recorded the migration as applied — the p0 would persist under a green deploy. It also re-reads the catalog after the ALTER and raises if the column is still too narrow, and raises if the column is absent entirely.
  • Treats atttypmod = -1 (unbounded varchar) as already-wide. 000067's character_maximum_length IS NULL branch would have narrowed such a column.

Down — narrows only from VARCHAR(255) (the exact state the up produces; guarding on "anything wider than 20" would impose a state 000095 never created). It refuses with a named error rather than truncating when any account_id exceeds 20 characters, and documents the CUDLY_FORCE_MIGRATION_VERSION=95 recovery for the dirty state a refusal leaves behind (force to current, since the up's effects are present).

Dependent objects: none. No view, materialized view, FK or partition references purchase_history, so no drop/recreate dance (contrast 000067, which had to drop three views). Widening a varchar is binary-coercible, so Postgres skips the table rewrite and leaves idx_purchase_history_account_timestamp in place.

Go side: no change needed. scanPurchaseHistoryRow scans account_id into a plain string (NOT NULL in 000001, never relaxed since), and all eight purchase_history SELECT projections stay in lockstep. Verified rather than assumed.

Other identifier columns

Swept every VARCHAR column in the migrations holding an account/subscription/project/tenant identifier. Only three are VARCHAR(20):

Column Verdict
purchase_history.account_id fixed here
savings_snapshots.account_id already widened by 000067
ri_exchange_history.account_id CHECK (account_id ~ '^\d{12}$') — explicitly AWS-12-digit by construction, not a defect

Everything else is VARCHAR(255), or VARCHAR(36) for Azure GUID columns in cloud_accounts (exact fit).

Verification

A green suite is not evidence, so:

Pre-fix failure, verbatim (migrated to version 92, then the real INSERT column set):

000095_..._test.go:112: pre-fix rejection for 3f2504e0-4f89-11d3-9a0c-0305e82c3301:
  ERROR: value too long for type character varying(20) (SQLSTATE 22001)
000095_..._test.go:112: pre-fix rejection for cudly-production-analytics-001:
  ERROR: value too long for type character varying(20) (SQLSTATE 22001)

Both round-trip untruncated after 000095.

Falsification — with the up migration neutralized to SELECT 1;, the post-migration assertions fail:

--- FAIL: TestMigration095_PurchaseHistoryAccountIDWidth
    --- FAIL: .../post-migration/azure_subscription_GUID_(36_chars)
    --- FAIL: .../post-migration/gcp_project_ID_(30_chars)

So the test is not vacuous — it could not have stayed green with the bug present.

The test uses SavePurchaseHistory's own INSERT column set, not a narrower proxy. It pins to version 92 via MigrateToVersion rather than migrating to HEAD (so the next migration PR's CI is unaffected), and the two rollback tests use RollbackMigrations(1) so they reverse exactly 000095 even once 000094 lands.

Also covered: the lossless rollback path (narrowing actually happens with only AWS IDs present) and up-migration idempotency (re-executing the file is a no-op) — without these, the ALTER branch after the down's guard and the header's auto-heal claim would ship untested.

Gates

  • go test -tags integration ./internal/database/postgres/migrations/ — full package, exit 0
  • go test ./internal/database/postgres/... ./internal/config/... — 703 passed
  • golangci-lint at the CI-pinned v2.10.1, with and without --build-tags=integration — 0 issues, exit 0
  • go build ./..., go vet, gocyclo -over 10 — clean (no non-test Go code added)
  • Full pre-commit hooks, no --no-verify

purchase_history.account_id has been VARCHAR(20) since 000001, which fits
an AWS account ID (12 digits) and nothing else. An Azure subscription ID is
a 36-character GUID and a GCP project ID runs to 30 characters, so
SavePurchaseHistory's INSERT is rejected with SQLSTATE 22001 after the
commitment has already been purchased and billed.

The error is not swallowed: savePurchaseHistory returns it and the caller
stamps a history_write_failed audit-gap marker on the execution (#621).
But the purchase_history row itself is lost, so the billed commitment is
invisible in the History view, absent from GetActivePurchaseHistory (and
therefore undercounted in analytics), and unseen by the grace-period and
suppression logic.

The identical widening was already applied to the sibling column: 000067
widened savings_snapshots.account_id for exactly this reason and 000074
repaired it on partially-migrated databases. purchase_history was missed.

- up: probe-guarded ALTER so it is correct on a fresh database, an
  already-deployed one, and on re-run under auto-heal. Resolves the table
  via 'purchase_history'::regclass so the probe follows search_path exactly
  as the ALTER does, and re-reads the catalog afterwards so the migration
  cannot be recorded as applied without having widened the column.
- down: narrows only from VARCHAR(255), and refuses with a named error
  rather than truncating when any account_id exceeds 20 characters.
  Documents the CUDLY_FORCE_MIGRATION_VERSION=95 recovery for the dirty
  state a refusal leaves behind.

Rows already lost to 22001 were never inserted and cannot be recovered by
this migration; affected executions are identifiable via the
history_write_failed marker on purchase_executions.error.

Regression test replicates the real failing scenario with SavePurchaseHistory's
own INSERT column set and a 36-char Azure GUID / 30-char GCP project ID:
both are rejected with 22001 at version 92 and round-trip untruncated after
000095. Also covers the lossless rollback path and up-migration idempotency.

Closes #1603
@coderabbitai

coderabbitai Bot commented Jul 29, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 56 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 09e4cf7a-1096-45ae-b1c9-237f094b7f3f

📥 Commits

Reviewing files that changed from the base of the PR and between 6ded401 and 6eb2777.

📒 Files selected for processing (5)
  • internal/database/postgres/migrations/000095_purchase_history_account_id_width.down.sql
  • internal/database/postgres/migrations/000095_purchase_history_account_id_width.up.sql
  • internal/database/postgres/migrations/000095_purchase_history_account_id_width_test.go
  • internal/database/postgres/migrations/helpers_test.go
  • internal/database/postgres/migrations/migrate_autoheal_test.go

Comment @coderabbitai help to get the list of available commands.

cristim added 2 commits July 29, 2026 11:42
The shared insert helper hardcoded provider='azure' with Azure's service,
region and resource_type, so the GCP case inserted an Azure-shaped row and
only the account_id differed. In a test whose entire point is that Azure
and GCP identifiers overflow VARCHAR(20), a GCP case carrying
'westeurope'/'Standard_D4s_v3' misrepresents the scenario it claims to
cover.

Group the identifier with its provider/service/region/resource_type in a
commitmentRow fixture and pass those through as bind parameters, so each
case inserts a row its own provider would actually produce. Adds an AWS
fixture for the lossless-rollback test, which previously used a bare
12-digit literal.

Test-only; no migration or production code changes.
TestMigrations_AutoHealDirty rolls back one migration and asserted the
resulting version equals headVersion-1. That encodes an invariant this
repository does not hold: migration numbers are not contiguous, because
renumbering to dodge collisions with in-flight PRs leaves gaps. The set
already skips 61-62, 68-69, 82, 84-85.

Those gaps were harmless only because none of them sat immediately below
head, so headVersion-1 happened to be a real migration. Adding 000095 on
top of main's 000093 puts a gap directly below head for the first time,
and golang-migrate's Steps(-1) lands on 93 -- the next version that
actually exists -- so the assertion failed with expected 94, actual 93.

Derive the expected version from the .up.sql filenames instead. This is
the root fix rather than a renumber: the next migration to land above a
gap would have hit the same assertion, and renumbering 000095 to 000094
would have collided with the in-flight #1523.

Test-only; no migration or production code changes.
@cristim

cristim commented Aug 3, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Aug 3, 2026 •

Copy link
Copy Markdown
Contributor
✅ Action performed

Full review finished.


Your included review limit is currently reached under our Fair Usage Limits Policy. Your recent PR review activity is in the 95th percentile or higher among CodeRabbit users, so adaptive limits apply. This review may still proceed through usage-based billing if eligible. Your next included review will be available in 56 minutes.

@cristim
cristim merged commit 9de48ae into main Aug 3, 2026
19 checks passed
@cristim
cristim deleted the fix/1603-account-id-width branch August 3, 2026 11:29
@cristim

cristim commented Aug 3, 2026

Copy link
Copy Markdown
Member Author

Adversarial review record (merged)

Independent reviewer, distinct from the author. Recording it here because the merge rested on this, not on the CI checks — this PR's CodeRabbit status read success while its description said "Review rate limited", i.e. a green status over a review that never ran (true of 11 of 13 open PRs at the time).

Verified against a real Postgres testcontainer, not by reading the diff.

The regression test genuinely bites. Pre-migration schema rejected both realistic values live: SQLSTATE 22001 for a 36-char Azure subscription GUID and a 30-char GCP project ID, inserted through the same NOT NULL column set SavePurchaseHistory uses. Falsified too — replacing the up migration with SELECT 1; makes the post-migration assertions fail, so the test could not have stayed green with the bug present.

Migration numbering: 000095 clear of origin/main's highest (000093) and of every open PR (only #1523 carries 000094). Order-independent.

A latent CI trap was found and fixed: TestMigrations_AutoHealDirty asserted rollback lands on headVersion - 1, a contiguity assumption this repo violates (already skips 61-62, 68-69, 82, 84-85). The replacement derives the target from .up.sql filenames and was exercised against every gap in the repo, not just the one this PR creates.

Down migration fails loudly rather than truncating. Scanner/projection parity confirmed — account_id in all five SELECT projections and the shared scanner, NOT NULL since 000001, never altered. Catalog verified post-migration: 9 indexes valid, 6 constraints validated, 0 dependent views/matviews.

Two non-blocking notes: TestMigration095_DownNarrowsWhenSafe is vacuous as a regression guard (passes under falsification), and the down migration cannot distinguish "the up widened it" from "an operator hand-widened it beforehand" — worst case a schema regression, never truncation.

Sibling issues filed: #1676, #1679.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/s Hours impact/all-users Affects every user priority/p0 Drop everything; same-day fix severity/critical Major harm when it happens triaged Item has been triaged type/bug Defect urgency/now Drop other things

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(db): purchase_history.account_id VARCHAR(20) drops the audit row after every Azure/GCP purchase

1 participant