Skip to content

test(scheduler): verify incomplete savings plans persistence - #490

Merged
cristim merged 1 commit into
mainfrom
test/sp-recommendation-completeness
Oct 5, 2026
Merged

cristim merged 1 commit into
mainfrom
test/sp-recommendation-completeness

Conversation

@cristim

@cristim cristim commented Oct 5, 2026

Copy link
Copy Markdown
Member

Savings Plans collections can return usable rows with rejected details or failed requests. Platform now pins the published AWS completeness fix and verifies that those partial collections preserve their surviving offers without authorizing stale-row eviction.

The existing PostgreSQL regression covers 24 cases: 11 RI controls and 13 SP cases, including malformed details, failed plan types, late-page survivors, configured fallbacks, exact persisted financial values and account-scoped isolation. Only the integration test and AWS module pin/checksums change.

Refs LeanerCloud/cloud-commitments-go#170. This completes its Platform consumer task; the producer, MCP and CLI changes are already merged.

Independent reviewer: gpt-6-astra, under the user's authorized local-verification alternative. Reviewed head: c47100d405b7e24aa2cc0034c125029f04e146da; base: 4fe935e3dee6c0a7464cda279fbc308cbca728d5. Full committed diff review found no actionable findings. Both staged review passes were clean.

Verification:

  • Old published SDK: 9 SP cases fail behaviorally; all 11 RI and 4 valid/empty SP controls pass.
  • Fixed published SDK: all 24 connected cases pass, including a fresh committed-head run with race detection and -count=1.
  • Removing only the diagnostic warning makes the mixed case fail while valid/empty controls pass.
  • Full repository race/short suite, build, vet, pinned integration-tagged lint and schema validation pass. Normal installed commit hooks pass.

Tests use synthetic AWS responses through the real SDK, scheduler and store into native PostgreSQL 17.11. A local overlay replaces only the test database-creation helper; production code, migrations and assertions remain unchanged. Fixture databases are retained. No live AWS or deployed-feature verification is claimed. The broader suite uses synthetic credentials and configured rejection proxies; its entire network boundary was not independently audited.

Added test comment ratio: 0%. Exact source hashes, commands, environments, parent-red proof, mutation proof and committed-head logs are recorded in the attached independent review.

Pin the published AWS completeness fix and cover malformed details,
failed plan types, late-page survivors and conservative fallback eviction.
Assert persisted financial values and account-scoped row identities.

Refs LeanerCloud/cloud-commitments-go#170
@cristim cristim added urgency/this-sprint Within the current sprint triaged Item has been triaged priority/p1 Next up; this sprint severity/high Significant harm impact/many Affects most users effort/m Days type/bug Defect labels Oct 5, 2026
@coderabbitai

coderabbitai Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

  • Run on-demand review

This review includes 2 billable files and costs up to $0.50.

  • Ask an admin to make reviews automatic

Open in CodeRabbit

Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing.

Or wait 43 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available. Your 72 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Repository: LeanerCloud/cloud-commitments-platform/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Essentials
  • Run ID: 396f6dc7-dbec-42b5-b20c-0826325c0223
📥 Commits

Reviewing files that changed from the base of the PR and between 115ff70 and c47100d.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (2)
  • go.mod
  • internal/scheduler/recommendation_completeness_integration_test.go
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@cristim

cristim commented Oct 5, 2026

Copy link
Copy Markdown
Member Author

Task C review and baseline

Pass 1: one finding, persisted identity assertion checked broad allowed values instead of exact survivor tuples. Author revised.
Pass 2: no actionable findings at test SHA256 6f93c9b8237ebe20b3f359d8b53a5212fe650ff103ea6c86df5fcd276efdaebc. Re-read full revised test, published9962786 SP parser/collection, scheduler recommendationID/tagAccount, and PostgresStore account filters/upsert eviction. Exact expected persisted key set includes account UUID/service/term/payment, configured fallback1yr/no-upfront, failed Database type exclusion, and complete sibling independently. Ambient disabled-account seed guards NULL-scope eviction. All six review dimensions and approved Task C checked. Shipping pin remains006ef for baseline.

Three clean runtime-script reviews before execution:

  1. Correctness/completeness: bind exact base HEAD, test hash, sole expected dirty path, old published pin, fixed Go1.26.6; full integration selector and separate four SP control selector; race/count1/integration-tag all explicit. No actionable findings.
  2. Security/data ownership: native helper reused from independently reviewed472 fixture, prefix changed only to go170_UUID. Verifies exact datadir/address/port/server17.11 before CREATE DATABASE; quoted random fresh database; cleanup closes connection, resets/truncation forbidden. Synthetic credentials, empty HOME, rejected external proxy, offline readonly modules, GOWORKoff. No actionable findings.
  3. Concurrency/evidence: root assigned sole Go capacity; canonical shared OS lock fails fast if occupied;1200second command timeout; exclusive new logs/manifests/results; input/output hashes; only helper overlay, no product mutation. No actionable findings.

Matrix is24subcases (11RI+13SP),25test events including parent. Evidence is synthetic HTTP through actual SDK/scheduler/store into local PostgreSQL17.11, not live AWS or DockerPG16.

Baseline executed: modules-parent exit 0 selects published AWS006ef. Full parent run session69886 exited1: all24subcases ran,15passed (all11RI plus4SP controls),9SP incomplete cases failed at successful-account/eviction-roster assertions. These are intended behavior failures, not compile/setup errors. They detect unauthorized eviction eligibility before persistence; no destructive baseline DELETE is claimed. Parent-controls session49901 exited0 with separately selected SP_valid, SP_empty, SP_clean_fallback, SP_empty_fallback all passing. Logs, exact argv/env/input hashes, result status and retained UUID database names are adjacent. Source hash unchanged; only planned test dirty. Fixed-pin proof remains pending root's repin.

Fixed runner three clean reviews: (1) binds same reviewed test hash and exact three planned dirty files, requires published9962786, named fixed and broad check modes; (2) mutant removes only the AWS diagnostic Warnf call in an external scheduler overlay, preserves completeness/survivors/eviction, records mutant hash and runs mixed plus valid/empty controls; (3) all immutable-output, endpoint ownership, retained DB, lock, offline/synthetic env and timeout guards retained. Pinned golangci2.10.1 matches Makefile and installed version; lint explicitly includes integration-tagged changed test. No actionable findings.

Fixed draft verification complete

Independently reviewed all three final draft files against base 4fe935e. Module diff changes only AWS006ef to published9962786 and its two sums; pkg and Azure/GCP pins unchanged, no replace. No actionable findings. The only dirty files are the approved test/go.mod/go.sum; diff-check passes.

Fresh fixed modules provenance exit0. Fixed connected suite session75969 exit0: all24subcases PASS (25events including parent), package19.087s. Assertions exercise exact persisted key sets, costs, request tuples, count-bearing diagnostic output, account-filter isolation, partial old-row retention, clean-sibling eviction, and ambient/disabled-account boundaries.

Diagnostic-only mutant session23838 exit1 as required: SP_mixed reaches the diagnostic assertion after successful exact persistence checks, then fails because output lacks failed_details=6 failed_scopes=0. SP_valid and SP_empty controls PASS. Direct source diff confirms only the two-line scheduler AWS warning call is removed; no completeness or row behavior changed.

Full repository race/short suite session96610 exit0, build53991 exit0, vet92916 exit0, pinned integration-tagged scheduler lint91765 exit0 with 0 issues. All used fixed Go1.26.6/offline published modules/synthetic credentials under the canonical lock. The focused SDK fixture uses an injected no-dial HTTP client and verified local database endpoint. For the broad suite, synthetic credentials and rejection proxies are configured protections, not proof that every client honors proxies; no universal network-call assertion was installed. No product source edits by reviewer. All sessions terminal and fixture databases retained. Commands, environments, inputs, source hashes and terminal states reside in adjacent manifests/results/logs.

Final draft SHA256:

  • go.mod: 7d8b588f6d3afbaf63eb4cda3cebc3fee1e20d083d6ca58a83aa09d316a7c232
  • go.sum: 8bf2b56bf150e7a0588c7975942e0ad86c2f6b016b7140b0e9de0bbefea146f5
  • integration test: 6f93c9b8237ebe20b3f359d8b53a5212fe650ff103ea6c86df5fcd276efdaebc

Draft verification approves these exact inputs for staging/normal hooks. Committed-SHA review and connected runtime remain pending the author's commit, as do remote CI/publication/merge gates. This evidence substitutes only native PostgreSQL helper creation for Docker; it does not assert live AWS, browser, deployed, or PostgreSQL16-container parity.

Staged review gate

Both passes independently read the entire git diff --cached for all three files against HEAD 4fe935e. Pass 1: no actionable code findings. Pass 2: no actionable findings. Each pass covered completeness, correctness, security, bugs, duplication/reuse, over-engineering, scope and comments against the approved Task C and actual published SDK/scheduler/store contracts. The staged input hashes equal the verified draft hashes above; no unstaged changes were present. Staged diff SHA256: def9fac9f09635bc1b7cd34f18c53cbe6f515fe684593565221f880b2dc3f53c.

Read the five requested feedback memories: collection-result verification, SDK test network confinement, integration-tagged lint, cross-repository issue references, and installed push hooks. Collection outcomes and exact persisted rows are independently asserted. No issue references or hook changes occur in the diff. Test additions contain 0 comment lines out of 220 added lines (0%). Overall staged diff is 223 insertions and 14 deletions in three files. No unrelated production changes, Azure/GCP drift, new abstraction, or duplicated fixture was introduced.

Evidence wording above now distinguishes the focused no-dial fixture from broad-suite proxy configuration. Pinned integration-tagged lint passed; separate golangci-lint config verify remains assigned to the author for the hook/commit phase. These two clean staged reviews approve the exact staged content for normal hooks and commit, without carrying approval forward to an unreviewed committed SHA. Fresh committed-SHA review and connected runtime remain required.

Committed runner review before execution: three clean passes. Pass 1 verifies exact c47100d, clean tree, hashes for all three files, published pin, uncached race/integration selector. Pass 2 verifies the reused helper checks exact local PostgreSQL endpoint before additive UUID database creation and disables destructive cleanup; overlay is helper-only and synthetic no-dial SDK fixture is unchanged. Pass 3 verifies canonical lock, bounded timeout, offline modules, new exclusive committed-prefixed manifests/logs/results, and before/after source state recording. Only modules and fixed suite modes are allowed. Full committed three-file diff reread: no actionable findings; matches staged content.

Exact committed verdict

Approved c47100d, tree 112d4a2528ccc3c4a3d7a1267800470c6b2fb5e5, parent 4fe935e. All three committed file hashes match the reviewed and verified draft. Fresh full committed diff review found no actionable findings across all six dimensions, reuse and scope, including monetary values and account isolation. Approval is for this exact SHA and local verification gate; the author owns publication, CI and merge.

Fresh python3 committed-run.py modules-fixed exited 0. committed-modules-fixed.log confirms published AWS9962786 and pkg b3b4 without Replace, and the expected module sums. Fresh python3 committed-run.py fixed ran through approved local loopback execution, session 28050, terminal exit 0. Actual command is recorded in committed-fixed-manifest.json: Go1.26.6 test, readonly modules, p=1, race, count=1, integration tag, native-helper-only overlay, exact TestAWSRecommendationCompletenessPersistence selector. All 24 named subcases PASS (11 RI and 13 SP); parent PASS; package 18.322s. Reviewed actual terminal log and result file, not only runner status.

The connected run proves exact persisted key sets and monetary fields, survivor retention, complete empty eviction, successful account roster, registered/ambient/disabled account boundaries, exact outbound request tuples, and count-bearing incomplete diagnostics against real migrations and local PostgreSQL17.11. Prior baseline and diagnostic-only mutant evidence uses identical test bytes and remains applicable. Broad race/short suite, build, vet and integration-tagged lint were not repeated because committed product bytes match their passing draft inputs. The author reports successful normal hooks and pinned linter config verification; those are attributed author observations rather than an independently rerun claim here.

Final fresh git queries confirm exact HEAD/tree/parent above and empty porcelain status. Result manifest likewise records unchanged HEAD/test hash and clean status. All 24 newly created UUID databases are retained. No product edits, public writes, live-cloud credentials, or existing database deletion occurred in this final gate. This is synthetic SDK HTTP through production scheduler/store into native local PostgreSQL, not live AWS or Docker PostgreSQL16 proof. Sole Go runtime capacity is released to the author.

@cristim

cristim commented Oct 5, 2026

Copy link
Copy Markdown
Member Author

PR490 independent review

Verdict: no actionable findings at c47100d against base 4fe935e. Reviewed by independent gpt-6-astra under the user's explicit reviewer and synthetic-verification substitution. No product edits or public writes.

Merge boundary update: coordinator reports main advanced to 115ff70 while this review ran. That object was not present locally at the last check. This verdict covers the exact head and original base only, not the combined candidate on new main. The new delta reportedly touches scheduler read-time filters and recommendation overrides. Inspect those exact changes and verify the merged candidate's connected scenario before treating this as merge approval; do not infer combined behavior from the 24/24 exact-head result.

Follow-up source inspection after coordinator fetch: 115ff70 is now local. git diff 115ff707...c47100d4 --stat still reports the same three-file PR diff. Compared 4fe935e to 115ff70: scheduler.go changes only GetRecommendationByID, with nil-account global-config resolution; scheduler_overrides.go and config/recommendation_overrides.go apply global policy to ambient recommendations. Collection, fallback, conversion, persistence SQL, migrations and module manifests are unchanged. Other changed production paths are purchase state transition, UI confirmation/history and cleanup scripts, not this collection path. No rebase is required. One additive merged-candidate run of the completeness suite and TestGlobalFiltersPersistedAmbientAndRegistered is requested to verify the changed read-time consumer with the updated SDK; full-suite repetition is unnecessary for this boundary.

The complete PR diff is three files, 223 insertions and 14 deletions: published AWS provider pin and sums, plus scheduler integration tests. Prerequisite scheduler behavior is already in the base.

Dimension Result
Completeness 13 SP cases added alongside 11 existing cases. Parse errors, failed type, late page, fallback completeness, clean empty results, registered siblings, disabled and ambient scopes covered.
Correctness Assertions agree with published SDK parsing, request expansion, typed error aggregation and Platform account-scoped persistence.
Security No auth or purchase changes. Synthetic credentials and injected no-dial AWS transport; uncollected accounts and other providers retain rows.
Bugs Race-enabled connected runs pass; pagination survivors and sticky incomplete fallback are exercised. No actionable concurrency or lifecycle defect found in product diff.
Duplication / reuse Existing completeness fixture and production collection/persistence functions extended; no second product implementation.
Over-engineering / scope Two focused assertion helpers and table cases remain in the existing file. No unrelated production refactor or dependency replacement.

Source contract citations (paths below are relative to the Platform worktree, except SDK):

  • SDK recommendations/parser_sp.go:31 counts rejected details and :42 returns survivors with typed incompleteness. :156, :160, :164 parse commitment/savings/upfront; :188 gives 4 x 730 = 2920 on-demand cost; :220 and :226 give zero all-upfront or 2 x 730 = 1460 recurring cost. :238 and :240 preserve savings 10 and upfront 3. Test internal/scheduler/recommendation_completeness_integration_test.go:378 onward checks exact values and nil distinctions.
  • SDK recommendations/collection_sp.go:35 through :53 merges type outcomes; :62 through :99 keeps earlier-page rows when a later page fails. :37 through :40 specifies linked account scope and request fields. Test :443 through :456 checks the exact request multiset, including next token and configured fallback lookback.
  • SDK recommendations/collection_error.go:25 accumulates nested detail/scope counts; recommendations/client.go:428 through :455 propagates typed partials while retaining rows. Test :305 checks diagnostic counts.
  • Platform internal/scheduler/scheduler.go:988 recognizes incomplete results and reports them; :1018 and :1035 handle discovery and fallback; :1041 keeps completeness false after either partial result. Test :195 and :196 covers both fallback directions.
  • Platform internal/scheduler/scheduler.go:594 through :600 excludes incomplete accounts from the successful roster; :1051 through :1068 tags account and exact identity. Test :279, :293, :312 through :324, and :362 onward checks rosters, persistence, account filters and exact identities.
  • Platform internal/config/store_postgres_recommendations.go:96 through :107 deletes stale rows only for successful provider/account pairs. Seed/assertions at test :244 through :261 and :293 onward check stale, clean sibling, ambient, disabled-account and other-provider retention/eviction.

SDK source root: /Users/cristi/go/pkg/mod/github.com/!leaner!cloud/cloud-commitments-go/providers/aws@v0.0.0-20261003204812-9962786e0695.

Independent reproduction, all with Go 1.26.6, race enabled, native PostgreSQL 17.11, real migrations and production scheduler/store, synthetic AWS HTTP responses:

Run Case RUN/PASS/FAIL Including parent test Exit
old SDK 006ef5c8d0a2 plus final tests 24 / 15 / 9 25 / 15 / 10 1
shipping SDK 9962786e0695 plus final tests 24 / 24 / 0 25 / 25 / 0 0
diagnostic-loss mutation 3 / 2 / 1 4 / 2 / 2 1

The nine baseline failures occur at actual successful-account roster assertions (:283 or :279), not build or fixture failures. All eleven preexisting cases and SP valid/empty/clean-fallback/empty-fallback controls pass. Mutating only the warning at scheduler.go:991 leaves persistence working but causes SP_mixed to fail at test:305 because failed_details=6 failed_scopes=0 is absent; SP_valid and SP_empty pass. No test skips or race reports were observed.

Commands executed: python3 <this-directory>/verify.py parent, then fixed, then mutant. Each mode's manifest.json contains the exact expanded Go command, safe environment, cwd, HEAD and complete input SHA256 values; raw.log and result.json retain raw results. Parent uses an external baseline.mod/baseline.sum copied as raw Git bytes from the base, without changing shipping manifests. Fixed uses shipping manifests without replace directives. The sole local fixed overlay substitutes the container setup helper with a verified native local PostgreSQL helper; mutation additionally overlays scheduler.go. New UUID databases are retained. The canonical persistent test-suite flock serialized each run.

Input SHA256:

  • Final test: 6f93c9b8237ebe20b3f359d8b53a5212fe650ff103ea6c86df5fcd276efdaebc
  • Shipping go.mod: 7d8b588f6d3afbaf63eb4cda3cebc3fee1e20d083d6ca58a83aa09d316a7c232
  • Shipping go.sum: 8bf2b56bf150e7a0588c7975942e0ad86c2f6b016b7140b0e9de0bbefea146f5
  • Runner: da8a79e25216cb9d32e55494a12d083805fa695dc5baaabbac4a41f8fbd4332d
  • Native fixture helper: cde9f99169440de5d8c9f01f616cb667cd722967a55b9c98f3ee7e2d8cdafc1a

Runner review: three passes checked input/revision binding, scope and safe writes; synthetic network and database confinement; exclusive lock, timeout, process-session cleanup and log/result integrity. Python syntax compiled. Existing source graph was consulted only for navigation; current source controlled the verdict.

Limits: synthetic AWS payloads, not live cloud verification; this is connected scheduler/PostgreSQL integration, not browser UI or purchase testing. Full suite/build/vet/lint/hooks were not independently rerun because the brief records those checks on these same bytes and no new concern justified them. CI and public review posting remain the coordinator's responsibility. A fixed-run approval-review timeout occurred before process creation; the permitted single retry succeeded. All three execution sessions are terminal and their subprocess leaders were reaped; global process listing was sandbox-blocked. Final manifests verify unchanged clean product tree and the exact reviewed HEAD.

@cristim

cristim commented Oct 5, 2026

Copy link
Copy Markdown
Member Author

PR490 moved-base candidate verification

Bound candidate: staged Git tree d314013ef523f7d0a96248271956d15fe311d188, from HEAD 115ff7077cab53022bdf93b5c510876fd37634c6 and MERGE_HEAD c47100d405b7e24aa2cc0034c125029f04e146da, in /Users/cristi/.claude/worktrees/pr490-merge-candidate-20261005. This is an uncommitted merge candidate, not a new reviewed commit SHA.

Source review: the PR three-dot diff remains go.mod, go.sum and recommendation_completeness_integration_test.go only (223 insertions, 14 deletions). The 4fe935e to 115ff70 base delta changes relevant behavior only on recommendation reads: scheduler.GetRecommendationByID, filterRecsByResolvedConfigs, and config.ResolveAccountConfigsForRecs now apply global service policy to ambient rows. Collection, fallback completeness, conversion, account identity and database eviction are unchanged. No rebase or product correction is required. The exact-head review and baseline/mutation evidence remain in the sibling go170-pr490-cold-20261005 directory.

Verification scope: TestAWSRecommendationCompletenessPersistence (24 subcases) plus TestGlobalFiltersPersistedAmbientAndRegistered (11 subcases). The latter checks persisted ambient and registered rows for enabled/disabled, minimum count, include/exclude engine/region/type, exclusion precedence, and registered-override precedence; visible rows also pass detail-view hidden-reason assertions. This targets the actual changed consumer boundary without repeating unrelated full-suite work.

Runner: python3 /Users/cristi/.claude/projects/cudly-resume/evidence/go170-pr490-candidate-20261005/verify.py fixed. Three clean final review passes covered revision/index/staging and hash binding; exact native PostgreSQL helper plus synthetic AWS confinement; canonical flock, owned process cleanup, immutable evidence and exact expected run counts. Syntax compiled before execution. Run uses Go1.26.6, -race -count=1 -tags=integration -mod=readonly -p=1, published AWS SDK9962786, production scheduler/store and native PostgreSQL17.11. Only testhelper setup is overlaid. All fixture databases are fresh UUID names and retained.

Result: PASS, exit 0. Exact counts are 37 RUN / 37 PASS / 0 FAIL, comprising 24 completeness subcases, 11 global-filter subcases and their two parent tests. No skips or race reports. Runtime package result: ok github.com/LeanerCloud/cloud-commitments-platform/internal/scheduler 18.713s. The runner rechecked HEAD, MERGE_HEAD, tree, exact staged paths and all recorded input hashes after execution; no unstaged changes. Session82127 is terminal and the subprocess was reaped.

Verdict: no actionable findings for staged candidate tree d314013ef523f7d0a96248271956d15fe311d188 combining the exact heads named above. This closes the moved-base behavioral verification gap. Approval is tree-bound, not a claim that an uncreated merge commit was reviewed. fixed/manifest.json contains the exact command, environment and SHA256 hashes; fixed/raw.log and fixed/result.json retain the independent execution evidence.

Limits: synthetic AWS HTTP payloads with real SDK and PostgreSQL, not live cloud or UI/purchase verification. Coordinator must compare the eventual GitHub merge tree to this bound tree before extending this candidate proof to that merge. No new commit, source mutation, public write, or merge is performed by this reviewer.

@cristim
cristim merged commit ed44785 into main Oct 5, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/m Days impact/many Affects most users priority/p1 Next up; this sprint severity/high Significant harm triaged Item has been triaged type/bug Defect urgency/this-sprint Within the current sprint

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant