Repository navigation
fix(azure): pair the recommended SKU with the count in its own units - #1771
Conversation
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 5 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (4)
Comment |
Azure's consumption API carries two quantities on a reservation recommendation and they are not interchangeable: RecommendedQuantity counts the actual SKU (SKUProperties[SKUName]), RecommendedQuantityNormalized counts NormalizedSize. The resource type and the count were resolved independently, so the Legacy/EA ladder took its SKU from NormalizedSize while the count came from RecommendedQuantity. Azure recommending 4 x Standard_D8s_v3, normalized to 16 x Standard_D2s_v3, was converted to Standard_D2s_v3 x4 -- a quarter of the recommended capacity. Verified end to end: it reaches the wire as sku.name=Standard_D2s_v3 with quantity=4 on both calculatePrice and purchase, which Azure accepts because both values are individually valid. The recommendation's savings figure is Azure's number for the FULL quantity and rode along unchanged, so $400/mo was reported against a purchase delivering roughly a quarter of it. Under-buy, not over-buy: it fails safe on commitment risk -- nobody is locked into capacity they cannot use -- but unsafe on reported savings, because the number shown to whoever approves the purchase is wrong. A decision-quality defect rather than a spend defect. The two are now resolved together so no branch can emit a mixed pair. The actual SKU is preferred, matching the Modern ladder, so EA and MCA subscriptions with identical usage yield the same resource type; its partner RecommendedQuantity is also the *float64 field, avoiding a *float32 rounding question on a purchase path. When only NormalizedSize is available its normalized count is the only valid partner, and if that is absent the count is left unset rather than borrowed -- an unpaired count is the defect itself, and zero is refused by the Count <= 0 guard every service applies before purchasing. Modern was not the reported instance and its first rung is unchanged, but with SKUName absent its NormalizedSize rung mixed the same two fields. Paired here rather than left as the one branch that can still emit mismatched units. The fixture builders now set both quantities, because a fixture naming a NormalizedSize while setting only RecommendedQuantity describes a payload with no valid pairing. Refs #1540
c219fad to
cd80e33
Compare
|
Merging. Closes #1540. Merging on independent adversarial review — CodeRabbit has no verdict at this head. Legacy (EA) recommendations paired the normalized SKU with the un-normalized quantity. It reaches a purchase, not just a display. The direction finding is the part worth preserving, and it survived attack: under-buy — fails safe on commitment risk, unsafe on reported savings. Nobody buys a commitment they should not; the savings figure shown to whoever approves the purchase is wrong. That is a decision-quality defect. Calling it "wrong purchases" would overstate it and "display only" would understate it, and the third framing is the accurate one. The reviewer attacked that claim directly rather than accepting it — traced both resolvers by hand and stress-tested the float32/float64 conversion ( Scope was wider than the issue was filed with: seven consuming services, plus provider-agnostic CLI sizing and duplicate suppression in The Modern/MCA negative was re-examined, not written off. The SKUName rung — what real MCA payloads take — pairs with Mutations re-run independently rather than trusted from the PRs table:
All restored cleanly afterward, verified by diff against the PR commit. One verified, non-blocking finding, recorded for a fast-follow. The deliberate reordering to check The reviewer then asked the question that makes this reportable rather than alarming: does it weaken the direction claim? It does not. Under old code that same input hit the original #1540 bug — a silent under-buy where the purchase succeeds. Under new code it produces a SKU name Azure rejects outright: fails loud, no money moves. Safer-or-equal, with a different failure mode — availability/completeness, not money-safety. Suggested fix is to let the last-resort loop win only when Note on the CI history here: this PR spent two cycles failing on Gates at |
Closes #1540
The defect
Azure's consumption API carries two quantities on a reservation recommendation and they are not interchangeable:
RecommendedQuantitySKUProperties[SKUName])RecommendedQuantityNormalizedNormalizedSizeResourceTypeandCountwere resolved independently, so the Legacy/EA ladder took its SKU fromNormalizedSizewhile the count came fromRecommendedQuantity. Azure recommending 4 ×Standard_D8s_v3, normalized to 16 ×Standard_D2s_v3, becameStandard_D2s_v3× 4 — a quarter of the recommended capacity.RecommendedQuantityNormalizedappeared nowhere in the repo before this change. The field was never read, so no path produced the correct pair.It reaches a purchase, not a display
Carried through the compute converter into
buildReservationBody, these are the bytes sent tocalculatePriceandpurchase:{"sku":{"name":"Standard_D2s_v3"}, "properties":{"quantity":4,"term":"P1Y","reservedResourceType":"VirtualMachines", ...}}Azure has no reason to reject it:
Standard_D2s_v3is a real SKU and4is a valid quantity. The purchase succeeds and the customer is committed for the term. The only quantity guard anywhere on the path isrec.Count <= 0; nothing validated that the SKU and the count were in the same units.And the money figures ride along unchanged — same run:
EstimatedSavingsis Azure's number for the full recommended quantity — the converter says so itself (NetSavings is the savings from buying the full recommended quantity). So $400/mo was reported against a purchase delivering roughly a quarter of it.Direction
Under-buy — fails safe on commitment risk, unsafe on reported savings.
Nobody is locked into capacity they cannot use; that is the worse direction and it does not happen here. But the savings figure shown to whoever approves the purchase is wrong. A decision-quality defect rather than a spend defect — which is why "wrong purchases" would overstate it and "display only" would understate it.
Strictly, the direction is decided by Azure's data rather than by the code: under-buy iff
NormalizedSize≤ the actual SKU. Thearmconsumption@v1.1.0doc comments are terse ("The recommended Quantity Normalized") and never state units, so that half rests on Azure's documented normalize-to-base-size behaviour. In-repo corroboration: all 34 Legacy fixture call sites treatNormalizedSizeas the base size —Standard_D2s_v3,GeneralPurpose_Gen5_2,100RU,DW500c,Premium_P1.Scope — wider than the issue states
The issue cites one example consumer. It is seven services, all through the one shared
recommendations.Extract:compute,cache,cosmosdb,database,managedredis,search,synapse.Countalso feeds provider-agnostic logic incmd/helpers.go, which the issue does not mention:applySizing/ApplyCoveragescale bynewCount / rec.Count,ApplyTargetCoverageanchors on it,--max-instancesclamps on it, and the duplicate checker comparesexistingCount >= rec.Countand subtracts. None of that is gated on provider, so the wrong count reached sizing math and duplicate suppression as well as the purchase body.The fix
ResourceTypeandCountare now resolved together, so no branch can emit a mixed pair.ResourceType. Its partnerRecommendedQuantityis also the*float64field, avoiding a*float32rounding question on a purchase path.NormalizedSizeis available, its normalized count is the only valid partner.RecommendedQuantity— an unpaired count is precisely the defect above. Zero is refused by theCount <= 0guard every service applies before purchasing, and a warning is logged. (Note there is no empty-ResourceTypeguard anywhere inproviders/azure, so blanking the type instead would have put an emptysku.nameon the wire.)Behaviour change worth calling out explicitly
This changes what gets purchased for EA customers. The purchased SKU moves from the normalized size to the actual size for every Legacy recommendation carrying both fields — which is the common real-world payload shape. That is the point of the fix, but it is a real change and should not pass unnoticed.
Modern was extended too — a correction to my own earlier assessment
I reported during investigation that Modern/MCA was unaffected. That is true of the rung real MCA payloads take (
SKUNamepresent), and this PR leaves that rung byte-identical — pinned byTestExtract_Modern_ResourceTypePrefersSKUNameOverNormalizedSize.But while editing I found that with
SKUNameabsent, Modern'sNormalizedSizerung mixed the same two fields exactly as Legacy did. Fixing Legacy alone would have left one branch of the same function still able to emit mismatched units on a money path, so it is paired here rather than deferred. Flagging it as a deliberate, named extension rather than something slipped in.Tests — both directions, plus a downstream consumer
A test asserting only that the bad pairing is gone passes against a converter that produces nothing, so every case asserts the SKU and the quantity that must accompany it.
TestExtract_LegacySKUAndQuantityStayInMatchingUnits— three payload shapes; each valid pair must still be produced.TestExtract_LegacyNormalizedSizeWithoutNormalizedQuantityLeavesCountUnset— the one state with no valid pairing.TestExtract_Modern_NormalizedSizeRungIsCountedNormalizedand theSKUName-rung test above.TestPurchaseBody_SKUAndQuantityStayInMatchingUnits(services/compute) — the downstream half. The pairing is fixed in the shared converter, but the defect only mattered because it reached the wire; this parses the actual request body and assertssku.nameandquantitytogether. A converter-only test would have looked right while the bytes Azure receives stayed wrong, which is how the defect survived.Mutation-tested
Five mutations, each reverting one property of the fix. Every one is killed by a named test:
NormalizedSize+ un-normalized count)RecommendedQuantityRecommendedQuantityRecommendedQuantityM5 is the trap this design exists to avoid, and M1 shows the downstream test reporting the exact wrong wire values:
Fixtures
The Legacy and Modern builders now set both quantities, and
WithQuantity/WithModernQuantityset both: a fixture naming aNormalizedSizewhile setting onlyRecommendedQuantitydescribes a payload with no valid pairing.WithNormalizedQuantity/WithModernNormalizedQuantitymake them differ deliberately, andWithoutNormalizedQuantitymodels the unpairable payload. All 34 pre-existing Legacy fixture call sites are unchanged and still pass.Gates
gofmt -l .providers/azure:go build ./...+go vet ./...+go test -race -count=1 ./...gocyclo -over 10 -ignore "_test\.go"(root andproviders/azure)golangci-lintv2.10.1 onproviders/azure,commset diff vs baseThe set diff earned itself: the first pass came back 592 → 597, and the diff named all five as mine — one
behaviourmisspell, twomodellingmisspells, and twounnamedResultfrom the new two-value returns. All fixed; the re-run is identical to base on both the untouched files (540 = 540, exact line-preservingcomm) and the touched ones (by file+message, since my edits shift line numbers).Both golangci runs initially exited 3 with zero findings —
Error: parallel golangci-lint is running. That reads exactly like a clean run and was retried rather than trusted.mainis currently red onTestGrantCeiling_ConstraintContainment(internal/auth), from #1737 and #1758 colliding — confirmed failing at this branch's base commit, unrelated to this diff and untouched by it. This PR's CI will inherit it until that fix lands.