Repository navigation
fix(iac/azure): remove nonexistent Microsoft.Capacity action from reservation-purchaser role - #1800
Merged
Merged
Conversation
…ervation-purchaser role Microsoft.Capacity/reservationOrders/purchase/action does not exist in Azure's operation catalog. Validated all 12 actions in the custom reservation-purchaser role against the live catalog (az provider operation show, both namespace-level and resourceType-level operations): 11 valid, this one absent. Azure rejects the entire role definition with InvalidActionOrNotAction when it appears in the actions list, so the role was never created, which is why the Azure deploy has failed on every main run since 2026-07-19 (the bootstrap stack's data.azurerm_role_definition lookup finds nothing). It is also why the customer-facing ARM template in arm/CUDly-CrossSubscription fails for anyone deploying it to onboard a subscription. The role's actual purpose is legitimate: the built-in Reservation Purchaser role genuinely lacks Microsoft.Capacity/reservationOrders/write, Microsoft.Capacity/calculatePrice/action, and every Microsoft.BillingBenefits action, which is what causes 403s on production purchases. Only the one action name was wrong; the role's description and every comment citing it are corrected to name the real gap instead. Purchasing a reservation is Microsoft.Capacity/reservationOrders/write, already present in the list. Removed the action from every occurrence: the Terraform module, the customer-facing ARM template (both the actions list and two descriptions), the two comments in consuming modules, and all 34 scripts/testdata/role-parity/*-arm.json fixtures plus the TF baseline fixture that also carried it. The fixtures are replaced with a real, previously-absent action (Microsoft.BillingBenefits/register/action) rather than deleted outright, since the role-parity checker compares TF and ARM actions as a set (sort -u then diff) and most fixtures use a single shared TF baseline: a bare deletion would desync the actions count across 33 of the 35 fixtures and mask the scope/casing/collision axis each one actually exists to test. Full fixture suite verified green after the change (36/36). The parity checker itself never caught this because both the ARM and TF sides carried the same invalid action: parity is not validity, and neither side of the checker validates actions against Azure's actual catalog. That is a separate, larger effort (needs network access in CI) and is not attempted here. Also corrects the "Explain a missing bootstrap role" CI diagnostic in deploy-azure.yml, which assumed the bootstrap stack had simply never been applied. It had been applied and failed, so the role was rejected rather than skipped; the message now points at InvalidActionOrNotAction in the bootstrap stack's own apply log as the first thing to check. Verified by a real terraform apply against a live subscription: the role definition now creates successfully.
cristim
added
triaged
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (40)
📝 WalkthroughWalkthroughThe Azure reservation role now uses ChangesAzure reservation role correction
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related issues
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
This was referenced Aug 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Microsoft.Capacity/reservationOrders/purchase/actiondoes not exist in Azure's operation catalog. Validated all 12 actions the custom reservation-purchaser role granted, against the live catalog (az provider operation show, both namespace-leveloperations[]andresourceTypes[].operations[], forMicrosoft.CapacityandMicrosoft.BillingBenefits): 11 valid, this one absent.Azure rejects the entire role definition with
InvalidActionOrNotActionwhen an unknown action is present, so the role was never created. This is the root cause of two distinct failures:mainhas failed on every run since 2026-07-19. The runtime module'sdata.azurerm_role_definitionlookup finds nothing, because the bootstrap stack'sazurerm_role_definitionwas rejected outright.arm/CUDly-CrossSubscription/template.jsoncarries the same invalid action; anyone deploying it to onboard a subscription hits the sameInvalidActionOrNotActionrejection.Root cause vs. the role's actual purpose
The role's purpose is legitimate: the built-in "Reservation Purchaser" role genuinely lacks
Microsoft.Capacity/reservationOrders/write,Microsoft.Capacity/calculatePrice/action, and everyMicrosoft.BillingBenefitsaction, which is what causes 403s on production purchases. Only the one action name was wrong. Purchasing a reservation isPUT /providers/Microsoft.Capacity/reservationOrders/{id}, i.e.Microsoft.Capacity/reservationOrders/write, already present in the list. Fixed:terraform/modules/iam/azure/cudly-reservation-role/main.tf,arm/CUDly-CrossSubscription/template.json,iac/federation/azure-target/terraform/main.tf,terraform/modules/compute/azure/container-apps/main.tf.actionslist and from the ARM template'sactionslist and its role-assignment description.Provider registration was tested and ruled out, not merely considered. Both
Microsoft.CapacityandMicrosoft.BillingBenefitswereNotRegisteredon the subscription, which looked like a plausible cause since Azure validates role actions against provider catalogs. Both were registered, confirmedRegistered, and the identical apply was re-run: it failed with the identicalInvalidActionOrNotActionerror on the same action. Removing the action is what fixed it.The two registrations are additive and reversible and have been left in place, but they were an incidental change to the subscription rather than part of the fix, and are recorded here so they are not mistaken for one.
Verified in production, not just locally
Applied this fix against a live subscription and re-ran the previously-failing deploy:
data.azurerm_role_definition.cudly_reservation_purchaser: Read complete after 0sazurerm_role_assignment.reservations_purchaser: Creation complete after 26sThe three-week-old
InvalidActionOrNotAction/ role-not-found failure is gone.The deploy is still red after this fix, for an unrelated, pre-existing reason — noted here so a red run isn't mistaken for this PR failing. Two other role assignments (
cost_management_reader,subscription_reader) now fail with409 RoleAssignmentExists: they already exist in Azure but are absent from Terraform state, because neither sets an explicitname, soazurermmints a new UUID per apply while Azure dedupes on (scope, role, principal) — a missing state entry is a permanent 409 with no state to reconcile against. This is state drift, unrelated to the invalid-action bug, and is being tracked and fixed separately. Not addressed in this PR.Customer-facing blast radius
arm/CUDly-CrossSubscription/template.jsonis the template customers deploy directly (az deployment sub create) to grant CUDly cross-subscription access. It carried the identical invalid action, so it was almost certainly failing for customers too, not just in our own CI.Test fixtures: parity is not validity
scripts/testdata/role-parity/feedsscripts/check-azure-role-parity.sh, which asserts the ARM template and TF module agree on actions/notActions/dataActions/notDataActions (compared as a set: both sides are extracted then piped throughsort -u, thendiff'd) and that no grant escapes subscription scope. 34 of the 35 fixture files carried the same invalid action as a copy-pasted baseline (matching-tf.tf.fixtureplus every*-arm.jsonexceptdrifted-arm.json, which is the fixture that intentionally omits it to test the actions-mismatch axis).The parity checker never caught this because both sides carried the same invalid action. The fixtures proved ARM and TF agreed while both were invalid — nothing in the suite could have caught a shared mistake, because parity is not validity: the checker only proves the two sides agree, never that either names a real Azure action. Fixed the fixtures by substituting a real, previously-unused action (
Microsoft.BillingBenefits/register/action, already present in production) for the invalid one, uniformly across the TF baseline and every ARM fixture that carried it. A bare deletion was considered and rejected: since the checker's actions comparison is set-based (order doesn't matter, confirmed by readingextract_tf_list/extract_arm_list, bothsort -u, andcompare_action_lists'sdiff), deleting without replacing would desync the action count between the single shared TF baseline and 33 of the 35 ARM fixtures, which would make the actions check fail first on scope/casing/collision fixtures whose whole purpose is exercising a different axis — passing those tests for the wrong reason instead of the one they're built to catch, while breaking the 5 fixtures asserting a cleanexit 0. Verified:bash scripts/test-azure-role-parity.sh→ 36 passed, 0 failed.bash scripts/check-azure-role-parity.sh(against the real production files) → both checks pass.Not attempted here, tracked as a follow-up: validating actions against Azure's live operation catalog in CI. That needs network access from the CI runner and is a separate design decision from this fix.
CI diagnostic correction
.github/workflows/deploy-azure.yml's "Explain a missing bootstrap role" step assumed the bootstrap stack had simply never been applied. It had been applied and rejected, not skipped. Corrected the message: the first thing to check on any recurrence is nowInvalidActionOrNotActionin the bootstrap stack's own apply log, pointing at this fix.Verification
git grep -n -F "reservationOrders/purchase/action"→ zero hits.python3 -m json.tool arm/CUDly-CrossSubscription/template.json→ parses clean; same for all touched fixture JSON files.terraform fmt -check -recursive→ clean for all three touched Terraform directories.terraform validate→ passes for all three touched Terraform directories (terraform/modules/iam/azure/cudly-reservation-role,iac/federation/azure-target/terraform,terraform/modules/compute/azure/container-apps).bash scripts/test-azure-role-parity.sh→ 36 passed, 0 failed.bash scripts/check-azure-role-parity.sh(production files, no overrides) → actions/notActions/dataActions/notDataActions match; all scopes subscription-anchored.terraform applyagainst a live subscription confirmed the role definition now creates successfully (see "Verified in production" above).Closes #1794
Summary by CodeRabbit
Bug Fixes
Documentation