Repository navigation
fix(iac/azure): stop replacing the reservation-purchaser role assignment on every apply - #1804
Merged
Merged
Conversation
…ent on every apply module.compute_container_apps carries a module-level depends_on, and Terraform propagates a module-level depends_on to every data source inside the module. Both data.azurerm_subscription.current and data.azurerm_role_definition.cudly_reservation_purchaser were therefore read during apply, leaving scope and role_definition_id "(known after apply)" on every plan. Both attributes are ForceNew on azurerm_role_assignment, so all three subscription-scoped assignments were destroyed and recreated on every deploy: cost_management_reader, subscription_reader, and the money-path reservations_purchaser, whose grant is what permits calculatePrice and reservationOrders/write. Resolve both inputs in the caller, where nothing defers them. The role-definition lookup moves to terraform/environments/azure/compute.tf and reaches the module as reservation_role_definition_id; the RBAC scopes are built from var.subscription_id, which the deploy workflow already supplies via TF_VAR_subscription_id and which also configures the azurerm provider. The caller lowercases it once so the scope matches the casing Azure returns in resource IDs. The bootstrap-vs-runtime split is unchanged: the role definition is still owned by the human-applied ci-cd-permissions stack, and the runtime side still only looks it up and creates the assignment. The lookup now fails at plan rather than at apply when the bootstrap has not been applied; the workflow's "Explain a missing bootstrap role" step already reads both logs. Refs #1802
Contributor
📝 WalkthroughWalkthroughThe Azure environment now resolves the custom reservation-purchaser role at plan time and passes its ID and the normalized subscription ID to the Container Apps module. The module uses these values for subscription-scoped RBAC assignments and environment configuration. ChangesAzure RBAC plan-time resolution
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant AzureEnvironment
participant AzureRoleDefinition
participant ContainerAppsModule
participant AzureRBAC
AzureEnvironment->>AzureRoleDefinition: Look up custom role by name and subscription scope
AzureRoleDefinition-->>AzureEnvironment: Return role definition ID
AzureEnvironment->>ContainerAppsModule: Pass subscription ID and role definition ID
ContainerAppsModule->>AzureRBAC: Create subscription-scoped role assignments
Possibly related issues
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@terraform/environments/azure/compute.tf`:
- Line 12: Add validation blocks to the subscription ID variable declarations,
including the bootstrap variable, requiring an Azure subscription GUID with four
hexadecimal groups followed by eight hexadecimal characters in the standard
hyphenated format, case-insensitively. Keep lower(var.subscription_id) for
canonical ARM identifiers and reuse the same validation pattern for both
variables.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 8b1cca03-2730-47d6-bad1-66c84613b2ef
📒 Files selected for processing (4)
.github/workflows/deploy-azure.ymlterraform/environments/azure/compute.tfterraform/modules/compute/azure/container-apps/main.tfterraform/modules/compute/azure/container-apps/variables.tf
subscription_id now builds the reconstructed role-definition lookup name
("CUDly Reservation Purchaser (custom) - <guid>") as well as every
subscription-scoped RBAC scope. A value that is not a GUID at all, such as a
subscription display name, makes that lookup search for a role that cannot
exist and surfaces as "role not found", which reads as a missing bootstrap
rather than as bad input.
Validate the GUID shape case-insensitively on both the container-apps module
input and the root variable, then keep lower() for the canonical ARM
identifier: the two do different jobs, and dropping lower() would reintroduce
the case-only diff that replaces these ForceNew assignments on every refresh.
The root declaration is validated as well as the module input because only a
root variable validation is guaranteed to run before the role-definition data
source in compute.tf; verified by observing the validation error abort the plan
with the downstream read never attempted.
This is input validation, not a security control. A GUID-shaped string is
accepted whether or not that subscription exists, is reachable, or is the
right one, and the comments say so.
Refs #1802
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1802
Root cause
module "compute_container_apps"carries a module-leveldepends_on(
terraform/environments/azure/compute.tf), and Terraform propagates amodule-level
depends_onto every data source inside the module. Both datasources in that module were therefore deferred, which deploy run 31545175691
shows directly:
scopeandrole_definition_idare both ForceNew onazurerm_role_assignment,so the same run planned all three subscription-scoped assignments for
replacement, and those are exactly the three destroys the issue reports:
Fix
Resolve both inputs in the caller, where nothing defers them.
terraform/environments/azure/compute.tfand reaches the module through a newreservation_role_definition_idinput. The root module has nodepends_on,so the read happens at plan.
subscription_idinput rather than fromdata.azurerm_subscription.current.id. The deploy workflow already suppliesthat value as
TF_VAR_subscription_id, and it is what configures theazurermprovider, so it is statically known at plan.resource IDs, so a scope that differed from the canonical form only by case
would diff on every refresh and replace these ForceNew assignments forever,
which is the failure this PR exists to remove.
Both new variables have exactly one setter: the
module "compute_container_apps"block in
terraform/environments/azure/compute.tf.data.azurerm_subscription.currentstays in the module, now only for theAZURE_TENANT_IDenv var. It is still deferred, which is fine: an env var isnot ForceNew.
What was considered and rejected
terraform_remote_stateagainst the bootstrap stack. It removes thename-based lookup, but it gives the runtime stack a hard dependency on the
bootstrap stack's state file and its backend credentials. That coupling is a
worse problem than the one being fixed.
depends_on. It is the actual root cause and itis a one-line change, and every one of its four targets is already an implicit
dependency through referenced outputs. It was still rejected: module-level
depends_onalso orders the container app after resources that no referencedoutput covers (NSG associations, the jwt/session secrets, the database
firewall rules), and dropping that ordering is a greenfield-apply behaviour
change well outside what this issue asks for.
bounds. The bootstrap-vs-runtime split is unchanged here: the definition is
still owned by the human-applied
ci-cd-permissionsstack, and the runtimeside still only looks it up and creates the assignment.
Are both attributes now known at plan time?
Yes.
scopeis"/subscriptions/${var.subscription_id}"inside the module, fedfrom
lower(var.subscription_id)at the root. No data source involved.role_definition_idisvar.reservation_role_definition_id, fed from a rootdata source whose
nameandscopeare both var-derived, which has nodepends_on, and whose provider config is fully known.If either were still
(known after apply)the fix would not work, so this isthe load-bearing claim in the PR.
Verification
Offline, at the pattern level, with a Terraform sandbox that reproduces the
mechanism: one
externaldata source read from the root module and an identicalone read from inside a module carrying
depends_onon a resource that changeson every plan, each feeding a
triggers_replace(the ForceNew stand-in).First plan:
Second apply, no intervening change:
terraform_data.assignment_from_rootdoes not appear in the second plan at all.That is the before and after of this change, reproduced end to end.
Separately verified in the same way that a
data.x[0]reference from acount = 0module block is never evaluated, so thecompute_platform = "aks"path is unaffected.
Repo checks, all exit 0:
What is not proven
The real test is two consecutive applies against the live stack with no
intervening change, confirming the second plan reports no changes for
module.compute_container_apps[0].azurerm_role_assignment.reservations_purchaser.That has not run. This is unverified until two consecutive CI applies show it,
and
validatepassing is not evidence that it is fixed.One residual: if the
AZURE_SUBSCRIPTION_IDsecret's GUID casing differs fromAzure's canonical lowercase, the first plan after this change would still show
these three assignments replaced once, converging afterwards.
lower()is whatmakes it converge instead of churning forever.
Customer-side module
iac/federation/azure-target/terraformdoes not share this defect, and isnot touched here. Its
azurerm_role_assignment.cudly_reservationstakesscope = "/subscriptions/${local.subscription_id}"androle_definition_idfrom
module.cudly_reservation_role.role_definition_resource_id, a module inthe same state, so it is known from state after the first apply. No module in
that stack carries a
depends_on, so itsdata.azurerm_subscription.currentisread at plan.
Summary by CodeRabbit
Input validation on
subscription_id(added after review)subscription_idnow builds the reconstructed role-definition lookup name aswell as every subscription-scoped RBAC scope, so a value that is not a GUID at
all (a subscription display name, say) would search for a role that cannot exist
and surface as "role not found" - indistinguishable from a missing bootstrap.
Both
subscription_iddeclarations this PR is responsible for now carry acase-insensitive GUID
validationblock:terraform/modules/compute/azure/container-apps/variables.tf(the input thisPR introduces)
terraform/environments/azure/variables.tf(the root variable, whose job thisPR materially changed)
The root one is not redundant: only a root variable validation is guaranteed to
run before the role-definition data source in
compute.tf. Verified by planninga reduced copy of the same shape with
subscription_id="My Production Subscription"and observing the validation error abort the plan with thedownstream read never attempted. The pattern was exercised in both directions,
4 accepted (including all-uppercase) and 6 rejected.
lower()stays where it was. Validation andlower()do different jobs:validation rejects a non-GUID,
lower()pins an accepted GUID to the casingAzure returns in resource IDs. Removing
lower()would reintroduce thecase-only diff that replaces these ForceNew assignments on every refresh.
This is input validation and fail-fast, not a security control. A
GUID-shaped string is accepted whether or not that subscription exists, is
reachable, or is the right one. A shape check admits every value of that shape
and does not restrict which subscription can be targeted. The code comments say
the same.
The bootstrap stack's own
subscription_id(
terraform/environments/azure/ci-cd-permissions/variables.tf) was deliberatelyleft alone: this PR neither introduces nor changes it, and widening into an
untouched variable in a separate human-applied stack is out of scope for #1802.
Declined in writing on the review thread; a reasonable standalone follow-up.