Repository navigation
sec(iac/gcp): pin WIF trust to the exact role ARN and narrow the SA grant - #1675
Conversation
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 35 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (5)
Comment |
|
@coderabbitai full review |
✅ Action performedFull review finished. Your included review limit is currently reached under our Fair Usage Limits Policy. Your recent PR review activity is in the 95th percentile or higher among CodeRabbit users, so adaptive limits apply. This review may still proceed through usage-based billing if eligible. Your next included review will be available in 56 minutes. |
|
@coderabbitai full review |
✅ Action performedFull review finished. Your included review limit is currently reached under our Fair Usage Limits Policy. Your recent PR review activity is in the 95th percentile or higher among CodeRabbit users, so adaptive limits apply. This review may still proceed through usage-based billing if eligible. Your next included review will be available in 10 minutes. |
…rant
The GCP WIF Terraform module mapped attribute.aws_role to the raw
assertion.arn and gated on a substring test:
"attribute.aws_role" = "assertion.arn"
attribute_condition = "attribute.aws_role.contains('assumed-role/${var.aws_role_name}/')"
AWS's IAM path grammar is (/)|(/[!-~]+/), so an attacker holding only
iam:CreateUser in the trusted account can create a user at path
/assumed-role/<pinned role>/. The resulting ARN
arn:aws:iam::<acct>:user/assumed-role/<pinned role>/<name> carries the
substring the condition looked for, and the pool-wide
principalSet://.../* impersonation grant admitted every identity in the
pool, yielding full impersonation of the CUDly service account and its
compute.commitments.create/update authority.
Normalise the session ARN down to the role ARN in the attribute mapping,
then compare with == against arn:aws:sts::<acct>:assumed-role/<role>.
An IAM user ARN normalises to an arn:aws:iam::...:user/... value that
cannot equal an arn:aws:sts:: role ARN, and equality (not a prefix or
substring test) means a longer sibling role such as
CUDly-Execution-Admin does not inherit CUDly-Execution's trust.
Narrow the impersonation grant from the pool-wide wildcard to
principalSet://.../attribute.aws_role/<exact role ARN>, restoring a
second independent gate so a future mapping bug cannot widen access
pool-wide on its own. The normalised attribute strips the per-session
suffix, which is what makes an exact-value principalSet possible; the
variable session name was the stated reason the wildcard was believed
necessary.
Mapping expression and member form are byte-identical to the shell
sibling fixed in #1651, so the two customer-facing onboarding paths
produce the same provider. That script's pool_wide_grants() scan flags a
Terraform-created wildcard grant as legacy, so before this change a
customer who onboarded with Terraform and later ran the script was
correctly told their setup was unsafe.
Upgrade ordering for already-applied deployments: attribute_mapping and
attribute_condition update in place (neither is ForceNew), while member
is ForceNew. depends_on reconfigures the provider before the member
swap, and create_before_destroy adds the narrow member before the old
pool-wide one is removed, so no window exists in which the service
account has no matching grant.
Also add validation blocks on aws_role_name and oidc_subject, which are
interpolated into the CEL condition and into an IAM principal
identifier: aws_role_name = "x') || true || ('" rendered an always-true
condition. And stop shipping the identity pin commented out and labelled
"Recommended" in the served tfvars template; main.tf requires it, so the
downloaded bundle could not apply.
Closes #1667
The wildcard guard added alongside the narrowed impersonation grant read `principalSet?://[^"]*\*`, where the `?` applies to the preceding `t` rather than to `Set`. It therefore matched only `principalSe`/ `principalSet`, and a wildcard reintroduced on the OIDC branch (`principal://.../*`) would have slipped past the very check that exists to reject it. Group the optional segment: `principal(Set)?://[^"]*\*`. Confirmed by running the test against the pre-fix main.tf: the guard now reports the wildcard member specifically, not just the missing exact-value member. Also drop the em-dashes from the lines this change added, and correct the comment claiming the mapping is byte-identical to the shell sibling: the attribute.aws_role expression, the condition and the member form are identical, but the module additionally maps attribute.account, which the script does not. The script compares conditions rather than mappings, so the extra attribute stays inert there.
golangci-lint at the version CI pins (v2.10.1) flags "behavioural" and "labelled" via the misspell linter. Both are in a comment and an assertion message, so the rewrite touches no identifier and no asserted string.
The Terraform module maps three attributes on the AWS branch (google.subject, attribute.aws_role, attribute.account) while arm/CUDly-CrossSubscription/setup-gcp-wif.sh maps only the first two. Nothing reads attribute.account: its only two references in the tree were the mapping line itself and the comment above it. It is not named by the attribute_condition, by the impersonation grant's principalSet, or by any test. Today that divergence is inert, because the script compares only the provider's attributeCondition. Once the script also compares attributeMapping as an exact set, it stops being inert: both paths default to pool 'cudly-pool' and provider 'cudly-provider', so a customer onboarded via the Terraform bundle who then runs the script against the same project lands on this module's provider, fails the set comparison on the one extra key, and is told to delete the provider and re-run. That detaches every live federated session, and the next terraform apply then fights the recreated provider. The verdict would also be wrong: the mapping is not unsafe, it just carries one extra unused key. Removing the key rather than teaching the script to expect it keeps the comparison an exact set, which is what stops an unexpected attribute from being introduced later and keyed on by a pool-wide principalSet://.../attribute.X/... grant. Account pinning is unaffected: the provider's aws.account_id block and the account number inside the pinned role ARN both still constrain it. Also corrects the comment above the mapping. It specifically anticipated this attribute and concluded it was inert because the script "compares conditions, not mappings" - true when written, false as soon as the script compares mappings, and a future reader would have trusted it. Migration: attribute_mapping is not ForceNew in the google provider (the field is Optional with no ForceNew, and Update appends "attributeMapping" to the PATCH updateMask), so dropping a key is an in-place update. It rides along with the change this branch already makes to the same field and adds no replacement risk. TestGCPTargetMappingMatchesSetupScript locks the two paths to the same mapping set, parsing both files rather than restating either, and fails if a key is ever added on one side alone. Refs #1667
91ca3c0 to
40d5207
Compare
Adversarial review record (merged)Independent reviewer, distinct from the author. Recording it here because the merge rested on this — CodeRabbit's status on this PR read Exactness verified, no bypass found. The condition is Non-obvious correctness point: Google documents two AWS mapping variants. The simpler Pool-wide binding genuinely narrowed to the documented exact-attribute-value Migration confirmed from provider source, not the PR's word. Tests mutation-tested, not merely run pre-fix: four mutations of Finding recorded, not blocking
Two intentional narrowings worth knowing: the pinned ARN hardcodes Hygiene issue found
|
Closes #1667
Recovered from two interrupted sessions (see "Provenance" at the bottom). Both halves of the fix land here.
The defect
iac/federation/gcp-target/terraform/main.tfmapped the attribute raw and gated on a substring:AWS's IAM path grammar is
(/)|(/[!-~]+/), so an attacker holding onlyiam:CreateUserin the trusted account creates a user at path/assumed-role/CUDly-Execution/. The resulting ARNarn:aws:iam::<acct>:user/assumed-role/CUDly-Execution/evilcontains the substring the condition looked for, and the pool-wideprincipalSet://.../*grant admitted every identity in the pool. Result: full impersonation of the CUDly service account and itscompute.commitments.create/updateauthority.IAM role paths do not work here, because STS strips the path from
assumed-roleARNs. It is specifically the IAM user ARN, which retains its path, that defeated the substring test.This is customer-facing:
iac/embed.goembedsfederation, andinternal/api/handler_federation.goservesfederation/gcp-targetas the default bundle fortarget=gcp.Half 1 — exact match, not substring
Normalise the session ARN down to the role ARN in the mapping, then compare with
==:The mapping expression is byte-identical to the one merged into the shell sibling in #1651 (
arm/CUDly-CrossSubscription/setup-gcp-wif.sh:209), so the two customer-facing onboarding paths now produce the same provider. Before this change, that script'spool_wide_grants()scan correctly flagged a Terraform-created grant as legacy and refused to report success.On "genuinely exact, not another prefix construct" — three substring-for-exact-match defects have surfaced in this codebase, so the replacement is tested behaviourally, not just eyeballed. An IAM user ARN normalises to
arn:aws:iam::<acct>:user/assumed-role/<role>, which cannot equal anarn:aws:sts::role ARN; and==(notstartsWith) meansCUDly-Execution-Admindoes not inheritCUDly-Execution's trust.iac/gcp_target_wif_test.gotranscribes the CELextract()semantics into Go and drives a table covering both:arn:aws:sts::<acct>:assumed-role/CUDly-Execution/cudly-sessionarn:aws:sts::<acct>:assumed-role/CUDly-Execution/i-0abc123(different session)arn:aws:sts::<acct>:assumed-role/CUDly-Execution-Admin/s(longer sibling role)arn:aws:iam::<acct>:user/assumed-role/CUDly-Execution/evil(the reported bypass)arn:aws:iam::<acct>:user/assumed-role/CUDly-Execution/assumed-role/CUDly-Execution/evil(repeated marker)arn:aws:sts::<acct>:assumed-role/Attacker/assumed-role-CUDly-Execution(crafted session name)arn:aws:sts::210987654321:assumed-role/CUDly-Execution/s(wrong account)arn:aws-us-gov:sts::<acct>:assumed-role/CUDly-Execution/s(partition mismatch — fails closed)arn:aws:iam::<acct>:user/alice,...:rootThe bypass test carries a fixture guard that first asserts the exploit ARN does satisfy the old
contains()condition — so if the fixture ever stops reproducing the reported bug, the test fails loudly rather than passing vacuously. A separate test pins the CEL expression inmain.tfbyte-for-byte, so the Go transcription cannot silently drift from what Terraform actually applies, and rejects a re-introducedcontains(/startsWith(onattribute.aws_role.Half 2 — the pool-wide binding is narrowed (it does ship)
Tightening only the condition would have left the blast radius of the next mapping bug intact, so this was treated as required, not optional. The old inline comment claimed exact-match was impossible because session ARNs carry variable session names — that is true of the raw ARN, but the normalised
attribute.aws_rolestrips the session suffix, which is precisely what makes an exact-value principalSet work. The same member form is already merged and shipping in the shell path (setup-gcp-wif.sh:267).A regression test rejects any wildcard in a principal identifier (
principal(Set)?://[^"]*\*), on either branch. The guard first shipped asprincipalSet?://..., where the?bound to the precedingtand so never matched a wildcard reintroduced on the OIDC branch; fixed in the follow-up commit on this branch.Documented caveat added in code: principal identifiers are pool-scoped, not provider-scoped, so any provider added to this pool that can mint the same attribute value satisfies the grant.
var.pool_idshould stay dedicated to CUDly.Migration verdict: safe in-place upgrade, no replacement of the pool or provider
Determined from the provider schema, not memory.
attribute_mappingandattribute_conditionupdate IN PLACE. Inhashicorp/terraform-provider-google,google/services/iambeta/resource_iam_workload_identity_pool_provider.go, neither schema entry carriesForceNew:Only
workload_identity_pool_idandworkload_identity_pool_provider_idareForceNew: true. The registry docs contain no "Changing this forces a new resource to be created" text for either field. The pool and the provider are not replaced — this is not a breaking change for existing customers.memberongoogle_service_account_iam_memberIS ForceNew. From the shared generatorgoogle/tpgiamresource/resource_iam_member.go:So the grant is replaced, and ordering is the whole risk. The dangerous window the issue warns about is real: if the narrow member were created while
attribute.aws_rolestill resolved to the raw session ARN, it would match nothing, and if the old pool-wide member were removed first, coverage would lapse. A safe sequence exists and is now pinned in the config:Net apply order: reconfigure provider (in place) → add narrow member → remove old pool-wide member. At least one matching grant is in place at every step.
create_before_destroyis meaningful here becausegoogle_service_account_iam_memberis the non-authoritative form. Registry docs, verbatim: "google_service_account_iam_member: Non-authoritative. Updates the IAM policy to grant a role to a new member. Other members for the role for the service account are preserved." Two members granting the same role to different principals coexist during the window; the authoritative_binding/_policyforms would not permit this.TestGCPTargetUpgradeOrderingIsPinnedasserts both directives are present, because losing either turns a security fix into a customer outage.One point I could not hard-confirm: I found no Google doc stating explicitly whether an attribute value containing colons (a full ARN) must be URL-encoded inside a
principalSet://identifier. Google's docs do show unencoded slashes in attribute values (attribute.repository/my-org/my-repo), and the provider's own registry example comparesattribute.aws_roleto a full unencoded ARN. Decisive for this PR: the identical unencoded-ARN member string is already merged and shipping in the shell path (#1651), so the two paths agree either way — which was the goal. Flagging it rather than presenting it as settled.Interop window with #1651 (already merged as 3fc9b57)
#1651 is on
main, this is not. Until this merges and each Terraform-onboardedcustomer re-applies, the two customer-facing onboarding paths actively contradict
each other:
setup-gcp-wif.shnow scans the service account policy forprincipalSet://.../workloadIdentityPools/<pool>/*and refuses to reportsuccess while one exists, calling it a legacy grant from an older run of
itself.
iac/federation/gcp-target, not by the script. The verdict is correct (thegrant really is pool-wide) but the attribution is not, and the script's
suggested remedy,
--remove-legacy-pool-binding, would delete a member thatTerraform still owns in state, so the next
terraform applyrecreates it.That window closes per-customer on
terraform applyafter this merges: the applyreplaces the wildcard member with the exact-value one, and the provider condition
this writes is byte-identical to the one the script expects, so a subsequent
script run reports success instead of failing its condition comparison.
Operators who want the wildcard gone before re-applying should re-apply rather
than run
--remove-legacy-pool-binding, for the state-drift reason above.Cross-PR interop fix:
attribute.accountdropped (found by adversarial review of #1673 against #1675)Found only by reviewing this PR against open #1673, not by reviewing either
PR on its own. Neither is broken alone; the incompatibility exists only once both
merge.
main.tfmapped three attributes on the AWS branch whilearm/CUDly-CrossSubscription/setup-gcp-wif.shmaps two:google.subject=assertion.arnattribute.aws_role= (the normalising CEL)attribute.account=assertion.accountOn
maintoday this is inert, because the script compares onlyattributeCondition. #1673 adds an exact-SET comparison ofattributeMapping, not a subset test. Both paths default to poolcudly-pooland provider
cudly-provider(verified atvariables.tf:9,15andsetup-gcp-wif.sh:46-47), so a customer onboarded via the Terraform bundle whothen runs the script lands on this module's provider and hits a hard
die:whose only offered remedy is deleting the provider, which detaches every live
federated session and which the next
terraform applythen fights. The verdictwould also be wrong: the mapping is not unsafe, it carries one extra inert key.
Reproduced by running #1673's
normalize_mappingverbatim against a TF-createdprovider's mapping: 3 parsed pairs vs 2, so MISMATCH, so
die. The OIDC branchmatches on both sides and was never affected.
Fix: drop
"attribute.account" = "assertion.account". Nothing reads it. Itsonly two references in the tree were the mapping line and the comment above it;
it is not named by the
attribute_condition, by the impersonation grant'sprincipalSet, or by any test. It is pre-existing onmain(last touched by#510), so this PR did not introduce it, but making the two paths agree is this
PR's stated goal and this was dead configuration.
Both alternatives are worse. Relaxing #1673's check to a subset test re-opens the
door to an unexpected extra attribute that a pool-wide
principalSet://.../attribute.X/...grant could key on. Addingattribute.accountto the script's
EXPECTED_MAPPINGforces #1673 to change and leaves deadconfiguration on both sides.
Account pinning is unaffected: the provider's
aws { account_id = ... }blockand the account number inside the pinned role ARN both still constrain it.
Migration: safe, in place, no replacement.
attribute_mappingis notForceNew (same schema check as the section above;
Updateappends"attributeMapping"to the PATCHupdateMask), so dropping a key is an in-placeupdate. It rides along with the change this PR already makes to that same field
and costs no additional migration risk.
The comment above the mapping is corrected, not just trimmed. It specifically
anticipated this attribute and reached the wrong conclusion ("it compares
conditions, not mappings, so the extra attribute is inert there"): accurate
against
maintoday, false the moment #1673 merges. A comment a future readerwould trust is worse than no comment, so it now states the invariant that
actually holds and the consequence of breaking it.
TestGCPTargetMappingMatchesSetupScriptlocks the parity. It parses the AWSmapping out of both files rather than restating either, and fails if a key is
ever added on one side alone. Confirmed to fail with
attribute.accountre-addedand pass without it, and to pass against both the current script and #1673's
EXPECTED_MAPPINGform.Also fixed (issue items c and d)
aws_role_nameandoidc_subjectare interpolated into a single-quoted CEL string literal and into an IAM principal identifier, with no validation.aws_role_name = "x') || true || ('"rendered an always-true condition. Addedvalidationblocks rejecting' " \* $, plus format checks (AWS's own role-name charset[A-Za-z0-9_+=,.@-]{1,64}; subject charset and GCP's 127-chargoogle.subject` cap). The character check is kept separate from the format check because Terraform reports every failing validation, and "this would rewrite the attribute condition" is more actionable than a bare charset regex.internal/iacfiles/templates/gcp-wif.tfvars.tmplshipped the identity pin commented out and labelled "Recommended" whilemain.tfrequires it, so the downloaded bundle could notterraform apply. Now emitted uncommented and labelled REQUIRED, with the AWS branch naming the account the role must live in. The value is intentionally left empty: an empty string hits the lifecycle precondition and fails the apply with an explanation, whereas aREPLACE_MEplaceholder would satisfy the variable validation and silently pin trust to a nonexistent role.These template changes are required by the fix, not incidental —
main.tf's precondition rejects an emptyaws_role_name, so a tfvars still shipping it commented out would break the apply.templates_test.gocovers both theawsandazurerender branches.Verification
terraform fmt -check -diff— clean.terraform init -backend=false && terraform validate—Success! The configuration is valid.go test ./iac/... ./internal/iacfiles/...— 25 pass (24 +TestGCPTargetMappingMatchesSetupScript).main.tfand the one extracted fromsetup-gcp-wif.sh, each sorted and rendered askey=valuelines,cmpequal (sha256686df559...). Before droppingattribute.accountthey differed by exactly that line.normalize_mappingrun verbatim against a simulatedgcloud ... describe --format='value(attributeMapping)'of a TF-created provider: MISMATCH (woulddie) before, MATCH (reuses the provider) after.main.tf+ the tfvars template to the parent commit and re-running gives 4 failures (TestGCPTargetPinsRoleARNByEquality,TestGCPTargetGrantsImpersonationToOneIdentity,TestGCPTargetUpgradeOrderingIsPinned,TestGCPWIFTfvarsMarksPinnedIdentityRequired); all pass after. Tests that would have stayed green with the bug present do not count as verification.--no-verify).Not verified: a live
terraform applyagainst a real GCP project and a real AWS token exchange. The migration verdict rests on the provider schema and the already-merged shell path, not on an executed upgrade.Provenance
Recovered from two interrupted sessions that left uncommitted work in two worktrees:
wt-1667on branchsec/1667-gcp-wif-tf-exact-match, based on an older main — 4 files staged, never committed.wt-1667on branchsec/1667-gcp-wif-tf-bypass, based on current main — the same 4 files with more complete content, plus an untrackediac/gcp_target_wif_test.gocarrying the behavioural test suite.I read both diffs before choosing. The second was both more complete and already exactly at
origin/main(0 ahead / 0 behind), so it needed no rebase; the first attempt's stale base was avoided rather than rebased. Adopted from the second attempt: all five files. Verified independently rather than taken on trust: the ForceNew/migration claims in its code comments (checked against provider source — they hold), that the regression tests fail pre-fix (they do), that the mapping expression matches the merged shell sibling byte-for-byte (it does), that the template changes are required by the fix (they are), andterraform fmt/validate/go test.