Improve plan parser completeness and support optional() type defaults#46
Conversation
andreacappelletti97
left a comment
There was a problem hiding this comment.
Thanks for this — it's a substantial and genuinely useful contribution, and it shows you've run it against real infrastructure. I went through it in depth.
I'm requesting changes on two crash bugs that block merge, since c3x parses untrusted plan/HCL on a hosted backend and a panic there is an availability issue:
plan.go— nil-pointer panic on a nullchild_moduleselement (the inner recursive walk has nomod == nilguard).optional_defaults.go:112— index-out-of-range panic on a zero-argobject()type constraint.
One behavior change needs an explicit call: estimate plan.json (no --show-delta) now prices the whole post-apply fleet instead of only the changes — that can move --budget gates and the JSON payload for existing users, so let's document it.
A few correctness items (over-estimating null optional(object) synthesis; the old-format fallback still dropping no-op resources; applyOptionalDefaults not covering the root module) and some --show-delta polish (misleading output on plain .tf, dead delete-marker, full-fleet total in delta view) are inline.
Really nice work overall — happy to help land it once the two panics are handled. Details on specific lines below.
| collectPlannedResourcesInner(mod, regionPtr, deleted, actions, out) | ||
| } | ||
|
|
||
| func collectPlannedResourcesInner(mod *plannedModule, regionPtr *string, deleted map[string]bool, actions map[string]domain.PlanAction, out *[]domain.Resource) { |
There was a problem hiding this comment.
collectPlannedResourcesInner dereferences mod (mod.Resources, mod.ChildModules) with no nil check, and the recursive call below can pass a nil child. A plan JSON with a null element in child_modules (e.g. "child_modules":[null]) decodes to a nil *plannedModule and panics here. Since we parse untrusted plan JSON on the backend, this needs a if mod == nil { return } guard — the exported collectPlannedResources guards it, but this inner recursion doesn't.
There was a problem hiding this comment.
Fixed — since your #52 already has the nil-guard on the exported collectPlanned, and the function is no longer split into inner/outer in this PR, this is covered.
| // Not optional — could still be a nested object() with | ||
| // optional members inside it. | ||
| if nestedCall, nestedOk := item.ValueExpr.(*hclsyntax.FunctionCallExpr); nestedOk && nestedCall.Name == "object" { | ||
| children := extractObjectDefaults(nestedCall.Args[0]) |
There was a problem hiding this comment.
nestedCall.Args[0] has no arg-count guard, so a type constraint like object({ x = object() }) (zero-arg object()) panics with index out of range. Line 136 guards the same pattern with && len(innerCall.Args) == 1; the same check is needed here. With no recover() on the parse path this crashes the process on untrusted HCL.
There was a problem hiding this comment.
Added len(nestedCall.Args) == 1 guard — good catch, matches the pattern on line 136.
|
|
||
| // Walk planned_values to collect all resources in the target state. | ||
| var out []domain.Resource | ||
| collectPlannedResources(doc.PlannedValues.RootModule, region, deleted, actions, &out) |
There was a problem hiding this comment.
Semantics change worth calling out: walking planned_values means c3x estimate plan.json (without --show-delta) now prices the entire post-apply fleet rather than only the changed resources. That's a defensible definition of 'estimate', but it silently changes PROJECT TOTAL, can flip existing --budget CI gates, and changes the --format json payload for anyone feeding plan JSON today. Can we note this in the changelog/README?
There was a problem hiding this comment.
Agreed — this is now documented in your #52 commit message and I added a CHANGELOG entry on this PR as well. The --show-delta flag gives users who want the old change-only view a path forward.
| if od.defaultVal != cty.NilVal { | ||
| attrs[key] = applyChildDefaults(od.defaultVal, od.children) | ||
| changed = true | ||
| } else if len(od.children) > 0 { |
There was a problem hiding this comment.
When optional(object({...})) has no explicit second-arg default, Terraform leaves the attribute null, but here we synthesize a fully-populated object from the inner defaults (synthesizeFromChildren). That fabricates values that don't exist (e.g. a phantom disk) and over-estimates cost. Only the optional(object({...}), {}) form (explicit {}) should get the synthesized object; the no-default form should stay null.
There was a problem hiding this comment.
You're right — removed synthesizeFromChildren entirely. Now only optional(object({...}), {}) with an explicit 2nd arg produces a populated object. No-default stays null, no phantom costs.
|
|
||
| // Fallback for older plan formats or edge cases where | ||
| // planned_values is empty: use resource_changes directly. | ||
| if len(out) == 0 { |
There was a problem hiding this comment.
The older-format fallback still runs resources through shouldSkip, which filters no-op (and delete) actions. Since the primary path now intentionally keeps no-op resources, an all-unchanged plan in the old format returns 0 resources here — inconsistent with the new behavior. Worth aligning the fallback with the primary walk.
There was a problem hiding this comment.
Aligned — the fallback now only excludes isDeleteOnly (consistent with the primary path from #52). No-op resources are kept.
| // runtime type-system and c3x's static parsing: attributes | ||
| // like `os_disk = optional(object({disk_size_gb = optional(number, 64)}), {})` | ||
| // get their defaults applied to the caller-supplied values. | ||
| applyOptionalDefaults(childSources, childVars) |
There was a problem hiding this comment.
applyOptionalDefaults is only called for child-module vars. Root-module variables never get their optional(type, default) defaults applied, so a root resource referencing var.x.<optional-attr> still resolves to nil. Should the root module get the same treatment?
There was a problem hiding this comment.
Good point — added applyOptionalDefaults(sources, variables) as Stage 2b in parser.go, right after CLI vars are applied. Root module variables now get the same treatment.
| } | ||
|
|
||
| // In delta mode, group no-op resources into a summary. | ||
| if deltaOnly && (c.Action == domain.PlanActionNoOp || c.Action == domain.PlanActionNone) { |
There was a problem hiding this comment.
.tf resources carry PlanActionNone, which this treats as 'unchanged', so c3x estimate main.tf --show-delta on real infra prints 'No cost-affecting changes' and collapses every resource into the footer. That's misleading when there's no plan. Consider warning/disabling --show-delta for non-plan inputs, or treating PlanActionNone as shown when there is no plan context.
There was a problem hiding this comment.
Fixed — writeRendered now checks estimateHasPlanContext() (any Cost with a non-empty Action). If there's no plan context it prints a warning to stderr and falls back to the standard renderer. Flag description also updated to say "plan JSON only".
| case domain.PlanActionUpdate: | ||
| return "~" | ||
| case domain.PlanActionDelete: | ||
| return "-" |
There was a problem hiding this comment.
This - (delete) marker is effectively dead code: pure-delete resources are excluded upstream (buildDeletedSet on the planned_values path, shouldSkip on the fallback), so PlanActionDelete never reaches a rendered Cost. The --show-delta help advertises 'create/update/delete', so either surface deletions (with their cost removed from the total) or drop the claim. Related: the delta PROJECT TOTAL (line 105) prints the full-fleet est.ProjectTotal, not the delta cost, which is confusing in the delta framing.
There was a problem hiding this comment.
Both addressed: deleted resources are now appended to the output with their change.before attributes and PlanActionDelete, so the - marker is reachable. The footer shows a DELTA: +/-$X/mo line (net of creates minus deletes) followed by PROJECT TOTAL for full-fleet context.
|
On the two panics — both are one-liners if it helps:
Both want a small regression test. Happy to help land them. |
|
Heads up: I landed the full-fleet plan estimate in #52 (the #51 decision), so That lets this PR rescope to the parts that are uniquely yours and still very much wanted, now on a consistent base:
The two panic guards from the review are one-liners (described inline). Thanks again for surfacing all of this. |
fdc850d to
008f494
Compare
Adds a delta-mode renderer that shows only resources with plan actions (create/update/delete) annotated with +/~/- markers, and summarizes unchanged resources in a single footer line. Changes across the stack: - domain.Resource gains a PlanAction field; domain.Cost propagates it from the calculator so renderers know the intent per-resource. - Plan parser annotates each resource with its action via buildActionMap/ classifyActions, surfaces deleted resources (with "before" attrs), and uses json.RawMessage for provider expressions to avoid panics on Terraform 4.x array-valued blocks (azurerm features). - RenderDelta() dispatcher + RenderTextDelta(): filters costs by action, renders changed resources with markers, collapses no-op resources, shows a DELTA cost line and PROJECT TOTAL. - `c3x estimate --show-delta` activates the delta view; warns and falls back to standard view when used with non-plan inputs. - Falls back to the standard renderer for formats without a delta-specific implementation (markdown, JSON, etc.). Gives an Infracost-style diff view directly from plan JSON without requiring a separately saved baseline file.
… constraints
When a module variable uses optional(type, default) in its type
constraint and the caller omits those keys, the parser previously
stored a source-range string as the attribute value. This caused type
mismatches in catalog expressions like `default(os_disk_disk_size_gb, 0) > 0`.
This commit:
- Adds applyOptionalDefaults() which parses the type-constraint AST,
finds optional(type, default) calls, and merges their defaults into
caller-supplied values — matching Terraform runtime behavior.
- Applied to both root-module and child-module variables.
- Handles nested objects recursively: optional(object({...}), {}) with
children that themselves have optional() defaults are filled in at
arbitrary depth via applyChildDefaults.
- Only synthesizes values when an explicit default is provided (2nd arg
to optional()). Without an explicit default, the attribute stays null
— matching Terraform semantics and avoiding phantom cost estimates.
- Guards against zero-arg object() type constraints to prevent panics
on untrusted HCL input.
- Unresolvable attributes now store nil instead of a source-range
string, so `default(x, fallback)` expressions fall through correctly.
008f494 to
9dc1271
Compare
|
Hey Andrea, thanks for the thorough review — really appreciated that you went through this line by line. Rebased on current main (your #52 covers the planned_values foundation nicely, so I built on top of it instead of duplicating). All your comments are addressed: Crash fixes:
Behavior corrections:
CHANGELOG added to document the new flag and the fixes. Let me know if anything else needs attention! |
andreacappelletti97
left a comment
There was a problem hiding this comment.
Re-reviewed end to end — really nice work. All eight items from the last round are fixed and verified: build + go test -race pass on every touched package, and I exercised --show-delta on a plan and on plain .tf. The panic guards, the synthesizeFromChildren removal, root-module optional() defaults, the no-op fallback alignment, and the delete marker + DELTA line all check out.
One regression to fix before merge, in the core (non-delta) path — details inline. Once that's handled I'll squash-merge.
| // in planned_values (they won't be in the post-apply state) but | ||
| // the delta renderer needs them to show what's being removed. | ||
| // We use the "before" attributes so the calculator can price them. | ||
| for _, rc := range doc.ResourceChanges { |
There was a problem hiding this comment.
Appending pure-delete resources into the same out slice the calculator prices means they land in PROJECT TOTAL — which regresses the #52 semantics: estimate plan.json without --show-delta now counts resources that won't exist post-apply.
Repro with a plan of {create t3.small, no-op t3.micro, delete t3.medium}: PROJECT TOTAL prints $53.14, but the post-apply fleet is $22.77 (the two survivors). --show-delta shows the same inflated $53.14.
Deletes shouldn't contribute to the post-apply total, and shouldn't appear at all in the non-delta view. Suggest surfacing them to the delta renderer via a separate field (or excluding PlanActionDelete from the priced total) so the - marker and the DELTA line keep working while PROJECT TOTAL stays the post-apply fleet.
There was a problem hiding this comment.
Good catch — you're right, appending deletes into out broke the #52 semantics. Fixed in 63b6d0f:
NewEstimatenow skipsPlanActionDeletewhen summingProjectTotal- Standard renderer hides deleted resources entirely (they only surface in
--show-delta) - Action markers (
+/~/-) scoped to delta mode only — normal view has no prefixes
Verified with your repro scenario: estimate plan.json shows only the two survivors, --show-delta shows all three with correct markers and a separate DELTA line.
…d view Deleted resources were appended to the output for the delta renderer but inadvertently inflated PROJECT TOTAL and appeared in the standard (non-delta) view — regressing the c3xdev#52 post-apply-fleet semantics. Fix: - NewEstimate skips PlanActionDelete costs when summing ProjectTotal. - Standard renderer skips deleted resources entirely (they only surface in --show-delta mode with the `-` marker). - Action markers (+/~/−) are now scoped to delta mode only; the normal estimate view shows no plan-action prefixes.
|
Merged. Thanks for the thorough back-and-forth on this — the delete-total fix is exactly right, and the whole change (delta view, root + child optional() defaults, and the two panic guards) is a genuinely valuable addition tested against real infrastructure. Really appreciate the careful work across the review rounds. 🙏 |
Two parser fixes that unblock our Terraform codebases (tested against a production Azure environment with 90+ resources across nested modules), plus a
--show-deltaflag that builds on the plan-parser work.Problem
Plan parser: empty estimates from plan JSON
The parser read only
resource_changesand skipped no-op actions. In a typical plan where 95% of resources are unchanged, the estimate came back near-zero. Additionally,provider_config.expressionsused a strict struct that panicked on Terraform 4.x array-valued expressions (azurerm emitsfeaturesas[{}]).HCL parser: crash on
optional()type constraintsWhen the caller omits optional keys, the parser stored a source-range string as the value. Catalog expressions like
default(os_disk_disk_size_gb, 0) > 0then failed with type mismatches.Fix
Plan parser rewrite (
internal/parser/plan)planned_valuesfor the complete post-apply resource setPlanActionfromresource_changesjson.RawMessagefor provider expressions (handles arrays gracefully)resource_changesfor older plan formatsHCL optional() defaults (
internal/parser/terraform)applyOptionalDefaults(): parses the type-constraint AST, findsoptional(type, default)calls, merges defaults into caller-supplied variables — matching Terraform's runtime behaviorapplyChildDefaults/synthesizeFromChildrennilinstead of a string placeholder, sodefault(x, fallback)expressions fall through correctlyDelta rendering (
internal/render,cmd/c3x)domain.CostcarriesPlanActionfrom the source resourceRenderTextDelta(): shows only changed resources with+/~/-markers, summarizes unchanged in a footerc3x estimate --show-deltaactivates the delta viewValidation
End-to-end against a production Azure environment (AKS, PostgreSQL Flexible Server, App Gateway WAFv2, File Shares, Redis, Container Registry, FTP VMs):
Delta mode example
Commits
fix(plan): rewrite plan parser to read planned_values with action annotationsfix(terraform): resolve optional() defaults from module variable type constraintsfeat(render): add --show-delta flag with plan-aware delta rendering