Skip to content

Core 3.24.x Enum cast regression blocks core floor bump #8066

@MaxGhenis

Description

@MaxGhenis

Summary

policyengine-us can't cleanly bump its policyengine-core floor past 3.24.0 because core regressions break the tax_unit_itemizes.yaml integration test (and many others via the same cycle) with a TypeError during formula result casting.

Tracking the core fix here so the bump can land once core releases it.

Symptom under core 3.24.1

File ".../policyengine_core/simulations/simulation.py", line 1067, in _cast_formula_result
    return value.astype(variable.dtype)
TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'

Surfaces in:

  • policyengine_us/tests/policy/baseline/gov/irs/income/taxable_income/deductions/tax_unit_itemizes.yaml — "Integration test which fails without tolerance in the itemization choice" and "Oklahoma itemization test requiring higher tolerance"
  • any household simulation whose tax_unit_itemizes path crosses the itemizing reform branch (same class of test as refundable_ctc ↔ income_tax dependency cycle surfaces on itemizing branch #8059)

Root cause (in core, not us)

Two regressions in core 3.24.0 combine to break us:

  1. H3 _invalidate_all_caches over-reaches. Simulation.apply_reform wipes every variable's _memory_storage._arrays, including user-provided set_input values. policyengine-us's Simulation.__init__ applies a structural reform AFTER the situation is built, so every household input (state_fips, age, premium_tax_credit, …) gets wiped between init and the first calculate. The TypeError above is the downstream consequence: state_fips returns None, state_name.formula maps it through pd.Series.map({...})NaN, and the .astype(int64) cast blows up.

  2. C1 Holder.get_array fallback is too narrow. After fixing Basic prototype #1, the tax_unit_itemizes cycle surfaces (same stack as refundable_ctc ↔ income_tax dependency cycle surfaces on itemizing branch #8059). tax_liability_if_itemizing sets tax_unit_itemizes=True on an itemizing branch; ctc_limiting_tax_liability forks a no_salt sub-branch from it. Under 3.24.x, Holder.get_array on the no_salt branch only falls back to default, not to the parent itemizing branch, so tax_unit_itemizes re-runs its formula on no_salt → calls tax_liability_if_itemizing again → infinite recursion.

Plus a latent GroupPopulation.clone bug that only surfaces once #2 is fixed: the cloned holder's .simulation reference was pointing at the source sim, not the clone, so parent_branch walks started from the wrong node.

Core fix

PolicyEngine/policyengine-core#475 covers all three:

  1. Track user set_input provenance in Simulation._user_input_keys (both via Simulation.set_input and Holder.set_input — the latter covers the Simulation(situation=...) path through SimulationBuilder.finalize_variables_init) and replay on _invalidate_all_caches.
  2. Walk simulation.parent_branch in Holder.get_array before falling back to default.
  3. GroupPopulation.clone now passes the cloned population to holder.clone.

With that branch applied to core 3.24.1, the failing us tests pass; without it, they crash.

Plan

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions