Skip to content

refactor(v4): PR 4c-i of the 4.0 cut - decouple the TWFE event-study mode from the MultiPeriodDiD containers and migrate the MPD-fixture suites - #868

Merged
igerber merged 2 commits into
mainfrom
feat/v4-pr4c-i-twfe-es-decoupling
Oct 25, 2026
Merged

igerber merged 2 commits into
mainfrom
feat/v4-pr4c-i-twfe-es-decoupling

Conversation

@igerber

@igerber igerber commented Oct 24, 2026

Copy link
Copy Markdown
Owner

Summary

  • 4.0 cut, PR 4c-i (docs/v4-design.md section 9; item 4c is split into 4c-i/4c-ii in this diff - user decision 2026-10-24). The TWFE event-study mode is decoupled from the MultiPeriodDiD containers: DifferenceInDifferences._fit_event_study_core returns a private _EventStudyCorePayload, TwoWayFixedEffects.fit(..., event_study=True) builds its EventStudyResults from it via results_base._from_event_study_payload, and only MultiPeriodDiD.fit still wraps the payload into MultiPeriodDiDResults (estimators._mpd_results_from_payload). _from_mpd and the payload route share one calendar-surface builder (_calendar_surface_from_arrays; the sorted-label contract is an explicit raise), and _resolve_scalar_df_survey's metadata half is split out as _scalar_df_survey_from_metadata. No user-visible behavior changes; the ledger's M-010 / M-011 "decouple before deleting" obligation is met, so PR 4c-ii can delete the class.
  • Oracle captured BEFORE the refactor: tests/_capture_v4_merge_mpd_oracles.py ran on the unmodified tree at 68e383bd (python backend; the rust capture agreed at rtol 1e-12 / atol 1e-13) and wrote tests/fixtures/v4_twfe_event_study_oracles.json - every EventStudyResults field of the TWFE surface AND of its MultiPeriodDiD twin across eleven lanes (pooled default / explicit cluster / hc2_bm; within + explicit cluster / hc2 carve-out / Conley / survey TSL / JK1 replicates; string labels; a non-suffix partition; a rank-deficient NaN post period). Lanes live in the shared tests/helpers/twfe_es_oracle_lanes.py; the fixture echoes each lane so an edited lane fails as lane drift, not as a numeric regression.
  • The MultiPeriodDiD-fixture test suites are migrated to TwoWayFixedEffects().fit(..., event_study=True) while MultiPeriodDiD is still the live oracle (the 3a -> 3b pattern): honest_did, pretrends (plus a TWFE pre_periods= class), methodology pretrends/honest_did, visualization, plotly, Conley, survey, replicate weights, survey_phase6, vcov_type (plus a within+hc2 R-parity sibling class on the mpd_absorbed_fe_did golden), estimators (a TestTwoWayFixedEffectsEventStudy twin class), the event-study surface roster (two native producers), variance-conventions (two twfe_es_* rows, table regenerated) and the DiagnosticReport / BusinessReport generic fixtures (static TWFE fit, per the user's decision). MPD-only tests stay for 4c-ii: the class API / deprecation / avg_* pins, the MPD-vs-TWFE parity oracles, multi-absorb / fixed_effects= / wild-bootstrap designs, container-field assertions, the MPD-branch consumer tests, the native-MPD plot test and the report classes' event-study-lane and name-string-dispatch pins (kept on legacy_mpd_fit).
  • Spec item 4c-ii records the spec="within" benchmark-arm relabel (the arm fits the unit-FE design, absorb=["unit"] vs R feols(... | unit)), the M-016 view's record type and the build_event_study_surface sniff reorder; ledger M-010 / M-011 notes updated (no status flips); a TODO.md row tracks the one-way HC2-BM absorbed-FE R-parity lane that the within spec cannot express.

Methodology references (required if estimator / math changes)

  • Method name(s): N/A - no estimator or math changes. The event-study core's arithmetic is unchanged; the payload refactor is pinned bit-identical (pooled) / 1e-12 (within) to the pre-refactor capture and to the live MultiPeriodDiD twin.
  • Paper / source link(s): N/A
  • Any intentional deviations from the source (and why): None

Validation

  • Tests added/updated: new tests/test_twfe_event_study_decoupling.py (65 tests: committed-oracle comparison of the TWFE surface and the MPD twin per lane, lane-echo drift pin, payload-vs-_from_mpd byte identity, full-container TWFE-vs-MPD identity, the unsorted-dict sorted-zip contract, the df_survey resolver split, post_period_average identity with MPD's avg_att/avg_se and its NaN contract on a real rank-deficient fit; 65 passed on both backends and under the py3.9-leg pins scipy 1.13.1 / numpy 2.0.2); shared helpers tests/helpers/twfe_es_oracle_lanes.py, event_study_average.py, event_study_surface_builders.py; the ported suites above (honest_did 88, pretrends 74, methodology_honest_did 38, methodology_pretrends 37, visualization 30, plotly 43, conley 150, survey 138, replicate 37, phase6 80, vcov_type 137, estimators 174+, event_study_surface 195 + 2 skipped, variance_conventions incl. the two slow rows, diagnostic_report 167, business_report 202). Untouched MPD-consuming and roster suites re-run unmodified after the refactor (results_serialization, result_aliases, practitioner, base_estimator, aliases, profile_panel, balance_interop_contract, target_parameter, power's MPD tests, naming_guard, v4_matrix): all green. tests/test_v4_merge_mpd.py (72) unchanged and green on both backends. Doc gates (test_v4_matrix, test_naming_guard, test_tracking_files, test_doc_deps_integrity, test_changelog_fragments, test_docs_ia, test_guides, the variance-conventions table gate) green after the spec/ledger/TODO/changelog edits. CI-pinned mypy diff_diff: no issues; black/ruff clean; py3.9 compileall clean.
  • Backtest / simulation / notebook evidence (if applicable): N/A (no notebook touched; tests/test_t23_spillover_tva_drift.py, the TWFE event-study consumer, green on both backends).

Security / privacy

  • Confirm no secrets/PII in this PR: Yes

Changelog

  • changelog.d/ fragment added (or N/A - no user-visible change): Yes - changelog.d/20261024-v4-pr4c-i-twfe-es-decoupling.md (### Internal, ### Testing)

Follow-ups

  • PR 4c-ii: the deletions (MultiPeriodDiD, MultiPeriodDiDResults, PeriodEffect, EventStudy; _from_mpd, _mpd_results_from_payload, the payload's avg_* block), the ledger flips (M-010/M-011/M-012 -> removed; M-060/M-007/M-083 -> done; M-016 locator -> EventStudyResults.period_effects, shimmed), the M-016 record view, the M-083 raise, the pyproject.toml filter, the benchmark arm -> spec="within", the power registry entry, the rosters, the two markdown notebook cells, the build_event_study_surface sniff reorder, the legacy_mpd_fit fixtures and every MPD-only test named in the spec item, the MPD half of tests/test_twfe_event_study_decoupling.py and the oracle's mpd_twin records, the caller-less HonestDiD._estimate_max_pre_violation MPD branch, and a re-pin of the two twfe_es_* variance rows once the core's post-average block goes (their tail_df carries that call today).
  • TODO.md: TWFE event-study one-way HC2-BM absorbed-FE R parity via an explicit singleton-cluster column.
  • Cluster metadata (cluster_name / n_clusters / vcov_type) on EventStudyResults: the auto-clustering event-study mode has no surface for it (TODO candidate).

…mode from the MultiPeriodDiD containers and migrate the MPD-fixture suites

Cut-sequence item 4c-i (docs/v4-design.md section 9; item 4c split into
4c-i/4c-ii, user decision 2026-10-24). The TWFE event-study mode no longer
constructs a MultiPeriodDiDResults: DifferenceInDifferences._fit_event_study_core
returns a private _EventStudyCorePayload, TwoWayFixedEffects.fit(...,
event_study=True) builds its EventStudyResults from it via
results_base._from_event_study_payload, and MultiPeriodDiD.fit wraps the same
payload into its container (estimators._mpd_results_from_payload). _from_mpd and
the payload route share one calendar-surface builder
(_calendar_surface_from_arrays, sorted-label contract enforced with an explicit
raise), so the two paths cannot drift; _resolve_scalar_df_survey's metadata half
is split out as _scalar_df_survey_from_metadata. No user-visible behavior
change; mypy zero errors.

Oracle (captured BEFORE the refactor): tests/_capture_v4_merge_mpd_oracles.py
ran on the UNMODIFIED tree (68e383b, python backend; the rust capture agreed
at rtol 1e-12 / atol 1e-13) and wrote tests/fixtures/v4_twfe_event_study_oracles.json -
every EventStudyResults field of the TWFE surface AND of the MultiPeriodDiD
twin across eleven lanes (pooled default / explicit cluster / hc2_bm; within +
cluster / hc2 / Conley / survey TSL / JK1 replicates; string labels; a
non-suffix partition; a rank-deficient NaN post period). Lanes live in
tests/helpers/twfe_es_oracle_lanes.py (shared by the script and the test; the
fixture echoes each lane so an edited lane fails as lane drift).
tests/test_twfe_event_study_decoupling.py pins the refactored surface against
that capture (CI bounds at the scipy-drift tolerance 1e-9), the MPD twin
against its capture, payload-vs-_from_mpd byte identity, the full-container
TWFE-vs-MPD identity (exact pooled / 1e-12 within), the sorted-zip contract on
an unsorted hand-built dict, the df_survey resolver split, and the
post_period_average helper's MPD arithmetic incl. its NaN contract.

Suite migration (ported = replaced; MPD-only tests stay for 4c-ii): honest_did,
pretrends (+ a TWFE pre_periods= class; the MPD class is MPD-only because it
promotes declared-post labels), methodology pretrends/honest_did (hand-built
calendar surfaces, reference inserted so no coefficient is demoted),
visualization (the native-MPD plot test stays as the period_effects-branch pin),
plotly, Conley (unit= kept: Conley requires it and never auto-clusters),
survey, replicate weights, survey_phase6, vcov_type (incl. a within+hc2
R-parity sibling class; the one-way HC2-BM lane goes to TODO.md), estimators
(a TwoWayFixedEffectsEventStudy twin class; the class-scoped panels hoisted to
module builders), the event-study surface roster (two native producers),
variance-conventions (two twfe_es_* rows + regenerated table), and the
DiagnosticReport/BusinessReport generic fixtures (static TWFE fit, legacy MPD
fixture kept for the event-study-lane and dispatch pins). Ported avg_att/avg_se
reads use tests/helpers/event_study_average.py (MPD's arithmetic); hand-built
containers use tests/helpers/event_study_surface_builders.py.

Spec: item 4c split; 4c-ii records the spec="within" benchmark relabel, the
M-016 record type and the build_event_study_surface sniff reorder. Ledger:
M-010/M-011 notes only (no status flips). TODO row for the HC2-BM lane.
@igerber
igerber force-pushed the feat/v4-pr4c-i-twfe-es-decoupling branch from b75f9dd to d9cba7f Compare October 25, 2026 11:53
@github-actions

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: d9cba7f90763c7ca1a7407ea0085b6b80daacf89


Overall assessment: ✅ Looks good

No unmitigated P0 or P1 findings identified in the supplied diff.

Executive summary

  • The payload refactor preserves estimator arithmetic, inference calls, covariance indexing, and per-period degrees of freedom.
  • Calendar partitions, reference rows, and survey metadata remain correctly propagated.
  • The committed oracle covers eleven lanes, including rank deficiency, string labels, survey inference, and non-suffix partitions.
  • The deferred HC2-BM parity test is properly tracked.
  • Static checks passed; runtime tests could not start because this environment lacks pytest and numerical dependencies.

Methodology

No findings. The affected methods are TWFE event studies and MultiPeriodDiD. Comparison with docs/methodology/REGISTRY.md, cited docstrings, and available paper-review material found no introduced methodological mismatch.

The refactor retains safe_inference() and distinguishes per-period degrees of freedom from the post-average contrast df (diff_diff/estimators.py:L2282-L2349). The shared builder preserves that distinction and extracts the stored covariance submatrix without recomputation (diff_diff/results_base.py:L1267-L1308).

Code Quality

No findings. All 34 payload fields are supplied by the core, and both production callers consume the new return contract. Sorted labels remain aligned with estimates and covariance indices.

Performance

No findings. Added per-period arrays require linear storage in the number of periods; the TWFE path removes intermediate legacy-container construction.

Maintainability

No findings. Both adapters use one calendar-surface builder, reducing duplicated conversion logic while preserving the leaf-module dependency structure.

Tech Debt

P3 — Informational: tracked HC2-BM parity coverage (TODO.md:L89).

Impact: The one-way absorbed-FE HC2-BM R-parity successor remains follow-up work; existing MPD coverage remains present in this PR.

Concrete fix: No action required for approval. Follow the tracked task to verify singleton-cluster equivalence and add the TWFE R-parity lane.

Security

No findings in the changed code. No apparent credentials or sensitive data were introduced.

Documentation/Tests

No actionable findings. A changelog fragment is present, and changelog_compile.py check passed. All 25 changed Python files passed Python 3.9 syntax parsing. The oracle JSON parsed successfully, and its undefined-SE rows consistently contain NaN t-statistics, p-values, and confidence bounds.

Runtime regression results were not independently verified: the attempted pytest invocation failed with No module named pytest; NumPy, SciPy, and pandas are also unavailable.

@igerber igerber added the ready-for-ci Triggers CI test workflows label Oct 25, 2026
CI on every Linux and Windows leg failed the two committed-oracle tests on
the rank-deficient pooled_nan_post_period lane: the near-zero period-0
effect (att -3.8e-3, se 0.29, t -0.013) drifted 1.6e-13 (OpenBLAS) and
2.4e-13 (Windows) in t_stat against the macOS Accelerate capture while att
and se each passed at atol 1e-13. A ratio inherits atol / se in absolute
terms, so a flat atol on t_stat is tighter than the inputs' own contract
whenever se < 1.

Replace the flat tolerance with the first-order propagated bound
|d t| <= atol (1 + |t|) / se + 2 rtol |t| computed from the stored att/se
(admits the observed CI values with 2x margin; rejects 1e-12 on the small
row and 1e-10 on the t = 10 rows). Fixture README notes the rule.
@github-actions

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: aee1cc40d68c7174aacf9f0b926330ae2ca57bae


Overall assessment: ✅ Looks good

No unmitigated P0 or P1 findings identified. The prior review’s assessment remains supported.

Executive summary

  • The refactor preserves estimator arithmetic, covariance extraction, and inference.
  • Calendar labels, reference rows, partitions, and survey degrees of freedom remain aligned.
  • The revised oracle t-statistic tolerance follows error propagation from ATT/SE tolerances and preserves NaN checks.
  • Static checks passed; runtime regression tests remain unverified because dependencies are unavailable.

Methodology

No findings. Affected methods are TWFE event studies and legacy MultiPeriodDiD. Cross-checking docs/methodology/REGISTRY.md, cited docstrings, and available paper-review material found no introduced methodology mismatch.

Per-period inference retains safe_inference() and coefficient-specific Bell–McCaffrey degrees of freedom (diff_diff/estimators.py:L2282-L2308). The shared builder extracts stored covariance entries and preserves per-period df separately from the post-average contrast df (diff_diff/results_base.py:L1267-L1305).

Code Quality

No findings. All 34 payload fields are supplied, both production callers consume the new return contract, and the sorted-label guard protects label/value alignment.

Performance

No findings. The TWFE path removes intermediate legacy-container construction without adding statistical recomputation.

Maintainability

No findings. Both conversion routes share one calendar-surface builder, reducing opportunities for divergence.

Tech Debt

P3 — Informational: HC2-BM parity coverage remains tracked (TODO.md:L89).

Impact: The successor one-way absorbed-FE HC2-BM R-parity test remains follow-up work; existing MPD coverage remains available.

Concrete fix: No action required for approval. Complete the tracked singleton-cluster verification and add the successor parity test.

Security

No findings in the changed code. No apparent credentials or sensitive data were introduced.

Documentation/Tests

No actionable findings. The changelog fragment is present, and changelog_compile.py check passed.

Independent checks passed for Python 3.9 syntax across all 25 changed Python files, payload-field completeness, survey resolver fallbacks, and oracle integrity across eleven lanes and 172 rows—including undefined-SE inference consistency.

Runtime tests could not start: pytest, NumPy, SciPy, and pandas are unavailable in this environment. Numerical regression claims were therefore not independently reproduced.

@igerber
igerber merged commit d45d531 into main Oct 25, 2026
35 checks passed
@igerber
igerber deleted the feat/v4-pr4c-i-twfe-es-decoupling branch October 25, 2026 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-ci Triggers CI test workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant