Skip to content

Tighten /diagnose-issue: input parity in Step 6, verify reporter's claims in Step 1#897

Merged
PavelMakarchuk merged 2 commits into
mainfrom
audit/diagnose-issue-skill-2
May 12, 2026
Merged

Tighten /diagnose-issue: input parity in Step 6, verify reporter's claims in Step 1#897
PavelMakarchuk merged 2 commits into
mainfrom
audit/diagnose-issue-skill-2

Conversation

@PavelMakarchuk
Copy link
Copy Markdown
Collaborator

Summary

Two skill refinements based on running the workflow on taxsim issues #882 and #878:

  1. Step 6 — input parity is critical (commit 37dbc1f). When dropping into a direct Simulation to inspect intermediates, every non-zero TAXSIM input from txpydata.csv must be mapped to its PE-US variable. Forgetting one (especially tax_unit_childcare_expenses from childcare, which lives on the tax_unit not the person) produces different intermediates than the emulator and you mis-attribute a real bug to a "framework difference." Added a cross-walk table for the easy-to-miss mappings, plus a "verify input parity by running the emulator on the same row" instruction.

  2. Step 1 — treat the reporter's claim as a hypothesis (commit c7688dd). If the reporter cites a specific PE value, confirm it appears in the bundle's output.txt before building a diagnosis around it. Reporters sometimes paste values from a different case; without this check you can construct a wrong narrative around a wrong number.

Test plan

  • Re-ran the workflow on a fresh taxsim issue with the updated skill loaded.

🤖 Generated with Claude Code

PavelMakarchuk and others added 2 commits May 11, 2026 18:48
…Simulation

Lesson from re-examining taxsim #882: forgetting to pass
tax_unit_childcare_expenses in a direct Simulation situation
zeroed the federal CDCC, which shifted tax_liability_if_not_itemizing
by ~$300 and made it look like Microsim and Simulation produced
different answers. They actually agreed — I was comparing apples to
oranges because the inputs weren't identical.

Step 6 now mandates a TAXSIM-to-PE variable cross-walk before running
a direct Simulation, with a table of the easy-to-miss mappings
(childcare → tax_unit_childcare_expenses, proptax → real_estate_taxes,
mortgage → deductible_mortgage_interest, rentpaid → rent).
Debugging checklist updated to match.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… against output.txt

If the reporter cites a specific PE value, confirm it appears in the
bundle's output.txt before building a diagnosis around it. Reporters
sometimes paste values from a different case; without this check
you can construct a wrong narrative around a wrong number.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vercel
Copy link
Copy Markdown

vercel Bot commented May 12, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
policyengine-taxsim Ready Ready Preview, Comment May 12, 2026 6:49am

Request Review

@PavelMakarchuk PavelMakarchuk merged commit 8a99769 into main May 12, 2026
9 of 12 checks passed
@PavelMakarchuk PavelMakarchuk deleted the audit/diagnose-issue-skill-2 branch May 12, 2026 07:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant