Audit and tighten /diagnose-issue slash command#895
Merged
Conversation
Based on recent issue runs (#877, #879, #884, #885, #886, #889, #890): - Add Step 0 pre-triage with three short-circuit checks: (a) bug claim vs informational question, (b) compare reporter's PE version to local + CHANGELOG grep, (c) gh search for existing PE-US tracking issue/PR. - Reframe Critical Rule #1 from "PE / TAXSIM / TaxAct" to "current PE / reporter's PE / TaxAct" — matches what the bundle actually contains. - Add Critical Rule #4: verify against primary sources, not search summaries (search results on state tax law are frequently stale or wrong). - Expand Step 7: when PE and TaxAct disagree, fetch all three — statute text, current-year form PDF, instructions booklet — and cross-reference before concluding. - Document mstat=1 + depx>=1 -> HoH inference in Step 2. - Step 5 PDF snippet iterates all bundle PDFs instead of hardcoded form.pdf. - Step 9 drops stale issue_analysis tracker; adds cross-link step. - Remove the always-404 <issue>.yaml from the batch download. - Fix v37/v38/v39 variable-table placeholders that lost their state-substitution braces. - Fix self-referencing example link in Common Root Causes. Tested on issue #886 (WV CDCC): new Step 0c surfaced PR #3019 quickly; new Step 7 forced fetching the 2025 IT-140 booklet (page 7 recap + page 17 description), confirming PE correctly implements W.Va. Code §11-21-26. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two adjustments based on testing on issue #883: - Critical Rule #1: comparison is now "current PE vs TaxAct only". Reporter's PE version is still useful for Step 0b triage (version comparison + changelog grep) but never for the actual diagnosis comparison. Avoids confusion about which PE values to use. - Step 4: comparison table now has just two PE columns and an explicit rule that every PE value MUST come from a direct query (Step 3 CSV or `Simulation.calculate(...)`). Never infer a PE value from a gap between other variables. - Debugging checklist updated to match. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Based on running the /diagnose-issue workflow on ~10 recent issues (#877–#890), this PR tightens the skill to:
output.txtis useful for triage in Step 0b but never for the actual diagnosis comparison).Simulation.calculate(...)or the CSV output; never inferred from gaps between variables. (Lesson from ME 2025 HoH 259Kpensions 30Kgssi 30Kproptax 20Kmortgage 2depx #883 where guessing led to a vague write-up.)mstat=1+depx≥1→ HoH inference in Step 2.<issue>.yamlfrom the batch download, theissue_analysis/tracker step we never use, the broken self-referencing example URL.{state}braces, plus the misleadingstate_income_taxPE-variable name.Test plan
Validated on six issues during the audit:
🤖 Generated with Claude Code