Milestones 8 and 9: normalize, fingerprint, and deparse — every corpus suite at 100%

[kyleconroy]

Lands milestones 8 (Normalize + Fingerprint) and 9 (Deparse) from PLAN.md. With these, every suite's todo list is empty: all 217,229 corpus cases across the eight suites pass, and go test -race ./... is green. The remaining not-implemented entry points are Summary (milestone 10) and ParsePlPgSqlToJSON (milestone 11).

Milestone 8a: Fingerprint (internal/fingerprint)

Port of pg_query_fingerprint.c + its generated defs/conds at the pin. Like the milestone-4 JSON emitter, the per-node walk is protobuf-descriptor-driven rather than generated per-node code — the upstream defs are generated from the same struct metadata the pinned proto came from, so the descriptor already carries the field set, kinds, and (after sorting) the alphabetical field order. The hand-written layer mirrors the upstream generator's special-case tables: skip-nodes/skip-fields, the six sorted-dedup list field names with the listsort cache (the exponential-blowup guard the 1.1 MB stress insert needs), AEXPR_OP_ANY/AEXPR_IN folding, RangeVar digit-stripping and temp-table elision, TypeCast-of-constant elision, the ResTarget.name-under-SELECT-targetList rule, and the pre-PG15 legacy value-node field names.

The C's per-field XXH3 state snapshot/restore reduces to "elide the field name if the subtree appended zero bytes", so tokens stream into a byte buffer and internal/xxh3 stays one-shot.

Gate met: all 43,385 fingerprint todos graduated, including the 1.1 MB stress case.

Milestone 8b: Normalize (internal/normalize)

Port of pg_query_normalize.c: const_record_walker with its hand-written cases (DefElem string arguments located by text scan, subscription conninfo matching, the SELECT GROUP BY-to-target-list fingerprint matching that reuses parameter numbers, the utility-only gates), backed by a port of PostgreSQL 17's raw_expression_tree_walker whose unsupported node types stop the subtree walk the way the swallowed elog does upstream. Constant lengths come from the milestone-3 lexer.

One bug-compatibility find: the C's location sort is an unstable pg_qsort, and for duplicate constant locations (a MultiAssignRef source walked once per target column) the parameter number depends on which duplicate the Bentley–McIlroy partition leaves first — UPDATE t SET (c,b,a) = ('x', b+1, DEFAULT) WHERE c = 'y' normalizes to SET (c,b,a) = ($1, b+$4, DEFAULT) WHERE c = $7. pg_qsort is therefore ported too (internal/normalize/qsort.go).

Gate met: all 43,254 normalize and 25 normalize_utility todos graduated.

Milestone 9: Deparse (internal/deparse)

Port of all 293 functions of postgres_deparse.c (12,107 lines at the pin), targeting the protobuf structs directly. The part/group/nesting-level machinery — built for the pretty-print mode pg_query_go's API never exposes — still shapes plain-mode byte output (parts join with single spaces except after ( or before a part starting with )/;, part groups fold their major keyword into their first part, and the merge pass runs against the default 80-column limit), so it is ported exactly, while the pretty-print and comment paths themselves are omitted as unreachable. quote_identifier/quote_qualified_identifier are ported from ruleutils.c on top of the milestone-3 keyword tables. Deparse errors surface as parser.Error via panic/recover standing in for the C ereport longjmp; no corpus input triggers one.

proto3 cannot represent C's NULL-vs-empty-string distinction; where the deparser branches on it, call sites decided the port (e.g. deparseOptBooleanOrString's NULL guard is dead upstream — every caller passes strVal — so SET x TO '' falls through to print '').

Gate met: all 43,352 deparse todos graduated — byte-equality with the oracle's deparse output corpus-wide, which subsumes the reference's parse → deparse → reparse roundtrip criterion since the goldens are the oracle's own deparse output.

Also in this PR

• Normalize, NormalizeUtility, Fingerprint, FingerprintToUInt64, and Deparse/DeparseFromProtobuf are wired up in the public API and removed from the not-implemented compat test.
• cmd/difftodo now supports all corpus suites, not just parse.
• PLAN.md milestones 8–9 marked landed, with as-built notes.

🤖 Generated with Claude Code

https://claude.ai/code/session_0114AafjcMa5K8ZJJAZdGwzC

---

Generated by Claude Code