Milestone 4: expressions + SELECT — parser, JSON emitter, ParseToJSON

[kyleconroy]

Implements milestone 4 of PLAN.md: the a_expr/b_expr/c_expr precedence machinery and SELECT end-to-end, plus the JSON emitter that makes ParseToJSON (and therefore the parse corpus) verifiable.

JSON emitter (internal/emit)

Byte-parity with the pinned pg_query_outfuncs_json.c + pg_query_json_helper.c, driven by the protobuf descriptor rather than generated per-node code — the pinned proto was itself generated from the C struct metadata, so field declaration order (= C struct order), json_name (= C field name), and field kinds already encode the format. The reference's hand-written special cases are ported directly: A_Const's union wrapping, the five value nodes, the List variants, _outToken's exact escape set (</>, control bytes, NUL truncation), and the two always-emitted empty-string fields.

Validation: TestGoldenRoundTrip protojson-decodes all 42,970 corpus tree goldens and re-emits every one byte-identically — this covers every node type in the corpus, including statement types the parser won't build until milestones 5–7. Generated per-node emitters remain an option for milestone 12 if profiling wants them.

Parser (internal/parse)

Hand-written recursive descent over the pinned gram.y's expression and SELECT regions, one parse* method per production with attribution comments; gram.y's support functions live in gram_support.go under their C names.

• Precedence climbing mirrors bison's conflict resolution: left-assoc operators parse their right operand one level up, %nonassoc levels error when chained within one climb, %prec annotations are reproduced at their annotated sites. a_expr subquery_Op sub_type enters the loop at the operator token's own precedence but reduces at %prec Op (so 1 = 2 = ANY(...) fails at the second = while = ANY(...) + 1 binds the + outside). Qual-requiring JOINs absorb further joins into their right operand (bison shifts because the rule can't reduce until its join_qual); CROSS/NATURAL joins stay left-associative.
• Expressions: constants and const typecasts (AexprConst incl. func_name Sconst/PARAM and ConstTypename/ConstInterval forms), the full Typename families, indirection/slices, CASE, rows and OVERLAPS, arrays, sublinks, IS forms incl. SQL/JSON predicates.
• Functions: func_application (named args, VARIADIC, DISTINCT, agg star), WITHIN GROUP/FILTER/OVER, window specifications and frames, and the func_expr_common_subexpr set — CAST/EXTRACT/OVERLAY/POSITION/SUBSTRING/TRIM/TREAT/NULLIF/COALESCE/GREATEST/LEAST, SQL value functions, xml*, SQL/JSON constructors, query functions and aggregates, MERGE_ACTION.
• SELECT end-to-end: target list with bare-label aliases, FROM with joins/LATERAL/function tables/ROWS FROM/TABLESAMPLE/XMLTABLE/JSON_TABLE (incl. NESTED paths), DISTINCT [ON], GROUP BY incl. grouping sets, HAVING, WINDOW, set operations, CTEs (MATERIALIZED, SEARCH/CYCLE), VALUES, ORDER BY, LIMIT/OFFSET/FETCH (WITH TIES), FOR locking clauses, SELECT INTO.
• Errors: grammar errors report through the C stack's path (base_yyerror → parser_yyerror → scanner_yyerror), so messages carry scan.l error data, character-based cursor positions, and merged tokens (NOT_LA etc.) report only their first word; gram.y action ereports keep their original funcname and positions.
• LALR keeps sub-select and paren-expression paths alive simultaneously; the port uses bounded backtracking (token mark/reset) in exactly four places, each pre-gated by cheap lookahead.

Wiring and corpus

• Parse, ParseToProtobuf, and ParseToJSON are live; pg_query_go's ported parse tests now run for the implemented statement types.
• 16,300+ parse cases harvested off the todo list (sidecar diffs committed by the harness). Every remaining parse todo starts with — or embeds, e.g. DML inside CTEs — a milestone-5–7 statement type; no pure-SELECT case is left failing.
• cmd/difftest ships the interim -summary/-show failure classifier that drove the milestone (mutation fuzzing replaces it in milestone 12).
• PLAN.md gains "As-built notes (milestone 4)".

go build ./... and go test -race ./... are green; the main module remains cgo-free with google.golang.org/protobuf as its only dependency.

🤖 Generated with Claude Code

https://claude.ai/code/session_01543SdbotP1DX2XRMvfCHn5

---

Generated by Claude Code