Milestones 10 and 11: Summary and PL/pgSQL — every entry point implemented, corpus at 100% - #8
Merged
Merged
Conversation
Extend the oracle protocol with summary (truncation limit riding along as a directive line in summary_truncate case inputs) and parse_plpgsql ops, extract libpg_query's summary test calls and the plpgsql_regress + plpgsql_samples tiers, and wire the three new suites through cmd/regenerate, the corpus harness, and difftodo. - postgres_regress runs summary untruncated plus summary_truncate at limit 100; inline summary calls keep their upstream limits and also run at limit 50. - plpgsql_regress feeds the standard SQL suites too; all of those cases pass the existing implementation and are harvested immediately. - Goldens verified byte-identical across two regeneration runs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5Ummi5YEg7pxk5fvRUd3U
Port pg_query_summary.c, pg_query_summary_statement_type.c, and pg_query_summary_truncate.c into internal/summary: - A faithful raw_expression_tree_walker (nodeFuncs.c, PostgreSQL 17) over the protobuf tree, gated exactly as pg_query_raw_tree_walker_supports allows; list nodes are walked as nodes so the truncation pass's depth accounting matches. - The summary walk (tables with contexts, aliases, CTE names, functions, filter columns) including its quirks: WHERE clauses walked twice, the filter-column pass aborting at sub-SELECTs, and functions in WHERE clauses recorded once per pass. - The statement-type walk (insertion-ordered set of node type names). - Truncation: possible truncations collected with depth/length priority, sorted with the pg_qsort port (list_sort maps to pg_qsort at the pin, so tie order needs the exact algorithm), applied cumulatively against the milestone-9 deparser, with the multibyte-aware fallback chop. pg_query_go's summary_test.go is ported verbatim (import swap only). Gate: all 43,992 summary and 43,345 summary_truncate corpus cases pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5Ummi5YEg7pxk5fvRUd3U
…ToJSON Port the PL/pgSQL support libpg_query builds at the pin: - internal/plpgsql: pl_gram.y as recursive descent (statement linenos computed at bison's reduce points, since plpgsql_latest_lineno is stateful and feeds the compile error context), the pl_scanner.c token layer over the core lexer (PL reserved keywords re-classified from the SQL scanner's output, pushback stack, A.B.C composition), the pl_comp.c/pl_funcs.c subset (namespace chain, datums, recfields, exception conditions), pg_query_json_plpgsql.c's dump, and the pg_query_parse_plpgsql.c driver — including libpg_query's mocks: parse_datatype keeps the type text verbatim, %TYPE/%ROWTYPE lookups return NULL, make_return_stmt drops return-type checks, and function_parse_error_transpose always fails (errors carry the "near line N" context, no cursor). - internal/parse: ParseWithMode with the RAW_PARSE_PLPGSQL_EXPR and ASSIGN1..3 grammar modes (PLpgSQL_Expr / PLAssignStmt productions), which check_sql_expr uses to validate embedded SQL. - internal/rawwalk: the raw_expression_tree_walker port moves out of internal/summary so the driver's statement collection can share it. - cmd/generate -plpgsql emits internal/plpgsql/tables.go from the newly vendored pl_reserved_kwlist.h / pl_unreserved_kwlist.h / plerrcodes.h (internal/reference gains those plus patched pl_gram.y, pl_scanner.c). Gate: all 690 plpgsql corpus cases (plpgsql_regress + plpgsql_samples + per-statement tiers) pass, as does pg_query_go's TestParsePlPgSQL; the corpus todo list is now empty across all eleven suites — 308,561 cases. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5Ummi5YEg7pxk5fvRUd3U
PLAN.md gains the milestones 10–11 as-built notes (corpus tiers, the libpg_query mocks the PL/pgSQL port follows, the pg_qsort tie-order dependency in truncation, the raw-parse modes); README's status table catches up with milestones 3–11; CLAUDE.md and internal/reference/README cover the new generated file and vendored PL reference sources. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5Ummi5YEg7pxk5fvRUd3U
The suite order predates milestones 10-11; with it stale, a future pin advance would put todos back on those suites that next-test silently skips. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01C5Ummi5YEg7pxk5fvRUd3U
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lands the last two entry points of the pg_query_go API surface. Every public function is now implemented, and the corpus todo list is empty across all eleven suites: 2,067 golden files / 308,561 cases, all passing (including under
-race).Corpus first (7b53c48)
The milestone-2 deferrals land as their own commit, so the acceptance gates exist before the ports:
summary— postgres_regress + plpgsql_regress untruncated, plus libpg_query's summary test-call inputs. Upstream's summary tests are call sites rather thantests[]tables, socmd/regenerategrew a mechanical call extractor (summary("...", 0, limit), C line splices removed); inputs are transcribed mechanically, expectations always from the oracle.summary_truncate— the same regress statements at limit 100, the upstream truncate tests at their exact limits, and every summary input at limit 50. The limit rides along as a-- truncate_limit: Ndirective line in the case input, stripped identically by regenerate and the harness.plpgsql—plpgsql_regress/+plpgsql_samples.sql, oneParsePlPgSqlToJSONgolden per statement. The plpgsql_regress files also feed the five standard SQL suites; all of those passed the existing implementation on arrival.Goldens verified byte-identical across regeneration runs, before and after the ports.
Milestone 10: Summary (1d0bb54)
internal/summaryportspg_query_summary.c,..._statement_type.c, and..._truncate.cover a faithfulraw_expression_tree_walker(PostgreSQL 17nodeFuncs.c, gated bypg_query_raw_tree_walker_supports). The quirks the goldens encode are preserved deliberately: WHERE clauses are walked twice (functions there recorded once per pass), the filter-column pass aborts at sub-SELECTs,MERGE ... USINGsources are never tables, and list nodes count as depth levels in the truncation walk. Truncation tie-ordering needed the exactpg_qsortalgorithm again —list_sortmaps to it at the pin, so equal (depth, length) pairs are ordered by the partition scheme, not the comparator. pg_query_go'ssummary_test.gois ported verbatim (import swap only).Milestone 11: PL/pgSQL (eca5f82)
internal/plpgsqlports what libpg_query builds, not vanilla PostgreSQL —extract_source.rb's mocks are the authority:parse_datatypekeeps the type text verbatim (no catalog round trip),%TYPE/%ROWTYPElookups return NULL,make_return_stmtdrops the return-type checks, andfunction_parse_error_transposealways fails, so compile errors carry thecompilation of PL/pgSQL function "f" near line Ncontext and no cursor.A.B.Cdatum composition, and the statement-start/unreserved-keyword logic.pl_gram.ybecame recursive descent with statement linenos computed at bison's exact reduce points —plpgsql_latest_linenois stateful and feeds the error context, so call order is part of conformance.internal/parsegainsParseWithModewith theRAW_PARSE_PLPGSQL_EXPRandASSIGN1..3grammar modes (PLpgSQL_Expr/PLAssignStmtproductions) forcheck_sql_expr's embedded-SQL validation.internal/rawwalk;cmd/generate -plpgsqlemits the keyword/errcode tables from newly vendoredpl_reserved_kwlist.h/pl_unreserved_kwlist.h/plerrcodes.h.All 690 plpgsql cases passed on the port's first corpus run, as does the ported
TestParsePlPgSQL.Docs and tooling (3d30257, 4014bd8)
PLAN.md as-built notes for milestones 10–11, README status table caught up with milestones 3–11, CLAUDE.md's generated-file list, provenance for the vendored PL reference sources, CI's generate-clean job verifying
-plpgsqloutput, andcmd/next-testcovering the three new suites.Verification
go test ./...andgo test -race ./...fully green; corpus at 0 todos.🤖 Generated with Claude Code
https://claude.ai/code/session_01C5Ummi5YEg7pxk5fvRUd3U
Generated by Claude Code