Support INSERT ... VALUES in the polyglot SQL dialect by alexey-milovidov · Pull Request #110321 · ClickHouse/ClickHouse

alexey-milovidov · 2026-07-13T21:39:09Z

Enable INSERT ... VALUES with inline data in the polyglot SQL dialect (dialect = 'polyglot').

Previously, running e.g. INSERT INTO t VALUES (1), (2), (3) with dialect = 'polyglot' failed with Multi-statement queries are not supported in polyglot dialect mode. The underlying problem is that transpiling inside the parser cannot deliver inline data to the executor: the transpiled buffer is transient, and the executor overwrites ASTInsertQuery::tail with the external input stream, so the inline-data pointers (data/end) must reference a live query buffer.

This transpiles the query up front instead of inside the parser:

The server (executeQuery) transpiles a foreign-dialect query to ClickHouse SQL before parsing, keeps the transpiled text alive on the query context, and parses it with the standard parser. Inline INSERT data then points into a live buffer and is processed by the normal machinery. SET queries are still parsed as-is so dialect/polyglot_dialect can always be changed back.
The client (clickhouse-client/clickhouse-local) parses a non-ClickHouse-dialect query into an AST — which, for a foreign dialect, means transpiling it locally only to drive client-side handling (statement classification, output format, INSERT detection) — but then sends the original query text verbatim, without splitting off inline data. The server performs the authoritative transpilation whose result is actually executed, so inline INSERT data lives in a server-owned buffer and survives parsing. The client-side transpilation is throwaway; note this means the transpiler must also be available on the client (a client built without USE_POLYGLOT fails locally with SUPPORT_IS_DISABLED), and the client and server transpilers are assumed to agree — acceptable for this experimental dialect. Every parse-time setting the query was parsed under (dialect, allow_experimental_polyglot_dialect, polyglot_dialect, allow_settings_after_format_in_insert, implicit_select, and the parse limits max_query_size, max_parser_depth, max_parser_backtracks) is pinned in the per-query settings sent along with the verbatim text, so the query's own SETTINGS clause cannot change how the server reparses that same text (it still applies to the query's execution, and a SET still takes effect for subsequent queries).

All changes are gated on the dialect, so ordinary ClickHouse INSERTs are unaffected. Validated over the HTTP interface, the native client, and clickhouse-local (multi-row and single-row VALUES, INSERT ... SELECT, and PostgreSQL literal transpilation such as true/false); SET passthrough and multi-statement rejection are preserved. External insert data combined with a foreign-dialect INSERT is rejected with NOT_IMPLEMENTED instead of being silently dropped, on both surfaces: the client rejects piped stdin and INFILE (it sends the query verbatim and cannot forward a data tail), and the server rejects a non-empty HTTP request body appended to a streaming INSERT (POST /?query=INSERT ... &dialect=polyglot with a body). A foreign-dialect INSERT is transpiled as a whole, so the body would go through neither the transpiler nor the max_query_size guard, mixing two parsing rules in one INSERT. An empty body still works, which is the normal way to run a polyglot INSERT over HTTP.

Limitations (scoped, experimental): because a foreign-dialect query is transpiled as a whole (the transpiler rewrites the inline data too and cannot know where the SQL header ends without parsing the dialect), the inline INSERT ... VALUES data counts towards max_query_size — unlike a native ClickHouse INSERT, whose inline data is streamed and is not bounded by max_query_size. An oversized payload fails-close with a dedicated, actionable error rather than silently changing the INSERT size contract; increase max_query_size to submit larger inline payloads.

Only INSERT ... VALUES inline data is transpilable by the bundled dialects. INSERT ... FORMAT ... is not: FORMAT is a ClickHouse-only extension, so a foreign-dialect parser rejects the query at the inline data that follows (empirically, postgresql/mysql/sqlite/duckdb/snowflake/bigquery all fail at the first data row after FORMAT; a hypothetical identity transpiler even drops the raw FORMAT payload rather than re-emitting it). A foreign-dialect INSERT ... FORMAT therefore fails cleanly with a syntax error and inserts nothing — like EXPLAIN INSERT ... VALUES, which is also not transpilable by the bundled dialects (rejected at the VALUES token). The server-owned transpiled buffer that carries the inline data is itself format-agnostic and would handle FORMAT data if a transpiler ever produced such a query; the parser also defensively clears the inline-data pointers of an EXPLAIN-wrapped INSERT — the same way the client unwraps it — so both forms are safe if a future transpiler supports them.

Changelog category (leave one):

Experimental Feature

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Support INSERT ... VALUES with inline data when using the experimental polyglot SQL dialect.

Documentation entry for user-facing changes

Documentation is written (mandatory for new features)

Workflow [PR]
Sync PR [sync-upstream/pr/110321]

Inline INSERT data (`INSERT ... VALUES (...)` / `... FORMAT ...`) could not be used with `dialect = 'polyglot'`: it failed with "Multi-statement queries are not supported". The underlying problem is that transpiling inside the parser cannot deliver inline data to the executor - the transpiled buffer is transient, and the executor overwrites `ASTInsertQuery::tail` with the external input stream, so the data pointers (`data`/`end`) must reference a live query buffer. Transpile the query up front instead of inside the parser: - The server (`executeQuery`) transpiles a foreign-dialect query to ClickHouse SQL before parsing, keeps the transpiled text alive on the query context, and parses it with the standard parser. Inline INSERT data then points into a live buffer and is processed by the normal machinery. SET queries are still parsed as-is so `dialect`/`polyglot_dialect` can always be changed back. - The client (`clickhouse-client`/`clickhouse-local`) sends a query written in a non-ClickHouse dialect verbatim (without splitting off inline data) and lets the server transpile and read it, so transpilation happens exactly once, server-side. All changes are gated on the dialect, so ordinary ClickHouse INSERTs are unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

clickhouse-gh · 2026-07-13T21:39:56Z

Workflow [PR], commit [2f089c5]

Summary: ✅

Performance Comparison: Performance dashboard

AI Review

Summary

PR #110321 enables inline INSERT ... VALUES in the experimental polyglot dialect by retaining the transpiled SQL on the query context and sending the original client query for server-side transpilation. Review of commit 2f089c5e29f5955b5287ff73ab078c7d620d7b40, the regression coverage, and all prior discussion found no remaining actionable issues or new inline findings.

Missing context / blind spots

⚠️ Runtime verification is incomplete: the completed Fast test skipped all nine polyglot tests, and the Rust-enabled stateless jobs were still pending in the CI report. Successful execution of the focused regression tests in a build with USE_POLYGLOT would close this validation gap. No local build or tests were run.

Final Verdict

Status: ✅ Approve. No code changes requested; runtime CI validation remains pending.

…cted cleanly Extend `04512_polyglot_insert_values` to cover the case where a second statement follows inline `INSERT ... VALUES` data in the polyglot dialect (e.g. `INSERT INTO t VALUES (1); SELECT 2`). The whole remaining buffer is transpiled at once and the transpiler rejects the multi-statement input, so the query fails with a clean `SYNTAX_ERROR` rather than being silently mis-executed or reaching the server as unread `Values` tail. The test also asserts that no partial insert happens on the rejected path. This documents the behavior flagged in the PR review, which was verified not to reproduce as a silent regression.

Address review: on the verbatim polyglot path the client sends the query text as-is and never forwards external data, so piped stdin rows were silently dropped: \`printf '(2)\n' | clickhouse-client --dialect polyglot ... -q 'INSERT INTO t VALUES (1)'\` inserted only the inline row while in the \`clickhouse\` dialect both rows are inserted. Now a foreign-dialect \`INSERT\` (without \`SELECT\`) with data on stdin or \`INFILE\` fails with \`NOT_IMPLEMENTED\` before anything is sent, matching the existing checks for \`async_insert\` and inline-insert-data modes, which throw the same way instead of losing data. Add a test case asserting the error and that no partial insert happens. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

alexey-milovidov · 2026-07-15T02:46:16Z

The Stress test (arm_release) failure — Logical error: 'Block structure mismatch in JoinStep: [__table3.number, __table3.number] and [__table3.number] stream: different number of columns' (STID: 2228-453d) — is unrelated to this PR, which only touches dialect transpilation and the client insert path. The same STID has failed on many unrelated PRs over the last 30 days (e.g. #96225, #107650, #107567, #108096).

Tracked in #109215, and a fix is already in progress: #109114.

…t dialect `executeQueryImpl` reassigned `begin`/`end` to point at the transpiled ClickHouse SQL when parsing a `polyglot`-dialect query, and those same local variables were then used further down to build `query`/`query_for_logging`/`normalized_query_hash`. As a result, `system.query_log`/processlist showed the transpiled SQL instead of what the user actually submitted, and per-`normalized_query_hash` quotas grouped by the transpiled form. Additionally, when an `ASTInsertQuery::data` pointer (into the transpiled buffer) was used to cut the logged text short, it was sliced against the original (untranspiled) `begin`, mixing pointers from two unrelated buffers. Keep the transpile-and-reparse step local to the polyglot branch instead of reassigning the outer `begin`/`end`, and only use `insert_query->data` to shorten the logged query when it actually falls within `[begin, end)`. Added a regression test asserting `system.query_log.query` for a polyglot `INSERT` matches the original text. Addresses review feedback on #110321

alexey-milovidov · 2026-07-16T05:24:35Z

The Stateless tests (amd_asan_ubsan, distributed plan, parallel) failure (hundreds of unrelated tests failing with Code: 241. DB::Exception: (total) memory limit exceeded, e.g. 00033_aggregate_key_string, 01079_bad_alters_zookeeper_long, 02835_drop_user_during_session) is unrelated to this PR. This job is currently flaky/red on master itself (confirmed on several recent MasterCI runs, e.g. run 29466889572) due to the sanitizer memory-ratio change in #110293 backfiring under --distributed-plan's per-query memory multiplication. A fix is already in progress: #110574 (re-applies the 0.7 ratio and cuts this job's concurrency). No action needed here; this will clear once #110574 merges into master and this branch picks it up.

The style check requires `SYSTEM FLUSH LOGS log_name` instead of the global `SYSTEM FLUSH LOGS`, and the test only reads `system.query_log`.

For a `polyglot`-dialect inline `INSERT`, the parsed `ASTInsertQuery::data` pointer references the transpiled buffer owned by the query context, not the original query text `[begin, end)`. The previous logging truncation only cut the query short when `data` fell inside `[begin, end)`, so for polyglot inline `INSERT`s it never fired and the full `VALUES`/`FORMAT` payload was written to `system.query_log`, the process list, and `normalized_query_hash`, breaking the "`INSERT` logs omit inserted data" contract and potentially leaking row values. The inline-data boundary cannot be mapped back onto the original text because transpilation rewrites the query, so log the transpiled header up to the data instead: it carries the `INSERT` target and column list but no row values, and reflects what was actually executed. Non-`INSERT` polyglot queries keep logging the original text as before. Updated `04512_polyglot_insert_values` to assert the inline data is absent from `system.query_log` instead of asserting the (leaky) full original text. Addresses review feedback on #110321 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

In the multi-query path, `ClientBase` computed `insert->data - query_to_execute.data()` unconditionally for every `ASTInsertQuery`. For a foreign-dialect (`polyglot`) `INSERT`, the polyglot parser clears `insert->data` (the query is sent verbatim and the server reads the data from its own transpiled buffer), so this became `nullptr - ptr`, which is undefined behavior. The computed length was already unused on the verbatim path (guarded by a later `insert && insert->data && !send_query_verbatim` check), so guard the subtraction on `insert->data` being non-null. Addresses review feedback on #110321 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Address review (clickhouse-gh AI verdict): the PR contract claimed "transpilation happens exactly once, on the server", but the native client (`clickhouse-client`/`clickhouse-local`) is fundamentally AST-driven, so for a foreign dialect `ClientBase::parseQuery` transpiles the query locally via `ParserPolyglotQuery` to obtain an AST for client-side handling (statement classification, output format, INSERT detection). That transpiled text is thrown away; the client sends the *original* query verbatim and the server performs the authoritative transpilation whose result is actually executed. No behavior change: this only makes the comments (and the PR description) accurate about the two-stage transpilation and its accepted limitations for this experimental dialect — the client requires the transpiler to be built in (`USE_POLYGLOT`, else `SUPPORT_IS_DISABLED`), and the client and server transpilers are assumed to agree. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Address review: `ParserPolyglotQuery::parseImpl` only cleared the dangling `data`/`end` pointers of a top-level `ASTInsertQuery`. The client unwraps an `ASTExplainQuery` and dereferences the nested `ASTInsertQuery::data` the same way when it locates the inline-data boundary (`ClientBase::analyzeMultiQueryText`), so an `EXPLAIN INSERT ... VALUES` in polyglot mode would leave the nested insert's `data`/`end` pointing into the transient `transpiled` string that is freed on return - a use-after-free in `-n`/script mode. Introduce a `findInlineDataInsert` helper that unwraps a single `EXPLAIN` layer (mirroring the client) and clear the pointers of the explained `INSERT` too. This is a defensive correctness fix: no bundled transpiler dialect currently transpiles `EXPLAIN INSERT ... VALUES` (they reject it at the `VALUES` token), so the path is not yet reachable, but the fix future-proofs it against a transpiler that supports the form. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Address review: a foreign-dialect `INSERT ... VALUES`/`FORMAT` is transpiled as a whole, so its inline data counts towards `max_query_size` - unlike a native ClickHouse `INSERT`, whose inline data is streamed and is not bounded by `max_query_size`. This explicitly scopes the experimental feature to inline payloads that fit the parser size limit. The oversized case now fails-close with a dedicated, actionable error (added in the previous commit in `transpilePolyglotToClickHouse`) instead of silently changing the `INSERT` size contract; document the limitation on the `allow_experimental_polyglot_dialect` setting so users can discover it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Extend `04512_polyglot_insert_values.sh`: - `EXPLAIN SELECT` transpiles and runs (exercises the new `EXPLAIN`-unwrapping helper, which returns no inline-data `INSERT` for a non-insert). - `EXPLAIN INSERT ... VALUES` is rejected cleanly by the transpiler (no bundled dialect transpiles it) with no partial insert and no use-after-free. - An inline `INSERT` payload larger than `max_query_size` is rejected with the dedicated error and inserts nothing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

… mode The client parses scripts with allow_multi_statements enabled, which zeroes the generic per-query length limit, so `ParserPolyglotQuery` was constructed with `max_query_size = 0` and the oversized-query guard never ran on the client: an oversized polyglot inline `INSERT` in `--multiquery` mode was fully transpiled locally before the server rejected it. Construct the polyglot classifier with the real `max_query_size` (it always consumes the whole remaining buffer as a single query sent verbatim, so the per-query limit applies in every mode), and run the transpile - whose size guard rejects oversized input up front - before touching the token stream. Also stop the token-advance loop at `ErrorMaxQuerySizeExceeded`: the lexer emits it on every call once a size-capped stream passes its limit (never reaching `EndOfStream`), so iterating to the end could not terminate. Add a --multiquery regression proving the oversized query is rejected on the client, without a server round trip: its query_id must not appear in `system.query_log` (a server-side rejection would be recorded there as `ExceptionBeforeStart`). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

`transpiled_query` was stored as a plain `String` inside `ContextData`, and `Context::createCopy` performs a deep copy of it. Insert paths such as `AsynchronousInsertQueue::preprocessInsertQuery`, `StorageDistributed` distributed writes, and `DistributedSink` clone the query context, so a large polyglot inline `INSERT` payload could be duplicated several extra times in memory. Hold the buffer behind `std::shared_ptr<const String>` so context copies share it instead of cloning it; this also keeps the buffer alive independently of which context outlives which. Addresses the AI review finding on the PR.

alexey-milovidov · 2026-07-18T00:23:11Z

The Stress test (amd_tsan) failure — Logical error: Cannot convert nested result of function JSON_EXISTS with type UInt8 to the expected result type Tuple(UInt8, UInt8) (STID: 4811-66fc), from test 04325_sql_json_functions_with_dynamic_input.sql — is unrelated to this PR, which only touches dialect transpilation and the client insert path. The same STID has failed on many unrelated PRs and on master over the last week (e.g. #100391, #110171, #110633, #110530, #109347).

It is a master-only regression from #106877, reported in #110345, and a fix is already in progress: #109944.

The Stress test (arm_asan_ubsan, s3) hung check is the known issue #107941 (already labeled in the CI report).

…`clickhouse-local` The `input()` initializer of `LocalConnection` reparses the query it received to learn the `input` structure hint and the data format. It has a branch for every dialect but `polyglot`, so a foreign-dialect `INSERT ... SELECT * FROM input(...)` - the one INSERT shape whose data still travels outside the query text, and which the client therefore sends verbatim - was reparsed with the plain ClickHouse parser and rejected as soon as the original text used non-ClickHouse syntax, even though the server transpiles and accepts it. Capture `polyglot_dialect` and `allow_experimental_polyglot_dialect` together with the other parser-affecting settings and use `ParserPolyglotQuery` here, mirroring `ClientBase::parseQuery`. New test `05227_polyglot_local_input_function`: a PostgreSQL cast over `input()`, a BigQuery statement whose original text the ClickHouse lexer cannot even tokenize (`#comment`), and the same text in the plain dialect, which must still be a `SYNTAX_ERROR`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

alexey-milovidov · 2026-09-19T10:49:36Z

🕵 Thanks @groeneai — acknowledging the three follow-ups on the unrelated reds, and reporting what this PR's own push does.

ORDER BY ... WITH FILL around a NULL fill key (the Stress test (amd_debug) red, STID 2508-4384): Fix ORDER BY WITH FILL mis-placing generated rows around a NULL value #117832 is still open, so it is not on master and a master merge cannot clear that signature. Nothing to do here.
The fuzzed prepare_stateful_data wedge: Do not fuzz the stateful fixture load in the flaky check #117997 merged on 2026-09-13, i.e. before this branch's last master merge ebb2392ac21a (merge-base 2026-09-18), so it is already in.
NATS: Update the vendored nats-io client to fix a stalled NATS subscription and a JetStream fetch deadlock #119867 is closed and this branch carries master's pin 3e3a3f10, which by your own measurement does not fix the ordering defect. That is a master-side decision about which gitlink to land; this PR does not touch it.

On this PR's own red — Fast test failing 05019_trino_dialect, 05023_trino_conformance_tempto and 05153_trino_dialect_wrappers_and_set with Unexpected packet from server (expected Data, Exception, Log or TimezoneUpdate, got Progress). That one was mine, not a flake: the client's verbatim path was keyed on "any dialect other than clickhouse/clickhouse_json" and so caught trino, whose parser deliberately keeps ASTInsertQuery::data pointing into the original buffer. Fixed in 38bd4ecb92a7 (verbatim is now polyglot only); all three tests pass locally.

groeneai · 2026-09-19T11:27:45Z

Understood on all three. One thing to add on the gitlink: I am not opening a competing bump, because #120202 already pins b88704df, which is cb4edba9 plus one commit in src/unix/sock.c, so it carries both merged fork fixes.

Measured cost while master stays on 3e3a3f10: 514 red test_storage_nats rows across 59 pull requests in the last 10 days, including 16 jobs that lost 8 or more of the module's tests in a single shard, 6 of them to the Code: 210 connection-refused cascade that follows the server dying.

alexey-milovidov · 2026-09-25T08:51:39Z

🕵 @groeneai thanks. Noted that #120202 carries the NATS gitlink fix (b88704df). This PR doesn't touch that pin, so nothing further is needed here. I replied separately on the open bot thread about lexically foreign URL queries: the concatenating path transpiles URL query and body as one statement, so no code change.

Conflict resolution: - `IParser::consumesForeignText` from this PR is dropped in favor of master's `IParser::consumesRawText`, which covers the same need (skip the lexical shortcut and the unmatched-parentheses check for opaque dialect text) while keeping `ErrorMaxQuerySizeExceeded` fatal. `parseQuery.cpp` takes master's version. - `ParserPolyglotQuery`: keeps this PR's up-front `transpilePolyglotToClickHouse` and inline-data handling, plus master's `SET` / `SET ROLE` / `SET TRANSACTION SNAPSHOT` dispatch. That dispatch is extracted into `parsePolyglotNativeStatement` so that the server-side polyglot branch of `executeQuery` (which transpiles before parsing, and so no longer goes through `ParserPolyglotQuery`) classifies these statements exactly like the client, instead of using a bare `ParserSetQuery`. - `ClientBase`: keeps this PR's `ParserPolyglotQuery` arguments and master's new `logsql` dialect branch. - `Settings.cpp`: keeps this PR's documentation of `allow_experimental_polyglot_dialect` and master's settings-history entry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

alexey-milovidov · 2026-10-02T20:17:20Z

🕵 Merged master (e6853bbd5dd) to resolve the conflicts. Notes on the resolution:

This PR's IParser::consumesForeignText is dropped in favor of master's equivalent IParser::consumesRawText.
Master changed how the polyglot dialect recognizes ClickHouse SET statements (SET ROLE, SET TRANSACTION SNAPSHOT, foreign prefixes such as MySQL SET SESSION ...). That logic now lives in parsePolyglotNativeStatement, and the server-side polyglot branch of executeQuery uses it too, replacing the bare ParserSetQuery this PR had used. Client and server now classify these statements the same way. I checked over HTTP that SET ROLE NONE, SET TRANSACTION SNAPSHOT 1 (without the feature flag), SET max_threads = 1 garbage and SET SESSION sql_mode = ... behave as on master.
The open bot thread on lexically foreign URL queries was answered earlier (no code change: the concatenating path transpiles the URL query and the body together as one statement), so I resolved it.

`transpiled_query` was turned into a `std::shared_ptr` so that `Context::createCopy` shares the transpiled buffer instead of cloning it, but the `ContextData` copy constructor never copied the member. Child contexts (async insert queue, distributed writes) therefore had an empty `transpiled_query`, did not keep the buffer that the inline `INSERT` data points into alive, and `hasTranspiledInlineData` returned false for them. Related: #110321 (comment) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

# Conflicts: # src/Client/ClientBase.cpp # src/Client/ClientBase.h # src/Client/LocalConnection.cpp

…ntext consumers Cover the asynchronous insert queue (`async_insert = 1, wait_for_async_insert = 1`) and the `Distributed` sink, both of which run on a `Context::createCopy` copy and need the transpiled query that owns the inline data. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ritten query When a client-side AST->SQL rewrite (`--allow_merge_tree_settings`, or client-side query parameter substitution for an old server) replaces the verbatim polyglot text, the server parses plain SQL and no longer needs the parser-only settings pinned for the verbatim text. Restore the settings as they were before the pinning, so `polyglot_dialect`, `max_parser_depth` and similar are not sent as changed settings (which could fail under a profile that keeps one of them readonly). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Resolved the conflict in `src/Client/ClientBase.h`: kept the PR's new members and took master's comment, which renames `allow_experimental_logsql_dialect` to `enable_logsql_dialect`.

Conflicts: kept both new `QueryFlags` fields and both `HTTPHandler` flag assignments; the polyglot branch of `executeQuery` keeps this PR's up-front transpilation, now parsed with master's `*_for_query_text` limits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ings `executeQueryInBackground` replays the original query text with a copy of the query context, which by then carries the query's own `SETTINGS` clause. A query like `SELECT 41 + 1 SETTINGS allow_experimental_polyglot_dialect = 0` under the `polyglot` dialect was accepted by the foreground parse and then rejected with `SUPPORT_IS_DISABLED` by the background re-parse; `SETTINGS dialect = ...` and the parser limits had the same skew. Remember the parse-time values before applying the `SETTINGS` clause and restore them on the background context; the background run applies the clause again itself after parsing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…y mode In single-query mode `ClientBase::parseQuery` passed `max_query_size` as the lexer cap to `tryParseQuery`, whose lexical shortcut reported the generic `Max query size exceeded` before `ParserPolyglotQuery::parseImpl` ran, so the dedicated error explaining that inline `INSERT` data counts towards `max_query_size` in the polyglot dialect was never reached (and `04512_polyglot_insert_values` failed in a Rust-enabled build). The polyglot parser enforces `max_query_size` on the whole remaining input itself, before transpiling, so drop the outer lexer cap for it, as in multi-statement mode. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… comment With `dialect = 'polyglot'`, `ClientBase::analyzeMultiQueryText` treated an error token of the ClickHouse lexer (e.g. a leading BigQuery `#comment`) as "only comments left" and silently ended the script, so `clickhouse-client --queries-file` and `clickhouse-local` did not execute it. For the polyglot dialect only the real end of input ends the script now; anything else goes to the polyglot parser. Cover a leading-comment `INSERT ... VALUES` with a row-count assertion. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

A `WITH TOTALS` inside a FROM subquery, a CTE, a JOIN side or a view body puts a totals block into the outer query's result, which the oracles compare. That block is not distributive over a WHERE partition: a filter pushed below `TotalsHavingStep` recomputes the totals over the filtered rows (deliberately, see filterPushDown.cpp), and `UNION ALL` keeps the totals of its first branch only (`Pipe::uniteTotals`). So TLP WHERE's reference and its partitioned union disagree on the totals row, and the same holds for TLP DISTINCT and the distributed-plan oracle. The oracles already rejected a top-level `WITH TOTALS`. Add a recursive `hasWithTotalsAnywhere` screen to `check()`, next to the FINAL and WITH FILL screens, and to the stored-definition screen so a view hiding the same text is skipped too. Nested ROLLUP / CUBE / GROUPING SETS without TOTALS are left checked: their data rows stay partitionable. Seen on PR ClickHouse#110321 in `AST fuzzer (amd_release, oracle)`; reproduces on master. Mismatching attempts (10 fuzzed rounds each, clickhouse local), master build vs this change: subquery 44/50 -> 0/50, view 45/50 -> 0/50, CTE 46/50 -> 0/50, JOIN 26/30 -> 0/30. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

alexey-milovidov · 2026-10-23T23:54:11Z

🕵 @groeneai, investigate the failure: https://s3.amazonaws.com/clickhouse-test-reports/praktika.html?PR=110321&sha=7f4ebf08f19cc79f40eef60ff855db08df1961bc&name_0=PR&name_1=AST%20fuzzer%20%28amd_release%2C%20oracle%29 and provide a fix in a separate PR. If the fix is already in progress, link it here.

It is a TLP WHERE oracle mismatch on SELECT x, y, x, y FROM (SELECT DISTINCT -1 AS y, isNotNull(true AS x), 1 AS x, x GROUP BY GROUPING SETS ((y)) WITH TOTALS LIMIT 376): the reference prints x as true (Bool) and the partitioned query prints it as 1 (two aliases named x with different types). It is a SELECT with no polyglot dialect or INSERT, so it is unrelated to this PR.

The other red, Stress test (amd_debug) with STID 2508-2c9e, is already handled by #127367, which uses this PR's report as its reproducer.

groeneai · 2026-10-24T00:44:24Z

Both causes of this report are already handled:

Totals: TLP cannot partition a WITH TOTALS inside a FROM subquery. The filter pushed down through TotalsHavingStep changes the totals row, and UNION ALL keeps only the first arm's totals, so the result depends on arm order. AST fuzzer oracle: skip queries with WITH TOTALS in a subquery or view #127335 (merged 2026-10-22) makes the oracle skip these queries. This run's head 7f4ebf08f19c predates it.
true vs 1: this is a real analyzer bug. true AS x and 1 AS x are accepted as one alias because Bool and UInt8 constants compare equal, so the subquery has two x columns of different types. On master, SELECT true AS a, 1 AS a returns true and 1, while true AS a, 2 AS a is rejected. My open Fix wrong results when two DateTime constants differ only in the time zone #117161 fixes it: on its head 1a34a6991b2a, both that query and this report's subquery fail with MULTIPLE_EXPRESSIONS_FOR_ALIAS.

alexey-milovidov · 2026-10-25T10:59:23Z

🕵 Thanks, @groeneai. That covers both causes: the WITH TOTALS partitioning is skipped by the oracle since #127335, and the true AS x / 1 AS x alias ambiguity is tracked in #117161. Nothing in this PR needs to change. The current head 2f089c5e is fully green.

clickhouse-gh Bot added the pr-experimental Experimental Feature label Jul 13, 2026

clickhouse-gh Bot reviewed Jul 13, 2026

View reviewed changes

Comment thread src/Parsers/Polyglot/ParserPolyglotQuery.cpp Resolved

clickhouse-gh Bot reviewed Jul 14, 2026

View reviewed changes

Comment thread src/Client/ClientBase.cpp Resolved

alexey-milovidov and others added 2 commits July 15, 2026 02:04

Merge remote-tracking branch 'origin/master' into insert-values-polyglot

67cd894

clickhouse-gh Bot reviewed Jul 15, 2026

View reviewed changes

Comment thread src/Interpreters/executeQuery.cpp Resolved

alexey-milovidov added 2 commits July 16, 2026 11:54

Merge remote-tracking branch 'origin/master' into insert-values-polyglot

1df7670

Fix style check: scope SYSTEM FLUSH LOGS to query_log

23237b9

The style check requires `SYSTEM FLUSH LOGS log_name` instead of the global `SYSTEM FLUSH LOGS`, and the test only reads `system.query_log`.

clickhouse-gh Bot reviewed Jul 16, 2026

View reviewed changes

Comment thread src/Interpreters/executeQuery.cpp Resolved

Comment thread src/Parsers/Polyglot/ParserPolyglotQuery.cpp Resolved

alexey-milovidov and others added 3 commits July 17, 2026 03:56

Merge remote-tracking branch 'origin/master' into insert-values-polyglot

d2254f3

clickhouse-gh Bot reviewed Jul 17, 2026

View reviewed changes

Comment thread src/Client/ClientBase.cpp Resolved

clickhouse-gh Bot reviewed Jul 17, 2026

View reviewed changes

Comment thread src/Parsers/Polyglot/ParserPolyglotQuery.cpp Resolved

Comment thread src/Parsers/Polyglot/ParserPolyglotQuery.cpp Resolved

alexey-milovidov and others added 5 commits July 17, 2026 12:55

Merge remote-tracking branch 'origin/master' into insert-values-polyglot

6ef4580

clickhouse-gh Bot reviewed Jul 17, 2026

View reviewed changes

Comment thread src/Interpreters/Context.h Resolved

alexey-milovidov added 2 commits July 18, 2026 00:10

Merge remote-tracking branch 'origin/master' into insert-values-polyglot

3630b27

clickhouse-gh Bot reviewed Sep 19, 2026

View reviewed changes

Comment thread src/Server/HTTPHandler.cpp Resolved

clickhouse-gh Bot reviewed Oct 2, 2026

View reviewed changes

Comment thread src/Server/HTTPHandler.cpp Resolved

Comment thread src/Interpreters/executeQuery.cpp Resolved

clickhouse-gh Bot reviewed Oct 4, 2026

View reviewed changes

Comment thread src/Interpreters/Context.cpp Resolved

alexey-milovidov and others added 2 commits October 5, 2026 18:43

Merge remote-tracking branch 'origin/master' into HEAD

567d1d6

# Conflicts: # src/Client/ClientBase.cpp # src/Client/ClientBase.h # src/Client/LocalConnection.cpp

clickhouse-gh Bot reviewed Oct 5, 2026

View reviewed changes

Comment thread src/Client/ClientBase.cpp Resolved

alexey-milovidov and others added 2 commits October 6, 2026 13:47

Merge remote-tracking branch 'origin/master' into insert-values-polyglot

7055230

Resolved the conflict in `src/Client/ClientBase.h`: kept the PR's new members and took master's comment, which renames `allow_experimental_logsql_dialect` to `enable_logsql_dialect`.

clickhouse-gh Bot reviewed Oct 7, 2026

View reviewed changes

Comment thread src/Parsers/Polyglot/ParserPolyglotQuery.cpp Resolved

clickhouse-gh Bot reviewed Oct 13, 2026

View reviewed changes

Comment thread src/Interpreters/executeQuery.cpp Resolved

alexey-milovidov and others added 2 commits October 14, 2026 09:08

clickhouse-gh Bot reviewed Oct 15, 2026

View reviewed changes

Comment thread tests/queries/0_stateless/05057_polyglot_lexically_foreign_query.sh Resolved

alexey-milovidov and others added 4 commits October 16, 2026 14:00

Merge remote-tracking branch 'origin/master' into HEAD

c980318

Merge remote-tracking branch 'origin/master' into insert-values-polyglot

054d9d7

Merge remote-tracking branch 'origin/master' into insert-values-polyglot

7f4ebf0

Merge remote-tracking branch 'origin/master' into HEAD

2f089c5

Conversation

alexey-milovidov commented Jul 13, 2026 • edited by clickhouse-gh Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog category (leave one):

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Documentation entry for user-facing changes

Uh oh!

clickhouse-gh Bot commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

AI Review

Summary

Missing context / blind spots

Final Verdict

Uh oh!

Uh oh!

Uh oh!

alexey-milovidov commented Jul 15, 2026

Uh oh!

Uh oh!

alexey-milovidov commented Jul 16, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alexey-milovidov commented Jul 18, 2026

Uh oh!

alexey-milovidov commented Sep 19, 2026

Uh oh!

groeneai commented Sep 19, 2026

Uh oh!

Uh oh!

alexey-milovidov commented Sep 25, 2026

Uh oh!

alexey-milovidov commented Oct 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alexey-milovidov commented Oct 23, 2026

Uh oh!

groeneai commented Oct 24, 2026

Uh oh!

alexey-milovidov commented Oct 25, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

alexey-milovidov commented Jul 13, 2026 •

edited by clickhouse-gh Bot

Loading

clickhouse-gh Bot commented Jul 13, 2026 •

edited

Loading