feat: support Kafka tables in TypeScript and Python schemas

[KeKs0r]

Kafka queues currently require manual SQL because table() assumes MergeTree keys and settings. This adds native Kafka schema support in TypeScript first, followed by the Python equivalent, covering the Kafka → materialized view → MergeTree pipeline.

Fixes #203 (Kafka must-haves). Distributed remains a separate follow-up.

Behavior

• Define Kafka tables without sorting/storage clauses. Quote and escape literal settings while preserving the existing setting-string contract for other engines.
• Pull queues back into each native DSL and compare Kafka engine, columns, and settings in drift/check. Preserve meaningful whitespace, quoted delimiters, booleans, and escaped backslashes.
• Reject unsupported Kafka changes before writing migrations or advancing snapshots. Document explicit drop/create migrations with consuming views, synchronous queue drops, and the existing --allow-destructive gate.
• Document consumer offsets as runtime state and correct the issue's kafka_auto_offset_reset example: configure auto_offset_reset server-side instead.
• Add isolated Docker Redpanda/ClickHouse fixtures and a CI matrix for ClickHouse 25.3 and 26.3, exercising both CLIs.

Risks and boundaries

Replacing a queue interrupts ingestion and can replay or miss messages depending on consumer groups, offsets, retention, and in-flight work. There is no automatic replacement or exactly-once guarantee. Storage tables are retained in the documented workflow; operators must account for unmanaged views and other consumers. Pull warns about credential settings and rejects redacted placeholders during generation.

Validation

• GitHub checks are all green: full CI verify, Kafka on 25.3/26.3, text indexes on 26.3/26.8, CodeQL, and Fallow.
• Rebased onto the concurrently merged text-index support in feat(schema): add reliable full-text index support #215; retained both features.
• TypeScript typecheck, lint, and builds pass. All package test suites pass, with the 310-test CLI suite run serially after the local Bun parallel runner stalled without a reported failure.
• Python: 1,445 tests pass; one existing localhost-only integration test skips. Ruff and mypy pass. Pyright has the same 390 errors as current main, with no new errors.
• Both native Kafka end-to-end workflows pass on ClickHouse 25.3 and 26.3: message ingestion, pull/generate no-op, live drift/check, refused mutation without artifact changes, destructive gate, and resumed ingestion after explicit replacement.
• Docs build, Python wheel build/import smoke test, and workspace/packed dependency checks pass.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-27T02:27:37.595594Z	42695fd	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.