feat: 0.5.0 payload - CONCURRENTLY by default on both surfaces + statement_timeout - #6
Merged
Merged
Conversation
PlannedOperation gains a frozen-dataclass field `concurrent: bool = False` and to_json_dict emits it per operation. Additive to the JSON v1 contract: the schema's required list and the golden payload gain the key; existing keys/values are byte-stable. Defaults keep every hand-built-plan construction site (tests, users) unchanged.
Final pipeline stage _render_concurrent (after _dedup_enum_types, before Plan) rewrites standalone add_index ops on tables NOT created in the same plan to CREATE [UNIQUE] INDEX CONCURRENTLY via an anchored, idempotent, count=1 regex, setting the new concurrent flag via dataclasses.replace. The ordering invariant (dedup never sees a CONCURRENTLY render, so the byte-containment match holds by construction) is commented in plan(). DiffEngine.plan gains concurrently=True to skip injection entirely. Executor _is_concurrent becomes a union: op.concurrent (authoritative for generated plans) OR the SQL substring (hand-built Plans). classify docstring updated: add_index stays RISKY in both renderings.
A7: _session_gucs contextmanager SETs session-level GUCs on the autocommit segment's connection and RESETs each in finally; a reset failure is suppressed and the connection invalidated so the pool never hands out a session with a shrunken budget (0.4.2 lesson). The segment now runs with the same lock_timeout as the txn segment — previously it had no lock budget at all. B12: apply_plan gains statement_timeout (None = no GUC anywhere); not-None reaches the txn segment as SET LOCAL alongside lock_timeout (inline-ms, no bind params) and the concurrent segment via _session_gucs + RESET. Negative values are rejected up front, mirroring the 0.4.2 budget-validation pattern. Tests: concurrent-segment lock budget + pooled RESET observability, statement_timeout reaching both segments (current_setting probes) with None touching no GUC, negative validation, and the live end-to-end push (valid index in pg_indexes, honest partial_failure).
…sion api.plan/push/revision gain concurrently=True; api.push gains statement_timeout=None. THREADING HAZARD fixed: both knobs flow api.push -> with_advisory_lock -> the lock winner's reverify.plan(...) AND the final apply_plan — without it the winner re-planned with the default rendering, silently ignoring the caller's request. Pinned by test_concurrently_opt_out_end_to_end (spy on the re-plan kwargs + the captured plan's plain SQL + live apply). CLI: --no-concurrently on push and revision, --statement-timeout on push, following the existing flag patterns.
Fast path (no CONCURRENTLY in the file's text) stays byte-identical to 0.4.2: whole text, one txn, one exec_driver_sql call — the parser never executes, so dollar-quoted bodies (and their internal -- lines) are safe. Files containing CONCURRENTLY replay per-op on the op-label delimiters: plain segment first in one txn (create-to-index deps inside one file), concurrent ops on a dedicated lazily-created autocommit connection (B11: one per walk, session lock_timeout via _session_gucs, RESET + invalidate-then-close via ExitStack), versions row only after all concurrent ops succeed. A failed concurrent op blocks the file: partial_failure, no versions row, strict-order stop, and an honest note when the plain segment already committed. Implementation discovery (live-verified against the dev DB): the server executes a multi-statement simple-protocol string as ONE implicit transaction block even on an autocommit connection, so CREATE INDEX CONCURRENTLY cannot ride a multi-statement exec_driver_sql call. The concurrent lane therefore executes statement-per-execute via _split_statements — a dollar-quote, string-literal and comment aware boundary scanner (psql-equivalent). Generated ops are single statements (splitter is a no-op); only hand-edited label-less bodies exercise it, matching the spec §7 at-the-author's-risk stance. Plain segment and fast path never split. Module docstring rewritten to the hybrid contract (B9).
run_migrate gains statement_timeout (seconds, None = set nothing): SET LOCAL in every per-file transaction (fast path and the mixed path's plain segment) and a session statement_timeout on the CONCURRENTLY autocommit connection — set once at connection creation via _session_gucs alongside lock_timeout, RESET before the connection closes. Negative values are rejected up front with a typed SqlpushError, before any connection or file work (0.4.2 validation pattern). Plumbed through api.migrate(statement_timeout=None) and the migrate verb's --statement-timeout flag, following push's flag.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The 0.5.0 payload: CONCURRENTLY by default on both surfaces, plus
statement_timeout— implementing the adjudicated design (12 decisions, oracle-authored; one mid-cycle design repair, see below).Push surface
plan/push/revision(one injection point, final pipeline stage after both dedups — the byte-containment dedup invariant holds by construction; new-table indexes stay in-transaction plain;create_hypertableops unaffected).revision-generated chain files inherit the same rendering.concurrently=False(API) /--no-concurrently(push, revision) — the documented INVALID-index recovery escape. Threaded through the advisory-lock winner re-plan (the pinned hazard, test-pinned).concurrentboolean on every operation in the v1 plan JSON — additive only, golden contract updated.lock_timeoutnow boundsCREATE INDEX CONCURRENTLYtoo (session SET + RESET with invalidate-on-failed-reset hygiene), not just the transactional segment.statement_timeouton push (SET LOCALin the txn segment, session+RESET on autocommit; default None = server default untouched; negative rejected up front).Chain surface
-- op N [label]delimiters (the chain-spec §7 mechanism): plain ops first in one transaction, concurrent ops statement-by-statement on a dedicated autocommit connection, versions row last — first concurrent failure blocks the file (partial_failure, no versions row, honest note when the plain segment already committed).statement_timeouton migrate (SET LOCALper file, session GUC on the autocommit connection,--statement-timeout).''/""doubling, comments; confined to that lane only). Survived a 25-case adversarial audit with no silent mis-split; deviations only on exotic input, all fail-loud.Docs
[Unreleased]: 6 Added bullets (API + CLI named) + a Known-limitations section (hand-edit tokenization cost per chain spec §7, INVALID-index recovery, mixed-file crash window, fail-loud re-runs). README aligned (behavior, JSON additive field, roadmap); stale asyncpg roadmap line dropped (shipped in 0.4.2).Verification
166 passed, 1 xfailed(baseline 144 + 22 new) against livetimescaledb-ha:pg17; DB-free subset skips cleanlyruff check,ruff format --check,ty checkall cleanRelease surface
Exit codes unchanged. This PR is the 0.5.0 payload: on merge, cut
v0.5.0.