Unit 4a: DuckDB execution layer and hot-store/tree reader

[mairas]

Goal: Build the DuckDB execution layer and the reader that unions the hot store with the Parquet tree. No API surface yet.

Requirements: R12

Dependencies: Unit 3b

Files:

• Create: src/query/duck.ts, src/query/reader.ts
• Test: src/test/duck.test.ts, src/test/reader.test.ts

Approach:

• This has no sqhp analogue. sqhp's query-client.ts is a 235-line HTTP client to a running server; this is process spawn, lifecycle, timeout, concurrency control and configuration lockdown.
• One HTTP request compiles to at most one spawned process. sqhp issues one query per pathSpec sequentially, and a non-numeric path costs two. Following that shape with spawn-per-query turns a ten-series Grafana panel into ten to twenty spawns, each ~120 MB and each paying process start plus extension load plus Parquet footer reads. Compile all pathSpecs into a single statement.
• Cap concurrent query processes and state the behaviour at the cap — queue or 503. State whether a roll (~218 MB) and a query (~120 MB) may run concurrently, and what the summed transient is. Nothing else in the plan couples these.
• Disable DuckDB extension autoinstall and autoload, restrict file access to the data directory, and lock the configuration after startup. Load sqlite_scanner before the restriction goes on. The restriction must permit the hot store's SQLite WAL and shm files, wherever Unit 2 put them.
• Reader unions hot store and tree so a query ending at "now" is complete within the flush window. Measured: 16–37 ms single-path against a one-roll tree.
• Reopen Unit 3a's layout decision on query evidence. Partitioning governs whether getPaths is a directory listing or a scan, and whether a time range prunes files. State what a reversal costs, since existing tree data must be re-rolled.

Test scenarios:

• Happy path: a raw range spanning hot store and tree returns correct rows with no gap or duplicate at the seam.
• Happy path: a multi-path request produces exactly one spawned process.
• Edge case: a range entirely inside the hot store never touches the tree.
• Edge case: a range before the earliest partition returns empty, not an error.
• Error path: N concurrent requests hold the process cap; behaviour at the cap matches the stated rule.
• Error path: a query that would trigger an extension load fails cleanly rather than fetching over the network.
• Error path: a partially written partition from a failed roll is skipped, not returned truncated.
• Integration: end-to-end wall time from spawn to result, not in-engine query time — the spike figures may be the latter.

Verification:

• A seam-spanning range query returns correct rows.
• Spawn count per request and the concurrency cap are asserted, not assumed.