Skip to content

feat: add generation-2 request template storage - #1499

Merged
pjb157 merged 1 commit into
peter/retention-lifecyclefrom
peter/template-generation
Aug 17, 2026
Merged

feat: add generation-2 request template storage#1499
pjb157 merged 1 commit into
peter/retention-lifecyclefrom
peter/template-generation

Conversation

@pjb157

@pjb157 pjb157 commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Summary

First PR of the template-generation retention workstream: an expand-only weekly-partitioned generation-2 store for request templates, so template content can eventually be deleted by partition drop and the legacy heap retired as one relation. Stacked on #1481 (retained-response retention), whose journal/bucket/fence machinery the later retirement PRs reuse.

Nothing moves and nothing can be deleted after this PR: no writes are cut over, no lifecycle flags exist yet, and the migration never scans, rewrites, or locks the existing request_templates heap.

What it adds

  • request_templates_g2 — range-partitioned by created_on into weekly children (request_templates_g2_yIYYYwIW), carrying every legacy column plus the partition key, with the same secondary index shapes. Scheduled deletion for this store will be whole-partition drop, gated on indexed metadata proofs that no file, live batch, or archived batch still references the window.
  • request_template_buckets — one lifecycle row per weekly partition (active → retiring → retired); the read fence for future retirement.
  • request_template_routes — content-free template_id → week oracle. A route exists only for generation-2 templates; absence means the legacy heap. This is how readers, erasure, and the claim path resolve a template's generation without probing partitions.
  • ensure_request_template_partition(s) — advisory-locked, idempotent weekly partition creation with exact-bounds validation before attach; rejects non-Monday week starts.
  • active_request_templates is redefined as a UNION across both generations with an identical column shape, so every existing consumer (including the claim path) is generation-transparent before any write moves. The generation-2 arm is empty until cutover and fenced buckets are excluded.
  • The down migration fails closed while any generation-2 row, route, or lifecycle state exists.

Testing

  • New integration suite: partition strategy and column parity, exact weekly bounds, idempotent/concurrent-safe ensure helpers, non-Monday rejection, view parity across generations, soft-deleted-file filtering in both arms, bucket fencing hiding generation-2 rows, and route-directed point-read partition pruning.
  • Full fusillade-arsenal library + integration suites green; fresh up/down/up migration cycle verified on PostgreSQL 17, including the fail-closed down with rows present.
  • fmt, clippy (zero warnings), migration checksums (up + down), and sqlx offline metadata unchanged.

Follow-ups (separate PRs, in order)

Write cutover behind a flag → scheduled input-file retention → batch-archive partition retirement → reference-gated weekly template drops → legacy heap seal-and-drop.

🤖 Generated with Claude Code

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 17, 2026

Copy link
Copy Markdown

Deploying control-layer with  Cloudflare Pages  Cloudflare Pages

Latest commit: 2c3d1f1
Status: ✅  Deploy successful!
Preview URL: https://80ff9b23.control-layer.pages.dev
Branch Preview URL: https://peter-template-generation.control-layer.pages.dev

View logs

@pjb157

pjb157 commented Aug 17, 2026

Copy link
Copy Markdown
Contributor Author

Consolidated into #1481 per review direction — the retention PR now carries the full deletion policy across data classes.

@pjb157
pjb157 merged commit 2c3d1f1 into peter/retention-lifecycle Aug 17, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant