feat(serenity): split publish out of create + publish-after-populate finalize (LLMO-5492)

[andreeastroe96]

What & why

LLMO-5492 — T12 · push generated prompts + models to Semrush, publish-after-populate.

The onboarding fan-out (performSerenityFanOut → handleCreateMarket) published each Semrush project at create time — i.e. published an empty project. Prompts/models only live in our Postgres (llmo_customer_config) and were never pushed upstream.

⚠️ Stacked on feat/LLMO-5201-5202-cohort-gate-markets — review/merge that first. Base is set accordingly.

Behavior-preserving in prod: the new path is inert until SERENITY_DEFER_PUBLISH=true and the DRS-completion trigger calls finalizeSerenityProjects. Safe to merge but dormant until both land.

Scope (this PR)

AC#1 — split publish out of the create path (flag-gated)

• SERENITY_DEFER_PUBLISH env flag + isSerenityDeferPublishEnabled(env) — default OFF, so projects are never left unpublished until the finalize trigger is live.
• handleCreateMarket / handleCreatePrompts take a { publish = true } option (default preserves the standalone-endpoint contract).
• performSerenityFanOut passes publish: !deferPublish — provisions drafts when the flag is on.

AC#2 — publish-after-populate mechanism

• New src/support/serenity/handlers/finalize.js → finalizeSerenityProjects(...): push prompts (publish deferred) → set models per slice (handleUpdateModels) → publish each project exactly once.
• Publish is gated on population: if prompts were requested but every push failed, publish is skipped so an empty project never goes live. Per-slice/per-project failures are recorded and don't abort the rest.
• Model sync and publish are idempotent; prompt push is not dedup'd upstream, so the (future, separate-repo) trigger must deliver the prompt payload exactly once. Returns { prompts, models, published, publishFailed }.

AC#3 — publish-completion polling ✅ (now implemented)

Previously deferred as "blocked — rest-transport.js has no getProjectStatus." Unblocked once serenity-docs #12 (§6) confirmed the project carries a publish_status attribute (draft | publishing | initial_publish_failed | live | live_with_unpublished_updates).

• rest-transport.js — new getProjectStatus(ws, pid) → GET /v1/workspaces/{ws}/projects/{pid}. Uses the v1 default view deliberately (it echoes publish_status faithfully; the v2/live=true view empties a never-published draft's config per §10).
• src/support/serenity/handlers/publish-status.js (new) — the reusable read/classify/poll mechanism. classifyPublishStatus maps live/live_with_unpublished_updates → published, initial_publish_failed → failed, draft/publishing/unknown → pending. pollProjectPublished(transport, ws, pid, { attempts, intervalMs, log, sleep }) bounds the poll; a status-read error is non-fatal (logged, retried, reported pending). The DRS/audit worker's unbounded ≤900s reconcile loop consumes this same helper.
• finalize.js — after the async publish (202) is accepted, a bounded best-effort confirm within the Lambda's wall budget. Opt-in via typeof transport.getProjectStatus === 'function', so the existing 6-arg callers/tests are unchanged; new 7th options arg { confirmAttempts, confirmIntervalMs }. Confirmed live → published; initial_publish_failed → publishFailed (surfaced early); still draft/publishing within budget or status unreadable → stays published (accepted on 202; the worker reconciles) so an in-progress async publish is never mislabeled as failed.

Deferred / out of scope

• AC#2 trigger ("when DRS prompt-generation completes") — lives in audit-worker/DRS, a separate repo.
• The unbounded ≤900s reconcile poll — also the DRS/audit worker's job (separate repo, SQS-driven). This Lambda only does the bounded best-effort confirm within its ~15s wall budget; both consume the shared publish-status.js helper.
• AC#4 (stop storing prompts in our DB) — deferred per the ticket.

Tests

• New: flag helper (5), handleCreateMarket publish:false/default (2), handleCreatePrompts publish:false (1), finalizeSerenityProjects (publish-after-populate incl. empty-publish guard + partial-success, plus AC3 confirm cases: live→published, initial_publish_failed→publishFailed, publishing-within-budget→published, read-error→published, mixed), publish-status.js (read/classify + poll: first-read live, initial_publish_failed, draft→publishing→live, exhaust→pending, read-error→pending, single-attempt no-sleep), rest-transport getProjectStatus (URL + GET + raw JSON).
• eslint clean. Code review + security review run (see Jira labels).

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[github-actions]

This PR will trigger a minor release when merged.