Connection-per-session model exhausts Postgres connections under concurrent sandbox creation (no backpressure)

[Hazzng]

Summary

Under high concurrency, brand-new sandbox creation fails with Postgres sorry, too many clients already once the number of concurrent warm sessions exceeds the database connection budget. The server degrades gracefully (sanitized { ok: false }, no crash), but there is no backpressure before Postgres hard-rejects, so a burst of new sandboxes silently fails instead of being queued or rejected with a meaningful 503.

This is a property of the existing connection-per-session architecture — surfaced during stress testing of #120 (MCP static-header auth), but not caused by it. Filing as an enhancement so it isn't lost.

Root cause

• Each warm sandbox session owns its own PostgresDialect, whose pool is bounded by PG_POOL_MAX (default 2) — see src/sql-fs/dialects/postgres.ts:57.
• There is no global cap on the number of concurrent warm sessions (only an idle reaper via SESSION_IDLE_MS). Peak connection demand ≈ PG_POOL_MAX × concurrent_warm_sessions + (per-tenant meta backend).
• Against a Postgres with max_connections = 100, the effective ceiling is roughly ~48 simultaneous fresh sandboxes before new connections are refused.

Reproduction

Local server (pnpm dev) + Dockerized Postgres 16 (max_connections = 100), driving the MCP endpoint with N concurrent distinct identities, each creating a sandbox + running bash_exec:

Concurrent identities	Result
20	20/20 ok, 0 errors, 0 leftovers
45 (×2 back-to-back)	45/45 ok, 0 errors
200	70/200 ok — 130 failed with sorry, too many clients already

Server log during the 200-way burst:

{"event":"destroy_sandbox_error","sandboxId":"...","error":"sorry, too many clients already"}
{"event":"destroy_sandbox_error","sandboxId":"...","error":"remaining connection slots are reserved for roles with the SUPERUSER attribute"}

The server stayed up and responsive throughout (no crash, no unhandled rejection); failed operations returned the sanitized error path. Cross-owner isolation held (0 leaks) even under exhaustion.

Impact

• Bursty multi-user load (e.g. many LibreChat end-users via the new static-header MCP auth, each getting an isolated sandbox) can transiently exhaust the connection budget and fail sandbox creation/deletion with an opaque internal error.
• Failures surface as a generic sanitized error rather than a clear "capacity" signal, making it hard for clients to back off/retry.

Proposed enhancements (pick some)

1. Backpressure / global connection budget: cap concurrent warm sessions (or total DB connections) and queue or fast-fail with 503 Service Unavailable + Retry-After once the budget is reached, instead of letting Postgres reject mid-transaction.
2. Shared connection pool across sessions (or a tenant-level pool) rather than one pool per session, so connection count scales with concurrency rather than with session count.
3. Pooler guidance in docs: explicitly document running behind PgBouncer / the Neon pooler endpoint for multi-user deployments, and surface PG_POOL_MAX + expected max_connections math (the code comment at postgres.ts:53-56 already hints at this).
4. Clearer error mapping: translate too many clients (53300) to a dedicated CAPACITY/503 response code so clients can retry intelligently.

Notes

• No code change required for feat(mcp): static-header (API-key) auth for /mcp (LibreChat) #120 — that PR only touches the auth/MCP-wiring layer.
• Environment knobs involved today: PG_POOL_MAX (default 2), SESSION_IDLE_MS, and Postgres max_connections.