Skip to content

[T3 Connect] Reuse one HTTP client per replica for procedure requests - #6023

Draft
bradleyshep wants to merge 1 commit into
masterfrom
bradley/procedure-http-client-reuse
Draft

bradleyshep wants to merge 1 commit into
masterfrom
bradley/procedure-http-client-reuse

Conversation

@bradleyshep

@bradleyshep bradleyshep commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Part of the work to move T3 Code's T3 Connect relay onto SpacetimeDB.

Description of Changes

InstanceEnv::http_request (a procedure's ctx.http.fetch) built a new reqwest::Client for every request, so no connection or TLS session was ever reused (the TODO(perf) this resolves). A module that sends to an HTTP service repeatedly, such as Apple's push service, which recommends keeping connections open, paid a full TCP and TLS handshake per request.

The client now lives on the ReplicaContext, built on first use and shared by all of the replica's module instances, so their requests share one connection pool. It is dropped with the replica and survives module updates. Behaviour is otherwise unchanged:

  • the same FilteredDnsResolver;
  • the same redirect policy, which refuses private and special-purpose IP literals;
  • the per-request timeout, still set on each request;
  • the disabled-config check, metrics and error mapping.

A client that fails to build still surfaces as an HTTP error to the procedure, and the next request tries again.

The client is per replica rather than process-wide because a pooled connection belongs to the tokio runtime that opened it. Production runs procedure HTTP on one host runtime, but tests create a runtime per test.

Related: #6012 lets these requests negotiate HTTP/2. Together they give APNs a reused HTTP/2 connection.

API and ABI breaking changes

None.

Rollback safety impact

n/a

Expected complexity level and risk

  1. The request path is unchanged apart from where the client comes from.

Worth a reviewer's eye:

  • Idle connections now stay pooled under reqwest's default pool settings instead of closing after each request. With many databases on one host, that means more idle sockets; pool_idle_timeout and pool_max_idle_per_host can be set in build_module_http_client if needed.
  • A server can close a pooled connection while it's idle. hyper retries a request on a reused connection that turned out closed before the request was sent (retry_canceled_requests, on by default); a request already sent when the server hung up fails like any other network error. reqwest drops idle connections after 90 s.
  • The DNS filter runs when a connection opens. A reused connection only ever goes to an address that already passed it, and redirects are still checked on every request.

Testing

  • New test http_requests_reuse_connection: two requests through InstanceEnv::http_request to a local keep-alive server arrive over one TCP connection. The test fails (2 connections) with the old per-request client.
  • cargo test -p spacetimedb-core --lib --features allow_loopback_http_for_tests instance_env (23 passing), and without the feature (19)
  • cargo clippy -p spacetimedb-core --all-targets: no new warnings
  • cargo fmt --check
  • CI: cargo test --all doesn't enable allow_loopback_http_for_tests for core, so the loopback HTTP tests (this one and the existing decompression tests) never ran in CI. tools/ci/commands/test now runs them in a focused step; the exact command runs 4 tests locally, and 5 with [T3 Connect] Negotiate HTTP/2 for procedure HTTPS requests #6012 applied too (its ALPN test included; all pass).
  • Both this PR and [T3 Connect] Negotiate HTTP/2 for procedure HTTPS requests #6012 add tests at the end of the same test module in instance_env.rs, so whichever merges second needs a trivial rebase (keep both tests).

@bradleyshep
bradleyshep force-pushed the bradley/procedure-http-client-reuse branch from 01f2302 to 4de5d9c Compare September 30, 2026 17:23
@bradleyshep
bradleyshep force-pushed the bradley/procedure-http-client-reuse branch from 4de5d9c to f258cd6 Compare September 30, 2026 17:45
@bradleyshep bradleyshep changed the title Reuse one HTTP client per replica for procedure requests [T3 Connect] Reuse one HTTP client per replica for procedure requests Sep 30, 2026
`InstanceEnv::http_request` built a new reqwest client for every
`ctx.http.fetch`, so no connection or TLS session was ever reused, and a
module sending to an HTTP/2 service like APNs paid a full TCP and TLS
handshake per request (the TODO(perf) this resolves).

The client now lives on the ReplicaContext, built on first use and shared by
the replica's module instances. It keeps the same DNS filter and redirect
policy; per-request timeouts stay on the request. A failed build still
surfaces as an HTTP error to the procedure and is retried on the next request.
@bradleyshep
bradleyshep force-pushed the bradley/procedure-http-client-reuse branch from f258cd6 to d1a35f8 Compare October 1, 2026 17:26

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant