Summary
server.codeium.com (the endpoint serving swe-2 / devin-agent chat streams) forcibly closes TCP connections that carry no bytes for ~60 seconds. Any model generation with a silent window longer than 60s — long thinking pauses, or large buffered tool-call arguments — is killed mid-stream. Clients see The socket connection was closed unexpectedly and cannot recover, because retrying replays the same long generation.
Reproduction
Raw TLS connection to server.codeium.com:443, no request sent:
handshake ok at 0.6s, holding idle...
recv returned at 60.6s: b'' # server-initiated close
- Reproduced via direct route and via an HTTP/SOCKS proxy — identical 60.6s, so it's the server/LB, not the path.
- Control:
api.github.com closes idle TLS at ~5.4s; the 60s value matches a typical managed load-balancer idle timeout (e.g. AWS ALB default).
In production traffic (via the omp coding agent, devin provider, model swe-2): 15 stream failures logged, durations [60, 60, 61, 61, 61, 62, 62, 62, 62, 63, 64, 66, 76, 118, 530]s — clustering exactly at the 60s idle window. Successful streams in the same sessions ran up to 1198s, confirming the limit is silence, not total duration.
Impact
Agentic coding clients routinely trigger >60s silent windows: the model thinks for a long stretch, then emits a large tool call (e.g. dispatching several subagents with inlined specs) whose arguments are generated server-side before any bytes stream out. Every such turn dies deterministically, and client retries fail identically — the feature is effectively unusable for large single generations.
Request
Emit periodic heartbeat/keepalive frames on chat streams (e.g. an SSE comment : keepalive or a no-op protobuf message every ~15–30s) so intermediate timeouts never see a fully idle connection. Alternatively, raise the idle timeout for streaming endpoints well above the worst-case generation latency.
Happy to provide packet captures or additional timing data if helpful.
Summary
server.codeium.com(the endpoint servingswe-2/devin-agentchat streams) forcibly closes TCP connections that carry no bytes for ~60 seconds. Any model generation with a silent window longer than 60s — long thinking pauses, or large buffered tool-call arguments — is killed mid-stream. Clients seeThe socket connection was closed unexpectedlyand cannot recover, because retrying replays the same long generation.Reproduction
Raw TLS connection to
server.codeium.com:443, no request sent:api.github.comcloses idle TLS at ~5.4s; the 60s value matches a typical managed load-balancer idle timeout (e.g. AWS ALB default).In production traffic (via the omp coding agent,
devinprovider, modelswe-2): 15 stream failures logged, durations[60, 60, 61, 61, 61, 62, 62, 62, 62, 63, 64, 66, 76, 118, 530]s — clustering exactly at the 60s idle window. Successful streams in the same sessions ran up to 1198s, confirming the limit is silence, not total duration.Impact
Agentic coding clients routinely trigger >60s silent windows: the model thinks for a long stretch, then emits a large tool call (e.g. dispatching several subagents with inlined specs) whose arguments are generated server-side before any bytes stream out. Every such turn dies deterministically, and client retries fail identically — the feature is effectively unusable for large single generations.
Request
Emit periodic heartbeat/keepalive frames on chat streams (e.g. an SSE comment
: keepaliveor a no-op protobuf message every ~15–30s) so intermediate timeouts never see a fully idle connection. Alternatively, raise the idle timeout for streaming endpoints well above the worst-case generation latency.Happy to provide packet captures or additional timing data if helpful.