Repository navigation
CONNECT-IP: an HTTP/3 path that shrinks below 1280 bytes black-holes the tunnel #1247
Description
What happens
When the path toward an HTTP/3 CONNECT-IP client shrinks mid-tunnel so that it can no longer carry 1280-byte inner packets, the tunnel stays open and delivers nothing larger than the new path size. The server sends no ICMPv6 error, does not end the request, and counts nothing: galactic_connectip_tunnel_ends_total{cause="path_mtu"} and galactic_connectip_packets_denied_total{reason="path_mtu"} stay at zero. The connector sees a silent black hole until it reconnects on its own.
RFC 9484 section 10.1 calls for ending the tunnel when the path cannot carry the IPv6 minimum, because the sender cannot shrink below 1280 bytes.
Cause
The server sends QUIC packets of a fixed size, so it can't tell when the path shrinks under them. quic-go (v0.63.0) has no black hole detection: it never lowers a packet size, it sets IPV6_PMTUDISC_PROBE so the kernel ignores Packet Too Big for the socket, and it reports a lost datagram as nothing. No quic-go release changes this; quic-go/quic-go#3955 and quic-go/quic-go#5914 are open upstream.
The admission probe checks only that a 1280-byte packet fits the server's own packet size. It does not confirm the client received it, so it also admits a request over a path that is already too small.
How to reproduce
A loopback UDP relay between client and server that drops server-to-client datagrams above 1300 bytes, applied after the handshake, with the gVPC sending 1280-byte packets to the client. The tunnel stays open for as long as the test waits. In the containerlab lab, lowering the provider pod's host-side veth MTU to 1300–1340 during a tunnel does the same.
Expected
Once the path stops carrying 1280-byte packets, the request ends with cause path_mtu, one packet is counted as reason="path_mtu", and the admission probe refuses a new request until the path recovers.
Notes
The other half of #1237, a Packet Too Big for packets larger than the tunnel carries on a path that shrank but still carries 1280 bytes, is fixed by sending fixed-size QUIC packets. Draft #1243 prototyped detection in the server from the connection's qlog trace and the client's acknowledgements. It works on loopback, but it depends on qlog event details that quic-go doesn't treat as a stable API, and one of its tests failed 1 run in 30. A fix needs an agreed approach: that one, a quic-go change upstream, or something else.
Activity
fix(connectip): Stop QUIC path MTU discovery so Packet Too Big holds
privateip commentedon Oct 29, 2026
Confirmed on main at 32704f5.
A loopback test, run through the existing
shrinkingPathrelay ininternal/edge/connectip, opens an HTTP/3 tunnel, delivers one 1280-byte packet, and then makes the relay drop every server-to-client datagram larger than 1300 bytes. It then injects a 1280-byte packet toward the client every 100 ms for 10 s. The relay dropped all 100 packets. The backend session stayed open, andpackets_denied_total{reason="path_mtu"}didn't increase.The code matches the cause described above. With
DisablePathMTUDiscoveryand a fixed 1350-byte packet,SendDatagramnever returnsDatagramTooLargeErrorfor a 1280-byte packet, so then <= MinInnerMTUbranch inproxySessioncan't run on HTTP/3.probeInnerMTUchecks only that the datagram fits locally. quic-go v0.63.0'sConnectionStatsgives only connection-wide loss counts, so it can't tell full-size packets lost on a shrunk path from random loss. Detecting this needs per-packet acknowledgement data, which quic-go exposes only through the connection's qlog trace.