Skip to content

Implement and verify CONNECT-IP MTU and ICMPv6 behavior #982

New issue
New issue
Closed
Task
Closed
Implement and verify CONNECT-IP MTU and ICMPv6 behavior#982
Task
Assignees
privateip
Labels
enhancementNew feature or request

Description

@privateip
privateip
opened on Oct 22, 2026

Parent: #970

Deliverable

Correct packet-size handling and ICMPv6 signaling across the connector tunnel and gVPC forwarding path, with reproducible path-MTU tests.

Dependencies

#980, #981

Galactic Edge integration (#992)

Implement this milestone as a consumer of the Galactic Edge provider contract. #992 owns provider authorization, host attachment lifecycle, shared-state fencing and maintenance, independently enforced forwarding expiry, prefix conflict arbitration/publication, and gVPC traffic policy. MASQUE owns client authentication, connector /96 assignment and retention, session packet checks, transport behavior and its provider adapter. Use the foundation's versioned contract and existing provider helpers; do not introduce a second host-networking controller, lease store or BGP publisher. Any required foundation change must be tracked and qualified under #992 before this milestone claims integrated acceptance.

Work

  • Account for actual QUIC datagram capacity and framing overhead when admitting and forwarding inner IPv6 packets.
  • Implement correct Packet Too Big behavior for oversized packets and explicit tunnel failure when the QUIC path cannot support the required IPv6 minimum, following RFC 9484.
  • Exercise the complete path, including return traffic and gVPC encapsulation/decapsulation; fix any packet-size handling gap that prevents acceptance.
  • Validate HTTP/2 capsule reassembly across transport segments without imposing the QUIC single-datagram size limit. Maintain configured resource bounds.

Acceptance criteria

  • Both transports pass bidirectional 1280-byte inner IPv6 delivery against real workloads.
  • Controlled path-MTU reductions during an active QUIC tunnel produce correct ICMPv6 feedback or required tunnel termination, not silent black holes.
  • Packets larger than the usable tunnel or downstream MTU exercise the appropriate error path; feedback reaches the correct connector and obeys policy.
  • Tests record inner packet size, outer path MTU and observed behavior and cover local and remote workload paths.

Additional integration acceptance

  • Record attachment/device MTU, SRv6/underlay overhead and transport payload capacity independently; coordinate any host-side correction with the foundation rather than granting MASQUE host privileges.
  • Resolve Edge pod attachments: unchecked caller inputs and weak isolation tests #989's attachment-MTU-ceiling finding with a linked fix or current-path proof. Device drift/readback and insufficient underlay capacity must produce explicit behavior.
  • Include local-NAT-egress and remote-workload paths, both directions, live MTU changes, control-packet policy and tenant-safe feedback. PacketAccess readiness alone is not MTU acceptance.

Scope follows #970: CONNECT-IP with HTTP/2 fallback; no public API development, plugin/portal integration, product-client development or production rollout.

Activity

  1. privateip
    added
    enhancementNew feature or request
    on Oct 22, 2026
  2. privateip
    added a parent issue on Oct 22, 2026
  3. privateip
    changed the title [-]MASQUE: implement and verify CONNECT-IP MTU and ICMPv6 behavior[/-] [+]Implement and verify CONNECT-IP MTU and ICMPv6 behavior[/+] on Oct 22, 2026
  4. privateip
    added the
    Task
    issue type on Oct 22, 2026
  5. privateip
    self-assigned this
    on Oct 28, 2026

  6. privateip commented on Oct 28, 2026

    @privateip
    privateip
    on Oct 28, 2026
    CollaboratorAuthor

    Verdict: valid in part. Checked against the top of the #980/#981 stack (#1207, 78a1b53), since this issue builds on both.

    Already in place

    • Admission sends a 1280-byte inner probe datagram over HTTP/3 and answers 503 when the path cannot carry it.
    • Over HTTP/3, a gVPC packet too large for the current datagram size gets a Packet Too Big that passes the session policy and reaches the sender (TestProxyReturnsPacketTooBig).
    • Over HTTP/2, capsules are parsed incrementally, a DATAGRAM capsule is capped at 1508 bytes, and a packet over 1500 bytes gets a Packet Too Big.

    Gaps

    1. Mid-tunnel path MTU drop below the IPv6 minimum over HTTP/3. connect-ip-go answers every packet that does not fit with a Packet Too Big that reports 1280, even when the packet is 1280 bytes or smaller. The sender cannot go lower, so packets are dropped with no termination. RFC 9484 section 10.1 requires the tunnel to end instead. Reproduced with a test that feeds proxySession a tunnel unable to send any packet: after a 1232-byte packet, the session stayed open for 2 s and the backend received a Packet Too Big reporting MTU 1280.
    2. The HTTP/2 Packet Too Big reports 1280 although that transport carries 1500 bytes, so senders shrink more than they need to.
    3. Edge pod attachments: unchecked caller inputs and weak isolation tests #989's ceiling finding still holds on the edge foundation. galactic-edge --mtu has a floor and no ceiling. Edge host links write no tenant_gw_table row, so usid_egress drops a packet larger than encap_mtu_table with no_gateway and sends no error. A connector packet toward a remote workload larger than the uplink MTU less 40 bytes is lost silently.

    The lab rows (real workloads, live MTU reduction against a real QUIC path, local NAT egress) need the lab and are not covered by this verdict.

  7. privateip commented on Oct 28, 2026

    @privateip
    privateip
    on Oct 28, 2026
    CollaboratorAuthor

    Fix plan, approved.

    PR 1, stacked on #1207 (fix/issue-982)

    • proxySession ends the session when the tunnel returns a Packet Too Big for a packet of 1280 bytes or less. That means the QUIC path can no longer carry the IPv6 minimum, so RFC 9484 section 10.1 calls for termination. The server logs it and counts it in a new reason="path_mtu" session-end metric, so the drop is visible.
    • The HTTP/2 Packet Too Big reports 1500, the largest packet that transport carries, instead of 1280.
    • Regression tests: a tunnel unable to send a 1280-byte packet ends the session and returns no Packet Too Big. A tunnel that refuses only larger packets keeps the session and returns the error. An HTTP/2 Packet Too Big reports 1500.
    • doc.go and docs/edge/masque-service.md describe the behavior.

    PR 2, from main, foundation under #992 and #989 (fix/issue-982-mtu-ceiling)

    • galactic-edge reads encap_mtu_table on every attachment pass. When the value is nonzero and --mtu exceeds it, the attachment is not provisioned. It gets Programmed=False, reason MTUExceedsUnderlay, naming both values. A zero value (check off, or galactic-cni not started yet) skips the check.
    • Unit test with a fake map. The provider contract doc gets updated.

    Risk: a tunnel whose QUIC path drops below the minimum now ends instead of black-holing. The connector reconnects, and the 1280-byte admission probe refuses the new request while the path stays too small. galactic-edge is not deployed outside labs, so the ceiling refuses nothing in production today.

    #982 stays open for the lab rows.

  8. privateip
    added 2 commits that reference this issue on Oct 28, 2026

    fix(containerlab): Patch only the edge agent's MTU argument

    9b0c48c

    fix(containerlab): Fail the lab build if the edge MTU argument moves

    5874f0a

  9. privateip commented on Oct 28, 2026

    @privateip
    privateip
    on Oct 28, 2026
    CollaboratorAuthor

    The fixes are in two PRs. Neither closes this issue, which stays open for the lab rows.

    Still open here: bidirectional 1280-byte delivery against real workloads on both transports, a live path MTU drop against a real QUIC path, local NAT egress, and lab edge conformance at the new attachment MTU.

  10. privateip
    added a commit that references this issue on Oct 29, 2026

    fix(connectip): End the session when the path drops below 1280 bytes

    7f5057d

  11. privateip commented on Oct 29, 2026

    @privateip
    privateip
    on Oct 29, 2026
    CollaboratorAuthor

    Closing this with one criterion not met. The live path-MTU case moves to #1237; the rest have evidence. #1208 ends an HTTP/3 tunnel whose path can't carry 1280 bytes and fixes the HTTP/2 Packet Too Big size. #1209 refuses an attachment MTU above the underlay ceiling. #1240 adds the MTU rows to the containerlab lab suite.

    Criterion Evidence Status
    1280-byte packets both ways on both transports against real workloads mtu-1280-both: on HTTP/3 and HTTP/2, 500/500 1280-byte replies from the edge-node workload, the compute-node workload and an off-fabric host through NAT, and 40/40 1280-byte pings from each workload to the connector Met
    A live path-MTU reduction on a QUIC tunnel gets ICMPv6 feedback or ends the tunnel, not a black hole mtu-live-drop fails. With the provider's host-side MTU at 1360 to 1420, 1400-byte packets are dropped with no Packet Too Big. At 1300 to 1340, the tunnel stays open and delivers nothing, and no path_mtu end is counted Not met, moved to #1237
    Packets larger than the tunnel or downstream MTU take the error path Unit tests cover Packet Too Big on both transports and the tunnel end below 1280 bytes when the size is known (#1208). The live case is the failure above Met for a known path size; live case moved to #1237
    Tests record inner size, path MTU and behavior on local and remote paths mtu-1280-both and mtu-live-drop record them for the edge-node, compute-node and NAT paths Met
    #989's attachment MTU ceiling; local NAT egress and remote paths #1209 refuses an MTU above the underlay ceiling with MTUExceedsUnderlay, and the edge conformance suite passed all 24 rows at the lab's new 1460 MTU. NAT and remote paths are in mtu-1280-both Met

    #1237 has the reproduction and stays open under #970, so the epic can't close until a live MTU drop is handled. #1211 to #1214 stay open as hardening for the edge MTU work.

  12. privateip
    closed this as completedon Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

Task

Fields

Priority

None yet

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions