feat(proto): MTU discovery probes should be congestion controlled by flub · Pull Request #573 · n0-computer/noq

flub · 2026-04-09T10:28:58Z

Description

Previously we sent MTU probes without checking congestion control
limits. But at the same time they are accounted as bytes-in-flight,
which means you can easily go over the congestion control window if
you lose connectivity during MTU probing. Which in turn makes
recovering from that congestion slower because more bytes are
considered to be in-flight that need to arrive or be declared lost
before the window grows enough again.

This had to split up figuring out the next MTU probe to send from
considering it being sent. Because we need to know the size for the
congestion controller check, but if it fails that check we can not
send the probe.

Breaking Changes

n/a

Notes & open questions

Based on #595, that needs to merge first.

github-actions · 2026-04-09T10:31:33Z

Documentation for this PR has been generated and is available at: https://n0-computer.github.io/noq/pr/573/docs/noq/

Last updated: 2026-04-16T08:50:13Z

github-actions · 2026-04-09T10:42:55Z

Performance Comparison Report

6fcf4306b7beb5e950348fab668e1213574f41cb - artifacts

Raw Benchmarks (localhost)

Scenario	noq	upstream	Delta	CPU (avg/max)
large-single	5468.1 Mbps	7939.1 Mbps	-31.1%	93.9% / 99.6%
medium-concurrent	5329.0 Mbps	7743.5 Mbps	-31.2%	94.6% / 124.0%
medium-single	4221.2 Mbps	4662.0 Mbps	-9.5%	86.7% / 96.0%
small-concurrent	3849.1 Mbps	5239.0 Mbps	-26.5%	101.8% / 190.0%
small-single	3533.4 Mbps	4662.7 Mbps	-24.2%	88.4% / 96.6%

Netsim Benchmarks (network simulation)

Condition	noq	upstream	Delta
ideal	2863.1 Mbps	4008.9 Mbps	-28.6%
lan	768.5 Mbps	810.4 Mbps	-5.2%
lossy	69.8 Mbps	69.9 Mbps	~0%
wan	83.8 Mbps	83.8 Mbps	~0%

Summary

noq is 25.6% slower on average

---

d674ba9e427384ba37e8d180824f045534223494 - artifacts

Raw Benchmarks (localhost)

Scenario	noq	upstream	Delta	CPU (avg/max)
large-single	6010.6 Mbps	8133.5 Mbps	-26.1%	96.9% / 98.5%
medium-concurrent	5349.2 Mbps	7551.2 Mbps	-29.2%	95.6% / 97.5%
medium-single	4126.9 Mbps	4189.6 Mbps	-1.5%	96.1% / 98.4%
small-concurrent	3968.2 Mbps	4949.0 Mbps	-19.8%	97.3% / 99.6%
small-single	3625.8 Mbps	4443.5 Mbps	-18.4%	95.9% / 98.3%

Netsim Benchmarks (network simulation)

Condition	noq	upstream	Delta
ideal	3100.8 Mbps	3608.9 Mbps	-14.1%
lan	782.4 Mbps	796.4 Mbps	-1.8%
lossy	69.9 Mbps	55.9 Mbps	+25.0%
wan	83.8 Mbps	83.8 Mbps	~0%

Summary

noq is 19.8% slower on average

---

768a1cc8012d77a97301be70011aba94ad4f2254 - artifacts

Raw Benchmarks (localhost)

Scenario	noq	upstream	Delta	CPU (avg/max)
large-single	5797.2 Mbps	7859.8 Mbps	-26.2%	97.8% / 149.0%
medium-concurrent	5372.5 Mbps	7730.2 Mbps	-30.5%	96.6% / 147.0%
medium-single	4123.9 Mbps	4633.3 Mbps	-11.0%	91.9% / 101.0%
small-concurrent	3947.1 Mbps	5378.4 Mbps	-26.6%	94.7% / 102.0%
small-single	3612.1 Mbps	4706.7 Mbps	-23.3%	97.4% / 152.0%

Netsim Benchmarks (network simulation)

Condition	noq	upstream	Delta
ideal	3098.3 Mbps	4083.3 Mbps	-24.1%
lan	796.4 Mbps	810.3 Mbps	-1.7%
lossy	69.8 Mbps	69.8 Mbps	~0%
wan	83.8 Mbps	83.8 Mbps	~0%

Summary

noq is 23.9% slower on average

---

967299f4bd8e51056c4c1cdfc8e70fd96e6c67c2 - artifacts

No results available

---

9ad95a86ee20d477086556fb780bf1dc0c52caf3 - artifacts

Raw Benchmarks (localhost)

Scenario	noq	upstream	Delta	CPU (avg/max)
large-single	5596.8 Mbps	8042.1 Mbps	-30.4%	97.1% / 98.5%
medium-concurrent	5314.3 Mbps	7765.6 Mbps	-31.6%	95.9% / 97.8%
medium-single	3842.2 Mbps	4748.9 Mbps	-19.1%	96.0% / 98.0%
small-concurrent	3842.9 Mbps	5187.4 Mbps	-25.9%	97.3% / 99.5%
small-single	3548.9 Mbps	4821.3 Mbps	-26.4%	96.7% / 100.0%

Netsim Benchmarks (network simulation)

Condition	noq	upstream	Delta
ideal	3114.5 Mbps	N/A	N/A
lan	782.4 Mbps	N/A	N/A
lossy	69.8 Mbps	N/A	N/A
wan	83.8 Mbps	N/A	N/A

Summary

noq is 27.5% slower on average

---

6391dea8e6a5e7ae4199fd87c8b6577d6d00ede2 - artifacts

Raw Benchmarks (localhost)

Scenario	noq	upstream	Delta	CPU (avg/max)
large-single	5335.0 Mbps	7835.7 Mbps	-31.9%	95.0% / 146.0%
medium-concurrent	5353.3 Mbps	7418.7 Mbps	-27.8%	96.4% / 147.0%
medium-single	4045.9 Mbps	4527.8 Mbps	-10.6%	89.1% / 97.0%
small-concurrent	3766.4 Mbps	5279.3 Mbps	-28.7%	94.8% / 102.0%
small-single	3511.4 Mbps	4699.9 Mbps	-25.3%	94.9% / 151.0%

Netsim Benchmarks (network simulation)

Condition	noq	upstream	Delta
ideal	3002.9 Mbps	N/A	N/A
lan	782.4 Mbps	N/A	N/A
lossy	69.8 Mbps	N/A	N/A
wan	0.0 Mbps	N/A	N/A

Summary

noq is 26.0% slower on average

---

5fb5b6a1799adc4242a0372acb9da18a9fb2da56 - artifacts

Raw Benchmarks (localhost)

Scenario	noq	upstream	Delta	CPU (avg/max)
large-single	5647.6 Mbps	7927.0 Mbps	-28.8%	97.9% / 148.0%
medium-concurrent	5330.2 Mbps	7646.9 Mbps	-30.3%	96.9% / 147.0%
medium-single	4141.4 Mbps	4748.3 Mbps	-12.8%	92.5% / 100.0%
small-concurrent	3933.2 Mbps	5262.3 Mbps	-25.3%	95.2% / 102.0%
small-single	3561.4 Mbps	4786.0 Mbps	-25.6%	98.5% / 151.0%

Netsim Benchmarks (network simulation)

Condition	noq	upstream	Delta
ideal	2983.8 Mbps	3980.5 Mbps	-25.0%
lan	782.4 Mbps	810.3 Mbps	-3.4%
lossy	69.8 Mbps	55.9 Mbps	+25.0%
wan	83.8 Mbps	83.8 Mbps	~0%

Summary

noq is 24.8% slower on average

---

9477186fea5e64ba0cffe386940afed9f74e7662 - artifacts

Raw Benchmarks (localhost)

Scenario	noq	upstream	Delta	CPU (avg/max)
large-single	5639.9 Mbps	7935.0 Mbps	-28.9%	95.1% / 100.0%
medium-concurrent	5293.3 Mbps	7892.8 Mbps	-32.9%	94.5% / 98.1%
medium-single	3715.4 Mbps	4632.3 Mbps	-19.8%	94.8% / 149.0%
small-concurrent	3909.6 Mbps	5208.1 Mbps	-24.9%	101.0% / 154.0%
small-single	3528.3 Mbps	4696.1 Mbps	-24.9%	92.7% / 102.0%

Netsim Benchmarks (network simulation)

Condition	noq	upstream	Delta
ideal	N/A	4078.7 Mbps	N/A
lan	N/A	810.3 Mbps	N/A
lossy	N/A	69.9 Mbps	N/A
wan	N/A	83.8 Mbps	N/A

Summary

noq is 27.3% slower on average

---

1536900c490bec013585473c4ea6178bf9cc8514 - artifacts

No results available

---

b5943aa52538c03534fa8275cfb23ea4cc1d09bd - artifacts

Raw Benchmarks (localhost)

Scenario	noq	upstream	Delta	CPU (avg/max)
large-single	5431.8 Mbps	7981.6 Mbps	-31.9%	93.1% / 98.0%
medium-concurrent	5296.1 Mbps	7760.4 Mbps	-31.8%	91.3% / 96.9%
medium-single	3818.4 Mbps	4603.0 Mbps	-17.0%	93.6% / 99.9%
small-concurrent	3857.7 Mbps	5159.7 Mbps	-25.2%	98.5% / 152.0%
small-single	3438.4 Mbps	4668.3 Mbps	-26.3%	90.9% / 101.0%

Netsim Benchmarks (network simulation)

Condition	noq	upstream	Delta
ideal	3095.1 Mbps	4148.8 Mbps	-25.4%
lan	782.5 Mbps	810.4 Mbps	-3.4%
lossy	69.8 Mbps	55.9 Mbps	+25.0%
wan	0.0 Mbps	83.8 Mbps	-100.0%

Summary

noq is 26.9% slower on average

---

a39d6eece38699e5360f676ac6b3ea8512242559 - artifacts

Raw Benchmarks (localhost)

Scenario	noq	upstream	Delta	CPU (avg/max)
large-single	5538.0 Mbps	7916.8 Mbps	-30.0%	96.2% / 98.0%
medium-concurrent	5406.6 Mbps	7848.8 Mbps	-31.1%	96.1% / 97.9%
medium-single	4500.1 Mbps	4559.7 Mbps	-1.3%	95.8% / 98.1%
small-concurrent	3794.0 Mbps	5038.4 Mbps	-24.7%	97.5% / 99.6%
small-single	3522.2 Mbps	4648.5 Mbps	-24.2%	96.1% / 98.3%

Netsim Benchmarks (network simulation)

Condition	noq	upstream	Delta
ideal	2984.9 Mbps	4022.8 Mbps	-25.8%
lan	0.0 Mbps	810.4 Mbps	-100.0%
lossy	55.9 Mbps	55.9 Mbps	~0%
wan	83.8 Mbps	83.8 Mbps	~0%

Summary

noq is 26.0% slower on average

matheus23

Submitting because I have comments (I think I started reviewing this when it wasn't in draft mode, but I don't remember).

This seems to be a somewhat minimal change set that fixes tests: #578

You probably already know this. Maybe it's helpful, maybe not.

Previously we sent MTU probes without checking congestion control limits. But at the same time they are accounted as bytes-in-flight, which means you can easily go over the congestion control window if you lose connectivity during MTU probing. Which in turn makes recovering from that congestion slower because more bytes are considered to be in-flight that need to arrive or be declared lost before the window grows enough again.

(As long as `probe_in_flight` hasn't been called yet)

flub marked this pull request as draft April 9, 2026 10:29

n0bot bot added this to iroh Apr 9, 2026

github-project-automation bot moved this to 🚑 Needs Triage in iroh Apr 9, 2026

matheus23 reviewed Apr 10, 2026

View reviewed changes

Comment thread noq-proto/src/connection/mod.rs

Comment thread noq-proto/src/connection/mod.rs Outdated

matheus23 force-pushed the flub/congestion-controlled-mtu branch 2 times, most recently from 967299f to 9ad95a8 Compare April 15, 2026 15:16

This was referenced Apr 16, 2026

refactor(proto): Split up computing the required pacer delay from updating the pacer #595

Open

Don't set pacing timer in poll_transmit_mtu_probe #578

Closed

matheus23 force-pushed the flub/congestion-controlled-mtu branch from 9477186 to 1536900 Compare April 16, 2026 08:24

matheus23 changed the base branch from main to matheus23/split-pacer April 16, 2026 08:24

matheus23 marked this pull request as ready for review April 16, 2026 08:24

flub and others added 2 commits April 16, 2026 10:27

Write a test verifying that next_probe is idempotent

b5943aa

(As long as `probe_in_flight` hasn't been called yet)

matheus23 force-pushed the flub/congestion-controlled-mtu branch from 1536900 to b5943aa Compare April 16, 2026 08:28

matheus23 self-assigned this Apr 16, 2026

Make clippy happy

a39d6ee

Conversation

flub commented Apr 9, 2026 • edited by matheus23 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Breaking Changes

Notes & open questions

Uh oh!

github-actions bot commented Apr 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Apr 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Performance Comparison Report

Raw Benchmarks (localhost)

Netsim Benchmarks (network simulation)

Summary

Raw Benchmarks (localhost)

Netsim Benchmarks (network simulation)

Summary

Raw Benchmarks (localhost)

Netsim Benchmarks (network simulation)

Summary

Raw Benchmarks (localhost)

Netsim Benchmarks (network simulation)

Summary

Raw Benchmarks (localhost)

Netsim Benchmarks (network simulation)

Summary

Raw Benchmarks (localhost)

Netsim Benchmarks (network simulation)

Summary

Raw Benchmarks (localhost)

Netsim Benchmarks (network simulation)

Summary

Raw Benchmarks (localhost)

Netsim Benchmarks (network simulation)

Summary

Raw Benchmarks (localhost)

Netsim Benchmarks (network simulation)

Summary

Uh oh!

matheus23 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

flub commented Apr 9, 2026 •

edited by matheus23

Loading

github-actions bot commented Apr 9, 2026 •

edited

Loading

github-actions bot commented Apr 9, 2026 •

edited

Loading

matheus23 left a comment •

edited

Loading