Skip to content

refactor(multipath): challenges on a path generation are on path#712

Closed
flub wants to merge 1 commit into
mainfrom
flub/path-challenge-on-path
Closed

refactor(multipath): challenges on a path generation are on path#712
flub wants to merge 1 commit into
mainfrom
flub/path-challenge-on-path

Conversation

@flub

@flub flub commented Jun 17, 2026

Copy link
Copy Markdown
Collaborator

Description

The token of any PATH_CHALLENGE sent as a on-path challenge is stored
on the path generation state (PathData). Thus any response that
matches the token is always on path. The PathData::network_path of a
path generation can not change, only the local_ip can be filled in if
initially missing.

This may not have been the case when this was originally introduced.

Breaking Changes

none

Notes & open questions

This is the more controversial part of #711, so targets that PR. This
state was originally introduced in #443.

Change checklist

  • Self-review.
  • Documentation updates following the style guide, if relevant.

@github-actions

github-actions Bot commented Jun 17, 2026

Copy link
Copy Markdown

Documentation for this PR has been generated and is available at: https://n0-computer.github.io/noq/pr/712/docs/noq/

Last updated: 2026-06-18T09:42:38Z

@github-actions

github-actions Bot commented Jun 17, 2026

Copy link
Copy Markdown

Performance Comparison Report

1a1234d34f66cb3b2a4dbe525aa402f4edaa0463 - artifacts

Raw Benchmarks (localhost)

Scenario noq upstream Delta CPU (avg/max)
large-single 5751.6 Mbps 8042.4 Mbps -28.5% 92.4% / 97.7%
medium-concurrent 5503.7 Mbps 7809.1 Mbps -29.5% 92.7% / 99.0%
medium-single 3957.9 Mbps 4749.1 Mbps -16.7% 93.7% / 102.0%
small-concurrent 3798.2 Mbps 5160.1 Mbps -26.4% 97.2% / 155.0%
small-single 3543.5 Mbps 4841.6 Mbps -26.8% 89.2% / 97.2%

Netsim Benchmarks (network simulation)

Condition noq upstream Delta
ideal 3057.6 Mbps 4092.8 Mbps -25.3%
lan 782.4 Mbps 810.4 Mbps -3.5%
lossy 69.8 Mbps 69.8 Mbps ~0%
wan 83.8 Mbps 83.8 Mbps ~0%

Summary

noq is 25.5% slower on average

---
8d9cdcab6fc1c9a5f4a805defbb05333cb52f58b - artifacts

Raw Benchmarks (localhost)

Scenario noq upstream Delta CPU (avg/max)
large-single 5328.5 Mbps 7990.3 Mbps -33.3% 98.6% / 149.0%
medium-concurrent 5512.7 Mbps 7809.9 Mbps -29.4% 93.9% / 99.0%
medium-single 4049.0 Mbps 4749.5 Mbps -14.7% 92.4% / 101.0%
small-concurrent 3805.8 Mbps 5089.2 Mbps -25.2% 99.5% / 154.0%
small-single 3439.1 Mbps 4824.8 Mbps -28.7% 98.0% / 153.0%

Netsim Benchmarks (network simulation)

Condition noq upstream Delta
ideal 3045.1 Mbps 4009.0 Mbps -24.0%
lan 782.4 Mbps 810.3 Mbps -3.5%
lossy 69.9 Mbps 55.9 Mbps +25.0%
wan 83.8 Mbps 83.8 Mbps ~0%

Summary

noq is 26.3% slower on average

---
bc32f38a9c268f2034ef3bc8dcfe2b0f5a21b1ca - artifacts

Raw Benchmarks (localhost)

Scenario noq upstream Delta CPU (avg/max)
large-single 5353.3 Mbps 7832.6 Mbps -31.7% 95.2% / 100.0%
medium-concurrent 5414.2 Mbps 7694.6 Mbps -29.6% 96.5% / 150.0%
medium-single 3817.4 Mbps 4749.1 Mbps -19.6% 88.6% / 96.5%
small-concurrent 3702.1 Mbps 5318.2 Mbps -30.4% 94.9% / 103.0%
small-single 3405.9 Mbps 4796.9 Mbps -29.0% 94.5% / 103.0%

Netsim Benchmarks (network simulation)

Condition noq upstream Delta
ideal 3071.0 Mbps N/A N/A
lan 796.4 Mbps N/A N/A
lossy 69.9 Mbps N/A N/A
wan 83.8 Mbps N/A N/A

Summary

noq is 28.6% slower on average

Comment on lines +520 to +525
// An on-path challenge that was sent on a different 4-tuple would be stored
// in a different path generation.
debug_assert!(
info.network_path.is_probably_same_path(&self.network_path),
"response can only be for same network path"
);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sooo, I asked GLM 5.2 to review this assert (yes I'm still testing various open-weight models 🙃), and I think it's correct in saying that it is possible that this assert will fail:

  • Connection::handle_network_change will reset PathData::network_path::local_ip to None if the path is deemed recoverable.
  • info.network_path can have local_ip set to Some(X), with self.network_path.remote matching.
  • Assuming the remote addresses match, info.network_path.is_probably_same_path(&self.network_path) simplifies to info.network_path.local_ip.is_none() || info.network_path.local_ip == self.network_path.local_ip which simplifies to info.network_path.local_ip == self.network_path.local_ip which is false, because the left one is Some(X) and the right one was cleared to None by handle_network_change.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this is weird. iroh's use of the handle_network_change API explicitly says a path is recoverable if the local IP still exists. Yet in the noq implementation we clear the local_ip for the recoverable paths.

Should we just not clear the local_ip for recoverable paths?

Should handle_network_change create a new path generation with PathData::from_previous if the path is recoverable?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO setting local_ip = None in handle_network_change makes sense. But the check in iroh doesn't. The idea of handling the network change gracefully is to be able to re-use a path that existed on a 4-tuple with a different local interface. E.g. someone might've switched from WiFi to Cellular and thus lost their Wifi interface, but gained the Cellular one, which became the OS default interface. So when the Wifi path's local IP gets set to None, the OS chooses the cellular interface for the next send and the path migrates. That seems like it's the correct way to handle recoverable paths.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given how broken iroh is, this makes me wonder if we can remove all of the network change hint stuff in iroh and everything will keep working 👀
Let me try that :D

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The purpose was to switch between closing the old and new paths if the 4-tuples changed, which is what multipath says you should do, and try and keep the same path in case the 4-tuple hasn't changed. So to me iroh is doing fine, but noq clearing the local IP is not.

I just pushed a commit that removes the clearing of the local IP and patchbay is happy with it. So I'm currently inclined to think that this is reasonable.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like patchbay doesn't care about calling noq's handle_network_change

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's another case where local_ip might not match. At the end of process_payload the local_ip is updated for any non-probing packet which has a different local_ip. That means the local_ip can be different all the time during process_payload

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened n0-computer/iroh#4356 for discussion

@n0bot n0bot Bot added this to iroh Jun 17, 2026
@github-project-automation github-project-automation Bot moved this from 🚑 Needs Triage to 🏗 In progress in iroh Jun 17, 2026
@github-project-automation github-project-automation Bot moved this to 🚑 Needs Triage in iroh Jun 17, 2026
Base automatically changed from flub/open-status-pns to main June 18, 2026 09:32
The token of any PATH_CHALLENGE sent as a on-path challenge is stored
on the path generation state (PathData). Thus any response that
matches the token is always on path. The PathData::network_path of a
path generation can not change, only the local_ip can be filled in if
initially missing.

This may not have been the case when this was originally introduced.
@flub
flub force-pushed the flub/path-challenge-on-path branch from 8d9cdca to bc32f38 Compare June 18, 2026 09:40
@flub

flub commented Jun 18, 2026

Copy link
Copy Markdown
Collaborator Author

So, I believe this is wrong. Packets are processed before migration and local_ip updates are handled. This means while processing a packet you always have a valid packet from a peer but the 4-tuples may not match. For path validation this indeed makes a difference: you can receive valid responses that do not validate the path if your local IP changed in between. I believe there's still a bug, but it is so different that I want to create a new PR.

@flub flub closed this Jun 18, 2026
@github-project-automation github-project-automation Bot moved this from 🏗 In progress to ✅ Done in iroh Jun 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

2 participants