Implement #160 Related Origins support (WebAuthn L3 § 5.11)#173
Implement #160 Related Origins support (WebAuthn L3 § 5.11)#173HarveyOrourke15 wants to merge 5 commits intolinux-credentials:masterfrom
Conversation
- Introduced `validate_related_origins` function to validate related origins according to WebAuthn §5.11.1. - Implemented `RelatedOriginsHttpClient` trait for pluggable HTTP client functionality. - Added `ReqwestRelatedOriginsClient` for fetching the well-known JSON document. - Created `RelatedOriginsDocument` struct to deserialize the JSON response. - Added error handling for various failure scenarios in related origins validation. - Implemented unit tests for `validate_related_origins` with mock client. - Updated `RelyingPartyId` struct to include validation against origins. - Added dependencies for `reqwest`, `url`, `bytes`, and `publicsuffix` in `Cargo.toml`.
|
Thank you for your work on this @HarveyOrourke15! We'll try and review this as soon as possible. |
There was a problem hiding this comment.
Pull request overview
Adds WebAuthn L3 §5.11 “Related Origins” validation to libwebauthn, including a default reqwest-backed fetcher, domain label extraction helper, and new tests/examples.
Changes:
- Introduces
ops::webauthn::related_originsmodule with a public validation API and pluggable HTTP client trait. - Implements
ReqwestRelatedOriginsClientwith HTTPS + content-type/length + streaming caps. - Adds new dependencies (
reqwest,bytes,url,publicsuffix), plus a debug example and an integration test.
Reviewed changes
Copilot reviewed 6 out of 8 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
libwebauthn/src/ops/webauthn/related_origins/mod.rs |
Core validator + error types + registrable label helper + unit tests |
libwebauthn/src/ops/webauthn/related_origins/client.rs |
Reqwest-based well-known document fetcher |
libwebauthn/src/ops/webauthn/mod.rs |
Exposes related_origins module |
libwebauthn/tests/related_origins.rs |
Adds an Amazon-related integration test |
libwebauthn/examples/debug_related_origins.rs |
Adds a manual/debug executable for validation |
libwebauthn/Cargo.toml |
Adds new dependencies for fetching/parsing |
libwebauthn/Cargo.lock |
Locks transitive deps for the new crates |
.gitignore |
Ignores VS Code launch config |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if let Some(list_opt) = PS_LIST.get_or_init(|| publicsuffix::List::fetch().ok()) { | ||
| if let Ok(parsed) = list_opt.parse_domain(host) { | ||
| // If the parsed result provides the registrable root (eTLD+1), | ||
| // take the left-most label of that root as the registrable label. | ||
| if let Some(root) = parsed.root() { |
There was a problem hiding this comment.
publicsuffix::List::fetch() downloads the PSL over the network at runtime on first call. This can block the async runtime and introduces nondeterministic latency/failures (and may be disallowed in many deployments). Consider bundling a PSL snapshot with the crate (or using a non-network API from publicsuffix) so label extraction is offline and deterministic.
There was a problem hiding this comment.
We had this discussion before, with no clear solution. The list changes too often to rely only on compile time-snapshots. A hybrid approach would be thinkable, where we use both, try to fetch an updated version, and if that fails, fall back to the compile-time list.
But that would also increase the dependency tree and general bloaty-ness.
@AlfioEmanueleFresta , do you have a strong opinion on this either way?
There was a problem hiding this comment.
Fetching a new list upon each request is expensive. A very outdated PSL is a security risk. I don't think we should try and manage a PSL ourselves if we can at all avoid it.
I think the ideal solution would be to use the system-managed PSL in /usr/share/publicsuffix. From a quick search, it seems to be widely available on most Linux distros. It's updated via package managers. We could fallback to downloading our own, but this should be cached for a reasonable time. I'm not sure what time time should be.
A compromise might be that of using a package like public-suffix, which ships with its own PSL. However, it doesn't seem to be updated very frequently (last update 10 months ago) and it also relies on the version of libwebauthn and publix-suffix in the chain of dependencies, which can get obsolete quickly.
WDYT?
There was a problem hiding this comment.
I think the ideal solution would be to use the system-managed PSL in /usr/share/publicsuffix. From a quick search, it seems to be widely available on most Linux distros. It's updated via package managers. We could fallback to downloading our own, but this should be cached for a reasonable time. I'm not sure what time time should be.
I think using this package makes sense, but I think reading this file belongs in the credentialsd layer, not libwebauthn. For example, if someone wanted to build a browser, they probably already have a method to determine PSLs, so they would want to leverage their own infrastructure rather than requiring ours. In other words, I think libwebauthn could:
- accept the public suffix list in a specific format, or
- use a callback trait to allow the caller to verify a specific domain,
and leave it up to callers to provide the PSL verification. If necessary, we could provide a default PSL implementation under a non-default feature flag as a convenience.
There was a problem hiding this comment.
I agree with @iinuwa:
- The client should be able to override PSL behaviour.
- I am also keen for
libwebauthnto provide a safe, and implementation for the PSL.
I think a good compromise would be to:
- Define a
PublicSuffixListtrait, with a singleasyncmethod to get the registrable domain as needed by the spec; - Have
libwebauthninclude a simple implementation, based onpublicsuffixreading a PSL from a static DAT file path.
E.g.:
#[async_trait]
pub trait PublicSuffixList: Send + Sync {
/// Returns the registrable domain (eTLD+1) for the given host, if any.
async fn registrable_domain(&self, host: &str) -> Option<String>;
}And:
pub struct DatFilePublicSuffixList {
fn new(path: &PathBuf) -> Option<Self> { ... }
fn from_system_file() -> Option<Self> {
Self::new(PathBuf::from("/usr/share/publicsuffix/public_suffix_list.dat"))
}
}Clients like credentialsd can then decide whether to:
- (a) Depend on the system package which provides the DAT file at a known location, using libwebauthn's verifier with the appropriate file path, or
- (b) to provide theyr own implementation, which downloads & caches a PSL file
As for this specific PR, I think it would be fine just to implement and use the DatFilePublicSuffixList DAT-based PSL verifier component.
Building a PublicSuffixList trait, and allowing specifying/providing a PSL can be done as a follow up - by the same author or someone else.
WDYT?
| use std::time::Duration; | ||
| use std::sync::OnceLock; | ||
|
|
||
| use bytes::BytesMut; | ||
| use futures::TryStreamExt; | ||
|
|
There was a problem hiding this comment.
These imports are only used by the client submodule, not within this module, so they will trigger unused_imports warnings. Prefer moving the Duration/BytesMut/TryStreamExt imports into client.rs (and avoiding use super::* there) to keep module-level imports warning-free.
| use std::time::Duration; | |
| use std::sync::OnceLock; | |
| use bytes::BytesMut; | |
| use futures::TryStreamExt; | |
| use std::sync::OnceLock; |
| let client = match client::ReqwestRelatedOriginsClient::new() { | ||
| Ok(c) => c, | ||
| Err(e) => { | ||
| eprintln!("related_origins: failed to create HTTP client: {:?}", e); | ||
| return Err(e); |
There was a problem hiding this comment.
Library code prints directly to stderr via eprintln!, which is inconsistent with the rest of the crate’s tracing-based logging and makes output harder to control for consumers. Consider using tracing::error! (or removing the logging and just returning the error).
There was a problem hiding this comment.
Correct. Please replace eprintln!() with tracing.
| // Client implementation is kept in the `client` submodule. | ||
| pub mod client; | ||
|
|
||
| /// Implimentation of https://url.spec.whatwg.org/#host-public-suffix |
There was a problem hiding this comment.
Typo in doc comment: "Implimentation" should be "Implementation".
| /// Implimentation of https://url.spec.whatwg.org/#host-public-suffix | |
| /// Implementation of https://url.spec.whatwg.org/#host-public-suffix |
| // Fetch the well-known URL for the RP ID | ||
| let response = client.fetch_well_known(rp_id_requested).await.map_err(|_| RelatedOriginsError::SecurityError)?; | ||
|
|
There was a problem hiding this comment.
The fetch_well_known error is always mapped to SecurityError, discarding the underlying RelatedOriginsError and making HttpError effectively unreachable for request/response failures. If the spec requires returning SecurityError, consider at least logging the underlying error (via tracing) for diagnostics, or otherwise preserve it for callers of the lower-level API.
There was a problem hiding this comment.
Agreed, tracing the error details would be good.
| async fn fetch_well_known(&self, rp_id: &str) -> Result<RelatedOriginsHttpResponse, RelatedOriginsError> { | ||
| let url = format!("https://{}/.well-known/webauthn", rp_id); | ||
|
|
||
| // When issuing the request, avoid cookies/auth and set no Referer: | ||
| let req = self.inner.get(&url).header(reqwest::header::REFERER, ""); |
There was a problem hiding this comment.
rp_id is interpolated directly into a URL string. Without validating/normalizing it, a caller could pass values containing characters like @, /, or : to alter the request target (SSRF / request smuggling style issues). Consider validating rp_id using the existing RelyingPartyId::try_from domain checks (and using the normalized ASCII value) before constructing the URL.
There was a problem hiding this comment.
additionally: Since this is something all clients would need to do, I wonder if we shouldn't just do this on the outside and pass the final well-known url in here.
| serde_json = "1.0.141" | ||
| url = "2.4.0" | ||
| publicsuffix = "1.5" | ||
| reqwest = { version = "0.11", features = ["rustls-tls", "stream"] } |
There was a problem hiding this comment.
reqwest is added with rustls-tls, but without default-features = false it will still pull in default TLS/native-tls and extra dependencies. If the intent is rustls-only (and to avoid OpenSSL/native TLS in builds), set default-features = false and enable only the needed features.
| reqwest = { version = "0.11", features = ["rustls-tls", "stream"] } | |
| reqwest = { version = "0.11", default-features = false, features = ["rustls-tls", "stream"] } |
There was a problem hiding this comment.
I don't think that is true for newer reqwest-versions. Somebody would need to verify that.
| async fn amazon_validate_related_origins() -> Result<(), Box<dyn std::error::Error>> { | ||
| let caller_origin = "https://www.amazon.com"; | ||
| let rp_id = "amazon.com"; | ||
|
|
||
| let res = validate_related_origins(caller_origin, rp_id).await?; |
There was a problem hiding this comment.
This test performs a live network call to amazon.com, which will be flaky (DNS/timeout/rate limiting) and can fail in offline/sandboxed CI environments. Consider marking it #[ignore], gating behind an env/feature flag, or replacing it with a deterministic test using a mocked HTTP client / local test server.
There was a problem hiding this comment.
Hm, if we simply mark it #[ignore], it will just never get called/tested. I'm not sure how often it would actually fail in CI. @AlfioEmanueleFresta thoughts?
There was a problem hiding this comment.
I don't think our CI has network access other than crates.io (or, I hope it doesn't :)).
I would rather we build a test implementation of the HTTP client trait, and provide a mock list. This would allow tests to run automatically in CI, without an external dependency.
We should also avoid using real domain names in favour of reserved example domains.
| struct PolicyConfig { | ||
| pub max_body_bytes: usize, | ||
| pub max_origins: usize, | ||
| pub max_origin_len: usize, | ||
| pub max_labels: usize, |
There was a problem hiding this comment.
PolicyConfig::max_body_bytes is defined, but the validator never enforces it (it relies on the reqwest client implementation). Since RelatedOriginsHttpClient is pluggable, consider enforcing response.body.len() <= max_body_bytes here (or removing the field if it’s intentionally client-only) so limits are consistent across implementations.
msirringhaus
left a comment
There was a problem hiding this comment.
Thanks for this! Looking good overall!
I left comments to some minor things
| let resp = req.send().await.map_err(|e| RelatedOriginsError::HttpError(e.to_string()))?; | ||
|
|
||
| let status = resp.status(); | ||
| if status != StatusCode::OK { |
There was a problem hiding this comment.
All these checks happen outside in validate_related_origins_with_client() as well (status code, content type, content length, ..). Is there a reason to duplicate this here?
| struct PolicyConfig { | ||
| pub max_body_bytes: usize, | ||
| pub max_origins: usize, | ||
| pub max_origin_len: usize, | ||
| pub max_labels: usize, |
| async fn fetch_well_known(&self, rp_id: &str) -> Result<RelatedOriginsHttpResponse, RelatedOriginsError> { | ||
| let url = format!("https://{}/.well-known/webauthn", rp_id); | ||
|
|
||
| // When issuing the request, avoid cookies/auth and set no Referer: | ||
| let req = self.inner.get(&url).header(reqwest::header::REFERER, ""); |
There was a problem hiding this comment.
additionally: Since this is something all clients would need to do, I wonder if we shouldn't just do this on the outside and pass the final well-known url in here.
| // Fetch the well-known URL for the RP ID | ||
| let response = client.fetch_well_known(rp_id_requested).await.map_err(|_| RelatedOriginsError::SecurityError)?; | ||
|
|
There was a problem hiding this comment.
Agreed, tracing the error details would be good.
| let response = client.fetch_well_known(rp_id_requested).await.map_err(|_| RelatedOriginsError::SecurityError)?; | ||
|
|
||
| if response.status != 200 { | ||
| return Err(RelatedOriginsError::SecurityError); |
There was a problem hiding this comment.
tracing-calls here and below, for each early return would be good, to pinpoint errors quickly.
| let doc = r#"{"origins":["https://different.example.co.uk"]}"#; | ||
| let client = MockClient { body: doc.into() }; | ||
| let res = validate_related_origins_with_client("https://caller.example.com", "example.com", &client).await.unwrap(); | ||
| assert!(!res); |
| let doc = r#"{"origins":[]}"#; | ||
| let client = MockClient { body: doc.into() }; | ||
| let res = validate_related_origins_with_client("https://caller.example.com", "example.com", &client).await; | ||
| match res { |
There was a problem hiding this comment.
assert_eq!(res, Err(RelatedOriginsError::SecurityError)) works here as well and is far shorter than the match. (One can also add additional output after, like assert_eq!(res, Err(..), "Needs to be SecurityError, because of..").)
| let doc = r#"{"origins":["http://example.com"]}"#; | ||
| let client = MockClient { body: doc.into() }; | ||
| let res = validate_related_origins_with_client("https://caller.example.com", "example.com", &client).await; | ||
| match res { |
There was a problem hiding this comment.
Same here. assert_eq!(res, Err(RelatedOriginsError::SecurityError), "expected SecurityError for non-https origin, got {:?}", res) should do the trick
| let doc = serde_json::json!({"origins": [origin]}).to_string(); | ||
| let client = MockClient { body: doc }; | ||
| let res = validate_related_origins_with_client("https://caller.example.com", "example.com", &client).await; | ||
| match res { |
| if let Some(list_opt) = PS_LIST.get_or_init(|| publicsuffix::List::fetch().ok()) { | ||
| if let Ok(parsed) = list_opt.parse_domain(host) { | ||
| // If the parsed result provides the registrable root (eTLD+1), | ||
| // take the left-most label of that root as the registrable label. | ||
| if let Some(root) = parsed.root() { |
There was a problem hiding this comment.
We had this discussion before, with no clear solution. The list changes too often to rely only on compile time-snapshots. A hybrid approach would be thinkable, where we use both, try to fetch an updated version, and if that fails, fall back to the compile-time list.
But that would also increase the dependency tree and general bloaty-ness.
@AlfioEmanueleFresta , do you have a strong opinion on this either way?
|
My apologies for not being able to look into this more right now, but I'd like to suggest that we implement some sort of caching policy for validating related origins. One could imagine that if someone cancels a WebAuthn flow on a related origin request because they need to use a different authenticator, we don't want to make them fetch the related origins list again within seconds of each other. |
AlfioEmanueleFresta
left a comment
There was a problem hiding this comment.
Thank you for all your work so far on this @HarveyOrourke15!
| if let Some(list_opt) = PS_LIST.get_or_init(|| publicsuffix::List::fetch().ok()) { | ||
| if let Ok(parsed) = list_opt.parse_domain(host) { | ||
| // If the parsed result provides the registrable root (eTLD+1), | ||
| // take the left-most label of that root as the registrable label. | ||
| if let Some(root) = parsed.root() { |
There was a problem hiding this comment.
I agree with @iinuwa:
- The client should be able to override PSL behaviour.
- I am also keen for
libwebauthnto provide a safe, and implementation for the PSL.
I think a good compromise would be to:
- Define a
PublicSuffixListtrait, with a singleasyncmethod to get the registrable domain as needed by the spec; - Have
libwebauthninclude a simple implementation, based onpublicsuffixreading a PSL from a static DAT file path.
E.g.:
#[async_trait]
pub trait PublicSuffixList: Send + Sync {
/// Returns the registrable domain (eTLD+1) for the given host, if any.
async fn registrable_domain(&self, host: &str) -> Option<String>;
}And:
pub struct DatFilePublicSuffixList {
fn new(path: &PathBuf) -> Option<Self> { ... }
fn from_system_file() -> Option<Self> {
Self::new(PathBuf::from("/usr/share/publicsuffix/public_suffix_list.dat"))
}
}Clients like credentialsd can then decide whether to:
- (a) Depend on the system package which provides the DAT file at a known location, using libwebauthn's verifier with the appropriate file path, or
- (b) to provide theyr own implementation, which downloads & caches a PSL file
As for this specific PR, I think it would be fine just to implement and use the DatFilePublicSuffixList DAT-based PSL verifier component.
Building a PublicSuffixList trait, and allowing specifying/providing a PSL can be done as a follow up - by the same author or someone else.
WDYT?
I agree with your concern re. the performance of repeat requests. However, as the spec for validating related origins is non-trivial, and is part of WebAuthn, IMO it belongs to As the most expensive part of the computation is retrieving the well-known list, I believe most of the performance concerns should be addressed by allowing the client to provide its own HTTP client/request logic - as intended by this PR and #160. The caller would be free to implement caching policies as it best sees fit, based on HTTP response. Crucially, they would not need to re-implement related origin validation. WDYT? |
This pull request introduces a new implementation for validating related origins in accordance with the WebAuthn specification (§5.11.1).
related_originsmodule tolibwebauthn.validate_related_origins) and a pluggable HTTP client trait (RelatedOriginsHttpClient) for extensible validation and document fetching.ReqwestRelatedOriginsClientusing thereqwestcrate, enforces HTTPS, content type, content length, and streaming limits for secure fetching of well-known documents.reqwest,bytes,url, andpublicsuffixtoCargo.tomlto support HTTP requests, streaming, URL parsing, and domain label extraction.Testing and examples:
debug_related_origins.rs) demonstrating environment-driven validation calls for debugging and manual testing.