Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
0ab059c
chore: claim session outcomes
kixelated Oct 16, 2026
37ebd83
feat(stats): count refused and failed sessions
kixelated Oct 16, 2026
7fbe2cd
Merge remote-tracking branch 'origin/main' into quest/m1/session-outc…
kixelated Oct 16, 2026
d1d7fd1
fix(net): keep virtual stream closure out of connection metadata
kixelated Oct 16, 2026
c93e963
fix(net): preserve the session boundary on local transport close
kixelated Oct 16, 2026
bab5bd5
Merge remote-tracking branch 'origin/main' into iterate-5659
kixelated Oct 16, 2026
74942bc
docs(stats): show outcome counters in the example and trim to reader-…
kixelated Oct 16, 2026
8e8a5e7
Merge remote-tracking branch 'origin/main' into merge-5659
kixelated Oct 16, 2026
d3c10b1
fix(net): classify moq-quic close kinds after the backend move
kixelated Oct 16, 2026
a739346
Merge remote-tracking branch 'origin/main' into merge-5659
kixelated Oct 16, 2026
50e067b
fix(relay): count failed SETUP replies and exhausted peer reconnects
kixelated Oct 16, 2026
4fd790a
Merge remote-tracking branch 'origin/main' into r-5659
kixelated Oct 17, 2026
ef8589b
test(uring): resolve crate::quic at the merged integration test root
kixelated Oct 17, 2026
a2131c0
Merge remote-tracking branch 'origin/main' into r-5659
kixelated Oct 17, 2026
7f347d2
fix(relay): classify a refreshed session's end by its union, not its …
kixelated Oct 17, 2026
1d4989d
fix(relay): count a lease that ends its session as a failure
kixelated Oct 17, 2026
1505ddf
Merge remote-tracking branch 'origin/main' into quest/m1/session-outc…
kixelated Oct 17, 2026
9ed7287
fix(relay): count an admitted session refused while preparing its lease
kixelated Oct 17, 2026
dcccb6e
fix(relay): count any typed revocation that races a clean close
kixelated Oct 17, 2026
e454fb8
docs(net): say which error a code-less close surfaces as
kixelated Oct 17, 2026
6cc4a7d
fix(relay): keep a lease ended by shutdown out of the failures
kixelated Oct 17, 2026
e75a0f1
fix(relay): count a refreshed union that ended unobserved, and any te…
kixelated Oct 17, 2026
1d1589a
fix(relay): classify an unobserved union end like Lease::ended
kixelated Oct 17, 2026
c88602a
Merge remote-tracking branch 'origin/main' into quest/m1/session-outc…
kixelated Oct 17, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions doc/bin/relay/http.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,10 @@ A few worth alerting on:
- `moq_relay_sessions_refused_total`, by `reason`. A rise in `unavailable`
points at the auth server; a rise in `refused` at the credentials clients
present. Each attempt counts, so a client that retries is counted again.
- `moq_relay_sessions_failed_total`, by `tier` and `kind` (`idle_timeout`,
`transport`, `application`, `expired`): admitted sessions that ended
abnormally. A clean close is not a failure. Reported only while
[stats](/bin/relay/config#stats) are enabled.
- `moq_relay_stale_bytes_total` and friends: content dropped for drifting past
a subscriber's latency budget.
- `moq_relay_draining_sessions`: sessions sent a GOAWAY during a
Expand Down
22 changes: 21 additions & 1 deletion doc/concept/stats.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,15 @@ should prefer the canonical name and fall back to the legacy one.
### Sessions

```json
{ "acme": { "requests": 3, "sessions_started": 12, "sessions_ended": 10, "sessions": 12, "sessions_closed": 10, "announces_peak": 40, "subscriptions_peak": 900 } }
{
"acme": {
"requests": 3, "sessions_started": 12, "sessions_ended": 10,
"sessions": 12, "sessions_closed": 10,
"sessions_refused": { "refused": 0, "unavailable": 0, "request": 0, "forbidden": 0, "lan": 0 },
"sessions_failed": { "idle_timeout": 1, "transport": 0, "application": 0, "expired": 0 },
"announces_peak": 40, "subscriptions_peak": 900
}
}
```

`sessions_started` and `sessions_ended` count connects and disconnects, whether
Expand All @@ -207,6 +215,18 @@ group on the tier for `totals.json`. `sessions` and `sessions_closed` are their
legacy spellings. A session moved to a new tier ends on the old one and starts
on the new.

`sessions_refused` counts sessions turned away before admission, by reason.
They never start a session. The relay's own refusals have no resolved root, so
they count under the empty root on the default tier; an embedder can count one
it refuses itself with `stats::Handle::refuse`, under the root and tier it
resolved.

`sessions_failed` counts admitted sessions that ended abnormally, by kind: a
subset of `sessions_ended`, so the live count is unchanged. A clean close by
either side, a drain, or a shutdown is not a failure. A grant that expires
mid-session counts as `expired`; one revoked or narrowed out from under it
counts as `application`. A reader treats a missing object as zeros.

`requests` counts admitted request/response exchanges under the token's root and
tier, including HTTP `/fetch` and `/announced`. They never start or end a session.
A fetch's delivered group bytes count against the broadcast path on that tier;
Expand Down
1 change: 0 additions & 1 deletion quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,6 @@ lands before the cut.
- [Generated C++ shape](/quest/m1/cpp-generated-shape.md) - `moq::Client` is the generated type itself and `moq::expected` is one type at every C++ standard
- [Generated C bindings](/quest/m1/c/README.md) - C generated from moq-ffi ships as `moq-c` 0.8.0 and replaces the hand-written libmoq
- [Rust PROBE on demand](/quest/m1/probe-on-demand-rs.md) - a Rust client opens PROBE only while a `recv_bandwidth()` consumer is held, and `Stats` drops `estimated_recv_rate`
- [Session outcomes](/quest/m1/session-outcomes.md) - the relay's session stats count refusals by reason and ends by kind, per root and tier, including refusals an embedder records on its stats handle
- [Typed refusal reason](/quest/m1/refusal-reason.md) - an auth server's 403 names the reason (including `expired`) and the root and tier, so the relay's session outcomes count and attribute refusals
- [JS track handover](/quest/m1/js-group-handover.md) - a JS track subscription resumes across a route swap from the first frame it lacks, so `test/drain` passes at zero latency budget
- [Scope track priority](/quest/m1/track-priority-scope.md) - priority orders one owner's streams, and a shared cluster session is fair across tenants
Expand Down
1 change: 0 additions & 1 deletion quest/m1/drain-cleanup.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,4 +89,3 @@ Wire: none.
## Related

- [Handshake deadline covers admission](/quest/m1/handshake-admission-deadline.md) - bounds pending admissions, which the drain deadline must keep cutting
- [Session outcomes](/quest/m1/session-outcomes.md) - counts how sessions end, which needs this end recorded
10 changes: 2 additions & 8 deletions quest/m1/refusal-reason.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ refusal to that root and tier.
Decided 2026-10-08: moved out of the auth line to m1, since it extends
session outcomes and the HTTP auth contract, not in-band AUTH.

Decided 2026-10-08 (split from [Session outcomes](/quest/m1/session-outcomes.md)):
Decided 2026-10-08:

- **Why.** `TokenExpired` is raised only in moq-auth's key check
(`rs/moq-auth/src/key.rs`), on the auth server. `serve.rs` sends every
Expand All @@ -40,8 +40,7 @@ Decided 2026-10-08 (split from [Session outcomes](/quest/m1/session-outcomes.md)
path and `Admission::refuse` one refusal shape (reason, optional root and
tier), under the same bound, and update `From<moq_auth::Error>`, which sends
anything but `Refused` to `Unavailable` today. A refusal an embedder
records on its stats handle ([Session outcomes](/quest/m1/session-outcomes.md))
takes `expired` too.
records through `stats::Handle::refuse` takes `expired` too.
- **Relay.** Map the typed error to a `Refusal` reason (adding `expired`) in
`/metrics`, and count it in the root's `Presence` on that tier, or the
default tier when the body names none. A refusal without a root stays
Expand All @@ -55,8 +54,3 @@ Decided 2026-10-08 (split from [Session outcomes](/quest/m1/session-outcomes.md)

Public API: moq-auth's refusal error and the HTTP refusal body. Wire: the
auth HTTP contract only, not MoQ.

## Required

- [Session outcomes](/quest/m1/session-outcomes.md) - the refusal counters
this attributes and extends with `expired`
98 changes: 0 additions & 98 deletions quest/m1/session-outcomes.md

This file was deleted.

4 changes: 2 additions & 2 deletions quest/m2/grant-label.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,9 +78,9 @@ Public API: a new optional `Grant::label` in `moq-auth` and `@moq/auth`, and
`stats::Session::set_tier` becomes a `(tier, label)` retag. Wire: the auth
HTTP grant body and the stats track's per-label byte fields, not MoQ.

The per-root session outcome counters follow the same label.

## Related

- [Session outcomes](/quest/m1/session-outcomes.md) - its per-root counters
follow the same label
- [Typed refusal reason](/quest/m1/refusal-reason.md) - a refusal body names
a root too; decide there whether it may name a label
1 change: 0 additions & 1 deletion quest/m2/uring-connection-log.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,4 +54,3 @@ Public API: none expected. Wire: none.

- [IETF on the ring](/quest/m2/uring-ietf.md) - also touches the ring's accept path
- [Relay io_uring packages](/quest/m2/relay-io-uring-package.md) - ships the ring, whose operators read these logs
- [Session outcomes](/quest/m1/session-outcomes.md) - counts refusals and ends on both paths, from the same duplicated admission
9 changes: 2 additions & 7 deletions quest/m2/wire-stats/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,8 @@ see cost per tenant.
which loses a crashed session and lags a long one.
- **One source registry for sessions and embedders.** An embedder registers
a gateway connection's wire source against a (tier, root) on its stats
handle, the same path the relay's own sessions use, like
[session outcomes](/quest/m1/session-outcomes.md)' embedder refusals.
handle, the same path the relay's own sessions use, like the existing
`stats::Handle::refuse` API.

## Required

Expand All @@ -64,8 +64,3 @@ see cost per tenant.
reports its TCP bytes
- [moq-srt wire source](/quest/m2/wire-stats/srt.md) - an SRT connection
reports its wire bytes

## Related

- [Session outcomes](/quest/m1/session-outcomes.md) - the other per-root
`Presence` counters and the embedder stats-handle pattern
3 changes: 3 additions & 0 deletions rs/moq-ffi/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -302,6 +302,9 @@ impl From<moq_net::Error> for MoqError {
fn from(err: moq_net::Error) -> Self {
match err {
moq_net::Error::Transport(message) => Self::Transport { reason: message },
moq_net::Error::IdleTimeout => Self::Transport {
reason: "transport idle timeout".into(),
},
moq_net::Error::NotFound | moq_net::Error::NotFetchable => Self::NotFound,
moq_net::Error::Closed | moq_net::Error::GoingAway | moq_net::Error::SessionClosed => Self::Closed,
moq_net::Error::Cancel => Self::Cancelled,
Expand Down
14 changes: 13 additions & 1 deletion rs/moq-net/src/coding/codes.rs
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ pub trait StreamCodes {
return self.decode_stream_code(code).into();
}

Error::Transport(err.to_string())
Error::from_transport(err)
}
}

Expand Down Expand Up @@ -174,6 +174,18 @@ mod tests {
}
}

/// `Closed` also describes virtual streams, so it cannot claim that the transport's
/// connection closed. Native transport adapters supply that metadata themselves.
#[test]
fn a_virtual_stream_close_does_not_claim_the_connection_closed() {
let version = ietf::Version::Draft16;
assert!(matches!(version.transport_error(Error::Closed), Error::Transport(_)));
assert!(matches!(
version.transport_error(Error::IdleTimeout),
Error::IdleTimeout
));
}

/// A session close is not stream-scoped, so it keeps the session registry whichever
/// wire the stream is on.
#[test]
Expand Down
21 changes: 20 additions & 1 deletion rs/moq-net/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -542,6 +542,9 @@ pub enum Error {
/// Session and stream unknowns are [`Self::Session`] / [`Self::Stream`], not this.
#[error("remote error: code={0}")]
Remote(u32),
/// The underlying connection reached its transport idle deadline.
#[error("transport idle timeout")]
IdleTimeout,
}

impl Error {
Expand Down Expand Up @@ -569,6 +572,10 @@ impl Error {
/// [`StreamError::from_code`]. Reading a stream reset with the session table (or the
/// reverse) silently mistranslates, since e.g. 0 is "no error" for a session but an
/// internal error for a stream.
///
/// Without either code, the transport's [`close_kind`](crate::transport::Error::close_kind)
/// decides: an idle timeout is [`Error::IdleTimeout`], a local close is
/// `Error::Session(SessionError::Cancel)`, and anything else is [`Error::Transport`].
pub fn from_transport(err: impl crate::transport::Error) -> Self {
if let Some((code, _reason)) = err.session_error() {
return SessionError::from_code(code).into();
Expand All @@ -578,7 +585,12 @@ impl Error {
return StreamError::from_code(code).into();
}

Self::Transport(err.to_string())
match err.close_kind() {
crate::transport::CloseKind::IdleTimeout => Self::IdleTimeout,
// A connection ending must not become the model's clean end-of-content.
crate::transport::CloseKind::Closed => SessionError::Cancel.into(),
_ => Self::Transport(err.to_string()),
Comment thread
kixelated marked this conversation as resolved.
}
}
}

Expand Down Expand Up @@ -692,6 +704,13 @@ impl From<&Error> for StreamError {
}

impl crate::transport::Error for Error {
fn close_kind(&self) -> crate::transport::CloseKind {
match self {
Self::IdleTimeout => crate::transport::CloseKind::IdleTimeout,
// Closed also ends model handles and virtual streams, not just connections.
_ => crate::transport::CloseKind::Transport,
}
}
fn session_error(&self) -> Option<(u32, String)> {
None
}
Expand Down
9 changes: 7 additions & 2 deletions rs/moq-net/src/ietf/auth.rs
Original file line number Diff line number Diff line change
Expand Up @@ -288,7 +288,7 @@ where
}

match &err {
Error::Cancel | Error::Unsupported | Error::Transport(_) | Error::Session(_) => {
Error::Cancel | Error::Unsupported | Error::Transport(_) | Error::IdleTimeout | Error::Session(_) => {
tracing::debug!(%err, "auth token ended")
}
err => tracing::warn!(%err, "auth token ended"),
Expand Down Expand Up @@ -467,7 +467,12 @@ impl Serve {
Ok(()) => Error::Cancel,
Err(err) => {
match &err {
Error::Cancel | Error::Unsupported | Error::Stream(_) | Error::Session(_) | Error::Transport(_) => {
Error::Cancel
| Error::Unsupported
| Error::Stream(_)
| Error::Session(_)
| Error::Transport(_)
| Error::IdleTimeout => {
tracing::debug!(%err, "auth request ended")
}
err => tracing::warn!(%err, "auth request error"),
Expand Down
9 changes: 7 additions & 2 deletions rs/moq-net/src/ietf/session.rs
Original file line number Diff line number Diff line change
Expand Up @@ -627,7 +627,7 @@ where
/// closes over that reset with its own PROTOCOL_VIOLATION before it reads our code.
fn close<S: crate::transport::poll::Session>(session: &mut S, res: &Result<(), Error>) {
match res {
Err(err @ Error::Transport(_)) => {
Err(err @ (Error::Transport(_) | Error::IdleTimeout)) => {
tracing::info!(%err, "session terminated");
session.close(super::error::to_session_code(&SessionError::Internal), "");
}
Expand Down Expand Up @@ -1102,7 +1102,12 @@ impl<S: crate::transport::poll::Boxable> Uni<S> {
fn died_before_header(err: &Error) -> bool {
matches!(
err,
Error::Cancel | Error::Stream(_) | Error::Remote(_) | Error::Transport(_) | Error::Decode(DecodeError::Short)
Error::Cancel
| Error::Stream(_)
| Error::Remote(_)
| Error::Transport(_)
| Error::IdleTimeout
| Error::Decode(DecodeError::Short)
)
}

Expand Down
11 changes: 9 additions & 2 deletions rs/moq-net/src/lite/publisher.rs
Original file line number Diff line number Diff line change
Expand Up @@ -439,7 +439,12 @@ impl<S: crate::transport::poll::Session> AuthServe<S> {
Ok(()) => Error::Cancel,
Err(err) => {
match &err {
Error::Cancel | Error::Unsupported | Error::Stream(_) | Error::Session(_) | Error::Transport(_) => {
Error::Cancel
| Error::Unsupported
| Error::Stream(_)
| Error::Session(_)
| Error::Transport(_)
| Error::IdleTimeout => {
tracing::debug!(%err, "auth stream ended")
}
err => tracing::warn!(%err, "auth stream error"),
Expand Down Expand Up @@ -899,7 +904,8 @@ impl<S: crate::transport::poll::Session> AnnounceServe<S> {
Error::Cancel
| Error::Stream(crate::StreamError::Cancel)
| Error::Session(crate::SessionError::Cancel)
| Error::Transport(_) => {
| Error::Transport(_)
| Error::IdleTimeout => {
tracing::debug!(prefix = %origin.absolute(""), "announcing cancelled");
}
err => {
Expand Down Expand Up @@ -1392,6 +1398,7 @@ fn is_cancel(err: &Error) -> bool {
| Error::Stream(crate::StreamError::Cancel)
| Error::Session(crate::SessionError::Cancel)
| Error::Transport(_)
| Error::IdleTimeout
)
}

Expand Down
Loading
Loading