Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/bin/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -250,7 +250,7 @@ moq ... import capture --camera 0 --width 1280 --height 720 --fps 30 --bitrate 3

Video goes through the platform hardware encoder, with an H.264 software
fallback when built with `openh264`, and audio is Opus; [moq-video](/lib/rs/moq-video) lists the
backends. The camera opens only while someone is watching, and `--bitrate` is a
backends. The camera stays open while publishing, and `--bitrate` is a
ceiling that backends with live bitrate control lower to fit the connection.
`moq devices` prints every source id. On Windows, `display:N` is an
enumeration index rather than a stable monitor id, so re-run `moq devices` if a
Expand Down
9 changes: 6 additions & 3 deletions doc/lib/rs/moq-video.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,15 +26,18 @@ feature (vendored C++); the published bindings and binaries enable it. H.265 is
hardware-only, AV1 decodes through NVDEC and MediaCodec, and VP8 and VP9
decode in software through the opt-in `vpx` feature.

Capture is pull-driven: `encode::publish_capture` advertises the track up
front and opens the camera only while someone subscribes. A viewer who joins
`encode::publish_capture` advertises the track up front, then keeps the camera
open and encoding whether or not anyone subscribes, so a late viewer can fetch
the recent past. Set `encode::Capture::on_demand` to open the camera only while
someone subscribes, at the cost of that past. A viewer who joins
once the current group's deltas outweigh twice its keyframe gets a fresh
keyframe, so the GOP is only the longest a group runs. Through a moq-lite
relay, only the join that opens the relay's upstream subscription counts. A
build with the `v4l2` feature keeps every `Auto` or `Hardware` selection to its
GOP, whichever encoder it lands on, since a V4L2 driver may refuse the request.
Backends that can
change bitrate live follow the connection's send estimate. Where the platform
change bitrate live follow the connection's send estimate while someone
watches, and return to the configured bitrate while nobody does. Where the platform
allows it, frames stay on the GPU between capture, decode, encode, and render.

Opt-in features: `capture`, `vaapi` (Intel and AMD on Linux; the build needs
Expand Down
1 change: 0 additions & 1 deletion quest/m1/eager-encode/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,6 @@ The epic's own work, once the children land:
## Required

- [@moq/publish encodes while enabled](/quest/m1/eager-encode/js.md) - audio and video encoders drop the demand gate, with an on-demand opt-in
- [moq-video captures while enabled](/quest/m1/eager-encode/video.md) - the camera stays open and encodes while enabled, with an on-demand opt-in
- [moq-cli on-demand flag](/quest/m1/eager-encode/cli.md) - `moq publish` capture encodes always, unless `--on-demand`
- [Raw audio on demand](/quest/m1/eager-encode/raw-audio.md) - audio encodes on demand again, from raw frames kept up to `maxAge`

Expand Down
11 changes: 4 additions & 7 deletions quest/m1/eager-encode/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,12 @@ as today.
Decided 2026-10-28 (maintainer); see the [epic](/quest/m1/eager-encode/README.md).
`rs/moq-cli/src/publish.rs:592` calls both `publish_capture`s, so the default
follows the libraries. The flag selects their on-demand option
(`moq_audio::encode::Capture::on_demand` for audio). Update the
comment there and `doc/bin/cli.md:250`. Name the flag under the repository's
CLI rule.
(`moq_video::encode::Capture::on_demand` and
`moq_audio::encode::Capture::on_demand`). The comment there and
`doc/bin/cli.md:250` describe the always-on camera; add the flag and the
microphone. Name the flag under the repository's CLI rule.

Tests: the CLI's capture args parse `--on-demand`, and the default publishes
without a subscriber.

Public API: CLI default change, plus a new flag. Wire: none.

## Required

- [moq-video captures while enabled](/quest/m1/eager-encode/video.md) - the video option this selects
24 changes: 0 additions & 24 deletions quest/m1/eager-encode/video.md

This file was deleted.

7 changes: 3 additions & 4 deletions rs/moq-cli/src/publish.rs
Original file line number Diff line number Diff line change
Expand Up @@ -589,10 +589,9 @@ impl Publish {
// Each enabled medium publishes its own track onto the shared
// broadcast + catalog. Both stamp frames on the catalog's
// advertised clock, so HLS/DASH wall times match the mapping on
// the wire. Video encodes on demand (camera opens only while
// subscribed). Both run on this task rather than a spawn: on
// macOS the audio future holds ObjC handles across an await,
// so it is `!Send`.
// the wire. Video captures for the whole run. Both run on this
// task rather than a spawn: on macOS the audio future holds ObjC
// handles across an await, so it is `!Send`.
let video_fut = {
let broadcast = broadcast.clone();
let catalog = catalog.clone();
Expand Down
2 changes: 1 addition & 1 deletion rs/moq-net/src/model/bandwidth.rs
Original file line number Diff line number Diff line change
Expand Up @@ -922,7 +922,7 @@ mod tests {
assert_eq!(second_share.rate(), Some(bps(1_000_000)));
}

/// An unwatched track releases its reservation to the others, since `publish_capture`
/// An unwatched track releases its reservation to the others, since an on-demand sender
/// stops encoding entirely while nothing is subscribed, but still reads the slice it
/// would get back, which is what a stopped rendition its viewers left resumes on.
#[test]
Expand Down
2 changes: 0 additions & 2 deletions rs/moq-transcode/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1029,8 +1029,6 @@ mod tests {

/// A source that resizes mid-stream takes the ladder with it.
///
/// `moq_video::encode::publish_capture` opens its source twice by design (once
/// to probe the mode, once when the first subscriber arrives), and a window
/// A source that changes aspect ratio keeps every rung height while moving
/// every rung width, so a rung retires and its replacement serves the same
/// height. That replacement must not reuse the retired track's name: it is a
Expand Down
4 changes: 2 additions & 2 deletions rs/moq-transcode/src/pipeline.rs
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
//! The ladder a transcoder publishes for one source, and the rungs serving it.
//!
//! The ladder is sized against the source picture, so it has to follow it.
//! `moq_video::encode::publish_capture` opens its source twice by design (once
//! to probe the mode, once when the first subscriber arrives), and a window
//! An on-demand `moq_video::encode::publish_capture` opens its source twice
//! (once to probe the mode, once when the first subscriber arrives), and a window
//! capture derives its geometry from the window on every open, so the picture a
//! ladder was resolved against is routinely not the one the source ends up
//! carrying. A reconnecting publisher and a renegotiated screen share do the
Expand Down
5 changes: 3 additions & 2 deletions rs/moq-video/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,8 +127,9 @@ way, awaited.

Public entry points:

- `encode::publish_capture(...)` captures a webcam, encodes it, and publishes on
demand: the track and catalog are advertised up front, but the camera opens
- `encode::publish_capture(...)` captures a webcam, encodes it, and publishes it:
the track and catalog are advertised up front, and the camera stays open
whether or not anyone is watching. With `Capture::on_demand`, the camera opens
only while a subscriber is watching and is released when the last one leaves.
- `encode::Producer` publishes frames you encoded yourself (`publish(&[Encoded])`),
handling the catalog and framing. Each is published at its own timestamp.
Expand Down
10 changes: 8 additions & 2 deletions rs/moq-video/src/capture/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -113,8 +113,9 @@ pub enum Source {

/// A screen or window chosen through the Linux system picker.
///
/// Clone the selection for demand-driven reopens; create a new one to ask
/// for a different source. Restoration depends on compositor support.
/// Clone the selection to reopen the same source, as an on-demand capture
/// does for each viewer; create a new one to ask for a different source.
/// Restoration depends on compositor support.
#[cfg(all(target_os = "linux", feature = "pipewire"))]
Portal(portal::Selection),
}
Expand Down Expand Up @@ -425,6 +426,11 @@ impl Synthetic {
pub(crate) fn close(&self) {
self.chan.close();
}

/// Whether the stream reading this device was dropped, as a closed camera is.
pub(crate) fn released(&self) -> bool {
Arc::strong_count(&self.chan) == 1
}
}

/// Open the capture source described by `config`.
Expand Down
18 changes: 9 additions & 9 deletions rs/moq-video/src/capture/pipewire.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,11 @@
//! pull-style pump).
//!
//! Two screen quirks worth knowing:
//! - `publish_capture` releases the capture while unwatched and reopens it on
//! demand. A fresh portal session would re-prompt the picker every time, so the
//! portal's restore token is kept with the selection and consumed by the
//! next [`open`], which restores the same grant without a dialog (on
//! compositors that support persistence). The token is forgotten when the
//! - An on-demand `publish_capture` releases the capture while unwatched and
//! reopens it for the next viewer. A fresh portal session would re-prompt the
//! picker every time, so the portal's restore token is kept with the selection
//! and consumed by the next [`open`], which restores the same grant without a
//! dialog (on compositors that support persistence). The token is forgotten when the
//! compositor ends the stream (the user hit "stop sharing"), so a revoked
//! grant is asked for again rather than silently resumed.
//! - Compositors only deliver frames on damage, so a static screen would starve
Expand Down Expand Up @@ -154,7 +154,7 @@ async fn start(config: &Config, capture: Capture, session: Option<SessionGuard>)
return_tx,
}) {
// Surface a setup failure through the awaiting `open`; a mid-stream
// failure just ends the stream (the encode loop reopens on demand).
// failure just ends the stream (the encode loop reopens it).
match state.borrow_mut().geo_tx.take() {
Some(tx) => drop(tx.send(Err(e))),
None => chan.fail(e),
Expand Down Expand Up @@ -301,8 +301,8 @@ async fn portal_negotiate(cursor: bool, selection: &portal::Selection) -> Result
}

/// Closes the portal session when dropped, so the compositor's "screen is being
/// shared" indicator turns off and sessions don't pile up across demand-driven
/// reopens. The close call is async and `Drop` is not, so a task spawned here
/// shared" indicator turns off and sessions don't pile up across an on-demand
/// capture's reopens. The close call is async and `Drop` is not, so a task spawned here
/// waits for the guard to drop. Closing does not invalidate the restore token.
struct SessionGuard {
_close: tokio::sync::oneshot::Sender<()>,
Expand Down Expand Up @@ -863,7 +863,7 @@ fn run_loop(args: CaptureLoop) -> Result<(), Error> {
if done {
// The producer ended the stream while our loop was live (the
// user hit "stop sharing", or the output or camera went away).
// Forget the screen restore token so the demand-driven reopen
// Forget the screen restore token so the next reopen
// asks again instead of silently resuming a grant the user just
// revoked. Our own teardown quits the loop before anything
// disconnects, so it never reaches this path.
Expand Down
4 changes: 2 additions & 2 deletions rs/moq-video/src/encode/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@
//!
//! A [`Producer`] advertises its catalog rendition as soon as the track exists,
//! resolved from the encoder itself via [`Config::probe`], so a subscriber can
//! discover a track nothing has encoded yet. That's what makes on-demand
//! encoding possible at all.
//! discover a track nothing has encoded yet. That's what lets an on-demand
//! capture wait for its first viewer at all.
//!
//! `Capture` (with `capture`) / [`Kind`] / [`Config`] configure them.
//! [`Config::preset`] picks a [`Preset`], how the encoder trades latency for
Expand Down
Loading
Loading