Repository navigation
feat!: demand reads as wanted everywhere - #7036
Conversation
Quest-Claim: c4bd47d367cf460e
kio, moq-net, @moq/net, moq-ffi, moq-c, and the wrappers name every demand signal wanted: a snapshot wanted() plus wanted_changed() and its poll. bandwidth::Producer gets a demand() handle, track::Fill merges is_used() and next() into wanted(), and @moq/net drops the internal Broadcast.Demand. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Format, gate the async-only bandwidth poll, regenerate the Dart bindings, fix the remaining FFI test, and delete the finished quest. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
WalkthroughThe change replaces used/unused demand terminology with wanted/unwanted terminology across demand primitives, fill APIs, protocol handling, and language bindings. Fill APIs add wanted-state queries and change notifications, while removing the prior used-state and next/unused methods. Network lifecycle checks and tests now use wanted-state behavior. The C demand constants are renamed without changing their numeric values. JavaScript broadcast-level aggregate demand tracking is removed, while pending lookup counting remains. Merge Risk:
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9cfe968760
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed commit: 9cfe96876085a61bfd1b869b594131ccfc9b2718.
P2 — Do not advance the shared fill watcher after foreign cancellation (rs/moq-ffi/src/producer.rs:545–547). The new wanted_changed drives its shared cursor inside ffi::detached. UniFFI cancellation stops polling before rust_future_free drops the future, so the spawned worker can still consume a change and update Fill.prev after the caller cancelled. For example: observe Some(5), park another call, cancel it without freeing it, then let the last reader leave. The worker consumes None; after freeing the cancelled call, the next wanted_changed can park forever even though wanted() is already terminal None. The old level-triggered unused() did not consume a shared cursor. Drive this stateful wait in place, following Task::run's existing cancellation approach, and add a cancel-before-free regression like raw_read_frame_cancelled_before_free_leaves_the_frame.
I also agree with the existing cache-arrival wakeup finding: wanted() reads the live cache, but the new poll does not register for that input. Referencing it rather than adding a duplicate inline comment.
Direction: the consistent snapshot/change vocabulary and removal of unused JS broadcast aggregation are reasonable simplifications. The new stateful Fill watch needs both complete wake registration and cancellation-safe cursor ownership; these are behavioral changes beyond the mechanical renames.
Verification: static review of the diff and relevant kio, demand, forwarding, JS cancellation and binding paths, including an independent Fill/FFI pass. No tests, loom runs or binding builds executed.
Convert the calls #6934 added to the wanted vocabulary: broadcast demand's track check, the ietf subscriber's request rejection, and the track tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fill::poll_wanted_changed now registers on the track state, so a parked wait wakes when the owed group arrives through the live feed and the cache answers it. MoqFill::wanted_changed drives its watcher in place through ffi::Task instead of a detached task, so a call the foreign side cancelled but has not freed cannot consume the change the next call waits for. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Both findings are fixed in 5fe5d63:
(Written by Claude Opus 5.5) |
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed commit: f44f94e.
Both previously reported P2s are addressed:
- The cancellation issue in #7036 (review):
MoqFill::wanted_changednow usesTask::runin place, so a cancelled but unfreed call cannot keep advancing the shared cursor. The new regression models stopping polling before freeing the future. - The cache-arrival wakeup issue in #7036 (comment): the poll now arms the track state before reading
wanted(), and the live-arrival regression asserts both a wake and the changed result without separately driving the reader.
Direction: these focused fixes supply cancellation-safe cursor ownership and the missing wake source without undoing the consistent demand API. No new actionable defects found in the reviewed fix delta and integration paths.
GitHub-only static comparison and surrounding implementation/test inspection. No tests, loom runs, binding builds, or foreign-runtime cancellation experiments executed; reported validation was not independently reproduced.
(Written by OpenAI)
|
Landing summary for the /quest-merge pass:
(Written by Claude Opus 5.5) |
# Conflicts: # rs/moq-net/src/model/origin.rs
|
Landing The old merge queue ejected this as (Written by Claude Opus 5.5) |
|
quest land removed this pull request from the queue: the batch check failed (exit 101). vm_service 15.3.0 (15.4.0 available)
web 1.1.1 (1.1.2 available)
Got dependencies!
14 packages have newer versions incompatible with dependency constraints.
Try `dart pub outdated` for more information.
Resolving dependencies...
Downloading packages...
_fe_analyzer_shared 105.0.0 (111.0.0 available)
analyzer 14.1.0 (14.7.0 available)
code_assets 2.0.0 (2.1.0 available)
coverage 1.15.1 (1.16.0 available)
crypto 3.0.7 (3.0.8 available)
frontend_server_client 4.0.0 (4.2.0 available)
http_multi_server 3.2.2 (3.2.3 available)
! moq_ffi 0.0.0-dev from path ../moq_ffi (overridden)
shelf 1.4.2 (1.4.3 available)
shelf_static 1.1.3 (1.2.0 available)
test 1.32.0 (1.33.0 available)
test_api 0.7.14 (0.7.15 available)
test_core 0.6.20 (0.6.21 available)
vm_service 15.3.0 (15.4.0 available)
web 1.1.1 (1.1.2 available)
Got dependencies!
14 packages have newer versions incompatible with dependency constraints.
Try `dart pub outdated` for more information.
Publishing moq 0.1.0 to https://pub.dev:
├── CHANGELOG.md (<1 KB)
├── LICENSE (1 KB)
├── MIGRATE.md (16 KB)
├── README.md (1 KB)
├── analysis_options.yaml (<1 KB)
├── bench
│ └── ffi.dart (5 KB)
├── lib
│ ├── audio.dart (2 KB)
│ ├── json.dart (2 KB)
│ ├── media.dart (3 KB)
│ ├── moq.dart (<1 KB)
│ ├── src
│ │ ├── aliases.dart (7 KB)
│ │ ├── client.dart (10 KB)
│ │ ├── epoch.dart (<1 KB)
│ │ ├── server.dart (5 KB)
│ │ ├── stream.dart (2 KB)
│ │ ├── timestamp.dart (1 KB)
│ │ └── version.dart (<1 KB)
│ └── video.dart (3 KB)
├── pubspec.yaml (<1 KB)
└── test
├── cancellation_test.dart (7 KB)
├── docs
│ ├── prelude.dart (<1 KB)
│ └── samples.dart (4 KB)
└── moq_test.dart (27 KB)
Total compressed archive size: 30 KB.
Validating package...
Package validation found the following hint:
* Non-dev dependencies are overridden in pubspec.yaml.
This indicates you are not testing your package against the same versions of its
dependencies that users will have when they use it.
This might be necessary for packages with cyclic dependencies.
Please be extra careful when publishing.
The server may enforce additional checks.
Package has 0 warnings and 1 hint.
ci: check rs failed
ci: check rs_shapes failed
ci: seconds per module
255 check py
59 test py
280 check rs (failed)
169 check rs_shapes (failed)
311 check c
253 check cpp
27 check obs_compile
335 check js
59 test js
28 check drafts
24 check markdown
3 check gh
57 test gh
147 check kt
9 check carla
0 check swift
96 check go
1 check drill
3 check quest
1 check obs
22 check shell
0 check large_files
1 check toml
0 check nix
0 check justfile
183 check dart
1003 wall time
error: recipe `ci` failed on line 104 with exit code 101
. |
|
Codex land fixer: (written by Codex) |
|
The host land gate's batch failed on this PR: (written by Claude Opus 5.5) |
Main gated write_filled behind the async feature, but this PR's fill_wanted_changed_follows_the_group_owed test is sync and runs in the explicit-time shape too. Ungate the helper so both shapes compile. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Re-landing The batch failure was a semantic conflict with main: main gated the track test helper (Written by Claude Opus 5.5) |
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed commit: 7c378e11ae79f4262d98cc908af9e8e7619fb664 (update since f44f94edbf3d47001ce9d1c3ceb346bb187a5ea0, separating merged-main changes from this PR).
No new actionable bugs found in the reviewed update. Ungating write_filled in rs/moq-net/src/model/track.rs:18016 is the appropriate minimal fix: its body is synchronous, and fill_wanted_changed_follows_the_group_owed must be able to call it without the async feature. The origin integration preserves main's closed-track handling while applying the wanted vocabulary.
Both prior findings remain fixed: the Fill poll still registers the live-cache wake source, and the FFI watcher still runs in place via Task::run. These are not new findings.
Direction: retain the consistent demand API and the small feature-shape build fix; no additional abstraction is needed for this update.
Verification: GitHub-only static inspection of commit ancestry, the build fix, relevant merged integration, and the prior fix paths. No tests, feature-shape builds, binding builds, or loom runs executed; local CI claims were not independently reproduced.
…ser-baseline Resolve the fill conflicts with #7036: keep main's `Fill::wanted` name and its no-reader early return in the new `wanted_except`, and drop the removed `is_used` guard from `serve_fill`, since both run lookups now end the loop once no reader waits. Unblock the two quests main added that required this one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keeps REQUEST_UPDATE's AUTHORIZATION TOKEN on the namespace update main made crate-visible, and follows main's rename of `poll_unused` to `poll_wanted_changed` (#7036) where a refused discovery token waits for its verification credit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
main (#7036) removed Fill::unused()/is_used() and renamed next() to wanted(). A lookup owes no group, so nothing now tells it when its readers stop waiting: an abandoned lookup finishes its store read and answers nobody. Rename the test calls and fix the Fill::time() docs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#7036 deleted the broadcast-level Demand, so the contribution tests watch the catalog and video tracks' wanted demand instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
main renders /metrics through prometheus-client now (#7363), so the demux drop counter becomes an `encode_demux` family on `Scrape`, and the pinned series test covers it. Also rename the two `kio::Producer::is_used` calls to `wanted`, which #7036 renamed; the previous merge left them uncompiled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem
Demand was spelled two ways:
wanted/wanted_changedontrack::Demandandgroup::Demand, andused/unused/is_usedon kio channels,bandwidth::Producer,track::Fill,@moq/net, moq-ffi, moq-c, and the wrappers. This finishes quest/m1/api-audit/wanted.md: every demand signal is a snapshotwanted()pluswanted_changed()and its poll, and a method nobody used is deleted rather than renamed.Changes
Producer/ProducerWeakis_used/used/unused/poll_used/poll_unusedbecomewanted()andpoll_wanted_changed(waiter, last).Producer::wanted_changed(last)stays async for its two callers, moq-net's server close and moq-scene's source.ProducerWeak's async pair, which only tests used, is gone.write_unused/Unused::{Idle, Used, Closed}becomewrite_unwanted/Unwanted::{Idle, Wanted, Closed}.bandwidth::Producerloses its five consumer-count methods and gainsdemand() -> bandwidth::Demand(wanted() -> bool,wanted_changed(),poll_wanted_changed()), shaped likegroup::Demand.track::Fillmergesis_used()andnext()intowanted() -> Option<u64>: the next group owed, orNoneonce nothing is owed or no reader is left.wanted_changed()/poll_wanted_changed()replaceunused()/poll_unused(). A parked poll wakes on readers leaving, narrowing, or taking a group, on an answer, and on an order change.Fillis nowClone, and each clone watches on its own.track::Producer::abort_unusedisabort_unwanted.poll_wanted/poll_unwanted/is_wanted,abort_unwanted,reject_unwanted, and the origin front'sEvent::{Wanted, Unwanted}.Broadcast.Demand,Broadcast.Producer.demand(), and the per-track aggregation behind them are deleted. They were@internaland only tests and a bench read them.@internalGroup.Demand.usediswanted, andunused()is gone. Fetch coalescing waits onwantedthrough one module-private helper.Lister.used,Listing.used, andObserved.usedarewanted.MoqFill::next/is_used/unusedbecomewanted()andwanted_changed(), with a separate watcher the wayMoqTrackDemandkeeps one.MOQ_DEMAND_USED/MOQ_DEMAND_UNUSEDareMOQ_DEMAND_WANTED/MOQ_DEMAND_UNWANTED, with the same values.Fillexposewanted/wanted_changedin place ofnext/is_used/unused. Kotlin and Dart alias the generated type, and the Dart bindings are regenerated. The staleUsed/Unusedadvice ingo/wrapper/README.mdanddoc.gois fixed.Impact
Public API, all breaking. Wire: none.
is_used()withwanted(), andused()/unused()withwanted_changed(false)/wanted_changed(true)(poll_wanted_changed(waiter, last)). These now return the new value (Result<bool, Closed>) instead of().write_unusediswrite_unwanted, andUnused::UsedisUnwanted::Wanted.bandwidth::Producer: replaceis_used()withproducer.demand().wanted(), and theused/unusedwaits with a loop over a heldproducer.demand()'swanted_changed().track::Fill: serve withwhile let Some(sequence) = fill.wanted()where you checkedis_used() && next(). Awaitwanted_changed()until it returnsNonewhere you awaitedunused().abort_unusedisabort_unwanted.MoqFill.next()iswanted(),is_used()iswanted() != null, andunused()is a loop overwanted_changed()until it returns null.MOQ_DEMAND_USED/MOQ_DEMAND_UNUSEDtoMOQ_DEMAND_WANTED/MOQ_DEMAND_UNWANTED.@moq/net: only@internalitems changed, so no published surface moved.For the release cut, the existing Unreleased notes need these corrections:
rs/moq-json/MIGRATE.mdandrs/moq-flate/MIGRATE.md("Demand is read throughdemand()"): the call isproducer.demand().wanted().is_some(), notproducer.demand().is_used().js/net/MIGRATE.md:usedandunused()is now false. Both are@internal: broadcast demand is gone, and group demand iswanted.Group.Demand.unused()andBroadcast.Demand.unused()reject on close is moot.producer.unused()isproducer.demand().unused()" should point atdemand().wanted.Decisions settled
The maintainer accepted every recommendation below in a /quest-complete session.
wanted_changedtakes the last value seen. The quest nameswanted_changed()with no argument. A kioProduceris a bare channel handle, so trackingprevper handle would put a field on every channel. Instead it iswanted_changed(last: bool)/poll_wanted_changed(waiter, last), which returns once presence differs fromlast. moq-net'sgroup::Demand,bandwidth::Demand, and the internal track poll keep the per-handleprevon top of it. Settled: keep it. The alternative, a stateful handle, costs memory on every kio channel.track::Fillis nowClone. moq-ffi needs a second handle to parkwanted_changedon without holding a lock across the sync getters, mirroringMoqTrackDemand. A clone shares the fill and keeps it unsettled until the last one drops, asGroupFillalready does. Settled: keep it. The alternative is a mutex-wrappedFillinside moq-ffi.abort_unused→abort_unwanted(public ontrack::Producer),write_unused/Unused→write_unwanted/Unwanted(kio), plus the crate-internal helpers. None is a demand signal, but each named the vocabulary this PR retires. Settled: keep them. Leaving them would leaveunusedin kio and moq-net's public API.Choices made
Fill::wanted_changedreturnsOption<u64>, not aResult. A live fill handle keeps its answer channel open, so there is no closed state to report. A track abort shows up as its readers leaving, which ends atNone.Fill::is_wanted(sequence)/unwanted(sequence)stay as they are. The per-group signal only ever goes one way, so awanted_changedshape does not fit it.@moq/net. Pending info lookups only fedBroadcast.Demand, so with it gone they count toward nothing.Lookupsstill letsresolveTrackInfoclose a request it opened.Demandtrait (used/unusedwrappingwanted_changed) and the carla demo's Python helpers. moq-stats's privateis_usedbecameis_wantedbecause it was one line.MIGRATE.mdfiles are untouched; the corrections for the cut are listed under Impact above.Tests
fill_wanted_changed_follows_the_group_owed(moq-net) checks that a parkedpoll_wanted_changedwakes on narrowing, on an answer, and on the last reader leaving, and ends atNone.wanted_changedin both directions.cargo check/clippy -D warnings(all targets) on kio, moq-net, moq-ffi, moq-c, moq-tokio, moq-transcode, moq-archive, moq-scene, and moq-stats.clippy --no-default-featureson moq-net and kio.cargo test --libon moq-net, moq-ffi, moq-c, and moq-tokio.fetch_cancelintegration tests and moq-transcode's abandoned-fetch test.bun run checkand the touched test files in js/net, json, flate, hang, and e2ee.just go check(vet, build,go test -race) andjust py check(ruff, pyright, samples).just rs loom, the Python unit tests, the full js/net suite after the last edits, and the Swift, Kotlin, and Dart builds.log::testcases (reader_outlives_writer,a_read_outlasts_eviction,eviction_frees_whatever_readers_do) failed under the parallel run but pass alone.log.rsis untouched here, so this is a pre-existing flake (see Follow-ups).Follow-ups
broadcast::Demandand edits this quest file. Whichever lands second converts the other's kio calls (is_used/poll_used/poll_unused→wanted/poll_wanted_changed) and resolves the quest-file conflict. This PR deletes the quest file.log::testeviction tests are flaky when run in parallel, likely from a shared global collection. They were not investigated here.Review
Codex (
quest review), gpt-6.1-sol, medium effort, reviewed head7c378e11ae79f4262d98cc908af9e8e7619fb664againstorigin/main(1d8bf24998): no findings. That head merges main conflict-free and adds one fix for a semantic conflict thequest landbatch caught: main gated the test helperwrite_filledbehind theasyncfeature, while this PR's sync testfill_wanted_changed_follows_the_group_owedcalls it, so--no-default-featurestest builds failed withE0425. The helper is now ungated, so the test also runs in the explicit-time shape. The earlier round on7800792668(no findings) covered the hand-resolvedorigin.rsconflict, and rounds onf44f94edbfaddressed both of the online reviewer's P2s (see the review thread).Local checks on that head:
just ci origin/mainpassed every module (py, rs, rs_shapes, c, cpp, obs_compile, js, markdown, gh, kt, swift, go, drill, quest, shell, large_files, toml, nix, justfile, dart).(Written by Claude Opus 5.5)
🤖 Generated with Claude Code