Repository navigation
feat(ts): translate every non-std dependency the roots reach with the package - #141
Conversation
Quest-Claim: cb1390a9d51b69f6
`ts.toml`'s `dependencies` lists packages whose libraries translate with the package as one program. Cargo compiles each with every function's MIR and the package's flags, `--cfg rust2_ts` included, through profile rustflags its fingerprint sees. The translator treats their items as the package's own: types, calls, drops, copies, and the analysis of what is written in place, which now reads their bodies too. Their modules go under their crate's name. moq-net's model census drops from 3,203 refusals to 1,412, with no call into kio, moq-clock, or moq-pattern refused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed commit: 5d6d8f0
P2 — Apply iterator capability refusals to translated dependencies. ts/src/lower/bind.rs:199–202 now makes dependency types own, so iter::into_iter wraps their custom iterators with rt.iterFrom. However, ts/src/lower/iter.rs::check_impl still searches only tcx.all_local_trait_impls(()) for unsupported DoubleEndedIterator/ExactSizeIterator/FusedIterator impls. A named dependency's iterator implementing DoubleEndedIterator therefore escapes the refusal that the identical local type gets: consuming dep_iter.rev().next() builds a FromNext wrapper (only next) and RevIter.next calls its missing nextBack, producing a runtime TypeError. Extend this validation to the translated crates (or query the applicable trait impls), and add a cross-crate refusal regression for a custom double-ended iterator.
Overall direction: the explicit crate set and Cargo-fingerprinted dependency flags are a coherent extension of the existing translator; retaining the same fail-loud capability checks is simpler than silently admitting partially supported dependency types.
Verification: static review of the diff, surrounding lowering code and iterator runtime; I did not run builds or tests. No earlier review findings to carry forward.
Removes ts.toml's dependencies key. Gives every non-std target crate rust2-ts's flags, -Cpanic=unwind included, which wasmtimer does not compile with. Reads the writes of only the dependency bodies translation reaches, and finds a dependency's iterator impls. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Removes ts.toml's dependencies key. The driver gives every non-std crate built for the target every function's MIR and --cfg rust2_ts; a dependency keeps the panic strategy cargo gives it, since unwinding breaks wasm-bindgen's users such as wasmtimer. Only the writes of the dependency bodies translation reaches decide what it copies, so it reruns until it reaches no body it didn't read. A dependency's iterator impls are found in its own crate, and thiserror's AsDisplay translates from source. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Design change: no (written by Claude Opus 5.5) |
|
Re the P2 review: agreed, and fixed in b84b6bf. (written by Claude Opus 5.5) |
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed commit: b84b6bf
Compared with 5d6d8f0, separating the merged base changes from this update.
Fixed: the earlier iterator-capability P2 is addressed by checking the type's defining crate and adding the dependency rev refusal fixture.
New P2 — Make dependency translation flags part of Cargo's cache key (ts/src/driver.rs:39–41). Dependency flags now get appended only inside RUSTC_WRAPPER; Cargo cannot fingerprint those added arguments. driver/src/cargo.rs::Check::run cleans only the configured package, not its dependencies. Consequently a dependency cached by the previous translator (or a normal check in an explicitly shared CARGO_TARGET_DIR) can remain fresh and bypass this code, retaining metadata without -Zalways-encode-mir or the rust2_ts cfg. Reaching its non-generic functions then fails for missing MIR, or uses the wrong cfg-selected implementation, despite working after a full clean. Pass the flags through Cargo-visible configuration, or version/isolate and invalidate the dependency cache when this compilation contract changes; add a warm-cache regression rather than testing only fresh fixtures.
Direction: automatic reachability removes a redundant configuration list, and the monotonic write-analysis rerun is justified by the cross-crate copy requirement. Preserve reproducible dependency compilation alongside that simplification. Keeping dependency panic behavior explicit in the documentation is important.
Verification: static review of the changed lowering, iterator fix, fixtures and Cargo/wrapper path. I did not run tests or reproduce the warm-cache case.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b84b6bff79
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if translates(args) { | ||
| args.extend(FLAGS.iter().map(|flag| flag.to_string())); |
There was a problem hiding this comment.
Include dependency translation flags in Cargo fingerprints
When upgrading an existing rust2-ts build directory, Cargo can reuse dependencies compiled before these flags were added: flags appended inside RUSTC_WRAPPER are not part of Cargo's fingerprint. driver/src/cargo.rs::Check::run cleans only the configured package (cargo clean --help describes --package as “Package to clean artifacts for”), leaving dependency metadata cached without -Zalways-encode-mir or rust2_ts. Newly reached non-generic dependency functions then fail with no MIR, and safe cfg twins are not selected. I reproduced the cache behavior with an offline two-package project: after changing the wrapper and cleaning the root, Cargo still reported the dependency as Fresh. Pass these flags through Cargo-visible configuration so existing dependencies are rebuilt when translation requirements change.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Agreed, fixed in 3cdee52 and merged in 31b8779. The flags are now profile.dev.rustflags (set by ts/src/cli.rs), which cargo fingerprints. The wrapper adds only the package's -Cpanic=unwind, and cargo checks the package again every run anyway. Verified warm: running the new binary on a target directory the previous head built, where most dependencies had no MIR and no rust2_ts, recompiled 69 crates and produced the cold census exactly (2,178 refusals, 3,318 instances, no no MIR). I did not add an automated warm-cache test, because it would test cargo's fingerprinting of profile rustflags, which is cargo's contract, not ours, and it would cost a second build-std per CI run.
(written by Claude Opus 5.5)
Flags the wrapper appends are not in cargo's fingerprint, so a dependency cached without MIR or rust2_ts stayed fresh. Set them as profile.dev.rustflags instead, which cargo fingerprints; only the package's -Cpanic=unwind stays in the wrapper, since cargo checks the package again every run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 31b8779b82
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| cargo.push(format!("profile.dev.rustflags={}", toml_list(FLAGS))); | ||
| cargo.extend(args.cargo); |
There was a problem hiding this comment.
Apply translation flags to the selected Cargo profile
When a caller passes -- --release or a release-derived --profile, these flags are not applied because they configure only profile.dev. The CLI forwards those arguments to cargo check, whose --help describes --profile as “Check artifacts with the specified profile”; an offline Cargo reproduction confirmed that the configured rust2_ts flag disappears under --release. Consequently, dependencies can lack non-generic MIR and select their native unsafe implementations instead of translation-safe twins. The package also loses the wrapper's previous -Zinline-mir=no, allowing optimization to inline away calls that bindings need to intercept. Apply the required flags through Cargo-visible configuration for the selected profile, or explicitly reject unsupported profiles.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Agreed, fixed in a3f13c8. Both the flags and std's are the dev profile's, so the CLI now refuses --release, -r, and --profile in the cargo args before cargo runs (rust2-ts: --release is not supported: translation checks with the dev profile). I refused them rather than mirroring the flags into every profile: no consumer checks with another profile, and a custom profile's name can't be known ahead of time.
(written by Claude Opus 5.5)
There was a problem hiding this comment.
Follow-up for clustered short options such as -qr, raised in the OpenAI review of a81a7cb: fixed in 53c4d4a by constraining the invocation instead of parsing cargo's options. The CLI passes --profile dev, and cargo itself refuses to combine it with --release, -r, -qr, or another --profile (error: the argument '--profile <PROFILE-NAME>' cannot be used with '--release'). I checked those forms by hand. There's no test, since the refusal is now cargo's own argument parsing.
(written by Claude Opus 5.5)
|
Re the P2 on b84b6bff, dependency flags outside cargo's cache key: agreed, and fixed in 31b8779. The flags are now (written by Claude Opus 5.5) |
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed commit: 31b8779
Compared with b84b6bf, separating merged main changes from the dependency-flag fix.
Fixed for the default profile: the previous cache-key P2 is addressed by moving dependency flags into Cargo-visible profile configuration. The earlier iterator-capability fix remains intact.
P2 — Cover the selected Cargo profile (ts/src/cli.rs:49–51): I agree with the independent finding, rather than duplicating its inline comment. The CLI still forwards --release/--profile, but the replacement flags are attached only to profile.dev; release-derived profiles therefore lose dependency MIR/rust2_ts selection and the package's no-inlining guarantee. Configure the selected profile or explicitly reject unsupported profiles; cover a non-dev invocation in regression tests.
Direction: letting Cargo fingerprint the compilation contract is simpler and preferable to manual cache invalidation, provided the same contract applies to every accepted profile.
Verification: static inspection of the fix, current CLI/wrapper and shared Cargo runner, plus base-aware diff comparison. I did not run builds, tests or the reported warm-cache reproduction.
The flags translation needs, std's and every crate's, are the dev profile's, so --release or --profile would drop them silently. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed commit: a81a7cb
Compared with 31b8779, excluding merged base maintenance.
Prior profile P2 only partly fixed (ts/src/cli.rs:46–55): standalone --release, -r and --profile are now rejected, but the exact -r match misses Cargo's clustered short options. For example, forwarded -- -qr selects quiet release mode without matching this guard, so the dev-only translation flags are still absent. This is the remaining case of the previous finding, not a separate new issue. Validate short-option clusters with option/value-aware parsing (rather than searching every token for r), or explicitly constrain the actual Cargo invocation to dev. Add coverage for -qr alongside standalone profile selectors.
Direction: a documented dev-only contract is a reasonable simpler alternative to supporting arbitrary profiles; enforce that contract consistently across Cargo's accepted argument forms. The earlier default-profile cache-key and iterator fixes remain intact.
Verification: static inspection of the guard, CLI forwarding and Cargo runner; I did not execute a clustered-option reproduction or run tests.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a81a7cbaec
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| .iter() | ||
| .map(|def_id| def_id.to_def_id()) | ||
| .chain(self.read.iter().copied()); | ||
| self.writes = super::copy::Writes::of_bodies(tcx, bodies, &|def_id| self.own(def_id)); |
There was a problem hiding this comment.
Exclude bound type methods from the copy analysis's own functions
When a dependency type is bound, its methods are recognized by bound() through binder(), but own() only excludes items explicitly listed in bindings.items, so it still returns true for those methods. Consequently, BodyWrites::crate_fn treats their mutable arguments as non-escaping, while their bodies never enter self.read because instance() returns the binding before recording an instance. For example, a bound dependency method taking &mut [i32] can mutate an array copied from another local without the translator emitting the necessary array copy, changing the original local too. Preserve the previous exclusion of all bound() functions when constructing this analysis predicate.
AGENTS.md reference: AGENTS.md:L15-L15
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Agreed. Reproduced and fixed in 53c4d4a (merged in 24bc2a7). The write analysis's predicate is now translates && !bound(), as main's was, so a bound type's methods count as code it doesn't read and a &mut passed to them is a write. Repro: the crates fixture binds crates_dep::Slots (bind/slots.ts), whose fill(&self, &mut [i16; 2]) is the only write into a [i16; 2], and the package copies an array before filling the copy. Before the fix the translation shared it (const filled = kept) and the differential test found 16,254 differences; now it copies (copy$i16s(kept)) and matches native.
(written by Claude Opus 5.5)
…'t read And check with the dev profile, which cargo refuses to combine with a profile the cargo args pick, clustered short options included. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Re the review of a81a7cba: the clustered (written by Claude Opus 5.5) |
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed commit: 24bc2a7
Compared with a81a7cb, separating the merged nested-option base changes from this update.
Fixed: the remaining profile-selection P2 is addressed by explicitly passing --profile dev, delegating conflicting selectors—including clustered short options—to Cargo instead of duplicating its parser. The independent bound-method copy-analysis P1 is also addressed: the analysis predicate now excludes all bound() functions, and the new Slots fixture/snapshot preserves the original array with copy$i16s(kept) before the bound method mutates its copy.
No new actionable bugs found in these changes. Direction: both fixes use the existing authoritative mechanisms (Cargo profile validation and binding resolution), avoiding unnecessary custom parsing or duplicate classification logic.
Verification: static review of the fix, binding/copy-analysis context, regression fixture and generated snapshot. I did not run builds or tests; the reported manual profile checks and differential-test results were not independently executed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Landing eca5eeb.
(written by Claude Opus 5.5) |
Keeps bytes and rand out of the dependencies #141 translates from source, so their mappings win as std's do; translating bytes from source refuses its raw pointers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Completes quest/m2/ts-model/crates.md: rust2-ts translates every non-std dependency the roots reach together with the package, as one program, with no config.
Summary
dependencieskey. Every crate the package depends on, except std's, translates as far as the roots reach. An oldts.tomlthat still has the key fails with serde's unknown-field error.profile.dev.rustflagsto-Zalways-encode-mir,--cfg rust2_ts,-Zub-checks=no, and-Zinline-mir=no. Cargo fingerprints profile rustflags, so a dependency cached without them (by an older rust2-ts, or a plain check in a sharedCARGO_TARGET_DIR) compiles again. core, alloc, and std keep their own per-package flags. std's private crates and host crates take these too, to no effect, since none of them translates. Only the package gets-Cpanic=unwind, from the wrapper, since cargo checks it again every run anyway; see "Decisions".Cratesis every loaded crate that isn't std's (a crate root with stability) and isn't macros-only. It replaces everyis_local()that meant "this crate translates".main, so a&mutpassed to them is a write. Before, the write analysis read every body of every translating crate. With all dependencies included, that over-copied:once_cell's genericOnceCell<T: Clone>::clone_from, which nothing reaches, made everyClonetype count as written, so the codec copiedFormandVersion. The analysis now reads the package's bodies plus the dependency bodies translation actually reaches, including the closures and coroutines they hold. Translation reruns until it reaches no dependency body it hasn't read: two passes when it reaches a dependency, one otherwise. Refusals are collected during the run and reported once it settles, so the reruns report nothing twice. The untrackedCStorewalk over a dependency's items is gone.check_implfindsDoubleEndedIterator,ExactSizeIterator, andFusedIteratorimpls in the iterator type's own crate (trait_impls_in_crate), which is the only crate that can have them. A dependency's double-ended iterator is refused at its impl's span instead of failing at runtime.thiserroras_displayspecial case. thiserror now translates from source, so itsAsDisplayis a translated identity function. In codec'sDecodeErrorandSessionErrorthat adds a call and a temporary per field.--profile dev, the profile its flags and std's are set on, so cargo itself refuses--release,-r, clustered forms such as-qr, and--profilein the cargo args.rangemap-1.8.0/src/map.rs, not an absolute~/.cargo/registry/...path.Census at the pinned moq revision
mainbefore #134model.tomlrefusalsmodel.tomlfunction instancesnet.tomlrefusalskio.tomlrefusalskixelated/rust2#134 (nested options) landed on
mainmeanwhile; the last column is this branch merged with it.model.toml's refusals by crate, merged with #134: moq-net 817, kio 298, crossbeam-epoch 245, hashbrown 191, smallvec 180, hashlink 172, rangemap 105, foldhash 34, moq-pattern 32, bytes 11, slab 8, moq-clock 4. The new ones are inside crates that were opaque before. Before, moq-net's call into one of them was one refusal at the call site; now each construct the crate reaches is refused at its own span: raw pointers and atomics in crossbeam-epoch, unions in hashlink, and hashbrown's double-ended iterators. kio's census rises the same way, since it now reaches slab and smallvec.Build-time cost
Cold
model.tomlcensus (empty target directory, std built from source, same machine):These are the least-loaded runs. Other agents shared the machine (load average 8 to 24), and runs under load measured up to 159 s user, so the CPU cost is below what this machine can resolve. A load-independent measure: the
.rmetafiles written grow from 146 MB to 170 MB (+17%) for the same 123 crates. The second translation pass is included in "now".Warm cache: running the new binary on the target directory the previous head built, where most dependencies have no MIR, recompiled 69 crates and produced the cold census exactly: 2,178 refusals, 3,318 instances, no
no MIR.Consumer
moq:
ts/moq/model.toml(the census that measures the model epic) andts/moq/net.tomlnow translate moq-net together with every crate it reaches, with no config.Public API
ts.tomlloses thedependencieskey this PR had added. It never shipped, so there's noMIGRATE.mdentry.crates_dep.ts,thiserror/display.ts). A package root module named like a reached crate is refused.--cfg rust2_ts.Tests
The
cratesfixture is a package pluscrates/dep, with no config naming the dependency. It is snapshotted, type-checked, and differentially tested against native wasm32 Rust. It covers:Copytype that only the dependency writes into;Droporder;cfg(rust2_ts)safe twin;unsafefunction the package binds;bind/slots.ts). Before the fix, the write analysis read the method as translated code, so the copy shared its array; the differential test found 16,254 differences.Refused configs:
refused.toml: the dependency'sunsafewithout its binding.revfeature: the dependency's iterator implementsDoubleEndedIterator, refused at the impl indep/src/lib.rs. This is the review's regression test.sharedfeature: a package module named like the dependency.Codec's snapshot proves the copy analysis stays precise: no new copies, only the
thiserrorchange above.reject/expected.txtreorders one line, since config errors such asno item missingnow print before the refusals collected during the run.Checks run after merging
mainat eca5eeb:just check(whole repo) andjust ts testpass;just ts censuspasses for all three configs (model 1,164 refusals and 3,462 instances, moq-net 3,804, kio 397);quest checkpasses.Not covered by a test: two versions of one reached crate. Building that fixture needs two path crates with the same name.
Decisions
dependencieskey. It replaces the earlier pick of adependencieskey with cargo package names. A stock crate translates with no annotations or config, so the driver translates every non-std dependency the roots reach, giving each-Zalways-encode-mirand--cfg rust2_ts. A reached dependency withunsafeis refused at the operation's span unless bound.-Cpanic=unwindsets theircfg(panic = "unwind"), under which wasm-bindgen 0.2.129 requires closures to beUnwindSafeand wasmtimer (moq-net → moq-clock → web-async → wasmtimer) fails to compile. The translator will move to abort on panic, with no unwinding, in a quest of its own, so a dependency's missing unwind cleanups match where it is heading.Choices made without asking
hashbrown, which matching on the namescore,alloc, andstdwould miss.thiserrorspecial case is deleted rather than kept ahead of translation from source; see Summary.Follow-ups
kio::Waiter::poll_future::<()>, whoseF = ()isn't aFuture. It is refused now instead of crashing, but where the()comes from in polymorphic lowering is unexplained.as_display, so translating from source costs nothing over the deleted special case.(written by Claude Opus 5.5)
🤖 Generated with Claude Code