Skip to content

feat(ts): drop the closures std's iterators and collections are done with - #139

Merged
kixelated merged 13 commits into
mainfrom
quest/m2/ts-drop
Oct 28, 2026
Merged

kixelated merged 13 commits into
mainfrom
quest/m2/ts-drop

Conversation

@kixelated

@kixelated kixelated commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Finishes quest/m2/ts-drop.md: lands its closure leftovers, moves the rest to quest/m3/ts-drop-order.md, and deletes it.

Problem

The quest's first leftover: a closure that owns a value that drops, given to std's iterators or collections, was refused. An adaptor (map, filter, ...) had no way to drop the closure it holds, so dropping the adaptor refused. successors of items that drop refused too, since nothing dropped the item it holds next. Also, a function that calls the closure and then lets it go (any, find, position, a collection's retain) would have leaked it, so it refused too.

Consumer: moq-net's model, per the quest. A fresh census at the pinned moq revision (88448d9) on main shows no drop refusal in moq-net, its model, or kio. The only drop-adjacent calls left are MaybeUninit::assume_init_drop (ts-unsafe) and thread::panicking (ts-shared). So this is generic std coverage, the kind the maintainer asked to plan anyway.

Approach

  • Adaptors. rt.Iter's closure-holding adaptors share a Calling base class. Its [Symbol.dispose] drops the wrapped iterator, then the closure, and the closure still drops if the iterator's drop panics. That is the order of std's fields (Map { iter, f } and the rest). The same holds for rt.iterFromFn, and for rt.iterSuccessors, which now also takes a dropper for the item it holds next. It drops that item before its closure, and drops the item it passed to a closure that panics. This settles the "Next" note that #142 added to the quest. The translator accepts a dropping closure kept by a std function's output only if that output is one of these adaptors (iter::holds_closure). disposable uses the same rule for std iterator types.
  • Calls that keep nothing. Some std functions call a closure by Fn or FnMut, keep nothing of it (its type is in neither the output nor another input), and take nothing else that drops by value. Their closure drops as the call returns or unwinds: rt.dropAfter(f, () => call). Rust drops a function's parameters as it ends, after anything the function dropped itself, so nothing is ordered between the two. When more than one such closure is passed, the last parameter drops first.
  • Still refused, now with a span:
    • A consumer that std folds (count, fold, sum, collect, ...) of an iterator that holds a dropping closure beside anything else that drops. std's fold drops an adaptor's closure as the leaf fold ends, before the source, but dropping the adaptor drops it after (folds and held in iter.rs).
    • A dropping closure given to a function that also takes another dropping value by value (for_each of items that drop).

Proof: fixtures/task/src/drops/closures.rs adds the iterated scenario. The drop scenarios go from 8 panic points to 11 so that successors panics at a point of its own (253 cases still fit a byte). Each point runs through the differential test against wasm32 and the unwinding test against the host:

  • map, filter in a for, a skip_while/take_while/inspect chain, filter_map + flat_map, map_while
  • from_fn; successors of guards, dropped holding one
  • any, find, position, a folded sum and for_each, and max_by_key over items made in flight
  • peekable().count(), skip(1).count() and skip(1).last() of a map, which drop the inner adaptor before the one holding it: its closure drops once
  • B-tree and deque retain

Each piece was broken on purpose, and a test caught it:

Mutation Differential (wasm32) Unwinding (host)
no closure drop in Calling 45 fails
closure dropped before the iterator 45 fails
dropAfter drops nothing 45 fails
dropAfter drops only on return, not on a panic 0 51
from_fn without dispose 45 fails
successors without dispose 45 fails
successors' held item not dropped 142 fails
successors' closure dropped before its item 142 fails
successors' item not dropped when its closure panics 0 17
the new fold refusal lifted, map(..).count() over guards 0 20
Calling drops on every dispose, not just the first 142 fails

Impact

Census at the pinned moq revision, main then this branch: unchanged at 9328 (moq-net), 3203 (model), and 328 (kio) refusals, none about drops in either.

Bench, measured before the successors change (the codec never calls successors, so its bundle tree-shakes it out) (just ts bench --route base=..., base being this branch's translation with main's rt/iter.ts): every codec, iterator, and handoff workload is within A/A noise in Node, Bun, Chromium, and Firefox. The codec's minified bundle grows 135 bytes raw / 39 gzip (32733 to 32868 / 10287 to 10326): iterMap and iterFilter now carry the closure-dropping Calling class. The handoff bundle is unchanged.

Public API:

  • The translator's output: a closure that owns a value that drops, given to the adaptors above or to a std function that only calls it, translates instead of refusing.
  • rt.iterSuccessors takes an optional dropper for its items.
  • New runtime export dropAfter. The runtime ships with each translation, so no consumer calls it by hand.
  • Newly refused, with a span: a consumer that std folds of an iterator holding a dropping closure beside anything else that drops. Before this PR, such iterators failed earlier, at the closure.

Choices made

  • Two closures in one call. Several spent closures in one call drop last parameter first, as a function's parameters do. No mapped std function takes two, so no test reaches that order.

  • How the fold refusal counts. It treats two dropping closures in one chain as a conflict even where std's order would match rt's (map(f).filter(g) over a borrowed slice). Following std's order per adaptor (TakeWhile, Chain, Peekable each fold their own way) would mean mirroring each adaptor's fold in the runtime.

  • The reject fixture. owning_closure and called_closure now cover what still refuses: a count that folds a closure over owned items, for_each over owned items, and or_insert_with. owning_lazy goes: nothing in it refuses anymore.

  • Review fixes (Codex). Calling and SuccessorsIter drop what they hold once, since some consumers drop an inner adaptor before the one holding it. Only a closure that a function trait bounds skips the operand drop check, so a Pattern closure that drops (str::contains) refuses again; it is covered in the reject fixture. A flat_map whose source is fused keeps its closure once it ends, as std's Fuse<Map<I, F>> does.

Alternatives

  • Each consumer mirrors std's fold. Give every rt.Iter a fold that reproduces std's, so by-value consumers drop closures where std's fold does, rather than refusing. This removes the remaining fold refusal for items and closures alike, at the cost of reimplementing about 20 adaptors' fold. No census instance needs it.
  • Droppers instead of rt.dropAfter. Pass each runtime function a dropper for its closure: every runtime function that takes a closure would grow a parameter, to serve the rare closure that drops.

Decision

The maintainer decided on 2026-10-27, picking option 1 of the three proposed:

  1. Land this, move the remaining refusals to a new m3 quest, and delete ts-drop.
  2. Land this, keep the remaining refusals here, and leave ts-async blocked.
  3. Continue here with a per-adaptor fold.

So the remaining refusals, none of which appears in moq's census, move to quest/m3/ts-drop-order.md. It is listed in m3's Required and records that rust2 aims for all of std. quest/m2/ts-drop.md and every link to it are deleted. That unblocks ts-async and ts-shared, and it drops the entry from the model epic's Required. ts/README's "Not yet" links to the new quest.

Follow-ups

  • Created: quest/m3/ts-drop-order.md, which holds the remaining refusals (fold paths, hash-map halves, trusted zip, flatten, B-tree collect, in-place zip collect, DebugList::entries).
  • ts/rustc-ice-2026-10-27T22_10_28-1632761.txt, a rustc crash dump committed by #130, gets its own PR.

(written by Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 3 commits October 27, 2026 17:13
Quest-Claim: caa8b83e2a77590c
…with

A closure that owns a value that drops, given to a std iterator adaptor
(`map`, `filter`, `filter_map`, `skip_while`, `take_while`, `map_while`,
`inspect`, `flat_map`, `from_fn`, `successors`), drops after the
iterator it wraps, as std's fields order them. Given to a function that
calls it by `Fn` or `FnMut`, keeps nothing of it, and takes nothing else
that drops by value (`any`, `find`, `position`, a collection's `retain`),
it drops as the call returns or unwinds (`rt.dropAfter`). A consumer that
std folds still refuses an iterator holding such a closure beside
anything else that drops, since `fold` drops the closure before the
source.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Drop in the TypeScript translator feat(ts): drop the closures std's iterators and collections are done with Oct 28, 2026
kixelated and others added 5 commits October 27, 2026 19:26
`rt.iterSuccessors` takes what drops its item and drops the one it
holds next before its closure, as std's fields drop, and the item it
passed to a closure that panics. This lifts the last `successors`
refusal, so the reject fixture's `owning_lazy` goes. The task fixture's
drop scenarios gain three panic points so `successors` panics at one of
its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
moq-net, its model, and kio have no drop refusal left, so the drop quest
is done; what it still refuses, none in moq's census, moves to
quest/m3/ts-drop-order.md, as rust2 aims for all of std. ts-async,
ts-shared, and the model epic no longer wait on it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review October 28, 2026 03:06
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-28T08:05:48.255239Z 51bc42a New commits
🔒 Security Review ✅ Completed 2026-10-28T08:06:28.226365Z 51bc42a New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d9cc96db59

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ts/src/lower/library.rs Outdated
inputs.into_iter().find(|&ty| b.t.drops(ty))
inputs
.into_iter()
.find(|&ty| !matches!(ty.kind(), ty::Closure(..)) && b.t.drops(ty))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve the drop check for closures passed as patterns

When a string predicate owns a dropping value, excluding every closure here silently accepts calls that still leak their captures. For example, "abc".contains(move |c| { guard.touch(); c == 'a' }) previously refused translation, but now maps to strContains without any closure cleanup. called_closures does not handle this case because the parameter is bounded by Pattern, not a function trait, and neither strContains nor its Searcher disposes the predicate. Keep rejecting these operands until their mappings explicitly drop them.

AGENTS.md reference: AGENTS.md:L15-L15

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in db3a53b. Reproduced first: pattern_closure in the reject fixture (s.contains(move |c: char| ...) owning a Noisy) translated silently. Now only a closure whose parameter a function trait bounds skips dropping_operand (called_by in library.rs, shared with called_closures), so a Pattern closure that drops refuses again, and the reject snapshot holds that refusal.

(written by Opus 5.5)

Comment thread ts/runtime/rt/iter.ts
Comment on lines +113 to +117
override [Symbol.dispose](): void {
both(
() => dispose(this.inner),
() => dispose(this.f),
);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Prevent consumers from disposing adaptor closures twice

For an accepted chain such as [1, 2].iter().map(move |n| { guard.touch(); *n }).peekable().count(), this unconditional cleanup drops the captured guard twice. The existing count implementation calls consumed(it.inner, ...) for Peekable, disposing the map, and then ownedCount disposes the peekable, which disposes that same map again. A runtime reproduction logs two mapper drops, while native Rust logs one; Skip and last have similar paths. Transfer ownership in those consumers or make the adaptor cleanup single-use, including when disposal panics.

AGENTS.md reference: AGENTS.md:L15-L15

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in db3a53b. Reproduced first: the task fixture's iterated scenario now runs map(..).peekable().count(), map(..).skip(1).count(), and map(..).skip(1).last() with closures that own guards, which gave 142 differences. Calling and SuccessorsIter now drop what they hold only the first time, a panic included, since the flag is set before either drop runs. That fixes every consumer that drops an inner iterator before its holder, not just these three. Differential and unwinding tests: 0 differences. Breaking the guard brings back the 142.

(written by Opus 5.5)

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: d9cc96db592be3d9c00cb8083cfccae37e0895dc.

Two actionable issues confirmed by tracing the changed lowering and runtime; both already have independent inline findings, so referencing them rather than duplicating:

  • P1, ts/src/lower/library.rs:583-585: blanket exclusion of closure operands lets Pattern-bound string predicates bypass the drop check, although called_closures handles only function-trait bounds and the string runtime never disposes them. Preserve rejection until cleanup is implemented. Existing finding.
  • P1, ts/runtime/rt/iter.ts:113-117: peekable().count() and skip().count()/last() can dispose an inner mapper through consumed, then dispose it again through their outer iterator. The newly supported captured guards therefore drop twice. Ownership must be consumed once, including on panic; the new SuccessorsIter closure cleanup needs the same audit. Existing finding.

Direction: shared adaptor cleanup and a conservative refusal for unimplemented fold order are preferable to duplicating every fold here. The added differential/panic scenarios are useful, but should include nested by-value consumers and non-function-trait closure operands before widening acceptance.

Verification: static review of the diff, lowering, runtime consumers and fixture coverage only; builds and tests were not run. No additional distinct actionable bugs found.

(Written by OpenAI)

… that drops

`count` of a `peekable` or `skip`, and `last` of a `skip`, drop the
adaptor they wrap and then the one holding it, so `Calling` and
`SuccessorsIter` now drop what they hold only the first time. Only a
closure that a function trait bounds skips the check for operands that
drop, since only those does `called_closures` decide: one given as a
`Pattern`, as `str::contains` takes, refuses again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: db3a53b62c675ab5ed4f5cb617e510f9b10b22f7, focusing on changes since d9cc96db592be3d9c00cb8083cfccae37e0895dc.

Both issues referenced in the previous review are fixed in the inspected code:

  • Pattern closure leak: called_by now consistently limits the drop-check exemption to function-trait parameters; the new reject fixture covers a dropping contains predicate.
  • Repeated adaptor cleanup: Calling and SuccessorsIter mark themselves dropped before cleanup, preventing a second disposal even when the first panics. Added fixture cases exercise nested Peekable and Skip consumers.

No new actionable bugs found in this update. The shared bound check and single-use cleanup are focused fixes without broadening the supported surface.

Verification: static inspection of the delta and relevant lowering/runtime paths; builds and tests were not run.

(Written by OpenAI)

# Conflicts:
#	ts/README.md
#	ts/fixtures/reject/census.md
#	ts/fixtures/reject/expected.txt
#	ts/src/lower/library.rs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ff6e4ab907

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ts/src/lower/iter.rs
"adapters",
_,
"Map" | "Filter" | "FilterMap" | "SkipWhile" | "TakeWhile" | "MapWhile" | "Inspect"
] | ["iter", "adapters", "flatten", "FlatMap"]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include the mapper's captures when deciding FlatMap fusion

When the source iterator drops nothing but the flat_map closure owns a dropping value, accepting FlatMap here exposes an early-drop regression. The flat_map mapping computes fused(b, self_ty) from only the source, so a borrowed slice produces false even though its internal Map now drops the closure. FlattenIter.nextOuter consequently disposes the mapper on exhaustion, whereas Rust retains it until the fused flat-map iterator is dropped. In a native/runtime reproduction using [1u32].iter().flat_map(move |&n| { guard.touch(); [n] }), exhausting next() and then logging exhausted drops the guard before the log in TypeScript but after it in Rust. Include the mapper's dropping captures in the fusion decision before accepting this case.

AGENTS.md reference: AGENTS.md:L15-L15

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in c54d1b6. Reproduced first: the task fixture's iterated scenario now ends a [1u32].iter().flat_map(..) whose closure owns a guard, logs, then drops it, which gave 142 differences against native Rust. fused now also counts what flat_map's closure drops, since std's FlatMap keeps Fuse<Map<I, F>> whole once a fused source ends. Differential and unwinding tests: 0 differences.

(written by Opus 5.5)

std's `FlatMap` holds `Fuse<Map<I, F>>`, which keeps the mapped source, `F` included,
once it ends if `I` is a `FusedIterator`. `fused` looked only at the source, so a
`flat_map` over a borrowed slice dropped its closure as it ended.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: c54d1b687ec38fabb0628cc6a66496b0a8d02b64, focusing on the substantive update since the last review after accounting for main merges.

The independent FlatMap early-drop finding is fixed: ts/src/lower/iter.rs now includes the mapper closure's drop requirement when selecting fused retention. The runtime keeps the wrapped MapIter through exhaustion and disposes its closure when the flat-map is dropped. The added fixture exhausts a borrowed-slice flat-map, logs, then drops it, covering that ordering. The previously reviewed Pattern and repeated-cleanup fixes remain intact.

No new actionable bugs found in this update. Extending the existing fusion decision is a focused solution without adding runtime complexity.

Verification: static inspection of the delta, generated fixture, and relevant runtime exhaustion/disposal paths; builds and tests were not run.

# Conflicts:
#	ts/README.md
#	ts/fixtures/reject/census.md
#	ts/fixtures/reject/expected.txt
#	ts/fixtures/reject/ts.toml
@kixelated

Copy link
Copy Markdown
Collaborator Author

Merge summary:

  • Merged main three times as it moved. The conflicts were in the reject fixture's snapshots (regenerated), ts.toml roots (kept pattern_closure and main's parts_summed; owning_lazy and duration_assign are gone on both sides), the ts/README "Not yet" list (kept both sides' entries, with the drop-order link pointing to quest/m3/ts-drop-order.md), and library.rs (kept main's takes_none/sentinel_option beside this PR's Closures). After main's format-spec change, drops/closures.ts now formats Option<u32> with u32Debug.
  • Fixed Codex's P2 in c54d1b6: a flat_map over a fused source keeps its closure once it ends, as std's Fuse<Map<I, F>> does. Reproduced first: 142 differences, now 0.
  • Codex reviewed head 51bc42a with no findings. CI passed. quest check passed against main. The local merge with main's newer fix(ts): view a Vec by its own element type #158 is clean and just ts test passes on it.

(written by Opus 5.5)

@kixelated
kixelated merged commit 400914a into main Oct 28, 2026
1 check passed
@kixelated
kixelated deleted the quest/m2/ts-drop branch October 28, 2026 08:44
kixelated added a commit that referenced this pull request Oct 28, 2026
…-more

Resolved by hand against #139: main's drops sweep is now 23 scenarios
by 11 panic points, 253 of the 256 values its byte holds, so `edited`
can't be a 24th. It runs as task op 13 instead, at `arg % 8`, with
`held` kept at op 12 so the existing scripts read the same.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant