diff --git a/packages/codemode/README.md b/packages/codemode/README.md index 2cc67fe36a4b..b48a58a816b8 100644 --- a/packages/codemode/README.md +++ b/packages/codemode/README.md @@ -246,12 +246,12 @@ CodeMode executes a deliberately bounded JavaScript subset. It supports: - Regular expressions - `/literals/` and `new RegExp(...)` with `test`/`exec` (stateful `lastIndex` for `g`), plus string `match`/`matchAll`/`replace`/`replaceAll`/`split`/`search` with patterns. Match results are arrays carrying `index` and named `groups` as own properties (`input` is omitted). `replace` and `replaceAll` accept function replacers with captures, offset, input, and named groups; callbacks run sequentially, may await tool calls, and have their results coerced to strings. Invalid patterns, invalid flags, and missing-`g` calls fail with catchable errors that say what was wrong and how to fix it (escaping hints, the exact `/pattern/g` to write). Patterns run on the host engine, so pathological backtracking is bounded only by the execution timeout. - `Map` and `Set` - construction from entries/arrays/strings, `get`/`set`/`add`/`has`/`delete`/`clear`/`size`/`forEach`, and `keys`/`values`/`entries` returning **arrays** (not iterators). - URL helpers - `URL` resolution and mutation, linked `URLSearchParams`, `URL.canParse`/`URL.parse`, URI and URI-component encoding/decoding, and query parameter construction, lookup, mutation, sorting, callbacks, and materialization. URLSearchParams iteration methods return arrays, matching the Map/Set convention. -- First-class promises - an un-awaited `tools.ns.tool(...)` is a promise value whose call starts immediately on a supervised fiber; `await` resolves it (awaiting a non-promise value is a no-op, and `return tools.ns.tool(...)` resolves like an async-function return). `Promise.all`, `Promise.allSettled`, and `Promise.race` accept any array mixing promises and plain values (built inline, beforehand, or via spread); `Promise.resolve`/`Promise.reject` construct settled promises. `Promise.allSettled` rejection reasons are the same plain `{ name?, message }` data a `catch` binding sees, and `Promise.race` interrupts its losing in-flight calls. At most 8 tool calls run concurrently. When a program completes, still-running un-awaited calls are awaited before the execution ends; a failure from a call that was never awaited surfaces as an unhandled-rejection diagnostic. +- First-class promises - an un-awaited `tools.ns.tool(...)` is a promise value whose call starts immediately on a supervised fiber; `await` resolves it (awaiting a non-promise value is a no-op, and `return tools.ns.tool(...)` resolves like an async-function return). Promises support `then`/`catch`/`finally`, including returned-promise flattening and rejection recovery. `Promise.all`, `Promise.allSettled`, and `Promise.race` accept any array mixing promises and plain values (built inline, beforehand, or via spread); `Promise.resolve` preserves an existing promise or wraps a plain value, and `Promise.reject` creates a rejected promise. `Promise.allSettled` rejection reasons use the same values a `catch` binding sees, and `Promise.race` lets losing promises continue. At most 8 tool calls run concurrently. When a program completes, still-running un-awaited calls are awaited before the execution ends; a failure from a call that was never awaited surfaces as an unhandled-rejection diagnostic. - `throw value` and `throw new Error(message)` for explicit program failure. `Error` (and `TypeError`/`RangeError`/`SyntaxError`/`ReferenceError`/`EvalError`/`URIError`) are real constructors, callable with or without `new`; error values are plain `{ name, message }` data that additionally satisfy `instanceof Error` (a specific type matches itself and `Error`, as in JS). Every caught failure - thrown errors, interpreter runtime errors, and tool failures - is `instanceof Error` in a `catch` block; a thrown non-error value (`throw "text"`) is not, matching JS. Caught failures carry the `name` the equivalent real-JS failure would have - `JSON.parse` and invalid regex patterns produce a `SyntaxError` (satisfying `instanceof SyntaxError`), an unknown identifier a `ReferenceError`, assigning to a constant a `TypeError`, a bad `normalize` form a `RangeError`; failures with no specific analogue (including tool failures) are named `"Error"`. `instanceof` also recognizes `Date`, `RegExp`, `Map`, `Set`, `URL`, `URLSearchParams`, `Array`, `Object`, and `Promise`; any other right-hand side is a catchable error. Inside a program, standard-library values stay live everywhere: the internal data checkpoints (`Object.*` helpers, spread, coercion inputs) preserve the instances, so `Object.values({ d: date })[0].getTime()` and a spread copy of an object holding a Map keep working. Only at the host boundary (final result, tool arguments, `JSON.stringify`) do they serialize exactly as `JSON.stringify` would: Date and URL become strings (an invalid Date becomes `null`), while RegExp, Map, Set, and URLSearchParams become `{}`. Promise values never cross a data boundary: an un-awaited promise in a result or tool argument produces a diagnostic that says to await it, instead of serializing to `{}`. -It does not expose `eval`, dynamic imports, modules, classes, generators, timers, host globals, prototype mutation, custom promise constructors (`new Promise`), promise chaining (`.then`/`.catch`/`.finally` - `await` with `try`/`catch` is the supported style), or arbitrary method calls. Unsupported syntax returns an `UnsupportedSyntax` diagnostic with a source location when available. +It does not expose `eval`, dynamic imports, modules, classes, generators, timers, host globals, prototype mutation, custom promise constructors (`new Promise`), or arbitrary method calls. Unsupported syntax returns an `UnsupportedSyntax` diagnostic with a source location when available. CodeMode is an orchestration language, not a general JavaScript runtime. diff --git a/packages/codemode/codemode.md b/packages/codemode/codemode.md index f7f798f2e1b5..ce08add6e8f9 100644 --- a/packages/codemode/codemode.md +++ b/packages/codemode/codemode.md @@ -63,8 +63,10 @@ path lookup, namespace browsing, deterministic ranking, and pagination. ### Tool execution Calling a tool starts its Effect eagerly on a supervised fiber. The returned sandbox promise is run-once and can be -awaited directly or through the supported `Promise` combinators. At most eight tool calls execute concurrently. -Unfinished calls are drained before successful program completion, and an unhandled call failure becomes a diagnostic. +awaited directly, chained with `then`/`catch`/`finally`, or passed through the supported `Promise` combinators. At most +eight tool calls execute concurrently. +Unfinished tracked promises are drained before successful program completion, and an unhandled rejection becomes a +diagnostic. The public execution-policy knobs are `timeoutMs`, `maxToolCalls`, and `maxOutputBytes`. The package supplies no defaults because budgets are host policy. The interpreter also enforces fixed internal boundaries for tool-call @@ -109,6 +111,56 @@ MCP tools use this canonical path: they register as grouped tools and are deferr output schemas are preserved in generated signatures. Direct Core tools remain direct and are not ambient globals inside CodeMode. +## Promise Status + +The runtime currently provides eager, run-once promises for tool calls and async functions; `await`; +`then`/`catch`/`finally`; and chainable `all`/`allSettled`/`race`/`resolve`/`reject`. `Promise.all` rejects promptly while +siblings continue, and `Promise.race` leaves losers running as JavaScript does. Tracked work remains supervised in one +execution scope, at most eight tool calls run concurrently, and ordinary success or failure drains unfinished work before +closing. Timeout and external interruption cancel immediately instead. + +### Confirmed defects + +- [ ] Return rejected promises for invalid `Promise.all`/`allSettled`/`race` inputs instead of throwing during the call. +- [ ] Align handler callability with the values CodeMode reports as functions, or document the narrower callback + allowlist. For example, unsupported constructor-like callables are currently treated as absent handlers. + +### Deliberate deviations and open decisions + +- CodeMode drains unfinished work before ordinary success or failure closes. This keeps tool effects supervised, but a + race loser or fail-fast `Promise.all` sibling that never settles can hold execution open indefinitely when the host + supplies no timeout. +- Promise resolution unwraps only `SandboxPromise`; arbitrary `{ then(resolve, reject) }` values remain data. Full + thenable assimilation requires internal callable resolver values, first-settlement arbitration, recursive adoption, + and cycle detection. Decide whether that machinery belongs in the bounded runtime. +- `new Promise`, `Promise.any`, resolver APIs, subclasses/species, and the broader prototype surface are unavailable. + Consider `Promise.any` independently; custom constructors and subclassing are not current goals. +- Combinators currently accept arrays plus CodeMode's spreadable strings, Maps, and Sets, while documentation and + diagnostics describe array inputs. Choose and document one contract. +- `Promise.race([])` raises a clear error instead of creating a permanently pending promise. +- Rejection tracking is execution-scoped and checked at drain time, not an ECMAScript microtask-level unhandled + rejection model. + +### Covered regressions + +- Nested unreturned tool calls remain alive after an async function or `then` handler settles. +- Abandoned failing tool calls, async functions, and immediate `Promise.reject` values report unhandled rejections. +- Ordinary program failure drains pending work and preserves the original error; a rejecting race winner drains its slow + loser. +- Timeouts interrupt all in-flight promise fibers with parallel teardown, while host interruption propagates instead of + becoming a diagnostic. +- Promise reactions and plain, pending, or settled `await` continuations start in deterministic FIFO order. Nested + reactions preserve enqueue order, while an async reaction can suspend without blocking the next queued reaction. +- Awaiting the same promise twice settles it once. + +### Missing coverage + +- Nested unreturned work from `catch` and `finally` handlers. +- Abandoned chained and combinator rejections. +- External interruption while handled pending work remains. +- Never-settling race losers and fail-fast `Promise.all` siblings under an explicit timeout. +- Shared or duplicate promises across combinators, discarded inner chains, and detailed reaction ordering. + ## Intentionally Unsupported These are product boundaries rather than DSL backlog: @@ -148,9 +200,6 @@ the adapter TODO. Delete entries when completed. The supported JavaScript subset should grow when common model-generated code improves tool orchestration. These are current omissions to implement, not intentional product boundaries. -- [ ] Design proper multi-stage promise pipelines. Supporting `.then`, `.catch`, and `.finally` should preserve promise - assimilation, cancellation, failure handling, and concurrent per-item pipelines rather than adding syntax-only - shims. Consider `Promise.any` in the same pass. - [ ] Support async iteration and `for await...of`. Define behavior first for the runtime's supported promise and collection values, then extend it to bounded host streams when a stream boundary exists. - [ ] Support callback-bearing standard-library variants that models commonly generate: the mapper argument to diff --git a/packages/codemode/src/interpreter/model.ts b/packages/codemode/src/interpreter/model.ts index 08d47cd37e8c..3ee4a7d5ffe1 100644 --- a/packages/codemode/src/interpreter/model.ts +++ b/packages/codemode/src/interpreter/model.ts @@ -123,7 +123,7 @@ export type DiagnosticKind = export const OptionalShortCircuit: unique symbol = Symbol("codemode.optional-short-circuit") export const supportedSyntaxMessage = - "Supported orchestration syntax: tools.* calls (they return promises - resolve them with await), data literals, destructuring, optional chaining, template literals, conditionals, switch, loops (incl. for...of and for...in over object/array/tools keys), arrow functions, spread, try/catch, array methods (map/filter/find/findIndex/some/every/reduce/flatMap/forEach/sort/slice/concat/indexOf/lastIndexOf/at/flat/reverse/includes/join), string methods (incl. match/matchAll/replace/split with regular expressions), Date/RegExp/Map/Set/URL/URLSearchParams, URI encoding helpers, Object/Math/JSON helpers, captured console.log/warn/error/dir/table, and Promise.all/allSettled/race/resolve/reject over arrays mixing promises and plain values for parallel tool calls (promise chaining with .then/.catch is not supported - use await with try/catch)." + "Supported orchestration syntax: tools.* calls (they return promises - resolve them with await or .then/.catch/.finally), data literals, destructuring, optional chaining, template literals, conditionals, switch, loops (incl. for...of and for...in over object/array/tools keys), arrow functions, spread, try/catch, array methods (map/filter/find/findIndex/some/every/reduce/flatMap/forEach/sort/slice/concat/indexOf/lastIndexOf/at/flat/reverse/includes/join), string methods (incl. match/matchAll/replace/split with regular expressions), Date/RegExp/Map/Set/URL/URLSearchParams, URI encoding helpers, Object/Math/JSON helpers, captured console.log/warn/error/dir/table, and Promise.all/allSettled/race/resolve/reject over arrays mixing promises and plain values for parallel tool calls." export class InterpreterRuntimeError extends Error { readonly node?: AstNode diff --git a/packages/codemode/src/interpreter/runtime.ts b/packages/codemode/src/interpreter/runtime.ts index 2825e744b5f6..4e1d473475d5 100644 --- a/packages/codemode/src/interpreter/runtime.ts +++ b/packages/codemode/src/interpreter/runtime.ts @@ -1,5 +1,5 @@ import { parse } from "acorn" -import { Cause, Effect, Exit, Fiber, Semaphore } from "effect" +import { Cause, Deferred, Effect, Exit, Fiber, Queue, Scope, Semaphore } from "effect" import { DiagnosticCategory, ModuleKind, ScriptTarget, flattenDiagnosticMessageText, transpileModule } from "typescript" import { copyIn, @@ -148,6 +148,11 @@ const parseProgram = (code: string): ProgramNode => { return parsed as ProgramNode } +type PromiseReaction = { + readonly effect: Effect.Effect + readonly settlement: Deferred.Deferred +} + const publicErrorMessage = (message: string): string => message.replace(/\/(?:Users|home|private|tmp|var\/folders)\/[^\s"'`]+/g, "") @@ -219,7 +224,7 @@ const normalizeError = (error: unknown): Diagnostic => { } } -// Shared by catch bindings, Promise.allSettled rejection reasons, and Promise.race losers. +// Shared by catch bindings and Promise.allSettled rejection reasons. const caughtErrorValue = (thrown: unknown): unknown => { if (thrown instanceof ProgramThrow) return thrown.value if (thrown instanceof InterpreterRuntimeError) return createErrorValue(thrown.errorName, thrown.message) @@ -607,9 +612,13 @@ class Interpreter { private readonly toolKeys: (path: ReadonlyArray) => ReadonlyArray private readonly logs: Array private lastValue: unknown + // Every promise fiber belongs to the execution rather than the async function or handler + // that happened to create it. The execution scope still interrupts all work on teardown. + private readonly promiseScope: Scope.Scope + private readonly reactionQueue: Queue.Queue> // Caps how many eagerly forked tool calls run at once (the parallel-call concurrency cap). private readonly callPermits: Semaphore.Semaphore - // Fiber-backed promises whose settlement no program construct has observed yet. Successful + // Fiber-backed promises whose settlement no program construct has observed yet. Ordinary // program completion drains these (like a runtime waiting on in-flight work at exit) and // surfaces a never-awaited failure as an unhandled-rejection diagnostic. private readonly pendingSettlements: Set @@ -617,6 +626,8 @@ class Interpreter { constructor( invokeTool: (path: ReadonlyArray, args: Array) => Effect.Effect, toolKeys: (path: ReadonlyArray) => ReadonlyArray, + promiseScope: Scope.Scope, + reactionQueue: Queue.Queue>, logs: Array = [], shared?: { callPermits: Semaphore.Semaphore; pendingSettlements: Set }, ) { @@ -626,6 +637,8 @@ class Interpreter { this.toolKeys = toolKeys this.logs = logs this.lastValue = undefined + this.promiseScope = promiseScope + this.reactionQueue = reactionQueue this.callPermits = shared?.callPermits ?? Semaphore.makeUnsafe(TOOL_CALL_CONCURRENCY) this.pendingSettlements = shared?.pendingSettlements ?? new Set() globalScope.set("tools", { mutable: false, value: new ToolReference([]) }) @@ -668,7 +681,7 @@ class Interpreter { // top-level declarations (`let undefined = 5`, `const Object = ...`) shadow builtins like // JS module scope, instead of colliding with the seeded globals. this.pushScope() - return Effect.gen(function* () { + const evaluate = Effect.gen(function* () { self.hoistFunctions(program.body) let value: unknown = undefined let returned = false @@ -695,31 +708,44 @@ class Interpreter { // resolves before crossing the data boundary - `return tools.ns.tool(...)` works // without an explicit await, exactly as in JS. if (value instanceof SandboxPromise) value = yield* self.settlePromise(value) - yield* self.drainPendingSettlements() return value + }) + return Effect.gen(function* () { + const result = yield* Effect.exit(evaluate) + if (Exit.isFailure(result) && Cause.hasInterruptsOnly(result.cause)) { + return yield* Effect.failCause(result.cause) + } + + const drained = yield* Effect.exit(self.drainPendingSettlements()) + if (Exit.isFailure(result)) return yield* Effect.failCause(result.cause) + if (Exit.isFailure(drained)) return yield* Effect.failCause(drained.cause) + return result.value }).pipe(Effect.ensuring(Effect.sync(() => self.popScope()))) } // Awaits every fiber-backed promise the program abandoned (fire-and-forget tool calls), so // their work completes before the execution ends - mirroring a JS runtime waiting on // in-flight I/O at exit. A failure nobody could have handled becomes an unhandled-rejection - // diagnostic (interrupted calls, e.g. Promise.race losers, are ignored). + // diagnostic. private drainPendingSettlements(): Effect.Effect { const self = this return Effect.gen(function* () { + let unhandled: InterpreterRuntimeError | undefined while (self.pendingSettlements.size > 0) { const promise = self.pendingSettlements.values().next().value if (promise === undefined) break const exit = yield* self.observePromise(promise) - if (Exit.isSuccess(exit) || Cause.hasInterruptsOnly(exit.cause)) continue + if (Exit.isSuccess(exit) || Cause.hasInterruptsOnly(exit.cause) || promise.handled) continue + if (unhandled !== undefined) continue const failure = normalizeError(Cause.squash(exit.cause)) - throw new InterpreterRuntimeError( + unhandled = new InterpreterRuntimeError( `Unhandled rejection from an un-awaited promise: ${failure.message}`, undefined, failure.kind, ["Await promises so failures can be caught and handled."], ) } + if (unhandled !== undefined) throw unhandled }) } @@ -735,7 +761,7 @@ class Interpreter { } private createPromise(effect: Effect.Effect): Effect.Effect { - return Effect.map(Effect.forkChild(effect, { startImmediately: true }), (fiber) => { + return Effect.map(Effect.forkIn(effect, this.promiseScope, { startImmediately: true }), (fiber) => { const promise = new SandboxPromise(fiber) this.pendingSettlements.add(promise) return promise @@ -752,28 +778,49 @@ class Interpreter { // `await promise`: succeed with the fulfilled value or re-raise the failure so try/catch // observes it exactly like a synchronous throw at the await site. - private settlePromise(promise: SandboxPromise, node?: AstNode): Effect.Effect { + private settlePromise(promise: SandboxPromise): Effect.Effect { const self = this - return Effect.flatMap(this.observePromise(promise), (exit) => self.unwrapPromiseExit(promise, exit, node)) + return Effect.flatMap(this.observePromise(promise), (exit) => self.unwrapPromiseExit(exit)) } - private unwrapPromiseExit( - promise: SandboxPromise | undefined, - exit: Exit.Exit, - node?: AstNode, - ): Effect.Effect { - if (Exit.isSuccess(exit)) return Effect.succeed(exit.value) - // A call Promise.race interrupted after losing settles as a catchable program failure; - // any other interruption is execution teardown (timeout/host) and must keep propagating - // as interruption rather than becoming program-visible data. - if (promise?.interrupted === true && Cause.hasInterruptsOnly(exit.cause)) { - return Effect.fail( - new InterpreterRuntimeError( - "This tool call was interrupted because another value settled a Promise.race first.", - node, + private createReaction( + source: Effect.Effect, never, R>, + reaction: (exit: Exit.Exit, chained: SandboxPromise) => Effect.Effect, + ): Effect.Effect { + const settlement = Deferred.makeUnsafe() + const chained = new SandboxPromise(undefined, Deferred.await(settlement)) + this.pendingSettlements.add(chained) + return Effect.as( + Effect.forkIn( + Effect.flatMap(source, (exit) => + Effect.sync(() => + Queue.offerUnsafe(this.reactionQueue, { + settlement, + effect: Effect.suspend(() => reaction(exit, chained)), + }), + ), ), - ) - } + this.promiseScope, + { startImmediately: true }, + ), + chained, + ) + } + + private awaitValue(value: unknown): Effect.Effect { + return Effect.flatMap( + this.createReaction( + value instanceof SandboxPromise + ? Effect.exit(this.settlePromise(value)) + : Effect.succeed(Exit.succeed(value)), + (exit) => this.unwrapPromiseExit(exit), + ), + (continuation) => this.settlePromise(continuation), + ) + } + + private unwrapPromiseExit(exit: Exit.Exit): Effect.Effect { + if (Exit.isSuccess(exit)) return Effect.succeed(exit.value) return Effect.failCause(exit.cause) } @@ -1544,12 +1591,9 @@ class Interpreter { case "UpdateExpression": return this.evaluateUpdateExpression(node) case "AwaitExpression": { - // `await` resolves a promise value; awaiting anything else is a passthrough no-op, - // matching real JS semantics for non-thenables. - const self = this - return Effect.flatMap(this.evaluateExpression(getNode(node, "argument")), (value) => - value instanceof SandboxPromise ? self.settlePromise(value, node) : Effect.succeed(value), - ) + // Even an already-settled value resumes in a later reaction turn, after work that was + // already queued, matching JavaScript's await continuation ordering. + return Effect.flatMap(this.evaluateExpression(getNode(node, "argument")), (value) => this.awaitValue(value)) } case "NewExpression": return this.evaluateNewExpression(node) @@ -2230,7 +2274,11 @@ class Interpreter { ) } if (ref.name === "reject") { - return Effect.sync(() => new SandboxPromise(undefined, Effect.fail(new ProgramThrow(args[0])))) + return Effect.sync(() => { + const promise = new SandboxPromise(undefined, Effect.fail(new ProgramThrow(args[0]))) + this.pendingSettlements.add(promise) + return promise + }) } const items = Array.isArray(args[0]) ? args[0] : spreadItems(args[0]) @@ -2251,68 +2299,59 @@ class Interpreter { ? Effect.map(this.observePromise(item), (exit) => ({ index, item, exit })) : Effect.succeed({ index, item: undefined, exit: Exit.succeed(item) }), ) - return Effect.gen(function* () { - const remaining = [...observations] - const values: Array = [] - values.length = items.length - while (remaining.length > 0) { - const winner = yield* Effect.raceAll(remaining) - const position = remaining.indexOf(observations[winner.index]) - if (position >= 0) remaining.splice(position, 1) - if (Exit.isSuccess(winner.exit)) { - values[winner.index] = winner.exit.value - continue + return this.createPromise( + Effect.gen(function* () { + const remaining = [...observations] + const values: Array = [] + values.length = items.length + while (remaining.length > 0) { + const winner = yield* Effect.raceAll(remaining) + const position = remaining.indexOf(observations[winner.index]) + if (position >= 0) remaining.splice(position, 1) + if (Exit.isSuccess(winner.exit)) { + values[winner.index] = winner.exit.value + continue + } + for (const item of items) { + if (!(item instanceof SandboxPromise) || item === winner.item) continue + item.handled = true + self.pendingSettlements.add(item) + } + return yield* self.unwrapPromiseExit(winner.exit) } - yield* self.createPromise( - Effect.asVoid( - Effect.forEach( - items, - (item) => (item instanceof SandboxPromise ? self.observePromise(item) : Effect.void), - { concurrency: "unbounded" }, - ), - ), - ) - return yield* self.unwrapPromiseExit(winner.item, winner.exit, node) - } - return values - }) + return values + }), + ) } case "allSettled": { const observations = items.map((item) => - item instanceof SandboxPromise - ? Effect.map(this.observePromise(item), (exit) => ({ promise: item as SandboxPromise | undefined, exit })) - : Effect.succeed({ promise: undefined as SandboxPromise | undefined, exit: Exit.succeed(item as unknown) }), + item instanceof SandboxPromise ? this.observePromise(item) : Effect.succeed(Exit.succeed(item as unknown)), ) - return Effect.gen(function* () { - const outcomes: Array = [] - for (const observation of observations) { - const { exit, promise } = yield* observation - if (Exit.isSuccess(exit)) { + return this.createPromise( + Effect.gen(function* () { + const outcomes: Array = [] + for (const observation of observations) { + const exit = yield* observation + if (Exit.isSuccess(exit)) { + outcomes.push( + Object.assign(Object.create(null) as SafeObject, { status: "fulfilled", value: exit.value }), + ) + continue + } + if (Cause.hasInterruptsOnly(exit.cause)) { + // Execution teardown (timeout/host interruption), not a program-level rejection. + return yield* Effect.failCause(exit.cause) + } outcomes.push( - Object.assign(Object.create(null) as SafeObject, { status: "fulfilled", value: exit.value }), + Object.assign(Object.create(null) as SafeObject, { + status: "rejected", + reason: caughtErrorValue(Cause.squash(exit.cause)), + }), ) - continue } - const raceInterrupted = promise?.interrupted === true && Cause.hasInterruptsOnly(exit.cause) - if (Cause.hasInterruptsOnly(exit.cause) && !raceInterrupted) { - // Execution teardown (timeout/host interruption), not a program-level rejection. - return yield* Effect.failCause(exit.cause) - } - const thrown = raceInterrupted - ? new InterpreterRuntimeError( - "This tool call was interrupted because another value settled a Promise.race first.", - node, - ) - : Cause.squash(exit.cause) - outcomes.push( - Object.assign(Object.create(null) as SafeObject, { - status: "rejected", - reason: caughtErrorValue(thrown), - }), - ) - } - return outcomes - }) + return outcomes + }), + ) } case "race": { if (items.length === 0) { @@ -2326,28 +2365,26 @@ class Interpreter { ? Effect.map(this.observePromise(item), (exit) => ({ index, exit })) : Effect.succeed({ index, exit: Exit.succeed(item as unknown) }), ) - return Effect.gen(function* () { - // First settlement (fulfilled OR rejected) wins; the observations never fail, so - // racing them yields exactly that. Losing in-flight calls are then interrupted. - const winner = yield* Effect.raceAll(observations) - for (const [index, item] of items.entries()) { - if (index === winner.index || !(item instanceof SandboxPromise) || item.fiber === undefined) continue - item.interrupted = true - yield* Fiber.interrupt(item.fiber) - } - const winningItem = items[winner.index] - return yield* self.unwrapPromiseExit( - winningItem instanceof SandboxPromise ? winningItem : undefined, - winner.exit, - node, - ) - }) + return this.createPromise( + Effect.gen(function* () { + // First settlement (fulfilled OR rejected) wins. Losers continue like native + // promises, while a supervised drain keeps their failures handled and their work + // inside the execution lifetime. + const winner = yield* Effect.raceAll(observations) + for (const [index, item] of items.entries()) { + if (index === winner.index || !(item instanceof SandboxPromise)) continue + item.handled = true + self.pendingSettlements.add(item) + } + return yield* self.unwrapPromiseExit(winner.exit) + }), + ) } } } private invokeFunction(fn: CodeModeFunction, args: Array): Effect.Effect { - const invocation = new Interpreter(this.invokeTool, this.toolKeys, this.logs, { + const invocation = new Interpreter(this.invokeTool, this.toolKeys, this.promiseScope, this.reactionQueue, this.logs, { callPermits: this.callPermits, pendingSettlements: this.pendingSettlements, }) @@ -2390,6 +2427,9 @@ class Interpreter { args: Array, node: AstNode, ): Effect.Effect { + if (ref.receiver instanceof SandboxPromise) { + return this.invokePromisePrototypeMethod(ref.receiver, ref.name, args, node) + } if (typeof ref.receiver === "string") { if ( (ref.name === "replace" || ref.name === "replaceAll") && @@ -2426,6 +2466,86 @@ class Interpreter { throw new InterpreterRuntimeError(`Method '${ref.name}' is not available in CodeMode.`, node) } + private invokePromisePrototypeMethod( + promise: SandboxPromise, + name: string, + args: Array, + node: AstNode, + ): Effect.Effect { + const callback = (value: unknown) => { + if ( + !(value instanceof ToolReference && value.path.length > 0) && + !(value instanceof PromiseMethodReference) && + !(value instanceof CodeModeFunction) && + !(value instanceof IntrinsicReference) && + !(value instanceof GlobalMethodReference) && + !(value instanceof CoercionFunction) && + !(value instanceof UriFunction) && + !(value instanceof ErrorConstructorReference) + ) + return undefined + return (callbackArgs: Array, chained: SandboxPromise) => + Effect.flatMap( + value instanceof ToolReference + ? this.createToolCallPromise(value.path, callbackArgs) + : value instanceof PromiseMethodReference + ? this.invokePromiseMethod(value, callbackArgs, node) + : value instanceof CodeModeFunction + ? this.invokeFunction(value, callbackArgs) + : value instanceof IntrinsicReference + ? this.invokeIntrinsic(value, callbackArgs, node) + : value instanceof GlobalMethodReference + ? Effect.sync(() => { + if (value.namespace === "console") return this.invokeConsole(value.name, callbackArgs, node) + if ( + value.namespace === "Object" && + callbackArgs[0] instanceof ToolReference && + !objectMethodsPreservingIdentity.has(value.name) + ) + return this.invokeObjectMethodOnTools(value.name, callbackArgs[0], node) + return invokeGlobalMethod(value, callbackArgs, node) + }) + : value instanceof CoercionFunction + ? Effect.succeed(invokeCoercion(value, callbackArgs, node)) + : value instanceof UriFunction + ? Effect.succeed(invokeUriFunction(value, callbackArgs, node)) + : Effect.succeed( + createErrorValue( + value.name, + callbackArgs[0] === undefined ? "" : coerceToString(callbackArgs[0]), + ), + ), + (result) => { + if (result === chained) { + return Effect.fail( + new InterpreterRuntimeError("Chaining cycle detected for promise.", node).as("TypeError"), + ) + } + return result instanceof SandboxPromise ? this.settlePromise(result) : Effect.succeed(result) + }, + ) + } + const onFulfilled = name === "then" ? callback(args[0]) : undefined + const onRejected = name === "then" ? callback(args[1]) : name === "catch" ? callback(args[0]) : undefined + const onFinally = name === "finally" ? callback(args[0]) : undefined + const settlement = Effect.exit(this.settlePromise(promise)) + + return this.createReaction( + settlement, + (exit, chained) => + Effect.gen(function* () { + if (onFinally !== undefined) yield* onFinally([], chained) + if (Exit.isSuccess(exit)) { + if (onFulfilled === undefined) return exit.value + return yield* onFulfilled([exit.value], chained) + } + if (onRejected !== undefined) + return yield* onRejected([caughtErrorValue(Cause.squash(exit.cause))], chained) + return yield* Effect.failCause(exit.cause) + }), + ) + } + private invokeStringReplacer( value: string, name: "replace" | "replaceAll", @@ -3219,17 +3339,9 @@ class Interpreter { return new ComputedValue(undefined) } - // Any property access on a promise is a confused program (`p.then(...)`, `p.value`); - // reading `undefined` here would hide the missing await, so both paths get an explicit, - // await-hinting error instead of the forgiving unknown-property fallthrough. if (objectValue instanceof SandboxPromise) { if (key === "then" || key === "catch" || key === "finally") { - throw new InterpreterRuntimeError( - `Promise.prototype.${String(key)} is not supported in CodeMode; use await instead (with try/catch to handle failures) - e.g. \`const result = await tools.ns.tool(...)\`.`, - propertyNode, - "UnsupportedSyntax", - [supportedSyntaxMessage], - ) + return new IntrinsicReference(objectValue, key) } throw new InterpreterRuntimeError( "This value is an un-awaited Promise and has no readable properties; await it first - e.g. `const result = await tools.ns.tool(...)`.", @@ -3518,18 +3630,33 @@ export const executeWithLimits = >( }) } - const operation = Effect.gen(function* () { - const program = parseProgram(options.code) - const interpreter = new Interpreter>(tools.invoke, tools.keys, logs) - const value = yield* interpreter.run(program) - const result = copyOut(copyIn(value, "Execution result"), true) as DataValue - return { - ok: true, - value: result, - ...logged(), - toolCalls: tools.calls, - } satisfies Result - }).pipe((program) => { + const operation = Effect.scoped( + Effect.gen(function* () { + const scope = yield* Effect.acquireRelease(Scope.make("parallel"), (scope, exit) => Scope.close(scope, exit)) + const reactionQueue = yield* Queue.unbounded>>() + yield* Effect.forever( + Effect.gen(function* () { + const reaction = yield* Queue.take(reactionQueue) + yield* Effect.yieldNow + yield* Effect.forkIn( + Effect.flatMap(Effect.exit(reaction.effect), (exit) => Deferred.done(reaction.settlement, exit)), + scope, + { startImmediately: true }, + ) + }), + ).pipe(Effect.forkIn(scope, { startImmediately: true })) + const program = parseProgram(options.code) + const interpreter = new Interpreter>(tools.invoke, tools.keys, scope, reactionQueue, logs) + const value = yield* interpreter.run(program) + const result = copyOut(copyIn(value, "Execution result"), true) as DataValue + return { + ok: true, + value: result, + ...logged(), + toolCalls: tools.calls, + } satisfies Result + }), + ).pipe((program) => { const timeoutMs = limits.timeoutMs if (timeoutMs === undefined) return program return program.pipe( diff --git a/packages/codemode/src/tool-runtime.ts b/packages/codemode/src/tool-runtime.ts index 7d67ba6c79f8..8a80493fec0b 100644 --- a/packages/codemode/src/tool-runtime.ts +++ b/packages/codemode/src/tool-runtime.ts @@ -616,7 +616,7 @@ export const prepare = (tools: HostTools, catalogBudget = defaultCatalogBu "## Language", "", "Use common JavaScript data operations, functions, control flow, selected standard-library methods, and awaited tool calls. Built-ins include Date, RegExp, Map, Set, URL, URLSearchParams, and URI encoding helpers.", - "Modules/imports, classes, generators, timers, fetch, eval, prototype access, unlisted methods, and promise chaining are unavailable. Use Code Mode tools for external operations. Use await with try/catch.", + "Modules/imports, classes, generators, timers, fetch, eval, prototype access, and unlisted methods are unavailable. Use Code Mode tools for external operations. Use await with try/catch.", "Dates and URLs serialize to strings at data boundaries; Map/Set/RegExp/URLSearchParams serialize to `{}`.", ] diff --git a/packages/codemode/src/values.ts b/packages/codemode/src/values.ts index 4ca305d815eb..b1a3e947c54a 100644 --- a/packages/codemode/src/values.ts +++ b/packages/codemode/src/values.ts @@ -1,7 +1,7 @@ import type { Effect, Fiber } from "effect" export class SandboxPromise { - interrupted = false + handled = false constructor( readonly fiber: Fiber.Fiber | undefined, readonly immediate?: Effect.Effect, diff --git a/packages/codemode/test/codemode.test.ts b/packages/codemode/test/codemode.test.ts index 221b5e07dfe1..d69fcb11dee3 100644 --- a/packages/codemode/test/codemode.test.ts +++ b/packages/codemode/test/codemode.test.ts @@ -652,9 +652,12 @@ describe("CodeMode public contract", () => { expect(instructions).toContain("not a general-purpose runtime") expect(instructions).not.toContain("Standard modern JavaScript works") expect(instructions).not.toContain("TypeScript type annotations") - for (const missing of ["Modules/imports", "classes", "generators", "fetch", "promise chaining"]) { + for (const missing of ["Modules/imports", "classes", "generators", "fetch"]) { expect(instructions).toContain(missing) } + expect(instructions).toContain("selected standard-library methods, and awaited tool calls") + expect(instructions).toContain("Use await with try/catch") + expect(instructions).not.toContain("promise chaining are unavailable") expect(instructions).toContain("URL, URLSearchParams, and URI encoding helpers") expect(instructions).not.toContain("host globals") expect(instructions).toContain("Use Code Mode tools for external operations") diff --git a/packages/codemode/test/promise.test.ts b/packages/codemode/test/promise.test.ts index 313b9c22cc0c..6e5a5e07e8b1 100644 --- a/packages/codemode/test/promise.test.ts +++ b/packages/codemode/test/promise.test.ts @@ -212,6 +212,22 @@ describe("first-class promise values", () => { expect(diagnostic.suggestions?.join(" ")).toContain("Await promises") }) + test("an unhandled rejection does not stop later work from draining", async () => { + const trace = makeTrace() + const diagnostic = await error( + ` + tools.host.fail({}) + tools.host.sleepy({ id: 1, ms: 30 }) + return "done" + `, + { trace }, + ) + expect(diagnostic.kind).toBe("ToolFailure") + expect(diagnostic.message).toContain("Unhandled rejection from an un-awaited promise") + expect(trace.completed).toBe(1) + expect(trace.interrupted).toBe(0) + }) + test("a never-awaited failing async function surfaces as an unhandled promise rejection", async () => { const diagnostic = await error(` const fail = async () => { throw new Error("boom") } @@ -235,6 +251,58 @@ describe("first-class promise values", () => { expect(diagnostic.kind).toBe("ToolFailure") expect(diagnostic.message).toContain("Lookup refused") }) + + test("keeps unreturned calls alive after their async function settles", async () => { + const trace = makeTrace() + expect( + await value( + ` + const start = async () => { + tools.host.sleepy({ id: 1, ms: 30 }) + return "started" + } + await start() + return "done" + `, + { trace }, + ), + ).toBe("done") + expect(trace.completed).toBe(1) + expect(trace.interrupted).toBe(0) + }) + + test("keeps unreturned calls alive after their promise handler settles", async () => { + const trace = makeTrace() + expect( + await value( + ` + await Promise.resolve().then(() => { + tools.host.sleepy({ id: 1, ms: 30 }) + }) + return "done" + `, + { trace }, + ), + ).toBe("done") + expect(trace.completed).toBe(1) + expect(trace.interrupted).toBe(0) + }) + + test("drains pending work after program failure and preserves the original failure", async () => { + const trace = makeTrace() + const diagnostic = await error( + ` + tools.host.fail({}) + tools.host.sleepy({ id: 1, ms: 30 }) + throw new Error("original") + `, + { trace }, + ) + expect(diagnostic.kind).toBe("ExecutionFailure") + expect(diagnostic.message).toBe("Uncaught: original") + expect(trace.completed).toBe(1) + expect(trace.interrupted).toBe(0) + }) }) describe("promises at data boundaries", () => { @@ -412,45 +480,36 @@ describe("Promise.allSettled", () => { }) describe("Promise.race", () => { - test("first settlement wins and losers are interrupted", async () => { + test("first settlement wins and losers continue", async () => { const trace = makeTrace() const result = await value( ` const fast = tools.host.sleepy({ id: 1, ms: 10 }) - const slow = tools.host.sleepy({ id: 2, ms: 5000 }) + const slow = tools.host.sleepy({ id: 2, ms: 30 }) return await Promise.race([fast, slow]) `, { trace }, ) expect(result).toBe(1) - expect(trace.interrupted).toBe(1) - expect(trace.completed).toBe(1) + expect(trace.interrupted).toBe(0) + expect(trace.completed).toBe(2) }) - test("awaiting an interrupted loser afterwards is a catchable program failure", async () => { + test("a losing chain continues and remains awaitable", async () => { expect( await value(` - const fast = tools.host.sleepy({ id: 1, ms: 10 }) - const slow = tools.host.sleepy({ id: 2, ms: 5000 }) - const winner = await Promise.race([fast, slow]) - try { - await slow - return "no" - } catch (e) { - return { winner, caught: e.message } - } + const slow = tools.host.sleepy({ id: 2, ms: 30 }).then((id) => id * 2) + const winner = await Promise.race([slow, "fast"]) + return [winner, await slow] `), - ).toEqual({ - winner: 1, - caught: "This tool call was interrupted because another value settled a Promise.race first.", - }) + ).toEqual(["fast", 4]) }) test("a rejection can win the race", async () => { expect( await value(` try { - await Promise.race([tools.host.fail({}), tools.host.sleepy({ id: 1, ms: 5000 })]) + await Promise.race([tools.host.fail({}), tools.host.sleepy({ id: 1, ms: 30 })]) return "no" } catch (e) { return e.message @@ -459,12 +518,37 @@ describe("Promise.race", () => { ).toBe("Lookup refused") }) + test("a rejecting winner still drains its slow loser", async () => { + const trace = makeTrace() + const diagnostic = await error( + `return await Promise.race([tools.host.fail({}), tools.host.sleepy({ id: 1, ms: 30 })])`, + { trace }, + ) + expect(diagnostic.kind).toBe("ToolFailure") + expect(diagnostic.message).toBe("Lookup refused") + expect(trace.completed).toBe(1) + expect(trace.interrupted).toBe(0) + }) + test("a plain value wins over pending promises", async () => { const trace = makeTrace() expect( - await value(`return await Promise.race([tools.host.sleepy({ id: 1, ms: 5000 }), "immediate"])`, { trace }), + await value(`return await Promise.race([tools.host.sleepy({ id: 1, ms: 30 }), "immediate"])`, { trace }), ).toBe("immediate") - expect(trace.interrupted).toBe(1) + expect(trace.interrupted).toBe(0) + expect(trace.completed).toBe(1) + }) + + test("a losing rejection remains handled", async () => { + expect( + await value(` + const rejectLater = async () => { + await tools.host.sleepy({ id: 1, ms: 20 }) + throw new Error("late") + } + return await Promise.race([rejectLater(), "winner"]) + `), + ).toBe("winner") }) test("an empty race is a clear error instead of hanging", async () => { @@ -492,6 +576,36 @@ describe("Promise.resolve / Promise.reject", () => { `), ).toBe("nope") }) + + test("an abandoned rejected promise surfaces as an unhandled rejection", async () => { + const diagnostic = await error(` + Promise.reject(new Error("boom")) + return "done" + `) + expect(diagnostic.kind).toBe("ExecutionFailure") + expect(diagnostic.message).toContain("Unhandled rejection from an un-awaited promise") + expect(diagnostic.message).toContain("boom") + }) +}) + +describe("Promise combinator values", () => { + test("all, allSettled, and race return chainable promises", async () => { + expect( + await value(` + const all = Promise.all([Promise.resolve(1), 2]) + const settled = Promise.allSettled([Promise.resolve(3)]) + const race = Promise.race([Promise.resolve(4)]) + return [ + all instanceof Promise, + settled instanceof Promise, + race instanceof Promise, + await all.then((values) => values.join(",")), + await settled.then((values) => values[0].value), + await race.then((value) => value + 1), + ] + `), + ).toEqual([true, true, true, "1,2", 3, 5]) + }) }) describe("timeout interruption of forked calls", () => { @@ -525,18 +639,185 @@ describe("timeout interruption of forked calls", () => { expect(result.error.kind).toBe("TimeoutExceeded") expect(trace.interrupted).toBe(2) }) + + test("interrupts promise fibers concurrently during scope teardown", async () => { + const cleanup = { active: 0, overlapped: false } + const tool = Tool.make({ + description: "Wait until interrupted", + input: Schema.Struct({}), + output: Schema.Never, + run: () => + Effect.never.pipe( + Effect.onInterrupt(() => + Effect.gen(function* () { + cleanup.active += 1 + yield* Effect.sleep(20) + cleanup.overlapped ||= cleanup.active > 1 + cleanup.active -= 1 + }), + ), + ), + }) + const result = await Effect.runPromise( + CodeMode.execute({ + tools: { host: { first: tool, second: tool } }, + code: ` + tools.host.first({}) + tools.host.second({}) + return await Promise.resolve("waiting") + `, + limits: { timeoutMs: 50 }, + }), + ) + expect(result.ok).toBe(false) + if (result.ok) return + expect(result.error.kind).toBe("TimeoutExceeded") + expect(cleanup.overlapped).toBe(true) + }) }) -describe("unsupported promise surface", () => { - test(".then/.catch/.finally give a clear await-instead error", async () => { - for (const method of ["then", "catch", "finally"]) { - const diagnostic = await error(`return tools.host.sleepy({ id: 1 }).${method}((x) => x)`) - expect(diagnostic.kind).toBe("UnsupportedSyntax") - expect(diagnostic.message).toContain(`Promise.prototype.${method} is not supported`) - expect(diagnostic.message).toContain("await") - } +describe("promise chaining", () => { + test("then transforms values and flattens returned promises", async () => { + expect( + await value(` + return tools.host.sleepy({ id: 2 }) + .then((id) => tools.host.sleepy({ id: id + 1 })) + .then((id) => id * 2) + `), + ).toBe(6) + }) + + test("handlers run after synchronous statements", async () => { + expect( + await value(` + const order = [] + const chained = Promise.resolve().then(() => order.push("then")) + order.push("sync") + await chained + return order + `), + ).toEqual(["sync", "then"]) + }) + + test("nested reactions run before downstream reactions queued later", async () => { + expect( + await value(` + const order = [] + await Promise.resolve() + .then(() => { + order.push(1) + Promise.resolve().then(() => order.push(2)) + }) + .then(() => order.push(3)) + return order + `), + ).toEqual([1, 2, 3]) + }) + + test("plain and settled await resume after reactions that are already queued", async () => { + expect( + await value(` + const order = [] + Promise.resolve().then(() => order.push(1)) + await 0 + order.push(2) + Promise.resolve().then(() => order.push(3)) + await Promise.resolve() + order.push(4) + return order + `), + ).toEqual([1, 2, 3, 4]) }) + test("reactions registered on the same pending promise preserve order", async () => { + expect( + await value(` + const order = [] + const pending = tools.host.sleepy({ id: 1 }) + pending.then(() => order.push(1)) + pending.then(() => order.push(2)) + await pending + return order + `), + ).toEqual([1, 2]) + }) + + test("an async reaction does not block the next queued reaction", async () => { + expect( + await value(` + const order = [] + const first = Promise.resolve().then(async () => { + order.push(1) + await tools.host.sleepy({ id: 1 }) + order.push(3) + }) + Promise.resolve().then(() => order.push(2)) + await first + return order + `), + ).toEqual([1, 2, 3]) + }) + + test("catch receives normalized errors and recovers the chain", async () => { + expect(await value(`return tools.host.fail({}).catch((error) => error.message)`)).toBe("Lookup refused") + expect( + await value(` + return Promise.resolve(1) + .then(() => { throw new Error("boom") }) + .catch((error) => error.message) + `), + ).toBe("boom") + }) + + test("then rejection handlers and omitted handlers pass through settlement", async () => { + expect(await value(`return Promise.resolve(4).then(undefined).catch(undefined)`)).toBe(4) + expect(await value(`return Promise.reject("nope").then(undefined, (reason) => reason + "!")`)).toBe("nope!") + }) + + test("supported builtin callables can be handlers", async () => { + expect(await value(`return Promise.resolve({ a: 1 }).then(JSON.stringify)`)).toBe('{"a":1}') + expect(await value(`return Promise.resolve(4).then(Promise.resolve)`)).toBe(4) + }) + + test("finally awaits its callback and preserves the original settlement", async () => { + expect( + await value(` + let cleanup = 0 + const result = await Promise.resolve(7).finally(async () => { + await tools.host.sleepy({ id: 1 }) + cleanup = 1 + return 99 + }) + return [result, cleanup] + `), + ).toEqual([7, 1]) + expect( + await value(`return Promise.reject(new Error("original")).finally(() => 99).catch((error) => error.message)`), + ).toBe("original") + }) + + test("a rejected finally callback replaces the original settlement", async () => { + expect( + await value(` + return Promise.resolve(1) + .finally(() => Promise.reject(new Error("cleanup"))) + .catch((error) => error.message) + `), + ).toBe("cleanup") + }) + + test("a self-resolving chain rejects instead of deadlocking", async () => { + expect( + await value(` + let chained + chained = Promise.resolve().then(() => chained) + return chained.catch((error) => [error.name, error.message]) + `), + ).toEqual(["TypeError", "Chaining cycle detected for promise."]) + }) +}) + +describe("unsupported promise surface", () => { test("other property reads on a promise hint at the missing await", async () => { const diagnostic = await error(`return tools.host.sleepy({ id: 1 }).value`) expect(diagnostic.kind).toBe("InvalidDataValue") diff --git a/packages/core/test/mcp.test.ts b/packages/core/test/mcp.test.ts index 26977a87293b..50866cdd5d42 100644 --- a/packages/core/test/mcp.test.ts +++ b/packages/core/test/mcp.test.ts @@ -248,6 +248,7 @@ it.effect("advertises MCP output schemas to Code Mode", () => const execute = (yield* toolDefinitions(registry)).find((tool) => tool.name === "execute") expect(execute?.description).toContain("tools.demo.search(input: {}): Promise<{\n ok: boolean,\n}>") + expect(execute?.description).not.toContain("promise chaining are unavailable") }), )