Skip to content

Catalog output drift: signatures declare a subset of the runtime return, and the drift test cuts return types away #39

Description

@lidge-jun

Summary

#33 closed input drift between the catalog and the runtime, and added test/catalog-drift.test.js to keep it closed. The output side of the same class is open and untested: several signatures declare a strict subset of what the runtime returns, and the missing keys are disproportionately the completeness signals a caller most needs.

The drift test cannot catch this, by construction — it cuts the return type off before comparing:

// test/catalog-drift.test.js:32-36
// The return type is cut away first. Promise<{ok,items}> carries braces and identifiers of
// its own, and reading those as arguments reported `ok` as an undeclared option.
function argumentPart(signature) {

That was the right fix for #33. It also means nothing checks the half that remains.

Measured drift

Live call, not a fixture:

const r = await browse.searchMany(["aside codemode"], { engine: "duckduckgo" });
Object.keys(r);
// ["engine","items","ok","partial","deduped","filtered","dateFilter","suspectEmpty"]

Declared: Promise<{engine,items,ok}>. Undeclared: partial, deduped, filtered, dateFilter, suspectEmpty — five keys, including both completeness signals.

Other instances found by reading the runtime against the catalog:

Action Signature declares Runtime also returns
browse.exec {ok,items,timings,partial,leakedUrls} schema, status, runId, requested, completed, unreturned, ledger, effects, complete, truncated, actionLog, contentVerified, suspectEmpty, tabs, … (src/host/browse/session.js:498-557)
browse.attach {ok,tab,href,hash,title,scrollY,render,contentVerified,runId,effects} text (the body), snapshot, pageUrl, fragmentDropped, actions, data, code, note (src/host/browse/attach.js:390-425)
api.batch {ok,items} partial (src/host/browse/adapters.js:86)
browse.prefetch {items,warmed} ok, note (src/host/browse/watch.js:108)
recipes.run {recipe,url,ok,items} partial (src/host/browse/watch.js:86)

browse.attach is the sharpest: an agent that trusts the signature does not learn that the page body is on text. That is the same shape as #33's own complaint that the first thing a reader saw omitted actions/approveWrites.

Why it matters more than cosmetics

The undeclared keys are not incidental extras. partial, complete, truncated, dateFilter, suspectEmpty, contentVerified are precisely the fields this project added to stop silent-success bugs (#24, #29, #30, #31, #32). A signal that exists at runtime but is absent from the discovery surface is, for a model reading describe first, functionally not there — which is this repo's own stated rule about the registry.

Requested change

  1. Extend test/catalog-drift.test.js with an output direction: for actions with a callable fixture, assert every key in the runtime return appears in the signature's Promise<{…}> (or is explicitly allow-listed).
  2. Update the five signatures above. Priority order: browse.attach (text), browse.searchMany (dateFilter, suspectEmpty), browse.exec (status, complete, contentVerified).
  3. For envelopes too large to inline, keep the signature honest with the load-bearing keys and move the rest into notes or outputsbrowse.exec and browse.captureMany are the only two actions using outputs today (src/host/browse/actions-schema.js:138, :217), so the mechanism exists and is simply underused.

Environment

aside-codemode v0.8.1 (dev), macOS, MCP execute_code. searchMany drift reproduced live; the rest read from source.
Related: #33 (input drift, closed).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions