test(tool): pin every tool's parameters schema before migration#23244
Open
kitlangton wants to merge 1 commit intodevfrom
Open
test(tool): pin every tool's parameters schema before migration#23244kitlangton wants to merge 1 commit intodevfrom
kitlangton wants to merge 1 commit intodevfrom
Conversation
Pre-migration safety net for the upcoming tool-by-tool zod\u2192Schema conversion. Every tool's parameters schema now has: 1. A JSON Schema snapshot (`z.toJSONSchema` with `io: "input"`) \u2014 this captures exactly what the LLM sees at tool registration time, so any drift caused by a future migration fails the snapshot. 2. Parse-accept/parse-reject assertions per tool pinning the user-visible behavioural contract (required fields, refinement bounds, enum membership, default values). To make the snapshots possible without standing up each tool's full Effect runtime, every tool file now exports its parameters schema as `Parameters` at module scope: - 9 tools already had a module-level const \u2014 just added `export`, and standardised the name to `Parameters` (uppercase) where it was previously `parameters`. - 9 tools had their schema inline inside `Tool.define` \u2014 hoisted to module scope under the same `Parameters` name and wired back through. Zero behaviour change: Tool.define still sees the same schema, runtime validation path is identical, SDK (types.gen.ts + openapi.json) is byte-identical, and the full 2054-test suite passes. 18 JSON Schema snapshots and 43 explicit parse/reject assertions for the 18 built-in tools (apply_patch, bash, codesearch, edit, glob, grep, invalid, lsp, multiedit, plan, question, read, skill, task, todo, webfetch, websearch, write).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Pre-migration safety net for the upcoming tool-by-tool
zod → Schemaconversion. Every tool's parameters schema now has:z.toJSONSchemawithio: "input") — captures exactly what the LLM sees at tool registration time, so any drift caused by a future migration fails the snapshot.Refactor
To make the snapshots possible without standing up each tool's full Effect runtime, every tool file now exports its parameters schema as
Parametersat module scope:export, and standardised the name toParameters(uppercase) where it was previouslyparameters.Tool.define(...)— hoisted to module scope under the sameParametersname and wired back through.The tool framework's
Tool.define(...)call site is unchanged — it just referencesParametersby name instead of an inlinez.object({...}).Coverage
18 JSON Schema snapshots and 43 explicit parse/reject assertions for:
apply_patch,bash,codesearch,edit,glob,grep,invalid,lsp,multiedit,plan,question,read,skill,task,todo,webfetch,websearch,writeVerification
bun typecheckcleanbun run test test/— 2054 pass / 0 fail (full suite)types.gen.tsandopenapi.json)Next
With this in place, each subsequent PR can migrate one tool's
Parametersfromz.object({...})toSchema.Struct({...})(or similar), let theeffect-zodwalker emit the zod the tool framework still consumes, and rely on these tests to catch any JSON Schema or parse behaviour drift.