Add robust external response.schema_ref support (JSON + Zod modules) with metadata, validation, and tests

README.md

@@ -261,7 +261,7 @@ In browser or client-side code, keep provider credentials on the server. Use the
 
 ### Provider-specific fields and raw passthrough
 
-Use normalized fields first (`sampling`, `response`, `cache`, `tools`) so prompts stay portable. `response.schema` is the neutral JSON Schema path; adapters emit it as OpenAI/OpenRouter/LLMAsAService `response_format`, OpenAI Responses `text.format`, Anthropic `output_config.format`, and Gemini `generationConfig.responseJsonSchema`.
+Use normalized fields first (`sampling`, `response`, `cache`, `tools`) so prompts stay portable. `response.schema` is the neutral JSON Schema path; adapters emit it as OpenAI/OpenRouter/LLMAsAService `response_format`, OpenAI Responses `text.format`, Anthropic `output_config.format`, and Gemini `generationConfig.responseJsonSchema`. You can also provide `response.schema_ref` to load schema from a prompt-relative `.json` file or `.js/.mjs/.cjs` zod module (mutually exclusive with `response.schema`).
 
 Use `provider_options` when PromptOpsKit has a known provider-specific mapping, such as Anthropic `top_k`, Gemini's native `response_schema`, OpenRouter routing fields, or LLMAsAService gateway routing/customer metadata.
 
@@ -656,7 +656,7 @@ Prompt files use YAML front matter with these fields:
 | `fallback_models` | `string[]` | Fallback model list |
 | `reasoning` | `object` | `{ effort, budget_tokens }` |
 | `sampling` | `object` | `{ temperature, top_p, frequency_penalty, presence_penalty, stop, max_output_tokens }` |
-| `response` | `object` | `{ format, stream, schema, schema_name, schema_description, schema_strict }` |
+| `response` | `object` | `{ format, stream, schema, schema_ref, schema_name, schema_description, schema_strict }` |
 | `cache` | `object` | Provider-specific cache controls (`openai`, `anthropic`, `gemini`/`google`) |
 | `tools` | `array` | Tool references (string names or inline definitions) |
 | `provider_options` | `object` | Provider-specific non-portable options (`anthropic`, `gemini`, `openrouter`, `llmasaservice`) |

docs/index.md

@@ -17,6 +17,7 @@ Open-source developer toolkit for managing prompts, system instructions, tools,
 - [CLI](./cli.md) — Command-line interface: init, validate, compile, render, inspect, skill
 - [API Reference](./api-reference.md) — TypeScript API: `createPromptOpsKit`, `renderPrompt`, standalone functions
 - [Schema](./schema.md) — Full YAML front matter schema reference
+- [Response Schema DX](./schema-dx.md) — Design proposal for sharing one response schema between prompt files and runtime validation code
 - [Vendor Schema Gap Analysis](./vendor-schema-gap-analysis.md) — Snapshot comparison against published OpenAI, Anthropic, Gemini, OpenRouter, and LLMAsAService gateway schema capabilities
 - [Testing](./testing.md) — Test helpers, mock assets, and sidecar test files
 - [Validation](./validation.md) — Schema validation, "did you mean?" suggestions, variable checks, early regex validation, and YAML regex quoting guidance

docs/overrides.md

@@ -51,7 +51,7 @@ Only these fields can be overridden in `environments` and `tiers`:
 | `fallback_models` | `string[]` | Fallback model list |
 | `reasoning` | `object` | `{ effort, budget_tokens }` |
 | `sampling` | `object` | `{ temperature, top_p, frequency_penalty, presence_penalty, stop, max_output_tokens }` |
-| `response` | `object` | `{ format, stream, schema, schema_name, schema_description, schema_strict }` |
+| `response` | `object` | `{ format, stream, schema, schema_ref, schema_name, schema_description, schema_strict }` |
 | `cache` | `object` | Provider-specific cache controls (`openai`, `anthropic`, `gemini`/`google`) |
 | `raw` | `object` | Provider-specific request-body passthrough blocks |
 | `tools` | `array` | Tool references |

docs/schema-dx.md

@@ -0,0 +1,170 @@
+# Response Schema DX: Author Once, Use in Prompt + Code
+
+This proposal describes how PromptOpsKit can let teams define a response schema once and use it in:
+
+1. Prompt front matter (`response.schema`), and
+2. Runtime TypeScript validation/parsing.
+
+## Goals
+
+- **Single source of truth** for structured output contracts.
+- **Low-friction authoring** (no mandatory codegen for simple teams).
+- **Strong typing** in application code for teams that want it.
+- **Portable behavior** across providers that support JSON Schema.
+- **Incremental adoption** without breaking existing prompts.
+
+## Recommended design
+
+Use a **three-lane model** so users can choose complexity level.
+
+### Lane A: Inline schema (existing behavior)
+
+Keep current `response.schema` behavior unchanged.
+
+Best for:
+- quick experiments
+- one-off prompts
+- teams without build tooling
+
+### Lane B: External schema file references (new)
+
+Allow prompt front matter to reference a schema asset file.
+
+Example:
+
+```yaml
+response:
+  format: json
+  schema_ref: ./schemas/support-reply.schema.json
+  schema_name: support_reply
+  schema_strict: true
+```
+
+Resolution rules:
+- `schema_ref` is resolved relative to the prompt file.
+- File must be JSON Schema object.
+- Compiler inlines resolved schema into compiled prompt output for providers.
+
+Why this helps DX:
+- keeps prompts readable
+- enables sharing schema across multiple prompts
+- lets app code import the same schema file directly
+
+### Lane C: Optional codegen for typed validators (new, opt-in)
+
+Add an optional command:
+
+```bash
+promptopskit schema generate
+```
+
+This command can emit:
+- `*.schema.json` copies (or normalized forms)
+- TS validator modules (e.g. Zod/Valibot wrappers)
+- inferred TS types for app code
+
+Why optional:
+- some teams only need raw JSON Schema + Ajv
+- others want end-to-end typed contracts
+
+## Authoring/usage patterns
+
+### Pattern 1: JSON Schema + Ajv (minimal dependencies)
+
+- Define schema in `*.schema.json`.
+- Reference it from prompt via `schema_ref`.
+- In app runtime, import same JSON schema into Ajv to validate model output.
+
+Pros: simplest, ecosystem-standard.
+
+### Pattern 2: Type-first (Zod) with schema export
+
+- Define schema in code (`z.object(...)`).
+- Export JSON Schema artifact during build (`zod-to-json-schema`).
+- Prompt references generated JSON file.
+
+Pros: strong TS ergonomics.
+Cons: build step required.
+
+### Pattern 3: Schema-first with generated TS types
+
+- Define `*.schema.json`.
+- Generate TS types and/or validators.
+
+Pros: prompt and runtime both consume same source.
+
+## Proposed front matter additions
+
+Under `response`:
+
+- `schema_ref?: string` — path to external JSON Schema file or zod module.
+- Keep existing `schema?: object`.
+- Validation rule: exactly one of `schema` or `schema_ref` should be provided.
+
+Potential future extension:
+
+- `schema_ref_name?: string` for referencing named schemas from a registry file.
+
+## Compiler/runtime behavior
+
+1. Parser reads prompt.
+2. If `schema_ref` exists, loader resolves and loads schema source (`.json` or zod module).
+3. Validate schema is object-shaped JSON.
+4. Normalize to internal `response.schema`.
+5. Continue provider mapping unchanged.
+
+This preserves downstream adapter logic and minimizes invasive changes.
+
+## CLI UX
+
+### `compile`
+
+- Include resolved schema in compiled artifact.
+- Include source metadata:
+  - `response.schema_source.resolved_path`
+  - `response.schema_source.hash` (optional)
+
+### `validate`
+
+- Verify referenced schema file exists.
+- Validate JSON parse + object shape.
+- Warn on unsupported JSON Schema constructs per provider (best-effort).
+
+### `inspect`
+
+- Show whether schema is inline or file-based.
+- Display resolved path and size/hash.
+
+## Error messages (important for DX)
+
+Make messages actionable:
+
+- `POK050: response.schema_ref "./schemas/reply.json" not found (resolved from prompts/support/reply.md)`
+- `POK050: response.schema_ref "./schemas/reply.json" must resolve to a JSON object schema`
+- `POK051: zod schema modules must export a Zod schema as default export or named export "schema"`
+- `POK051: response.schema_ref "./schemas/reply.schema.ts" has unsupported extension ".ts". Use .json, .js, .mjs, or .cjs`
+- `response.schema and response.schema_ref are mutually exclusive`
+
+## Migration strategy
+
+- No breaking changes for existing prompts.
+- Teams can migrate prompt-by-prompt:
+  1. move inline schema to file
+  2. replace `schema` with `schema_ref`
+  3. run `promptopskit validate`
+
+## Recommendation
+
+`schema_ref` (Lane B) is now implemented.
+
+Next step: evaluate **Lane C** (optional codegen) once real user needs clarify:
+- target validator libs
+- desired output layout
+- monorepo integration expectations
+
+This sequencing maximizes immediate DX value with low maintenance cost.
+
+
+## Security note for executable schema modules
+
+`schema_ref` supports executable JS modules (`.js/.mjs/.cjs`) for zod exports. These modules are imported during load/validate/compile, so only reference trusted repository code.

docs/schema.md

@@ -84,6 +84,7 @@ sampling:
 ## `response`
 
 ```yaml
+# Inline schema
 response:
   format: json    # text | json | markdown
   stream: true
@@ -95,13 +96,24 @@ response:
   schema_name: support_reply
   schema_description: Support reply payload
   schema_strict: true
+
+# External schema reference (mutually exclusive with `schema`)
+response:
+  format: json
+  schema_ref: ./schemas/support-reply.schema.json
+
+# External zod module reference (must export default or named `schema`)
+response:
+  format: json
+  schema_ref: ./schemas/support-reply.schema.mjs
 ```
 
 | Field | Type | Description |
 |-------|------|-------------|
 | `format` | `'text' \| 'json' \| 'markdown'` | Response format |
 | `stream` | `boolean` | Enable streaming |
 | `schema` | `object` | Portable JSON Schema object for structured output |
+| `schema_ref` | `string` | Relative path to external schema (`.json`) or zod module (`.js/.mjs/.cjs`) |
 | `schema_name` | `string` | Optional schema name (used by OpenAI/OpenAI Responses) |
 | `schema_description` | `string` | Optional schema description (used by OpenAI/OpenAI Responses/OpenRouter/LLMAsAService structured outputs) |
 | `schema_strict` | `boolean` | Strict schema enforcement toggle (OpenAI/OpenAI Responses) |

package.json

@@ -139,7 +139,8 @@
   },
   "dependencies": {
     "gray-matter": "^4.0.3",
-    "zod": "^3.23.0"
+    "zod": "^3.23.0",
+    "zod-to-json-schema": "^3.25.2"
   },
   "devDependencies": {
     "@types/node": "^25.6.0",

src/parser/loader.ts

@@ -6,6 +6,7 @@ import type { ParseResult } from './parser.js';
 import { extractSections } from './sections.js';
 import { PromptDefaultsSchema } from '../schema/index.js';
 import type { PromptDefaults } from '../schema/index.js';
+import { resolveResponseSchemaRef } from './response-schema-ref.js';
 
 const DEFAULTS_FILE_NAME = 'defaults.md';
 
@@ -27,7 +28,17 @@ export async function loadPromptFile(filePath: string, options: LoadPromptOption
   // walks above the prompt tree when no explicit root is provided.
   const root = options.defaultsRoot ?? dirname(filePath);
   const defaults = await loadDefaultsForPath(filePath, root);
-  const asset = applyDefaults(parsed.asset, defaults);
+  const withDefaults = applyDefaults(parsed.asset, defaults);
+  const resolved = await resolveResponseSchemaRef(withDefaults, filePath);
+  const asset = (resolved.response?.schema !== undefined && !resolved.response?.schema_source)
+    ? {
+      ...resolved,
+      response: {
+        ...resolved.response,
+        schema_source: { mode: 'inline' as const },
+      },
+    }
+    : resolved;
 
   return {
     ...parsed,