Skip to content

Parallel APIs second pass: add specs array to counterfact.yaml#1845

Closed
Copilot wants to merge 11 commits intomainfrom
copilot/add-specs-array-to-counterfact-yaml
Closed

Parallel APIs second pass: add specs array to counterfact.yaml#1845
Copilot wants to merge 11 commits intomainfrom
copilot/add-specs-array-to-counterfact-yaml

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 13, 2026

Summary

Adds a specs array to counterfact.yaml that mounts multiple OpenAPI documents at distinct URL base paths from a single server instance. When specs is present it takes precedence over spec.

specs:
  - source: ./billing.yaml
    base: billing

  - source: https://example.com/identity.yaml
    base: identity

Each spec is fully isolated: its own Registry, Dispatcher, CodeGenerator, and ModuleLoader. One Koa middleware is created per spec (each pre-configured with the spec's route prefix); they are passed as an array to createKoaApp(), which registers them with app.use() in succession — no existing class needed to be made multi-API-aware, and no composite fan-out middleware is required.

Single-spec and multi-spec configurations share a single code path: a plain openApiPath config is normalised at startup to specs = [{ source, base: "" }], so there is no divergence between the two modes. koaMiddleware in the returned object is always an array (length 1 for single-spec, N for N specs).

adminApiMiddleware is constructed in app.ts and passed into createKoaApp() as an optional parameter, so createKoaApp() no longer needs to import or build it internally and no longer requires registry or contextRegistry in its signature.

A single shared Transpiler instance covers all specs. Transpiler(rootPath, moduleKind) watches rootPath recursively and applies the convention that any .ts file under a routes/ subdirectory is compiled to the sibling .cache/ directory (e.g. {base}/routes/foo.ts{base}/.cache/foo.cjs). In app.ts, one new Transpiler(config.basePath, "commonjs") covers all specs automatically — no per-spec instances, no array of path mappings.

Generated files land under {destination}/{base}/routes/ and {destination}/{base}/types/. For single-spec mode (normalised with base: ""), files continue to land directly under {destination}/routes/ and {destination}/types/ as before.

Original Prompt

Add a specs array to counterfact.yaml that mounts multiple OpenAPI documents at distinct URL base paths from a single Counterfact server instance.

Manual acceptance tests

  • counterfact.yaml with a specs array starts the server; requests to /billing/... and /identity/... are served by their respective specs
  • specs takes precedence over a spec key in the same config file — the single spec value is ignored
  • Request to /billing/invoices is validated against billing.yaml; request to /identity/users is validated against identity.yaml
  • A request path with no matching base prefix (e.g. /gamma/...) falls through to a 404 — not silently matched by another spec
  • Single-spec mode (spec: or positional openapi.yaml argument) is unaffected
  • Running with _ (no spec) still works unchanged

Tasks

  • Added SpecEntry interface (source, base) and specs?: SpecEntry[] to Config
  • Added createSpecBundle() — creates the full per-spec service set (Registry, Dispatcher, CodeGenerator, ModuleLoader); handles source === "_" (no document loaded) and optionally wires scenarios for the primary spec; no transpiler is created here
  • createKoaApp() now accepts Koa.Middleware | Koa.Middleware[] and registers each with app.use() in order (removed buildMultiSpecMiddleware()); registry and contextRegistry parameters removed — createKoaApp() now accepts an optional pre-built adminMiddleware?: Koa.Middleware instead
  • adminApiMiddleware is constructed in app.ts (conditionally when config.startAdminApi) and passed to createKoaApp() — keeping middleware construction co-located with configuration logic
  • Transpiler refactored to Transpiler(rootPath, moduleKind): watches rootPath recursively; any .ts file under a routes/ subdirectory is compiled to the sibling .cache/ directory; one shared instance per server covers all specs
  • counterfact() normalises a plain single-spec config to specs = [{ source: config.openApiPath, base: "" }] at startup, then runs a single unified loop — no separate single-spec branch; koaMiddleware in the returned object is always an array; one shared Transpiler is created with config.basePath as the root
  • Specs with base === "" preserve config.routePrefix; named specs use "/" + base as their route prefix
  • bin/counterfact.js: reads specs from config file; sets openApiPath = "_" when specs is active; passes specs into the config object
  • Tests: specs YAML parsing in load-config-file.test.ts; multi-spec shape and per-spec middleware prefix routing in app.test.ts; single-spec test updated to assert koaMiddleware is always an array with length 1
  • Transpiler tests rewritten to use the rootPath + routes/.cache/ convention; two new tests added: one verifying files outside routes/ are ignored, one verifying multi-spec compilation under the same root
  • Unit tests (test/app.multispec.test.ts): code generation verifies route files appear under {base}/routes/ for each spec without cross-contamination, type files generated under {base}/types/, and startRepl() is wired to the primary spec's registry
  • Black-box tests (test-black-box/test_parallel_apis.py) with dedicated OpenAPI fixtures (alpha.yaml, beta.yaml): distinct responses per prefix, file isolation between specs, 404 for unmatched prefix, and specs precedence over spec
  • Docs: added counterfact.yaml and parallel-APIs section to docs/reference.md

Copilot AI linked an issue Apr 13, 2026 that may be closed by this pull request
Copilot AI changed the title [WIP] Add specs array to counterfact.yaml for mounting multiple OpenAPI documents Parallel APIs second pass: add specs array to counterfact.yaml Apr 13, 2026
Copilot AI requested a review from pmcelhaney April 13, 2026 21:04
Copy link
Copy Markdown
Collaborator

@pmcelhaney pmcelhaney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add more tests. Make sure the code is generated correctly. Make sure the REPL works.

Also add black box tests.

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 13, 2026

Add more tests. Make sure the code is generated correctly. Make sure the REPL works.

Also add black box tests.

Added in commits 27e03fe and 39a0281:

Unit tests (test/app.multispec.test.ts):

  • Code generation: verifies route files appear under {base}/routes/ for each spec, that files don't bleed between specs (alpha has no items.ts, beta has no ping.ts), and that {base}/types/_.context.ts is generated for each spec
  • REPL: verifies startRepl() is returned as a function in multi-spec mode, and that the primary (first spec's) registry is the one wired to the REPL

Black-box tests (test-black-box/test_parallel_apis.py) with matching OpenAPI fixtures (alpha.yaml, beta.yaml):

  • /alpha/ping returns "alpha-pong" and /beta/ping returns "beta-pong" — specs serve their own distinct responses
  • Route files land under out/alpha/routes/ and out/beta/routes/ without cross-contamination
  • Requests to /gamma/ping (unmatched prefix) return 404
  • When both spec and specs are in counterfact.yaml, specs wins: /alpha/ping returns 200, /ping at root returns 404

)
if response.status_code == 200:
return
except requests.exceptions.RequestException:
Copilot AI requested a review from pmcelhaney April 13, 2026 22:04
Copy link
Copy Markdown
Collaborator

@pmcelhaney pmcelhaney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of a multispecmiddleware, pass an array of middleware to createKoaApp() and use() each one in succession.

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 13, 2026

Instead of a multispecmiddleware, pass an array of middleware to createKoaApp() and use() each one in succession.

Done in commit ac3a2bf. buildMultiSpecMiddleware() is removed. createKoaApp() now accepts Koa.Middleware | Koa.Middleware[] and calls app.use(m) for each entry. In multi-spec mode app.ts builds one koaMiddleware per spec (each already knows its routePrefix and calls next() for paths outside it) and passes the array directly — the Koa chain does the fan-out naturally.

Copilot AI requested a review from pmcelhaney April 13, 2026 22:30
Copy link
Copy Markdown
Collaborator

@pmcelhaney pmcelhaney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When the config specifies a single spec, translate that to a config that has a specs property with one item.

After that there should be no divergence for single vs. multiple specs.

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 13, 2026

When the config specifies a single spec, translate that to a config that has a specs property with one item.

After that there should be no divergence for single vs. multiple specs.

Done in commit e409237. At the top of counterfact() a single-spec config is now normalized to specs = [{ source: config.openApiPath, base: "" }], then only the unified loop runs — the separate single-spec branch is removed. createSpecBundle() handles source === "_" and scenarios are wired to the first spec only. Specs with base === "" preserve config.routePrefix; named specs use "/" + base.

Copilot AI requested a review from pmcelhaney April 13, 2026 22:53
Copy link
Copy Markdown
Collaborator

@pmcelhaney pmcelhaney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adminApiMiddleware should be built in app.js and passed in to createKoaApp()

const primaryBundle = specBundles[0] as SpecBundle;
const primarySpec = specs[0] as (typeof specs)[number];

// Build one per-spec middleware. For the normalised single-spec entry
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is now an array

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in commit ec465e0 — the single-spec test now asserts Array.isArray(result.koaMiddleware) is true and result.koaMiddleware has length 1, reflecting that the unified code path always returns an array.

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 13, 2026

adminApiMiddleware should be built in app.js and passed in to createKoaApp()

Done in commit ec465e0. adminApiMiddleware is now built in app.ts and passed to createKoaApp() as an optional adminMiddleware parameter. createKoaApp() no longer imports or constructs it internally, and no longer needs registry or contextRegistry in its signature.

Copilot AI requested a review from pmcelhaney April 13, 2026 23:09
Copy link
Copy Markdown
Collaborator

@pmcelhaney pmcelhaney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should only be one transpiler, not one per bundle.

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 13, 2026

There should only be one transpiler, not one per bundle.

Done in 4557caa. Transpiler now takes a single rootPath string (plus moduleKind). It watches rootPath recursively and applies the convention routes/ → .cache/ internally — no per-spec instances, no array of path mappings. In app.ts a single new Transpiler(config.basePath, "commonjs") covers all specs automatically.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parallel APIs second pass

2 participants