feat(AF-612): import a Postman collection as a connector schema#620
Merged
Merged
Conversation
Teams without an OpenAPI/WSDL/SDL/proto document usually do have a Postman
collection that is already the de-facto contract. PostmanCollectionParser
accepts a Collection v2.x export behind the existing ApiSchemaParser SPI, so
neither SchemaParserRegistry nor POST /api-connectors/{id}/schemas changed.
Folders flatten into a slugified, deterministic operationId; collection
variable[] values are substituted and every remaining {{var}} (and Postman's
:id form) becomes a {var} path template. Postman stores examples rather than
schemas, so requestSchema/responseSchema are inferred by JsonShapeInferrer --
a real fidelity gap, stated in the upload UI.
Two security properties drove the design. Only the declared auth *type* is
read (surfaced as api_schemas.detected_auth_method so the admin knows what to
re-enter), never a credential value; and event blocks -- arbitrary
pre-request/test JavaScript -- are dropped entirely. Since raw_content
persists the document verbatim, neither could hold by parsing alone, so the
SPI now returns ParsedApiSchema (operations + detectedAuthMethod +
sanitizedContent) and the parser hands back a redacted copy that is stored in
place of the upload. The other four parsers return null for both new fields.
A v1 export is rejected 422 pointing at Postman's v2.1 export, as are
documents over 5 MiB or 2000 requests.
Frontend: Postman option on the schema-type select, a caveat banner covering
both notices, the file picker converted to an Upload.Dragger with the size
guard and error surface it previously lacked, and a detected-auth column.
Docs: 03-data-model, 04-api-spec, 05-backend, 17-api-governance,
website/index.html, website/docs/index.html.
Closes #612
Contributor
Contributor
Coverage Report for Frontend Coverage (frontend)
File Coverage
|
||||||||||||||||||||||||||||||||||||||
Contributor
Contributor
Backend Code Coverage
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #612
Makes a Postman Collection v2.x export a first-class
apigovschema source.PostmanCollectionParserslots in behind the existingApiSchemaParserSPI, soSchemaParserRegistrypicks it up fromsupportedType()andPOST /api-connectors/{id}/schemasneeded no new route.Mapping
Folders flatten into a slugified, deterministic
operationId(billing/invoices/create-invoice, collisions suffixed-2/-3). Collectionvariable[]values are substituted; every remaining{{var}}— and Postman's:idparam form — becomes a{var}path template, leavingbase_urlto the admin. Read/write classification is the usual safe-method rule, so the existingrequire_review_reads/require_review_writesgates apply unchanged.Postman stores examples, not schemas, so
requestSchema/responseSchemaare inferred from the saved example bodies by a newJsonShapeInferrer. That fidelity gap is real and is stated in the upload UI.Security shaped the design
api_schemas.raw_contentpersists the uploaded document verbatim, and Postman exports routinely carry live tokens plus arbitrary pre-request/test JavaScript. So "no secret is persisted" and "eventblocks are dropped" could not hold by parsing alone.The
ApiSchemaParserSPI therefore now returnsapigov.api.ParsedApiSchema(operations +detectedAuthMethod+sanitizedContent) instead of a bare list. The Postman parser returns a redacted copy of the collection — everyeventblock and every auth credential array removed, at collection and per-request level — andDefaultApiSchemaServicestores that instead of the upload. Only the declared auth type is read, surfaced asapi_schemas.detected_auth_methodso the admin knows what to re-enter on the connector. The other four parsers returnnullfor both new fields and are otherwise untouched.A v1 export is rejected
422pointing at Postman's v2.1 export; so are documents over 5 MiB or 2000 requests (parser constants mirroringMAX_FETCH_BYTES— no new env var).Frontend
Postman option on the schema-type select; a caveat banner carrying both required notices (inferred schemas, credentials ignored); the file picker converted from a plain button
Uploadto anUpload.Draggerfollowing theCustomDriverUploadModalpattern, adding the size guard and inline error surface it previously lacked; a detected-auth column on the schema table. Also fixescanUpload, which accepted a whitespace-only file.Two deliberate deviations from the issue
messages.propertieskey.ApiGovExceptionHandlerresolves one localized detail (error.api_schema_parse) and attaches the parser's English message as a non-localizedreasonproperty — the same shape all four existing parsers use. Adding per-rejection keys would have been inconsistent with that.MessagesParityTestis untouched; the seven frontend locales are updated.requestSchema/responseSchemahave no consumer yet. The feat(AF-499): Data Lifecycle Manager — retention policies, right-to-erasure, proxy-enforced pseudonymization & compliance #518SCHEMA_FIELDmasking matcher takes an admin-typed dot-path, not a parsed schema. The inferred shapes are persisted inparsed_operationsfor a future consumer, per an explicit scoping decision.Migrations
V122—ALTER TYPE api_schema_type ADD VALUE 'POSTMAN_COLLECTION', with the paired.sql.conf(executeInTransaction=false) required by theV81precedent.V123— nullableapi_schemas.detected_auth_method api_auth_method.Docs & website
docs/03-data-model.md,docs/04-api-spec.md(new Postman collection import section),docs/05-backend.md,docs/17-api-governance.md,website/index.html,website/docs/index.html. No new env var, sodocs/09-deployment.mdand the CLAUDE.md env table are deliberately unchanged.Verification
ApplicationModulesTest,ApiPackageDependencyTest,MessagesParityTesttest:coverage/ buildNot run locally: Testcontainers integration tests and the Playwright e2e suite — Docker is unavailable in my environment. All 699 backend errors in the full run traced to
Could not find a valid Docker environment; every one aborts at container startup before reaching application code. That means the two migrations are unverified against a real PostgreSQL — they follow theV81/V100patterns exactly and the entity mapping matchesApiConnectorEntity.authMethod, but CI is the first real check. The new e2e spec (admin imports a Postman collection as a connector schema) also has not been executed; it asserts the flattened ids, the{id}template, the detected-auth tag, and that neither the export's token nor its script text survives into the operation catalog. Please watch thebackendande2eCI jobs.Noted, not fixed
DefaultApiSchemaService.fetchguardssourceUrlwith a scheme check only — no loopback/link-local/RFC1918 block, sohttp://169.254.169.254/…is reachable by any connector-manage holder. Postman reuses that guard unchanged, as the issue asked. Hardening it affects all five schema types equally and belongs in its own issue.