Skip to content

feat(AF-614): filter connector operations at schema import#618

Merged
babltiga merged 2 commits into
mainfrom
feature/AF-614-filter-operations-at-schema-import
Jul 20, 2026
Merged

feat(AF-614): filter connector operations at schema import#618
babltiga merged 2 commits into
mainfrom
feature/AF-614-filter-operations-at-schema-import

Conversation

@babltiga

Copy link
Copy Markdown
Contributor

Closes #614

What

Lets an admin declare, per schema upload, which operations to keep — so an API connector's governed operation catalog only ever contains operations they intend to expose. Real-world OpenAPI documents carry /internal/**, /actuator/**, deprecated, and admin-only surfaces that previously could only be kept out downstream, per-grant.

Design decision (the one the issue flagged)

Persist the filter + apply on read, not parse-time filtering:

  • parsed_operations keeps the complete parsed catalog; the new nullable api_schemas.operation_filter JSONB stores the filter.
  • DefaultApiSchemaService.listOperations applies it — the single choke point that makes filtered-out operations unreachable from /api-editor, text-to-API, and the allowed_operations grant picker alike (they all read through it).
  • operation_count is the post-filter (kept) count; totalOperationCount exposes the pre-filter size.
  • PUT /api-connectors/{id}/schemas/{schemaId}/filter re-edits the filter without re-uploading the document or re-fetching a remote sourceUrl. Parse-time filtering wouldn't have kept the excluded surface off disk anyway — raw_content stores the document regardless.

Filter dimensions

Path glob, HTTP verb, operation-id glob, tag, and excludeDeprecated. Evaluated as keep when every non-empty include dimension matches AND no exclude dimension matchesexclude wins. Globs use the same *-only syntax as the routing-policy table globs; verbs and tags are exact case-insensitive matches.

ApiOperation gains tags + deprecated (boxed Boolean on purpose — schemas parsed before this change deserialize as null rather than tripping Jackson's FAIL_ON_NULL_FOR_PRIMITIVES). Both are OpenAPI-only; for GraphQL SDL / WSDL / gRPC proto the operation-id glob is the load-bearing dimension since path is synthesized.

POST .../schemas/preview dry-runs a filter (kept/excluded counts + the dropped operations) without persisting or auditing.

API_SCHEMA_UPLOADED audit metadata now carries the filter, total_operation_count, and excluded_count, for uploads and filter edits alike (the edit path is tagged action=filter_updated, reusing the action to avoid AuditAction enum fan-out).

An absent or empty filter is exactly the previous behaviour.

Frontend

ApiConnectorSettingsPage → Schema tab gains a collapsible Import filter (optional) section (exclude/include path globs, exclude verbs, exclude operation-id globs, exclude tags, exclude-deprecated checkbox), a Preview button reporting "Keeps K of N operations" plus the dropped list, an "N of M operations imported" line after upload, and a per-row Edit filter modal. Client-side caps (100 entries / 200 chars) mirror the backend Bean Validation.

Docs & website updated

  • docs/03-data-model.mdoperation_filter column, post-filter operation_count semantics
  • docs/04-api-spec.md — request shape, the two new endpoints, and an Operation import filter section
  • docs/05-backend.md — apigov "Operation import filter (AF-614)" prose
  • docs/06-frontend.mdApiConnectorSettingsPage row
  • website/index.html + website/docs/index.html — user-facing description of the filter

No new env var, no new dependency, no new AuditAction → no CLAUDE.md change.

Tests

  • OperationFilterMatcherTest (14) — each dimension independently, exclude-beats-include, empty-filter passthrough, glob edge cases, tags, deprecated
  • OperationFilterTest (4) — canonicalization, defensive copying, isEmpty
  • DefaultApiSchemaServiceTest — filter persisted, full catalog retained, applied on read, operation_count post-filter, updateFilter recomputes, previewFilter doesn't persist
  • ApiSchemaControllerTest — upload/preview/edit paths and audit metadata
  • OpenApiSchemaParserTest — tags + deprecated captured
  • ApiConnectorSettingsPage.test.tsx — 5 new tests (net-new coverage; the Schema tab was previously untested)

Verification: backend Failures: 0 across 5085 tests — the 687 errors are all Could not find a valid Docker environment (Testcontainers *IntegrationTest classes; Docker isn't running locally, CI will run them). ApplicationModulesTest, ApiPackageDependencyTest, and MessagesParityTest green. Frontend: 1258/1258 tests, lint (0 errors), typecheck, and build all green.

No new Playwright spec: schema upload isn't covered in e2e/tests/ today, and no existing spec drives the Schema tab or the operations catalog (verified), so nothing drifts. The new flow is covered by component tests instead.

Real-world OpenAPI documents carry operations nobody should reach through
AccessFlow (/internal/**, /actuator/**, deprecated surfaces). Admins can now
declare an operation filter per schema so the connector's governed catalog only
contains operations they intend to expose.

The filter is persisted on api_schemas.operation_filter (new nullable JSONB) and
applied on the read path rather than at parse time: parsed_operations keeps the
complete catalog, so the filter is re-editable via PUT .../schemas/{id}/filter
without re-fetching a remote sourceUrl. listOperations is the single choke point,
so filtered-out operations are unreachable from /api-editor, text-to-API, and the
allowed_operations grant picker alike. operation_count is the post-filter count.

Dimensions: path glob, HTTP verb, operation-id glob, tag, and excludeDeprecated.
Include and exclude compose as "keep when every non-empty include matches and no
exclude matches" - exclude wins. ApiOperation gains tags + deprecated (boxed, so
pre-AF-614 rows deserialize as null instead of tripping FAIL_ON_NULL_FOR_PRIMITIVES);
both are OpenAPI-only. POST .../schemas/preview dry-runs a filter without
persisting. API_SCHEMA_UPLOADED audit metadata records the filter, total count,
and excluded count for uploads and filter edits.

An absent or empty filter is exactly the previous behaviour.

Closes #614
@github-actions

github-actions Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Frontend Test Results

1 258 tests   1 258 ✅  4m 18s ⏱️
  166 suites      0 💤
    1 files        0 ❌

Results for commit 6cd2058.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Backend Test Results

5 148 tests  +26   5 148 ✅ +26   16m 59s ⏱️ +19s
  660 suites + 2       0 💤 ± 0 
  660 files   + 2       0 ❌ ± 0 

Results for commit 6cd2058. ± Comparison against base commit ab52346.

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Backend Code Coverage

Overall Project 93.2% -0.06% 🍏
Files changed 90.85% 🍏

File Coverage
ApiOperationResponse.java 100% 🍏
OperationFilterPreviewResponse.java 100% 🍏
OperationFilterRequest.java 100% 🍏
ApiSchemaResponse.java 100% 🍏
ApiSchemaView.java 100% 🍏
OperationFilter.java 100% 🍏
ApiOperation.java 100% 🍏
OperationFilterMatcher.java 99.01% -0.99% 🍏
ApiSchemaController.java 97.54% -2.46% 🍏
GlobMatcher.java 96.55% -3.45% 🍏
OperationFilterPreview.java 89.47% -10.53% 🍏
OpenApiSchemaParser.java 89.36% 🍏
UploadApiSchemaRequest.java 87.5% -12.5% 🍏
DefaultApiSchemaService.java 86.49% -4.14% 🍏
OperationFilterResponse.java 11.86% -88.14% 🍏

The "edits an existing schema filter" test timed out at 15s on CI. It leaned on
whole-page findByRole(..., { name }) queries, which compute accessible names
across every mounted element — and AntD keeps all six tab panels mounted, so the
scan covered the entire settings page. getByRole('button', { name: 'Save' }) was
also ambiguous: the Config tab has its own Save button.

Query the modal through within(dialog) and use text queries instead. The file
drops from ~11.7s to ~1.5s, so the global 15s timeout is ample headroom again.
@github-actions

Copy link
Copy Markdown
Contributor

Coverage Report for Frontend Coverage (frontend)

Status Category Percentage Covered / Total
🟢 Lines 93.75% (🎯 90%) 2058 / 2195
🟢 Statements 91.72% (🎯 90%) 2296 / 2503
🟢 Functions 90.92% (🎯 90%) 631 / 694
🟢 Branches 83.82% (🎯 80%) 1296 / 1546
File Coverage
File Stmts Branches Functions Lines Uncovered Lines
Changed Files
frontend/src/api/apiConnectors.ts 88.23% 100% 85.71% 87.75% 30, 32, 95-111
Generated in workflow #795 for commit 6cd2058 by the Vitest Coverage Report Action

@babltiga
babltiga merged commit 1a344de into main Jul 20, 2026
28 checks passed
@babltiga
babltiga deleted the feature/AF-614-filter-operations-at-schema-import branch July 20, 2026 07:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

apigov: filter operations at schema import time (exclude /internal, deprecated, by verb/tag/path glob)

1 participant