Skip to content

Fix caller permissions and Studio source probes with blocking type checks - #184

Merged
dwickyfp merged 2 commits into
mainfrom
feat/governed-ingestion-pipes
Oct 14, 2026
Merged

dwickyfp merged 2 commits into
mainfrom
feat/governed-ingestion-pipes

Conversation

@dwickyfp

@dwickyfp dwickyfp commented Oct 13, 2026 •

Copy link
Copy Markdown
Owner

Pipe Manager opened a root connection for authenticated requests. Route its operations through Nova's shared SQL pipeline with the caller, active role, session, and security-context version. Validate identifiers and properties, use the pinned engine's metadata fields, and redact stored source statements and errors.

Reject CREATE PIPE and execution-enabling ALTER PIPE during Ranger planning because StarRocks 4.1.4 background pipe tasks do not preserve the initiating security context. Inspection, suspension, and deletion remain subject to authorization. Record pipe write and external-I/O effects for shared SQL consumers.

Studio source-availability checks fetched rows to determine access. Plan EXPLAIN SELECT ... LIMIT 1 through the same caller-scoped query service instead, retaining Ranger policy checks and engine authorization without fetching source data. Keep existing deadlines and concurrency bounds. Real-engine regression tests cover empty tables, active-role changes, SELECT revocation, missing sources, and audit records; patched-FE acceptance also checks Ranger denial.

Resolve application mypy errors with typed model contracts, checked nullable values, and development-only third-party stubs. Enable the Pydantic plugin and make full-application mypy blocking in CI. Generated ANTLR artifacts retain their regeneration checks; existing locked dependency versions are unchanged. Preserve provider defaults and startup behavior when Redis is unavailable; leader-lock operations still require Redis.

Validation on 0bd35d8:

  • Backend unit and branch coverage: 7,487 passed; 61% rounded coverage.
  • Agent eval: 48/48 scenarios and 165/165 checks.
  • Seeded StarRocks/Redis/MinIO integration: 215 passed, 16 skipped, 30 deselected; no failures or errors. The previously failing Studio production-API scenario and the new source-access regression passed. All four pipe cases passed.
  • Fresh real-engine scheduler verification after the final leader-lock change: 7 passed.
  • Full-application mypy passed on macOS and Linux; changed-file Ruff, grammar pin/drift, and the user-data system/root boundary scan passed.
  • Separate patched-FE/Ranger acceptance passed for active roles, row filters, masking, session/transaction isolation, denied roles, source probes, root guards, agent tools, and pipe refusal.

The 16 integration skips require opt-in installations, live providers, or specialized engine configuration; they are not counted as passes. The pipe lifecycle test uses synthetic data with auto-ingest disabled and does not establish loading throughput or background delivery guarantees. Existing native pipes must be suspended before enabling Ranger; the planning gate cannot stop already-running engine tasks.

GitHub CI passed all five jobs for 0bd35d8e46addb798f29306bb6a5af0be94d9987: Actions run 38500463113. Backend unit: 7,487 passed with 61% rounded coverage; eval: 48/48 scenarios and 165/165 checks; integration: 215 passed, 16 skipped, 30 deselected, zero failures/errors; frontend lint/build and 1,197 tests passed; static checks and parser regeneration/drift passed. Patched-FE/Ranger acceptance was verified locally and is separate from default CI.

@dwickyfp dwickyfp changed the title Run Pipe Manager operations with caller permissions and gate Ranger ingestion Fix caller permissions and Studio source probes with blocking type checks Oct 14, 2026
@dwickyfp
dwickyfp marked this pull request as ready for review October 14, 2026 10:35
@dwickyfp
dwickyfp merged commit 2a4def7 into main Oct 14, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant