ACE-041: PII masking (mask-else-refuse)#128
Draft
sandeep-agami wants to merge 11 commits into
Draft
Conversation
TEST-FIRST checkpoint. Adds tests/test_sensitive_projection_maskable.py — a broad matrix asserting that check_sensitive_projection classifies each offending sensitive projection as maskable (with its 0-based output-column index) vs must-refuse (buried in expression/function/scalar-subquery/star). Fails on the not-yet-added SensitiveCheckResult.projections field, as intended. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…, tests green Extend the sensitive-projection guard so each offending projection also carries the traceability a later masking slice needs, without changing runtime behaviour (the executor still refuses on ANY sensitive projection, exactly as today). - New frozen SensitiveProjection(column, maskable, output_index) and a SensitiveCheckResult.projections: list[SensitiveProjection] field (as_dict serializes it). Legacy action/columns/reason/suggestion are byte-identical. - Per projection (per set-operation arm), classify by SHAPE: (a) * / t.* -> must-refuse (no single deterministic output index), (b) col / t.col / AS x -> maskable at its 0-based projection index, (c) sensitive col buried in expr/func/scalar-subquery -> must-refuse. Fail-closed: only a bare/qualified/aliased single-column projection is maskable. - Reuses _output_selects (every arm), _resolve_col_table, _direct_from_tables, _count_protects, _sensitive_by_table, build_guard_context — single sqlglot parse, no second parser. Offending-set computation is preserved exactly; two small helpers (_sensitive_col_ref, _classify_projection) factor the shape logic. Caller execute_sql.py::_model_safety unchanged (reads action/reason/suggestion only). 33 new tests pass; full suite 1562 passed / 1 skipped, ruff clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the data_protection stub regression pin with the real contract: a `provable` (deterministically maskable) verdict maps to `mask` in every tier; `heuristic`/`uncertain`/unexpected certainty fails closed to `reject` in every tier. Two maskable cases fail red against the current fail-closed stub, as intended (test-first). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implement policy()'s data_protection branch. It keys on `certainty`, exactly like the safety branch: `provable` (the gate proved every offending sensitive projection is a deterministically maskable 1:1 image of one output column) -> `mask`; any weaker certainty (`heuristic`/`uncertain`/unexpected) -> `reject`, fail-closed, since the raw value cannot be safely masked. It enforces in every tier — tier is deliberately not consulted, unlike governance. Which output columns to mask travels separately on the guard result, not on the Verdict; policy() only decides mask-vs-reject. Re-vendored via `dev.py sync-lib`; plugins/agami/lib/guardrail.py is byte-identical to src (no unexpected drift). Guardrail tests pass against both src and the vendored lib; full suite green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… tests Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…SSN" bypass) _sensitive_by_table now folds table+column keys and preserves the declared spelling in the ref value; _sensitive_col_ref and the star branch fold-match. Flips the (now spec-obsolete) case-sensitive pin to assert mask, adds SSN/ mixed-case/quoted matrix rows + the non-sensitive same-name decoy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…d[{mask}]
_model_safety now returns (sql, Refusal|None, MaskPlan|None): a maskable
sensitive projection routes through a data_protection Verdict + policy -> MASK
plan (query proceeds), an untraceable one fails closed. execute_guarded redacts
result.rows at the single shared post-execute point (covers _run_<db> and any
injected ports.Executor). ExecResult.masked_columns + a {"masked": …} stderr
marker carry the applied[{mask}] note on both surfaces. Unparseable SQL with
declared PII fails closed (no ACE-037 upstream gate on this branch).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ty stubs; sync-lib
- test_ah012: real-PII maskable projection now MASKS (query runs, value redacted,
applied[{mask}] recorded); added an untraceable-projection refuse counterpart.
- test_sample_database: raw-PII bare column now REDACTED end-to-end (token on stdout,
{masked} marker on stderr); added an untraceable-still-refuses counterpart.
- Mechanical: _model_safety callers/stubs widened 2->3 tuple (ace028, ace051, ah012).
- sync-lib: vendored plugins/agami/lib/execute_sql.py byte-identical to source.
These are spec-driven data-protection behaviour changes; NO F9 safety test modified.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Finding 1 (correctness/fail-safe): _apply_mask_plan now builds the
applied[{mask}] / masked_columns receipt via a new _mask_note helper. It
keeps every declared sensitive table.column ref AND appends the output
label (from result.columns) of any extra positionally-aligned position a
set operation forced us to redact — e.g.
"SELECT ssn, name FROM customers UNION ALL SELECT name, ssn FROM customers"
redacts BOTH output columns (each carries a raw SSN in some arm) and the
note now reads ("customers.ssn", "name") instead of under-reporting as
("customers.ssn",). A position is "already named" iff its output label
case-folds to the bare column name of a declared ref; the normal
single-column case is therefore unchanged and still reports exactly
{"mask": "customers.ssn"} (success criterion 1). MaskPlan.columns is left
as the declared refs so _model_safety's plan is unchanged; the enrichment
happens once, post-execution, so both surfaces (in-process rows +
subprocess {"masked": ...} stderr line) inherit it. Vendored copy synced.
Finding 2 (known limitation, NOT fixed): documented the derived-table /
CTE alias-rename bypass at semantic_model/runtime.py::_output_selects with
an "# ACE-041 known limitation" comment pointing at the xfail regressions.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Finding 1 — cross-position set operations (pin the behaviour + the
enriched receipt): UNION ALL / UNION / INTERSECT / EXCEPT of
(ssn, name) vs (name, ssn) redacts EVERY output position and leaks no raw
SSN on both surfaces (in-process rows + CSV wire); the note is asserted as
("customers.ssn", "name"). The partial case
"SELECT ssn, id FROM customers UNION ALL SELECT cust_id, id FROM orders"
masks only column 0 and names only its declared ref. The single-column
note is re-pinned unchanged, and _mask_note is unit-tested directly.
Finding 2 — xfail the derived-table / CTE alias-rename bypass:
"WITH t AS (SELECT ssn AS s FROM customers) SELECT s FROM t" and
"SELECT z FROM (SELECT ssn AS z FROM customers) q" assert the DESIRED
secure behaviour (outer projection masked OR refused). They xfail today
(raw SSN flows through) with strict=False, so a future alias-lineage fix
flips them to xpass as the tracking signal.
Finding 3 — corpus gaps now asserted (were probe-only): a decimal
sensitive column masks to "***" on both surfaces; a NULL at a masked
index becomes the token deterministically (native + in-process); an empty
result still records applied[{mask}] with zero rows; and a REAL
subprocess fork ("python -m execute_sql" over an on-disk model + sqlite)
asserts "***" in the emitted CSV, the raw value absent, and the
{"masked": ...} stderr line present. The injected ports.Executor path is
already covered by the existing slice-3 tool-edge test (not duplicated).
Added an autouse fixture that resets the process-global injected executor
between tests. No existing test weakened; F9 safety tests untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Turns the refuse-only sensitive-projection guard into mask-else-refuse (F10 data-protection): a raw projection of a model-declared
sensitivecolumn is redacted to***instead of blocking the whole query; a projection that can't be safely traced to a single output column is refused (fail-closed). Enforced once at the shared executor chokepoint, so it holds on every engine and on both surfaces.Spec: ACE-041Changes
runtime.check_sensitive_projection→ per-projectionSensitiveProjection{column, maskable, output_index}(bare /t.col/ aliased → maskable at its output index; expression / function / scalar-subquery /*→ must-refuse; every set-op arm walked).guardrail.policy()data_protectionbranch:certainty="provable" → "mask", else"reject"(fail-closed, enforces in every tier).execute_sql._model_safetybuilds adata_protectionVerdict(provableiff every offending projection is maskable) →policy→ aMaskPlanor a refusal;execute_guardedapplies redaction in a new shared post-executor.execute()step (covers any injectedports.Executor; composes with the already-merged row cap).applied[{mask}]recorded on both surfaces (in-process 4-tuple →_finalize_execution; subprocess{"masked":[...]}stderr line). Cross-position set-op notes report every masked output position.SELECT "SSN"/SELECT SSNbypass, consistent with the table/column-scope gates.Test plan
Executor, numeric / NULL / empty results, and row-cap composition.dev.py checkgreen: ruff (check + format), 1608 passed / 1 skipped / 2 xfailed, gitleaks, vendored-lib drift exit 0.Sign-off items (why this is a draft)
***.WITH t AS (SELECT ssn AS s FROM customers) SELECT s FROM t) escapes both mask and refuse. Pre-existing in the name-based sensitive-projection gate (present on the base branch), not introduced here. Documented in-code + pinned by@pytest.mark.xfailtripwire tests; follow-up spec ACE-062 filed.Operator note:
--no-safetyskips masking along with all model guards (operator-only; not reachable from the MCP tool).