diff --git a/README.md b/README.md index 353830ea..406b9595 100644 --- a/README.md +++ b/README.md @@ -49,7 +49,8 @@ container credentials, and the default AWS credential chain. Local sign-in honors Codex's configured credential backend, including a system keyring required by a managed device. Codex Security keeps login and scan -credentials in the same private, persistent state directory. +sign-in scoped to a dedicated Codex home in the same private, persistent state +directory. If both a ChatGPT sign-in and an API key are available, interactive scans ask which credential to use. CI and other noninteractive scans keep the existing @@ -67,12 +68,26 @@ keys: unset OPENAI_API_KEY CODEX_API_KEY ``` -Scan history is stored in the Codex Security workbench state directory. If that -directory cannot be written, set `CODEX_SECURITY_STATE_DIR` to a writable -directory outside the repository. +Scan history and saved findings are stored in one Codex Security state database. +Linked Git worktrees are discovered automatically and grouped when they use the +same state directory. Leave `CODEX_SECURITY_STATE_DIR` unset, or select one +stable, writable directory outside the repository. Changing or unsetting it +selects separate history and an isolated Codex credential home and sign-in +scope; restore the previous value to reopen its existing scans and sign-in. + +```bash +# Run from another linked Git worktree: +npx @openai/codex-security scans list +npx @openai/codex-security findings list + +# Reopen an existing, separately selected state directory: +export CODEX_SECURITY_STATE_DIR=/path/to/existing/codex-security-state +npx @openai/codex-security scans list +``` `findings list [repository]` shows open findings across a repository's scans -and identifies findings not confirmed in its latest scan. +and identifies findings not confirmed in its latest completed scan across +linked worktrees. `scans compare BEFORE_SCAN_ID AFTER_SCAN_ID` automatically matches findings by root cause, reuses saved matches, and identifies new, persisting, reopened, diff --git a/sdk/typescript/README.md b/sdk/typescript/README.md index 5c2cf069..1f30747b 100644 --- a/sdk/typescript/README.md +++ b/sdk/typescript/README.md @@ -426,7 +426,7 @@ The CLI and SDK recognize the following user-configurable environment: | `OPENAI_API_KEY`, `CODEX_API_KEY` | Scan authentication; `OPENAI_API_KEY` wins when both are present. | | `CODEX_SECURITY_LOG_LEVEL` | CLI-only; set to `debug` for verbose diagnostics. | | `LOG_LEVEL` | CLI-only fallback when `CODEX_SECURITY_LOG_LEVEL` is unset. | -| `CODEX_SECURITY_STATE_DIR` | Override the private scan-history, workbench, and default artifact directory. | +| `CODEX_SECURITY_STATE_DIR` | Select the history database, artifact directory, and dedicated Codex credential home. | | `CODEX_HOME` | Set the ambient Codex home for file-backed sign-in and default state; defaults to `~/.codex`. | | `CODEX_CLI_PATH` | Use another Codex executable for authentication, plugin setup, scans, and nested workers. | | `PYTHON` | Select a Python interpreter when `--python` or SDK `pythonPath` is not set. | @@ -538,11 +538,15 @@ same command to resume. ### Scan history and reruns -`npx @openai/codex-security scans list` lists scans for the current repository. Pass a -repository path to inspect another checkout, `--scan-root DIR` to list scans -whose artifacts are under a particular root. `scans show SCAN_ID` includes the -scan configuration, results, coverage, and artifact locations. Add -`--show-linked-findings` to include finding links from previous scans. +`npx @openai/codex-security scans list` lists scans for the current repository. +Linked Git worktrees are discovered automatically and grouped when they share +the selected state database. Findings indicate whether they were confirmed in +the repository's latest completed scan across those linked worktrees. Pass a +repository path to inspect another checkout. `--scan-root DIR` only filters +scans already recorded in that database by their artifact directory; it never +imports scan results from another state directory. `scans show SCAN_ID` +includes the scan configuration, results, coverage, and artifact locations. +Add `--show-linked-findings` to include finding links from previous scans. `scans logs SCAN_ID` shows complete session events from the scan and its workers, which can include source code and credentials. @@ -553,9 +557,27 @@ least eight characters. Scan history uses `$CODEX_SECURITY_STATE_DIR/workbench.sqlite3` when `CODEX_SECURITY_STATE_DIR` is set. Otherwise, it uses `$CODEX_HOME/state/plugins/codex-security/workbench.sqlite3`; `CODEX_HOME` -defaults to `~/.codex`. Scan credentials are never stored in the scan -configuration. Recorded failure summaries and bulk-scan receipts omit messages -that contain recognizable credentials. +defaults to `~/.codex`. Saved findings use the same selected database. +Changing or unsetting `CODEX_SECURITY_STATE_DIR` selects separate scan history +and an isolated Codex credential home and sign-in scope; scans from the +previous state remain hidden until you select that state again. Keep the +setting stable across linked worktrees to share scans and findings +automatically. + +```bash +# Inspect shared history from another linked Git worktree: +npx @openai/codex-security scans list +npx @openai/codex-security findings list + +# Reopen an existing state directory and its sign-in: +export CODEX_SECURITY_STATE_DIR=/path/to/existing/codex-security-state +npx @openai/codex-security scans list +npx @openai/codex-security findings list +``` + +Scan credentials are never stored in the scan configuration. Recorded failure +summaries and bulk-scan receipts omit messages that contain recognizable +credentials. The scan sandbox permits writes to the selected state directory so SQLite can maintain its database and journal files. If the host itself cannot write to the diff --git a/sdk/typescript/_bundled_plugin/scripts/workbench_feedback.py b/sdk/typescript/_bundled_plugin/scripts/workbench_feedback.py index 51b08a16..ac3e1651 100644 --- a/sdk/typescript/_bundled_plugin/scripts/workbench_feedback.py +++ b/sdk/typescript/_bundled_plugin/scripts/workbench_feedback.py @@ -16,12 +16,24 @@ FINDING_SUMMARY_BYTES, FINDING_TITLE_BYTES, ) +from workbench_native_indexes import _indexed_findings, repository_target_ids from workbench_validation import bounded_output_text def get_scan_feedback(connection: sqlite3.Connection, scan: sqlite3.Row) -> dict[str, Any]: + target_ids = sorted(repository_target_ids(connection, scan["target_id"])) + if not target_ids: + return {"scanId": scan["id"], "targetId": scan["target_id"], "falsePositives": []} + + indexed_findings = { + finding_id: finding + for finding in _indexed_findings(connection) + if finding["target_id"] in target_ids + for finding_id in finding["matched_finding_ids"] + } + target_placeholders = ", ".join("?" for _ in target_ids) rows = connection.execute( - """ + f""" WITH ranked_decisions AS ( SELECT findings.id AS finding_id, findings.fingerprint, findings.rule_id, findings.identity_anchor, findings.identity_instance, occurrences.title, @@ -49,7 +61,7 @@ def get_scan_feedback(connection: sqlite3.Connection, scan: sqlite3.Row) -> dict candidate.sort_order LIMIT 1 ) - WHERE source_scans.target_id = ? + WHERE source_scans.target_id IN ({target_placeholders}) AND source_scans.id != ? AND source_scans.status = 'complete' ) @@ -61,12 +73,22 @@ def get_scan_feedback(connection: sqlite3.Connection, scan: sqlite3.Row) -> dict AND note IS NOT NULL AND trim(note) != '' ORDER BY updated_at DESC, source_completed_at DESC, source_scan_id DESC, finding_id DESC - LIMIT 50 """, - (scan["target_id"], scan["id"]), + (*target_ids, scan["id"]), ) false_positives = [] + reviewed_components: set[str] = set() for row in rows: + finding = indexed_findings.get(row["finding_id"]) + if ( + finding is None + or finding["status"] != "closed" + or finding["close_reason"] != "false_positive" + or finding["occurrence_id"] in reviewed_components + ): + continue + reviewed_components.add(finding["occurrence_id"]) + identity = {"anchor": row["identity_anchor"]} if row["identity_instance"] is not None: identity["instance"] = row["identity_instance"] @@ -91,6 +113,8 @@ def get_scan_feedback(connection: sqlite3.Connection, scan: sqlite3.Row) -> dict "updatedAt": row["updated_at"], } ) + if len(false_positives) == 50: + break return {"scanId": scan["id"], "targetId": scan["target_id"], "falsePositives": false_positives} diff --git a/sdk/typescript/_bundled_plugin/scripts/workbench_native_indexes.py b/sdk/typescript/_bundled_plugin/scripts/workbench_native_indexes.py index 8ce2492b..9d1df103 100644 --- a/sdk/typescript/_bundled_plugin/scripts/workbench_native_indexes.py +++ b/sdk/typescript/_bundled_plugin/scripts/workbench_native_indexes.py @@ -13,19 +13,65 @@ sys.path.insert(0, str(Path(__file__).resolve().parent)) import workbench_scan_history as scan_history from workbench_constants import FINDING_SUMMARY_BYTES, FINDING_TITLE_BYTES, FINDINGS_PAGE_MAX +from workbench_target_state import _require_current_target_owner from workbench_validation import bounded_output_text +def repository_target_ids(connection: sqlite3.Connection, target_id: str) -> set[str]: + if not _has_repository_identities(connection): + return {target_id} + + requested_target = connection.execute( + "SELECT current_path, repository_identity FROM security_targets WHERE id = ?", + (target_id,), + ).fetchone() + if requested_target is None: + return {target_id} + try: + _require_current_target_owner( + connection, + target_id, + requested_target["current_path"], + requested_target["repository_identity"], + ) + except SystemExit: + return set() + + rows = connection.execute( + """ + SELECT id + FROM security_targets + WHERE id = ? + OR ( + repository_identity IS NOT NULL + AND repository_identity = ? + ) + """, + (target_id, requested_target["repository_identity"]), + ) + return {row["id"] for row in rows} or {target_id} + + +def _has_repository_identities(connection: sqlite3.Connection) -> bool: + return any( + column["name"] == "repository_identity" + for column in connection.execute("PRAGMA table_info(security_targets)") + ) + + def list_global_findings( connection: sqlite3.Connection, args: argparse.Namespace, ) -> dict[str, Any]: limit = min(args.limit, FINDINGS_PAGE_MAX) query = args.query.strip().casefold() if args.query else "" + target_ids = ( + None if args.target_id is None else repository_target_ids(connection, args.target_id) + ) findings = ( row for row in _indexed_findings(connection) - if (args.target_id is None or row["target_id"] == args.target_id) + if (target_ids is None or row["target_id"] in target_ids) and (args.severity is None or row["severity"] == args.severity) and (args.status is None or row["status"] == args.status) and ( @@ -75,40 +121,85 @@ def list_global_findings( def _indexed_findings(connection: sqlite3.Connection) -> Iterator[dict[str, Any]]: - parents: dict[tuple[str, str], tuple[str, str]] = {} + parents: dict[ + tuple[tuple[str, str], str], tuple[tuple[str, str], str] + ] = {} + has_repository_identities = _has_repository_identities(connection) + identity_column = "targets.repository_identity" if has_repository_identities else "NULL" + before_identity_column = ( + "before_targets.repository_identity" if has_repository_identities else "NULL" + ) + after_identity_column = ( + "after_targets.repository_identity" if has_repository_identities else "NULL" + ) + alias_condition = ( + "before_targets.repository_identity IS NOT NULL " + "AND before_targets.repository_identity = after_targets.repository_identity" + if has_repository_identities + else "0" + ) + + def repository_identity(target_id: str, identity: str | None) -> tuple[str, str]: + return ("target", target_id) if identity is None else ("repository", identity) - def group(identity: tuple[str, str]) -> tuple[str, str]: + def group(identity: tuple[tuple[str, str], str]) -> tuple[tuple[str, str], str]: while identity in parents: identity = parents[identity] return identity for match in connection.execute( - """ - SELECT before_scans.target_id, before.finding_id AS before_finding_id, + f""" + SELECT before_scans.target_id AS before_target_id, + after_scans.target_id AS after_target_id, + {before_identity_column} AS before_repository_identity, + {after_identity_column} AS after_repository_identity, + before.finding_id AS before_finding_id, after.finding_id AS after_finding_id FROM scan_comparison_matches AS matches JOIN finding_occurrences AS before ON before.id = matches.before_occurrence_id JOIN scans AS before_scans ON before_scans.id = before.scan_id + JOIN security_targets AS before_targets ON before_targets.id = before_scans.target_id JOIN finding_occurrences AS after ON after.id = matches.after_occurrence_id JOIN scans AS after_scans ON after_scans.id = after.scan_id - WHERE before_scans.target_id = after_scans.target_id + JOIN security_targets AS after_targets ON after_targets.id = after_scans.target_id + WHERE before_scans.target_id = after_scans.target_id OR ({alias_condition}) """ ): - before = group((match["target_id"], match["before_finding_id"])) - after = group((match["target_id"], match["after_finding_id"])) + before = group( + ( + repository_identity( + match["before_target_id"], match["before_repository_identity"] + ), + match["before_finding_id"], + ) + ) + after = group( + ( + repository_identity( + match["after_target_id"], match["after_repository_identity"] + ), + match["after_finding_id"], + ) + ) if before != after: parents[after] = before - latest_scan_by_target = dict( - connection.execute( - "SELECT target_id, id FROM scans " - "WHERE status = 'complete' ORDER BY started_at, id" + latest_scan_by_repository = { + repository_identity(row["target_id"], row["repository_identity"]): row["id"] + for row in connection.execute( + f""" + SELECT scans.target_id, scans.id, {identity_column} AS repository_identity + FROM scans + JOIN security_targets AS targets ON targets.id = scans.target_id + WHERE scans.status = 'complete' + ORDER BY scans.started_at, scans.id + """ ) - ) + } - grouped: dict[tuple[str, str], list[sqlite3.Row]] = {} + grouped: dict[tuple[tuple[str, str], str], list[sqlite3.Row]] = {} for row in connection.execute( - """ + f""" SELECT occurrences.id AS occurrence_id, occurrences.finding_id, @@ -118,6 +209,7 @@ def group(identity: tuple[str, str]) -> tuple[str, str]: scans.started_at AS scan_started_at, scans.target_id, targets.current_path AS target_path, + {identity_column} AS repository_identity, scans.scope, MAX(scans.updated_at, COALESCE(triage.updated_at, '')) AS updated_at, triage.status AS decision_status, @@ -140,7 +232,15 @@ def group(identity: tuple[str, str]) -> tuple[str, str]: LEFT JOIN finding_triage AS triage ON triage.occurrence_id = occurrences.id """, ): - grouped.setdefault(group((row["target_id"], row["finding_id"])), []).append(row) + grouped.setdefault( + group( + ( + repository_identity(row["target_id"], row["repository_identity"]), + row["finding_id"], + ) + ), + [], + ).append(row) findings = [] for occurrences in grouped.values(): @@ -161,7 +261,10 @@ def group(identity: tuple[str, str]) -> tuple[str, str]: findings.append( { **dict(latest), - "confirmed_in_latest_scan": latest_scan_by_target.get(latest["target_id"]) + "close_reason": decision["close_reason"] if decision is not None else None, + "confirmed_in_latest_scan": latest_scan_by_repository.get( + repository_identity(latest["target_id"], latest["repository_identity"]) + ) == latest["scan_id"], "known_since": scans[0][0], "known_scan_ids": [scan_id for _, scan_id in scans], @@ -204,16 +307,28 @@ def list_repositories( ): latest_scan_by_target.setdefault(row["target_id"], scans_by_id[row["id"]]) - open_findings_by_target = Counter( - row["target_id"] for row in _indexed_findings(connection) if row["status"] == "open" - ) targets = {row["id"]: row for row in connection.execute("SELECT * FROM security_targets")} + + def repository_group(target_id: str) -> tuple[str, str]: + target = targets[target_id] + identity = ( + target["repository_identity"] if "repository_identity" in target.keys() else None + ) + return ("target", target_id) if identity is None else ("repository", identity) + + open_findings_by_repository = Counter( + repository_group(row["target_id"]) + for row in _indexed_findings(connection) + if row["status"] == "open" + ) repositories = [ { "checkoutAvailable": Path(target["current_path"]).is_dir(), "displayName": target["display_name"], "latestScan": latest_scan, - "openFindingsCount": open_findings_by_target.get(target_id, 0), + "openFindingsCount": open_findings_by_repository.get( + repository_group(target_id), 0 + ), "scanCount": scan_count_by_target[target_id], "targetId": target_id, "targetPath": target["current_path"], diff --git a/sdk/typescript/_bundled_plugin/scripts/workbench_scan_history.py b/sdk/typescript/_bundled_plugin/scripts/workbench_scan_history.py index 323fdbe7..66421233 100644 --- a/sdk/typescript/_bundled_plugin/scripts/workbench_scan_history.py +++ b/sdk/typescript/_bundled_plugin/scripts/workbench_scan_history.py @@ -12,40 +12,66 @@ # Some plugin hosts launch Python with safe-path isolation enabled. sys.path.insert(0, str(Path(__file__).resolve().parent)) +from filesystem_identity import serialize_filesystem_identity from report_projection import SEVERITY_ORDER from workbench_constants import FINDINGS_PAGE_MAX from workbench_scan_usage import stored_scan_cost_fields from workbench_target import git_output +from workbench_target_state import ( + _require_current_target_owner, + repository_identity, + repository_relative_path, +) def _same_repository( before: sqlite3.Row, after: sqlite3.Row, *, - after_identity: tuple[str | None, tuple[str, str] | None] | None = None, + after_identity: tuple[str | None, tuple[str, str] | None, str | None] | None = None, ) -> bool: - if before["target_id"] == after["target_id"]: + before_target_id = before["target_id"] + if before_target_id and before_target_id == after["target_id"]: return True before_target = Path(before["target_path"]) after_target = Path(after["target_path"]) - before_git_dir = git_output( - before_target, "rev-parse", "--path-format=absolute", "--git-common-dir" + if str(before_target.resolve()) == str(after_target.resolve()): + return True + before_stored_identity = ( + before["repository_identity"] if "repository_identity" in before.keys() else None + ) + after_stored_identity = ( + after["repository_identity"] if "repository_identity" in after.keys() else None ) - after_git_dir = ( - git_output(after_target, "rev-parse", "--path-format=absolute", "--git-common-dir") - if after_identity is None + after_grouping_identity = ( + after_stored_identity + if after_stored_identity is not None else after_identity[0] + if after_identity is not None + else None ) - if ( - before_git_dir is not None - and after_git_dir is not None - and Path(before_git_dir).resolve() == Path(after_git_dir).resolve() - ): + if before_stored_identity and before_stored_identity == after_grouping_identity: return True - before_origin = _repository_origin(before_target) - return before_origin is not None and before_origin == ( + + before_live_identity = repository_identity(before_target) + after_live_identity = ( + repository_identity(after_target) if after_identity is None else after_identity[0] + ) + if before_live_identity is not None and before_live_identity == after_live_identity: + return True + if before_live_identity is None or after_live_identity is None: + return False + + after_origin = ( _repository_origin(after_target) if after_identity is None else after_identity[1] ) + if after_origin is None or _repository_origin(before_target) != after_origin: + return False + before_relative = repository_relative_path(before_target) + after_relative = ( + repository_relative_path(after_target) if after_identity is None else after_identity[2] + ) + return before_relative is not None and before_relative == after_relative def _repository_origin(target: Path) -> tuple[str, str] | None: @@ -82,30 +108,114 @@ def list_scans( values: list[Any] = [] if args is not None and args.repository: repository = Path(args.repository).expanduser().resolve() + target_columns = { + row["name"] for row in connection.execute("PRAGMA table_info(security_targets)") + } + identity_column = "repository_identity" in target_columns + requested_identity_column = ( + ", (SELECT repository_identity FROM security_targets " + "WHERE current_path = ?) AS repository_identity" + if identity_column + else "" + ) + requested_values = ( + (str(repository), str(repository), str(repository)) + if identity_column + else (str(repository), str(repository)) + ) requested_repository = connection.execute( - """ + f""" SELECT COALESCE((SELECT id FROM security_targets WHERE current_path = ?), '') AS target_id, ? AS target_path + {requested_identity_column} """, - (str(repository), str(repository)), + requested_values, ).fetchone() + scan_columns = { + row["name"] for row in connection.execute("PRAGMA table_info(scans)") + } + check_ownership = {"target_device", "target_inode"} <= scan_columns + ownership_matches = not check_ownership or _repository_ownership_matches( + connection, + repository, + requested_repository["target_id"], + requested_repository["repository_identity"] if identity_column else None, + ) + live_identity = repository_identity(repository) + identity_matches = ( + not identity_column + or requested_repository["repository_identity"] is None + or requested_repository["repository_identity"] == live_identity + ) + ownership_matches = ownership_matches and identity_matches requested_identity = ( - git_output(repository, "rev-parse", "--path-format=absolute", "--git-common-dir"), - _repository_origin(repository), + ( + requested_repository["repository_identity"] + if ( + identity_column + and ownership_matches + and requested_repository["repository_identity"] is not None + ) + else live_identity + ), + None, + None, + ) + related_targets = ( + [ + target + for target in connection.execute( + "SELECT id AS target_id, current_path AS target_path, repository_identity " + "FROM security_targets" + if identity_column + else "SELECT id AS target_id, current_path AS target_path FROM security_targets" + ) + if _same_repository( + target, + requested_repository, + after_identity=requested_identity, + ) + ] + if ownership_matches + else [] + ) + repository_clauses = ( + [ + _owned_scan_clause( + repository, + check_ownership + and not ( + identity_column + and requested_repository["repository_identity"] is not None + and requested_repository["repository_identity"] == live_identity + and ownership_matches + and repository_relative_path(repository) not in (None, ".") + ), + values, + ) + ] + if identity_matches + else ["0"] ) - related_target_ids = [ - target["target_id"] - for target in connection.execute( - "SELECT id AS target_id, current_path AS target_path FROM security_targets" + for target in related_targets: + target_path = Path(target["target_path"]) + target_identity = target["repository_identity"] if identity_column else None + verified_target = ( + check_ownership + and target_identity is not None + and repository_relative_path(target_path) not in (None, ".") + and _repository_ownership_matches( + connection, target_path, target["target_id"], target_identity + ) + ) + repository_clauses.append( + _owned_scan_clause( + target_path, + check_ownership and not verified_target, + values, + target_id=target["target_id"], + ) ) - if _same_repository(target, requested_repository, after_identity=requested_identity) - ] - repository_clauses = ["scans.target_path = ?"] - values.append(str(repository)) - if related_target_ids: - placeholders = ", ".join("?" for _ in related_target_ids) - repository_clauses.append(f"scans.target_id IN ({placeholders})") - values.extend(related_target_ids) clauses.append(f"({' OR '.join(repository_clauses)})") if args is not None and args.scan_root: scan_root = str(Path(args.scan_root).expanduser().resolve()) @@ -219,6 +329,53 @@ def list_scans( return result +def _owned_scan_clause( + target: Path, + check_ownership: bool, + values: list[Any], + *, + target_id: str | None = None, +) -> str: + if target_id is None: + values.append(str(target)) + target_clause = "scans.target_path = ?" + else: + values.append(target_id) + target_clause = "scans.target_id = ?" + if not check_ownership: + return target_clause + try: + metadata = target.stat() + except OSError: + return target_clause + values.extend( + ( + serialize_filesystem_identity(metadata.st_dev), + serialize_filesystem_identity(metadata.st_ino), + ) + ) + return ( + f"({target_clause} AND (" + "(scans.target_device IS NULL AND scans.target_inode IS NULL) " + "OR (scans.target_device = ? AND scans.target_inode = ?)" + "))" + ) + + +def _repository_ownership_matches( + connection: sqlite3.Connection, + repository: Path, + target_id: str | None, + stored_identity: str | None = None, +) -> bool: + try: + repository.stat() + _require_current_target_owner(connection, target_id or "", str(repository), stored_identity) + except (OSError, SystemExit): + return False + return True + + def list_unmatched_scan_pairs( connection: sqlite3.Connection, args: argparse.Namespace, @@ -227,19 +384,63 @@ def list_unmatched_scan_pairs( read_coverage: Callable[[sqlite3.Row], dict[str, Any]], ) -> dict[str, Any]: repository = Path(args.repository).expanduser().resolve() + identity_column = any( + row["name"] == "repository_identity" + for row in connection.execute("PRAGMA table_info(security_targets)") + ) + requested_identity_column = ( + ", (SELECT repository_identity FROM security_targets " + "WHERE current_path = ?) AS repository_identity" + if identity_column + else "" + ) + requested_values = ( + (str(repository), str(repository), str(repository)) + if identity_column + else (str(repository), str(repository)) + ) requested = connection.execute( - """ + f""" SELECT COALESCE((SELECT id FROM security_targets WHERE current_path = ?), '') AS target_id, ? AS target_path + {requested_identity_column} """, - (str(repository), str(repository)), + requested_values, ).fetchone() + if identity_column and requested["target_id"]: + _require_current_target_owner( + connection, + requested["target_id"], + str(repository), + requested["repository_identity"], + ) + scan_query = ( + "SELECT scans.*, targets.repository_identity " + "FROM scans LEFT JOIN security_targets AS targets ON targets.id = scans.target_id " + "WHERE scans.status = 'complete' ORDER BY scans.started_at, scans.id" + if identity_column + else "SELECT * FROM scans WHERE status = 'complete' ORDER BY started_at, id" + ) + live_identity = repository_identity(repository) + requested_identity = ( + requested["repository_identity"] + if identity_column and requested["repository_identity"] is not None + else live_identity + ) + requested_group = ( + requested_identity, + _repository_origin(repository), + repository_relative_path(repository), + ) selected = [ scan - for scan in connection.execute( - "SELECT * FROM scans WHERE status = 'complete' ORDER BY started_at, id" + for scan in connection.execute(scan_query) + if str(Path(scan["target_path"]).resolve()) == str(repository) + or _same_repository( + scan, + requested, + after_identity=requested_group, ) - if Path(scan["target_path"]).resolve() == repository or _same_repository(scan, requested) ] available = [] @@ -305,8 +506,12 @@ def compare_scans( include_matching_inputs: bool = False, require_matches: bool = False, ) -> dict[str, Any]: - before = require_scan(connection, args.before_scan_id) - after = require_scan(connection, args.after_scan_id) + before = _scan_with_repository_identity( + connection, require_scan(connection, args.before_scan_id) + ) + after = _scan_with_repository_identity( + connection, require_scan(connection, args.after_scan_id) + ) if before["id"] == after["id"]: raise SystemExit("Select two different scans to compare.") if before["status"] != "complete" or after["status"] != "complete": @@ -445,8 +650,12 @@ def save_scan_comparison( require_scan: Callable[[sqlite3.Connection, str], sqlite3.Row], read_coverage: Callable[[sqlite3.Row], dict[str, Any]], ) -> dict[str, Any]: - before = require_scan(connection, args.before_scan_id) - after = require_scan(connection, args.after_scan_id) + before = _scan_with_repository_identity( + connection, require_scan(connection, args.before_scan_id) + ) + after = _scan_with_repository_identity( + connection, require_scan(connection, args.after_scan_id) + ) if before["id"] == after["id"]: raise SystemExit("Select two different scans to compare.") if before["status"] != "complete" or after["status"] != "complete": @@ -528,6 +737,28 @@ def save_scan_comparison( return compare_scans(connection, args, require_scan=require_scan, read_coverage=read_coverage) +def _scan_with_repository_identity( + connection: sqlite3.Connection, scan: sqlite3.Row +) -> sqlite3.Row: + if "repository_identity" in scan.keys(): + return scan + if not any( + row["name"] == "repository_identity" + for row in connection.execute("PRAGMA table_info(security_targets)") + ): + return scan + enriched = connection.execute( + """ + SELECT scans.*, targets.repository_identity + FROM scans + LEFT JOIN security_targets AS targets ON targets.id = scans.target_id + WHERE scans.id = ? + """, + (scan["id"],), + ).fetchone() + return enriched if enriched is not None else scan + + def finding_matches( connection: sqlite3.Connection, occurrence_id: str, scan_id: str, started_at: str ) -> tuple[list[dict[str, Any]], str, list[str]]: diff --git a/sdk/typescript/_bundled_plugin/scripts/workbench_schema.py b/sdk/typescript/_bundled_plugin/scripts/workbench_schema.py index 93b8f297..50074a0b 100644 --- a/sdk/typescript/_bundled_plugin/scripts/workbench_schema.py +++ b/sdk/typescript/_bundled_plugin/scripts/workbench_schema.py @@ -643,6 +643,17 @@ ADD COLUMN max_time_hours REAL NOT NULL DEFAULT 96; """, ), + ( + 30, + "persist repository identities", + """ + ALTER TABLE security_targets + ADD COLUMN repository_identity TEXT; + + CREATE INDEX security_targets_by_repository_identity + ON security_targets(repository_identity); + """, + ), ) @@ -704,11 +715,19 @@ def apply_migrations( "max_time_hours", "REAL NOT NULL DEFAULT 96", ) + elif version == 30: + should_backfill_targets = ( + repair_repository_identity_migration(connection) + or should_backfill_targets + ) continue if version == 6: repair_thread_scoped_workspaces_migration(connection) elif version == 16: should_backfill_targets = repair_stable_targets_migration(connection) + elif version == 30: + repair_repository_identity_migration(connection) + should_backfill_targets = True else: for statement in sql_statements(sql): connection.execute(statement) @@ -1154,38 +1173,85 @@ def repair_stable_targets_migration(connection: sqlite3.Connection) -> bool: and "target_id" in scan_columns and existing_objects == {"security_targets", "scans_by_target"} ): + needs_backfill = bool( + connection.execute( + """ + SELECT EXISTS ( + SELECT 1 FROM workspaces AS workspace + WHERE (workspace.target_id IS NULL AND workspace.target_path IS NOT NULL) + OR ( + workspace.target_id IS NOT NULL + AND NOT EXISTS ( + SELECT 1 FROM security_targets + WHERE security_targets.id = workspace.target_id + ) + ) + ) OR EXISTS ( + SELECT 1 FROM scans AS scan + WHERE scan.target_id IS NULL + OR NOT EXISTS ( + SELECT 1 FROM security_targets + WHERE security_targets.id = scan.target_id + ) + ) + """ + ).fetchone()[0] + ) + if not needs_backfill: + return False + else: + migration_sql = next(sql for version, _, sql in MIGRATIONS if version == 16) + for statement in sql_statements(migration_sql): + if statement.startswith("ALTER TABLE workspaces"): + add_column_if_missing( + connection, + "workspaces", + "target_id", + "TEXT REFERENCES security_targets(id)", + ) + continue + if statement.startswith("ALTER TABLE scans"): + add_column_if_missing( + connection, + "scans", + "target_id", + "TEXT REFERENCES security_targets(id)", + ) + continue + statement = statement.replace("CREATE TABLE ", "CREATE TABLE IF NOT EXISTS ", 1) + statement = statement.replace("CREATE INDEX ", "CREATE INDEX IF NOT EXISTS ", 1) + connection.execute(statement) + + for table in ("workspaces", "scans"): + connection.execute( + f""" + UPDATE {table} + SET target_id = NULL + WHERE target_id IS NOT NULL + AND NOT EXISTS ( + SELECT 1 FROM security_targets + WHERE security_targets.id = {table}.target_id + ) + """ + ) + return True + + +def repair_repository_identity_migration(connection: sqlite3.Connection) -> bool: + columns = { + row["name"] for row in connection.execute("PRAGMA table_info(security_targets)") + } + index_exists = connection.execute( + "SELECT 1 FROM sqlite_master WHERE type = 'index' " + "AND name = 'security_targets_by_repository_identity'" + ).fetchone() + if "repository_identity" in columns and index_exists is not None: return False - migration_sql = next(sql for version, _, sql in MIGRATIONS if version == 16) - for statement in sql_statements(migration_sql): - if statement.startswith("ALTER TABLE workspaces"): - add_column_if_missing( - connection, - "workspaces", - "target_id", - "TEXT REFERENCES security_targets(id)", - ) - continue - if statement.startswith("ALTER TABLE scans"): - add_column_if_missing( - connection, - "scans", - "target_id", - "TEXT REFERENCES security_targets(id)", - ) - continue - statement = statement.replace("CREATE TABLE ", "CREATE TABLE IF NOT EXISTS ", 1) - statement = statement.replace("CREATE INDEX ", "CREATE INDEX IF NOT EXISTS ", 1) - connection.execute(statement) + add_column_if_missing(connection, "security_targets", "repository_identity", "TEXT") connection.execute( - """ - UPDATE scans - SET target_id = NULL - WHERE target_id IS NOT NULL - AND NOT EXISTS ( - SELECT 1 FROM security_targets WHERE security_targets.id = scans.target_id - ) - """ + "CREATE INDEX IF NOT EXISTS security_targets_by_repository_identity " + "ON security_targets(repository_identity)" ) return True diff --git a/sdk/typescript/_bundled_plugin/scripts/workbench_target_state.py b/sdk/typescript/_bundled_plugin/scripts/workbench_target_state.py index 47d2ea6d..6f1fd81c 100644 --- a/sdk/typescript/_bundled_plugin/scripts/workbench_target_state.py +++ b/sdk/typescript/_bundled_plugin/scripts/workbench_target_state.py @@ -4,16 +4,250 @@ import argparse import hashlib +import os import sqlite3 +import subprocess +import sys from datetime import datetime, timezone from pathlib import Path +# Some plugin hosts launch Python with safe-path isolation enabled. +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from filesystem_identity import ( + serialize_filesystem_identity, + stored_filesystem_identity_matches, +) +from workbench_target import git_output + def stable_target_id(target: Path) -> str: digest = hashlib.sha256(f"local-workspace\0{target}".encode()).hexdigest() return f"target_sha256_{digest}" +def repository_relative_path(target: Path) -> str | None: + """Return the target's normalized location within its own Git worktree.""" + worktree = _repository_worktree(target) + return worktree[1] if worktree is not None else None + + +def _repository_worktree(target: Path) -> tuple[Path, str] | None: + worktree_root = git_output(target, "rev-parse", "--show-toplevel") + if worktree_root is None: + return None + try: + canonical_root = Path(os.path.realpath(worktree_root)) + relative = Path(os.path.realpath(target)).relative_to(canonical_root) + except (OSError, ValueError): + return None + return canonical_root, relative.as_posix() + + +def _repository_birth_time_ns(path: str, metadata: os.stat_result) -> int | None: + birth_time_ns = getattr(metadata, "st_birthtime_ns", None) + if birth_time_ns is not None: + return birth_time_ns if birth_time_ns > 0 else None + if os.name == "nt": + return metadata.st_ctime_ns if metadata.st_ctime_ns > 0 else None + birth_time = getattr(metadata, "st_birthtime", None) + if birth_time is not None: + return int(birth_time * 1_000_000_000) if birth_time > 0 else None + if not sys.platform.startswith("linux"): + return None + try: + result = subprocess.run( + ["stat", "--format=%.9W", "--", path], + check=False, + capture_output=True, + text=True, + env={**os.environ, "LC_ALL": "C"}, + ) + except OSError: + return None + seconds, separator, nanoseconds = result.stdout.strip().partition(".") + if ( + result.returncode != 0 + or separator != "." + or not seconds.isdecimal() + or len(nanoseconds) != 9 + or not nanoseconds.isdecimal() + ): + return None + birth_time_ns = int(seconds) * 1_000_000_000 + int(nanoseconds) + return birth_time_ns if birth_time_ns > 0 else None + + +def repository_identity(target: Path | str) -> str | None: + """Identify matching Git worktree targets without storing remote credentials.""" + target = Path(target) + common_directory = git_output( + target, "rev-parse", "--path-format=absolute", "--git-common-dir" + ) + if common_directory is None: + return None + worktree = _repository_worktree(target) + if worktree is None: + return None + worktree_root, relative = worktree + registered = git_output(target, "worktree", "list", "--porcelain", "-z") + if registered is None: + return None + canonical_root = os.fspath(worktree_root) + if not any( + os.path.realpath(record.removeprefix("worktree ")) == canonical_root + for record in registered.split("\0") + if record.startswith("worktree ") + ): + return None + canonical_directory = os.path.realpath(common_directory) + try: + metadata = Path(canonical_directory).stat() + except OSError: + return None + birth_time_ns = _repository_birth_time_ns(canonical_directory, metadata) + if birth_time_ns is None: + return None + device = serialize_filesystem_identity(metadata.st_dev) + inode = serialize_filesystem_identity(metadata.st_ino) + material = ( + f"git-common-dir\0{canonical_directory}\0{device}\0{inode}\0" + f"{birth_time_ns}\0{relative}" + ) + return f"repository_sha256_{hashlib.sha256(material.encode()).hexdigest()}" + + +def _supports_repository_identity(connection: sqlite3.Connection) -> bool: + return any( + row["name"] == "repository_identity" + for row in connection.execute("PRAGMA table_info(security_targets)") + ) + + +def _verified_repository_identity( + connection: sqlite3.Connection, target_id: str, target_path: str +) -> str | None: + target = Path(target_path) + try: + metadata = target.stat() + except OSError: + return None + + scan_columns = { + row["name"] for row in connection.execute("PRAGMA table_info(scans)") + } + if not {"target_id", "target_path"} <= scan_columns: + return None + if not {"target_device", "target_inode"} <= scan_columns: + historical_scan = connection.execute( + "SELECT 1 FROM scans WHERE target_id = ? OR target_path = ? LIMIT 1", + (target_id, target_path), + ).fetchone() + return None if historical_scan is not None else repository_identity(target) + + scans = connection.execute( + """ + SELECT target_device, target_inode + FROM scans + WHERE target_id = ? OR target_path = ? + """, + (target_id, target_path), + ) + if any( + not stored_filesystem_identity_matches(scan["target_device"], metadata.st_dev) + or not stored_filesystem_identity_matches(scan["target_inode"], metadata.st_ino) + for scan in scans + ): + return None + return repository_identity(target) + + +def _require_current_target_owner( + connection: sqlite3.Connection, + target_id: str, + target_path: str, + stored_identity: str | None, +) -> None: + target = Path(target_path) + try: + metadata = target.stat() + except OSError: + return + scan_columns = { + row["name"] for row in connection.execute("PRAGMA table_info(scans)") + } + if not {"target_id", "target_path", "target_device", "target_inode"} <= scan_columns: + return + scans = connection.execute( + """ + SELECT target_device, target_inode + FROM scans + WHERE target_id = ? OR target_path = ? + """, + (target_id, target_path), + ) + historical_scan = False + recorded_owner = False + malformed_owner = False + mismatch = False + for scan in scans: + historical_scan = True + device, inode = scan["target_device"], scan["target_inode"] + if device is None and inode is None: + continue + if device is None or inode is None: + malformed_owner = True + break + recorded_owner = True + if not stored_filesystem_identity_matches( + device, metadata.st_dev + ) or not stored_filesystem_identity_matches(inode, metadata.st_ino): + mismatch = True + verified_repository = ( + stored_identity is not None and repository_identity(target) == stored_identity + ) + if ( + malformed_owner + or mismatch + and ( + not verified_repository + or repository_relative_path(target) in (None, ".") + ) + or stored_identity is not None + and ( + not verified_repository or historical_scan and not recorded_owner + ) + ): + raise SystemExit( + f"The repository checkout at {target_path} no longer matches its recorded " + "security scan history; refusing to reuse its target." + ) + + +def backfill_repository_identities(connection: sqlite3.Connection) -> None: + if not _supports_repository_identity(connection): + return + targets = connection.execute( + """ + SELECT id, current_path + FROM security_targets + WHERE repository_identity IS NULL + """ + ).fetchall() + for target in targets: + identity = _verified_repository_identity( + connection, str(target["id"]), target["current_path"] + ) + if identity is not None: + connection.execute( + """ + UPDATE security_targets + SET repository_identity = ? + WHERE id = ? AND repository_identity IS NULL + """, + (identity, target["id"]), + ) + + def backfill_security_targets(connection: sqlite3.Connection) -> None: rows = connection.execute( """ @@ -24,7 +258,7 @@ def backfill_security_targets(connection: sqlite3.Connection) -> None: ).fetchall() for row in rows: target_path = row["target_path"] - target_id = ensure_security_target(connection, target_path) + target_id = ensure_security_target(connection, target_path, verify_ownership=False) connection.execute( "UPDATE workspaces SET target_id = ? WHERE target_path = ? AND target_id IS NULL", (target_id, target_path), @@ -33,25 +267,58 @@ def backfill_security_targets(connection: sqlite3.Connection) -> None: "UPDATE scans SET target_id = ? WHERE target_path = ? AND target_id IS NULL", (target_id, target_path), ) + backfill_repository_identities(connection) -def ensure_security_target(connection: sqlite3.Connection, target_path: str) -> str: +def ensure_security_target( + connection: sqlite3.Connection, target_path: str, *, verify_ownership: bool = True +) -> str: + supports_identity = _supports_repository_identity(connection) existing = connection.execute( - "SELECT id FROM security_targets WHERE current_path = ?", + "SELECT id, repository_identity FROM security_targets WHERE current_path = ?" + if supports_identity + else "SELECT id FROM security_targets WHERE current_path = ?", (target_path,), ).fetchone() if existing is not None: - return str(existing["id"]) + target_id = str(existing["id"]) + if verify_ownership and supports_identity: + _require_current_target_owner( + connection, target_id, target_path, existing["repository_identity"] + ) + if supports_identity and existing["repository_identity"] is None: + identity = _verified_repository_identity(connection, target_id, target_path) + if identity is not None: + connection.execute( + """ + UPDATE security_targets + SET repository_identity = ? + WHERE id = ? AND repository_identity IS NULL + """, + (identity, target_id), + ) + return target_id target_id = stable_target_id(Path(target_path)) timestamp = datetime.now(timezone.utc).isoformat().replace("+00:00", "Z") - connection.execute( - """ - INSERT OR IGNORE INTO security_targets ( - id, current_path, display_name, created_at, updated_at - ) VALUES (?, ?, ?, ?, ?) - """, - (target_id, target_path, Path(target_path).name, timestamp, timestamp), - ) + if supports_identity: + identity = _verified_repository_identity(connection, target_id, target_path) + connection.execute( + """ + INSERT OR IGNORE INTO security_targets ( + id, current_path, display_name, created_at, updated_at, repository_identity + ) VALUES (?, ?, ?, ?, ?, ?) + """, + (target_id, target_path, Path(target_path).name, timestamp, timestamp, identity), + ) + else: + connection.execute( + """ + INSERT OR IGNORE INTO security_targets ( + id, current_path, display_name, created_at, updated_at + ) VALUES (?, ?, ?, ?, ?) + """, + (target_id, target_path, Path(target_path).name, timestamp, timestamp), + ) return target_id diff --git a/sdk/typescript/src/api.ts b/sdk/typescript/src/api.ts index 3fde8e86..92d99361 100644 --- a/sdk/typescript/src/api.ts +++ b/sdk/typescript/src/api.ts @@ -1286,9 +1286,7 @@ export class CodexSecurity { scanId, repository: repo, previousFindings: previousFindings.filter( - (finding) => - finding["scanId"] !== scanId && - finding["targetId"] === targetId, + (finding) => finding["scanId"] !== scanId, ), falsePositives: falsePositiveExamples as Record[], findings: result.findings.findings, diff --git a/sdk/typescript/src/cli.ts b/sdk/typescript/src/cli.ts index 7d5a77c6..917b6df5 100644 --- a/sdk/typescript/src/cli.ts +++ b/sdk/typescript/src/cli.ts @@ -919,11 +919,12 @@ export async function main( ); return presentHistory( await history( - ["list-repositories"], + ["list-scans", "--repository", repository], async (value): Promise => { - const target = (value["repositories"] as JsonObject[]).find( - (entry) => entry["targetPath"] === repository, - ); + const scans = value["scans"] as JsonObject[]; + const target = + scans.find((scan) => scan["targetPath"] === repository) ?? + scans[0]; const findings = target === undefined ? [] diff --git a/sdk/typescript/tests-ts/cli-workbench.test.ts b/sdk/typescript/tests-ts/cli-workbench.test.ts index 9766a261..2e2ea953 100644 --- a/sdk/typescript/tests-ts/cli-workbench.test.ts +++ b/sdk/typescript/tests-ts/cli-workbench.test.ts @@ -18,12 +18,7 @@ describe("CLI workbench", () => { const stdout = capture(); const calls: Array = []; const responses: JsonObject[] = [ - { - repositories: [ - { targetId: "other", targetPath: `${repository}-clone` }, - { targetId: "selected", targetPath: repository }, - ], - }, + { scans: [{ targetId: "selected", targetPath: repository }] }, { findings: [{ title: "Finding 1" }], nextOffset: 1 }, { findings: [{ title: "Finding 2" }], nextOffset: null }, ]; @@ -37,7 +32,7 @@ describe("CLI workbench", () => { }), ), ).toBe(0); - expect(calls[0]).toEqual(["list-repositories"]); + expect(calls[0]).toEqual(["list-scans", "--repository", repository]); expect(calls[1]).toEqual([ "list-global-findings", "--target-id", @@ -75,6 +70,77 @@ describe("CLI workbench", () => { } }); + test.each([ + { requestedScan: false, selectedTarget: "linked" }, + { requestedScan: true, selectedTarget: "requested" }, + ])( + "lists findings through one trusted target when requested scan exists: $requestedScan", + async ({ requestedScan, selectedTarget }) => { + const repository = resolve("/current/repository"); + const linked = resolve("/current/repository-worktree"); + const findings: JsonObject[] = [ + { occurrenceId: "requested-finding", status: "open" }, + { occurrenceId: "linked-finding", status: "open" }, + ]; + const calls: Array = []; + const stdout = capture(); + expect( + await main( + ["findings", "list", "--json"], + stdout.stream, + capture().stream, + dependencies({ + onWorkbench: (args): JsonObject => { + calls.push(args); + return args[0] === "list-scans" + ? { + scans: [ + { targetId: "linked", targetPath: linked }, + ...(requestedScan + ? [{ targetId: "requested", targetPath: repository }] + : []), + ], + } + : { findings, nextOffset: null }; + }, + }), + ), + ).toBe(0); + expect(calls).toEqual([ + ["list-scans", "--repository", repository], + [ + "list-global-findings", + "--target-id", + selectedTarget, + "--status", + "open", + ], + ]); + expect(JSON.parse(stdout.text())).toEqual({ repository, findings }); + }, + ); + + test("returns no findings when no related repository has been scanned", async () => { + const calls: Array = []; + const stdout = capture(); + const repository = resolve("/current/repository"); + expect( + await main( + ["findings", "list", "--json"], + stdout.stream, + capture().stream, + dependencies({ + onWorkbench: (args) => { + calls.push(args); + return { scans: [] }; + }, + }), + ), + ).toBe(0); + expect(calls).toEqual([["list-scans", "--repository", repository]]); + expect(JSON.parse(stdout.text())).toEqual({ repository, findings: [] }); + }); + test("lists repository and scan-root history without starting Codex", async () => { const repository = resolve("/current/repository"); const cases: Array<[string[], string[]]> = [ diff --git a/sdk/typescript/tests-ts/repository-feedback-aliases.test.ts b/sdk/typescript/tests-ts/repository-feedback-aliases.test.ts new file mode 100644 index 00000000..35709a5a --- /dev/null +++ b/sdk/typescript/tests-ts/repository-feedback-aliases.test.ts @@ -0,0 +1,206 @@ +import { spawnSync } from "node:child_process"; +import { join } from "node:path"; +import { expect, test } from "bun:test"; +import { PLUGIN_ROOT } from "./plugin-root.js"; + +test("reuses reviewed feedback only across matching persisted repository identities", () => { + const python = (Bun.which("python3") ?? Bun.which("python"))!; + const probe = ` +import json, os, sqlite3, sys, tempfile +sys.path.insert(0, sys.argv[1]) +from filesystem_identity import serialize_filesystem_identity +from workbench_feedback import get_scan_feedback + +scenario = sys.argv[2] +connection = sqlite3.connect(":memory:") +connection.row_factory = sqlite3.Row +connection.executescript(""" +CREATE TABLE security_targets(id TEXT, current_path TEXT, display_name TEXT, origin TEXT); +CREATE TABLE scans(id TEXT, target_id TEXT, status TEXT, completed_at TEXT, started_at TEXT, updated_at TEXT, scope TEXT); +CREATE TABLE findings(id TEXT PRIMARY KEY, fingerprint TEXT, rule_id TEXT, identity_anchor TEXT, identity_instance TEXT); +CREATE TABLE finding_occurrences(id TEXT, finding_id TEXT, scan_id TEXT, title TEXT, summary TEXT, severity TEXT, created_at TEXT); +CREATE TABLE finding_triage(occurrence_id TEXT, status TEXT, close_reason TEXT, note TEXT, updated_at TEXT); +CREATE TABLE finding_locations(id INTEGER PRIMARY KEY, occurrence_id TEXT, relative_path TEXT, start_line INTEGER, end_line INTEGER, role TEXT, sort_order INTEGER); +CREATE TABLE scan_comparison_matches(before_occurrence_id TEXT, after_occurrence_id TEXT); +""") +if scenario == "identities": + connection.execute("ALTER TABLE security_targets ADD COLUMN repository_identity TEXT") + +origin = "https://example.invalid/synthetic/repository" +for target, identity in [ + ("primary", "common-git-directory::."), + ("linked", "common-git-directory::."), + ("same-origin-clone", "independent-git-directory::."), + ("different-scope", "common-git-directory::packages/api"), + ("unknown-first", None), + ("unknown-second", None), +]: + connection.execute( + "INSERT INTO security_targets(id, current_path, display_name, origin) VALUES (?, ?, ?, ?)", + (target, f"/{target}", target, origin), + ) + if scenario == "identities": + connection.execute( + "UPDATE security_targets SET repository_identity = ? WHERE id = ?", + (identity, target), + ) + +def add_scan(scan_id, target, day, status="complete"): + timestamp = f"2026-03-{day:02d}T00:00:00Z" + completed_at = timestamp if status == "complete" else None + connection.execute("INSERT INTO scans VALUES (?, ?, ?, ?, ?, ?, ?)", (scan_id, target, status, completed_at, timestamp, timestamp, "repository")) + +def add_finding(scan_id, finding_id, day, *, status="closed", reason="false_positive", note="Reviewed and safe"): + occurrence_id = f"{scan_id}:{finding_id}" + connection.execute( + "INSERT OR IGNORE INTO findings VALUES (?, ?, ?, ?, ?)", + (finding_id, f"fingerprint-{finding_id}", "synthetic-rule", f"anchor-{finding_id}", None), + ) + connection.execute( + "INSERT INTO finding_occurrences VALUES (?, ?, ?, ?, ?, ?, ?)", + (occurrence_id, finding_id, scan_id, finding_id, "Synthetic summary", "high", f"2026-03-{day:02d}T00:00:00Z"), + ) + connection.execute( + "INSERT INTO finding_locations(occurrence_id, relative_path, start_line, end_line, role, sort_order) VALUES (?, ?, ?, ?, ?, ?)", + (occurrence_id, "src/auth.py", 3, 4, "root_control", 0), + ) + connection.execute( + "INSERT INTO finding_triage VALUES (?, ?, ?, ?, ?)", + (occurrence_id, status, reason, note, f"2026-03-{day:02d}T12:00:00Z"), + ) + +for scan_id, target, day in [ + ("primary-reviewed", "primary", 1), + ("linked-reviewed", "linked", 2), + ("linked-reopened", "linked", 3), + ("clone-reviewed", "same-origin-clone", 4), + ("scope-reviewed", "different-scope", 5), + ("unknown-first-reviewed", "unknown-first", 6), + ("unknown-second-reviewed", "unknown-second", 7), +]: + add_scan(scan_id, target, day) + +add_finding("primary-reviewed", "primary-false-positive", 1) +add_finding("primary-reviewed", "reopened-across-alias", 1) +add_finding("primary-reviewed", "renamed-before-reopening", 1) +add_finding("linked-reviewed", "linked-false-positive", 2) +add_finding("linked-reviewed", "linked-wont-fix", 2, reason="wont_fix") +add_finding("linked-reviewed", "linked-no-note", 2, note=" ") +add_finding("linked-reopened", "reopened-across-alias", 3, status="open", reason=None, note=None) +add_finding("linked-reopened", "renamed-after-reopening", 3, status="open", reason=None, note=None) +connection.execute( + "INSERT INTO scan_comparison_matches VALUES (?, ?)", + ("primary-reviewed:renamed-before-reopening", "linked-reopened:renamed-after-reopening"), +) +add_finding("clone-reviewed", "clone-false-positive", 4) +add_finding("scope-reviewed", "scope-false-positive", 5) +add_finding("unknown-first-reviewed", "unknown-first-false-positive", 6) +add_finding("unknown-second-reviewed", "unknown-second-false-positive", 7) +add_scan("linked-incomplete", "linked", 8, status="running") +add_finding("linked-incomplete", "linked-incomplete-false-positive", 8) + +for target in ["primary", "linked", "same-origin-clone", "different-scope", "unknown-first", "unknown-second"]: + add_scan(f"current-{target}", target, 9, status="running") + +def feedback(target): + scan = connection.execute("SELECT * FROM scans WHERE id = ?", (f"current-{target}",)).fetchone() + return get_scan_feedback(connection, scan) + +result = { + "primary": feedback("primary"), + "linked": feedback("linked"), + "clone": feedback("same-origin-clone"), + "differentScope": feedback("different-scope"), + "unknownFirst": feedback("unknown-first"), + "unknownSecond": feedback("unknown-second"), +} +if scenario == "identities": + connection.execute("ALTER TABLE scans ADD COLUMN target_path TEXT") + connection.execute("ALTER TABLE scans ADD COLUMN target_device INTEGER") + connection.execute("ALTER TABLE scans ADD COLUMN target_inode INTEGER") + with tempfile.TemporaryDirectory() as reused_path: + metadata = os.stat(reused_path) + connection.execute( + "UPDATE security_targets SET current_path = ? WHERE id = ?", + (reused_path, "primary"), + ) + connection.execute( + "UPDATE scans SET target_path = ?, target_device = ?, target_inode = ? WHERE target_id = ?", + ( + reused_path, + serialize_filesystem_identity(metadata.st_dev), + serialize_filesystem_identity(metadata.st_ino + 1), + "primary", + ), + ) + result["reusedPath"] = feedback("primary") + result["deletedAlias"] = feedback("linked") +print(json.dumps(result)) +`; + + const run = (scenario: "identities" | "legacy") => { + const execution = spawnSync( + python, + ["-I", "-B", "-c", probe, join(PLUGIN_ROOT, "scripts"), scenario], + { encoding: "utf8", timeout: 10_000 }, + ); + expect(execution.status, execution.stderr).toBe(0); + return JSON.parse(execution.stdout) as Record< + string, + { + scanId: string; + targetId: string; + falsePositives: Array>; + } + >; + }; + + const identities = run("identities"); + expect(identities["primary"]).toMatchObject({ + scanId: "current-primary", + targetId: "primary", + falsePositives: [ + { + findingId: "linked-false-positive", + sourceScanId: "linked-reviewed", + reason: "Reviewed and safe", + }, + { + findingId: "primary-false-positive", + sourceScanId: "primary-reviewed", + }, + ], + }); + expect(identities["linked"]?.falsePositives).toEqual( + identities["primary"]?.falsePositives, + ); + expect(identities["clone"]?.falsePositives).toMatchObject([ + { findingId: "clone-false-positive" }, + ]); + expect(identities["differentScope"]?.falsePositives).toMatchObject([ + { findingId: "scope-false-positive" }, + ]); + expect(identities["unknownFirst"]?.falsePositives).toMatchObject([ + { findingId: "unknown-first-false-positive" }, + ]); + expect(identities["unknownSecond"]?.falsePositives).toMatchObject([ + { findingId: "unknown-second-false-positive" }, + ]); + expect(identities["reusedPath"]?.falsePositives).toEqual([]); + expect(identities["deletedAlias"]?.falsePositives).toEqual( + identities["linked"]?.falsePositives, + ); + + const legacy = run("legacy"); + expect(legacy["primary"]?.falsePositives).toMatchObject([ + { findingId: "reopened-across-alias", sourceScanId: "primary-reviewed" }, + { + findingId: "renamed-before-reopening", + sourceScanId: "primary-reviewed", + }, + { findingId: "primary-false-positive", sourceScanId: "primary-reviewed" }, + ]); + expect(legacy["linked"]?.falsePositives).toMatchObject([ + { findingId: "linked-false-positive", sourceScanId: "linked-reviewed" }, + ]); +}); diff --git a/sdk/typescript/tests-ts/repository-findings-worktrees.test.ts b/sdk/typescript/tests-ts/repository-findings-worktrees.test.ts new file mode 100644 index 00000000..f231e3aa --- /dev/null +++ b/sdk/typescript/tests-ts/repository-findings-worktrees.test.ts @@ -0,0 +1,177 @@ +import { mkdir } from "node:fs/promises"; +import { join } from "node:path"; +import { afterEach, describe, expect, test } from "bun:test"; +import { CodexSecurity } from "../src/index.js"; +import { + completedEvents, + createApiTestFixtures, + preparedRuntime, +} from "./support/api-events.js"; + +const fixtures = createApiTestFixtures(); +const TestClient = CodexSecurity as unknown as new ( + config: Record, + dependencies: Record, +) => CodexSecurity; + +afterEach(fixtures.cleanup); + +describe("repository findings across linked worktrees", () => { + test("matches only repository-identity-scoped history from another worktree", async () => { + const root = await fixtures.temporaryDirectory(); + const repository = join(root, "repository"); + const codexHome = join(root, "codex-home"); + const scanDir = join(root, "scan"); + await mkdir(repository); + await mkdir(codexHome); + await mkdir(scanDir, { mode: 0o700 }); + + const targetId = "target_sha256_example"; + const linkedFinding = { + findingId: "linked_worktree_finding", + occurrenceId: "linked_worktree_occurrence", + scanId: "linked_worktree_scan", + targetId: "target_linked_worktree", + }; + const unrelatedFinding = { + findingId: "independent_clone_finding", + occurrenceId: "independent_clone_occurrence", + scanId: "independent_clone_scan", + targetId: "target_independent_clone", + }; + const currentFinding = { + findingId: "csf_852f90d6e1177502ff113d4a", + occurrenceId: "occ_e79cb19591e696572a1c22be", + scanId: "scan_example_001", + targetId, + }; + const mergedFinding = { + ...linkedFinding, + title: "Unsafe archive extraction", + summary: "Archive entries can escape their destination.", + severity: { level: "high" as const }, + status: "open" as const, + confirmedInLatestScan: true, + knownScanIds: [linkedFinding.scanId, currentFinding.scanId], + }; + const commands: Array = []; + const matchedInputs: Array<{ + before: readonly Record[]; + after: readonly Record[]; + }> = []; + + const client = new TestClient( + {}, + { + environment: {}, + prepareRuntime: async () => preparedRuntime(codexHome), + resolvePluginPython: async () => "/managed/python", + prepareOutputDir: async () => scanDir, + repositoryRevision: async () => "deadbeef", + runWorkbench: async (_options: unknown, args: readonly string[]) => { + commands.push(args); + if (args[0] === "register-cli-scan") { + return { + scanId: currentFinding.scanId, + targetId, + targetRevision: "deadbeef", + scanDir: args[args.indexOf("--scan-dir") + 1], + contract: { target: { allowedKinds: ["git_revision"] } }, + }; + } + if (args[0] === "get-scan-feedback") { + return { + scanId: currentFinding.scanId, + targetId, + falsePositives: [], + }; + } + if (args[0] === "list-global-findings") { + expect(args).toContain("--target-id"); + expect(args).toContain(targetId); + return { + findings: args.includes("--status") + ? [mergedFinding] + : [linkedFinding, currentFinding], + }; + } + if (args[0] === "list-unmatched-scan-pairs") { + return { + batches: [ + { + afterScanId: currentFinding.scanId, + afterFindings: [currentFinding], + beforeScans: [ + { + scanId: linkedFinding.scanId, + findings: [linkedFinding], + }, + { + scanId: unrelatedFinding.scanId, + findings: [unrelatedFinding], + }, + ], + }, + ], + }; + } + return {}; + }, + async matchFindings(input: (typeof matchedInputs)[number]) { + matchedInputs.push(input); + return { + matches: [ + { + beforeOccurrenceIds: [linkedFinding.occurrenceId], + afterOccurrenceIds: [currentFinding.occurrenceId], + confidence: "high", + reason: "The linked worktree has the same root cause.", + }, + ], + uncertain: [], + }; + }, + createCodex: () => ({ + startThread: () => ({ + id: null, + async runStreamed() { + await fixtures.copyCompletedScan(root); + return { events: completedEvents() }; + }, + }), + }), + }, + ); + + const result = await client.run(repository); + + expect(matchedInputs).toEqual([ + { before: [linkedFinding], after: [currentFinding] }, + ]); + expect( + commands.filter(([command]) => command === "save-scan-comparison"), + ).toEqual([ + [ + "save-scan-comparison", + "--before-scan-id", + linkedFinding.scanId, + "--after-scan-id", + currentFinding.scanId, + "--matches-json", + JSON.stringify({ + matches: [ + { + beforeOccurrenceIds: [linkedFinding.occurrenceId], + afterOccurrenceIds: [currentFinding.occurrenceId], + confidence: "high", + reason: "The linked worktree has the same root cause.", + }, + ], + uncertain: [], + }), + ], + ]); + expect(result.repositoryFindings).toEqual([mergedFinding]); + await client.close(); + }); +}); diff --git a/sdk/typescript/tests-ts/repository-findings.test.ts b/sdk/typescript/tests-ts/repository-findings.test.ts index 1a001995..9a82060a 100644 --- a/sdk/typescript/tests-ts/repository-findings.test.ts +++ b/sdk/typescript/tests-ts/repository-findings.test.ts @@ -114,3 +114,271 @@ print(json.dumps(result)) "renamed-again", ]); }); + +test("shares findings only between explicitly identified repository and scope aliases", () => { + const python = (Bun.which("python3") ?? Bun.which("python"))!; + const probe = ` +import argparse, json, os, sqlite3, sys, tempfile +sys.path.insert(0, sys.argv[1]) +from filesystem_identity import serialize_filesystem_identity +import workbench_native_indexes as indexes + +connection = sqlite3.connect(":memory:") +connection.row_factory = sqlite3.Row +connection.executescript(""" +CREATE TABLE security_targets(id TEXT, current_path TEXT, display_name TEXT, repository_identity TEXT, origin TEXT); +CREATE TABLE scans(id TEXT, target_id TEXT, scope TEXT, updated_at TEXT, status TEXT, started_at TEXT); +CREATE TABLE finding_occurrences(id TEXT, finding_id TEXT, severity TEXT, created_at TEXT, scan_id TEXT, title TEXT, summary TEXT); +CREATE TABLE finding_triage(occurrence_id TEXT, status TEXT, updated_at TEXT, close_reason TEXT); +CREATE TABLE finding_locations(occurrence_id TEXT, relative_path TEXT, role TEXT, sort_order INTEGER); +CREATE TABLE scan_comparison_matches(before_occurrence_id TEXT, after_occurrence_id TEXT); +""") +origin = "https://example.invalid/synthetic/repository" +connection.executemany("INSERT INTO security_targets VALUES (?, ?, ?, ?, ?)", [ + ("primary", "/primary", "Primary", "common-git-directory::.", origin), + ("linked", "/linked", "Linked", "common-git-directory::.", origin), + ("empty-alias", "/empty-alias", "Empty alias", "common-git-directory::.", origin), + ("same-origin-clone", "/clone", "Clone", "independent-git-directory::.", origin), + ("different-scope", "/primary/packages/api", "Scoped", "common-git-directory::packages/api", origin), + ("unknown-first", "/unknown-first", "Unknown first", None, origin), + ("unknown-second", "/unknown-second", "Unknown second", None, origin), +]) + +def add_scan(scan_id, target, day): + timestamp = f"2026-02-{day:02d}T00:00:00Z" + connection.execute("INSERT INTO scans VALUES (?, ?, ?, ?, ?, ?)", (scan_id, target, "repository", timestamp, "complete", timestamp)) + +def add_finding(occurrence, finding, scan, severity="high"): + started = connection.execute("SELECT started_at FROM scans WHERE id = ?", (scan,)).fetchone()[0] + connection.execute("INSERT INTO finding_occurrences VALUES (?, ?, ?, ?, ?, ?, ?)", (occurrence, finding, severity, started, scan, finding, "Summary")) + connection.execute("INSERT INTO finding_locations VALUES (?, ?, ?, ?)", (occurrence, "src/auth.py", "root_control", 0)) + +for scan_id, target, day in [ + ("primary-old", "primary", 1), + ("empty-alias-scan", "empty-alias", 1), + ("linked-reviewed", "linked", 2), + ("primary-open", "primary", 3), + ("linked-latest", "linked", 4), + ("clone-scan", "same-origin-clone", 5), + ("scope-scan", "different-scope", 6), + ("unknown-first-scan", "unknown-first", 7), + ("unknown-second-scan", "unknown-second", 8), +]: + add_scan(scan_id, target, day) + +for occurrence, finding, scan in [ + ("dismissed-primary", "dismissed-original", "primary-old"), + ("dismissed-linked", "dismissed-renamed", "linked-reviewed"), + ("fixed-primary", "fixed-original", "primary-old"), + ("fixed-linked", "fixed-rediscovered", "linked-reviewed"), + ("wont-fix-primary", "wont-fix-original", "primary-old"), + ("wont-fix-linked", "wont-fix-renamed", "linked-reviewed"), + ("open-primary", "primary-only", "primary-open"), + ("same-id-primary", "same-id", "primary-open"), + ("open-linked", "linked-only", "linked-latest"), + ("same-id-linked", "same-id", "linked-latest"), + ("clone-occurrence", "clone-only", "clone-scan"), + ("scope-occurrence", "scope-only", "scope-scan"), + ("unknown-first-occurrence", "unknown-first-only", "unknown-first-scan"), + ("unknown-second-occurrence", "unknown-second-only", "unknown-second-scan"), +]: + add_finding(occurrence, finding, scan) + +connection.executemany("INSERT INTO scan_comparison_matches VALUES (?, ?)", [ + ("dismissed-primary", "dismissed-linked"), + ("fixed-primary", "fixed-linked"), + ("wont-fix-primary", "wont-fix-linked"), + ("open-linked", "clone-occurrence"), + ("open-linked", "scope-occurrence"), + ("unknown-first-occurrence", "unknown-second-occurrence"), +]) +connection.executemany("INSERT INTO finding_triage VALUES (?, ?, ?, ?)", [ + ("dismissed-primary", "closed", "2026-02-01T12:00:00Z", "false_positive"), + ("fixed-primary", "closed", "2026-02-01T12:00:00Z", "already_fixed"), + ("wont-fix-primary", "closed", "2026-02-01T12:00:00Z", "wont_fix"), +]) + +def findings(target, status="open", limit=20, offset=0): + arguments = argparse.Namespace(limit=limit, offset=offset, query=None, severity=None, status=status, target_id=target) + return indexes.list_global_findings(connection, arguments) + +indexes.scan_history.list_scans = lambda database: { + "scans": [ + {"scanId": scan["id"], "targetId": scan["target_id"]} + for scan in database.execute("SELECT id, target_id FROM scans") + ] +} +repository_arguments = argparse.Namespace( + query=None, target_id=None, status="open_findings", limit=None, offset=0 +) +result = { + "primary": findings("primary"), + "linked": findings("linked"), + "closed": findings("primary", "closed"), + "all": findings("primary", None), + "clone": findings("same-origin-clone"), + "differentScope": findings("different-scope"), + "unknownFirst": findings("unknown-first"), + "unknownSecond": findings("unknown-second"), + "firstPage": findings("primary", limit=2), + "secondPage": findings("primary", limit=2, offset=2), + "thirdPage": findings("primary", limit=2, offset=4), + "repositories": indexes.list_repositories(connection), + "openRepositories": indexes.list_repositories(connection, repository_arguments), +} +connection.execute( + "INSERT INTO finding_triage VALUES (?, ?, ?, ?)", + ("same-id-primary", "closed", "2026-02-05T00:00:00Z", "false_positive"), +) +result["sameIdDismissed"] = findings("primary") +result["sameIdClosed"] = findings("linked", "closed") +connection.execute( + "INSERT INTO finding_triage VALUES (?, ?, ?, ?)", + ("same-id-linked", "open", "2026-02-06T00:00:00Z", None), +) +result["sameIdReopened"] = findings("primary") +connection.execute("ALTER TABLE scans ADD COLUMN target_path TEXT") +connection.execute("ALTER TABLE scans ADD COLUMN target_device INTEGER") +connection.execute("ALTER TABLE scans ADD COLUMN target_inode INTEGER") +with tempfile.TemporaryDirectory() as reused_path: + metadata = os.stat(reused_path) + connection.execute( + "UPDATE security_targets SET current_path = ? WHERE id = ?", + (reused_path, "primary"), + ) + connection.execute( + "UPDATE scans SET target_path = ?, target_device = ?, target_inode = ? WHERE target_id = ?", + ( + reused_path, + serialize_filesystem_identity(metadata.st_dev), + serialize_filesystem_identity(metadata.st_ino + 1), + "primary", + ), + ) + result["reusedPath"] = findings("primary") + result["deletedAlias"] = findings("linked") +print(json.dumps(result)) +`; + + const execution = spawnSync( + python, + ["-I", "-B", "-c", probe, join(PLUGIN_ROOT, "scripts")], + { encoding: "utf8", timeout: 10_000 }, + ); + expect(execution.status, execution.stderr).toBe(0); + + const result = JSON.parse(execution.stdout) as Record< + string, + { + findings: Array>; + nextOffset: number | null; + repositories?: Array>; + } + >; + const primary = result["primary"]!.findings; + expect(result["linked"]!.findings).toEqual(primary); + expect(primary.map((finding) => finding["findingId"])).toEqual([ + "linked-only", + "same-id", + "primary-only", + "fixed-rediscovered", + ]); + expect( + primary.filter((finding) => finding["findingId"] === "same-id"), + ).toMatchObject([ + { + targetId: "linked", + occurrenceCount: 2, + knownScanIds: ["primary-open", "linked-latest"], + matchedFindingIds: ["same-id"], + }, + ]); + expect( + primary.find((finding) => finding["findingId"] === "linked-only"), + ).toMatchObject({ + confirmedInLatestScan: true, + matchedFindingIds: ["linked-only"], + }); + expect( + primary.find((finding) => finding["findingId"] === "primary-only"), + ).toMatchObject({ confirmedInLatestScan: false }); + expect( + primary.find((finding) => finding["findingId"] === "fixed-rediscovered"), + ).toMatchObject({ + status: "open", + knownScanIds: ["primary-old", "linked-reviewed"], + matchedFindingIds: ["fixed-original", "fixed-rediscovered"], + occurrenceCount: 2, + }); + expect(result["closed"]!.findings).toMatchObject([ + { + findingId: "dismissed-renamed", + status: "closed", + matchedFindingIds: ["dismissed-original", "dismissed-renamed"], + }, + { + findingId: "wont-fix-renamed", + status: "closed", + matchedFindingIds: ["wont-fix-original", "wont-fix-renamed"], + }, + ]); + expect(result["all"]!.findings).toHaveLength(6); + expect(result["clone"]!.findings).toMatchObject([ + { findingId: "clone-only", targetId: "same-origin-clone" }, + ]); + expect(result["differentScope"]!.findings).toMatchObject([ + { findingId: "scope-only", targetId: "different-scope" }, + ]); + expect(result["unknownFirst"]!.findings).toMatchObject([ + { findingId: "unknown-first-only", targetId: "unknown-first" }, + ]); + expect(result["unknownSecond"]!.findings).toMatchObject([ + { findingId: "unknown-second-only", targetId: "unknown-second" }, + ]); + expect( + Object.fromEntries( + result["repositories"]!.repositories!.map((repository) => [ + repository["targetId"], + repository["openFindingsCount"], + ]), + ), + ).toEqual({ + primary: 4, + linked: 4, + "empty-alias": 4, + "same-origin-clone": 1, + "different-scope": 1, + "unknown-first": 1, + "unknown-second": 1, + }); + expect( + result["openRepositories"]!.repositories!.map( + (repository) => repository["targetId"], + ), + ).toContain("empty-alias"); + expect(result["firstPage"]!.nextOffset).toBe(2); + expect(result["secondPage"]!.nextOffset).toBeNull(); + expect(result["thirdPage"]!.nextOffset).toBeNull(); + expect([ + ...result["firstPage"]!.findings, + ...result["secondPage"]!.findings, + ...result["thirdPage"]!.findings, + ]).toEqual(primary); + expect( + result["sameIdDismissed"]!.findings.some( + (finding) => finding["findingId"] === "same-id", + ), + ).toBe(false); + expect( + result["sameIdClosed"]!.findings.find( + (finding) => finding["findingId"] === "same-id", + ), + ).toMatchObject({ status: "closed", occurrenceCount: 2 }); + expect( + result["sameIdReopened"]!.findings.find( + (finding) => finding["findingId"] === "same-id", + ), + ).toMatchObject({ status: "open", occurrenceCount: 2 }); + expect(result["reusedPath"]!.findings).toEqual([]); + expect(result["deletedAlias"]!.findings).toHaveLength(4); +}); diff --git a/sdk/typescript/tests-ts/workbench-canonical-paths.test.ts b/sdk/typescript/tests-ts/workbench-canonical-paths.test.ts index dd04b823..0d64d4bf 100644 --- a/sdk/typescript/tests-ts/workbench-canonical-paths.test.ts +++ b/sdk/typescript/tests-ts/workbench-canonical-paths.test.ts @@ -52,6 +52,60 @@ const simulatedPathProbe = [ "print(json.dumps({'accepted': accepted, 'nativePathEquality': path_type(supplied) == path_type(resolved), 'resolvedPath': result}))", ].join("\n"); +const caseSensitiveWindowsHistoryProbe = [ + "import json, sqlite3, sys", + "from pathlib import PureWindowsPath", + "from types import SimpleNamespace", + "sys.path.insert(0, sys.argv[1])", + "import workbench_scan_history as history", + "class WindowsPath(PureWindowsPath):", + " def expanduser(self):", + " return self", + " def resolve(self, strict=False):", + " return self", + "history.Path = WindowsPath", + "connection = sqlite3.connect(':memory:')", + "connection.row_factory = sqlite3.Row", + "connection.executescript('''", + "CREATE TABLE security_targets (id TEXT, current_path TEXT, repository_identity TEXT);", + "CREATE TABLE scans (id TEXT, target_id TEXT, target_path TEXT, status TEXT, started_at TEXT);", + "CREATE TABLE scan_comparisons (before_scan_id TEXT, after_scan_id TEXT);", + "''')", + "def target(identifier, path, identity):", + " return connection.execute('SELECT ? AS target_id, ? AS target_path, ? AS repository_identity', (identifier, str(WindowsPath(path)), identity)).fetchone()", + "upper_root = target('upper-root', 'D:/Repository', 'root-upper')", + "lower_root = target('lower-root', 'D:/repository', 'root-lower')", + "upper_scope = target('upper-scope', 'D:/Repository/Service', 'scope-upper')", + "lower_scope = target('lower-scope', 'D:/Repository/service', 'scope-lower')", + "linked_scope = target('linked-scope', 'E:/Linked/Service', 'scope-upper')", + "clone_scope = target('clone-scope', 'D:/Clone/Service', 'scope-clone')", + "same_path = target('same-path', 'D:/Repository/Service', None)", + "targets = [upper_root, lower_root, upper_scope, lower_scope, linked_scope, clone_scope]", + "identities = {entry['target_path']: entry['repository_identity'] for entry in targets}", + "history.repository_identity = lambda path: identities.get(str(path))", + "history.repository_relative_path = lambda path: WindowsPath(path).name", + "history._repository_origin = lambda path: ('example.test', 'synthetic/repository')", + "root_group = ('root-upper', None, None)", + "scope_group = ('scope-upper', None, None)", + "checks = {", + " 'nativeWindowsScopeEquality': WindowsPath(upper_scope['target_path']) == WindowsPath(lower_scope['target_path']),", + " 'nativeWindowsRootEquality': WindowsPath(upper_root['target_path']) == WindowsPath(lower_root['target_path']),", + " 'caseSensitiveScopesMatch': history._same_repository(lower_scope, upper_scope, after_identity=scope_group),", + " 'caseSensitiveRootsMatch': history._same_repository(lower_root, upper_root, after_identity=root_group),", + " 'linkedWorktreeMatches': history._same_repository(linked_scope, upper_scope, after_identity=scope_group),", + " 'sameOriginCloneMatches': history._same_repository(clone_scope, upper_scope, after_identity=scope_group),", + " 'exactResolvedPathMatches': history._same_repository(same_path, upper_scope, after_identity=scope_group),", + "}", + "for entry in (upper_scope, lower_scope):", + " connection.execute('INSERT INTO security_targets VALUES (?, ?, ?)', (entry['target_id'], entry['target_path'], entry['repository_identity']))", + "connection.execute('INSERT INTO scans VALUES (?, ?, ?, ?, ?)', ('lower-scan', lower_scope['target_id'], lower_scope['target_path'], 'complete', '2026-08-15T00:00:00Z'))", + "history._require_current_target_owner = lambda *args: None", + "history._same_repository = lambda *args, **kwargs: False", + "selected = history.list_unmatched_scan_pairs(connection, SimpleNamespace(repository=upper_scope['target_path'], force=False), backfill_finding_details=lambda *args: None, read_coverage=lambda scan: {})", + "checks['caseSensitiveUnmatchedScanCount'] = selected['scanCount']", + "print(json.dumps(checks))", +].join("\n"); + const realFilesystemProbe = [ "import json, sys", "from pathlib import Path", @@ -178,6 +232,19 @@ describe("bundled workbench canonical paths", () => { }); }); + test("keeps case-sensitive Windows repository roots and scopes distinct", () => { + expect(runPythonProbe(caseSensitiveWindowsHistoryProbe)).toEqual({ + nativeWindowsScopeEquality: true, + nativeWindowsRootEquality: true, + caseSensitiveScopesMatch: false, + caseSensitiveRootsMatch: false, + linkedWorktreeMatches: true, + sameOriginCloneMatches: false, + exactResolvedPathMatches: true, + caseSensitiveUnmatchedScanCount: 0, + }); + }); + testCaseSensitive( "rejects case-differing symlinks at every workbench and finalizer boundary", async () => { diff --git a/sdk/typescript/tests-ts/workbench-repository-identity.test.ts b/sdk/typescript/tests-ts/workbench-repository-identity.test.ts new file mode 100644 index 00000000..a87bc300 --- /dev/null +++ b/sdk/typescript/tests-ts/workbench-repository-identity.test.ts @@ -0,0 +1,826 @@ +import { execFileSync, spawnSync } from "node:child_process"; +import { + mkdirSync, + mkdtempSync, + realpathSync, + rmSync, + writeFileSync, +} from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, describe, expect, test } from "bun:test"; +import { PLUGIN_ROOT } from "./plugin-root.js"; + +const temporaryRoots: string[] = []; +const remote = + "https://fixture-user:SYNTHETIC_PASSWORD@example.test/acme/project.git"; + +afterEach(() => { + for (const root of temporaryRoots.splice(0)) { + rmSync(root, { recursive: true, force: true }); + } +}); + +function git(repository: string, ...args: string[]): string { + return execFileSync( + "git", + [ + "-c", + "user.name=Fixture", + "-c", + "user.email=fixture@example.test", + ...args, + ], + { cwd: repository, encoding: "utf8", stdio: ["ignore", "pipe", "pipe"] }, + ).trim(); +} + +function fixture(): { + root: string; + repository: string; + worktree: string; + clone: string; +} { + const root = realpathSync( + mkdtempSync(join(tmpdir(), "codex-security-repository-identity-")), + ); + temporaryRoots.push(root); + const repository = join(root, "repository"); + const worktree = join(root, "linked-worktree"); + const clone = join(root, "same-origin-clone"); + mkdirSync(repository); + git(repository, "init", "-q"); + for (const service of ["service-a", "service-b", "MixedCase"]) { + mkdirSync(join(repository, service)); + writeFileSync(join(repository, service, "service.py"), "value = 1\n"); + } + git(repository, "add", "."); + git(repository, "commit", "-qm", "fixture"); + git(repository, "remote", "add", "origin", remote); + git(repository, "worktree", "add", "--detach", "-q", worktree, "HEAD"); + execFileSync("git", ["clone", "-q", repository, clone], { + stdio: ["ignore", "pipe", "pipe"], + }); + git(clone, "remote", "set-url", "origin", remote); + return { root, repository, worktree, clone }; +} + +const probe = String.raw` +import argparse +import json +import os +import shutil +import sqlite3 +import subprocess +import sys +from pathlib import Path +from types import SimpleNamespace +from unittest.mock import patch + +sys.path.insert(0, sys.argv[1]) + +import workbench_scan_history as history +from filesystem_identity import serialize_filesystem_identity +from workbench_native_indexes import repository_target_ids +from workbench_schema import MIGRATIONS, apply_migrations +from workbench_target_state import ( + _repository_birth_time_ns, + backfill_repository_identities, + backfill_security_targets, + ensure_security_target, + repository_identity, + repository_relative_path, + stable_target_id, +) + +scenario = sys.argv[2] +root, repository, worktree, clone = map(Path, sys.argv[3:7]) +timestamp = "2026-08-01T00:00:00Z" +connection = sqlite3.connect(":memory:") +connection.row_factory = sqlite3.Row +connection.execute("PRAGMA foreign_keys = ON") +apply_migrations(connection, MIGRATIONS, lambda: timestamp, backfill_security_targets) + + +def git(target, *args): + return subprocess.run( + [ + "git", + "-c", "user.name=Fixture", + "-c", "user.email=fixture@example.test", + "-C", str(target), + *args, + ], + check=True, + capture_output=True, + text=True, + ).stdout.strip() + + +def add_scan(scan_id, target, ownership="current", verify_ownership=True): + target = Path(target) + target_id = ensure_security_target( + connection, str(target), verify_ownership=verify_ownership + ) + metadata = target.stat() + device = serialize_filesystem_identity(metadata.st_dev) + inode = serialize_filesystem_identity(metadata.st_ino) + if ownership == "missing": + device, inode = None, None + elif ownership == "malformed": + inode = None + elif ownership == "mismatch": + inode = serialize_filesystem_identity(metadata.st_ino + 1) + + workspace_id = f"workspace-{scan_id}" + connection.execute( + "INSERT INTO workspaces (id, target_path, target_id, created_at, updated_at) " + "VALUES (?, ?, ?, ?, ?)", + (workspace_id, str(target), target_id, timestamp, timestamp), + ) + connection.execute( + "INSERT INTO scans (id, workspace_id, target_path, target_id, target_device, " + "target_inode, target_revision, scope, mode, scan_dir, status, phase, " + "started_at, created_at, updated_at) " + "VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)", + ( + scan_id, workspace_id, str(target), target_id, device, inode, + "synthetic-revision", ".", "standard", str(root / "scans" / scan_id), + "complete", "reporting", timestamp, timestamp, timestamp, + ), + ) + connection.execute( + "INSERT INTO scan_progress (scan_id, updated_at) VALUES (?, ?)", + (scan_id, timestamp), + ) + return target_id + + +def listed(target): + arguments = argparse.Namespace( + repository=str(target), + scan_root=None, + target_id=None, + mode=None, + status=None, + query=None, + limit=None, + offset=0, + ) + return sorted(scan["scanId"] for scan in history.list_scans(connection, arguments)["scans"]) + + +def forged_worktree(): + forged = root / "forged-git-pointer" + forged.mkdir() + common = git(repository, "rev-parse", "--path-format=absolute", "--git-common-dir") + (forged / ".git").write_text(f"gitdir: {common}\n") + return forged + + +if scenario == "identity": + targets = { + "repository": repository, + "worktree": worktree, + "serviceA": repository / "service-a", + "worktreeServiceA": worktree / "service-a", + "serviceB": repository / "service-b", + "mixedCase": repository / "MixedCase", + "worktreeMixedCase": worktree / "MixedCase", + "clone": clone, + } + target_ids = { + name: ensure_security_target(connection, str(target)) + for name, target in targets.items() + } + identities = { + name: repository_identity(target) + for name, target in targets.items() + } + common = os.path.realpath(git( + repository, "rev-parse", "--path-format=absolute", "--git-common-dir" + )) + forged_identity = repository_identity(forged_worktree()) + (repository / "service-a" / "service.py").write_text("value = 2\n") + git(repository, "add", ".") + git(repository, "commit", "-qm", "ordinary update") + commit_independent = repository_identity(repository) == identities["repository"] + additional_worktree = root / "additional-worktree" + git(repository, "worktree", "add", "--detach", "-q", str(additional_worktree), "HEAD") + worktree_independent = repository_identity(repository) == identities["repository"] + git(repository, "worktree", "remove", "--force", str(additional_worktree)) + git( + repository, + "remote", "set-url", "origin", + "https://different-user:DIFFERENT_SYNTHETIC_PASSWORD@example.test/other/repo.git", + ) + remote_independent = repository_identity(repository) == identities["repository"] + description = Path(common) / "description" + description.write_text("An ordinary user-edited Git description.\n") + description_edit_independent = repository_identity(repository) == identities["repository"] + os.chmod(description, 0o400) + description_mode_independent = repository_identity(repository) == identities["repository"] + os.chmod(description, 0o600) + description.unlink() + description_absence_independent = repository_identity(repository) == identities["repository"] + custom_template = root / "custom-template" + custom_template.mkdir() + git(custom_template, "init", "-q", "--template=") + markerless_repository_identity = repository_identity(custom_template) + with patch("workbench_target_state._repository_birth_time_ns", side_effect=(41, 42)): + recycled_inode_distinguished = ( + repository_identity(repository) != repository_identity(repository) + ) + original_normcase = os.path.normcase + os.path.normcase = lambda path: os.fspath(path).lower() + try: + case_preserved = ( + repository_relative_path(repository / "MixedCase") == "MixedCase" + and repository_identity(repository / "MixedCase") == identities["mixedCase"] + ) + finally: + os.path.normcase = original_normcase + + legacy_metadata = SimpleNamespace(st_ctime_ns=41) + modern_metadata = SimpleNamespace(st_birthtime_ns=43, st_ctime_ns=41) + with patch.object(os, "name", "nt"): + windows_legacy_birth_time = _repository_birth_time_ns(common, legacy_metadata) + windows_modern_birth_time = _repository_birth_time_ns(common, modern_metadata) + linux_birth_times = {} + with patch.object(sys, "platform", "linux"), patch.object(os, "name", "posix"): + for label, output, status in ( + ("valid", "42.000000123\n", 0), + ("unavailable", "0.000000000\n", 0), + ("malformed", "42.123\n", 0), + ("failed", "42.000000123\n", 1), + ): + with patch("workbench_target_state.subprocess.run") as stat_command: + stat_command.return_value = SimpleNamespace(stdout=output, returncode=status) + linux_birth_times[label] = _repository_birth_time_ns(common, legacy_metadata) + if label == "valid": + linux_locale = stat_command.call_args.kwargs["env"]["LC_ALL"] + with patch("workbench_target_state.subprocess.run", side_effect=OSError("missing stat")): + linux_birth_times["missing"] = _repository_birth_time_ns(common, legacy_metadata) + print(json.dumps({ + "identities": identities, + "forgedIdentity": forged_identity, + "commitIndependent": commit_independent, + "worktreeIndependent": worktree_independent, + "remoteIndependent": remote_independent, + "descriptionEditIndependent": description_edit_independent, + "descriptionModeIndependent": description_mode_independent, + "descriptionAbsenceIndependent": description_absence_independent, + "markerlessRepositoryIdentity": markerless_repository_identity, + "recycledInodeDistinguished": recycled_inode_distinguished, + "casePreserved": case_preserved, + "windowsLegacyBirthTime": windows_legacy_birth_time, + "windowsModernBirthTime": windows_modern_birth_time, + "linuxBirthTimes": linux_birth_times, + "linuxLocale": linux_locale, + "targetIdsPreserved": all( + target_ids[name] == stable_target_id(target) + for name, target in targets.items() + ), + "stored": { + row["current_path"]: row["repository_identity"] + for row in connection.execute( + "SELECT current_path, repository_identity FROM security_targets" + ) + }, + })) + +elif scenario == "history": + add_scan("canonical-root", repository) + add_scan("linked-root", worktree) + add_scan("spoof-root", clone) + add_scan("canonical-a", repository / "service-a") + add_scan("linked-a", worktree / "service-a") + add_scan("canonical-b", repository / "service-b") + add_scan("spoof-a", clone / "service-a") + add_scan("forged-root", forged_worktree()) + before = { + "root": listed(repository), + "serviceA": listed(repository / "service-a"), + "serviceB": listed(repository / "service-b"), + } + git(repository, "worktree", "remove", "--force", str(worktree)) + after = { + "root": listed(repository), + "serviceA": listed(repository / "service-a"), + } + arguments = argparse.Namespace(repository=str(repository), force=False) + matched = history.list_unmatched_scan_pairs( + connection, + arguments, + backfill_finding_details=lambda _connection, _scan: None, + read_coverage=lambda _scan: {}, + ) + + def unavailable(scan): + if scan["id"] == "linked-root": + raise SystemExit("Saved scan artifacts are unavailable.") + return {} + + unavailable_matches = history.list_unmatched_scan_pairs( + connection, + arguments, + backfill_finding_details=lambda _connection, _scan: None, + read_coverage=unavailable, + ) + + def require_scan(database, scan_id): + return database.execute("SELECT * FROM scans WHERE id = ?", (scan_id,)).fetchone() + + comparison_args = argparse.Namespace( + before_scan_id="canonical-root", + after_scan_id="linked-root", + matches_json=json.dumps({"matches": [], "uncertain": []}), + ) + compared = history.compare_scans( + connection, + comparison_args, + require_scan=require_scan, + read_coverage=lambda _scan: {"completeness": "complete"}, + ) + connection.commit() + saved = history.save_scan_comparison( + connection, + comparison_args, + now=lambda: timestamp, + require_scan=require_scan, + read_coverage=lambda _scan: {"completeness": "complete"}, + ) + explicit_clone = history.compare_scans( + connection, + argparse.Namespace(before_scan_id="canonical-root", after_scan_id="spoof-root"), + require_scan=require_scan, + read_coverage=lambda _scan: {"completeness": "complete"}, + ) + + print(json.dumps({ + "before": before, + "after": after, + "matchedScanCount": matched["scanCount"], + "matchingBatches": len(matched["batches"]), + "unavailableScans": unavailable_matches["unavailableScans"], + "compared": compared["beforeScanId"] == "canonical-root" + and compared["afterScanId"] == "linked-root", + "saved": saved["beforeScanId"] == "canonical-root" + and saved["afterScanId"] == "linked-root", + "explicitCloneCompared": explicit_clone["afterScanId"] == "spoof-root", + })) + +elif scenario == "backfill": + add_scan("valid", repository) + add_scan("reused", clone, "mismatch") + add_scan("mixed-current", repository / "service-a") + add_scan("mixed-previous", repository / "service-a", "mismatch") + add_scan("malformed", repository / "service-b", "malformed") + add_scan("missing", worktree) + missing_path = str(worktree) + git(repository, "worktree", "remove", "--force", missing_path) + connection.execute("UPDATE security_targets SET repository_identity = NULL") + before_ids = { + row["current_path"]: row["id"] + for row in connection.execute("SELECT id, current_path FROM security_targets") + } + backfill_repository_identities(connection) + rows = { + row["current_path"]: {"id": row["id"], "identity": row["repository_identity"]} + for row in connection.execute( + "SELECT id, current_path, repository_identity FROM security_targets" + ) + } + print(json.dumps({ + "valid": rows[str(repository)]["identity"], + "reused": rows[str(clone)]["identity"], + "mixed": rows[str(repository / "service-a")]["identity"], + "malformed": rows[str(repository / "service-b")]["identity"], + "missing": rows[missing_path]["identity"], + "idsPreserved": all(rows[path]["id"] == target_id for path, target_id in before_ids.items()), + })) + +elif scenario == "replacement": + add_scan("previous-owner", worktree) + add_scan("trusted-alias", repository) + git(repository, "worktree", "remove", "--force", str(worktree)) + worktree.mkdir() + git(worktree, "init", "-q") + (worktree / "replacement.py").write_text("value = 2\n") + git(worktree, "add", ".") + git(worktree, "commit", "-qm", "replacement") + git(worktree, "remote", "add", "origin", sys.argv[7]) + before = listed(worktree) + try: + add_scan("replacement-owner", worktree) + except SystemExit as error: + registration_error = str(error) + else: + registration_error = None + try: + history.list_unmatched_scan_pairs( + connection, + argparse.Namespace(repository=str(worktree), force=False), + backfill_finding_details=lambda _connection, _scan: None, + read_coverage=lambda _scan: {}, + ) + except SystemExit as error: + matching_error = str(error) + else: + matching_error = None + after = listed(worktree) + + empty_target = clone / "service-a" + ensure_security_target(connection, str(empty_target)) + connection.execute( + "UPDATE security_targets SET repository_identity = ? WHERE current_path = ?", + (repository_identity(repository), str(empty_target)), + ) + try: + ensure_security_target(connection, str(empty_target)) + except SystemExit as error: + empty_target_error = str(error) + else: + empty_target_error = None + + explicit_clone_id = add_scan("explicit-clone", clone) + connection.execute( + "UPDATE security_targets SET repository_identity = ? WHERE id = ?", + (repository_identity(repository), explicit_clone_id), + ) + try: + ensure_security_target(connection, str(clone)) + except SystemExit as error: + explicit_clone_error = str(error) + else: + explicit_clone_error = None + + unverified_target = clone / "service-b" + add_scan("unverified-owner", unverified_target, "missing") + try: + ensure_security_target(connection, str(unverified_target)) + except SystemExit as error: + unverified_owner_error = str(error) + else: + unverified_owner_error = None + print(json.dumps({ + "before": before, + "after": after, + "registrationError": registration_error, + "matchingError": matching_error, + "emptyTargetError": empty_target_error, + "explicitCloneError": explicit_clone_error, + "unverifiedOwnerError": unverified_owner_error, + "replacementScanCreated": connection.execute( + "SELECT 1 FROM scans WHERE id = 'replacement-owner'" + ).fetchone() is not None, + })) + +elif scenario == "recreated-directory": + target = repository / "service-a" + linked = worktree / "service-a" + target_id = add_scan("before-recreation", target) + linked_id = add_scan("linked-scope", linked) + original_metadata = target.stat() + original_identity = repository_identity(target) + original_scan = connection.execute( + "SELECT target_device, target_inode FROM scans WHERE id = 'before-recreation'" + ).fetchone() + + shutil.rmtree(target) + (root / "retired-directory-inode").mkdir() + target.mkdir() + (target / "service.py").write_text("value = 2\n") + recreated_metadata = target.stat() + before_rescan = listed(target) + before_aliases = repository_target_ids(connection, target_id) + recreated_id = add_scan("after-recreation", target) + repeated_id = add_scan("repeated-rescan", target) + after_rescan = listed(target) + after_aliases = repository_target_ids(connection, target_id) + + malformed_target = repository / "service-b" + add_scan("malformed-owner", malformed_target, "malformed") + try: + ensure_security_target(connection, str(malformed_target)) + except SystemExit as error: + malformed_owner_error = str(error) + else: + malformed_owner_error = None + + plain = root / "recreated-nongit" + plain.mkdir() + add_scan("plain-before-recreation", plain) + shutil.rmtree(plain) + (root / "retired-nongit-inode").mkdir() + plain.mkdir() + try: + ensure_security_target(connection, str(plain)) + except SystemExit as error: + nongit_owner_error = str(error) + else: + nongit_owner_error = None + + root_target_id = add_scan("linked-root-before", worktree) + add_scan("canonical-root-alias", repository) + root_identity = repository_identity(worktree) + root_metadata = worktree.stat() + git_pointer = (worktree / ".git").read_text() + shutil.rmtree(worktree) + (root / "retired-worktree-root-inode").mkdir() + worktree.mkdir() + (worktree / ".git").write_text(git_pointer) + try: + ensure_security_target(connection, str(worktree)) + except SystemExit as error: + recreated_root_error = str(error) + else: + recreated_root_error = None + + preserved_scan = connection.execute( + "SELECT target_device, target_inode FROM scans WHERE id = 'before-recreation'" + ).fetchone() + print(json.dumps({ + "ownerChanged": ( + original_metadata.st_dev != recreated_metadata.st_dev + or original_metadata.st_ino != recreated_metadata.st_ino + ), + "identityPreserved": repository_identity(target) == original_identity, + "targetIdPreserved": recreated_id == target_id and repeated_id == target_id, + "historicalOwnerPreserved": ( + preserved_scan["target_device"] == original_scan["target_device"] + and preserved_scan["target_inode"] == original_scan["target_inode"] + ), + "beforeRescan": before_rescan, + "afterRescan": after_rescan, + "beforeAliases": sorted(before_aliases), + "afterAliases": sorted(after_aliases), + "expectedAliases": sorted((target_id, linked_id)), + "malformedOwnerError": malformed_owner_error, + "nongitOwnerError": nongit_owner_error, + "rootOwnerChanged": root_metadata.st_ino != worktree.stat().st_ino, + "rootIdentityPreserved": repository_identity(worktree) == root_identity, + "recreatedRootError": recreated_root_error, + "recreatedRootScans": listed(worktree), + "recreatedRootAliases": sorted(repository_target_ids(connection, root_target_id)), + })) + +elif scenario == "git-replacement": + add_scan("original-repository", repository) + add_scan("original-alias", worktree) + original_identity = repository_identity(repository) + original_metadata = repository.stat() + + def remove_read_only(operation, path, _error): + os.chmod(path, 0o700) + operation(path) + + shutil.rmtree(repository / ".git", onerror=remove_read_only) + git(repository, "init", "-q") + git(repository, "add", ".") + git(repository, "commit", "-qm", "replacement") + replacement_identity = repository_identity(repository) + try: + ensure_security_target(connection, str(repository)) + except SystemExit as error: + registration_error = str(error) + else: + registration_error = None + replacement_metadata = repository.stat() + print(json.dumps({ + "originalIdentity": original_identity, + "replacementIdentity": replacement_identity, + "checkoutOwnerUnchanged": ( + original_metadata.st_dev == replacement_metadata.st_dev + and original_metadata.st_ino == replacement_metadata.st_ino + ), + "visibleScans": listed(repository), + "registrationError": registration_error, + })) + +elif scenario == "unverified-owner": + ensure_security_target(connection, str(repository)) + add_scan("trusted-alias", worktree) + git(repository, "worktree", "remove", "--force", str(worktree)) + before = listed(repository) + add_scan("unverified-owner", repository, "missing") + after = listed(repository) + print(json.dumps({ + "before": before, + "after": after, + })) + +elif scenario == "nongit": + first = root / "plain-a" + second = root / "plain-b" + first.mkdir() + second.mkdir() + add_scan("plain-a", first) + add_scan("legacy-a", first, "missing") + add_scan("malformed-a", first, "malformed", verify_ownership=False) + add_scan("mismatched-a", first, "mismatch", verify_ownership=False) + add_scan("plain-b", second) + before = connection.execute( + "SELECT NULL AS target_id, ? AS target_path", (str(first),) + ).fetchone() + after = connection.execute( + "SELECT NULL AS target_id, ? AS target_path", (str(second),) + ).fetchone() + same = connection.execute( + "SELECT NULL AS target_id, ? AS target_path", (str(first),) + ).fetchone() + print(json.dumps({ + "firstIdentity": repository_identity(first), + "secondIdentity": repository_identity(second), + "differentNullIdsMatch": history._same_repository(before, after), + "sameNullPathMatches": history._same_repository(before, same), + "firstScans": listed(first), + "secondScans": listed(second), + })) +`; + +function runProbe( + scenario: string, + repositories: ReturnType, +): Record { + const python = Bun.which("python3") ?? Bun.which("python") ?? Bun.which("py"); + expect(python).not.toBeNull(); + if (python === null) throw new Error("A Python interpreter is required."); + const execution = spawnSync( + python, + [ + "-I", + "-B", + "-c", + probe, + join(PLUGIN_ROOT, "scripts"), + scenario, + repositories.root, + repositories.repository, + repositories.worktree, + repositories.clone, + remote, + ], + { encoding: "utf8", timeout: 20_000 }, + ); + expect(execution.status, execution.stderr).toBe(0); + expect(execution.stderr).toBe(""); + return JSON.parse(execution.stdout) as Record; +} + +describe("durable workbench repository identities", () => { + test("hashes Git common directories and repository-relative target scopes", () => { + const repositories = fixture(); + const result = runProbe("identity", repositories); + const identities = result["identities"] as Record; + + expect(identities["repository"]).toMatch( + /^repository_sha256_[a-f0-9]{64}$/, + ); + expect(identities["worktree"]).toBe(identities["repository"]); + expect(identities["worktreeServiceA"]).toBe(identities["serviceA"]); + expect(identities["worktreeMixedCase"]).toBe(identities["mixedCase"]); + expect(identities["serviceA"]).not.toBe(identities["repository"]); + expect(identities["serviceA"]).not.toBe(identities["serviceB"]); + expect(identities["clone"]).not.toBe(identities["repository"]); + expect(result["forgedIdentity"]).toBeNull(); + expect(result["commitIndependent"]).toBe(true); + expect(result["worktreeIndependent"]).toBe(true); + expect(result["remoteIndependent"]).toBe(true); + expect(result["descriptionEditIndependent"]).toBe(true); + expect(result["descriptionModeIndependent"]).toBe(true); + expect(result["descriptionAbsenceIndependent"]).toBe(true); + expect(result["markerlessRepositoryIdentity"]).toMatch( + /^repository_sha256_[a-f0-9]{64}$/, + ); + expect(result["recycledInodeDistinguished"]).toBe(true); + expect(result["casePreserved"]).toBe(true); + expect(result["windowsLegacyBirthTime"]).toBe(41); + expect(result["windowsModernBirthTime"]).toBe(43); + expect(result["linuxBirthTimes"]).toEqual({ + valid: 42_000_000_123, + unavailable: null, + malformed: null, + failed: null, + missing: null, + }); + expect(result["linuxLocale"]).toBe("C"); + expect(result["targetIdsPreserved"]).toBe(true); + expect(JSON.stringify(result["stored"])).not.toContain( + "SYNTHETIC_PASSWORD", + ); + }, 30_000); + + test("retains removed worktrees while excluding spoofed origins and unrelated scopes", () => { + const result = runProbe("history", fixture()); + const before = result["before"] as Record; + const after = result["after"] as Record; + + expect(before["root"]).toEqual(["canonical-root", "linked-root"]); + expect(before["serviceA"]).toEqual(["canonical-a", "linked-a"]); + expect(before["serviceB"]).toEqual(["canonical-b"]); + expect(after["root"]).toEqual(["canonical-root", "linked-root"]); + expect(after["serviceA"]).toEqual(["canonical-a", "linked-a"]); + expect(result["matchedScanCount"]).toBe(3); + expect(result["matchingBatches"]).toBe(2); + expect(result["unavailableScans"]).toBe(1); + expect(result["compared"]).toBe(true); + expect(result["saved"]).toBe(true); + expect(result["explicitCloneCompared"]).toBe(true); + }, 30_000); + + test("backfills only current owners and preserves every existing target ID", () => { + const result = runProbe("backfill", fixture()); + + expect(result["valid"]).toMatch(/^repository_sha256_[a-f0-9]{64}$/); + expect(result["reused"]).toBeNull(); + expect(result["mixed"]).toBeNull(); + expect(result["malformed"]).toBeNull(); + expect(result["missing"]).toBeNull(); + expect(result["idsPreserved"]).toBe(true); + }, 30_000); + + test("does not expose a previous checkout owner or its trusted aliases", () => { + const result = runProbe("replacement", fixture()); + + expect(result["before"]).toEqual([]); + expect(result["after"]).toEqual([]); + expect(result["registrationError"]).toContain( + "refusing to reuse its target", + ); + expect(result["matchingError"]).toContain("refusing to reuse its target"); + expect(result["emptyTargetError"]).toContain( + "refusing to reuse its target", + ); + expect(result["replacementScanCreated"]).toBe(false); + expect(result["explicitCloneError"]).toContain( + "refusing to reuse its target", + ); + expect(result["unverifiedOwnerError"]).toContain( + "refusing to reuse its target", + ); + }, 30_000); + + test("rejects an in-place Git directory replacement under the same checkout", () => { + const result = runProbe("git-replacement", fixture()); + + expect(result["checkoutOwnerUnchanged"]).toBe(true); + expect(result["replacementIdentity"]).not.toBe(result["originalIdentity"]); + expect(result["visibleScans"]).toEqual([]); + expect(result["registrationError"]).toContain( + "refusing to reuse its target", + ); + }, 30_000); + + test("rescans recreated directories when their Git repository and scope are unchanged", () => { + const result = runProbe("recreated-directory", fixture()); + + expect(result["ownerChanged"]).toBe(true); + expect(result["identityPreserved"]).toBe(true); + expect(result["targetIdPreserved"]).toBe(true); + expect(result["historicalOwnerPreserved"]).toBe(true); + expect(result["beforeRescan"]).toEqual([ + "before-recreation", + "linked-scope", + ]); + expect(result["afterRescan"]).toEqual([ + "after-recreation", + "before-recreation", + "linked-scope", + "repeated-rescan", + ]); + expect(result["beforeAliases"]).toEqual(result["expectedAliases"]); + expect(result["afterAliases"]).toEqual(result["expectedAliases"]); + expect(result["malformedOwnerError"]).toContain( + "refusing to reuse its target", + ); + expect(result["nongitOwnerError"]).toContain( + "refusing to reuse its target", + ); + expect(result["rootOwnerChanged"]).toBe(true); + expect(result["rootIdentityPreserved"]).toBe(true); + expect(result["recreatedRootError"]).toContain( + "refusing to reuse its target", + ); + expect(result["recreatedRootScans"]).toEqual([]); + expect(result["recreatedRootAliases"]).toEqual([]); + }, 30_000); + + test("does not expand trusted aliases when historical checkout ownership is unverified", () => { + const result = runProbe("unverified-owner", fixture()); + + expect(result["before"]).toEqual(["trusted-alias"]); + expect(result["after"]).toEqual(["unverified-owner"]); + }, 30_000); + + test("keeps non-Git paths isolated and never equates unrelated null target IDs", () => { + const result = runProbe("nongit", fixture()); + + expect(result["firstIdentity"]).toBeNull(); + expect(result["secondIdentity"]).toBeNull(); + expect(result["differentNullIdsMatch"]).toBe(false); + expect(result["sameNullPathMatches"]).toBe(true); + expect(result["firstScans"]).toEqual(["legacy-a", "plain-a"]); + expect(result["secondScans"]).toEqual(["plain-b"]); + }, 30_000); +}); diff --git a/sdk/typescript/tests-ts/workbench-target-migration.test.ts b/sdk/typescript/tests-ts/workbench-target-migration.test.ts new file mode 100644 index 00000000..d1ad7989 --- /dev/null +++ b/sdk/typescript/tests-ts/workbench-target-migration.test.ts @@ -0,0 +1,389 @@ +import { spawnSync } from "node:child_process"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { describe, expect, test } from "bun:test"; +import { PLUGIN_ROOT } from "./plugin-root.js"; + +const migrationProbe = String.raw` +import json +import sqlite3 +import sys +from pathlib import Path + +sys.path.insert(0, sys.argv[1]) + +from workbench_schema import MIGRATIONS, apply_migrations +from workbench_target_state import backfill_security_targets, stable_target_id + +scenario = sys.argv[2] +root = Path(sys.argv[3]) +timestamp = "2026-08-01T00:00:00Z" +backfill_calls = [] + +def backfill(connection): + backfill_calls.append(True) + backfill_security_targets(connection) + +connection = sqlite3.connect(":memory:") +connection.row_factory = sqlite3.Row +connection.execute("PRAGMA foreign_keys = ON") +apply_migrations(connection, MIGRATIONS, lambda: timestamp, backfill) +backfill_calls.clear() + +existing_path = str(root / "existing-repository") +missing_path = str(root / "deleted-repository") +connection.execute( + "INSERT INTO security_targets (id, current_path, display_name, created_at, updated_at) VALUES (?, ?, ?, ?, ?)", + ("target-existing", existing_path, "existing-repository", timestamp, timestamp), +) +connection.executemany( + "INSERT INTO workspaces (id, target_path, target_id, created_at, updated_at) VALUES (?, ?, ?, ?, ?)", + [ + ("workspace-existing", existing_path, "target-existing", timestamp, timestamp), + ("workspace-empty", None, None, timestamp, timestamp), + ], +) + +def insert_scan(scan_id, workspace_id, target_path, target_id): + connection.execute( + "INSERT INTO scans (id, workspace_id, target_path, target_id, target_revision, scope, mode, scan_dir, status, phase, started_at, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)", + ( + scan_id, + workspace_id, + target_path, + target_id, + "synthetic-revision", + ".", + "standard", + str(root / scan_id), + "complete", + "reporting", + timestamp, + timestamp, + timestamp, + ), + ) + +insert_scan("scan-existing", "workspace-existing", existing_path, "target-existing") + +if scenario == "orphan-scan": + insert_scan("scan-orphan", "workspace-existing", existing_path, None) + expected_target_id = "target-existing" + orphan_path = existing_path +elif scenario in ("dangling-scan", "dangling-workspace", "dangling-targets"): + connection.commit() + connection.execute("PRAGMA foreign_keys = OFF") + if scenario != "dangling-scan": + connection.execute( + "INSERT INTO workspaces (id, target_path, target_id, created_at, updated_at) VALUES (?, ?, ?, ?, ?)", + ("workspace-orphan", missing_path, "target-dangling", timestamp, timestamp), + ) + if scenario != "dangling-workspace": + workspace = ( + "workspace-existing" if scenario == "dangling-scan" else "workspace-orphan" + ) + insert_scan("scan-orphan", workspace, missing_path, "target-dangling") + connection.commit() + connection.execute("PRAGMA foreign_keys = ON") + expected_target_id = stable_target_id(Path(missing_path)) + orphan_path = missing_path +else: + connection.execute( + "INSERT INTO workspaces (id, target_path, target_id, created_at, updated_at) VALUES (?, ?, ?, ?, ?)", + ("workspace-orphan", missing_path, None, timestamp, timestamp), + ) + expected_target_id = stable_target_id(Path(missing_path)) + orphan_path = missing_path + if scenario == "orphan-workspace-and-scan": + insert_scan("scan-orphan", "workspace-orphan", missing_path, None) + +connection.commit() +violations_before_repair = len(connection.execute("PRAGMA foreign_key_check").fetchall()) +apply_migrations(connection, MIGRATIONS, lambda: timestamp, backfill) +apply_migrations(connection, MIGRATIONS, lambda: timestamp, backfill) + +def target_id(table, row_id): + row = connection.execute( + f"SELECT target_id FROM {table} WHERE id = ?", (row_id,) + ).fetchone() + return row["target_id"] if row is not None else None + +print(json.dumps({ + "backfillCalls": len(backfill_calls), + "emptyWorkspaceTargetId": target_id("workspaces", "workspace-empty"), + "existingScanTargetId": target_id("scans", "scan-existing"), + "existingWorkspaceTargetId": target_id("workspaces", "workspace-existing"), + "expectedTargetId": expected_target_id, + "foreignKeysEnforced": bool(connection.execute("PRAGMA foreign_keys").fetchone()[0]), + "foreignKeyViolationsBeforeRepair": violations_before_repair, + "foreignKeyViolationsAfterRepair": len( + connection.execute("PRAGMA foreign_key_check").fetchall() + ), + "migrationRecorded": connection.execute( + "SELECT name FROM schema_migrations WHERE version = 16" + ).fetchone()[0] == "stable repository targets", + "orphanPathExists": Path(orphan_path).exists(), + "orphanScanTargetId": target_id("scans", "scan-orphan"), + "orphanWorkspaceTargetId": target_id("workspaces", "workspace-orphan"), + "targetCount": connection.execute( + "SELECT count(*) FROM security_targets" + ).fetchone()[0], +})) +`; + +const repositoryIdentityMigrationProbe = String.raw` +import json +import sqlite3 +import sys + +sys.path.insert(0, sys.argv[1]) + +from workbench_schema import MIGRATIONS, apply_migrations +from workbench_target_state import backfill_security_targets + +scenario = sys.argv[2] +timestamp = "2026-08-01T00:00:00Z" +backfill_calls = [] + +def backfill(connection): + backfill_calls.append(True) + backfill_security_targets(connection) + +connection = sqlite3.connect(":memory:") +connection.row_factory = sqlite3.Row +historical_migration = next( + (migration for migration in MIGRATIONS if migration[0] == 29), None +) +historical = tuple( + migration for migration in MIGRATIONS if migration[0] < 30 and migration[0] != 29 +) +apply_migrations(connection, historical, lambda: timestamp, backfill) +backfill_calls.clear() +connection.execute( + "INSERT INTO security_targets (id, current_path, display_name, created_at, updated_at) VALUES (?, ?, ?, ?, ?)", + ("target-existing", "/synthetic/deleted-repository", "repository", timestamp, timestamp), +) + +if scenario == "recorded-without-column": + connection.execute( + "INSERT INTO schema_migrations VALUES (?, ?, ?)", + (30, "persist repository identities", timestamp), + ) +elif scenario == "recorded-without-index": + connection.execute( + "ALTER TABLE security_targets ADD COLUMN repository_identity TEXT" + ) + connection.execute( + "INSERT INTO schema_migrations VALUES (?, ?, ?)", + (30, "persist repository identities", timestamp), + ) + +connection.commit() +migrations = MIGRATIONS +if scenario == "out-of-order-historical-migration": + without_historical = tuple( + migration for migration in MIGRATIONS if migration[0] != 29 + ) + apply_migrations(connection, without_historical, lambda: timestamp, backfill) + migrations = tuple(sorted( + ( + *without_historical, + historical_migration or ( + 29, + "synthetic historical migration", + """ + CREATE TABLE historical_migration_fixture ( + id INTEGER PRIMARY KEY, + value TEXT + ); + + CREATE INDEX historical_migration_fixture_by_value + ON historical_migration_fixture(value); + """, + ), + ), + key=lambda migration: migration[0], + )) +apply_migrations(connection, migrations, lambda: timestamp, backfill) +apply_migrations(connection, migrations, lambda: timestamp, backfill) + +columns = { + row["name"]: row + for row in connection.execute("PRAGMA table_info(security_targets)") +} +indexes = { + row["name"]: row + for row in connection.execute("PRAGMA index_list(security_targets)") +} +print(json.dumps({ + "backfillCalls": len(backfill_calls), + "hasRepositoryIdentityColumn": "repository_identity" in columns, + "hasRepositoryIdentityIndex": "security_targets_by_repository_identity" in indexes, + "historicalMigrationApplied": connection.execute( + "SELECT 1 FROM schema_migrations WHERE version = 29" + ).fetchone() is not None, + "historicalMigrationInSource": historical_migration is not None, + "repositoryIdentityColumnIsNullable": not bool( + columns["repository_identity"]["notnull"] + ), + "repositoryIdentityIndexIsUnique": bool( + indexes["security_targets_by_repository_identity"]["unique"] + ), + "migrationName": connection.execute( + "SELECT name FROM schema_migrations WHERE version = 30" + ).fetchone()[0], + "targetId": connection.execute( + "SELECT id FROM security_targets" + ).fetchone()[0], +})) +`; + +describe("stable workbench target migration", () => { + test.each([ + ["orphan-scan", "reuses an existing target for an orphaned scan"], + ["orphan-workspace", "repairs a workspace without an orphaned scan"], + ["dangling-scan", "repairs a dangling scan foreign key independently"], + [ + "dangling-workspace", + "repairs a dangling workspace foreign key independently", + ], + [ + "dangling-targets", + "repairs dangling workspace and scan foreign keys atomically", + ], + [ + "orphan-workspace-and-scan", + "repairs a workspace and scan after their repository is deleted", + ], + ] as const)("%s: %s", (scenario) => { + const python = + Bun.which("python3") ?? Bun.which("python") ?? Bun.which("py"); + expect(python).not.toBeNull(); + if (python === null) throw new Error("A Python interpreter is required."); + + const execution = spawnSync( + python, + [ + "-I", + "-B", + "-c", + migrationProbe, + join(PLUGIN_ROOT, "scripts"), + scenario, + join(tmpdir(), "codex-security-stable-target-migration", scenario), + ], + { encoding: "utf8", timeout: 10_000 }, + ); + + expect(execution.status, execution.stderr).toBe(0); + expect(execution.stderr).toBe(""); + + const result = JSON.parse(execution.stdout) as { + backfillCalls: number; + emptyWorkspaceTargetId: string | null; + existingScanTargetId: string; + existingWorkspaceTargetId: string; + expectedTargetId: string; + foreignKeysEnforced: boolean; + foreignKeyViolationsBeforeRepair: number; + foreignKeyViolationsAfterRepair: number; + migrationRecorded: boolean; + orphanPathExists: boolean; + orphanScanTargetId: string | null; + orphanWorkspaceTargetId: string | null; + targetCount: number; + }; + + expect(result).toMatchObject({ + backfillCalls: 1, + emptyWorkspaceTargetId: null, + existingScanTargetId: "target-existing", + existingWorkspaceTargetId: "target-existing", + foreignKeysEnforced: true, + foreignKeyViolationsBeforeRepair: + scenario === "dangling-targets" + ? 2 + : scenario.startsWith("dangling-") + ? 1 + : 0, + foreignKeyViolationsAfterRepair: 0, + migrationRecorded: true, + orphanPathExists: false, + targetCount: scenario === "orphan-scan" ? 1 : 2, + }); + expect(result.orphanScanTargetId).toBe( + scenario === "orphan-workspace" || scenario === "dangling-workspace" + ? null + : result.expectedTargetId, + ); + expect(result.orphanWorkspaceTargetId).toBe( + scenario === "orphan-scan" || scenario === "dangling-scan" + ? null + : result.expectedTargetId, + ); + }); + + test.each([ + [ + "unapplied", + "applies and backfills the new repository-identity migration", + ], + [ + "recorded-without-column", + "repairs a recorded migration missing its column and index", + ], + [ + "recorded-without-index", + "repairs a recorded migration missing only its index", + ], + [ + "out-of-order-historical-migration", + "applies an unavailable migration after a newer migration is recorded", + ], + ] as const)("%s: %s", (scenario) => { + const python = + Bun.which("python3") ?? Bun.which("python") ?? Bun.which("py"); + expect(python).not.toBeNull(); + if (python === null) throw new Error("A Python interpreter is required."); + + const execution = spawnSync( + python, + [ + "-I", + "-B", + "-c", + repositoryIdentityMigrationProbe, + join(PLUGIN_ROOT, "scripts"), + scenario, + ], + { encoding: "utf8", timeout: 10_000 }, + ); + + expect(execution.status, execution.stderr).toBe(0); + expect(execution.stderr).toBe(""); + const result = JSON.parse(execution.stdout) as { + backfillCalls: number; + hasRepositoryIdentityColumn: boolean; + hasRepositoryIdentityIndex: boolean; + historicalMigrationApplied: boolean; + historicalMigrationInSource: boolean; + migrationName: string; + repositoryIdentityColumnIsNullable: boolean; + repositoryIdentityIndexIsUnique: boolean; + targetId: string; + }; + expect(result).toEqual({ + backfillCalls: 1, + hasRepositoryIdentityColumn: true, + hasRepositoryIdentityIndex: true, + historicalMigrationApplied: + scenario === "out-of-order-historical-migration" || + result.historicalMigrationInSource, + historicalMigrationInSource: result.historicalMigrationInSource, + migrationName: "persist repository identities", + repositoryIdentityColumnIsNullable: true, + repositoryIdentityIndexIsUnique: false, + targetId: "target-existing", + }); + }); +});