diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md index 6e96e357d..7e0b69118 100644 --- a/DEVELOPER_GUIDE.md +++ b/DEVELOPER_GUIDE.md @@ -1763,7 +1763,7 @@ access. | Ad-hoc search SARIF | `search --format sarif` stores completion metadata on each SARIF run. The run and its single `queries[]` summary report `source_result_count`, `source_result_count_authoritative`, emitted `result_count`, the applied `limit_per_query` / `result_limit`, conservative `minimum_omitted_result_count`, and `truncated` state. Source and emitted counts use the final SARIF result/location unit, including exact-search occurrence expansion. Guarded searches retain their bounded candidate budget instead of failing during a completion recount; their source count is an explicitly non-authoritative lower bound and their truncation state remains conservative. Facet-filtered exact searches use an exhaustive source count rather than the display candidate window. Ad-hoc search does not expose a continuation cursor, so `cursoring_available` is `false` and `next_cursor` is null; a shell-quoted `replay_command` preserves option-like queries and active search controls. The completion vocabulary intentionally matches recipe SARIF, and empty runs carry the same fields with zero counts. | | Ad-hoc issue-draft selection | `search --format issue-drafts` reads the complete filtered ad-hoc population, then applies `--first-per-file`, deterministic `--sample`, and `min(--limit, --total-limit)` in that order. Guarded searches retain their finite candidate inspection contract: `source_total_count` is omitted, `source_minimum_count` reports the observed lower bound, `source_total_count_authoritative=false`, `source_fetch_limit` reports the bounded fetch, and `truncated=true` preserves incomplete-population state. Existing `result_count`, `result_limit`, `omitted_count`, and `truncated` fields describe the returned selection accurately; additive `source_total_count`, `returned_count`, `limit_per_query`, `total_limit`, `first_per_file`, and `sample` fields make the applied contract auditable. Replay commands are serialized from normalized parsed options, use POSIX-safe single-quote escaping, and retain raw/exact/prefix modes, path/language/facet/guard filters, selection controls, evidence formatting, duplicate preflight, and issue hints. | | Recipe SARIF | `search --recipe --format sarif` emits one result per bounded recipe result. Rule IDs use `recipe/query`; standard `fingerprints.cdidx/v1` values are derived from the normalized source location; result properties preserve recipe/query identity, severity, confidence, and per-query truncation; run properties preserve scope, applied result limits, aggregate counts, and conservative omitted-result metadata. `--max-json-bytes` preflights the complete schema-valid document and exact UTF-8 byte count, including escaping and the final newline, through a counting writer before materializing only the selected prefix. If the full document does not fit, serialization omits only whole trailing results and adds run/query source, emitted, omitted, byte-strategy, and replay metadata; emitted rules and locations remain intact. This truncation exits `11` unless `--allow-partial` is set. A cap below the zero-result document minimum emits no SARIF and reports the required bytes; explicit `--json` may place a versioned error object on stdout when that object fits the cap. Replay metadata removes the byte cap when a complete report exceeds the parser's maximum supported cap. Bound SARIF with `--limit` / `--total-limit`; row selectors such as `--sample`, `--first-per-file`, and `--per-file-limit` are rejected instead of being silently ignored. Recipe severity maps `critical` / `high` to `error`, `medium` to `warning`, and `low` / `info` to `note`. | -| Recipe classifier output | Recipe run JSON may add `audit_classifications` to individual `CompactSearchResult` rows when a recipe classifier can classify the hit, and query/count payloads may add `classifier_counts` when classified rows are present. These fields are additive; use them to separate triage domains such as DTO/result-wrapper `.Result` properties versus Task/ValueTask blocking waits without changing the raw search query. | +| Recipe classifier output | Recipe run JSON may add `audit_classifications` to individual `CompactSearchResult` rows when a recipe classifier can classify the hit, and query/count payloads may add `classifier_counts` when classified rows are present. These fields are additive; use them to separate triage domains such as DTO/result-wrapper `.Result` properties versus Task/ValueTask blocking waits without changing the raw search query. JSON read/write recipes also classify source-proximate `cdidx-audit: json-trust` annotations by origin, direction, sensitivity, trust, and rationale. Classification reads bounded indexed source rather than the projected snippet, including for guard-projected rows, and lexically verifies a real C# line comment so regular/verbatim/raw string contents and conditional-compilation regions cannot provide trust evidence. It evaluates every retained match site after overlap deduplication, collapses declaration-type facets—including expression-bodied method/local-function return types and generic return types split before the audited type—when the same containing statement has a later constructor facet across the same or following lines, consumes each annotation at the first lexical audited match across all selected JSON child queries, and binds it only to the next operation by parsing C# tokens. Annotation lookup searches the bounded indexed prefix rather than a fixed line gap. Nullable declarations, direct casts, nested-generic first arguments, and declaration-resolved direct receivers remain valid while an earlier statement, evaluated operand, indexer target, unresolved bare receiver, one-hop/chained property-valued assignment or invocation receiver, preprocessor directive, completed expression, control-flow block, or comma-separated operation is rejected as `not_adjacent`. Rows with distinct evidence remain conservatively `mixed_boundaries`; missing, invalid, direction-mismatched, or `review_required` evidence remains `ambiguous_trust`. The classifier groups rows by file and reconstructs the maximum required bounded prefix once; its per-query lexical cache retains only that one file prefix, records the prefix actually reconstructed, and remembers source-budget exhaustion so high-line matches neither repeat reconstruction nor poison a lower-line result. It never removes the raw result. Source-backed classification runs only for JSON/NDJSON/compact/count-JSON shapes that serialize row classifications or classifier counts; text, scalar count, compact summary, SARIF, issue-draft, and `--search-fields` projection paths skip it. | | NDJSON terminal records | Default NDJSON for `search`, `symbols`, and `files` appends one final `terminal_record` after result rows; search also emits it for zero-result responses, while raw `symbols` and `files` keep zero-result NDJSON empty. Recipe/audit search row streams share the same writer. Terminals report returned and observed total counts, `total_count_authoritative` / `total_count_lower_bound`, selection or interruption reason, applied limits, omitted rows, and recovery guidance. `--max-json-bytes` covers the complete stdout stream, including newlines and this terminal record; when additive selector-accounting fields prevent the terminal from fitting, the writer omits those optional fields before declaring the terminal impossible. A cap that still cannot fit the terminal fails before stdout. Capped output rejects `--profile`, `--verbose`, and `--json-envelope`. Byte-cap partial output exits with `CommandExitCodes.PartialResult` (`11`) unless `--allow-partial` explicitly opts into exit `0`. `--results-only` is the explicit terminal-record opt-out for these NDJSON row streams and is rejected with array, compact, summary, or count output. | | C# outline callable display | `DbSymbolReader.Outline` derives `display_name` only at read time and never changes canonical `symbols.name`, qualified paths, folded identity, or exact-query aliases. A complete C# generic method signature normally uses arity placeholders (`` or ``); if one would collide with a concrete parameter type, it deterministically selects collision-free `TArg` placeholders. Replacement applies only to unqualified method-type-parameter references, preserving qualified concrete types and escaped-keyword distinctions. Literal-aware scanning keeps delimiters in attributes and default values from changing parameter boundaries. The display omits `where` constraints, drops non-identity `this` / `params` / `scoped`, and retains overload-significant `ref` / `out` / `in` (including `ref readonly`). Non-generic and non-C# formatting stays on the existing path. Missing, truncated, or syntactically incomplete persisted signatures retain the legacy `Name@line` fallback for old-index compatibility. | | `outline` / `unused` cursor binding | `outline --json` accepts `--kind `, `--limit` / `--top`, opaque `--cursor `, and `--outline-fields ` for bounded machine output. Controlled outline responses keep the normal envelope and add `total_symbol_count`, `returned_symbol_count`, `cursor_offset`, `next_cursor`, `has_more`, and `result_stable_at`, plus `kind_filter` and `selected_fields` when active. Projection parsing canonicalizes aliases and removes duplicates before validation; unknown field names are reported together as one terminal usage error with valid candidates, while the empty-selection error is reserved for deliberately empty CSV input. `outline` and `unused` cursors bind their offset to the normalized path/scope, filters, ordering, and index generation; reuse after changing those inputs or refreshing the index fails with explicit restart-required guidance. Legacy `outline:` / `unused:` inputs remain accepted for transition, but every newly emitted cursor is opaque and bound. | @@ -5238,7 +5238,7 @@ help はすべてこのレジストリを参照します。field 名は大文字 | ad-hoc search SARIF | `search --format sarif` は completion metadata を SARIF の各 run に格納します。run と単一の `queries[]` summary は `source_result_count`、`source_result_count_authoritative`、出力済み `result_count`、適用された `limit_per_query` / `result_limit`、保守的な `minimum_omitted_result_count`、`truncated` state を返します。source / emitted count は exact search の occurrence 展開を含む最終的な SARIF result / location 単位を使用します。guard 付き search は completion の再計数で失敗せず bounded candidate budget を維持し、source count を明示的に non-authoritative な lower bound として返して truncation state を保守的に保ちます。facet filter 付き exact search は表示用 candidate window ではなく exhaustive な source count を使います。ad-hoc search は継続 cursor を公開しないため、`cursoring_available` は `false`、`next_cursor` は null となり、shell quote 済みの `replay_command` が option のような query と有効な search control を保持します。completion vocabulary は意図的に recipe SARIF と共通化し、空 run も count が 0 の同じ field を保持します。 | | ad-hoc issue-draft selection | `search --format issue-drafts` は filter 済みの ad-hoc 母集団全体を読み、`--first-per-file`、決定的な `--sample`、`min(--limit, --total-limit)` の順に適用します。guard 付き検索は有限の candidate inspection 契約を維持し、`source_total_count` を省略し、観測下限を `source_minimum_count`、非 authoritative 状態を `source_total_count_authoritative=false`、bounded fetch を `source_fetch_limit` で報告し、母集団が未完了であることを `truncated=true` で保持します。既存の `result_count`、`result_limit`、`omitted_count`、`truncated` field は返却 selection を正確に表し、additive な `source_total_count`、`returned_count`、`limit_per_query`、`total_limit`、`first_per_file`、`sample` field により適用済み契約を監査できます。replay command は正規化済み parse option から serialize し、POSIX-safe な単一引用符 escape を使い、raw / exact / prefix mode、path / language / facet / guard filter、selection control、evidence formatting、duplicate preflight、issue hint を維持します。 | | Recipe SARIF | `search --recipe --format sarif` は、上限付き recipe result ごとに result を1件出力します。rule ID は `recipe/query` を使い、標準の `fingerprints.cdidx/v1` は正規化済み source location から導出します。result properties は recipe/query identity、severity、confidence、query ごとの truncation を保持し、run properties は scope、適用済み result limit、集計 count、保守的な omitted-result metadata を保持します。`--max-json-bytes` は escape と末尾改行を含む schema-valid な完全 document と正確な UTF-8 byte 数を counting writer で検査し、選択した prefix だけを materialize します。完全な document が収まらない場合、末尾の result だけを1件単位で省略し、run / query に source、emitted、omitted、byte strategy、replay metadata を追加します。出力済み result の rule と location は維持し、この truncation は `--allow-partial` がなければ `11` を返します。result 0件の document の最小値より小さい cap では SARIF を出力せず必要 byte 数を報告し、明示的な `--json` では error object 自体が cap に収まる場合に version 付き error を stdout へ出力することがあります。完全な report が parser の対応可能な最大 cap を超える場合、replay metadata は byte cap を外します。SARIF の上限には `--limit` / `--total-limit` も使い、`--sample`、`--first-per-file`、`--per-file-limit` のような row selector は黙って無視せず拒否します。recipe severity は `critical` / `high` を `error`、`medium` を `warning`、`low` / `info` を `note` に対応付けます。 | -| Recipe classifier output | recipe classifier が hit を分類できる場合、recipe run JSON は個別の `CompactSearchResult` row に `audit_classifications` を追加することがあり、分類済み row がある query / count payload は `classifier_counts` を追加することがあります。これらは additive field です。raw search query を変えずに、DTO / result-wrapper の `.Result` property と Task / ValueTask の blocking wait などの triage domain を分離するために使います。 | +| Recipe classifier output | recipe classifier が hit を分類できる場合、recipe run JSON は個別の `CompactSearchResult` row に `audit_classifications` を追加することがあり、分類済み row がある query / count payload は `classifier_counts` を追加することがあります。これらは additive field です。raw search query を変えずに、DTO / result-wrapper の `.Result` property と Task / ValueTask の blocking wait などの triage domain を分離するために使います。JSON の read / write recipe は source-proximate な `cdidx-audit: json-trust` 注釈も origin、direction、sensitivity、trust、rationale で分類します。分類は guard により1行へ投影された row を含め、投影済み snippet ではなく上限付きの indexed source を読み、実際の C# line comment であることを lexical に検証するため regular / verbatim / raw string の内容や条件コンパイル領域を trust evidence にしません。overlap dedup 後に残った各 match site を評価し、expression-bodied method / local function の戻り型、および audit 対象型より前で改行された generic 戻り型を含め、同じ containing statement の同一行または後続行に constructor facet がある宣言型 facet は畳み込み、選択されたすべての JSON child query を横断した最初の lexical な audit 対象 match で各注釈を消費します。注釈探索は固定の行差ではなく上限付き indexed prefix を検索します。C# token 解析では nullable 宣言、直接 cast、nested-generic の first argument、宣言から解決できる直接 receiver は有効なまま維持する一方、途中の実行コード、先行 statement、評価済み operand、indexer 代入先、解決不能な単純名 receiver、1段 / 連鎖した property-valued な代入 / 呼び出し receiver、preprocessor directive、完了済み expression、control-flow block、カンマ区切りの操作がある注釈は `not_adjacent` として次の操作へ流用しません。異なる evidence を持つ row は保守的に `mixed_boundaries` とし、注釈の欠落、不正、direction 不一致、`review_required` を `ambiguous_trust` とします。分類は row を file ごとにまとめて最大必要行までの上限付き prefix を1回復元します。query ごとの lexical cache はその file prefix 1件だけを保持し、実際に復元できた行数と source budget の枯渇を記録するため、高い行で再構築を繰り返さず、低い行の結果も汚染しません。raw result は削除しません。source-backed 分類は row classification または classifier count を serialize する JSON / NDJSON / compact / count JSON shape だけで実行し、text、scalar count、compact summary、SARIF、issue-draft、`--search-fields` projection path では省略します。 | | NDJSON terminal record | `search`、`symbols`、`files` の既定 NDJSON は result row の後に最後の `terminal_record` を 1 件追加します。`search` は 0 件応答にも終端を出力しますが、raw `symbols` / `files` の 0 件 NDJSON は空のままです。recipe / audit search の row stream も同じ writer を使います。終端は返却件数と観測済み総件数、`total_count_authoritative` / `total_count_lower_bound`、selection または中断理由、適用上限、省略行数、復旧案内を報告します。`--max-json-bytes` は改行と終端レコードを含む stdout stream 全体を対象にし、追加 selector-accounting field が原因で終端が収まらない場合は、終端自体を不可能と判定する前にそれらの任意 field を省略します。それでも終端が収まらない cap は stdout 出力前に失敗します。上限付き出力は `--profile`、`--verbose`、`--json-envelope` を拒否します。byte cap による部分出力は、`--allow-partial` で終了コード `0` を明示許可しない限り `CommandExitCodes.PartialResult`（`11`）を返します。`--results-only` はこれらの NDJSON row stream から終端レコードを明示的に除外するための option であり、array / compact / summary / count 出力との組み合わせは拒否されます。 | | C# outline callable 表示 | `DbSymbolReader.Outline` は read 時にだけ `display_name` を導出し、canonical な `symbols.name`、qualified path、folded identity、完全一致 query alias は変更しません。完全な C# generic method signature は通常 arity placeholder（`` または ``）で表示し、具体的な parameter type と衝突する場合は決定的で衝突しない `TArg` placeholder を選びます。置換対象は修飾されていない method type parameter の参照だけであり、修飾された具体型と escaped keyword の区別は保持します。literal-aware な走査により、attribute と既定値内の区切り文字は parameter 境界を変えません。`where` constraint と identity に影響しない `this` / `params` / `scoped` は省略し、overload を区別する `ref` / `out` / `in`（`ref readonly` を含む）は保持します。非 generic および C# 以外の formatting は既存経路のままです。永続 signature が欠落、切り詰め、または構文的に不完全な場合は、旧 index 互換のため legacy `Name@line` fallback を維持します。 | | `outline` / `unused` cursor の束縛 | `outline --json` は bounded な機械向け出力として `--kind `、`--limit` / `--top`、opaque な `--cursor `、`--outline-fields ` を受け付けます。制御付き outline 応答は通常の envelope を維持し、`total_symbol_count`、`returned_symbol_count`、`cursor_offset`、`next_cursor`、`has_more`、`result_stable_at` を追加し、該当時は `kind_filter` と `selected_fields` も返します。projection parser は検証前に alias を canonicalize して重複を除きます。未知field名はvalid候補を伴う1つの終端usage errorにまとめ、empty-selection errorは意図的に空のCSV入力にだけ使います。`outline` と `unused` の cursor は offset を正規化済み path/scope、filter、ordering、index generation に束縛するため、条件変更後または index 更新後の再利用は restart-required の明示案内付きで失敗します。移行用に legacy の `outline:` / `unused:` 入力は受理しますが、新しく出力する cursor はすべて opaque かつ束縛済みです。 | diff --git a/TESTING_GUIDE.md b/TESTING_GUIDE.md index 078164245..f8aad29e1 100644 --- a/TESTING_GUIDE.md +++ b/TESTING_GUIDE.md @@ -230,6 +230,8 @@ Use `docs/test-doc-maintenance-plan.md` before moving oversized suites or adding Command-specific output format coverage uses a command/format matrix that checks both parser acceptance and the matching usage line; recognized shared formats without a command implementation need a separate usage-error assertion. Ad-hoc search SARIF completion coverage shares one fixture across complete, 1-of-126 limited, facet-filtered occurrence-expanded limited, bounded guarded, empty, and synthetically merged multi-run documents. Assert source/emitted/omitted counts and source-count authority in SARIF result units, applied limits, conservative truncation, null cursor state, raw-FTS and option-like-query replay commands, guard-preserving replay, and unchanged rule/location/severity fields on every run. Recipe SARIF coverage must assert bounded result counts, `recipe/query` rule identity, source locations, severity mapping, confidence, conservative truncation metadata, stable `fingerprints.cdidx/v1` values across identical runs, and the same `query_freshness` run properties as aggregate JSON. Query-freshness coverage must keep successful matched and zero-match executions separate from stale index/recipe/query versions and invalid or missing child executions, preserve the compatibility cardinality fields, and reconcile clean/stale/invalid state counts in mixed runs. Byte-budget coverage must count the complete UTF-8 stdout including JSON escaping and the final newline, exercise exact-fit and one-byte-under boundaries, Unicode, empty and multi-query runs, an individually oversized result, captured/redirected stdout, and replay metadata. Every successful output must parse as complete SARIF, omit only whole results, retain matching rules and locations, and stay within the requested cap. Below-minimum failures must emit no SARIF; non-explicit JSON failures leave stdout empty, while explicit `--json` may emit a bounded versioned error object. Also cover counting-writer measurement and replay recovery when the complete size exceeds the maximum accepted byte cap. + JSON trust-boundary classifier coverage must keep controlled private-local writers, public writers, a review-required public writer, a same-chunk row with distinct private/public boundaries, annotations followed by two adjacent operations on separate lines and by an earlier statement, evaluated arithmetic/comparison operand, a conditional branch followed by a nested assignment, indexer assignment target, unresolved bare receiver, one-hop/chained property-valued assignment or invocation receiver, conditional-compilation directive, control-flow block, or comma-separated operation on the matched line, nullable-declaration, direct-cast, first-named-argument, nested-generic-first-argument, expression-bodied method with a default parameter, and declaration-resolved direct invocation-receiver positive controls, explicit `Utf8JsonWriter` declaration-plus-constructor and expression-bodied return-type-plus-constructor controls, missing/invalid/direction-mismatched annotations, annotation-shaped inline/verbatim/raw string contents, and untrusted external/network/file parsers in one indexed fixture. Run with a one-line projected snippet and guard-projected controls to prove classification always uses bounded indexed source, assert per-row origin/direction/trust evidence and per-category counts, verify an annotation cannot bleed past intervening executable code, a nested second occurrence, or a later audited API family selected through another child query on either earlier or matched lines, verify review-required and mixed rows remain conservatively ambiguous, verify lexical string and conditional-compilation contents stay ambiguous, and verify that untrusted parser findings are still returned rather than suppressed. Add overlapping-chunk dedup coverage where stale facets carry a removed public site but only the retained private line may affect classification. Keep the lexical source cache bounded independently of matching file count; group same-file rows and prefetch their maximum required line once, then add an order-regression where multiple over-budget high-line rows are ranked before a classifiable lower-line row so exhausted-prefix caching and lower-line reuse remain covered. Keep source-backed classification out of text/scalar-count/compact-summary/SARIF/issue-draft/`--search-fields` paths that do not serialize it. + Keep long multiline declaration/assignment continuations, declaration types split from their constructors, fully qualified local declaration types, generic-wrapper expression-bodied return types whose audited type occupies a non-terminal argument, and generic return types split before the audited type as positive JSON trust-boundary controls so formatting, qualification, and wrapper shape cannot change annotation binding. Keep overlapping child-query substrings on one call as a negative reuse control so only the first lexical match consumes the annotation. MCP schema-origin coverage keeps identical audit phrases in `McpToolCatalog.cs` top-level tool descriptions, concatenated description segments, nested schema-property prose, and executable C# in one indexed fixture; assert explicit `schema_description` search metadata and the recipe's JSON, SARIF, and issue-draft outputs so origin filtering cannot drift across projections. Recipe row-selection coverage reuses one multi-file, multi-chunk fixture across aggregate JSON, compact JSON, NDJSON, and issue-draft source metadata. Assert emitted/matched/omitted counts, `selection_reason` / `selection_omitted_count`, first-per-file path uniqueness, selector-preserving replay commands, suppressed raw cursors when a later limit truncates selected rows, and rejection of incoming cursors with either selector. A separate candidate-window fixture must exceed the default low-limit fetch envelope and prove that `--sample ` observes at least its requested candidate target; validate rejected selectors for non-row recipe shapes without opening a database. Unused default-suppression row, JSON count, summary-only, and text count envelopes, including the `--all` count control, share one unused-symbol fixture. @@ -1192,6 +1194,8 @@ dotnet test --filter "FullyQualifiedName~GitHelperTests" コマンド別の出力形式 coverage は command / format matrix で parser の受理と対応する usage line の両方を検証してください。共通 parser が認識してもコマンド側に実装がない形式には、別途 usage error の assertion が必要です。 ad-hoc search SARIF の completion coverage は complete、1-of-126 の limited、facet filter 付き occurrence 展開後の limited、bounded guard、empty、合成した multi-run document で1つの fixture を共有します。SARIF result 単位の source / emitted / omitted count と source count の確定性、適用済み limit、保守的な truncation、null cursor state、raw FTS と option のような query の replay command、guard を保持する replay、および各 run で rule / location / severity field が不変であることを検証してください。 Recipe SARIF coverage では、上限付き result count、`recipe/query` rule identity、source location、severity mapping、confidence、保守的な truncation metadata、同一 run 間で安定する `fingerprints.cdidx/v1`、aggregate JSON と同じ `query_freshness` run properties を検証してください。query freshness coverage では、成功した matched / zero-match execution を stale な index / recipe / query version および invalid / missing child execution と分離し、互換用の件数フィールドを維持し、mixed run の clean / stale / invalid state count が整合することを検証してください。byte-budget coverage では JSON escape と末尾改行を含む完全な UTF-8 stdout を数え、exact-fit と1 byte不足の境界、Unicode、空 run と複数 query の run、単体で oversized な result、capture / redirect した stdout、replay metadata を扱ってください。成功した出力はすべて完全な SARIF として parse でき、result を1件単位でのみ省略し、対応する rule / location を維持し、要求 cap 以下でなければなりません。最小値未満の失敗では SARIF を出力せず、明示 JSON でない失敗は stdout を空にし、明示的な `--json` では上限内の version 付き error object を出力できることも検証してください。counting writer による計測と、完全な size が受理可能な最大 byte cap を超える場合の replay recovery も扱ってください。 + JSON trust-boundary classifier の coverage は、controlled な private-local writer、public writer、review-required な public writer、同一 chunk 内に異なる private / public boundary を持つ row、別々の行で隣接した2操作に続く注釈、および match と同じ行の先行 statement、評価済みの算術 / 比較 operand、条件分岐に続く nested 代入、indexer 代入先、解決不能な単純名 receiver、1段 / 連鎖した property-valued な代入 / 呼び出し receiver、条件コンパイル directive、control-flow block、カンマ区切りの操作に続く注釈、nullable 宣言、直接 cast、first named argument、nested-generic の first argument、default parameter を持つ expression-bodied method、宣言から解決できる直接 invocation receiver の positive control、明示的な `Utf8JsonWriter` の宣言と constructor、および expression-bodied な戻り型と constructor の control、注釈の欠落 / 不正 / direction 不一致、注釈形式の inline / verbatim / raw string 内容、untrusted な external / network / file parser を1つの indexed fixture に含めます。投影 snippet を1行にした case と guard により、分類が常に上限付きの indexed source を使うことを証明します。row ごとの origin / direction / trust evidence と category ごとの count、以前の行または match と同じ行にある実行コード、nested した2番目の occurrence、または別の child query で選択された後続の audit 対象 API family を越えて注釈が流用されないこと、review-required / mixed row が保守的に ambiguous のままであること、lexical な string と条件コンパイル領域の内容が ambiguous のままであることを検証し、untrusted parser finding が抑制されず返却されることを確認してください。overlap chunk の dedup coverage では、削除済み public site を stale facet が保持していても、残った private line だけが分類へ影響することを検証します。lexical source cache は match する file 数に依存しない上限を維持し、同じ file の row をまとめて最大必要行を1回 prefetch します。そのうえで、複数の上限超過 high-line row を分類可能な lower-line row より先に rank する順序回帰により、枯渇済み prefix の cache と lower-line reuse の両方を検証します。source-backed 分類を serialize しない text / scalar count / compact summary / SARIF / issue-draft / `--search-fields` path では実行しないでください。 + 長い複数行へ継続する宣言 / 代入、constructor と別行に分割された宣言型、完全修飾された local 宣言型、audit 対象型が末尾以外の型引数にある generic wrapper を持つ expression-bodied な戻り型、および audit 対象型より前で改行された generic 戻り型を JSON trust-boundary の positive control とし、formatting、修飾、wrapper の形によって注釈の束縛が変わらないことを維持してください。同じ呼び出しで重なる child-query substring は negative reuse control とし、最初の lexical match だけが注釈を消費することを維持してください。 MCP schema-origin coverage では、同一の audit phrase を `McpToolCatalog.cs` の top-level tool description、連結された description segment、nested schema property の prose、実行可能な C# に置いた1つの indexed fixture を共有し、明示的な `schema_description` 検索 metadata と recipe の JSON、SARIF、issue-draft 出力を検証して、projection 間で origin filter が drift しないようにしてください。 recipe row-selection coverage は aggregate JSON、compact JSON、NDJSON、issue-draft の source metadata で1つの multi-file / multi-chunk fixture を共有します。emitted / matched / omitted count、`selection_reason` / `selection_omitted_count`、first-per-file の path uniqueness、selector を保持する replay command、後続 limit が選択済み row を truncate する場合の raw cursor 抑止、両 selector と受け取った cursor の併用拒否を検証してください。別の candidate-window fixture では既定の low-limit fetch envelope を超え、`--sample ` が少なくとも要求 candidate 数を観測することを証明し、row を持たない recipe shape での selector 拒否は database を開かずに確認してください。 unused default-suppressionのrow、JSON count、summary-only、text count envelopeは、`--all` count controlも含めて1つのunused-symbol fixtureを共有してください。 diff --git a/USER_GUIDE.md b/USER_GUIDE.md index 82a5c08a7..b69e9a0a5 100644 --- a/USER_GUIDE.md +++ b/USER_GUIDE.md @@ -1501,6 +1501,49 @@ inferring freshness from match counts. Text recipe output summarizes the same states, and SARIF recipe runs expose the same `query_freshness` object in run properties. Output-limited recipe runs use matched-count metadata for this summary, so queries with known omitted matches are not reported as zero-match. +The `relaxed-json-encoder` query and JSON read/write queries also publish the +`json_trust_boundary` classifier. Place a source-proximate annotation immediately +before the operation when the trust boundary is known: + +```text +// cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=operator_only_local_jsonl +``` + +`origin` accepts `private_local`, `public_api`, `network`, `file`, `external`, or +`unknown`; `direction` is `read` or `write`; `sensitivity` accepts `diagnostic`, +`public`, `untrusted`, `confidential`, or `unknown`; and `trust` accepts +`controlled`, `untrusted`, or `review_required`. `rationale` is a stable token +of up to 80 ASCII letters, digits, `_`, `-`, or `.`. A valid annotation can +classify a controlled private writer, an external/public writer, or an untrusted +parser. The marker must be a real C# line comment; annotation-shaped text inside +regular, verbatim, or raw strings is ignored. Annotations inside conditional-compilation +regions, or separated from the operation by a preprocessor directive, are not trust +evidence. Missing, malformed, lexically invalid, directionally inconsistent, or `review_required` evidence remains +`ambiguous_trust`. Every match line is checked from indexed source even when +guard filtering projects the result to one line. If one result contains matches +with distinct trust evidence, it is conservatively reported as +`ambiguous_trust` with `annotation_status:mixed_boundaries`. An annotation binds +only to the next operation: intervening executable code, including an earlier +statement, evaluated operand, completed expression, control-flow block, or comma-separated operation on the matched line, leaves the later match +`ambiguous_trust` with `annotation_status:not_adjacent`. The annotation only enriches `audit_classifications`; it never +suppresses the underlying recipe result, so external parsing remains visible. +An incomplete declaration or assignment prefix may continue through the bounded +indexed-source statement onto the audited operation line without breaking adjacency; +annotation lookup is not limited to a fixed three-line gap. +Across all selected JSON child queries, the first lexical audited match consumes +the annotation; later matches on the line remain ambiguous even when they belong +to a different API family or overlap the first match as another child-query +substring of the same call. +The C# syntax check distinguishes nullable declaration punctuation such as +`JsonNode? value = ...` and first arguments of nested-generic calls from +conditional/comparison operands, evaluated indexer targets, or property-valued +assignment and invocation receivers before the JSON operation. Even an unresolved bare receiver +or a one-hop member receiver remains conservative when its local/type identity cannot be proven +from the audited declaration prefix. Direct casts are part of the +audited operation, and declaration-type occurrences—including expression-bodied +method or local-function return types, fully qualified local declaration types, +and types at any position in generic return wrappers—before a constructor in the +same containing statement do not consume the annotation, even across line breaks. Add `--show-excluded` to a recipe run when you need the effective path scope and exclusion diagnostics in JSON output. Recipe runs support text output, aggregate JSON with `--json` / `--format json`, @@ -1513,7 +1556,9 @@ recipe discovery, use `cdidx recipes --names --json` for a deterministic name list or `cdidx recipes --summary-only --json` for compact metadata. Recipe row streams can be projected with `--search-fields` including `query_name` and `recipe`, bounded across child queries with `--total-limit`, and byte-bounded -with `--max-json-bytes` for NDJSON. Recipe count output can use +with `--max-json-bytes` for NDJSON. Because the projection whitelist does not +include classification fields, `--search-fields` skips source-backed classification. +Recipe count output can use `--format count --summary-only --max-json-bytes ` to emit only recipe/scope names, aggregate counts, per-query counts, and query freshness. Recipe count aggregations support `--count-by path|file|symbol|origin|return-type|subsystem`, @@ -4886,6 +4931,34 @@ recipe definition と query definition の version により、cache consumer を推測せず drift を検出できます。text の recipe output も同じ状態を要約し、SARIF recipe run は run properties の `query_freshness` に同じ object を出力します。出力制限された recipe run では matched-count metadata を使うため、省略済みの match がある query は zero-match として報告されません。 +`relaxed-json-encoder` query と JSON の read / write query は +`json_trust_boundary` classifier も公開します。trust boundary が判明している場合は、対象操作の +直前に source-proximate な注釈を置きます。 + +```text +// cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=operator_only_local_jsonl +``` + +`origin` は `private_local`、`public_api`、`network`、`file`、`external`、`unknown`、 +`direction` は `read` または `write`、`sensitivity` は `diagnostic`、`public`、 +`untrusted`、`confidential`、`unknown`、`trust` は `controlled`、`untrusted`、 +`review_required` を受け付けます。`rationale` は ASCII の英数字、`_`、`-`、`.` からなる +80文字以下の安定した token です。有効な注釈は controlled private writer、external / public +writer、untrusted parser を分類できます。marker は実際の C# line comment でなければならず、 +regular / verbatim / raw string 内にある注釈形式の text は無視します。条件コンパイル領域内の注釈、 +または preprocessor directive をまたいで対象操作へ到達する注釈は trust evidence にしません。注釈の欠落、不正、 +lexical context 不正、read / write の不一致、または `review_required` の evidence は +`ambiguous_trust` のままです。guard filter により result が1行へ投影される場合も、各 match line を +indexed source から検査します。1つの result に異なる trust evidence を持つ match が含まれる場合は、 +`annotation_status:mixed_boundaries` を伴う `ambiguous_trust` として保守的に報告します。 +注釈は次の操作1件だけに束縛され、途中の実行コードや、match と同じ行の先行 statement、評価済み operand、完了済み expression、control-flow block、カンマ区切りの操作がある後続 match は +`annotation_status:not_adjacent` を伴う `ambiguous_trust` のままです。 +この注釈は `audit_classifications` に根拠を追加するだけで、 +元の recipe result を抑制しないため、external parsing は引き続き表示されます。 +未完了の宣言または代入 prefix が上限付き indexed-source statement 内で audit 対象操作の行まで継続する場合は、隣接性を失いません。注釈探索は固定の3行差に制限されません。 +選択されたすべての JSON child query を横断して、最初の lexical な audit 対象 match が注釈を消費します。後続 match は別の API family に属する場合や、同じ呼び出しに対する別の child-query substring として最初の match に重なる場合も曖昧なままです。 +C# 構文検査では `JsonNode? value = ...` のような nullable 宣言の記号や nested-generic 呼び出しの first argument と、JSON 操作より前に評価される条件式・比較式の operand、indexer 代入先、property-valued な代入 / 呼び出し receiver を区別します。audited declaration prefix から local / type と証明できない単純名 receiver や1段の member receiver も保守的に扱います。 +直接 cast は audit 対象操作の一部として扱い、expression-bodied method / local function の戻り型、完全修飾された local 宣言型、generic return wrapper 内の任意位置にある型を含め、同じ containing statement の constructor より前にある宣言型 occurrence は改行をまたいでも注釈を消費しません。 `--show-excluded` を recipe と併用すると、有効な path scope と除外診断を出力に含めます。 recipe run が対応する形式は text output、`--json` / `--format json` の aggregate JSON、 `--json=ndjson` または `--results-only` の NDJSON row stream、`--format count` の @@ -4896,6 +4969,8 @@ automation 向けの recipe 発見では、決定的な名前一覧だけなら compact metadata が必要なら `cdidx recipes --summary-only --json` を使います。recipe row stream は `query_name` と `recipe` を含む `--search-fields` で投影でき、`--total-limit` で child query 全体の emitted row 数を制限でき、NDJSON では `--max-json-bytes` で byte 数を制限できます。 +projection の allowlist は classification field を含まないため、`--search-fields` は +source-backed 分類を実行しません。 recipe count output は `--format count --summary-only --max-json-bytes ` により、recipe / scope 名、 aggregate count、query ごとの count、query freshness だけを出力できます。recipe の count aggregation は `--count-by path|file|symbol|origin|return-type|subsystem`、 `--group-by file|symbol|origin|return-type|subsystem --count`、`--unique path|file|symbol|origin|return-type|subsystem` に対応します。 diff --git a/changelog.d/unreleased/4913.security.md b/changelog.d/unreleased/4913.security.md new file mode 100644 index 000000000..52d7d84a2 --- /dev/null +++ b/changelog.d/unreleased/4913.security.md @@ -0,0 +1,21 @@ +--- +category: security +issues: + - 4913 +affected: + - src/CodeIndex/Cli/SearchAuditRecipes.cs + - src/CodeIndex/Cli/QueryCommandRunner.SearchRecipes.cs + - src/CodeIndex/Diagnostics/LocalJsonlJsonWriterOptions.cs + - tests/CodeIndex.Tests/QueryCommandRunnerSearchTests.cs + - USER_GUIDE.md + - DEVELOPER_GUIDE.md + - TESTING_GUIDE.md +--- + +## English + +- **Unsafe-relaxed JSON audits now expose explicit trust-boundary evidence (#4913)** — JSON read/write recipe results classify controlled private-local writers, external or public writers, untrusted parsers, and ambiguous boundaries from bounded source-proximate origin, direction, sensitivity, trust, and rationale annotations. Trust evidence must be a lexically verified C# line comment consumed by the next operation across line, long multiline statement continuation, expression, unresolved bare or one-hop/chained property assignment/invocation receiver, conditional-compilation directive, overlapping selected child-query API, control-flow, nested-call, cast, fully qualified, split-line, or outer/split-generic declaration/constructor, and comma-separated boundaries; `review_required` and mixed-boundary rows remain conservatively ambiguous, same-file rows share one maximum bounded prefix reconstruction, compact summaries and projected outputs skip discarded source work, and classifications never suppress the underlying finding. + +## 日本語 + +- **unsafe-relaxed JSON audit が明示的な trust-boundary evidence を公開するようになりました (#4913)** — JSON の read / write recipe result は、source-proximate で上限付きの origin、direction、sensitivity、trust、rationale 注釈から、controlled な private-local writer、external / public writer、untrusted parser、曖昧な境界を分類します。trust evidence は line、長い複数行の statement continuation、expression、解決不能な単純名または1段 / 連鎖した property-valued な代入 / 呼び出し receiver、条件コンパイル directive、重なりを持つ選択済み child query の API、control-flow、nested call、cast、完全修飾 / 改行分割 / 外側または改行された generic の宣言 / constructor、カンマ区切りの各境界で次の操作に消費される lexical 検証済みの C# line comment に限ります。`review_required` と mixed-boundary row は保守的に曖昧なまま、同じ file の row は最大必要行までの上限付き prefix 復元を1回共有し、compact summary と投影出力では破棄される source 処理を省き、分類によって元の finding を抑制しません。 diff --git a/src/CodeIndex/Cli/QueryCommandRunner.SearchRecipes.cs b/src/CodeIndex/Cli/QueryCommandRunner.SearchRecipes.cs index 419c80ba0..fcd65c3c0 100644 --- a/src/CodeIndex/Cli/QueryCommandRunner.SearchRecipes.cs +++ b/src/CodeIndex/Cli/QueryCommandRunner.SearchRecipes.cs @@ -7,6 +7,7 @@ using CodeIndex.Database; using CodeIndex.Indexer; using CodeIndex.Models; +using CodeIndex.Semantics; namespace CodeIndex.Cli; @@ -782,6 +783,7 @@ private static int RunSearchRecipe(QueryCommandOptions options, JsonSerializerOp options, userExact, freshnessContext: null, + includeAuditClassifications: options.SearchFields == null, out _, out var rowMinimumMatchedTotal, out _, @@ -847,6 +849,7 @@ private static int RunSearchRecipe(QueryCommandOptions options, JsonSerializerOp options, userExact, freshnessContext, + includeAuditClassifications: options.Json && options.OutputFormat != OutputFormatSarif, out var total, out _, out var freshnessObservations, @@ -1843,6 +1846,7 @@ private static int RunSearchRecipeIssueDrafts( options, userExact, freshnessContext, + includeAuditClassifications: false, out var total, out _, out var freshnessObservations, @@ -2136,6 +2140,7 @@ private static List CollectSearchRecipeQueryR QueryCommandOptions options, bool userExact, SearchQueryFreshnessContext? freshnessContext, + bool includeAuditClassifications, out int total, out int minimumMatchedTotal, out List freshnessObservations, @@ -2185,7 +2190,8 @@ private static List CollectSearchRecipeQueryR rows = ApplySearchRecipeSemanticFilter(reader, options, recipeQuery, rows); var outputSelection = ApplySearchOutputSelection(rows, options, resultLimit, sourceTotalAuthoritative); rows = outputSelection.Rows; - ApplySearchRecipeAuditClassifications(recipeQuery, rows); + if (includeAuditClassifications) + ApplySearchRecipeAuditClassifications(reader, recipeQuery, recipeQueries, rows); var minimumOmitted = Math.Max(0, outputSelection.OriginalCount - rows.Count); var selectionReason = GetSearchRecipeSelectionReason(outputSelection); total += rows.Count; @@ -2313,7 +2319,8 @@ private static List CollectSearchRecip rows = ApplySearchRecipeSemanticFilter(reader, options, recipeQuery, rows); var outputSelection = ApplySearchOutputSelection(rows, options, resultLimit, sourceTotalAuthoritative); rows = outputSelection.Rows; - ApplySearchRecipeAuditClassifications(recipeQuery, rows); + if (!options.SummaryOnly) + ApplySearchRecipeAuditClassifications(reader, recipeQuery, recipeQueries, rows); var minimumOmitted = Math.Max(0, outputSelection.OriginalCount - rows.Count); var selectionReason = GetSearchRecipeSelectionReason(outputSelection); total += rows.Count; @@ -2463,7 +2470,8 @@ private static List CountSearchRecipeQueryResu results = ApplySearchRecipeFileRejectQueries(reader, results, options, recipeQuery); var rows = BuildSearchDisplayRows(results, options, exact, recipeQuery.Query, rawFtsOverride: false, recipeQuery: recipeQuery); rows = ApplySearchRecipeSemanticFilter(reader, options, recipeQuery, rows); - ApplySearchRecipeAuditClassifications(recipeQuery, rows); + if (options.Json && !options.SummaryOnly) + ApplySearchRecipeAuditClassifications(reader, recipeQuery, recipeQueries, rows); var count = rows.Count; var fileCountForQuery = rows.Select(row => row.Result.Path).Distinct(StringComparer.Ordinal).Count(); foreach (var path in rows.Select(row => row.Result.Path)) @@ -2508,22 +2516,204 @@ private static List CountSearchRecipeQueryResu return queryCounts; } - private static void ApplySearchRecipeAuditClassifications(SearchAuditRecipeQuery recipeQuery, List rows) + private static void ApplySearchRecipeAuditClassifications( + DbReader reader, + SearchAuditRecipeQuery recipeQuery, + IReadOnlyList selectedQueries, + List rows) { var taskResultClassifier = recipeQuery.Classifiers .FirstOrDefault(classifier => string.Equals(classifier.Name, "task_result_intent", StringComparison.Ordinal)); - if (taskResultClassifier == null) + var jsonTrustBoundaryClassifier = recipeQuery.Classifiers + .FirstOrDefault(classifier => string.Equals(classifier.Name, "json_trust_boundary", StringComparison.Ordinal)); + if (taskResultClassifier == null && jsonTrustBoundaryClassifier == null) return; - foreach (var row in rows) + if (taskResultClassifier != null) { - var classification = TryClassifyTaskResultIntent(taskResultClassifier, row); - if (classification == null) - continue; + foreach (var row in rows) + AddSearchRecipeAuditClassification(row, TryClassifyTaskResultIntent(taskResultClassifier, row)); + } + + if (jsonTrustBoundaryClassifier == null || !recipeQuery.JsonTrustDirection.HasValue) + return; + + var selectedJsonTrustQueries = selectedQueries + .Where(query => query.JsonTrustDirection.HasValue + && query.Classifiers.Any(classifier => + string.Equals(classifier.Name, "json_trust_boundary", StringComparison.Ordinal))) + .Select(query => query.Query) + .Where(query => !string.IsNullOrWhiteSpace(query)) + .Distinct(StringComparer.Ordinal) + .ToList(); + var jsonTrustLexicalContextCache = new JsonTrustLexicalContextCache(); + foreach (var fileRows in rows.GroupBy(row => row.Result.Path, StringComparer.Ordinal)) + { + var groupedRows = fileRows.ToList(); + var maximumRequiredLine = groupedRows + .Select(GetJsonTrustRequiredLine) + .Where(line => line > 0 && line <= CSharpSemanticTokenClassifier.DefaultExcerptSourceLineLimit) + .DefaultIfEmpty() + .Max(); + if (maximumRequiredLine > 0) + { + _ = GetJsonTrustLexicalContext( + reader, + groupedRows[0], + maximumRequiredLine, + jsonTrustLexicalContextCache); + } + foreach (var row in groupedRows) + { + AddSearchRecipeAuditClassification( + row, + ClassifyJsonTrustBoundary( + jsonTrustBoundaryClassifier, + recipeQuery.JsonTrustDirection.Value, + row, + reader, + jsonTrustLexicalContextCache, + selectedJsonTrustQueries)); + } + } + } + + private static int GetJsonTrustRequiredLine(SearchDisplayRow row) + => row.Compact.MatchLines + .Where(line => line > 0) + .DefaultIfEmpty(row.Compact.FocusLine.GetValueOrDefault(row.Result.StartLine)) + .Max(); + + private static void AddSearchRecipeAuditClassification( + SearchDisplayRow row, + SearchAuditClassificationJsonResult? classification) + { + if (classification == null) + return; + + row.Compact.AuditClassifications ??= []; + row.Compact.AuditClassifications.Add(classification); + } + + private static SearchAuditClassificationJsonResult ClassifyJsonTrustBoundary( + SearchRecipeClassifierJsonResult classifier, + SearchRecipeJsonTrustDirection expectedDirection, + SearchDisplayRow row, + DbReader reader, + JsonTrustLexicalContextCache lexicalContextCache, + IReadOnlyList selectedJsonTrustQueries) + { + var matchLines = row.Compact.MatchLines + .Where(line => line > 0) + .Distinct() + .OrderBy(line => line) + .ToList(); + if (matchLines.Count == 0) + matchLines.Add(row.Compact.FocusLine.GetValueOrDefault(row.Result.StartLine)); + + var matchSites = row.Compact.MatchFacets + .Where(facet => facet.Line > 0 + && facet.Column > 0 + && matchLines.Contains(facet.Line)) + .Select(facet => new JsonTrustMatchSite(facet.Line, facet.Column, facet.Length)) + .Distinct() + .OrderBy(site => site.Line) + .ThenBy(site => site.Column) + .ToList(); + foreach (var line in matchLines.Where(line => matchSites.All(site => site.Line != line))) + matchSites.Add(new JsonTrustMatchSite(line, null, null)); + matchSites = matchSites + .OrderBy(site => site.Line) + .ThenBy(site => site.Column) + .ToList(); + + var lexicalContext = GetJsonTrustLexicalContext( + reader, + row, + matchSites.Max(site => site.Line), + lexicalContextCache); + if (lexicalContext != null) + { + matchSites = matchSites + .Where(site => !IsJsonTrustDeclarationFacetBeforeLaterMatch(site, matchSites, lexicalContext)) + .ToList(); + } + var consumedAnnotationLines = new HashSet(); + var matchEvidence = new List(matchSites.Count); + foreach (var site in matchSites) + { + var candidate = GetJsonTrustBoundaryEvidence( + expectedDirection, + site.Line, + site.Column, + lexicalContext, + selectedJsonTrustQueries); + if (candidate.AnnotationLine is { } annotationLine + && !consumedAnnotationLines.Add(annotationLine)) + { + candidate = new JsonTrustBoundaryEvidence( + "unknown", + expectedDirection == SearchRecipeJsonTrustDirection.Read ? "read" : "write", + "unknown", + "review_required", + "annotation_not_bound_to_operation", + "not_adjacent", + annotationLine); + } + matchEvidence.Add(candidate); + } + var evidence = matchEvidence[0]; + var mixedBoundaries = matchEvidence + .Skip(1) + .Any(candidate => !HasEquivalentJsonTrustBoundary(evidence, candidate)); + var matchCategories = matchEvidence + .Select(candidate => candidate.AnnotationStatus == "valid" + ? ClassifyValidJsonTrustBoundary(candidate) + : "ambiguous_trust") + .Distinct(StringComparer.Ordinal) + .OrderBy(category => category, StringComparer.Ordinal) + .ToList(); + if (mixedBoundaries) + { + evidence = new JsonTrustBoundaryEvidence( + "unknown", + expectedDirection == SearchRecipeJsonTrustDirection.Read ? "read" : "write", + "unknown", + "review_required", + "multiple_trust_boundaries", + "mixed_boundaries", + null); + } - row.Compact.AuditClassifications ??= []; - row.Compact.AuditClassifications.Add(classification); + var categoryName = mixedBoundaries + ? "ambiguous_trust" + : matchCategories[0]; + var categoryMetadata = classifier.Categories + .First(category => string.Equals(category.Name, categoryName, StringComparison.Ordinal)); + var details = new List + { + $"origin:{evidence.Origin}", + $"direction:{evidence.Direction}", + $"sensitivity:{evidence.Sensitivity}", + $"trust:{evidence.Trust}", + $"rationale:{evidence.Rationale}", + $"annotation_status:{evidence.AnnotationStatus}", + }; + if (evidence.AnnotationLine.HasValue) + details.Add($"annotation_line:{evidence.AnnotationLine.Value.ToString(CultureInfo.InvariantCulture)}"); + if (mixedBoundaries) + { + details.Add($"match_line_count:{matchLines.Count.ToString(CultureInfo.InvariantCulture)}"); + if (matchSites.Count != matchLines.Count) + details.Add($"match_site_count:{matchSites.Count.ToString(CultureInfo.InvariantCulture)}"); + details.Add($"boundary_categories:{string.Join(',', matchCategories)}"); } + return new SearchAuditClassificationJsonResult( + classifier.Name, + categoryMetadata.Name, + categoryMetadata.Description, + categoryMetadata.ReviewGuidance, + details); } private static List ApplySearchRecipeSemanticFilter( @@ -2614,6 +2804,1037 @@ private static List ApplySearchRecipeSemanticFilter( details); } + private static bool IsJsonTrustDeclarationFacetBeforeLaterMatch( + JsonTrustMatchSite site, + IReadOnlyList matchSites, + JsonTrustLexicalContext lexicalContext) + { + var laterMatch = matchSites + .Where(candidate => candidate.Column.HasValue + && (candidate.Line > site.Line + || candidate.Line == site.Line && candidate.Column > site.Column)) + .OrderBy(candidate => candidate.Line) + .ThenBy(candidate => candidate.Column) + .FirstOrDefault(); + if (!site.Column.HasValue + || !site.Length.HasValue + || site.Line <= 0 + || site.Line > lexicalContext.MaskedLines.Length + || laterMatch == default + || laterMatch.Line > lexicalContext.MaskedLines.Length) + { + return false; + } + + var declarationPrefix = new StringBuilder(); + for (var lineNumber = site.Line; lineNumber <= laterMatch.Line; lineNumber++) + { + var sourceLine = GetJsonTrustCodeBeforeLineComment(lexicalContext.MaskedLines[lineNumber - 1]); + var prefixLength = lineNumber == laterMatch.Line + ? Math.Clamp(laterMatch.Column!.Value - 1, 0, sourceLine.Length) + : sourceLine.Length; + declarationPrefix.Append(sourceLine.AsSpan(0, prefixLength)).Append(' '); + } + + var line = declarationPrefix.ToString(); + var index = Math.Clamp(site.Column.Value - 1 + site.Length.Value, 0, line.Length); + SkipJsonTrustDeclarationTypeSuffix( + line, + ref index, + GetJsonTrustUnclosedGenericDepthBeforeSite(lexicalContext, site)); + + if (index >= line.Length || !(char.IsLetter(line[index]) || line[index] is '_' or '@')) + return false; + + index++; + while (index < line.Length && (char.IsLetterOrDigit(line[index]) || line[index] == '_')) + index++; + SkipJsonTrustWhitespace(line, ref index); + if (line.AsSpan(index).IndexOfAny(';', '{', '}') >= 0) + return false; + + return index >= line.Length + || line[index] is '=' or ';' or ',' or ')' + || IsJsonTrustMethodDeclarationSuffix(line, index); + } + + private static void SkipJsonTrustDeclarationTypeSuffix( + string line, + ref int index, + int enclosingGenericDepth) + { + while (enclosingGenericDepth > 0 && index < line.Length) + { + switch (line[index]) + { + case '<': + enclosingGenericDepth++; + break; + case '>': + enclosingGenericDepth--; + break; + case ';' or '=' or '{' or '}': + return; + } + + index++; + } + + while (true) + { + SkipJsonTrustWhitespace(line, ref index); + if (index < line.Length && line[index] == '?') + { + index++; + continue; + } + if (index + 1 < line.Length && line[index] == '[' && line[index + 1] == ']') + { + index += 2; + continue; + } + return; + } + } + + private static int GetJsonTrustUnclosedGenericDepthBeforeSite( + JsonTrustLexicalContext lexicalContext, + JsonTrustMatchSite site) + { + var closingDepth = 0; + var unclosedDepth = 0; + for (var lineNumber = site.Line; lineNumber >= 1; lineNumber--) + { + var line = GetJsonTrustCodeBeforeLineComment(lexicalContext.MaskedLines[lineNumber - 1]); + var startIndex = lineNumber == site.Line + ? Math.Clamp(site.Column.GetValueOrDefault(1) - 2, -1, line.Length - 1) + : line.Length - 1; + for (var index = startIndex; index >= 0; index--) + { + switch (line[index]) + { + case '>': + closingDepth++; + break; + case '<' when closingDepth > 0: + closingDepth--; + break; + case '<': + unclosedDepth++; + break; + case ';' or '=' or '{' or '}': + return unclosedDepth; + } + } + } + + return unclosedDepth; + } + + private static bool IsJsonTrustMethodDeclarationSuffix(string line, int index) + { + if (index >= line.Length || line[index] == '(') + return index < line.Length; + if (line[index] != '<') + return false; + + var depth = 0; + for (; index < line.Length; index++) + { + if (line[index] == '<') + { + depth++; + continue; + } + if (line[index] != '>') + continue; + + depth--; + if (depth != 0) + continue; + + index++; + SkipJsonTrustWhitespace(line, ref index); + return index < line.Length && line[index] == '('; + } + + return false; + } + + private static void SkipJsonTrustWhitespace(string line, ref int index) + { + while (index < line.Length && char.IsWhiteSpace(line[index])) + index++; + } + + private static string ClassifyValidJsonTrustBoundary(JsonTrustBoundaryEvidence evidence) + { + if (evidence.Trust == "review_required") + return "ambiguous_trust"; + + var externalOrigin = evidence.Origin is "public_api" or "network" or "file" or "external"; + if (evidence.Direction == "write" && externalOrigin) + return "external_or_public_writer"; + if (evidence.Direction == "read" && externalOrigin && evidence.Trust == "untrusted") + return "untrusted_parser"; + if (evidence.Direction == "write" + && evidence.Origin == "private_local" + && evidence.Trust == "controlled" + && evidence.Sensitivity is "diagnostic" or "confidential") + { + return "controlled_private_writer"; + } + + return "ambiguous_trust"; + } + + private static bool HasEquivalentJsonTrustBoundary( + JsonTrustBoundaryEvidence left, + JsonTrustBoundaryEvidence right) + => string.Equals(left.Origin, right.Origin, StringComparison.Ordinal) + && string.Equals(left.Direction, right.Direction, StringComparison.Ordinal) + && string.Equals(left.Sensitivity, right.Sensitivity, StringComparison.Ordinal) + && string.Equals(left.Trust, right.Trust, StringComparison.Ordinal) + && string.Equals(left.Rationale, right.Rationale, StringComparison.Ordinal) + && string.Equals(left.AnnotationStatus, right.AnnotationStatus, StringComparison.Ordinal); + + private static JsonTrustBoundaryEvidence GetJsonTrustBoundaryEvidence( + SearchRecipeJsonTrustDirection expectedDirection, + int focusLine, + int? focusColumn, + JsonTrustLexicalContext? lexicalContext, + IReadOnlyList selectedJsonTrustQueries) + { + const string marker = "// cdidx-audit: json-trust "; + var expectedDirectionText = expectedDirection == SearchRecipeJsonTrustDirection.Read ? "read" : "write"; + var nearestLine = -1; + if (lexicalContext != null && focusLine > 0 && focusLine <= lexicalContext.SourceLines.Length) + { + var annotationIndex = Array.BinarySearch(lexicalContext.AnnotationLines, focusLine); + if (annotationIndex < 0) + annotationIndex = ~annotationIndex - 1; + if (annotationIndex >= 0) + nearestLine = lexicalContext.AnnotationLines[annotationIndex]; + } + + if (nearestLine < 0) + { + return new JsonTrustBoundaryEvidence( + "unknown", + expectedDirectionText, + "unknown", + "review_required", + "missing_explicit_trust_annotation", + "missing", + null); + } + + if (!lexicalContext!.MaskedLines[nearestLine - 1].TrimStart().StartsWith(marker, StringComparison.Ordinal)) + { + return new JsonTrustBoundaryEvidence( + "unknown", + expectedDirectionText, + "unknown", + "review_required", + "invalid_explicit_trust_annotation", + "invalid", + nearestLine); + } + + if (lexicalContext.ConditionalCompilationLines[nearestLine - 1]) + { + return new JsonTrustBoundaryEvidence( + "unknown", + expectedDirectionText, + "unknown", + "review_required", + "conditional_compilation_annotation", + "not_adjacent", + nearestLine); + } + + if (HasInterveningJsonTrustExecutableLine( + lexicalContext, + nearestLine, + focusLine, + focusColumn, + selectedJsonTrustQueries)) + { + return new JsonTrustBoundaryEvidence( + "unknown", + expectedDirectionText, + "unknown", + "review_required", + "annotation_not_bound_to_operation", + "not_adjacent", + nearestLine); + } + + var annotation = lexicalContext.SourceLines[nearestLine - 1].TrimStart(); + var payload = annotation[marker.Length..].Trim(); + if (!TryParseJsonTrustBoundaryAnnotation(payload, out var parsed)) + { + return new JsonTrustBoundaryEvidence( + "unknown", + expectedDirectionText, + "unknown", + "review_required", + "invalid_explicit_trust_annotation", + "invalid", + nearestLine); + } + + if (!string.Equals(parsed.Direction, expectedDirectionText, StringComparison.Ordinal)) + return parsed with { AnnotationStatus = "direction_mismatch", AnnotationLine = nearestLine }; + + return parsed with { AnnotationStatus = "valid", AnnotationLine = nearestLine }; + } + + private static bool HasInterveningJsonTrustExecutableLine( + JsonTrustLexicalContext lexicalContext, + int annotationLine, + int operationLine, + int? operationColumn, + IReadOnlyList selectedJsonTrustQueries) + { + var statementPrefix = new StringBuilder(); + for (var line = annotationLine + 1; line < operationLine; line++) + { + var maskedLine = GetJsonTrustCodeBeforeLineComment(lexicalContext.MaskedLines[line - 1]); + if (string.IsNullOrWhiteSpace(maskedLine)) + continue; + if (maskedLine.AsSpan().TrimStart().StartsWith("#", StringComparison.Ordinal)) + return true; + + statementPrefix.Append(maskedLine).Append(' '); + if (HasPriorSelectedJsonTrustMatchOnLine( + lexicalContext, + line, + maskedLine.Length + 1, + selectedJsonTrustQueries)) + { + return true; + } + } + + if (operationColumn.HasValue + && operationLine > 0 + && operationLine <= lexicalContext.MaskedLines.Length) + { + var operationText = lexicalContext.MaskedLines[operationLine - 1]; + var prefixLength = Math.Clamp(operationColumn.Value - 1, 0, operationText.Length); + statementPrefix.Append(operationText.AsSpan(0, prefixLength)); + if (HasPriorSelectedJsonTrustMatchOnLine( + lexicalContext, + operationLine, + operationColumn.Value, + selectedJsonTrustQueries)) + return true; + } + + return HasPriorJsonTrustOperationOnLine(statementPrefix.ToString().AsSpan()); + } + + private static string GetJsonTrustCodeBeforeLineComment(string maskedLine) + { + var commentIndex = maskedLine.IndexOf("//", StringComparison.Ordinal); + return commentIndex >= 0 ? maskedLine[..commentIndex] : maskedLine; + } + + private static bool HasPriorSelectedJsonTrustMatchOnLine( + JsonTrustLexicalContext lexicalContext, + int operationLine, + int operationColumn, + IReadOnlyList selectedJsonTrustQueries) + { + if (operationLine <= 0 + || operationLine > lexicalContext.MaskedLines.Length + || selectedJsonTrustQueries.Count == 0) + { + return false; + } + + var line = lexicalContext.MaskedLines[operationLine - 1]; + var prefixLength = Math.Clamp(operationColumn - 1, 0, line.Length); + var normalizedLine = CSharpVerbatimNameNormalizer.Normalize(line, out var rawIndexMap); + var normalizedPrefixLength = Array.BinarySearch(rawIndexMap, prefixLength); + if (normalizedPrefixLength < 0) + normalizedPrefixLength = ~normalizedPrefixLength; + var currentSite = new JsonTrustMatchSite(operationLine, operationColumn, null); + foreach (var query in selectedJsonTrustQueries) + { + var normalizedQuery = CSharpVerbatimNameNormalizer.Normalize(query); + var searchStart = 0; + while (searchStart < normalizedPrefixLength) + { + var occurrence = normalizedLine.IndexOf(normalizedQuery, searchStart, StringComparison.Ordinal); + if (occurrence < 0 || occurrence >= normalizedPrefixLength) + break; + + var normalizedEnd = occurrence + normalizedQuery.Length; + if (normalizedEnd > normalizedPrefixLength) + return true; + var rawStart = rawIndexMap[occurrence]; + var rawEnd = rawIndexMap[normalizedEnd - 1] + 1; + var priorSite = new JsonTrustMatchSite(operationLine, rawStart + 1, rawEnd - rawStart); + if (!IsJsonTrustDeclarationFacetBeforeLaterMatch( + priorSite, + [priorSite, currentSite], + lexicalContext)) + { + return true; + } + + searchStart = occurrence + Math.Max(1, normalizedQuery.Length); + } + } + + return false; + } + + private static bool HasPriorJsonTrustOperationOnLine(ReadOnlySpan prefix) + { + var tokens = TokenizeJsonTrustCSharpPrefix(prefix); + if (tokens.Count == 0) + return false; + + var expressionStart = 0; + if (TryGetJsonTrustExpressionBodiedMethodStart(tokens, out var expressionBodyStart)) + expressionStart = expressionBodyStart; + + var assignmentIndex = -1; + for (var index = tokens.Count - 1; index >= expressionStart; index--) + { + if (tokens[index] == "=") + { + assignmentIndex = index; + break; + } + } + + if (assignmentIndex >= 0 + && HasEvaluatedJsonTrustAssignmentTarget(tokens, expressionStart, assignmentIndex)) + { + return true; + } + + if (assignmentIndex >= 0) + expressionStart = assignmentIndex + 1; + if (HasEvaluatedJsonTrustInvocationReceiver(tokens, expressionStart)) + return true; + + for (var index = expressionStart; index < tokens.Count; index++) + { + var token = tokens[index]; + if (token == "(" + && TrySkipJsonTrustCast(tokens, expressionStart, index, out var castCloseIndex)) + { + index = castCloseIndex; + continue; + } + + if (token == "<" + && TrySkipJsonTrustGenericArgumentList(tokens, index, out var genericCloseIndex)) + { + index = genericCloseIndex; + continue; + } + + if (token == ")" && IsJsonTrustCastClosingParenthesis(tokens, expressionStart, index)) + continue; + + if (token is ";" or "," or "{" or "}" or ")" or "]" or "=>" + or "==" or "!=" or "<=" or ">=" or "++" or "--" + or "+" or "-" or "*" or "/" or "%" or "&" or "|" or "^" + or "&&" or "||" or "??" or "?" or "<" or ">" or "<<" or ">>" + or "+=" or "-=" or "*=" or "/=" or "%=" or "&=" or "|=" or "^=" + or "is" or "as" or "and" or "or") + { + return true; + } + + if (IsJsonTrustNumericToken(token)) + return true; + } + + return false; + } + + private static bool TrySkipJsonTrustCast( + IReadOnlyList tokens, + int expressionStart, + int openIndex, + out int closeIndex) + { + closeIndex = -1; + if (openIndex < expressionStart + || openIndex >= tokens.Count + || tokens[openIndex] != "(") + { + return false; + } + + var depth = 0; + for (var index = openIndex; index < tokens.Count; index++) + { + if (tokens[index] == "(") + { + depth++; + continue; + } + if (tokens[index] != ")") + continue; + + depth--; + if (depth != 0) + continue; + + if (!IsJsonTrustCastClosingParenthesis(tokens, expressionStart, index)) + return false; + + closeIndex = index; + return true; + } + + return false; + } + + private static bool HasEvaluatedJsonTrustInvocationReceiver( + IReadOnlyList tokens, + int expressionStart) + { + var closingDepth = 0; + for (var openIndex = tokens.Count - 1; openIndex >= expressionStart; openIndex--) + { + if (tokens[openIndex] == ")") + { + closingDepth++; + continue; + } + if (tokens[openIndex] != "(") + continue; + if (closingDepth > 0) + { + closingDepth--; + continue; + } + + var methodNameIndex = openIndex - 1; + if (methodNameIndex >= expressionStart && tokens[methodNameIndex] == ">") + { + var genericDepth = 0; + for (; methodNameIndex >= expressionStart; methodNameIndex--) + { + if (tokens[methodNameIndex] == ">") + { + genericDepth++; + continue; + } + if (tokens[methodNameIndex] != "<") + continue; + + genericDepth--; + if (genericDepth == 0) + { + methodNameIndex--; + break; + } + } + } + + if (methodNameIndex < expressionStart + 2 + || !IsJsonTrustIdentifierToken(tokens[methodNameIndex]) + || tokens[methodNameIndex - 1] is not ("." or "?.")) + { + continue; + } + + // A direct call on a simple local/type receiver (source.Build(...)) does not + // evaluate a property before its first argument. An additional member hop + // (source.Factory.Build(...)) can execute a getter and therefore consumes the + // trust annotation before the JSON operation. + var receiverMemberIndex = methodNameIndex - 2; + if (receiverMemberIndex >= expressionStart + 2 + && IsJsonTrustIdentifierToken(tokens[receiverMemberIndex]) + && tokens[receiverMemberIndex - 1] is "." or "?.") + { + return true; + } + + if (receiverMemberIndex < expressionStart + || !IsJsonTrustIdentifierToken(tokens[receiverMemberIndex])) + { + return true; + } + + var receiver = tokens[receiverMemberIndex]; + if (!IsJsonTrustReceiverDeclaredBeforeExpression(tokens, expressionStart, receiver)) + return true; + } + + return false; + } + + private static bool IsJsonTrustReceiverDeclaredBeforeExpression( + IReadOnlyList tokens, + int expressionStart, + string receiver) + { + for (var index = 1; index < expressionStart; index++) + { + if (!string.Equals(tokens[index], receiver, StringComparison.Ordinal)) + continue; + if (index + 1 >= expressionStart + || tokens[index + 1] is not ("," or ")" or "=")) + { + continue; + } + + var precedingToken = tokens[index - 1]; + if (IsJsonTrustIdentifierToken(precedingToken) + || precedingToken is ">" or "]" or "?" or "*") + { + return true; + } + } + + return false; + } + + private static bool TryGetJsonTrustExpressionBodiedMethodStart( + IReadOnlyList tokens, + out int expressionStart) + { + expressionStart = -1; + var arrowIndex = -1; + for (var index = tokens.Count - 1; index >= 0; index--) + { + if (tokens[index] == "=>") + { + arrowIndex = index; + break; + } + } + if (arrowIndex < 2 || tokens[arrowIndex - 1] != ")") + return false; + + var depth = 0; + var openIndex = -1; + for (var index = arrowIndex - 1; index >= 0; index--) + { + if (tokens[index] == ")") + { + depth++; + continue; + } + if (tokens[index] != "(") + continue; + + depth--; + if (depth == 0) + { + openIndex = index; + break; + } + } + + if (openIndex < 2 || !HasJsonTrustMethodNameBeforeParenthesis(tokens, openIndex)) + return false; + + expressionStart = arrowIndex + 1; + return true; + } + + private static bool IsJsonTrustIdentifierToken(string token) + => token.Length > 0 && (char.IsLetter(token[0]) || token[0] is '_' or '@'); + + private static bool HasJsonTrustMethodNameBeforeParenthesis( + IReadOnlyList tokens, + int openIndex) + { + var nameIndex = openIndex - 1; + if (IsJsonTrustIdentifierToken(tokens[nameIndex])) + return true; + if (tokens[nameIndex] != ">") + return false; + + var depth = 0; + for (var index = nameIndex; index >= 0; index--) + { + if (tokens[index] == ">") + { + depth++; + continue; + } + if (tokens[index] != "<") + continue; + + depth--; + if (depth == 0) + return index > 0 && IsJsonTrustIdentifierToken(tokens[index - 1]); + } + + return false; + } + + private static bool IsJsonTrustCastClosingParenthesis( + IReadOnlyList tokens, + int expressionStart, + int closeIndex) + { + var depth = 0; + var openIndex = -1; + for (var index = closeIndex; index >= expressionStart; index--) + { + if (tokens[index] == ")") + { + depth++; + continue; + } + if (tokens[index] != "(") + continue; + + depth--; + if (depth == 0) + { + openIndex = index; + break; + } + } + if (openIndex < 0 || openIndex + 1 >= closeIndex) + return false; + + if (openIndex > expressionStart + && tokens[openIndex - 1] is not ("return" or "throw" or "await" or "(" or "," or ":" + or "=" or "=>" or "!" or "~" or "+" or "-")) + { + return false; + } + + var hasTypeIdentifier = false; + var nestedParentheses = 0; + for (var index = openIndex + 1; index < closeIndex; index++) + { + var token = tokens[index]; + if (token == "(") + { + nestedParentheses++; + continue; + } + if (token == ")" && nestedParentheses > 0) + { + nestedParentheses--; + continue; + } + if (token.Length > 0 && (char.IsLetter(token[0]) || token[0] is '_' or '@')) + { + hasTypeIdentifier = true; + continue; + } + if (token is "." or "::" or "?" or "[" or "]" or "<" or ">" or "," or "*") + continue; + + return false; + } + + return hasTypeIdentifier && nestedParentheses == 0; + } + + private static bool TrySkipJsonTrustGenericArgumentList( + IReadOnlyList tokens, + int openIndex, + out int closeIndex) + { + closeIndex = -1; + var depth = 0; + for (var index = openIndex; index < tokens.Count; index++) + { + var token = tokens[index]; + if (token == "<") + { + depth++; + continue; + } + if (token != ">") + continue; + + depth--; + if (depth != 0) + continue; + + if (index + 1 < tokens.Count + && tokens[index + 1] is "(" or "." or "?." or "::") + { + closeIndex = index; + return true; + } + return false; + } + + return false; + } + + private static bool HasEvaluatedJsonTrustAssignmentTarget( + IReadOnlyList tokens, + int expressionStart, + int assignmentIndex) + { + if (IsJsonTrustDeclarationAssignmentTarget(tokens, assignmentIndex)) + return false; + + var genericDepth = 0; + for (var index = expressionStart; index < assignmentIndex; index++) + { + var token = tokens[index]; + if (token == "<") + { + genericDepth++; + continue; + } + if (token == ">" && genericDepth > 0) + { + genericDepth--; + continue; + } + if (token == "," && genericDepth > 0) + continue; + if (token == "]" && index > 0 && tokens[index - 1] == "[") + continue; + if (token == "]") + return true; + + if (token is "." or "?.") + return true; + if (token is "::" or "[") + continue; + + if (token is ";" or "," or "{" or "}" or ")" or "=>" or "?" or ":" + or "==" or "!=" or "<=" or ">=" or "++" or "--" + or "+" or "-" or "*" or "/" or "%" or "&" or "|" or "^" + or "&&" or "||" or "??" or "<<" or ">>" + or "is" or "as" or "and" or "or") + { + return true; + } + } + + return false; + } + + private static bool IsJsonTrustDeclarationAssignmentTarget( + IReadOnlyList tokens, + int assignmentIndex) + { + if (assignmentIndex < 2 || !IsJsonTrustIdentifierToken(tokens[assignmentIndex - 1])) + return false; + + var precedingToken = tokens[assignmentIndex - 2]; + return IsJsonTrustIdentifierToken(precedingToken) + || precedingToken is ">" or "]" or "?" or "*" or ")"; + } + + private static List TokenizeJsonTrustCSharpPrefix(ReadOnlySpan prefix) + { + var tokens = new List(); + for (var index = 0; index < prefix.Length;) + { + if (char.IsWhiteSpace(prefix[index])) + { + index++; + continue; + } + + if (char.IsLetter(prefix[index]) || prefix[index] is '_' or '@') + { + var start = index++; + while (index < prefix.Length && + (char.IsLetterOrDigit(prefix[index]) || prefix[index] == '_')) + { + index++; + } + tokens.Add(prefix[start..index].ToString().TrimStart('@')); + continue; + } + + if (char.IsDigit(prefix[index])) + { + var start = index++; + while (index < prefix.Length && + (char.IsLetterOrDigit(prefix[index]) || prefix[index] is '_' or '.')) + { + index++; + } + tokens.Add(prefix[start..index].ToString()); + continue; + } + + if (index + 1 < prefix.Length) + { + var pair = prefix.Slice(index, 2); + if (pair is "=>" or "::" or "?." or "??" or "==" or "!=" + or "<=" or ">=" or "++" or "--" or "&&" or "||" + or "+=" or "-=" or "*=" or "/=" + or "%=" or "&=" or "|=" or "^=") + { + tokens.Add(pair.ToString()); + index += 2; + continue; + } + } + + tokens.Add(prefix[index].ToString()); + index++; + } + + return tokens; + } + + private static bool IsJsonTrustNumericToken(string token) + => token.Length > 0 && char.IsDigit(token[0]); + + private static JsonTrustLexicalContext? GetJsonTrustLexicalContext( + DbReader reader, + SearchDisplayRow row, + int requiredLine, + JsonTrustLexicalContextCache cache) + { + if (!string.Equals(row.Result.Lang, "csharp", StringComparison.OrdinalIgnoreCase)) + return null; + + if (requiredLine <= 0 || requiredLine > CSharpSemanticTokenClassifier.DefaultExcerptSourceLineLimit) + return null; + + if (string.Equals(cache.Path, row.Result.Path, StringComparison.Ordinal)) + { + if (cache.LoadedThroughLine >= requiredLine) + return cache.Context; + if (cache.SourceLimitReached) + return null; + } + + var indexedLines = reader.GetIndexedSourceLinesForSemanticTokens( + row.Result.Path, + requiredLine, + CSharpSemanticTokenClassifier.DefaultExcerptSourceCharacterLimit); + JsonTrustLexicalContext? context = null; + if (indexedLines.Count > 0) + { + var sourceLines = indexedLines.Select(line => line ?? string.Empty).ToArray(); + var maskedLines = StructuralLineMasker.MaskLines("csharp", sourceLines); + var annotationLines = sourceLines + .Select((line, index) => (Line: line, Number: index + 1)) + .Where(candidate => candidate.Line.TrimStart().StartsWith( + "// cdidx-audit: json-trust ", + StringComparison.Ordinal)) + .Select(candidate => candidate.Number) + .ToArray(); + var conditionalCompilationLines = GetJsonTrustConditionalCompilationLines(maskedLines); + context = new JsonTrustLexicalContext( + sourceLines, + maskedLines, + annotationLines, + conditionalCompilationLines); + } + + // Retain only one bounded source prefix so count-mode memory does not grow with file count. + cache.Path = row.Result.Path; + cache.LoadedThroughLine = indexedLines.Count; + cache.SourceLimitReached = indexedLines.Count < requiredLine; + cache.Context = context; + return context; + } + + private static bool[] GetJsonTrustConditionalCompilationLines(IReadOnlyList maskedLines) + { + var conditionalLines = new bool[maskedLines.Count]; + var depth = 0; + for (var index = 0; index < maskedLines.Count; index++) + { + var trimmed = maskedLines[index].AsSpan().TrimStart(); + if (IsJsonTrustConditionalCompilationDirective(trimmed, "if", allowExpressionStartWithoutWhitespace: true)) + { + depth++; + conditionalLines[index] = true; + continue; + } + + conditionalLines[index] = depth > 0; + if (IsJsonTrustConditionalCompilationDirective(trimmed, "endif", allowExpressionStartWithoutWhitespace: false)) + { + depth = Math.Max(0, depth - 1); + } + } + + return conditionalLines; + } + + private static bool IsJsonTrustConditionalCompilationDirective( + ReadOnlySpan line, + ReadOnlySpan directive, + bool allowExpressionStartWithoutWhitespace) + { + var index = 0; + while (index < line.Length && char.IsWhiteSpace(line[index])) + index++; + if (index >= line.Length || line[index] != '#') + return false; + + index++; + while (index < line.Length && char.IsWhiteSpace(line[index])) + index++; + if (!line[index..].StartsWith(directive, StringComparison.Ordinal)) + return false; + + index += directive.Length; + return index == line.Length + || char.IsWhiteSpace(line[index]) + || (allowExpressionStartWithoutWhitespace && line[index] is '(' or '!'); + } + + private static bool TryParseJsonTrustBoundaryAnnotation( + string payload, + out JsonTrustBoundaryEvidence evidence) + { + evidence = null!; + var values = new Dictionary(StringComparer.Ordinal); + foreach (var token in payload.Split(' ', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries)) + { + var equalsIndex = token.IndexOf('='); + if (equalsIndex <= 0 || equalsIndex == token.Length - 1) + return false; + if (!values.TryAdd(token[..equalsIndex], token[(equalsIndex + 1)..])) + return false; + } + + if (values.Count != 5 + || !values.TryGetValue("origin", out var origin) + || !values.TryGetValue("direction", out var direction) + || !values.TryGetValue("sensitivity", out var sensitivity) + || !values.TryGetValue("trust", out var trust) + || !values.TryGetValue("rationale", out var rationale) + || origin is not ("private_local" or "public_api" or "network" or "file" or "external" or "unknown") + || direction is not ("read" or "write") + || sensitivity is not ("diagnostic" or "public" or "untrusted" or "confidential" or "unknown") + || trust is not ("controlled" or "untrusted" or "review_required") + || !IsValidJsonTrustBoundaryRationale(rationale)) + { + return false; + } + + evidence = new JsonTrustBoundaryEvidence( + origin, + direction, + sensitivity, + trust, + rationale, + string.Empty, + null); + return true; + } + + private static bool IsValidJsonTrustBoundaryRationale(string value) + => value.Length is > 0 and <= 80 + && value.All(ch => char.IsAsciiLetterOrDigit(ch) || ch is '_' or '-' or '.'); private static SearchRecipeSemanticEvidence GetRegexOperationEvidence( DbReader reader, SearchDisplayRow row, @@ -3163,6 +4384,31 @@ private sealed record TaskResultIntentEvidence( int? Line, string Reason); + private readonly record struct JsonTrustMatchSite(int Line, int? Column, int? Length); + + private sealed record JsonTrustBoundaryEvidence( + string Origin, + string Direction, + string Sensitivity, + string Trust, + string Rationale, + string AnnotationStatus, + int? AnnotationLine); + + private sealed record JsonTrustLexicalContext( + string[] SourceLines, + string[] MaskedLines, + int[] AnnotationLines, + bool[] ConditionalCompilationLines); + + private sealed class JsonTrustLexicalContextCache + { + public string? Path { get; set; } + public int LoadedThroughLine { get; set; } + public bool SourceLimitReached { get; set; } + public JsonTrustLexicalContext? Context { get; set; } + } + private static List CollectSearchRecipeAggregationResults( DbReader reader, IReadOnlyList recipeQueries, diff --git a/src/CodeIndex/Cli/SearchAuditRecipes.cs b/src/CodeIndex/Cli/SearchAuditRecipes.cs index e42a459b2..9b395c973 100644 --- a/src/CodeIndex/Cli/SearchAuditRecipes.cs +++ b/src/CodeIndex/Cli/SearchAuditRecipes.cs @@ -239,6 +239,17 @@ internal static class SearchAuditRecipes ], ["guard_filters", "guard_evidence", "guard_checks", "risk_evidence", "match_origins"], "Parser guard classifiers are triage hints; keep input-size, depth, and cancellation checks close to the parse boundary when possible."); + private static readonly SearchRecipeClassifierJsonResult JsonTrustBoundaryClassifier = new( + "json_trust_boundary", + "Classifies JSON reads and writes by explicit origin, direction, sensitivity, trust, and rationale evidence without suppressing the underlying finding.", + [ + new("controlled_private_writer", "A JSON writer is explicitly scoped to a controlled private-local sink.", "Verify that the sink remains private, local, non-HTML, and non-script, and that the annotation rationale still matches every consumer."), + new("external_or_public_writer", "A JSON writer crosses an external, network, file, or public-API boundary instead of a private-local sink.", "Keep the finding prioritized for escaping, content-type, redaction, destination-ownership, and downstream-consumer review."), + new("untrusted_parser", "A JSON parser or deserializer consumes explicitly untrusted external, network, file, or public-API input.", "Keep the finding prioritized for byte, depth, item-count, cancellation, and schema validation controls."), + new("ambiguous_trust", "The JSON trust boundary is missing, invalid, directionally inconsistent, or still marked review-required.", "Add or correct a source-proximate cdidx-audit json-trust annotation; do not suppress the raw recipe result.") + ], + ["audit_classifications.evidence", "source_annotation", "recipe_query_direction", "risk_evidence", "match_origins"], + "Use `// cdidx-audit: json-trust origin= direction= sensitivity= trust= rationale=` immediately before the operation. Annotations classify but never suppress findings; missing or invalid evidence remains ambiguous."); private static readonly SearchRecipeClassifierJsonResult ProcessLaunchClassifier = new( "process_launch_boundary", "Classifies process-launch hits by shell use, ArgumentList use, working directory, environment forwarding, and shared launch wrappers.", @@ -1311,6 +1322,8 @@ private static SearchAuditRecipeQuery TimestampBoundaryQuery( "positive: machine-only payloads with explicit content-type and no HTML/script embedding are lower risk." ], MatchOrigins = ["code"], + Classifiers = [JsonTrustBoundaryClassifier], + JsonTrustDirection = SearchRecipeJsonTrustDirection.Write, }, new( "temp-file-name", @@ -2031,6 +2044,8 @@ private static SearchAuditRecipeQuery TimestampBoundaryQuery( "risk: JsonDocument.Parse builds a full DOM and should show byte, depth, and item-count limits before user-controlled payloads reach it.", "positive: BoundedJson.ParseDocument or a size-gated structured-data fallback is upstream guard evidence for intentional DOM parsing." ], + Classifiers = [JsonTrustBoundaryClassifier], + JsonTrustDirection = SearchRecipeJsonTrustDirection.Read, }, new( "json-node-parse", @@ -2045,6 +2060,8 @@ private static SearchAuditRecipeQuery TimestampBoundaryQuery( "risk: JsonNode.Parse materializes a mutable DOM and should be paired with payload and depth bounds for API, config, or protocol inputs.", "positive: BoundedJson.ParseNode, bounded frame readers, or fixed-size local metadata files make the materialization auditable." ], + Classifiers = [JsonTrustBoundaryClassifier], + JsonTrustDirection = SearchRecipeJsonTrustDirection.Read, }, new( "json-serializer-deserialize", @@ -2059,6 +2076,8 @@ private static SearchAuditRecipeQuery TimestampBoundaryQuery( "risk: JsonSerializer.Deserialize can materialize an entire object graph before callers enforce semantic item limits.", "positive: BoundedJson.Deserialize, MaxDepth options, and fixed protocol frame byte caps show upstream parse bounds." ], + Classifiers = [JsonTrustBoundaryClassifier], + JsonTrustDirection = SearchRecipeJsonTrustDirection.Read, }, new( "json-async-deserialize", @@ -2073,6 +2092,8 @@ private static SearchAuditRecipeQuery TimestampBoundaryQuery( "risk: streaming deserialization still needs cancellation, per-item limits, and a bounded source stream.", "positive: WithCancellation, explicit record caps, and max-byte snapshot reads show streaming backpressure evidence." ], + Classifiers = [JsonTrustBoundaryClassifier], + JsonTrustDirection = SearchRecipeJsonTrustDirection.Read, }, new( "json-serializer-options", @@ -2115,6 +2136,8 @@ private static SearchAuditRecipeQuery TimestampBoundaryQuery( "risk: JsonSerializer.Serialize can materialize unbounded output when result sets scale with workspace size.", "positive: bounded result limits, Utf8JsonWriter streaming, output caps, or small fixed DTOs make serialization size explicit." ], + Classifiers = [JsonTrustBoundaryClassifier], + JsonTrustDirection = SearchRecipeJsonTrustDirection.Write, }, new( "utf8-json-writer", @@ -2129,6 +2152,8 @@ private static SearchAuditRecipeQuery TimestampBoundaryQuery( "risk: streaming writers still need bounded destinations, cancellation or flush ownership, and redaction policy at user-facing boundaries.", "positive: writing directly to a caller-owned stream, LocalJsonlJsonWriterOptions, or fixed-size diagnostic payloads can explain the writer." ], + Classifiers = [JsonTrustBoundaryClassifier], + JsonTrustDirection = SearchRecipeJsonTrustDirection.Write, } ], ParserGuardClassifier, GuardEvidenceClassifier)), SourceScopedRecipe( @@ -4376,6 +4401,7 @@ internal sealed record SearchAuditRecipeQuery( public List ExcludeOrigins { get; init; } = []; public List ResultKinds { get; init; } = []; public List Classifiers { get; init; } = []; + public SearchRecipeJsonTrustDirection? JsonTrustDirection { get; init; } public SearchResultRanking ResultRanking { get; init; } public SearchRecipeStringComparisonTaxonomyJsonResult? StringComparisonTaxonomy { get; init; } public SearchRecipeBroadCatchTaxonomyJsonResult? BroadCatchTaxonomy { get; init; } @@ -4390,6 +4416,12 @@ internal enum SearchRecipeSemanticFilter ShellExecuteAssignment, } +internal enum SearchRecipeJsonTrustDirection +{ + Read, + Write, +} + internal sealed record SearchRecipeListJsonResult( [property: JsonPropertyName("api_version")] string ApiVersion, [property: JsonPropertyName("count")] int Count, diff --git a/src/CodeIndex/Diagnostics/LocalJsonlJsonWriterOptions.cs b/src/CodeIndex/Diagnostics/LocalJsonlJsonWriterOptions.cs index e1650651a..f588af67d 100644 --- a/src/CodeIndex/Diagnostics/LocalJsonlJsonWriterOptions.cs +++ b/src/CodeIndex/Diagnostics/LocalJsonlJsonWriterOptions.cs @@ -17,6 +17,7 @@ internal static JsonWriterOptions Create() return new JsonWriterOptions { Indented = false, + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=operator_only_local_jsonl Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping, }; } diff --git a/tests/CodeIndex.Tests/QueryCommandRunnerSearchTests.cs b/tests/CodeIndex.Tests/QueryCommandRunnerSearchTests.cs index d10950458..fd39828b8 100644 --- a/tests/CodeIndex.Tests/QueryCommandRunnerSearchTests.cs +++ b/tests/CodeIndex.Tests/QueryCommandRunnerSearchTests.cs @@ -3304,6 +3304,1547 @@ static void AssertTaskResultClassification(JsonElement result, string expectedCa } } + [Fact] + public void RunSearch_JsonTrustBoundaryClassifiesPrivateWritersAndUntrustedParsers_Issue4913() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_search_json_trust_boundary_4913"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/private-writer.cs", + "csharp", + """ + using System.Text.Encodings.Web; + + public static class PrivateWriter + { + public static JavaScriptEncoder Create() + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=operator_only_local_jsonl + return JavaScriptEncoder.UnsafeRelaxedJsonEscaping; + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/public-writer.cs", + "csharp", + """ + using System.Text.Encodings.Web; + + public static class PublicWriter + { + public static JavaScriptEncoder Create() + { + // cdidx-audit: json-trust origin=public_api direction=write sensitivity=public trust=untrusted rationale=http_response_body + return JavaScriptEncoder.UnsafeRelaxedJsonEscaping; + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/review-required-public-writer.cs", + "csharp", + """ + using System.Text.Encodings.Web; + + public static class ReviewRequiredPublicWriter + { + public static JavaScriptEncoder Create() + { + // cdidx-audit: json-trust origin=public_api direction=write sensitivity=public trust=review_required rationale=public_sink_needs_review + return JavaScriptEncoder.UnsafeRelaxedJsonEscaping; + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/mixed-boundary-writer.cs", + "csharp", + """ + using System.Text.Encodings.Web; + + public static class MixedBoundaryWriter + { + public static JavaScriptEncoder Create() + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=private_diagnostic + var privateEncoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping; + // cdidx-audit: json-trust origin=public_api direction=write sensitivity=public trust=untrusted rationale=public_response + var publicEncoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping; + return publicEncoder; + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/annotation-bleed-writer.cs", + "csharp", + """ + using System.Text.Encodings.Web; + + public static class AnnotationBleedWriter + { + public static JavaScriptEncoder Create() + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=only_first_operation + var first = JavaScriptEncoder.UnsafeRelaxedJsonEscaping; + var second = JavaScriptEncoder.UnsafeRelaxedJsonEscaping; + return second; + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/same-line-intervening-writer.cs", + "csharp", + """ + using System.Text.Encodings.Web; + + public static class SameLineInterveningWriter + { + public static JavaScriptEncoder Create() + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=next_operation_only + Console.WriteLine("intervening"); return JavaScriptEncoder.UnsafeRelaxedJsonEscaping; + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/same-line-block-intervening-writer.cs", + "csharp", + """ + using System.Text.Encodings.Web; + + public static class SameLineBlockInterveningWriter + { + public static JavaScriptEncoder Create(bool skip) + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=next_operation_only + if (skip) { } return JavaScriptEncoder.UnsafeRelaxedJsonEscaping; + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/same-line-multiple-writer.cs", + "csharp", + """ + using System.Text.Encodings.Web; + + public static class SameLineMultipleWriter + { + public static object Create() + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=first_operation_only + return (JavaScriptEncoder.UnsafeRelaxedJsonEscaping, JavaScriptEncoder.UnsafeRelaxedJsonEscaping); + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/same-line-prior-getter-writer.cs", + "csharp", + """ + using System.Text.Encodings.Web; + + public sealed class Source + { + public string Value => "prefix"; + } + + public static class SameLinePriorGetterWriter + { + public static object Create(Source source) + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=next_operation_only + return source.Value + JavaScriptEncoder.UnsafeRelaxedJsonEscaping; + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/first-named-argument-writer.cs", + "csharp", + """ + using System.Text.Encodings.Web; + + public static class FirstNamedArgumentWriter + { + public static JavaScriptEncoder Create() + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=first_argument_executes_first + return Consume(encoder: JavaScriptEncoder.UnsafeRelaxedJsonEscaping); + } + + private static JavaScriptEncoder Consume(JavaScriptEncoder encoder) => encoder; + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/nullable-declaration-writer.cs", + "csharp", + """ + using System.Text.Encodings.Web; + + public static class NullableDeclarationWriter + { + public static JavaScriptEncoder? Create() + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=nullable_declaration_is_not_an_operand + JavaScriptEncoder? encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping; + return encoder; + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/comparison-operand-writer.cs", + "csharp", + """ + using System.Text.Encodings.Web; + + public static class ComparisonOperandWriter + { + public static bool Create(object existing) + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=next_operation_only + return existing == JavaScriptEncoder.UnsafeRelaxedJsonEscaping; + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/nested-generic-first-argument-writer.cs", + "csharp", + """ + using System.Collections.Generic; + using System.Text.Encodings.Web; + + public static class NestedGenericFirstArgumentWriter + { + public static JavaScriptEncoder Create() + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=nested_generic_first_argument + return Consume>(JavaScriptEncoder.UnsafeRelaxedJsonEscaping); + } + + private static JavaScriptEncoder Consume(JavaScriptEncoder encoder) => encoder; + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/indexer-target-writer.cs", + "csharp", + """ + using System.Text.Encodings.Web; + + public sealed class IndexHolder + { + public int Index => 0; + } + + public static class IndexerTargetWriter + { + public static JavaScriptEncoder Create(JavaScriptEncoder[] sink, IndexHolder holder) + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=next_operation_only + sink[holder.Index] = JavaScriptEncoder.UnsafeRelaxedJsonEscaping; + return sink[holder.Index]; + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/property-receiver-writer.cs", + "csharp", + """ + using System.Text.Encodings.Web; + + public sealed class ReceiverSink + { + public ReceiverOptions Options => new(); + } + + public sealed class ReceiverOptions + { + public JavaScriptEncoder? Encoder { get; set; } + } + + public static class PropertyReceiverWriter + { + public static void Create(ReceiverSink sink) + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=next_operation_only + sink.Options.Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping; + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/single-hop-property-receiver-writer.cs", + "csharp", + """ + using System.Text.Encodings.Web; + + public sealed class SingleHopReceiverOptions + { + public JavaScriptEncoder? Encoder { get; set; } + } + + public static class SingleHopPropertyReceiverWriter + { + private static SingleHopReceiverOptions Options => new(); + + public static void Create() + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=next_operation_only + Options.Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping; + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/private-looking-writer.cs", + "csharp", + """ + using System.Text.Encodings.Web; + + public static class AmbiguousWriter + { + public static JavaScriptEncoder Create() + { + var fakeAnnotation = "// cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=string_literal_is_not_evidence"; + Console.WriteLine(fakeAnnotation); + return JavaScriptEncoder.UnsafeRelaxedJsonEscaping; + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/direction-mismatch-writer.cs", + "csharp", + """ + using System.Text.Encodings.Web; + + public static class DirectionMismatchWriter + { + public static JavaScriptEncoder Create() + { + // cdidx-audit: json-trust origin=network direction=read sensitivity=untrusted trust=untrusted rationale=inbound_payload + return JavaScriptEncoder.UnsafeRelaxedJsonEscaping; + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/invalid-annotation-writer.cs", + "csharp", + """ + using System.Text.Encodings.Web; + + public static class InvalidAnnotationWriter + { + public static JavaScriptEncoder Create() + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled + return JavaScriptEncoder.UnsafeRelaxedJsonEscaping; + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/raw-string-writer.cs", + "csharp", + """" + using System.Text.Encodings.Web; + + public static class RawStringWriter + { + public static JavaScriptEncoder Create() + { + var documentation = """ + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=raw_string_is_not_evidence + """; + return JavaScriptEncoder.UnsafeRelaxedJsonEscaping; + } + } + """"); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/verbatim-string-writer.cs", + "csharp", + """ + using System.Text.Encodings.Web; + + public static class VerbatimStringWriter + { + public static JavaScriptEncoder Create() + { + var documentation = @" + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=verbatim_string_is_not_evidence + "; + return JavaScriptEncoder.UnsafeRelaxedJsonEscaping; + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/nested-serializer-writer.cs", + "csharp", + """ + using System.Text.Json; + + public static class NestedSerializerWriter + { + public static string Create(object payload) + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=only_next_json_operation + return JsonSerializer.Serialize(JsonSerializer.Serialize(payload)); + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/cast-serializer-writer.cs", + "csharp", + """ + using System.Text.Json; + + public static class CastSerializerWriter + { + public static object Create(object payload) + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=cast_applies_after_serialization + return (object)JsonSerializer.Serialize(payload); + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/multiline-serializer-writer.cs", + "csharp", + """ + using System.Text.Json; + + public static class MultilineSerializerWriter + { + public static string Create(object payload) + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=continued_assignment_prefix + var json = + JsonSerializer.Serialize(payload); + return json; + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/conditional-assignment-serializer-writer.cs", + "csharp", + """ + using System.Text.Json; + + public static class ConditionalAssignmentSerializerWriter + { + public static string Create(bool condition, object payload, ref string result) + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=conditional_operand_executes_first + return condition ? (result = JsonSerializer.Serialize(payload)) : result; + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/default-parameter-expression-bodied-serializer-writer.cs", + "csharp", + """ + using System.Text.Json; + + public static class DefaultParameterExpressionBodiedSerializerWriter + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=default_parameter_is_declaration_only + public static string Create(object payload, int count = 1) => JsonSerializer.Serialize(payload); + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/property-invocation-receiver-writer.cs", + "csharp", + """ + using System.Text.Json; + + public sealed class PropertyInvocationSource + { + public PropertyInvocationBuilder Factory => new(); + } + + public sealed class PropertyInvocationBuilder + { + public string Build(string json) => json; + } + + public static class PropertyInvocationReceiverWriter + { + public static string Create(PropertyInvocationSource source, object payload) + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=property_getter_executes_first + return source.Factory.Build(JsonSerializer.Serialize(payload)); + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/direct-invocation-receiver-writer.cs", + "csharp", + """ + using System.Text.Json; + + public sealed class DirectInvocationSource + { + public string Build(string json) => json; + } + + public static class DirectInvocationReceiverWriter + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=declared_direct_receiver_is_side_effect_free + public static string Create(DirectInvocationSource source, object payload) + => source.Build(JsonSerializer.Serialize(payload)); + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/bare-property-invocation-receiver-writer.cs", + "csharp", + """ + using System.Text.Json; + + public sealed class BarePropertyInvocationBuilder + { + public string Build(string json) => json; + } + + public static class BarePropertyInvocationReceiverWriter + { + private static BarePropertyInvocationBuilder Factory => new(); + + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=bare_property_getter_executes_first + public static string Create(object payload) + => Factory.Build(JsonSerializer.Serialize(payload)); + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/conditional-compilation-writer.cs", + "csharp", + """ + using System.Text.Json; + + public static class ConditionalCompilationWriter + { + # if false + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=inactive_annotation + # endif + public static string Create(object payload) => JsonSerializer.Serialize(payload); + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/parenthesized-conditional-compilation-writer.cs", + "csharp", + """ + using System.Text.Json; + + public static class ParenthesizedConditionalCompilationWriter + { + #if(false) + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=inactive_parenthesized_annotation + #endif + public static string Create(object payload) => JsonSerializer.Serialize(payload); + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/negated-conditional-compilation-writer.cs", + "csharp", + """ + using System.Text.Json; + + public static class NegatedConditionalCompilationWriter + { + #if!DEBUG + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=inactive_negated_annotation + #endif + public static string Create(object payload) => JsonSerializer.Serialize(payload); + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/repeated-assignment-property-receiver-writer.cs", + "csharp", + """ + using System.Text.Json; + + public sealed class RepeatedAssignmentBuilder + { + public RepeatedAssignmentBuilder Build(string json) => this; + } + + public static class RepeatedAssignmentPropertyReceiverWriter + { + private static RepeatedAssignmentBuilder Factory { get; set; } = new(); + + public static void Create(object payload) + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=repeated_assignment_property_getter_executes_first + Factory = Factory.Build(JsonSerializer.Serialize(payload)); + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/explicit-utf8-writer.cs", + "csharp", + """ + using System.IO; + using System.Text.Json; + + public static class ExplicitUtf8Writer + { + public static void Create(Stream stream) + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=private_stream_writer + using Utf8JsonWriter writer = new Utf8JsonWriter(stream); + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/multiline-explicit-utf8-writer.cs", + "csharp", + """ + using System.IO; + using System.Text.Json; + + public static class MultilineExplicitUtf8Writer + { + public static void Create(Stream stream) + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=multiline_typed_declaration + Utf8JsonWriter writer = + new Utf8JsonWriter(stream); + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/long-multiline-utf8-writer.cs", + "csharp", + """ + using System.IO; + using System.Text.Json; + + public static class LongMultilineUtf8Writer + { + public static void Create(Stream stream) + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=bounded_statement_continuation + using + var + writer + = + new + Utf8JsonWriter(stream); + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/expression-bodied-utf8-writer.cs", + "csharp", + """ + using System.IO; + using System.Text.Json; + + public static class ExpressionBodiedUtf8Writer + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=expression_bodied_private_writer + public static Utf8JsonWriter Create(Stream stream) => new Utf8JsonWriter(stream); + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/generic-return-utf8-writer.cs", + "csharp", + """ + using System.IO; + using System.Text.Json; + + public sealed record GenericWriterHolder(T Value); + + public static class GenericReturnUtf8Writer + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=generic_return_private_writer + public static GenericWriterHolder Create(Stream stream) => new(new Utf8JsonWriter(stream)); + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/outer-generic-return-utf8-writer.cs", + "csharp", + """ + using System; + using System.IO; + using System.Text.Json; + + public static class OuterGenericReturnUtf8Writer + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=outer_generic_return_private_writer + public static Tuple Create(Stream stream) => new(new Utf8JsonWriter(stream), 1); + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/split-generic-return-utf8-writer.cs", + "csharp", + """ + using System.IO; + using System.Text.Json; + + public sealed record SplitGenericWriterHolder(T Value); + + public static class SplitGenericReturnUtf8Writer + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=split_generic_return_private_writer + public static SplitGenericWriterHolder< + Utf8JsonWriter> Create(Stream stream) => + new(new Utf8JsonWriter(stream)); + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/qualified-utf8-writer.cs", + "csharp", + """ + using System.IO; + + public static class QualifiedUtf8Writer + { + public static void Create(Stream stream) + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=qualified_local_declaration + System.Text.Json.Utf8JsonWriter writer = new System.Text.Json.Utf8JsonWriter(stream); + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/cross-query-writer.cs", + "csharp", + """ + using System.IO; + using System.Text.Json; + + public static class CrossQueryWriter + { + public static string Create(Stream stream) + { + // cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=first_selected_json_operation_only + return JsonSerializer.Serialize(new Utf8JsonWriter(stream)); + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/overlapping-parser.cs", + "csharp", + """ + using System.IO; + using System.Text.Json; + + public static class OverlappingParser + { + public static object Parse(Stream stream) + { + // cdidx-audit: json-trust origin=network direction=read sensitivity=untrusted trust=untrusted rationale=first_overlapping_query_only + return JsonSerializer.@DeserializeAsyncEnumerable(stream); + } + } + """); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/composite-cast-parser.cs", + "csharp", + """ + using System.Collections.Generic; + using System.Text.Json; + + public static class CompositeCastParser + { + public static IEnumerable? Parse(string payload) + { + // cdidx-audit: json-trust origin=file direction=read sensitivity=untrusted trust=untrusted rationale=generic_nullable_cast_input + return (IEnumerable?)JsonSerializer.Deserialize>(payload); + } + } + """); + + foreach (var (path, origin, rationale) in new[] + { + ("src/external-parser.cs", "external", "plugin_payload"), + ("src/network-parser.cs", "network", "http_request_body"), + ("src/file-parser.cs", "file", "user_selected_file"), + }) + { + TestProjectHelper.InsertIndexedFile( + dbPath, + path, + "csharp", + $$""" + using System.Text.Json; + + public static class Parser + { + public static JsonDocument Parse(string payload) + { + // cdidx-audit: json-trust origin={{origin}} direction=read sensitivity=untrusted trust=untrusted rationale={{rationale}} + return JsonDocument.Parse(payload); + } + } + """); + } + + var (writerExitCode, writerStdout, writerStderr) = CaptureConsole(() => QueryCommandRunner.RunSearch( + ["--recipe", "dogfood-risk-patterns/relaxed-json-encoder", "--db", dbPath, "--json", "--limit", "30", "--snippet-lines", "1"], + _jsonOptions)); + var (parserExitCode, parserStdout, parserStderr) = CaptureConsole(() => QueryCommandRunner.RunSearch( + ["--recipe", "json-parse-apis/json-document-parse", "--db", dbPath, "--json", "--limit", "10", "--snippet-lines", "1"], + _jsonOptions)); + var (nestedExitCode, nestedStdout, nestedStderr) = CaptureConsole(() => QueryCommandRunner.RunSearch( + ["--recipe", "json-parse-apis/json-serializer-serialize", "--db", dbPath, "--path", "src/nested-serializer-writer.cs", "--path", "src/cast-serializer-writer.cs", "--path", "src/multiline-serializer-writer.cs", "--path", "src/conditional-assignment-serializer-writer.cs", "--path", "src/default-parameter-expression-bodied-serializer-writer.cs", "--path", "src/property-invocation-receiver-writer.cs", "--path", "src/direct-invocation-receiver-writer.cs", "--path", "src/bare-property-invocation-receiver-writer.cs", "--path", "src/conditional-compilation-writer.cs", "--path", "src/parenthesized-conditional-compilation-writer.cs", "--path", "src/negated-conditional-compilation-writer.cs", "--path", "src/repeated-assignment-property-receiver-writer.cs", "--json", "--limit", "20", "--snippet-lines", "1"], + _jsonOptions)); + var (utf8ExitCode, utf8Stdout, utf8Stderr) = CaptureConsole(() => QueryCommandRunner.RunSearch( + ["--recipe", "json-parse-apis/utf8-json-writer", "--db", dbPath, "--path", "src/explicit-utf8-writer.cs", "--path", "src/multiline-explicit-utf8-writer.cs", "--path", "src/long-multiline-utf8-writer.cs", "--path", "src/expression-bodied-utf8-writer.cs", "--path", "src/generic-return-utf8-writer.cs", "--path", "src/outer-generic-return-utf8-writer.cs", "--path", "src/split-generic-return-utf8-writer.cs", "--path", "src/qualified-utf8-writer.cs", "--json", "--limit", "10", "--snippet-lines", "1"], + _jsonOptions)); + var (crossQueryExitCode, crossQueryStdout, crossQueryStderr) = CaptureConsole(() => QueryCommandRunner.RunSearch( + [ + "--recipe", "json-parse-apis", + "--include-query", "utf8-json-writer", + "--include-query", "json-serializer-serialize", + "--db", dbPath, + "--path", "src/cross-query-writer.cs", + "--json", + "--limit", "10", + "--snippet-lines", "1", + ], + _jsonOptions)); + var (overlapQueryExitCode, overlapQueryStdout, overlapQueryStderr) = CaptureConsole(() => QueryCommandRunner.RunSearch( + [ + "--recipe", "json-parse-apis", + "--include-query", "json-serializer-deserialize", + "--include-query", "json-async-deserialize", + "--db", dbPath, + "--path", "src/overlapping-parser.cs", + "--path", "src/composite-cast-parser.cs", + "--json", + "--limit", "10", + "--snippet-lines", "1", + ], + _jsonOptions)); + var (guardedExitCode, guardedStdout, guardedStderr) = CaptureConsole(() => QueryCommandRunner.RunSearch( + [ + "--recipe", "dogfood-risk-patterns/relaxed-json-encoder", + "--db", dbPath, + "--path", "src/private-writer.cs", + "--require-before", "JavaScriptEncoder", + "--guard-scope", "same-line", + "--json", + "--limit", "10", + "--snippet-lines", "1", + ], + _jsonOptions)); + var (guardedCountExitCode, guardedCountStdout, guardedCountStderr) = CaptureConsole(() => QueryCommandRunner.RunSearch( + [ + "--recipe", "dogfood-risk-patterns/relaxed-json-encoder", + "--db", dbPath, + "--path", "src/private-writer.cs", + "--require-before", "JavaScriptEncoder", + "--guard-scope", "same-line", + "--format", "count", + ], + _jsonOptions)); + var (bleedGuardExitCode, bleedGuardStdout, bleedGuardStderr) = CaptureConsole(() => QueryCommandRunner.RunSearch( + [ + "--recipe", "dogfood-risk-patterns/relaxed-json-encoder", + "--db", dbPath, + "--path", "src/annotation-bleed-writer.cs", + "--require-before", "second", + "--guard-scope", "same-line", + "--json", + "--limit", "10", + "--snippet-lines", "1", + ], + _jsonOptions)); + var (projectedExitCode, projectedStdout, projectedStderr) = CaptureConsole(() => QueryCommandRunner.RunSearch( + [ + "--recipe", "dogfood-risk-patterns/relaxed-json-encoder", + "--db", dbPath, + "--path", "src/private-writer.cs", + "--results-only", + "--search-fields", "path,query_name", + "--limit", "10", + ], + _jsonOptions)); + var (compactSummaryExitCode, compactSummaryStdout, compactSummaryStderr) = CaptureConsole(() => QueryCommandRunner.RunSearch( + [ + "--recipe", "dogfood-risk-patterns/relaxed-json-encoder", + "--db", dbPath, + "--path", "src/private-writer.cs", + "--format", "compact", + "--summary-only", + "--json", + "--limit", "10", + ], + _jsonOptions)); + + Assert.Equal(CommandExitCodes.Success, writerExitCode); + Assert.Equal(string.Empty, writerStderr); + using (var document = ParseJsonOutput(writerStdout)) + { + var query = Assert.Single(document.RootElement.GetProperty("queries").EnumerateArray()); + Assert.Equal(21, query.GetProperty("count").GetInt32()); + var trustClassifier = Assert.Single( + query.GetProperty("classifiers").EnumerateArray(), + classifier => classifier.GetProperty("name").GetString() == "json_trust_boundary"); + Assert.Contains( + trustClassifier.GetProperty("evidence_fields").EnumerateArray(), + field => field.GetString() == "source_annotation"); + Assert.Contains( + trustClassifier.GetProperty("evidence_fields").EnumerateArray(), + field => field.GetString() == "recipe_query_direction"); + AssertJsonTrustClassifierCounts( + query, + ("controlled_private_writer", 4), + ("external_or_public_writer", 1), + ("ambiguous_trust", 16)); + + var results = query.GetProperty("results").EnumerateArray().ToArray(); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/private-writer.cs"), + "controlled_private_writer", + "origin:private_local", + "trust:controlled"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/public-writer.cs"), + "external_or_public_writer", + "origin:public_api", + "trust:untrusted"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/review-required-public-writer.cs"), + "ambiguous_trust", + "origin:public_api", + "trust:review_required"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/mixed-boundary-writer.cs"), + "ambiguous_trust", + "annotation_status:mixed_boundaries", + "boundary_categories:controlled_private_writer,external_or_public_writer"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/annotation-bleed-writer.cs"), + "ambiguous_trust", + "annotation_status:mixed_boundaries", + "boundary_categories:ambiguous_trust,controlled_private_writer"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/same-line-intervening-writer.cs"), + "ambiguous_trust", + "rationale:annotation_not_bound_to_operation", + "annotation_status:not_adjacent"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/same-line-block-intervening-writer.cs"), + "ambiguous_trust", + "rationale:annotation_not_bound_to_operation", + "annotation_status:not_adjacent"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/same-line-multiple-writer.cs"), + "ambiguous_trust", + "annotation_status:mixed_boundaries", + "boundary_categories:ambiguous_trust,controlled_private_writer"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/same-line-prior-getter-writer.cs"), + "ambiguous_trust", + "rationale:annotation_not_bound_to_operation", + "annotation_status:not_adjacent"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/first-named-argument-writer.cs"), + "controlled_private_writer", + "rationale:first_argument_executes_first", + "annotation_status:valid"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/nullable-declaration-writer.cs"), + "controlled_private_writer", + "rationale:nullable_declaration_is_not_an_operand", + "annotation_status:valid"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/comparison-operand-writer.cs"), + "ambiguous_trust", + "rationale:annotation_not_bound_to_operation", + "annotation_status:not_adjacent"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/nested-generic-first-argument-writer.cs"), + "controlled_private_writer", + "rationale:nested_generic_first_argument", + "annotation_status:valid"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/indexer-target-writer.cs"), + "ambiguous_trust", + "rationale:annotation_not_bound_to_operation", + "annotation_status:not_adjacent"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/property-receiver-writer.cs"), + "ambiguous_trust", + "rationale:annotation_not_bound_to_operation", + "annotation_status:not_adjacent"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/single-hop-property-receiver-writer.cs"), + "ambiguous_trust", + "rationale:annotation_not_bound_to_operation", + "annotation_status:not_adjacent"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/private-looking-writer.cs"), + "ambiguous_trust", + "origin:unknown", + "annotation_status:missing"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/direction-mismatch-writer.cs"), + "ambiguous_trust", + "direction:read", + "annotation_status:direction_mismatch"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/invalid-annotation-writer.cs"), + "ambiguous_trust", + "origin:unknown", + "annotation_status:invalid"); + foreach (var path in new[] { "src/raw-string-writer.cs", "src/verbatim-string-writer.cs" }) + { + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == path), + "ambiguous_trust", + "origin:unknown", + "annotation_status:invalid"); + } + } + + Assert.Equal(CommandExitCodes.Success, guardedExitCode); + Assert.Equal(string.Empty, guardedStderr); + using (var document = ParseJsonOutput(guardedStdout)) + { + var query = Assert.Single(document.RootElement.GetProperty("queries").EnumerateArray()); + AssertJsonTrustClassifierCounts(query, ("controlled_private_writer", 1)); + AssertJsonTrustClassification( + Assert.Single(query.GetProperty("results").EnumerateArray()), + "controlled_private_writer", + "origin:private_local", + "annotation_status:valid"); + } + + Assert.Equal(CommandExitCodes.Success, guardedCountExitCode); + Assert.Equal(string.Empty, guardedCountStderr); + using (var document = ParseJsonOutput(guardedCountStdout)) + { + var query = Assert.Single(document.RootElement.GetProperty("queries").EnumerateArray()); + AssertJsonTrustClassifierCounts(query, ("controlled_private_writer", 1)); + } + + Assert.Equal(CommandExitCodes.Success, bleedGuardExitCode); + Assert.Equal(string.Empty, bleedGuardStderr); + using (var document = ParseJsonOutput(bleedGuardStdout)) + { + var query = Assert.Single(document.RootElement.GetProperty("queries").EnumerateArray()); + AssertJsonTrustClassifierCounts(query, ("ambiguous_trust", 1)); + AssertJsonTrustClassification( + Assert.Single(query.GetProperty("results").EnumerateArray()), + "ambiguous_trust", + "rationale:annotation_not_bound_to_operation", + "annotation_status:not_adjacent"); + } + + Assert.Equal(CommandExitCodes.Success, projectedExitCode); + Assert.Equal(string.Empty, projectedStderr); + using (var document = ParseJsonOutput(projectedStdout)) + { + var result = document.RootElement; + Assert.Equal("src/private-writer.cs", result.GetProperty("path").GetString()); + Assert.Equal("relaxed-json-encoder", result.GetProperty("query_name").GetString()); + Assert.False(result.TryGetProperty("audit_classifications", out _)); + } + + Assert.Equal(CommandExitCodes.Success, parserExitCode); + Assert.Equal(string.Empty, parserStderr); + using (var document = ParseJsonOutput(parserStdout)) + { + var query = Assert.Single(document.RootElement.GetProperty("queries").EnumerateArray()); + Assert.Equal(3, query.GetProperty("count").GetInt32()); + AssertJsonTrustClassifierCounts(query, ("untrusted_parser", 3)); + + foreach (var result in query.GetProperty("results").EnumerateArray()) + { + AssertJsonTrustClassification( + result, + "untrusted_parser", + "direction:read", + "trust:untrusted"); + } + } + + Assert.Equal(CommandExitCodes.Success, nestedExitCode); + Assert.Equal(string.Empty, nestedStderr); + using (var document = ParseJsonOutput(nestedStdout)) + { + var query = Assert.Single(document.RootElement.GetProperty("queries").EnumerateArray()); + AssertJsonTrustClassifierCounts( + query, + ("ambiguous_trust", 8), + ("controlled_private_writer", 4)); + var results = query.GetProperty("results").EnumerateArray().ToArray(); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/nested-serializer-writer.cs"), + "ambiguous_trust", + "rationale:multiple_trust_boundaries", + "annotation_status:mixed_boundaries", + "match_site_count:2", + "boundary_categories:ambiguous_trust,controlled_private_writer"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/cast-serializer-writer.cs"), + "controlled_private_writer", + "rationale:cast_applies_after_serialization", + "annotation_status:valid"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/multiline-serializer-writer.cs"), + "controlled_private_writer", + "rationale:continued_assignment_prefix", + "annotation_status:valid"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/conditional-assignment-serializer-writer.cs"), + "ambiguous_trust", + "rationale:annotation_not_bound_to_operation", + "annotation_status:not_adjacent"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/default-parameter-expression-bodied-serializer-writer.cs"), + "controlled_private_writer", + "rationale:default_parameter_is_declaration_only", + "annotation_status:valid"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/property-invocation-receiver-writer.cs"), + "ambiguous_trust", + "rationale:annotation_not_bound_to_operation", + "annotation_status:not_adjacent"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/direct-invocation-receiver-writer.cs"), + "controlled_private_writer", + "rationale:declared_direct_receiver_is_side_effect_free", + "annotation_status:valid"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/bare-property-invocation-receiver-writer.cs"), + "ambiguous_trust", + "rationale:annotation_not_bound_to_operation", + "annotation_status:not_adjacent"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/conditional-compilation-writer.cs"), + "ambiguous_trust", + "rationale:conditional_compilation_annotation", + "annotation_status:not_adjacent"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/parenthesized-conditional-compilation-writer.cs"), + "ambiguous_trust", + "rationale:conditional_compilation_annotation", + "annotation_status:not_adjacent"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/negated-conditional-compilation-writer.cs"), + "ambiguous_trust", + "rationale:conditional_compilation_annotation", + "annotation_status:not_adjacent"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/repeated-assignment-property-receiver-writer.cs"), + "ambiguous_trust", + "rationale:annotation_not_bound_to_operation", + "annotation_status:not_adjacent"); + } + + Assert.Equal(CommandExitCodes.Success, utf8ExitCode); + Assert.Equal(string.Empty, utf8Stderr); + using (var document = ParseJsonOutput(utf8Stdout)) + { + var query = Assert.Single(document.RootElement.GetProperty("queries").EnumerateArray()); + AssertJsonTrustClassifierCounts(query, ("controlled_private_writer", 8)); + var results = query.GetProperty("results").EnumerateArray().ToArray(); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/explicit-utf8-writer.cs"), + "controlled_private_writer", + "rationale:private_stream_writer", + "annotation_status:valid"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/multiline-explicit-utf8-writer.cs"), + "controlled_private_writer", + "rationale:multiline_typed_declaration", + "annotation_status:valid"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/long-multiline-utf8-writer.cs"), + "controlled_private_writer", + "rationale:bounded_statement_continuation", + "annotation_status:valid"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/expression-bodied-utf8-writer.cs"), + "controlled_private_writer", + "rationale:expression_bodied_private_writer", + "annotation_status:valid"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/generic-return-utf8-writer.cs"), + "controlled_private_writer", + "rationale:generic_return_private_writer", + "annotation_status:valid"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/outer-generic-return-utf8-writer.cs"), + "controlled_private_writer", + "rationale:outer_generic_return_private_writer", + "annotation_status:valid"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/split-generic-return-utf8-writer.cs"), + "controlled_private_writer", + "rationale:split_generic_return_private_writer", + "annotation_status:valid"); + AssertJsonTrustClassification( + Assert.Single(results, result => result.GetProperty("path").GetString() == "src/qualified-utf8-writer.cs"), + "controlled_private_writer", + "rationale:qualified_local_declaration", + "annotation_status:valid"); + } + + Assert.Equal(CommandExitCodes.Success, crossQueryExitCode); + Assert.Equal(string.Empty, crossQueryStderr); + using (var document = ParseJsonOutput(crossQueryStdout)) + { + var queries = document.RootElement.GetProperty("queries").EnumerateArray().ToArray(); + var serializerQuery = Assert.Single( + queries, + query => query.GetProperty("name").GetString() == "json-serializer-serialize"); + var writerQuery = Assert.Single( + queries, + query => query.GetProperty("name").GetString() == "utf8-json-writer"); + AssertJsonTrustClassifierCounts(serializerQuery, ("controlled_private_writer", 1)); + AssertJsonTrustClassification( + Assert.Single(serializerQuery.GetProperty("results").EnumerateArray()), + "controlled_private_writer", + "rationale:first_selected_json_operation_only", + "annotation_status:valid"); + AssertJsonTrustClassifierCounts(writerQuery, ("ambiguous_trust", 1)); + AssertJsonTrustClassification( + Assert.Single(writerQuery.GetProperty("results").EnumerateArray()), + "ambiguous_trust", + "rationale:annotation_not_bound_to_operation", + "annotation_status:not_adjacent"); + } + + Assert.Equal(CommandExitCodes.Success, overlapQueryExitCode); + Assert.Equal(string.Empty, overlapQueryStderr); + using (var document = ParseJsonOutput(overlapQueryStdout)) + { + var queries = document.RootElement.GetProperty("queries").EnumerateArray().ToArray(); + var serializerQuery = Assert.Single( + queries, + query => query.GetProperty("name").GetString() == "json-serializer-deserialize"); + var asyncQuery = Assert.Single( + queries, + query => query.GetProperty("name").GetString() == "json-async-deserialize"); + AssertJsonTrustClassifierCounts(serializerQuery, ("untrusted_parser", 2)); + AssertJsonTrustClassification( + Assert.Single( + serializerQuery.GetProperty("results").EnumerateArray(), + result => result.GetProperty("path").GetString() == "src/overlapping-parser.cs"), + "untrusted_parser", + "rationale:first_overlapping_query_only", + "annotation_status:valid"); + AssertJsonTrustClassification( + Assert.Single( + serializerQuery.GetProperty("results").EnumerateArray(), + result => result.GetProperty("path").GetString() == "src/composite-cast-parser.cs"), + "untrusted_parser", + "rationale:generic_nullable_cast_input", + "annotation_status:valid"); + AssertJsonTrustClassifierCounts(asyncQuery, ("ambiguous_trust", 1)); + AssertJsonTrustClassification( + Assert.Single(asyncQuery.GetProperty("results").EnumerateArray()), + "ambiguous_trust", + "rationale:annotation_not_bound_to_operation", + "annotation_status:not_adjacent"); + } + + Assert.Equal(CommandExitCodes.Success, compactSummaryExitCode); + Assert.Equal(string.Empty, compactSummaryStderr); + using (var document = ParseJsonOutput(compactSummaryStdout)) + { + var query = Assert.Single(document.RootElement.GetProperty("queries").EnumerateArray()); + Assert.False(query.TryGetProperty("classifier_counts", out _)); + Assert.False(query.TryGetProperty("results", out _)); + } + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + + static void AssertJsonTrustClassifierCounts( + JsonElement query, + params (string Category, int Count)[] expected) + { + var counts = query.GetProperty("classifier_counts").EnumerateArray().ToArray(); + var trustCounts = Assert.Single(counts, count => count.GetProperty("classifier").GetString() == "json_trust_boundary"); + var categories = trustCounts.GetProperty("categories").EnumerateArray().ToArray(); + + foreach (var (category, count) in expected) + { + Assert.Contains(categories, item => + item.GetProperty("name").GetString() == category + && item.GetProperty("count").GetInt32() == count); + } + } + + static void AssertJsonTrustClassification( + JsonElement result, + string expectedCategory, + params string[] expectedEvidence) + { + var classification = Assert.Single( + result.GetProperty("audit_classifications").EnumerateArray(), + item => item.GetProperty("classifier").GetString() == "json_trust_boundary"); + Assert.Equal(expectedCategory, classification.GetProperty("category").GetString()); + var evidence = classification.GetProperty("evidence").EnumerateArray().Select(item => item.GetString()).ToArray(); + Assert.All(expectedEvidence, item => Assert.Contains(item, evidence)); + } + } + + [Fact] + public void RunSearch_JsonTrustBoundaryCacheReusesExhaustedActualPrefix_Issue4913() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_search_json_trust_cache_4913"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + const string path = "src/cache-order-writer.cs"; + var lowChunk = string.Join( + '\n', + "public static class CacheOrderWriter {", + "// cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=bounded_low_prefix", + "var low = JavaScriptEncoder.UnsafeRelaxedJsonEscaping;", + new string(' ', 4 * 1024 * 1024)); + const string highChunk = """ + // cdidx-audit: json-trust origin=public_api direction=write sensitivity=public trust=untrusted rationale=beyond_source_budget + var high = JavaScriptEncoder.UnsafeRelaxedJsonEscaping; + """; + const string laterHighChunk = """ + // cdidx-audit: json-trust origin=public_api direction=write sensitivity=public trust=untrusted rationale=also_beyond_source_budget + var laterHigh = JavaScriptEncoder.UnsafeRelaxedJsonEscaping; + """; + + using (var db = new DbContext(DbOpenIntent.WriteIndex, dbPath)) + { + db.InitializeSchema(); + var writer = new DbWriter(db.Connection); + var fileId = writer.UpsertFile(new FileRecord + { + Path = path, + Lang = "csharp", + Size = lowChunk.Length + highChunk.Length + laterHighChunk.Length, + Lines = 8, + Modified = new DateTime(2026, 8, 1, 0, 0, 0, DateTimeKind.Utc), + }); + writer.InsertChunks( + [ + new ChunkRecord + { + FileId = fileId, + ChunkIndex = 0, + StartLine = 5, + EndLine = 6, + Content = highChunk, + }, + new ChunkRecord + { + FileId = fileId, + ChunkIndex = 1, + StartLine = 7, + EndLine = 8, + Content = laterHighChunk, + }, + new ChunkRecord + { + FileId = fileId, + ChunkIndex = 2, + StartLine = 1, + EndLine = 4, + Content = lowChunk, + }, + ]); + } + + var (exitCode, stdout, stderr) = CaptureConsole(() => QueryCommandRunner.RunSearch( + [ + "--recipe", "dogfood-risk-patterns/relaxed-json-encoder", + "--db", dbPath, + "--path", path, + "--json", + "--no-dedup", + "--limit", "10", + "--snippet-lines", "1", + ], + _jsonOptions)); + + Assert.Equal(CommandExitCodes.Success, exitCode); + Assert.Equal(string.Empty, stderr); + using var document = ParseJsonOutput(stdout); + var query = Assert.Single(document.RootElement.GetProperty("queries").EnumerateArray()); + var results = query.GetProperty("results").EnumerateArray().ToArray(); + Assert.Equal(3, results.Length); + Assert.Contains(6, results[0].GetProperty("match_lines").EnumerateArray().Select(line => line.GetInt32())); + AssertJsonTrustClassification( + results[0], + "ambiguous_trust", + "rationale:missing_explicit_trust_annotation"); + Assert.Contains(8, results[1].GetProperty("match_lines").EnumerateArray().Select(line => line.GetInt32())); + AssertJsonTrustClassification( + results[1], + "ambiguous_trust", + "rationale:missing_explicit_trust_annotation"); + Assert.Contains(3, results[2].GetProperty("match_lines").EnumerateArray().Select(line => line.GetInt32())); + AssertJsonTrustClassification( + results[2], + "controlled_private_writer", + "rationale:bounded_low_prefix", + "annotation_status:valid"); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + + static void AssertJsonTrustClassification( + JsonElement result, + string expectedCategory, + params string[] expectedEvidence) + { + var classification = Assert.Single( + result.GetProperty("audit_classifications").EnumerateArray(), + item => item.GetProperty("classifier").GetString() == "json_trust_boundary"); + Assert.Equal(expectedCategory, classification.GetProperty("category").GetString()); + var evidence = classification.GetProperty("evidence").EnumerateArray().Select(item => item.GetString()).ToArray(); + Assert.All(expectedEvidence, item => Assert.Contains(item, evidence)); + } + } + + [Fact] + public void RunSearch_JsonTrustBoundaryIgnoresDeduplicatedOverlapFacets_Issue4913() + { + var projectRoot = TestProjectHelper.CreateTempProject("cdidx_search_json_trust_overlap_4913"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + const string path = "src/overlap-writer.cs"; + var lines = Enumerable.Range(1, 220) + .Select(line => $"// filler {line}") + .ToArray(); + lines[0] = "using System.Text.Encodings.Web;"; + lines[74] = "// cdidx-audit: json-trust origin=public_api direction=write sensitivity=public trust=untrusted rationale=external_overlap"; + lines[75] = "var externalEncoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping;"; + lines[140] = "// cdidx-audit: json-trust origin=private_local direction=write sensitivity=diagnostic trust=controlled rationale=private_retained"; + lines[141] = "var privateEncoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping;"; + var content = string.Join('\n', lines); + + using (var db = new DbContext(DbOpenIntent.WriteIndex, dbPath)) + { + db.InitializeSchema(); + var writer = new DbWriter(db.Connection); + var fileId = writer.UpsertFile(new FileRecord + { + Path = path, + Lang = "csharp", + Size = content.Length, + Lines = lines.Length, + Modified = new DateTime(2026, 8, 1, 0, 0, 0, DateTimeKind.Utc), + }); + writer.InsertChunks( + [ + new ChunkRecord + { + FileId = fileId, + ChunkIndex = 0, + StartLine = 1, + EndLine = 80, + Content = string.Join('\n', lines.Take(80)), + }, + new ChunkRecord + { + FileId = fileId, + ChunkIndex = 1, + StartLine = 71, + EndLine = 150, + Content = string.Join('\n', lines.Skip(70).Take(80)), + }, + new ChunkRecord + { + FileId = fileId, + ChunkIndex = 2, + StartLine = 141, + EndLine = 220, + Content = string.Join('\n', lines.Skip(140).Take(80)), + }, + ]); + } + + var (exitCode, stdout, stderr) = CaptureConsole(() => QueryCommandRunner.RunSearch( + [ + "--recipe", "dogfood-risk-patterns/relaxed-json-encoder", + "--db", dbPath, + "--path", path, + "--json", + "--limit", "10", + "--snippet-lines", "1", + ], + _jsonOptions)); + + Assert.Equal(CommandExitCodes.Success, exitCode); + Assert.Equal(string.Empty, stderr); + using var document = ParseJsonOutput(stdout); + var query = Assert.Single(document.RootElement.GetProperty("queries").EnumerateArray()); + var retainedPrivateResult = Assert.Single( + query.GetProperty("results").EnumerateArray(), + result => result.GetProperty("match_lines").EnumerateArray().Any(line => line.GetInt32() == 142)); + AssertJsonTrustClassification( + retainedPrivateResult, + "controlled_private_writer", + "rationale:private_retained", + "annotation_status:valid"); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + + static void AssertJsonTrustClassification( + JsonElement result, + string expectedCategory, + params string[] expectedEvidence) + { + var classification = Assert.Single( + result.GetProperty("audit_classifications").EnumerateArray(), + item => item.GetProperty("classifier").GetString() == "json_trust_boundary"); + Assert.Equal(expectedCategory, classification.GetProperty("category").GetString()); + var evidence = classification.GetProperty("evidence").EnumerateArray().Select(item => item.GetString()).ToArray(); + Assert.All(expectedEvidence, item => Assert.Contains(item, evidence)); + } + } + [Fact] public void RunSearch_TimestampTimezoneRecipeDocumentsClockBoundaries_Issue4321() {