diff --git a/CHANGELOG.md b/CHANGELOG.md index 076d602c8..d8afb8bb2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,10 @@ Full release notes with details on each version: [GitHub Releases](https://github.com/safishamsi/graphify/releases) +## Unreleased + +- Feature: `graphify curate` — durable human corrections that survive a rebuild. Every edge in `graph.json` is extractor-derived, so a hand correction was transient by construction, in both directions. A **deleted** edge came back: the semantic cache is keyed by file content, not graph state, so an unchanged doc keeps its cached edges and the next `extract` re-injected the edge (and wrote it with `force=True`, past the shrink guard). An **added** edge was destroyed: `build_merge` replaces per `source_file`, dropping everything belonging to a re-extracted file from the base before merging — a human-authored edge on that file is dropped, and no extractor re-emits it. Neither is reported, and the node count *grows* while the second one happens, so the loss reads as a successful update. (Measured on a 3,384-node Angular graph: one `graphify update` took it to 3,425 nodes and silently dropped 3 verified edges.) `save-result --outcome dead_end` records that an edge is false but is advisory — it informs a future reader through `LESSONS.md` and never gates extraction, so the graph and the lessons drift apart. A new `graphify-out/curation.json` overlay (`curate deny` / `curate pin` / `list` / `apply`) is now applied at the end of `build_from_json` — the funnel `build`, `build_merge`, `watch` and the skill's agent path all construct the graph through — so corrections are re-applied on every build, and clustering, god nodes and `GRAPH_REPORT.md` see the corrected graph (a disproved edge stops being re-advertised in Surprising Connections, not merely absent from `graph.json`). The `--no-cluster` write paths, which never build a NetworkX graph, apply the overlay to the payload directly. `curation.json` joins the dated-folder backup set (#834). Inert when absent; `GRAPHIFY_NO_CURATION=1` disables it. + ## 0.9.15 (2026-07-13) - Fix: detection now honors nested `.gitignore`/`.graphifyignore` files below the scan root, not just those at the scan root and above (#1847, thanks @Mohak-Agrawal). git applies a `.gitignore` to everything under its own directory, but graphify only loaded ignore files from the VCS-root-down-to-scan-root chain — so a `vendor/sub/.gitignore` deeper in the tree was never read and its exclusions leaked into the graph. Each directory's own ignore files are now read during the walk and anchored to that directory, preserving last-match-wins precedence (nearer files win, including over `.git/info/exclude`) and the parent-exclusion rule. diff --git a/README.md b/README.md index 0a883e75b..b993d4028 100644 --- a/README.md +++ b/README.md @@ -430,6 +430,37 @@ graphify-out/cost.json # local only --- +## Curating the graph + +Extraction is not perfect. Sometimes an LLM asserts an edge that doesn't exist, and sometimes it misses one you can prove is real. Fixing `graph.json` by hand does not stick: + +- **A deleted edge comes back.** The semantic cache is keyed by file *content*, not by graph state, so an unchanged doc keeps its cached edges and the next `graphify extract` re-injects the edge you deleted. +- **An added edge is destroyed.** `build_merge` replaces per `source_file`: everything belonging to a re-extracted file is dropped from the base graph before merging. A hand-authored edge on that file is dropped, and no extractor re-emits it. The node count *grows* while this happens, so nothing warns you. + +`graphify curate` records the correction in `graphify-out/curation.json` and re-applies it on **every** build: + +```bash +# an edge the extractor keeps inventing +graphify curate deny page_not_found patient_search --relation semantically_similar_to \ + --reason "lexical collision on 'not found'; no shared service, model, or import" + +# an edge you verified in the source but extraction missed +graphify curate pin usage_table medicare_table --relation shares_data_with --score 0.95 \ + --source-file src/usage_table.py --source-location L45 \ + --reason "both read CareManagementReport via subReportLoaded" + +graphify curate list # review the overlay +graphify curate apply # apply to graph.json now, without waiting for a rebuild +``` + +The overlay is applied when the graph is *built*, not when it is written — so clustering, god nodes and `GRAPH_REPORT.md` all see the corrected graph. A disproved edge stops being re-advertised in Surprising Connections, rather than merely vanishing from `graph.json`. + +Denies are matched on the unordered pair, so endpoint order doesn't matter; omit `--relation` to deny every edge between two nodes. A pinned edge whose endpoints aren't both in the graph is skipped, never invented. `curation.json` is small, reviewable, and worth committing — it is the record of what your team has actually verified. Set `GRAPHIFY_NO_CURATION=1` to ignore it for one run. + +This complements `save-result --outcome dead_end`, which records *that* a path was false for future readers of `LESSONS.md`. Curation makes the graph itself stop asserting it. + +--- + ## Using the graph directly ```bash @@ -637,6 +668,13 @@ graphify-out/ /graphify path "DigestAuth" "Response" /graphify explain "SwinTransformer" +graphify curate deny Foo Bar --relation calls --reason "lexical collision; no call site" # mark an edge false — removed on every build +graphify curate deny Foo Bar # deny every edge between the pair, whatever the relation +graphify curate pin Foo Bar --relation shares_data_with --score 0.95 # add a verified edge — re-added on every build +graphify curate list # show the overlay (graphify-out/curation.json) +graphify curate apply # apply it to graph.json without a rebuild +GRAPHIFY_NO_CURATION=1 graphify update . # ignore the overlay for one run + graphify save-result --question "Q" --answer "A" --nodes Foo Bar --outcome useful # record how a Q&A turned out (work memory; outcome ∈ useful|dead_end|corrected) graphify reflect # aggregate graphify-out/memory/ outcomes into reflections/LESSONS.md graphify reflect --if-stale # no-op when LESSONS.md is already newer than every input (cheap to run each session) diff --git a/graphify/__main__.py b/graphify/__main__.py index 82e422e0d..7067590b9 100644 --- a/graphify/__main__.py +++ b/graphify/__main__.py @@ -572,6 +572,16 @@ def _run_cli() -> None: print(" --outcome O work-memory signal: useful|dead_end|corrected") print(" --correction TEXT what the right answer was (pairs with --outcome corrected)") print(" --memory-dir DIR memory directory (default: graphify-out/memory)") + print(" curate durable human corrections that survive every rebuild") + print(" deny SRC TGT mark an edge false; removed on every build") + print(" --relation R deny only this relation (default: all edges between the pair)") + print(" --reason TEXT why it is false") + print(" --evidence REF e.g. src/foo.py:42") + print(" pin SRC TGT add a verified edge; re-added on every build") + print(" --relation R required, e.g. calls, references, shares_data_with") + print(" --confidence C EXTRACTED | INFERRED | AMBIGUOUS (default: INFERRED)") + print(" list show the current overlay") + print(" apply apply the overlay to graph.json without a rebuild") print(" reflect aggregate graphify-out/memory/ outcomes into a deterministic lessons doc") print(" --memory-dir DIR memory directory (default: graphify-out/memory)") print(" --out FILE output path (default: graphify-out/reflections/LESSONS.md)") diff --git a/graphify/build.py b/graphify/build.py index 428fa1cd4..c7c24d444 100644 --- a/graphify/build.py +++ b/graphify/build.py @@ -28,6 +28,7 @@ import unicodedata from pathlib import Path import networkx as nx +from .curation import apply_curation, format_stats, load_curation from .ids import make_id, normalize_id as _normalize_id from .paths import default_graph_json as _default_graph_json from .validate import validate_extraction @@ -380,13 +381,24 @@ def _doc_twin_remap(nodes: list) -> dict[str, str]: return remap -def build_from_json(extraction: dict, *, directed: bool = False, root: str | Path | None = None) -> nx.Graph: +def build_from_json( + extraction: dict, + *, + directed: bool = False, + root: str | Path | None = None, + curation: dict | None = None, +) -> nx.Graph: """Build a NetworkX graph from an extraction dict. directed=True produces a DiGraph that preserves edge direction (source→target). directed=False (default) produces an undirected Graph for backward compatibility. root: if given, absolute source_file paths from semantic subagents are made relative to root so all nodes share a consistent path key (#932). + curation: user-owned overlay applied last (denied edges removed, verified edges + pinned). Defaults to loading ``graphify-out/curation.json``; pass an explicit + dict to override, or ``{}`` to skip. Applied here rather than at write time so + clustering, god-nodes and GRAPH_REPORT.md all see the corrected graph — a + disproved edge must stop being re-advertised, not merely stop being written. """ _root = str(Path(root).resolve()) if root else None # NetworkX <= 3.1 serialised edges as "links"; remap to "edges" for compatibility. @@ -759,6 +771,17 @@ def build_from_json(extraction: dict, *, directed: bool = False, root: str | Pat if isinstance(he, dict) and he.get("source_file"): he["source_file"] = _norm_source_file(he["source_file"], _root) G.graph["hyperedges"] = hyperedges + + # Curation is applied LAST, on every build path (build, build_merge, watch, and + # the skill's agent path all funnel through here). Human corrections are otherwise + # transient: build_merge's replace-per-source drops pinned edges belonging to a + # re-extracted file, and the content-keyed semantic cache resurrects denied ones. + _curation = load_curation() if curation is None else curation + if _curation: + _stats = apply_curation(G, _curation) + _msg = format_stats(_stats) + if _msg: + print(_msg, file=sys.stderr) return G @@ -769,6 +792,7 @@ def build( dedup: bool = True, dedup_llm_backend: str | None = None, root: str | Path | None = None, + curation: dict | None = None, ) -> nx.Graph: """Merge multiple extraction results into one graph. @@ -797,7 +821,7 @@ def build( combined["nodes"], combined["edges"], communities={}, dedup_llm_backend=dedup_llm_backend, ) - return build_from_json(combined, directed=directed, root=root) + return build_from_json(combined, directed=directed, root=root, curation=curation) def _norm_label(label: str | None) -> str: @@ -870,6 +894,7 @@ def build_merge( directed: bool = False, dedup: bool = True, dedup_llm_backend: str | None = None, + curation: dict | None = None, root: str | Path | None = None, ) -> nx.Graph: """Load existing graph.json, merge new chunks into it, and save back. @@ -952,7 +977,10 @@ def _kept(item: dict) -> bool: base = [{"nodes": existing_nodes, "edges": existing_edges}] if had_graph else [] all_chunks = base + list(new_chunks) - G = build(all_chunks, directed=directed, dedup=dedup, dedup_llm_backend=dedup_llm_backend, root=root) + G = build( + all_chunks, directed=directed, dedup=dedup, + dedup_llm_backend=dedup_llm_backend, root=root, curation=curation, + ) # Prune set for deleted source files — both the raw form (matches nodes that # kept absolute source_file) and the normalised relative form (matches nodes diff --git a/graphify/cli.py b/graphify/cli.py index 0bb124777..f40d13828 100644 --- a/graphify/cli.py +++ b/graphify/cli.py @@ -538,6 +538,161 @@ def dispatch_command(cmd: str) -> None: correction=opts.correction, ) print(f"Saved to {out}") + elif cmd == "curate": + # graphify curate deny SRC TGT [--relation R] [--reason TEXT] [--evidence REF] + # graphify curate pin SRC TGT --relation R [--confidence C] [--score F] + # [--source-file F] [--source-location L] [--reason TEXT] + # graphify curate list + # graphify curate apply [--graph PATH] + import argparse as _ap + + from graphify.curation import ( + CURATION_SCHEMA_VERSION, + apply_curation_to_payload, + curation_path, + empty_curation, + format_stats, + load_curation, + save_curation, + ) + + p = _ap.ArgumentParser( + prog="graphify curate", + description=( + "Durable human corrections. Denied edges stay denied and pinned edges " + "stay pinned across rebuilds, which delete-from-graph.json alone does not." + ), + ) + sub = p.add_subparsers(dest="action", required=True) + + p_deny = sub.add_parser("deny", help="mark an edge false; it is removed on every build") + p_deny.add_argument("source") + p_deny.add_argument("target") + p_deny.add_argument("--relation", default=None, + help="deny only this relation (default: every edge between the pair)") + p_deny.add_argument("--reason", default=None) + p_deny.add_argument("--evidence", default=None, help="e.g. src/foo.py:42") + + p_pin = sub.add_parser("pin", help="add a verified edge; it is re-added on every build") + p_pin.add_argument("source") + p_pin.add_argument("target") + p_pin.add_argument("--relation", required=True) + p_pin.add_argument("--confidence", default="INFERRED", + choices=("EXTRACTED", "INFERRED", "AMBIGUOUS")) + p_pin.add_argument("--score", type=float, default=None, dest="confidence_score") + p_pin.add_argument("--source-file", default=None, dest="source_file") + p_pin.add_argument("--source-location", default=None, dest="source_location") + p_pin.add_argument("--reason", default=None) + + sub.add_parser("list", help="show the current overlay") + + p_apply = sub.add_parser( + "apply", help="apply the overlay to an existing graph.json without a rebuild") + p_apply.add_argument("--graph", default=str(Path(_GRAPHIFY_OUT) / "graph.json")) + + for _sp in (p_deny, p_pin): + _sp.add_argument("--out-dir", default=_GRAPHIFY_OUT) + + opts = p.parse_args(sys.argv[2:]) + + if opts.action == "list": + cur = load_curation(_GRAPHIFY_OUT) + if not cur: + print(f"No curation overlay at {curation_path(_GRAPHIFY_OUT)}") + return + denies = cur.get("deny_edges", []) + adds = cur.get("add_edges", []) + print(f"{curation_path(_GRAPHIFY_OUT)} (schema v{cur.get('version', '?')})") + print(f"\nDENIED ({len(denies)}):") + for e in denies: + rel = e.get("relation") or "*" + why = f" — {e['reason']}" if e.get("reason") else "" + ev = f" [{e['evidence']}]" if e.get("evidence") else "" + print(f" {e.get('source')} --{rel}-- {e.get('target')}{why}{ev}") + print(f"\nPINNED ({len(adds)}):") + for e in adds: + why = f" — {e['reason']}" if e.get("reason") else "" + print( + f" {e.get('source')} --{e.get('relation')}--> {e.get('target')} " + f"[{e.get('confidence', 'INFERRED')}]{why}" + ) + return + + if opts.action == "apply": + graph_path = Path(opts.graph) + if not graph_path.exists(): + print(f"error: no graph at {graph_path}", file=sys.stderr) + raise SystemExit(1) + cur = load_curation(graph_path.parent) + if not cur: + print(f"No curation overlay at {curation_path(graph_path.parent)}") + return + data = json.loads(graph_path.read_text(encoding="utf-8")) + stats = apply_curation_to_payload(data, cur) + msg = format_stats(stats) + if msg is None: + print("[graphify] curation: graph already matches the overlay") + return + graph_path.write_text(json.dumps(data, indent=2), encoding="utf-8") + print(msg) + print(f"[graphify] wrote {graph_path}") + return + + # deny / pin — mutate the overlay file + cur = load_curation(opts.out_dir) or empty_curation() + cur.setdefault("version", CURATION_SCHEMA_VERSION) + cur.setdefault("deny_edges", []) + cur.setdefault("add_edges", []) + + if opts.action == "deny": + entry = {"source": opts.source, "target": opts.target} + if opts.relation: + entry["relation"] = opts.relation + if opts.reason: + entry["reason"] = opts.reason + if opts.evidence: + entry["evidence"] = opts.evidence + dupe = any( + e.get("source") == entry["source"] + and e.get("target") == entry["target"] + and e.get("relation") == entry.get("relation") + for e in cur["deny_edges"] + ) + if dupe: + print("[graphify] curation: already denied") + return + cur["deny_edges"].append(entry) + path = save_curation(cur, opts.out_dir) + rel = opts.relation or "*" + print(f"[graphify] denied {opts.source} --{rel}-- {opts.target} in {path}") + print("[graphify] the edge is removed on every build from now on") + return + + entry = { + "source": opts.source, + "target": opts.target, + "relation": opts.relation, + "confidence": opts.confidence, + } + if opts.confidence_score is not None: + entry["confidence_score"] = opts.confidence_score + for k in ("source_file", "source_location", "reason"): + if getattr(opts, k, None): + entry[k] = getattr(opts, k) + dupe = any( + e.get("source") == entry["source"] + and e.get("target") == entry["target"] + and e.get("relation") == entry["relation"] + for e in cur["add_edges"] + ) + if dupe: + print("[graphify] curation: already pinned") + return + cur["add_edges"].append(entry) + path = save_curation(cur, opts.out_dir) + print(f"[graphify] pinned {opts.source} --{opts.relation}--> {opts.target} in {path}") + print("[graphify] the edge is re-added on every build from now on") + return elif cmd == "reflect": import argparse as _ap @@ -2487,6 +2642,16 @@ def _progress(idx: int, total: int, _result: dict) -> None: _node_sf.get(_e.get("source")) or _node_sf.get(_e.get("target")) or "" ) _backup(graphify_out) + # --no-cluster never builds a NetworkX graph, so it bypasses + # build_from_json's curation pass. Apply to the payload directly. + from graphify.curation import ( + apply_curation_to_payload as _apply_cur, + format_stats as _cur_stats, + load_curation as _load_cur, + ) + _cur_msg = _cur_stats(_apply_cur(merged, _load_cur(graphify_out))) + if _cur_msg: + print(_cur_msg) graph_json_path.write_text( json.dumps(merged, indent=2), encoding="utf-8" ) diff --git a/graphify/curation.py b/graphify/curation.py new file mode 100644 index 000000000..dd29c9251 --- /dev/null +++ b/graphify/curation.py @@ -0,0 +1,303 @@ +# user-owned curation overlay: deny false edges, pin verified ones +"""Durable human corrections to the graph. + +Every edge in graph.json is derived from an extractor, so any correction a human +makes to graph.json is transient by construction: + +- **Deleted false edges come back.** The semantic cache is keyed by file content, + not by graph state. An unchanged doc keeps its cached edges forever, so the next + ``graphify extract`` re-injects a deleted edge and writes it back with + ``force=True`` — past the shrink guard. +- **Added verified edges are destroyed.** ``build_merge`` replaces per source_file: + every ``source_file`` present in the new chunks is dropped from the base graph + before merging. A human-authored edge whose ``source_file`` names a re-extracted + file is dropped, and no extractor re-emits it. The node count *grows* while this + happens, so nothing warns. + +``save-result --outcome dead_end`` records that an edge is false, but it is advisory +— it informs a future agent through LESSONS.md and never gates extraction. So the +graph and the lessons drift apart: the graph keeps asserting an edge the operator +has already disproved. + +This module adds the missing layer: ``graphify-out/curation.json``, a small +user-owned file that is applied at the end of :func:`graphify.build.build_from_json` +— the single funnel through which ``build``, ``build_merge``, ``watch`` and the +skill's agent path all construct the graph. Applying it there (rather than at write +time) means clustering, god-nodes, surprising-connections and GRAPH_REPORT.md all +see the curated graph, so a disproved edge stops being re-advertised. + +The overlay is declarative and idempotent: it is re-applied on every build, so it +survives rebuilds that would otherwise erase it. Set ``GRAPHIFY_NO_CURATION=1`` to +disable (mirrors ``GRAPHIFY_NO_BACKUP``). + +Schema (``graphify-out/curation.json``):: + + { + "version": 1, + "deny_edges": [ + {"source": "a_mod_foo", "target": "b_mod_bar", "relation": "calls", + "reason": "lexical collision; no call site", "evidence": "b/bar.py:12"} + ], + "add_edges": [ + {"source": "a_mod_foo", "target": "b_mod_bar", + "relation": "shares_data_with", "confidence": "INFERRED", + "confidence_score": 0.95, "source_file": "a/foo.py", + "source_location": "L45", "reason": "both read Report.subLoaded"} + ] + } + +``relation`` is optional on a deny entry; omitting it denies every edge between the +pair. Endpoint order is not significant — an undirected graph canonicalizes it — so +a deny matches the pair in either orientation. +""" +from __future__ import annotations + +import json +import os +from pathlib import Path +from typing import Any + +import networkx as nx + +CURATION_FILENAME = "curation.json" +CURATION_SCHEMA_VERSION = 1 + +# Mirrors export._CONFIDENCE_SCORE_DEFAULTS so a pinned edge that omits an explicit +# score still lands with the same default the extractor path would have given it. +_DEFAULT_SCORES = {"EXTRACTED": 1.0, "INFERRED": 0.5, "AMBIGUOUS": 0.2} + + +def curation_path(out_dir: str | Path | None = None) -> Path: + """Path to the curation file. Defaults to ``graphify-out/curation.json`` in CWD.""" + return Path(out_dir or "graphify-out") / CURATION_FILENAME + + +def empty_curation() -> dict[str, Any]: + return {"version": CURATION_SCHEMA_VERSION, "deny_edges": [], "add_edges": []} + + +def _pair(entry: dict) -> tuple[str, str] | None: + src, tgt = entry.get("source"), entry.get("target") + if not isinstance(src, str) or not isinstance(tgt, str) or not src or not tgt: + return None + return (src, tgt) + + +def _deny_key(src: str, tgt: str, relation: str | None) -> tuple[str, str, str | None]: + """Order-insensitive key. Undirected storage canonicalizes endpoint order, so a + deny written as (a, b) must also match an edge stored as (b, a).""" + lo, hi = sorted((src, tgt)) + return (lo, hi, relation) + + +def load_curation(out_dir: str | Path | None = None) -> dict[str, Any] | None: + """Load the curation overlay. Returns None when absent, disabled, or unreadable. + + Never raises: a malformed curation file must not break a build. It warns and is + ignored, because silently applying half a corrupt overlay is worse than applying + none of it. + """ + if os.environ.get("GRAPHIFY_NO_CURATION"): + return None + path = curation_path(out_dir) + if not path.exists(): + return None + try: + data = json.loads(path.read_text(encoding="utf-8")) + except (OSError, ValueError) as exc: + print(f"[graphify] warning: ignoring unreadable {path}: {exc}") + return None + if not isinstance(data, dict): + print(f"[graphify] warning: ignoring {path}: expected a JSON object") + return None + version = data.get("version") + if version is not None and version != CURATION_SCHEMA_VERSION: + print( + f"[graphify] warning: ignoring {path}: schema version {version!r} " + f"(this graphify understands {CURATION_SCHEMA_VERSION})" + ) + return None + for key in ("deny_edges", "add_edges"): + if not isinstance(data.get(key, []), list): + print(f"[graphify] warning: ignoring {path}: {key} must be a list") + return None + return data + + +def save_curation(curation: dict[str, Any], out_dir: str | Path | None = None) -> Path: + """Write the overlay deterministically (sorted keys) so re-runs are byte-identical.""" + path = curation_path(out_dir) + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text( + json.dumps(curation, indent=2, sort_keys=True, ensure_ascii=False) + "\n", + encoding="utf-8", + ) + return path + + +def apply_curation(G: nx.Graph, curation: dict[str, Any] | None) -> dict[str, int]: + """Apply the overlay to a built graph, in place. Returns per-action counts. + + Denies run before adds, so an entry can legitimately deny an extractor's + mistyped edge and pin the corrected one over the same pair. + + A pinned edge whose endpoints are not both present is skipped, not invented: + the overlay corrects the graph, it does not fabricate nodes. That also keeps the + edge from dangling and being silently swallowed by the dangling-edge drop. + """ + stats = {"denied": 0, "added": 0, "skipped_missing_endpoint": 0} + if not curation: + return stats + + denies: set[tuple[str, str, str | None]] = set() + for entry in curation.get("deny_edges", []): + if not isinstance(entry, dict): + continue + pair = _pair(entry) + if pair is None: + continue + denies.add(_deny_key(pair[0], pair[1], entry.get("relation"))) + + if denies: + doomed = [] + for src, tgt, attrs in G.edges(data=True): + # Match on the edge's true direction (_src/_tgt) when present — undirected + # storage may have flipped the stored endpoints (#563). + e_src = attrs.get("_src", src) + e_tgt = attrs.get("_tgt", tgt) + rel = attrs.get("relation") + if ( + _deny_key(e_src, e_tgt, rel) in denies + or _deny_key(e_src, e_tgt, None) in denies + ): + doomed.append((src, tgt)) + for src, tgt in doomed: + if G.has_edge(src, tgt): + G.remove_edge(src, tgt) + stats["denied"] += 1 + + for entry in curation.get("add_edges", []): + if not isinstance(entry, dict): + continue + pair = _pair(entry) + if pair is None: + continue + src, tgt = pair + if src not in G or tgt not in G: + stats["skipped_missing_endpoint"] += 1 + continue + if G.has_edge(src, tgt): + existing = G.get_edge_data(src, tgt) or {} + if existing.get("relation") == entry.get("relation"): + continue # already present — idempotent re-apply + confidence = entry.get("confidence", "INFERRED") + attrs = { + "relation": entry.get("relation", "references"), + "confidence": confidence, + "confidence_score": entry.get( + "confidence_score", _DEFAULT_SCORES.get(confidence, 1.0) + ), + "source_file": entry.get("source_file", ""), + "source_location": entry.get("source_location"), + "weight": entry.get("weight", 1.0), + "curated": True, + "_src": src, + "_tgt": tgt, + } + G.add_edge(src, tgt, **attrs) + stats["added"] += 1 + + return stats + + +def apply_curation_to_payload( + data: dict[str, Any], curation: dict[str, Any] | None +) -> dict[str, int]: + """Apply the overlay to a raw graph payload, in place. + + For the ``--no-cluster`` write paths, which dump an extraction dict directly and + never construct a NetworkX graph. Handles both the ``links`` key (node_link_data) + and ``edges`` (raw extraction). + """ + stats = {"denied": 0, "added": 0, "skipped_missing_endpoint": 0} + if not curation: + return stats + + edge_key = "links" if "links" in data else "edges" + edges = data.get(edge_key) + if not isinstance(edges, list): + return stats + + denies: set[tuple[str, str, str | None]] = set() + for entry in curation.get("deny_edges", []): + if isinstance(entry, dict) and (pair := _pair(entry)): + denies.add(_deny_key(pair[0], pair[1], entry.get("relation"))) + + if denies: + kept = [] + for e in edges: + src, tgt = e.get("source"), e.get("target") + if isinstance(src, str) and isinstance(tgt, str) and ( + _deny_key(src, tgt, e.get("relation")) in denies + or _deny_key(src, tgt, None) in denies + ): + stats["denied"] += 1 + continue + kept.append(e) + edges = kept + + node_ids = { + n.get("id") for n in data.get("nodes", []) if isinstance(n, dict) + } + present = { + _deny_key(e["source"], e["target"], e.get("relation")) + for e in edges + if isinstance(e.get("source"), str) and isinstance(e.get("target"), str) + } + for entry in curation.get("add_edges", []): + if not isinstance(entry, dict): + continue + pair = _pair(entry) + if pair is None: + continue + src, tgt = pair + if src not in node_ids or tgt not in node_ids: + stats["skipped_missing_endpoint"] += 1 + continue + relation = entry.get("relation", "references") + if _deny_key(src, tgt, relation) in present: + continue + confidence = entry.get("confidence", "INFERRED") + edges.append({ + "source": src, + "target": tgt, + "relation": relation, + "confidence": confidence, + "confidence_score": entry.get( + "confidence_score", _DEFAULT_SCORES.get(confidence, 1.0) + ), + "source_file": entry.get("source_file", ""), + "source_location": entry.get("source_location"), + "weight": entry.get("weight", 1.0), + "curated": True, + }) + stats["added"] += 1 + + data[edge_key] = edges + return stats + + +def format_stats(stats: dict[str, int]) -> str | None: + """One-line summary for the CLI, or None when the overlay changed nothing.""" + bits = [] + if stats.get("denied"): + bits.append(f"{stats['denied']} edge(s) denied") + if stats.get("added"): + bits.append(f"{stats['added']} edge(s) pinned") + if stats.get("skipped_missing_endpoint"): + bits.append( + f"{stats['skipped_missing_endpoint']} pin(s) skipped (endpoint not in graph)" + ) + if not bits: + return None + return "[graphify] curation: " + ", ".join(bits) diff --git a/graphify/export.py b/graphify/export.py index 0e95b467f..1fa51d3cb 100644 --- a/graphify/export.py +++ b/graphify/export.py @@ -29,6 +29,7 @@ "manifest.json", ".graphify_semantic_marker", "cost.json", + "curation.json", ] diff --git a/graphify/watch.py b/graphify/watch.py index 14454bf08..65e6fdca5 100644 --- a/graphify/watch.py +++ b/graphify/watch.py @@ -10,6 +10,11 @@ from pathlib import Path # Single source of truth in graphify.paths (#1423); re-exported as _GRAPHIFY_OUT. +from graphify.curation import ( + apply_curation_to_payload, + format_stats as _format_curation_stats, + load_curation, +) from graphify.paths import GRAPHIFY_OUT as _GRAPHIFY_OUT _PENDING_FILENAME = ".pending_changes" _PENDING_DRAIN_MAX_PASSES = 20 @@ -941,6 +946,14 @@ def _add_deleted_source(path: Path) -> None: "nodes": _dedupe_nodes(result.get("nodes", [])), "links": _dedupe_edges(result.get("edges", [])), } + # --no-cluster dumps the payload directly and never builds a NetworkX + # graph, so it bypasses build_from_json's curation pass. Apply here too, + # or a denied edge survives on exactly this path. + _cur_msg = _format_curation_stats( + apply_curation_to_payload(candidate_graph_data, load_curation(out)) + ) + if _cur_msg: + print(_cur_msg) candidate_graph_text = _json_text(candidate_graph_data) same_graph = False if existing_graph.exists(): @@ -994,7 +1007,9 @@ def _add_deleted_source(path: Path) -> None: "total_words": detected.get("total_words", 0), } - G = build_from_json(result) + # Pass the overlay explicitly: watch's out dir is not always CWD-relative, + # and build_from_json's default lookup is. + G = build_from_json(result, curation=load_curation(out)) candidate_topology = _topology_from_graph(G) if existing_graph_data: try: diff --git a/tests/test_curation.py b/tests/test_curation.py new file mode 100644 index 000000000..dbfa9e3b5 --- /dev/null +++ b/tests/test_curation.py @@ -0,0 +1,263 @@ +"""Curation overlay: human corrections must survive every rebuild. + +Two failure modes this locks down, both observed in the wild on a 3.4k-node graph: + +1. `build_merge` replaces per source_file, so a pinned edge whose `source_file` + names a re-extracted file is dropped and no extractor re-emits it. The node + count *grows* while this happens, so no guard fires. +2. The semantic cache is keyed by file content, not graph state, so an unchanged + doc keeps its cached edges and `extract` re-injects an edge that was deleted + from graph.json. + +The overlay is applied at the end of build_from_json — the funnel every path +(build, build_merge, watch, skill) goes through — so both are neutralized. +""" +from __future__ import annotations + +import json + +import networkx as nx +import pytest + +from graphify.build import build, build_from_json, build_merge +from graphify.curation import ( + CURATION_SCHEMA_VERSION, + apply_curation, + apply_curation_to_payload, + empty_curation, + load_curation, + save_curation, +) + + +def _chunk(nodes, edges): + return {"nodes": nodes, "edges": edges, "hyperedges": []} + + +def _n(nid, sf): + return {"id": nid, "label": nid, "file_type": "code", "source_file": sf} + + +def _e(src, tgt, rel="references", sf="a.py"): + return { + "source": src, "target": tgt, "relation": rel, + "confidence": "EXTRACTED", "confidence_score": 1.0, "source_file": sf, + } + + +# --- deny ------------------------------------------------------------------- + +def test_deny_removes_edge_on_build(): + ext = _chunk([_n("a", "a.py"), _n("b", "b.py")], [_e("a", "b", "calls")]) + cur = {"version": 1, "deny_edges": [{"source": "a", "target": "b", "relation": "calls"}]} + G = build_from_json(ext, curation=cur) + assert not G.has_edge("a", "b"), "denied edge must not survive the build" + assert "a" in G and "b" in G, "deny removes the edge, not the nodes" + + +def test_deny_is_order_insensitive(): + """Undirected storage canonicalizes endpoint order; a deny written (a,b) must + still match an edge stored (b,a).""" + ext = _chunk([_n("a", "a.py"), _n("b", "b.py")], [_e("b", "a", "calls")]) + cur = {"deny_edges": [{"source": "a", "target": "b", "relation": "calls"}]} + G = build_from_json(ext, curation=cur) + assert not G.has_edge("a", "b") + + +def test_deny_without_relation_denies_every_edge_between_pair(): + ext = _chunk([_n("a", "a.py"), _n("b", "b.py")], [_e("a", "b", "calls")]) + cur = {"deny_edges": [{"source": "a", "target": "b"}]} + G = build_from_json(ext, curation=cur) + assert not G.has_edge("a", "b") + + +def test_deny_leaves_other_relations_alone(): + ext = _chunk( + [_n("a", "a.py"), _n("b", "b.py"), _n("c", "c.py")], + [_e("a", "b", "calls"), _e("a", "c", "calls")], + ) + cur = {"deny_edges": [{"source": "a", "target": "b", "relation": "calls"}]} + G = build_from_json(ext, curation=cur) + assert not G.has_edge("a", "b") + assert G.has_edge("a", "c"), "an unrelated edge must be untouched" + + +# --- pin -------------------------------------------------------------------- + +def test_pin_adds_verified_edge(): + ext = _chunk([_n("a", "a.py"), _n("b", "b.py")], []) + cur = {"add_edges": [{ + "source": "a", "target": "b", "relation": "shares_data_with", + "confidence": "INFERRED", "confidence_score": 0.95, + }]} + G = build_from_json(ext, curation=cur) + assert G.has_edge("a", "b") + attrs = G.get_edge_data("a", "b") + assert attrs["relation"] == "shares_data_with" + assert attrs["confidence_score"] == 0.95 + assert attrs["curated"] is True, "pinned edges must be attributable" + + +def test_pin_skips_missing_endpoint_rather_than_inventing_nodes(): + """The overlay corrects the graph; it must not fabricate nodes, or the edge + dangles and is silently swallowed by the dangling-edge drop.""" + ext = _chunk([_n("a", "a.py")], []) + cur = {"add_edges": [{"source": "a", "target": "ghost", "relation": "calls"}]} + G = build_from_json(ext, curation=cur) + assert "ghost" not in G + assert not G.has_edge("a", "ghost") + + +def test_pin_is_idempotent(): + ext = _chunk([_n("a", "a.py"), _n("b", "b.py")], []) + cur = {"add_edges": [{"source": "a", "target": "b", "relation": "calls"}]} + G = build_from_json(ext, curation=cur) + stats = apply_curation(G, cur) # re-apply on the same graph + assert stats["added"] == 0, "re-applying must not duplicate a pinned edge" + assert G.number_of_edges() == 1 + + +def test_deny_runs_before_pin_so_a_mistyped_edge_can_be_retyped(): + ext = _chunk([_n("a", "a.py"), _n("b", "b.py")], + [_e("a", "b", "conceptually_related_to")]) + cur = { + "deny_edges": [{"source": "a", "target": "b", "relation": "conceptually_related_to"}], + "add_edges": [{"source": "a", "target": "b", "relation": "shares_data_with"}], + } + G = build_from_json(ext, curation=cur) + assert G.get_edge_data("a", "b")["relation"] == "shares_data_with" + + +# --- the two regressions ---------------------------------------------------- + +def test_pinned_edge_survives_build_merge_replace_per_source(tmp_path): + """Regression: build_merge drops every edge whose source_file is re-extracted. + A pinned edge on a re-extracted file must come back.""" + root = tmp_path / "corpus" + root.mkdir() + graph_path = tmp_path / "graph.json" + + cur = {"add_edges": [{ + "source": "a", "target": "b", "relation": "shares_data_with", + "source_file": "a.py", # the file that gets re-extracted + }]} + + base = _chunk([_n("a", "a.py"), _n("b", "b.py")], []) + G0 = build_from_json(base, curation=cur) + assert G0.has_edge("a", "b") + graph_path.write_text( + json.dumps(nx.node_link_data(G0, edges="edges")), encoding="utf-8" + ) + + # a.py changes and is re-extracted — its prior contribution is replaced wholesale + new_chunk = _chunk([_n("a", "a.py")], []) + G1 = build_merge([new_chunk], graph_path, dedup=False, root=root, curation=cur) + + assert G1.has_edge("a", "b"), ( + "pinned edge was destroyed by replace-per-source — the exact bug this fixes" + ) + + +def test_denied_edge_stays_denied_when_extraction_reasserts_it(): + """Regression: the content-keyed semantic cache re-injects a deleted edge on + every extract. Denying it must hold even though the extraction still carries it.""" + cur = {"deny_edges": [{"source": "a", "target": "b", "relation": "semantically_similar_to"}]} + # the extractor (or its cache) keeps asserting the edge, run after run + for _ in range(3): + ext = _chunk( + [_n("a", "a.py"), _n("b", "b.py")], + [_e("a", "b", "semantically_similar_to")], + ) + G = build_from_json(ext, curation=cur) + assert not G.has_edge("a", "b"), "a disproved edge must not keep coming back" + + +def test_curation_applies_through_build(): + ext = _chunk([_n("a", "a.py"), _n("b", "b.py")], [_e("a", "b", "calls")]) + cur = {"deny_edges": [{"source": "a", "target": "b"}]} + G = build([ext], dedup=False, curation=cur) + assert not G.has_edge("a", "b"), "build() must funnel curation through build_from_json" + + +# --- payload path (--no-cluster) -------------------------------------------- + +def test_apply_curation_to_payload_denies_and_pins(): + data = { + "nodes": [{"id": "a"}, {"id": "b"}, {"id": "c"}], + "links": [ + {"source": "a", "target": "b", "relation": "calls"}, + {"source": "a", "target": "c", "relation": "calls"}, + ], + } + cur = { + "deny_edges": [{"source": "a", "target": "b", "relation": "calls"}], + "add_edges": [{"source": "b", "target": "c", "relation": "shares_data_with"}], + } + stats = apply_curation_to_payload(data, cur) + assert stats["denied"] == 1 + assert stats["added"] == 1 + pairs = {(e["source"], e["target"]) for e in data["links"]} + assert ("a", "b") not in pairs + assert ("a", "c") in pairs + assert ("b", "c") in pairs + + +def test_apply_curation_to_payload_handles_raw_extraction_edges_key(): + data = {"nodes": [{"id": "a"}, {"id": "b"}], + "edges": [{"source": "a", "target": "b", "relation": "calls"}]} + apply_curation_to_payload(data, {"deny_edges": [{"source": "a", "target": "b"}]}) + assert data["edges"] == [] + + +# --- file I/O --------------------------------------------------------------- + +def test_load_curation_absent_returns_none(tmp_path): + assert load_curation(tmp_path) is None + + +def test_save_then_load_roundtrip(tmp_path): + cur = empty_curation() + cur["deny_edges"].append({"source": "a", "target": "b", "reason": "lexical collision"}) + save_curation(cur, tmp_path) + loaded = load_curation(tmp_path) + assert loaded["version"] == CURATION_SCHEMA_VERSION + assert loaded["deny_edges"][0]["reason"] == "lexical collision" + + +def test_malformed_curation_is_ignored_not_fatal(tmp_path, capsys): + (tmp_path / "curation.json").write_text("{not json", encoding="utf-8") + assert load_curation(tmp_path) is None + assert "warning" in capsys.readouterr().out.lower() + + +def test_future_schema_version_is_ignored(tmp_path, capsys): + (tmp_path / "curation.json").write_text( + json.dumps({"version": 99, "deny_edges": []}), encoding="utf-8" + ) + assert load_curation(tmp_path) is None + assert "version" in capsys.readouterr().out.lower() + + +def test_env_var_disables_curation(tmp_path, monkeypatch): + save_curation({"version": 1, "deny_edges": [{"source": "a", "target": "b"}]}, tmp_path) + monkeypatch.setenv("GRAPHIFY_NO_CURATION", "1") + assert load_curation(tmp_path) is None + + +def test_build_from_json_without_curation_is_unchanged(): + """The overlay must be inert when absent — no behavior change for existing users.""" + ext = _chunk([_n("a", "a.py"), _n("b", "b.py")], [_e("a", "b", "calls")]) + G = build_from_json(ext, curation={}) + assert G.has_edge("a", "b") + + +@pytest.mark.parametrize("bad", [ + {"deny_edges": [{"source": "a"}]}, # no target + {"deny_edges": [{"target": "b"}]}, # no source + {"deny_edges": ["not-a-dict"]}, # wrong type + {"add_edges": [{"source": "", "target": "b"}]}, # empty id +]) +def test_malformed_entries_are_skipped_not_fatal(bad): + ext = _chunk([_n("a", "a.py"), _n("b", "b.py")], [_e("a", "b", "calls")]) + G = build_from_json(ext, curation=bad) + assert G.has_edge("a", "b"), "a malformed entry must be skipped, not crash the build"