fix(watch): preserve semantic edges of re-extracted sources (#1865)#1868
Open
xor-xe wants to merge 1 commit into
Open
fix(watch): preserve semantic edges of re-extracted sources (#1865)#1868xor-xe wants to merge 1 commit into
xor-xe wants to merge 1 commit into
Conversation
…-Labs#1865) A full graphify update evicted every edge whose source_file was a re-extracted source, including LLM semantic edges (e.g. semantically_similar_to) from Markdown docs that also have an AST extractor. The AST pass cannot regenerate those edges, so they were silently lost while their concept nodes survived — node eviction is provenance-aware via _origin (Graphify-Labs#1116), edge eviction was not. Tag AST-extracted edges with the same _origin=ast marker nodes already carry, and scope rebuild-driven edge eviction to that tier: re-extraction replaces a source's AST edges, while its semantic edges survive until a semantic re-extraction supersedes them. Deletion-driven eviction stays provenance-blind, so edges of deleted or excluded sources are still purged regardless of tier. Edges from graphs built before this change lack the marker; a stale AST edge from a file changed exactly once between the old and new version can linger until its source is deleted — the same migration trade-off the Graphify-Labs#1116 node marker made.
|
An explanation of this PR using Lenzon - hope that it helps explain this clearly. https://www.lenzon.ai/viewer/cmrjtm9ky000codzhxj2cbext?voice=google-chirp3 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1865
What was broken: A full
graphify update(and the post-checkout hook) silently evicted LLM-extracted semantic edges (semantically_similar_to, …) whosesource_fileis a document that also has an AST extractor (.md,.mdx,.qmd). Concept nodes survived; the edges connecting them were dropped, and since the graph grew overall the shrink guard never fired.Root cause:
_reconcile_existing_graphpreserves nodes provenance-aware (_origin == "ast", #1116) but evicted edges purely bysource_filemembership in the re-extracted set. On a full rebuild that set includes every doc with an AST extractor, so semantic edges were treated as owned by an AST pass that cannot regenerate them.Change: AST-extracted edges now carry the same
_origin: "ast"marker as nodes, and rebuild-driven edge eviction only claims that tier — a re-extraction replaces a source's AST edges while its semantic edges survive until a semantic re-extraction supersedes them. Deletion-driven eviction stays provenance-blind, so edges of deleted or excluded sources are still purged.graph.jsonalready persists_-prefixed edge attrs (_src/_tgt) andto_graphmlalready strips them, so the marker stays internal.Migration note: edges from graphs built before this change lack the marker; a stale AST edge from a file changed exactly once across the version boundary can linger until its source is deleted — the same trade-off the #1116 node marker made. Regenerating the graph clears it.
Testing: new regression test (full + incremental) reproducing the issue's scenario — fails on current
v8, passes here — plus an assertion that stale AST-tier edges of a re-extracted source are still evicted. Full suite: 3195 passed, 3 skipped. Verified end-to-end with the issue's CLI repro.