fix(core): ignore wikilinks in inline code - #1337
Conversation
|
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b9506c7c96
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| content = token.tag or token.content | ||
| if not token.children: | ||
| return content | ||
| return "".join(child.content for child in token.children if child.type != "code_inline") |
There was a problem hiding this comment.
When an inline token has children, joining child.content no longer preserves the source Markdown: MarkdownIt has already consumed escapes and formatting, while this filter also removes every code span wholesale. Consequently, \[\[Literal]] is reconstructed as [[Literal]] and creates a phantom relation, while an explicit relation such as - implemented_by [[Parser]] (`parse()`) loses its context entirely. These incorrect derived relations are recreated on every parse or reindex, so code ranges should be masked without rebuilding the remaining text from normalized child contents.
AGENTS.md reference: AGENTS.md:L161-L166
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed in f79a4a4. Relation parsing now masks matched code spans from the original inline source, preserving positions and retaining escaped Markdown and relation context. It also skips escaped wikilink delimiters. Added coverage for escaped literals, inline-code context, and a leading code span. Validation: 94 Markdown tests passed; ty, Ruff, format, and diff checks passed.
b9506c7 to
f79a4a4
Compare
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f79a4a4c7d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| position = 0 | ||
|
|
||
| while position < len(content): | ||
| if content[position] != "`": |
There was a problem hiding this comment.
When a wikilink is surrounded by backslash-escaped backticks, such as \see [[Target]]`, CommonMark treats those backticks as literal punctuation rather than code delimiters. This scanner nevertheless matches the runs and masks the entire wikilink, permanently omitting its links_to` relation on every parse and reindex; skip escaped opener runs before searching for their closer.
AGENTS.md reference: AGENTS.md:L161-L166
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed in 0f6c538. The inline-code scanner now skips backtick openers escaped by an odd-length backslash run, matching CommonMark tokenization, so literal escaped backticks keep their real wikilinks. Added a regression assertion for \see [[Target]]``. Validation: 94 Markdown tests passed; ty, Ruff, format, and diff checks passed.
f79a4a4 to
0f6c538
Compare
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0f6c53848c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| content = (token.tag or token.content).strip() | ||
| return _parse_explicit_relation(content) | ||
| source_content = _relation_content(token) | ||
| content = _mask_inline_code(source_content) |
There was a problem hiding this comment.
When a list-item relation has a non-parenthesized trailing code span, such as - calls [[Parser]] parse()``, masking turns the entire tail into whitespace before _parse_explicit_relation applies its prose-tail guard. The line therefore becomes a typed `calls` relation and silently omits `parse()` instead of falling back to `links_to` as it did before this change. Reindexing reparses the same source and recreates this incorrect relation, so preserve the presence of non-context source text while excluding only wikilink delimiters inside code.
AGENTS.md reference: AGENTS.md:L161-L165
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed in e52c4da with a root-cause change. The handwritten backtick scanner has been removed: a wrapped MarkdownIt backtick rule records the exact code spans that the parser recognizes. Relation parsing masks only bracket characters inside those spans, so non-context trailing code remains visible to the prose-tail guard and correctly falls back to links_to. Added regressions for trailing code, leading code, escaped text, and a literal directive in code.
| scan = position | ||
| while scan < len(content): | ||
| if content[scan] != "`": | ||
| scan += 1 | ||
| continue |
There was a problem hiding this comment.
When an inline token contains many unmatched backtick runs of distinct lengths, every candidate opener scans the entire remaining suffix looking for a same-length closer, making this loop quadratic in the number of runs. Because relation_rule invokes the scanner for every inline token even when it contains no wikilink, a generated 321 KB line with 800 increasing runs takes roughly 10 seconds in this loop alone and can stall parsing or reindexing; cache unsuccessful delimiter lengths or scan/index the runs once.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed in e52c4da by delegating all code-span recognition to MarkdownIts own cached inline backtick rule rather than rescanning delimiter runs. Added an 800 increasing-length unmatched-run regression that preserves the real wikilink; local parser smoke completed in 0.043s. Full validation: 95 Markdown tests passed; ty, Ruff, format, and diff checks passed.
0f6c538 to
e52c4da
Compare
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e52c4dab34
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if masked[position] in "[]": | ||
| masked[position] = " " |
There was a problem hiding this comment.
When a real outer wikilink contains bracket text quoted by an inline code span, such as [[API [v2]]] or [[Outer [[literal]] Target]], this loop replaces those literal brackets with spaces and parse_inline_relations then extracts the target from that masked string. The source-preserving extraction added for explicit relations does not cover this implicit path, so every parse or reindex persistently creates an edge to the wrong target; scan using the masked content but extract the target from the original source positions.
AGENTS.md reference: AGENTS.md:L161-L165
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Confirmed as valid against the current branch, so I am not rejecting it. Addressed in 7caf720 as part of the source-preservation invariant: masked text is now used only to locate valid wikilink boundaries, while parse_inline_relations extracts the target from the same positions in original source text. Added regressions for [[API [v2]]] and [[Outer [[literal]] Target]]; both now retain their exact targets. Validation: 95 Markdown tests passed; ty, Ruff, format, and diff checks passed.
Signed-off-by: buduoqiu <shenyaodong@hhu.edu.cn>
e52c4da to
7caf720
Compare
2 participants
Fixes #1332.
MarkdownIt already emits
code_inlinetokens before Basic Memory relation parsing. This change filters those literal spans when deriving relations, so quoted wikilinks no longer create graph edges. Ordinary prose links and explicit relation syntax remain unchanged.Tests:
uv run pytest --no-cov tests/markdown -q(93 passed)uv run ty check src tests test-intuv run ruff check src/basic_memory/markdown/plugins.py tests/markdown/test_relation_edge_cases.pyuv run ruff format --check src/basic_memory/markdown/plugins.py tests/markdown/test_relation_edge_cases.pyuv run basic-memory doctor