Skip to content

feat: infer language for untagged code blocks - #60

Merged
denolfe merged 1 commit into
mainfrom
feat/language-detection
Oct 8, 2026
Merged

denolfe merged 1 commit into
mainfrom
feat/language-detection

Conversation

@denolfe

@denolfe denolfe commented Oct 8, 2026 •

Copy link
Copy Markdown
Owner

Untagged fenced code blocks get syntax highlighting when their content clearly matches a bundled grammar. Blocks that match no rule stay plain.

Key Changes

  • Rule-based language detection
    • A new detectLang module checks block content against high-precision rules. The first match wins, and no match returns undefined.
    • Rules cover shebang interpreters, a $ shell prompt, valid JSON, HTML, Go, Python, Rust, and TOML.
    • HTML matches a doctype or a short list of lowercase tags. Blocks containing ={ (a JSX expression attribute) are excluded.
  • Inferred language drives highlighting
    • Code nodes carry the guess as inferredLang. It is set only when the fence has no info string (the text after the opening backticks).
    • The guess picks the grammar. The block title shows only what the author wrote.
    • Render mode (--render) preloads the inferred grammar too.
  • Test fixture
    • test/lang-detection.md lists detected, plain, and tagged cases. Each heading states the expected result.

The following prior art was investigated, but rejected for various reasons:

  • flourite: regex-based detector, evaluated and rejected. It labeled file trees, logs, and prose as Lua, and it has no bash, JSON, or TOML support.
  • highlight.js: highlightAuto evaluated and rejected. Short snippets came back as nix, applescript, and mipsasm.
  • GitHub Linguist shebang parser: source of the shebang interpreter rules.

Untagged fences get syntax highlighting when a high-precision rule
matches: shebang interpreter, `$ ` prompt, valid JSON, HTML (doctype or a
basic lowercase tag, excluding JSX), Go package clause, Python def/import,
Rust fn/use, or a TOML table. Anything else stays plain so trees, logs,
and output are not miscolored. The inferred language drives highlighting
only; the block title still shows just the author's tag.
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown

Startup benchmark (--render test/exhaustive.md, linux-x64)

build mean ratio verdict
baseline (main@714aff7) 294.0ms ± 5.7ms —
PR 300.6ms ± 13.5ms 1.02× ✅ ok

Thresholds: warn ≥ 1.1×, fail ≥ 1.25×. Baseline built from main.

First-frame (large.md, runs=10)

metric baseline (main@714aff7) PR ratio verdict
first-frame 213.0ms 226.1ms 1.06× ✅ ok

Thresholds: warn ≥ 1.1×, fail ≥ 1.25×. Baseline built from main.

Scroll (large.md)

metric baseline (main@714aff7) PR ratio verdict
full_mount 10338.0ms 10594.0ms 1.02× ✅ ok
step_p50 27.0ms 27.4ms 1.01× ✅ ok
step_search_p50 28.4ms 28.5ms 1.01× ✅ ok
stepMatch_p50 21.3ms 21.5ms 1.01× ✅ ok
commit 129.0ms 130.8ms 1.01× ✅ ok

Thresholds: warn ≥ 1.1×, fail ≥ 1.25×. Baseline built from main.

@denolfe
denolfe merged commit 8726ca7 into main Oct 8, 2026
8 checks passed
@denolfe
denolfe deleted the feat/language-detection branch October 8, 2026 02:07
denolfe added a commit that referenced this pull request Oct 8, 2026
- feat: infer language for untagged code blocks (#60) (8726ca7)
- feat: add diff syntax highlighting to code blocks (#59) (714aff7)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant