fix(CommandPalette): keep astral characters intact when truncating search results - #6817
fix(CommandPalette): keep astral characters intact when truncating search results#6817arsalan507 wants to merge 1 commit into
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughHTML truncation now iterates over Unicode code points instead of UTF-16 code units. Highlight truncation budgets also use code-point counts. New tests verify that emoji and other astral characters remain intact, long prefixes receive an ellipsis, and highlighted matches remain present. Estimated code review effort: 2 (Simple) | ~10 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
src/runtime/utils/search.tsParsing error: Unexpected token { test/utils/search.spec.tsParsing error: Unexpected token : Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
commit: |
…arch results `truncateHTMLFromStart` walked the highlighted snippet one UTF-16 code unit at a time. An astral character (emoji, most CJK extension blocks) occupies two code units, so a truncation boundary landing between them sliced the pair in half and emitted an unpaired surrogate, rendering as `<?>`. Iterate by code point instead, and measure the length budget the same way so both sides stay in the same units. Behaviour for BMP-only content is unchanged.
16016db to
6e97151
Compare
🔗 Linked issue
None open — filing this directly as it's a small, self-contained bug.
❓ Type of change
📚 Description
truncateHTMLFromStartinsrc/runtime/utils/search.tswalks the highlighted snippet in reverse one UTF-16 code unit at a time (html[i]). An astral character — emoji, most CJK extension blocks, many symbols — occupies two code units. When the truncation boundary lands between them, the surrogate pair is sliced in half and the output carries an unpaired surrogate, which renders as�.This is user-visible in
CommandPaletteanduseContentSearch, where indexed content is more likely to contain emoji than a hand-written command label: a search result whose match sits after an emoji gets a�at the truncation point.Reproduction — sweeping filler lengths 1–40 of
😀before the match, lone surrogates appear from filler length 7 onward and for every length after:Not a security issue. A lone surrogate can only render as a broken glyph or U+FFFD — it cannot manufacture HTML syntax, and no raw
<,>,&,"or'appears outside the<mark>tags the highlighter inserts itself. This is a correctness and appearance bug.🔧 The fix
Two hunks:
truncateHTMLFromStart— iterate withArray.from(html)so a surrogate pair is never split. TheinsideTaglogic is untouched, since<and>are always single code units.Worth calling out explicitly for review: hunk 2 is a consistency fix, not required to remove the lone surrogates — hunk 1 alone is sufficient. It does mean an emoji now counts as one character against the truncation budget rather than two, so emoji-containing snippets truncate slightly later than before. That seems closer to the intent of a visible-length budget, but happy to drop hunk 2 if you'd rather keep the diff minimal.
Behaviour for BMP-only content is byte-identical, which is why all pre-existing tests pass untouched.
✅ Verification
Added three tests to
test/utils/search.spec.ts:never splits an astral character, at any truncation boundary— the 1–40 sweep, asserting no lone surrogate at any lengthkeeps emoji before the match intactstill truncates a long prefix down to an ellipsis— guards against over-correcting into "never truncate"Verified the first two fail against unmodified
search.ts(expected [ 7, 8, 9, … ] to deeply equal []) and pass with the fix. The third passes both before and after by design.📝 Checklist