fix(CommandPalette): keep astral characters intact when truncating search results - #347
Open
arsalan507 wants to merge 1 commit into
Open
fix(CommandPalette): keep astral characters intact when truncating search results#347arsalan507 wants to merge 1 commit into
arsalan507 wants to merge 1 commit into
Conversation
…arch results `truncateHTMLFromStart` walked the highlighted snippet one UTF-16 code unit at a time. An astral character (emoji, most CJK extension blocks) occupies two code units, so a truncation boundary landing between them sliced the pair in half and emitted an unpaired surrogate, rendering as `<?>`. Iterate by code point instead, and measure the length budget the same way so both sides stay in the same units. Behaviour for BMP-only content is unchanged. Closes bitrix24#339
arsalan507
force-pushed
the
fix/search-truncate-astral-characters
branch
from
August 9, 2026 14:10
ee48021 to
f08572d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #339.
Upstream first
Per the note at the end of #339 —
src/runtime/utils/search.tsis byte-identical between this repo andnuxt/ui(re-verified bydiffagainstnuxt/ui@v4today, 3459 bytes both), and so istest/utils/search.spec.ts.So the fix went upstream first: nuxt/ui#6817. This PR is the identical patch, applied here with
git applyfrom the upstream commit so the two files stay in sync byte-for-byte.Entirely your call whether to land this now or wait for upstream and take it through the normal sync — the issue left that open ("unless it is wanted here sooner"). If you'd rather wait, close this and it'll arrive on its own; nothing here diverges from upstream.
The fix
Two hunks, exactly as #339 prescribed:
truncateHTMLFromStart— iterate withArray.from(html)so the loop advances one code point at a time and never slices a surrogate pair in half. TheinsideTagtracking is untouched, since<and>are always single code units.Worth flagging for review: hunk 1 alone is sufficient to remove the lone surrogates; hunk 2 is the consistency half. It does mean an emoji now counts as one character against the truncation budget rather than two, so emoji-containing snippets truncate slightly later than before. That reads as closer to the intent of a visible-length budget, but it is a behaviour change and I'm happy to drop it if you'd prefer the minimal diff.
BMP-only content is byte-identical either way, which is why all pre-existing tests pass untouched.
Verification
Reproduced your sweep first, before changing anything — lone surrogates appear from filler length 7 onward and for every length after, exactly as reported:
Three tests added to
test/utils/search.spec.ts:never splits an astral character, at any truncation boundary— the 1–40 sweep, asserting no lone surrogate at any lengthkeeps emoji before the match intactstill truncates a long prefix down to an ellipsis— guards against over-correcting into "never truncate"The first two were verified failing against unmodified
search.ts(expected [ 7, 8, 9, … ] to deeply equal []) and passing with the fix. The third passes both before and after by design.Consumers checked:
src/runtime/components/CommandPalette.vueandsrc/runtime/composables/useContentSearch.tsare the only two touchingutils/search.