Skip to content

fix(CommandPalette): keep astral characters intact when truncating search results - #347

Open
arsalan507 wants to merge 1 commit into
bitrix24:mainfrom
arsalan507:fix/search-truncate-astral-characters
Open

fix(CommandPalette): keep astral characters intact when truncating search results#347
arsalan507 wants to merge 1 commit into
bitrix24:mainfrom
arsalan507:fix/search-truncate-astral-characters

Conversation

@arsalan507

Copy link
Copy Markdown

Closes #339.

Upstream first

Per the note at the end of #339src/runtime/utils/search.ts is byte-identical between this repo and nuxt/ui (re-verified by diff against nuxt/ui@v4 today, 3459 bytes both), and so is test/utils/search.spec.ts.

So the fix went upstream first: nuxt/ui#6817. This PR is the identical patch, applied here with git apply from the upstream commit so the two files stay in sync byte-for-byte.

Entirely your call whether to land this now or wait for upstream and take it through the normal sync — the issue left that open ("unless it is wanted here sooner"). If you'd rather wait, close this and it'll arrive on its own; nothing here diverges from upstream.

The fix

Two hunks, exactly as #339 prescribed:

  1. truncateHTMLFromStart — iterate with Array.from(html) so the loop advances one code point at a time and never slices a surrogate pair in half. The insideTag tracking is untouched, since < and > are always single code units.
  2. The caller — measure the length budget in code points too ("and count length the same way"), so it stays in the same units as the counter inside the function.

Worth flagging for review: hunk 1 alone is sufficient to remove the lone surrogates; hunk 2 is the consistency half. It does mean an emoji now counts as one character against the truncation budget rather than two, so emoji-containing snippets truncate slightly later than before. That reads as closer to the intent of a visible-length budget, but it is a behaviour change and I'm happy to drop it if you'd prefer the minimal diff.

BMP-only content is byte-identical either way, which is why all pre-existing tests pass untouched.

Verification

Reproduced your sweep first, before changing anything — lone surrogates appear from filler length 7 onward and for every length after, exactly as reported:

n=10, before: "...\uDE00😀😀😀😀😀😀<mark>match</mark>"
n=10, after:  "😀😀😀😀😀😀😀😀😀😀<mark>match</mark>"

Three tests added to test/utils/search.spec.ts:

  • never splits an astral character, at any truncation boundary — the 1–40 sweep, asserting no lone surrogate at any length
  • keeps emoji before the match intact
  • still truncates a long prefix down to an ellipsis — guards against over-correcting into "never truncate"

The first two were verified failing against unmodified search.ts (expected [ 7, 8, 9, … ] to deeply equal []) and passing with the fix. The third passes both before and after by design.

test/utils/search.spec.ts             26 passed  (both projects; 3 new tests × 2)
test/utils/                          192 passed
CommandPalette + DashboardSearch +
ChatPalette + DashboardSearchButton  160 passed
eslint src/runtime/utils/search.ts test/utils/search.spec.ts    clean

Consumers checked: src/runtime/components/CommandPalette.vue and src/runtime/composables/useContentSearch.ts are the only two touching utils/search.

…arch results

`truncateHTMLFromStart` walked the highlighted snippet one UTF-16 code unit at
a time. An astral character (emoji, most CJK extension blocks) occupies two
code units, so a truncation boundary landing between them sliced the pair in
half and emitted an unpaired surrogate, rendering as `<?>`.

Iterate by code point instead, and measure the length budget the same way so
both sides stay in the same units. Behaviour for BMP-only content is unchanged.

Closes bitrix24#339
@arsalan507
arsalan507 force-pushed the fix/search-truncate-astral-characters branch from ee48021 to f08572d Compare August 9, 2026 14:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(search): truncateHTMLFromStart splits astral characters, emitting lone surrogates

1 participant