#

utf8-utf32

Here is 1 public repository matching this topic...

99degree / gpu_assisted_bpe

Use my old buddy iGPU HD620 to do token matching.

gpu vulkan tokens tokenization utf-32 byte-pair-encoding byte-pair tokenizerbpe utf8-utf32

Updated Apr 1, 2026
C++

Improve this page

Add a description, image, and links to the utf8-utf32 topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the utf8-utf32 topic, visit your repo's landing page and select "manage topics."