Skip to content

fix(llama): grow the detokenize buffer when a token piece exceeds 32 bytes - #2363

Open
Anai-Guo wants to merge 1 commit into
abetlen:mainfrom
Anai-Guo:fix-detokenize-oversized-piece
Open

fix(llama): grow the detokenize buffer when a token piece exceeds 32 bytes#2363
Anai-Guo wants to merge 1 commit into
abetlen:mainfrom
Anai-Guo:fix-detokenize-oversized-piece