Add optional int8 weight-only quantization - #7

Open

Talpik wants to merge 8 commits into

FireRedTeam:mainfrom

Talpik:feat/int8-quantization

Commits on Aug 26, 2026

Fix ValueError in FireRedTTS3Instruct.generate_tts

vladimir.talpa
and
claude
committed
c17592c
Support MPS (Apple Silicon) and CPU in addition to CUDA

vladimir.talpa
and
claude
committed
7cc41f7
Correct the MPS dtype benchmark in the README

vladimir.talpa
and
claude
committed
482e459
Let FIRERED_WEIGHT_DTYPE default to the checkpoint's own dtype

vladimir.talpa
and
claude
committed
48f9049
Add scripts/convert_to_bf16.py and document both bf16 routes

vladimir.talpa
and
claude
committed
6a50490
Add optional int8 weight-only quantization (FIRERED_QUANT=int8)

vladimir.talpa
and
claude
committed
aecce0c
Document FIRERED_QUANT=int8 in the README

vladimir.talpa
and
claude
committed
8c39bc8
Note that int8 needs no dtype variable on a pre-converted checkpoint

vladimir.talpa
and
claude
committed
b728021