Skip to content

feat: load scaled FP8 weights without upfront conversion - #1913

Merged
leejet merged 1 commit into
masterfrom
feat/native-fp8-weight-loading
Aug 27, 2026
Merged

feat: load scaled FP8 weights without upfront conversion#1913
leejet merged 1 commit into
masterfrom
feat/native-fp8-weight-loading

Conversation

@leejet

@leejet leejet commented Aug 26, 2026

Copy link
Copy Markdown
Owner

Summary

  • preserve E4M3 and E5M2 safetensors weights in their original one-byte representation
  • cast active FP8 linear weights to BF16 at runtime and apply scalar or per-channel weight scales
  • normalize .scale_weight tensor names to .weight_scale
  • align sd_type_t with the new ggml FP8 types and update the ggml submodule

Related Issue / Discussion

N/A

Additional Information

N/A

Checklist

@leejet
leejet merged commit be0e344 into master Aug 27, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant