[leehack]

Fixes #31.

LoRAs no longer crash the process on FP8 weights, FP8 adapters or Metal; auto-fit plans FP8 checkpoints at the size Metal holds them; the FP8 test covers E5M2. Every crash below is pre-existing (same on main and the pre-#30 pin).

Root causes

Crash	Where	Cause
FP8 base + any LoRA, immediately (and auto, which picks it for unquantized weights), CPU: SIGSEGV	LoraModel::build_lora_graph → ggml_ext_cast_f32 → ggml_compute_forward_mul_mat_one_chunk calls address 0	The F32 cast is mul_mat(weight, ones); ggml-cpu has no FP8 vec_dot. The write-back would then abort too: no F32→FP8 from_float.
FP8 lora_down, any mode, CPU: SIGSEGV	get_out_diff → ggml_ext_linear(x, lora_down) → same null vec_dot	LoRA tensors are multiplied as stored; only Linear casts FP8 weights first.
LoRA on an F16 weight, immediately/auto, Metal: SIGABRT ggml-backend.cpp:1089 "pre-allocated tensor … (ADD)"	ggml_add_inplace(weight F16, diff F32) in build_lora_graph	ggml-metal ADD needs src0->type == src1->type; the weight lives in an MTL0 buffer, so the scheduler cannot move the op. Every model has F16 weights: upstream creates Conv2d, grouped Conv2d and Conv3d weights as F16 whatever the file holds (ggml_block.hpp:519/604/761), so F32 and BF16 checkpoints abort too with a LoRA on a convolution; FP8 bases, which #30 loads as F16 on Metal, abort with any LoRA.

All three confirmed under lldb on a RelWithDebInfo build (operand types printed at the faulting node) and reproduced on SD-Turbo F16/FP8 with the audit's synthetic LoRAs.

Fix

• patches/lora_immediate_fallback.patch (diffusion_engine.cpp, configure_weight_loading): FP8 weights, and any model whose TE, diffusion or VAE runtime backend rejects an in-place F16 += F32 add (a supports_op probe) outside host memory, take the LoRA at_runtime. auto selects it silently; immediately warns, as upstream already does for INT8 tensorwise weights. In practice Metal always applies LoRAs at runtime; CPU, BLAS (host buffers, where the scheduler runs the add on the CPU) and Vulkan merges are unchanged. Since dc48762 the probe no longer depends on stored F16 weights (audit P2).
• patches/fp8_metal_f16_load.patch: FP8 LoRA tensors load as F16 on every backend (the register_param_tensors retype now also applies to ModelComponent::LoRA), and backend_fit.cpp estimate_components counts FP8 weights at F16 size when a Metal device exists (it overestimates a component that the plan keeps off Metal).
• tests/test_fp8_runtime.py: E5M2 and BF16 copies; linear, FP8-down and convolution (x_embedder.proj) LoRAs in each apply mode on F32/F16/BF16/E4M3/E5M2 weights per device, checking the image against at_runtime, the applied-tensor log and the fallback warning; TE and TAESD on MTL0 behind a CPU diffusion model; FP8 weights a blocks=f16 type rule loads as F16 merge without fallback (CPU); auto-fit plan on Metal (a model with an ignored 8M-element DiT tensor, since the plan prints whole MiB); retirement check now also matches FP8 helpers (fp8, any case) and _fp8 kernel names.

Not chosen: fixing the merge graph in lora.hpp (a header; the carry mechanism compiles patched copies of .cpp sources only, and diverging inline copies would violate ODR), adding FP8 kernels or an F32→FP8 conversion to ggml (upstream ggml).

Evidence (Apple M4 Max, macOS arm64, release builds)

Tiny PixArt model, 3 weight types × 2 LoRAs × 3 modes + plain, on CPU and MTL0 (42 runs, at 5077a47):

Build	Result
main 1694916	9 SIGSEGV on CPU (FP8 base immediately/auto; FP8 lora_down in every mode), 8 SIGABRT on MTL0 (F16/FP8 base immediately/auto)
This PR	42/42 exit 0; every LoRA image identical to that weight type's at_runtime image, and different from the plain image

SD-Turbo (sd_turbo-f16.gguf, sd_turbo-fp8-all.safetensors), audit LoRAs: all 12 previously crashing or reference cases exit 0; immediately logs the fallback; F16 CPU immediately output is byte-identical to the #30 head.

Test measurements at dc48762: weight types vs F32 control mean ≤ 0.031 (CPU), 0 (Metal); LoRA modes vs at_runtime ≤ 0.112 (CPU), 0 (Metal), tolerance 0.5; LoRA effect 1.6 (linear), 0.66 (conv). Application is checked from the runtime log rather than an effect threshold. Auto-fit DiT plan with the pad: F16 32 MiB, FP8 16 MiB on main, 32 MiB here.

Audit P2 at 5077a47 (llamadart-evidence/sd-native-pr-32/audit/): F32 and BF16 weights with a convolution LoRA abort MTL0 under immediately/auto; at dc48762 they fall back and pass.

Mutations at dc48762 (each rebuilt and run with SD_REQUIRE_RUNTIME=1 SD_REQUIRE_METAL_GENERATION=1)

6c4d699 changes only the test: the module test first generates the F32 control on Metal, so it skips on GitHub's macOS runners (MTL0 listed, cannot generate; dc48762's run SIGBUSed there). The probe-only-DIFFUSION mutant is re-checked against it: still killed (te, vae); the whole module passes locally on Metal.

Mutant	Killed by
5077a47's fallback patch (F16-gated probe, audit P2)	F32/BF16 conv LoRA immediately/auto on MTL0 (8 subtests)
Probe only DIFFUSION / no VAE / no TE	module fallback test (te, vae)
Ignore expected_type	type-rule merge test
FP8 blocker matches only E4M3	E5M2-weight subtests (CPU, BLAS)
Host-buffer shortcut always true / probe always true	MTL0 subtests (16)
No FP8 blocker / auto ignores blocker / immediately ignores blocker	26 / 15 / 27 subtests
LoRA retype Metal-only	FP8-down subtests

Survives by design: removing the host-buffer shortcut only makes BLAS fall back to at_runtime (same image); the test asserts warnings on CPU and Metal only.

Mutations at 5077a47

Mutant	Killed by
main patches (failing-first)	LoRA CPU/BLAS/MTL0 (21 subtests), auto-fit
FP8 blocker removed	FP8-base immediately/auto (CPU, BLAS, MTL0)
F16 probe skipped / probe always true	F16-base immediately/auto on MTL0
auto ignores blocker	auto subtests (CPU, MTL0)
immediately ignores blocker	immediately subtests
LoRA retype Metal-only	FP8 lora_down on CPU/BLAS
E5M2 dropped from the load retype (#31 item 1)	test_fp8_generates_on_each_gpu (E5M2, MTL0)
auto-fit retype removed / E4M3 only	test_auto_fit_plans_fp8_as_f16_on_metal

Retirement regexes checked on samples: match GGML_TYPE_F8_E4M3, ggml_is_type_fp8(…), kernel_mul_mm_f8_e4m3_f32, kernel_mul_mv_fp8_f32; not kernel_mul_mv_ext_q8_0_f32_r1_8 or the pinned sources.

Also at dc48762: validate_artifacts.py macos-arm64 ok; unittest discover -s tests with SD_REQUIRE_RUNTIME, SD_REQUIRE_METAL_GENERATION: 73 tests, 7 skipped, all pass; patched upstream sources are clang-format clean.

Reachability from llamadart

• LoRA crashes: not reachable from llamadart main (0a33025e9). It never sets sd_img_gen_params_t.loras, the classifier rejects LoRA files ("does not load yet"), and the C API parses no prompt LoRA syntax. llamadart keeps the default lora_apply_mode (auto), so image LoRA support (llamadart#1223) without this fix would abort on Apple GPUs with ordinary F16 checkpoints and crash the CPU with FP8 checkpoints or FP8 adapters.
• Auto-fit: reachable today. llamadart leaves auto_fit on and relies on it for models that do not fit; an FP8 checkpoint on an Apple GPU was planned at half its resident size.

Open items

• Behavior: on Metal immediately now always applies at runtime. A LoRA on TAESD weights, which only the immediately mode merged, is then not applied (upstream applies no LoRA to TAESD at runtime); only the generic fallback warning is logged.

• Converting weights to FP8 with a tensor type rule aborts at load (ggml.c:8312, ggml_quantize_chunk) with or without a LoRA: pre-existing upstream, not addressed.

• Upstream reports for the three root causes (stable-diffusion.cpp lora.hpp merge, leejet/ggml CPU FP8 MUL_MAT/CPY support claims) are not filed; patches/README asks for an upstream link to retire the patches.

• Vulkan: CI lavapipe only (source: Vulkan adds F16+F32, rejects FP8 MUL_MAT so Linear casts). CUDA not run.

• ASan exit teardown on Metal and iOS devices not run on this head.