Skip to content

Opt-in weight integrity verification at load time (pinned SHA-256 hashes / manifest) #14784

Open

Description

@UniversePeak

Is your feature request related to a problem? Please describe.

from_pretrained loads whatever weight files are on disk without any integrity verification at load time. Transport is content-addressed (LFS etag / Xet) and hf cache verify (huggingface_hub v0.32+) can re-check a cache directory against what the Hub currently serves, but both of those guarantees end before the actual load: a checkpoint that was tampered with after verification, a modified snapshot in a shared cache, or any local directory passed directly to from_pretrained is loaded as-is, and there is no way to pin approved weight hashes that get enforced when the model loads.

This matters more as automated pipelines and agents consume public checkpoints without a human in the loop. Recent work shows the threat is practical: backdoored latent world-model checkpoints can hijack downstream control while passing pre-deployment checks (arXiv 2609.15781, "When the World Lies", Sep 2026), and scanner-evading deserialization attacks on model files keep evolving (arXiv 2607.17503, "ShadowPickle", Jul 2026). Pinning a revision protects identity ("this is what was uploaded at that commit") but does not let a user detect that the bytes being loaded today differ from the ones they approved, and hf cache verify is a manual, cache-only step outside the load path.

Describe the solution you'd like.

An opt-in integrity check in the existing load path, e.g.:

pipe = DiffusionPipeline.from_pretrained(
    "org/model",
    expected_hashes={"unet/diffusion_pytorch_model.safetensors": "<sha256>", ...},
)

or equivalently integrity_manifest="path/to/manifest.json" (or a manifest discovered inside the local directory). Proposed behavior:

  • Before weight materialization, hash each weight file with SHA-256 and compare against the pinned value.
  • On mismatch, raise a clear dedicated error naming the file and both hashes; an optional flag could downgrade this to a warning for soft adoption.
  • While the files are open anyway, surface tensor-shape mismatches between checkpoint and config as a clear early error instead of a deep load-time failure.
  • Default behavior unchanged: no kwarg, no verification, no extra I/O.

Scope note: this verifies your approved bytes are the bytes being loaded (tampering, substitution, cache drift). It cannot certify that an upload you approved is benign — that would need external attestation and is out of scope.

Describe the alternatives you've considered.

  • Revision pinning — necessary but not sufficient: no detection of post-download tampering, and no contract for local directories.
  • hf cache verify — great post-hoc audit, but manual, cache-directory-only, checks against the Hub's current state rather than hashes the user pinned, and not wired into loading.
  • Verification in huggingface_hub itself — plausible home, but users express trust at from_pretrained, so surfacing the contract in diffusers would match where the decision is actually made; happy to be told the Hub is the better layer.
  • safetensors — removes arbitrary-code-execution risk on load, but says nothing about whether the weights are the ones the user expects.

Additional context.

Happy to work on this if maintainers think it fits. Concrete plan: a small additive change on the existing checkpoint-loading path plus regression tests — a known-clean checkpoint must load unchanged, and a tampered checkpoint (one flipped byte → SHA-256 mismatch) must raise the dedicated error; also report the false-positive rate on clean checkpoints.

Activity

  1. chinmayirm commented on Sep 22, 2026

    @chinmayirm

    Hi @UniversePeak, thanks for writing this up. I agree with the problem and with keeping the check opt-in.

    Root cause:

    from_pretrained resolves a file (hf_hub_download / snapshot_download, or a local path) and then load_state_dict in model_loading_utils.py opens it. Nothing between those two steps compares the bytes to a hash the caller pinned. The LFS etag / Xet check and hf cache verify both finish before that open, and a local directory never goes through them. A revision pin names a commit. It does not tell you that the file about to be materialized is the one you approved. safetensors closes the pickle path, and it still loads whatever tensors are in the file.

    Proposed approach:

    One helper, called from ModelMixin.from_pretrained after the checkpoint path is resolved and before load_state_dict or the FlashPack read. DiffusionPipeline.from_pretrained passes the same map through, with keys relative to the repo root (unet/diffusion_pytorch_model.safetensors), which is the contract in the request. A direct UNet2DConditionModel.from_pretrained uses keys relative to that directory.

    pipe = DiffusionPipeline.from_pretrained(
        "org/model",
        expected_hashes={"unet/diffusion_pytorch_model.safetensors": "<sha256>"},
    )
    # or integrity_manifest="/path/outside/the/checkpoint/manifest.json"

    The manifest path should be one the caller supplies. A json discovered inside the snapshot sits next to the weights, so whoever can replace the checkpoint can replace the manifest too. The pin has to come from outside that directory.

    Behavior when the kwarg is set:

    • Stream SHA-256 in fixed-size chunks. Do not read the file into RAM, and do not mmap it until the digest matches.
    • Every file that is about to be materialized must be in the map, including each shard. An unlisted weight file is a failure, otherwise a substituted shard never gets checked. Pins for files that are not loaded are reported and ignored.
    • On mismatch, raise a dedicated error with the relative path, the pinned digest, and the digest just computed. on_mismatch="warn" is the explicit downgrade. The default when verification is requested is to raise.
    • Shape check stays inside this opt-in path so the default load does no extra I/O. The safetensors header already has dtype and shape. Compare that to the empty module from from_config before any tensor is allocated, and raise a dedicated shape error instead of failing deep in the copy.
    • No kwarg: today's path, unchanged.

    This checks that the approved bytes are the bytes being loaded. It does not say those bytes are benign.

    Tests:

    On a tiny local checkpoint, no Hub download:

    • No kwarg, and a correct pin: the loaded state dict matches an unverified load.
    • One flipped byte: the dedicated error, with that path and both digests, and no module receives the tensor.
    • A weight file missing from the map: error.
    • A safetensors header whose shape disagrees with the config: the shape error, before materialization.
    • on_mismatch="warn": load continues, warning names the file.

    SHA-256 of the same bytes does not false-positive. The clean-pin test is the check that an unmodified file still loads.

    I would be glad to open a PR for this once the approach looks right to you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions