Skip to content

pnnx: write bool attributes as fp32 so the ncnn .bin stays in step (fixes #6855) - #6866

Open
giuliocorradi wants to merge 1 commit into
Tencent:masterfrom
giuliocorradi:pnnx-bool-attribute-fp32
Open

pnnx: write bool attributes as fp32 so the ncnn .bin stays in step (fixes #6855)#6866
giuliocorradi wants to merge 1 commit into
Tencent:masterfrom
giuliocorradi:pnnx-bool-attribute-fp32

Conversation

@giuliocorradi

Copy link
Copy Markdown

Fixes #6855.

A bool tensor attribute is written to model.ncnn.bin raw — one byte per element — but the MemoryData layer emitted for it is sized from the tensor shape, and MemoryData::load_model() reads that many float32 (load_type=1, untagged).

ModelBin is a sequential reader, so after an n-element bool the cursor is 3n/4 floats out of step and every weight-bearing layer after that point loads shifted data. Nothing reports it: pnnx exits 0, load_param() and load_model() both return 0, and inference produces plausible numbers that are simply wrong.

The fix widens bool to fp32 on write, mirroring the i64 -> i32 handling directly above it.

Correction to the repro in the issue

The reproduction I posted in #6855 does not actually trigger the bug — I should have checked it before filing. In that snippet the bool is a register_buffer consumed by a multiply, so torch type-promotes it to float during tracing and a type-9 attribute never reaches save_ncnn. Both patched and unpatched builds emit an identical 13224-byte .bin for it.

The real trigger is a bool that is folded into a constant and stays bool. A minimal one:

class M(nn.Module):
    def __init__(self):
        super().__init__()
        self.c1 = nn.Conv2d(3, 400, 1)
        self.c2 = nn.Conv2d(400, 8, 1)
        self.register_buffer("t", torch.randn(1, 400, 1, 1))
    def forward(self, x):
        m = self.t > 0                      # folds to a bool CONSTANT
        y = self.c1(x)
        return self.c2(torch.where(m, y, torch.zeros_like(y)))

torch.logical_and(self.a > 0, self.b > 0) in place of the comparison behaves identically.

Evidence

Two pnnx binaries built from identical source, differing only in save_ncnn.cpp:

construct unpatched patched delta
torch.where + folded bool 113240 114440 +1200
logical_and + folded bool 113240 114440 +1200

400 elements × (4 − 1) bytes = 1200, exactly the shortfall.

On a real model — an ultralytics YOLOE export, which is where I hit this — the graph carries MemoryData pnnx_fold_idx.1 0 1 335 0=400, a 400-element folded bool, and the same +1200 delta appears (47467496 → 47468696). There the shifted reads corrupted the anchor grid, which arrived as [-0.032, -0.04, ...] instead of [0.5, 1.5, 2.5, ...], along with the mask and proto convolutions. Layers earlier in the file were untouched, which is what makes the damage look selective and misleading.

One caveat on method, since it nearly misled me: I first tried to A/B the PyPI wheel against a locally built binary. That is not a valid comparison — the two differ in more than this patch (they emit different blob numbering and a different pnnx_188 shape), so the byte-level tail comparison does not line up. The numbers above are from same-source builds only.

Note for a runnable check

The minimal repro above exercises the writer but produces a graph containing torch.where, which ncnn has no layer for, so it cannot be executed end to end to show wrong output — only the wrong .bin size. The end-to-end corruption is visible on the YOLOE export.

🤖 Generated with Claude Code

A bool tensor attribute is written to model.ncnn.bin raw, one byte per
element, but the MemoryData layer emitted for it is sized from the tensor
SHAPE and ncnn reads that many float32 (load_type=1, untagged).

ModelBin is a sequential reader, so after an n-element bool the cursor is
3n/4 floats out of step and every weight-bearing layer after that point
loads shifted data. Nothing reports it: pnnx exits 0, load_param() and
load_model() both return 0, and inference produces plausible numbers that
are simply wrong.

Widen bool to fp32 on write, mirroring the existing i64 --> i32 handling
directly above, so what is written matches what will be read.

Verified with two binaries built from identical source differing only in
this file, on a 400-element folded bool constant:

    construct                      unpatched   patched    delta
    torch.where + folded bool         113240    114440    +1200
    logical_and + folded bool         113240    114440    +1200

400 * (4 - 1) = 1200 bytes, exactly the shortfall.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tencent-adm

Copy link
Copy Markdown
Member

CLA assistant check
Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pnnx: bool attributes desynchronise the ncnn .bin — MemoryData declares element count but only 1 byte/element is written

2 participants