gpu-vision, vision, serve: MiniCPM-V 4.7 images end to end, a tile capped at 70 tokens

[midagedev]

What and why

MiniCPM-V 4.7 images for 0.3.3. The tower runs end to end on the card. It is held against mainline 358a420e's nodes, and a tile group is encoded and prompted as a canvas. The serve seat caps a tile at 70 tokens. A larger tile is refused by the tower's existing buffer check.

• gpu-vision: the minicpmv4_7 tower arm, vis_attn_group and vis_mean_groups (held by gate_minicpmv_tower against their host rules), the vit_merger and the 2x2 merger.
• tools/ref, refset: the minicpmv4_7 oracle and family. The family refuses another mmproj, oracle build or rope order by name. gate-gpu-minicpm-text runs the e2e's self-consistency clauses on MiniCPM's file.
• vision, gpu, serve: Prepared is Grid or Tiles. The rope follows the file's section order (zyxt). A grid encode refuses a tile group by name.
• serve: ImageNumber, PlaceholderCount and ImageLast are HTTP 400. A build without the vision feature refuses --mmproj by name.

Code shape

• No second copy.
• Model code holds only architecture facts.
• Dead code is gone.
• No silent failure.
• Comments state what is true now.
• No relaxed gate. vis_attn_group and vis_mean_groups are new rows in tools/ref/ptx-shapes.tsv, pinned at 0 spill.
• fmt and clippy clean; lint 41 = ratchet.

Duplication found or removed: none.

What I ran

• Rebased on 27b4ce3. Three WIP commits were squashed into one, and the tree equals the pre-squash tree minus a stale handoff file.
• Mac static (full, no base): fmt-check 0, lint 41 = ratchet 41, combos 42 shapes 0 red, check-arch/comments/defaults/levers/loads/ptx-names/records/rustflags/unsafe/waits 0. check-recipes was red on the remote only (one git range self-test). tools/check-recipes.sh on the same tree here: check-recipes: ok.
• GPU, workstation (A6000 and 3090, one job at a time), on 4aeb8ca: the fixture tier of just affected (159 items) ran 156 green, and the 3 that lost the ssh session reran green (gate-gpu-qwen35moe-attn, gate-gpu-routed-legs, gate-ds41-bind). The real tier of the vision path, all rc 0: gate-refset (198 passed), gate-vision, gate-gpu-vision-lib, gate-gpu-minicpmv-tower, gate-gpu-minicpm-text, gate-gpu-vision, gate-gpu-qwen3vl-tower, gate-gpu-qwen35moe-media, gate-gpu-qwen35-media, gate-gpu-qwen3-serve, gate-gpu-ds41-media. e3ff6dd (the test helper only): gate-refset 198 passed, Mac static fmt 0 and lint 41.
• Not run: a ptx-scan pair against the base. The two new vis_ entries are pinned in tools/ref/ptx-shapes.tsv at 0 spill.

Prepared by the bloomery assistant on behalf of @midagedev.

🤖 Generated with Claude Code

[github-actions]

Verdict: MUST_FIX: Shared sizing, text validation, and reference utilities have new copies.

Must fix

1. Use the existing card sizing owner

crates/gpu-vision/src/minicpmv.rs:95 duplicates crates/vision/src/arch/minicpmv/card.rs:41. The new Lens also repeats card::ScratchLens, and ff_padded repeats card::padded_ff. Kind B (R14). One placement fact has two implementations.
Centralize before the next sizing change.

Delete local ff_padded, weight_bytes_of, Lens and its impl; use:

use vision::arch::minicpmv::card::{
    self, ScratchLens as Lens, padded_ff as ff_padded,
};
pub use vision::arch::minicpmv::card::weights as weight_bytes_of;

pub fn scratch_bytes_of(hp: &Hparams, max_patches: usize) -> u64 {
    card::ScratchLens::of(hp, max_patches).bytes()
}

Rename lens uses: stream → xa, att → attn, rows → out, including self.lens and tests. Allocate xb/h from lens.xb/h.

2. Call the shared projector validation

crates/gpu-gates/src/bin/shared/serve_seats/qwen3.rs:1342 copies the vocabulary, RoPE and PLE checks in crates/gpu-gates/src/vision_seat.rs:388. These checks are common policy. Kind B (R14).
Reuse them before the policy needs another edit.

Remove the unused use vision::MediaModel;. Replace let media = Media::new(...) through the returned Opened with:

let projector = vision_seat::Projector {
    path: path.to_owned(),
    rule,
    card,
    out_dim: hp.out_dim,
    media: Arc::new(Media::new(hp, marks, PAD_TEXT, pad)),
    cap: limits.max(),
    limits,
};
projector.check_text(text, token)?;
let seat = vision_seat::encoder_seat(place, projector.card.total())?;
Ok(vision_seat::Opened {
    path: projector.path,
    seat,
    media: projector.media,
    cap: projector.cap,
    card: projector.card,
    limits: projector.limits,
})

3. Share the prompt manifest parsers

crates/refset/src/arch/minicpmv4_7/prompt.rs:700 copies crates/refset/src/clefvis.rs:1195 exactly. read_chat at line 664 copies crates/refset/src/clefvis.rs:1501, the parser for the same header format. Kind B (R14).
Share before the header format changes.

Make the existing clefvis.rs functions pub(crate). Delete both new copies; replace the ChatLine import with:

use crate::clefvis::{ChatLine, read_chat, unescape};

4. Keep one refusal helper in the test harness

crates/refset/src/arch/minicpmv4_7/prompt_tests.rs:154 and crates/refset/src/arch/minicpmv4_7/vis_tests.rs:140 repeat the same refusal assertion; crates/refset/src/arch/glm5next/vis_tests.rs:279 is a third copy. Kind B (R14).
Share before another family copies it.

Put this in clefvis/testkit.rs; delete the three copies and import testkit::refusal:

pub(crate) fn refusal<T>(r: Result<T, crate::RefError>) -> String {
    match r {
        Ok(_) => panic!("the set was taken"),
        Err(e) => e.to_string(),
    }
}

Non-blocking

⚠ Engine image-prompt gate: crates/gpu-gates/src/bin/shared/serve_seats/qwen3.rs:739 has no comparison against the new prompt oracle. Tower/text gates do not run this image prompt. An equal-area Grid replacing Canvas here survives them. Compare the seat's hidden rows and positions before extending this path.

⚠ Host stage boundaries: crates/gpu-vision/src/minicpmv.rs:326 combines validation, weight packing/upload, scratch allocation and kernel loading. Split those host stages (R12). Split crates/gpu-gates/src/bin/gate_minicpmv_tower.rs:1039 by load, teacher-forced checks, reuse and oracle comparison.

Checked

Duplicates: all four shared-owner defects confirmed.
Complexity: load and tower clauses mix host stages; splitting remains non-blocking.
Silent failure: the base issue reproduced.
Tests: equal-row-count Grid passes span checks; tower/text gates miss the seat path. No hardware run.
Gates: two zero-spill rows added; unsafe-count increase has a dated PIN and reason.
Escape catalog: no override applies.

Notes

The PR body has no complete card gate run, FAIL-first output, or PTX scan evidence. GPU verification remains pending.

_{Card check: needs-gpu. This pull request changes crates/gpu-gates/Cargo.toml, crates/gpu-gates/src/bin/gate_ds41_media.rs, crates/gpu-gates/src/bin/gate_minicpmv_tower.rs … and carries no gate run on a card (CONTRIBUTING's evidence list); a maintainer adds gpu-verified after one.}

_{🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)}

Filed outside this PR: #151

[midagedev]

The four must-fix items are folded in 976afe6 and 4aeb8ca (head 4aeb8ca).

1. Card sizing: gpu-vision's minicpmv uses card::{padded_ff, weights, ScratchLens}. The local copies are gone. The lens formulas were compared field by field before the delete.
2. Text validation: open_minicpm builds a vision_seat::Projector and calls check_text and encoder_seat.
3. Manifest parsers: unescape is shared. read_chat was not a pure copy (its errors name the set's family), so clefvis now has parse_chat(fields, at), and both families call it with their own location.
4. testkit::refusal replaces the three copies.

Evidence on 4aeb8ca: full Mac static EXIT 0 (fmt 0, lint 41 = ratchet, 42 combos 0 red, every check-* 0). Workstation, real tier: gate-refset (198 passed), gate-vision, gate-gpu-vision-lib, gate-gpu-minicpm-text, gate-gpu-vision, gate-gpu-qwen3vl-tower, gate-gpu-qwen35moe-media, gate-gpu-qwen35-media, gate-gpu-ds41-media all rc 0. gate-gpu-minicpmv-tower and gate-gpu-qwen3-serve lost the ssh session during the build and will be rerun. The fixture-tier batch is running.

The two non-blocking items (a seat-level oracle comparison, the host stage split) are not in this change.

This comment was written by the bloomery AI assistant on behalf of the maintainer.

[github-actions]

Since the last review:

• Card sizing: fixed.
• Projector validation: fixed.
• Prompt manifest parsers: fixed.

The earlier findings still open are listed below.

Verdict: MUST_FIX: The new shared refusal helper still has two identical implementations.

Must fix

1. Finish migrating the refusal helper (still open)

crates/refset/src/clefvis/testkit.rs:294 repeats the same match, panic and error rendering in crates/refset/src/arch/qwen_vis_tests.rs:77 and crates/refset/src/arch/qwen35/clefvis/tests.rs:54. Kind B, R14. The refusal contract still has three owners.
Finish the migration before another family copies it.

Keep the generic helper. In both Qwen files, delete the local fn refusal and use crate::RefError; (that import serves only the deleted function). Add:

use crate::clefvis::testkit::refusal;

Non-blocking

⚠ Engine image-prompt gate (still open): crates/gpu-gates/src/bin/shared/serve_seats/qwen3.rs:739 lacks a comparison against the prompt oracle. The tower and text gates do not exercise this call, so replacing Canvas with an equal-area Grid is unguarded there. Add a seat gate comparing hidden rows and canvas positions.

⚠ Host stage boundaries (still open): crates/gpu-vision/src/minicpmv.rs:259 combines validation, packing/upload, scratch allocation and kernel loading. Split these host stages (R12) so each has one job. Split crates/gpu-gates/src/bin/gate_minicpmv_tower.rs:1039 into load, teacher-forced checks, reuse and oracle comparison.

Checked

Duplicates: the 2 new functions were compared with existing owners; sizing and prompt parsing now use shared owners.
Complexity: this increment adds no branches to the host loader or tower gate.
Silent failure: projector validation, chat parsing and prompt unescaping propagate named errors.
Tests: by inspection, changing Scratch::new's xb allocation at line 564 to zero would not fail the changed sizing unit test: it never allocates Scratch. Existing literal card-byte assertions check formula changes. No build or tests run.
Gates: no bands, tolerances, pins or test cases removed in this increment.
Performance: no added launches, copies or per-step allocations in the incremental diff.
Escape catalog: no C1–C4 pattern matches the incremental diff.

Notes

The PR body lacks complete card gate output, FAIL-first evidence and PTX scan evidence. GPU evidence is false.

_{Card check: needs-gpu. This pull request changes crates/gpu-gates/Cargo.toml, crates/gpu-gates/src/bin/gate_ds41_media.rs, crates/gpu-gates/src/bin/gate_minicpmv_tower.rs … and carries no gate run on a card (CONTRIBUTING's evidence list); a maintainer adds gpu-verified after one.}

_{🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)}

[midagedev]

e3ff6dd finishes the refusal helper: qwen_vis_tests.rs and qwen35/clefvis/tests.rs drop their copies and the RefError import, and take clefvis::testkit::refusal. gate-refset 198 passed on the workstation; Mac static fmt 0, lint 41.

The PR body now lists the card runs on 4aeb8ca: the fixture tier of just affected all green after three ssh-cut reruns, and the eleven real-tier vision-path gates all rc 0.

The two non-blocking items stay out of this change.

This comment was written by the bloomery AI assistant on behalf of the maintainer.

[github-actions]

Since the last review:

• Refusal helper: fixed. Both Qwen test modules use the shared owner.

Verdict: CLEAN: The incremental diff replaces duplicate refusal helpers with an equivalent shared implementation; the earlier findings remain non-blocking.

Non-blocking

⚠ Engine image-prompt gate (still open): crates/gpu-gates/src/bin/shared/serve_seats/qwen3.rs:739 constructs canvas spans and passes them into the session, but no gate compares this path against the image-prompt oracle. Existing canvas tests cover row assembly and rotation tables separately. Add a seat gate comparing hidden rows and positions that fails if this call substitutes an equal-area Grid for Canvas.

⚠ Host stage boundaries (still open): crates/gpu-vision/src/minicpmv.rs:259 combines validation, packing/upload, scratch allocation and kernel loading. Split these host stages by job (R12). Split crates/gpu-gates/src/bin/gate_minicpmv_tower.rs:1039 into load, teacher-forced checks, reuse and oracle comparison.

Checked

Duplicates: 0 new items; both deleted helper bodies match clefvis::testkit::refusal.
Complexity: only imports and two deleted match helpers change; no production function changes.
Silent failure: 24 refusal call sites now use the shared helper; unexpected Ok still panics and Err preserves its text.
Tests: no test bodies or assertions changed. Removing clefvis.rs:1117's pad/span guard makes qwen_vis_tests.rs:123 panic through the shared helper, by inspection. No build or tests run.
Gates: no thresholds, pins or test cases changed or removed.
Performance: test-only migration; no engine hot path changes.
Escape catalog: no C1–C4 pattern matches this increment.

Notes

The PR body still lacks the card/driver line and complete gate output required for GPU evidence, FAIL-first evidence and PTX scan evidence. GPU evidence is false.

_{Card check: needs-gpu. This pull request changes crates/gpu-gates/Cargo.toml, crates/gpu-gates/src/bin/gate_ds41_media.rs, crates/gpu-gates/src/bin/gate_minicpmv_tower.rs … and carries no gate run on a card (CONTRIBUTING's evidence list); a maintainer adds gpu-verified after one.}

_{🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)}