Skip to content

docs: a placed qwen35moe prompt walks units of 512, not 8-row passes - #154

Merged
midagedev merged 2 commits into
mainfrom
perf-docs-placed
Oct 14, 2026
Merged

midagedev merged 2 commits into
mainfrom
perf-docs-placed

Conversation

@midagedev

Copy link
Copy Markdown
Owner

Docs only. No code, lever, default or gate change.

docs/research/opt-axes.md (matrix cell and row 20) and docs/research/adaptive-picks.md §3.4 still said a placed qwen35moe load runs every prompt as 8-row passes. Since the placed prompt moved to PLACED_ROWS (512) units, auto walks a 1,464-id prompt as ubatch:512x2,440 (body35.rs placed_open, prefill.rs PrefillPlan::new). Eight-row passes remain only under --prefill pass and for a prompt of at most 8 ids.

The old 8-row decomposition in §3.4 stays, labelled as the old walk. The 12 GB rate for 512-row units is the derivation in docs/cards/q35pfa.card, unmeasured on that card.

🤖 Generated with Claude Code

opt-axes row 20 and adaptive-picks 3.4 still described the walk before
the placed prompt moved to PLACED_ROWS units. The 8-row decomposition
stays, struck and labelled as the old walk.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Oct 14, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: MUST_FIX: The new pass-path claim contradicts the current planner.

Must fix

1. Include short final tails in the pass-path description

docs/research/adaptive-picks.md:172 and docs/research/opt-axes.md:217 exclude passes on long prompts under auto. A placed 520-id prompt is planned as ubatch:512 pass:8: crates/gpu/src/arch/qwen3moe/prefill.rs:647 explicitly sends a final tail of at most 8 ids to the pass path. The qwen35moe caller uses this planner (body35.rs:2459). This is a proven misleading description (kind B).

Correct this now before the false boundary is reused in numeric or performance analysis.

Replace adaptive-picks.md line 172 with:

   `placed.rs`). `--prefill pass` uses passes of at most 8 ids; `auto` also uses a pass for a prompt or final tail of at most 8 ids. The

Replace opt-axes.md line 217 with:

| 20 | `q35hybpp` | D, A | A placed qwen35moe prompt walks units of up to `PLACED_ROWS` (512) on the wide arm (`placed.rs`, `body35.rs` `placed_open`); `--prefill pass` uses passes of at most 8 ids, and `auto` also uses a pass for a prompt or final tail of at most 8 ids. What is left is the host union's compute, and `xstream` for this family. Unmeasured on a 12 GB card (`docs/cards/q35pfa.card`) |

Checked

Duplicates: 0 new functions, types or constants; no dryer pairs supplied.
Complexity: no Rust functions changed.
Silent failure: 0 changed sites.
Tests: no tests changed; mutation check not applicable.
Gates: no gates or pins changed.
Escape catalog: no C1–C4 patterns matched.
Performance: no runtime path changed; the cited rate is explicitly derived and unmeasured.

Notes

dryer and crapper did not run because no Rust files changed.

🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)

Comment thread docs/research/adaptive-picks.md Outdated
1. **Where.** ~~A placed qwen35moe load runs every prompt as passes of at most 8 rows through the host tier's batch
port, whatever its length.~~ A placed qwen35moe load now walks a prompt in units of `PLACED_ROWS` = 512 rows on
the wide arm, each unit reading a host expert once (`crates/gpu/src/arch/qwen3moe/body35.rs` `placed_open`,
`placed.rs`). Eight-row passes remain only under `--prefill pass` and for a prompt of at most 8 ids. The

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Under auto, a placed 520-id prompt is planned as ubatch:512 pass:8; short final tails also take the pass path.

Suggested change
`placed.rs`). Eight-row passes remain only under `--prefill pass` and for a prompt of at most 8 ids. The
`placed.rs`). `--prefill pass` uses passes of at most 8 ids; `auto` also uses a pass for a prompt or final tail of at most 8 ids. The

Comment thread docs/research/opt-axes.md Outdated
| 18 | S13 N-node placement | F | Where the book sits across Sparks. Contributor holds the placement half |
| 19 | `HOST_LOCK` | F | Page-cache budget. Landed as a lever |
| 20 | `q35hybpp` | D, A | Placed Qwen3.5 prompts walk at most `MAX_PASS_ROWS` (8) rows (`model.rs`). The long-prompt cut is the card's roofline. Owned by common |
| 20 | `q35hybpp` | D, A | A placed qwen35moe prompt walks units of `PLACED_ROWS` (512) on the wide arm (`placed.rs`, `body35.rs` `placed_open`); 8-row passes are `--prefill pass` or a prompt of at most 8 ids. What is left is the host union's compute, and `xstream` for this family. Unmeasured on a 12 GB card (`docs/cards/q35pfa.card`) |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Under auto, a placed 520-id prompt is planned as ubatch:512 pass:8; short final tails also take the pass path.

Suggested change
| 20 | `q35hybpp` | D, A | A placed qwen35moe prompt walks units of `PLACED_ROWS` (512) on the wide arm (`placed.rs`, `body35.rs` `placed_open`); 8-row passes are `--prefill pass` or a prompt of at most 8 ids. What is left is the host union's compute, and `xstream` for this family. Unmeasured on a 12 GB card (`docs/cards/q35pfa.card`) |
| 20 | `q35hybpp` | D, A | A placed qwen35moe prompt walks units of up to `PLACED_ROWS` (512) on the wide arm (`placed.rs`, `body35.rs` `placed_open`); `--prefill pass` uses passes of at most 8 ids, and `auto` also uses a pass for a prompt or final tail of at most 8 ids. What is left is the host union's compute, and `xstream` for this family. Unmeasured on a 12 GB card (`docs/cards/q35pfa.card`) |

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@midagedev

Copy link
Copy Markdown
Owner Author

Folded in 5997e0d1: both lines now say auto also takes a pass for a prompt or final tail of at most 8 ids (prefill.rs PrefillPlan::new, tail ≤ MAX_TOKENS). Shingler's wording applied as suggested. No planner code changed. Please re-review.

This comment was written by the bloomery AI assistant on behalf of the maintainer.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the last review: must-fix 1 is fixed in docs/research/adaptive-picks.md:172 and docs/research/opt-axes.md:217.

Verdict: CLEAN: The revised prompt-path descriptions match the planner.

Checked

Duplicates: 0 new functions, types or constants; no dryer pairs supplied.
Complexity: no Rust functions changed; no crapper results supplied.
Silent failure: 0 changed sites.
Tests: no tests changed; mutation check not applicable. Source checks confirm the 8/9-id and 512/513/520/521-id boundaries; empty prompts are refused.
Gates: no gates or pins changed.
Escape catalog: no C1–C4 patterns matched.
Performance: no runtime path changed; the cited rate remains explicitly derived and unmeasured.

🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)

@midagedev
midagedev merged commit 4713ad0 into main Oct 14, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant