fix(pflash): score the user query and reject non-finite scores

[Graffioh]

Summary

• maps the final normalized user message, including string /v1/responses input, into the drafter-rendered token stream and uses its last eight tokens as the scorer query
• fails closed if that user query cannot be mapped, rather than falling back to the assistant/chat suffix
• keeps scorer-query selection separate from DFLASH_COMPRESS_QUERY_TOKENS, which remains the lexical-anchor window
• propagates the query window through local Qwen3/Qwen3.5, layer-split, and IPC paths
• assembles scorer-Q windows that cross a 4,096-token forward chunk boundary
• checks score-graph execution and readback, rejecting NaN/Inf values before top-k reduction

Why

The previous fixed lookahead=8 window landed on the rendered assistant suffix rather than on the question. On Strix Halo/gfx1151, corrupted HIP forward values could also reach the scorer; NaN comparisons then left earlier maxima in place and silently produced a selection.

Query selection now starts from the same normalized message stream used by prompt rendering. That keeps OpenAI Chat, Anthropic Messages, Responses arrays, and Responses string input under one user-message contract.

Changed flow

flowchart LR
    A["HTTP request<br/>chat / messages / responses"] --> B["normalize_chat_messages"]
    B --> C["final user text"]
    C --> D{"map suffix in<br/>drafter tokens"}
    D -->|missing| X["reject compression"]
    D -->|mapped window| E{"drafter placement"}
    E -->|local or layer split| F["Qwen3 / Qwen3.5 scorer"]
    E -->|remote| G["PFlash IPC daemon"]
    G --> F
    F --> H["capture scorer Q<br/>across 4,096-token chunks"]
    H --> I["score graph"]
    I -->|failed or non-finite| X
    I -->|finite| J["pool and chunk top-k"]

Loading

FlowKV continues to score the tail of each independently compressed aged message. Whole-prompt PFlash supplies the explicit user-query window.

Verification

Current single-commit tree 78ac6178e:

• HIP Release build for gfx1151: test_server_unit
• HIP_VISIBLE_DEVICES=1 ./build-hip-gfx1151/test_server_unit: 440 passed, 0 failed
• regression coverage includes Responses string-input normalization, user-query mapping, weak-match rejection, cross-chunk capture slicing, and NaN/Inf detection
• git diff --check

Finite end-to-end Responses string-input validation on Radeon AI PRO R9700/gfx1201, using the supported ROCm Phase 1 q8 fallback:

• input: 12,025 tokens as a plain /v1/responses string
• scorer query mapped to the actual user suffix [12008, 12016)
• PFlash compressed 12025 -> 6009 tokens (50.0% kept)
• response completed successfully with ORCHID; effective input was 6,009 tokens
• PFlash forward+score: 6.50 s; target prefill: 7.45 s; decode: 30.3 tok/s

Fresh CI on 78ac6178e:

• Linux build, server unit tests, and megakernel imports: passed
• Windows MSVC + CUDA build and smoke: passed
• Radeon AI PRO R9700/gfx1201: passed
• Strix Halo/gfx1151: passed
• RTX 3090/sm_86: passed
• workspace lock/sync/import checks: passed
• speed profile: passed
• DGX GB10/sm_121: cancelled after its self-hosted runner remained unavailable; no test failure

The same long Responses request on the optional ROCm Phase 2 sparse-attention build produced 131072/131072 non-finite tail scores at layer 1 and failed closed as intended. The result was unchanged under the documented legacy DFLASH_FP_NOPE_TAIL=0 scoring mode.

Earlier live 16K Strix Halo/gfx1151 validation of the same scoring mechanism:

• uncompressed control: 2/2 correct
• PFlash: failed closed with explicit non-finite score diagnostics
• query mapping log identified the actual user-query window, e.g. [16374, 16382), rather than the rendered assistant suffix

Paired long-context benchmark campaign: Luce-Org/luce_box#85.

Boundaries

The underlying ROCm Phase 2/gfx1151 numeric corruption is intentionally not hidden or fixed here. This change makes it observable and prevents quality results from being reported from invalid scores. The supported ROCm Phase 1 path has a finite successful end-to-end result above.

Review status

Ready for human review. The implementation, focused contract tests, finite end-to-end proof, and all runnable CI checks are green. The PR remains marked draft until the author chooses to flip the GitHub review state.

[cubic-dev-ai]

All reported issues were addressed across 14 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>