Support request-level cache for hybrid sliding-window attention - #1545

Open

shihaobai wants to merge 15 commits into

mainfrom

bsh/hybrid-sliding-window-cache

Commits on Sep 4, 2026

feat: support request-level sliding-window cache
shihaobai
committed
57e69cc

Commits on Sep 6, 2026

refactor: resolve decode kv indexes in request manager
shihaobai
committed
ffd84b0
refactor: keep sliding runtime state model-local
shihaobai
committed
f80ec44
fix: harden hybrid sliding-window state lifecycle
shihaobai
committed
87ee9b4

Commits on Sep 7, 2026

fix: bound sliding window commits to retained tokens
shihaobai
committed
8c25c4d
docs: defer Gemma hybrid cache guide until page redesign
shihaobai
committed
fd7c653
feat: support CPU cache for hybrid sliding windows
shihaobai
committed
f32a597
refactor: simplify hybrid cache state interfaces
shihaobai
committed
adb4155
refactor: address sliding window state without token tables
shihaobai
committed
01016ac
refactor: unify sliding-window runtime KV
shihaobai
committed
864182e

Commits on Sep 9, 2026

Commits on Sep 10, 2026

Merge main into bsh/hybrid-sliding-window-cache
shihaobai
committed
e7331e8
Optimize Gemma4 sliding-window slot reuse and attention state
shihaobai
committed
47c67bd