feat: support chunkprefill and prefill cuda graph#371

Open

Simon12345777 wants to merge 7 commits into

InfiniTensor:mainfrom

Simon12345777:main

Commits on May 12, 2026

add chunkprefill and prefill cuda graph
Simon12345777
committed

Commits on May 13, 2026

add chunk_prefill_compiler.cpp/.hpp
Simon12345777
committed

Commits on May 15, 2026

fix attn_metadata bug
Simon12345777
committed

Commits on May 20, 2026

fix schedule priority; add anti-starve mechanism; fix issue incompatible with Prefix Sharing
Simon12345777
committed

Commits on May 21, 2026

change priority to prefill first and fix GIL problems
Simon12345777
committed

Commits on May 26, 2026

fix decode and chunk yield issue
Simon12345777
committed

Commits on May 28, 2026

add batch chunk
Simon12345777
committed