Repository navigation
Commit 283cd1b
committed
feat(batch): add LlamaBatchExt and harden context resource management
- Support text, embedding, and paired MTP inputs with explicit batch ranges
- Route extended batches through llama_process for encoding and decoding
- Add synchronized graph reservation with dimension validation
- Retain attached LoRA adapters and guard closed-context operations
- Complete resource cleanup before reporting errors and prevent double frees
- Document batch insertion methods, ownership, and lifecycle
- Consolidate batch and context regression tests in test_runtime
Signed-off-by: JamePeng <jame_peng@sina.com>1 parent 4b0ed29 commit 283cd1b
3 files changed
Lines changed: 1198 additions & 127 deletions
0 commit comments