블로그 글 추가: 2026-06-25-lightseek-tokenspeed-kernel, TokenSpeed-Kernel: 멀티 실리콘 LLM 추론을 위한 이식 가능한 API와 고성능 커널#96
Merged
Merged
Conversation
PR Preview
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
번역 글 소개
TokenSpeed-Kernel: 멀티 실리콘 LLM 추론을 위한 이식 가능한 API와 고성능 커널 번역 글을 추가합니다.
TokenSpeed-kernel은 LLM 추론의 백엔드 복잡성을 해결하기 위한 독립형 오픈소스 서브시스템으로, 깔끔한 계층형 API와 레지스트리·선택 메커니즘으로 고수준 런타임을 하드웨어별 커널 코드로부터 분리합니다.
글에서는 GPT-OSS 120B를 예시로, 같은 공개 API(
mha_prefill,moe_apply등) 뒤에서 AMD(Gluon/CDNA4)와 NVIDIA(FlashInfer/TensorRT-LLM) 커널이 형제 구현으로 동작하는 방식을 설명합니다. AMD MI355X에서 Gluon 커널은 어텐션 프리필에서 Triton 대비 1.42.3배, 종단 간 출력 처리량에서 1.63.6배 향상을 보였으며, 해당 커널은tokenspeed-kernel-amd패키지로 분리 공개되어 vLLM에도 채택되었습니다.