Skip to content

Add return tokens id option#3

Merged
Thachnh merged 2 commits intodeep-main-v1.3.0rc5-eagle3-k25visionfrom
deep-main-v1.3.0rc5-eagle3-k25vision-return-token
Apr 2, 2026
Merged

Add return tokens id option#3
Thachnh merged 2 commits intodeep-main-v1.3.0rc5-eagle3-k25visionfrom
deep-main-v1.3.0rc5-eagle3-k25vision-return-token

Conversation

@quan-deepinfra
Copy link
Copy Markdown

No description provided.

Comment thread tensorrt_llm/serve/openai_server.py Outdated
async def chat_stream_generator(
promise: RequestOutput, postproc_params: PostprocParams) -> AsyncGenerator[str, None]:
nonlocal did_complete
is_first_chunk = request.return_token_ids
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this code is confusing. either change the name to something like `return_token_ids_in_first_chunk

@Thachnh Thachnh merged commit 0ec4ba4 into deep-main-v1.3.0rc5-eagle3-k25vision Apr 2, 2026
1 of 3 checks passed
@Thachnh Thachnh deleted the deep-main-v1.3.0rc5-eagle3-k25vision-return-token branch April 2, 2026 21:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants