missing parameters in OpenAIChatCompletionRequest

[Kevoting]

Hello,
I believe there is an issue with the "OpenAIChatCompletionRequest" class, the do_sample boolean is specified, but setting it has no effect, which causes the model to repeat the same message after regeneration. The "seed" parameter is ignored if "do_sample" is "false".

Tested with Gemma 2 using the LLM worker. If I hardcode the value to true in the code, it picks up the seed, and the message is different.

Also, i don't see a way to set "stream_chunk_tokens" it is not included in OpenAIChatCompletionRequest and is ignored. But setting it in the code does work.

[SearchSavior]

The "seed" parameter is ignored if "do_sample" is "false".

@Kevoting this behavior is correct; do_sample false would be greedy decoding without samplers.

However some fields are missing `https://github.com/SearchSavior/OpenArc/blob/main/src%2Fserver%2Fmodels%2Fov_genai.py#L7

In that way what you found is the Optional pydantic typing causing mischief.

A small PR adding what you found would be much appreciated.

Try changing stream_chunk_tokens, I found it hard to follow at any other value than 1, so the real fix ought to be removal from performance metrics which I think has been overdue.

Thanks for the issue, Lmk what you think.