What problem are you facing?
The local end-to-end test added in #365 brings the full path up on kind and asserts a live 200 through the control-plane gateway on both /v1/chat/completions and /v1/messages. That proves the path is wired and routing works — but it stops at "the endpoint answers." It doesn't assert behavior: response shapes, error paths, streaming, multi-turn, or the semantics the compositions are meant to guarantee. A regression that still returns 200 with a wrong body would pass unnoticed.
How could Modelplane help solve your problem?
Suggested by @haarchri in the #365 review (#365 (comment)): once the project is running in kind, run a Python test-suite against the published ModelService address to assert real behavior rather than just status codes. It would run in the same cloud-free harness and could grow to cover the OpenAI and Anthropic surfaces, error handling, and streaming.
This pairs naturally with the CPU-based-model idea (also raised on #365): the current mock returns canned responses, so meaningful behavioral assertions want either a richer mock or a small CPU model to exercise.
Follow-up to the local e2e added in #365. Related: #367 (KWOK nodes for scheduler-placement testing).
What problem are you facing?
The local end-to-end test added in #365 brings the full path up on
kindand asserts a live 200 through the control-plane gateway on both/v1/chat/completionsand/v1/messages. That proves the path is wired and routing works — but it stops at "the endpoint answers." It doesn't assert behavior: response shapes, error paths, streaming, multi-turn, or the semantics the compositions are meant to guarantee. A regression that still returns 200 with a wrong body would pass unnoticed.How could Modelplane help solve your problem?
Suggested by @haarchri in the #365 review (#365 (comment)): once the project is running in
kind, run a Python test-suite against the publishedModelServiceaddress to assert real behavior rather than just status codes. It would run in the same cloud-free harness and could grow to cover the OpenAI and Anthropic surfaces, error handling, and streaming.This pairs naturally with the CPU-based-model idea (also raised on #365): the current mock returns canned responses, so meaningful behavioral assertions want either a richer mock or a small CPU model to exercise.
Follow-up to the local e2e added in #365. Related: #367 (KWOK nodes for scheduler-placement testing).