Problem
Every row of .llms-benchmark/README.md was measured under oddyssey 1.12.0. Release 1.13.0 (e7fd9fa) changed what a run reads and runs: one observation depth (#620 - the mission no longer names a depth), environment-scoped stack_config and the persisted grafana context in the MCP server (#621, #622), and the two-runs-per-model protocol of launch-llms-benchmark (#634). Rows taken under 1.12.0 are not comparable with a row taken under 1.13.0.
What to do
Re-run the whole table under 1.13.0 with the launch-llms-benchmark protocol as it stands at e7fd9fa - every model on its CLI, two runs each, the better run in the table - on one branch, and propose the re-sorted ranking in one PR.
Rows (14): z-ai/glm-5.3, deepseek/deepseek-v4.1-flash, z-ai/glm-5.3-flashx, google/gemini-3.7-flash, google/gemini-3.8-flash, z-ai/glm-5.3-flash, qwen/qwen3.8-27b, qwen/qwen3.8-max-0902 on opencode; openai/gpt-5.6-luna, openai/gpt-5.6-terra, openai/gpt-5.6-sol on copilot; anthropic/claude-opus-5, anthropic/claude-fable-5.1, anthropic/claude-sonnet-5 on claude.
A model that fails a run moves to the end of the queue and is replayed; a model that fails twice keeps its 1.12.0 row marked provisional, with the failure recorded in the PR.
Problem
Every row of
.llms-benchmark/README.mdwas measured under oddyssey 1.12.0. Release 1.13.0 (e7fd9fa) changed what a run reads and runs: one observation depth (#620 - the mission no longer names a depth), environment-scopedstack_configand the persisted grafana context in the MCP server (#621, #622), and the two-runs-per-model protocol oflaunch-llms-benchmark(#634). Rows taken under 1.12.0 are not comparable with a row taken under 1.13.0.What to do
Re-run the whole table under 1.13.0 with the
launch-llms-benchmarkprotocol as it stands at e7fd9fa - every model on its CLI, two runs each, the better run in the table - on one branch, and propose the re-sorted ranking in one PR.Rows (14):
z-ai/glm-5.3,deepseek/deepseek-v4.1-flash,z-ai/glm-5.3-flashx,google/gemini-3.7-flash,google/gemini-3.8-flash,z-ai/glm-5.3-flash,qwen/qwen3.8-27b,qwen/qwen3.8-max-0902on opencode;openai/gpt-5.6-luna,openai/gpt-5.6-terra,openai/gpt-5.6-solon copilot;anthropic/claude-opus-5,anthropic/claude-fable-5.1,anthropic/claude-sonnet-5on claude.A model that fails a run moves to the end of the queue and is replayed; a model that fails twice keeps its 1.12.0 row marked provisional, with the failure recorded in the PR.