feat(benchmark): Vertex AI multi-model benchmark generation (#6913 foundation)#9638

Open

MarkusNeusinger wants to merge 11 commits into

mainfrom

claude/vertex-ai-model-testing-57y0cm

Commits on Jul 11, 2026

feat(benchmark): Vertex AI multi-model benchmark generation
claude
committed
docs: add PR ref to changelog entry
claude
committed
fix(benchmark): scrub render env, slug on normalized model id, runner tests
claude
committed
fix(benchmark): isolated-mode render interpreter, guard summary step
claude
committed
fix(benchmark): null token accounting for unreported usage, routing docstring
claude
committed
fix(benchmark): remap HOME to a scratch dir for the render subprocess
claude
committed
fix(benchmark): clear stale renders per attempt, honest extraction-failure retry
claude
committed
fix(benchmark): prompt contract wording, google-auth import guard, docs
claude
committed
fix(benchmark): canvas gate covers both themes, record fence compliance
claude
committed
test(benchmark): cover provider transports and main loop (codecov patch gate)
claude
committed
fix(benchmark): validate numeric CLI args, single-line summary errors
claude
committed