[rgarcia]

Summary

Generic harness changes for benchmarks that drive the browser through Kernel MCP's browser REPL or computer controls, per-trial proxies for site-config, and per-trial browser VM setup that configurators can request. No new benchmark here; the retail-cart benchmark is #10, stacked on this PR.

• Arm and suite args: arms take toolset_args (passed to the toolset) and agent_kwargs (passed to the Harbor agent); benchmarks take suite_args. bench prepare/run take --suite-arg and --toolset-arg overrides, also as workflow inputs.

• Attempts: bench run --attempts N runs each task N times per arm (Harbor --n-attempts), also as a workflow input.

• Browser REPL and computer controls: kernel-mcp-local takes toolsets, e.g. ["repl"] for browser_repl instead of execute_playwright_code, or ["repl", "computer"] to add OS-level computer_action. Tool calls are logged at the server (mcp-call-log), and kernel-mcp-verify reads that log, so it works whatever agent harness wraps the calls.

• Per-trial proxies in site-config: the configurator used to create one proxy per distinct recipe and reuse it by name. An ISP proxy is a single static IP, so every trial of every task with that recipe left from the same IP. The configurator now emits proxy_create (a POST /proxies body); KernelTaskEnvironment creates a uniquely named proxy before each trial's browser and deletes it after, and bench run deletes any bench-<run_id>- proxies a crashed trial left behind.

• Proxy failures are identifiable: a trial whose proxy cannot be created fails with ProxyCreationError. The report now lists infrastructure failures by arm, exception type and task, and Braintrust metadata carries errorType.

• Per-trial VM setup: a configurator's browser output may include chromium_env (KERNEL_* Chromium launcher variables, or a list of sets with one picked per trial) and vm_setup (shell commands run as root in the browser's VM). KernelTaskEnvironment applies both right after creating the browser and before the first page load: it restarts Chromium through supervisord with the variables and checks the page reports the requested macOS user agent or WebGL renderer. A trial where either fails fails with VmSetupError.

• GPU browsers: Harbor requests memory in container mode, which the API rejects with gpu=true, so it's dropped for GPU browsers.

Testing

• ruff check, ruff format --check, pytest (19 passed); bun test (21 passed), tsc --noEmit, prettier --check

• bench prepare kernel-mcp-clawbench for the static arms, as in CI

• Live site-config run: two trials on the same ISP recipe got separate proxies with different egress IPs, both deleted at teardown, none left after the run. Trials whose proxy failed to create showed up as ProxyCreationError in the infrastructure failures table.

• Live: Chromium restarted with KERNEL_MAC_PERSONA=1 and with a KERNEL_WEBGL_RENDERER override on GPU browsers, each confirmed from the page; vm_setup commands run as root on a live browser; both used across the 10-attempt retail-cart runs in Add retail-cart benchmark: guest add-to-cart outcomes on bot-protected sites #10.