Skip to content

LLM Benchmarks: Use stable model IDs and clean up build outputs - #6034

Open
bradleyshep wants to merge 4 commits into
masterfrom
bradley/llm-benchmark-use-api-model-name
Open

bradleyshep wants to merge 4 commits into
masterfrom
bradley/llm-benchmark-use-api-model-name

Conversation

@bradleyshep

Copy link
Copy Markdown
Contributor

Description of Changes

Use stable API model IDs for benchmark results and uploads. Display-name changes will no longer create separate entries for the same model.

Remove task-local Rust, C#, and TypeScript build outputs after publishing, including failed attempts. Preserve source files and shared caches. Add disk-usage logs before and after benchmarks.

API and ABI breaking changes

No public API or ABI changes. Benchmark result keys now contain model IDs instead of display names.

Rollback safety impact

No database schema changes. Reverting restores display-name keys and can create duplicate model entries on subsequent uploads.

Expected complexity level and risk

2/5. Changes are limited to the benchmark tool and workflow. Cleanup only targets task-local build directories.

The C# server SIGKILL remains unresolved; this PR does not claim to fix it.

Testing

  • All 54 benchmark library unit tests passed, including model-ID upload and cleanup regression tests.
  • Formatting and whitespace checks passed.
  • Rust and TypeScript benchmark jobs completed successfully.
  • All nine C# models completed both modes with 83 results uploaded per model/mode, using smaller recovery jobs after the full C# job failed.

@bradleyshep bradleyshep changed the title Use stable model IDs and clean up LLM benchmark build outputs LLM Benchmarks: Use stable model IDs and clean up LLM benchmark build outputs Oct 1, 2026
@bradleyshep bradleyshep changed the title LLM Benchmarks: Use stable model IDs and clean up LLM benchmark build outputs LLM Benchmarks: Use stable model IDs and clean up build outputs Oct 1, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant