-
Notifications
You must be signed in to change notification settings - Fork 2
feat(bench): update the llms-benchmark #650
Copy link
Copy link
Closed
Labels
enhancementNew feature or requestNew feature or requestlocalLocal otel-lgtm stackLocal otel-lgtm stackpriority: mediumReal friction, workaround existsReal friction, workaround existsrelease: v1.13.2Shipped in release v1.13.2Shipped in release v1.13.2
Description
Activity
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or requestlocalLocal otel-lgtm stackLocal otel-lgtm stackpriority: mediumReal friction, workaround existsReal friction, workaround existsrelease: v1.13.2Shipped in release v1.13.2Shipped in release v1.13.2
Umbrella issue for a round of updates to the llms-benchmark: its results tables in
.llms-benchmark/README.mdand the/launch-llms-benchmarkcommand that produces them.Planned so far:
medium;/launch-llms-benchmarktakes the effort as an optional third argument (mediumby default), passed to each CLI's effort flag; model, effort and CLI together identify a row.More changes to the benchmark will be added here as the work proceeds; the PR that closes this issue lists them all.