Skip to content

Divyam router re-submission - #227

Closed
nilam-divyam wants to merge 1 commit into
RouteWorks:mainfrom
Divyam-AI:divyam-router-resubmission
Closed

nilam-divyam wants to merge 1 commit into
RouteWorks:mainfrom
Divyam-AI:divyam-router-resubmission

Conversation

@nilam-divyam

@nilam-divyam nilam-divyam commented Oct 12, 2026 •

Copy link
Copy Markdown

Re-evaluating Divyam router with affiliation Divyam (https://divyam.ai)

  • Updated predictions (divyam-router.json) and robustness results (divyam-router-robustness.json); all entries complete, no null results
  • Config adds deepseek/deepseek-v4-flash to the model pool

Supersedes #226.

Re-evaluation of divyam-router with updated predictions, robustness
results, and config (adds deepseek/deepseek-v4-flash to the model pool).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nilam-divyam

Copy link
Copy Markdown
Author

/evaluate

@github-actions

Copy link
Copy Markdown

Router Evaluation Results

Router: divyam-router
Dataset Split: full

RouterArena Metrics

Metric Value
RouterArena Score 0.7627
Accuracy 78.74%
Total Cost $3.328727
Avg Cost per Query $0.000396
Avg Cost per 1K Queries $0.3963
Number of Queries 8400
Abnormal Entries 0
Robustness Score 0.8952

Optimality Metrics

Metric Value
Opt.Sel (Optimal Selection) 0.0473
Opt.Cost (Cost Efficiency) 0.2207
Opt.Acc (Accuracy vs Optimal) 0.9334

Evaluation completed by RouterArena automated workflow

@nilam-divyam
nilam-divyam marked this pull request as draft October 12, 2026 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant