Skip to content

Update R**2 Reasonometry Router with externally learned routes - #233

Open
sashakolpakov wants to merge 1 commit into
RouteWorks:mainfrom
sashakolpakov:submission-r2-reasonometry-v23-20261014
Open

sashakolpakov wants to merge 1 commit into
RouteWorks:mainfrom
sashakolpakov:submission-r2-reasonometry-v23-20261014

Conversation

@sashakolpakov

Copy link
Copy Markdown
Contributor

Summary

This updates the existing R**2: Reasonometry Router entry with a newly executed,
externally learned frozen route vector.

  • RouterArena was used only as the terminal evaluation set.
  • RouterArena prompts, answers, model outcomes, costs, and scores were not used
    to fit the router or select its routes.
  • Every submitted row contains a complete response and token usage returned by
    an actual OpenRouter call.
  • Five unsuccessful transport attempts in the new default-model campaign use
    real, same-model Gemini repair receipts already accepted in the preceding
    v8.4 submission. No response was selected by benchmark correctness.

Current-main private preflight

Metric Value
RouterArena score 0.761311
Accuracy 78.9366%
Total tariff cost $4.066588
Cost per 1,000 queries $0.484118
Queries 8,400
Abnormal entries 0
Robustness 99.7619%

The official RouterArena evaluation is authoritative.

Metering and completeness audit

  • 8,400 unique OpenRouter generation receipts for 8,400 rows
  • 1,013 distinct input-token counts
  • 555 distinct output-token counts
  • 7,041 distinct input/output/total usage triples
  • zero occurrences of the historical (40, 3, 43) tuple
  • most common usage tuple: 7 rows, or 0.0833%
  • prompt-character/input-token correlation: 0.9070
  • zero empty or failed generations
  • zero local endpoint markers
  • zero projected-usage markers

Please run /evaluate on the full and robustness files.

@sashakolpakov

Copy link
Copy Markdown
Contributor Author

/evaluate

@github-actions

Copy link
Copy Markdown

Router Evaluation Results

Router: r2-reasonometry-router
Dataset Split: full

RouterArena Metrics

Metric Value
RouterArena Score 0.7613
Accuracy 78.94%
Total Cost $4.066588
Avg Cost per Query $0.000484
Avg Cost per 1K Queries $0.4841
Number of Queries 8400
Abnormal Entries 0
Robustness Score 0.9976

Evaluation completed by RouterArena automated workflow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant