Skip to content

Context-Aware Model Selection — selector suggest mode + logging #14

Description

@DevVig

Summary
Introduce a lightweight selector that recommends a model per task, with default static mapping preserved. Run in "suggest" mode by default and log rationale and final choice for auditability.

Why

  • Optimize cost/performance without hidden complexity
  • Keep choices reproducible and reviewable

Scope

  • Add selector function and logging; keep static mapping as default
  • Caps for latency/cost; do not introduce multi-turn loops

Tasks

  • Implement a2a/models.py: select_model_with_reasoning(task, context) -> str (temp=0)
  • Default to existing static map; add --selector-suggest flag to allow suggestion display
  • Add logging: docs/timeline/model-selection.jsonl with {task, context_keys, recommended, chosen, rationale, ts}
  • Config caps: max model/latency budgets in .a2dev/perf.yaml
  • Unit tests: default path unaffected; suggest path logs without changing chosen model
  • Update docs: configuration and usage

Acceptance Criteria

  • No behavior change without flags
  • Suggest mode logs recommendations and rationale
  • All selections remain reproducible with static map unless explicitly overridden

Risks & Mitigations

  • Hidden complexity -> keep suggestion-only default; explicit override required to change model
  • Latency -> single call with short context keys, not full corpus

References

  • suggestion.md (Context-Aware Model Selection)
  • a2a/models.py

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions