Skip to content

[COVAL-7144] Reconcile the CLI with the Oct 5 API catalog and release 0.9.0 - #131

Merged
callumreid merged 3 commits into
mainfrom
chore/weekly-api-parity
Oct 5, 2026
Merged

callumreid merged 3 commits into
mainfrom
chore/weekly-api-parity

Conversation

@callumreid

@callumreid callumreid commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Reconciles the CLI with the 2026-10-05 public API catalog (183 operations, 437 request fields) and releases 0.9.0.

⚠️ Merge before Mon 2026-10-12 10:00 UTC (03:00 PT). The weekly parity workflow rebuilds chore/weekly-api-parity from main and force-pushes whenever the report differs. That would drop these hand commits and reset this title and body; it already erased the 9/21 hand commit on 9/28.

Changes

  • New request fields
    • Metric: Agentic judge judge_mode / enabled_tools; SQL and custom-trace aggregation_method / unit / value_source / span_name / metric_attribute; audio presets and thresholds; metric_metadata.
    • Persona advanced settings: silent_mode, multi_phone_config, initialization_parameters, custom_persona_data, voice, custom_voice_id.
    • Review annotation: structured ground truth.
    • Review project: label configuration, and project_type on update.
    • Run template: test_case_ids.
  • Explicit null clears on PATCH. These forward null instead of dropping it:
    • persona advanced settings;
    • run-template test_case_ids (the API rejects [], so null is the only way to remove a test-case subset);
    • metric aggregation_method / unit;
    • review-annotation annotations / ground_truth_json / ground_truth_set_value.
  • Removed: review-annotations create --priority. The create contract no longer accepts priority and the server ignored it, so the flag did nothing. Scripts that pass it now get a clap error. update --priority is unchanged.
  • Removed: the unused persona_metrics launch field. The CLI always sent None, so the wire format doesn't change.
  • Coverage manifest: removes 15 exceptions the spec now covers, including the run-template singular-ID drift from COVAL-5825. Records complete-conversation and sofia/delegation-token as reviewed gaps.
  • Version: 0.8.3 → 0.9.0 (minor, because a flag is removed). Merging triggers Release on version bump.

Validation

  • cargo fmt --check and cargo clippy --all-targets -- -D warnings: clean.
  • cargo test -- --test-threads=2: 9 + 194 passed.
    • In a fully parallel run on a loaded host, test_update_check_notifies_when_outdated can trip its 500 ms budget. That flake predates this PR.
  • New explicit-null tests fail on the previous models (positive control) and pass now.
  • Audit script tests: OK. Live audit: PASS, 119/183 operations, 437/437 request fields. The regenerated api-coverage-report.md is byte-identical to the committed one. release_version.py reports v0.9.0.
  • Live api.coval.dev/v1/openapi is byte-identical to docs origin/main for all 23 specs. Every added field's name and optionality matches the spec.

Follow-ups

  • Docs still list review-annotations create --priority (cli/human-review.mdx, llms-full.txt). Remove it when 0.9.0 ships.
  • The workflow should stop overwriting a rolling branch that has non-bot commits.

Part of COVAL-2079 (the CLI/API lockstep umbrella stays open).

🤖 Generated with Claude Code

@callumreid
callumreid force-pushed the chore/weekly-api-parity branch from a6b2188 to 286e1be Compare September 28, 2026 10:09
@callumreid callumreid changed the title chore(cli): reconcile weekly API parity [COVAL-2079] Reconcile weekly API parity Oct 5, 2026
….9.0

- Run-template test_case_ids, metric aggregation_method and unit, and the
  review-annotation structured ground-truth fields now forward an explicit null,
  which the API uses to clear them. Plain Option dropped the null, so a
  test-case subset could never be removed (the API rejects an empty list).
- Use caller data for the initialization_parameters test value; it is a persona
  setting, not an account attribute.
- Bump to 0.9.0: review-annotations create --priority is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@callumreid callumreid changed the title [COVAL-2079] Reconcile weekly API parity [COVAL-7144] Reconcile the CLI with the Oct 5 API catalog and release 0.9.0 Oct 5, 2026
@callumreid
callumreid requested a review from kdmelon October 5, 2026 19:07

@kdmelon kdmelon left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE — first review.

Severity counts: P0 0, P1 0, P2 0, P3 0.

Validation: inspected the complete current-head diff and request serialization paths; all current-head CI, CodeQL, and cross-platform builds are passing. No existing review threads.

Verification limitations: this is a client contract-parity change; review used the PR's mock-server serialization coverage and current-head CI rather than a live authenticated API round trip.

@callumreid
callumreid merged commit f332304 into main Oct 5, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants