Skip to content

Set up AI models in Settings and test a local model from Studio - #40

Merged
ancongui merged 38 commits into
mainfrom
feat/ai-models-screen
Oct 13, 2026
Merged

ancongui merged 38 commits into
mainfrom
feat/ai-models-screen

Conversation

@ancongui

Copy link
Copy Markdown
Contributor

Summary

Studio gets one place to set up AI: Settings › AI models (/settings/ai).

  • Provider cards for Ollama, OpenAI, Anthropic and Azure OpenAI ("Connected · N" once a connection exists).
  • AI connections with test results from this Studio session; people with only connection.bind see them
    read-only, people with neither see which role they need.
  • Readiness checklist: the Agentic worker, published AI actions, a worker online, an AI connection, its key and
    authorization, a model that answered a test in the last 24 hours, and Weave AI. One-click Publish and
    Authorize for people allowed to run them, Test connection, copyable operator commands and Recheck.
  • Weave AI card: turn Weave AI on, choose its connection and model. It replaces the Weave AI settings inside the
    Ask Weave AI panel, whose Set up Weave AI now opens this card.
  • AI connection side sheet, the only AI connection form. It also opens from Build › Connections › New AI
    connection, an AI task's connection step and Home's Connect a model. It offers plain HTTP only for endpoints a
    platform operator approved, takes a secret handle (never a key) and shows the weave platform secret set --handle
    command, lists the approved models the provider serves (size, tool support, context) with Refresh, and tests a
    model through the AI gateway: running with Cancel, success, success without tool support, failure with details.
  • The Studio host waits up to 140 seconds for a model test (a cold local model can take two minutes) and stops the
    platform call when the browser leaves or the person cancels. Every other call keeps its 35-second limit.
  • weave platform up --username gives its account the local owner roles (developer, deployer, operator, viewer,
    task_participant, tenant_admin, lumi_manager, execution_manager, worker_operator, deployment_reader,
    deployment_planner, deployment_approver, deployment_operator). New weave platform user grant --username NAME --role ROLE adds roles to an account this platform created; it first checks that the sign-in still belongs to the
    recorded person, using a private 0600 record of local accounts.

Removed: the AI setup wizard's AI connection flow, the old AI connection form, and the Weave AI settings wizard in the
Ask Weave AI panel. No platform API, migration or role is added; the platform stays the authority for every
permission, origin and credential rule.

Decisions

  • A connection is saved first and tested second: tests and model lists take a saved revision.
  • No stored default model per connection. Studio suggests qwen3:4b when it is approved and installed, otherwise the
    first approved, installed, tool-capable model. Test results are kept in memory for the Studio session only; the
    readiness checklist is the durable record.
  • On Settings › AI models, a readiness link that points to this page is named by the card it moves to (Go to Weave
    AI
    , Go to providers) and moves focus there.
  • No context-size choice for Weave AI yet; saving keeps the stored profile's options, reasoning and limits.
  • Weave AI can't use a connection without a key yet. The platform refuses to save Weave AI on a keyless
    connection, so the Weave AI card shows keyless connections (such as a local Ollama endpoint) disabled with that
    reason, and on a local platform every readiness item except Weave AI turns ready.
  • Accounts created before this version have no local record, so user grant refuses them and points to Settings ›
    People and access.
  • Endpoints and connections marked "Development only" say so; the private-origin list itself is not shown.

Verification

Mocked and offline (macOS)

  • make check: all 17 stages passed (6584 unit and contract tests, 208 agentic worker tests, 29 file worker tests,
    docs build, installed artifacts). mkdocs build --strict and the docs checker are clean.
  • Studio: tsc and Prettier clean; vitest 1796 passed (129 files); production build 732.98 kB initial (about
    190 kB transfer).
  • Playwright (Chromium headless shell, macOS, no retries, 2 workers), full suite at the PR head: 2203 passed,
    23 skipped, 0 failed (30.7 min). The AI suites (ai-setup, weave-ai, ai-models-layout, settings-routes,
    ai-task-profile) account for 61 of them. An earlier full run had one test race in inspector-ux.spec.ts, fixed in
    this PR.
  • Visual tour at seven sizes with the contrast check, plus the geometric layout spec at 1440x900, 600x500, 360x640 and
    touch. Linux font check in mcr.microsoft.com/playwright:v1.55.1-noble: the AI scenes passed; the Weave AI scene
    needs the Python host, which that container does not have.

Real services (macOS host, Linux containers in Colima)

  • Acceptance harness, ai profile, on a fresh local platform: bring-up passed, including the check that the account
    weave platform up --username creates gets the owner roles.
  • weave platform ai enable --ollama container --model qwen3:4b: ready in about 3 minutes; its own test answered in
    8.5 s (CPU only).
  • Real Studio host, real sign-in: Settings › AI models, Test connection on ollama-local answered in 10.1 s with
    qwen3:4b. After Recheck, every readiness item except Weave AI is ready. No console errors; no credential left after
    removing the platform; the harness secret scan found no canary.

Not verified here

  • Cloud providers (OpenAI, Anthropic, Azure OpenAI) against real accounts.
  • macOS and Windows CI runs and the Linux real-platform run (CI).
  • Weave AI on a local model.

Known gaps

  • Weave AI can't use a connection without a key (local Ollama) yet.
  • The private-origin entries an operator approved are not listed in Studio.
  • Authorize covers missing grants only; a connection whose authorization check fails for another reason needs the
    operator.

Follow-ups

  • Weave AI on a local model: accept keyless connections and a prompt that fits an 8k context.
  • Show the approved private-origin entries (and "Legacy setting") with the Compose operations work.
  • Authorize when the grant readiness check fails for another reason.
  • weave platform ai disable --remove-model-data failed at the api-start stage after the live run (/health/ready
    kept answering 503); the model data was still removed. Reproduce with the installation kept and read the API start
    log. Not caused by this change.

Andres Contreras added 30 commits October 12, 2026 21:34
…th AI setup and size its dropdowns like its buttons
@ancongui
ancongui merged commit a410027 into main Oct 13, 2026
17 checks passed
@ancongui
ancongui deleted the feat/ai-models-screen branch October 13, 2026 15:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant