diff --git a/CHANGELOG.md b/CHANGELOG.md index c3fcba1d..8ad4dfd2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,6 +20,16 @@ SPDX-License-Identifier: Apache-2.0 ## Unreleased +- Studio has one place to set up AI: **Settings › AI models**. It shows provider cards, your AI connections, a + readiness checklist with **Publish**, **Authorize** and **Test connection**, and the Weave AI card. The AI + connection side sheet is the only AI connection form; it also opens from Build › Connections, an AI task and + Home's **Connect a model**. It offers plain HTTP only for endpoints a platform operator approved, asks for a secret + handle (never a key), lists the approved models the provider serves, and tests a model through the AI gateway with + tool support. The old AI setup wizard and the Weave AI settings in the Ask Weave AI panel are gone. +- Studio waits up to 140 seconds for an AI connection test, so a local model loading for the first time no longer + fails the test early; canceling the test in Studio stops it on the platform. +- `weave platform up --username` grants the new account the local owner roles, and + `weave platform user grant --username NAME --role ROLE` adds roles to an account the local platform created. - Run state, history, tasks, deadlines, and incidents name each run of a step by its instance key, which equals the step ID outside loops, so existing workflows keep byte-identical transitions and history. Deadline scans, incident diff --git a/docs/guides/ai-workers.md b/docs/guides/ai-workers.md index 9dc66f86..de556f8f 100644 --- a/docs/guides/ai-workers.md +++ b/docs/guides/ai-workers.md @@ -116,18 +116,21 @@ used, which is separate from changing the grant list. Follow for the supported file and environment providers. Saving a handle in Studio does not provision its secret or grant access to it. -Studio's **New AI connection** uses the standard OpenAI or Anthropic endpoint -automatically. **Advanced: custom endpoint** is available for an approved proxy -or compatible service. Azure requires the resource endpoint and API version -from its administrator. Review shows the exact saved endpoint; hostname case -and the default HTTPS port are normalized, while the base path and trailing -slash are preserved. The worker or Weave AI gateway must still allow that exact -endpoint. Existing endpoint policies are not changed by saving a connection. - -**Saved · Not tested** means that the connection configuration passed validation. -It does not prove provider access, model availability, or worker/gateway readiness. -Workflow AI profiles and Weave AI settings remain separate and select their own model -or Azure deployment using the approved connection. +AI connections are created and tested in Studio's **Settings › AI models**. The +same AI connection side sheet opens from **Build › Connections › New AI +connection** and from an AI task's connection step. Choose a provider and one of +the platform's approved endpoints; a custom address must use `https://`, and plain +`http://` works only for an address a platform operator approved. Cloud providers +take a **Secret handle**, never the key itself, and the sheet shows the +`weave platform secret set --handle HANDLE` command that stores the key. Azure +also needs its **API version**. The worker or Weave AI gateway must still allow +the saved endpoint; saving a connection does not change endpoint policy. + +A saved connection has passed validation only. **Test connection** in the side +sheet sends one request through the AI gateway to the model you choose and says +whether it answered and can call tools; the model list shows the platform's +approved models that the provider serves. Workflow AI profiles and Weave AI +choose their own model or Azure deployment on the approved connection. **Hosting Weave in Azure does not select an AI provider.** The Azure operator still deploys the worker, supplies its configuration and token file, permits its @@ -256,31 +259,26 @@ after an ambiguous network failure. ## Configure the provider connection -In Studio, open **Settings → AI models → New AI connection**. The same action is -available from **Connections** and the AI task inspector. You need -`connection.manage` in the environment. If the provider Connector is not yet -published, the form explains the prerequisite; an author cannot bypass it. - -Studio alpha11 guides this setup through three steps. Alpha10 presents the -same fields in a single form. - -1. **Provider:** enter a **Connection name** that identifies the intended account - or purpose, choose **Provider**, and select **Continue**. The model or Azure - deployment name belongs in the workflow profile. -2. **Access:** copy the exact operator-approved **Provider endpoint**. For Azure, - also enter the explicit **Azure API version**. Enter the **API key secret - handle** supplied by the operator, never the raw API key, then select - **Review connection**. The Azure endpoint hint has no trailing slash; copy - the approved endpoint exactly rather than inventing a different spelling. -3. **Review:** check the name, provider, endpoint, API version when applicable, - and secret handle. **Back** preserves these details so you can correct them. - Select **Create AI connection** only when they are ready. Earlier steps do - not create a connection or send a model request. -4. Record the created connection and revision. Studio restricts allowed - destinations to the endpoint's HTTPS origin, and the operator's worker policy - must also permit the endpoint. The platform checked configuration, not live - provider connectivity. Complete worker release and credential grants before - binding it to an executable workflow. +In Studio, open **Settings › AI models** and choose **Connect** on a provider +card. The same side sheet opens from **Build › Connections › New AI connection** +and from the AI task's connection step. Creating a connection needs +`connection.manage` in the environment; people with only `connection.bind` see +AI connections read-only. + +1. **Provider** and **Endpoint:** pick the provider and one of the approved + endpoints the platform lists, or **Custom address…** for an approved proxy or + compatible service. For OpenAI and Azure OpenAI choose the **API style**; for + Azure also enter the **API version**. +2. **Connection name:** a name that identifies the account or purpose. +3. **Secret handle:** for cloud providers, the handle the operator maps to the + key, never the key itself. Local endpoints use no credential. +4. Select **Save connection**. Studio restricts allowed destinations to the + endpoint's origin, and the operator's worker policy must also permit it. +5. In **Test a model**, choose a model and select **Test connection**. The result + says whether the model answered and whether it can call tools; **Show details** + lists the error code when it did not. Complete the worker release and + credential grants (the **Readiness** checklist's **Publish** and **Authorize**) + before binding the connection to an executable workflow. The API also publishes the canonical provider Connector and the AI task Action for you: `ai_setup.publish` (`definition.publish` at project scope) publishes the diff --git a/docs/guides/docker-development.md b/docs/guides/docker-development.md index 84643035..60694d73 100644 --- a/docs/guides/docker-development.md +++ b/docs/guides/docker-development.md @@ -64,16 +64,15 @@ longer than later starts. The CLI prints the API address and sign-in instruction `platform status` also shows the interactive API docs address. Copy the generated password when it appears: it is shown once, and subsequent starts do not change it. -A default development account can author and run workflows, inspect executions, -and participate in human tasks. To also administer connections and reviewer groups, -choose these roles when creating the account: +Without `--role`, `up` gives this first account the local owner roles in the demo +workspace: `developer`, `deployer`, `operator`, `viewer`, `task_participant`, `tenant_admin`, `lumi_manager`, `execution_manager`, `worker_operator`, `deployment_reader`, `deployment_planner`, `deployment_approver` and `deployment_operator`. That is enough to author, run and operate workflows, +administer connections, and set up AI models and Weave AI in Studio. To manage +reviewer groups, add `task_manager` later; `user grant` adds roles to an account +this platform created and never removes any: ```sh -# Development only: create an account that can follow the administration guides. -# Use this INSTEAD of the first up command, with a new username. -weave platform up --username localadmin --role tenant_admin --role developer \ - --role deployer --role operator --role viewer --role task_participant \ - --role task_manager +# Development only: add a role to the account `up` created. +weave platform user grant --username developer --role task_manager ``` Your installation lives in `.local/platform`. Its containers and volumes have diff --git a/docs/guides/local-platform.md b/docs/guides/local-platform.md index bff4ad4b..8395e34d 100644 --- a/docs/guides/local-platform.md +++ b/docs/guides/local-platform.md @@ -237,9 +237,9 @@ read runs, and work on human tasks: | `viewer` | Demo environment | Read runs, deliveries, status, and the catalog | | `task_participant` | Demo environment | Read, claim, complete, and release assigned human tasks | -**Choose the roles now: you cannot add them later with this command.** An -existing username is never changed, and `--role` replaces the defaults, so list -every role you want. The defaults miss two roles that later guides need: +**`user` never changes an existing username**, and `--role` replaces the +defaults. To add roles to an account this platform created, use `user grant` +(below). The defaults miss two roles that later guides need: - **Integration connections** need `tenant_admin` (step 8 and [Call a REST API without code](../connectors/http-without-code.md)). @@ -257,9 +257,25 @@ weave platform user --username YOUR_NAME --role tenant_admin --role developer \ Expected: the same output, with `tenant_admin (demo tenant)` first in `Roles:`. `weave platform user --help` lists every role you can grant; `platform_admin` and -`worker` are never offered. **Already created your person with fewer roles?** -Create a second username with the full set, then sign in as it with -`weave auth login --switch-account`. +`worker` are never offered. + +**Already created your person with fewer roles?** Add the missing ones with +`user grant`. Grants are added, never removed, and are scoped like the roles +`user` gives; sign in again to use them: + +```sh +# Development only: add roles to an account this platform created. +weave platform user grant --username YOUR_NAME --role tenant_admin --role lumi_manager +``` + +`user grant` works for accounts that `weave platform user` or +`weave platform up --username` created with this version or later, which keep a +private record of the account. For an older account, assign roles in Studio's +**Settings › People and access** instead. + +The account `weave platform up --username NAME` creates (see the +[Docker development guide](docker-development.md)) gets the local owner roles +instead of the defaults: `developer`, `deployer`, `operator`, `viewer`, `task_participant`, `tenant_admin`, `lumi_manager`, `execution_manager`, `worker_operator`, `deployment_reader`, `deployment_planner`, `deployment_approver` and `deployment_operator`. ## 6. Sign in from the CLI diff --git a/docs/guides/ollama.md b/docs/guides/ollama.md index 664f7084..22b96e37 100644 --- a/docs/guides/ollama.md +++ b/docs/guides/ollama.md @@ -97,6 +97,21 @@ On macOS, Docker cannot use the Apple GPU, so `--ollama host` is much faster when Ollama runs on your Mac; `weave platform ai status` warns about CPU-only inference when Ollama runs in a container on macOS. +## Set up models in Studio + +Open **Settings › AI models**. The **Readiness** checklist shows what the platform checked: the Agentic worker, the +published AI actions, a worker online, an AI connection, its key and authorization, a model that answered a test in +the last 24 hours, and Weave AI. Each item you can fix has its button: **Publish** publishes the built-in AI actions, +**Authorize** lets the worker use your credentialed connections, and **Test connection** opens the connection to test. + +Choose **Connect** on the Ollama card (**Manage** after `weave platform ai enable`) to open the AI connection side +sheet. Pick an approved endpoint; plain `http://` addresses work only when a platform operator approved them, and +local endpoints use no credential. **Save connection**, choose a model (Studio suggests `qwen3:4b` when it is +installed) and **Test connection**. A first test of a local model can take up to two minutes while the model loads; +**Cancel** stops it. + +Weave AI can't use a connection without a key yet, so on a local platform every item but **Weave AI** turns ready. + ## Choose a model | Use | Model | Download | diff --git a/docs/guides/people-and-access.md b/docs/guides/people-and-access.md index afcc9d66..7916f445 100644 --- a/docs/guides/people-and-access.md +++ b/docs/guides/people-and-access.md @@ -405,6 +405,19 @@ command, `weave auth setup http://127.0.0.1:API_PORT`. The default roles are different set instead. An existing username is never reset. Never use this command with a shared or production identity provider. +`weave platform up --username NAME` creates its first account with the local +owner roles instead: `developer`, `deployer`, `operator`, `viewer`, `task_participant`, `tenant_admin`, `lumi_manager`, `execution_manager`, `worker_operator`, `deployment_reader`, `deployment_planner`, `deployment_approver` and `deployment_operator`. To add roles later to an account this platform +created, use `user grant`; it adds grants, never removes them, and first checks +that the sign-in still belongs to the recorded person: + +```sh +# Development only: add roles to an account this platform created. +weave platform user grant --username alice --role tenant_admin --role lumi_manager +``` + +Accounts created by earlier versions have no local record, so `user grant` +refuses them; assign their roles in Studio's **Settings › People and access**. + **To practise platform-administrator commands locally,** act as the local host application, which `weave platform setup` bootstrapped as platform administrator. `weave platform user` never offers `platform_admin`. From the diff --git a/docs/guides/studio-step-reference.md b/docs/guides/studio-step-reference.md index 26127397..9902bd6f 100644 --- a/docs/guides/studio-step-reference.md +++ b/docs/guides/studio-step-reference.md @@ -257,7 +257,8 @@ slot without changing the model, select that slot directly in the inspector. not contact the model or verify credentials. 2. **Connection:** choose the **AI connection slot**, or select **Add AI connection slot** to include a new slot when you apply. An authorized administrator can - also open **New AI connection** to create an environment connection using an + also select **New AI connection**, which opens the AI connection side sheet + over the editor, to create an environment connection using an operator-provisioned secret handle. Activation binds the slot to that environment connection; this wizard does not activate the workflow. 3. **Review:** check the profile, provider, model, slot, and list of affected AI @@ -275,7 +276,8 @@ read. See [Share context between AI steps](shared-ai-context.md) for a complete example and the limits across branches, waits, and retries. Provider credentials belong to an authorized connection, not the prompt or -profile. Workflow AI profiles are separate from Studio's Weave AI settings. +profile. Workflow AI profiles are separate from Weave AI's settings in +**Settings › AI models**. The [AI worker guide](ai-workers.md) covers deployment, supported providers, credential binding and execution limits. A configured inspector alone does not install the action or start a worker. diff --git a/docs/guides/studio.md b/docs/guides/studio.md index ad683a16..3d363d24 100644 --- a/docs/guides/studio.md +++ b/docs/guides/studio.md @@ -457,6 +457,14 @@ proves who you are; your administrator's link and grants on the platform decide what you may do. Follow [People and access](people-and-access.md) for that onboarding step. +**Settings has a tab for each kind of setup, each at its own address.** +**Platforms** (`/settings/platforms`) holds your saved platforms; **People and +access** (`/settings/people`) appears for administrators; **AI models** +(`/settings/ai`) appears once a platform is connected and holds provider cards, +your AI connections, the readiness checklist and the Weave AI card (see +[Set up models in Studio](ollama.md#set-up-models-in-studio)); **Preferences** +(`/settings/preferences`) holds Studio's own options. + ### Manage saved platforms **Saved platforms are shared with the `weave` command line on this computer.** diff --git a/docs/guides/weave-ai.md b/docs/guides/weave-ai.md index 4971a8eb..ca9140df 100644 --- a/docs/guides/weave-ai.md +++ b/docs/guides/weave-ai.md @@ -107,9 +107,10 @@ explains the shared operator/administrator handoff. **New conversation** clears the current exchange. Conversations remain in memory and clear when the workspace, identity, or sign-in session changes. They are not -workflow history. Managers can open **Weave AI settings** to configure the enabled -flag, pinned connection revision, and model profile separately from workflow AI -profiles. Studio supplies the fixed reply schema from the canonical contract; +workflow history. Managers configure Weave AI in the **Weave AI** card of +**Settings › AI models**; **Set up Weave AI** in the panel opens it. The card sets +the enabled flag, pinned connection revision, and model separately from workflow +AI profiles. Studio supplies the fixed reply schema from the canonical contract; changing a workflow's AI profile does not configure Weave AI. ## Explain Operations records @@ -209,26 +210,27 @@ separate from the administrator's editable environment Weave AI configuration. ### Save the settings in Studio -1. Open **Settings → AI models → Weave AI settings**, or open **Weave AI settings** from - the assistant. These settings belong to the current environment. -2. The **Model → Connection → Review** wizard keeps changes local until - **Save Weave AI settings**. Choose the provider, an explicit model, and maximum - response length. For Azure, supply the Azure deployment name. Expand - **Advanced model settings** to change generation controls, reasoning, call - limits, or time limits. Studio supplies the fixed reply schema and checks - incompatible settings before you continue. This local validation does not - contact the provider. Select **Continue to connection**. -3. Choose **Provider connection**. The list displays the connection name and - revision and filters for the selected provider. Weave AI pins the exact revision, - so a later connection revision does not silently change the assistant. -4. If needed, select **New AI connection** and follow the - [provider connection steps](ai-workers.md#configure-the-provider-connection). - Use **Refresh connections** to reload available choices. -5. Select **Review settings**, check the model and pinned connection, then select - **Save Weave AI settings**. **Back** preserves your draft. Use a simple request - without attachments to confirm the complete path, as described +1. Open **Settings › AI models** and find the **Weave AI** card, or select + **Set up Weave AI** in the Ask Weave AI panel. These settings belong to the + current environment. Configuring them needs the Weave AI manager role, and + choosing the connection also needs the tenant administrator role. +2. Select **Turn on Weave AI**, choose the **Connection**, and choose the + **Model**; Studio suggests one of the platform's approved models for that + connection. Weave AI pins the connection's exact revision, so a later + revision does not silently change the assistant. When the connection has a + newer revision, the card says so and saving moves Weave AI to it; a newer + revision without a key can't be used, so choose another connection. + Connections without a key (such as a local Ollama endpoint) can't be used for + Weave AI yet and are listed as unavailable. +3. Select **Save Weave AI settings**. Saving keeps the stored profile's + generation options, reasoning, call and time limits; a new configuration + starts from Weave AI's defaults. Studio supplies the fixed reply schema. Use a + simple request without attachments to confirm the complete path, as described [below](#confirm-setup-without-sharing-sensitive-source). +To create the connection first, use the +[provider connection steps](ai-workers.md#configure-the-provider-connection). + A workflow AI profile never fills these fields automatically. Likewise, a saved Weave AI profile does not change a workflow's model or authorize its worker. diff --git a/docs/operations/upgrades.md b/docs/operations/upgrades.md index a6bb9524..65f09ac5 100644 --- a/docs/operations/upgrades.md +++ b/docs/operations/upgrades.md @@ -45,6 +45,7 @@ the desktop app never migrates a platform. | alpha13 | `0030_worker_presence` | No new schema migration. Workers retry explicit capacity rejections while they read task context or credentials, and an Agentic preparation timeout before provider execution counts as not started. Upgrade the Agentic and Files workers to 0.1.5 together with the server. | | alpha14 | `0030_worker_presence` | No new schema migration. Adds the detached Docker development platform (`weave platform up`); existing foreground installations keep working and are never converted. The Agentic and Files workers 0.1.6 pin this server version. | | alpha15 | `0030_worker_presence` | No new schema migration and no role change. Upgrade CLIs and SDKs together with the server, review native executor `capacity` and the legacy private-network settings before restarting, and finish runs that use text templates before any rollback to alpha14; see [what changes in alpha15](#what-changes-in-alpha15). The Agentic and Files workers 0.1.7 pin this server version. | +| Unreleased | `0030_worker_presence` | No schema change. `weave platform up --username` grants the local owner roles (developer, deployer, operator, viewer, task_participant, tenant_admin, lumi_manager, execution_manager, worker_operator, deployment_reader, deployment_planner, deployment_approver, deployment_operator); existing accounts keep their roles until `weave platform user grant` adds more. `weave platform user` is now a command group; `weave platform user --username` works as before. | ![Schema, compatibility and execution acceptance gates](../diagrams/operations-upgrade.svg) diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 88f80714..c2af598a 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -359,13 +359,14 @@ initial steps in one `up` command. | Command | Effect | | --- | --- | | `doctor [--source REPO] [--context NAME]` | Check the source version, uv, and local Docker without changing anything | -| `up [--source REPO] [--context NAME] [--subnet CIDR] [--username NAME] [--role ROLE ...]` | New in alpha14: prepare once and leave the API running in Docker; create the demo workspace and optionally a sign-in account; repeating preserves data and the account | +| `up [--source REPO] [--context NAME] [--subnet CIDR] [--username NAME] [--role ROLE ...]` | New in alpha14: prepare once and leave the API running in Docker; create the demo workspace and optionally a sign-in account, which gets the local owner roles unless `--role` names others; repeating preserves data and the account | | `logs [--lines COUNT]` | Read the owned Docker API logs; default 100 lines, maximum 1,000 | | `setup [--source REPO] [--context NAME] [--subnet CIDR]` | Build and install an isolated server, start its own dependencies, and set up identity; `--subnet` sets an unused private (RFC 1918) IPv4 Docker subnet when the default pools are exhausted | | `start` | Resume the saved mode: detached API for a Docker installation, foreground API for a host installation | | `status` | Show the saved stage, API and identity readiness, URLs, whether a first run is saved, and the sign-in command | | `demo` | Create one authorized demo run; repeating it reuses the saved receipt | | `user --username NAME [--role ROLE ...]` | New in 0.1.0a7: create a development sign-in account, a linked person, and roles in the demo workspace | +| `user grant --username NAME --role ROLE [--role ROLE ...]` | Add roles in the demo workspace to an account this platform's `user` or `up` created; grants are added, never removed | | `integrations enable` | New in 0.1.0a7: publish the installed `weave-http@2.0.0` Connector, register this runtime as its local release, and grant a dedicated native principal | | `integrations disable` | New in 0.1.0a7: stop running connector actions locally after the next restart; works without a running API | | `integrations grant --connection REVISION_ID --access ACCESS` | New in 0.1.0a7: let the local release use one connection's secret handles; `ACCESS` is `read` or `write`, repeat `--access` for both | diff --git a/src/firefly_weave/cli/platform.py b/src/firefly_weave/cli/platform.py index 22fcbbbd..493c1c08 100644 --- a/src/firefly_weave/cli/platform.py +++ b/src/firefly_weave/cli/platform.py @@ -23,7 +23,7 @@ import sys from collections.abc import Callable from pathlib import Path -from typing import Any +from typing import Any, NoReturn import click @@ -216,6 +216,10 @@ def level(scope: dict[str, str]) -> str: click.echo(f"Username: {value['username']}") click.echo(f"Password: {value['password']}") click.echo(value["warning"] + " Copy it now; it cannot be shown again.") + if value.get("hint"): + click.echo(value["hint"]) + if value.get("receipt_hint"): + click.echo(value["receipt_hint"]) click.echo("Roles: " + ", ".join(f"{grant['role']} (demo {level(grant['scope'])})" for grant in value["grants"])) click.echo("Next, connect to this platform and sign in with that account:") click.echo(" " + value["next"][0]) @@ -223,10 +227,26 @@ def level(scope: dict[str, str]) -> str: click.echo(" " + value["next"][1]) -@platform.command() +def _usage(message: str, output: str) -> NoReturn: + # The CLI hides click's own usage messages because they may echo option values; this fixed text has none. + emit_result(error_result("WV-CLI-USAGE", message), output) + raise SystemExit(EXIT_USAGE) + + +def _show_grants(value: dict[str, Any]) -> None: + def level(scope: dict[str, str]) -> str: + return "environment" if "environment_id" in scope else "project" if "project_id" in scope else "tenant" + + click.echo( + f"Granted to {value['username']}: " + + ", ".join(f"{item['role']} (demo {level(item['scope'])})" for item in value["grants"]) + ) + click.echo("Sign in again in Studio or the CLI to use the new roles.") + + +@platform.group(invoke_without_command=True) @click.option( "--username", - required=True, help="New sign-in name: 3 to 64 lowercase letters and digits, optionally joined by '.', '_', or '-'.", ) @click.option( @@ -239,8 +259,8 @@ def level(scope: dict[str, str]) -> str: "One of: " + ", ".join(lifecycle.PERSON_ROLES) + ".", ) @click.option("--output", type=click.Choice(["text", "json"]), default="text") -@click.pass_obj -def user(directory: Path, username: str, roles: tuple[str, ...], output: str) -> None: +@click.pass_context +def user(context: click.Context, username: str | None, roles: tuple[str, ...], output: str) -> None: """Create a person who can sign in to this local platform (development only). Creates an account in the owned Keycloak with a generated password, a @@ -251,8 +271,45 @@ def user(directory: Path, username: str, roles: tuple[str, ...], output: str) -> Default roles are the smallest set to author, publish, activate, run, inspect runs, and work on human tasks: developer on the demo project, and deployer, operator, viewer, and task_participant on the demo environment. + Use `user grant` to add roles to an account later. + """ + if context.invoked_subcommand is not None: + if username is not None or roles: + _usage( + "Put --username and --role after the subcommand: " + "weave platform user grant --username NAME --role ROLE.", + output, + ) + return + if username is None: + _usage("Missing option '--username'. Use --help for supported arguments.", output) + directory: Path = context.obj + name: str = username + _call(lambda: lifecycle.user(directory, name, roles), output, _show_person) + + +@user.command("grant") +@click.option("--username", required=True, help="An account this platform created with `user` or `up`.") +@click.option( + "--role", + "roles", + multiple=True, + required=True, + type=click.Choice(lifecycle.PERSON_ROLES), + metavar="ROLE", + help="Role to add in the demo workspace; repeat for several roles. One of: " + + ", ".join(lifecycle.PERSON_ROLES) + + ".", +) +@click.option("--output", type=click.Choice(["text", "json"]), default="text") +@click.pass_obj +def user_grant(directory: Path, username: str, roles: tuple[str, ...], output: str) -> None: + """Add roles in the demo workspace to an existing local account (development only). + + Grants are added, never removed, and are scoped like the roles `user` gives. + The account must have been created by this platform's `user` or `up`. """ - _call(lambda: lifecycle.user(directory, username, roles), output, _show_person) + _call(lambda: lifecycle.grant(directory, username, roles), output, _show_grants) @platform.group() @@ -656,7 +713,8 @@ def ai_models_remove(directory: Path, provider: str, model: str, output: str) -> "roles", multiple=True, type=click.Choice(lifecycle.PERSON_ROLES), - help="Role for the initial account; repeat for several roles. Requires --username.", + help="Role for the initial account; repeat for several roles. Requires --username. Without --role the " + "account gets the local owner roles: " + ", ".join(lifecycle.LOCAL_OWNER_ROLES) + ".", ) @click.option( "--allow-private-origin", diff --git a/src/firefly_weave/sdk/platform.py b/src/firefly_weave/sdk/platform.py index e96ecf7e..5344aa7b 100644 --- a/src/firefly_weave/sdk/platform.py +++ b/src/firefly_weave/sdk/platform.py @@ -107,6 +107,26 @@ "email_manager", "execution_manager", ) +# The account `up --username` creates gets every role a local owner needs to set up the demo +# workspace from Studio: AI connections, Weave AI, executions, workers and deployments. Other +# accounts keep the defaults above; `user grant` adds roles to any account this platform created. +LOCAL_OWNER_ROLES = ( + "developer", + "deployer", + "operator", + "viewer", + "task_participant", + "tenant_admin", + "lumi_manager", + "execution_manager", + "worker_operator", + "deployment_reader", + "deployment_planner", + "deployment_approver", + "deployment_operator", +) +_PEOPLE = "people.json" +_PEOPLE_FORMAT = "weave/local-people-v1" class PlatformError(ValueError): @@ -363,16 +383,22 @@ def _load(directory: Path, *, complete: bool = True) -> dict[str, Any]: @contextmanager -def _exclusive(directory: Path, name: str, busy: str) -> Iterator[None]: +def _exclusive(directory: Path, name: str, busy: str, *, wait: float = 0.0) -> Iterator[None]: + """Hold a private lock file; with ``wait``, retry for that many seconds before giving up.""" import fcntl directory = _private_directory(directory) descriptor = os.open(directory / name, os.O_RDWR | os.O_CREAT | os.O_NOFOLLOW, 0o600) try: - try: - fcntl.flock(descriptor, fcntl.LOCK_EX | fcntl.LOCK_NB) - except BlockingIOError: - raise PlatformError(busy) from None + deadline = time.monotonic() + wait + while True: + try: + fcntl.flock(descriptor, fcntl.LOCK_EX | fcntl.LOCK_NB) + break + except BlockingIOError: + if time.monotonic() >= deadline: + raise PlatformError(busy) from None + time.sleep(0.05) yield finally: os.close(descriptor) @@ -1514,8 +1540,8 @@ def _check_username(username: str) -> None: ) -def _person_roles(roles: Sequence[str]) -> tuple[str, ...]: - selected = tuple(roles) or DEFAULT_PERSON_ROLES +def _person_roles(roles: Sequence[str], default: Sequence[str] = DEFAULT_PERSON_ROLES) -> tuple[str, ...]: + selected = tuple(roles) or tuple(default) if len(set(selected)) != len(selected) or any(role not in PERSON_ROLES for role in selected): raise PlatformError("Choose each role at most once from: " + ", ".join(PERSON_ROLES) + ".") return selected @@ -1533,6 +1559,47 @@ def _person_grants(roles: tuple[str, ...], scope: dict[str, str]) -> list[dict[s ] +_UNREADABLE_PEOPLE = "The private record of local accounts is not readable; it was not changed." +# How long a user command waits for another one that is writing the record. +_PEOPLE_LOCK_SECONDS = 10.0 +_NOT_RECORDED = ( + "This account was not recorded for weave platform user grant; " + "assign more roles in Studio, Settings › People and access." +) + + +_RECEIPT_NOT_SAVED = ( + "The account was created, but up-user.json still marks it as attempted. " + "Sign in with it; run up without --username from now on." +) + + +def _people(directory: Path) -> dict[str, Any]: + path = directory / _PEOPLE + if not path.exists(): + return {"format": _PEOPLE_FORMAT, "people": {}} + try: + value = strict_json(read_file(path, 1048576, private=True)) + except (DeploymentError, ValueError): + raise PlatformError(_UNREADABLE_PEOPLE) from None + if ( + not isinstance(value, dict) + or value.get("format") != _PEOPLE_FORMAT + or not isinstance(value.get("people"), dict) + ): + raise PlatformError(_UNREADABLE_PEOPLE) + return value + + +def _remember_person(directory: Path, username: str, principal: str, subject: str) -> None: + """Record which Weave person a local sign-in belongs to, so later role grants find it.""" + with _exclusive(directory, ".people.lock", "Another platform user command is active.", wait=_PEOPLE_LOCK_SECONDS): + path = directory / _PEOPLE + value = _people(directory) + value["people"][username] = {"principal_id": principal, "subject": subject} + _write(path, value, replace=path.exists()) + + def _exists(username: str) -> str: return ( f"A local sign-in account named {username} already exists. " @@ -1771,11 +1838,18 @@ def user( raise PlatformError("Private identity configuration is incomplete; retain it for recovery.") _token(state) host_token = strict_json(read_file(directory / "host-token.json", 65536, private=True))["access_token"] + # Read the record before creating anything: once the account exists, its password must reach the caller. + _people(directory) password = secrets.token_urlsafe(24) # Imported on use: offline commands start without event-loop, TLS, or HTTP modules. import asyncio principal, subject = asyncio.run(_enroll(state, username, password, grants, host_token, admin_secret, transport)) + try: + _remember_person(directory, username, principal, subject) + recorded = True + except Exception: # noqa: BLE001 - the account exists; never lose its shown-once password + recorded = False return { "ok": True, "username": username, @@ -1789,6 +1863,111 @@ def user( "grants": grants, "api_url": api, "next": ["weave auth setup " + api, "weave studio"], + "recorded": recorded, + **({} if recorded else {"hint": _NOT_RECORDED}), + } + + +async def _grant_roles( + state: dict[str, Any], + username: str, + person: dict[str, Any], + grants: list[dict[str, Any]], + host_token: str, + admin_secret: str, + transport: httpx.AsyncBaseTransport | None, +) -> None: + import httpx + + from firefly_weave.contracts.access import Scope + from firefly_weave.contracts.members import MemberGrantRequest, PrincipalIdentityRequest + from firefly_weave.sdk.client import WeaveClient + from firefly_weave.sdk.errors import WeaveError + from firefly_weave.sdk.members import MembersClient + + keycloak = f"http://localhost:{state['ports']['keycloak']}" + users = keycloak + "/admin/realms/weave/users" + tenant = Scope(tenant_id=UUID(grants[0]["scope"]["tenant_id"])) + async with ( + httpx.AsyncClient(timeout=10, trust_env=False, follow_redirects=False, transport=transport) as identity, + WeaveClient(_summary(state)["api_url"], lambda: host_token, tenant, timeout=15, transport=transport) as client, + ): + admin = await _keycloak_admin(identity, keycloak, admin_secret) + if await _keycloak_user(identity, users, admin, username) != person.get("subject"): + raise PlatformError( + f"The local sign-in account {username} is not the one this platform recorded; no role was granted." + ) + members = MembersClient(client) + principal = UUID(str(person["principal_id"])) + try: + # Linking the recorded sign-in again changes nothing, and is refused if it belongs to another person. + await members.link_identity( + principal, + PrincipalIdentityRequest( + provider_id=_PROVIDER_ID, issuer=keycloak + "/realms/weave", subject=str(person["subject"]) + ), + ) + for item in grants: + scope = { + key: UUID(item["scope"][key]) for key in ("project_id", "environment_id") if key in item["scope"] + } + await members.grant( + MemberGrantRequest.model_validate({"principal_id": principal, "role": item["role"], **scope}) + ) + except WeaveError as error: + raise PlatformError( + f"The local API refused the role grant ({error.code}). Roles granted before the refusal stay in place." + ) from None + + +def grant( + directory: Path, + username: str, + roles: Sequence[str], + *, + transport: httpx.AsyncBaseTransport | None = None, +) -> dict[str, Any]: + """Grant more roles in the demo workspace to a person that `user` or `up` created (development only).""" + _check_username(username) + if not roles: + raise PlatformError("Choose at least one role with --role.") + selected = _person_roles(roles) + if not (directory / "platform.json").exists(): + raise PlatformError("No local platform exists here yet. Create it first: " + _command(directory, "setup")) + state = _load(directory) + if not (directory / "first-run.json").exists(): + raise PlatformError("Create the demo workspace first: " + _command(directory, "demo")) + person = _people(directory)["people"].get(username) + if not isinstance(person, dict): + raise PlatformError( + f"This platform has no record of {username}. Accounts created before this version are not recorded: " + "assign roles in Studio under Settings › People and access, or create a new account with " + + _command(directory, "user --username NAME") + ) + grants = _person_grants(selected, _demo_receipt(directory)["scope"]) + try: + import httpx # noqa: F401 + except ModuleNotFoundError: + raise PlatformError("Granting roles requires the client extra: install firefly-weave[client].") from None + _check_engine(state) + _verify_runtime(state) + api = _summary(state)["api_url"] + if not _probe(api + "/health/ready"): + raise PlatformError("Start the API in another terminal (" + _command(directory, "start") + "), then retry.") + admin_secret = _env_file(directory / "identity.env").get("WEAVE_KC_ADMIN_SECRET") + if not admin_secret: + raise PlatformError("Private identity configuration is incomplete; retain it for recovery.") + _token(state) + host_token = strict_json(read_file(directory / "host-token.json", 65536, private=True))["access_token"] + import asyncio + + asyncio.run(_grant_roles(state, username, person, grants, host_token, admin_secret, transport)) + return { + "ok": True, + "username": username, + "principal_id": str(person["principal_id"]), + "grants": grants, + "development_only": True, } @@ -1808,7 +1987,7 @@ def up( if roles and username is None: raise PlatformError("Account roles require --username.") - selected = _person_roles(roles) + selected = _person_roles(roles, LOCAL_OWNER_ROLES) if username is not None: _check_username(username) origins = platform_origins.requested(private_origins) @@ -1845,9 +2024,14 @@ def up( ) result["account"] = {"username": username, "existing": True, "roles": saved_roles} else: + # An unreadable record of people fails before the attempt is marked, so a retry stays possible. + _people(directory) _write(path, {"stage": "attempted", "username": username, "roles": list(selected)}) account = user(directory, username, selected) - _write(path, {"stage": "ready", "username": username, "roles": list(selected)}, replace=True) + try: + _write(path, {"stage": "ready", "username": username, "roles": list(selected)}, replace=True) + except Exception: # noqa: BLE001 - the account exists; never lose its shown-once password + account = {**account, "receipt_saved": False, "receipt_hint": _RECEIPT_NOT_SAVED} result["account"] = account return result diff --git a/src/firefly_weave/studio/service.py b/src/firefly_weave/studio/service.py index 0116a265..3f860866 100644 --- a/src/firefly_weave/studio/service.py +++ b/src/firefly_weave/studio/service.py @@ -23,9 +23,10 @@ import re import secrets import time -from collections.abc import Callable +from collections.abc import Callable, Mapping from dataclasses import dataclass, field from pathlib import Path +from types import MappingProxyType from typing import TYPE_CHECKING, Any from urllib.parse import urlsplit from uuid import UUID @@ -39,6 +40,7 @@ from firefly_weave import __version__ from firefly_weave.contracts.connector_descriptors import ADAPTER from firefly_weave.contracts.surface import OPERATIONS +from firefly_weave.operations.ephemeral import until_disconnect from firefly_weave.sdk.auth import AuthError from firefly_weave.sdk.credentials import CredentialError @@ -49,6 +51,13 @@ BODY_LIMIT = 2 * 1024 * 1024 RESPONSE_LIMIT = 8 * 1024 * 1024 COOKIE = "weave_studio_session" +# Seconds the bridge waits for the platform. A model test calls the model itself, and a local model +# that loads for the first time can take two minutes; the platform stops that test after 130 seconds, +# so the bridge waits a little longer for its answer and cancels it if the browser leaves first. +BRIDGE_SECONDS = 35.0 +LONG_OPERATIONS: Mapping[str, float] = MappingProxyType({"ai_connections.test": 140.0}) +# The status a bridge answer gets when its browser left before it was ready (nothing is sent). +CLIENT_GONE = 499 STUDIO_FAMILIES = frozenset( { "identity", @@ -252,6 +261,10 @@ def pair(self, code: str) -> Response: return response def check_scope(self, path: str, method: str) -> int: + return self.match(path, method)[0] + + def match(self, path: str, method: str) -> tuple[int, str | None]: + """The bridge status for a path and the operation it names; None when no operation matches.""" profile = self.options.profile assert profile is not None for operation in OPERATIONS.values(): @@ -273,13 +286,13 @@ def check_scope(self, path: str, method: str) -> int: ("environment", profile.environment_id), ): if name in values and (selected is None or values[name] != str(selected)): - return 403 + return 403, operation.id # Variable segments are resource IDs, the finite catalog kind or a bounded adapter # name. A segment that fits another operation's template keeps matching (for example # /connector-descriptors also fits the {collection} template). if all(self.segment(name, value) for name, value in values.items()): - return 200 - return 404 + return 200, operation.id + return 404, None @staticmethod def segment(name: str, value: str) -> bool: @@ -300,7 +313,7 @@ async def bridge(self, request: Request, path: str) -> Response: return problem(409, "WV-STUDIO-OFFLINE", "Connect to a platform in Settings to use connected operations") if any(part in path for part in ("%", "\\", "?", "#", "..")): return problem(404, "WV-STUDIO-ROUTE", "Unknown platform operation") - status = self.check_scope(path, request.method) + status, operation = self.match(path, request.method) if status != 200: return problem(status, "WV-STUDIO-ROUTE", "Operation is outside the selected profile or API surface") try: @@ -329,44 +342,62 @@ async def bridge(self, request: Request, path: str) -> Response: headers[name] = request.headers[name] # Do not inherit upstream cookie authentication across browser operations. self.client.cookies.clear() - async with asyncio.timeout(35): - async with self.client.stream( - request.method, - profile.base_url.rstrip("/") + path, - params=request.query_params, - headers=headers, - content=await request.body(), - ) as upstream: - if upstream.is_redirect: - return problem(502, "WV-STUDIO-REDIRECT", "The platform returned an unexpected redirect") - if upstream.headers.get("content-encoding", "identity").strip().lower() != "identity": - return problem(502, "WV-STUDIO-ENCODING", "Platform response must use identity encoding") - body = bytearray() - async for chunk in upstream.aiter_bytes(): - body.extend(chunk) - if len(body) > RESPONSE_LIMIT: + long = LONG_OPERATIONS.get(operation or "") + seconds = long if long is not None else BRIDGE_SECONDS + content = await request.body() + + async def forward() -> Response: + async with asyncio.timeout(seconds): + async with self.client.stream( + request.method, + profile.base_url.rstrip("/") + path, + params=request.query_params, + headers=headers, + content=content, + timeout=seconds if long is not None else httpx.USE_CLIENT_DEFAULT, + ) as upstream: + if upstream.is_redirect: + return problem(502, "WV-STUDIO-REDIRECT", "The platform returned an unexpected redirect") + if upstream.headers.get("content-encoding", "identity").strip().lower() != "identity": + return problem(502, "WV-STUDIO-ENCODING", "Platform response must use identity encoding") + body = bytearray() + async for chunk in upstream.aiter_bytes(): + body.extend(chunk) + if len(body) > RESPONSE_LIMIT: + return problem( + 502, + "WV-STUDIO-RESPONSE-LIMIT", + "Platform response exceeds Studio's limit; narrow the query", + ) + if ( + self.connection_generation != generation + or self.options.profile is not profile + or self.options.token_provider is not provider + ): return problem( - 502, - "WV-STUDIO-RESPONSE-LIMIT", - "Platform response exceeds Studio's limit; narrow the query", + 503, + "WV-STUDIO-CONNECTION-CHANGED", + "Connection changed; discard the previous result. " + "A submitted command may have completed in its original scope", ) - if ( - self.connection_generation != generation - or self.options.profile is not profile - or self.options.token_provider is not provider - ): - return problem( - 503, - "WV-STUDIO-CONNECTION-CHANGED", - "Connection changed; discard the previous result. " - "A submitted command may have completed in its original scope", - ) - safe_headers = { - key: value - for key, value in upstream.headers.items() - if key.lower() in {"content-type", "etag", "retry-after"} - } - return Response(bytes(body), status_code=upstream.status_code, headers=safe_headers) + safe_headers = { + key: value + for key, value in upstream.headers.items() + if key.lower() in {"content-type", "etag", "retry-after"} + } + return Response(bytes(body), status_code=upstream.status_code, headers=safe_headers) + + if long is not None: + # A person can cancel a long model test; the platform call stops when the browser leaves. + try: + return await until_disconnect(forward(), request.receive) + except asyncio.CancelledError: + task = asyncio.current_task() + if task is not None and task.cancelling(): + raise + # The browser left: nobody reads this answer, and the server logs no traceback for it. + return Response(status_code=CLIENT_GONE) + return await forward() except (AuthError, CredentialError): return problem(401, "WV-AUTH-REQUIRED", "Sign in again with weave auth login using the selected profile") except (httpx.HTTPError, TimeoutError): diff --git a/studio/src/app/api.ts b/studio/src/app/api.ts index 93558e93..97593711 100644 --- a/studio/src/app/api.ts +++ b/studio/src/app/api.ts @@ -88,6 +88,33 @@ const replayable = (status: number, code: string, retryAfter: string | null) => (status === 503 && code === "WV-COMPATIBILITY" && !(Number(retryAfter) > longestPause)); +/** + * A signal that aborts when `signal` does, or after `timeout` milliseconds. + * `dispose` drops the timer and the listener once the request has its answer. + */ +function cancellable( + signal: AbortSignal, + timeout: number, +): { signal: AbortSignal; dispose: () => void } { + const controller = new AbortController(); + const follow = () => controller.abort(signal.reason); + const timer = setTimeout( + () => + controller.abort( + new DOMException("The request took too long.", "TimeoutError"), + ), + timeout, + ); + const dispose = () => { + clearTimeout(timer); + signal.removeEventListener("abort", follow); + }; + controller.signal.addEventListener("abort", dispose, { once: true }); + if (signal.aborted) follow(); + else signal.addEventListener("abort", follow, { once: true }); + return { signal: controller.signal, dispose }; +} + export class StudioApi { session: Session = { paired: false, @@ -100,9 +127,26 @@ export class StudioApi { /** Pauses before replaying a request the host or platform turned away for capacity. */ wait = (milliseconds: number) => new Promise((resolve) => setTimeout(resolve, milliseconds)); + /** A pause that a cancel ends at once, so a canceled request is never sent again. */ + private async pause(milliseconds: number, signal?: AbortSignal) { + if (!signal) return this.wait(milliseconds); + signal.throwIfAborted(); + let stop = () => {}; + const canceled = new Promise((_resolve, reject) => { + stop = () => reject(signal.reason); + signal.addEventListener("abort", stop, { once: true }); + }); + try { + await Promise.race([this.wait(milliseconds), canceled]); + } finally { + signal.removeEventListener("abort", stop); + } + } /** * Calls the paired local host. `timeout` (milliseconds) bounds the wait; * platform checks that reach a remote server or identity provider pass more. + * `signal` lets the caller cancel (a model test); a canceled request is never + * sent again. */ async request( path: string, @@ -110,6 +154,7 @@ export class StudioApi { body?: unknown, headers: Record = {}, timeout = 15000, + signal?: AbortSignal, ): Promise { if (!path.startsWith("/studio/") || path.startsWith("//")) throw Error("Studio requests must use the same-origin host boundary."); @@ -121,16 +166,24 @@ export class StudioApi { // request, so a read, or a change that carries an idempotency key, can be sent again. const safe = method === "GET" || "Idempotency-Key" in h; for (let attempt = 1; ; attempt++) { - const response = await fetch(path, { - method, - credentials: "same-origin", - headers: h, - body: body === undefined ? undefined : JSON.stringify(body), - signal: AbortSignal.timeout(timeout), - }); - const value: unknown = await response - .json() - .catch(() => ({ message: `HTTP ${response.status}` })); + const scoped = signal ? cancellable(signal, timeout) : null; + let response: Response; + let value: unknown; + try { + response = await fetch(path, { + method, + credentials: "same-origin", + headers: h, + body: body === undefined ? undefined : JSON.stringify(body), + signal: scoped?.signal ?? AbortSignal.timeout(timeout), + }); + const answer = response; + value = await answer + .json() + .catch(() => ({ message: `HTTP ${answer.status}` })); + } finally { + scoped?.dispose(); + } if (response.ok) return value as T; const error = new ApiError(response.status, value); if ( @@ -142,7 +195,10 @@ export class StudioApi { response.headers.get("Retry-After"), ) ) { - await this.wait(retryDelay(response.headers.get("Retry-After"))); + await this.pause( + retryDelay(response.headers.get("Retry-After")), + signal, + ); continue; } if ( diff --git a/studio/src/app/app.html b/studio/src/app/app.html index b81ac1cf..3f2fdff9 100644 --- a/studio/src/app/app.html +++ b/studio/src/app/app.html @@ -359,6 +359,7 @@

Your workflows, woven together.

[principal]="identity?.principal_id ?? ''" [loading]="loading" [signInToRead]="signInToRead" + [canConnectModel]="can('connection.manage')" [workspaceName]=" profile ? profile.name + @@ -368,6 +369,7 @@

Your workflows, woven together.

(createWorkflow)="newWorkflow()" (importWorkflow)="importWorkflow($event)" (connect)="openWizard('server')" + (connectModel)="openAiModels()" (navigate)="navigate($event)" (openLocal)="openLocalDraft($event)" (continueEditing)="continueEditing()" @@ -622,6 +624,18 @@

Server response

@defer (when lumiOpen) { } + @if (aiSheet && profile) { + @defer (on immediate) { + + } + } diff --git a/studio/src/app/app.ts b/studio/src/app/app.ts index d3b212df..dc82c811 100644 --- a/studio/src/app/app.ts +++ b/studio/src/app/app.ts @@ -151,6 +151,11 @@ import { DesignerView } from "./designer/designer-view"; // Integration and template UI loads lazily (@defer): only types are imported // from these modules besides the component classes listed in `imports`. import { IntegrationDialogs } from "./integrations/integration-dialogs"; +import { + AiConnectionSheet, + type AiSheetRequest, + type AiTestRecord, +} from "./integrations/ai-connection-sheet"; import { TemplateGallery } from "./templates/template-gallery"; import { sheetWhen } from "./modal-sheet"; import { ToastHost, ToastService, type ToastAction } from "./toast"; @@ -356,6 +361,7 @@ const sideEffects: Record = { StepPicker, StartRunDialog, IntegrationDialogs, + AiConnectionSheet, TemplateGallery, ToastHost, DesignerView, @@ -410,7 +416,6 @@ export class App implements CanvasHost { ? (this.operationsView()?.lumiAttachments ?? []) : []; } - lumiSettingsRequested = false; platformMenuOpen = false; platformBusy = ""; // Fences platform checks and status reads against later platform changes. @@ -1082,7 +1087,6 @@ export class App implements CanvasHost { } | null = null; /** The open "New API connection" dialog and its prefill. */ connectionDialog: { - kind?: "ai"; fromBuilder: HttpActionUse["connection"] | null; fromWorkflow?: boolean; } | null = null; @@ -4766,10 +4770,35 @@ export class App implements CanvasHost { }; this.cdr.markForCheck(); } - /** Opens "Connect to an API", prefilled from the builder's hand-off for this slot. */ - openAiConnectionDialog() { - this.connectionDialog = { kind: "ai", fromBuilder: null }; + /** The AI connection side sheet, open over any view. */ + aiSheet: AiSheetRequest | null = null; + /** Grows whenever an AI connection or its test changes; AI screens reload on it. */ + aiSetupGeneration = 0; + /** Test results of this Studio session, by connection revision; never stored. */ + readonly aiTests = new Map(); + /** Opens the AI connection side sheet. */ + openAiConnectionDialog(request: AiSheetRequest = {}) { + this.aiSheet = request; + this.cdr.markForCheck(); + } + aiConnectionSaved() { + this.aiSetupGeneration++; + if (this.view === "connections") void this.refresh(); } + aiConnectionTested(record: AiTestRecord) { + this.aiTests.set(record.revision, record); + this.aiSetupGeneration++; + } + /** Settings › AI models, from Weave AI, Home or a connection. */ + async openAiModels() { + this.lumiOpen = false; + await this.navigate("settings"); + // "Keep editing" (or a step that can't be left yet) stays in the designer, + // at its own address. + if (this.view !== "settings") return; + await this.router.navigateByUrl("/settings/ai", { replaceUrl: true }); + } + /** Opens "Connect to an API", prefilled from the builder's hand-off for this slot. */ openConnectionDialog(slot = "") { const use = this.lastUse; this.connectionDialog = { @@ -7038,6 +7067,9 @@ export class App implements CanvasHost { } /** Forgets everything read from the platform; the open workflow is kept. */ private clearPlatformState() { + this.aiSheet = null; + this.aiTests.clear(); + this.aiSetupGeneration++; this.stepDataGeneration++; this.importGeneration++; this.listSequence++; diff --git a/studio/src/app/dialog.ts b/studio/src/app/dialog.ts index 87491d98..0a493fa5 100644 --- a/studio/src/app/dialog.ts +++ b/studio/src/app/dialog.ts @@ -66,16 +66,22 @@ export class Autofocus implements AfterViewInit { selector: "weave-modal", standalone: true, imports: [Icon], - template: `
+ template: `
+

+ Then run weave platform start. On a shared platform, + ask a platform operator to grant this secret handle. +

+ @if (problem("handle"); as message) { + + } +
+ } + + @if (saved) { +
+

Test a model

+

+ Choose a model to test. AI steps and Weave AI choose their own + model. +

+ @if (draft.card === "azure") { +
+ + +

+ Use the deployment name, not the model family. +

+
+ } @else { +
+ + @if (canManage()) { + + } +
+ @if (models && !models.models.length) { +

No models on this server yet. Pull one:

+
+ ollama pull qwen3:4b + +
+ } + @if (modelsNote) { +

{{ modelsNote }}

+ } + } + @if (canManage()) { +
+ @if (testing) { + + } + + {{ view.text }} + @if (testing) { + + } +
+ @if (view.details.length) { +
+ Show details +
+ @for (detail of view.details; track detail[0]) { +
{{ detail[0] }}
+
{{ detail[1] }}
+ } +
+
+ } + } +
+ } + @if (problem("form") || error; as message) { + + } +
+ @if (canManage()) { + + + } @else { + + } +
+

{{ announcement }}

+ + `, +}) +export class AiConnectionSheet implements OnInit, OnDestroy { + api = input.required(); + canManage = input.required(); + request = input.required(); + connectionSaved = output(); + tested = output(); + closed = output(); + readonly cards = PROVIDER_CARDS; + private readonly dialogs = inject(DialogService); + private readonly cdr = inject(ChangeDetectorRef); + private readonly element = inject>(ElementRef); + private readonly injector = inject(Injector); + private setup!: AiSetupApi; + policy: "loaded" | "absent" | null = null; + endpoints: AiEndpoint[] = []; + connectorId = ""; + draft: SheetDraft = draftFor("ollama", []); + /** The saved revision this sheet shows, or null before the first save. */ + saved: AiConnection | null = null; + models: AiModels | null = null; + model = ""; + modelsNote = ""; + test: TestState = { kind: "idle" }; + loading = true; + busy = false; + error = ""; + errorCode = ""; + announcement = ""; + problems: SheetProblem[] = []; + /** The draft as saved (or as opened); differences are unsaved changes. */ + private opened = ""; + private controller: AbortController | null = null; + private attempt: { body: ConnectionRequestBody; key: string } | null = null; + private alive = true; + + heading() { + return ( + this.saved?.name ?? this.request().connection?.name ?? "New AI connection" + ); + } + get presets() { + return presetsFor(this.draft.card, this.endpoints); + } + get styles() { + return API_STYLES[this.draft.card === "azure" ? "azure" : "openai"]; + } + get customAllowed() { + return ( + this.draft.card === "ollama" || + this.policy === "absent" || + !this.presets.length + ); + } + get noKey() { + return keyless(this.draft, this.endpoints); + } + get secretCommand() { + return secretSetCommand(this.draft.handle); + } + get changed() { + return draftKey(this.draft) !== this.opened; + } + get dirty() { + return !this.saved || this.changed; + } + get testing() { + return this.test.kind === "running"; + } + get canTest() { + return ( + this.canManage() && + !!this.saved && + !this.changed && + !!this.model.trim() && + !this.testing && + !this.busy + ); + } + get view() { + return testView(this.test, this.saved?.name ?? this.draft.name.trim()); + } + get modelChoices() { + return modelOptions(this.models?.models ?? []); + } + problem(field: SheetField) { + return this.problems.find((item) => item.field === field)?.message ?? ""; + } + same(preset: AiEndpoint) { + return endpointFor(this.draft.endpoint, [preset]) !== undefined; + } + value(event: Event) { + return (event.target as HTMLInputElement).value; + } + + async ngOnInit() { + this.setup = new AiSetupApi(this.api()); + const request = this.request(); + this.saved = request.connection ?? null; + const card = request.card ?? "ollama"; + this.draft = draftFor(card, [], this.saved); + try { + const [endpoints, connectorId] = await Promise.all([ + this.setup.endpoints(), + this.canManage() && !this.saved + ? this.setup.connectorId() + : Promise.resolve(""), + ]); + if (!this.alive) return; + this.policy = endpoints.policy; + this.endpoints = endpoints.endpoints ?? []; + this.connectorId = connectorId; + this.draft = draftFor(card, this.endpoints, this.saved); + this.opened = draftKey(this.draft); + if (this.saved) await this.loadModels(false); + if (this.alive && this.saved && request.focus === "model") + afterNextRender( + () => + this.element.nativeElement + .querySelector("#ai-model-title") + ?.scrollIntoView({ block: "nearest" }), + { injector: this.injector }, + ); + } catch (error) { + if (this.alive) this.fail(error); + } finally { + if (this.alive) { + this.loading = false; + this.cdr.markForCheck(); + // The fields were disabled while loading, so the panel held focus. + this.focusSoon( + this.saved && request.focus === "model" + ? "#ai-deployment, .model-row [role='combobox'], [data-test-connection]" + : ".sheet-fields input:checked, .sheet-fields input:not(:disabled)", + ); + } + } + } + ngOnDestroy() { + this.alive = false; + this.cancelTest(); + } + chooseCard(card: ProviderCard) { + if (card === this.draft.card) return; + this.draft = draftFor(card, this.endpoints); + this.problems = []; + } + usePreset(preset: AiEndpoint) { + this.draft = { ...this.draft, endpoint: preset.url, custom: false }; + this.problems = this.problems.filter((item) => item.field !== "endpoint"); + } + async save() { + if (this.busy || this.loading || !this.canManage() || !this.dirty) return; + this.error = this.errorCode = ""; + this.problems = []; + if (!this.connectorId && this.saved) + this.connectorId = await this.setup.connectorId().catch(() => ""); + let body: ConnectionRequestBody; + try { + body = connectionRequest( + this.draft, + this.connectorId, + this.endpoints, + this.policy ?? "absent", + ); + } catch (error) { + if (!(error instanceof FieldError)) throw error; + this.problems = [{ field: error.field, message: error.message }]; + return; + } + // A retry of an uncertain save sends the same request with the same key, so it never adds a second revision. + if ( + !this.attempt || + JSON.stringify(this.attempt.body) !== JSON.stringify(body) + ) + this.attempt = { body, key: crypto.randomUUID() }; + this.busy = true; + try { + const connection = await this.setup.create( + this.attempt.body, + this.attempt.key, + ); + if (!this.alive) return; + this.attempt = null; + this.saved = connection; + this.opened = draftKey(this.draft); + this.test = { kind: "idle" }; + this.announcement = `Saved ${connection.name}, revision ${connection.revision}.`; + this.connectionSaved.emit(connection); + await this.loadModels(false); + // Saving disabled the focused Save button: continue at the model. + this.focusSoon( + "#ai-deployment, .model-row [role='combobox'], [data-test-connection]", + ); + } catch (error) { + if (!this.alive) return; + if (error instanceof ApiError && error.code === "WV-IDEMPOTENCY-CONFLICT") + this.attempt = null; + this.problems = sheetProblems( + error instanceof ApiError ? error.detail : null, + ); + if (!this.problems.length) this.fail(error); + } finally { + if (this.alive) { + this.busy = false; + this.cdr.markForCheck(); + } + } + } + async loadModels(refresh: boolean) { + const saved = this.saved; + if (!saved) return; + this.modelsNote = "Loading models…"; + this.cdr.markForCheck(); + try { + let models = await this.setup.models( + saved.id, + refresh && this.canManage(), + ); + if (!refresh && models.discovery === "unknown" && this.canManage()) + models = await this.setup.models(saved.id, true); + if (!this.alive || this.saved !== saved) return; + this.models = models; + this.model = + this.draft.card === "azure" + ? this.model + : defaultModel(models.models, this.model); + this.modelsNote = + models.discovery === "ok" || models.discovery === "unknown" + ? "" + : aiMessage(models.discovery, { + endpoint: originOf(saved.config.endpoint), + }); + } catch (error) { + if (this.alive) this.modelsNote = describeError(error).message; + } finally { + if (this.alive) this.cdr.markForCheck(); + } + } + async runTest() { + const saved = this.saved; + const model = this.model.trim(); + if (!this.canTest || !saved) return; + const controller = new AbortController(); + this.controller = controller; + const origin = originOf(saved.config.endpoint); + this.test = { kind: "running", model }; + // The disabled Test button can't keep focus; Cancel is the next thing to press. + this.focusSoon(".test-status button", true); + try { + const result = await this.setup.test(saved.id, model, controller.signal); + if (!this.alive || controller.signal.aborted) return; + this.test = { kind: "done", result, origin }; + this.tested.emit({ revision: saved.id, result, at: Date.now() }); + } catch (error) { + if (!this.alive) return; + this.test = controller.signal.aborted + ? { kind: "canceled" } + : refusal(error, origin); + } finally { + if (this.controller === controller) this.controller = null; + if (this.alive) { + this.cdr.markForCheck(); + this.focusSoon("[data-test-connection]"); + } + } + } + /** + * Focuses a control of the sheet once it is rendered, when focus was lost + * (its control was disabled or removed) or `always` is set and focus is + * still in the sheet. + */ + private focusSoon(selector: string, always = false) { + afterNextRender( + () => { + const sheet = + this.element.nativeElement.querySelector(".modal-panel"); + const active = document.activeElement; + const lost = !active || active === document.body || active === sheet; + if (!sheet || !(lost || (always && sheet.contains(active)))) return; + sheet.querySelector(selector)?.focus(); + }, + { injector: this.injector }, + ); + } + cancelTest() { + this.controller?.abort( + new DOMException("The test was canceled.", "AbortError"), + ); + } + async copy(text: string) { + try { + await navigator.clipboard.writeText(text); + this.announcement = "Copied."; + } catch { + this.announcement = "Copy isn't available here. Select the text instead."; + } + this.cdr.markForCheck(); + } + async close() { + if (this.busy) return; + if (this.canManage() && this.changed) { + const discard = await this.dialogs.confirm({ + title: "Discard your changes?", + message: "This AI connection has changes you haven't saved.", + confirmLabel: "Discard changes", + cancelLabel: "Keep editing", + danger: true, + }); + if (!discard || !this.alive) return; + } + this.cancelTest(); + this.closed.emit(); + } + private fail(error: unknown) { + const plain = describeError(error); + this.error = plain.message; + this.errorCode = plain.code; + } +} diff --git a/studio/src/app/integrations/ai-models-model.ts b/studio/src/app/integrations/ai-models-model.ts new file mode 100644 index 00000000..8fc2c11a --- /dev/null +++ b/studio/src/app/integrations/ai-models-model.ts @@ -0,0 +1,761 @@ +/* +Copyright 2026 Firefly Software Foundation. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +Author: Firefly Software Foundation +SPDX-License-Identifier: Apache-2.0 +*/ +// The rules Settings › AI models applies, kept out of the components so they +// can be tested alone: which endpoints a provider offers, how a connection +// request is built, which model to suggest, what a test result means and how +// the readiness checklist reads. The platform checks all of it again. +import { ApiError } from "../api"; +import { describeError } from "../errors"; +import type { SelectOption } from "../forms/ui/select"; +import { plainServerMessage } from "./connection-copy"; +import { secretRefs, type AiConnection } from "./ai-provider-connection"; +import type { + AiEndpoint, + AiModel, + AiProvider, + AiTestResult, + BindableConnection, + ConnectionRequestBody, + LumiConfiguration, + Readiness, + ReadinessFix, + ReadinessId, + ReadinessStatus, +} from "./ai-setup-api"; + +/** The reserved handle of connections that use no credential. */ +export const NO_CREDENTIAL = "no-credential"; +export const SUGGESTED_MODEL = "qwen3:4b"; +export const PLAIN_HTTP_REFUSED = + "This address is not approved for plain HTTP. A platform operator must add it to the private-origin policy."; +export const GATEWAY_MISSING = + "Tests need the AI gateway. Run `weave platform ai enable`, or ask an operator to deploy it."; +export const RATE_LIMITED = + "Too many AI connection tests. Wait a minute, then test again."; +const NAME = /^[A-Za-z0-9][A-Za-z0-9_.-]*$/; +/** The platform's secret handle rule; a pasted key never matches it. */ +const HANDLE = /^[a-z0-9][a-z0-9_.-]{0,63}$/; +/** The command that stores a key under a handle; a pasted key is never echoed. */ +export const secretSetCommand = (handle: string): string => + `weave platform secret set --handle ${HANDLE.test(handle.trim()) ? handle.trim() : "HANDLE"}`; + +export type ProviderCard = "ollama" | "openai" | "anthropic" | "azure"; +export interface CardInfo { + id: ProviderCard; + title: string; + kind: string; + icon: "model" | "cloud"; + providers: readonly AiProvider[]; +} +export const PROVIDER_CARDS: readonly CardInfo[] = [ + { + id: "ollama", + title: "Ollama", + kind: "Local models", + icon: "model", + providers: ["openai-chat"], + }, + { + id: "openai", + title: "OpenAI", + kind: "Cloud", + icon: "cloud", + providers: ["openai-responses", "openai-chat"], + }, + { + id: "anthropic", + title: "Anthropic", + kind: "Cloud", + icon: "cloud", + providers: ["anthropic"], + }, + { + id: "azure", + title: "Azure OpenAI", + kind: "Cloud", + icon: "cloud", + providers: ["azure-responses", "azure-chat"], + }, +]; +export const cardInfo = (card: ProviderCard): CardInfo => + PROVIDER_CARDS.find((item) => item.id === card)!; +export const API_STYLES: Readonly< + Record<"openai" | "azure", readonly { value: AiProvider; label: string }[]> +> = { + openai: [ + { value: "openai-responses", label: "Responses" }, + { value: "openai-chat", label: "Chat Completions" }, + ], + azure: [ + { value: "azure-responses", label: "Responses" }, + { value: "azure-chat", label: "Chat Completions" }, + ], +}; +const STANDARD: Partial> = { + openai: "https://api.openai.com/v1", + anthropic: "https://api.anthropic.com", +}; +const DEFAULT_NAME: Record = { + ollama: "ollama-local", + openai: "openai", + anthropic: "anthropic", + azure: "azure-openai", +}; +const DEFAULT_HANDLE: Record = { + ollama: "", + openai: "openai-api-key", + anthropic: "anthropic-api-key", + azure: "azure-openai-api-key", +}; + +/** The address as the platform stores it: origin plus path, without a lone trailing slash. */ +export function normalizedEndpoint(value: string): string { + const text = value.trim(); + const url = new URL(text); + return ( + url.origin + + (url.pathname === "/" && !text.endsWith("/") ? "" : url.pathname) + ); +} +export function originOf(value: string): string { + try { + return new URL(value).origin; + } catch { + return value; + } +} +export function endpointFor( + url: string, + endpoints: readonly AiEndpoint[], +): AiEndpoint | undefined { + try { + const wanted = normalizedEndpoint(url); + return endpoints.find( + (endpoint) => normalizedEndpoint(endpoint.url) === wanted, + ); + } catch { + return undefined; + } +} +export function presetsFor( + card: ProviderCard, + endpoints: readonly AiEndpoint[], +): AiEndpoint[] { + const providers = cardInfo(card).providers; + return endpoints.filter((endpoint) => + card === "ollama" + ? endpoint.compat === "ollama" + : endpoint.compat === null && + endpoint.providers.some((provider) => providers.includes(provider)), + ); +} +export const keylessConnection = (connection: AiConnection): boolean => + secretRefs(connection)["apiKey"] === NO_CREDENTIAL; +export function cardOf( + connection: AiConnection, + endpoints: readonly AiEndpoint[], +): ProviderCard { + if ( + keylessConnection(connection) || + endpointFor(connection.config.endpoint, endpoints)?.compat === "ollama" + ) + return "ollama"; + const provider = connection.config.provider; + return provider === "anthropic" + ? "anthropic" + : provider.startsWith("azure-") + ? "azure" + : "openai"; +} +export function cardOfBindable( + item: BindableConnection, + endpoints: readonly AiEndpoint[], +): ProviderCard { + if ( + endpoints.some( + (endpoint) => + endpoint.compat === "ollama" && + originOf(endpoint.url) === item.endpoint_origin, + ) + ) + return "ollama"; + const provider = item.provider ?? ""; + return provider === "anthropic" + ? "anthropic" + : provider.startsWith("azure-") + ? "azure" + : "openai"; +} + +export interface SheetDraft { + card: ProviderCard; + name: string; + endpoint: string; + /** The address was typed, not chosen from the approved endpoints. */ + custom: boolean; + provider: AiProvider; + apiVersion: string; + handle: string; +} +export function draftFor( + card: ProviderCard, + endpoints: readonly AiEndpoint[], + existing?: AiConnection | null, +): SheetDraft { + const presets = presetsFor(card, endpoints); + if (existing) { + const handle = secretRefs(existing)["apiKey"] ?? ""; + return { + card, + name: existing.name, + endpoint: existing.config.endpoint, + custom: !endpointFor(existing.config.endpoint, presets), + provider: existing.config.provider as AiProvider, + apiVersion: existing.config.apiVersion ?? "", + handle: handle === NO_CREDENTIAL ? "" : handle, + }; + } + const preferred = + presets.find((endpoint) => endpoint.url === STANDARD[card]) ?? presets[0]; + return { + card, + name: DEFAULT_NAME[card], + endpoint: preferred?.url ?? STANDARD[card] ?? "", + custom: !preferred, + provider: cardInfo(card).providers[0], + apiVersion: "", + handle: DEFAULT_HANDLE[card], + }; +} +/** What makes two drafts the same request. */ +export const draftKey = (draft: SheetDraft): string => + JSON.stringify([ + draft.card, + draft.name.trim(), + draft.endpoint.trim(), + draft.provider, + draft.apiVersion.trim(), + draft.handle.trim(), + ]); +/** The chosen endpoint is an approved preset that sends no credential. */ +export function keyless( + draft: SheetDraft, + endpoints: readonly AiEndpoint[], +): boolean { + if (draft.custom) return false; + return ( + endpointFor(draft.endpoint, presetsFor(draft.card, endpoints)) + ?.credential === "none" + ); +} + +export type SheetField = + | "name" + | "endpoint" + | "apiVersion" + | "handle" + | "model" + | "form"; +export class FieldError extends Error { + constructor( + readonly field: SheetField, + message: string, + ) { + super(message); + } +} +/** The request for a new connection revision; throws FieldError for the field to fix. */ +export function connectionRequest( + draft: SheetDraft, + connectorId: string, + endpoints: readonly AiEndpoint[], + policy: "loaded" | "absent", +): ConnectionRequestBody { + const name = draft.name.trim(); + if (!NAME.test(name)) + throw new FieldError( + "name", + "Letters, numbers, dots, dashes or underscores.", + ); + let url: URL; + try { + url = new URL(draft.endpoint.trim()); + } catch { + throw new FieldError("endpoint", "Enter the model endpoint's address."); + } + if (url.username || url.password || url.search || url.hash) + throw new FieldError( + "endpoint", + "Use an address without credentials, query parameters or fragments.", + ); + const preset = draft.custom + ? undefined + : endpointFor(draft.endpoint, presetsFor(draft.card, endpoints)); + if (url.protocol === "http:") { + if (!preset) throw new FieldError("endpoint", PLAIN_HTTP_REFUSED); + } else if (url.protocol !== "https:") + throw new FieldError("endpoint", "Use an https:// address."); + else if (policy === "loaded" && draft.card !== "ollama" && !preset) + throw new FieldError( + "endpoint", + "Choose an approved endpoint. A platform operator approves model endpoints.", + ); + if (draft.card === "azure" && !draft.apiVersion.trim()) + throw new FieldError("apiVersion", "Azure requires an API version."); + const noKey = preset?.credential === "none"; + const handle = draft.handle.trim(); + if (!noKey && handle === NO_CREDENTIAL) + throw new FieldError( + "handle", + "no-credential is reserved for approved local endpoints without a key.", + ); + if (!noKey && !HANDLE.test(handle)) + throw new FieldError( + "handle", + "Enter the secret handle your operator approved, never the key itself.", + ); + if (!connectorId) + throw new FieldError( + "form", + "Publish the AI actions first, from the readiness checklist.", + ); + return { + name, + connector_version_id: connectorId, + config: { + provider: draft.provider, + endpoint: normalizedEndpoint(draft.endpoint), + secretSlot: "apiKey", + ...(draft.card === "azure" + ? { apiVersion: draft.apiVersion.trim() } + : {}), + }, + secretRef: { apiKey: noKey ? NO_CREDENTIAL : handle }, + allowed_destinations: [url.origin], + }; +} + +export interface SheetProblem { + field: SheetField; + message: string; + code?: string; +} +const record = (value: unknown): Record => + value !== null && typeof value === "object" && !Array.isArray(value) + ? (value as Record) + : {}; +function fieldFor(pointer: string): SheetField { + if (pointer === "/name") return "name"; + if ( + pointer.startsWith("/config/endpoint") || + pointer.startsWith("/allowed_destinations") + ) + return "endpoint"; + if (pointer.startsWith("/config/apiVersion")) return "apiVersion"; + if (pointer.startsWith("/secretRef")) return "handle"; + return "form"; +} +/** Field problems from a 422 WV-CONNECTION answer, each on the field its pointer names. */ +export function sheetProblems(detail: unknown): SheetProblem[] { + const body = record(detail); + const items = Array.isArray(body["diagnostics"]) + ? body["diagnostics"] + : record(body["result"])["diagnostics"]; + if (!Array.isArray(items)) return []; + return items.slice(0, 20).map((value) => { + const item = record(value); + const message = item["message"]; + const code = String(item["code"] ?? ""); + return { + field: fieldFor(typeof item["path"] === "string" ? item["path"] : ""), + message: + typeof message === "string" && message.trim() && message.length <= 240 + ? plainServerMessage(message) + : "The platform rejected this setting.", + code: /^WV-CONNECTION-[A-Z]+$/.test(code) ? code : undefined, + }; + }); +} + +export function formatSize(bytes?: number): string { + if (bytes === undefined) return ""; + return bytes >= 1e9 + ? `${(bytes / 1e9).toFixed(1)} GB` + : `${Math.max(1, Math.round(bytes / 1e6))} MB`; +} +export function modelFacts(model: AiModel): string { + if (!model.approved) return "Not approved"; + if (!model.available) return "Not installed on this server"; + return [ + formatSize(model.size_bytes), + model.tools === "yes" ? "tools" : model.tools === "no" ? "no tools" : "", + model.context_tokens + ? `${Math.round(model.context_tokens / 1024)}k context` + : "", + ] + .filter(Boolean) + .join(" · "); +} +export function modelOptions(models: readonly AiModel[]): SelectOption[] { + return models.map((model) => ({ + value: model.name, + label: model.name, + description: modelFacts(model), + disabled: !model.approved || !model.available, + })); +} +export function defaultModel(models: readonly AiModel[], current = ""): string { + const usable = models.filter((model) => model.approved && model.available); + return ( + ( + usable.find((model) => model.name === current) ?? + usable.find((model) => model.name === SUGGESTED_MODEL) ?? + usable.find((model) => model.tools === "yes") ?? + usable[0] + )?.name ?? "" + ); +} + +/** The platform's copy for each model error code (contracts/ai.py); a test keeps them equal. */ +export const AI_COPY: Readonly> = { + LLM_POLICY: + "This model is not approved for this endpoint. Ask an operator to approve it.", + LLM_CONNECTION: + "The AI connection does not match this step. Choose the connection again.", + LLM_MODEL_NOT_FOUND: + "{model} is not installed on {endpoint}. Pull it with `ollama pull {model}`.", + LLM_NO_TOOL_SUPPORT: + "{model} cannot call tools. Choose a tool-capable model, such as qwen3:4b.", + LLM_UNREACHABLE: + "Weave could not reach the model endpoint. Check that Ollama is running.", + LLM_TIMEOUT: + "The model did not answer within {n} seconds. Local models can be slow on first load.", + LLM_CONTEXT_LIMIT: + "The prompt is too long for this model's context ({n} tokens).", + LLM_AUTH: "The provider refused the credential.", + LLM_RATE_LIMITED: "The provider is rate limiting requests.", + LLM_CAPACITY: "Weave is busy; the step will retry.", + LLM_OUTPUT: "The model's answer did not match the result fields.", + LLM_LIMIT: "The step reached its token or request limit.", + LLM_INPUT: "The AI step input is not valid.", + LLM_OPTIONS: "This provider does not accept these model options.", + LLM_REASONING: "The reasoning pattern did not reach a final answer.", + LLM_PROVIDER: "The model provider returned an unexpected error.", +}; +export function aiMessage( + code: string, + values: { model?: string; endpoint?: string; n?: number }, +): string { + return (AI_COPY[code] ?? AI_COPY["LLM_PROVIDER"]) + .replaceAll("{model}", values.model ?? "The model") + .replaceAll("{endpoint}", values.endpoint ?? "the endpoint") + .replaceAll("{n}", (values.n ?? 0).toLocaleString("en-US")); +} + +export type TestState = + | { kind: "idle" } + | { kind: "running"; model: string } + | { kind: "canceled" } + | { kind: "done"; result: AiTestResult; origin: string } + | { + kind: "refused"; + message: string; + code: string; + status: number; + origin: string; + }; +export interface TestView { + tone: "neutral" | "success" | "warning" | "danger"; + text: string; + details: [string, string][]; +} +/** The gateway's own test budget, which a timed-out test reports. */ +const TEST_SECONDS = 120; +export function testView(state: TestState, connection: string): TestView { + switch (state.kind) { + case "idle": + return { tone: "neutral", text: "Not tested", details: [] }; + case "canceled": + return { tone: "neutral", text: "Test canceled.", details: [] }; + case "running": + return { + tone: "neutral", + text: `Calling ${state.model} on ${connection}…`, + details: [], + }; + case "refused": + return { + tone: "danger", + text: state.message, + details: [ + ["Code", state.code], + ["Endpoint", state.origin], + [ + "HTTP status", + state.status + ? `${Math.floor(state.status / 100)}xx` + : "No response", + ], + ], + }; + case "done": { + const result = state.result; + const seconds = + result.latency_ms === null + ? "" + : ` in ${(result.latency_ms / 1000).toFixed(1)} s`; + const details: [string, string][] = [ + ["Code", result.code], + ["Endpoint", state.origin], + [ + "Latency", + result.latency_ms === null ? "Unknown" : `${result.latency_ms} ms`, + ], + ]; + if (!result.ok) + return { + tone: "danger", + text: aiMessage(result.code, { + model: result.model, + endpoint: state.origin, + n: + result.code === "LLM_TIMEOUT" + ? TEST_SECONDS + : (result.context_tokens ?? 0), + }), + details, + }; + if (result.tool_calling === "unsupported") + return { + tone: "warning", + text: `Connected. ${result.model} answered${seconds} but cannot call tools. Use it for AI tasks, not AI agents.`, + details, + }; + return { + tone: "success", + text: + result.tool_calling === "supported" + ? `Connected. ${result.model} answered${seconds} and can call tools.` + : `Connected. ${result.model} answered${seconds}.`, + details, + }; + } + } +} +/** A test the platform refused before calling the model. Never shows the response body. */ +export function refusal(error: unknown, origin: string): TestState { + const status = error instanceof ApiError ? error.status : 0; + const code = error instanceof ApiError ? error.code : ""; + const message = + code === "WV-AI-GATEWAY-MISSING" + ? GATEWAY_MISSING + : code === "WV-AI-RATE-LIMITED" + ? RATE_LIMITED + : describeError(error).message; + return { + kind: "refused", + message, + code: code || "WV-STUDIO", + status, + origin, + }; +} + +export const READINESS_LABELS: Readonly> = { + worker_installed: "Agentic worker installed", + actions_published: "AI actions published", + worker_online: "Worker online", + connection: "AI connection", + key_available: "Key available", + connection_authorized: "Connection authorized", + model_responds: "Model responds", + weave_ai: "Weave AI", +}; +const DETAILS: Readonly> = { + worker_installed: "Installed", + worker_missing: "Not installed", + actions_published: "Published", + actions_missing: "Not published yet", + catalog_mismatch: "Published versions differ from this platform's", + worker_online: "Online", + worker_offline: "Offline", + connection_available: "Set up", + connection_missing: "No AI connection yet", + key_grant_available: "Available", + key_missing: "Key not found", + no_credential: "Not needed", + connection_authorized: "Authorized", + authorization_missing: "Not authorized yet", + test_passed: "Answered in the last 24 hours", + test_failed: "The last test failed", + test_required: "Not tested yet", + test_stale: "Last tested more than a day ago", + weave_ai_ready: "Ready", + weave_ai_missing: "Not set up", + weave_ai_disabled: "Turned off", + gateway_missing: "Needs the AI gateway", + record_unavailable: "Couldn't check", + limit_exceeded: "Too many records to check", +}; +const ICONS: Readonly> = { + done: "ok", + action: "warning", + not_needed: "off", + unknown: "help", +}; +export interface ReadinessRow { + id: ReadinessId; + label: string; + status: ReadinessStatus; + tone: "success" | "warning" | "neutral"; + icon: string; + detail: string; + fix?: ReadinessFix; +} +export function readinessRows( + result: Readiness, + roleLabel: (role: string) => string, + lastTest?: { model: string; latency_ms: number | null }, +): ReadinessRow[] { + return result.items.map((item) => ({ + id: item.id, + label: READINESS_LABELS[item.id], + status: item.status, + tone: + item.status === "done" + ? "success" + : item.status === "action" + ? "warning" + : "neutral", + icon: ICONS[item.status], + detail: item.detail.startsWith("requires_") + ? `Requires ${roleLabel(item.detail.slice("requires_".length))}` + : item.id === "model_responds" && item.status === "done" && lastTest + ? `${lastTest.model}${lastTest.latency_ms === null ? "" : ` · ${(lastTest.latency_ms / 1000).toFixed(1)} s`}` + : (DETAILS[item.detail] ?? "Couldn't check"), + fix: item.fix, + })); +} +export function allReady( + result: Readiness, + except: readonly ReadinessId[] = [], +): boolean { + return result.items.every( + (item) => + except.includes(item.id) || + item.status === "done" || + item.status === "not_needed", + ); +} + +export type FixAction = + | { kind: "button"; label: string } + | { kind: "copy"; label: string; command: string } + | { kind: "link"; label: string } + | { kind: "text"; label: string }; +const NEEDS: Readonly> = { + "ai_setup.publish": "definition.publish", + "ai_setup.grant": "connection.manage", + "ai_connections.test": "connection.manage", + "lumi.configuration.write": "lumi.manage", +}; +/** + * What a person can do about an item; null when the fix is not theirs to run. + * The checklist sits on Settings › AI models, so a link to that page is named + * by the card it moves to instead of the page already open. + */ +export function fixAction( + fix: ReadinessFix | undefined, + can: (capability: string) => boolean, + id?: ReadinessId, +): FixAction | null { + if (!fix) return null; + if (fix.kind === "operation") + return fix.operation && can(NEEDS[fix.operation]) + ? { kind: "button", label: fix.label } + : null; + if (fix.kind === "command" && fix.command) + return { kind: "copy", label: fix.label, command: fix.command }; + if (fix.kind === "navigate") + return { + kind: "link", + label: + fix.route !== "settings/ai" + ? fix.label + : id === "weave_ai" + ? "Go to Weave AI" + : "Go to providers", + }; + // "Requires {role}" repeats the row's detail, which names the role in words. + return fix.label.startsWith("Requires ") + ? null + : { kind: "text", label: fix.label }; +} + +export interface WeaveAiDraft { + enabled: boolean; + connection: string; + model: string; +} +/** Limits for a new Weave AI configuration; a saved one keeps its own. */ +export const WEAVE_AI_DEFAULTS: Readonly> = { + options: { max_tokens: 1024 }, + reasoning: { pattern: "none", maxSteps: 6 }, + maxCalls: 8, + timeoutSeconds: 180, +}; +export function weaveAiBody( + draft: WeaveAiDraft, + connection: AiConnection, + current: LumiConfiguration | null, + replySchema: unknown, +): Record { + const model = draft.model.trim(); + if (!model) throw new FieldError("model", "Choose a model."); + const { + provider: _provider, + model: _model, + outputSchema: _schema, + ...kept + } = (current?.profile ?? WEAVE_AI_DEFAULTS) as Record; + return { + enabled: draft.enabled, + connection_revision_id: connection.id, + profile: { + ...kept, + provider: connection.config.provider, + model, + outputSchema: replySchema, + }, + }; +} +export function weaveAiChoices( + connections: readonly AiConnection[], +): SelectOption[] { + return connections.map((connection) => { + const noKey = keylessConnection(connection); + return { + value: connection.id, + label: `${connection.name} · revision ${connection.revision}`, + description: noKey + ? "Weave AI can't use a connection without a key yet" + : connection.config.endpoint, + disabled: noKey, + }; + }); +} diff --git a/studio/src/app/integrations/ai-provider-connection-form.ts b/studio/src/app/integrations/ai-provider-connection-form.ts deleted file mode 100644 index 97470f76..00000000 --- a/studio/src/app/integrations/ai-provider-connection-form.ts +++ /dev/null @@ -1,531 +0,0 @@ -/* -Copyright 2026 Firefly Software Foundation. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -Author: Firefly Software Foundation -SPDX-License-Identifier: Apache-2.0 -*/ -import { - ChangeDetectionStrategy, - ChangeDetectorRef, - Component, - ElementRef, - Injector, - afterNextRender, - OnDestroy, - OnInit, - inject, - input, - output, -} from "@angular/core"; -import { ApiError, type StudioApi } from "../api"; -import { describeError } from "../errors"; -import { Select } from "../forms/ui/select"; -import { - aiConnector, - aiConnectionRequest, - aiProviderEndpoint, - aiProviders, - aiSetupRows, - type AiConnection, - type AiConnectionDraft, -} from "./ai-provider-connection"; - -@Component({ - selector: "weave-ai-provider-connection-form", - standalone: true, - changeDetection: ChangeDetectionStrategy.Eager, - imports: [Select], - template: ` - @if (!canManage()) { -

- Ask an administrator for connection.manage in this environment to create - a provider connection. -

- } @else if (done) { -

Connection created

-

{{ done.name }} · revision {{ done.revision }}

-

Saved · Not tested

-

- The configuration was validated. Provider access has not been tested. No - model request was sent. A platform operator must authorize the worker - release to use this connection before a workflow can run; Weave AI uses - its separately configured gateway. -

- - } @else { -
    - @for (label of stages; track label; let index = $index) { -
  1. - - {{ label }} -
    2. - } -
- @if (loading) { -

Checking the published AI connector…

- } @else if (!connectorId) { -

- Publish {{ connector }} in this project first. A platform operator - installs the Agentic worker and provisions the scoped API key handle. -

- - } -
-
- @if (step === 0) { -

Choose a provider

-

- Name this connection so authors can recognize it. Choose models or - Azure deployments later in workflow AI profiles or Weave AI - settings. -

- - - } @else if (step === 1) { -

Provider access

-

- Choose the credential handle approved for {{ providerLabel }} in - this environment. -

- @if (draft.provider.startsWith("azure-")) { - -

- Enter the base endpoint for your Azure resource. -

- -

- Use the API version approved for your deployment. -

- } @else { -

Standard endpoint: {{ standardEndpoint }}

-
- Advanced: custom endpoint - -

- Override only for an approved proxy or compatible service. The - worker or Weave AI gateway must allow the exact endpoint. -

-
- } - -

- Enter the handle supplied by your platform operator. The operator - stores and grants the secret for this environment. Never paste the - API key. -

- } @else { -

Review your connection

-

- Check these details before creating an immutable connection - revision. -

-
-
-
Connection name
-
{{ draft.name.trim() }}
-
-
-
Provider
-
{{ providerLabel }}
-
-
-
Endpoint
-
{{ reviewedEndpoint }}
-
- @if (draft.provider.startsWith("azure-")) { -
-
Azure API version
-
{{ draft.apiVersion.trim() }}
-
- } -
-
API key secret handle
-
{{ draft.handle.trim() }}
-
-
-

- The platform will validate the configuration. No model request or - connectivity test is sent. Your operator still needs to authorize - the worker release or configure Weave AI's gateway before use. -

- } -
- @if (step > 0) { - - } - -
-
-
- } - @if (error) { - - } - `, - styles: ` - :host { - display: grid; - gap: 16px; - min-width: 0; - } - .progress { - display: grid; - grid-template-columns: repeat(3, minmax(0, 1fr)); - gap: 8px; - list-style: none; - padding: 0 0 16px; - margin: 0; - border-bottom: 1px solid var(--line); - } - .progress li { - display: flex; - align-items: center; - gap: 6px; - color: var(--muted); - font-size: 13px; - } - .progress li[aria-current] { - color: var(--text); - font-weight: 600; - } - .number { - display: grid; - place-items: center; - width: 24px; - height: 24px; - flex: 0 0 24px; - border: 1px solid var(--field-border); - border-radius: 50%; - } - [aria-current] .number { - background: var(--accent); - border-color: var(--accent); - color: var(--on-accent); - } - .complete .number { - background: var(--selected); - color: var(--text); - } - fieldset { - display: grid; - gap: 14px; - border: 0; - padding: 0; - margin: 0; - min-width: 0; - } - .wizard-title { - margin: 0; - font-size: 17px; - line-height: 1.4; - } - label { - display: grid; - gap: 6px; - } - details[open] { - display: grid; - gap: 10px; - } - summary { - cursor: pointer; - padding: 8px 0; - } - input { - min-width: 0; - width: 100%; - box-sizing: border-box; - } - p { - margin: 0; - overflow-wrap: anywhere; - } - .actions { - display: flex; - justify-content: flex-end; - flex-wrap: wrap; - gap: 8px; - border-top: 1px solid var(--line); - margin-top: 4px; - padding-top: 16px; - } - .actions button { - min-height: 40px; - } - .actions button:first-child:not(.primary) { - margin-right: auto; - } - .review { - margin: 0; - display: grid; - gap: 12px; - min-width: 0; - } - .review div { - display: grid; - grid-template-columns: minmax(100px, 1fr) minmax(0, 1.5fr); - gap: 12px; - } - dt { - color: var(--muted); - } - dd { - margin: 0; - overflow-wrap: anywhere; - font-weight: 500; - } - .review-note { - border-left: 3px solid var(--accent); - padding: 10px 12px; - background: var(--raised); - } - @media (max-width: 420px) { - .progress li { - flex-direction: column; - align-items: flex-start; - } - .review div { - grid-template-columns: minmax(0, 1fr); - gap: 3px; - } - } - `, -}) -export class AiProviderConnectionForm implements OnInit, OnDestroy { - api = input.required(); - canManage = input.required(); - created = output(); - finished = output(); - providers = aiProviders; - connector = aiConnector; - draft: AiConnectionDraft = { - name: "", - provider: "", - endpoint: "", - apiVersion: "", - handle: "", - }; - readonly stages = ["Provider", "Access", "Review"]; - step = 0; - get providerLabel() { - return ( - this.providers.find((provider) => provider.value === this.draft.provider) - ?.label ?? "your provider" - ); - } - get standardEndpoint() { - return aiProviderEndpoint(this.draft.provider); - } - get reviewedEndpoint() { - return aiConnectionRequest(this.draft, this.connectorId).config.endpoint; - } - chooseProvider(provider: string) { - if (provider === this.draft.provider) return; - this.draft.provider = provider; - this.draft.endpoint = aiProviderEndpoint(provider); - this.draft.apiVersion = ""; - } - connectorId = ""; - loading = false; - busy = false; - error = ""; - done: AiConnection | null = null; - private alive = true; - private cdr = inject(ChangeDetectorRef); - private host = inject>(ElementRef); - private injector = inject(Injector); - private focus(selector = ".wizard-title") { - const step = this.step; - afterNextRender( - () => { - if (this.alive && this.step === step) - this.host.nativeElement.querySelector(selector)?.focus(); - }, - { injector: this.injector }, - ); - } - back() { - if (this.busy || this.loading || this.step === 0) return; - this.step--; - this.error = ""; - this.focus(); - } - private attempt: { - body: ReturnType; - key: string; - } | null = null; - ngOnInit() { - if (this.canManage()) void this.load(); - } - ngOnDestroy() { - this.alive = false; - } - value(event: Event) { - return (event.target as HTMLInputElement).value; - } - async load() { - this.loading = true; - this.error = ""; - try { - const rows = await aiSetupRows(this.api(), "connectors", false); - if (!this.alive) return; - this.connectorId = String( - rows.find( - (row) => - !row["unavailable"] && - !row["retired"] && - `${row["name"]}@${row["version"]}` === aiConnector, - )?.["id"] ?? "", - ); - } catch (error) { - if (this.alive) - this.error = - error instanceof ApiError && error.status === 403 - ? "Reading the published provider connector requires catalog.read in this project. Ask an administrator for that grant as well as connection.manage in this environment." - : describeError(error).message; - } finally { - if (this.alive) { - this.loading = false; - this.cdr.markForCheck(); - } - } - } - async submit(event: Event) { - event.preventDefault(); - if (this.busy || this.loading || !this.canManage() || !this.connectorId) - return; - this.error = ""; - try { - if (this.step === 0) { - if (!/^[A-Za-z0-9][A-Za-z0-9_.-]*$/.test(this.draft.name.trim())) - throw Error( - "Enter a connection name using letters, numbers, dots, dashes or underscores.", - ); - if ( - !this.providers.some( - (provider) => provider.value === this.draft.provider, - ) - ) - throw Error("Choose a provider."); - this.step = 1; - this.focus(); - return; - } - const body = aiConnectionRequest(this.draft, this.connectorId); - if (this.step === 1) { - this.step = 2; - this.focus(); - return; - } - if ( - !this.attempt || - JSON.stringify(this.attempt.body) !== JSON.stringify(body) - ) - this.attempt = { body, key: crypto.randomUUID() }; - this.busy = true; - const result = await this.api().mutate( - `${this.api().environment}/connections`, - "POST", - this.attempt.body, - undefined, - this.attempt.key, - ); - if (!this.alive) return; - this.done = result; - this.created.emit(result); - this.focus(); - } catch (error) { - if (this.alive) { - this.error = describeError(error).message; - this.focus(".error"); - if ( - error instanceof ApiError && - error.code === "WV-IDEMPOTENCY-CONFLICT" - ) - this.attempt = null; - } - } finally { - if (this.alive) { - this.busy = false; - this.cdr.markForCheck(); - } - } - } -} diff --git a/studio/src/app/integrations/ai-provider-connection.ts b/studio/src/app/integrations/ai-provider-connection.ts index 3305a14f..2ad993cb 100644 --- a/studio/src/app/integrations/ai-provider-connection.ts +++ b/studio/src/app/integrations/ai-provider-connection.ts @@ -24,84 +24,26 @@ export interface AiConnection { revision: number; connector: string; config: { provider: string; endpoint: string; apiVersion?: string }; + /** The secret handle by slot; never a secret value. */ + secretRef?: Record; } -export interface AiConnectionDraft { - name: string; - provider: string; - endpoint: string; - apiVersion: string; - handle: string; -} -export const aiProviders = [ - { value: "openai-responses", label: "OpenAI Responses" }, - { value: "openai-chat", label: "OpenAI Chat" }, - { value: "azure-responses", label: "Azure OpenAI Responses" }, - { value: "azure-chat", label: "Azure OpenAI Chat" }, - { value: "anthropic", label: "Anthropic" }, -]; -export function aiProviderEndpoint(provider: string): string { - if (provider === "openai-chat" || provider === "openai-responses") - return "https://api.openai.com/v1"; - return provider === "anthropic" ? "https://api.anthropic.com" : ""; +/** The secret handles of a connection revision, whichever spelling the platform used. */ +export function secretRefs(connection: AiConnection): Record { + const value = + connection.secretRef ?? + (connection as unknown as { secret_refs?: Record }) + .secret_refs; + return value && typeof value === "object" ? value : {}; } -export function aiConnectionRequest( - draft: AiConnectionDraft, - connector: string, -) { - if (!/^[A-Za-z0-9][A-Za-z0-9_.-]*$/.test(draft.name.trim())) - throw Error( - "Enter a connection name using letters, numbers, dots, dashes or underscores.", - ); - if (!aiProviders.some((p) => p.value === draft.provider)) - throw Error("Choose a provider."); - let endpoint: URL; - const endpointValue = - draft.endpoint.trim() || aiProviderEndpoint(draft.provider); - try { - endpoint = new URL(endpointValue); - } catch { - throw Error("Enter the provider's trusted HTTPS endpoint."); +/** The newest revision of each connection name, sorted by name. */ +export function latestByName(connections: AiConnection[]): AiConnection[] { + const latest = new Map(); + for (const connection of connections) { + const known = latest.get(connection.name); + if (!known || connection.revision > known.revision) + latest.set(connection.name, connection); } - if ( - endpoint.protocol !== "https:" || - endpoint.username || - endpoint.password || - endpoint.search || - endpoint.hash - ) - throw Error( - "Use an HTTPS endpoint without credentials, query parameters or fragments.", - ); - if (draft.provider.startsWith("azure-") && !draft.apiVersion.trim()) - throw Error( - "Azure requires an API version supplied by your platform operator.", - ); - if (!/^[A-Za-z0-9][A-Za-z0-9_.-]*$/.test(draft.handle.trim())) - throw Error( - "Enter an operator-approved API key secret handle, never the key itself.", - ); - if (!connector) - throw Error( - `${aiConnector} must be published in this project before creating this connection.`, - ); - return { - name: draft.name.trim(), - connector_version_id: connector, - config: { - provider: draft.provider, - endpoint: - endpoint.origin + - (endpoint.pathname === "/" && !endpointValue.endsWith("/") - ? "" - : endpoint.pathname), - secretSlot: "apiKey", - ...(draft.provider.startsWith("azure-") - ? { apiVersion: draft.apiVersion.trim() } - : {}), - }, - secretRef: { apiKey: draft.handle.trim() }, - allowed_destinations: [endpoint.origin], - }; + return [...latest.values()].sort((a, b) => a.name.localeCompare(b.name)); } /** Bounded lists fail visibly rather than hiding an older pinned revision. */ export async function aiSetupRows( diff --git a/studio/src/app/integrations/ai-setup-api.ts b/studio/src/app/integrations/ai-setup-api.ts new file mode 100644 index 00000000..bc23cce5 --- /dev/null +++ b/studio/src/app/integrations/ai-setup-api.ts @@ -0,0 +1,284 @@ +/* +Copyright 2026 Firefly Software Foundation. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +Author: Firefly Software Foundation +SPDX-License-Identifier: Apache-2.0 +*/ +// Calls to the platform's AI setup operations. The types mirror the platform's +// payloads and keep their snake_case. The platform checks every permission; +// Studio only shows what a person may do. +import { ApiError, type StudioApi } from "../api"; +import { + aiConnections, + aiConnector, + aiSetupRows, + latestByName, + type AiConnection, +} from "./ai-provider-connection"; + +export type AiProvider = + | "openai-chat" + | "openai-responses" + | "azure-chat" + | "azure-responses" + | "anthropic"; +export interface AiEndpoint { + id: string; + label: string; + url: string; + providers: AiProvider[]; + compat: "ollama" | null; + credential: "none" | "required"; + models: "served" | "listed"; + context_tokens: number | null; + development_only: boolean; +} +export interface AiEndpoints { + policy: "loaded" | "absent"; + endpoints: AiEndpoint[]; +} +export interface AiModel { + name: string; + approved: boolean; + available: boolean; + tools: "yes" | "no" | "unknown"; + context_tokens?: number; + size_bytes?: number; + family?: string; +} +export interface AiModels { + approval: "served" | "listed" | "unknown"; + discovery: string; + discovered_at: string | null; + models: AiModel[]; +} +export interface AiTestResult { + ok: boolean; + code: string; + latency_ms: number | null; + model: string; + tool_calling: "supported" | "unsupported" | "unknown"; + context_tokens?: number; +} +export type ReadinessId = + | "worker_installed" + | "actions_published" + | "worker_online" + | "connection" + | "key_available" + | "connection_authorized" + | "model_responds" + | "weave_ai"; +export type ReadinessStatus = "done" | "action" | "not_needed" | "unknown"; +export interface ReadinessFix { + kind: "operation" | "command" | "navigate" | "ask"; + label: string; + operation?: + | "ai_setup.publish" + | "ai_setup.grant" + | "ai_connections.test" + | "lumi.configuration.write" + | null; + command?: string | null; + route?: "settings/ai" | "operate/workers" | null; +} +export interface ReadinessItem { + id: ReadinessId; + status: ReadinessStatus; + detail: string; + fix?: ReadinessFix; +} +export interface Readiness { + items: ReadinessItem[]; +} +export interface BindableConnection { + connection_id: string; + revision_id: string; + name: string; + connector: string; + label: string; + enabled: boolean; + provider?: string; + endpoint_origin?: string; +} +interface BindablePage { + items: BindableConnection[]; + next_cursor: string | null; +} +export interface ConnectionRequestBody { + name: string; + connector_version_id: string; + config: { + provider: AiProvider; + endpoint: string; + secretSlot: "apiKey"; + apiVersion?: string; + }; + secretRef: { apiKey: string }; + allowed_destinations: string[]; +} +export interface LumiConfiguration { + enabled: boolean; + connection_revision_id: string; + profile: Record; + revision: number; +} +export interface LumiStatus { + configured: boolean; + provider?: string | null; + model?: string | null; +} + +/** A model test: the platform stops it after 130 seconds; Studio waits a little longer. */ +export const TEST_TIMEOUT_MS = 150_000; +/** Model discovery: the AI gateway gives a server 15 seconds. */ +export const DISCOVERY_TIMEOUT_MS = 30_000; +/** An ordinary platform call: the Studio host stops waiting after 35 seconds. */ +export const BRIDGE_TIMEOUT_MS = 35_000; + +export class AiSetupApi { + constructor(private readonly api: StudioApi) {} + private get environment() { + return this.api.environment; + } + endpoints() { + return this.api.request(`${this.environment}/ai/endpoints`); + } + models(revision: string, refresh = false, signal?: AbortSignal) { + return this.api.request( + `${this.environment}/ai/models?connection=${encodeURIComponent(revision)}&refresh=${refresh}`, + "GET", + undefined, + {}, + DISCOVERY_TIMEOUT_MS, + signal, + ); + } + readiness() { + return this.api.request(`${this.environment}/ai/readiness`); + } + test(revision: string, model: string, signal?: AbortSignal) { + return this.api.request( + `${this.environment}/ai/connections/${encodeURIComponent(revision)}/test`, + "POST", + { model, probe_tools: true }, + {}, + TEST_TIMEOUT_MS, + signal, + ); + } + publish() { + return this.api.request>( + `${this.environment}/ai/setup/publish`, + "POST", + undefined, + {}, + BRIDGE_TIMEOUT_MS, + ); + } + grant(revision: string) { + return this.api.request<{ granted: boolean; not_needed?: boolean }>( + `${this.environment}/ai/setup/grant`, + "POST", + { connection_revision_id: revision }, + ); + } + /** The newest revision of each AI connection (needs connection.manage). */ + async connections(): Promise { + return latestByName( + aiConnections(await aiSetupRows(this.api, "connections", true)), + ); + } + /** One connection revision by its ID, or null when the platform has no such revision. */ + async connection(revision: string): Promise { + const found = await this.api.request>( + `${this.environment}/connections/${encodeURIComponent(revision)}`, + ); + return found?.id === revision && typeof found.name === "string" + ? (found as AiConnection) + : null; + } + /** AI connections this person may choose (needs connection.bind). */ + async bindable(): Promise { + const items: BindableConnection[] = []; + let cursor: string | null = null; + for (let page = 0; page < 20; page++) { + const query: string = `connector=${encodeURIComponent(aiConnector)}&limit=100${cursor ? `&cursor=${encodeURIComponent(cursor)}` : ""}`; + const result: BindablePage = await this.api.request( + `${this.environment}/bindable-connections?${query}`, + ); + items.push(...result.items); + if (!result.next_cursor) return items; + cursor = result.next_cursor; + } + throw Error( + "Too many connections to list here. Ask an administrator to narrow the environment's configuration.", + ); + } + /** The published AI connector version, or "" before the AI actions are published. */ + async connectorId(): Promise { + const rows = await aiSetupRows(this.api, "connectors", false); + return String( + rows.find( + (row) => + !row["unavailable"] && + !row["retired"] && + `${row["name"]}@${row["version"]}` === aiConnector, + )?.["id"] ?? "", + ); + } + /** Creates a connection revision; a retry with the same key never adds a second one. */ + create(body: ConnectionRequestBody, key: string) { + return this.api.mutate( + `${this.environment}/connections`, + "POST", + body, + undefined, + key, + ); + } + async lumiConfiguration(): Promise { + try { + return await this.api.request( + `${this.environment}/lumi/configuration`, + ); + } catch (error) { + if (error instanceof ApiError && error.status === 404) return null; + throw error; + } + } + saveLumi(body: Record, revision: number | null) { + return this.api.request( + `${this.environment}/lumi/configuration`, + "PUT", + body, + revision === null ? {} : { "If-Match": `"${revision}"` }, + ); + } + lumiStatus() { + return this.api.request(`${this.environment}/lumi/status`); + } + /** The fixed reply schema every Weave AI configuration carries. */ + async lumiReplySchema(): Promise { + const schema = await this.api.request>( + "/studio/contracts/lumi-configuration", + ); + const defs = (schema["$defs"] ?? {}) as Record; + const reply = + defs["LumiProfile"]?.["properties"]?.["outputSchema"]?.["const"]; + if (!reply) + throw Error("The host did not provide the fixed Weave AI reply schema."); + return reply; + } +} diff --git a/studio/src/app/integrations/editor-host.ts b/studio/src/app/integrations/editor-host.ts index ad7127bd..666dc38f 100644 --- a/studio/src/app/integrations/editor-host.ts +++ b/studio/src/app/integrations/editor-host.ts @@ -68,7 +68,6 @@ export interface EditorHost { apiBuilder: { context: UseContext; tab: BuilderTab; step: string } | null; lastUse: HttpActionUse | null; connectionDialog: { - kind?: "ai"; fromBuilder: HttpActionUse["connection"] | null; /** Opened from a workflow: "Back to the workflow" returns there. */ fromWorkflow?: boolean; diff --git a/studio/src/app/integrations/integration-dialogs.ts b/studio/src/app/integrations/integration-dialogs.ts index a62f1696..b01c98ae 100644 --- a/studio/src/app/integrations/integration-dialogs.ts +++ b/studio/src/app/integrations/integration-dialogs.ts @@ -17,11 +17,11 @@ SPDX-License-Identifier: Apache-2.0 */ // The quick-integration dialogs the shell opens: the API action builder (with // the readiness checklist), "New API connection" and "New workflow from a -// template". Load it lazily (@defer) and place it before , +// template". AI connections open in their own side sheet, which the shell +// hosts. Load it lazily (@defer) and place it before , // so the builder's own confirmations open above it. Each dialog loads its // own chunk when it first opens. import { ChangeDetectionStrategy, Component, input } from "@angular/core"; -import { AiProviderConnectionForm } from "./ai-provider-connection-form"; import { Modal } from "../dialog"; import type { EditorHost } from "./editor-host"; import { HttpActionBuilder, type HttpActionUse } from "./http-action-builder"; @@ -48,7 +48,6 @@ import { Modal, HttpActionBuilder, HttpConnectionForm, - AiProviderConnectionForm, IntegrationReadiness, TemplateGallery, ], @@ -82,38 +81,25 @@ import { @if (host().connectionDialog; as dialog) { @if (host().profile) { - @if (dialog.kind === "ai") { - @defer (on immediate) { - - } - } @else { - @defer (on immediate) { - - } @placeholder { -

Loading…

- } + @defer (on immediate) { + + } @placeholder { +

Loading…

} } diff --git a/studio/src/app/lumi/lumi-panel.ts b/studio/src/app/lumi/lumi-panel.ts index c7768e33..3c305105 100644 --- a/studio/src/app/lumi/lumi-panel.ts +++ b/studio/src/app/lumi/lumi-panel.ts @@ -25,20 +25,8 @@ import { inject, } from "@angular/core"; import type { App } from "../app"; -import { ApiError } from "../api"; import { parse } from "yaml"; -import { Select } from "../forms/ui/select"; -import { AiProviderConnectionForm } from "../integrations/ai-provider-connection-form"; -import { - aiConnections, - aiSetupRows, - type AiConnection, -} from "../integrations/ai-provider-connection"; import { Modal } from "../dialog"; -import { AiProfileEditor } from "../integrations/ai-profile-editor"; -import { AiSetupWizard } from "../integrations/ai-setup-wizard"; -import type { Schema } from "../task-schema"; -import { missingData } from "../forms/core/form-model"; import { describeError } from "../errors"; import { exportFile } from "../export-file"; import { @@ -58,13 +46,7 @@ const uuid = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i; selector: "weave-lumi-panel", standalone: true, changeDetection: ChangeDetectionStrategy.Eager, - imports: [ - Modal, - AiProfileEditor, - AiSetupWizard, - Select, - AiProviderConnectionForm, - ], + imports: [Modal], template: ` @if (host.lumiOpen) {
-
+
@if (status?.configured) { {{ status.provider }} · {{ status.model }} } - @if (host.profile && host.can("lumi.manage")) { - }
@@ -103,155 +77,16 @@ const uuid = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;

Ask an administrator for access to Weave AI in this environment.

- } @else if (!settings && !status?.configured && !loading) { + } @else if (!status?.configured && !loading) {

- Weave AI is unavailable in this environment. An administrator - configures its model and connection; a platform operator enables the - Weave AI gateway. Saving model settings alone does not deploy the - gateway. + Weave AI is not set up in this environment. A Weave AI manager sets + it up in Settings › AI models.

} @if (loading) {

Loading Weave AI…

} - @if (settings) { -
-

- Choose Weave AI's model and connection for this environment. - Workflow AI profiles have their own settings. -

- @if (!host.can("connection.manage")) { -

- To select or save a provider connection, ask an administrator - for connection.manage in this environment as well as Weave AI - manager. -

- } - @if (configSchema) { - -
- -
-
-

- Choose Weave AI's provider connection. Credentials stay with - the platform operator. -

- -

- Weave AI pins this exact revision. Choose a connection with - the same provider as the profile; changing a workflow - profile will not change these settings. -

- @if (configConnection && !selectedConnection) { -

- Choose an available connection that matches this provider. -

- } - @if (host.can("connection.manage")) { - @if (!connectionOptions.length) { -

- No matching provider connections yet. Create one with an - operator-approved secret handle, or change the provider. -

- } -
- -
- @if (newConnection) { - - } - } -
-
-
- @for (row of settingsSummary; track row.label) { -
-
{{ row.label }}
-
{{ row.value }}
-
- } -
-

- Saving applies these settings to Weave AI in the current - environment. It does not change workflow AI profiles or send - a model request. The platform operator must enable the Weave - AI gateway and authorize this connection before you can ask - Weave AI. -

-
-
- @if (configError) { - - } -
-
- -
- - } @else if (configBusy) { -

Loading Weave AI settings…

- } - @if (!configSchema && configError) { - - } -
- -
-
- } @else if (review; as item) { + @if (review; as item) {

{{ item.title }}

@@ -442,35 +277,6 @@ const uuid = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i; [hidden] { display: none !important; } - .settings-connection { - display: grid; - gap: 12px; - min-width: 0; - } - .settings-summary { - margin: 0; - border-top: 1px solid var(--line); - } - .settings-summary div { - display: grid; - grid-template-columns: minmax(100px, 1fr) minmax(0, 2fr); - gap: 16px; - padding: 12px 0; - border-bottom: 1px solid var(--line); - } - .settings-summary dt { - color: var(--muted); - } - .settings-summary dd { - margin: 0; - overflow-wrap: anywhere; - } - @media (max-width: 480px) { - .settings-summary div { - grid-template-columns: 1fr; - gap: 4px; - } - } .lumi-conversation { display: grid; gap: 16px; @@ -494,7 +300,6 @@ const uuid = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i; max-width: 100%; white-space: normal; } - .lumi-settings, .lumi-review, fieldset { display: grid; @@ -533,8 +338,6 @@ const uuid = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i; }) export class LumiPanel implements DoCheck { @Input({ required: true }) host!: App; - readonly validateAiProfile = (profile: Record) => - this.host.api.validateLlmProfile({ ...profile, outputSchema: {} }); private cdr = inject(ChangeDetectorRef); private element = inject>(ElementRef); private keepFocus() { @@ -549,7 +352,6 @@ export class LumiPanel implements DoCheck { back() { this.keepFocus(); this.review = null; - this.settings = false; } conversation = new LumiConversation(); message = ""; @@ -562,58 +364,6 @@ export class LumiPanel implements DoCheck { null; review: ProposalReview | null = null; validating = false; - settings = false; - settingsStep = 0; - readonly settingsHeadings = [ - "Choose Weave AI's model", - "Choose a provider connection", - "Review Weave AI settings", - ]; - get settingsSummary() { - const profile = object(this.config["profile"]); - const connection = this.connections.find( - (c) => c.id === this.configConnection, - ); - return [ - { - label: "Assistant", - value: this.config["enabled"] ? "Enabled" : "Disabled", - }, - { label: "Provider", value: profile["provider"] }, - { label: "Model or deployment", value: profile["model"] }, - { - label: "Connection", - value: connection - ? `${connection.name} · revision ${connection.revision}` - : "Unavailable", - }, - { - label: "Endpoint", - value: connection?.config.endpoint ?? "Unavailable", - }, - { - label: "Reasoning", - value: object(profile["reasoning"])["pattern"] ?? "none", - }, - { label: "Maximum calls", value: profile["maxCalls"] }, - { label: "Timeout", value: `${profile["timeoutSeconds"]} seconds` }, - { - label: "Maximum output tokens", - value: object(profile["options"])["max_tokens"] ?? "Provider default", - }, - ]; - } - configBusy = false; - configError = ""; - configSchema: Schema | null = null; - configInitial: Record = {}; - config: Record = {}; - configValid = true; - configRevision: number | null = null; - configConnection = ""; - connections: AiConnection[] = []; - newConnection = false; - private replySchema: unknown; private wasOpen = false; ngDoCheck() { const key = JSON.stringify([ @@ -631,10 +381,6 @@ export class LumiPanel implements DoCheck { } if (this.host.lumiOpen && !this.wasOpen) void this.loadStatus(); this.wasOpen = this.host.lumiOpen; - if (this.host.lumiOpen && this.host.lumiSettingsRequested) { - this.host.lumiSettingsRequested = false; - void this.openSettings(); - } } private reset() { this.message = ""; @@ -643,23 +389,12 @@ export class LumiPanel implements DoCheck { this.review = null; this.status = null; this.busy = false; - this.settings = false; - this.configSchema = null; - this.config = {}; - this.configInitial = {}; - this.configRevision = null; - this.configConnection = ""; - this.connections = []; - this.newConnection = false; this.error = ""; this.loading = false; - this.configBusy = false; this.validating = false; - this.configError = ""; - this.replySchema = undefined; } clear() { - if (this.busy || this.configBusy) return; + if (this.busy) return; this.conversation.clear(); this.message = ""; this.review = null; @@ -881,179 +616,4 @@ export class LumiPanel implements DoCheck { item.source, ); } - get connectionOptions() { - const provider = object(this.config["profile"])["provider"]; - return this.connections - .filter((c) => c.config.provider === provider) - .map((c) => ({ - value: c.id, - label: `${c.name} · revision ${c.revision}`, - description: c.config.endpoint, - })); - } - get selectedConnection() { - return this.connectionOptions.some( - (option) => option.value === this.configConnection, - ); - } - async loadConnections() { - if (!this.host.can("connection.manage")) return; - const generation = this.conversation.generation; - try { - const rows = await aiSetupRows(this.host.api, "connections", true); - if (generation === this.conversation.generation) - this.connections = aiConnections(rows); - } catch (error) { - if (generation === this.conversation.generation) - this.configError = describeError(error).message; - } finally { - this.cdr.markForCheck(); - } - } - connectionCreated(connection: AiConnection) { - this.connections = [ - ...this.connections.filter((c) => c.id !== connection.id), - connection, - ]; - this.configConnection = connection.id; - } - get modelComplete() { - return ( - !!this.configSchema && - this.configValid && - !missingData(this.configSchema, this.config).length - ); - } - get configComplete() { - return ( - this.host.can("connection.manage") && - this.selectedConnection && - this.modelComplete - ); - } - async openSettings() { - if (this.configBusy || !this.host.profile || !this.host.can("lumi.manage")) - return; - this.keepFocus(); - this.settings = true; - this.settingsStep = 0; - this.newConnection = false; - this.configSchema = null; - this.keepFocus(); - this.configBusy = true; - this.configError = ""; - const generation = this.conversation.generation; - try { - const schema = await this.host.api.request>( - "/studio/contracts/lumi-configuration", - ); - const defs = object(schema["$defs"]); - const profile = object(defs["LumiProfile"]); - this.replySchema = object(object(profile["properties"])["outputSchema"])[ - "const" - ]; - if (!this.replySchema) - throw Error( - "The host did not provide the fixed Weave AI reply schema.", - ); - const properties = { ...object(profile["properties"]) }; - delete properties["outputSchema"]; - const { connection_revision_id: _, ...configProperties } = object( - schema["properties"], - ); - const editable = { - ...schema, - properties: configProperties, - required: (schema["required"] as string[]).filter( - (key) => key !== "connection_revision_id", - ), - $defs: { - ...defs, - LumiProfile: { - ...profile, - properties, - required: (profile["required"] as string[]).filter( - (key) => key !== "outputSchema", - ), - }, - }, - }; - let configuration: Record = { - enabled: true, - profile: { - options: { max_tokens: 2048 }, - reasoning: { pattern: "none", maxSteps: 6 }, - maxCalls: 8, - timeoutSeconds: 120, - }, - }; - try { - configuration = await this.host.api.request>( - `${this.host.api.environment}/lumi/configuration`, - ); - } catch (error) { - if (!(error instanceof ApiError && error.status === 404)) throw error; - } - if (generation !== this.conversation.generation) return; - this.configRevision = configuration["revision"] ?? null; - const { revision, connection_revision_id, ...data } = configuration; - this.configConnection = connection_revision_id ?? ""; - const cleanProfile = { ...object(data["profile"]) }; - delete cleanProfile["outputSchema"]; - this.configInitial = { ...data, profile: cleanProfile }; - this.config = structuredClone(this.configInitial); - this.configSchema = editable as Schema; - await this.loadConnections(); - } catch (error) { - if (generation === this.conversation.generation) - this.configError = describeError(error).message; - } finally { - if (generation === this.conversation.generation) { - this.configBusy = false; - this.cdr.markForCheck(); - } - } - } - async saveSettings() { - if ( - this.configBusy || - this.settingsStep !== 2 || - !this.configComplete || - !this.host.can("lumi.manage") - ) - return; - this.configBusy = true; - this.configError = ""; - const generation = this.conversation.generation; - try { - const body = { - ...this.config, - connection_revision_id: this.configConnection, - profile: { - ...object(this.config["profile"]), - outputSchema: this.replySchema, - }, - }; - const result = await this.host.api.request>( - `${this.host.api.environment}/lumi/configuration`, - "PUT", - body, - this.configRevision === null - ? {} - : { "If-Match": `"${this.configRevision}"` }, - ); - if (generation !== this.conversation.generation) return; - this.configRevision = result["revision"]; - this.settings = false; - await this.loadStatus(); - } catch (error) { - if (generation === this.conversation.generation) - this.configError = describeError(error).message; - } finally { - if (generation === this.conversation.generation) { - this.configBusy = false; - this.cdr.markForCheck(); - } - } - } } diff --git a/studio/src/app/modal-sheet.ts b/studio/src/app/modal-sheet.ts index 5110b539..e7fd3bc3 100644 --- a/studio/src/app/modal-sheet.ts +++ b/studio/src/app/modal-sheet.ts @@ -64,6 +64,7 @@ export const keepUsable = [ "weave-dialog-host", "weave-step-picker", "weave-integration-dialogs", + "weave-ai-connection-sheet", "weave-start-run-dialog", "weave-activation-dialog", "weave-resolve-incident-dialog", diff --git a/studio/src/app/operations/records-view.ts b/studio/src/app/operations/records-view.ts index ac903b6a..d77783b5 100644 --- a/studio/src/app/operations/records-view.ts +++ b/studio/src/app/operations/records-view.ts @@ -29,6 +29,7 @@ import { FilePicker } from "../forms/ui/file-picker"; import { filesIn } from "../forms/core/file-reference"; import type { Schema } from "../task-schema"; import { ConnectionDetail } from "../integrations/connection-detail"; +import { aiConnector } from "../integrations/ai-provider-connection"; import { NewMenu } from "../templates/new-menu"; import { localKeepLabel, type LocalDraftEntry } from "../local-drafts"; import { isDesktopShell } from "../export-file"; @@ -639,6 +640,19 @@ const runStatusFilters: [string, string][] = [ platform version.

} @else { + @if (record["connector"] === aiConnector) { +

+ AI connections are set up and tested in Settings › AI + models. + +

+ } (); readonly sheetWhen = sheetWhen; + readonly aiConnector = aiConnector; readonly statusFilters = runStatusFilters; readonly taskStatuses: [string, string][] = [ ["ready", "Ready to claim"], diff --git a/studio/src/app/settings/ai-models-page.css b/studio/src/app/settings/ai-models-page.css new file mode 100644 index 00000000..a268cd9d --- /dev/null +++ b/studio/src/app/settings/ai-models-page.css @@ -0,0 +1,182 @@ +/* +Copyright 2026 Firefly Software Foundation. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +Author: Firefly Software Foundation +SPDX-License-Identifier: Apache-2.0 +*/ +:host { + display: block; +} +.ai-intro { + margin: var(--space-4) 0; + color: var(--muted); +} +.ai-layout { + display: grid; + gap: var(--space-6); + grid-template-columns: minmax(0, 1fr); + grid-template-areas: "readiness" "providers" "weave"; +} +@media (min-width: 1024px) { + .ai-layout { + grid-template-columns: minmax(0, 1fr) 360px; + grid-template-areas: "providers readiness" "weave weave"; + align-items: start; + } +} +.ai-readiness { + grid-area: readiness; +} +.ai-providers { + grid-area: providers; +} +.weave-ai { + grid-area: weave; +} +.ai-panel { + min-width: 0; + padding: var(--space-6); + background: var(--surface); + border: 1px solid var(--border); + border-radius: var(--radius-md); +} +.panel-heading { + display: flex; + align-items: center; + justify-content: space-between; + gap: var(--space-2); + min-height: var(--control-md); + margin-bottom: var(--space-3); +} +.panel-heading h2 { + margin: 0; + font: var(--type-title-sm); +} +/* The four provider cards fill their rows evenly: four across, two by two, + or one per row, by the width the panel gives them (cards of 160 px or more). */ +.ai-providers { + container-type: inline-size; +} +.provider-cards { + list-style: none; + margin: 0; + padding: 0; + display: grid; + gap: var(--space-3); + grid-template-columns: minmax(0, 1fr); +} +@container (min-width: 332px) { + .provider-cards { + grid-template-columns: repeat(2, minmax(0, 1fr)); + } +} +@container (min-width: 676px) { + .provider-cards { + grid-template-columns: repeat(4, minmax(0, 1fr)); + } +} +.provider-card { + display: grid; + grid-template-rows: auto auto 1fr auto; + gap: var(--space-2); + min-width: 0; + padding: var(--space-4); + background: var(--raised); + border: 1px solid var(--border); + border-radius: var(--radius-md); +} +.provider-card p { + margin: 0; +} +.provider-title { + display: flex; + align-items: center; + gap: var(--space-2); + font: var(--type-label); +} +.provider-kind { + color: var(--muted); + font: var(--type-small); +} +.provider-card > button { + justify-self: start; +} +.connections-heading { + margin: var(--space-6) 0 var(--space-2); + font: var(--type-label); +} +.connection-list { + list-style: none; + margin: 0; + padding: 0; + border-top: 1px solid var(--line); +} +.connection-row { + display: grid; + grid-template-columns: minmax(0, 1fr) auto; + align-items: center; + gap: var(--space-2) var(--space-4); + min-height: var(--control-sm); + padding: var(--space-2) 0; + border-bottom: 1px solid var(--line); +} +.connection-main { + display: grid; + gap: 2px; + min-width: 0; +} +.connection-name { + font: var(--type-label); + overflow-wrap: anywhere; +} +.connection-meta { + display: flex; + flex-wrap: wrap; + align-items: center; + gap: var(--space-1) var(--space-2); + min-width: 0; + color: var(--muted); + font: var(--type-small); +} +.connection-meta .endpoint { + font: var(--type-mono); + overflow-wrap: anywhere; +} +.connection-test { + font: var(--type-small); + color: var(--muted); +} +.connection-test[data-tone="success"] { + color: var(--success-ink); +} +.connection-test[data-tone="danger"] { + color: var(--danger-ink); +} +.connection-actions { + display: flex; + align-items: center; + gap: var(--space-2); +} +.empty { + margin: var(--space-2) 0; + color: var(--muted); +} +@media (max-width: 599px) { + .ai-panel { + padding: var(--space-4); + } + .connection-row { + grid-template-columns: minmax(0, 1fr); + } +} diff --git a/studio/src/app/settings/ai-models-page.ts b/studio/src/app/settings/ai-models-page.ts new file mode 100644 index 00000000..484685fc --- /dev/null +++ b/studio/src/app/settings/ai-models-page.ts @@ -0,0 +1,530 @@ +/* +Copyright 2026 Firefly Software Foundation. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +Author: Firefly Software Foundation +SPDX-License-Identifier: Apache-2.0 +*/ +// Settings › AI models: one place to connect models, check that AI steps can +// run, and set up Weave AI. The shell owns the AI connection sheet; this page +// asks it to open and reloads whenever an AI connection or test changes. The +// platform checks every permission; this page only offers what a person may do. +import { + ChangeDetectionStrategy, + ChangeDetectorRef, + Component, + DoCheck, + ElementRef, + OnDestroy, + inject, + input, +} from "@angular/core"; +import { ApiError, type Profile, type StudioApi } from "../api"; +import { describeError } from "../errors"; +import { Icon } from "../icon"; +import { RowMenu, type RowMenuItem } from "../row-menu"; +import type { + AiSheetRequest, + AiTestRecord, +} from "../integrations/ai-connection-sheet"; +import type { AiConnection } from "../integrations/ai-provider-connection"; +import { + PROVIDER_CARDS, + allReady, + cardInfo, + cardOf, + cardOfBindable, + endpointFor, + fixAction, + keylessConnection, + readinessRows, + type ProviderCard, + type ReadinessRow, +} from "../integrations/ai-models-model"; +import { + AiSetupApi, + type AiEndpoint, + type BindableConnection, + type Readiness, +} from "../integrations/ai-setup-api"; +import { AiReadinessList } from "./ai-readiness-list"; +import { WeaveAiCard } from "./weave-ai-card"; + +/** What the page reads from and asks of the shell. */ +export interface AiModelsHost { + readonly api: StudioApi; + readonly profile: Profile | null; + readonly aiSetupGeneration: number; + readonly aiTests: ReadonlyMap; + can(capability: string, resource?: string): boolean; + roleLabel(role: string): string; + openAiConnectionDialog(request?: AiSheetRequest): void; + navigate(view: "workers"): Promise; + notify(text: string): void; +} +/** A connection row: managed connections can be tested; chosen ones only viewed. */ +interface Row { + connection: AiConnection; + card: ProviderCard; + readOnly: boolean; +} +const fromBindable = (item: BindableConnection): AiConnection => ({ + id: item.revision_id, + name: item.name, + revision: 0, + connector: item.connector, + config: { + provider: item.provider ?? "openai-chat", + endpoint: item.endpoint_origin ?? "", + }, +}); + +@Component({ + selector: "weave-ai-models-page", + standalone: true, + changeDetection: ChangeDetectionStrategy.Eager, + imports: [Icon, RowMenu, AiReadinessList, WeaveAiCard], + styleUrl: "./ai-models-page.css", + template: `

+ Connect a model once. Workflows and Weave AI use the connections you set + up here. +

+
+
+
+

Readiness

+ +
+ @if (readinessError) { + + } @else if (!readiness) { +

Checking…

+ } @else { + @if (ready) { +

+ AI steps can run in this environment. +

+ } + + } +
+
+
+

Providers

+
+
    + @for (card of cards; track card.id) { + @let total = count(card.id); + @let action = cardAction(card.id); +
  • +

    + {{ card.title }} +

    +

    {{ card.kind }}

    +

    + @if (total) { + Connected · {{ total }} + } @else { + Not set up + } +

    + @if (action) { + + } +
    • + } +
+

+ Connections +

+ @if (listError) { + + } @else if (!canManage() && !canBind()) { +

Seeing AI connections requires the Deployer role.

+ } @else { +
    + @for (item of visible; track item.connection.id) { +
  • +
    + {{ + item.connection.name + }} + + {{ providerLabel(item) }} + {{ + item.connection.config.endpoint + }} + @if (developmentOnly(item)) { + Development only + } + + @if (!item.readOnly) { + {{ testText(item) }} + } +
    +
    + @if (item.readOnly) { + + } @else { + + + } +
    +
    • + } @empty { +
  • + {{ + listed + ? "No AI connections yet. Connect a provider above." + : "Loading…" + }} +
    • + } +
+ } +
+ +
+

{{ announcement }}

`, +}) +export class AiModelsPage implements DoCheck, OnDestroy { + host = input.required(); + readonly cards = PROVIDER_CARDS; + private readonly cdr = inject(ChangeDetectorRef); + private readonly element = inject>(ElementRef); + /** The AI setup generation and the person's access the page last loaded for. */ + private loadedFor = ""; + private loads = 0; + private alive = true; + private setup: AiSetupApi | null = null; + readiness: Readiness | null = null; + readinessError = ""; + checking = false; + endpoints: AiEndpoint[] = []; + connections: AiConnection[] = []; + bindable: BindableConnection[] | null = null; + listed = false; + listError = ""; + running = ""; + failures: Record = {}; + announcement = ""; + + ngDoCheck() { + // Capabilities arrive after the page may first render; what the page may + // list depends on them, so a change in access reloads it too. + const key = `${this.host().aiSetupGeneration}:${this.canManage()}:${this.canBind()}`; + if (key === this.loadedFor) return; + this.loadedFor = key; + void this.load(); + } + ngOnDestroy() { + this.alive = false; + } + canManage() { + return this.host().can("connection.manage"); + } + canBind() { + return this.host().can("connection.bind"); + } + async load() { + const load = ++this.loads; + this.setup = new AiSetupApi(this.host().api); + await Promise.all([this.recheck(load), this.loadConnections(load)]); + } + async recheck(load = this.loads) { + const setup = this.setup; + if (!setup) return; + this.checking = true; + try { + const readiness = await setup.readiness(); + if (this.alive && load === this.loads) { + this.readiness = readiness; + this.readinessError = ""; + } + } catch (error) { + if (this.alive && load === this.loads) + this.readinessError = + error instanceof ApiError && error.status === 403 + ? "Checking AI readiness requires the Viewer role." + : describeError(error).message; + } finally { + if (this.alive) { + this.checking = false; + this.cdr.markForCheck(); + } + } + } + private async loadConnections(load: number) { + const setup = this.setup!; + try { + const endpoints = await setup + .endpoints() + .catch(() => ({ policy: "absent" as const, endpoints: [] })); + const connections = this.canManage() ? await setup.connections() : []; + const bindable = + !this.canManage() && this.canBind() ? await setup.bindable() : null; + if (!this.alive || load !== this.loads) return; + this.endpoints = endpoints.endpoints ?? []; + this.connections = connections; + this.bindable = bindable; + this.listError = ""; + this.listed = true; + } catch (error) { + if (this.alive && load === this.loads) + this.listError = describeError(error).message; + } finally { + if (this.alive) this.cdr.markForCheck(); + } + } + get visible(): Row[] { + if (this.bindable) + return this.bindable.map((item) => ({ + connection: fromBindable(item), + card: cardOfBindable(item, this.endpoints), + readOnly: true, + })); + return this.connections.map((connection) => ({ + connection, + card: cardOf(connection, this.endpoints), + readOnly: false, + })); + } + get ready() { + return !!this.readiness && allReady(this.readiness); + } + get rows(): ReadinessRow[] { + if (!this.readiness) return []; + const latest = [...this.host().aiTests.values()] + .filter( + (record) => + record.result.ok && + this.connections.some((c) => c.id === record.revision), + ) + .sort((a, b) => b.at - a.at)[0]; + return readinessRows( + this.readiness, + (role) => this.host().roleLabel(role), + latest?.result, + ); + } + get actions() { + const host = this.host(); + return this.rows.map((row) => + fixAction(row.fix, (capability) => host.can(capability), row.id), + ); + } + count(card: ProviderCard) { + return this.visible.filter((row) => row.card === card).length; + } + cardAction(card: ProviderCard): string { + if (this.canManage()) return this.count(card) ? "Manage" : "Connect"; + return this.count(card) ? "View" : ""; + } + providerLabel(row: Row) { + const title = cardInfo(row.card).title; + const provider = row.connection.config.provider; + return row.card === "openai" || row.card === "azure" + ? `${title} · ${provider.endsWith("-chat") ? "Chat Completions" : "Responses"}` + : title; + } + developmentOnly(row: Row) { + const origin = row.connection.config.endpoint; + return ( + endpointFor(origin, this.endpoints)?.development_only ?? + this.endpoints.some( + (endpoint) => + endpoint.development_only && endpoint.url.startsWith(origin + "/"), + ) + ); + } + testText(row: Row) { + const record = this.host().aiTests.get(row.connection.id); + if (!record) return "Not tested in this session"; + const result = record.result; + if (!result.ok) return `Last test failed · ${result.code}`; + return `${result.model}${result.latency_ms === null ? "" : ` · ${(result.latency_ms / 1000).toFixed(1)} s`}`; + } + testTone(row: Row) { + const record = this.host().aiTests.get(row.connection.id); + return record ? (record.result.ok ? "success" : "danger") : null; + } + openCard(card: ProviderCard) { + const first = this.visible.find((row) => row.card === card); + this.host().openAiConnectionDialog({ + card, + connection: first?.connection ?? null, + }); + } + openRow(row: Row, focus?: "model") { + this.host().openAiConnectionDialog({ + card: row.card, + connection: row.connection, + focus, + }); + } + menu(row: Row): RowMenuItem[] { + return [ + { label: "Edit connection", run: () => this.openRow(row) }, + { + label: "Copy connection ID", + run: () => void this.copy(row.connection.id), + }, + ]; + } + async act(row: ReadinessRow) { + const fix = row.fix; + const host = this.host(); + const setup = this.setup; + if (!fix || !setup || this.running) return; + if (fix.kind === "navigate") { + if (fix.route === "operate/workers") await host.navigate("workers"); + else + this.focus( + row.id === "weave_ai" ? "#weave-ai-title" : "#ai-providers-title", + ); + return; + } + if (fix.kind !== "operation") return; + if (fix.operation === "ai_connections.test") { + const target = + this.connections.find( + (connection) => !host.aiTests.get(connection.id)?.result.ok, + ) ?? this.connections[0]; + if (target) + host.openAiConnectionDialog({ + card: cardOf(target, this.endpoints), + connection: target, + focus: "model", + }); + return; + } + if (fix.operation === "lumi.configuration.write") { + this.focus("#weave-ai-title"); + return; + } + this.running = row.id; + this.failures = {}; + try { + if (fix.operation === "ai_setup.publish") { + await setup.publish(); + this.announcement = "Published the AI actions."; + } else if (fix.operation === "ai_setup.grant") { + const targets = this.connections.filter( + (connection) => !keylessConnection(connection), + ); + // One refused grant doesn't stop the others; the checklist shows + // what was granted before the refusal is reported. + let refused: unknown = null; + for (const connection of targets) + await setup.grant(connection.id).catch((error: unknown) => { + refused ??= error; + }); + if (refused) { + await this.recheck(); + throw refused; + } + this.announcement = + targets.length === 1 + ? `Authorized ${targets[0].name}.` + : `Authorized ${targets.length} AI connections.`; + } + host.notify(this.announcement); + await this.recheck(); + } catch (error) { + const plain = describeError(error); + if (this.alive) + this.failures = { + [row.id]: plain.code + ? `${plain.message} (${plain.code})` + : plain.message, + }; + } finally { + if (this.alive) { + this.running = ""; + this.cdr.markForCheck(); + } + } + } + async copy(text: string) { + try { + await navigator.clipboard.writeText(text); + this.announcement = "Copied."; + } catch { + this.announcement = "Copy isn't available here. Select the text instead."; + } + this.cdr.markForCheck(); + } + private focus(selector: string) { + this.element.nativeElement.querySelector(selector)?.focus(); + } +} diff --git a/studio/src/app/settings/ai-readiness-list.ts b/studio/src/app/settings/ai-readiness-list.ts new file mode 100644 index 00000000..078894d8 --- /dev/null +++ b/studio/src/app/settings/ai-readiness-list.ts @@ -0,0 +1,170 @@ +/* +Copyright 2026 Firefly Software Foundation. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +Author: Firefly Software Foundation +SPDX-License-Identifier: Apache-2.0 +*/ +// The AI readiness checklist: one row per fact the platform checked, with the +// fix the person may run. The page runs the fixes; this list only shows them. +import { + ChangeDetectionStrategy, + Component, + input, + output, +} from "@angular/core"; +import { Icon } from "../icon"; +import type { FixAction, ReadinessRow } from "../integrations/ai-models-model"; + +@Component({ + selector: "weave-ai-readiness-list", + standalone: true, + changeDetection: ChangeDetectionStrategy.Eager, + imports: [Icon], + template: `
    + @for (row of rows(); track row.id; let index = $index) { + @let action = actions()[index]; +
  • + + {{ row.label }} · {{ row.detail }} + @if (action) { + @if (action.kind === "button") { + + } @else if (action.kind === "link") { + + } @else if (action.kind === "copy") { + + + @for (word of words(action.command); track $index) {{{ word }}{{ $last ? "" : " " }}} + + + } @else { + {{ action.label }} + } + } + @if (failures()[row.id]; as failure) { + + } +
    • + } +
`, + styles: ` + .readiness-list { + list-style: none; + margin: 0; + padding: 0; + display: grid; + } + .readiness-row { + display: grid; + grid-template-columns: 16px minmax(0, 1fr) auto; + align-items: center; + column-gap: var(--space-2); + row-gap: var(--space-1); + min-height: var(--control-sm); + padding: var(--space-1) 0; + border-bottom: 1px solid var(--line); + } + .readiness-icon[data-tone="success"] { + color: var(--success-ink); + } + .readiness-icon[data-tone="warning"] { + color: var(--warning-ink); + } + .readiness-icon[data-tone="neutral"] { + color: var(--muted); + } + .readiness-text { + min-width: 0; + overflow-wrap: anywhere; + } + .readiness-detail, + .readiness-ask { + color: var(--muted); + } + .readiness-ask { + font: var(--type-small); + } + .readiness-command, + .readiness-error { + grid-column: 2 / -1; + margin: 0; + } + .readiness-command { + display: flex; + align-items: center; + gap: var(--space-2); + min-width: 0; + } + .readiness-command code { + flex: 1 1 auto; + min-width: 0; + overflow-wrap: anywhere; + font: var(--type-mono); + background: var(--sunken); + padding: var(--space-1) var(--space-2); + border-radius: var(--radius-xs); + } + /* A command wraps between its words, never after a word's hyphen. */ + .command-word { + white-space: nowrap; + } + @media (max-width: 767px), (pointer: coarse) { + .readiness-row { + min-height: var(--control-touch); + } + } + `, +}) +export class AiReadinessList { + rows = input.required(); + actions = input.required<(FixAction | null)[]>(); + /** The item whose fix is running, or "". */ + running = input(""); + failures = input>({}); + act = output(); + copy = output(); + /** A command's words, shown so a line never breaks inside one. */ + words(command: string) { + return command.split(/\s+/).filter(Boolean); + } +} diff --git a/studio/src/app/settings/settings-page.css b/studio/src/app/settings/settings-page.css index a823eeea..b2729374 100644 --- a/studio/src/app/settings/settings-page.css +++ b/studio/src/app/settings/settings-page.css @@ -54,14 +54,23 @@ SPDX-License-Identifier: Apache-2.0 border-radius: 2px 2px 0 0; background: var(--accent); } -/* On a phone three tabs share one row: each shrinks to its longest word and - wraps the rest of its label, so the strip never widens the page. */ +/* On a phone the tabs wrap into rows that fill the strip, so four tabs never + widen the page; each tab draws its own bottom line, so every row has one. */ @media (max-width: 767px) { + .settings-tabs { + flex-wrap: wrap; + gap: 0; + border-bottom: 0; + } .settings-tabs button { - flex: 0 1 auto; + flex: 1 1 auto; padding: var(--space-1) var(--space-2); white-space: normal; line-height: 1.25; + box-shadow: inset 0 -1px 0 var(--line); + } + .settings-tabs button[aria-selected="true"]::after { + bottom: 0; } } .people-layout { diff --git a/studio/src/app/settings/settings-page.ts b/studio/src/app/settings/settings-page.ts index 49b22610..4e679bd7 100644 --- a/studio/src/app/settings/settings-page.ts +++ b/studio/src/app/settings/settings-page.ts @@ -16,8 +16,9 @@ Author: Firefly Software Foundation SPDX-License-Identifier: Apache-2.0 */ // Settings: Platforms (saved platforms, how Studio starts, keyboard controls), -// People and access for administrators, and Preferences, each tab at its own -// address (/settings/). The shell (App) +// People and access for administrators, AI models once a platform is +// connected, and Preferences, each tab at its own address (/settings/). +// The shell (App) // owns the state and the commands; this page loads lazily (@defer). import { ChangeDetectionStrategy, @@ -32,6 +33,7 @@ import { ModalSheet, sheetWhen } from "../modal-sheet"; import { PlatformList } from "../platform-list"; import { RowMenu, type RowMenuItem } from "../row-menu"; import { statusLabel } from "../status-labels"; +import { AiModelsPage } from "./ai-models-page"; import { settingsPath, settingsTab, @@ -55,7 +57,7 @@ interface AccountRow { selector: "weave-settings-page", changeDetection: ChangeDetectionStrategy.Eager, standalone: true, - imports: [Icon, PlatformList, RowMenu, ModalSheet], + imports: [Icon, PlatformList, RowMenu, ModalSheet, AiModelsPage], styleUrl: "./settings-page.css", template: `@let h = host();
@@ -66,40 +68,6 @@ interface AccountRow {

- @if (h.profile) { -
-

AI models

-

- Provider connections are shared environment resources. Weave AI uses - its own model settings; workflow AI profiles stay in each workflow. -

- @if (h.can("connection.manage")) { - - } - @if (h.can("lumi.manage")) { - - } - @if (!h.can("lumi.manage") || !h.can("connection.manage")) { -

- Ask an administrator for Weave AI manager and connection.manage - grants in this environment to set up Weave AI and its provider - connection. -

- } -

- For a workflow model, add an AI task and open Configure workflow AI - profiles. A platform operator installs the Agentic worker and Weave AI - gateway and provisions approved secret handles. -

-
- } @let current = tab();
}
+ } @else if (current === "ai") { +
+ @defer (on immediate) { + + } @placeholder { +

Loading AI models…

+ } +
} @else {
tab.id); - const index = order.indexOf(this.tab()); + // Arrows move from the focused tab, which may differ from the open one. + const focused = (event.target as HTMLElement | null)?.id?.replace( + /^settings-tab-/, + "", + ); + const from = order.includes(focused as SettingsTab) + ? (focused as SettingsTab) + : this.tab(); + const index = order.indexOf(from); const next = event.key === "Home" ? 0 diff --git a/studio/src/app/settings/settings-routes.ts b/studio/src/app/settings/settings-routes.ts index 32475d5c..1b8bd71e 100644 --- a/studio/src/app/settings/settings-routes.ts +++ b/studio/src/app/settings/settings-routes.ts @@ -15,34 +15,43 @@ limitations under the License. Author: Firefly Software Foundation SPDX-License-Identifier: Apache-2.0 */ -// Settings has one address per tab: /settings/platforms, /settings/people -// and /settings/preferences; /settings opens Platforms. People and access -// shows only to people who can manage members or grants. +// Settings has one address per tab: /settings/platforms, /settings/people, +// /settings/ai and /settings/preferences; /settings opens Platforms. People +// and access shows only to people who can manage members or grants; AI models +// shows once a platform is connected. import type { Routes } from "@angular/router"; -export type SettingsTab = "platforms" | "people" | "preferences"; +export type SettingsTab = "platforms" | "people" | "ai" | "preferences"; export const SETTINGS_TABS: readonly { id: SettingsTab; label: string }[] = [ { id: "platforms", label: "Platforms" }, { id: "people", label: "People and access" }, + { id: "ai", label: "AI models" }, { id: "preferences", label: "Preferences" }, ]; +/** Which optional tabs this person sees. */ +export interface SettingsAccess { + people: boolean; + ai: boolean; +} export const settingsPath = (tab: SettingsTab): string => `/settings/${tab}`; /** The tabs this person sees, in order. */ -export function settingsTabs(canSeePeople: boolean) { - return SETTINGS_TABS.filter((tab) => tab.id !== "people" || canSeePeople); +export function settingsTabs(access: SettingsAccess) { + return SETTINGS_TABS.filter( + (tab) => + (tab.id !== "people" || access.people) && (tab.id !== "ai" || access.ai), + ); } /** The tab an address opens; an unknown or hidden tab opens Platforms. */ export function settingsTab( pathname: string, - canSeePeople: boolean, + access: SettingsAccess, ): SettingsTab { const segment = pathname.split("/")[2] ?? ""; return ( - settingsTabs(canSeePeople).find((tab) => tab.id === segment)?.id ?? - "platforms" + settingsTabs(access).find((tab) => tab.id === segment)?.id ?? "platforms" ); } diff --git a/studio/src/app/settings/weave-ai-card.ts b/studio/src/app/settings/weave-ai-card.ts new file mode 100644 index 00000000..d522a50c --- /dev/null +++ b/studio/src/app/settings/weave-ai-card.ts @@ -0,0 +1,405 @@ +/* +Copyright 2026 Firefly Software Foundation. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +Author: Firefly Software Foundation +SPDX-License-Identifier: Apache-2.0 +*/ +// Weave AI's settings in Settings › AI models: on or off, the AI connection +// and the model. Its limits (output tokens, calls, time) keep their saved +// values; a new configuration starts from Weave AI's defaults. +import { + ChangeDetectionStrategy, + ChangeDetectorRef, + Component, + OnDestroy, + DoCheck, + inject, + input, + output, +} from "@angular/core"; +import { ApiError, type StudioApi } from "../api"; +import { describeError } from "../errors"; +import { Select } from "../forms/ui/select"; +import type { AiConnection } from "../integrations/ai-provider-connection"; +import { + FieldError, + defaultModel, + keylessConnection, + modelOptions, + weaveAiBody, + weaveAiChoices, + type WeaveAiDraft, +} from "../integrations/ai-models-model"; +import { + AiSetupApi, + type AiModels, + type LumiConfiguration, + type LumiStatus, +} from "../integrations/ai-setup-api"; + +@Component({ + selector: "weave-ai-card", + standalone: true, + changeDetection: ChangeDetectionStrategy.Eager, + imports: [Select], + template: `
+

Weave AI

+ @if (status) { + {{ + status.configured ? "Set up · " + status.model : "Not set up" + }} + } +
+

+ Weave AI answers questions about your workflows and runs, and drafts + changes for you to review. +

+ @if (!manager()) { +

+ {{ + canUse() + ? "Configuring Weave AI requires the Weave AI manager role." + : "Using Weave AI requires the Weave AI user role." + }} +

+ } @else if (!canChoose()) { +

+ Choosing Weave AI's connection also requires the Tenant administrator + role. +

+ } @else if (loading) { +

Loading Weave AI settings…

+ } @else { +
+ +
+ + +
+ @if (connectionNote; as note) { +

{{ note }}

+ } + @if (!usable.length) { +

+ Weave AI needs an AI connection with a secret handle. Connections + without a key can't be used for Weave AI yet. +

+ } + @if (error) { + + } +
+ +
+
+ } +

{{ announcement }}

`, + styles: ` + :host { + display: block; + } + .panel-heading { + display: flex; + align-items: center; + justify-content: space-between; + gap: var(--space-2); + min-height: var(--control-md); + margin-bottom: var(--space-3); + } + h2 { + margin: 0; + font: var(--type-title-sm); + } + .card-intro, + .empty { + margin: 0 0 var(--space-4); + color: var(--muted); + } + .weave-ai-form { + display: grid; + gap: var(--space-4); + } + .weave-ai-fields { + display: grid; + gap: var(--space-4); + grid-template-columns: repeat(auto-fit, minmax(220px, 1fr)); + } + .card-actions { + display: flex; + justify-content: flex-end; + } + /* The dropdowns match the buttons: 36 px, or 44 px for touch. */ + @media (max-width: 767px), (pointer: coarse) { + .weave-ai-fields weave-select { + --select-height: var(--control-touch); + } + } + `, +}) +export class WeaveAiCard implements DoCheck, OnDestroy { + api = input.required(); + /** lumi.manage */ + manager = input(false); + /** lumi.manage and connection.manage: saving reads the chosen connection. */ + canChoose = input(false); + /** lumi.use */ + canUse = input(false); + connections = input([]); + /** The shell's AI setup generation: the card reloads when it changes. */ + generation = input(0); + saved = output(); + private readonly cdr = inject(ChangeDetectorRef); + private setup!: AiSetupApi; + private replySchema: unknown = null; + private alive = true; + loading = true; + busy = false; + error = ""; + announcement = ""; + status: LumiStatus | null = null; + current: LumiConfiguration | null = null; + models: AiModels | null = null; + draft: WeaveAiDraft = { enabled: true, connection: "", model: "" }; + /** The saved connection revision, read when the list may not hold it. */ + pinned: AiConnection | null = null; + /** The saved connection revision could not be read. */ + pinnedMissing = false; + + /** + * The connection Save uses: the chosen one, or for a saved revision that + * is no longer the newest, the newest revision of that connection. + */ + get chosen(): AiConnection | null { + const id = this.draft.connection; + if (!id) return null; + const listed = this.connections().find((item) => item.id === id); + if (listed) return listed; + const newest = this.newest; + return newest && !keylessConnection(newest) ? newest : null; + } + /** The newest revision of the saved connection, when the list has a newer one. */ + private get newest(): AiConnection | null { + const pinned = this.pinned; + if (!pinned || pinned.id !== this.draft.connection) return null; + return ( + this.connections().find( + (item) => item.name === pinned.name && item.id !== pinned.id, + ) ?? null + ); + } + /** Why the saved connection is not the one shown, when it isn't. */ + get connectionNote(): string { + const id = this.draft.connection; + if (!id || this.connections().some((item) => item.id === id)) return ""; + const newest = this.newest; + if (newest && this.pinned) + return keylessConnection(newest) + ? `The newest revision of ${newest.name} (revision ${newest.revision}) has no key, and Weave AI can't use a connection without a key yet. Choose a connection with a key, then save.` + : `Weave AI uses revision ${this.pinned.revision} of ${this.pinned.name}; saving moves it to revision ${newest.revision}.`; + if (this.pinnedMissing || (this.pinned && this.connections().length)) + return "Weave AI's connection is no longer available. Choose a connection, then save."; + return ""; + } + + get choices() { + return weaveAiChoices(this.connections()); + } + get usable() { + return this.connections().filter( + (connection) => !keylessConnection(connection), + ); + } + get modelChoices() { + return modelOptions(this.models?.models ?? []); + } + checked(event: Event) { + return (event.target as HTMLInputElement).checked; + } + /** The access the card last loaded for; capabilities can arrive after it renders. */ + private loadedFor = ""; + private loads = 0; + ngDoCheck() { + const key = `${this.generation()}:${this.canUse()}:${this.manager()}:${this.canChoose()}`; + if (key !== this.loadedFor) { + this.loadedFor = key; + void this.load(); + return; + } + // The list of connections can arrive after the card loaded: list the + // models of the revision Save will use once it is known. + const target = this.loading ? "" : (this.chosen?.id ?? ""); + if (target && target !== this.modelsFor) void this.loadModels(target); + } + private async load() { + const load = ++this.loads; + this.setup = new AiSetupApi(this.api()); + this.loading = true; + try { + if (this.canUse() || this.manager()) + this.status = await this.setup.lumiStatus().catch(() => null); + if (this.canChoose()) { + const [current, reply] = await Promise.all([ + this.setup.lumiConfiguration(), + this.setup.lumiReplySchema(), + ]); + const pinnedId = current?.connection_revision_id ?? ""; + let pinned: AiConnection | null = null; + let unreadable = ""; + if (pinnedId) + try { + pinned = await this.setup.connection(pinnedId); + } catch (error) { + // Only a missing revision means it is gone; any other failure is + // reported as it is, and Save waits for a connection. + if (!(error instanceof ApiError && error.status === 404)) + unreadable = describeError(error).message; + } + if (!this.alive || load !== this.loads) return; + this.current = current; + this.pinned = pinned; + this.pinnedMissing = !!pinnedId && !pinned && !unreadable; + this.error = unreadable + ? `Studio couldn't read the connection Weave AI uses. ${unreadable}` + : ""; + this.replySchema = reply; + this.draft = { + enabled: current?.enabled ?? true, + connection: current?.connection_revision_id ?? "", + model: String(current?.profile?.["model"] ?? ""), + }; + this.modelsFor = ""; + const target = this.chosen; + if (target) await this.loadModels(target.id); + } + } catch (error) { + if (this.alive && load === this.loads) + this.error = describeError(error).message; + } finally { + if (this.alive && load === this.loads) { + this.loading = false; + this.cdr.markForCheck(); + } + } + } + ngOnDestroy() { + this.alive = false; + } + async chooseConnection(id: string) { + if (id === this.draft.connection) return; + this.draft = { ...this.draft, connection: id, model: "" }; + await this.loadModels(id); + } + /** The connection revision whose models were last asked for. */ + private modelsFor = ""; + private async loadModels(id: string) { + this.modelsFor = id; + try { + const models = await this.setup.models(id); + if (!this.alive || this.modelsFor !== id) return; + this.models = models; + this.draft = { + ...this.draft, + model: defaultModel(models.models, this.draft.model), + }; + } catch (error) { + if (this.alive) this.error = describeError(error).message; + } finally { + if (this.alive) this.cdr.markForCheck(); + } + } + async save() { + if (this.busy) return; + const connection = this.chosen; + if (!connection) { + this.error = "Choose a connection for Weave AI."; + return; + } + this.error = ""; + let body: Record; + try { + body = weaveAiBody( + this.draft, + connection, + this.current, + this.replySchema, + ); + } catch (error) { + if (error instanceof FieldError) { + this.error = error.message; + return; + } + throw error; + } + this.busy = true; + try { + this.current = await this.setup.saveLumi( + body, + this.current?.revision ?? null, + ); + if (!this.alive) return; + this.status = await this.setup.lumiStatus().catch(() => this.status); + this.announcement = "Saved Weave AI settings."; + this.saved.emit(); + } catch (error) { + if (!this.alive) return; + this.error = + error instanceof ApiError && error.code === "WV-LUMI-REVISION" + ? "Weave AI settings changed since you opened them. Reload the page to see them, then save again." + : describeError(error).message; + } finally { + if (this.alive) { + this.busy = false; + this.cdr.markForCheck(); + } + } + } +} diff --git a/studio/src/styles.css b/studio/src/styles.css index 7ab1ab09..f86ab9b6 100644 --- a/studio/src/styles.css +++ b/studio/src/styles.css @@ -1111,6 +1111,33 @@ dd { .modal-panel.wide { width: min(900px, 100%); } +/* A side sheet: the same dialog docked at the right edge, full height. */ +.modal-backdrop.side { + justify-content: flex-end; + align-items: stretch; + padding: 0; +} +.modal-panel.side { + width: min(480px, 100%); + max-height: none; + height: 100%; + border-radius: 0; + border-left: 1px solid var(--border); +} +@media (max-width: 767px) { + .modal-panel.side { + width: 100%; + border-left: 0; + } +} +/* The sheet's content grows, so its footer rests on the bottom edge. */ +.modal-panel.side { + display: flex; + flex-direction: column; +} +.modal-panel.side > :not(.modal-header) { + flex: 1 0 auto; +} .modal-panel:focus { outline: none; } @@ -1218,6 +1245,14 @@ dd { margin-inline: 0; } } +/* A side sheet's footer holds two answers at most: they share a row while + the sheet is wider than a phone. */ +@container dialog (min-width: 360px) { + .modal-panel.side :is(.dialog-actions, .dialog-footer) { + flex-direction: row; + align-items: center; + } +} /* Notices: warning by default; data-tone="success" | "danger" | "info" | "neutral". .error-notice is the danger tone under its old name. */ .notice { diff --git a/studio/tests/ai-models-model.test.ts b/studio/tests/ai-models-model.test.ts new file mode 100644 index 00000000..e6840bc8 --- /dev/null +++ b/studio/tests/ai-models-model.test.ts @@ -0,0 +1,698 @@ +/* +Copyright 2026 Firefly Software Foundation. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +Author: Firefly Software Foundation +SPDX-License-Identifier: Apache-2.0 +*/ +import { execFileSync } from "node:child_process"; +import { describe, expect, it } from "vitest"; +import { ApiError } from "../src/app/api"; +import type { AiConnection } from "../src/app/integrations/ai-provider-connection"; +import type { + AiEndpoint, + AiModel, + Readiness, +} from "../src/app/integrations/ai-setup-api"; +import { + AI_COPY, + FieldError, + PLAIN_HTTP_REFUSED, + aiMessage, + allReady, + cardOf, + connectionRequest, + defaultModel, + draftFor, + fixAction, + keyless, + secretSetCommand, + modelOptions, + presetsFor, + readinessRows, + refusal, + sheetProblems, + testView, + weaveAiBody, + weaveAiChoices, +} from "../src/app/integrations/ai-models-model"; +import { python, pythonAvailable } from "./python-path"; + +const ollama: AiEndpoint = { + id: "ollama-local", + label: "Ollama (Weave-managed)", + url: "http://ollama:11434/v1", + providers: ["openai-chat"], + compat: "ollama", + credential: "none", + models: "served", + context_tokens: 8192, + development_only: true, +}; +const openai: AiEndpoint = { + id: "openai", + label: "OpenAI", + url: "https://api.openai.com/v1", + providers: ["openai-chat", "openai-responses"], + compat: null, + credential: "required", + models: "listed", + context_tokens: 128000, + development_only: false, +}; +const endpoints = [ollama, openai]; +const keylessConnection: AiConnection = { + id: "r1", + name: "ollama-local", + revision: 1, + connector: "weave-agentic-provider@1.0.0", + config: { provider: "openai-chat", endpoint: "http://ollama:11434/v1" }, + secretRef: { apiKey: "no-credential" }, +}; +const cloudConnection: AiConnection = { + id: "r2", + name: "openai", + revision: 3, + connector: "weave-agentic-provider@1.0.0", + config: { + provider: "openai-responses", + endpoint: "https://api.openai.com/v1", + }, + secretRef: { apiKey: "openai-api-key" }, +}; +const qwen: AiModel = { + name: "qwen3:4b", + approved: true, + available: true, + tools: "yes", + context_tokens: 8192, + size_bytes: 2_500_000_000, +}; +const gemma: AiModel = { + name: "gemma3:270m", + approved: true, + available: true, + tools: "no", + context_tokens: 8192, + size_bytes: 291_000_000, +}; + +describe("provider presets and connection requests", () => { + it("offers only Ollama endpoints on the Ollama card and cloud endpoints elsewhere", () => { + expect(presetsFor("ollama", endpoints)).toEqual([ollama]); + expect(presetsFor("openai", endpoints)).toEqual([openai]); + expect(presetsFor("anthropic", endpoints)).toEqual([]); + }); + it("builds a keyless Ollama connection from an approved preset", () => { + const draft = draftFor("ollama", endpoints); + expect(keyless(draft, endpoints)).toBe(true); + expect(connectionRequest(draft, "connector", endpoints, "loaded")).toEqual({ + name: "ollama-local", + connector_version_id: "connector", + config: { + provider: "openai-chat", + endpoint: "http://ollama:11434/v1", + secretSlot: "apiKey", + }, + secretRef: { apiKey: "no-credential" }, + allowed_destinations: ["http://ollama:11434"], + }); + }); + it("shows the operator command only for a handle, never for a pasted key", () => { + expect(secretSetCommand("team-openai")).toBe( + "weave platform secret set --handle team-openai", + ); + const key = `sk-proj-${"Ab3dE".repeat(19)}x`; + expect(secretSetCommand(key)).toBe( + "weave platform secret set --handle HANDLE", + ); + expect(secretSetCommand("")).toBe( + "weave platform secret set --handle HANDLE", + ); + }); + it("asks for a handle on a custom address even when it equals a keyless preset", () => { + const draft = { + ...draftFor("ollama", endpoints), + custom: true, + endpoint: "http://ollama:11434/v1", + }; + expect(keyless(draft, endpoints)).toBe(false); + }); + it.each([ + "http://10.0.0.5:11434/v1", + "http://localhost:11434/v1", + "http://ollama.example.com/v1", + ])("refuses plain HTTP that no preset approves: %s", (endpoint) => { + const draft = { ...draftFor("ollama", endpoints), custom: true, endpoint }; + expect(() => + connectionRequest(draft, "connector", endpoints, "loaded"), + ).toThrow(PLAIN_HTTP_REFUSED); + }); + it("asks for a secret handle on credentialed endpoints and never sends the reserved one", () => { + const draft = draftFor("openai", endpoints); + expect(draft).toMatchObject({ + name: "openai", + provider: "openai-responses", + endpoint: "https://api.openai.com/v1", + handle: "openai-api-key", + }); + expect( + connectionRequest(draft, "connector", endpoints, "loaded").secretRef, + ).toEqual({ apiKey: "openai-api-key" }); + expect( + connectionRequest( + { ...draft, handle: "a".repeat(64) }, + "connector", + endpoints, + "loaded", + ).secretRef, + ).toEqual({ apiKey: "a".repeat(64) }); + // The platform's handle rule: 1 to 64 lowercase letters, digits, ".", "_" + // or "-". Real keys (mixed case, long) never leave the browser. + for (const handle of [ + "no-credential", + "sk live key", + "", + `sk-proj-${"Ab3dE".repeat(19)}x`, + `sk-ant-api03-${"Zq9".repeat(30)}`, + "OpenAI-Key", + "a".repeat(65), + ]) { + const error = (() => { + try { + connectionRequest( + { ...draft, handle }, + "connector", + endpoints, + "loaded", + ); + } catch (caught) { + return caught; + } + })(); + expect(error).toBeInstanceOf(FieldError); + expect((error as FieldError).field).toBe("handle"); + } + }); + it("accepts only approved cloud endpoints while a policy is loaded", () => { + const draft = { + ...draftFor("openai", endpoints), + custom: true, + endpoint: "https://proxy.example.com/v1", + }; + expect(() => + connectionRequest(draft, "connector", endpoints, "loaded"), + ).toThrow("approved endpoint"); + expect( + connectionRequest(draft, "connector", endpoints, "absent").config + .endpoint, + ).toBe("https://proxy.example.com/v1"); + }); + it.each([ + "https://user:password@model.example", + "https://model.example/?api-key=secret", + "https://model.example/#secret", + "ftp://model.example", + ])("refuses an endpoint carrying anything but an address: %s", (endpoint) => { + const draft = { ...draftFor("openai", []), custom: true, endpoint }; + expect(() => connectionRequest(draft, "c", [], "absent")).toThrow( + FieldError, + ); + }); + it("requires Azure's API version and keeps it only for Azure", () => { + const azure = { + ...draftFor("azure", []), + endpoint: "https://team.openai.azure.com", + custom: true, + }; + expect(() => connectionRequest(azure, "c", [], "absent")).toThrow( + "API version", + ); + expect( + connectionRequest( + { ...azure, apiVersion: "2025-04-01-preview" }, + "c", + [], + "absent", + ).config, + ).toMatchObject({ + provider: "azure-responses", + apiVersion: "2025-04-01-preview", + }); + }); + it("needs the published AI connector before saving", () => { + expect(() => + connectionRequest(draftFor("ollama", endpoints), "", endpoints, "loaded"), + ).toThrow("Publish"); + }); + it("puts each server problem on its field", () => { + const detail = { + code: "WV-CONNECTION", + result: { + diagnostics: [ + { + path: "/config/endpoint", + code: "WV-CONNECTION-CONFIG", + message: "Plain HTTP needs a private-origin entry.", + }, + { + path: "/secretRef/apiKey", + code: "WV-CONNECTION-SECRET", + message: "The secret handle is not available.", + }, + { + path: "/name", + code: "WV-CONNECTION-CONFIG", + message: "Choose another name.", + }, + ], + }, + }; + expect(sheetProblems(detail).map((p) => p.field)).toEqual([ + "endpoint", + "handle", + "name", + ]); + }); + it("knows which card a saved connection belongs to", () => { + expect(cardOf(keylessConnection, endpoints)).toBe("ollama"); + expect(cardOf(cloudConnection, endpoints)).toBe("openai"); + expect( + cardOf( + { + ...cloudConnection, + config: { + provider: "anthropic", + endpoint: "https://api.anthropic.com", + }, + }, + endpoints, + ), + ).toBe("anthropic"); + }); +}); + +describe("models", () => { + it("suggests qwen3:4b, then a tool-capable model, and never one that is not approved or installed", () => { + expect(defaultModel([gemma, qwen])).toBe("qwen3:4b"); + expect(defaultModel([gemma, { ...qwen, name: "llama3.2:3b" }])).toBe( + "llama3.2:3b", + ); + expect(defaultModel([{ ...qwen, approved: false }, gemma])).toBe( + "gemma3:270m", + ); + expect(defaultModel([{ ...qwen, available: false }])).toBe(""); + expect(defaultModel([gemma, qwen], "gemma3:270m")).toBe("gemma3:270m"); + }); + it("describes each model with size, tools and context, and disables the rest", () => { + expect(modelOptions([qwen, { ...gemma, approved: false }])).toEqual([ + { + value: "qwen3:4b", + label: "qwen3:4b", + description: "2.5 GB · tools · 8k context", + disabled: false, + }, + { + value: "gemma3:270m", + label: "gemma3:270m", + description: "Not approved", + disabled: true, + }, + ]); + }); +}); + +describe("test states", () => { + const origin = "http://ollama:11434"; + it("reads a tool-capable success, a no-tools warning and a failure", () => { + expect( + testView( + { + kind: "done", + origin, + result: { + ok: true, + code: "ok", + latency_ms: 2400, + model: "qwen3:4b", + tool_calling: "supported", + }, + }, + "ollama-local", + ).text, + ).toBe("Connected. qwen3:4b answered in 2.4 s and can call tools."); + const warning = testView( + { + kind: "done", + origin, + result: { + ok: true, + code: "ok", + latency_ms: 800, + model: "gemma3:270m", + tool_calling: "unsupported", + }, + }, + "ollama-local", + ); + expect(warning).toMatchObject({ + tone: "warning", + text: "Connected. gemma3:270m answered in 0.8 s but cannot call tools. Use it for AI tasks, not AI agents.", + }); + const failed = testView( + { + kind: "done", + origin, + result: { + ok: false, + code: "LLM_MODEL_NOT_FOUND", + latency_ms: 30, + model: "qwen3:4b", + tool_calling: "unknown", + }, + }, + "ollama-local", + ); + expect(failed.text).toBe( + "qwen3:4b is not installed on http://ollama:11434. Pull it with `ollama pull qwen3:4b`.", + ); + expect(failed.details).toEqual([ + ["Code", "LLM_MODEL_NOT_FOUND"], + ["Endpoint", "http://ollama:11434"], + ["Latency", "30 ms"], + ]); + expect( + testView({ kind: "running", model: "qwen3:4b" }, "ollama-local").text, + ).toBe("Calling qwen3:4b on ollama-local…"); + expect(testView({ kind: "idle" }, "x").text).toBe("Not tested"); + }); + it("names the gateway's test budget when a model does not answer in time", () => { + const timeout = testView( + { + kind: "done", + origin, + result: { + ok: false, + code: "LLM_TIMEOUT", + latency_ms: null, + model: "qwen3:4b", + tool_calling: "unknown", + }, + }, + "ollama-local", + ); + expect(timeout.text).toContain("120 seconds"); + }); + it("says there was no response when the request never reached the platform", () => { + const offline = refusal(new TypeError("Failed to fetch"), origin); + expect(testView(offline, "x").details).toContainEqual([ + "HTTP status", + "No response", + ]); + }); + it("explains a missing gateway and the test limit without response bodies", () => { + const missing = refusal( + new ApiError(503, { + code: "WV-AI-GATEWAY-MISSING", + message: "secret detail from a body", + }), + origin, + ); + expect(missing).toMatchObject({ + kind: "refused", + message: + "Tests need the AI gateway. Run `weave platform ai enable`, or ask an operator to deploy it.", + }); + expect(JSON.stringify(testView(missing, "x"))).not.toContain( + "secret detail", + ); + expect(testView(missing, "x").details).toContainEqual([ + "HTTP status", + "5xx", + ]); + expect( + refusal(new ApiError(429, { code: "WV-AI-RATE-LIMITED" }), origin), + ).toMatchObject({ + message: "Too many AI connection tests. Wait a minute, then test again.", + }); + }); +}); + +describe.skipIf(!pythonAvailable())("error copy matches the platform", () => { + it("has the same text and placeholders as the server's table", () => { + const output = execFileSync( + python, + [ + "-c", + "import json; from firefly_weave.contracts.ai import AI_ERRORS, ai_message; " + + "print(json.dumps({'copy': {c: e.message for c, e in AI_ERRORS.items()}, " + + "'filled': {c: ai_message(c, model='qwen3:4b', endpoint='http://ollama:11434', n=8192) for c in AI_ERRORS}}))", + ], + { cwd: "..", encoding: "utf8" }, + ); + const server = JSON.parse(output) as { + copy: Record; + filled: Record; + }; + expect(AI_COPY).toEqual(server.copy); + for (const [code, text] of Object.entries(server.filled)) + expect( + aiMessage(code, { + model: "qwen3:4b", + endpoint: "http://ollama:11434", + n: 8192, + }), + ).toBe(text); + }); +}); + +describe("readiness", () => { + const item = (id: string, status: string, detail: string, fix?: object) => ({ + id, + status, + detail, + ...(fix ? { fix } : {}), + }); + const local = { + items: [ + item("worker_installed", "done", "worker_installed"), + item("actions_published", "action", "actions_missing", { + kind: "operation", + label: "Publish", + operation: "ai_setup.publish", + }), + item("worker_online", "done", "worker_online"), + item("connection", "done", "connection_available"), + item("key_available", "not_needed", "no_credential"), + item("connection_authorized", "not_needed", "no_credential"), + item("model_responds", "action", "test_required", { + kind: "operation", + label: "Test connection", + operation: "ai_connections.test", + }), + item("weave_ai", "unknown", "requires_lumi_user", { + kind: "ask", + label: "Requires lumi_user", + }), + ], + } as Readiness; + it("labels each item, its state and who may fix it", () => { + const rows = readinessRows(local, (role) => + role === "lumi_user" ? "Weave AI user" : role, + ); + expect(rows.map((row) => [row.label, row.detail, row.icon])).toEqual([ + ["Agentic worker installed", "Installed", "ok"], + ["AI actions published", "Not published yet", "warning"], + ["Worker online", "Online", "ok"], + ["AI connection", "Set up", "ok"], + ["Key available", "Not needed", "off"], + ["Connection authorized", "Not needed", "off"], + ["Model responds", "Not tested yet", "warning"], + ["Weave AI", "Requires Weave AI user", "help"], + ]); + expect( + readinessRows( + { + items: local.items.map((i) => + i.id === "model_responds" + ? { ...i, status: "done", detail: "test_passed" } + : i, + ), + } as Readiness, + (role) => role, + { model: "qwen3:4b", latency_ms: 2400 }, + )[6].detail, + ).toBe("qwen3:4b · 2.4 s"); + }); + it("is ready only when every item is done or not needed", () => { + expect(allReady(local)).toBe(false); + const done = { + items: local.items.map((i) => ({ + ...i, + status: i.status === "not_needed" ? "not_needed" : "done", + })), + } as Readiness; + expect(allReady(done)).toBe(true); + expect( + allReady( + { + items: done.items.map((i) => + i.id === "weave_ai" ? { ...i, status: "action" } : i, + ), + } as Readiness, + ["weave_ai"], + ), + ).toBe(true); + }); + it("names the card an AI settings link goes to on the page itself", () => { + const settings = { + kind: "navigate", + label: "Open AI settings", + route: "settings/ai", + } as const; + expect(fixAction(settings, () => true, "weave_ai")).toEqual({ + kind: "link", + label: "Go to Weave AI", + }); + expect(fixAction(settings, () => true, "connection")).toEqual({ + kind: "link", + label: "Go to providers", + }); + expect( + fixAction( + { kind: "navigate", label: "Open workers", route: "operate/workers" }, + () => true, + "worker_online", + ), + ).toEqual({ kind: "link", label: "Open workers" }); + }); + it("offers an operation only to people who may run it", () => { + const publish = { + kind: "operation", + label: "Publish", + operation: "ai_setup.publish", + } as const; + expect(fixAction(publish, () => true)).toEqual({ + kind: "button", + label: "Publish", + }); + expect(fixAction(publish, (c) => c !== "definition.publish")).toBeNull(); + expect( + fixAction( + { kind: "operation", label: "Authorize", operation: "ai_setup.grant" }, + (c) => c !== "connection.manage", + ), + ).toBeNull(); + expect( + fixAction( + { + kind: "command", + label: "Ask an operator to enable AI", + command: "weave platform ai enable --ollama auto", + }, + () => false, + ), + ).toEqual({ + kind: "copy", + label: "Ask an operator to enable AI", + command: "weave platform ai enable --ollama auto", + }); + expect( + fixAction({ kind: "ask", label: "Requires lumi_user" }, () => false), + ).toBeNull(); + expect( + fixAction( + { kind: "ask", label: "Ask an operator to set the connection secret" }, + () => false, + ), + ).toEqual({ + kind: "text", + label: "Ask an operator to set the connection secret", + }); + }); +}); + +describe("Weave AI", () => { + it("keeps the saved profile's limits and takes provider and model from the choice", () => { + const body = weaveAiBody( + { enabled: true, connection: "r2", model: "gpt-5-mini" }, + cloudConnection, + { + enabled: false, + connection_revision_id: "old", + revision: 4, + profile: { + provider: "anthropic", + model: "x", + options: { max_tokens: 2048 }, + reasoning: { pattern: "none", maxSteps: 6 }, + maxCalls: 4, + timeoutSeconds: 90, + outputSchema: { old: true }, + }, + }, + { type: "object" }, + ); + expect(body).toEqual({ + enabled: true, + connection_revision_id: "r2", + profile: { + options: { max_tokens: 2048 }, + reasoning: { pattern: "none", maxSteps: 6 }, + maxCalls: 4, + timeoutSeconds: 90, + provider: "openai-responses", + model: "gpt-5-mini", + outputSchema: { type: "object" }, + }, + }); + }); + it("uses 1,024 output tokens and three minutes for a new configuration", () => { + expect( + weaveAiBody( + { enabled: true, connection: "r2", model: "gpt-5-mini" }, + cloudConnection, + null, + {}, + ).profile, + ).toMatchObject({ + options: { max_tokens: 1024 }, + maxCalls: 8, + timeoutSeconds: 180, + }); + expect(() => + weaveAiBody( + { enabled: true, connection: "r2", model: " " }, + cloudConnection, + null, + {}, + ), + ).toThrow("Choose a model"); + }); + it("lists connections without a key as not usable for Weave AI yet", () => { + expect(weaveAiChoices([keylessConnection, cloudConnection])).toEqual([ + { + value: "r1", + label: "ollama-local · revision 1", + description: "Weave AI can't use a connection without a key yet", + disabled: true, + }, + { + value: "r2", + label: "openai · revision 3", + description: "https://api.openai.com/v1", + disabled: false, + }, + ]); + }); +}); diff --git a/studio/tests/ai-provider-connection.test.ts b/studio/tests/ai-provider-connection.test.ts index ed30d50c..ff6cc6fd 100644 --- a/studio/tests/ai-provider-connection.test.ts +++ b/studio/tests/ai-provider-connection.test.ts @@ -17,82 +17,12 @@ SPDX-License-Identifier: Apache-2.0 */ import { describe, expect, it } from "vitest"; import { - aiConnectionRequest, aiConnections, aiConnector, + latestByName, + secretRefs, } from "../src/app/integrations/ai-provider-connection"; -const draft = { - name: "azure-models", - provider: "azure-responses", - endpoint: "https://approved.openai.azure.com/", - apiVersion: "2025-04-01-preview", - handle: "model-key", -}; -describe("AI provider connection authority", () => { - it.each([ - ["openai-chat", "https://api.openai.com/v1"], - ["openai-responses", "https://api.openai.com/v1"], - ["anthropic", "https://api.anthropic.com"], - ])("uses the standard endpoint for %s", (provider, endpoint) => { - const body = aiConnectionRequest( - { ...draft, provider, endpoint: "" }, - "connector", - ); - expect(body.config.endpoint).toBe(endpoint); - expect(body.allowed_destinations).toEqual([new URL(endpoint).origin]); - }); - it("requires Azure's resource endpoint", () => { - expect(() => - aiConnectionRequest({ ...draft, endpoint: "" }, "connector"), - ).toThrow("HTTPS endpoint"); - }); - it("canonicalizes the saved endpoint and destination together without removing the base path", () => { - const body = aiConnectionRequest( - { ...draft, endpoint: "https://APPROVED.openai.azure.com:443/custom/" }, - "connector", - ); - expect(body.config.endpoint).toBe( - "https://approved.openai.azure.com/custom/", - ); - expect(body.allowed_destinations).toEqual([ - "https://approved.openai.azure.com", - ]); - }); - it.each([ - "http://model.invalid", - "https://user:password@model.invalid", - "https://model.invalid/?api-key=secret", - "https://model.invalid/#secret", - ])( - "rejects an endpoint carrying authority outside HTTPS origin: %s", - (endpoint) => { - expect(() => - aiConnectionRequest({ ...draft, endpoint }, "connector"), - ).toThrow(); - }, - ); - it("requires the explicit Azure API version and operator handle", () => { - expect(() => - aiConnectionRequest({ ...draft, apiVersion: "" }, "connector"), - ).toThrow("Azure requires"); - expect(() => - aiConnectionRequest({ ...draft, handle: "key with spaces" }, "connector"), - ).toThrow("secret handle"); - }); - it("keeps provider/model separate and derives only the approved origin", () => { - const body = aiConnectionRequest(draft, "connector"); - expect(body.allowed_destinations).toEqual([ - "https://approved.openai.azure.com", - ]); - expect(body.config).not.toHaveProperty("model"); - expect(body.secretRef).toEqual({ apiKey: "model-key" }); - expect( - aiConnectionRequest( - { ...draft, provider: "openai-responses" }, - "connector", - ).config, - ).not.toHaveProperty("apiVersion"); - }); +describe("AI connection revisions", () => { it("excludes unavailable and unrelated connection revisions", () => { const valid = { id: "ai", @@ -108,3 +38,30 @@ describe("AI provider connection authority", () => { ).toEqual([valid]); }); }); +describe("AI connection lists", () => { + const base = { + connector: aiConnector, + config: { provider: "openai-chat", endpoint: "http://ollama:11434/v1" }, + }; + it("keeps the latest revision of each name, sorted by name", () => { + // The platform's order is not alphabetical, so the result shows the sort. + const rows = [ + { ...base, id: "b1", name: "openai", revision: 1 }, + { ...base, id: "a1", name: "ollama-local", revision: 1 }, + { ...base, id: "a2", name: "ollama-local", revision: 2 }, + ]; + expect(latestByName(rows as never).map((row) => row.id)).toEqual([ + "a2", + "b1", + ]); + }); + it("reads secret handles under either wire spelling", () => { + expect( + secretRefs({ secretRef: { apiKey: "no-credential" } } as never), + ).toEqual({ apiKey: "no-credential" }); + expect(secretRefs({ secret_refs: { apiKey: "k" } } as never)).toEqual({ + apiKey: "k", + }); + expect(secretRefs({} as never)).toEqual({}); + }); +}); diff --git a/studio/tests/api.test.ts b/studio/tests/api.test.ts index 705384c8..36c19b09 100644 --- a/studio/tests/api.test.ts +++ b/studio/tests/api.test.ts @@ -17,6 +17,10 @@ SPDX-License-Identifier: Apache-2.0 */ import { describe, it, expect, vi, afterEach } from "vitest"; import { StudioApi, ApiError } from "../src/app/api"; +import { + AiSetupApi, + BRIDGE_TIMEOUT_MS, +} from "../src/app/integrations/ai-setup-api"; afterEach(() => vi.unstubAllGlobals()); describe("same-origin credential boundary", () => { it("rejects arbitrary destinations before any fetch", async () => { @@ -259,3 +263,150 @@ describe("capacity rejections", () => { expect(fetch).toHaveBeenCalledTimes(1); }); }); +describe("cancellable requests", () => { + it("stops a request when its caller cancels it", async () => { + vi.stubGlobal( + "fetch", + vi.fn( + (_path: string, options: RequestInit) => + new Promise((_resolve, reject) => + options.signal!.addEventListener("abort", () => + reject(options.signal!.reason), + ), + ), + ), + ); + const api = new StudioApi(); + const controller = new AbortController(); + const pending = api.request( + "/studio/api/x", + "POST", + {}, + {}, + 150000, + controller.signal, + ); + controller.abort(new DOMException("The test was canceled.", "AbortError")); + await expect(pending).rejects.toMatchObject({ name: "AbortError" }); + }); + it("still stops a cancellable request at its deadline", async () => { + vi.useFakeTimers(); + try { + vi.stubGlobal( + "fetch", + vi.fn( + (_path: string, options: RequestInit) => + new Promise((_resolve, reject) => + options.signal!.addEventListener("abort", () => + reject(options.signal!.reason), + ), + ), + ), + ); + const pending = new StudioApi().request( + "/studio/api/x", + "GET", + undefined, + {}, + 100, + new AbortController().signal, + ); + let outcome: unknown = "pending"; + pending.then( + () => (outcome = "resolved"), + (error: unknown) => (outcome = error), + ); + await vi.advanceTimersByTimeAsync(150); + // Checked once, so a deadline that never fires fails here instead of hanging the run. + expect(outcome).toMatchObject({ name: "TimeoutError" }); + } finally { + vi.useRealTimers(); + } + }); + it("leaves no deadline timer or listener behind after an answer", async () => { + vi.useFakeTimers(); + try { + vi.stubGlobal( + "fetch", + vi.fn(async () => new Response(JSON.stringify({ ok: true }))), + ); + const controller = new AbortController(); + const removed = vi.spyOn(controller.signal, "removeEventListener"); + await expect( + new StudioApi().request( + "/studio/api/x", + "GET", + undefined, + {}, + 150000, + controller.signal, + ), + ).resolves.toEqual({ ok: true }); + expect(vi.getTimerCount()).toBe(0); + expect(removed).toHaveBeenCalledWith("abort", expect.any(Function)); + } finally { + vi.useRealTimers(); + } + }); + it("stops at once when canceled during a capacity pause, and sends nothing more", async () => { + vi.useFakeTimers(); + try { + const fetch = vi.fn( + async () => + new Response( + JSON.stringify({ code: "WV-OPERATION-CAPACITY", message: "Busy" }), + { status: 429, headers: { "Retry-After": "5" } }, + ), + ); + vi.stubGlobal("fetch", fetch); + const controller = new AbortController(); + const pending = new StudioApi().request( + "/studio/api/x", + "GET", + undefined, + {}, + 150000, + controller.signal, + ); + let outcome: unknown = "pending"; + pending.then( + () => (outcome = "resolved"), + (error: unknown) => (outcome = error), + ); + await vi.advanceTimersByTimeAsync(100); + expect(fetch).toHaveBeenCalledTimes(1); + controller.abort( + new DOMException("The test was canceled.", "AbortError"), + ); + await vi.advanceTimersByTimeAsync(0); + expect(outcome).toMatchObject({ name: "AbortError" }); + await vi.advanceTimersByTimeAsync(10000); + expect(fetch).toHaveBeenCalledTimes(1); + } finally { + vi.useRealTimers(); + } + }); +}); + +describe("AI setup calls", () => { + it("waits for publishing only as long as the Studio host does", async () => { + const api = new StudioApi(); + api.session.profile = { + name: "Test", + baseUrl: "https://weave.example", + tenantId: "t", + projectId: "p", + environmentId: "dev", + }; + const request = vi.spyOn(api, "request").mockResolvedValue({}); + await new AiSetupApi(api).publish(); + expect(request).toHaveBeenCalledWith( + "/studio/api/api/v1/tenants/t/projects/p/environments/dev/ai/setup/publish", + "POST", + undefined, + {}, + BRIDGE_TIMEOUT_MS, + ); + expect(BRIDGE_TIMEOUT_MS).toBe(35_000); + }); +}); diff --git a/studio/tests/browser/ai-models-layout.spec.ts b/studio/tests/browser/ai-models-layout.spec.ts new file mode 100644 index 00000000..0e220f65 --- /dev/null +++ b/studio/tests/browser/ai-models-layout.spec.ts @@ -0,0 +1,376 @@ +/* +Copyright 2026 Firefly Software Foundation. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +Author: Firefly Software Foundation +SPDX-License-Identifier: Apache-2.0 +*/ +// Settings › AI models and its side sheet, measured: edges and centers line +// up, controls keep the desktop density (36 px fields and buttons, 32 px rows) +// or the touch one (44 px), and nothing is cut off or scrolls sideways. +import { expect, test, type Locator, type Page } from "@playwright/test"; +import { connected } from "./support"; +import { expectNotClippedBy } from "./ux-assertions"; +import { + box, + boxes, + clippedText, + contentEdges, + density, + expectAligned, + textBox, +} from "./step-geometry"; +import { + AI_CAPABILITIES, + mockAiPlatform, + ollamaConnection, + openaiConnection, + openAiModels, + readiness, + sheet, +} from "./ai-models-po"; + +const sizes = [ + { tag: "1440x900", width: 1440, height: 900, touch: false }, + { tag: "600x500", width: 600, height: 500, touch: false }, + { tag: "360x640", width: 360, height: 640, touch: true }, + { tag: "1024x768-touch", width: 1024, height: 768, touch: true }, +]; + +async function open(page: Page) { + await connected(page, { capabilities: AI_CAPABILITIES }); + await mockAiPlatform(page, { + connections: [ollamaConnection, openaiConnection], + readiness: readiness({ + actions_published: { + status: "action", + detail: "actions_missing", + fix: { + kind: "operation", + label: "Publish", + operation: "ai_setup.publish", + }, + }, + worker_installed: { + status: "action", + detail: "worker_missing", + fix: { + kind: "command", + label: "Ask an operator to enable AI", + command: "weave platform ai enable --ollama auto", + }, + }, + }), + }); + await openAiModels(page); + await expect(page.locator(".readiness-row")).toHaveCount(8); + await expect(page.locator(".connection-row")).toHaveCount(2); + await page.evaluate(() => document.fonts.ready); +} +/** Groups boxes that share a top edge (one visual row). */ +function byRow(items: T[]): T[][] { + const rows: T[][] = []; + for (const item of [...items].sort((a, b) => a.y - b.y)) { + const row = rows.find((group) => Math.abs(group[0].y - item.y) <= 1); + if (row) row.push(item); + else rows.push([item]); + } + return rows; +} +async function minimumHeight(controls: Locator, minimum: number, what: string) { + for (const control of await boxes(controls)) + expect( + control.height, + `${what} is ${control.height} px high`, + ).toBeGreaterThanOrEqual(minimum - 0.5); +} + +for (const size of sizes) + test.describe(`AI models layout at ${size.tag}`, () => { + test.use({ + viewport: { width: size.width, height: size.height }, + hasTouch: size.touch, + }); + + test("the page lines up and keeps its density", async ({ page }) => { + await open(page); + const d = await density(page); + const root = page.locator("weave-ai-models-page"); + expect( + await page.evaluate( + () => document.documentElement.scrollWidth <= innerWidth, + ), + ).toBe(true); + expect(await clippedText(root, "")).toEqual([]); + + // Panels share the page's left edge; side by side from 1024 px, stacked (checklist first) below. + const readinessPanel = await box(page.locator(".ai-readiness")); + const providersPanel = await box(page.locator(".ai-providers")); + const weavePanel = await box(page.locator("weave-ai-card")); + if (size.width >= 1024) { + expectAligned([readinessPanel.y, providersPanel.y], "panel tops"); + expect(readinessPanel.x).toBeGreaterThan(providersPanel.right); + expectAligned( + [providersPanel.x, weavePanel.x], + "left edges of providers and Weave AI", + ); + expectAligned( + [readinessPanel.right, weavePanel.right], + "right edges of readiness and Weave AI", + ); + } else { + expect(readinessPanel.bottom).toBeLessThanOrEqual(providersPanel.y); + expectAligned( + [readinessPanel.x, providersPanel.x, weavePanel.x], + "panel left edges", + ); + } + + // Each heading starts at its panel's content edge, like the first thing below it. + const providers = page.locator(".ai-providers"); + const edge = (await contentEdges(providers)).left; + expectAligned( + [ + edge, + (await textBox(page.locator("#ai-providers-title"))).x, + (await box(page.locator(".provider-card"))).x, + ], + "providers heading and first card", + ); + const readinessEdge = (await contentEdges(page.locator(".ai-readiness"))) + .left; + expectAligned( + [ + readinessEdge, + (await textBox(page.locator("#ai-readiness-title"))).x, + (await box(page.locator(".readiness-icon"))).x, + ], + "readiness heading and first icon", + ); + + // Cards in one row share their top, height and button line. + const cards = await boxes(page.locator(".provider-card")); + const buttons = await boxes(page.locator(".provider-card > button")); + // Four cards fill their rows evenly: 4, 2 + 2 or one per row, never 3 + 1. + const perRow = byRow(cards).map((row) => row.length); + expect([[4], [2, 2], [1, 1, 1, 1]]).toContainEqual(perRow); + if (size.width >= 1440) expect(perRow).toEqual([4]); + for (const row of byRow(cards)) { + expectAligned( + row.map((card) => card.height), + "card heights in a row", + ); + } + for (const row of byRow(buttons)) + expectAligned( + row.map((button) => button.y), + "card buttons in a row", + ); + await minimumHeight( + page.locator(".provider-card > button"), + d.control, + "a card button", + ); + + // Checklist rows: the icon sits on the text's center; fixes end on one right edge. + const rows = page.locator(".readiness-row"); + for (let index = 0; index < 8; index++) { + const row = rows.nth(index); + const rowBox = await box(row); + expect(rowBox.height).toBeGreaterThanOrEqual(d.row - 0.5); + expectAligned( + [ + (await box(row.locator(".readiness-icon"))).middle, + (await box(row.locator(".readiness-text"))).middle, + ], + `checklist row ${index + 1}: icon and text centers`, + ); + } + const fixes = await boxes(page.locator(".readiness-action")); + expectAligned( + fixes.map((fix) => fix.right), + "checklist fixes' right edges", + ); + await minimumHeight( + page.locator(".readiness-action"), + d.control, + "a checklist fix", + ); + // A command wraps between its words, never inside one ("--ollama"). + const command = page.locator(".readiness-command code"); + // Selecting it copies the command as written. + expect(await command.evaluate((code) => code.textContent)).toBe( + "weave platform ai enable --ollama auto", + ); + expect( + await command.evaluate((code) => { + const text = code.textContent ?? ""; + const split: string[] = []; + for (const match of text.matchAll(/\S+/g)) { + const range = document.createRange(); + let offset = 0; + const walker = document.createTreeWalker( + code, + NodeFilter.SHOW_TEXT, + ); + for (let node = walker.nextNode(); node; node = walker.nextNode()) { + const length = node.textContent!.length; + const start = match.index - offset; + const end = start + match[0].length; + if (start >= 0 && start < length) range.setStart(node, start); + if (end > 0 && end <= length) range.setEnd(node, end); + offset += length; + } + const lines = new Set( + [...range.getClientRects()] + .filter((rect) => rect.width > 0) + .map((rect) => Math.round(rect.top)), + ); + if (lines.size > 1) split.push(match[0]); + } + return split; + }), + "words split across lines", + ).toEqual([]); + + // Connection rows: actions centered on the row beside the text, or under it on a phone. + for (const row of await page.locator(".connection-row").all()) { + const main = await box(row.locator(".connection-main")); + const actions = await box(row.locator(".connection-actions")); + if (size.width >= 600) + expectAligned( + [main.middle, actions.middle], + "connection text and actions centers", + 1.5, + ); + else + expectAligned( + [main.x, actions.x], + "connection text and actions left edges", + ); + await minimumHeight( + row.locator("button"), + d.control, + "a connection action", + ); + } + // Weave AI's dropdowns are exactly as high as the buttons beside them; + // side by side they share a top, stacked they share a left edge. + const card = page.locator("weave-ai-card"); + // The bordered field, not the text box inside it. + const dropdowns = await boxes(card.locator("weave-select .choice-field")); + expect(dropdowns).toHaveLength(2); + expectAligned( + [d.control, ...dropdowns.map((dropdown) => dropdown.height)], + "Weave AI dropdown heights", + 0.5, + ); + if (Math.abs(dropdowns[0].y - dropdowns[1].y) <= 1) + expectAligned( + dropdowns.map((dropdown) => dropdown.y), + "Weave AI dropdown tops", + ); + else + expectAligned( + dropdowns.map((dropdown) => dropdown.x), + "Weave AI dropdown left edges", + ); + // A dropdown's chevron sits inside its 44 px field, which opens the + // same list: the field is the target, measured above. + if (d.touch) + await minimumHeight( + root.locator("button:not(.choice-field button)"), + 44, + "a touch target", + ); + }); + + test("the side sheet fills its edge and lines up its fields and footer", async ({ + page, + }) => { + await open(page); + await page.getByRole("button", { name: "Connect Anthropic" }).click(); + const dialog = sheet(page); + await expect(dialog.getByLabel("Secret handle")).toBeVisible(); + const d = await density(page); + const panel = await box(dialog); + expectAligned([panel.right, size.width], "sheet right edge"); + expectAligned([panel.y, 0], "sheet top"); + expectAligned([panel.bottom, size.height], "sheet bottom"); + expectAligned( + [panel.width, size.width < 768 ? size.width : 480], + "sheet width", + ); + + // The heading's first line sits on the close button's center. + const heading = await textBox(dialog.locator(".modal-header h2")); + const close = await box( + dialog.getByRole("button", { name: "Close AI connection" }), + ); + expectAligned( + [heading.middle, close.middle], + "heading and close centers", + ); + + // Labels, fields and group legends share the form's left edge. + const left = (await contentEdges(dialog.locator(".ai-sheet"))).left; + expectAligned( + [ + left, + (await textBox(dialog.locator(".choice-group legend").first())).x, + (await box(dialog.getByLabel("Connection name"))).x, + (await box(dialog.getByLabel("Secret handle"))).x, + ], + "sheet field edges", + ); + // Radio rows: the radio sits on its label's center. + for (const row of await dialog.locator(".choice-row").all()) { + expect((await box(row)).height).toBeGreaterThanOrEqual(d.row - 0.5); + expectAligned( + [ + (await box(row.locator("input"))).middle, + (await box(row.locator(".choice-text"))).middle, + ], + "radio and label centers", + 1.5, + ); + } + await minimumHeight( + dialog.locator("input:not([type=radio])"), + d.control, + "a sheet field", + ); + + // Footer answers: one row ending at the content edge, or stacked full width on a phone. + const footer = dialog.locator(".dialog-footer"); + const answers = await boxes(footer.locator("button")); + if (new Set(answers.map((answer) => Math.round(answer.y))).size === 1) + expectAligned( + [answers.at(-1)!.right, (await contentEdges(footer)).right], + "primary answer right edge", + ); + else + expectAligned( + answers.map((answer) => answer.width), + "stacked answers' widths", + ); + await minimumHeight( + footer.locator("button"), + d.control, + "a footer answer", + ); + for (const answer of await footer.locator("button").all()) + await expectNotClippedBy(answer, dialog); + expect(await clippedText(dialog, "")).toEqual([]); + }); + }); diff --git a/studio/tests/browser/ai-models-po.ts b/studio/tests/browser/ai-models-po.ts new file mode 100644 index 00000000..2c1aef3e --- /dev/null +++ b/studio/tests/browser/ai-models-po.ts @@ -0,0 +1,375 @@ +/* +Copyright 2026 Firefly Software Foundation. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +Author: Firefly Software Foundation +SPDX-License-Identifier: Apache-2.0 +*/ +// A mocked platform for Settings › AI models: approved endpoints, discovery, +// readiness, setup fixes, connections and Weave AI, recording every request +// the screen sends. Routes registered here win over support.ts's catch-all. +import { expect, type Locator, type Page, type Route } from "@playwright/test"; +import { allCapabilities } from "./support"; + +const ENV = + "**/studio/api/api/v1/tenants/tenant/projects/project/environments/development"; +const PROJECT = "**/studio/api/api/v1/tenants/tenant/projects/project"; +export const AI_CAPABILITIES = [ + ...allCapabilities, + "status.read", + "connection.bind", + "lumi.use", + "lumi.manage", +]; +export const CONNECTOR_ID = "11111111-1111-4111-8111-111111111111"; +export const OLLAMA_REVISION = "22222222-2222-4222-8222-222222222222"; +export const OPENAI_REVISION = "33333333-3333-4333-8333-333333333333"; +export const OLLAMA_ENDPOINT = { + id: "ollama-local", + label: "Ollama (Weave-managed)", + url: "http://ollama:11434/v1", + providers: ["openai-chat"], + compat: "ollama", + credential: "none", + models: "served", + context_tokens: 8192, + development_only: true, +}; +export const OPENAI_ENDPOINT = { + id: "openai", + label: "OpenAI", + url: "https://api.openai.com/v1", + providers: ["openai-chat", "openai-responses"], + compat: null, + credential: "required", + models: "listed", + context_tokens: 128000, + development_only: false, +}; +const revision = ( + id: string, + name: string, + config: object, + apiKey: string, + origin: string, +) => ({ + id, + name, + revision: 1, + connector: "weave-agentic-provider@1.0.0", + connector_digest: "digest", + adapter: "agentic", + connector_version_id: CONNECTOR_ID, + config: { ...config, secretSlot: "apiKey" }, + secretRef: { apiKey }, + allowed_destinations: [origin], +}); +export const ollamaConnection = revision( + OLLAMA_REVISION, + "ollama-local", + { provider: "openai-chat", endpoint: "http://ollama:11434/v1" }, + "no-credential", + "http://ollama:11434", +); +export const openaiConnection = revision( + OPENAI_REVISION, + "openai", + { provider: "openai-responses", endpoint: "https://api.openai.com/v1" }, + "openai-api-key", + "https://api.openai.com", +); +export const QWEN = { + name: "qwen3:4b", + approved: true, + available: true, + tools: "yes", + context_tokens: 8192, + size_bytes: 2_500_000_000, + family: "qwen3", +}; +export const GEMMA = { + name: "gemma3:270m", + approved: true, + available: true, + tools: "no", + context_tokens: 8192, + size_bytes: 291_000_000, + family: "gemma3", +}; +type Item = { + id: string; + status: string; + detail: string; + fix?: Record; +}; +/** A local platform after `weave platform ai enable`: everything but Weave AI. */ +export const READY_LOCAL: Item[] = [ + { id: "worker_installed", status: "done", detail: "worker_installed" }, + { id: "actions_published", status: "done", detail: "actions_published" }, + { id: "worker_online", status: "done", detail: "worker_online" }, + { id: "connection", status: "done", detail: "connection_available" }, + { id: "key_available", status: "not_needed", detail: "no_credential" }, + { + id: "connection_authorized", + status: "not_needed", + detail: "no_credential", + }, + { id: "model_responds", status: "done", detail: "test_passed" }, + { + id: "weave_ai", + status: "action", + detail: "weave_ai_missing", + fix: { kind: "navigate", label: "Open AI settings", route: "settings/ai" }, + }, +]; +export function readiness(overrides: Record> = {}) { + return { + items: READY_LOCAL.map((item) => ({ + ...item, + ...(overrides[item.id] ?? {}), + })), + }; +} +export interface AiPlatform { + capabilities: string[]; + readiness: { items: Item[] }; + connections: Record[]; + models: Record[]; + /** The test answer: a result body, or a refusal with its status. */ + testAnswer: + | Record + | { status: number; body: Record }; + testDelay: number; + /** Answers for the next connection creates, in order; then each create succeeds. */ + createAnswers: { status: number; body: Record }[]; + /** Answers for the next grants, in order; then each grant succeeds. */ + grantAnswers: { status: number; body: Record }[]; + lumi: Record | null; + // Recorded requests. + creates: { body: Record; key: string }[]; + tests: { url: string; body: Record }[]; + modelRequests: string[]; + publishes: number; + grants: Record[]; + lumiWrites: { body: Record; ifMatch: string | null }[]; + readinessReads: number; +} +export async function mockAiPlatform( + page: Page, + options: Partial = {}, +): Promise { + const state: AiPlatform = { + capabilities: AI_CAPABILITIES, + readiness: readiness(), + connections: [], + models: [QWEN, GEMMA], + testAnswer: { + ok: true, + code: "ok", + latency_ms: 2400, + model: "qwen3:4b", + tool_calling: "supported", + }, + testDelay: 0, + createAnswers: [], + grantAnswers: [], + lumi: null, + creates: [], + tests: [], + modelRequests: [], + publishes: 0, + grants: [], + lumiWrites: [], + readinessReads: 0, + ...options, + }; + const json = (route: Route, body: unknown, status = 200) => + route.fulfill({ status, json: body }); + await page.route(`${ENV}/ai/endpoints`, (r) => + json(r, { + policy: "loaded", + endpoints: [OLLAMA_ENDPOINT, OPENAI_ENDPOINT], + }), + ); + await page.route(`${ENV}/ai/readiness`, (r) => { + state.readinessReads++; + return json(r, state.readiness); + }); + await page.route(`${ENV}/ai/models?*`, (r) => { + state.modelRequests.push(r.request().url()); + return json(r, { + approval: "served", + discovery: "ok", + discovered_at: "2026-10-13T10:00:00Z", + models: state.models, + }); + }); + await page.route(`${ENV}/ai/connections/*/test`, async (r) => { + state.tests.push({ + url: r.request().url(), + body: r.request().postDataJSON(), + }); + if (state.testDelay) + await new Promise((resolve) => setTimeout(resolve, state.testDelay)); + const answer = state.testAnswer; + await ( + "status" in answer && "body" in answer + ? json(r, answer.body, answer.status as number) + : json(r, answer) + ).catch(() => undefined); + }); + await page.route(`${ENV}/ai/setup/publish`, (r) => { + state.publishes++; + state.readiness = { + items: state.readiness.items.map((item) => + item.id === "actions_published" + ? { id: item.id, status: "done", detail: "actions_published" } + : item, + ), + }; + return json(r, { connector: {}, actions: [{}] }); + }); + await page.route(`${ENV}/ai/setup/grant`, (r) => { + state.grants.push(r.request().postDataJSON()); + const scripted = state.grantAnswers.shift(); + if (scripted) return json(r, scripted.body, scripted.status); + state.readiness = { + items: state.readiness.items.map((item) => + item.id === "connection_authorized" + ? { id: item.id, status: "done", detail: "connection_authorized" } + : item, + ), + }; + return json(r, { granted: true }); + }); + await page.route(`${ENV}/connections?*`, (r) => + json(r, { items: state.connections, next_cursor: null }), + ); + await page.route(`${ENV}/connections`, (r) => { + if (r.request().method() !== "POST") return r.fallback(); + const body = r.request().postDataJSON() as Record; + state.creates.push({ + body, + key: r.request().headers()["idempotency-key"] ?? "", + }); + const scripted = state.createAnswers.shift(); + if (scripted) return json(r, scripted.body, scripted.status); + // Saving a name again adds its next revision; earlier ones stay readable. + const earlier = state.connections.filter((c) => c["name"] === body["name"]); + const revision = + Math.max(0, ...earlier.map((c) => Number(c["revision"]))) + 1; + const first = + body["name"] === "ollama-local" ? OLLAMA_REVISION : OPENAI_REVISION; + const created = { + ...body, + id: + revision === 1 + ? first + : `${first.slice(0, -2)}${String(revision).padStart(2, "0")}`, + revision, + connector: "weave-agentic-provider@1.0.0", + connector_digest: "digest", + adapter: "agentic", + }; + state.connections = [...state.connections, created]; + return json(r, created, 201); + }); + await page.route(`${ENV}/connections/*`, (r) => { + const id = new URL(r.request().url()).pathname.split("/").pop(); + const found = state.connections.find((c) => c["id"] === id); + if (r.request().method() !== "GET" || !found) return r.fallback(); + return json(r, found); + }); + await page.route(`${ENV}/bindable-connections?*`, (r) => + json(r, { + items: state.connections.map((c) => ({ + connection_id: c["id"], + revision_id: c["id"], + name: c["name"], + connector: "weave-agentic-provider@1.0.0", + label: c["name"], + enabled: true, + provider: (c["config"] as Record)["provider"], + endpoint_origin: new URL( + (c["config"] as Record)["endpoint"], + ).origin, + })), + next_cursor: null, + }), + ); + await page.route(`${PROJECT}/connectors?*`, (r) => + json(r, { + items: [ + { id: CONNECTOR_ID, name: "weave-agentic-provider", version: "1.0.0" }, + ], + next_cursor: null, + }), + ); + await page.route(`${ENV}/lumi/configuration`, (r) => { + if (r.request().method() === "PUT") { + const body = r.request().postDataJSON() as Record; + state.lumiWrites.push({ + body, + ifMatch: r.request().headers()["if-match"] ?? null, + }); + state.lumi = { + ...body, + revision: ((state.lumi?.["revision"] as number) ?? 0) + 1, + }; + return json(r, state.lumi); + } + return state.lumi + ? json(r, state.lumi) + : json( + r, + { code: "WV-NOT-FOUND", message: "Weave AI is not configured" }, + 404, + ); + }); + await page.route(`${ENV}/lumi/status`, (r) => + json( + r, + state.lumi + ? { + configured: true, + provider: "openai-responses", + model: "gpt-5-mini", + } + : { configured: false }, + ), + ); + await page.route("**/studio/contracts/lumi-configuration", (r) => + json(r, { + $defs: { + LumiProfile: { + properties: { outputSchema: { const: { type: "object" } } }, + }, + }, + }), + ); + return state; +} +/** Settings › AI models, loaded. */ +export async function openAiModels(page: Page) { + await page.goto("/settings/ai"); + await expect(page.getByRole("tab", { name: "AI models" })).toHaveAttribute( + "aria-selected", + "true", + ); +} +/** The AI connection side sheet, by its title when one is given. */ +export function sheet(page: Page, name?: string): Locator { + return name + ? page.getByRole("dialog", { name }) + : page.locator("weave-ai-connection-sheet").getByRole("dialog"); +} diff --git a/studio/tests/browser/ai-setup.spec.ts b/studio/tests/browser/ai-setup.spec.ts index 374601d0..9930b7b8 100644 --- a/studio/tests/browser/ai-setup.spec.ts +++ b/studio/tests/browser/ai-setup.spec.ts @@ -15,767 +15,1139 @@ limitations under the License. Author: Firefly Software Foundation SPDX-License-Identifier: Apache-2.0 */ +// Settings › AI models and its AI connection side sheet against a mocked +// platform, with real clicks at 1440x900 and 600x500. import { execFileSync } from "node:child_process"; -import { test, expect, type Page, type Route } from "@playwright/test"; -import { parse } from "yaml"; -import { - allCapabilities, - connected, - newWorkflow, - sourceText, - selectChoice, -} from "./support"; -import { DesignerPage } from "./designer-po"; +import { expect, test, type Page } from "@playwright/test"; import { python } from "../python-path"; +import { connected, newWorkflow, selectChoice } from "./support"; +import { DesignerPage } from "./designer-po"; +import { + AI_CAPABILITIES, + CONNECTOR_ID, + OLLAMA_REVISION, + OPENAI_REVISION, + mockAiPlatform, + ollamaConnection, + openAiModels, + openaiConnection, + readiness, + sheet, + type AiPlatform, +} from "./ai-models-po"; -const schema = (module: string, name: string) => - JSON.parse( - execFileSync( - python, - [ - "-c", - `import json; from firefly_weave.contracts.${module} import ${name}; print(json.dumps(${name}.model_json_schema(by_alias=True)))`, - ], - { cwd: "..", encoding: "utf8" }, - ), - ); -const lumiSchema = schema("lumi", "LumiConfigurationRequest"); -const llmSchema = schema("llm", "LLMProfile"); -const connectorId = "11111111-1111-4111-8111-111111111111"; -const connectionId = "22222222-2222-4222-8222-222222222222"; -async function setup( - page: Page, - capabilities = [...allCapabilities, "lumi.use", "lumi.manage"], -) { - await connected(page, { capabilities }); - const connections: Record[] = []; - const configurations: any[] = []; - const creates: any[] = []; - await page.route("**/studio/contracts/lumi-configuration", (r) => - r.fulfill({ json: lumiSchema }), - ); - await page.route("**/studio/contracts/llm-profile", (r) => - r.fulfill({ json: llmSchema }), - ); - await page.route("**/studio/contracts/llm-profile/validate", (r) => - r.fulfill({ json: { valid: true, issues: [] } }), - ); - await page.route("**/projects/project/connectors?*", (r) => - r.fulfill({ - json: { - items: [ - { id: connectorId, name: "weave-agentic-provider", version: "1.0.0" }, - ], - next_cursor: null, - }, - }), - ); - await page.route("**/environments/development/connections?*", (r) => - r.fulfill({ json: { items: connections, next_cursor: null } }), - ); - await page.route("**/environments/development/connections", async (r) => { - const body = r.request().postDataJSON(); - creates.push(body); - const connection = { - ...body, - id: connectionId, - revision: 1, - connector: "weave-agentic-provider@1.0.0", - }; - connections.push(connection); - await r.fulfill({ json: connection }); +async function start(page: Page, options: Partial = {}) { + await connected(page, { + capabilities: options.capabilities ?? AI_CAPABILITIES, }); - await page.route("**/lumi/status", (r) => - r.fulfill({ json: { configured: false } }), - ); - await page.route("**/lumi/configuration", async (r) => { - if (r.request().method() === "PUT") { - const body = r.request().postDataJSON(); - configurations.push(body); - await r.fulfill({ json: { ...body, revision: 1 } }); - } else - await r.fulfill({ - status: 404, - json: { code: "WV-NOT-FOUND", message: "Weave AI is not configured" }, - }); - }); - return { connections, configurations, creates }; + const platform = await mockAiPlatform(page, options); + await openAiModels(page); + return platform; +} +/** Opens a new AI connection from the screen's Connect button. */ +async function newConnection(page: Page, provider: "Ollama" | "OpenAI") { + await page.getByRole("button", { name: `Connect ${provider}` }).click(); + return sheet(page); } -for (const width of [1440, 600]) - test.describe(`AI setup at ${width}`, () => { - test.use({ viewport: { width, height: 800 } }); - test("fresh Azure provider connection and Weave AI settings use named revisions and approved handles", async ({ + +for (const size of [ + { width: 1440, height: 900 }, + { width: 600, height: 500 }, +]) + test.describe(`AI connection sheet at ${size.width}x${size.height}`, () => { + test.use({ viewport: size }); + + test("connects Ollama from an approved preset without a key, then tests a tool-capable model", async ({ page, }) => { - const capture = await setup(page); - await page - .getByRole("button", { name: "Connections", exact: true }) - .click(); - await page - .getByRole("button", { name: "New AI connection", exact: true }) - .click(); - const form = page.getByRole("dialog", { - name: "New AI connection", - exact: true, - }); - await form - .getByLabel("Connection name", { exact: true }) - .fill("azure-models"); - await selectChoice( - form.getByLabel("Provider", { exact: true }), - "azure-responses", - ); - await form.getByRole("button", { name: "Continue", exact: true }).click(); - await form - .getByLabel("Provider endpoint", { exact: true }) - .fill("https://approved-models.openai.azure.com/"); - await form - .getByLabel("API key secret handle", { exact: true }) - .fill("azure-model-key"); - await form - .getByRole("button", { name: "Review connection", exact: true }) - .click(); - await expect(form).toContainText("Azure requires an API version"); - expect(capture.creates).toHaveLength(0); - await form - .getByLabel("Azure API version", { exact: true }) - .fill("2025-04-01-preview"); - await form - .getByRole("button", { name: "Review connection", exact: true }) - .click(); - await form - .getByRole("button", { name: "Create AI connection", exact: true }) - .click(); - await expect(form).toContainText("Connection created"); - expect( - await page.evaluate( - () => document.documentElement.scrollWidth <= innerWidth, - ), - ).toBe(true); - expect(capture.creates[0]).toEqual({ - name: "azure-models", - connector_version_id: connectorId, - config: { - provider: "azure-responses", - endpoint: "https://approved-models.openai.azure.com/", - secretSlot: "apiKey", - apiVersion: "2025-04-01-preview", - }, - secretRef: { apiKey: "azure-model-key" }, - allowed_destinations: ["https://approved-models.openai.azure.com"], - }); - await form.getByRole("button", { name: "Done", exact: true }).click(); - await page - .getByRole("button", { name: "Ask Weave AI", exact: true }) - .click(); - const lumi = page.getByRole("dialog", { - name: "Ask Weave AI", - exact: true, - }); - await lumi - .getByRole("button", { name: "Weave AI settings", exact: true }) - .click(); - await selectChoice( - lumi.getByLabel("Provider", { exact: true }), - "azure-responses", + const platform = await start(page); + const form = await newConnection(page, "Ollama"); + await expect( + form.getByRole("radio", { name: /Ollama \(Weave-managed\)/ }), + ).toBeChecked(); + await expect(form).toContainText("Development only"); + await expect(form.getByLabel("Secret handle")).toHaveCount(0); + await expect(form).toContainText( + "Local endpoints use no credential. Weave never sends credentials to a model over plain HTTP.", ); - await lumi - .getByLabel("Model", { exact: true }) - .fill("assistant-deployment"); - await lumi - .getByRole("button", { name: "Continue to connection", exact: true }) - .click(); - await selectChoice( - lumi.getByLabel("Provider connection", { exact: true }), - connectionId, + await expect( + form.getByRole("button", { name: "Test connection" }), + ).toBeDisabled(); + await form.getByRole("button", { name: "Save connection" }).click(); + await expect(form.getByLabel("Model", { exact: true })).toHaveValue( + "qwen3:4b", ); + expect(platform.creates).toEqual([ + { + key: expect.any(String), + body: { + name: "ollama-local", + connector_version_id: CONNECTOR_ID, + config: { + provider: "openai-chat", + endpoint: "http://ollama:11434/v1", + secretSlot: "apiKey", + }, + secretRef: { apiKey: "no-credential" }, + allowed_destinations: ["http://ollama:11434"], + }, + }, + ]); + await form.getByRole("button", { name: "Test connection" }).click(); await expect( - lumi.getByLabel("Connection revision id", { exact: true }), - ).toHaveCount(0); - await lumi - .getByRole("button", { name: "Review settings", exact: true }) - .click(); - await lumi - .getByRole("button", { name: "Save Weave AI settings", exact: true }) - .click(); - await expect(lumi.getByLabel("Message to Weave AI")).toBeVisible(); + form.getByRole("status").filter({ hasText: "Connected." }), + ).toHaveText("Connected. qwen3:4b answered in 2.4 s and can call tools."); + expect(platform.tests).toEqual([ + { + url: expect.stringContaining( + `/ai/connections/${OLLAMA_REVISION}/test`, + ), + body: { model: "qwen3:4b", probe_tools: true }, + }, + ]); + expect( + await page.evaluate(() => JSON.stringify(localStorage)), + ).not.toContain("qwen3:4b"); expect( await page.evaluate( () => document.documentElement.scrollWidth <= innerWidth, ), ).toBe(true); - await page.screenshot({ - path: `../.superpowers/editor-ux/ai-setup-${width}.png`, - }); - expect(capture.configurations).toHaveLength(1); - expect(capture.configurations[0].connection_revision_id).toBe( - connectionId, - ); - expect(capture.configurations[0].profile.model).toBe( - "assistant-deployment", - ); - expect(capture.configurations[0].profile.outputSchema).toEqual( - lumiSchema.$defs.LumiProfile.properties.outputSchema.const, - ); - await expect(lumi).toContainText("gateway"); }); - test("fresh workflow AI profile and connection slot remain separate from Weave AI", async ({ + + test("refuses a custom plain HTTP address before sending anything", async ({ page, }) => { - const capture = await setup(page); - await newWorkflow(page); - const designer = new DesignerPage(page); - if (width <= 767) - await page - .getByRole("button", { name: "Insert step", exact: true }) - .click(); - await page - .locator(".palette") - .getByRole("button", { name: "AI task", exact: true }) - .click(); - await designer.selectStep("ask-ai-1"); - const form = page.locator("weave-llm-inspector"); + const platform = await start(page); + const form = await newConnection(page, "Ollama"); + await form.getByRole("radio", { name: "Custom address…" }).check(); await form - .getByRole("button", { - name: "Configure workflow AI profiles", - exact: true, - }) - .click(); - await selectChoice( - form.getByLabel("Provider", { exact: true }), - "azure-responses", + .getByLabel("Endpoint address") + .fill("http://10.0.0.5:11434/v1"); + await form.getByRole("button", { name: "Save connection" }).click(); + await expect(form.getByRole("alert")).toHaveText( + "This address is not approved for plain HTTP. A platform operator must add it to the private-origin policy.", ); - await form - .getByLabel("Model", { exact: true }) - .fill("workflow-deployment"); - await form.getByLabel("Max tokens", { exact: true }).fill("512"); - await form - .getByRole("button", { name: "Continue to connection", exact: true }) - .click(); - await form - .getByRole("button", { name: "Add AI connection slot", exact: true }) - .click(); - await expect( - form.getByLabel("AI connection slot", { exact: true }), - ).toHaveValue("ai"); - await form - .getByRole("button", { name: "Review settings", exact: true }) - .click(); - await form - .getByRole("button", { - name: "Apply workflow AI settings", - exact: true, - }) - .click(); - await form - .locator('[data-field="prompt"]') - .getByLabel("AI prompt", { exact: true }) - .fill("Summarize input"); - await expect( - form.getByLabel("Workflow AI profile", { exact: true }), - ).toHaveValue("default"); - const doc = parse(await sourceText(page)); - expect(doc.spec.llmProfiles.default.provider).toBe("azure-responses"); - expect(doc.spec.llmProfiles.default.model).toBe("workflow-deployment"); - expect(doc.spec.connections.ai.connector).toBe( - "weave-agentic-provider@1.0.0", + await expect(form.getByLabel("Endpoint address")).toHaveAttribute( + "aria-invalid", + "true", ); - expect(doc.spec.steps[0].connection).toBe("ai"); - expect(capture.configurations).toEqual([]); + expect(platform.creates).toEqual([]); }); }); -test("Weave AI manager without connection management sees the required grant instead of an editable UUID", async ({ + +test("shows running, canceled, no-tools, failed and gateway-missing test states", async ({ page, }) => { - const capture = await setup(page, ["lumi.use", "lumi.manage"]); - await page.getByRole("button", { name: "Ask Weave AI", exact: true }).click(); - const lumi = page.getByRole("dialog", { name: "Ask Weave AI", exact: true }); - await lumi - .getByRole("button", { name: "Weave AI settings", exact: true }) - .click(); - await expect(lumi).toContainText("connection.manage"); - await selectChoice( - lumi.getByLabel("Provider", { exact: true }), - "openai-responses", + const platform = await start(page, { + connections: [ollamaConnection], + testDelay: 60_000, + }); + await page.getByRole("button", { name: "Test ollama-local" }).click(); + const form = sheet(page, "ollama-local"); + await form.getByRole("button", { name: "Test connection" }).click(); + await expect( + form.getByRole("status").filter({ hasText: "Calling" }), + ).toHaveText("Calling qwen3:4b on ollama-local…"); + await form.getByRole("button", { name: "Cancel test" }).click(); + await expect( + form.getByRole("status").filter({ hasText: "canceled" }), + ).toHaveText("Test canceled."); + platform.testDelay = 0; + platform.testAnswer = { + ok: true, + code: "ok", + latency_ms: 800, + model: "gemma3:270m", + tool_calling: "unsupported", + }; + await form.getByLabel("Model", { exact: true }).click(); + await page.getByRole("option", { name: /gemma3:270m/ }).click(); + await form.getByRole("button", { name: "Test connection" }).click(); + const status = form.locator(".test-status"); + await expect(status).toHaveAttribute("data-tone", "warning"); + await expect(status).toContainText( + "cannot call tools. Use it for AI tasks, not AI agents.", + ); + platform.testAnswer = { + ok: false, + code: "LLM_MODEL_NOT_FOUND", + latency_ms: 30, + model: "gemma3:270m", + tool_calling: "unknown", + }; + await form.getByRole("button", { name: "Test connection" }).click(); + await expect(status).toContainText( + "gemma3:270m is not installed on http://ollama:11434.", + ); + await form.getByText("Show details").click(); + await expect(form.locator(".test-details")).toContainText( + "LLM_MODEL_NOT_FOUND", + ); + platform.testAnswer = { + status: 503, + body: { + code: "WV-AI-GATEWAY-MISSING", + message: "body text that must not show", + }, + }; + await form.getByRole("button", { name: "Test connection" }).click(); + await expect(status).toContainText("Tests need the AI gateway."); + await expect(form).not.toContainText("body text that must not show"); +}); + +test("asks for an OpenAI secret handle, never a key, and shows the operator command", async ({ + page, +}) => { + const platform = await start(page); + const form = await newConnection(page, "OpenAI"); + await expect(form.getByRole("radio", { name: "Responses" })).toBeChecked(); + await expect(form.getByLabel("Secret handle")).toHaveValue("openai-api-key"); + await expect(form).toContainText( + "Weave stores a reference to your key, never the key.", ); - await lumi.getByLabel("Model", { exact: true }).fill("test-model"); - await lumi - .getByRole("button", { name: "Continue to connection", exact: true }) - .click(); await expect( - lumi.getByRole("button", { name: "Review settings", exact: true }), - ).toBeDisabled(); + form + .locator("code") + .filter({ hasText: "weave platform secret set --handle openai-api-key" }), + ).toBeVisible(); + // A pasted key is never echoed: the command keeps its placeholder. + const pasted = "sk-proj-CANARY0123456789abcdefCANARY"; + await form.getByLabel("Secret handle").fill(pasted); await expect( - lumi.getByLabel("Provider connection", { exact: true }), - ).toBeDisabled(); - expect(capture.configurations).toEqual([]); - expect(capture.creates).toEqual([]); + form + .locator("code") + .filter({ hasText: "weave platform secret set --handle HANDLE" }), + ).toBeVisible(); + await expect(form).not.toContainText(pasted); + await form.getByLabel("Secret handle").fill("no-credential"); + await form.getByRole("button", { name: "Save connection" }).click(); + await expect(form.getByRole("alert")).toContainText( + "no-credential is reserved", + ); + await form.getByLabel("Secret handle").fill("team-openai"); + await form.getByRole("button", { name: "Save connection" }).click(); + await expect.poll(() => platform.creates.length).toBe(1); + expect(platform.creates[0].body).toMatchObject({ + name: "openai", + config: { + provider: "openai-responses", + endpoint: "https://api.openai.com/v1", + }, + secretRef: { apiKey: "team-openai" }, + }); +}); + +test("asks before closing with unsaved changes, and an uncertain save reuses its key", async ({ + page, +}) => { + const platform = await start(page, { + createAnswers: [ + { + status: 502, + body: { + code: "WV-STUDIO-UPSTREAM", + message: "Platform request failed", + }, + }, + ], + }); + const form = await newConnection(page, "Ollama"); + await form.getByLabel("Connection name").fill("team-ollama"); + await page.keyboard.press("Escape"); + const question = page.getByRole("dialog", { name: "Discard your changes?" }); + await question.getByRole("button", { name: "Keep editing" }).click(); + await expect(form).toBeVisible(); + await form.getByRole("button", { name: "Save connection" }).click(); + await expect(form.getByRole("alert")).toContainText( + "Platform request failed", + ); + await form.getByRole("button", { name: "Save connection" }).click(); + await expect.poll(() => platform.creates.length).toBe(2); + expect(platform.creates[1].key).toBe(platform.creates[0].key); + await form.getByRole("button", { name: "Close AI connection" }).click(); + await expect(form).toHaveCount(0); +}); + +test("is read-only for people who cannot manage connections", async ({ + page, +}) => { + await start(page, { + capabilities: ["catalog.read", "status.read", "connection.bind"], + connections: [ollamaConnection], + }); + await page.getByRole("button", { name: "View ollama-local" }).click(); + const form = sheet(page, "ollama-local"); + await expect(form).toContainText( + "Creating AI connections requires the tenant administrator role.", + ); await expect( - lumi.getByLabel("Connection revision id", { exact: true }), + form.getByRole("button", { name: "Save connection" }), ).toHaveCount(0); + await expect( + form.getByRole("button", { name: "Test connection" }), + ).toHaveCount(0); + await expect(form.getByLabel("Connection name")).toBeDisabled(); }); -test("Settings exposes admin AI setup while viewers cannot configure or spend", async ({ +test("Build › Connections opens the same sheet and points AI connections to Settings", async ({ page, }) => { - await setup(page); - await page.getByRole("button", { name: "Settings", exact: true }).click(); + await start(page, { connections: [ollamaConnection] }); + await page.getByRole("button", { name: "Connections", exact: true }).click(); await page - .getByRole("button", { name: "Weave AI settings", exact: true }) + .getByRole("button", { name: "New AI connection", exact: true }) .click(); + const form = sheet(page); await expect( - page.getByRole("heading", { - name: "Choose Weave AI's model", + form + .getByRole("group", { name: "Provider" }) + .getByRole("radio", { name: /^Ollama/ }), + ).toBeChecked(); + await form.getByRole("button", { name: "Close AI connection" }).click(); + await expect(form).toHaveCount(0); + await page + .getByRole("button", { name: /ollama-local/ }) + .first() + .click(); + await page + .getByRole("button", { name: "Manage in Settings › AI models" }) + .click(); + await expect(page).toHaveURL(/\/settings\/ai$/); +}); + +test("an AI task's connection step opens the same sheet over the editor", async ({ + page, +}) => { + await start(page); + // The profile editor reads its fields from the real LLM profile schema. + const llmSchema = execFileSync( + python, + [ + "-c", + "import json; from firefly_weave.contracts.llm import LLMProfile; print(json.dumps(LLMProfile.model_json_schema(by_alias=True)))", + ], + { cwd: "..", encoding: "utf8" }, + ); + await page.route("**/studio/contracts/llm-profile", (r) => + r.fulfill({ contentType: "application/json", body: llmSchema }), + ); + await page.route("**/studio/contracts/llm-profile/validate", (r) => + r.fulfill({ json: { valid: true, issues: [] } }), + ); + await page.getByRole("button", { name: "Home", exact: true }).click(); + await newWorkflow(page); + await page + .locator(".palette") + .getByRole("button", { name: "AI task", exact: true }) + .click(); + await new DesignerPage(page).selectStep("ask-ai-1"); + const inspector = page.locator("weave-llm-inspector"); + await inspector + .getByRole("button", { + name: "Configure workflow AI profiles", exact: true, - }), - ).toBeVisible(); - const lumi = page.getByRole("dialog", { name: "Ask Weave AI", exact: true }); + }) + .click(); await selectChoice( - lumi.getByLabel("Provider", { exact: true }), - "openai-responses", + inspector.getByLabel("Provider", { exact: true }), + "openai-chat", ); - await lumi.getByLabel("Model", { exact: true }).fill("test-model"); - await lumi + await inspector.getByLabel("Model", { exact: true }).fill("qwen3:4b"); + await inspector.getByLabel("Max tokens", { exact: true }).fill("512"); + await inspector .getByRole("button", { name: "Continue to connection", exact: true }) .click(); - await expect( - lumi.getByText(/No matching provider connections/), - ).toBeVisible(); + await inspector + .getByRole("button", { name: "New AI connection", exact: true }) + .click(); + await expect(sheet(page)).toBeVisible(); + await sheet(page) + .getByRole("button", { name: "Close AI connection" }) + .click(); + await expect(inspector).toBeVisible(); }); -test("viewers see setup guidance without manager controls", async ({ +test("Home offers Connect a model to people who manage connections", async ({ page, }) => { - const capture = await setup(page, ["catalog.read"]); - await page.getByRole("button", { name: "Settings", exact: true }).click(); - await expect( - page.getByRole("button", { name: "Weave AI settings", exact: true }), - ).toHaveCount(0); + await start(page); + await page.getByRole("button", { name: "Home", exact: true }).click(); + await page.getByRole("button", { name: "Connect a model" }).click(); + await expect(page).toHaveURL(/\/settings\/ai$/); +}); + +for (const size of [ + { width: 1440, height: 900 }, + { width: 600, height: 500 }, +]) + test.describe(`AI models page at ${size.width}x${size.height}`, () => { + test.use({ viewport: size }); + + test("shows providers, connections and the checklist", async ({ page }) => { + await start(page, { connections: [ollamaConnection, openaiConnection] }); + const cards = page.locator(".provider-card"); + await expect(cards).toHaveCount(4); + await expect(cards.nth(0)).toContainText("Ollama"); + await expect(cards.nth(0)).toContainText("Connected · 1"); + await expect(cards.nth(2)).toContainText("Not set up"); + await expect( + page.getByRole("button", { name: "Manage Ollama" }), + ).toBeVisible(); + await expect( + page.getByRole("button", { name: "Connect Anthropic" }), + ).toBeVisible(); + const row = page + .locator(".connection-row") + .filter({ hasText: "ollama-local" }); + await expect(row).toContainText("http://ollama:11434/v1"); + await expect(row).toContainText("Development only"); + await expect(row).toContainText("Not tested in this session"); + const items = page.locator(".readiness-row"); + await expect(items).toHaveCount(8); + await expect(items.nth(4)).toContainText("Key available · Not needed"); + await expect(items.nth(7)).toContainText("Weave AI · Not set up"); + expect( + await page.evaluate( + () => document.documentElement.scrollWidth <= innerWidth, + ), + ).toBe(true); + if (size.width < 1024) { + const readinessBox = await page.locator(".ai-readiness").boundingBox(); + const providersBox = await page.locator(".ai-providers").boundingBox(); + expect(readinessBox!.y + readinessBox!.height).toBeLessThanOrEqual( + providersBox!.y, + ); + } + }); + + test("Publish and Authorize run once and the checklist rechecks", async ({ + page, + }) => { + const platform = await start(page, { + connections: [ollamaConnection, openaiConnection], + readiness: readiness({ + actions_published: { + status: "action", + detail: "actions_missing", + fix: { + kind: "operation", + label: "Publish", + operation: "ai_setup.publish", + }, + }, + key_available: { status: "done", detail: "key_grant_available" }, + connection_authorized: { + status: "action", + detail: "authorization_missing", + fix: { + kind: "operation", + label: "Authorize", + operation: "ai_setup.grant", + }, + }, + }), + }); + const publish = page.getByRole("button", { + name: "Publish", + exact: true, + }); + await publish.dblclick(); + await expect(page.locator(".readiness-row").nth(1)).toContainText( + "AI actions published · Published", + ); + expect(platform.publishes).toBe(1); + await page + .getByRole("button", { name: "Authorize", exact: true }) + .click(); + await expect(page.locator(".readiness-row").nth(5)).toContainText( + "Connection authorized · Authorized", + ); + expect(platform.grants).toEqual([ + { connection_revision_id: OPENAI_REVISION }, + ]); + }); + }); + +test("a viewer sees the checklist without setup controls", async ({ page }) => { + await start(page, { + capabilities: ["catalog.read", "status.read"], + connections: [ollamaConnection], + readiness: readiness({ + connection: { + status: "unknown", + detail: "requires_deployer", + fix: { kind: "ask", label: "Requires deployer" }, + }, + key_available: { + status: "unknown", + detail: "requires_tenant_admin", + fix: { kind: "ask", label: "Requires tenant_admin" }, + }, + connection_authorized: { + status: "unknown", + detail: "requires_tenant_admin", + fix: { kind: "ask", label: "Requires tenant_admin" }, + }, + model_responds: { + status: "unknown", + detail: "requires_tenant_admin", + fix: { kind: "ask", label: "Requires tenant_admin" }, + }, + weave_ai: { + status: "unknown", + detail: "requires_lumi_user", + fix: { kind: "ask", label: "Requires lumi_user" }, + }, + }), + }); + await expect(page.locator(".readiness-row").nth(4)).toContainText( + "Key available · Requires Tenant administrator", + ); + await expect(page.locator(".readiness-row").nth(7)).toContainText( + "Weave AI · Requires Weave AI user", + ); + expect(await page.locator(".ai-readiness").textContent()).not.toContain( + "tenant_admin", + ); + await expect(page.getByRole("button", { name: /^Connect / })).toHaveCount(0); + await expect(page.locator(".ai-providers")).toContainText( + "Seeing AI connections requires the Deployer role.", + ); +}); + +test("the operator command is copied and the workers link opens Operate", async ({ + page, + context, +}) => { + await context.grantPermissions(["clipboard-read", "clipboard-write"]); + await start(page, { + readiness: readiness({ + worker_installed: { + status: "action", + detail: "worker_missing", + fix: { + kind: "command", + label: "Ask an operator to enable AI", + command: "weave platform ai enable --ollama auto", + }, + }, + worker_online: { + status: "action", + detail: "worker_offline", + fix: { + kind: "navigate", + label: "Open workers", + route: "operate/workers", + }, + }, + }), + }); + await page + .getByRole("button", { + name: "Copy weave platform ai enable --ollama auto", + }) + .click(); + expect(await page.evaluate(() => navigator.clipboard.readText())).toBe( + "weave platform ai enable --ollama auto", + ); + await page.getByRole("button", { name: "Open workers" }).click(); + await expect(page).toHaveURL(/\/operate\/workers$/); +}); + +test("the Weave AI and providers links name their card and move focus to it", async ({ + page, +}) => { + await page.setViewportSize({ width: 1280, height: 520 }); + await start(page, { + connections: [], + readiness: readiness({ + connection: { + status: "action", + detail: "connection_missing", + fix: { + kind: "navigate", + label: "Open AI settings", + route: "settings/ai", + }, + }, + }), + }); + const checklist = page.locator(".ai-readiness"); + await expect(checklist).not.toContainText("Open AI settings"); + await checklist.getByRole("button", { name: "Go to Weave AI" }).click(); + const weaveAi = page.locator("#weave-ai-title"); + await expect(weaveAi).toBeFocused(); + await expect(weaveAi).toBeInViewport(); + await page.evaluate(() => scrollTo(0, 0)); + await checklist.getByRole("button", { name: "Go to providers" }).click(); + const providers = page.locator("#ai-providers-title"); + await expect(providers).toBeFocused(); + await expect(providers).toBeInViewport(); + await expect(page).toHaveURL(/\/settings\/ai$/); +}); + +test("saves Weave AI on a cloud connection with a suggested model, keeping its limits", async ({ + page, +}) => { + const platform = await start(page, { + connections: [ollamaConnection, openaiConnection], + models: [ + { + name: "gpt-5-mini", + approved: true, + available: true, + tools: "yes", + context_tokens: 128000, + }, + ], + }); + const card = page.locator("weave-ai-card"); + await expect(card.getByRole("heading", { name: "Weave AI" })).toBeVisible(); + await card.getByLabel("Connection", { exact: true }).click(); await expect( - page.getByRole("button", { name: "New AI connection", exact: true }), - ).toHaveCount(0); - await page.getByRole("button", { name: "Ask Weave AI", exact: true }).click(); - const panel = page.getByRole("dialog", { name: "Ask Weave AI", exact: true }); + page.getByRole("option", { name: /ollama-local/ }), + ).toHaveAttribute("aria-disabled", "true"); + await page.getByRole("option", { name: /openai · revision 1/ }).click(); + await expect(card.getByLabel("Model", { exact: true })).toHaveValue( + "gpt-5-mini", + ); + await card.getByRole("button", { name: "Save Weave AI settings" }).click(); + await expect.poll(() => platform.lumiWrites.length).toBe(1); + expect(platform.lumiWrites[0]).toEqual({ + ifMatch: null, + body: { + enabled: true, + connection_revision_id: OPENAI_REVISION, + profile: { + options: { max_tokens: 1024 }, + reasoning: { pattern: "none", maxSteps: 6 }, + maxCalls: 8, + timeoutSeconds: 180, + provider: "openai-responses", + model: "gpt-5-mini", + outputSchema: { type: "object" }, + }, + }, + }); + await expect.poll(() => platform.readinessReads).toBeGreaterThan(1); +}); + +test("explains that connections without a key can't be used for Weave AI yet", async ({ + page, +}) => { + await start(page, { connections: [ollamaConnection] }); + await expect(page.locator("weave-ai-card")).toContainText( + "Weave AI needs an AI connection with a secret handle. Connections without a key can't be used for Weave AI yet.", + ); +}); + +test("a Weave AI manager without connection management sees why the card is read-only", async ({ + page, +}) => { + await start(page, { + capabilities: ["catalog.read", "status.read", "lumi.use", "lumi.manage"], + }); + await expect(page.locator("weave-ai-card")).toContainText( + "Choosing Weave AI's connection also requires the Tenant administrator role.", + ); await expect( - panel.getByRole("button", { name: "Weave AI settings", exact: true }), + page.getByRole("button", { name: "Save Weave AI settings" }), ).toHaveCount(0); - await expect( - panel.getByRole("button", { name: "Send message", exact: true }), - ).toBeDisabled(); - expect(capture.creates).toEqual([]); - expect(capture.configurations).toEqual([]); }); -test("AI model basics and advanced options preserve edits, invalid drafts and explicit clears", async ({ +test("saving Weave AI on a configured connection keeps its limits and sends its revision", async ({ page, }) => { - const capture = await setup(page); - capture.connections.push({ - id: connectionId, - name: "approved-ai", - revision: 1, - connector: "weave-agentic-provider@1.0.0", - config: { - provider: "openai-responses", - endpoint: "https://api.openai.com/v1", + const platform = await start(page, { + connections: [ollamaConnection, openaiConnection], + models: [ + { + name: "gpt-5-mini", + approved: true, + available: true, + tools: "yes", + context_tokens: 128000, + }, + ], + lumi: { + enabled: true, + connection_revision_id: OPENAI_REVISION, + profile: { + provider: "openai-responses", + model: "gpt-5-mini", + options: { max_tokens: 512 }, + reasoning: { pattern: "none", maxSteps: 6 }, + maxCalls: 4, + timeoutSeconds: 120, + outputSchema: { type: "object" }, + }, + revision: 7, + }, + }); + const card = page.locator("weave-ai-card"); + await expect(card.getByLabel("Model", { exact: true })).toHaveValue( + "gpt-5-mini", + ); + await card.getByRole("switch", { name: "Turn on Weave AI" }).uncheck(); + await card.getByRole("button", { name: "Save Weave AI settings" }).click(); + await expect.poll(() => platform.lumiWrites.length).toBe(1); + expect(platform.lumiWrites[0].ifMatch).toBe('"7"'); + expect(platform.lumiWrites[0].body).toMatchObject({ + enabled: false, + profile: { + options: { max_tokens: 512 }, + maxCalls: 4, + timeoutSeconds: 120, + model: "gpt-5-mini", }, }); - const profile = { +}); + +const GPT = { + name: "gpt-5-mini", + approved: true, + available: true, + tools: "yes", + context_tokens: 128000, +}; +const pinnedOpenAi = { + enabled: true, + connection_revision_id: OPENAI_REVISION, + profile: { provider: "openai-responses", - model: "approved-model", - options: { max_tokens: 1024, temperature: 0.5, top_p: 0.8 }, - reasoning: { pattern: "none", maxSteps: 7 }, - maxCalls: 9, - timeoutSeconds: 90, - outputSchema: lumiSchema.$defs.LumiProfile.properties.outputSchema.const, - }; - let saved = { - enabled: true, - connection_revision_id: connectionId, - profile, - revision: 1, - }; - await page.route("**/lumi/configuration", async (route) => { - if (route.request().method() === "PUT") { - const body = route.request().postDataJSON(); - capture.configurations.push(body); - saved = { ...body, revision: saved.revision + 1 }; - } - await route.fulfill({ json: saved }); + model: "gpt-5-mini", + outputSchema: { type: "object" }, + }, + revision: 7, +}; + +test("after its connection gets a new revision, Weave AI says so and saves on the new one", async ({ + page, +}) => { + const platform = await start(page, { + connections: [ollamaConnection, openaiConnection], + models: [GPT], + lumi: pinnedOpenAi, }); - await page.getByRole("button", { name: "Settings", exact: true }).click(); - await page - .getByRole("button", { name: "Weave AI settings", exact: true }) - .click(); - const lumi = page.getByRole("dialog", { name: "Ask Weave AI", exact: true }); - await expect(lumi.getByLabel("Model", { exact: true })).toHaveValue( - "approved-model", + const card = page.locator("weave-ai-card"); + await expect(card.getByLabel("Model", { exact: true })).toHaveValue( + "gpt-5-mini", ); - await expect(lumi.getByLabel("Max tokens", { exact: true })).toHaveValue( - "1024", + // Edit the connection Weave AI uses: saving adds revision 2, and the card + // reads Weave AI's settings again. + let reads = 0; + page.on("request", (request) => { + if ( + request.method() === "GET" && + request.url().endsWith("/lumi/configuration") + ) + reads++; + }); + await page.getByRole("button", { name: "Test openai" }).click(); + const form = sheet(page, "openai"); + await form.getByLabel("Secret handle").fill("openai-team-key"); + await form.getByRole("button", { name: "Save as a new revision" }).click(); + await expect(form.getByRole("status").last()).toContainText("revision 2"); + await page.keyboard.press("Escape"); + await expect(form).toHaveCount(0); + await expect.poll(() => reads).toBeGreaterThan(0); + + await expect(card).toContainText( + "Weave AI uses revision 1 of openai; saving moves it to revision 2.", ); - await expect( - lumi.getByRole("spinbutton", { - name: "Temperature (optional)", - exact: true, - }), - ).not.toBeVisible(); - await page.screenshot({ - path: "../.superpowers/editor-ux/ai-profile-basic.png", + await expect(card.getByLabel("Connection", { exact: true })).toHaveValue( + "openai · revision 2", + ); + await card.getByRole("switch", { name: "Turn on Weave AI" }).uncheck(); + await card.getByRole("button", { name: "Save Weave AI settings" }).click(); + await expect.poll(() => platform.lumiWrites.length).toBe(1); + const second = platform.connections.find( + (c) => c["name"] === "openai" && c["revision"] === 2, + )!; + expect(platform.lumiWrites[0].ifMatch).toBe('"7"'); + expect(platform.lumiWrites[0].body).toMatchObject({ + enabled: false, + connection_revision_id: second["id"], + profile: { model: "gpt-5-mini" }, }); - const advanced = lumi.getByText("Advanced model settings", { exact: true }); - await advanced.click(); - await page.screenshot({ - path: "../.superpowers/editor-ux/ai-profile-advanced.png", +}); + +test("when Weave AI's connection can't be found, the card asks for one before saving", async ({ + page, +}) => { + const platform = await start(page, { + connections: [ollamaConnection], + models: [GPT], + lumi: pinnedOpenAi, }); - await lumi - .getByRole("spinbutton", { name: "Temperature (optional)", exact: true }) - .fill("3"); - await advanced.click(); - await lumi.getByLabel("Model", { exact: true }).fill("updated-model"); + const card = page.locator("weave-ai-card"); + await expect(card).toContainText( + "Weave AI's connection is no longer available. Choose a connection, then save.", + ); await expect( - lumi.getByRole("button", { name: "Continue to connection", exact: true }), + card.getByRole("button", { name: "Save Weave AI settings" }), ).toBeDisabled(); - await advanced.click(); - await expect( - lumi.getByRole("spinbutton", { - name: "Temperature (optional)", - exact: true, - }), - ).toHaveValue("3"); - await lumi - .getByRole("spinbutton", { name: "Temperature (optional)", exact: true }) - .fill(""); - await advanced.click(); - await lumi.getByLabel("Max tokens", { exact: true }).fill("2048"); - await lumi - .getByRole("button", { name: "Continue to connection", exact: true }) - .click(); - await lumi - .getByRole("button", { name: "Review settings", exact: true }) - .click(); - await lumi - .getByRole("button", { name: "Save Weave AI settings", exact: true }) - .click(); - await expect(lumi.getByLabel("Message to Weave AI")).toBeVisible(); - expect(capture.configurations[0].profile).toEqual({ - ...profile, - model: "updated-model", - options: { max_tokens: 2048, top_p: 0.8 }, + expect(platform.lumiWrites).toEqual([]); +}); + +test("when the newest revision of Weave AI's connection has no key, the card says so and keeps Save off", async ({ + page, +}) => { + const keyless = { + ...openaiConnection, + id: "55555555-5555-4555-8555-555555555555", + revision: 2, + secretRef: { apiKey: "no-credential" }, + }; + const platform = await start(page, { + connections: [ollamaConnection, openaiConnection, keyless], + models: [GPT], + lumi: pinnedOpenAi, }); - await lumi - .getByRole("button", { name: "Weave AI settings", exact: true }) - .click(); - await lumi.getByText("Advanced model settings", { exact: true }).click(); + const card = page.locator("weave-ai-card"); + await expect(card).toContainText( + "The newest revision of openai (revision 2) has no key, and Weave AI can't use a connection without a key yet. Choose a connection with a key, then save.", + ); + await expect(card).not.toContainText("saving moves it"); await expect( - lumi.getByRole("spinbutton", { - name: "Temperature (optional)", - exact: true, - }), - ).toHaveValue(""); + card.getByRole("button", { name: "Save Weave AI settings" }), + ).toBeDisabled(); + expect(platform.lumiWrites).toEqual([]); +}); + +test("after its connection gets a new revision, Weave AI lists the models of the revision it will save", async ({ + page, +}) => { + const second = { + ...openaiConnection, + id: "66666666-6666-4666-8666-666666666666", + revision: 2, + }; + const platform = await start(page, { + connections: [ollamaConnection, openaiConnection, second], + models: [GPT], + lumi: pinnedOpenAi, + }); + const card = page.locator("weave-ai-card"); + await expect(card).toContainText( + "Weave AI uses revision 1 of openai; saving moves it to revision 2.", + ); + await expect + .poll(() => platform.modelRequests.some((url) => url.includes(second.id))) + .toBe(true); + expect( + platform.modelRequests.filter((url) => url.includes(OPENAI_REVISION)), + ).toEqual([]); }); -test("provider setup explains missing catalog permission without creating a connection", async ({ +test("when Weave AI's connection can't be read, the card says what happened and keeps Save off", async ({ page, }) => { - const capture = await setup(page); - await page.route("**/projects/project/connectors?*", (r) => + await connected(page, { capabilities: AI_CAPABILITIES }); + const platform = await mockAiPlatform(page, { + connections: [ollamaConnection], + models: [GPT], + lumi: pinnedOpenAi, + }); + await page.route(`**/connections/${OPENAI_REVISION}`, (r) => r.fulfill({ status: 403, - json: { code: "WV-FORBIDDEN", message: "Access denied" }, + json: { code: "WV-FORBIDDEN", message: "Not allowed" }, }), ); - await page.getByRole("button", { name: "Connections", exact: true }).click(); - await page - .getByRole("button", { name: "New AI connection", exact: true }) - .click(); - const form = page.getByRole("dialog", { - name: "New AI connection", - exact: true, - }); - await expect(form).toContainText("catalog.read"); + await openAiModels(page); + const card = page.locator("weave-ai-card"); + await expect(card.getByRole("alert")).toContainText( + "Studio couldn't read the connection Weave AI uses.", + ); + await expect(card).not.toContainText("no longer available"); await expect( - form.getByRole("button", { name: "Continue", exact: true }), + card.getByRole("button", { name: "Save Weave AI settings" }), ).toBeDisabled(); - expect(capture.creates).toEqual([]); + expect(platform.lumiWrites).toEqual([]); }); -test("canonical model validation blocks pending and invalid settings and ignores stale replies", async ({ +const teamOpenAi = { + ...openaiConnection, + id: "44444444-4444-4444-8444-444444444444", + name: "openai-team", + secretRef: { apiKey: "openai-team-key" }, +}; +const authorizeFix = { + status: "action", + detail: "authorization_missing", + fix: { + kind: "operation", + label: "Authorize", + operation: "ai_setup.grant", + }, +}; + +test("Authorize goes on past a refused grant, then rechecks and says what failed", async ({ page, }) => { - await setup(page); - const requests: Route[] = []; - await page.route("**/studio/contracts/llm-profile/validate", (route) => { - requests.push(route); - }); - await page.getByRole("button", { name: "Settings", exact: true }).click(); - await page - .getByRole("button", { name: "Weave AI settings", exact: true }) - .click(); - const lumi = page.getByRole("dialog", { name: "Ask Weave AI", exact: true }); - await selectChoice( - lumi.getByLabel("Provider", { exact: true }), - "openai-responses", - ); - await lumi.getByLabel("Model", { exact: true }).fill("first-model"); - await expect.poll(() => requests.length).toBe(1); - const next = lumi.getByRole("button", { - name: "Continue to connection", - exact: true, - }); - await expect(next).toBeDisabled(); - await lumi.getByLabel("Model", { exact: true }).fill("second-model"); - await expect.poll(() => requests.length).toBe(2); - const message = - "A model call timeout cannot exceed the total AI step timeout"; - await requests[1].fulfill({ - json: { valid: false, issues: [{ path: [], message }] }, - }); - await expect(lumi.getByRole("alert")).toContainText(message); - await requests[0].fulfill({ json: { valid: true, issues: [] } }); - await expect(next).toBeDisabled(); - await expect(lumi.getByRole("alert")).toContainText(message); - await lumi.getByLabel("Model", { exact: true }).fill("corrected-model"); - await expect.poll(() => requests.length).toBe(3); - expect(requests[2].request().postDataJSON()).toMatchObject({ - provider: "openai-responses", - model: "corrected-model", - outputSchema: {}, + const platform = await start(page, { + connections: [openaiConnection, teamOpenAi], + readiness: readiness({ + key_available: { status: "done", detail: "key_grant_available" }, + connection_authorized: authorizeFix, + }), + grantAnswers: [ + { + status: 500, + body: { code: "WV-INTERNAL", message: "The grant was not saved." }, + }, + ], }); - expect(requests[2].request().postDataJSON()).not.toHaveProperty("profile"); - await requests[2].fulfill({ json: { valid: true, issues: [] } }); - await expect(next).toBeEnabled(); - await expect(lumi.getByRole("alert")).toHaveCount(0); + await expect(page.locator(".readiness-row")).toHaveCount(8); + const reads = platform.readinessReads; + await page.getByRole("button", { name: "Authorize", exact: true }).click(); + await expect(page.locator(".readiness-error")).toBeVisible(); + expect(platform.grants).toHaveLength(2); + await expect.poll(() => platform.readinessReads).toBeGreaterThan(reads); }); -test("retrying an uncertain connection create reuses the same idempotency key", async ({ +test("the checklist's Test connection opens the first connection without a passing test", async ({ page, }) => { - await setup(page); - const requests: { body: unknown; key: string | undefined }[] = []; - await page.route("**/environments/development/connections", async (r) => { - const body = r.request().postDataJSON(); - requests.push({ body, key: r.request().headers()["idempotency-key"] }); - if (requests.length === 1) - await r.fulfill({ status: 503, json: { message: "Temporary failure" } }); - else - await r.fulfill({ - json: { - ...body, - id: connectionId, - revision: 1, - connector: "weave-agentic-provider@1.0.0", + await start(page, { + connections: [ollamaConnection, openaiConnection], + readiness: readiness({ + model_responds: { + status: "action", + detail: "test_required", + fix: { + kind: "operation", + label: "Test connection", + operation: "ai_connections.test", }, - }); + }, + }), }); - await page.getByRole("button", { name: "Connections", exact: true }).click(); + // ollama-local passes a test in this session; openai has none. + await page.getByRole("button", { name: "Test ollama-local" }).click(); + const tested = sheet(page, "ollama-local"); + await tested.getByRole("button", { name: "Test connection" }).click(); + await expect(tested.locator(".test-status")).toContainText("can call tools"); + await page.keyboard.press("Escape"); + await expect(tested).toHaveCount(0); await page - .getByRole("button", { name: "New AI connection", exact: true }) + .locator(".ai-readiness") + .getByRole("button", { name: "Test connection", exact: true }) .click(); - const form = page.getByRole("dialog", { - name: "New AI connection", - exact: true, + await expect(sheet(page, "openai")).toBeVisible(); +}); + +test("Tab pressed while focus is on the page goes back into the dialog", async ({ + page, +}) => { + await start(page, { connections: [ollamaConnection] }); + await page.getByRole("button", { name: "Test ollama-local" }).click(); + const form = sheet(page, "ollama-local"); + await expect( + form.getByRole("button", { name: "Test connection" }), + ).toBeEnabled(); + // Focus falls to the body and Tab arrives before the dialog's focus + // guard runs: the key itself brings focus back to the dialog. + const caught = await page.evaluate(() => { + (document.activeElement as HTMLElement).blur(); + const tab = new KeyboardEvent("keydown", { + key: "Tab", + bubbles: true, + cancelable: true, + }); + document.dispatchEvent(tab); + return { + prevented: tab.defaultPrevented, + inDialog: !!document.activeElement?.closest('[role="dialog"]'), + }; }); - await form.getByLabel("Connection name", { exact: true }).fill("provider"); - await selectChoice( - form.getByLabel("Provider", { exact: true }), - "openai-responses", - ); - await form.getByRole("button", { name: "Continue", exact: true }).click(); - await form.getByText("Advanced: custom endpoint", { exact: true }).click(); - await form - .getByLabel("Provider endpoint", { exact: true }) - .fill("https://APPROVED.example.com:443/v1/"); - await form - .getByLabel("API key secret handle", { exact: true }) - .fill("model-key"); - await form - .getByRole("button", { name: "Review connection", exact: true }) - .click(); - const submit = form.getByRole("button", { - name: "Create AI connection", - exact: true, + expect(caught).toEqual({ prevented: true, inDialog: true }); + await page.keyboard.press("Tab"); + expect(await focusInSheet(page)).not.toBeNull(); +}); +/** The accessible name of the focused element, when it sits in the AI connection sheet. */ +const focusInSheet = (page: Page) => + page.evaluate(() => { + const active = document.activeElement as HTMLElement | null; + if (!active?.closest('[role="dialog"]')) return null; + return ( + active.getAttribute("aria-label") ?? + active.textContent?.trim() ?? + active.tagName + ); }); - await submit.click(); - await expect(form.getByRole("alert")).toBeVisible(); - await submit.click(); - await expect(form).toContainText("Connection created"); - expect(requests).toHaveLength(2); - expect(requests[0].key).toBeTruthy(); - expect(requests[1]).toEqual(requests[0]); - expect(requests[0].body).toMatchObject({ - config: { endpoint: "https://approved.example.com/v1/" }, - allowed_destinations: ["https://approved.example.com"], + +test("keeps keyboard focus in the sheet while a test runs, ends and saves", async ({ + page, +}) => { + const platform = await start(page, { + connections: [ollamaConnection], + testDelay: 60_000, }); + await page.getByRole("button", { name: "Test ollama-local" }).click(); + const form = sheet(page, "ollama-local"); + const testButton = form.getByRole("button", { name: "Test connection" }); + await expect(testButton).toBeEnabled(); + await testButton.focus(); + await page.keyboard.press("Enter"); + await expect(form.getByRole("button", { name: "Cancel test" })).toBeFocused(); + await page.keyboard.press("Enter"); + await expect( + form.getByRole("status").filter({ hasText: "canceled" }), + ).toHaveText("Test canceled."); + await expect(testButton).toBeFocused(); + platform.testDelay = 0; + await page.keyboard.press("Enter"); + await expect( + form.getByRole("status").filter({ hasText: "Connected." }), + ).toBeVisible(); + expect(await focusInSheet(page)).not.toBeNull(); + await page.keyboard.press("Tab"); + expect(await focusInSheet(page)).not.toBeNull(); + await page.keyboard.press("Escape"); + await expect(form).toHaveCount(0); + + await page.getByRole("button", { name: "Connect OpenAI" }).click(); + const created = sheet(page); + await created.getByRole("button", { name: "Save connection" }).focus(); + await page.keyboard.press("Enter"); + await expect(created.getByLabel("Model", { exact: true })).toBeFocused(); + await page.keyboard.press("Escape"); + await expect(page.getByRole("dialog")).toHaveCount(0); }); -for (const width of [1440, 360]) - test.describe(`AI connection wizard at ${width}`, () => { - test.use({ viewport: { width, height: 800 } }); - test("standard providers need no endpoint entry and distinguish saving from testing", async ({ - page, - }) => { - const capture = await setup(page); - await page - .getByRole("button", { name: "Connections", exact: true }) - .click(); - await page - .getByRole("button", { name: "New AI connection", exact: true }) - .click(); - const form = page.getByRole("dialog", { - name: "New AI connection", - exact: true, - }); - await form.getByLabel("Connection name", { exact: true }).fill("team-ai"); - await selectChoice( - form.getByLabel("Provider", { exact: true }), - "openai-responses", - ); - await form.getByRole("button", { name: "Continue", exact: true }).click(); - await expect( - form.getByRole("heading", { name: "Provider access", exact: true }), - ).toBeVisible(); - await expect( - form.getByLabel("Provider endpoint", { exact: true }), - ).not.toBeVisible(); - await expect(form).toContainText("https://api.openai.com/v1"); - await form - .getByLabel("API key secret handle", { exact: true }) - .fill("model-key"); - await form.getByRole("button", { name: "Back", exact: true }).click(); - await selectChoice( - form.getByLabel("Provider", { exact: true }), - "anthropic", - ); - await form.getByRole("button", { name: "Continue", exact: true }).click(); - await expect(form).toContainText("https://api.anthropic.com"); - await form - .getByRole("button", { name: "Review connection", exact: true }) - .click(); - await form - .getByRole("button", { name: "Create AI connection", exact: true }) - .click(); - await expect(form).toContainText("Connection created"); - await expect(form).toContainText("Not tested"); - expect(capture.creates[0].config.endpoint).toBe( - "https://api.anthropic.com", +test("opening the sheet moves keyboard focus into it, onto the chosen provider", async ({ + page, +}) => { + await start(page, {}); + await page.getByRole("button", { name: "Connect Ollama" }).click(); + const form = sheet(page); + await expect( + form.getByRole("button", { name: "Save connection" }), + ).toBeEnabled(); + await expect( + form + .getByRole("group", { name: "Provider" }) + .getByRole("radio", { checked: true }), + ).toBeFocused(); + for (let press = 0; press < 3; press++) { + await page.keyboard.press("Tab"); + expect(await focusInSheet(page)).not.toBeNull(); + } + await page.keyboard.press("Escape"); + await expect( + page.getByRole("button", { name: "Connect Ollama" }), + ).toBeFocused(); +}); + +test("a dialog takes focus back when its focused control is disabled", async ({ + page, +}) => { + await start(page, { connections: [ollamaConnection], testDelay: 60_000 }); + await page.getByRole("button", { name: "Test ollama-local" }).click(); + const form = sheet(page, "ollama-local"); + await expect( + form.getByRole("button", { name: "Test connection" }), + ).toBeEnabled(); + // A control that disables itself while focused, as a busy button does. + await page.evaluate(() => { + const button = [ + ...document.querySelectorAll('[role="dialog"] button'), + ].find((item) => item.textContent?.trim() === "Test connection")!; + button.focus(); + button.disabled = true; + }); + await expect.poll(() => focusInSheet(page)).not.toBeNull(); + await page.keyboard.press("Escape"); + await expect(form).toHaveCount(0); +}); + +test("the desktop sheet keeps its footer in one row at the bottom edge", async ({ + page, +}) => { + await page.setViewportSize({ width: 1440, height: 900 }); + await start(page); + const form = await newConnection(page, "Ollama"); + await expect( + form.getByRole("button", { name: "Save connection" }), + ).toBeVisible(); + const measure = () => + page.evaluate(() => { + const panel = document.querySelector(".modal-panel.side")!; + const footer = panel.querySelector(".dialog-footer")!; + const buttons = [...footer.querySelectorAll("button")].map((button) => + button.getBoundingClientRect(), ); - expect(capture.creates[0].allowed_destinations).toEqual([ - "https://api.anthropic.com", - ]); - expect( - await page.evaluate( - () => document.documentElement.scrollWidth <= innerWidth, - ), - ).toBe(true); + return { + panelBottom: panel.getBoundingClientRect().bottom, + footerBottom: footer.getBoundingClientRect().bottom, + tops: buttons.map((box) => Math.round(box.top)), + heights: buttons.map((box) => Math.round(box.height)), + }; }); - test("wizard guides keyboard review without creating early and retains Back edits", async ({ - page, - }) => { - const capture = await setup(page); - await page - .getByRole("button", { name: "Connections", exact: true }) - .click(); - await page - .getByRole("button", { name: "New AI connection", exact: true }) - .click(); - const form = page.getByRole("dialog", { - name: "New AI connection", - exact: true, - }); - const progress = form.getByRole("list", { name: "AI connection setup" }); - await expect(progress.locator('[aria-current="step"]')).toHaveText( - /Provider/, - ); - await expect( - form.getByLabel("Provider endpoint", { exact: true }), - ).toHaveCount(0); - await form.getByRole("button", { name: "Continue", exact: true }).click(); - await expect(form.getByRole("alert")).toContainText("connection name"); - expect(capture.creates).toHaveLength(0); - await selectChoice( - form.getByLabel("Provider", { exact: true }), - "azure-chat", - ); - await form - .getByLabel("Connection name", { exact: true }) - .fill("approved-azure"); - await form.getByLabel("Connection name", { exact: true }).press("Enter"); - await expect( - form.getByRole("heading", { - name: "Provider access", - exact: true, - }), - ).toBeFocused(); - await expect(progress.locator('[aria-current="step"]')).toHaveText( - /Access/, - ); - await expect( - form.getByLabel("Provider endpoint", { exact: true }), - ).toHaveAttribute( - "placeholder", - "https://your-resource.openai.azure.com", - ); - await expect(form).toContainText("base endpoint for your Azure resource"); - await form - .getByLabel("Provider endpoint", { exact: true }) - .fill("https://approved.openai.azure.com"); - await form - .getByLabel("Azure API version", { exact: true }) - .fill("2024-10-21"); - await form - .getByLabel("API key secret handle", { exact: true }) - .fill("approved-key"); - await form - .getByLabel("API key secret handle", { exact: true }) - .press("Enter"); - await expect( - form.getByRole("heading", { - name: "Review your connection", - exact: true, - }), - ).toBeFocused(); - await expect(progress.locator('[aria-current="step"]')).toHaveText( - /Review/, + for (const step of ["new", "saved"]) { + if (step === "saved") + await form.getByRole("button", { name: "Save connection" }).click(); + if (step === "saved") + await expect(form.getByLabel("Model", { exact: true })).toHaveValue( + "qwen3:4b", ); - await expect(form).toContainText("Azure OpenAI Chat"); - await expect(form).toContainText("https://approved.openai.azure.com"); - await expect(form).toContainText("approved-key"); - await expect(form).toContainText("No model request or connectivity test"); - expect(capture.creates).toHaveLength(0); - await form.getByRole("button", { name: "Back", exact: true }).click(); - await expect( - form.getByLabel("API key secret handle", { exact: true }), - ).toHaveValue("approved-key"); - await form.getByRole("button", { name: "Back", exact: true }).click(); - await expect( - form.getByLabel("Connection name", { exact: true }), - ).toHaveValue("approved-azure"); - await form - .getByLabel("Connection name", { exact: true }) - .fill("reviewed-azure"); - await form.getByRole("button", { name: "Continue", exact: true }).click(); - await expect( - form.getByLabel("Azure API version", { exact: true }), - ).toHaveValue("2024-10-21"); - await form - .getByRole("button", { name: "Review connection", exact: true }) - .click(); - await expect(form).toContainText("reviewed-azure"); - expect(capture.creates).toHaveLength(0); - expect( - await page.evaluate( - () => document.documentElement.scrollWidth <= innerWidth, - ), - ).toBe(true); - await page.screenshot({ - path: `../.superpowers/editor-ux/ai-wizard-review-${width}.png`, - }); - await form - .getByRole("button", { name: "Create AI connection", exact: true }) - .press("Enter"); - await expect(form).toContainText("Connection created"); - expect(capture.creates).toHaveLength(1); - expect(capture.creates[0].name).toBe("reviewed-azure"); + const box = await measure(); + expect(box.tops).toHaveLength(2); + expect(new Set(box.tops).size, step).toBe(1); + expect( + box.heights.every((height) => height === 36), + step, + ).toBe(true); + expect(Math.abs(box.panelBottom - box.footerBottom), step).toBeLessThan(1); + } +}); + +for (const size of [ + { width: 1440, height: 900, control: 36 }, + { width: 600, height: 500, control: 44 }, +]) + test(`the Model field lines up with Refresh models at ${size.width}x${size.height}`, async ({ + page, + }) => { + await page.setViewportSize(size); + await start(page, { connections: [ollamaConnection] }); + await page.getByRole("button", { name: "Test ollama-local" }).click(); + const form = sheet(page, "ollama-local"); + await expect(form.getByLabel("Model", { exact: true })).toHaveValue( + "qwen3:4b", + ); + const box = await page.evaluate(() => { + const panel = document.querySelector(".modal-panel.side")!; + const rect = (selector: string) => + panel.querySelector(selector)!.getBoundingClientRect(); + const field = rect(".model-row .choice-field"); + const refresh = rect('[aria-label="Refresh models"]'); + const name = rect("#ai-connection-name"); + return { field, refresh, name }; }); + expect(Math.round(box.field.height)).toBe(size.control); + expect(Math.round(box.refresh.height)).toBe(size.control); + expect(Math.abs(box.field.bottom - box.refresh.bottom)).toBeLessThan(0.5); + expect(Math.abs(box.field.left - box.name.left)).toBeLessThan(0.5); }); + +test("the sheet takes the connection's name once it is saved, and names its test section once", async ({ + page, +}) => { + await start(page); + const form = await newConnection(page, "Ollama"); + await expect(form.getByRole("heading", { level: 2 })).toHaveText( + "New AI connection", + ); + await form.getByRole("button", { name: "Save connection" }).click(); + await expect(form.getByRole("heading", { level: 2 })).toHaveText( + "ollama-local", + ); + await expect(form.getByRole("heading", { level: 3 })).toHaveText( + "Test a model", + ); + await expect(form.getByText("Model", { exact: true })).toHaveCount(1); +}); diff --git a/studio/tests/browser/ai-task-profile.spec.ts b/studio/tests/browser/ai-task-profile.spec.ts new file mode 100644 index 00000000..f06d9697 --- /dev/null +++ b/studio/tests/browser/ai-task-profile.spec.ts @@ -0,0 +1,307 @@ +/* +Copyright 2026 Firefly Software Foundation. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +Author: Firefly Software Foundation +SPDX-License-Identifier: Apache-2.0 +*/ +// The AI task's workflow profiles. AI connections themselves are covered by ai-setup.spec.ts. +import { execFileSync } from "node:child_process"; +import { test, expect, type Page, type Route } from "@playwright/test"; +import { parse } from "yaml"; +import { + allCapabilities, + connected, + newWorkflow, + sourceText, + selectChoice, +} from "./support"; +import { DesignerPage } from "./designer-po"; +import { python } from "../python-path"; + +const schema = (module: string, name: string) => + JSON.parse( + execFileSync( + python, + [ + "-c", + `import json; from firefly_weave.contracts.${module} import ${name}; print(json.dumps(${name}.model_json_schema(by_alias=True)))`, + ], + { cwd: "..", encoding: "utf8" }, + ), + ); +const lumiSchema = schema("lumi", "LumiConfigurationRequest"); +const llmSchema = schema("llm", "LLMProfile"); +const connectorId = "11111111-1111-4111-8111-111111111111"; +const connectionId = "22222222-2222-4222-8222-222222222222"; +async function setup( + page: Page, + capabilities = [...allCapabilities, "lumi.use", "lumi.manage"], +) { + await connected(page, { capabilities }); + const connections: Record[] = []; + const configurations: any[] = []; + const creates: any[] = []; + await page.route("**/studio/contracts/lumi-configuration", (r) => + r.fulfill({ json: lumiSchema }), + ); + await page.route("**/studio/contracts/llm-profile", (r) => + r.fulfill({ json: llmSchema }), + ); + await page.route("**/studio/contracts/llm-profile/validate", (r) => + r.fulfill({ json: { valid: true, issues: [] } }), + ); + await page.route("**/projects/project/connectors?*", (r) => + r.fulfill({ + json: { + items: [ + { id: connectorId, name: "weave-agentic-provider", version: "1.0.0" }, + ], + next_cursor: null, + }, + }), + ); + await page.route("**/environments/development/connections?*", (r) => + r.fulfill({ json: { items: connections, next_cursor: null } }), + ); + await page.route("**/environments/development/connections", async (r) => { + const body = r.request().postDataJSON(); + creates.push(body); + const connection = { + ...body, + id: connectionId, + revision: 1, + connector: "weave-agentic-provider@1.0.0", + }; + connections.push(connection); + await r.fulfill({ json: connection }); + }); + await page.route("**/lumi/status", (r) => + r.fulfill({ json: { configured: false } }), + ); + await page.route("**/lumi/configuration", async (r) => { + if (r.request().method() === "PUT") { + const body = r.request().postDataJSON(); + configurations.push(body); + await r.fulfill({ json: { ...body, revision: 1 } }); + } else + await r.fulfill({ + status: 404, + json: { code: "WV-NOT-FOUND", message: "Weave AI is not configured" }, + }); + }); + return { connections, configurations, creates }; +} +for (const width of [1440, 600]) + test.describe(`AI setup at ${width}`, () => { + test.use({ viewport: { width, height: 800 } }); + test("fresh workflow AI profile and connection slot remain separate from Weave AI", async ({ + page, + }) => { + const capture = await setup(page); + await newWorkflow(page); + const designer = new DesignerPage(page); + if (width <= 767) + await page + .getByRole("button", { name: "Insert step", exact: true }) + .click(); + await page + .locator(".palette") + .getByRole("button", { name: "AI task", exact: true }) + .click(); + await designer.selectStep("ask-ai-1"); + const form = page.locator("weave-llm-inspector"); + await form + .getByRole("button", { + name: "Configure workflow AI profiles", + exact: true, + }) + .click(); + await selectChoice( + form.getByLabel("Provider", { exact: true }), + "azure-responses", + ); + await form + .getByLabel("Model", { exact: true }) + .fill("workflow-deployment"); + await form.getByLabel("Max tokens", { exact: true }).fill("512"); + await form + .getByRole("button", { name: "Continue to connection", exact: true }) + .click(); + await form + .getByRole("button", { name: "Add AI connection slot", exact: true }) + .click(); + await expect( + form.getByLabel("AI connection slot", { exact: true }), + ).toHaveValue("ai"); + await form + .getByRole("button", { name: "Review settings", exact: true }) + .click(); + await form + .getByRole("button", { + name: "Apply workflow AI settings", + exact: true, + }) + .click(); + await form + .locator('[data-field="prompt"]') + .getByLabel("AI prompt", { exact: true }) + .fill("Summarize input"); + await expect( + form.getByLabel("Workflow AI profile", { exact: true }), + ).toHaveValue("default"); + const doc = parse(await sourceText(page)); + expect(doc.spec.llmProfiles.default.provider).toBe("azure-responses"); + expect(doc.spec.llmProfiles.default.model).toBe("workflow-deployment"); + expect(doc.spec.connections.ai.connector).toBe( + "weave-agentic-provider@1.0.0", + ); + expect(doc.spec.steps[0].connection).toBe("ai"); + expect(capture.configurations).toEqual([]); + }); + }); + +/** Opens the workflow AI profile dialog of a new AI task. */ +async function openProfiles(page: Page) { + await newWorkflow(page); + await page + .locator(".palette") + .getByRole("button", { name: "AI task", exact: true }) + .click(); + await new DesignerPage(page).selectStep("ask-ai-1"); + const form = page.locator("weave-llm-inspector"); + await form + .getByRole("button", { + name: "Configure workflow AI profiles", + exact: true, + }) + .click(); + return form; +} + +test("AI model basics and advanced options preserve edits, invalid drafts and explicit clears", async ({ + page, +}) => { + await setup(page); + const form = await openProfiles(page); + await selectChoice( + form.getByLabel("Provider", { exact: true }), + "openai-responses", + ); + await form.getByLabel("Model", { exact: true }).fill("approved-model"); + await form.getByLabel("Max tokens", { exact: true }).fill("1024"); + await expect( + form.getByRole("spinbutton", { + name: "Temperature (optional)", + exact: true, + }), + ).not.toBeVisible(); + await page.screenshot({ + path: "../.superpowers/editor-ux/ai-profile-basic.png", + }); + const advanced = form.getByText("Advanced model settings", { exact: true }); + await advanced.click(); + await page.screenshot({ + path: "../.superpowers/editor-ux/ai-profile-advanced.png", + }); + await form + .getByRole("spinbutton", { name: "Temperature (optional)", exact: true }) + .fill("3"); + await advanced.click(); + await form.getByLabel("Model", { exact: true }).fill("updated-model"); + await expect( + form.getByRole("button", { name: "Continue to connection", exact: true }), + ).toBeDisabled(); + await advanced.click(); + await expect( + form.getByRole("spinbutton", { + name: "Temperature (optional)", + exact: true, + }), + ).toHaveValue("3"); + await form + .getByRole("spinbutton", { name: "Temperature (optional)", exact: true }) + .fill(""); + await advanced.click(); + await form.getByLabel("Max tokens", { exact: true }).fill("2048"); + await form + .getByRole("button", { name: "Continue to connection", exact: true }) + .click(); + await form + .getByRole("button", { name: "Add AI connection slot", exact: true }) + .click(); + await form + .getByRole("button", { name: "Review settings", exact: true }) + .click(); + await form + .getByRole("button", { name: "Apply workflow AI settings", exact: true }) + .click(); + const profile = parse(await sourceText(page)).spec.llmProfiles.default; + expect(profile).toMatchObject({ + provider: "openai-responses", + model: "updated-model", + options: { max_tokens: 2048 }, + }); + expect(profile["options"]).not.toHaveProperty("temperature"); +}); + +test("canonical model validation blocks pending and invalid settings and ignores stale replies", async ({ + page, +}) => { + await setup(page); + const requests: Route[] = []; + await page.route("**/studio/contracts/llm-profile/validate", (route) => { + requests.push(route); + }); + const form = await openProfiles(page); + await selectChoice( + form.getByLabel("Provider", { exact: true }), + "openai-responses", + ); + await form.getByLabel("Max tokens", { exact: true }).fill("512"); + await form.getByLabel("Model", { exact: true }).fill("first-model"); + const first = requests.length; + await expect.poll(() => requests.length).toBeGreaterThan(0); + const next = form.getByRole("button", { + name: "Continue to connection", + exact: true, + }); + await expect(next).toBeDisabled(); + await form.getByLabel("Model", { exact: true }).fill("second-model"); + await expect.poll(() => requests.length).toBeGreaterThan(Math.max(first, 1)); + const stale = requests.slice(0, -1); + const latest = requests[requests.length - 1]; + const message = + "A model call timeout cannot exceed the total AI step timeout"; + await latest.fulfill({ + json: { valid: false, issues: [{ path: [], message }] }, + }); + await expect(form.getByRole("alert")).toContainText(message); + for (const route of stale) + await route.fulfill({ json: { valid: true, issues: [] } }); + await expect(next).toBeDisabled(); + await expect(form.getByRole("alert")).toContainText(message); + const before = requests.length; + await form.getByLabel("Model", { exact: true }).fill("corrected-model"); + await expect.poll(() => requests.length).toBe(before + 1); + const corrected = requests[before]; + expect(corrected.request().postDataJSON()).toMatchObject({ + provider: "openai-responses", + model: "corrected-model", + outputSchema: {}, + }); + expect(corrected.request().postDataJSON()).not.toHaveProperty("profile"); + await corrected.fulfill({ json: { valid: true, issues: [] } }); + await expect(next).toBeEnabled(); + await expect(form.getByRole("alert")).toHaveCount(0); +}); diff --git a/studio/tests/browser/inspector-ux.spec.ts b/studio/tests/browser/inspector-ux.spec.ts index 01f8d5db..422ca8e3 100644 --- a/studio/tests/browser/inspector-ux.spec.ts +++ b/studio/tests/browser/inspector-ux.spec.ts @@ -403,6 +403,8 @@ for (const viewport of [ const inputs = designer.inspector.locator( '[data-field="spec/inputSchema"]', ); + // evaluate() does not wait for the summary inside the field to render. + await expect(inputs.locator("button.sd-summary").first()).toBeVisible(); const colors = await inputs.evaluate((element) => [ getComputedStyle(element.querySelector("button.sd-summary")!).color, getComputedStyle(element).color, diff --git a/studio/tests/browser/real-platform.spec.ts b/studio/tests/browser/real-platform.spec.ts index 5f759628..99d5f698 100644 --- a/studio/tests/browser/real-platform.spec.ts +++ b/studio/tests/browser/real-platform.spec.ts @@ -43,6 +43,8 @@ SPDX-License-Identifier: Apache-2.0 // quick-integration test needs it. // WEAVE_E2E_ACME_CONTROL The fixture's loopback control URL; when set, the // test also reads the fixture's request journal. +// WEAVE_E2E_AI_MODEL The model weave platform ai enable approved on this +// platform (qwen3:4b). Test 12 needs it. // // Build the app first (`npm run build`): the host serves // studio/dist/studio/browser. Every test starts its own host with a fresh @@ -1270,3 +1272,62 @@ test("11: sign out with revocation, then remove the platform", async ({ expectNoCredentialsLeft(); expect(s.errors).toEqual([]); }); + +test("12: the AI models screen reaches ready on a local model", async ({ + page, +}) => { + test.setTimeout(480_000); + const model = process.env["WEAVE_E2E_AI_MODEL"]?.trim() ?? ""; + test.skip( + !model, + "Set WEAVE_E2E_AI_MODEL after weave platform ai enable on this platform.", + ); + const { person } = setup(); + const s = await session(page); + const name = uniqueName("e2e-ai-models"); + await pair(page, s.host!, "12-pairing"); + await connectAndSignIn(page, name, person); + await openSidebar(page, "Settings"); + await page.getByRole("tab", { name: "AI models" }).click(); + await expect(page).toHaveURL(/\/settings\/ai$/); + const rows = page.locator(".readiness-row"); + await expect(rows).toHaveCount(8, { timeout: 30_000 }); + await shot(page, "12-ai-models"); + + // Test the local model from Studio; a cold model may take up to two minutes. + await page.getByRole("button", { name: "Test ollama-local" }).click(); + const form = page.getByRole("dialog", { name: "ollama-local" }); + await expect(form.getByLabel("Model", { exact: true })).toHaveValue(model, { + timeout: 30_000, + }); + await form.getByRole("button", { name: "Test connection" }).click(); + await expect(form.locator(".test-status")).toContainText( + `Connected. ${model} answered`, + { timeout: 170_000 }, + ); + await shot(page, "12-ai-connection-tested"); + await form.getByRole("button", { name: "Close AI connection" }).click(); + + // With only `weave platform ai enable` run, every item but Weave AI is ready. + await page.getByRole("button", { name: "Recheck" }).click(); + await expect(rows.nth(6)).toHaveAttribute("data-status", "done", { + timeout: 30_000, + }); + const statuses = await rows.evaluateAll((items) => + items.map((item) => [ + item.querySelector(".readiness-label")?.textContent ?? "", + item.getAttribute("data-status") ?? "", + ]), + ); + expect( + statuses + .filter(([label]) => label !== "Weave AI") + .every(([, status]) => status === "done" || status === "not_needed"), + JSON.stringify(statuses), + ).toBe(true); + await expect(rows.nth(7)).toHaveAttribute("data-status", "action"); + await shot(page, "12-ai-models-ready"); + expect(s.errors).toEqual([]); + await removePlatform(page, name); + expectNoCredentialsLeft(); +}); diff --git a/studio/tests/browser/settings-routes.spec.ts b/studio/tests/browser/settings-routes.spec.ts index 8da247b9..03c2595f 100644 --- a/studio/tests/browser/settings-routes.spec.ts +++ b/studio/tests/browser/settings-routes.spec.ts @@ -16,7 +16,7 @@ Author: Firefly Software Foundation SPDX-License-Identifier: Apache-2.0 */ import { expect, test } from "@playwright/test"; -import { offline } from "./support"; +import { allCapabilities, connected, offline } from "./support"; test("Settings tabs have their own addresses, and Preferences turns on the new editor", async ({ page, @@ -67,3 +67,68 @@ test("/settings opens Platforms, and the tabs follow the arrow keys", async ({ await expect(page.getByRole("tab", { name: "Platforms" })).toBeFocused(); await expect(page).toHaveURL(/\/settings\/platforms$/); }); + +test("AI models has its own address once a platform is connected", async ({ + page, +}) => { + await page.setViewportSize({ width: 1440, height: 900 }); + await connected(page, { capabilities: allCapabilities }); + await page.goto("/settings/ai"); + const tab = page.getByRole("tab", { name: "AI models" }); + await expect(tab).toHaveAttribute("aria-selected", "true"); + await expect(page.getByRole("tabpanel", { name: "AI models" })).toContainText( + "Connect a model once. Workflows and Weave AI use the connections you set up here.", + ); + await expect( + page.getByRole("button", { name: "Weave AI settings" }), + ).toHaveCount(0); + await page.getByRole("tab", { name: "Platforms" }).focus(); + await page.keyboard.press("ArrowRight"); + await expect(page.getByRole("tab", { name: "AI models" })).toBeFocused(); + await expect(page).toHaveURL(/\/settings\/ai$/); +}); + +test("AI models is not offered while working locally", async ({ page }) => { + await page.setViewportSize({ width: 1440, height: 900 }); + await offline(page); + await page.goto("/settings/ai"); + await expect(page).toHaveURL(/\/settings\/ai$/); + await expect(page.getByRole("tab", { name: "AI models" })).toHaveCount(0); + await expect(page.getByRole("tab", { name: "Platforms" })).toHaveAttribute( + "aria-selected", + "true", + ); +}); + +for (const size of [ + { width: 360, height: 740 }, + { width: 600, height: 500 }, +]) + test(`all four Settings tabs fit the strip at ${size.width}x${size.height}`, async ({ + page, + }) => { + await page.setViewportSize(size); + await connected(page, { + capabilities: [...allCapabilities, "grant.admin"], + }); + await page.goto("/settings/ai"); + const tabs = page.getByRole("tab"); + await expect(tabs).toHaveCount(4); + const strip = await page.getByRole("tablist").boundingBox(); + expect( + await page.evaluate( + () => + document.documentElement.scrollWidth <= innerWidth && + document.querySelector(".page-content")!.scrollWidth <= + document.querySelector(".page-content")!.clientWidth, + ), + ).toBe(true); + for (const tab of await tabs.all()) { + const box = (await tab.boundingBox())!; + expect(box.x).toBeGreaterThanOrEqual(strip!.x - 0.5); + expect(box.x + box.width).toBeLessThanOrEqual( + strip!.x + strip!.width + 0.5, + ); + expect(box.height).toBeGreaterThanOrEqual(43.5); + } + }); diff --git a/studio/tests/browser/visual-tour.spec.ts b/studio/tests/browser/visual-tour.spec.ts index 96fb1388..bc68eaf5 100644 --- a/studio/tests/browser/visual-tour.spec.ts +++ b/studio/tests/browser/visual-tour.spec.ts @@ -60,6 +60,15 @@ import { yamlFile, } from "./step-details-po"; import { showParameters } from "./step-geometry"; +import { + AI_CAPABILITIES, + mockAiPlatform, + ollamaConnection, + openaiConnection, + openAiModels, + readiness, + sheet, +} from "./ai-models-po"; import { CanvasPage, openWorkflow, useNewEditor } from "./canvas-po"; import { debugSession, @@ -1459,6 +1468,9 @@ const scenes: Record = { .getByLabel("Message to Weave AI") .fill("Help me add a review step."); await panel.getByLabel("Include current source", { exact: true }).check(); + await expect( + panel.getByRole("button", { name: "Set up Weave AI", exact: true }), + ).toBeVisible(); await shot("lumi-context", { end: ".modal-panel" }); await panel .getByRole("button", { name: "Send message", exact: true }) @@ -1469,11 +1481,6 @@ const scenes: Record = { .getByRole("button", { name: "Review workflow draft", exact: true }) .click(); await shot("lumi-review", { end: ".modal-panel" }); - await panel - .getByRole("button", { name: "Weave AI settings", exact: true }) - .click(); - await expect(panel.getByLabel("Model", { exact: true })).toBeVisible(); - await shot("lumi-settings", { end: ".modal-panel" }); }, // ------------------------------------------------------- step details async "step-details-api"({ page, shot, size }) { @@ -2038,6 +2045,134 @@ ${paths} await shot("33-settings-assign-role", { end: ".page-content" }); }, + async "settings-ai"({ page, shot }) { + await connected(page, { capabilities: AI_CAPABILITIES }); + await mockAiPlatform(page, { + connections: [ollamaConnection, openaiConnection], + }); + await openAiModels(page); + await expect(page.locator(".readiness-row")).toHaveCount(8); + await expect(page.locator("weave-ai-card")).toContainText( + "Save Weave AI settings", + ); + await shot("35-settings-ai-ready", { end: ".page-content" }); + }, + + async "settings-ai-read-only"({ page, shot }) { + await connected(page, { + capabilities: [ + "catalog.read", + "status.read", + "connection.bind", + "lumi.use", + ], + }); + await mockAiPlatform(page, { + connections: [ollamaConnection], + readiness: readiness({ + key_available: { status: "unknown", detail: "requires_tenant_admin" }, + connection_authorized: { + status: "unknown", + detail: "requires_tenant_admin", + }, + model_responds: { status: "unknown", detail: "requires_tenant_admin" }, + }), + }); + await openAiModels(page); + await expect( + page.getByRole("button", { name: "View ollama-local" }), + ).toBeVisible(); + await shot("35-settings-ai-read-only", { end: ".page-content" }); + }, + + async "settings-ai-setup"({ page, shot }) { + await connected(page, { capabilities: AI_CAPABILITIES }); + await mockAiPlatform(page, { + connections: [openaiConnection], + readiness: readiness({ + worker_installed: { + status: "action", + detail: "worker_missing", + fix: { + kind: "command", + label: "Ask an operator to enable AI", + command: "weave platform ai enable --ollama auto", + }, + }, + actions_published: { + status: "action", + detail: "actions_missing", + fix: { + kind: "operation", + label: "Publish", + operation: "ai_setup.publish", + }, + }, + worker_online: { + status: "action", + detail: "worker_offline", + fix: { + kind: "navigate", + label: "Open workers", + route: "operate/workers", + }, + }, + key_available: { status: "done", detail: "key_grant_available" }, + connection_authorized: { + status: "action", + detail: "authorization_missing", + fix: { + kind: "operation", + label: "Authorize", + operation: "ai_setup.grant", + }, + }, + model_responds: { + status: "action", + detail: "test_required", + fix: { + kind: "operation", + label: "Test connection", + operation: "ai_connections.test", + }, + }, + }), + }); + await openAiModels(page); + await expect( + page.getByRole("button", { name: "Publish", exact: true }), + ).toBeVisible(); + await shot("36-settings-ai-setup", { end: ".page-content" }); + }, + + async "ai-connection-sheet"({ page, shot }) { + await connected(page, { capabilities: AI_CAPABILITIES }); + const platform = await mockAiPlatform(page); + await openAiModels(page); + await page.getByRole("button", { name: "Connect Ollama" }).click(); + const form = sheet(page); + await expect( + form.getByRole("button", { name: "Save connection" }), + ).toBeEnabled(); + await shot("37-ai-connection-sheet", { end: ".modal-panel" }); + await form.getByRole("button", { name: "Save connection" }).click(); + await form.getByRole("button", { name: "Test connection" }).click(); + await expect(form.locator(".test-status")).toContainText("can call tools"); + await shot("37-ai-connection-tested", { end: ".modal-panel" }); + platform.testAnswer = { + ok: false, + code: "LLM_UNREACHABLE", + latency_ms: 12, + model: "qwen3:4b", + tool_calling: "unknown", + }; + await form.getByRole("button", { name: "Test connection" }).click(); + await expect(form.locator(".test-status")).toContainText( + "could not reach the model endpoint", + ); + await form.getByText("Show details").click(); + await shot("37-ai-connection-failed", { end: ".modal-panel" }); + }, async "designer-lanes"({ page, shot }) { await offline(page); await page diff --git a/studio/tests/browser/weave-ai.spec.ts b/studio/tests/browser/weave-ai.spec.ts index 11ec61a5..828ea415 100644 --- a/studio/tests/browser/weave-ai.spec.ts +++ b/studio/tests/browser/weave-ai.spec.ts @@ -15,7 +15,6 @@ limitations under the License. Author: Firefly Software Foundation SPDX-License-Identifier: Apache-2.0 */ -import { execFileSync } from "node:child_process"; import { expect, test } from "@playwright/test"; import { allCapabilities, @@ -25,7 +24,6 @@ import { sourceText, } from "./support"; import { DesignerPage } from "./designer-po"; -import { python } from "../python-path"; const proposal = `apiVersion: weave/v1alpha1 kind: Workflow @@ -179,186 +177,63 @@ test("context defaults off, stale proposals cannot replace later edits, and relo await expect(panel).not.toContainText("Private conversation"); }); -for (const viewport of [ - { width: 1440, height: 900 }, - { width: 390, height: 700 }, -]) - test(`Weave AI settings wizard preserves drafts and saves reviewed configuration at ${viewport.width}px`, async ({ - page, - }, testInfo) => { - await page.setViewportSize(viewport); - const schema = JSON.parse( - execFileSync( - python, - [ - "-c", - "import json; from firefly_weave.contracts.lumi import LumiConfigurationRequest; print(json.dumps(LumiConfigurationRequest.model_json_schema(by_alias=True)))", - ], - { cwd: "..", encoding: "utf8" }, - ), - ); - const fixed = schema.$defs.LumiProfile.properties.outputSchema.const; - await connected(page, { - capabilities: [...allCapabilities, "lumi.use", "lumi.manage"], - }); - await page.route("**/lumi/status", (r) => - r.fulfill({ - json: { - configured: true, - provider: "openai-responses", - model: "before", - }, - }), - ); - await page.route("**/studio/contracts/lumi-configuration", (r) => - r.fulfill({ json: schema }), - ); - await page.route("**/studio/contracts/llm-profile/validate", (r) => - r.fulfill({ json: { valid: true, issues: [] } }), - ); - const original = { - enabled: true, - connection_revision_id: "11111111-1111-4111-8111-111111111111", - profile: { - provider: "openai-responses", - model: "before", - options: { max_tokens: 512 }, - reasoning: { pattern: "none", maxSteps: 6 }, - maxCalls: 8, - timeoutSeconds: 120, - outputSchema: fixed, - }, - revision: 7, - }; - await page.route("**/environments/development/connections?*", (r) => - r.fulfill({ - json: { - items: [ - { - id: original.connection_revision_id, - name: "lumi-provider", - revision: 1, - connector: "weave-agentic-provider@1.0.0", - config: { - provider: "openai-responses", - endpoint: "https://api.openai.com", - }, - }, - ], - next_cursor: null, - }, - }), - ); - let saved: any; - let etag: string | undefined; - let finishSave: () => void = () => {}; - const saving = new Promise((resolve) => { - finishSave = resolve; - }); - await page.route("**/lumi/configuration", async (r) => { - if (r.request().method() === "PUT") { - saved = r.request().postDataJSON(); - etag = r.request().headers()["if-match"]; - await saving; - return r.fulfill({ json: { ...saved, revision: 8 } }); - } - return r.fulfill({ json: original }); - }); - await newWorkflow(page); - await page - .getByRole("button", { name: "Ask Weave AI", exact: true }) - .click(); - const panel = page.getByRole("dialog", { - name: "Ask Weave AI", - exact: true, - }); - await panel - .getByRole("button", { name: "Weave AI settings", exact: true }) - .click(); - await expect( - panel.getByRole("button", { name: "New conversation", exact: true }), - ).toBeHidden(); - await panel.getByLabel("Model", { exact: true }).fill("configured-model"); - await panel.getByLabel("Max tokens", { exact: true }).fill("1024"); - await expect(panel.getByText("Output schema", { exact: true })).toHaveCount( - 0, - ); - await expect( - panel.getByRole("button", { - name: "Save Weave AI settings", - exact: true, - }), - ).toHaveCount(0); - await panel - .getByRole("button", { name: "Continue to connection", exact: true }) - .click(); - await expect( - panel.getByRole("heading", { - name: "Choose a provider connection", - exact: true, - }), - ).toBeFocused(); - expect(saved).toBeUndefined(); - await panel.getByRole("button", { name: "Back", exact: true }).click(); - await expect(panel.getByLabel("Model", { exact: true })).toHaveValue( - "configured-model", - ); - await expect(panel.getByLabel("Max tokens", { exact: true })).toHaveValue( - "1024", - ); - await panel - .getByRole("button", { name: "Continue to connection", exact: true }) - .click(); - await panel - .getByRole("button", { name: "Review settings", exact: true }) - .click(); - await expect( - panel.getByRole("heading", { - name: "Review Weave AI settings", - exact: true, - }), - ).toBeFocused(); - await expect(panel.locator(".settings-summary")).toContainText( - "configured-model", - ); - await expect(panel.locator(".settings-summary")).toContainText( - "https://api.openai.com", - ); - await expect(panel.locator(".settings-summary")).toContainText( - "lumi-provider", - ); - expect(saved).toBeUndefined(); - await panel.screenshot({ - path: testInfo.outputPath(`lumi-review-${viewport.width}.png`), - }); - expect(await panel.evaluate((el) => el.scrollWidth <= el.clientWidth)).toBe( - true, - ); - await panel - .getByRole("button", { name: "Save Weave AI settings", exact: true }) - .click(); - await expect( - panel.getByRole("button", { name: "Weave AI settings", exact: true }), - ).toBeHidden(); - await expect( - panel.getByRole("button", { name: "New conversation", exact: true }), - ).toBeHidden(); - await expect( - panel.getByRole("button", { name: "Reload settings", exact: true }), - ).toBeDisabled(); - finishSave(); - await expect(panel.getByLabel("Message to Weave AI")).toBeVisible(); - await expect( - panel.getByRole("button", { name: "Weave AI settings", exact: true }), - ).toBeVisible(); - await expect( - panel.getByRole("button", { name: "New conversation", exact: true }), - ).toBeEnabled(); - expect(etag).toBe('"7"'); - expect(saved.profile.model).toBe("configured-model"); - expect(saved.profile.options.max_tokens).toBe(1024); - expect(saved.profile.outputSchema).toEqual(fixed); +test("Set up Weave AI opens Settings › AI models", async ({ page }) => { + await connected(page, { + capabilities: [...allCapabilities, "lumi.use", "lumi.manage"], + }); + await page.route("**/lumi/status", (r) => + r.fulfill({ json: { configured: false } }), + ); + await newWorkflow(page); + await page.getByRole("button", { name: "Ask Weave AI", exact: true }).click(); + const panel = page.getByRole("dialog", { name: "Ask Weave AI", exact: true }); + await expect(panel).toContainText( + "Weave AI is not set up in this environment.", + ); + await panel + .getByRole("button", { name: "Set up Weave AI", exact: true }) + .click(); + await expect(panel).toHaveCount(0); + await expect(page).toHaveURL(/\/settings\/ai$/); +}); + +test("Set up Weave AI with unsaved edits keeps the designer and its address when the person keeps editing", async ({ + page, +}) => { + await connected(page, { + capabilities: [...allCapabilities, "lumi.use", "lumi.manage"], }); + await page.route("**/lumi/status", (r) => + r.fulfill({ json: { configured: false } }), + ); + await newWorkflow(page); + await page + .locator(".palette") + .getByRole("button", { name: "AI task", exact: true }) + .click(); + await page.keyboard.press("Escape"); + const designer = page.url(); + expect(designer).toMatch(/\/designer$/); + await page.getByRole("button", { name: "Ask Weave AI", exact: true }).click(); + await page + .getByRole("dialog", { name: "Ask Weave AI", exact: true }) + .getByRole("button", { name: "Set up Weave AI", exact: true }) + .click(); + await page + .getByRole("dialog", { name: /^Leave/ }) + .getByRole("button", { name: "Keep editing" }) + .click(); + await expect(page.locator(".palette")).toBeVisible(); + await expect(page.getByRole("tab", { name: "AI models" })).toHaveCount(0); + await expect(page).toHaveURL(designer); + // The designer's address is still in history: Back leaves it, and + // Forward returns to it, not to Settings. + await page.goBack(); + await expect(page).not.toHaveURL(designer); + await expect(page).not.toHaveURL(/\/settings/); + await page.goForward(); + await expect(page).toHaveURL(designer); +}); test("non-workflow proposals are validated before saving a reviewed file", async ({ page, diff --git a/studio/tests/ndv-sample-account.test.ts b/studio/tests/ndv-sample-account.test.ts index 0ae6d689..a6e75882 100644 --- a/studio/tests/ndv-sample-account.test.ts +++ b/studio/tests/ndv-sample-account.test.ts @@ -53,6 +53,8 @@ function fixture() { contractRevision: 0, knownRuns: new Map(), workflowVersions: new Map(), + aiTests: new Map(), + aiSetupGeneration: 0, contractCache: new Map(), cdr: { markForCheck() {} }, closeRecord() {}, diff --git a/studio/tests/settings-routes.test.ts b/studio/tests/settings-routes.test.ts index 60903adc..6b61b640 100644 --- a/studio/tests/settings-routes.test.ts +++ b/studio/tests/settings-routes.test.ts @@ -24,46 +24,53 @@ import { settingsTabs, } from "../src/app/settings/settings-routes"; +const everything = { people: true, ai: true }; +const nothing = { people: false, ai: false }; describe("Settings addresses", () => { it("gives each tab its own address", () => { expect(SETTINGS_TABS.map((tab) => settingsPath(tab.id))).toEqual([ "/settings/platforms", "/settings/people", + "/settings/ai", "/settings/preferences", ]); }); - it("opens the tab an address names", () => { - expect(settingsTab("/settings/preferences", false)).toBe("preferences"); - expect(settingsTab("/settings/people", true)).toBe("people"); - expect(settingsTab("/settings/platforms", true)).toBe("platforms"); + expect(settingsTab("/settings/preferences", nothing)).toBe("preferences"); + expect(settingsTab("/settings/people", everything)).toBe("people"); + expect(settingsTab("/settings/ai", everything)).toBe("ai"); + expect(settingsTab("/settings/platforms", everything)).toBe("platforms"); }); - it("opens Platforms for /settings, an unknown tab or a tab the person can't see", () => { - expect(settingsTab("/settings", true)).toBe("platforms"); - expect(settingsTab("/settings/ai", true)).toBe("platforms"); - expect(settingsTab("/settings/people", false)).toBe("platforms"); + expect(settingsTab("/settings", everything)).toBe("platforms"); + expect(settingsTab("/settings/administration", everything)).toBe( + "platforms", + ); + expect(settingsTab("/settings/people", nothing)).toBe("platforms"); + expect(settingsTab("/settings/ai", nothing)).toBe("platforms"); }); - - it("shows People and access only to people who can manage it", () => { - expect(settingsTabs(false).map((tab) => tab.label)).toEqual([ + it("shows People and access to managers and AI models once a platform is connected", () => { + expect(settingsTabs(nothing).map((tab) => tab.label)).toEqual([ "Platforms", "Preferences", ]); - expect(settingsTabs(true).map((tab) => tab.label)).toEqual([ + expect( + settingsTabs({ people: false, ai: true }).map((tab) => tab.label), + ).toEqual(["Platforms", "AI models", "Preferences"]); + expect(settingsTabs(everything).map((tab) => tab.label)).toEqual([ "Platforms", "People and access", + "AI models", "Preferences", ]); }); - it("routes /settings to Platforms and each tab to the shell", () => { const paths = routes.map((route) => route.path); expect(routes.find((route) => route.path === "settings")).toMatchObject({ redirectTo: "settings/platforms", pathMatch: "full", }); - for (const tab of ["platforms", "people", "preferences"]) + for (const tab of ["platforms", "people", "ai", "preferences"]) expect(paths).toContain(`settings/${tab}`); expect(paths.indexOf("settings/preferences")).toBeLessThan( paths.indexOf("**"), diff --git a/studio/tests/weave-ai-naming.test.ts b/studio/tests/weave-ai-naming.test.ts index 2afb31ec..9475dc56 100644 --- a/studio/tests/weave-ai-naming.test.ts +++ b/studio/tests/weave-ai-naming.test.ts @@ -55,9 +55,10 @@ describe("Weave AI naming", () => { expect(panel).toContain('heading="Ask Weave AI"'); expect(panel).toContain('closeLabel="Close Weave AI"'); expect(panel).toContain('aria-label="Message to Weave AI"'); - const settings = read("app/settings/settings-page.ts"); - expect(settings).toContain("

AI models

"); - expect(settings).not.toContain("AI setup"); + expect(read("app/settings/settings-routes.ts")).toContain( + '{ id: "ai", label: "AI models" }', + ); + expect(read("app/settings/settings-page.ts")).not.toContain("AI setup"); }); it("keeps the lumi identifiers the API, roles and storage use", () => { diff --git a/tests/acceptance/journeys.toml b/tests/acceptance/journeys.toml index 3294013c..d93f6b64 100644 --- a/tests/acceptance/journeys.toml +++ b/tests/acceptance/journeys.toml @@ -52,7 +52,7 @@ "language-docs" = false "ai-gateway" = true "ai-setup" = false -"ai-models-screen" = false +"ai-models-screen" = true "ai-task" = false "weave-ai-ollama" = false "agents" = false diff --git a/tests/acceptance/test_j0_bring_up.py b/tests/acceptance/test_j0_bring_up.py index ad256739..90189220 100644 --- a/tests/acceptance/test_j0_bring_up.py +++ b/tests/acceptance/test_j0_bring_up.py @@ -29,7 +29,7 @@ import pytest from firefly_weave import __version__ -from firefly_weave.sdk.platform import PERSON_ROLES +from firefly_weave.sdk.platform import LOCAL_OWNER_ROLES, PERSON_ROLES pytestmark = pytest.mark.acceptance @@ -67,6 +67,10 @@ def ready(api_url: str, timeout: float = 60) -> bool: return False +def _expect_roles(grants: list[dict[str, object]], roles: tuple[str, ...]) -> None: + assert sorted({str(grant["role"]) for grant in grants}) == sorted(roles) + + @pytest.mark.journey_step("J0.1") def test_j0_01_version(run): assert run.require(run.weave("--version")).stdout == f"Firefly Weave {__version__}\n" @@ -101,6 +105,10 @@ def test_j0_03_up_with_the_fixture_origin(run): for purpose in ("event-delivery", "http-connector") ], } + run.check( + "J0.3/owner-roles", + lambda: _expect_roles(value["account"]["grants"], LOCAL_OWNER_ROLES), + ) @pytest.mark.journey_step("J0.4") diff --git a/tests/unit/cli/test_platform_docker.py b/tests/unit/cli/test_platform_docker.py index f2315fcb..f53b9e85 100644 --- a/tests/unit/cli/test_platform_docker.py +++ b/tests/unit/cli/test_platform_docker.py @@ -221,6 +221,32 @@ def setup(*args, **kwargs): assert calls == ["setup", "start", "user", "start"] +def test_up_gives_its_new_account_the_local_owner_roles(tmp_path, monkeypatch): + directory = tmp_path / "owner" + state = { + "mode": "docker", + "context": "owned", + "directory": str(directory), + "stage": "ready", + "id": "d" * 24, + "ports": {"container_api": 55201}, + } + seen = [] + + def setup(*args, **kwargs): + directory.mkdir(mode=0o700) + + monkeypatch.setattr(platform, "setup", setup) + monkeypatch.setattr(platform, "_load", lambda p: state) + monkeypatch.setattr(platform, "start", lambda *a: None) + monkeypatch.setattr(platform, "demo", lambda *a: {"receipt": {"status": "succeeded"}}) + monkeypatch.setattr(platform, "user", lambda d, u, roles: seen.append(tuple(roles)) or {"username": u}) + platform.up(directory, tmp_path, "owned", username="owner") + assert seen == [platform.LOCAL_OWNER_ROLES] + receipt = json.loads((directory / "up-user.json").read_text()) + assert receipt["roles"] == list(platform.LOCAL_OWNER_ROLES) + + def test_uncertain_account_creation_blocks_replay(owned, monkeypatch): directory, state = owned platform._write(directory / "up-user.json", {"stage": "attempted", "username": "developer"}) @@ -231,6 +257,38 @@ def test_uncertain_account_creation_blocks_replay(owned, monkeypatch): platform.up(directory, Path("."), None, username="developer") +def test_up_refuses_an_unreadable_people_record_before_marking_an_account_attempt(owned, monkeypatch): + directory, state = owned + (directory / "people.json").write_text("{not json") + (directory / "people.json").chmod(0o600) + monkeypatch.setattr(platform, "start", lambda *a: None) + monkeypatch.setattr(platform, "demo", lambda *a: {"receipt": {"status": "succeeded"}}) + monkeypatch.setattr(platform, "user", lambda *a: pytest.fail("created an account")) + with pytest.raises(platform.PlatformError, match="not readable"): + platform.up(directory, Path("."), None, username="developer") + assert not (directory / "up-user.json").exists() + + +def test_up_returns_the_new_password_when_its_receipt_cannot_be_completed(owned, monkeypatch): + directory, state = owned + monkeypatch.setattr(platform, "start", lambda *a: None) + monkeypatch.setattr(platform, "demo", lambda *a: {"receipt": {"status": "succeeded"}}) + monkeypatch.setattr(platform, "user", lambda *a: {"username": "developer", "password": "once"}) + write = platform._write + + def failing_write(path, value, **kwargs): + if value.get("stage") == "ready": + raise OSError("No space left on device") + return write(path, value, **kwargs) + + monkeypatch.setattr(platform, "_write", failing_write) + result = platform.up(directory, Path("."), None, username="developer") + assert result["account"]["password"] == "once" + assert result["account"]["receipt_saved"] is False + assert "up-user.json" in result["account"]["receipt_hint"] + assert "once" not in (directory / "up-user.json").read_text() + + def test_container_configuration_mounts_only_selected_values_and_literal_environment(owned, monkeypatch): from firefly_weave.sdk import platform_docker diff --git a/tests/unit/cli/test_platform_sign_in.py b/tests/unit/cli/test_platform_sign_in.py index ffefbb43..2274833f 100644 --- a/tests/unit/cli/test_platform_sign_in.py +++ b/tests/unit/cli/test_platform_sign_in.py @@ -20,6 +20,7 @@ import gzip import importlib.util import json +import os from pathlib import Path from urllib.parse import parse_qs from uuid import UUID, uuid4 @@ -672,15 +673,33 @@ def test_cli_json_prints_one_object_with_the_password_once(demo_workspace, monke "password", "principal_id", "provider_id", + "recorded", "subject", "username", "warning", } + assert value["recorded"] is True assert result.output.count(value["password"]) == 1 assert value["grants"][0] == {"role": "developer", "scope": {"tenant_id": TENANT, "project_id": PROJECT}} assert UUID(value["principal_id"]) and value["next"][0] == "weave auth setup " + API +def test_cli_text_shows_the_password_and_where_to_assign_roles_when_the_record_is_busy(demo_workspace, monkeypatch): + import fcntl + + directory, _, _ = demo_workspace + monkeypatch.setattr(platform, "_PEOPLE_LOCK_SECONDS", 0.1) + holder = os.open(directory / ".people.lock", os.O_RDWR | os.O_CREAT, 0o600) + try: + fcntl.flock(holder, fcntl.LOCK_EX | fcntl.LOCK_NB) + result = _invoke(directory, monkeypatch, "--username", "grace") + finally: + os.close(holder) + assert result.exit_code == 0, result.output + assert "Password: " in result.output + assert "Settings › People and access" in result.output + + def test_cli_text_shows_password_once_with_warning_and_next_commands(demo_workspace, monkeypatch): directory, _, _ = demo_workspace result = _invoke(directory, monkeypatch, "--username", "grace", "--role", "viewer") @@ -710,3 +729,232 @@ def test_cli_help_documents_development_only_default_roles(): assert "development only" in text.lower() for role in ("developer", "deployer", "operator", "viewer", "task_participant"): assert role in text + + +def test_local_owner_roles_cover_every_setup_screen_and_stay_person_roles(): + assert set(platform.LOCAL_OWNER_ROLES) <= set(platform.PERSON_ROLES) + assert set(platform.DEFAULT_PERSON_ROLES) < set(platform.LOCAL_OWNER_ROLES) + for role in ("tenant_admin", "lumi_manager", "execution_manager", "worker_operator", "deployment_operator"): + assert role in platform.LOCAL_OWNER_ROLES + assert not set(platform.LOCAL_OWNER_ROLES) & {"platform_admin", "worker", "deployment_runner"} + + +def test_user_records_the_account_for_later_grants(demo_workspace): + directory, _, _ = demo_workspace + value = platform.user(directory, "ada.lovelace", transport=httpx.MockTransport(Identity())) + path = directory / "people.json" + assert path.stat().st_mode & 0o777 == 0o600 + record = json.loads(path.read_text()) + assert record == { + "format": "weave/local-people-v1", + "people": {"ada.lovelace": {"principal_id": PRINCIPAL, "subject": SUBJECT}}, + } + assert value["password"] not in path.read_text() + + +@pytest.mark.parametrize("damage", ["mode", "json", "shape"]) +def test_user_refuses_an_unreadable_people_record_before_creating_anything(demo_workspace, damage): + directory, _, _ = demo_workspace + path = directory / "people.json" + if damage == "mode": + path.write_text(json.dumps({"format": "weave/local-people-v1", "people": {}})) + path.chmod(0o644) + else: + path.write_text("{not json" if damage == "json" else json.dumps({"format": "other"})) + path.chmod(0o600) + identity = Identity() + with pytest.raises(platform.PlatformError, match="private record of local accounts is not readable"): + platform.user(directory, "ada.lovelace", transport=httpx.MockTransport(identity)) + assert identity.requests == [] + + +def test_user_keeps_the_password_when_another_command_holds_the_record(demo_workspace, monkeypatch): + import fcntl + + directory, _, _ = demo_workspace + monkeypatch.setattr(platform, "_PEOPLE_LOCK_SECONDS", 0.2) + holder = os.open(directory / ".people.lock", os.O_RDWR | os.O_CREAT, 0o600) + try: + fcntl.flock(holder, fcntl.LOCK_EX | fcntl.LOCK_NB) + value = platform.user(directory, "ada.lovelace", transport=httpx.MockTransport(Identity())) + finally: + os.close(holder) + assert len(value["password"]) >= 32 + assert value["recorded"] is False + assert "Settings › People and access" in value["hint"] + assert not (directory / "people.json").exists() + + +def test_user_waits_for_a_record_another_command_is_writing(demo_workspace): + import fcntl + import threading + + directory, _, _ = demo_workspace + holder = os.open(directory / ".people.lock", os.O_RDWR | os.O_CREAT, 0o600) + fcntl.flock(holder, fcntl.LOCK_EX | fcntl.LOCK_NB) + release = threading.Timer(0.3, os.close, (holder,)) + release.start() + try: + value = platform.user(directory, "ada.lovelace", transport=httpx.MockTransport(Identity())) + finally: + release.join() + assert value["recorded"] is True + assert "ada.lovelace" in json.loads((directory / "people.json").read_text())["people"] + + +def _recorded(directory, username="ada.lovelace"): + platform._write( + directory / "people.json", + {"format": "weave/local-people-v1", "people": {username: {"principal_id": PRINCIPAL, "subject": SUBJECT}}}, + ) + + +def test_grant_adds_roles_to_a_recorded_account_after_proving_its_sign_in(demo_workspace): + directory, _, _ = demo_workspace + _recorded(directory) + identity = Identity() + identity.users["ada.lovelace"] = SUBJECT + value = platform.grant( + directory, "ada.lovelace", ("tenant_admin", "lumi_manager"), transport=httpx.MockTransport(identity) + ) + assert identity.paths() == [ + ("POST", "/realms/master/protocol/openid-connect/token"), + ("GET", "/admin/realms/weave/users"), + ("POST", f"/api/v1/admin/principals/{PRINCIPAL}/identity-links"), + *[("POST", f"/api/v1/tenants/{TENANT}/members")] * 2, + ] + link = json.loads(identity.requests[2].content) + assert link == {"provider_id": "local-keycloak", "issuer": KEYCLOAK + "/realms/weave", "subject": SUBJECT} + grants = [json.loads(request.content) for request in identity.requests[3:]] + assert [(g["role"], g["project_id"], g["environment_id"]) for g in grants] == [ + ("tenant_admin", None, None), + ("lumi_manager", PROJECT, ENVIRONMENT), + ] + assert value == { + "ok": True, + "username": "ada.lovelace", + "principal_id": PRINCIPAL, + "grants": [ + {"role": "tenant_admin", "scope": {"tenant_id": TENANT}}, + { + "role": "lumi_manager", + "scope": {"tenant_id": TENANT, "project_id": PROJECT, "environment_id": ENVIRONMENT}, + }, + ], + "development_only": True, + } + + +def test_grant_refuses_an_unrecorded_account_before_any_request(demo_workspace): + directory, _, _ = demo_workspace + + def refuse(request): + raise AssertionError("no request may be sent") + + with pytest.raises(platform.PlatformError, match="no record of grace"): + platform.grant(directory, "grace", ("viewer",), transport=httpx.MockTransport(refuse)) + + +def test_grant_refuses_a_replaced_sign_in_account(demo_workspace): + directory, _, _ = demo_workspace + _recorded(directory) + identity = Identity() + identity.users["ada.lovelace"] = "66666666-6666-4666-8666-666666666666" + with pytest.raises(platform.PlatformError, match="not the one this platform recorded"): + platform.grant(directory, "ada.lovelace", ("viewer",), transport=httpx.MockTransport(identity)) + assert ("POST", f"/api/v1/tenants/{TENANT}/members") not in identity.paths() + + +def test_grant_refuses_when_the_sign_in_belongs_to_another_principal(demo_workspace): + directory, _, _ = demo_workspace + _recorded(directory) + identity = Identity(link_status=409) + identity.users["ada.lovelace"] = SUBJECT + with pytest.raises(platform.PlatformError, match="refused the role grant"): + platform.grant(directory, "ada.lovelace", ("viewer",), transport=httpx.MockTransport(identity)) + assert ("POST", f"/api/v1/tenants/{TENANT}/members") not in identity.paths() + + +@pytest.mark.parametrize( + ("roles", "message"), + [((), "at least one role"), (("platform_admin",), "Choose each role"), (("viewer", "viewer"), "Choose each role")], +) +def test_grant_checks_roles_before_any_request(demo_workspace, roles, message): + directory, _, _ = demo_workspace + _recorded(directory) + with pytest.raises(platform.PlatformError, match=message): + platform.grant( + directory, "ada.lovelace", roles, transport=httpx.MockTransport(lambda request: pytest.fail("sent")) + ) + + +def test_cli_user_grant_prints_the_grants(demo_workspace, monkeypatch): + directory, _, _ = demo_workspace + _recorded(directory) + identity = Identity() + identity.users["ada.lovelace"] = SUBJECT + monkeypatch.setattr(platform, "grant", functools.partial(platform.grant, transport=httpx.MockTransport(identity))) + result = CliRunner().invoke( + cli, + [ + "platform", + "--directory", + str(directory), + "user", + "grant", + "--username", + "ada.lovelace", + "--role", + "tenant_admin", + ], + ) + assert result.exit_code == 0, result.output + assert "Granted to ada.lovelace: tenant_admin (demo tenant)" in result.output + + +def test_cli_user_options_belong_after_the_grant_subcommand(demo_workspace): + directory, _, _ = demo_workspace + result = CliRunner().invoke( + cli, + [ + "platform", + "--directory", + str(directory), + "user", + "--username", + "ada", + "grant", + "--username", + "ada", + "--role", + "viewer", + ], + ) + assert result.exit_code == 2 + assert "after the subcommand" in result.output + + +def test_cli_user_without_a_username_or_subcommand_is_a_usage_error(demo_workspace): + directory, _, _ = demo_workspace + result = CliRunner().invoke(cli, ["platform", "--directory", str(directory), "user"]) + assert result.exit_code == 2 + assert "--username" in result.output + + +def test_cli_text_says_when_the_initial_account_receipt_was_not_completed(capsys): + from firefly_weave.cli import platform as cli_platform + + cli_platform._show_person( + { + "username": "owner", + "password": "shown-once", + "warning": "Development only.", + "grants": [], + "next": ["weave auth setup http://localhost:1", "weave studio"], + "receipt_saved": False, + "receipt_hint": platform._RECEIPT_NOT_SAVED, + } + ) + output = capsys.readouterr().out + assert output.count("shown-once") == 1 + assert platform._RECEIPT_NOT_SAVED in output diff --git a/tests/unit/test_acceptance_journeys.py b/tests/unit/test_acceptance_journeys.py index a8ca77fe..654c5648 100644 --- a/tests/unit/test_acceptance_journeys.py +++ b/tests/unit/test_acceptance_journeys.py @@ -129,12 +129,12 @@ def test_the_committed_map_is_valid(): enablement = journeys.Enablement.load(ROOT / "tests/acceptance/journeys.toml") assert enablement.capabilities["acceptance-foundations"] is True landed = [name for name, value in enablement.capabilities.items() if value] - assert landed == ["text-operators", "instance-keys", "ai-gateway", "acceptance-foundations"] + assert landed == ["text-operators", "instance-keys", "ai-gateway", "ai-models-screen", "acceptance-foundations"] assert enablement.journey_steps("J0") == tuple(f"J0.{n}" for n in range(1, 15)) assert enablement.profiles["pr"] == ("J0", "J1", "J2", "J3", "J5", "J8", "J9") assert enablement.criteria["J3"] == ("SC1", "SC3", "SC7") assert enablement.applies("J0.12", "ai") and not enablement.applies("J0.12", "pr") - assert enablement.missing("J0.3/owner-roles") == ("ai-models-screen|compose-operations",) + assert enablement.missing("J0.3/owner-roles") == () def test_versions_toml_pins_match_the_files_that_use_them(): diff --git a/tests/unit/test_studio_bridge_deadlines.py b/tests/unit/test_studio_bridge_deadlines.py new file mode 100644 index 00000000..eb41695a --- /dev/null +++ b/tests/unit/test_studio_bridge_deadlines.py @@ -0,0 +1,205 @@ +# Copyright 2026 Firefly Software Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# Author: Firefly Software Foundation +# SPDX-License-Identifier: Apache-2.0 + +"""A model test may take longer than other platform calls, and stops when its browser leaves.""" + +import asyncio +from pathlib import Path +from types import MappingProxyType + +import httpx +from starlette.requests import Request +from starlette.testclient import TestClient + +from firefly_weave.studio import service as studio_service +from firefly_weave.studio.host import make_studio_app +from firefly_weave.studio.service import StudioOptions, StudioProfile, StudioService + +ORIGIN = "http://127.0.0.1:8766" +CODE = "terminal-only-pairing-code" +TENANT = "00000000-0000-0000-0000-000000000001" +PROJECT_ID = "00000000-0000-0000-0000-000000000002" +ENVIRONMENT_ID = "00000000-0000-0000-0000-000000000003" +REVISION = "00000000-0000-0000-0000-0000000000aa" +ENVIRONMENT = f"/api/v1/tenants/{TENANT}/projects/{PROJECT_ID}/environments/{ENVIRONMENT_ID}" +MODEL_TEST = f"{ENVIRONMENT}/ai/connections/{REVISION}/test" +READINESS = f"{ENVIRONMENT}/ai/readiness" +PROFILE = StudioProfile( + name="Dev", base_url="https://api.example", tenant_id=TENANT, project_id=PROJECT_ID, environment_id=ENVIRONMENT_ID +) + + +def assets(tmp_path: Path) -> Path: + folder = tmp_path / "assets" + folder.mkdir() + (folder / "index.html").write_text("Weave Studio") + return folder + + +def paired(tmp_path: Path, upstream) -> TestClient: + browser = TestClient( + make_studio_app( + StudioOptions( + origin=ORIGIN, + assets=assets(tmp_path), + pairing_code=CODE, + profile=PROFILE, + token_provider=lambda: "host-token", + transport=httpx.MockTransport(upstream), + ) + ), + base_url=ORIGIN, + ) + response = browser.post("/studio/session", json={"code": CODE}, headers={"Origin": ORIGIN}) + assert response.status_code == 200 + browser.headers.update({"Origin": ORIGIN, "X-Weave-CSRF": response.json()["csrfToken"]}) + return browser + + +def test_the_bridge_names_the_operation_a_path_matches(tmp_path): + service = StudioService(StudioOptions(origin=ORIGIN, assets=assets(tmp_path), profile=PROFILE)) + assert service.match(MODEL_TEST, "POST") == (200, "ai_connections.test") + assert service.match(READINESS, "GET") == (200, "ai_readiness.read") + other = MODEL_TEST.replace(ENVIRONMENT_ID, "00000000-0000-0000-0000-000000000009") + assert service.match(other, "POST") == (403, "ai_connections.test") + assert service.match("/api/v1/unknown", "GET") == (404, None) + assert service.check_scope(MODEL_TEST, "POST") == 200 + + +def test_a_model_test_may_outlast_the_ordinary_deadline(tmp_path, monkeypatch): + monkeypatch.setattr(studio_service, "BRIDGE_SECONDS", 0.2) + monkeypatch.setattr(studio_service, "LONG_OPERATIONS", MappingProxyType({"ai_connections.test": 3.0})) + + reads: dict[str, float] = {} + + async def upstream(request: httpx.Request) -> httpx.Response: + reads[request.url.path.rsplit("/", 1)[-1]] = request.extensions["timeout"]["read"] + await asyncio.sleep(0.5) + if request.url.path.endswith("/test"): + return httpx.Response(200, json={"ok": True, "code": "ok", "latency_ms": 500, "model": "qwen3:4b"}) + return httpx.Response(200, json={"items": []}) + + with paired(tmp_path, upstream) as browser: + tested = browser.post("/studio/api" + MODEL_TEST, json={"model": "qwen3:4b", "probe_tools": True}) + assert tested.status_code == 200, tested.text + assert tested.json()["model"] == "qwen3:4b" + slow = browser.get("/studio/api" + READINESS) + assert slow.status_code == 502 + assert slow.json()["code"] == "WV-STUDIO-UPSTREAM" + # The HTTP client's own read timeout follows the longer deadline, so it cannot cut a slow test short. + assert reads == {"test": 3.0, "readiness": 30} + + +async def test_a_model_test_stops_when_the_browser_leaves(tmp_path): + started, cancelled = asyncio.Event(), asyncio.Event() + + async def upstream(request: httpx.Request) -> httpx.Response: + started.set() + try: + await asyncio.sleep(60) + except asyncio.CancelledError: + cancelled.set() + raise + return httpx.Response(200, json={}) + + service = StudioService( + StudioOptions( + origin=ORIGIN, + assets=assets(tmp_path), + profile=PROFILE, + token_provider=lambda: "host-token", + transport=httpx.MockTransport(upstream), + ) + ) + messages = [{"type": "http.request", "body": b'{"model":"qwen3:4b","probe_tools":true}', "more_body": False}] + + async def receive(): + if messages: + return messages.pop(0) + await started.wait() + return {"type": "http.disconnect"} + + scope = { + "type": "http", + "method": "POST", + "path": "/studio/api" + MODEL_TEST, + "raw_path": ("/studio/api" + MODEL_TEST).encode(), + "query_string": b"", + "headers": [(b"content-type", b"application/json")], + "scheme": "http", + "server": ("127.0.0.1", 8766), + "root_path": "", + } + try: + # Nobody is left to read the answer; the bridge ends quietly instead of raising into the server's log. + response = await asyncio.wait_for(service.bridge(Request(scope, receive), MODEL_TEST), 10) + assert response.status_code == 499 + assert response.body == b"" + assert cancelled.is_set() + finally: + await service.close() + + +async def test_a_model_test_passes_on_its_own_cancellation(tmp_path): + """A server shutting down cancels the bridge itself; that is not a browser leaving.""" + started = asyncio.Event() + + async def upstream(request: httpx.Request) -> httpx.Response: + started.set() + await asyncio.sleep(60) + return httpx.Response(200, json={}) + + service = StudioService( + StudioOptions( + origin=ORIGIN, + assets=assets(tmp_path), + profile=PROFILE, + token_provider=lambda: "host-token", + transport=httpx.MockTransport(upstream), + ) + ) + messages = [{"type": "http.request", "body": b'{"model":"qwen3:4b","probe_tools":true}', "more_body": False}] + + async def receive(): + if messages: + return messages.pop(0) + await asyncio.sleep(60) + return {"type": "http.disconnect"} + + scope = { + "type": "http", + "method": "POST", + "path": "/studio/api" + MODEL_TEST, + "raw_path": ("/studio/api" + MODEL_TEST).encode(), + "query_string": b"", + "headers": [(b"content-type", b"application/json")], + "scheme": "http", + "server": ("127.0.0.1", 8766), + "root_path": "", + } + try: + bridge = asyncio.create_task(service.bridge(Request(scope, receive), MODEL_TEST)) + await asyncio.wait_for(started.wait(), 10) + bridge.cancel() + try: + await asyncio.wait_for(bridge, 10) + except asyncio.CancelledError: + pass + else: + raise AssertionError("the bridge swallowed its own cancellation") + assert bridge.cancelled() + finally: + await service.close()