diff --git a/docs/capabilities.md b/docs/capabilities.md index 85ee0d43..6dba89e1 100644 --- a/docs/capabilities.md +++ b/docs/capabilities.md @@ -166,8 +166,11 @@ is retained as evidence; it is not counted as a successful model run. ## What is not included -- **BPMN.** Weave does not import or export BPMN files. Subprocesses, loops, - compensation, and error boundary events are not part of the workflow language; +- **BPMN.** Weave does not import or export BPMN files. Compensation and error + boundary events are not part of the workflow language. The language defines + **Loop over items** and **Call a workflow** (see [Steps](contracts.md#steps)), + but this version does not compile or run them yet: the compiler reports + `WV-COMP-UNSUPPORTED_FEATURE`. [Coming from BPM/BPMN](concepts.md#coming-from-bpmbpmn) lists the alternatives. - **Named enterprise adapters.** SAP and Oracle are not offered as executable named adapters. Reaching a system through a generic connector does not certify diff --git a/docs/concepts.md b/docs/concepts.md index 6c390e6a..ed7735b4 100644 --- a/docs/concepts.md +++ b/docs/concepts.md @@ -305,8 +305,8 @@ Use this table to translate what you know: | Error end event, terminate end event | **Fail** (`fail`) | Ends the whole run as failed with your code and message, including any other parallel branches | | Error boundary event | Not in the language | A failed Action retries when that is safe; otherwise the run is suspended with an incident for an operator. To branch on a business outcome, return it in the Action's output and test it with a Decision | | Message throw or end event | **Call an action**, or an integration event subscription | A subscription notifies another system when a run's status changes | -| Call activity, subprocess | Not in the language | Keep the part inside the same workflow, or make it a separate workflow that your product starts | -| Loop, multi-instance activity | Not in the language | Use a Parallel with a fixed set of branches, or start one run per item | +| Call activity, subprocess | **Call a workflow** (`callWorkflow`) | Defined in the [workflow language](contracts.md#steps), but this version does not compile or run it: the compiler reports `WV-COMP-UNSUPPORTED_FEATURE`. Until then, keep the part inside the same workflow, or make it a separate workflow that your product starts | +| Loop, multi-instance activity | **Loop over items** (`forEach`) | Defined in the [workflow language](contracts.md#steps), but this version does not compile or run it: the compiler reports `WV-COMP-UNSUPPORTED_FEATURE`. Until then, use a Parallel with a fixed set of branches, or start one run per item | | Compensation | Not in the language | Model compensating Actions as ordinary steps; Weave never undoes an external effect | | Instance migration | Not available | A running run keeps its version; new runs use the new activation | | Process monitoring, cockpit | **Runs**, history, and incidents in Studio and the CLI; logs, metrics, and traces | Studio draws a run's graph and marks its current step **Now**; see [follow and manage runs](guides/studio.md#save-publish-activate-and-run), [execution management](guides/execution-management.md), and [observability](operations/observability.md) | diff --git a/docs/contracts.md b/docs/contracts.md index 56a6e044..7fb9b669 100644 --- a/docs/contracts.md +++ b/docs/contracts.md @@ -146,10 +146,11 @@ snake_case keys in a document are rejected. Serialize Python models with `model_dump(by_alias=True)` or `model_dump_json(by_alias=True)`. **Some fields may be omitted but never set to null.** These are the workflow's -`timeoutSeconds`, an action's `connection` and `routing`, and an action step's -`connection`. When present they must have their declared type; an explicit `null` -is rejected. Both serializers preserve the omission, and the Python model shows -an omitted field as `None`. +`timeoutSeconds` and `callable` (and its `allowedCallers`), an action's +`connection` and `routing`, an action step's `connection`, and a call step's +`onFailure` and `businessKey`. When present they must have their declared type; +an explicit `null` is rejected. Both serializers preserve the omission, and the +Python model shows an omitted field as `None`. `load_definition` checks shape only. It does not parse source text, validate the embedded JSON Schemas, resolve dependencies, compile, or authorize anything. The @@ -170,7 +171,10 @@ object with exactly one of these keys: | `op` | An operator with expression arguments: `{op: {name, args}}` | `{op: {name: eq, args: [{ref: /input/urgent}, {literal: true}]}}` | Operators are `eq`, `ne`, `lt`, `lte`, `gt`, `gte`, `and`, `or`, `not`, `exists`, -`coalesce`, `contains`, `notContains`, `in`, `notIn`, `startsWith`, and `endsWith`. +`coalesce`, `contains`, `notContains`, `in`, `notIn`, `startsWith`, `endsWith`, +`concat`, and `join`. `concat` joins one or more strings, numbers, or Booleans +into one string; `join` takes a list of such values and a separator string. Numbers +are written as JavaScript writes them (`2.0` becomes `2`, `1e21` stays `1e+21`). There are no function calls, scripts, environment variables, or file access. The compiler checks arity, types, and scope; the [compiler reference](reference/compiler.md#cli-and-published-catalog-contract) @@ -194,16 +198,29 @@ helps if you know process modeling from another tool: | `kind` | Studio name | Closest BPMN idea | Other fields | | --- | --- | --- | --- | | `action` | **Call an action** | Service task | Required `uses` (exact action reference) and `with` (input expression); optional `connection` slot name | +| `llm` | **AI task** | Service task | Required `uses` (exact AI action reference), `profile` (an `llmProfiles` entry), `prompt` and `context` expressions, and `connection` slot name | | `transform` | **Transform** | Script or business rule task (expressions only) | Required `value` expression | +| `decisionTable` | **Decision table** | Business rule task | Required `uses` (exact decision table reference) and `with` (input expression) | | `switch` | **Decision** | Exclusive gateway | Nonempty `cases: [{when, steps, output}]` and required `default: {steps, output}` (the **Otherwise** path in Studio); the first true case wins | | `parallel` | **Parallel** | Parallel gateway (split and join) | Nonempty `branches: {name: {steps, output}}` and a positive integer `concurrency`; every branch completes before the step continues | | `wait` | **Wait for time** | Timer intermediate event | Required positive integer `durationSeconds` | | `signal` | **Wait for signal** | Message intermediate event | Required `name`, positive integer `timeoutSeconds`, and a `payloadSchema` object | | `humanTask` | **Human task** | User task | Required `assignment` (the name of an assignment bound at activation), `title` and `context` expressions, and `formSchema`; `decisions` defaults to `approve`, `reject` (1 to 32 unique names); optional positive `dueSeconds` and `expirySeconds` | | `fail` | **Fail** | Error end event | Required business-error `code` (a name such as `customer-not-found`) and a nonempty `message` | +| `forEach` | **Loop over items** | Multi-instance subprocess | Required `items` (an expression that gives a list) and `body: {steps, output}`; optional `concurrency` (default 1), `maxItems` (default 1000), and `collect` (`all`, the default, or `nonNull`). The defaults are written into the stored document | +| `callWorkflow` | **Call a workflow** | Call activity | Required `uses` (exact workflow reference) and `with` (input expression); optional `mode` (`wait`, the default, or `detach`), `onFailure` (`stop` or `continue`, only with `wait`), and `businessKey` expression | -Weave is not a BPMN engine and does not import BPMN files. Subprocesses (call -activities), loops, and compensation are not part of the language. +Weave is not a BPMN engine and does not import BPMN files. Loops and calls to +other workflows are part of the language; compensation is not. + +**New language constructs.** `forEach`, `callWorkflow`, `concat`, and `join` +each need a language feature: `flow.forEach`, `flow.callWorkflow`, `text.concat`, +and `text.join`. Their document shape is final, so `load_definition` accepts +them, but this version of the compiler reports `WV-COMP-UNSUPPORTED_FEATURE` at +each use instead of compiling it. A platform runs a construct only when its +[language manifest](#language-manifest) lists the feature. Step IDs can never +contain `[`, `#`, or `~`, which keeps +[instance keys](reference/compiler.md#instance-keys) unambiguous. Branches may contain zero steps, but each must declare its `output`. The compiler, not the shape check, enforces unique step IDs, unique signal names, branch scope, @@ -231,6 +248,34 @@ definition; they are not legal on a workflow step. | `output` | Yes | Expression that computes the run output | | `timeoutSeconds` | No | Positive whole-run timeout; omit it for no workflow-wide timeout | | `connections` | No (default `{}`) | **Connection slots**: named requirements `{connector: , required: }`; `required` defaults to `true`. Each slot is bound to an integration connection at activation | +| `callable` | No | Lets other workflows call this exact version with `callWorkflow`: `{}` allows any workflow in the project, and `{allowedCallers: [order-intake]}` allows only the named workflows (1 to 100 unique names). Studio will write it from a **Called by a workflow** trigger; the manifest marks the field `pending` until then | + +A callable workflow, and a loop over a list that builds text for each item: + +```yaml +# notify-customer@1.0.0 accepts calls from order-intake only. +spec: + callable: + allowedCallers: [order-intake] +--- +# One reminder per invoice, four at a time, collected in invoice order. +- id: notify + kind: forEach + items: {ref: /input/invoices} + concurrency: 4 + body: + steps: + - id: subject + kind: transform + value: + op: + name: concat + args: [{literal: "Invoice "}, {ref: /item/number}, {literal: " is overdue"}] + output: {ref: /steps/subject/output} +``` + +The full examples are in +[examples/language](../examples/language/). ## Actions @@ -272,6 +317,24 @@ Manifests contain no Python source or import paths. To build one, see [Author a connector](connectors/authoring.md); to call a REST API without writing one, see [Call a REST API without code](connectors/http-without-code.md). +## Language manifest + +The **language manifest** lists every step kind, operator, and workflow field the +language defines, the features each one needs, the features the platform runs, +and the language limits. Studio will read it to decide what to offer. Read it with +`weave remote language` or `GET /api/v1/tenants/{tenant}/projects/{project}/language` +(`language.read`, capability `catalog.read`); Studio's local host serves the same +document at `/studio/contracts/language`, and `weave schema export` writes its +schema as `language-manifest.schema.json`. + +| Field | Meaning | +| --- | --- | +| `version`, `language_version` | `weave/language-manifest-v1` and `weave/v1alpha1` | +| `ir_versions` | The executable IR versions the platform accepts | +| `features` | The language features the platform runs; empty in this version | +| `limits` | `max_loop_items`, `default_loop_max_items`, `max_loop_depth`, `max_concurrency`, `max_run_iterations`, `max_call_depth` | +| `step_kinds`, `operators`, `workflow_fields` | One entry each, with the `feature` it needs (if any) and `studio`: `ready` when Studio edits it, `pending` until then | + ## Values and budgets Definitions and payloads use plain JSON values: null, Boolean, integer, float, @@ -370,5 +433,7 @@ backend is absent. The `integration` and `e2e` pytest markers are registered. | A field is rejected although it looks right | Field names are camelCase and case-sensitive; snake_case keys are rejected | Use the published name, such as `timeoutSeconds` | | `null` is rejected for an optional field | Optional fields may be omitted but not set to null | Remove the field | | A step's `retry` or `timeoutSeconds` is rejected | Retry and per-attempt timeout belong to the action definition | Move them to the action; use `spec.timeoutSeconds` for a workflow-wide timeout | +| `WV-COMP-UNSUPPORTED_FEATURE` | The document uses `forEach`, `callWorkflow`, `concat`, or `join`, which this version of the compiler does not compile yet | Keep the document for a later version, or replace the construct with the steps the compiler supports | +| `onFailure` is rejected on a call step | `onFailure` applies only when the call waits for its result | Remove `onFailure`, or set `mode: wait` | | Partial validation passes, but there is no artifact | Partial validation never produces one | Compile with an explicit catalog | | Compilation passes, but the run fails to start | Compilation proves no worker, connection, or permission | Check the activation's bindings and your grants | diff --git a/docs/contributing/source-documentation.md b/docs/contributing/source-documentation.md index d2d5849e..f1bf4377 100644 --- a/docs/contributing/source-documentation.md +++ b/docs/contributing/source-documentation.md @@ -159,8 +159,11 @@ Choose the kind that matches the file: Most inventory entries are `commentless`: strict JSON, the Python, npm, and Cargo lockfiles, the desktop icons and installer background, and `py.typed`. Two YAML files are `immutable-fixture` entries because parser and CLI tests assert -their source positions. One entry is `generated`: Studio's schema test corpus, produced by -[`studio_schema_fixtures.py`](../../scripts/studio_schema_fixtures.py). No vendored +their source positions. `generated` entries name their generator in `source`. Examples are +Studio's schema test corpus, produced by +[`studio_schema_fixtures.py`](../../scripts/studio_schema_fixtures.py), the shared language +fixtures, produced by [`language_fixtures.py`](../../scripts/language_fixtures.py), and the +worker lockfiles, produced from the `pyproject.toml` beside each. No vendored third-party source is declared. Third-party dependencies keep their own distribution metadata and license terms; the inventory does not certify redistribution compliance. diff --git a/docs/contributing/source-inventory.toml b/docs/contributing/source-inventory.toml index bfb8492e..3e0dd390 100644 --- a/docs/contributing/source-inventory.toml +++ b/docs/contributing/source-inventory.toml @@ -340,6 +340,33 @@ copyright = "Copyright 2026 Firefly Software Foundation." author = "Firefly Software Foundation" license = "Apache-2.0" +[[exceptions]] +path = "studio/tests/fixtures/language/instance-keys.json" +kind = "generated" +source = "scripts/language_fixtures.py" +reason = "Strict JSON instance-key parse and format cases shared by the Python and Studio suites; regenerate with the generator, which also checks freshness." +copyright = "Copyright 2026 Firefly Software Foundation." +author = "Firefly Software Foundation" +license = "Apache-2.0" + +[[exceptions]] +path = "studio/tests/fixtures/language/manifest.json" +kind = "generated" +source = "scripts/language_fixtures.py" +reason = "Strict JSON snapshot of the language manifest shared by the Python and Studio suites; regenerate with the generator, which also checks freshness." +copyright = "Copyright 2026 Firefly Software Foundation." +author = "Firefly Software Foundation" +license = "Apache-2.0" + +[[exceptions]] +path = "studio/tests/fixtures/language/text-conversion.json" +kind = "generated" +source = "scripts/language_fixtures.py" +reason = "Strict JSON concat and join text conversion cases shared by the Python and Studio suites; regenerate with the generator, which also checks freshness." +copyright = "Copyright 2026 Firefly Software Foundation." +author = "Firefly Software Foundation" +license = "Apache-2.0" + [[exceptions]] path = "studio/tests/fixtures/shipped-schemas.json" kind = "generated" diff --git a/docs/reference/api.md b/docs/reference/api.md index d51ab6f2..b1a85aaf 100644 --- a/docs/reference/api.md +++ b/docs/reference/api.md @@ -522,6 +522,7 @@ operation in the [full API reference](api-explorer.md); to generate a client, | `compiler.evaluate_decision` | `POST /api/v1/tenants/{tenant}/projects/{project}/compiler/evaluate-decision` | compile | | `catalog.read` | `GET /api/v1/tenants/{tenant}/projects/{project}/catalog` | catalog.read | | `capabilities.read` | `GET /api/v1/tenants/{tenant}/projects/{project}/capabilities` | catalog.read | +| `language.read` | `GET /api/v1/tenants/{tenant}/projects/{project}/language` | catalog.read | | `schemas.read` | `GET /api/v1/tenants/{tenant}/projects/{project}/schemas` | catalog.read | | `connector_descriptors.list` | `GET /api/v1/tenants/{tenant}/projects/{project}/connector-descriptors` | catalog.read | | `connector_descriptors.read` | `GET /api/v1/tenants/{tenant}/projects/{project}/connector-descriptors/{adapter}` | catalog.read | diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 23f6a1e7..2e19cc5d 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -568,6 +568,8 @@ workspace, and you need each operation's grant. Find the body schemas in the ```sh # Read the project catalog. weave remote catalog --output json +# Read the language manifest: step kinds, operators, features, and limits. +weave remote language --output json # Compile a request file through the platform. weave remote compile --request compiler-request.json --output json # Save a new draft, then a change to revision 1 of it. diff --git a/docs/reference/compiler.md b/docs/reference/compiler.md index ada3d010..1f0d4d67 100644 --- a/docs/reference/compiler.md +++ b/docs/reference/compiler.md @@ -209,6 +209,11 @@ selection reads expression syntax, never operator-shaped literal data. Existing workflows retain their prior IR version and executable digest. Readers that do not support v1alpha3 reject these artifacts; upgrade runtime readers before activating them. The capabilities response advertises accepted IR versions. +Executables also define a sorted `features` list of the language features they +need (`text.concat`, `text.join`, `flow.forEach`, `flow.callWorkflow`, +`ai.agent`, `ai.memory`). It is omitted when empty, which it is for every +executable this version compiles, so existing digests do not change; a non-empty +list requires `weave/ir-v1alpha4`, which this version does not accept yet. `ir.py` defines every node, edge, control, join, guard, dependency, and executable field as a strict Pydantic model, and `export_schemas()` publishes `executable` and @@ -322,6 +327,37 @@ flow as: start, decide, take one branch, join, end. The rules behind it: node `path`, the input, a branch output, or the workflow output, and validate it against `schemaRef` using the locked schema resources as the local bundle. +### Instance keys + +A node can run more than once in one run: once per loop iteration, or once per +agent turn or tool call. Each run of a node has an **instance key**: the node ID, +then one `[index]` per enclosing loop (outer loop first), then optional +`#`-separated activation segments, then an optional `~count` for a loop's yield: + +```text +instance-key = node-id *("[" index "]") ["#" seg *("." seg)] ["~" count] +index = "0" / (%x31-39 *DIGIT) ; loop iteration, from 0 +count = %x31-39 *DIGIT ; yield count, from 1 +seg = 1*(ALPHA / DIGIT) ; turn, tool call, or review +``` + +Examples: `send` (no loop), `send[3]`, `send[3][0]`, `support#2.1`, +`support[3]#2.1.review`, and `notify[1]~2`. Indexes and counts have no leading +zeros and are at most 2^53 − 1. Node IDs never contain `[`, `]`, `#`, or `~`. +A node ID is a step ID or a synthetic ID (`@run`, `@start`, `@end`, `@legacy`, +`@join:`, `@branch::`): an optional leading `@`, then an ASCII letter +or digit, then ASCII letters, digits, `_`, `.`, `:`, `@` and `-` only. So +`firefly_weave.contracts.instance_keys.node_of(key)` returns the node ID by +cutting at the first separator; `split_instance` parses a whole key and +`format_instance` writes it back, and both reject any other spelling. API views +show `node_id` (the node ID), `instance_key` (the full key, or `""` when it equals +the node ID), and `iteration` (the loop indexes); `instance_view(key)` returns all +three. `INSTANCE_KEY_PATTERN` is the same grammar as an unanchored regular +expression. Python checks a key with `re.fullmatch(INSTANCE_KEY_PATTERN, key)`; +JavaScript (with the `u` flag) and JSON Schema `pattern` use the anchored form +`^(?:...)$`. API models declare key fields as `InstanceKeyText` (or +`InstanceKeyTextOrEmpty` where `""` is allowed). + ## Import an artifact `import_artifact(bytes_or_text_or_object, *, limits=ArtifactLimits())` re-checks @@ -462,6 +498,7 @@ is not classified. Safe ordinary artifacts keep their canonical digests. | `WV-COMP-TYPE_MISMATCH` at `/spec/output` | An expression's type cannot satisfy the target schema | Change the expression or the schema; the authoring guide shows an example | | `WV-COMP-UNAVAILABLE_REFERENCE` | An expression reads a step that has not run yet in its scope, or a value private to another branch | Read the step only after it runs, or read the decision or parallel step's output | | `WV-COMP-UNKNOWN_ACTION` (or `_CONNECTOR`, `_TASK`, `_ADAPTER`) | The catalog does not contain that exact reference | Add the exact version to the catalog, or fix the reference | +| `WV-COMP-UNSUPPORTED_FEATURE` | The workflow or decision table uses a construct whose language feature this compiler does not compile yet; the message names the feature | Keep the document for a later version, or use the steps and operators this version compiles | | `WV-COMP-CATALOG_PENDING` notes in Studio | Studio works without the catalog and checks references later | Nothing; connect and **Validate** against the project catalog | | `WV-COMP-CONNECTION` | A step names a connection slot the workflow does not declare, omits a slot its action requires, or uses a slot declared for a different connector | Declare the slot in `spec.connections` with the action's exact connector, or fix the step's `connection` | | `WV-COMP-CONFIG_CONTRACT` | An action's `config` does not fit its connector | Fix the configuration; [HTTP profiles](../connectors/http-profiles.md) lists the built-in HTTP rules | diff --git a/docs/reference/sdk.md b/docs/reference/sdk.md index 6b5d8196..f47d2c62 100644 --- a/docs/reference/sdk.md +++ b/docs/reference/sdk.md @@ -343,7 +343,7 @@ capability. | Methods | Purpose | | --- | --- | -| `compile`, `validate`, `catalog`, `schemas`, `capabilities` | Compile and validate; read the catalog and the supported language, limits, and connectors | +| `compile`, `validate`, `catalog`, `schemas`, `capabilities`, `language` | Compile and validate; read the catalog, the supported language, limits, and connectors, and the language manifest (`language()` returns a `LanguageManifest`) | | `list_connector_descriptors`, `read_connector_descriptor` | Installed connectors and the exact manifest to publish (new in 0.1.0a7) | | `save_draft`, `read_draft`, `export_draft`, `delete_draft`, `list_definitions` | Append-only drafts, their retirement, and catalog discovery | | `publish`, `read_definition`, `export_definition`, `retire_definition` | Immutable workflow, Action, and Connector versions | diff --git a/examples/language/notify-customer.workflow.yaml b/examples/language/notify-customer.workflow.yaml new file mode 100644 index 00000000..85dc6dae --- /dev/null +++ b/examples/language/notify-customer.workflow.yaml @@ -0,0 +1,46 @@ +# Copyright 2026 Firefly Software Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# Author: Firefly Software Foundation +# SPDX-License-Identifier: Apache-2.0 + +# A sub-workflow: "spec.callable" lets other workflows in the project call this exact version. +apiVersion: weave/v1alpha1 +kind: Workflow +metadata: + name: notify-customer + version: 1.0.0 +spec: + callable: {} + inputSchema: + type: object + properties: + customerId: {type: string} + message: {type: string} + required: [customerId, message] + additionalProperties: false + outputSchema: + type: object + properties: + customerId: {type: string} + delivered: {type: boolean} + required: [customerId, delivered] + additionalProperties: false + steps: + - id: record + kind: transform + value: + object: + customerId: {ref: /input/customerId} + delivered: {literal: true} + output: {ref: /steps/record/output} diff --git a/examples/language/notify-overdue.workflow.yaml b/examples/language/notify-overdue.workflow.yaml new file mode 100644 index 00000000..a3876b74 --- /dev/null +++ b/examples/language/notify-overdue.workflow.yaml @@ -0,0 +1,76 @@ +# Copyright 2026 Firefly Software Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# Author: Firefly Software Foundation +# SPDX-License-Identifier: Apache-2.0 + +# Loop over items with a text template. The compiler reports "not yet supported" for forEach and concat +# until their language milestones land; the document shape is final. +apiVersion: weave/v1alpha1 +kind: Workflow +metadata: + name: notify-overdue + version: 1.0.0 +spec: + inputSchema: + type: object + properties: + invoices: + type: array + items: + type: object + properties: + number: {type: string} + customerEmail: {type: string} + daysLate: {type: integer} + required: [number, customerEmail, daysLate] + additionalProperties: false + required: [invoices] + additionalProperties: false + outputSchema: + type: object + connections: + mail: + connector: weave-email@1.0.0 + steps: + - id: notify + kind: forEach + items: {ref: /input/invoices} + concurrency: 4 + maxItems: 500 + collect: all + body: + steps: + - id: send + kind: action + uses: mail.send-reminder@1.0.0 + connection: mail + with: + object: + to: {ref: /item/customerEmail} + subject: + op: + name: concat + args: + - {literal: "Invoice "} + - {ref: /item/number} + - {literal: " is overdue by "} + - {ref: /item/daysLate} + - {literal: " days"} + output: + object: + invoice: {ref: /item/number} + messageId: {ref: /steps/send/output/id} + output: + object: + sent: {ref: /steps/notify/output} diff --git a/examples/language/order-intake.workflow.yaml b/examples/language/order-intake.workflow.yaml new file mode 100644 index 00000000..c5de1862 --- /dev/null +++ b/examples/language/order-intake.workflow.yaml @@ -0,0 +1,64 @@ +# Copyright 2026 Firefly Software Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# Author: Firefly Software Foundation +# SPDX-License-Identifier: Apache-2.0 + +# Call a workflow and wait for its result, then start another call without waiting. The compiler reports +# "not yet supported" for callWorkflow, concat and join until their language milestones land. +apiVersion: weave/v1alpha1 +kind: Workflow +metadata: + name: order-intake + version: 1.0.0 +spec: + inputSchema: + type: object + properties: + customerId: {type: string} + orderId: {type: string} + tags: + type: array + items: {type: string} + required: [customerId, orderId, tags] + additionalProperties: false + outputSchema: + type: object + steps: + - id: notify + kind: callWorkflow + uses: notify-customer@1.0.0 + with: + object: + customerId: {ref: /input/customerId} + message: + op: + name: concat + args: [{literal: "Order "}, {ref: /input/orderId}, {literal: " received"}] + mode: wait + onFailure: continue + businessKey: {ref: /input/orderId} + - id: audit + kind: callWorkflow + uses: notify-customer@1.0.0 + with: + object: + customerId: {literal: auditor} + message: + op: + name: join + args: [{ref: /input/tags}, {literal: ", "}] + mode: detach + output: + object: + notification: {ref: /steps/notify/output/status} diff --git a/scripts/language_fixtures.py b/scripts/language_fixtures.py new file mode 100644 index 00000000..df8af8fd --- /dev/null +++ b/scripts/language_fixtures.py @@ -0,0 +1,222 @@ +# Copyright 2026 Firefly Software Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# Author: Firefly Software Foundation +# SPDX-License-Identifier: Apache-2.0 + +"""Write the shared language fixtures that the Python suite and Studio's vitest suite both run. + +The fixtures freeze the language contracts of language milestone M0 (language spec 14.5): instance-key parse and +format cases with the ``INSTANCE_KEY_PATTERN`` expression (3.2), the text conversion table for ``concat`` and +``join`` (4.3), and the language manifest snapshot (14.1). Expected values are written here by hand; the generator +refuses to write a fixture that the implementation disagrees with. Later milestones add template segments and +``scope_at`` cases. + +Run ``uv run --locked --all-extras python scripts/language_fixtures.py`` to regenerate the files, or add +``--check`` to fail when they are out of date. +""" + +import argparse +import json +import re +import sys +from pathlib import Path +from typing import Any + +import rfc8785 + +from firefly_weave.contracts.instance_keys import ( + INSTANCE_KEY_PATTERN, + InstanceKey, + InvalidInstanceKey, + format_instance, + instance_view, + node_of, + split_instance, +) +from firefly_weave.contracts.language import language_manifest + +ROOT = Path(__file__).resolve().parents[1] +OUTPUT = ROOT / "studio" / "tests" / "fixtures" / "language" +GENERATOR = "scripts/language_fixtures.py" + +# (key, node_id, indexes, segments, count) +VALID_KEYS: tuple[tuple[str, str, list[int], list[str], int | None], ...] = ( + ("send", "send", [], [], None), + ("send[3]", "send", [3], [], None), + ("send[3][0]", "send", [3, 0], [], None), + ("support#2", "support", [], ["2"], None), + ("support#2.1", "support", [], ["2", "1"], None), + ("support#2.1.review", "support", [], ["2", "1", "review"], None), + ("support[3]#2.1.review", "support", [3], ["2", "1", "review"], None), + ("notify~2", "notify", [], [], 2), + ("notify[1]~2", "notify", [1], [], 2), + ("@join:notify", "@join:notify", [], [], None), + ("@branch:notify:0[4]", "@branch:notify:0", [4], [], None), + ("a.b_c-d[9007199254740991]", "a.b_c-d", [9007199254740991], [], None), +) +# (key, reason). The Python unit test also covers two keys a JSON fixture cannot carry: a 5000-digit index and a +# lone surrogate. +INVALID_KEYS: tuple[tuple[str, str], ...] = ( + ("", "empty key"), + ("[3]", "no node ID"), + ("#2", "no node ID"), + ("~1", "no node ID"), + ("send[03]", "index with a leading zero"), + ("send[-1]", "negative index"), + ("send[]", "empty index"), + ("send[3", "unclosed index"), + ("send]3[", "bracket in the node ID"), + ("send[3]x", "text after an index"), + ("send#", "empty segment list"), + ("send#2.", "empty segment"), + ("send#.2", "empty segment"), + ("send#2..1", "empty segment"), + ("send#ü", "non-ASCII segment"), + ("send#2[3]", "index after a segment"), + ("send~0", "yield count zero"), + ("send~01", "yield count with a leading zero"), + ("send~1[2]", "index after the yield count"), + ("send~1#2", "segment after the yield count"), + ("send~1~2", "two yield counts"), + ("send[9007199254740992]", "index above 2**53 - 1"), + ("send~9007199254740992", "yield count above 2**53 - 1"), + ("a b", "whitespace in the node ID"), + ("send\n", "control character in the node ID"), + ("send\x00", "NUL in the node ID"), + ("se/nd", "slash in the node ID"), + ("ü[1]", "non-ASCII node ID"), + ("@", "synthetic marker without a name"), +) +# Values that concat and join convert to text; floats follow ECMAScript Number::toString (RFC 8785). +CONVERSION_VALUES: tuple[str | bool | int | float, ...] = ( + "Invoice ", + "", + "ü\U0001f600", + True, + False, + 0, + 7, + -12, + 9007199254740991, + 2.0, + -0.0, + 0.1, + 1.5, + -2.5, + 100.0, + 1e20, + 1e21, + 1e-7, + 0.000001, + 123456789012345680000.0, + 5e-324, + 1.7976931348623157e308, +) + + +def text_of(value: str | bool | int | float) -> str: + """The reference conversion of language spec 4.3.""" + if isinstance(value, str): + return value + if isinstance(value, bool): + return "true" if value else "false" + return rfc8785.dumps(value).decode() + + +def instance_key_cases() -> dict[str, Any]: + valid = [] + for key, node_id, indexes, segments, count in VALID_KEYS: + parsed = InstanceKey(node_id, tuple(indexes), tuple(segments), count) + view = instance_view(key) + if ( + split_instance(key) != parsed + or format_instance(parsed) != key + or node_of(key) != node_id + or re.fullmatch(INSTANCE_KEY_PATTERN, key) is None + ): + raise SystemExit(f"Implementation disagrees with the fixture for {key!r}") + valid.append( + { + "key": key, + "node_id": node_id, + "indexes": indexes, + "segments": segments, + "count": count, + "view": {"node_id": view.node_id, "instance_key": view.instance_key, "iteration": view.iteration}, + } + ) + invalid = [] + for key, reason in INVALID_KEYS: + if re.fullmatch(INSTANCE_KEY_PATTERN, key) is not None: + raise SystemExit(f"INSTANCE_KEY_PATTERN accepts the invalid key {key!r}") + try: + split_instance(key) + except InvalidInstanceKey: + invalid.append({"key": key, "reason": reason}) + else: + raise SystemExit(f"Implementation accepts the invalid key {key!r}") + return { + "generator": GENERATOR, + "grammar": "language spec 3.2", + "pattern": INSTANCE_KEY_PATTERN, + "valid": valid, + "invalid": invalid, + } + + +def conversion_cases() -> dict[str, Any]: + return { + "generator": GENERATOR, + "rule": "language spec 4.3", + "cases": [{"value": value, "text": text_of(value)} for value in CONVERSION_VALUES], + } + + +def manifest_snapshot() -> dict[str, Any]: + return {"generator": GENERATOR, "manifest": language_manifest().model_dump(mode="json")} + + +def render() -> dict[str, str]: + files = { + "instance-keys.json": instance_key_cases(), + "text-conversion.json": conversion_cases(), + "manifest.json": manifest_snapshot(), + } + return {name: json.dumps(value, indent=2, ensure_ascii=False) + "\n" for name, value in files.items()} + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=(__doc__ or "").splitlines()[0]) + parser.add_argument("--check", action="store_true", help="fail when a committed fixture is out of date") + parser.add_argument("--output", type=Path, default=OUTPUT) + args = parser.parse_args(argv) + stale = [] + for name, text in render().items(): + path = args.output / name + if args.check: + # read_text applies universal newlines, so CRLF checkouts compare equal. + if not path.exists() or path.read_text(encoding="utf-8") != text: + stale.append(name) + continue + path.parent.mkdir(parents=True, exist_ok=True) + path.write_bytes(text.encode("utf-8")) + print(f"{path}: written") + if stale: + print(f"Out of date in {args.output}: {', '.join(stale)}; run {GENERATOR}", file=sys.stderr) + return 1 + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/src/firefly_weave/api/compiler.py b/src/firefly_weave/api/compiler.py index 14bd3e14..e60fa2b6 100644 --- a/src/firefly_weave/api/compiler.py +++ b/src/firefly_weave/api/compiler.py @@ -163,6 +163,13 @@ async def capabilities(self, request: Request) -> dict[str, Any]: ) return result.model_dump(mode="json") + @operation("language.read") + async def language(self, request: Request) -> dict[str, Any]: + from firefly_weave.contracts.language import language_manifest + + await self.service.catalog(request.state.principal, request_scope(request), context=request.state.audit_context) + return language_manifest().model_dump(mode="json") + @operation("schemas.read") async def schemas(self, request: Request) -> dict[str, Any]: from firefly_weave.contracts.schema_export import export_schemas diff --git a/src/firefly_weave/cli/remote.py b/src/firefly_weave/cli/remote.py index 28f6f73c..e701d4d3 100644 --- a/src/firefly_weave/cli/remote.py +++ b/src/firefly_weave/cli/remote.py @@ -364,6 +364,7 @@ def family(name: str, prefix: str, *, include: tuple[str, ...] = ()) -> click.Gr "compiler.evaluate_decision", "catalog.read", "capabilities.read", + "language.read", "schemas.read", ): remote.add_command(command(operation_id, operation_id.split(".")[0] if operation_id.endswith(".read") else None)) diff --git a/src/firefly_weave/compiler/analyzer.py b/src/firefly_weave/compiler/analyzer.py index 1163f029..ff6cd244 100644 --- a/src/firefly_weave/compiler/analyzer.py +++ b/src/firefly_weave/compiler/analyzer.py @@ -40,6 +40,7 @@ measure_value, pointer_segments, ) +from firefly_weave.compiler.language_support import unsupported_uses from firefly_weave.compiler.llm import llm_action_valid, llm_input, llm_output_schema from firefly_weave.compiler.parser import ParsedSource from firefly_weave.compiler.schema_profile import DEFAULT_CONTRACT_LIMITS, SchemaLimits @@ -903,6 +904,20 @@ def run(self, *, partial: bool = False) -> AnalysisResult: resource = self.catalog.resolve("Schema", name) if resource is not None: self.resolved[("Schema", name)] = resource + # Constructs whose feature is not compiled yet stop here, before budgets and flow analysis read them. + unsupported = unsupported_uses(value) + for use in unsupported: + self.add( + Diagnostic( + code="WV-COMP-UNSUPPORTED_FEATURE", + severity="error", + stage="semantic", + path=use.path, + message=use.message, + ) + ) + if unsupported: + return self.finish(model.kind, definition) if model.kind != "Workflow": self.manifest(value, "", partial=partial) return self.finish(model.kind, definition) diff --git a/src/firefly_weave/compiler/ir.py b/src/firefly_weave/compiler/ir.py index 750a3656..fdece363 100644 --- a/src/firefly_weave/compiler/ir.py +++ b/src/firefly_weave/compiler/ir.py @@ -21,7 +21,7 @@ from collections import deque from typing import Annotated, Literal, cast -from pydantic import Field, model_validator +from pydantic import AfterValidator, Field, model_validator from firefly_weave.compiler.canonical import canonical_digest from firefly_weave.compiler.catalog import CatalogResource, FrozenDocument, ResourceKind @@ -56,6 +56,7 @@ load_definition, ) from firefly_weave.contracts.diagnostics import Diagnostic, SourceRange +from firefly_weave.contracts.language_features import LanguageFeature from firefly_weave.contracts.llm import LLMProfile from firefly_weave.contracts.values import JsonObject, JsonObjectData, JsonValue, UnicodeString @@ -64,6 +65,14 @@ IR_VERSION = "weave/ir-v1alpha1" HUMAN_IR_VERSION = "weave/ir-v1alpha2" COMPARISON_IR_VERSION = "weave/ir-v1alpha3" +# The IR version that carries language features (language spec 3.1); accepted from language milestone M1. +IR_VERSION_EXTENSIONS = "weave/ir-v1alpha4" + + +def _sorted_unique(features: list[LanguageFeature]) -> list[LanguageFeature]: + if features != sorted(set(features)): + raise ValueError("Language features must be sorted and unique") + return features class Dependency(ContractModel): @@ -438,9 +447,15 @@ class ExecutableBase(ContractModel): dependencies: list[Dependency] schemas: dict[Digest, JsonObjectData] guards: list[Guard] + # Required language features: sorted, unique, omitted when empty, and part of the artifact digest. + features: Annotated[list[LanguageFeature], AfterValidator(_sorted_unique)] = Field( + default_factory=list, exclude_if=lambda value: not value, json_schema_extra={"uniqueItems": True} + ) @model_validator(mode="after") def references(self) -> ExecutableBase: + if self.features and self.ir_version != IR_VERSION_EXTENSIONS: + raise ValueError("Language features require weave/ir-v1alpha4") identities = [(d.kind, d.reference) for d in self.dependencies] if len(set(identities)) != len(identities): raise ValueError("Duplicate dependency identity") diff --git a/src/firefly_weave/compiler/language_support.py b/src/firefly_weave/compiler/language_support.py new file mode 100644 index 00000000..987976cb --- /dev/null +++ b/src/firefly_weave/compiler/language_support.py @@ -0,0 +1,153 @@ +# Copyright 2026 Firefly Software Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# Author: Firefly Software Foundation +# SPDX-License-Identifier: Apache-2.0 + +"""Find language constructs whose feature this compiler does not compile yet. + +The definition schema accepts every frozen construct (language milestone M0); the compiler reports each use of +a feature outside ``COMPILED_FEATURES`` as ``WV-COMP-UNSUPPORTED_FEATURE`` instead of lowering it. Each language +milestone that lands a feature's compiler support adds that feature here. +""" + +from collections import deque +from dataclasses import dataclass +from typing import Final, cast + +from firefly_weave.compiler.source_map import pointer_child +from firefly_weave.contracts.language_features import KIND_FEATURES, OPERATOR_FEATURES, LanguageFeature +from firefly_weave.contracts.values import JsonObject, JsonValue + +COMPILED_FEATURES: Final[frozenset[LanguageFeature]] = frozenset() + +# Expression positions per step kind; blocks are walked separately. +_EXPRESSIONS: Final[dict[str, tuple[str, ...]]] = { + "action": ("with",), + "llm": ("prompt", "context"), + "transform": ("value",), + "decisionTable": ("with",), + "humanTask": ("title", "context"), + "forEach": ("items",), + "callWorkflow": ("with", "businessKey"), +} + + +@dataclass(frozen=True) +class UnsupportedUse: + path: str + construct: str + feature: LanguageFeature + + @property + def message(self) -> str: + return ( + f"{self.construct} needs the language feature {self.feature}, " + "which this version of Firefly Weave does not compile yet." + ) + + +def unsupported_uses(document: JsonObject) -> list[UnsupportedUse]: + """Every use of a kind or operator whose feature is not compiled, sorted by path; iterative, never recursive.""" + found: list[UnsupportedUse] = [] + expressions: list[tuple[JsonValue, str]] = [] + spec = document.get("spec") + if not isinstance(spec, dict): + return found + if document.get("kind") == "Workflow": + blocks: deque[tuple[JsonValue, str]] = deque([(spec.get("steps"), "/spec/steps")]) + expressions.append((spec.get("output"), "/spec/output")) + while blocks: + steps, path = blocks.popleft() + for index, step in enumerate(steps if isinstance(steps, list) else []): + if not isinstance(step, dict): + continue + location = f"{path}/{index}" + kind = step.get("kind") + feature = KIND_FEATURES.get(kind) if isinstance(kind, str) else None + if feature is not None and feature not in COMPILED_FEATURES: + found.append(UnsupportedUse(location + "/kind", f"The {kind} step", feature)) + for name in _EXPRESSIONS.get(str(kind), ()): + if name in step: + expressions.append((step[name], f"{location}/{name}")) + for branch, branch_path in _branches(step, location): + if "when" in branch: + expressions.append((branch["when"], branch_path + "/when")) + blocks.append((branch.get("steps"), branch_path + "/steps")) + expressions.append((branch.get("output"), branch_path + "/output")) + elif document.get("kind") == "DecisionTable": + rules = spec.get("rules") + for index, rule in enumerate(rules if isinstance(rules, list) else []): + if isinstance(rule, dict): + expressions.append((rule.get("when"), f"/spec/rules/{index}/when")) + expressions.append((rule.get("output"), f"/spec/rules/{index}/output")) + if "defaultOutput" in spec: + expressions.append((spec["defaultOutput"], "/spec/defaultOutput")) + for expression, path in expressions: + found.extend(_operators(expression, path)) + return sorted(found, key=lambda use: _order(use.path)) + + +def _branches(step: JsonObject, path: str) -> list[tuple[JsonObject, str]]: + kind = step.get("kind") + result: list[tuple[JsonObject, str]] = [] + if kind == "switch": + cases = step.get("cases") + for index, case in enumerate(cases if isinstance(cases, list) else []): + if isinstance(case, dict): + result.append((case, f"{path}/cases/{index}")) + if isinstance(step.get("default"), dict): + result.append((cast(JsonObject, step["default"]), path + "/default")) + elif kind == "parallel" and isinstance(step.get("branches"), dict): + for name, branch in sorted(cast(JsonObject, step["branches"]).items()): + if isinstance(branch, dict): + result.append((branch, pointer_child(path + "/branches", name))) + elif kind == "forEach" and isinstance(step.get("body"), dict): + result.append((cast(JsonObject, step["body"]), path + "/body")) + return result + + +def _operators(expression: JsonValue, path: str) -> list[UnsupportedUse]: + found: list[UnsupportedUse] = [] + pending: list[tuple[JsonValue, str]] = [(expression, path)] + while pending: + node, location = pending.pop() + if not isinstance(node, dict): + continue + if isinstance(node.get("op"), dict): + operation = cast(JsonObject, node["op"]) + name = operation.get("name") + feature = OPERATOR_FEATURES.get(name) if isinstance(name, str) else None + if feature is not None and feature not in COMPILED_FEATURES: + found.append(UnsupportedUse(location + "/op/name", f"The {name} operator", feature)) + arguments = operation.get("args") + for index, argument in enumerate(arguments if isinstance(arguments, list) else []): + pending.append((argument, f"{location}/op/args/{index}")) + elif isinstance(node.get("object"), dict): + for key, child in cast(JsonObject, node["object"]).items(): + pending.append((child, pointer_child(location + "/object", key))) + elif isinstance(node.get("array"), list): + for index, child in enumerate(cast(list[JsonValue], node["array"])): + pending.append((child, f"{location}/array/{index}")) + return found + + +def _order(path: str) -> tuple[tuple[int, int, str], ...]: + """Sort key: ASCII-digit segments by length and then text, other segments after them as text. + + For digit strings without leading zeros, length then text is numeric order. Nothing calls ``int()``, whose digit + limit would raise on a long key. + """ + return tuple( + (0, len(part), part) if part.isascii() and part.isdigit() else (1, 0, part) for part in path.split("/")[1:] + ) diff --git a/src/firefly_weave/compiler/scope.py b/src/firefly_weave/compiler/scope.py new file mode 100644 index 00000000..df2fcaef --- /dev/null +++ b/src/firefly_weave/compiler/scope.py @@ -0,0 +1,81 @@ +# Copyright 2026 Firefly Software Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# Author: Firefly Software Foundation +# SPDX-License-Identifier: Apache-2.0 + +"""Scope suggestions: which data an expression at a definition path may reference (language spec 14.3). + +The signature and the camelCase result shape are frozen in language milestone M0 so that the Studio host's +``POST /studio/local/scope`` and Studio's offline ``scope.ts`` fallback share one contract. Results come from the +compiler's own dominance and type analysis, including the inferred outputs of ``forEach``, ``callWorkflow`` and +``agent`` steps, which Studio does not reimplement. +""" + +from typing import Literal + +from pydantic import Field + +from firefly_weave.compiler.catalog import CatalogSnapshot +from firefly_weave.contracts.definitions import ( + ContractModel, + JsonPointer, + OmissionOnly, + ResourceName, + _is_absent, + _omit_absent_default, +) +from firefly_weave.contracts.values import JsonObject, JsonObjectData + +type ScopeSource = Literal["input", "step", "item", "index"] +type JsonType = Literal["array", "boolean", "integer", "null", "number", "object", "string"] + + +class ScopeEntry(ContractModel): + """One suggestion; field names match Studio's ``ScopeEntry`` (``studio/src/app/forms/core/scope.ts``).""" + + ref: JsonPointer + source: ScopeSource + # The loop step ID for item and index entries. + loop: OmissionOnly[ResourceName] = Field( + default=None, exclude_if=_is_absent, json_schema_extra=_omit_absent_default + ) + step_id: OmissionOnly[ResourceName] = Field( + default=None, alias="stepId", exclude_if=_is_absent, json_schema_extra=_omit_absent_default + ) + step_kind: OmissionOnly[str] = Field( + default=None, alias="stepKind", exclude_if=_is_absent, json_schema_extra=_omit_absent_default + ) + path: list[str] + label: str + breadcrumb: str + schema_value: JsonObjectData = Field(alias="schema") + types: list[JsonType] + type_label: str = Field(alias="typeLabel") + optional: bool + + +class ScopeResult(ContractModel): + found: bool + evaluated: bool + truncated: bool + entries: list[ScopeEntry] + + +def scope_at(document: JsonObject, path: JsonPointer, *, catalog: CatalogSnapshot | None = None) -> ScopeResult: + """Suggestions for the expression at ``path`` (an RFC 6901 pointer into ``document``). + + Without a catalog, resources resolve as in offline authoring and their outputs stay unconstrained. + Implemented with the loop compiler (language milestone M3); until then it raises ``NotImplementedError``. + """ + raise NotImplementedError("Scope suggestions are not available in this version of Firefly Weave") diff --git a/src/firefly_weave/contracts/definitions.py b/src/firefly_weave/contracts/definitions.py index 423aa587..d05b4055 100644 --- a/src/firefly_weave/contracts/definitions.py +++ b/src/firefly_weave/contracts/definitions.py @@ -23,6 +23,7 @@ from pydantic import BaseModel, BeforeValidator, ConfigDict, Discriminator, Field, Tag, TypeAdapter, model_validator from pydantic.json_schema import JsonSchemaValue, SkipJsonSchema +from firefly_weave.contracts.language_features import DEFAULT_LOOP_MAX_ITEMS from firefly_weave.contracts.llm import LLMProfile from firefly_weave.contracts.values import MAX_SAFE_INTEGER, JsonData, JsonObject, JsonObjectData, UnicodeString @@ -59,6 +60,8 @@ "notIn", "startsWith", "endsWith", + "concat", + "join", ] @@ -325,6 +328,41 @@ class FailStep(ContractModel): message: Annotated[UnicodeString, Field(min_length=1)] +class ForEachStep(ContractModel): + """Run ``body`` once per element of ``items`` and collect one result per element, in element order.""" + + id: ResourceName + kind: Literal["forEach"] + items: Expression + # Defaults are materialized in the canonical document, like ``retry`` and ``decisions``. + concurrency: PositiveInt = 1 + max_items: PositiveInt = Field(default=DEFAULT_LOOP_MAX_ITEMS, alias="maxItems") + collect: Literal["all", "nonNull"] = "all" + body: Branch + + +class CallWorkflowStep(ContractModel): + """Run an exact published workflow version as a separate child run.""" + + id: ResourceName + kind: Literal["callWorkflow"] + uses: VersionedReference + input: Expression = Field(alias="with") + mode: Literal["wait", "detach"] = "wait" + on_failure: OmissionOnly[Literal["stop", "continue"]] = Field( + default=None, alias="onFailure", exclude_if=_is_absent, json_schema_extra=_omit_absent_default + ) + business_key: OmissionOnly[Expression] = Field( + default=None, alias="businessKey", exclude_if=_is_absent, json_schema_extra=_omit_absent_default + ) + + @model_validator(mode="after") + def failure_needs_wait(self) -> CallWorkflowStep: + if self.mode == "detach" and self.on_failure is not None: + raise ValueError("onFailure applies only when the call waits for its result") + return self + + type Step = Annotated[ ActionStep | LLMStep @@ -335,11 +373,32 @@ class FailStep(ContractModel): | WaitStep | SignalStep | HumanTaskStep - | FailStep, + | FailStep + | ForEachStep + | CallWorkflowStep, Field(discriminator="kind"), ] +type CallerNames = Annotated[ + list[ResourceName], Field(min_length=1, max_length=100, json_schema_extra={"uniqueItems": True}) +] + + +class CallableSpec(ContractModel): + """A workflow's opt-in to being called; omitted allowedCallers means any workflow in the project.""" + + allowed_callers: OmissionOnly[CallerNames] = Field( + default=None, alias="allowedCallers", exclude_if=_is_absent, json_schema_extra=_omit_absent_default + ) + + @model_validator(mode="after") + def unique_callers(self) -> CallableSpec: + if self.allowed_callers is not None and len(set(self.allowed_callers)) != len(self.allowed_callers): + raise ValueError("allowedCallers must be unique") + return self + + class WorkflowSpec(ContractModel): input_schema: JsonObjectData = Field(alias="inputSchema") output_schema: JsonObjectData = Field(alias="outputSchema") @@ -352,6 +411,9 @@ class WorkflowSpec(ContractModel): ) steps: list[Step] output: Expression + callable: OmissionOnly[CallableSpec] = Field( + default=None, exclude_if=_is_absent, json_schema_extra=_omit_absent_default + ) class ConnectorAction(ContractModel): @@ -427,6 +489,8 @@ class DecisionTableDefinition(ContractModel): SwitchCase, SwitchStep, ParallelStep, + ForEachStep, + CallWorkflowStep, ): _model.model_rebuild() diff --git a/src/firefly_weave/contracts/instance_keys.py b/src/firefly_weave/contracts/instance_keys.py new file mode 100644 index 00000000..4fdf4d27 --- /dev/null +++ b/src/firefly_weave/contracts/instance_keys.py @@ -0,0 +1,169 @@ +# Copyright 2026 Firefly Software Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# Author: Firefly Software Foundation +# SPDX-License-Identifier: Apache-2.0 + +"""Instance keys: the runtime identity of one activation of a workflow node. + +Grammar (frozen in language milestone M0; overview contract C11):: + + instance-key = node-id *("[" index "]") ["#" seg *("." seg)] ["~" count] + index = "0" / (%x31-39 *DIGIT) ; loop iteration, outer loop first + count = %x31-39 *DIGIT ; loop yield count, from 1 + seg = 1*(ALPHA / DIGIT) ; repeated activation (agent turn, tool call, review) + +A node ID is an optional leading ``@``, then an ASCII letter or digit, then ASCII letters, digits, ``_``, ``.``, +``:``, ``@`` or ``-``. Author IDs are ``ResourceName`` values (``[A-Za-z0-9][A-Za-z0-9_.-]*``), and the synthetic +``@run``, ``@start``, ``@end``, ``@legacy``, ``@join:X`` and ``@branch:X:N`` fit the same set. Node IDs therefore +never contain ``[``, ``]``, ``#``, ``~``, whitespace, control characters or non-ASCII characters, so parsing is +unambiguous. Indexes and counts are at most 2**53 - 1 so that API views can carry them as JSON integers. + +``INSTANCE_KEY_PATTERN`` is this grammar, bound included, as unanchored regular-expression source. In Python, +match it with ``re.fullmatch``; ``split_instance`` does. Do not wrap it in ``^(?:...)$`` for Python ``re``, whose +``$`` also matches before a trailing newline, so a key such as ``send[1]`` plus a newline would pass. The +anchored form is for ECMAScript (with the ``u`` flag), JSON Schema ``pattern`` and pydantic's default Rust +regular-expression engine, whose ``$`` matches only at the end of the text. API models declare key fields as +``InstanceKeyText``, or ``InstanceKeyTextOrEmpty`` where ``""`` stands for the step itself; both rely on that +Rust engine. +""" + +from __future__ import annotations + +import re +from dataclasses import dataclass +from typing import Annotated + +from pydantic import Field + +from firefly_weave.contracts.values import MAX_SAFE_INTEGER + + +def _positive_at_most(limit: int) -> str: + """Regular-expression source for the decimals 1 to ``limit`` without sign or leading zeros.""" + digits = str(limit) + branches = [f"[1-9][0-9]{{0,{len(digits) - 2}}}"] if len(digits) > 1 else [] + for position, digit in enumerate(digits): + lowest = 1 if position == 0 else 0 + if int(digit) > lowest: + below = str(lowest) if int(digit) - 1 == lowest else f"[{lowest}-{int(digit) - 1}]" + rest = len(digits) - position - 1 + branches.append(digits[:position] + below + (f"[0-9]{{{rest}}}" if rest > 1 else "[0-9]" * rest)) + branches.append(digits) + return "|".join(branches) + + +_NODE_ID = r"@?[A-Za-z0-9][A-Za-z0-9_.:@-]*" +_INDEX = "0|" + _positive_at_most(MAX_SAFE_INTEGER) +_COUNT = _positive_at_most(MAX_SAFE_INTEGER) +_SEGMENT = r"[A-Za-z0-9]+" + +# The whole grammar without anchors. Python: re.fullmatch it (a "^(?:...)$" wrapper would accept a trailing newline, +# because Python's "$" matches before one). ECMAScript (u flag), JSON Schema and pydantic's default Rust engine, which +# the API types below use: wrap it in ^(?:...)$. +INSTANCE_KEY_PATTERN: str = rf"{_NODE_ID}(?:\[(?:{_INDEX})\])*(?:#{_SEGMENT}(?:\.{_SEGMENT})*)?(?:~(?:{_COUNT}))?" + +InstanceKeyText = Annotated[str, Field(max_length=512, pattern=rf"^(?:{INSTANCE_KEY_PATTERN})$")] +InstanceKeyTextOrEmpty = Annotated[str, Field(max_length=512, pattern=rf"^(?:{INSTANCE_KEY_PATTERN})?$")] + +_SEPARATORS = re.compile(r"[\[#~]") +_KEY = re.compile(INSTANCE_KEY_PATTERN) +_NODE = re.compile(_NODE_ID) +_SEGMENT_TEXT = re.compile(_SEGMENT) + + +class InvalidInstanceKey(ValueError): + """The text or parts do not form one canonical instance key.""" + + +@dataclass(frozen=True) +class InstanceKey: + """A parsed instance key; constructing one validates every part.""" + + node_id: str + indexes: tuple[int, ...] = () + segments: tuple[str, ...] = () + count: int | None = None + + def __post_init__(self) -> None: + if type(self.node_id) is not str or _NODE.fullmatch(self.node_id) is None: + raise InvalidInstanceKey( + "Node IDs start with an optional '@' and an ASCII letter or digit, then contain only ASCII " + "letters, digits, '_', '.', ':', '@' or '-'" + ) + if type(self.indexes) is not tuple or any( + type(index) is not int or not 0 <= index <= MAX_SAFE_INTEGER for index in self.indexes + ): + raise InvalidInstanceKey("Loop indexes are integers from 0 to 2**53 - 1") + if type(self.segments) is not tuple or any( + type(segment) is not str or _SEGMENT_TEXT.fullmatch(segment) is None for segment in self.segments + ): + raise InvalidInstanceKey("Activation segments contain only ASCII letters and digits") + if self.count is not None and (type(self.count) is not int or not 1 <= self.count <= MAX_SAFE_INTEGER): + raise InvalidInstanceKey("Yield counts are integers from 1 to 2**53 - 1") + + +@dataclass(frozen=True) +class InstanceView: + """The C11 API view of a key: static step ID, full key ('' when it equals the step ID), loop indexes.""" + + node_id: str + instance_key: str + iteration: list[int] + + +def node_of(key: str) -> str: + """The static node ID: the key cut at its first '[', '#' or '~'. Cheap; does not validate the rest.""" + if type(key) is not str: + raise InvalidInstanceKey("Instance keys are strings") + match = _SEPARATORS.search(key) + node = key if match is None else key[: match.start()] + if not node: + raise InvalidInstanceKey("Instance keys start with a node ID") + return node + + +def split_instance(key: str) -> InstanceKey: + """Parse a whole key; non-canonical keys (such as 'send[03]') are rejected.""" + if type(key) is not str: + raise InvalidInstanceKey("Instance keys are strings") + if _KEY.fullmatch(key) is None: + raise InvalidInstanceKey("Text is not a canonical instance key") + # The pattern proved the shape and the bounds, so splitting at the separators recovers the parts. + rest, _, count = key.partition("~") + rest, _, segments = rest.partition("#") + node, bracket, indexes = rest.partition("[") + return InstanceKey( + node, + tuple(int(index) for index in indexes[:-1].split("][")) if bracket else (), + tuple(segments.split(".")) if segments else (), + int(count) if count else None, + ) + + +def format_instance(key: InstanceKey) -> str: + """The canonical text of a parsed key; the inverse of split_instance.""" + if not isinstance(key, InstanceKey): + raise InvalidInstanceKey("Format an InstanceKey") + text = key.node_id + "".join(f"[{index}]" for index in key.indexes) + if key.segments: + text += "#" + ".".join(key.segments) + if key.count is not None: + text += f"~{key.count}" + return text + + +def instance_view(key: str) -> InstanceView: + """API view fields for a stored key, after checking that it is canonical.""" + parsed = split_instance(key) + return InstanceView(parsed.node_id, "" if key == parsed.node_id else key, list(parsed.indexes)) diff --git a/src/firefly_weave/contracts/language.py b/src/firefly_weave/contracts/language.py new file mode 100644 index 00000000..ca5e70dd --- /dev/null +++ b/src/firefly_weave/contracts/language.py @@ -0,0 +1,295 @@ +# Copyright 2026 Firefly Software Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# Author: Firefly Software Foundation +# SPDX-License-Identifier: Apache-2.0 + +"""The language manifest: the single source of step kinds, operators, workflow fields, features and limits. + +Served by ``language.read`` and the Studio host's ``GET /studio/contracts/language`` and exported as +``language-manifest`` (language spec 14.1, contract C5). ``studio`` is ``ready`` once Studio ships a +descriptor for the entry and ``pending`` until then; Studio's schema coverage test compares only ready entries. +Every change that adds a step kind, operator or workflow field to the definition models adds its entry here. +""" + +from collections.abc import Sequence +from typing import Final, Literal + +from pydantic import Field, model_validator + +from firefly_weave.compiler.ir import COMPARISON_IR_VERSION, HUMAN_IR_VERSION, IR_VERSION +from firefly_weave.contracts.definitions import ( + ContractModel, + OmissionOnly, + PositiveInt, + _is_absent, + _omit_absent_default, +) +from firefly_weave.contracts.language_features import ( + ADVERTISED_FEATURES, + DEFAULT_LOOP_MAX_ITEMS, + KIND_FEATURES, + MAX_CALL_DEPTH, + MAX_LOOP_DEPTH, + MAX_LOOP_ITEMS, + MAX_RUN_ITERATIONS, + OPERATOR_FEATURES, + WORKFLOW_FIELD_FEATURES, + LanguageFeature, +) +from firefly_weave.contracts.values import JsonObject, JsonObjectData + +type JsonType = Literal["array", "boolean", "integer", "null", "number", "object", "string"] +type StudioMark = Literal["ready", "pending"] +type KindGroup = Literal["actions", "ai", "data", "flow", "wait", "human"] + +MANIFEST_VERSION: Final = "weave/language-manifest-v1" +IR_VERSIONS: Final[tuple[str, ...]] = (IR_VERSION, HUMAN_IR_VERSION, COMPARISON_IR_VERSION) +# The compiler's default parallel concurrency ceiling (compile_source's max_parallel_concurrency). +MAX_PARALLEL_CONCURRENCY: Final = 1000 + + +class Arity(ContractModel): + min: PositiveInt + max: OmissionOnly[PositiveInt] = Field(default=None, exclude_if=_is_absent, json_schema_extra=_omit_absent_default) + + +class ManifestOperator(ContractModel): + name: str + feature: OmissionOnly[LanguageFeature] = Field( + default=None, exclude_if=_is_absent, json_schema_extra=_omit_absent_default + ) + arity: Arity + # One list of accepted JSON types per operand position; with repeat_last the last list repeats. + operand_types: list[list[JsonType]] = Field(min_length=1) + repeat_last: bool = Field(default=False, exclude_if=lambda value: not value) + # Accepted item types for operators that take a list. + item_types: OmissionOnly[list[JsonType]] = Field( + default=None, exclude_if=_is_absent, json_schema_extra=_omit_absent_default + ) + result: JsonObjectData + label: str = Field(min_length=1) + studio: StudioMark + + +class ManifestWorkflowField(ContractModel): + name: str + feature: OmissionOnly[LanguageFeature] = Field( + default=None, exclude_if=_is_absent, json_schema_extra=_omit_absent_default + ) + studio: StudioMark + + +class ManifestBlock(ContractModel): + """A nested Branch at a fixed path: its steps are at ``path + "/steps"`` and its output at ``path + "/output"``.""" + + name: str + path: str = Field(pattern=r"^(?:/[A-Za-z]+)+$") + label: str = Field(min_length=1) + + +class ManifestStepKind(ContractModel): + kind: str + feature: OmissionOnly[LanguageFeature] = Field( + default=None, exclude_if=_is_absent, json_schema_extra=_omit_absent_default + ) + schema_ref: str = Field(pattern=r"^workflow\.schema\.json#/\$defs/[A-Za-z]+$") + group: KindGroup + label: str = Field(min_length=1) + blocks: list[ManifestBlock] + scope_roots: list[str] + studio: StudioMark + + +class ManifestLimits(ContractModel): + max_loop_items: PositiveInt + default_loop_max_items: PositiveInt + max_loop_depth: PositiveInt + max_concurrency: PositiveInt + max_run_iterations: PositiveInt + max_call_depth: PositiveInt + + +class LanguageManifest(ContractModel): + version: Literal["weave/language-manifest-v1"] + language_version: Literal["weave/v1alpha1"] + ir_versions: list[str] = Field(min_length=1) + # The features the serving target runs (Capabilities.language_features), sorted and unique. + features: list[LanguageFeature] + limits: ManifestLimits + operators: list[ManifestOperator] + workflow_fields: list[ManifestWorkflowField] + step_kinds: list[ManifestStepKind] + + @model_validator(mode="after") + def unique_entries(self) -> "LanguageManifest": + for names in ( + [entry.name for entry in self.operators], + [entry.name for entry in self.workflow_fields], + [entry.kind for entry in self.step_kinds], + ): + if len(set(names)) != len(names): + raise ValueError("Manifest entries must be unique") + if self.features != sorted(set(self.features)): + raise ValueError("Manifest features must be sorted and unique") + return self + + +_ANY: Final[list[JsonType]] = ["array", "boolean", "integer", "null", "number", "object", "string"] +_ORDERED: Final[list[JsonType]] = ["integer", "number", "string"] +_TEXT_PARTS: Final[list[JsonType]] = ["string", "integer", "number", "boolean"] +_BOOLEAN: Final[JsonObject] = {"type": "boolean"} +_STRING: Final[JsonObject] = {"type": "string"} + + +def _operator( + name: str, + arity: tuple[int, int | None], + operand_types: list[list[JsonType]], + result: JsonObject, + label: str, + *, + repeat_last: bool = False, + item_types: list[JsonType] | None = None, + studio: StudioMark = "ready", +) -> ManifestOperator: + value: JsonObject = { + "name": name, + "arity": {"min": arity[0]} if arity[1] is None else {"min": arity[0], "max": arity[1]}, + "operand_types": [list(types) for types in operand_types], + "repeat_last": repeat_last, + "result": result, + "label": label, + "studio": studio, + } + if name in OPERATOR_FEATURES: + value["feature"] = OPERATOR_FEATURES[name] + if item_types is not None: + value["item_types"] = list(item_types) + return ManifestOperator.model_validate(value) + + +OPERATORS: Final[tuple[ManifestOperator, ...]] = ( + _operator("eq", (2, 2), [_ANY, _ANY], _BOOLEAN, "Is"), + _operator("ne", (2, 2), [_ANY, _ANY], _BOOLEAN, "Is not"), + _operator("lt", (2, 2), [_ORDERED, _ORDERED], _BOOLEAN, "Is less than"), + _operator("lte", (2, 2), [_ORDERED, _ORDERED], _BOOLEAN, "Is at most"), + _operator("gt", (2, 2), [_ORDERED, _ORDERED], _BOOLEAN, "Is greater than"), + _operator("gte", (2, 2), [_ORDERED, _ORDERED], _BOOLEAN, "Is at least"), + _operator("and", (1, None), [["boolean"]], _BOOLEAN, "All of", repeat_last=True), + _operator("or", (1, None), [["boolean"]], _BOOLEAN, "Any of", repeat_last=True), + _operator("not", (1, 1), [["boolean"]], _BOOLEAN, "Not"), + _operator("exists", (1, 1), [_ANY], _BOOLEAN, "Is present"), + _operator("coalesce", (1, None), [_ANY], {}, "First available of", repeat_last=True), + _operator("contains", (2, 2), [["array", "string"], _ANY], _BOOLEAN, "Contains"), + _operator("notContains", (2, 2), [["array", "string"], _ANY], _BOOLEAN, "Does not contain"), + _operator("in", (2, 2), [_ANY, ["array"]], _BOOLEAN, "Is in list"), + _operator("notIn", (2, 2), [_ANY, ["array"]], _BOOLEAN, "Is not in list"), + _operator("startsWith", (2, 2), [["string"], ["string"]], _BOOLEAN, "Starts with"), + _operator("endsWith", (2, 2), [["string"], ["string"]], _BOOLEAN, "Ends with"), + _operator("concat", (1, None), [_TEXT_PARTS], _STRING, "Combine text", repeat_last=True, studio="pending"), + _operator("join", (2, 2), [["array"], ["string"]], _STRING, "Join list", item_types=_TEXT_PARTS, studio="pending"), +) + +_WORKFLOW_FIELD_MARKS: Final[tuple[tuple[str, StudioMark], ...]] = ( + ("inputSchema", "ready"), + ("outputSchema", "ready"), + ("connections", "ready"), + ("timeoutSeconds", "ready"), + ("llmProfiles", "ready"), + ("steps", "ready"), + ("output", "ready"), + ("callable", "pending"), +) +WORKFLOW_FIELDS: Final[tuple[ManifestWorkflowField, ...]] = tuple( + ManifestWorkflowField.model_validate( + {"name": name, "studio": studio} + | ({"feature": WORKFLOW_FIELD_FEATURES[name]} if name in WORKFLOW_FIELD_FEATURES else {}) + ) + for name, studio in _WORKFLOW_FIELD_MARKS +) + + +def _kind( + kind: str, + model: str, + group: KindGroup, + label: str, + *, + blocks: Sequence[tuple[str, str, str]] = (), + scope_roots: Sequence[str] = (), + studio: StudioMark = "ready", +) -> ManifestStepKind: + value: JsonObject = { + "kind": kind, + "schema_ref": f"workflow.schema.json#/$defs/{model}", + "group": group, + "label": label, + "blocks": [{"name": name, "path": path, "label": text} for name, path, text in blocks], + "scope_roots": list(scope_roots), + "studio": studio, + } + if kind in KIND_FEATURES: + value["feature"] = KIND_FEATURES[kind] + return ManifestStepKind.model_validate(value) + + +# Lane A adds the agent entry with AgentStep: kind "agent", group "ai", label "AI agent", studio "pending". +STEP_KINDS: Final[tuple[ManifestStepKind, ...]] = ( + _kind("action", "ActionStep", "actions", "Action"), + _kind("llm", "LLMStep", "ai", "AI task"), + _kind("transform", "TransformStep", "data", "Transform"), + _kind("decisionTable", "DecisionTableStep", "data", "Decision table"), + _kind("switch", "SwitchStep", "flow", "Decision", blocks=[("default", "/default", "Otherwise")]), + _kind("parallel", "ParallelStep", "flow", "Parallel"), + _kind( + "forEach", + "ForEachStep", + "flow", + "Loop over items", + blocks=[("body", "/body", "For each item")], + scope_roots=["/item", "/index", "/loops"], + studio="pending", + ), + _kind("callWorkflow", "CallWorkflowStep", "flow", "Call a workflow", studio="pending"), + _kind("wait", "WaitStep", "wait", "Wait for time"), + _kind("signal", "SignalStep", "wait", "Wait for signal"), + _kind("humanTask", "HumanTaskStep", "human", "Human task"), + _kind("fail", "FailStep", "flow", "Stop with error"), +) + + +def language_manifest(features: Sequence[LanguageFeature] = ADVERTISED_FEATURES) -> LanguageManifest: + """The manifest of a target that runs ``features``; limits are this platform's language ceilings.""" + return LanguageManifest( + version=MANIFEST_VERSION, + language_version="weave/v1alpha1", + ir_versions=list(IR_VERSIONS), + features=sorted(set(features)), + limits=ManifestLimits( + max_loop_items=MAX_LOOP_ITEMS, + default_loop_max_items=DEFAULT_LOOP_MAX_ITEMS, + max_loop_depth=MAX_LOOP_DEPTH, + max_concurrency=MAX_PARALLEL_CONCURRENCY, + max_run_iterations=MAX_RUN_ITERATIONS, + max_call_depth=MAX_CALL_DEPTH, + ), + operators=list(OPERATORS), + workflow_fields=list(WORKFLOW_FIELDS), + step_kinds=list(STEP_KINDS), + ) + + +def supported_step_kinds(features: Sequence[LanguageFeature] = ADVERTISED_FEATURES) -> list[str]: + """The kinds a target running ``features`` executes: every kind without a feature, plus the listed ones.""" + return [entry.kind for entry in STEP_KINDS if entry.feature is None or entry.feature in features] diff --git a/src/firefly_weave/contracts/language_features.py b/src/firefly_weave/contracts/language_features.py new file mode 100644 index 00000000..a98d0f01 --- /dev/null +++ b/src/firefly_weave/contracts/language_features.py @@ -0,0 +1,48 @@ +# Copyright 2026 Firefly Software Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# Author: Firefly Software Foundation +# SPDX-License-Identifier: Apache-2.0 + +"""Language feature names and the definition constructs that need them (language spec 3.1, contract C5). + +A workflow IR lists the features it uses in ``features``; a platform runs it only when it advertises each one. +""" + +from collections.abc import Mapping +from types import MappingProxyType +from typing import Final, Literal, get_args + +type LanguageFeature = Literal["ai.agent", "ai.memory", "flow.callWorkflow", "flow.forEach", "text.concat", "text.join"] + +# Every feature the program defines, sorted. The values are frozen; adding one is a contract change. +LANGUAGE_FEATURES: Final[tuple[LanguageFeature, ...]] = tuple(sorted(get_args(LanguageFeature.__value__))) + +# Features whose runtime milestone has merged. Capabilities.language_features and the manifest serve these. +ADVERTISED_FEATURES: Final[tuple[LanguageFeature, ...]] = () + +# Step kinds, operators and workflow fields that need a feature. Lane A adds "agent": "ai.agent" with AgentStep. +KIND_FEATURES: Final[Mapping[str, LanguageFeature]] = MappingProxyType( + {"forEach": "flow.forEach", "callWorkflow": "flow.callWorkflow"} +) +OPERATOR_FEATURES: Final[Mapping[str, LanguageFeature]] = MappingProxyType( + {"concat": "text.concat", "join": "text.join"} +) +WORKFLOW_FIELD_FEATURES: Final[Mapping[str, LanguageFeature]] = MappingProxyType({"callable": "flow.callWorkflow"}) + +# Language limits the manifest publishes (language spec 5.3 and 6.4); the loop and call milestones enforce them. +DEFAULT_LOOP_MAX_ITEMS: Final = 1000 +MAX_LOOP_ITEMS: Final = 10_000 +MAX_LOOP_DEPTH: Final = 3 +MAX_RUN_ITERATIONS: Final = 100_000 +MAX_CALL_DEPTH: Final = 8 diff --git a/src/firefly_weave/contracts/public.py b/src/firefly_weave/contracts/public.py index e147e7fa..68f8ae43 100644 --- a/src/firefly_weave/contracts/public.py +++ b/src/firefly_weave/contracts/public.py @@ -28,6 +28,7 @@ from firefly_weave.contracts.catalog import Activation, Draft, PublishedVersion from firefly_weave.contracts.definitions import ContractModel from firefly_weave.contracts.diagnostics import Diagnostic +from firefly_weave.contracts.language import IR_VERSIONS, supported_step_kinds from firefly_weave.contracts.limits import Limits from firefly_weave.contracts.operational_policy import OperationsPolicy from firefly_weave.contracts.values import JsonData, JsonObjectData @@ -261,12 +262,9 @@ def policy_identity(self) -> "OperationalCapabilities": class Capabilities(ContractModel): wire_version: Literal["weave/api-v1"] = "weave/api-v1" language_versions: list[str] = Field(default_factory=lambda: ["weave/v1alpha1"]) - ir_versions: list[str] = Field( - default_factory=lambda: ["weave/ir-v1alpha1", "weave/ir-v1alpha2", "weave/ir-v1alpha3"] - ) - step_kinds: list[str] = Field( - default_factory=lambda: ["action", "decisionTable", "transform", "switch", "parallel", "wait", "signal", "fail"] - ) + ir_versions: list[str] = Field(default_factory=lambda: list(IR_VERSIONS)) + # Every kind this platform runs, from the language manifest (kinds whose feature it advertises included). + step_kinds: list[str] = Field(default_factory=supported_step_kinds) limits: dict[str, int] schemas: list[str] connectors: list[str] diff --git a/src/firefly_weave/contracts/schema_export.py b/src/firefly_weave/contracts/schema_export.py index 853d3c42..b345d294 100644 --- a/src/firefly_weave/contracts/schema_export.py +++ b/src/firefly_weave/contracts/schema_export.py @@ -152,6 +152,7 @@ def contract_models() -> dict[str, SchemaModel]: TaskGroupRequest, ) from firefly_weave.contracts.identity import IdentityView + from firefly_weave.contracts.language import LanguageManifest from firefly_weave.contracts.lumi import ( LumiAskRequest, LumiConfiguration, @@ -322,6 +323,7 @@ def contract_models() -> dict[str, SchemaModel]: "decision-evaluation": DecisionEvaluation, "problem": Problem, "capabilities": Capabilities, + "language-manifest": LanguageManifest, "draft-view": DraftView, "draft-retirement": DraftRetirement, "draft-export": DraftExport, diff --git a/src/firefly_weave/contracts/surface.py b/src/firefly_weave/contracts/surface.py index 73f0f8cf..e8c1744d 100644 --- a/src/firefly_weave/contracts/surface.py +++ b/src/firefly_weave/contracts/surface.py @@ -94,6 +94,7 @@ ) from firefly_weave.contracts.identity import IdentityView from firefly_weave.contracts.integration_events import DeliveryAttempt, DeliveryView, Subscription, SubscriptionRequest +from firefly_weave.contracts.language import LanguageManifest from firefly_weave.contracts.lumi import ( LumiAskRequest, LumiConfiguration, @@ -1181,6 +1182,7 @@ def native(self) -> OpenAPIOperation: ), Operation("catalog.read", PROJECT + "/catalog", "GET", CatalogLock, "catalog.read"), Operation("capabilities.read", PROJECT + "/capabilities", "GET", Capabilities, "catalog.read"), + Operation("language.read", PROJECT + "/language", "GET", LanguageManifest, "catalog.read"), Operation("schemas.read", PROJECT + "/schemas", "GET", dict[str, JsonObjectData], "catalog.read"), # Static catalog resources precede the {collection} matcher for first-match consumers. Operation( diff --git a/src/firefly_weave/sdk/client.py b/src/firefly_weave/sdk/client.py index 2b023707..ffe2f4b7 100644 --- a/src/firefly_weave/sdk/client.py +++ b/src/firefly_weave/sdk/client.py @@ -57,6 +57,7 @@ TaskGroupRequest, ) from firefly_weave.contracts.integration_events import DeliveryAttempt, DeliveryView, Subscription, SubscriptionRequest +from firefly_weave.contracts.language import LanguageManifest from firefly_weave.contracts.lumi import ( LumiAskRequest, LumiConfiguration, @@ -376,6 +377,10 @@ async def evaluate_decision(self, request: DecisionEvaluationRequest) -> Decisio async def capabilities(self) -> Capabilities: return cast(Capabilities, await self.invoke("capabilities.read")) + async def language(self) -> LanguageManifest: + """The platform's language manifest: step kinds, operators, workflow fields, features and limits.""" + return cast(LanguageManifest, await self.invoke("language.read")) + async def schemas(self) -> dict[str, JsonObject]: return cast(dict[str, JsonObject], await self.invoke("schemas.read")) diff --git a/src/firefly_weave/studio/host.py b/src/firefly_weave/studio/host.py index e1f32c52..905a7199 100644 --- a/src/firefly_weave/studio/host.py +++ b/src/firefly_weave/studio/host.py @@ -407,6 +407,12 @@ async def lumi_configuration_contract(self, request: Request) -> Response: LumiConfigurationRequest.model_json_schema(by_alias=True), headers={"Cache-Control": "no-store"} ) + @get_mapping("/studio/contracts/language") + async def language_contract(self, request: Request) -> Response: + from firefly_weave.contracts.language import language_manifest + + return JSONResponse(language_manifest().model_dump(mode="json"), headers={"Cache-Control": "no-store"}) + @post_mapping("/studio/local/validate") async def validate(self, request: Request) -> Response: try: diff --git a/src/firefly_weave/studio/service.py b/src/firefly_weave/studio/service.py index add2dbd2..c78aa4c0 100644 --- a/src/firefly_weave/studio/service.py +++ b/src/firefly_weave/studio/service.py @@ -58,6 +58,7 @@ "compiler", "catalog", "capabilities", + "language", "schemas", "activations", "runs", diff --git a/studio/src/app/designer/diagnostic-copy.ts b/studio/src/app/designer/diagnostic-copy.ts index 9c6aa782..808f7309 100644 --- a/studio/src/app/designer/diagnostic-copy.ts +++ b/studio/src/app/designer/diagnostic-copy.ts @@ -155,6 +155,10 @@ const compiler: Record = { text: "Nothing reads this step's output.", hint: "Use it in a later step or the workflow result, or delete the step.", }, + UNSUPPORTED_FEATURE: { + text: "This version of Weave doesn't support this step or formula yet.", + hint: "Keep the workflow for a later version, or use the steps and formulas this version offers.", + }, }; const expression: Record = { diff --git a/studio/tests/fixtures/language/instance-keys.json b/studio/tests/fixtures/language/instance-keys.json new file mode 100644 index 00000000..1eecc407 --- /dev/null +++ b/studio/tests/fixtures/language/instance-keys.json @@ -0,0 +1,308 @@ +{ + "generator": "scripts/language_fixtures.py", + "grammar": "language spec 3.2", + "pattern": "@?[A-Za-z0-9][A-Za-z0-9_.:@-]*(?:\\[(?:0|[1-9][0-9]{0,14}|[1-8][0-9]{15}|900[0-6][0-9]{12}|90070[0-9]{11}|90071[0-8][0-9]{10}|900719[0-8][0-9]{9}|9007199[0-1][0-9]{8}|90071992[0-4][0-9]{7}|900719925[0-3][0-9]{6}|9007199254[0-6][0-9]{5}|90071992547[0-3][0-9]{4}|9007199254740[0-8][0-9]{2}|90071992547409[0-8][0-9]|9007199254740990|9007199254740991)\\])*(?:#[A-Za-z0-9]+(?:\\.[A-Za-z0-9]+)*)?(?:~(?:[1-9][0-9]{0,14}|[1-8][0-9]{15}|900[0-6][0-9]{12}|90070[0-9]{11}|90071[0-8][0-9]{10}|900719[0-8][0-9]{9}|9007199[0-1][0-9]{8}|90071992[0-4][0-9]{7}|900719925[0-3][0-9]{6}|9007199254[0-6][0-9]{5}|90071992547[0-3][0-9]{4}|9007199254740[0-8][0-9]{2}|90071992547409[0-8][0-9]|9007199254740990|9007199254740991))?", + "valid": [ + { + "key": "send", + "node_id": "send", + "indexes": [], + "segments": [], + "count": null, + "view": { + "node_id": "send", + "instance_key": "", + "iteration": [] + } + }, + { + "key": "send[3]", + "node_id": "send", + "indexes": [ + 3 + ], + "segments": [], + "count": null, + "view": { + "node_id": "send", + "instance_key": "send[3]", + "iteration": [ + 3 + ] + } + }, + { + "key": "send[3][0]", + "node_id": "send", + "indexes": [ + 3, + 0 + ], + "segments": [], + "count": null, + "view": { + "node_id": "send", + "instance_key": "send[3][0]", + "iteration": [ + 3, + 0 + ] + } + }, + { + "key": "support#2", + "node_id": "support", + "indexes": [], + "segments": [ + "2" + ], + "count": null, + "view": { + "node_id": "support", + "instance_key": "support#2", + "iteration": [] + } + }, + { + "key": "support#2.1", + "node_id": "support", + "indexes": [], + "segments": [ + "2", + "1" + ], + "count": null, + "view": { + "node_id": "support", + "instance_key": "support#2.1", + "iteration": [] + } + }, + { + "key": "support#2.1.review", + "node_id": "support", + "indexes": [], + "segments": [ + "2", + "1", + "review" + ], + "count": null, + "view": { + "node_id": "support", + "instance_key": "support#2.1.review", + "iteration": [] + } + }, + { + "key": "support[3]#2.1.review", + "node_id": "support", + "indexes": [ + 3 + ], + "segments": [ + "2", + "1", + "review" + ], + "count": null, + "view": { + "node_id": "support", + "instance_key": "support[3]#2.1.review", + "iteration": [ + 3 + ] + } + }, + { + "key": "notify~2", + "node_id": "notify", + "indexes": [], + "segments": [], + "count": 2, + "view": { + "node_id": "notify", + "instance_key": "notify~2", + "iteration": [] + } + }, + { + "key": "notify[1]~2", + "node_id": "notify", + "indexes": [ + 1 + ], + "segments": [], + "count": 2, + "view": { + "node_id": "notify", + "instance_key": "notify[1]~2", + "iteration": [ + 1 + ] + } + }, + { + "key": "@join:notify", + "node_id": "@join:notify", + "indexes": [], + "segments": [], + "count": null, + "view": { + "node_id": "@join:notify", + "instance_key": "", + "iteration": [] + } + }, + { + "key": "@branch:notify:0[4]", + "node_id": "@branch:notify:0", + "indexes": [ + 4 + ], + "segments": [], + "count": null, + "view": { + "node_id": "@branch:notify:0", + "instance_key": "@branch:notify:0[4]", + "iteration": [ + 4 + ] + } + }, + { + "key": "a.b_c-d[9007199254740991]", + "node_id": "a.b_c-d", + "indexes": [ + 9007199254740991 + ], + "segments": [], + "count": null, + "view": { + "node_id": "a.b_c-d", + "instance_key": "a.b_c-d[9007199254740991]", + "iteration": [ + 9007199254740991 + ] + } + } + ], + "invalid": [ + { + "key": "", + "reason": "empty key" + }, + { + "key": "[3]", + "reason": "no node ID" + }, + { + "key": "#2", + "reason": "no node ID" + }, + { + "key": "~1", + "reason": "no node ID" + }, + { + "key": "send[03]", + "reason": "index with a leading zero" + }, + { + "key": "send[-1]", + "reason": "negative index" + }, + { + "key": "send[]", + "reason": "empty index" + }, + { + "key": "send[3", + "reason": "unclosed index" + }, + { + "key": "send]3[", + "reason": "bracket in the node ID" + }, + { + "key": "send[3]x", + "reason": "text after an index" + }, + { + "key": "send#", + "reason": "empty segment list" + }, + { + "key": "send#2.", + "reason": "empty segment" + }, + { + "key": "send#.2", + "reason": "empty segment" + }, + { + "key": "send#2..1", + "reason": "empty segment" + }, + { + "key": "send#ü", + "reason": "non-ASCII segment" + }, + { + "key": "send#2[3]", + "reason": "index after a segment" + }, + { + "key": "send~0", + "reason": "yield count zero" + }, + { + "key": "send~01", + "reason": "yield count with a leading zero" + }, + { + "key": "send~1[2]", + "reason": "index after the yield count" + }, + { + "key": "send~1#2", + "reason": "segment after the yield count" + }, + { + "key": "send~1~2", + "reason": "two yield counts" + }, + { + "key": "send[9007199254740992]", + "reason": "index above 2**53 - 1" + }, + { + "key": "send~9007199254740992", + "reason": "yield count above 2**53 - 1" + }, + { + "key": "a b", + "reason": "whitespace in the node ID" + }, + { + "key": "send\n", + "reason": "control character in the node ID" + }, + { + "key": "send\u0000", + "reason": "NUL in the node ID" + }, + { + "key": "se/nd", + "reason": "slash in the node ID" + }, + { + "key": "ü[1]", + "reason": "non-ASCII node ID" + }, + { + "key": "@", + "reason": "synthetic marker without a name" + } + ] +} diff --git a/studio/tests/fixtures/language/manifest.json b/studio/tests/fixtures/language/manifest.json new file mode 100644 index 00000000..b8c6bd55 --- /dev/null +++ b/studio/tests/fixtures/language/manifest.json @@ -0,0 +1,635 @@ +{ + "generator": "scripts/language_fixtures.py", + "manifest": { + "version": "weave/language-manifest-v1", + "language_version": "weave/v1alpha1", + "ir_versions": [ + "weave/ir-v1alpha1", + "weave/ir-v1alpha2", + "weave/ir-v1alpha3" + ], + "features": [], + "limits": { + "max_loop_items": 10000, + "default_loop_max_items": 1000, + "max_loop_depth": 3, + "max_concurrency": 1000, + "max_run_iterations": 100000, + "max_call_depth": 8 + }, + "operators": [ + { + "name": "eq", + "arity": { + "min": 2, + "max": 2 + }, + "operand_types": [ + [ + "array", + "boolean", + "integer", + "null", + "number", + "object", + "string" + ], + [ + "array", + "boolean", + "integer", + "null", + "number", + "object", + "string" + ] + ], + "result": { + "type": "boolean" + }, + "label": "Is", + "studio": "ready" + }, + { + "name": "ne", + "arity": { + "min": 2, + "max": 2 + }, + "operand_types": [ + [ + "array", + "boolean", + "integer", + "null", + "number", + "object", + "string" + ], + [ + "array", + "boolean", + "integer", + "null", + "number", + "object", + "string" + ] + ], + "result": { + "type": "boolean" + }, + "label": "Is not", + "studio": "ready" + }, + { + "name": "lt", + "arity": { + "min": 2, + "max": 2 + }, + "operand_types": [ + [ + "integer", + "number", + "string" + ], + [ + "integer", + "number", + "string" + ] + ], + "result": { + "type": "boolean" + }, + "label": "Is less than", + "studio": "ready" + }, + { + "name": "lte", + "arity": { + "min": 2, + "max": 2 + }, + "operand_types": [ + [ + "integer", + "number", + "string" + ], + [ + "integer", + "number", + "string" + ] + ], + "result": { + "type": "boolean" + }, + "label": "Is at most", + "studio": "ready" + }, + { + "name": "gt", + "arity": { + "min": 2, + "max": 2 + }, + "operand_types": [ + [ + "integer", + "number", + "string" + ], + [ + "integer", + "number", + "string" + ] + ], + "result": { + "type": "boolean" + }, + "label": "Is greater than", + "studio": "ready" + }, + { + "name": "gte", + "arity": { + "min": 2, + "max": 2 + }, + "operand_types": [ + [ + "integer", + "number", + "string" + ], + [ + "integer", + "number", + "string" + ] + ], + "result": { + "type": "boolean" + }, + "label": "Is at least", + "studio": "ready" + }, + { + "name": "and", + "arity": { + "min": 1 + }, + "operand_types": [ + [ + "boolean" + ] + ], + "repeat_last": true, + "result": { + "type": "boolean" + }, + "label": "All of", + "studio": "ready" + }, + { + "name": "or", + "arity": { + "min": 1 + }, + "operand_types": [ + [ + "boolean" + ] + ], + "repeat_last": true, + "result": { + "type": "boolean" + }, + "label": "Any of", + "studio": "ready" + }, + { + "name": "not", + "arity": { + "min": 1, + "max": 1 + }, + "operand_types": [ + [ + "boolean" + ] + ], + "result": { + "type": "boolean" + }, + "label": "Not", + "studio": "ready" + }, + { + "name": "exists", + "arity": { + "min": 1, + "max": 1 + }, + "operand_types": [ + [ + "array", + "boolean", + "integer", + "null", + "number", + "object", + "string" + ] + ], + "result": { + "type": "boolean" + }, + "label": "Is present", + "studio": "ready" + }, + { + "name": "coalesce", + "arity": { + "min": 1 + }, + "operand_types": [ + [ + "array", + "boolean", + "integer", + "null", + "number", + "object", + "string" + ] + ], + "repeat_last": true, + "result": {}, + "label": "First available of", + "studio": "ready" + }, + { + "name": "contains", + "arity": { + "min": 2, + "max": 2 + }, + "operand_types": [ + [ + "array", + "string" + ], + [ + "array", + "boolean", + "integer", + "null", + "number", + "object", + "string" + ] + ], + "result": { + "type": "boolean" + }, + "label": "Contains", + "studio": "ready" + }, + { + "name": "notContains", + "arity": { + "min": 2, + "max": 2 + }, + "operand_types": [ + [ + "array", + "string" + ], + [ + "array", + "boolean", + "integer", + "null", + "number", + "object", + "string" + ] + ], + "result": { + "type": "boolean" + }, + "label": "Does not contain", + "studio": "ready" + }, + { + "name": "in", + "arity": { + "min": 2, + "max": 2 + }, + "operand_types": [ + [ + "array", + "boolean", + "integer", + "null", + "number", + "object", + "string" + ], + [ + "array" + ] + ], + "result": { + "type": "boolean" + }, + "label": "Is in list", + "studio": "ready" + }, + { + "name": "notIn", + "arity": { + "min": 2, + "max": 2 + }, + "operand_types": [ + [ + "array", + "boolean", + "integer", + "null", + "number", + "object", + "string" + ], + [ + "array" + ] + ], + "result": { + "type": "boolean" + }, + "label": "Is not in list", + "studio": "ready" + }, + { + "name": "startsWith", + "arity": { + "min": 2, + "max": 2 + }, + "operand_types": [ + [ + "string" + ], + [ + "string" + ] + ], + "result": { + "type": "boolean" + }, + "label": "Starts with", + "studio": "ready" + }, + { + "name": "endsWith", + "arity": { + "min": 2, + "max": 2 + }, + "operand_types": [ + [ + "string" + ], + [ + "string" + ] + ], + "result": { + "type": "boolean" + }, + "label": "Ends with", + "studio": "ready" + }, + { + "name": "concat", + "feature": "text.concat", + "arity": { + "min": 1 + }, + "operand_types": [ + [ + "string", + "integer", + "number", + "boolean" + ] + ], + "repeat_last": true, + "result": { + "type": "string" + }, + "label": "Combine text", + "studio": "pending" + }, + { + "name": "join", + "feature": "text.join", + "arity": { + "min": 2, + "max": 2 + }, + "operand_types": [ + [ + "array" + ], + [ + "string" + ] + ], + "item_types": [ + "string", + "integer", + "number", + "boolean" + ], + "result": { + "type": "string" + }, + "label": "Join list", + "studio": "pending" + } + ], + "workflow_fields": [ + { + "name": "inputSchema", + "studio": "ready" + }, + { + "name": "outputSchema", + "studio": "ready" + }, + { + "name": "connections", + "studio": "ready" + }, + { + "name": "timeoutSeconds", + "studio": "ready" + }, + { + "name": "llmProfiles", + "studio": "ready" + }, + { + "name": "steps", + "studio": "ready" + }, + { + "name": "output", + "studio": "ready" + }, + { + "name": "callable", + "feature": "flow.callWorkflow", + "studio": "pending" + } + ], + "step_kinds": [ + { + "kind": "action", + "schema_ref": "workflow.schema.json#/$defs/ActionStep", + "group": "actions", + "label": "Action", + "blocks": [], + "scope_roots": [], + "studio": "ready" + }, + { + "kind": "llm", + "schema_ref": "workflow.schema.json#/$defs/LLMStep", + "group": "ai", + "label": "AI task", + "blocks": [], + "scope_roots": [], + "studio": "ready" + }, + { + "kind": "transform", + "schema_ref": "workflow.schema.json#/$defs/TransformStep", + "group": "data", + "label": "Transform", + "blocks": [], + "scope_roots": [], + "studio": "ready" + }, + { + "kind": "decisionTable", + "schema_ref": "workflow.schema.json#/$defs/DecisionTableStep", + "group": "data", + "label": "Decision table", + "blocks": [], + "scope_roots": [], + "studio": "ready" + }, + { + "kind": "switch", + "schema_ref": "workflow.schema.json#/$defs/SwitchStep", + "group": "flow", + "label": "Decision", + "blocks": [ + { + "name": "default", + "path": "/default", + "label": "Otherwise" + } + ], + "scope_roots": [], + "studio": "ready" + }, + { + "kind": "parallel", + "schema_ref": "workflow.schema.json#/$defs/ParallelStep", + "group": "flow", + "label": "Parallel", + "blocks": [], + "scope_roots": [], + "studio": "ready" + }, + { + "kind": "forEach", + "feature": "flow.forEach", + "schema_ref": "workflow.schema.json#/$defs/ForEachStep", + "group": "flow", + "label": "Loop over items", + "blocks": [ + { + "name": "body", + "path": "/body", + "label": "For each item" + } + ], + "scope_roots": [ + "/item", + "/index", + "/loops" + ], + "studio": "pending" + }, + { + "kind": "callWorkflow", + "feature": "flow.callWorkflow", + "schema_ref": "workflow.schema.json#/$defs/CallWorkflowStep", + "group": "flow", + "label": "Call a workflow", + "blocks": [], + "scope_roots": [], + "studio": "pending" + }, + { + "kind": "wait", + "schema_ref": "workflow.schema.json#/$defs/WaitStep", + "group": "wait", + "label": "Wait for time", + "blocks": [], + "scope_roots": [], + "studio": "ready" + }, + { + "kind": "signal", + "schema_ref": "workflow.schema.json#/$defs/SignalStep", + "group": "wait", + "label": "Wait for signal", + "blocks": [], + "scope_roots": [], + "studio": "ready" + }, + { + "kind": "humanTask", + "schema_ref": "workflow.schema.json#/$defs/HumanTaskStep", + "group": "human", + "label": "Human task", + "blocks": [], + "scope_roots": [], + "studio": "ready" + }, + { + "kind": "fail", + "schema_ref": "workflow.schema.json#/$defs/FailStep", + "group": "flow", + "label": "Stop with error", + "blocks": [], + "scope_roots": [], + "studio": "ready" + } + ] + } +} diff --git a/studio/tests/fixtures/language/text-conversion.json b/studio/tests/fixtures/language/text-conversion.json new file mode 100644 index 00000000..f4c9bdd3 --- /dev/null +++ b/studio/tests/fixtures/language/text-conversion.json @@ -0,0 +1,94 @@ +{ + "generator": "scripts/language_fixtures.py", + "rule": "language spec 4.3", + "cases": [ + { + "value": "Invoice ", + "text": "Invoice " + }, + { + "value": "", + "text": "" + }, + { + "value": "ü😀", + "text": "ü😀" + }, + { + "value": true, + "text": "true" + }, + { + "value": false, + "text": "false" + }, + { + "value": 0, + "text": "0" + }, + { + "value": 7, + "text": "7" + }, + { + "value": -12, + "text": "-12" + }, + { + "value": 9007199254740991, + "text": "9007199254740991" + }, + { + "value": 2.0, + "text": "2" + }, + { + "value": -0.0, + "text": "0" + }, + { + "value": 0.1, + "text": "0.1" + }, + { + "value": 1.5, + "text": "1.5" + }, + { + "value": -2.5, + "text": "-2.5" + }, + { + "value": 100.0, + "text": "100" + }, + { + "value": 1e+20, + "text": "100000000000000000000" + }, + { + "value": 1e+21, + "text": "1e+21" + }, + { + "value": 1e-07, + "text": "1e-7" + }, + { + "value": 1e-06, + "text": "0.000001" + }, + { + "value": 1.2345678901234568e+20, + "text": "123456789012345680000" + }, + { + "value": 5e-324, + "text": "5e-324" + }, + { + "value": 1.7976931348623157e+308, + "text": "1.7976931348623157e+308" + } + ] +} diff --git a/studio/tests/language-fixtures.test.ts b/studio/tests/language-fixtures.test.ts new file mode 100644 index 00000000..3da8c49b --- /dev/null +++ b/studio/tests/language-fixtures.test.ts @@ -0,0 +1,65 @@ +/* +Copyright 2026 Firefly Software Foundation. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +Author: Firefly Software Foundation +SPDX-License-Identifier: Apache-2.0 +*/ +// Lane L's shared language fixtures (language spec 14.5), generated by +// scripts/language_fixtures.py. The platform formats numbers for concat and +// join exactly as JavaScript's String(x), so Studio previews match it. +import { readFileSync } from "node:fs"; +import { resolve } from "node:path"; +import { describe, expect, it } from "vitest"; + +const fixture = (name: string) => + JSON.parse( + readFileSync( + resolve(import.meta.dirname, "fixtures/language", name), + "utf8", + ), + ); + +describe("shared language fixtures", () => { + it("formats every concat and join value as JavaScript does", () => { + const { cases } = fixture("text-conversion.json") as { + cases: { value: string | number | boolean; text: string }[]; + }; + expect(cases.length).toBeGreaterThan(10); + for (const { value, text } of cases) expect(String(value), text).toBe(text); + }); + it("accepts exactly the valid instance keys with the published pattern", () => { + const { pattern, valid, invalid } = fixture("instance-keys.json") as { + pattern: string; + valid: { key: string; node_id: string }[]; + invalid: { key: string }[]; + }; + const instanceKey = new RegExp(`^(?:${pattern})$`, "u"); + for (const { key, node_id } of valid) { + expect(instanceKey.test(key), key).toBe(true); + expect(key.startsWith(node_id), key).toBe(true); + expect(node_id, key).not.toMatch(/[[\]#~]/); + } + for (const { key } of invalid) + expect(instanceKey.test(key), key).toBe(false); + expect(invalid.length).toBeGreaterThan(10); + }); + it("publishes the manifest with Studio marks on every entry", () => { + const { manifest } = fixture("manifest.json") as { + manifest: Record; + }; + for (const group of ["step_kinds", "operators", "workflow_fields"]) + for (const entry of manifest[group]) + expect(["ready", "pending"]).toContain(entry.studio); + }); +}); diff --git a/studio/tests/schema-coverage.test.ts b/studio/tests/schema-coverage.test.ts index a6e46749..4daef497 100644 --- a/studio/tests/schema-coverage.test.ts +++ b/studio/tests/schema-coverage.test.ts @@ -30,6 +30,20 @@ const python = resolve( ? ".venv/Scripts/python.exe" : ".venv/bin/python", ); + +/** The language manifest's entries (language spec 14.1); Studio compares only `ready` ones. */ +interface Marked { + studio: "ready" | "pending"; + feature?: string; +} +interface Manifest { + step_kinds: (Marked & { kind: string })[]; + operators: (Marked & { name: string })[]; + workflow_fields: (Marked & { name: string })[]; +} +const ready = (entries: T[]) => + entries.filter((entry) => entry.studio === "ready"); + // CI installs the repository environment. Frontend-only contributors can still // run UI tests; this cross-language contract check explicitly reports a skip. const available = existsSync(python); @@ -43,9 +57,8 @@ const schema = available import json from typing import get_args from firefly_weave.contracts import definitions as d -models = [d.ActionStep, d.TransformStep, d.SwitchStep, d.ParallelStep, - d.WaitStep, d.SignalStep, d.HumanTaskStep, d.FailStep, - d.DecisionTableStep, d.LLMStep] +from firefly_weave.contracts.language import language_manifest +models = get_args(get_args(d.Step.__value__)[0]) def fields(model): return [f.alias or name for name, f in model.model_fields.items()] print(json.dumps({ @@ -53,6 +66,7 @@ print(json.dumps({ "workflow": fields(d.WorkflowSpec), "metadata": fields(d.Metadata), "case": fields(d.SwitchCase), "branch": fields(d.Branch), "operators": list(get_args(d.OperatorName.__value__)), + "manifest": language_manifest().model_dump(mode="json"), })) `, ], @@ -60,12 +74,28 @@ print(json.dumps({ ), ) : null; +const manifest = schema?.manifest as Manifest; describe.skipIf(!available)( "editor coverage of the real Python contracts", () => { - it("covers every step kind and configuration field", () => { - expect([...kinds].sort()).toEqual(Object.keys(schema.steps).sort()); + it("lists every Python step kind, operator and workflow field in the language manifest", () => { + expect(manifest.step_kinds.map((entry) => entry.kind).sort()).toEqual( + Object.keys(schema.steps).sort(), + ); + expect(manifest.operators.map((entry) => entry.name).sort()).toEqual( + [...schema.operators].sort(), + ); + expect( + manifest.workflow_fields.map((entry) => entry.name).sort(), + ).toEqual([...schema.workflow].sort()); + }); + it("covers every ready step kind and its configuration fields", () => { + expect([...kinds].sort()).toEqual( + ready(manifest.step_kinds) + .map((entry) => entry.kind) + .sort(), + ); for (const kind of kinds) { const fields = new Set( propertyFields(createStep(kind, "step")).map((f) => f.path[0]), @@ -76,21 +106,23 @@ describe.skipIf(!available)( expect([...fields].sort(), kind).toEqual(schema.steps[kind].sort()); } }); - it("covers workflow configuration and metadata without hiding fields in source mode", () => { + it("covers ready workflow configuration and metadata without hiding fields in source mode", () => { const fields = propertyFields(freshWorkflow()); const spec = fields .filter((f) => f.path[0] === "spec") .map((f) => f.path[1]); // Canvas owns steps, ConnectionSlotList the slots, and LlmInspector profiles. expect([...spec, "steps", "connections", "llmProfiles"].sort()).toEqual( - schema.workflow.sort(), + ready(manifest.workflow_fields) + .map((entry) => entry.name) + .sort(), ); const metadata = fields .filter((f) => f.path[0] === "metadata") .map((f) => f.path[1]); expect(metadata.sort()).toEqual(schema.metadata.sort()); }); - it("covers branch configuration and every supported expression operation", () => { + it("covers branch configuration and every ready expression operation", () => { const decision = propertyFields(createStep("switch", "decision")); const caseFields = decision .filter((f) => f.path[0] === "cases") @@ -101,7 +133,11 @@ describe.skipIf(!available)( .filter((f) => f.path[0] === "branches" && f.path[1] === "first") .map((f) => f.path[2]); expect([...branchFields, "steps"].sort()).toEqual(schema.branch.sort()); - expect([...supportedOperators].sort()).toEqual(schema.operators.sort()); + expect([...supportedOperators].sort()).toEqual( + ready(manifest.operators) + .map((entry) => entry.name) + .sort(), + ); }); }, ); diff --git a/tests/contracts/test_language_surface.py b/tests/contracts/test_language_surface.py new file mode 100644 index 00000000..267d4f06 --- /dev/null +++ b/tests/contracts/test_language_surface.py @@ -0,0 +1,123 @@ +# Copyright 2026 Firefly Software Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# Author: Firefly Software Foundation +# SPDX-License-Identifier: Apache-2.0 + +"""One language manifest across the platform operation, schema, OpenAPI, SDK, CLI and the Studio host.""" + +import asyncio +import json +from pathlib import Path +from types import SimpleNamespace +from unittest.mock import AsyncMock +from uuid import uuid4 + +import httpx +from click.testing import CliRunner +from jsonschema import Draft202012Validator +from starlette.requests import Request +from starlette.testclient import TestClient + +from firefly_weave.contracts.access import Scope +from firefly_weave.contracts.language import language_manifest + +ORIGIN = "http://127.0.0.1:8766" +CODE = "terminal-only-pairing-code" + + +def controller_fixture(): + from pyfly.container import Container + + from firefly_weave.api.compiler import CompilerController + from firefly_weave.definitions.service import DefinitionService + from firefly_weave.settings import Settings + + settings = Settings(database_url="postgresql+asyncpg://test:never-expose-this@localhost/test") + service = SimpleNamespace(catalog=AsyncMock(), capabilities=SimpleNamespace(resources={})) + graph = Container() + graph.register_instance(Settings, settings) + graph.register_instance(DefinitionService, service) + graph.register(CompilerController) + scope = Scope(tenant_id=uuid4(), project_id=uuid4()) + request = Request( + { + "type": "http", + "path_params": {"tenant": str(scope.tenant_id), "project": str(scope.project_id)}, + "state": {"principal": object(), "audit_context": object()}, + "app": SimpleNamespace(state=SimpleNamespace()), + } + ) + return graph.resolve(CompilerController), request, service, scope + + +async def test_language_read_is_authorized_like_capabilities_and_matches_every_declaration(monkeypatch): + from firefly_weave.cli.main import cli + from firefly_weave.contracts.openapi import export_openapi + from firefly_weave.contracts.schema_export import export_schemas + from firefly_weave.contracts.surface import OPERATIONS + from firefly_weave.sdk import client + + controller, request, service, scope = controller_fixture() + payload = await controller.language(request) + service.catalog.assert_awaited_once_with(request.state.principal, scope, context=request.state.audit_context) + assert payload == language_manifest().model_dump(mode="json") + operation = OPERATIONS["language.read"] + assert (operation.method, operation.canonical_path, operation.capability) == ( + "GET", + "/api/v1/tenants/{tenant}/projects/{project}/language", + "catalog.read", + ) + Draft202012Validator(export_schemas()["language-manifest"]).validate(payload) + spec = export_openapi() + schema = spec["paths"][operation.canonical_path]["get"]["responses"]["200"]["content"]["application/json"]["schema"] + Draft202012Validator({**schema, "components": spec["components"]}).validate(payload) + + original_client = client.WeaveClient + calls = [] + + def receive(http_request): + calls.append(http_request) + assert http_request.url.path.endswith("/language") + return httpx.Response(200, json=payload) + + def connected(*args, **kwargs): + return original_client(*args, **kwargs, transport=httpx.MockTransport(receive)) + + async with connected("https://api.invalid", lambda: "test-token", scope) as sdk: + assert (await sdk.language()) == language_manifest() + monkeypatch.setattr(client, "WeaveClient", connected) + monkeypatch.setenv("WEAVE_ACCESS_TOKEN", "test-token") + arguments = ["remote", "language", "--base-url", "https://api.invalid"] + arguments += ["--tenant", str(scope.tenant_id), "--project", str(scope.project_id)] + result = await asyncio.to_thread(CliRunner().invoke, cli, arguments) + assert result.exit_code == 0, result.output + assert json.loads(result.output) == payload + assert len(calls) == 2 + + +def test_studio_host_serves_the_manifest_only_to_a_paired_session(tmp_path: Path): + from firefly_weave.studio.host import make_studio_app + from firefly_weave.studio.service import StudioOptions + + assets = tmp_path / "assets" + assets.mkdir() + (assets / "index.html").write_text("Weave Studio") + app = make_studio_app(StudioOptions(origin=ORIGIN, assets=assets, pairing_code=CODE)) + with TestClient(app, base_url=ORIGIN) as browser: + assert browser.get("/studio/contracts/language").status_code == 401 + assert browser.post("/studio/session", json={"code": CODE}, headers={"Origin": ORIGIN}).status_code == 200 + response = browser.get("/studio/contracts/language") + assert response.status_code == 200 + assert response.headers["cache-control"] == "no-store" + assert response.json() == language_manifest().model_dump(mode="json") diff --git a/tests/fixtures/schemas/contracts.json b/tests/fixtures/schemas/contracts.json index f09b81f4..3d21bd61 100644 --- a/tests/fixtures/schemas/contracts.json +++ b/tests/fixtures/schemas/contracts.json @@ -439,5 +439,251 @@ "extra": 1 }, "valid": false + }, + { + "name": "language-loop-call-text", + "contract": "definition", + "valid": true, + "value": { + "apiVersion": "weave/v1alpha1", + "kind": "Workflow", + "metadata": { + "name": "sample", + "version": "1.0.0" + }, + "spec": { + "inputSchema": {}, + "outputSchema": {}, + "steps": [ + { + "id": "notify", + "kind": "forEach", + "items": { + "ref": "/input/invoices" + }, + "concurrency": 4, + "maxItems": 500, + "collect": "nonNull", + "body": { + "steps": [ + { + "id": "label", + "kind": "transform", + "value": { + "op": { + "name": "concat", + "args": [ + { + "literal": "Invoice " + }, + { + "ref": "/item/number" + } + ] + } + } + }, + { + "id": "enrich", + "kind": "callWorkflow", + "uses": "customer-enrichment@1.2.0", + "with": { + "object": {} + }, + "mode": "wait", + "onFailure": "continue", + "businessKey": { + "ref": "/item/number" + } + } + ], + "output": { + "op": { + "name": "join", + "args": [ + { + "ref": "/item/tags" + }, + { + "literal": ", " + } + ] + } + } + } + } + ], + "output": { + "ref": "/steps/t/output" + }, + "callable": { + "allowedCallers": [ + "order-intake" + ] + } + } + } + }, + { + "name": "language-call-detach-on-failure", + "contract": "definition", + "valid": false, + "value": { + "apiVersion": "weave/v1alpha1", + "kind": "Workflow", + "metadata": { + "name": "sample", + "version": "1.0.0" + }, + "spec": { + "inputSchema": {}, + "outputSchema": {}, + "steps": [ + { + "id": "enrich", + "kind": "callWorkflow", + "uses": "customer-enrichment@1.2.0", + "with": { + "object": {} + }, + "mode": "detach", + "onFailure": "stop" + } + ], + "output": { + "ref": "/steps/t/output" + } + } + } + }, + { + "name": "language-loop-null-max-items", + "contract": "definition", + "valid": false, + "value": { + "apiVersion": "weave/v1alpha1", + "kind": "Workflow", + "metadata": { + "name": "sample", + "version": "1.0.0" + }, + "spec": { + "inputSchema": {}, + "outputSchema": {}, + "steps": [ + { + "id": "notify", + "kind": "forEach", + "items": { + "ref": "/input/invoices" + }, + "concurrency": 4, + "maxItems": null, + "collect": "nonNull", + "body": { + "steps": [ + { + "id": "label", + "kind": "transform", + "value": { + "op": { + "name": "concat", + "args": [ + { + "literal": "Invoice " + }, + { + "ref": "/item/number" + } + ] + } + } + }, + { + "id": "enrich", + "kind": "callWorkflow", + "uses": "customer-enrichment@1.2.0", + "with": { + "object": {} + }, + "mode": "wait", + "onFailure": "continue", + "businessKey": { + "ref": "/item/number" + } + } + ], + "output": { + "op": { + "name": "join", + "args": [ + { + "ref": "/item/tags" + }, + { + "literal": ", " + } + ] + } + } + } + } + ], + "output": { + "ref": "/steps/t/output" + } + } + } + }, + { + "name": "language-callable-empty-callers", + "contract": "definition", + "valid": false, + "value": { + "apiVersion": "weave/v1alpha1", + "kind": "Workflow", + "metadata": { + "name": "sample", + "version": "1.0.0" + }, + "spec": { + "inputSchema": {}, + "outputSchema": {}, + "steps": [], + "output": { + "ref": "/steps/t/output" + }, + "callable": { + "allowedCallers": [] + } + } + } + }, + { + "name": "language-callable-duplicate-callers", + "contract": "definition", + "valid": false, + "value": { + "apiVersion": "weave/v1alpha1", + "kind": "Workflow", + "metadata": { + "name": "sample", + "version": "1.0.0" + }, + "spec": { + "inputSchema": {}, + "outputSchema": {}, + "steps": [], + "output": { + "ref": "/steps/t/output" + }, + "callable": { + "allowedCallers": [ + "a", + "a" + ] + } + } + } } ] diff --git a/tests/unit/compiler/test_authoring_validation.py b/tests/unit/compiler/test_authoring_validation.py index 1b10d4b7..ce1dbe12 100644 --- a/tests/unit/compiler/test_authoring_validation.py +++ b/tests/unit/compiler/test_authoring_validation.py @@ -633,4 +633,6 @@ def test_documented_examples_are_clean_offline(): if "kind:" not in text: continue result = validate_authoring(text, format="yaml", filename=path.name) - assert errors(result) == [], (path.name, codes(result)) + # examples/language shows frozen constructs that compile once their language milestone lands. + findings = [d for d in errors(result) if d.code != "WV-COMP-UNSUPPORTED_FEATURE"] + assert findings == [], (path.name, codes(result)) diff --git a/tests/unit/compiler/test_instance_keys.py b/tests/unit/compiler/test_instance_keys.py new file mode 100644 index 00000000..ad6dc098 --- /dev/null +++ b/tests/unit/compiler/test_instance_keys.py @@ -0,0 +1,233 @@ +# Copyright 2026 Firefly Software Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# Author: Firefly Software Foundation +# SPDX-License-Identifier: Apache-2.0 + +"""The frozen instance-key grammar (language spec 3.2, overview C11).""" + +import re + +import pytest +from hypothesis import given +from hypothesis import strategies as st +from pydantic import TypeAdapter, ValidationError + +from firefly_weave.contracts.instance_keys import ( + INSTANCE_KEY_PATTERN, + InstanceKey, + InstanceKeyText, + InstanceKeyTextOrEmpty, + InstanceView, + InvalidInstanceKey, + format_instance, + instance_view, + node_of, + split_instance, +) +from firefly_weave.contracts.values import MAX_SAFE_INTEGER + +# The cases of the shared fixture studio/tests/fixtures/language/instance-keys.json (scripts/language_fixtures.py), +# plus keys a JSON fixture cannot carry (a 5000-digit index and a lone surrogate). +VALID_KEYS = [ + ("send", InstanceKey("send")), + ("send[3]", InstanceKey("send", (3,))), + ("send[3][0]", InstanceKey("send", (3, 0))), + ("support#2", InstanceKey("support", (), ("2",))), + ("support#2.1", InstanceKey("support", (), ("2", "1"))), + ("support#2.1.review", InstanceKey("support", (), ("2", "1", "review"))), + ("support[3]#2.1.review", InstanceKey("support", (3,), ("2", "1", "review"))), + ("notify~2", InstanceKey("notify", (), (), 2)), + ("notify[1]~2", InstanceKey("notify", (1,), (), 2)), + ("@join:notify", InstanceKey("@join:notify")), + ("@branch:notify:0[4]", InstanceKey("@branch:notify:0", (4,))), + ("a.b_c-d[9007199254740991]", InstanceKey("a.b_c-d", (MAX_SAFE_INTEGER,))), +] +INVALID_KEYS = [ + "", + "[3]", + "#2", + "~1", + "send[03]", + "send[-1]", + "send[]", + "send[3", + "send]3[", + "send[3]x", + "send#", + "send#2.", + "send#.2", + "send#2..1", + "send#ü", + "send#2[3]", + "send~0", + "send~01", + "send~1[2]", + "send~1#2", + "send~1~2", + "send[9007199254740992]", + "send~9007199254740992", + "send[" + "9" * 5000 + "]", + "a b", + "send\n", + "send\x00", + "se/nd", + "ü[1]", + "send\ud800", + "@", +] +TEXT = TypeAdapter(InstanceKeyText) +TEXT_OR_EMPTY = TypeAdapter(InstanceKeyTextOrEmpty) + + +def accepts(validate, key) -> bool: + try: + validate(key) + except (InvalidInstanceKey, ValidationError): + return False + return True + + +@pytest.mark.parametrize(("key", "parsed"), VALID_KEYS) +def test_every_grammar_form_parses_and_formats_back(key, parsed): + assert split_instance(key) == parsed + assert format_instance(parsed) == key + assert node_of(key) == parsed.node_id + + +@pytest.mark.parametrize("key", INVALID_KEYS) +def test_non_canonical_keys_are_rejected(key): + with pytest.raises(InvalidInstanceKey): + split_instance(key) + + +@pytest.mark.parametrize( + ("key", "valid"), [(key, True) for key, _ in VALID_KEYS] + [(key, False) for key in INVALID_KEYS] +) +def test_the_pattern_and_api_types_accept_exactly_what_split_instance_accepts(key, valid): + assert accepts(split_instance, key) is valid + assert (re.fullmatch(INSTANCE_KEY_PATTERN, key) is not None) is valid + assert accepts(TEXT.validate_python, key) is valid + assert accepts(TEXT_OR_EMPTY.validate_python, key) is (valid or key == "") + if valid: + assert node_of(key) == split_instance(key).node_id + + +def test_the_anchored_form_is_for_ecmascript_and_pydantic_not_python_re(): + key = "send[1]\n" + # Python's "$" also matches before a trailing newline, so the anchored form is not a safe Python guard ... + assert re.match(rf"^(?:{INSTANCE_KEY_PATTERN})$", key) is not None + # ... while re.fullmatch on the unanchored pattern, split_instance and pydantic's default (Rust) engine reject it. + assert re.fullmatch(INSTANCE_KEY_PATTERN, key) is None + assert not accepts(split_instance, key) + assert not accepts(TEXT.validate_python, key) + assert not accepts(TEXT_OR_EMPTY.validate_python, key) + + +def test_the_pattern_bounds_indexes_and_counts_at_max_safe_integer(): + digits = str(MAX_SAFE_INTEGER) + numbers = {0, 1, 9, 10, 10**15 - 1, 10**15, 10**16 - 1, 10**16, MAX_SAFE_INTEGER + 1} + numbers |= {int(digits[:at] + digit + digits[at + 1 :]) for at in range(len(digits)) for digit in "0123456789"} + for number in sorted(numbers): + assert (re.fullmatch(INSTANCE_KEY_PATTERN, f"send[{number}]") is not None) is (number <= MAX_SAFE_INTEGER) + assert (re.fullmatch(INSTANCE_KEY_PATTERN, f"send~{number}") is not None) is (1 <= number <= MAX_SAFE_INTEGER) + + +def test_api_types_cap_keys_at_512_characters_and_publish_the_pattern(): + longest = "s" * 509 + "[1]" + assert TEXT.validate_python(longest) == longest + with pytest.raises(ValidationError): + TEXT.validate_python("s" + longest) + assert TEXT_OR_EMPTY.validate_python("") == "" + assert TEXT.json_schema() == {"type": "string", "maxLength": 512, "pattern": f"^(?:{INSTANCE_KEY_PATTERN})$"} + assert TEXT_OR_EMPTY.json_schema() == { + "type": "string", + "maxLength": 512, + "pattern": f"^(?:{INSTANCE_KEY_PATTERN})?$", + } + + +@pytest.mark.parametrize( + "parts", + [ + {"node_id": ""}, + {"node_id": "a[1]"}, + {"node_id": "a#b"}, + {"node_id": "a~1"}, + {"node_id": "a]"}, + {"node_id": "a b"}, + {"node_id": "ü"}, + {"node_id": "a", "indexes": (-1,)}, + {"node_id": "a", "indexes": (True,)}, + {"node_id": "a", "indexes": (MAX_SAFE_INTEGER + 1,)}, + {"node_id": "a", "indexes": [1]}, + {"node_id": "a", "segments": ("",)}, + {"node_id": "a", "segments": ("2.1",)}, + {"node_id": "a", "count": 0}, + {"node_id": "a", "count": True}, + ], +) +def test_parts_are_validated_when_constructed(parts): + with pytest.raises(InvalidInstanceKey): + InstanceKey(**parts) + + +def test_long_keys_parse_in_linear_time(): + key = "send" + "[1]" * 20_000 + "#" + ".".join(["a"] * 20_000) + "~7" + parsed = split_instance(key) + assert parsed.indexes == (1,) * 20_000 and len(parsed.segments) == 20_000 and parsed.count == 7 + assert format_instance(parsed) == key + + +@pytest.mark.parametrize("key", ["", "[3]", "#2", "~1"]) +def test_node_of_rejects_keys_without_a_node_id(key): + with pytest.raises(InvalidInstanceKey): + node_of(key) + + +def test_api_view_uses_static_node_id_and_empty_key_outside_loops_and_agents(): + assert instance_view("send") == InstanceView("send", "", []) + assert instance_view("send[3][0]") == InstanceView("send", "send[3][0]", [3, 0]) + assert instance_view("support[2]#3.1") == InstanceView("support", "support[2]#3.1", [2]) + with pytest.raises(InvalidInstanceKey): + instance_view("send[03]") + + +node_ids = st.from_regex(r"[A-Za-z0-9][A-Za-z0-9_.-]{0,20}|@join:[a-z]{1,5}|@branch:[a-z]{1,5}:[0-9]", fullmatch=True) +keys = st.builds( + InstanceKey, + node_ids, + st.lists(st.integers(0, MAX_SAFE_INTEGER), max_size=4).map(tuple), + st.lists(st.from_regex(r"[A-Za-z0-9]{1,6}", fullmatch=True), max_size=4).map(tuple), + st.none() | st.integers(1, MAX_SAFE_INTEGER), +) + + +@given(keys) +def test_split_and_format_round_trip_for_every_form(key): + text = format_instance(key) + assert split_instance(text) == key + assert format_instance(split_instance(text)) == text + assert re.fullmatch(INSTANCE_KEY_PATTERN, text) + + +@given(keys) +def test_node_of_returns_the_static_id_without_separators(key): + node = node_of(format_instance(key)) + assert node == key.node_id == split_instance(format_instance(key)).node_id + assert not {"[", "#", "~"} & set(node) + + +@given(st.text(alphabet="sa9810[]#~.ü", max_size=16)) +def test_the_pattern_agrees_with_split_instance_on_any_text(text): + assert (re.fullmatch(INSTANCE_KEY_PATTERN, text) is not None) is accepts(split_instance, text) diff --git a/tests/unit/compiler/test_ir_features.py b/tests/unit/compiler/test_ir_features.py new file mode 100644 index 00000000..6471d7b1 --- /dev/null +++ b/tests/unit/compiler/test_ir_features.py @@ -0,0 +1,78 @@ +# Copyright 2026 Firefly Software Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# Author: Firefly Software Foundation +# SPDX-License-Identifier: Apache-2.0 + +"""The IR feature set's frozen shape (language spec 3.1); language milestone M1 computes and checks it.""" + +import json +from pathlib import Path + +import pytest +from pydantic import ValidationError + +from firefly_weave.compiler.api import compile_source +from firefly_weave.compiler.canonical import canonical_digest +from firefly_weave.compiler.ir import IR_VERSION_EXTENSIONS, WorkflowIR +from firefly_weave.contracts.language_features import ADVERTISED_FEATURES, LANGUAGE_FEATURES +from firefly_weave.contracts.schema_export import export_schemas + +EXECUTABLE = json.loads(Path("tests/fixtures/canonical/empty-workflow.executable.json").read_text()) + + +def test_the_ir_version_extensions_is_named_but_nothing_is_advertised_yet(): + assert IR_VERSION_EXTENSIONS == "weave/ir-v1alpha4" + assert ADVERTISED_FEATURES == () + + +def test_an_empty_feature_set_is_omitted(): + model = WorkflowIR.model_validate(EXECUTABLE) + assert model.features == [] + assert "features" not in model.model_dump(by_alias=True) + assert "features" not in WorkflowIR.model_validate({**EXECUTABLE, "features": []}).model_dump(by_alias=True) + + +def test_compiled_workflows_keep_their_ir_version_and_digest(catalog, workflow_source): + result = compile_source(workflow_source, format="yaml", catalog=catalog) + executable = result.artifact.executable + assert "features" not in executable + assert executable["irVersion"] == "weave/ir-v1alpha1" + assert canonical_digest(WorkflowIR.model_validate(executable).model_dump(by_alias=True)) == result.artifact.digest + + +@pytest.mark.parametrize( + ("features", "message"), + [ + (["text.join", "text.concat"], "sorted and unique"), + (["text.concat", "text.concat"], "sorted and unique"), + (["text.concat"], "require weave/ir-v1alpha4"), + (["loops"], "Input should be"), + ], +) +def test_features_are_sorted_known_and_need_the_ir_version_extensions(features, message): + with pytest.raises(ValidationError, match=message): + WorkflowIR.model_validate({**EXECUTABLE, "features": features}) + + +@pytest.mark.parametrize("features", [[], ["text.concat"]]) +def test_the_ir_version_extensions_is_not_accepted_until_a_later_milestone_widens_the_model(features): + # M0 names weave/ir-v1alpha4 but the model still lists v1alpha1 to v1alpha3 only, so no executable can use it yet. + with pytest.raises(ValidationError, match="Input should be 'weave/ir-v1alpha1'"): + WorkflowIR.model_validate({**EXECUTABLE, "irVersion": IR_VERSION_EXTENSIONS, "features": features}) + + +def test_exported_executable_schema_carries_the_feature_vocabulary(): + schema = export_schemas()["executable"] + assert schema["$defs"]["WorkflowIR"]["properties"]["features"]["uniqueItems"] is True + assert schema["$defs"]["LanguageFeature"]["enum"] == list(LANGUAGE_FEATURES) diff --git a/tests/unit/compiler/test_language_definitions.py b/tests/unit/compiler/test_language_definitions.py new file mode 100644 index 00000000..a269bf5f --- /dev/null +++ b/tests/unit/compiler/test_language_definitions.py @@ -0,0 +1,166 @@ +# Copyright 2026 Firefly Software Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# Author: Firefly Software Foundation +# SPDX-License-Identifier: Apache-2.0 + +"""Definition models frozen in language milestone M0: forEach, callWorkflow, callable, concat and join.""" + +import json +from pathlib import Path + +import pytest +from jsonschema import Draft202012Validator +from pydantic import ValidationError + +from firefly_weave.contracts.definitions import load_definition +from firefly_weave.contracts.schema_export import export_schemas + +LOOP = { + "id": "notify", + "kind": "forEach", + "items": {"ref": "/input/invoices"}, + "body": {"steps": [], "output": {"ref": "/item"}}, +} +CALL = {"id": "enrich", "kind": "callWorkflow", "uses": "customer-enrichment@1.2.0", "with": {"object": {}}} + + +def workflow(steps: list[dict], **spec: object) -> dict: + return { + "apiVersion": "weave/v1alpha1", + "kind": "Workflow", + "metadata": {"name": "order-intake", "version": "1.0.0"}, + "spec": {"inputSchema": {}, "outputSchema": {}, "steps": steps, "output": {"literal": None}, **spec}, + } + + +def dumped(document: dict) -> dict: + return load_definition(document).model_dump(by_alias=True) + + +def test_loop_defaults_are_materialized_in_the_canonical_document(): + assert dumped(workflow([LOOP]))["spec"]["steps"][0] == { + "id": "notify", + "kind": "forEach", + "items": {"ref": "/input/invoices"}, + "concurrency": 1, + "maxItems": 1000, + "collect": "all", + "body": {"steps": [], "output": {"ref": "/item"}}, + } + + +def test_call_defaults_wait_and_omits_optional_fields(): + assert dumped(workflow([CALL]))["spec"]["steps"][0] == {**CALL, "mode": "wait"} + full = {**CALL, "mode": "wait", "onFailure": "continue", "businessKey": {"ref": "/input/customerId"}} + assert dumped(workflow([full]))["spec"]["steps"][0] == full + + +def test_callable_round_trips_and_an_absent_callable_is_not_written(): + assert dumped(workflow([], callable={}))["spec"]["callable"] == {} + callers = {"allowedCallers": ["order-intake", "invoice-run"]} + assert dumped(workflow([], callable=callers))["spec"]["callable"] == callers + assert "callable" not in dumped(workflow([]))["spec"] + + +def test_text_operators_are_part_of_the_expression_language(): + concat = {"op": {"name": "concat", "args": [{"literal": "Invoice "}, {"ref": "/input/number"}]}} + join = {"op": {"name": "join", "args": [{"ref": "/input/tags"}, {"literal": ", "}]}} + document = workflow([{"id": "t", "kind": "transform", "value": concat}], output=join) + assert dumped(document)["spec"]["steps"][0]["value"] == concat + + +def test_loop_body_holds_nested_new_kinds(): + nested = {**LOOP, "body": {"steps": [CALL, {**LOOP, "id": "inner"}], "output": {"literal": None}}} + assert [step["kind"] for step in dumped(workflow([nested]))["spec"]["steps"][0]["body"]["steps"]] == [ + "callWorkflow", + "forEach", + ] + + +@pytest.mark.parametrize( + "step", + [ + {**CALL, "mode": "detach", "onFailure": "continue"}, + {**CALL, "mode": "detach", "onFailure": "stop"}, + {**CALL, "onFailure": None}, + {**CALL, "businessKey": None}, + {**CALL, "mode": "later"}, + {**CALL, "uses": "customer-enrichment@latest"}, + {**CALL, "input": {"object": {}}}, + {**LOOP, "concurrency": 0}, + {**LOOP, "maxItems": 0}, + {**LOOP, "maxItems": None}, + {**LOOP, "collect": "some"}, + {**LOOP, "body": {"steps": []}}, + {key: value for key, value in LOOP.items() if key != "items"}, + {**LOOP, "id": "send[3]"}, + ], +) +def test_invalid_new_steps_are_rejected_by_the_model_and_the_exported_schema(step): + document = workflow([step]) + with pytest.raises(ValidationError): + load_definition(document) + if step.get("mode") == "detach": + # Cross-field rule: enforced by the model only, like retry delays. + return + assert not Draft202012Validator(export_schemas()["definition"]).is_valid(document) + + +@pytest.mark.parametrize( + "callable_spec", + [ + None, + {"allowedCallers": []}, + {"allowedCallers": ["a", "a"]}, + {"allowedCallers": None}, + {"allowed_callers": ["a"]}, + {"other": True}, + ], +) +def test_invalid_callable_is_rejected_by_the_model_and_the_exported_schema(callable_spec): + document = workflow([], callable=callable_spec) + with pytest.raises(ValidationError): + load_definition(document) + assert not Draft202012Validator(export_schemas()["definition"]).is_valid(document) + + +def test_callers_are_bounded_to_one_hundred_names(): + names = [f"caller-{index}" for index in range(101)] + with pytest.raises(ValidationError): + load_definition(workflow([], callable={"allowedCallers": names})) + assert ( + len(dumped(workflow([], callable={"allowedCallers": names[:100]}))["spec"]["callable"]["allowedCallers"]) == 100 + ) + + +def test_frozen_schema_definitions_are_exported_for_studio(): + definitions = export_schemas()["workflow"]["$defs"] + assert {"ForEachStep", "CallWorkflowStep", "CallableSpec"} <= definitions.keys() + assert definitions["ForEachStep"]["required"] == ["id", "kind", "items", "body"] + assert definitions["CallWorkflowStep"]["required"] == ["id", "kind", "uses", "with"] + callers = json.dumps(definitions["CallableSpec"]) + assert "allowedCallers" in callers + assert {"concat", "join"} <= set(definitions["OperatorName"]["enum"]) + + +def test_existing_examples_gain_no_new_keys(): + from firefly_weave.compiler.parser import parse_source + + for path in sorted(Path("examples").rglob("*.yaml")): + text = path.read_text(encoding="utf-8") + if "kind: Workflow" not in text or path.parent.name == "language": + continue + spec = dumped(parse_source(text, format="yaml").value)["spec"] + assert "callable" not in spec, path + assert "forEach" not in json.dumps(spec) and "callWorkflow" not in json.dumps(spec), path diff --git a/tests/unit/compiler/test_language_examples.py b/tests/unit/compiler/test_language_examples.py new file mode 100644 index 00000000..588ce442 --- /dev/null +++ b/tests/unit/compiler/test_language_examples.py @@ -0,0 +1,67 @@ +# Copyright 2026 Firefly Software Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# Author: Firefly Software Foundation +# SPDX-License-Identifier: Apache-2.0 + +"""The language examples have their final shape and report exactly the features not compiled yet.""" + +from pathlib import Path + +import pytest + +from firefly_weave.compiler.api import compile_source, validate_authoring +from firefly_weave.compiler.catalog import CatalogSnapshot +from firefly_weave.compiler.parser import parse_source +from firefly_weave.contracts.definitions import load_definition + +EXAMPLES = Path("examples/language") +EXPECTED = { + "notify-customer.workflow.yaml": [], + "notify-overdue.workflow.yaml": [ + ("/spec/steps/0/body/steps/0/with/object/subject/op/name", "text.concat"), + ("/spec/steps/0/kind", "flow.forEach"), + ], + "order-intake.workflow.yaml": [ + ("/spec/steps/0/kind", "flow.callWorkflow"), + ("/spec/steps/0/with/object/message/op/name", "text.concat"), + ("/spec/steps/1/kind", "flow.callWorkflow"), + ("/spec/steps/1/with/object/message/op/name", "text.join"), + ], +} + + +def test_every_language_example_is_listed(): + assert sorted(path.name for path in EXAMPLES.glob("*.yaml")) == sorted(EXPECTED) + + +@pytest.mark.parametrize("name", sorted(EXPECTED)) +def test_example_shape_is_final_and_unsupported_features_are_reported(name): + text = (EXAMPLES / name).read_text() + load_definition(parse_source(text, format="yaml").value) + result = validate_authoring(text, format="yaml", filename=name) + reported = sorted( + (d.path, d.message.split("language feature ")[1].split(",")[0]) + for d in result.diagnostics + if d.code == "WV-COMP-UNSUPPORTED_FEATURE" + ) + assert reported == EXPECTED[name] + assert [d.code for d in result.diagnostics if d.severity == "error"] == ["WV-COMP-UNSUPPORTED_FEATURE"] * len( + EXPECTED[name] + ) + + +def test_the_callable_example_compiles_today(): + text = (EXAMPLES / "notify-customer.workflow.yaml").read_text() + result = compile_source(text, format="yaml", catalog=CatalogSnapshot.from_definitions([])) + assert result.ok, [d.code for d in result.diagnostics] diff --git a/tests/unit/compiler/test_language_fixtures.py b/tests/unit/compiler/test_language_fixtures.py new file mode 100644 index 00000000..e208a1cc --- /dev/null +++ b/tests/unit/compiler/test_language_fixtures.py @@ -0,0 +1,93 @@ +# Copyright 2026 Firefly Software Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# Author: Firefly Software Foundation +# SPDX-License-Identifier: Apache-2.0 + +"""The shared language fixtures (language spec 14.5) are current and agree with the Python contracts.""" + +import json +import re +import runpy +from pathlib import Path + +import pytest +import rfc8785 + +from firefly_weave.contracts.instance_keys import ( + INSTANCE_KEY_PATTERN, + InstanceKey, + InvalidInstanceKey, + format_instance, + instance_view, + node_of, + split_instance, +) +from firefly_weave.contracts.language import language_manifest + +FIXTURES = Path("studio/tests/fixtures/language") + + +def fixture(name: str): + return json.loads((FIXTURES / name).read_text(encoding="utf-8")) + + +def test_committed_fixtures_are_what_the_generator_writes_today(): + rendered = runpy.run_path("scripts/language_fixtures.py")["render"]() + assert sorted(rendered) == ["instance-keys.json", "manifest.json", "text-conversion.json"] + for name, text in rendered.items(): + assert (FIXTURES / name).read_text(encoding="utf-8") == text, name + + +@pytest.mark.parametrize("case", fixture("instance-keys.json")["valid"], ids=lambda case: case["key"]) +def test_valid_instance_key_cases(case): + parsed = InstanceKey(case["node_id"], tuple(case["indexes"]), tuple(case["segments"]), case["count"]) + assert split_instance(case["key"]) == parsed + assert format_instance(parsed) == case["key"] + assert node_of(case["key"]) == split_instance(case["key"]).node_id == case["node_id"] + assert re.fullmatch(INSTANCE_KEY_PATTERN, case["key"]) + view = instance_view(case["key"]) + assert {"node_id": view.node_id, "instance_key": view.instance_key, "iteration": view.iteration} == case["view"] + + +@pytest.mark.parametrize("case", fixture("instance-keys.json")["invalid"], ids=lambda case: case["reason"]) +def test_invalid_instance_key_cases(case): + with pytest.raises(InvalidInstanceKey): + split_instance(case["key"]) + assert re.fullmatch(INSTANCE_KEY_PATTERN, case["key"]) is None + + +def test_instance_key_fixture_publishes_the_python_pattern(): + assert fixture("instance-keys.json")["pattern"] == INSTANCE_KEY_PATTERN + + +@pytest.mark.parametrize("case", fixture("text-conversion.json")["cases"], ids=lambda case: repr(case["value"])) +def test_text_conversion_cases_follow_ecmascript_number_formatting(case): + value = case["value"] + expected = value if isinstance(value, str) else rfc8785.dumps(value).decode() + if isinstance(value, bool): + expected = "true" if value else "false" + assert case["text"] == expected + + +def test_manifest_snapshot_is_the_served_manifest(): + assert fixture("manifest.json")["manifest"] == language_manifest().model_dump(mode="json") + + +def test_freshness_check_accepts_windows_line_endings_and_catches_drift(tmp_path): + generator = runpy.run_path("scripts/language_fixtures.py") + for name, text in generator["render"]().items(): + (tmp_path / name).write_bytes(text.replace("\n", "\r\n").encode("utf-8")) + assert generator["main"](["--check", "--output", str(tmp_path)]) == 0 + (tmp_path / "manifest.json").write_text("{}\n", encoding="utf-8") + assert generator["main"](["--check", "--output", str(tmp_path)]) == 1 diff --git a/tests/unit/compiler/test_language_manifest.py b/tests/unit/compiler/test_language_manifest.py new file mode 100644 index 00000000..1ab3d6f6 --- /dev/null +++ b/tests/unit/compiler/test_language_manifest.py @@ -0,0 +1,208 @@ +# Copyright 2026 Firefly Software Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# Author: Firefly Software Foundation +# SPDX-License-Identifier: Apache-2.0 + +"""The language manifest lists exactly what the definition models accept (language spec 14.1, contract C5).""" + +import inspect +from typing import get_args + +import pytest +from pydantic import ValidationError + +from firefly_weave.compiler.api import compile_source +from firefly_weave.compiler.ir import COMPARISON_IR_VERSION, HUMAN_IR_VERSION, IR_VERSION +from firefly_weave.contracts import definitions +from firefly_weave.contracts.language import LanguageManifest, language_manifest, supported_step_kinds +from firefly_weave.contracts.language_features import ( + DEFAULT_LOOP_MAX_ITEMS, + KIND_FEATURES, + LANGUAGE_FEATURES, + OPERATOR_FEATURES, + WORKFLOW_FIELD_FEATURES, +) +from firefly_weave.contracts.public import Capabilities +from firefly_weave.contracts.schema_export import export_schemas + + +def union_kinds() -> list[str]: + models = get_args(get_args(definitions.Step.__value__)[0]) + return [get_args(model.model_fields["kind"].annotation)[0] for model in models] + + +def test_feature_vocabulary_is_frozen(): + assert LANGUAGE_FEATURES == ( + "ai.agent", + "ai.memory", + "flow.callWorkflow", + "flow.forEach", + "text.concat", + "text.join", + ) + assert dict(KIND_FEATURES) == {"forEach": "flow.forEach", "callWorkflow": "flow.callWorkflow"} + assert dict(OPERATOR_FEATURES) == {"concat": "text.concat", "join": "text.join"} + assert dict(WORKFLOW_FIELD_FEATURES) == {"callable": "flow.callWorkflow"} + + +def test_every_definition_construct_has_exactly_one_manifest_entry(): + manifest = language_manifest() + assert sorted(entry.kind for entry in manifest.step_kinds) == sorted(union_kinds()) + assert [entry.name for entry in manifest.operators] == list(get_args(definitions.OperatorName.__value__)) + assert sorted(entry.name for entry in manifest.workflow_fields) == sorted( + field.alias or name for name, field in definitions.WorkflowSpec.model_fields.items() + ) + + +def test_entries_carry_exactly_the_features_of_the_vocabulary(): + manifest = language_manifest() + assert {entry.kind: entry.feature for entry in manifest.step_kinds if entry.feature} == { + kind: feature for kind, feature in KIND_FEATURES.items() if kind in union_kinds() + } + names = [entry.name for entry in manifest.operators] + assert {entry.name: entry.feature for entry in manifest.operators if entry.feature} == { + name: feature for name, feature in OPERATOR_FEATURES.items() if name in names + } + fields = [entry.name for entry in manifest.workflow_fields] + assert {entry.name: entry.feature for entry in manifest.workflow_fields if entry.feature} == { + name: feature for name, feature in WORKFLOW_FIELD_FEATURES.items() if name in fields + } + + +def test_action_entry_and_limits_match_the_spec_example(): + dump = language_manifest().model_dump(mode="json") + assert dump["step_kinds"][0] == { + "kind": "action", + "schema_ref": "workflow.schema.json#/$defs/ActionStep", + "group": "actions", + "label": "Action", + "blocks": [], + "scope_roots": [], + "studio": "ready", + } + assert dump["limits"] == { + "max_loop_items": 10000, + "default_loop_max_items": 1000, + "max_loop_depth": 3, + "max_concurrency": 1000, + "max_run_iterations": 100000, + "max_call_depth": 8, + } + assert (dump["version"], dump["language_version"]) == ("weave/language-manifest-v1", "weave/v1alpha1") + assert dump["ir_versions"] == [IR_VERSION, HUMAN_IR_VERSION, COMPARISON_IR_VERSION] + + +def test_schema_references_and_blocks_resolve_in_the_exported_workflow_schema(): + workflow = export_schemas()["workflow"] + for entry in language_manifest().step_kinds: + model = workflow["$defs"][entry.schema_ref.removeprefix("workflow.schema.json#/$defs/")] + assert model["properties"]["kind"]["const"] == entry.kind + for block in entry.blocks: + assert block.path.removeprefix("/") in model["properties"] + + +def test_limits_follow_their_sources(): + limits = language_manifest().limits + assert limits.default_loop_max_items == DEFAULT_LOOP_MAX_ITEMS + assert limits.max_concurrency == inspect.signature(compile_source).parameters["max_parallel_concurrency"].default + + +def test_features_are_what_the_target_runs(): + assert language_manifest().features == [] + assert language_manifest(["text.join", "text.concat"]).features == ["text.concat", "text.join"] + + +def test_capabilities_list_every_runnable_kind_from_the_manifest(): + capabilities = Capabilities(limits={}, schemas=[], connectors=[]) + assert capabilities.step_kinds == supported_step_kinds() + assert sorted(capabilities.step_kinds) == sorted( + ["action", "llm", "transform", "decisionTable", "switch", "parallel", "wait", "signal", "humanTask", "fail"] + ) + assert capabilities.ir_versions == [IR_VERSION, HUMAN_IR_VERSION, COMPARISON_IR_VERSION] + + +@pytest.mark.parametrize( + "change", + [ + lambda m: {**m, "features": ["text.join", "text.concat"]}, + lambda m: {**m, "features": ["loops"]}, + lambda m: {**m, "operators": [*m["operators"], m["operators"][0]]}, + lambda m: {**m, "step_kinds": [{**m["step_kinds"][0], "studio": "done"}]}, + lambda m: {**m, "step_kinds": [{**m["step_kinds"][0], "feature": None}]}, + lambda m: {**m, "version": "weave/language-manifest-v2"}, + lambda m: {**m, "unknown": True}, + ], +) +def test_manifest_contract_rejects_malformed_documents(change): + with pytest.raises(ValidationError): + LanguageManifest.model_validate(change(language_manifest().model_dump(mode="json"))) + + +def test_manifest_round_trips_and_its_schema_is_exported(): + manifest = language_manifest() + assert LanguageManifest.model_validate_json(manifest.model_dump_json()) == manifest + exported = export_schemas()["language-manifest"] + assert exported["$id"] == "language-manifest.schema.json" + assert set(exported["required"]) == { + "version", + "language_version", + "ir_versions", + "features", + "limits", + "operators", + "workflow_fields", + "step_kinds", + } + + +def test_new_constructs_stay_pending_until_studio_ships_them(): + manifest = language_manifest() + assert {entry.kind for entry in manifest.step_kinds if entry.studio == "pending"} == {"forEach", "callWorkflow"} + assert {entry.name for entry in manifest.operators if entry.studio == "pending"} == {"concat", "join"} + assert {entry.name for entry in manifest.workflow_fields if entry.studio == "pending"} == {"callable"} + + +def test_loop_and_join_entries_match_the_spec_example(): + dump = language_manifest().model_dump(mode="json") + assert next(entry for entry in dump["step_kinds"] if entry["kind"] == "forEach") == { + "kind": "forEach", + "feature": "flow.forEach", + "schema_ref": "workflow.schema.json#/$defs/ForEachStep", + "group": "flow", + "label": "Loop over items", + "blocks": [{"name": "body", "path": "/body", "label": "For each item"}], + "scope_roots": ["/item", "/index", "/loops"], + "studio": "pending", + } + assert next(entry for entry in dump["operators"] if entry["name"] == "join") == { + "name": "join", + "feature": "text.join", + "arity": {"min": 2, "max": 2}, + "operand_types": [["array"], ["string"]], + "item_types": ["string", "integer", "number", "boolean"], + "result": {"type": "string"}, + "label": "Join list", + "studio": "pending", + } + + +def test_loop_default_in_the_definition_is_the_manifest_default(): + max_items = definitions.ForEachStep.model_fields["max_items"] + assert max_items.default == language_manifest().limits.default_loop_max_items + + +def test_kinds_with_a_feature_run_only_where_the_feature_is_advertised(): + assert "forEach" not in supported_step_kinds() + assert "forEach" in supported_step_kinds(["flow.forEach"]) + assert "callWorkflow" in supported_step_kinds(["flow.callWorkflow"]) diff --git a/tests/unit/compiler/test_schemas.py b/tests/unit/compiler/test_schemas.py index da8b17fd..c4fc205c 100644 --- a/tests/unit/compiler/test_schemas.py +++ b/tests/unit/compiler/test_schemas.py @@ -358,7 +358,13 @@ def test_generated_shapes_match_models_for_representable_fixture_cases(): cases = json.loads((Path(__file__).parents[2] / "fixtures/schemas/contracts.json").read_text()) exports = exporter().export_schemas() - model_only = {"unsafe-integer", "strict-integer", "retry-cross-field", "diagnostic-unordered-source"} + model_only = { + "unsafe-integer", + "strict-integer", + "retry-cross-field", + "diagnostic-unordered-source", + "language-call-detach-on-failure", + } for case in cases: if case["name"] not in model_only: assert Draft202012Validator(exports[case["contract"]]).is_valid(case["value"]) is case["valid"], case[ diff --git a/tests/unit/compiler/test_scope_contract.py b/tests/unit/compiler/test_scope_contract.py new file mode 100644 index 00000000..c0b84840 --- /dev/null +++ b/tests/unit/compiler/test_scope_contract.py @@ -0,0 +1,102 @@ +# Copyright 2026 Firefly Software Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# Author: Firefly Software Foundation +# SPDX-License-Identifier: Apache-2.0 + +"""The frozen scope_at signature and result shape (language spec 14.3); M3 implements the analysis.""" + +import inspect + +import pytest +from pydantic import ValidationError + +from firefly_weave.compiler.catalog import CatalogSnapshot +from firefly_weave.compiler.scope import ScopeEntry, ScopeResult, scope_at + +SPEC_EXAMPLE = { + "found": True, + "evaluated": True, + "truncated": False, + "entries": [ + { + "ref": "/item", + "source": "item", + "loop": "notify", + "path": [], + "label": "Current item", + "breadcrumb": "Loop notify › Current item", + "schema": {"type": "object"}, + "types": ["object"], + "typeLabel": "Object", + "optional": False, + }, + { + "ref": "/index", + "source": "index", + "loop": "notify", + "path": [], + "label": "Item position (from 0)", + "breadcrumb": "Loop notify › Item position", + "schema": {"type": "integer", "minimum": 0, "maximum": 499}, + "types": ["integer"], + "typeLabel": "Whole number", + "optional": False, + }, + { + "ref": "/steps/check/output/eligible", + "source": "step", + "stepId": "check", + "stepKind": "action", + "path": ["eligible"], + "label": "eligible", + "breadcrumb": "check › eligible", + "schema": {"type": "boolean"}, + "types": ["boolean"], + "typeLabel": "Yes or no", + "optional": True, + }, + ], +} + + +def test_signature_is_frozen(): + signature = inspect.signature(scope_at) + assert list(signature.parameters) == ["document", "path", "catalog"] + assert signature.parameters["catalog"].kind is inspect.Parameter.KEYWORD_ONLY + assert signature.parameters["catalog"].default is None + assert signature.return_annotation in {ScopeResult, "ScopeResult"} + + +def test_result_shape_matches_the_studio_scope_entry_and_round_trips(): + result = ScopeResult.model_validate(SPEC_EXAMPLE) + assert result.model_dump(mode="json", by_alias=True) == SPEC_EXAMPLE + + +@pytest.mark.parametrize( + "entry", + [ + {**SPEC_EXAMPLE["entries"][0], "source": "loop"}, + {**SPEC_EXAMPLE["entries"][0], "loop": None}, + {**SPEC_EXAMPLE["entries"][0], "type_label": "Object"}, + {key: value for key, value in SPEC_EXAMPLE["entries"][0].items() if key != "optional"}, + ], +) +def test_malformed_entries_are_rejected(entry): + with pytest.raises(ValidationError): + ScopeEntry.model_validate(entry) + + +def test_analysis_is_not_available_before_the_loop_compiler(): + with pytest.raises(NotImplementedError): + scope_at({}, "/spec/output", catalog=CatalogSnapshot.empty()) diff --git a/tests/unit/compiler/test_unsupported_features.py b/tests/unit/compiler/test_unsupported_features.py new file mode 100644 index 00000000..dcd309ea --- /dev/null +++ b/tests/unit/compiler/test_unsupported_features.py @@ -0,0 +1,434 @@ +# Copyright 2026 Firefly Software Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# Author: Firefly Software Foundation +# SPDX-License-Identifier: Apache-2.0 + +"""The compiler reports each language construct whose feature it does not compile yet (language spec 18, M0).""" + +from collections.abc import Iterator +from typing import get_args + +import pytest +from pydantic import BaseModel + +import firefly_weave.compiler.language_support as support +from firefly_weave.compiler.api import compile_source, validate_authoring, validate_source +from firefly_weave.compiler.catalog import CatalogSnapshot +from firefly_weave.compiler.language_support import COMPILED_FEATURES, UnsupportedUse, unsupported_uses +from firefly_weave.contracts import definitions +from firefly_weave.contracts.language_features import ADVERTISED_FEATURES + +CODE = "WV-COMP-UNSUPPORTED_FEATURE" +CONCAT = {"op": {"name": "concat", "args": [{"literal": "Invoice "}, {"ref": "/input/number"}]}} +JOIN = {"op": {"name": "join", "args": [{"literal": ["a", "b"]}, {"literal": ", "}]}} +NULL = {"literal": None} +REF = {"ref": "/input/items"} + + +def workflow(steps: list[dict], output: dict | None = None) -> dict: + return { + "apiVersion": "weave/v1alpha1", + "kind": "Workflow", + "metadata": {"name": "w", "version": "1.0.0"}, + "spec": {"inputSchema": {}, "outputSchema": {}, "steps": steps, "output": output or {"literal": None}}, + } + + +EVERYTHING = workflow( + [ + { + "id": "notify", + "kind": "forEach", + "items": {"ref": "/input/items"}, + "body": { + "steps": [ + {"id": "label", "kind": "transform", "value": CONCAT}, + {"id": "child", "kind": "callWorkflow", "uses": "child@1.0.0", "with": {"object": {}}}, + ], + "output": {"ref": "/item"}, + }, + }, + { + "id": "branch", + "kind": "switch", + "cases": [{"when": {"literal": True}, "steps": [], "output": JOIN}], + "default": {"steps": [], "output": {"literal": None}}, + }, + ], + output={"object": {"text": {"array": [CONCAT]}}}, +) +EXPECTED = [ + ("/spec/output/object/text/array/0/op/name", "text.concat"), + ("/spec/steps/0/body/steps/0/value/op/name", "text.concat"), + ("/spec/steps/0/body/steps/1/kind", "flow.callWorkflow"), + ("/spec/steps/0/kind", "flow.forEach"), + ("/spec/steps/1/cases/0/output/op/name", "text.join"), +] + + +def _uses(document: dict) -> list[tuple[str, str]]: + return [(use.path, use.feature) for use in unsupported_uses(document)] + + +def _switch(cases: list[dict], default: dict | None = None) -> dict: + return {"id": "s", "kind": "switch", "cases": cases, "default": default or {"steps": [], "output": NULL}} + + +TRANSFORM = {"id": "inner", "kind": "transform", "value": CONCAT} +# One unsupported operator per case, at exactly the position named by the id. forEach and callWorkflow cases also +# carry their kind diagnostic, sorted by path. The scan never validates, so forEach and callWorkflow work before Task 5. +POSITIONS = [ + pytest.param( + workflow([{"id": "a", "kind": "action", "uses": "tool@1.0.0", "with": CONCAT}]), + [("/spec/steps/0/with/op/name", "text.concat")], + id="action-with", + ), + pytest.param( + workflow( + [{"id": "l", "kind": "llm", "uses": "model@1.0.0", "profile": "p", "prompt": CONCAT, "context": NULL}] + ), + [("/spec/steps/0/prompt/op/name", "text.concat")], + id="llm-prompt", + ), + pytest.param( + workflow( + [{"id": "l", "kind": "llm", "uses": "model@1.0.0", "profile": "p", "prompt": NULL, "context": CONCAT}] + ), + [("/spec/steps/0/context/op/name", "text.concat")], + id="llm-context", + ), + pytest.param( + workflow([{"id": "t", "kind": "transform", "value": CONCAT}]), + [("/spec/steps/0/value/op/name", "text.concat")], + id="transform-value", + ), + pytest.param( + workflow([{"id": "d", "kind": "decisionTable", "uses": "rules@1.0.0", "with": CONCAT}]), + [("/spec/steps/0/with/op/name", "text.concat")], + id="decision-table-with", + ), + pytest.param( + workflow([{"id": "h", "kind": "humanTask", "assignment": "a", "title": CONCAT, "context": NULL}]), + [("/spec/steps/0/title/op/name", "text.concat")], + id="human-task-title", + ), + pytest.param( + workflow([{"id": "h", "kind": "humanTask", "assignment": "a", "title": NULL, "context": CONCAT}]), + [("/spec/steps/0/context/op/name", "text.concat")], + id="human-task-context", + ), + pytest.param( + workflow([{"id": "f", "kind": "forEach", "items": CONCAT, "body": {"steps": [], "output": NULL}}]), + [("/spec/steps/0/items/op/name", "text.concat"), ("/spec/steps/0/kind", "flow.forEach")], + id="for-each-items", + ), + pytest.param( + workflow([{"id": "c", "kind": "callWorkflow", "uses": "child@1.0.0", "with": CONCAT}]), + [("/spec/steps/0/kind", "flow.callWorkflow"), ("/spec/steps/0/with/op/name", "text.concat")], + id="call-workflow-with", + ), + pytest.param( + workflow([{"id": "c", "kind": "callWorkflow", "uses": "child@1.0.0", "with": NULL, "businessKey": CONCAT}]), + [("/spec/steps/0/businessKey/op/name", "text.concat"), ("/spec/steps/0/kind", "flow.callWorkflow")], + id="call-workflow-business-key", + ), + pytest.param( + workflow([_switch([{"when": CONCAT, "steps": [], "output": NULL}])]), + [("/spec/steps/0/cases/0/when/op/name", "text.concat")], + id="switch-case-when", + ), + pytest.param( + workflow([_switch([{"when": NULL, "steps": [], "output": JOIN}])]), + [("/spec/steps/0/cases/0/output/op/name", "text.join")], + id="switch-case-output", + ), + pytest.param( + workflow([_switch([{"when": NULL, "steps": [TRANSFORM], "output": NULL}])]), + [("/spec/steps/0/cases/0/steps/0/value/op/name", "text.concat")], + id="switch-case-nested-step", + ), + pytest.param( + workflow([_switch([{"when": NULL, "steps": [], "output": NULL}], {"steps": [], "output": JOIN})]), + [("/spec/steps/0/default/output/op/name", "text.join")], + id="switch-default-output", + ), + pytest.param( + workflow([_switch([{"when": NULL, "steps": [], "output": NULL}], {"steps": [TRANSFORM], "output": NULL})]), + [("/spec/steps/0/default/steps/0/value/op/name", "text.concat")], + id="switch-default-nested-step", + ), + pytest.param( + workflow( + [{"id": "p", "kind": "parallel", "branches": {"a/b~c": {"steps": [], "output": CONCAT}}, "concurrency": 2}] + ), + [("/spec/steps/0/branches/a~1b~0c/output/op/name", "text.concat")], + id="parallel-branch-output-escaped-name", + ), + pytest.param( + workflow( + [ + { + "id": "p", + "kind": "parallel", + "branches": {"main": {"steps": [TRANSFORM], "output": NULL}}, + "concurrency": 2, + } + ] + ), + [("/spec/steps/0/branches/main/steps/0/value/op/name", "text.concat")], + id="parallel-branch-nested-step", + ), + pytest.param( + workflow([{"id": "f", "kind": "forEach", "items": REF, "body": {"steps": [], "output": CONCAT}}]), + [("/spec/steps/0/body/output/op/name", "text.concat"), ("/spec/steps/0/kind", "flow.forEach")], + id="for-each-body-output", + ), + pytest.param( + workflow([{"id": "f", "kind": "forEach", "items": REF, "body": {"steps": [TRANSFORM], "output": NULL}}]), + [("/spec/steps/0/body/steps/0/value/op/name", "text.concat"), ("/spec/steps/0/kind", "flow.forEach")], + id="for-each-body-nested-step", + ), + pytest.param( + workflow([], output=CONCAT), + [("/spec/output/op/name", "text.concat")], + id="workflow-output", + ), + pytest.param( + {"kind": "DecisionTable", "spec": {"rules": [{"id": "r", "when": CONCAT, "output": NULL}]}}, + [("/spec/rules/0/when/op/name", "text.concat")], + id="decision-rule-when", + ), + pytest.param( + {"kind": "DecisionTable", "spec": {"rules": [{"id": "r", "when": NULL, "output": CONCAT}]}}, + [("/spec/rules/0/output/op/name", "text.concat")], + id="decision-rule-output", + ), + pytest.param( + { + "kind": "DecisionTable", + "spec": {"rules": [{"id": "r", "when": NULL, "output": NULL}], "defaultOutput": JOIN}, + }, + [("/spec/defaultOutput/op/name", "text.join")], + id="decision-default-output", + ), +] + + +def test_nothing_is_compiled_or_advertised_that_the_compiler_cannot_compile(): + assert not COMPILED_FEATURES + assert set(ADVERTISED_FEATURES) <= COMPILED_FEATURES + + +def test_scan_finds_every_kind_and_operator_at_its_field(): + assert [(use.path, use.feature) for use in unsupported_uses(EVERYTHING)] == EXPECTED + + +def test_literal_data_and_other_documents_are_never_read_as_syntax(): + literal = workflow([], output={"literal": {"op": {"name": "concat", "args": []}}}) + assert unsupported_uses(literal) == [] + assert unsupported_uses({"kind": "Workflow", "spec": {"steps": [7, {"kind": 3}], "output": "x"}}) == [] + assert unsupported_uses({"kind": "Action", "spec": {}}) == [] + assert unsupported_uses({"kind": "Workflow"}) == [] + + +def test_decision_table_rules_are_scanned(): + table = { + "kind": "DecisionTable", + "spec": {"rules": [{"id": "r", "when": {"literal": True}, "output": CONCAT}], "defaultOutput": JOIN}, + } + assert [(use.path, use.feature) for use in unsupported_uses(table)] == [ + ("/spec/defaultOutput/op/name", "text.join"), + ("/spec/rules/0/output/op/name", "text.concat"), + ] + + +@pytest.mark.parametrize(("document", "expected"), POSITIONS) +def test_every_expression_position_is_scanned(document: dict, expected: list[tuple[str, str]]): + assert _uses(document) == expected + + +def test_long_numeric_keys_are_scanned_without_integer_conversion(): + long_key = "1" * 5000 + document = workflow([], output={"object": {long_key: CONCAT}}) + assert _uses(document) == [(f"/spec/output/object/{long_key}/op/name", "text.concat")] + + +def test_numeric_segments_keep_numeric_order_at_any_length(): + long_key = "1" * 5000 + document = workflow([], output={"object": {long_key: CONCAT, "2": CONCAT}}) + assert _uses(document) == [ + ("/spec/output/object/2/op/name", "text.concat"), + (f"/spec/output/object/{long_key}/op/name", "text.concat"), + ] + + +def _classes(annotation: object) -> Iterator[type[BaseModel]]: + """Pydantic models named by an annotation, looking through generics such as list, dict and OmissionOnly.""" + if isinstance(annotation, type) and issubclass(annotation, BaseModel): + yield annotation + return + for argument in get_args(annotation): + yield from _classes(argument) + + +def _expression_fields(model: type[BaseModel]) -> set[str]: + """Wire names of the fields typed as Expression, bare or wrapped (OmissionOnly[Expression], list, dict).""" + return { + field.alias or name + for name, field in model.model_fields.items() + if field.annotation is definitions.Expression or definitions.Expression in get_args(field.annotation) + } + + +def _holds_blocks(model: type[BaseModel]) -> bool: + return any( + issubclass(model_class, definitions.Branch) + for field in model.model_fields.values() + for model_class in _classes(field.annotation) + ) + + +def _step_models() -> tuple[type[BaseModel], ...]: + return get_args(get_args(definitions.Step.__value__)[0]) + + +def _kind(model: type[BaseModel]) -> str: + return get_args(model.model_fields["kind"].annotation)[0] + + +def _definition_models() -> list[type[BaseModel]]: + return [ + value + for value in vars(definitions).values() + if isinstance(value, type) and issubclass(value, BaseModel) and value.__module__ == definitions.__name__ + ] + + +# Expression grammar models: the operator scan walks them as expressions, not as scanned positions. +_EXPRESSION_GRAMMAR = { + definitions.LiteralExpression, + definitions.RefExpression, + definitions.ObjectExpression, + definitions.ArrayExpression, + definitions.Operation, + definitions.OpExpression, +} + + +def test_every_expression_field_of_every_step_kind_is_scanned(): + assert {"action", "transform", "switch", "parallel"} <= {_kind(model) for model in _step_models()} + for model in _step_models(): + assert _expression_fields(model) == set(support._EXPRESSIONS.get(_kind(model), ())), _kind(model) + + +def test_only_kinds_the_scan_walks_hold_blocks(): + holders = {_kind(model) for model in _step_models() if _holds_blocks(model)} + assert {"switch", "parallel"} <= holders <= {"switch", "parallel", "forEach"} + + +def test_every_expression_field_outside_steps_is_scanned(): + # Tripwire: a new Expression-typed model fails here until the scan and this table both learn its positions. + steps = set(_step_models()) + holders = { + model.__name__: _expression_fields(model) + for model in _definition_models() + if model not in steps | _EXPRESSION_GRAMMAR and _expression_fields(model) + } + assert holders == { + "Branch": {"output"}, + "SwitchCase": {"output", "when"}, + "DecisionRule": {"when", "output"}, + "DecisionTableSpec": {"defaultOutput"}, + "WorkflowSpec": {"output"}, + } + + +def test_message_names_the_construct_and_its_feature(): + use = UnsupportedUse("/spec/steps/0/kind", "The forEach step", "flow.forEach") + assert use.message == ( + "The forEach step needs the language feature flow.forEach, " + "which this version of Firefly Weave does not compile yet." + ) + + +@pytest.mark.parametrize( + "check", + [ + lambda d: compile_source(d, format="object", catalog=CatalogSnapshot.from_definitions([])), + lambda d: validate_source(d, format="object"), + lambda d: validate_authoring(d, format="object"), + ], +) +def test_every_compiler_entry_point_stops_at_constructs_it_cannot_compile(monkeypatch, check): + # Pretend two existing constructs need a feature, so the wiring is proven before new constructs parse. + monkeypatch.setattr(support, "KIND_FEATURES", {"wait": "flow.forEach"}) + monkeypatch.setattr(support, "OPERATOR_FEATURES", {"exists": "text.concat"}) + document = workflow( + [{"id": "pause", "kind": "wait", "durationSeconds": 5}], + output={"op": {"name": "exists", "args": [{"ref": "/input/a"}]}}, + ) + result = check(document) + assert not result.ok and result.artifact is None + assert [(d.code, d.path, d.severity, d.stage) for d in result.diagnostics] == [ + (CODE, "/spec/output/op/name", "error", "semantic"), + (CODE, "/spec/steps/0/kind", "error", "semantic"), + ] + + +@pytest.mark.parametrize( + "check", + [ + lambda d: compile_source(d, format="object", catalog=CatalogSnapshot.from_definitions([])), + lambda d: validate_source(d, format="object"), + lambda d: validate_authoring(d, format="object"), + ], +) +def test_new_constructs_parse_and_every_entry_point_reports_them(check): + result = check(EVERYTHING) + assert not result.ok and result.artifact is None + assert {d.code for d in result.diagnostics} == {CODE} + assert sorted(d.path for d in result.diagnostics) == [path for path, _ in EXPECTED] + + +def test_decision_tables_with_text_operators_do_not_compile(): + table = { + "apiVersion": "weave/v1alpha1", + "kind": "DecisionTable", + "metadata": {"name": "d", "version": "1.0.0"}, + "spec": { + "inputSchema": {}, + "outputSchema": {}, + "hitPolicy": "first", + "rules": [{"id": "r", "when": {"literal": True}, "output": CONCAT}], + }, + } + result = compile_source(table, format="object", catalog=CatalogSnapshot.from_definitions([])) + assert [(d.code, d.path) for d in result.diagnostics] == [(CODE, "/spec/rules/0/output/op/name")] + + +def test_callable_workflows_compile_because_callable_only_declares_an_interface(): + document = workflow([]) + document["spec"]["callable"] = {"allowedCallers": ["order-intake"]} + result = compile_source(document, format="object", catalog=CatalogSnapshot.from_definitions([])) + assert result.ok, [d.code for d in result.diagnostics] + + +def test_diagnostics_stay_capped_on_large_documents(): + many = workflow([{"id": f"t{i}", "kind": "transform", "value": CONCAT} for i in range(150)]) + result = validate_authoring(many, format="object") + assert len(result.diagnostics) == 100 + assert result.truncated and result.omitted_count > 0 + findings = [d for d in result.diagnostics if d.code != "WV-COMP-DIAGNOSTICS_TRUNCATED"] + assert len(findings) == len(result.diagnostics) - 1 # exactly one truncation marker + assert all(d.code == CODE for d in findings) diff --git a/tests/unit/test_studio_bridge_families.py b/tests/unit/test_studio_bridge_families.py index 1467878d..4eb5209d 100644 --- a/tests/unit/test_studio_bridge_families.py +++ b/tests/unit/test_studio_bridge_families.py @@ -78,6 +78,7 @@ def pair(browser): ("GET", f"{PROJECT}/connector-descriptors/Pets_API.v1"), ("GET", f"{PROJECT}/drafts"), ("GET", f"{PROJECT}/workflows/{RESOURCE}"), + ("GET", f"{PROJECT}/language"), ], ) def test_studio_journeys_are_bridged(tmp_path, method, path): @@ -109,6 +110,7 @@ def test_studio_journeys_are_bridged(tmp_path, method, path): ("POST", f"{OTHER_PROJECT}/debug/sessions", 403), ("GET", f"{OTHER_PROJECT}/connector-descriptors", 403), ("GET", f"{OTHER_PROJECT}/connector-descriptors/weave-http-v2", 403), + ("GET", f"{OTHER_PROJECT}/language", 403), ("GET", ENVIRONMENT.replace("000000000003", "000000000009") + "/worker-releases", 403), ], )