Compile workflow calls against the exact version they call, with activation pins

CHANGELOG.md

@@ -20,6 +20,25 @@ SPDX-License-Identifier: Apache-2.0
 
 ## Unreleased
 
+- The compiler checks and compiles **Call a workflow** (`callWorkflow`) against
+  the exact published version it calls: that version must opt in with
+  `spec.callable` (any workflow, or only `allowedCallers`), `with` is checked
+  against its input schema, a `businessKey` must be text of 1 to 200 characters,
+  and the step returns the called workflow's output, a `{status, runId, output,
+  error}` envelope with `onFailure: continue`, or `null` with `mode: detach`. New
+  diagnostics `WV-COMP-UNKNOWN_WORKFLOW`, `CALL_NOT_CALLABLE`, `CALL_RECURSION`,
+  `CALL_DEPTH` (more than 8 calls deep, the new compiler limit
+  `max_call_depth`), and `CALL_MODE` point at the field to fix. The called
+  version is locked as an interface dependency: the caller's artifact carries
+  that version's definition, so its digest matches the published one, but
+  compiling, importing, and activating the caller read only its input and output
+  schemas. Activation takes `workflow_activation_ids`, one live
+  activation of each called version in the same environment, and refuses wrong
+  or missing pins with `422 WV-READINESS-CALL`. This platform does not start child
+  runs yet, so it refuses to publish, import, or activate a workflow that calls
+  another with `WV-IR-UNSUPPORTED`; workflows without calls compile to the same
+  bytes as before. `weave workflow explain` and `weave workflow graph` show calls,
+  and the SDK's `WorkflowBuilder.with_callable()` writes `spec.callable`.
 - Loops run. A **Loop over items** (`forEach`) step runs its body once per
   element, `concurrency` at a time, in one run, and its output lists one result
   per element in element order, whatever order the iterations finish in. Each

docs/concepts.md

@@ -305,7 +305,7 @@ Use this table to translate what you know:
 | Error end event, terminate end event | **Fail** (`fail`) | Ends the whole run as failed with your code and message, including any other parallel branches |
 | Error boundary event | Not in the language | A failed Action retries when that is safe; otherwise the run is suspended with an incident for an operator. To branch on a business outcome, return it in the Action's output and test it with a Decision |
 | Message throw or end event | **Call an action**, or an integration event subscription | A subscription notifies another system when a run's status changes |
-| Call activity, subprocess | **Call a workflow** (`callWorkflow`) | Defined in the [workflow language](contracts.md#steps), but this version does not compile or run it: the compiler reports `WV-COMP-UNSUPPORTED_FEATURE`. Until then, keep the part inside the same workflow, or make it a separate workflow that your product starts |
+| Call activity, subprocess | **Call a workflow** (`callWorkflow`) | Defined in the [workflow language](contracts.md#steps). This version compiles and checks it against the exact version it calls, but the platform does not run calls yet and refuses to publish a workflow that uses one (`WV-IR-UNSUPPORTED`). Until then, keep the part inside the same workflow, or make it a separate workflow that your product starts |
 | Loop, multi-instance activity | **Loop over items** (`forEach`) | Runs its body once per element of a list, `concurrency` at a time, inside the same run, and collects one result per element in element order; see [loops at run time](reference/compiler.md#loops-at-run-time). A failure in one element stops the whole run, as in a Parallel |
 | Compensation | Not in the language | Model compensating Actions as ordinary steps; Weave never undoes an external effect |
 | Instance migration | Not available | A running run keeps its version; new runs use the new activation |

docs/contracts.md

@@ -220,10 +220,9 @@ each need a language feature: `flow.forEach`, `flow.callWorkflow`, `text.concat`
 and `text.join`. This version compiles and runs `concat`, `join`, and `forEach`;
 see [loops at run time](reference/compiler.md#loops-at-run-time). A platform
 that does not list `flow.forEach` refuses to publish, import, or activate a
-workflow that uses a loop (`WV-IR-UNSUPPORTED`). The shape of
-`callWorkflow` is final, so `load_definition` accepts it, but this version of the
-compiler reports `WV-COMP-UNSUPPORTED_FEATURE` at each use instead of compiling
-it. A platform runs a construct only when its
+workflow that uses a loop (`WV-IR-UNSUPPORTED`). It also compiles
+`callWorkflow` (see **Workflow calls** below) and refuses to publish a workflow
+that calls another until it runs calls. A platform runs a construct only when its
 [language manifest](#language-manifest) lists the feature. Step IDs can never
 contain `[`, `#`, or `~`, which keeps
 [instance keys](reference/compiler.md#instance-keys) unambiguous.
@@ -246,6 +245,37 @@ iterations; the compiler suggests a lower `maxItems` for the inner loop when the
 need more. The [compiler reference](reference/compiler.md#troubleshooting) lists
 the diagnostics.
 
+**Workflow calls.** A `callWorkflow` step names an exact published workflow
+version in `uses`, and that version must allow the call with `spec.callable`:
+`{}` allows any workflow in the project, and `allowedCallers` limits it to the
+named workflows. `with` is checked against the called version's `inputSchema`,
+and `businessKey`, when set, must be text of 1 to 200 characters. The step's
+output depends on how it runs:
+
+| `mode` and `onFailure` | Output |
+| --- | --- |
+| `wait`, without `onFailure` or with `stop` | The called version's output, checked against its `outputSchema` |
+| `wait` with `continue` | `{status, runId, output, error}`: `status` (`succeeded`, `failed`, `cancelled`, or `timed_out`) and `runId` are always present; read `output` and `error` with `coalesce`, since either may be missing |
+| `detach` | `null`; the step starts the call and continues |
+
+A call cannot lead back to the calling version, and the longest chain of calls a
+workflow starts may be at most 8 deep (`max_call_depth`). The compiler locks the
+called version as a `Workflow` dependency of the caller and reads only its input
+and output schemas: the called workflow's own steps, actions, and connections
+stay with its own activation. The caller's artifact still includes the called
+version's definition, so the dependency digest matches the published one. Each
+called version is included once, without its own dependencies, and counts toward
+the 32 MiB artifact limit. Anyone who can read the caller's version or export
+its artifact can see the called version's steps; secret values never appear,
+because literals bound to secret inputs are refused. When you activate a
+workflow that calls others, `workflow_activation_ids` pins one activation of
+each called version in the same environment, by reference (for example
+`{"notify-customer@1.0.0": "ACTIVATION_UUID"}`). Activation refuses a missing or
+extra pin, a pin to another workflow or version, a retired activation or
+version, a version that does not allow this workflow, and a chain of calls
+deeper than 8 with `422 WV-READINESS-CALL`; the activation records each pin in
+`workflow_call_pins` and the longest chain in `call_depth`.
+
 **Human tasks** need an environment assignment binding at activation, and
 completing one requires the current claim and task permission. A workflow with a
 human task compiles to IR version `weave/ir-v1alpha2` or higher, because the
@@ -383,6 +413,7 @@ The operator's `Limits()` defaults bound compilation:
 | `max_loop_items` | 10,000 | The largest `maxItems` a loop may declare |
 | `max_loop_depth` | 3 | Loops nested inside one another |
 | `max_run_iterations` | 100,000 | Iterations all loops of one workflow may need, each loop multiplied by the `maxItems` of the loops around it |
+| `max_call_depth` | 8 | The longest chain of workflow calls one workflow may start |
 
 Definitions cannot change these budgets. Building a model alone does not apply
 them; the compiler does.
@@ -458,7 +489,9 @@ backend is absent. The `integration` and `e2e` pytest markers are registered.
 | A field is rejected although it looks right | Field names are camelCase and case-sensitive; snake_case keys are rejected | Use the published name, such as `timeoutSeconds` |
 | `null` is rejected for an optional field | Optional fields may be omitted but not set to null | Remove the field |
 | A step's `retry` or `timeoutSeconds` is rejected | Retry and per-attempt timeout belong to the action definition | Move them to the action; use `spec.timeoutSeconds` for a workflow-wide timeout |
-| `WV-COMP-UNSUPPORTED_FEATURE` | The document uses `callWorkflow`, which this version of the compiler does not compile yet | Keep the document for a later version, or replace the construct with the steps the compiler supports |
+| `WV-COMP-UNSUPPORTED_FEATURE` | The document uses a construct whose language feature this compiler does not compile; the message names the feature | Upgrade the CLI or SDK that compiles it, or replace the construct with steps this compiler supports |
+| `WV-IR-UNSUPPORTED` when publishing a workflow with `callWorkflow` | The compiler accepts the call, but this platform does not run workflow calls yet, so it refuses to publish it; `result.missing_features` is `["flow.callWorkflow"]` | Keep the workflow as a draft until the platform lists `flow.callWorkflow` in its language manifest; the called workflow itself publishes normally |
+| `WV-READINESS-CALL` on activation | A called workflow version has no pinned activation; a pin names something the workflow does not call; the pinned activation or its version is retired, runs another version, or does not allow this workflow; the pinned version differs from the one the workflow was compiled against; or the chain of calls is deeper than 8 | Pin one live activation of each called version in `workflow_activation_ids` |
 | `WV-IR-UNSUPPORTED` when publishing a workflow with `forEach` | The platform you publish to is older and does not list `flow.forEach` in its language manifest; `result.missing_features` is `["flow.forEach"]` | Upgrade that platform, or keep the workflow as a draft until it lists the feature |
 | `WV-DECISION-OPERATOR` | A decision table rule uses `concat` or `join` | Build the text in a Transform step before the table and pass it in as input |
 | `WV-IR-UNSUPPORTED` on activation, a catalog read, or a run read | The platform does not list an IR version or a language feature the workflow uses; `result.missing_features` names the missing features and is empty when the IR version itself is unknown. A catalog list shows such a version as an unavailable item with `reason: ir_unsupported` and the same `missing_features`. A run that uses such a feature waits for an upgrade; signals and task results for it answer the same way, and run lists show it as unavailable | Upgrade the platform, or activate a version that does not use the feature |

docs/reference/compiler.md

@@ -230,8 +230,9 @@ Executables also define a sorted `features` list of the language features they
 need (`text.concat`, `text.join`, `flow.forEach`, `flow.callWorkflow`,
 `ai.agent`, `ai.memory`). It is omitted when empty, so existing digests do not
 change. A workflow that uses `concat` or `join` in any expression compiles to
-`weave/ir-v1alpha4` with `text.concat`, `text.join`, or both, and a workflow with
-a `forEach` step lists `flow.forEach`; v1alpha4 always has a non-empty list, and
+`weave/ir-v1alpha4` with `text.concat`, `text.join`, or both, a workflow with
+a `forEach` step lists `flow.forEach`, and a workflow with a `callWorkflow` step
+lists `flow.callWorkflow`; v1alpha4 always has a non-empty list, and
 the validator recomputes the version and the list from the graph. The version is the highest level any construct needs, so a workflow
 with a human task and `join` is v1alpha4, not v1alpha2. A platform imports and
 activates such an artifact only when its capabilities list every feature in
@@ -252,6 +253,14 @@ locks:
   code.
 - Schema guards and signal nodes reference interned schemas instead of copying
   them into every step.
+- A `callWorkflow` node refers to its called workflow's digest, locked as a
+  `Workflow` dependency whose document is that version's definition. The caller
+  uses only its `inputSchema`, `outputSchema`, and the named schemas they
+  reference: import never rechecks the called workflow's steps, and its actions
+  and connections are not dependencies of the caller. The node carries `with`,
+  `mode` (`wait` or `detach`), and, when set, `onFailure` and `businessKey`; the
+  guards `call_input` (at `…/with`) and `call_output` (at the step) hold the
+  called version's schemas, and a called version must allow the caller by name.
 
 Action and connector executables keep their strict manifest spec and
 dependencies; they have no workflow graph.
@@ -611,7 +620,13 @@ is not classified. Safe ordinary artifacts keep their canonical digests.
 | Incident `WV-RUNTIME-LOOP-LIMIT` on a loop | `items` had more elements than the loop's `maxItems` | Terminate the run; raise `maxItems` (at most 10,000) or split the list before the loop |
 | Incident `WV-RUNTIME-LOOP-OUTPUT` on a loop | The collected results would exceed the 1 MiB payload limit | Terminate the run; return only what later steps need from the body, or use `collect: nonNull` to drop empty results |
 | `WV-COMP-UNKNOWN_ACTION` (or `_CONNECTOR`, `_TASK`, `_ADAPTER`) | The catalog does not contain that exact reference | Add the exact version to the catalog, or fix the reference |
-| `WV-COMP-UNSUPPORTED_FEATURE` | The workflow uses a construct whose language feature this compiler does not compile yet (`callWorkflow`); the message names the feature | Keep the document for a later version, or use the steps and operators this version compiles |
+| `WV-COMP-UNSUPPORTED_FEATURE` | The workflow uses a construct whose language feature this compiler does not compile; the message names the feature | Upgrade the CLI or SDK that compiles it, or use the steps and operators this version compiles |
+| `WV-COMP-UNKNOWN_WORKFLOW` at `…/uses` | The catalog does not contain the exact workflow version a call names | Publish that version first, or fix the reference |
+| `WV-COMP-CALL_NOT_CALLABLE` at `…/uses` | The called version has no `spec.callable`, or its `allowedCallers` does not name this workflow | Publish a new version of the called workflow whose `callable` allows this workflow |
+| `WV-COMP-CALL_RECURSION` at `…/uses` | Following the calls of the called version leads back to this exact version | Call a workflow that does not lead back to this one |
+| `WV-COMP-CALL_DEPTH` at `…/uses` | The chain of calls this call starts is deeper than 8 (`max_call_depth`), or never ends | Call a workflow that starts fewer calls of its own |
+| `WV-COMP-CALL_MODE` at `…/onFailure` | `onFailure` is set on a call with `mode: detach` | Remove `onFailure`, or set `mode: wait` |
+| `WV-COMP-REFERENCE_PRESENCE` reading `/steps/<call>/output/output` | A call with `onFailure: continue` has a child output only when the child succeeded | Read it with `coalesce`, or branch on `/steps/<call>/output/status` first |
 | `WV-COMP-LOOP_ITEMS` at `…/items` | A loop's `items` can never be a list | Point `items` at a list, such as an array field of the input or of a step's output |
 | `WV-COMP-UNKNOWN_COMPATIBILITY` at `…/items` | The compiler cannot prove that `items` is a list; the run checks it | Declare the field as an array where it comes from |
 | `WV-COMP-LOOP_SCOPE` at `…/ref` | `/item`, `/index`, or `/loops/<id>` is read outside a loop body, in the loop's own `items`, or names a loop that does not enclose the expression | Read loop data only inside the loop's body; use `/loops/<id>/item` for an enclosing loop |

examples/language/order-intake.workflow.yaml

@@ -14,8 +14,8 @@
 # Author: Firefly Software Foundation
 # SPDX-License-Identifier: Apache-2.0
 
-# Call a workflow and wait for its result, then start another call without waiting. This version compiles
-# concat and join but reports "not yet supported" for callWorkflow; the document shape is final.
+# Call a workflow and wait for its result, then start another call without waiting. Both calls compile
+# against notify-customer@1.0.0; a platform publishes this workflow once it runs workflow calls.
 apiVersion: weave/v1alpha1
 kind: Workflow
 metadata:
@@ -26,7 +26,7 @@ spec:
     type: object
     properties:
       customerId: {type: string}
-      orderId: {type: string}
+      orderId: {type: string, minLength: 1, maxLength: 200}
       tags:
         type: array
         items: {type: string}

scripts/language_fixtures.py

@@ -18,7 +18,8 @@
 
 The fixtures pin the stable language contracts: instance-key parse and format cases with the
 ``INSTANCE_KEY_PATTERN`` expression, the text conversion table for ``concat`` and ``join``, template segments
-with their saved form, ``scope_at`` suggestions inside and around loops, and the language manifest snapshot.
+with their saved form, ``scope_at`` suggestions inside and around loops and after workflow calls, and the language
+manifest snapshot.
 Expected values are written here by hand; the generator refuses to write a fixture that the implementation
 disagrees with. Scope cases pin each suggestion's reference, source, loop and type label by hand and record the
 rest of each entry as ``scope_at`` returns it.
@@ -281,6 +282,18 @@ def _loop(identifier: str, items: str, *steps: dict[str, Any], **options: Any) -
 _LINES = _loop(
     "line", "/item", {"id": "pair", "kind": "transform", "value": {"ref": "/loops/batch/index"}}, maxItems=50
 )
+
+
+def _call(identifier: str, **options: Any) -> dict[str, Any]:
+    return {
+        "id": identifier,
+        "kind": "callWorkflow",
+        "uses": "notify-customer@1.0.0",
+        "with": {"ref": "/input"},
+        **options,
+    }
+
+
 _INPUT_ROWS = [
     ("/input", "input", None, "Group"),
     ("/input/invoices", "input", None, "List of groups"),
@@ -346,6 +359,39 @@ def _loop(identifier: str, items: str, *steps: dict[str, Any], **options: Any) -
         [*_INPUT_ROWS, ("/item", "item", "batch", "List of groups"), ("/index", "index", "batch", "Whole number")],
     ),
     ("a field that is not an expression has no scope", _workflow(_NOTIFY), "/spec/steps/0/maxItems", None),
+    (
+        "a call's input sees what is visible before the call",
+        _workflow(_call("notify")),
+        "/spec/steps/0/with",
+        _INPUT_ROWS,
+    ),
+    (
+        "after a call that waits, its result is the called workflow's output, unknown without a catalog",
+        _workflow(_call("notify"), output={"ref": "/steps/notify/output"}),
+        "/spec/output",
+        [*_INPUT_ROWS, ("/steps/notify/output", "step", None, "Any value")],
+    ),
+    (
+        "a call that continues on failure returns its status, run ID, the child's output and the error",
+        _workflow(_call("notify", onFailure="continue"), output={"ref": "/steps/notify/output"}),
+        "/spec/output",
+        [
+            *_INPUT_ROWS,
+            ("/steps/notify/output", "step", None, "Group"),
+            ("/steps/notify/output/status", "step", None, "Choice"),
+            ("/steps/notify/output/runId", "step", None, "Text"),
+            ("/steps/notify/output/output", "step", None, "Any value"),
+            ("/steps/notify/output/error", "step", None, "Group"),
+            ("/steps/notify/output/error/code", "step", None, "Text"),
+            ("/steps/notify/output/error/message", "step", None, "Text"),
+        ],
+    ),
+    (
+        "a call that does not wait returns nothing",
+        _workflow(_call("notify", mode="detach"), output={"ref": "/steps/notify/output"}),
+        "/spec/output",
+        [*_INPUT_ROWS, ("/steps/notify/output", "step", None, "Empty")],
+    ),
 )