Add mutable runtime recipe variables with step-time resolution and explicit nested export

[Copilot]

Linked issue

Implements general-purpose mutable runtime variables in a separate PR from #1179; this branch does not include or depend on attachment/feature-branch changes from that draft.

What changes

Adds an opt-in runtime variable system so recipes can set/mutate values during execution, bind action return values, and consume current values in later steps (including on_start -> read/wrangles/write) without changing existing static ${...} behavior.

• Runtime variable model

  • New execution-time references: ${runtime.name} and ${runtime.name.path}
  • Runtime resolution happens per-step (not eagerly at load)
  • Escaped literal support for runtime templates (\${runtime...})
  • Per-run mutable state cloning to avoid mutating caller-owned inputs

• Mutation + result binding

  • New variables run action:
    • set (atomic assignment)
    • update (deep merge for dict values)
    • mark_secret
    • inspect / max_items
  • New framework run parameter: result_variable to capture connector/action return values into runtime state (excluded from connector args)

• Isolation and export semantics

  • Runtime state is isolated per recipe.run() invocation
  • Nested recipes inherit a snapshot by default
  • Explicit parent export via export_runtime_variables on recipe connector
  • Deterministic conflict handling: export_conflict_policy: error|overwrite
  • Protected names blocked from assignment (row_count, column_count, columns, df, recipe_variables, applied_permission_group, internal runtime keys)

• Inspection/logging safety

  • log wrangle supports runtime_variables and runtime_max_items
  • Redaction for marked secrets + sensitive-name heuristics + signed URL query secrets
  • Bounded/truncated rendering for large containers and non-text payloads

• Schema/docs/tests

  • Updated run common schema for result_variable
  • Added connector schema for variables
  • README section documents syntax, compatibility boundaries, and nested/parallel semantics
  • Added focused tests for sequencing, reassignment, typed values, missing/escaped refs, no-input startup, nested export conflicts, branch isolation, redaction, and caller-input isolation

run:
  on_start:
    - custom.build_reference:
        result_variable: upload_ref
    - variables:
        update:
          upload_ref:
            status: ready
        inspect:
          - upload_ref.file_id

wrangles:
  - create.column:
      output: reference_id
      value: ${runtime.upload_ref.file_id}

How it was verified

Focused suites covering runtime behavior and regressions:

• tests/recipes/test_run.py
• tests/recipes/test_variables.py
• tests/connectors/test_recipe.py
• tests/connectors/test_concurrent.py
• tests/connectors/test_matrix.py
• targeted tests/recipes/wrangles/test_main.py log/python/matrix cases
• parallel_validation run (CodeQL clean; code-review subtool unavailable in this environment)

Compatibility and risk

• Backward compatible by default: existing ${variable} remains load-time/static; mutable behavior is opt-in via ${runtime...} and variables action.
• System/computed values remain read-only and separate from assignable runtime state.
• No global/process-wide runtime store introduced; isolation is preserved across parallel branches and concurrent recipe.run() calls unless explicitly exported.
• Rollback is low-risk: remove runtime-resolution/mutation paths and keep legacy static templating behavior.

Ready-for-review checklist

• One human delivery owner is assigned
• The linked issue and intended milestone are correct
• The branch is current with main and has no merge conflicts
• Focused tests pass
• New or changed behavior has direct test coverage
• Documentation/schema/configuration is updated where applicable
• The PR contains no unrelated changes
• The PR description reflects the branch's current scope and latest validation
• One primary reviewer is requested only when this PR is ready

See the pull request workflow.