Add ClassifierRetryPolicy and keep LLMRetryPolicy as the text-model policy it was

providers/common/ai/docs/changelog.rst

@@ -26,31 +26,23 @@ Changelog
 ---------
 
 .. note::
-  ``LLMRetryPolicy`` now asks the model only which category a failure is; whether that
-  category is retried, and after how long, comes from the policy's ``categories`` table
-  (``ErrorCategory(description, retry, delay, min_confidence)``), not from the model.
-  ``ErrorClassification`` and its ``should_retry``, ``suggested_delay_seconds`` and
-  ``reasoning`` fields are removed, so a Dag file that imports the class fails to parse,
-  and with it every Dag in that file. The new public names are ``ErrorCategory`` and
-  ``DEFAULT_CATEGORIES``.
-
-  A policy built with only ``llm_conn_id`` classifies into the same seven categories with
-  the same retry/fail split and the same 60s/10s/30s delays. The delays are now fixed by
-  the table rather than chosen by the model, so an error the model previously answered
-  with its own delay now waits the category's. Custom ``instructions`` that named a delay,
-  said "do NOT retry", or introduced category names outside the seven still parse but no
-  longer steer anything: the model is constrained to ``categories``, so a name of your own
-  is either mapped onto the nearest default or rejected by the schema and sent to the
-  fallback path. The 0.9.0 guide's Snowflake example asked for ``rate_limit`` after 120s
-  and now gets the default 60s. Move each such rule into an ``ErrorCategory`` entry and
-  keep ``instructions`` for teaching the model your error strings; passing custom
-  ``instructions`` without ``categories`` now raises a ``UserWarning`` at Dag parse time
-  saying so.
-
-  The ``retry_reason`` written on a retry is now a generated line
-  (``category=... confidence=... threshold=... action=... delay=...``) rather than the
-  model's prose, and a decision that came from ``fallback_rules`` has its reason prefixed
-  with ``LLM classification not applied (<why>);``. See :doc:`retry_policies`.
+  ``LLMBranchOperator`` and ``LLMOperator`` now push a ``decision`` XCom on every run, next to
+  ``return_value``, with the model's pick, the action taken, the confidence and probabilities
+  when the model reports them, and the ``decision_policy`` that applied. ``LLMBranchOperator``
+  also offers the downstream task IDs to the model in sorted order (it was set order, which
+  differed between workers), gained ``branches`` as a template field, and builds its option
+  type from pydantic-ai's ``Choices`` on 2.46+ or an equivalent enum whose member names are
+  generated; the option values are still the task IDs, so ``do_branch`` receives the same
+  strings as before.
+
+.. note::
+  ``execute_complete`` on ``LLMOperator``, ``LLMBranchOperator``, ``LLMSQLQueryOperator`` and
+  ``LLMSchemaCompareOperator`` gained a keyword argument, ``decision``. ``LLMOperator`` and
+  ``LLMBranchOperator`` pass it on resume, so a subclass of either that overrides
+  ``execute_complete`` with the old three-argument signature raises ``TypeError`` when the
+  reviewed task resumes; add ``decision=None`` to the override. The other two accept the
+  keyword but do not pass it yet. A review that was already pending when you upgraded
+  resumes without it and is unaffected.
 
 .. note::
   Configuring ``fallback_conn_ids`` on a connection (or the matching operator/decorator

providers/common/ai/docs/classifier_models.rst

@@ -112,14 +112,16 @@ Where it fits in this provider
      - A ``Literal``, ``Enum``, ``bool`` or bounded number works. Describe the field, which
        becomes the question, and describe each option, which is what tells them apart. An
        option with no description is read from its name alone.
-   * - :doc:`LLMRetryPolicy <retry_policies>`
+   * - :doc:`ClassifierRetryPolicy <retry_policies>`
      - Yes
      - The model names one of the policy's ``categories`` and nothing else; retry or
        fail, the delay and the confidence bar come from each category's entry in the
-       worker. Set ``model_id`` and ``min_confidence``, and an unsure answer goes to
-       ``fallback_rules`` and then the task's own retry behaviour, instead of ending the
-       task on the model's say-so. This is the surface where
-       the model's speed and price matter most: it runs on every task failure.
+       worker. Set ``min_confidence`` and an unsure answer goes to ``fallback_policy``
+       (typically an ``LLMRetryPolicy`` on a text model), then ``fallback_rules``, then
+       the task's own retry behaviour, instead of ending the task on the model's say-so.
+       ``LLMRetryPolicy`` itself asks for free text, which a classifier model refuses.
+       This is the surface where the model's speed and price matter most: it runs on
+       every task failure.
    * - Agents with toolsets
      - Partly
      - Which tool the text calls for is itself a pick, so a classifier model can make it.
@@ -140,8 +142,8 @@ and :class:`~airflow.providers.common.ai.operators.llm.LLMOperator` take a
 ``decision_policy`` whose ``min_confidence`` sends an unsure answer to a person, or fails
 the task, before anything downstream runs on it, and record the confidence, the
 probabilities and the bar in the ``decision`` XCom (see :doc:`operators/llm_branch`).
-:doc:`LLMRetryPolicy <retry_policies>` takes the same ``min_confidence`` and hands an unsure
-answer to its deterministic fallback rules. In the branch operator and the retry policy, a
+:doc:`ClassifierRetryPolicy <retry_policies>` takes the same ``min_confidence`` and hands an
+unsure answer to ``fallback_policy``, then its deterministic fallback rules. In the branch operator and the retry policy, a
 per-option bar lets the choice whose wrong pick costs most demand more certainty than the rest.
 
 Outside those, read it yourself. ``AgentOperator`` carries it inside the ``message_history``

providers/common/ai/docs/operators/llm_branch.rst

@@ -87,6 +87,15 @@ and a text model's structured output carries no confidence to read. With a
 classifier model such as TypeSafe's, the descriptions become the criteria of
 its choice question, which is the text it weighs each option by.
 
+A pick is relative: the model chooses the best fit among the downstream tasks
+offered, not whether any of them fits. If "none of these" or "not enough to
+tell" is a real outcome, give it a downstream task of its own (an
+``EmptyOperator`` that ends the run, or a task that opens a ticket) and
+describe it, rather than expecting the model to refuse. When you change a
+description or the set of branches, treat confidence values measured before
+as stale: the distribution the model returns is over the options it was
+given.
+
 Multiple Branches
 -----------------