Him188 · Oct 24, 2026 · Oct 16, 2026 · Oct 16, 2026 · Oct 19, 2026 · coderabbitai
diff --git a/agents/build/custom-llm.mdx b/agents/build/custom-llm.mdx
@@ -58,16 +58,16 @@
        "parameters": {
          "type": "object",
          "properties": {
            "order_number": { "type": "string", "description": "The order number, e.g. A12345." }
          },
          "required": ["order_number"]
        }
      }
    }
  ],
  "session_id": "sess_01j9x4k2m8v3q7n5p6r8t9w0y1",
  "user_id": "user-42",
  "fishaudio_extra_body": { "chat_id": "chat-9" }
 }
 ```

@@ -143,7 +143,26 @@
 
 - Each request gets one retry and a 10 second response cap.
 - When a generation still fails, the agent speaks a brief hold line and stays on the call.
-- After three consecutive failed generations the agent apologizes, hangs up, and the session ends with `end_reason: llm_endpoint_failure` (in the [sessions API](/agents/monitor/conversation-history#status-and-end-reason), the dashboard, and the `call.ended` webhook's `ended_reason`). Each failure also emits an `llm.endpoint_error` event on the session timeline.
+- After three consecutive failed generations the agent apologizes, hangs up, and the session ends with `end_reason: llm_endpoint_failure` (in the [sessions API](/agents/monitor/conversation-history#status-and-end-reason), the dashboard, and the `call.ended` webhook's `ended_reason`).
+
+### See what your endpoint returned
+
+Every failed generation is recorded, whether or not the call ended because of it. You can find them in three places:
+
+- **Conversations** in the dashboard shows each failure in the transcript, where it happened.
+- The [sessions API](/agents/monitor/conversation-history#llm-endpoint-errors) returns them as `llm_endpoint_errors`.
+- The [`call.ended` webhook](/agents/monitor/webhooks#payload) carries the same list.
+
+Each failure has a type:
+
+| `type`             | Meaning                                                                  |
+| ------------------ | ------------------------------------------------------------------------ |
+| `http_error`       | Your endpoint answered with a non-2xx status                             |
+| `timeout`          | No response within 10 seconds                                            |
+| `connection_error` | The request got no answer, for example a DNS, TLS or refused connection  |
+| `invalid_response` | A 2xx response that was not a usable reply, such as a broken stream      |
+
+For `http_error`, `response_body` holds the start of your response body, up to 300 characters. Your API key is masked. `connection_error` and `invalid_response` also carry a `reason`, such as `dns_failed` or `not_event_stream`. The lines the agent spoke in place of a reply have `fallback_reason: "llm_error"`.
 
 <Tip>
 Voice conversations are latency sensitive:

diff --git a/agents/monitor/conversation-history.mdx b/agents/monitor/conversation-history.mdx
@@ -4,9 +4,9 @@
 icon: "clock-rotate-left"
 ---

 Every production session your agents handle (from the API, the console, [public agents](/agents/deploy/public-agents), or [phone calls](/agents/telephony/inbound-calls)) is queryable over REST: a lightweight list for browsing, a merged timeline of messages and tool activity per session, and per-speaker recordings when the agent [records audio](#what-gets-stored). You can query a session while the call is still in progress.

 This is a server-side API: authenticate with your API key. The client SDKs deliberately expose no history interface; fetch history from your backend and pass it to your frontend as needed.

 ## What gets stored

@@ -67,8 +67,8 @@
      "metadata": { "crm_ticket": "T-4821" }
    }
  ],
  "has_more": true,
  "next_cursor": "…"
 }
 ```

@@ -189,20 +189,52 @@
 
 | `type`        | What it is                                                                                                                                                 |
 | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `message`     | One utterance. `role` is `user` or `assistant` (system prompt content is never exposed). `turn_id` groups items belonging to the same conversational turn. |
+| `message`     | One utterance. `role` is `user` or `assistant` (system prompt content is never exposed). `turn_id` groups items belonging to the same conversational turn. `fallback_reason` is `"llm_error"` on a fixed line the agent spoke because its LLM failed, and `null` otherwise. |
 | `tool_call`   | A tool invocation as it started. `input` is a JSON string, stored in full with no truncation.                                                              |
 | `tool_result` | The paired outcome, matched by `call_id`. `status` is `completed` or `failed`. `latency_ms` is the execution time.                                         |
 | `transfer`    | One [call transfer](/agents/telephony/transfers) attempt, successful or not. See [Transfer items](#transfer-items).                                        |

 Details worth knowing:

 - **Role vocabulary**: the history API's `assistant` is the same speaker the [SDK's live events](/agents/deploy/web-sdk) call `agent`.
 - **Order**: items are sorted by `created_at` ascending; a `tool_result`'s timestamp is its completion time, so long-running tools appear where they actually finished, with messages in between. Items sharing a timestamp order `message`, then `tool_call`, then `tool_result`.
 - **`tool_source`**: where the tool ran, one of `client`, `webhook`, `builtin` (platform tools such as call transfer and hang-up), `mcp` (tools from a connected MCP server), `background` (work the agent delegated to a background task), or `unknown` (calls recorded before source attribution). Treat it as an open set. See [Tools](/agents/build/tools).
 - **Payloads**: `input` and `output` are JSON strings, symmetric with the live SDK events, so one parser covers both. `output` and `error` are stored up to 256 KB; beyond that the text is cut and `output_truncated` is `true`.
 - **Mid-call queries**: you can fetch an `active` session; you get the items persisted so far, and `analysis` is `null` until [post-call analysis](/agents/monitor/post-call-analysis) completes.
 - A `tool_call` without a matching `tool_result` means the execution never resolved.
 
+### LLM endpoint errors
+
+Sessions on a [custom LLM](/agents/build/custom-llm) also carry `llm_endpoint_errors`. It lists each failed generation of your endpoint, oldest first. It is empty when the agent runs on a platform model or your endpoint never failed.
+
+```json
+{
+  "end_reason": "llm_endpoint_failure",
+  "llm_endpoint_errors": [
+    {
+      "occurred_at": "2026-10-15T15:45:10Z",
+      "turn_id": 3,
+      "type": "http_error",
+      "reason": null,
+      "status_code": 502,
+      "response_body": "{\"detail\":\"no tunnel\"}",
+      "duration_ms": 812,
+      "consecutive_failures": 1
+    }
+  ]
+}
+```
+
+- `turn_id` is the turn the failed reply belonged to. It uses the same numbering as `message` items.
+- `type` is `http_error`, `timeout`, `connection_error` or `invalid_response`. See [Failure behavior](/agents/build/custom-llm#failure-behavior). New types may be added.
+- `reason` is set on `connection_error` and `invalid_response` only. New reasons may be added.
+  - `connection_error`: `dns_failed`, `connection_refused`, `connection_reset`, `tls_failed`, `address_not_allowed` (an address the platform does not call) or `unknown`.
+  - `invalid_response`: `not_event_stream` (a streamed request got something else back), `stream_incomplete` (the stream ended with no `finish_reason`, usage or output), `empty_reply` (no text and no tool call) or `unknown`.
+- `status_code` is set on `http_error` only.
+- `response_body` is the start of what your endpoint sent back, on `http_error` and `not_event_stream` only. It is at most 300 characters, and your API key is masked. It is `null` when the body was empty.
+- `duration_ms` is the time from the request to the response or the failure. It is `null` when it was not measured.
+- `consecutive_failures` counts failures in a row. The agent hangs up at the third.
+
 <Warning>
   `items` is a discriminated union on `type`, and it evolves additively. Ignore
   item types and fields you don't recognize. New modalities and item kinds will

diff --git a/agents/monitor/webhooks.mdx b/agents/monitor/webhooks.mdx
@@ -92,7 +92,7 @@
 
 ## Payload
 
-Every payload carries the `event` name and a `session` object with the session facts. Note the field names differ from the [sessions API](/agents/monitor/conversation-history), which uses `session_id` / `started_at` / `ended_at` for the same facts. `call.ended` adds `ended_reason`; `call.analyzed` adds the `analysis` entity; `phone_call.dial_finished` adds top-level `dial_status` and `answered_by`. The `session` object is the same on every event. The `phone_call.dial_finished` example shows it in full, the others are shortened.
+Every payload carries the `event` name and a `session` object with the session facts. Note the field names differ from the [sessions API](/agents/monitor/conversation-history), which uses `session_id` / `started_at` / `ended_at` for the same facts. `call.ended` adds `ended_reason` and `llm_endpoint_errors`; `call.analyzed` adds the `analysis` entity; `phone_call.dial_finished` adds top-level `dial_status` and `answered_by`. The `session` object is the same on every event. The `phone_call.dial_finished` example shows it in full, the others are shortened.
 
 <CodeGroup>
 
@@ -105,16 +105,17 @@
    "branch_id": "b7d4…",
    "source": "phone",
    "status": "completed",
    "end_reason": "user_hangup",
    "conversation_started_at": "2026-07-23T12:01:12Z",
    "conversation_ended_at": "2026-07-23T12:04:16Z",
    "duration_seconds": 184,
    "end_user_id": "customer-42",
    "metadata": { "order_ref": "SO-1042" },
     "agent_name": "Support agent",
     "config_hash": "sha256:9c41…"
   },
-  "ended_reason": "user_hangup"
+  "ended_reason": "user_hangup",
+  "llm_endpoint_errors": []
 }
 ```
 
@@ -187,11 +188,13 @@
 
 `ended_reason` is the same value as `session.end_reason` and uses the [sessions API vocabulary](/agents/monitor/conversation-history#status-and-end-reason): `user_hangup`, `agent_hangup`, `escalated`, `conversation_timeout`, `heartbeat_timeout`, `agent_join_timeout`, `internal_error`, or `llm_endpoint_failure`. `call.ended` is only sent for sessions in which a participant connected, so the never-connected reasons (`room_join_timeout`, `cancelled`, `dial_failed`) do not appear here; query the sessions API with `status=failed` for those. In the rare case the reason is not settled yet at delivery time, both fields are `null`. New values may be added; treat unrecognized values as informational rather than rejecting the event.
 
+`call.ended` also carries `llm_endpoint_errors`. It lists the failures of your [custom LLM](/agents/build/custom-llm#see-what-your-endpoint-returned) endpoint during the session, in the same shape as the [sessions API](/agents/monitor/conversation-history#llm-endpoint-errors). It is empty on platform models. It is `null` in the rare case the list could not be read at delivery time.
+
 `dial_status` is `answered`, `busy`, `no_answer`, or `failed`. `answered_by` is deprecated: answering-machine detection has been removed, so it is `unknown` once the call is answered and `null` before (older events may carry `human` or `voicemail`). Both facts also appear on the session, alongside `direction` (`inbound` or `outbound`) and `batch_call_id` (reserved, always `null` today); these session fields are present in every webhook payload, with the dial fields `null` on inbound sessions. The `phone_call.dial_finished` snapshot is taken when the dial resolves, so on an answered call `conversation_ended_at` and `duration_seconds` are still `null`. The final numbers arrive with `call.ended`.
 
 `caller_number`, `dialed_number` and `sip_call_id` carry the same values as on the [sessions API](/agents/monitor/conversation-history): the E.164 numbers of the call (person-centric on outbound calls, so `caller_number` is the party you dialed) and the SIP `Call-ID` of the call's INVITE. All three are empty strings on web sessions, and `sip_call_id` is also empty on an unanswered outbound dial. How to use it against your carrier's records: [Match calls with your SIP records](/agents/telephony/byo-sip#match-calls-with-your-sip-records).

 `call_sid` is the carrier's id of the call. Today that is Twilio's CallSid on an inbound call through a Twilio number. It is the same value the [inbound call webhook](#inbound-call-webhook) request carries, so you can join that request to the session. It is an empty string on web sessions, outbound calls, and calls through your own SIP trunk.

 `end_user_id` and `metadata` are echoed exactly as you set them when creating the session. Use them to correlate the event with records in your own system. `branch_id` is an internal configuration-lineage identifier and is safe to ignore. What lands in `summary`, `data`, and `criteria_results` is defined by your [analysis configuration](/agents/monitor/post-call-analysis).

@@ -283,14 +286,14 @@
 | Fan-out   | Every configured endpoint receives every event                                                                    |
 | Guarantee | At-least-once, per endpoint                                                                                       |
 | Timeout   | 10 seconds per attempt                                                                                            |
 | Retries   | 2 after the first attempt (3 attempts total) per endpoint, with backoff of 1s / 5s, then that delivery is dropped |
 | Ordering  | `phone_call.dial_finished` (outbound only) before `call.ended` before `call.analyzed` for the same session        |

 Endpoints are delivered in parallel and independently: each gets its own attempts, its own retry budget, and its own signature keyed with its own secret. An endpoint that is down and exhausts all three attempts has no effect on the others.

 Respond with a `2xx` status within the timeout; a `500` response or a timed-out request counts as a failed attempt. Acknowledge first and process asynchronously. Slow handlers burn their own retry budget.

 **Idempotency.** At-least-once delivery means the same event can arrive more than once. Retries of one delivery carry an identical body, so dedupe `call.ended` and `phone_call.dial_finished` on (`event`, `session.id`), and `call.analyzed` on (`event`, `session.id`, `analysis.finished_at`). The extra element matters because a skipped or failed analysis can be re-run from the console: the recovered result arrives as a fresh `call.analyzed` with a newer `finished_at`, superseding the earlier one.

 <Note>
  [Preview calls](/agents/test/preview-calls) made from the Builder never
@@ -354,13 +357,13 @@
 }
 ```

 `caller_number` and `dialed_number` are E.164. `caller_number` is an empty string when the carrier withholds the caller's number, and `call_sid` is empty when the call did not come through a Twilio number.

 To match this request with the call, keep `call_sid`. Once the call is connected, `GET /v1/agent/sessions?call_sid=…` returns its session, and post-call webhooks carry the same value. This works only when `call_sid` is not empty, that is for calls through a Twilio number. During the call, webhook tools can send the session id with `{{system.session_id}}`. New fields can be added to this request, see [Forward compatibility](#forward-compatibility).

 ### Response

 Respond with a `2xx` status and a JSON object whose `dynamic_variables` follow the same rules as on session creation: names match `[A-Za-z][A-Za-z0-9_]*`, values are strings (up to 100,000 characters), numbers, or booleans, and at most 500 entries totaling 1 MiB are read.

 ```json Response
 {
@@ -392,7 +395,7 @@

 ## Auto-ticket unresolved calls

 `call.analyzed` closes the loop on conversations the agent couldn't: judge every call with a success criterion, and open a ticket in your helpdesk whenever the verdict isn't `success`.

 First give the agent's [analysis configuration](/agents/monitor/post-call-analysis) a criterion that captures resolution:

@@ -513,11 +516,11 @@

 </CodeGroup>

 `verifyWebhook` is the function from [Verify the signature](#verify-the-signature); `openTicket` stands in for your helpdesk's API. Escalating on anything but `success` includes `unknown` verdicts: the model couldn't judge the call, which usually deserves human eyes too. Tighten the check to `failure` only if unknowns prove noisy.

 Edges worth handling:

 - Calls with nothing to analyze arrive with `analysis.status: "skipped"`. The handler above tickets them as unjudged, so every call reaches the helpdesk without also watching `call.ended`. Drop that branch if silent calls don't belong in your queue.
 - Payloads carry no transcript. To include one in the ticket, fetch `GET /v1/agent/sessions/{session_id}` from your handler. See [conversation history](/agents/monitor/conversation-history).
 - Set `end_user_id` and `metadata` when creating sessions so tickets attach to the right customer record without a lookup.