Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions skills/assemblyai/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,7 @@ See `references/llm-gateway.md` for models, tool calling, structured outputs, an
| Voice Agent HTTP tool `headers` | Server-side HTTP tools take `headers` as a **list of `{name, value}` entries** (since June 2026), NOT a `{name: value}` dict. Reads return `{name, last_set_at}` only — values are write-only and never returned (not masked as `"***"`) |
| Voice Agent BYO LLM (`llm`) | Set `llm` on a **stored agent** (REST create/update, not `session.update`) to run on your own OpenAI-compatible endpoint: `[{base_url, model, api_key}]` — one entry only, must stream, `api_key` write-only. `"llm": []` reverts to the managed model. Point `base_url` at the LLM Gateway to use a frontier model on your AssemblyAI account |
| Voice Agent webhooks | Subscribe to Voice Agent events via `POST https://agents.assemblyai.com/v1/webhook-subscriptions` (events: `session.started`/`session.completed`/`call.connected`/`call.ended`/`call.failed`; scope with `agent_id` or omit for account-wide). See `references/api-reference.md` |
| Voice Agent tool `pattern` on spoken digits | A tool parameter's `pattern` is matched against the value **as the agent produces it from speech**, and cleanup is best-effort. Long digit sequences (card/account numbers) often arrive with interior spaces (`"4 2 4 2 …"`), so a strict `^\d{16}$` rejects them and traps the caller in a re-ask loop. Allow interior spaces, bound the **digit** count, add a spaced `example`, and strip non-digits in your handler. See `references/voice-agents.md` |

## Common Mistakes

Expand Down
10 changes: 10 additions & 0 deletions skills/assemblyai/references/api-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -286,6 +286,16 @@ Full list of supported PII policy values for `redact_pii_policies`:

`account_number`, `banking_information`, `blood_type`, `credit_card_cvv`, `credit_card_expiration`, `credit_card_number`, `date`, `date_interval`, `date_of_birth`, `drivers_license`, `drug`, `duration`, `email_address`, `event`, `filename`, `gender`, `gender_sexuality`, `healthcare_number`, `injury`, `ip_address`, `language`, `location`, `location_address`, `location_address_street`, `location_city`, `location_coordinate`, `location_country`, `location_state`, `location_zip`, `marital_status`, `medical_condition`, `medical_process`, `money_amount`, `nationality`, `number_sequence`, `occupation`, `organization`, `organization_medical_facility`, `passport_number`, `password`, `person_age`, `person_name`, `phone_number`, `physical_attribute`, `political_affiliation`, `religion`, `sexuality`, `statistics`, `time`, `url`, `us_social_security_number`, `username`, `vehicle_id`, `zodiac_sign`

### Granular `location_*` policies (matching is shape-dependent)

The `location_*` subtypes let you redact specific parts of a location while leaving others visible — but **which policy matches a span depends on the span's shape**, which is a common source of "why wasn't this redacted" mistakes:

- A **full contiguous mailing address** (street + city + state + ZIP together, e.g. `145 Windsor St., Toronto, ON M5A 2P5`) is one `location_address` span — redact it with `location_address`.
- A **standalone fragment** is tagged as its specific subtype: a lone street → `location_address_street`, a lone city → `location_city`, a lone state → `location_state`, a lone country → `location_country`, a lone ZIP → `location_zip`, coordinates → `location_coordinate`.
- A **combined phrase that mixes parts without forming a full address** — e.g. `Toronto, Canada` — is a single generic `location` span. The granular subtypes alone will **not** match it; you must include the broad `location` policy. (So adding only `location_city` expecting `Toronto, Canada` to be redacted will silently fail.)

These subtypes apply to **PII redaction only**. [Entity Detection](https://www.assemblyai.com/docs/speech-understanding/entity-detection) returns every location under the single `location` entity type — never as `location_city`, `location_zip`, etc.

---

## 14. Data Retention & Compliance
Expand Down
1 change: 0 additions & 1 deletion skills/assemblyai/references/llm-gateway.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,6 @@ Model IDs have NO provider prefix (e.g., use `claude-sonnet-4-5-20250929`, not `
| Model | ID |
|-------|----|
| Gemini 3.5 Flash | `gemini-3.5-flash` |
| Gemini 3 Flash Preview | `gemini-3-flash-preview` |
| Gemini 3.1 Flash Lite Preview ⚠️ US-only | `gemini-3.1-flash-lite-preview` |
| Gemini 2.5 Pro | `gemini-2.5-pro` |
| Gemini 2.5 Flash | `gemini-2.5-flash` |
Expand Down
19 changes: 19 additions & 0 deletions skills/assemblyai/references/voice-agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,25 @@ By default a voice agent uses AssemblyAI's **managed** conversational model. To

`session.tools` updates **replace** the previous array (not merge). Pattern: progressive tool reveal — start with minimal tools, add the next phase's tools after each successful `tool.result`.

### Parameter hints for spoken input (`pattern` / `examples`)

A parameter's JSON Schema `pattern` (Python regex, matched against the **whole** value — no `^`/`$` needed; escape as `\\d` in JSON) and `examples` describe the value **the agent produces from speech**, not the words the caller says. The agent's cleanup of spoken input is **best-effort, not guaranteed**: it is reliable for well-known shapes (phone numbers, emails, dates), but **long digit sequences** (card numbers, account numbers, claim IDs) often arrive **as spoken** — with interior spaces when the caller reads digits one at a time (`"4 2 4 2 …"`, `"4242 4242 4242 4242"`), or with digits missing if recognition dropped some.

**The trap (#1 cause of re-ask loops):** a `pattern` written only for the tidy form (`^\\d{16}$`) rejects the spaced form, so the agent silently re-asks — the caller just hears "I didn't catch that" on repeat. Instead, allow interior spaces, bound the **digit** count (not character count), and include a spaced `example`, then strip non-digits in your handler:

```json
{
"card_number": {
"type": "string",
"description": "The card number as spoken. May contain spaces between digits; 13–19 digits once spaces are removed.",
"pattern": " *([0-9] *){13,19}",
"examples": ["4242424242424242", "4242 4242 4242 4242", "4 2 4 2 4 2 4 2 4 2 4 2 4 2 4 2"]
}
}
```

This tolerates a **complete** number spoken with spaces; it should not (and does not) accept a genuinely incomplete one — a partial capture is correctly rejected and re-asked. Keep the `pattern` as a shape hint that admits real spoken input; do length/checksum validation in your handler.

### Mutability After `session.ready`

| Field | Mutable? |
Expand Down