diff --git a/CHANGELOG.md b/CHANGELOG.md index 6377ce0..d7c8749 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -537,6 +537,74 @@ version 2: its `{{ … }}` sequences become substitution points, and its OVOS-PIPELINE-1 §5.3, session registry → OVOS-SESSION-1 §2.2, `ovos.intent.unmatched` → PIPELINE-1 §9.3) and OVOS-SESSION-1 cited by its canonical title. +- §6.1 — the poll is the decision point (no second round, unlike + OVOS-COMMON-QUERY-1), so the 0.5 s RECOMMENDED ceiling is stated as + calibrated for local evaluators and a model-backed stage MUST configure + a ceiling above its own evaluation latency. A fallback skill SHOULD + cache its ping-time evaluation keyed `(session_id, utterance)`, reuse it + at dispatch, and discard it when a new utterance arrives + (OVOS-COMMON-QUERY-1 §5.2). The stage collection ceiling is defined — + `pool_size × per-poll ceiling` sequentially, one window for the + broadcast form — and a deployment MUST set the stage's match-timeout + bound at or above it (OVOS-PIPELINE-1 §4.4, the OVOS-COMMON-QUERY-1 + §2.1 pattern). Acknowledges that OVOS-STOP-1 §4.1 additionally caps its + ceiling at 1 s where this specification does not. +- §6.1 — the pong gains a REQUIRED `utterance` echo (the first candidate) + and round correlation: a pong whose `utterance` or session does not + match the poll in flight MUST be discarded, one round-trip per + `(skill_id, utterance)`. Absence of the field is tolerated as a legacy + producer (SHOULD accept); a present, non-matching value never is. +- §6.1 — the ping's reply derivation rationale corrected: the derivation + is `reply` because the queried skill always answers, so the query must + arrive response-ready — session propagation is common to every + derivation and does not discriminate. The pong likewise MUST be derived + through the reply derivation, so it reaches the plugin wherever the + skill runs. Dotted form cited to OVOS-MSG-1 §2.1.1. +- §7 (new note) — a fallback dispatch is a fresh activation: `fallback` + is a reserved intent_name whose `session.active_handlers` push is NOT + suppressed, per the per-row rule in the OVOS-PIPELINE-1 §7.3 registry. + This is what makes a running fallback skill stoppable (OVOS-STOP-1) and + lets a conversational fallback take the next utterance + (OVOS-CONVERSE-1). +- §5 step 1 — equal registered priorities MUST be ordered + deterministically; ascending `skill_id` is RECOMMENDED. Registration + arrival order is not an acceptable tie-break. +- §5 step 2 / §8.2 — range bounds are inclusive; configured ranges SHOULD + partition the priority space with the last stage open above; a skill in + no range is never queried and SHOULD be logged at WARN. The range + filter applies after the preference order, so + `session.fallback_handlers` orders the pool but never admits a skill + into a stage. +- §3.2 — deregistration is keyed by `context.session.session_id`, never + by `Message.data` (OVOS-INTENT-4 §11.3), and a deregistration whose + payload `skill_id` differs from `context.skill_id` MUST NOT be honoured. + Added to the §10 MUST list. +- §3.5 (new) — registry liveness: a skill SHOULD deregister on orderly + shutdown; the plugin SHOULD honour `ovos.skill.deregister` + (OVOS-INTENT-4 §8.4) for the session in `context`; the plugin MAY evict + after N consecutive poll timeouts and MUST re-admit on re-registration. + Eviction is availability, not access control. +- §3.3 / §8.1 — the default-response-skill rule moves to §8.1, its + normative home, and §3.3 becomes purely descriptive. The catch-all + guarantee comes from placement (last entry of + `session.fallback_handlers`, or the last stage's range), not from + registering at the number 100 — priority is an unbounded integer. + Added to the §10 deployment SHOULDs. +- §3.4 — the "available to all sessions" claim now cites + OVOS-INTENT-4 §11.1 for keying and §11.2 for inheritance. +- §8.1 / §8.2 — corrected: only the low-confidence catch-all fallback + stage SHOULD follow the persona stage; `fallback_high` and + `fallback_medium` MAY precede it, interleaved with intent-matching + stages by priority range, per §8.2. The §8.2 example pipeline is + restored to + `stop_high, converse, intent_high, fallback_high, intent_medium, + fallback_medium, common_query, persona, fallback_low`, aligned with + OVOS-PERSONA-1 §10's `common_query` → `persona` ordering, so the + confidence-tier interleaving from §8.2 is no longer lost to a + blanket persona-first placement. +- §1, §4 — one session field (`fallback_handlers`); access control reuses + the OVOS-PIPELINE-1 denylists. Poll-wait wording unified across §6.1 + and §10. ## OVOS-CONVERSE-1 — Active Handlers and Interactive Response ### 2 diff --git a/fallback.md b/fallback.md index b18b141..b1903d4 100644 --- a/fallback.md +++ b/fallback.md @@ -35,9 +35,11 @@ This specification defines: - **the fallback plugin role** (§2) — a pipeline plugin that delegates to registered fallback skills; - **skill registration** (§3) — how a skill declares itself as a - fallback handler with a default ordering priority; -- **session fields** (§4) — the two session-resident fields that - control pool ordering and access control; + fallback handler with a default ordering priority, and how the + registry stays live (§3.5); +- **session fields** (§4) — the one session-resident field that + controls pool ordering, and the reuse of the OVOS-PIPELINE-1 + denylists for access control; - **pool construction** (§5) — how the ordered handler pool is derived from registration, session preference, and policy; - **the match contract** (§6) — the sequential per-skill query, @@ -122,10 +124,24 @@ index a registration where the payload `skill_id` differs from | Field | Type | Required | Meaning | |-------|------|----------|---------| -| `skill_id` | string | yes | The skill's identity. | +| `skill_id` | string | yes | The skill's identity. MUST equal `context.skill_id` of this Message. | Removes the skill from the registry. Unknown `skill_id` is a no-op. +Deregistration is scoped exactly like registration. The plugin +**MUST** key the removal by `context.session.session_id` of the +deregistration Message — never by a `session_id` carried in +`Message.data` — per **OVOS-INTENT-4 §11.3**. A deregistration +arriving under the default session removes the `"default"`-scoped +entry only; it does not remove entries registered under a specific +`session_id`. + +The plugin **MUST NOT** honour a deregistration whose payload +`skill_id` differs from `context.skill_id`: without this check any +skill can evict any other skill from the registry. A mismatch is +malformed — log at WARN with both identifiers and the topic, and do +not act on it. + ### 3.3 Priority guidance Priority values have no normative meaning beyond ordering — lower @@ -148,29 +164,55 @@ membership** (which confidence class a number belongs to), not by numeric rescale — the numbers label the classes; they are not a continuous scale. -A skill author uncertain which tier applies **SHOULD** register at -a higher number rather than a lower one. Pre-empting a more +A skill author uncertain which tier applies is better off +registering at a higher number than a lower one. Pre-empting a more precise handler with a low-confidence catch-all degrades response quality silently. -**Default response skill.** A voice assistant SHOULD always -produce an answer rather than silent failure. Every deployment -SHOULD include a catch-all fallback skill — registered at the -highest priority number in the pool (e.g. `priority: 100`) — that -unconditionally returns `can_handle: true` and responds with a -graceful "I don't know how to answer that" message. When -`session.fallback_handlers` is set, this skill **SHOULD** be its -last entry, so it remains the last resort after all -higher-confidence handlers have declined. Without it, an utterance that no skill can handle -produces `ovos.intent.unmatched` with no user-facing response. +**The catch-all handler.** The low-confidence tier is where a +deployment's catch-all handler belongs — the skill that answers +"I don't know how to answer that" when nothing else will. The +normative rule for that skill is §8.1; this section only says where +in the number space it sits. Note that a large priority number does +not by itself make a skill last: priority is an unbounded integer, +so any number can be undercut by a larger one. A catch-all is last +because of *placement* — it is the last entry of +`session.fallback_handlers`, or the only registration inside the +last stage's priority range (§8.2) — not because of the number it +chose. ### 3.4 Session-scoped registration Registration is session-scoped per **OVOS-INTENT-4 §11.1**: the plugin keys each entry by `context.session.session_id` of the registration Message. Skills registered under `"default"` are -available to all sessions. Skills registered under a specific -`session_id` extend the pool for that session only. +available to all sessions, because every session inherits the +`"default"` scope (**OVOS-INTENT-4 §11.2**). Skills registered +under a specific `session_id` extend the pool for that session +only. + +### 3.5 Registry liveness + +A registry entry outlives the process that created it. A skill that +has stopped, crashed, or disconnected still occupies a pool slot and +still consumes a per-poll ceiling (§6.1) on every utterance that +reaches the stage. + +- A fallback skill **SHOULD** emit `ovos.fallback.deregister` + during orderly shutdown. +- The plugin **SHOULD** honour `ovos.skill.deregister` + (**OVOS-INTENT-4 §8.4**) by removing that skill's fallback + registration for the session read from + `context.session.session_id`. This is what lets a bridge clean up + a satellite's registrations when the satellite disconnects. +- The plugin **MAY** evict a skill that failed to pong within the + ceiling on N consecutive polls, where N is deployment-configured. + An evicted skill **MUST** be re-admitted on its next + `ovos.fallback.register`. + +Eviction is an availability optimisation, not access control. It +**MUST NOT** be the mechanism by which a deployment keeps a skill +out of the pool; `session.blacklisted_skills` is (§4). --- @@ -212,9 +254,30 @@ pool**: skills not in the list, sorted by registered priority ascending. If absent, sort all registered skills by registered priority ascending. + Two skills registered at the same priority **MUST** be ordered + deterministically — the same registry and the same session + always produce the same pool order. Ascending `skill_id` is the + **RECOMMENDED** tie-break. Registration arrival order is not an + acceptable tie-break: it makes the selected handler depend on + process start order. 2. **Stage range.** If this plugin instance is configured with a priority range (§8.2), retain only skills whose registered - priority falls within that range. + priority falls within that range. Both bounds are **inclusive**: + a range `[50, 74]` retains priority 50 and priority 74. + + Ranges configured across the loaded stages **SHOULD** partition + the priority space — no gaps, no overlaps. A skill whose + priority falls in no configured range is never queried by any + stage; the deployment is misconfigured, and a plugin that can + observe the gap **SHOULD** log it at WARN with the `skill_id` + and the priority. + + The range filter applies after the preference order of step 1. + A skill named in `session.fallback_handlers` is therefore still + dropped by the stage whose range excludes it — the preference + list orders the pool, it does not admit a skill into a stage. + A skill preferred by the session but out of every stage's range + is never queried at all. 3. **Availability.** Retain only skills present in the registry for the current `session_id` (including `"default"` registrations per §3.4). @@ -236,9 +299,14 @@ at a time in pool order. For each skill it sends: `.fallback.ping` -using the **dotted addressed** form, derived via `.reply()` from -the inbound utterance Message (**OVOS-MSG-1 §5.2**), so that -`context.session` and routing metadata propagate automatically. +using the **dotted addressed** form (**OVOS-MSG-1 §2.1.1**), +derived from the inbound utterance Message through the reply +derivation (**OVOS-MSG-1 §5.2**). The derivation is the reply one +because the queried skill always answers: a ping that arrives +response-ready lets the skill produce its pong by the same +derivation, and the pong then reaches the plugin wherever either +side runs. Carrying `context.session` forward is a property every +derivation has; it is not what selects `reply` here. Payload: @@ -251,16 +319,39 @@ The skill uses these to run its own evaluation logic and decide whether it can produce a meaningful response. This is the point at which the fallback skill parses the utterance — it may query a knowledge base, run a classifier, call an LLM, or apply any other -internal logic. The reply carries only the decision: +internal logic. The reply carries only the decision. The queried skill replies with: `.fallback.pong` +derived from the ping through the reply derivation (**OVOS-MSG-1 +§5.2**), so that the pong reaches the plugin wherever the skill +runs. + | Field | Type | Required | Meaning | |-------|------|----------|---------| | `skill_id` | string | yes | The responding skill's identity. MUST equal the topic prefix. | | `can_handle` | bool | yes | Whether this skill is willing to handle the current utterance. | +| `utterance` | string | yes | Echo of the utterance evaluated — the first element of the ping's `utterances`. | + +**Round correlation.** The plugin keys poll state by +`session_id`, read from `context.session`. A pong whose `utterance` +or whose session does not match the poll in flight **MUST** be +discarded: a stale or replayed pong otherwise decides a round it +never evaluated. The round-trip is one exchange per +`(skill_id, utterance)` pair. Because at most one fallback poll is +in flight per session (the session lock and round model of this +section), the utterance itself uniquely identifies the round — no +opaque poll id is needed, unlike OVOS-COMMON-QUERY-1 §6.4's +`query_id`, which serves overlapping contests. + +The `utterance` field is REQUIRED going forward. A pong that omits +it is a legacy producer; the plugin **SHOULD** accept such a pong, +correlating it by session and by the `skill_id` it is still +waiting on. A pong that carries `utterance` and does not match +**MUST** be discarded — tolerance covers absence, never +disagreement. The boolean's field name is protocol-specific: this spec and OVOS-STOP-1 use `can_handle`, OVOS-CONVERSE-1's poll uses `result`, @@ -268,15 +359,64 @@ and OVOS-COMMON-QUERY-1 uses `can_answer`. Each name is normative only within its own protocol. The plugin waits for each skill's reply before advancing to the -next. The plugin **MUST** bound each per-skill wait by a ceiling; -the **RECOMMENDED default is 0.5 s** (matching the analogous polls -of OVOS-CONVERSE-1 §4.2 and OVOS-STOP-1 §4.1), which a deployer MAY -raise or lower. Without a ceiling, one unresponsive skill stalls -the entire fallback stage — and with it the utterance — forever. -An absent or malformed pong (missing or non-boolean `can_handle`, -mismatched `skill_id`) **MUST** be treated as `can_handle: false` -and the skill skipped; this is uniform with the silence rules of -OVOS-CONVERSE-1 (§4.2 there) and OVOS-STOP-1 (§4.2 there). +next. The plugin **MUST** bound each poll wait by a ceiling. +Without a ceiling, one unresponsive skill stalls the entire +fallback stage — and with it the utterance — forever. An absent or +malformed pong (missing or non-boolean `can_handle`, mismatched +`skill_id`, mismatched `utterance`) **MUST** be treated as +`can_handle: false` and the skill skipped; this is uniform with the +silence rules of OVOS-CONVERSE-1 (§4.2 there) and OVOS-STOP-1 +(§4.2 there). + +**The poll is the decision.** Unlike OVOS-COMMON-QUERY-1, whose +ping only filters plausible answerers before a separate answer +round, the fallback ping is the whole contest: the pong is the +claim, and there is no second round in which a slow evaluator can +catch up. A skill that cannot answer the ping in time is not +delayed, it is skipped. This follows OVOS-CONVERSE-1 §4.2, where +the poll is likewise the decision point. + +**Ceiling calibration.** The **RECOMMENDED default ceiling is +0.5 s**, matching the analogous polls of OVOS-CONVERSE-1 §4.2 and +OVOS-STOP-1 §4.1 — with the caveat that OVOS-STOP-1 §4.1 also caps +its ceiling at 1 s, which this specification does not. That default +is calibrated for skills whose evaluation is local: a database +lookup, a classifier, a vocabulary test. It is not a budget for a +model-backed skill. A deployment that runs a model-backed fallback +stage — an LLM chatbot, a remote question-answering service — +**MUST** configure that stage's ceiling above the stage's actual +evaluation latency. Leaving the default in place silently converts +every such skill into a non-responder, and the utterance falls +through to a lower-confidence handler that answered faster. + +**Evaluation caching.** Because a willing skill is asked the same +question twice — once at the ping, once at the `:fallback` +dispatch (§7) — a fallback skill **SHOULD** cache the result of its +ping-time evaluation, keyed by the pair `(session_id, utterance)` +with `session_id` read from `context.session`, and reuse it when +the dispatch for that pair arrives. The entry **MUST** be discarded +when a new utterance arrives in the same session, and **MUST NOT** +be returned for any utterance other than the exact string it was +computed for. This mirrors OVOS-COMMON-QUERY-1 §5.2. Caching is a +skill-side optimisation: the protocol is correct without it, and a +skill that re-evaluates at dispatch is conformant. + +**Stage collection ceiling.** The worst-case time a fallback stage +spends inside `match` is bounded by: + +- **sequential form** — `pool_size × per-poll ceiling`, since each + wait runs to the ceiling before the next begins; +- **broadcast form** — one poll window, independent of pool size. + +Both are the stage's *collection ceiling*. OVOS-PIPELINE-1 §4.4 +lets the orchestrator bound each `match` by a timeout and skip a +plugin that exceeds it, so a deployment **MUST** set the fallback +stage's match-timeout bound at or above that stage's collection +ceiling, exactly as OVOS-COMMON-QUERY-1 §2.1 requires for the +common-query collection window. A shorter bound kills the stage +mid-poll on every utterance it handles, and the failure is silent: +the stage simply never claims. The broadcast form is the +straightforward way to keep the ceiling constant as the pool grows. **Broadcast-poll optimisation.** As an observably equivalent alternative to the sequential per-skill query, the plugin MAY emit @@ -333,6 +473,18 @@ The selected skill's handler: No fallback-specific response protocol is required beyond subscribing to `:fallback`. +**Activation.** `fallback` is a reserved intent_name, but it is not +one of the reserved names whose `session.active_handlers` push is +suppressed — the per-row rule is in the OVOS-PIPELINE-1 §7.3 +registry, and the fallback row does push. A fallback dispatch is a +fresh activation, not the continuation of an already-active skill, +so the dispatch pushes the selected `skill_id` onto +`session.active_handlers` per OVOS-PIPELINE-1 §7.1. This is what +makes a running fallback skill stoppable through OVOS-STOP-1, and +what lets a conversational fallback — a language-model chatbot, for +instance — take the next utterance through OVOS-CONVERSE-1 without +a second registration. + --- ## 8. Pipeline positioning @@ -340,14 +492,28 @@ subscribing to `:fallback`. ### 8.1 General placement Fallback stage(s) **SHOULD** be placed after all deterministic -intent-matching stages and after the persona stage (if present). -Utterances that reach a fallback stage were not claimed by any +intent-matching stages. Only the low-confidence, catch-all fallback +stage **SHOULD** additionally follow the persona stage (if present): +it is the last-resort tier (§3.3), and a persona claims everything +that reaches it (OVOS-PERSONA-1 §7.2), so anything placed behind +persona is unreachable while a persona is active — the catch-all +belongs there regardless. Higher-confidence fallback stages +(`fallback_high`, `fallback_medium`) **MAY** precede the persona +stage, interleaved with intent-matching stages by priority range, per +§8.2. Utterances that reach a fallback stage were not claimed by any earlier stage. -Every deployment **SHOULD** include a catch-all fallback skill -registered at the bottom of the priority pool (see §3.3) that -always returns `can_handle: true`. This ensures the user receives -a response to every utterance rather than silent `ovos.intent.unmatched`. +**Default response skill.** A voice assistant should always produce +an answer rather than fail silently. Every deployment **SHOULD** +include a catch-all fallback skill that unconditionally returns +`can_handle: true` and responds with a graceful "I don't know how +to answer that" message. That skill **SHOULD** be reachable only +after every other fallback handler has declined — as the last entry +of `session.fallback_handlers` when that list is set, and inside +the range of the last fallback stage when stages are ranged (§8.2). +§3.3 places it in the low-confidence tier. Without it, an utterance +that no skill can handle produces `ovos.intent.unmatched` +(**PIPELINE-1 §9.3**) and no user-facing response. ### 8.2 Multiple stages and priority interleaving @@ -368,6 +534,7 @@ session.pipeline: [ "fallback_high", ← priority range [0, 49] "intent_medium", "fallback_medium", ← priority range [50, 74] + "common_query", "persona", "fallback_low" ← priority range [75, 100] ] @@ -375,8 +542,19 @@ session.pipeline: [ A skill registered at `priority: 10` is queried by `fallback_high` before `intent_medium` runs. A skill registered at `priority: 80` is -queried by `fallback_low` only after both `intent_medium` and `persona` -have declined. A single-stage deployment sets no range restriction. +queried by `fallback_low` only after `intent_medium`, +`fallback_medium`, `common_query`, and `persona` have all declined. +`fallback_high` and `fallback_medium` interleave with the +intent-matching stages ahead of `persona`; only `fallback_low`, the +catch-all tier, follows it — per §8.1 and OVOS-PERSONA-1 §10, which +places persona after `common_query` and states the same ordering +from its side. A single-stage deployment sets no range restriction. + +The configured ranges are inclusive at both bounds and **SHOULD** +partition the priority space (§5 step 2). Because priority is an +unbounded integer, the last stage's range **SHOULD** be open above +— `[75, ∞)` rather than `[75, 100]` — so that a skill registering +above the convention is still queried somewhere. Within any stage, pool construction and session ordering (§5) work identically regardless of how many stages are present. @@ -406,13 +584,20 @@ identically regardless of how many stages are present. - subscribe to `ovos.fallback.register` and `ovos.fallback.deregister` (§3); - reject any registration where payload `skill_id` ≠ - `context.skill_id` (§3.1); -- construct the effective handler pool per §5 on each match call; + `context.skill_id` (§3.1), and any deregistration with the same + mismatch (§3.2); +- key registration and deregistration by + `context.session.session_id`, never by a `session_id` in + `Message.data` (§3.2, §3.4); +- construct the effective handler pool per §5 on each match call, + ordering equal priorities deterministically (§5 step 1); - query skills via `.fallback.ping` / `.pong` — either sequentially in pool order or via the observably equivalent broadcast-poll optimisation (§6.1); -- bound each per-poll wait by a ceiling and treat an absent or +- bound each poll wait by a ceiling and treat an absent or malformed pong as `can_handle: false` (§6.1); +- discard a pong whose `utterance` or session does not match the + poll in flight (§6.1); - select the first willing skill in pool order (§6.2); - return a `Match` with `intent_name: "fallback"` targeting the selected skill (§6.3); @@ -420,26 +605,51 @@ identically regardless of how many stages are present. ### A fallback pipeline plugin **SHOULD**: -- use the recommended 0.5 s default per-poll ceiling unless the +- use the recommended 0.5 s default poll ceiling unless the deployer configures otherwise (§6.1); -- when configured with a priority range, apply it as §5 step 2. +- when configured with a priority range, apply it as §5 step 2, and + log at WARN a registered skill that falls in no range (§5); +- tolerate a pong that omits `utterance` as a legacy producer + (§6.1); +- honour `ovos.skill.deregister` for the session read from + `context.session.session_id` (§3.5). ### A fallback pipeline plugin **MAY**: -- mutate session state via `Match.updated_session` (§6.3). +- mutate session state via `Match.updated_session` (§6.3); +- evict a skill after N consecutive poll timeouts, re-admitting it + on its next registration (§3.5). ### A skill registered as a fallback handler **MUST**: - emit `ovos.fallback.register` with its `skill_id` and `priority` before receiving fallback dispatches (§3.1); - subscribe to `.fallback.ping` and reply with - `.fallback.pong` (§6.1); + `.fallback.pong`, derived through the reply + derivation and echoing the evaluated `utterance` (§6.1); - subscribe to `:fallback` to receive dispatches (§7). +### A skill registered as a fallback handler **SHOULD**: + +- emit `ovos.fallback.deregister` during orderly shutdown (§3.5); +- cache its ping-time evaluation keyed `(session_id, utterance)` + and reuse it at dispatch (§6.1). + ### A deployment **SHOULD**: - position fallback stage(s) after deterministic intent-matching - and persona stages in `session.pipeline` (§8.1). + and persona stages in `session.pipeline` (§8.1); +- include a catch-all fallback skill, reachable only after every + other fallback handler has declined (§8.1); +- configure priority ranges that partition the priority space, with + the last stage open above (§8.2). + +### A deployment **MUST**: + +- set each fallback stage's match-timeout bound at or above that + stage's collection ceiling (§6.1, OVOS-PIPELINE-1 §4.4); +- raise the poll ceiling of a model-backed fallback stage above + that stage's evaluation latency (§6.1). ---