From 8b0f9c09eee5b62bce07a3456d62cc2bc9d5c27d Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Thu, 27 Aug 2026 16:20:04 +0100 Subject: [PATCH 01/17] Add the integration provider seam design spec The five-PR series (#1043 to #1047) opens the identity, device and geo seams. The nine vendor integrations already in core sit behind the integration registry instead, which is a private table, so none of them can move out until that table is opened. This spec defines the one core change that opens it: public registration builders with a second input on IntegrationRegistry, browser JavaScript carried on the registration, startup validation as a hook, the same treatment for auction providers and the bid renderer contract, and neutral replacements for the two places where a vendor reaches into core. It then sets out the migration of all nine existing integrations, one PR each. The change is complete in itself: after it, no vendor move needs a core change. Written against the series' tree with the file and line references for every claim about the current code. Documentation only. --- ...-08-27-integration-provider-seam-design.md | 207 ++++++++++++++++++ 1 file changed, 207 insertions(+) create mode 100644 docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md diff --git a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md new file mode 100644 index 000000000..6dbfcb7ee --- /dev/null +++ b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md @@ -0,0 +1,207 @@ +# Design Spec: The Integration Provider Seam + +**Status:** Proposed, 2026-08-27. Sixth PR in the provider series. It adds +this document only and targets `main` directly; it reads alongside the +series' specs, which land with PR #1047. +**Author:** 51Degrees (contributed), for Tech Lab review +**Related specs:** `2026-07-30-pluggable-providers-design.md`, +`2026-07-30-provider-migration-rollout-design.md`, +`provider-code-registry.md` +**Related PRs:** #1043, #1044, #1045, #1046, #1047, #1054 +**Last updated:** 2026-08-27 + +> **Why this spec exists.** PRs #1043 to #1047 open the identity, device and +> geo seams, so a vendor can ship an Edge Cookie provider in its own crate +> and an adapter injects it. The nine vendor integrations already inside +> `trusted-server-core` do not sit behind those seams. They hang off the +> integration registry, which is a private table in core, so none of them +> can move out until that table is opened. This spec defines the one core +> change that opens it, so the migration of every existing vendor is a +> single defined piece of work rather than an open question repeated once +> per vendor. + +## 1. The problem, with the code that causes it + +Every claim here was read from the tree at `split/5-response-hook-docs`, +which is `main` plus the five PRs. + +1. **The registry is closed.** `IntegrationRegistry::new` takes only + `&Settings` and iterates a fixed table + (`crates/trusted-server-core/src/integrations/registry.rs:797`, table at + `crates/trusted-server-core/src/integrations/mod.rs:290`). Both + `IntegrationBuilder` and `builders()` are `pub(crate)` with private + fields, so no adapter and no external crate can add to the list. The + payload type `IntegrationRegistration` + (`registry.rs:586`) is already public, so an outside crate can build a + registration but has nowhere to hand it. +2. **Browser JavaScript is fixed at build time.** `trusted-server-js` + discovers `lib/src/integrations/*/index.ts`, builds one file per + integration, and `build.rs` writes a fixed array of `include_str!` + entries consumed by `bundle.rs`. `IntegrationRegistry::js_module_ids` + only serves a module when + `trusted_server_js::module_bundle(id).is_some()` + (`registry.rs:1169`), so an integration outside that compile-time map + gets no script however it registers. +3. **Startup validation names every vendor.** `validate_enabled_integrations` + imports and calls each vendor's config type by name + (`crates/trusted-server-core/src/config.rs:136` to `:166`). +4. **Auction providers are a second closed table.** + `crates/trusted-server-core/src/auction/mod.rs:49` lists Prebid, APS and + the ad server mock. +5. **Two vendors reach further into core.** DataDome drives cache privacy + and the origin fetch decision through a marker type + (`html_processor.rs:303`, `publisher.rs:4367` to `:4387`, + `publisher.rs:2653`), and GPT diagnostics is called by name from all four + adapters (for example + `crates/trusted-server-adapter-fastly/src/app.rs:584`). + +The result is that the project carries nine vendors as core code (ten +registered integrations, since GPT registers a proxy and a diagnostics +integration). Tech Lab engineering time is spent on named commercial vendors, and +every new vendor is another core change, as PR #1054 shows. + +## 2. Principle + +A vendor integration is a provider like any other. Core owns the seam and +owns nothing behind it. Concretely: + +- Core defines the registration contract and the request pipeline. It names + no vendor. +- A vendor integration ships as its own crate with its Rust, its browser + JavaScript, its configuration type, its startup validation and its tests. +- An adapter composes the deployment by injecting the registrations it was + built with, exactly as it already injects the geo and device providers. +- Tech Lab engineering assesses and reviews vendor crates. It does not + maintain them. + +## 3. Design + +### 3.1 Opening the registry + +Make the builder contract public and give the registry a second input. + +- `IntegrationBuilder` becomes `pub` with a public constructor, and + `builders()` stays as the built-in set. +- `IntegrationRegistry::new` gains a companion, + `IntegrationRegistry::with_registrations(settings, extra)`, where `extra` + is a slice of externally supplied builders. `new` keeps its signature and + calls the companion with an empty slice, so no existing caller changes + behavior. +- Duplicate integration ids are a startup error, naming both sources, so a + vendor crate cannot silently shadow a built-in. + +### 3.2 Carrying browser JavaScript on the registration + +Add one optional field to `IntegrationRegistration`, next to `js_deferred` +and `js_disabled`: the module source and its hash, both `&'static str`, so a +crate can `include_str!` its own built bundle. + +`js_module_ids` keeps serving built-in ids from the compile-time map and +serves a carried module from the registration. `publisher.rs` composes the +served script and its hash from both sources rather than calling +`trusted_server_js::concatenate_modules` alone. The hash rule is unchanged, +so the served bundle stays cacheable and its integrity attribute stays +correct. + +### 3.3 Startup validation on the registration + +Replace the named list in `config.rs` with a validation hook on the +registration, so a vendor validates its own configuration and a missing +vendor cannot silently stop being validated. The existing test that asserts +every registered integration is covered by deploy validation +(`config.rs:408`) is rewritten against the hook, so the guarantee survives +in a vendor-neutral form. + +### 3.4 Auction providers + +Give `AuctionOrchestrator` the same treatment as the registry: a public +provider-builder type and a second input, so an auction-side vendor such as +APS can register from its own crate. Prebid stays in core as protocol +support rather than as a vendor integration. + +The bid renderer contract is generalized in the same change. Today +`BidRenderer` is an enum with one variant, `Aps(ApsRendererV1)` +(`crates/trusted-server-core/src/auction/types.rs:216`), serialized into the +OpenRTB response extension under a `type` tag +(`crates/trusted-server-core/src/auction/formats.rs:377`, +`crates/trusted-server-core/src/openrtb.rs:183`). It becomes an open +descriptor, a type tag and a payload the auction provider supplies, with +the same serialized form, so the response a page receives does not change +and the APS renderer type moves with APS. The ad server mock in core uses +the neutral form. + +### 3.5 The two neutral hooks + +- **Response shaping.** DataDome's marker becomes a neutral request + extension meaning "this response is personalized to the request, do not + share it", set by any integration. Core keeps the behavior, being the full + body buffer and private cache, and stops naming a vendor. +- **Request prepare and finalize.** The direct GPT diagnostics calls in the + four adapters move behind hooks on the registration, so an adapter runs + whatever its registrations declare. + +## 4. Migration of the nine existing integrations + +One vendor per PR, after this change lands. Each moves its Rust, its +TypeScript, its config type and its tests into +`crates/integrations/`, and the adapter that wants it depends on it. + +| Vendor | What it needs | +| ---------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | +| Didomi, Google Tag Manager, Lockr, Osano, Permutive, Sourcepoint | Move as they are. Coupled only through the builder table, deploy validation and the JS map. | +| APS | Also needs the auction provider seam and the generalized renderer contract in §3.4, both of which this change delivers. | +| GPT (the `gpt` proxy and `gpt_diagnostics`) | The proxy moves as it is. The diagnostics half needs the prepare and finalize hooks in §3.5. | +| DataDome | Needs the neutral response-shaping hook in §3.5, and about forty test literals move with it. | + +`[integrations.]` configuration tables need no change, because +`IntegrationSettings` is a flattened map that already accepts unknown vendor +keys (`crates/trusted-server-core/src/settings.rs:166`). + +## 5. What does not change + +The request pipeline, the hook traits and their order, the served script +format and its hash, every `[integrations.*]` table, the permission model, +and the Edge Cookie, device and geo seams from PRs #1043 to #1046. No +integration changes behavior. A deployment that lists the same integrations +gets the same responses. + +## 6. Acceptance + +1. **A round trip with a non-default implementation.** A test integration + defined outside `trusted-server-core`, carrying its own JavaScript, + registers through an adapter, appears in the served bundle with the + right hash, runs its hooks in the right order, and is rejected on a + duplicate id. A seam is only proven by an implementation that is not the + built-in one. +2. **Parity.** The existing integration and parity suites pass unchanged, + because the built-in set still registers through the same path. +3. **No vendor left behind.** The rewritten deploy-validation test shows + every registered integration validates its configuration. +4. All CI gates in `CLAUDE.md`, on all four adapters. + +## 7. Risk + +The change is wide but shallow. It touches the registry, the served script +path, deploy validation and four adapter entry points, and it changes no +integration's behavior. The largest risk is the served script, where a +mistake shows up as a wrong hash or a missing module, so §6's round trip +covers both. Doing this once is what removes the per-vendor core change +that the project pays for today, most recently in PR #1054. + +## 8. Sign-off + +| # | Decision | Status | +| --- | ------------------------------------------------------------------------------- | -------------------- | +| 1 | Vendor integrations belong outside core, behind the registration contract | Proposed | +| 2 | Tech Lab engineering reviews vendor crates, and does not maintain them | Proposed, governance | +| 3 | A registration may carry its own browser JavaScript | Proposed | +| 4 | Deploy validation moves onto the registration | Proposed | +| 5 | The nine existing integrations migrate one PR each, on the schedule in §4 | Proposed | +| 6 | This change is complete in itself: after it, no vendor move needs a core change | Proposed | + +## Revision record + +| Date | Change | +| ---------- | ----------------------------------------------------------------------------------------------------------------------------- | +| 2026-08-27 | First draft, written against `split/5-response-hook-docs`. | +| 2026-08-27 | Brought the bid renderer contract into scope (§3.4), so that after this change no vendor move needs a core change (§8 row 6). | From 15c263eb8a927f76d850b045fadb50c543a92c9a Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 28 Aug 2026 10:14:34 +0100 Subject: [PATCH 02/17] Revise the integration seam spec to carry identity, geo and device as registration capabilities Adopt the registration shape asked for in the review of PR 1043 and apply its rule consistently, so a module declares its identity, geo and device providers alongside its JavaScript and hooks (new section 3.6). Record the relationship to the pluggable-providers spec in PR 986, reorder the series so this spec and its core implementation precede PR 1043, and add the capabilities round trip to the acceptance list. --- ...-08-27-integration-provider-seam-design.md | 210 ++++++++++++++---- 1 file changed, 172 insertions(+), 38 deletions(-) diff --git a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md index 6dbfcb7ee..b86730f44 100644 --- a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md +++ b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md @@ -1,14 +1,18 @@ # Design Spec: The Integration Provider Seam -**Status:** Proposed, 2026-08-27. Sixth PR in the provider series. It adds -this document only and targets `main` directly; it reads alongside the -series' specs, which land with PR #1047. +**Status:** Proposed, 2026-08-27, revised 2026-08-28. This PR adds this +document only and targets `main` directly. Following the review of #1043 +(27 August) the seam it defines is a precondition for the provider series +rather than a follow-up to it, so the order is now this spec, then its +implementation in a seventh PR against `main` (51Degrees), then PRs #1043 +to #1047 reworked onto it. It reads alongside the series' specs, which land +with PR #1047. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-07-30-pluggable-providers-design.md`, `2026-07-30-provider-migration-rollout-design.md`, `provider-code-registry.md` -**Related PRs:** #1043, #1044, #1045, #1046, #1047, #1054 -**Last updated:** 2026-08-27 +**Related PRs:** #986, #1043, #1044, #1045, #1046, #1047, #1054 +**Last updated:** 2026-08-28 > **Why this spec exists.** PRs #1043 to #1047 open the identity, device and > geo seams, so a vendor can ship an Edge Cookie provider in its own crate @@ -20,14 +24,30 @@ series' specs, which land with PR #1047. > single defined piece of work rather than an open question repeated once > per vendor. +> **Relationship to #986 and the #1043 review.** The pluggable-providers +> spec in #986 (31 July) defines identity, device and geo as providers +> selected by `[ec] provider`, `[device] provider` and `[geo] provider` and +> wired by each adapter through a composition root. #1043 and #1044 +> implement that. The review of #1043 on 27 August asks instead that a +> vendor's identity provider be a capability declared on its integration +> registration, because a vendor ships its browser JavaScript and its +> identity function together. This revision adopts that end state (§3.6) +> and applies its rule consistently, so geo and device providers attach the +> same way. The lifecycle contract, the identifier envelope, the permission +> gating and the validation rules in #986 are unchanged. What changes is +> only where a vendor's provider is constructed and selected from. The +> registration shape needs a registry a vendor crate can register with, +> which is what §3.1 opens, so this spec precedes #1043 rather than +> following it. + ## 1. The problem, with the code that causes it -Every claim here was read from the tree at `split/5-response-hook-docs`, -which is `main` plus the five PRs. +Every claim here was read from `main` at b7fcb5d4c (28 August), which the +seventh PR targets; the five series PRs do not touch these files. 1. **The registry is closed.** `IntegrationRegistry::new` takes only `&Settings` and iterates a fixed table - (`crates/trusted-server-core/src/integrations/registry.rs:797`, table at + (`crates/trusted-server-core/src/integrations/registry.rs:792`, table at `crates/trusted-server-core/src/integrations/mod.rs:290`). Both `IntegrationBuilder` and `builders()` are `pub(crate)` with private fields, so no adapter and no external crate can add to the list. The @@ -40,7 +60,7 @@ which is `main` plus the five PRs. entries consumed by `bundle.rs`. `IntegrationRegistry::js_module_ids` only serves a module when `trusted_server_js::module_bundle(id).is_some()` - (`registry.rs:1169`), so an integration outside that compile-time map + (`registry.rs:1155`), so an integration outside that compile-time map gets no script however it registers. 3. **Startup validation names every vendor.** `validate_enabled_integrations` imports and calls each vendor's config type by name @@ -50,10 +70,10 @@ which is `main` plus the five PRs. the ad server mock. 5. **Two vendors reach further into core.** DataDome drives cache privacy and the origin fetch decision through a marker type - (`html_processor.rs:303`, `publisher.rs:4367` to `:4387`, + (`html_processor.rs:303`, `publisher.rs:4361` to `:4381`, `publisher.rs:2653`), and GPT diagnostics is called by name from all four adapters (for example - `crates/trusted-server-adapter-fastly/src/app.rs:584`). + `crates/trusted-server-adapter-fastly/src/app.rs:564`). The result is that the project carries nine vendors as core code (ten registered integrations, since GPT registers a proxy and a diagnostics @@ -88,7 +108,10 @@ Make the builder contract public and give the registry a second input. calls the companion with an empty slice, so no existing caller changes behavior. - Duplicate integration ids are a startup error, naming both sources, so a - vendor crate cannot silently shadow a built-in. + vendor crate cannot silently shadow a built-in. There is no such check + today, only a per-route conflict check and a debug-only assertion, and + `AuctionOrchestrator::register_provider` silently keeps the last writer, + so the builder carries a source label and both tables get the check. ### 3.2 Carrying browser JavaScript on the registration @@ -97,11 +120,22 @@ and `js_disabled`: the module source and its hash, both `&'static str`, so a crate can `include_str!` its own built bundle. `js_module_ids` keeps serving built-in ids from the compile-time map and -serves a carried module from the registration. `publisher.rs` composes the -served script and its hash from both sources rather than calling -`trusted_server_js::concatenate_modules` alone. The hash rule is unchanged, -so the served bundle stays cacheable and its integrity attribute stays -correct. +serves a carried module from the registration. The composition of the +served script moves from `trusted-server-js` into core, because every hop +after `js_module_ids` today re-enters `trusted-server-js` by id and silently +drops an id it does not know (`bundle.rs`, `concatenated_module_ids` and +`visit_concatenated_module_parts`), and the hash memo is keyed on the id +list alone. Core composes body and hash from (id, source, hash) triples +drawn from both sources, keeping the exact byte rule of today (core first, +`;\n` separator) so every existing `?v=` hash is unchanged. Three consumers +follow the registry rather than the compile-time list: the standalone +module route `parse_single_module_filename` (`publisher.rs`), the +`GPT_DIAGNOSTICS_INTEGRATION_ID` standalone special case, which becomes a +registration property, and `template_fingerprint`, which must cover carried +modules so a vendor crate rebuild invalidates the server-side template +cache. The served script keeps its cache rule, being the `?v=` query +matched at serve time (there is no integrity attribute on the tag today, and +this change adds none). ### 3.3 Startup validation on the registration @@ -109,8 +143,14 @@ Replace the named list in `config.rs` with a validation hook on the registration, so a vendor validates its own configuration and a missing vendor cannot silently stop being validated. The existing test that asserts every registered integration is covered by deploy validation -(`config.rs:408`) is rewritten against the hook, so the guarantee survives -in a vendor-neutral form. +(`config.rs:688` on `main`) is rewritten against the hook, so the guarantee +survives in a vendor-neutral form. Two details the map of `main` adds. The +enumeration the test needs is independent of which integrations a +configuration enables, so the registry exposes the full set of registrations +it was built from, not only the enabled ones. And `adserver_mock` is +validated today without being a registration (it exists only as an auction +provider), so auction-side registrations carry the same validation hook and +the test covers both tables. ### 3.4 Auction providers @@ -140,6 +180,67 @@ the neutral form. four adapters move behind hooks on the registration, so an adapter runs whatever its registrations declare. +### 3.6 Identity, geo and device as registration capabilities + +The rule. Things the host supplies are platform services, being the KV +store, the HTTP client, the host geo lookup, and the host TLS and HTTP/2 +signals. Things a vendor supplies are capabilities of that vendor's module. +An identity provider, a geo provider and a device provider are supplied by +vendors, with or without any host involved, so all three are module +capabilities, and the same registration carries them alongside the module's +JavaScript and hooks. + +- The registration builder gains three optional capabilities, at most one + of each per registration (names indicative, the shape is normative): + `.with_ec_provider(Arc)`, + `.with_geo_provider(Arc)` and + `.with_device_provider(Arc)`. The traits are the ones + #1043 and #1044 define, unchanged. +- Selection keeps the select-exactly-one semantics of #986. `[ec] provider`, + `[geo] provider` and `[device] provider` each name either a built-in (the + names #986 and #1043/#1044 already define, for example `hmac` and `none` + for identity) or the id of a registered module that declares the matching + capability. A selector that names a module which is registered but does + not declare the capability, or that names nothing registered, is a + startup error. A module that declares a capability the selector does not + name is inert for that capability and its other hooks still run, and + startup logs a warning naming the module and the unused capability, so an + operator can see a module shipping script for a provider that is not + selected. +- No provider is built into core. Everything goes through one method, so + the HMAC identity provider from #1043 and the User-Agent-only device + provider from #1044 become Tech Lab-owned modules in their own crates + under `crates/integrations/`, configured under `[integrations.]` and + validated through §3.3 like any other module, and the adapters register + them by default. Core keeps only the seam and the `none` state for each + capability (no identity, no location, unknown device signals). A + deployment that registers no identity module is stateless, as #986's + `provider = "none"` already means. +- Composition. The composition root resolves the selected provider for + each capability from the registry once at startup and places it in the + per-request services, so the request path is unchanged from #1043 and + #1044. Adapters stop injecting vendor providers directly (the + `ec_provider` slot on the runtime services builder and the injected + closures in `build_device_provider` and `build_geo_provider` go). Host + defaults are still supplied by the adapter as platform services and are + consumed by a built-in or a module through the request evidence and host + signal abstractions, exactly as now. A provider that needs a host signal + the running adapter does not expose is rejected at startup, as #986 + requires. +- A module that declares all three capabilities may share one backend call + per request across them, which is the shared-backend principle in + `CLAUDE.md`, and is the case that a split between a registry-attached + identity provider and platform-attached geo and device providers would + have made impossible. +- The host-signal device provider that #1044 ships as a separate crate is a + provider built on platform signals, so it registers as a module too. The + signals it reads stay platform. + +Effect on the series. #1043 and #1044 rework their construction and +selection path onto this section, move the HMAC and User-Agent-only +providers into module crates, and keep everything else. #1045, #1046 +and #1047 are unaffected beyond the rebase. + ## 4. Migration of the nine existing integrations One vendor per PR, after this change lands. Each moves its Rust, its @@ -157,13 +258,31 @@ TypeScript, its config type and its tests into `IntegrationSettings` is a flattened map that already accepts unknown vendor keys (`crates/trusted-server-core/src/settings.rs:166`). +Two more places every move must touch, found by mapping `main`: + +- `crates/trusted-server-core/src/migration_guards.rs` embeds every core + source file by relative path with `include_str!`, the thirteen vendor + files included, so a vendor move that leaves its entry behind breaks the + build rather than a test. The guard cannot derive its list from the + registrations, because `include_str!` paths are fixed at compile time, so + this change drops the nine vendors' files from the guard instead: a module + crate is outside the core neutrality guarantee, and a move then deletes + nothing there. +- The `ts audit` command carries its own vendor table (detection patterns + and configuration section names in + `crates/trusted-server-cli/src/commands/audit/analyzer.rs` and + `commands/audit/mod.rs`). It is outside the registry and outside this + change. Each vendor move takes its `ts audit` rows with it, and how the + CLI learns a vendor's detection pattern from a crate is a follow-up this + spec records but does not solve. + ## 5. What does not change The request pipeline, the hook traits and their order, the served script format and its hash, every `[integrations.*]` table, the permission model, -and the Edge Cookie, device and geo seams from PRs #1043 to #1046. No -integration changes behavior. A deployment that lists the same integrations -gets the same responses. +and the identity lifecycle, envelope and validation contracts from #986 as +implemented in PRs #1043 to #1046. No integration changes behavior. A +deployment that lists the same integrations gets the same responses. ## 6. Acceptance @@ -173,11 +292,17 @@ gets the same responses. right hash, runs its hooks in the right order, and is rejected on a duplicate id. A seam is only proven by an implementation that is not the built-in one. -2. **Parity.** The existing integration and parity suites pass unchanged, +2. **Capabilities round trip.** The same test integration declares an + identity, a geo and a device provider. With the three selectors naming + it, a request is served by all three (the minted identifier carries its + code, the resolved country and the device signals are its). With a + selector naming a module that lacks the capability, startup fails with + an error that names the module and the capability. +3. **Parity.** The existing integration and parity suites pass unchanged, because the built-in set still registers through the same path. -3. **No vendor left behind.** The rewritten deploy-validation test shows +4. **No vendor left behind.** The rewritten deploy-validation test shows every registered integration validates its configuration. -4. All CI gates in `CLAUDE.md`, on all four adapters. +5. All CI gates in `CLAUDE.md`, on all four adapters. ## 7. Risk @@ -185,23 +310,32 @@ The change is wide but shallow. It touches the registry, the served script path, deploy validation and four adapter entry points, and it changes no integration's behavior. The largest risk is the served script, where a mistake shows up as a wrong hash or a missing module, so §6's round trip -covers both. Doing this once is what removes the per-vendor core change +covers both, and the existing hash round-trip tests in `bundle.rs`, +`publisher.rs` and `tsjs.rs` pin every current `?v=` value. The second risk +is the renderer contract, where `BidRenderer::as_aps` is an exhaustive +single-arm match with eight test sites constructing the variant directly, +and the wire shape `{"type":"aps", ...}` must survive byte for byte. Doing this once is what removes the per-vendor core change that the project pays for today, most recently in PR #1054. ## 8. Sign-off -| # | Decision | Status | -| --- | ------------------------------------------------------------------------------- | -------------------- | -| 1 | Vendor integrations belong outside core, behind the registration contract | Proposed | -| 2 | Tech Lab engineering reviews vendor crates, and does not maintain them | Proposed, governance | -| 3 | A registration may carry its own browser JavaScript | Proposed | -| 4 | Deploy validation moves onto the registration | Proposed | -| 5 | The nine existing integrations migrate one PR each, on the schedule in §4 | Proposed | -| 6 | This change is complete in itself: after it, no vendor move needs a core change | Proposed | +| # | Decision | Status | +| --- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- | +| 1 | Vendor integrations belong outside core, behind the registration contract | Proposed | +| 2 | Tech Lab engineering reviews vendor crates, and does not maintain them | Proposed, governance | +| 3 | A registration may carry its own browser JavaScript | Proposed | +| 4 | Deploy validation moves onto the registration | Proposed | +| 5 | The nine existing integrations migrate one PR each, on the schedule in §4 | Proposed | +| 6 | This change is complete in itself: after it, no vendor move needs a core change | Proposed | +| 7 | Identity, geo and device providers are capabilities of a module registration (§3.6), the #1043 review's rule applied to all three | Proposed | +| 8 | No provider is built into core: HMAC and the User-Agent-only device provider are Tech Lab-owned modules configured under `[integrations.]`, and core keeps only `none` | Proposed | +| 9 | This spec and its core implementation precede #1043; 51Degrees implements the core seam, the nine vendor moves in §4 stay one PR each | Proposed | ## Revision record -| Date | Change | -| ---------- | ----------------------------------------------------------------------------------------------------------------------------- | -| 2026-08-27 | First draft, written against `split/5-response-hook-docs`. | -| 2026-08-27 | Brought the bid renderer contract into scope (§3.4), so that after this change no vendor move needs a core change (§8 row 6). | +| Date | Change | +| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| 2026-08-27 | First draft, written against `split/5-response-hook-docs`. | +| 2026-08-27 | Brought the bid renderer contract into scope (§3.4), so that after this change no vendor move needs a core change (§8 row 6). | +| 2026-08-28 | Corrected line references to `main` at b7fcb5d4c and added what mapping `main` found: composition of the served script moves into core (§3.2), registration enumeration and the auction-only `adserver_mock` case (§3.3), the duplicate-id gap (§3.1), the source-file guard and the `ts audit` vendor table (§4), the renderer risk (§7). | +| 2026-08-28 | Adopted the #1043 review's registration shape for identity and applied its rule to geo and device, with no provider built into core (§3.6, §6 item 2, §8 rows 7 to 9). Recorded the relationship to #986 and reordered the series so this spec and its implementation come first. | From 9b5b328423e51f51bfea1c7ceb0816293f30b614 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 28 Aug 2026 21:32:36 +0100 Subject: [PATCH 03/17] Record what implementing the integration seam found A probe integration built outside core and registered through an adapter exercised every seam end to end. Four things surfaced that reading the code did not: the operator CLI reaches deploy validation through a type that supplies no builders, so a vendor's own rules are skipped on the path an operator uses; a carried browser module's hash literal is hand-maintained and breaks on a checkout that rewrites line endings; a provider is resolved more than once per request, which the shared-backend principle needs a per-request context to fix; and one core reader still reads the APS renderer payload directly. Also records the construction-time verification of a carried module's declared hash. --- ...-08-27-integration-provider-seam-design.md | 85 ++++++++++++++++++- 1 file changed, 82 insertions(+), 3 deletions(-) diff --git a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md index b86730f44..373627392 100644 --- a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md +++ b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md @@ -133,8 +133,13 @@ module route `parse_single_module_filename` (`publisher.rs`), the `GPT_DIAGNOSTICS_INTEGRATION_ID` standalone special case, which becomes a registration property, and `template_fingerprint`, which must cover carried modules so a vendor crate rebuild invalidates the server-side template -cache. The served script keeps its cache rule, being the `?v=` query -matched at serve time (there is no integrity attribute on the tag today, and +cache. The registry verifies a carried module's declared hash against its source +when it is built, so a stale literal is a startup error rather than a +stale script served under a valid-looking URL. Covering carried modules in +the template cache fingerprint means the publisher entry point needs the +registry, so it takes the configuration and the registry as one argument +rather than two. The served script keeps its cache rule, being the +`?v=` query matched at serve time (there is no integrity attribute on the tag today, and this change adds none). ### 3.3 Startup validation on the registration @@ -317,7 +322,80 @@ single-arm match with eight test sites constructing the variant directly, and the wire shape `{"type":"aps", ...}` must survive byte for byte. Doing this once is what removes the per-vendor core change that the project pays for today, most recently in PR #1054. -## 8. Sign-off +## 8. What implementing this found + +A probe integration built outside `trusted-server-core` and registered +through an adapter exercised every seam end to end. Four things surfaced +that reading the code did not, and they are recorded here rather than left +for each vendor to rediscover. + +1. **A vendor's own deploy rules do not run through the operator CLI.** + `ts config validate` and `ts config push` reach validation through + `TrustedServerAppConfig`, which supplies no builders, so a vendor's + `[integrations.]` rules are skipped on exactly the path an operator + uses. The validation hook in §3.3 is only real once that path can carry + the builders a deployment was composed with. This needs a decision: + either the CLI is built per deployment with its vendor crates, or the + adapter validates at startup and the CLI checks only what core owns. +2. **A carried module's hash is hand-maintained and line-ending fragile.** + Core's own modules get their hashes generated at build time. A vendor + crate keeps a literal beside its `include_str!`, and on a checkout that + rewrites line endings the embedded file changes and the literal stops + matching, which fails startup on that machine only. A generated helper + or a documented build-script recipe removes the trap; the probe pins the + file's line endings and tests the literal, which every vendor would + otherwise have to reinvent. +3. **A provider is resolved more than once per request.** A proxy that + wants the resolved location calls the geo provider itself while the + request path has already called it. `CLAUDE.md`'s principle that a + vendor sharing one backend makes a single call per request needs a + per-request provider context to hang that on, which this change does not + introduce. +4. **One core reader still reaches into a vendor's payload.** The + `hb_adid` fallback in the publisher reads the APS renderer's fields, so + the APS migration needs a neutral answer for it rather than only the + seam in §3.4. + +5. **Request preparation covers different routes on each host.** Every + adapter runs preparers before routing, but not on the same set of + routes: one runs them on every route but the health check, one skips a + batch endpoint and its admin diagnostics deliberately, one covers three + of its paths, and one skips its inline admin stubs. A module that strips + its own reserved query or cookie is therefore protected on a different + set of routes depending on the host it is deployed to. Making that + uniform means routing each adapter's hand-written handlers through one + wrapper, which is worth doing before a vendor depends on it. + +6. **A module's validate function runs nowhere in a real deployment.** + Building the registry calls only a builder's build function, and the + operator CLI calls deploy validation without the builders, so a vendor + whose checks live in `validate` has them enforced on no path at all. The + probe works around it by repeating its check inside its build function, + which every vendor would have to copy. Either the registry runs + `validate` when it builds, or the operator path carries the builders, + and the second is item 1. This is the same gap as item 1 seen from the + other side, and together they mean §3.3 is not yet delivered in + practice even though the hook exists. +7. **The Fastly adapter cannot take a vendor crate at all.** Its + `build_state_with_registrations` is crate-private and it has no library + target, so composing a module into a Fastly deployment means editing the + adapter. The other three adapters expose both entry points. Fastly is + the primary deployment target, so this one decides whether the seam is + usable in production or only in the dev server. + +Items 1, 6 and 7 are the ones a vendor meets on its first day, and item 7 +decides whether any of this is reachable on the platform most deployments +use. Item 5 is the one that produces a bug report nobody can reproduce, +because whether it appears depends on which host the reporter runs. + +Taken together these say the seam is proven but not yet finished. A vendor +can register a module, ship its browser code, declare a geo provider and +serve a route, all from its own crate and proven end to end. It cannot yet +do that on Fastly, and its own configuration rules are not enforced +anywhere. Both are small changes against what this document already +defines, and both should land before the first vendor is asked to use it. + +## 9. Sign-off | # | Decision | Status | | --- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- | @@ -338,4 +416,5 @@ that the project pays for today, most recently in PR #1054. | 2026-08-27 | First draft, written against `split/5-response-hook-docs`. | | 2026-08-27 | Brought the bid renderer contract into scope (§3.4), so that after this change no vendor move needs a core change (§8 row 6). | | 2026-08-28 | Corrected line references to `main` at b7fcb5d4c and added what mapping `main` found: composition of the served script moves into core (§3.2), registration enumeration and the auction-only `adserver_mock` case (§3.3), the duplicate-id gap (§3.1), the source-file guard and the `ts audit` vendor table (§4), the renderer risk (§7). | +| 2026-08-28 | Recorded what implementing the seam found (§8): the operator CLI skips a vendor's deploy rules, a carried module's hash literal is fragile, providers resolve more than once per request, and one core reader still reads an APS payload. Recorded the construction-time hash check in §3.2. | | 2026-08-28 | Adopted the #1043 review's registration shape for identity and applied its rule to geo and device, with no provider built into core (§3.6, §6 item 2, §8 rows 7 to 9). Recorded the relationship to #986 and reordered the series so this spec and its implementation come first. | From 61eef6bd41e5c346017dfe3584795869ae48c8a7 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 30 Aug 2026 08:29:01 +0100 Subject: [PATCH 04/17] Move the provider series design specs ahead of their implementation The review of #1043 asked that spec changes land before the code that implements them, so a divergence is a decision taken in review rather than a ratification of something already merged. PRs #1043 to #1047 each carried the design document for their own step, and #1043 carried a 607-line spec describing device providers, geo providers, the permission model and the browser resolve endpoint, none of which is in that PR. Move all six series documents here, so this PR carries the complete normative set and no code: - 2026-07-30-pluggable-providers-design.md (from #1043) - provider-code-registry.md (from #1043) - 2026-07-30-permission-model-design.md (from #1045) - 2026-07-30-client-cycle-ec-resolve-design.md (from #1046, later revised by #1047) - 2026-07-30-integration-response-header-hook-design.md (from #1047) - 2026-07-30-provider-migration-rollout-design.md (from #1047) Each file is taken verbatim at the tip of the stack, so the later revisions are preserved: the provider-switching continuity section, the geo requires-signal floor, and the code-envelope paragraph #1047 added to the client-cycle spec. The revision-record tables are unchanged. No document's substance was edited. The only edits are to this spec's own status line, which said the PR adds one document and that the series specs land with #1047, and a revision-record row recording the move. --- ...26-07-30-client-cycle-ec-resolve-design.md | 260 ++++ ...integration-response-header-hook-design.md | 1219 +++++++++++++++++ .../2026-07-30-permission-model-design.md | 826 +++++++++++ .../2026-07-30-pluggable-providers-design.md | 641 +++++++++ ...07-30-provider-migration-rollout-design.md | 516 +++++++ ...-08-27-integration-provider-seam-design.md | 10 +- .../specs/provider-code-registry.md | 30 + 7 files changed, 3498 insertions(+), 4 deletions(-) create mode 100644 docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md create mode 100644 docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md create mode 100644 docs/superpowers/specs/2026-07-30-permission-model-design.md create mode 100644 docs/superpowers/specs/2026-07-30-pluggable-providers-design.md create mode 100644 docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md create mode 100644 docs/superpowers/specs/provider-code-registry.md diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md new file mode 100644 index 000000000..6949889a3 --- /dev/null +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -0,0 +1,260 @@ +# Design Spec: Client-Cycle Edge Cookie Providers and the Resolve Endpoint + +**Status:** Implemented (hardened v1) in PR #1046, on the threat model below. +The full anti-replay reservation machinery (§3.9) and the vendor envelope +verification it serves are deliberately not in v1: they land with the first +real vendor scheme, which brings the concrete envelope format the +reservation design must fit. §8 records exactly what v1 implements, what it +defers, and why the feature is normative rather than deferred. +**Author:** Engineering (revised against the implementation, 2026-08-25) +**Issue references:** #778 (series), successor spec of the 2026-07-31 draft +**Related specs:** `2026-07-30-pluggable-providers-design.md` +**Last updated:** 2026-08-25 + +> **Context.** PR #838 shipped, undeclared and unspec'd, a second provider +> _type_: a "client-cycle" EC provider whose identifier is established by a +> browser POST to a new public endpoint (`POST /_ts/api/v1/ec/resolve`), +> plus a demo provider (`client-fixed`) and a JS bundle. Review found the +> endpoint accepted cross-origin identity-setting posts with no origin +> check, minted cookies with no identity-graph row (violating an invariant +> the organic path enforces explicitly), was registered on only one of four +> adapters, and could never round-trip because the core did not recognize +> non-HMAC identifiers. None of that is an argument the feature is a bad +> idea. Vendor identity systems with a browser leg (for example +> signed-envelope schemes) are a real integration target, and for this +> project the client-side path is the preferred route for the first vendor +> integration. It is an argument that the feature needs a threat model +> before an implementation. This spec is that threat model; PR #1046 is the +> implementation measured against it. + +--- + +## 1. Overview + +A **client-cycle** EC provider establishes the identifier via a browser +round trip: server-injected first-party JS obtains or derives a value in the +page (typically a signed envelope from a vendor identity system), posts it to +a Trusted Server endpoint, and the endpoint, after provider-specific +verification, sets the first-party `ts-ec` cookie. + +This differs from server-side providers in one security-critical way: **the +identifier is attacker-influenceable input**, not server-derived evidence. +Everything in this spec follows from that. + +Why this feature is normative in the series rather than deferred: the first +vendor integration this project targets works client-side by design, since the +page script talks to the vendor's identity system and hands the result to +the edge, so the server-side path alone cannot carry it. The 2026-07-31 +draft deferred the feature for lack of a consumer; the consumer now exists +as a planned vendor provider, and v1 builds the endpoint that provider will +verify against. + +## 2. Threat model + +| Threat | Vector | Consequence if unmitigated | +| -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Cross-site identity fixation** | `text/plain` POST is a CORS-simple request: any page on the web can `fetch(resolveUrl, {method: "POST", credentials: "include", body: payload})` with no preflight | An attacker pins a chosen identity onto a victim's first-party cookie jar, a login-CSRF for the ad-identity layer, and the victim's activity accretes to an attacker-controlled ID | +| **Replay** | A captured valid payload (from the attacker's own session or a leak) replayed against another browser | Same as fixation, without needing to mint payloads | +| **Phantom identity** | Endpoint sets the cookie without an identity-graph row | Later requests carry an EC that the KV graph has never seen; downstream sync and withdrawal logic operate on an identity that half-exists (the organic generation path explicitly refuses to write a cookie when the graph write fails, for exactly this reason) | +| **Un-tombstoneable identity** | Core does not recognize the provider's identifier shape | Withdrawal cannot expire or tombstone the identity, which is a compliance failure and not only a bug | +| **Amplification** | The page script cannot observe an HttpOnly cookie, so it cannot know the cookie is already set | A POST on every page view of every session (PR #838's JS gated on reading a cookie its own server marked HttpOnly, making the guard permanently false) | + +## 3. Requirements on the endpoint + +`POST /_ts/api/v1/ec/resolve` MUST: + +1. **Reject cross-site requests with an origin check.** v1 authorizes a + request only when its `Origin` header names the publisher's configured + domain or a subdomain of it; a missing or foreign `Origin` is rejected + with `403`. Browsers always send `Origin` on POST `fetch`, so its + absence means a non-browser caller, which has no business on a + page-script endpoint. The 2026-07-31 draft asked for an exact origin + allowlist as new configuration plus an optional session-bound CSRF + token; v1 derives the allowed set from `publisher.domain` (existing, + operator-trusted configuration) and admits the publisher's own + subdomains, because the publisher controls their subdomain namespace, and the + draft's sibling-subdomain concern applies to `Sec-Fetch-Site: +same-site` (which v1 does not consult at all), not to a suffix match on + the publisher's own apex. An explicit multi-origin allowlist remains + open (§7.5) for publishers whose pages run on domains other than the + configured apex. +2. **Verify the payload per provider.** The endpoint hands the posted + payload to the selected provider's `resolve_from_client` and mints only + what the provider returns; whether the payload is trustworthy is the + provider's responsibility, stated on the trait. A real vendor provider + verifies a signed, audience-bound, expiring envelope; the draft's + session-binding and replay analysis (see §3.9) is the bar that + verification must clear when the vendor scheme lands. The demo provider + verifies a fixed constant and is compiled out of production builds + (§5). `resolve_from_client` is normative in v1, with a no-op default so + server-side providers are untouched. +3. **Preserve the identity-graph invariant.** Implemented: the graph row is + written before the cookie is set, keyed by the provider's + `normalize_id_for_kv` canonical form, exactly like the organic mint + path. No graph available → no mint (`204`), same as organic generation; + a graph write failure → `503`, no cookie. +4. **Round-trip through the lifecycle contract.** Implemented: read-back + goes through the selected provider's `accepts_id`, the KV key through + `normalize_id_for_kv`, and withdrawal reaches the row like any other + identity. A round-trip test drives an opaque client identifier through + organic deferral, resolve, cookie set, and verbatim read-back. +5. **Exist on every adapter, where parity means identical behavior, + including identical refusal.** Partially implemented, documented: the + Fastly adapter routes the endpoint (passing the same bot-gated identity + graph as organic generation, so unrecognized clients cannot mint + through resolve either). The Axum, Cloudflare, and Spin adapters + deliberately do not route it, matching `identify` and `batch-sync`, + which need the same platform KV wiring those adapters do not have; the + route list in the Spin adapter documents all three together. The + draft's stronger ask, identical startup rejection of the client-cycle + selection on adapters that cannot serve it, is the agreed follow-up + when the portability adapters gain KV (§7.4). +6. **Be uncacheable and permission-gated on the provider's full + declaration.** Implemented: every response carries `Cache-Control: +no-store`, and the gate is the selected provider's complete + `required_permissions()` through the same resolved permission state as + organic minting, not a hard-coded storage check. +7. **Bound every input.** Implemented: request body at most 65,536 bytes, where + an advertised `Content-Length` over the limit answers `413` before the + read, and the read body is re-checked so a missing or false length does + not bypass the bound. `Content-Type` allowlist: `text/plain` and + `application/json`, matched on the media type alone, case-insensitively, + ignoring parameters (the browser's default `text/plain;charset=UTF-8` + passes); anything else → `415`. The minted identifier must fit the + global identifier bounds (at most 256 bytes, cookie-safe alphabet, + shared with every other mint path); violation → `400`, never a rewrite. + Core then applies the provider's registered code envelope + (`provider-code-registry.md`): the cookie and the identity-graph key + carry `{code}~value`, so a client-set identity is namespaced to its + provider exactly like an edge-minted one (the demo's cookie value is + `cfix~an-ec`). Status codes are part of the contract: `400` out-of-bounds identifier, + `403` origin rejection, `409` different-identity conflict, `413` body, + `415` content type, `503` graph-write failure, `204` closed gate / no + provider / no graph / unverified payload. Tests exercise each rejection. +8. **Define behavior against an existing identity, with no silent + replacement.** Implemented: resolving to the same identity refreshes + idempotently; resolving to a different identity while the request + carries a recognized EC is rejected with `409`. Any legitimate + re-identification flow (account link, vendor migration) is an explicit + linking design this spec does not authorize (§7). +9. **Replay consumption and graph persistence as one idempotent + sequence.** Not in v1, by decision rather than omission. The draft's + reservation design (atomic single-key CAS reservation with owner hash, + lease epochs, fenced transitions, and family-epoch revocation + linearization) presupposes a payload with a unique id, a session + binding, and a validity window, which are properties of a concrete vendor + envelope format that does not exist yet, and a CAS-class storage + primitive no production adapter exposes today. Designing the + reservation against a hypothetical envelope would repeat the mistake + this series exists to fix. v1's stance: the endpoint is safe without it + for the providers v1 ships (the demo mints a constant, feature-gated + out of production), and the reservation lands with the first vendor + scheme, designed against its real envelope, with the draft's §3.9 as + the starting bar. Until then the draft's text is preserved below as the + agreed requirement. + +The 2026-07-31 draft's §3.9 reservation requirement is retained verbatim as +the bar for the vendor-scheme implementation: + +> Neither naive order works: consume-the-nonce-first makes a subsequent +> graph failure unretryable (the token is spent, the identity never +> existed); graph-first lets the losers of a replay race leave residual +> rows. Required shape: consumption is an atomic single-key reservation +> (CAS) keyed by the payload's unique id, with explicit states: `pending` → +> `committed` | `failed`, each carrying an owner hash (the session binding) +> and a monotonic lease epoch. Takeover of an expired `pending` lease +> increments the epoch, and every state transition is a fenced CAS on +> (state, epoch). `failed` is retryable by the same owner at a higher +> epoch. The graph write happens under the reservation and is deterministic +> under its key. A duplicate must never receive the cookie unless it proves +> the original session binding; a requester matching the reservation's +> owner hash has the `Set-Cookie` re-emitted (lost-response recovery), +> anyone else gets a terminal response with no cookie. The same-identity +> no-op first checks the family revocation record, and "revocation wins" is +> enforced by a CAS conditioned on the family epoch read at the start. + +## 4. Requirements on the page script + +- **The re-post guard must not depend on reading an HttpOnly cookie.** + Implemented as the draft's first option: the resolve response sets a + non-HttpOnly companion marker cookie (`ts-ecr=1`) carrying no identity, + which is the only signal the page has that a resolve succeeded. The + marker shares the Edge Cookie's scope and lifetime and is expired + together with it on withdrawal, so a visitor who later re-establishes + the permission can resolve again. +- **The page leg is permission-gated before vendor contact.** The demo + module contacts no vendor (it posts a constant), so the draft's + injection-time and live-CMP gating requirements bind the first vendor + module, not v1: a vendor module must not derive identity or contact the + vendor for a visitor whose resolved permissions do not satisfy the + provider's declaration, must re-check immediately before vendor contact + (consent can change between document delivery and asynchronous vendor + contact, including BFCache restoration), and its injection is keyed off + the provider selection exactly as the demo's is today. +- The JS module ships through the standard integration bundle mechanism, + loaded only when a client-cycle provider is the selected EC provider. + The interaction between provider-keyed bundle content and content-hash / + SRI pinning remains open (§7.6). +- **Any constant shared between Rust and TS is asserted equal by a test.** + Implemented: a Rust test reads the page-script source and asserts the + fixed word and the marker cookie name match their Rust constants, so a + rename on either side fails the build instead of silently breaking the + round trip. + +## 5. Demo providers + +Implemented as required: the `client-fixed` demonstration provider (fixed +identifier, constant-equality verification) is compiled only behind the +`client-fixed-demo` cargo feature. In a production build the settings +validator rejects the selection at startup with a direct message, and the +provider builder rejects it again as defense in depth. A fixed shared word +is not an identity; the demo exists to exercise verify-before-mint end to +end in tests and demonstrations. + +## 6. Testing + +- Endpoint unit tests cover: origin rejection (missing and foreign), + subdomain acceptance, content-type rejection, the body bound, the + identifier bound, the different-identity conflict, the no-graph refusal, + the closed permission gate, the unverified payload, and the marker, + cache-control, and graph-row effects of a success. +- A round-trip test drives a client identifier through organic deferral, + resolve, cookie set, and verbatim read-back recognition. +- The cross-language constant test pins the endpoint's shared constants to + the page-script source. +- Remaining for the vendor scheme: a real-browser integration round trip + (JS → POST → Set-Cookie → next request recognized) in the browser suite, + and the §3.9 reservation tests (crash-between-steps, lease takeover, + concurrent duplicates, owner-hash recovery, resolve-vs-revocation + races). + +## 7. Open questions, carried to the vendor-scheme issue + +0. The commit path spans keys (reservation, identity row, family-epoch + CAS): the cross-key atomicity or saga/compensation design is undefined, since + the single-key CAS steps are specified, their composition is not. +1. The first vendor scheme's envelope format, measured against §3.2 and + §3.9 (audience binding, expiry, unique id, session binding). +2. Whether the resolve flow needs consent-state echo in its response, and + the minimal disclosure if so. +3. Rate limiting / abuse posture at the edge for an unauthenticated POST. +4. Startup rejection of the client-cycle selection on adapters that cannot + route the endpoint, once the portability adapters gain platform KV. +5. An explicit multi-origin allowlist for publishers whose pages run on + domains other than the configured apex (v1 authorizes the apex and its + subdomains). +6. How JS module selection keyed off EC provider configuration coexists + with content-hashed/SRI-pinned bundles, meaning per-config hashes, cache + keying, and the config-push story for them. + +## 8. Revision record vs the 2026-07-31 draft + +| Draft position | v1 (PR #1046) | Why | +| ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | +| Feature deferred, `resolve_from_client` de-normalized | Feature normative; trait method kept with a no-op default | The first vendor integration works client-side by design, so the endpoint is on the critical path for the series' first real provider | +| Origin check via new allowlist config + CSRF token | Publisher apex + subdomains from existing `publisher.domain`; missing/foreign `Origin` → `403` | Uses existing operator-trusted configuration; explicit allowlist stays open for multi-domain publishers (§7.5) | +| §3.9 reservation/replay machinery required before code | Deferred to the vendor scheme; draft text retained verbatim as the bar | The design needs the real envelope's unique id and session binding, and a CAS-class primitive no production adapter exposes today | +| Identical behavior or identical startup refusal, 4 ways | Fastly routes it (bot-gated graph); portability adapters documented as deliberately not routing, like identify | Same platform-KV constraint as the existing EC API routes; startup rejection follow-up recorded (§7.4) | +| Marker cookie or injected variable (design must pick) | Marker cookie (`ts-ecr`), expired with the EC cookie | The draft's own first option; testable and observable | +| Demo gated by cargo feature or `#[cfg(test)]` | Both: `client-fixed-demo` feature + startup rejection in the settings validator | Defense in depth | +| Everything else in §3 (graph row, bounds, 409, no-store, full-declaration gate) | Implemented as specified | (none) | diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md new file mode 100644 index 000000000..0182495fb --- /dev/null +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -0,0 +1,1219 @@ +# Design Spec: Integration Response-Header Hook + +**Status:** Not implemented in the series. PR #1047 carries the +documentation set only; the hook was removed from the earlier draft of that +PR because it has no consumer, which is this spec's own §-rule for +speculative surface. The spec is retained as the design bar for the hook +when its first consumer arrives (an integration that must set response +headers such as Accept-CH or detection results). Revised 2026-08-25. +**Author:** Engineering +**Issue references:** #782 +**Related specs:** `2026-07-30-pluggable-providers-design.md` and the +baseline DataDome design +`2026-06-11-datadome-server-side-protection-design.md` +**Last updated:** 2026-08-25 + +> **Context.** Issue #782 already specifies this feature well; its done-when +> is the contract. PR #838 shipped the trait and registry wiring with **no +> adapter call site**. `apply_response_headers` had zero production +> callers, so the feature existed only in its own unit test. This short spec +> restates the contract plus the two details the issue left open (ordering +> and collision policy), and adds the rule that prevents a repeat, which is +> that the hook lands with a consumer or not at all. The implemented series +> (PRs #1043-#1047) applied that rule to this spec itself, see the Status +> above and §7. + +--- + +## 1. Overview + +The pre-existing DataDome design remains unchanged. For implementations +governed by this spec, §4a is the normative PR-specific delta and supersedes +that baseline wherever its generic request/effect API, header ordering, +session-by-header behavior, endpoint/request surface, cookie lifecycle, +challenge transport, or response-pointer behavior conflicts. The baseline +continues to provide historical context; it is not a second normative source +for those surfaces. + +Integrations can today rewrite request-path behavior (proxies, attribute +rewriters, head injectors), and a request filter can name response headers +to set through `RequestFilterEffects.response_headers`, but no integration +can inspect a response and decide its headers from it. The hook will add +that capability. An integration registers a response-header mutator +via its `IntegrationRegistration` builder, and every adapter applies all +registered mutators to the outbound response for HTML document responses it +processed. + +## 2. Contract + +- `IntegrationRegistration::builder(ID).with_response_mutator(...)` registers + a mutator; `IntegrationRegistry::apply_response_headers(...)` applies all + registered mutators in registration order. +- Every registration carries a nonzero `behavior_revision: u32`, bumped for + any change to its response decision, operation semantics, declared read set, + or security field list. Integration IDs match + `[a-z0-9][a-z0-9-]{0,63}` and are unique. The registry revision hashes the + ordered registration list. Order is behavior, so the array is never sorted. +- **The mutator API is structured operations, not header-map access.** A + mutator returns (or is handed a recorder for) typed operations, + `append(name, value)` and `replace(name, value)` (v1 is headers-only; + the cookie operation arrives with the deferred cookie surface, §3), + which **core validates and applies**, + attributing each to its integration id. PR #838's shape handed the + integration an unrestricted `&mut HeaderMap`, which makes §3's collision + policy unenforceable by construction: core cannot validate or attribute + writes it never sees. An API that cannot express a violation beats one + that promises to catch it. +- **Every adapter calls the apply point** on its outbound-response path for + processed documents. The call site lives in shared response-finalization + code where one exists; where adapters finalize independently, each adapter + gains the call and a test proving it. +- **Ordering is three stages, and the last one is inviolable:** core + response-header handling (EC Set-Cookie emission, EC header clearing, + privacy headers) → integration operations → **final cache/privacy + invariant enforcement**, which no integration operation can override. + Running the hook dead-last would be wrong: current `main` deliberately + runs cookie-cache protection _after_ arbitrary header changes, stripping + surrogate caching and forcing private/no-store on any response that sets + a cookie, a hook applied after that recheck could combine an appended + `Set-Cookie` with a replaced public `Cache-Control` into a + **shared-cacheable cookie response**. The invariant pass therefore runs + after all mutations, unconditionally, and it enforces more than the + cookie rule. Core **snapshots the complete pre-hook cache restriction + state**, whether the restriction came from core's own classification + (processed auction HTML is marked private even when no cookie is + emitted; today's final helper returns early without `Set-Cookie`) **or + from the origin** (an origin-supplied `private, no-store` that core + merely passed through), and the post-hook response may only be + **equal or stronger** on the privacy axis: integrations can tighten + caching, never loosen it, regardless of which header they replaced. + "Equal or stronger" is a defined merge over **independent sticky + directives, not a totally ordered lattice**, because `no-cache` and `private` + are orthogonal constraints (RFC 9111: `no-cache` permits shared + storage subject to revalidation; `private` forbids shared storage), so + "replace `private` with the stronger `no-cache`" would make a + personalized response shared-storable. The merge: each of the **six sticky directives**, `no-store`, `no-cache`, + `private`, `must-revalidate`, `proxy-revalidate`, `no-transform`, is + present-in-snapshot-or-mutation ⇒ present-in-final (this + "snapshot or mutation ⇒ final" rule scopes to exactly these six), with two refinements. First, `must-understand` is the deliberate + exception: **mutation-introduced `must-understand` is rejected** + (snapshot-present survives untouched), because under RFC 9111 + §5.2.2.3 a cache that understands the status may then ignore an + accompanying `no-store`, "adding" it weakens a stored `no-store`, so + it is not additive. Second, stickiness is **form-preserving, not + name-presence-only**: an **unqualified directive never becomes + field-qualified, and a qualified field set never shrinks**, where + `private` → `private="Set-Cookie"` keeps the directive name while + authorizing shared storage of everything but one field (RFC 9111: + qualified `private`/`no-cache` have materially weaker semantics), so + a mutation supplying a qualified form where the snapshot is + unqualified keeps the snapshot's bare form, and qualified snapshot + sets may only grow; `public` is + dropped whenever any restriction is present; **request-side authority + is part of the invariant**. If the request carried `Authorization` + and the origin did not itself authorize shared reuse (no `public`, + `must-revalidate`, or `s-maxage` from the origin), an integration may + not introduce `public`, `must-revalidate`, or `s-maxage` (RFC 9111 + §3.5 makes those the very directives that unlock shared caching of + authenticated responses); the invariant pass forces `private, +no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may + only shrink relative to the snapshot; `stale-while-revalidate`/ + `stale-if-error` may appear only if the snapshot had them **and their + durations may only shrink** (present-at-1s must not become + present-at-1y); CDN-specific cache fields are **reserved outright, by enumerated + name in the field registry**, `Surrogate-Control`, + `CDN-Cache-Control`, `Cloudflare-CDN-Cache-Control`, and + `Edge-Control`, each individually tested ("host equivalents" was not + a matching rule four adapters would implement identically). + Merging them per-directive on unrestricted responses was a hole (an + unrestricted `CDN-Cache-Control: max-age=60` could become a year), and + they are additionally stripped from any restricted response; and the + final `Vary` is the **union of the complete snapshot `Vary` set**, + origin-supplied members included, not only core-required ones, and + the mutation. Ordering is normative so the final `Vary` reaches TS's + own cache key, not just the wire: **mutation/invariant → final `Vary` + computation → cache-key construction → body/metadata commit**, with + three identity rules. Cache matching uses the **exact final publisher + request** (post-overlay view; keying from the redacted view would + collapse personalized variants). + + The final `Vary` name list has one cross-adapter grammar. Core collects every + `Vary` response field line from the snapshot plus mutation, parses each as an + HTTP comma-list, trims optional whitespace around every member, and requires + each non-empty member to be an RFC field-name token. It lowercases ASCII, + removes duplicates, and sorts unique names by unsigned ASCII byte order. An + empty member or invalid token introduced by a mutation invalidates that + batch. If the reverted snapshot itself is malformed, the invariant replaces + the final value with `Vary: *`, forces `no-store`, and writes no cache + artifact. `*` in either source likewise dominates every other member: the + normalized result is the single `*` and is uncacheable. For an ordinary + list, the wire response emits one lowercase `, `-joined value, while the + variant descriptor stores `vary_names` as the exact sorted JSON array of + lowercase strings. Thus field-line grouping, case, and input ordering cannot + produce different cache identities. + + A normalized name nominates one request header. Digest construction obtains + **all** values for that header from the exact final publisher request after + overlay, preserving received field-line order and value octets; it does not + comma-fold, trim, or split those request values. The `` bytes in the + HMAC input below are always the normalized lowercase ASCII name, and a name + is digested once even if it appeared repeatedly in `Vary`. Known-answer + normalization fixture: snapshot/mutation lines `Vary: X-Tenant , + Accept-Encoding` and `Vary: accept-encoding` produce wire value + `accept-encoding, x-tenant` and descriptor value + `["accept-encoding", "x-tenant"]`, then digest each nominated request + field's original instances in their received order. Fixtures also cover + differently grouped lines, case variants, duplicate names, empty members, + invalid tokens, `*`, absent versus present-empty request fields, and one value + containing a comma. + + **Every** `Vary`-nominated request value is stored **only as a keyed + digest**, every value, not a sensitivity classification an unknown + credential field could slip past: HMAC-SHA-256 with domain tag + `tsvry1|`, over an input that encodes **presence, instance count, and + length-prefixed octets**, since the earlier comma-join grammar had + deterministic collisions (absent hashed identically to + present-but-empty, violating RFC 9111 §4.1's absence-matches-only- + absence, and two members `a`,`b` collided with one member `a,b`), + which could select a representation built under a different + credential or tenant. Absent field → `tsvry1||a`; present → + `tsvry1||p|` with `count` as ASCII decimal, then, per member in received order, + `|:` with `len` the ASCII-decimal byte count. Output + lowercase hex (64 chars). **The key is a deployable contract, not an + implementation detail**: the setting + `[cache] vary_digest_key_secret_name` names a platform-secret-store + entry containing one versioned keyring JSON object: + `{ "schema_version": 1, "current_key_id": "", "keys": + [{ "id": "", "key_base64url": "" }] }`. `keys` is + sorted by `id`, contains 1..4 unique entries, rejects unknown fields, + and every value decodes to exactly 32 CSPRNG bytes. An id is derived, + not operator-invented: the first 16 lowercase hex characters of + SHA-256 over the raw key bytes; a supplied mismatch or one id bound to + different bytes is fatal. The current id must exist in the array. + Every cache entry stores its id; raw keys never enter config, cache + artifacts, logs, or metrics. Startup **fails** when response caching is + enabled and the keyring/current key does not resolve (digests are never + computed unkeyed). + + Lookup uses a stable per-representation **variant index** so the key ID is + discoverable before the variant artifact is addressed. The base index key is + the cache tuple excluding `Vary` values. Each bounded index descriptor stores + only the normalized final `Vary` name list, key ID, corresponding keyed + digests, artifact key, artifact expiry, and artifact revision tuple, never + raw request values. A reader loads the index, groups descriptors by key ID, + resolves each referenced key (one atomic keyring refresh if an ID is + unknown), recomputes the digests from the exact final publisher request, and + fetches only a descriptor whose complete name/digest tuple matches. A + still-unknown ID, malformed descriptor, missing artifact, expiry, or revision + mismatch is a miss for that descriptor, never a comparison under another + key. Multiple matching descriptors are corruption and make the entire base + lookup a miss with a metric; index order never chooses a winner. + + Publication writes and verifies the immutable artifact first, then CAS-adds + or replaces its complete descriptor in the index. Rekeying or a changed + `Vary` set inserts the new artifact/descriptor before removing the old + descriptor; a crash may leave a safely unreachable artifact or two + nonmatching descriptors but cannot point to a partial artifact. Index + capacity eviction removes expired descriptors first and otherwise the + least-recently-used complete descriptor; it never rewrites a digest under a + new key ID. This is the one meaning of “insert-new-then-index-update rekey” in + the capability matrix and 304 rules. + + Rotation atomically replaces the secret entry with a new valid keyring + containing the new current key **and all still-live previous keys**. Fleet + propagation may be mixed only in the safe direction: a process with the old + keyring can write/read the old id; the variant index exposes that id before + lookup, so a process seeing an unknown descriptor id refreshes the keyring + once, then treats a still-unknown descriptor as a cache miss. It never + guesses, probes with the current key, or computes unkeyed. A previous key may + be retired only after no unexpired index descriptor references it **and** the + maximum processed-artifact lifetime plus the adapter's qualified keyring + refresh bound has elapsed since it stopped being current. Key IDs are never + reused. Secret replacement atomicity, maximum propagation/refresh time, and + unknown-id refresh are explicit adapter capability cells and startup gates; + an unqualified adapter disables response caching rather than weakening the + grammar. Cross-adapter fixtures cover old→mixed→new propagation, unknown-id + refresh/miss, premature retirement rejection, and id/material mismatch. + Known-answer vectors under the all-zero 32-byte test key: + `tsvry1|authorization|p|1|10:Bearer abc` → + `c880c5e8c36febc0b1581c92f1d598fded34391626e67372ed63b2857d8a7b6b`; + absent-field form `tsvry1|x-tenant|a` → + `a2ae26cf529a5843a25f1448acc4e90016d4c1dce0ffda5662e3ac459433e1ab`. And + a response derived from a request carrying an **identity-bearing TS + overlay** (the DataDome ClientID overlay) is forced `private, no-store` + unless an explicit per-overlay contract says otherwise, because + the `Authorization` rule protects origin credentials and this rule + protects the identity TS itself injected. Parsing itself is a **shared core + parser with fail-closed normalization**, not four adapter interpretations: + a `Cache-Control` value that fails the shared grammar has an + **enumerated result, not a "most restrictive reading"** (restrictions + are independent axes, so no single most-restrictive point exists): + the response is treated as **uncacheable for the storage decision** + (`no-store`-equivalent in the invariant) and any mutation batch + merging against the malformed value is rejected whole; among + well-formed values, duplicate directives keep the strongest; quoted and unquoted + forms are equivalent; conflicting `max-age` values keep the smallest; + unknown extension directives are dropped **from mutations only, while + unknown directives already in the snapshot are preserved verbatim** (a + downstream cache may honor a restrictive extension TS does not + recognize; dropping it would weaken origin policy, RFC 9111 §5.2.3); + `Expires` participates in the freshness bound via **RFC 9111 §4.2.1's + freshness-lifetime algorithm, referenced directly**: the + `Expires`-derived lifetime is `Expires − Date` (absent `Date` → + response receipt time), invalid or duplicate date values are treated + as already expired (the RFC's conservative option, chosen + normatively), and `Age` is handled per the RFC, effective freshness + is the minimum across `max-age`, `s-maxage`, and that derived + lifetime and a mutation may **not introduce `max-age`/`s-maxage` + where the snapshot supplied no upper bound**, HTTP prefers `max-age` + over `Expires` (RFC 9111 §5.3), so introducing one would override an + origin's shorter or already-expired `Expires`; and `Vary: *` is + treated as uncacheable-by-shared-caches (no-store-equivalent for the + invariant). Conformance fixtures cover each rule. Middle-stage placement also keeps + the earlier property: an integration mutation is not silently stripped + by ordinary core handling, only by the invariant pass, which logs the + downgrade it applies. + +## 3. Collision policy + +- Mutators may not touch **reserved surface**, which is defined at two + granularities because `Set-Cookie` is multi-valued: (a) reserved header + _names_, HTTP framing and hop-by-hop headers (`Content-Length`, + `Transfer-Encoding`, `Connection`, `Trailer`, `Upgrade`, `TE`, + `Keep-Alive`), **freshness metadata** (`Age`, `Date`, `Expires`, since replacing `Age: 59` + with `Age: 0` on a cached `max-age=60` response, or pushing `Expires` + into the future, extends downstream freshness in exactly the way the + monotonic merge forbids for `max-age`, so these are reserved outright + rather than merged), **representation headers coupled to body bytes + the hook cannot see** (`Content-Encoding`, `Content-Range`, + `Content-Type`, `ETag`, `Last-Modified`, `Accept-Ranges`, and digest + headers, since + relabeling uncompressed bytes as Brotli, or advertising a validator or + digest for bytes the hook never saw, corrupts responses or poisons + caches), the `x-ts-*` namespace, and the + consent/privacy headers core emits; (b) reserved cookie _names_ within `Set-Cookie`, `ts-ec`, + `ts-eids`, and the other `ts-*` cookies core owns. **Cookie operations are deferred out of the v1 hook, headers only.** + The write-side gate alone ("persistent cookies require P1") was shown + insufficient: it never modeled reading, using, forwarding, or + withdrawing the cookie, so a P1-granted-then-withdrawn integration + cookie would keep arriving on every request with nothing required to + expire, hide, or stop egressing it, and an advertising-identifier + cookie needs P4 the contract never expressed. Rather than ship + "inside the permission model" as a claim the model does not back, + `append_set_cookie` and the typed cookie builder are **deferred** to a + follow-up spec whose entry bar is: declared per-cookie required + permissions, a typed authorized request-side view, stripping from + unauthorized integration/proxy inputs, mandatory expiry on destructive + P1 withdrawal, and startup-unique (name, domain, path) ownership. + Integration IDs are **startup-unique, enforced**: registry + construction rejects a duplicate ID (current code silently coalesces, + which corrupts attribution and budgets), with a duplicate-ID test in + the done-when. The registration's `behavior_revision` follows §2's bump + contract; configuration-dependent behavior is captured separately by the + effective-config digest. Model-only activation is separate from both, so the + **one cache revision tuple** contains exactly + `integration_registry_revision`, `effective_config_revision`, + `active_policy_digest`, `active_policy_ordinal`, `model_epoch`, + `activation_generation`, and `hook_invariant_revision` from the strong active + tuple at publication. Every processed artifact, mutation IR/read-set bundle, + variant descriptor, and variant-index update stores that complete tuple; a + lookup, local conditional, HEAD update, or 304 replay requires byte-for-byte + equality with current active. In particular, the `permissions_v2` model CAS + misses every `pre_epic_v1` artifact even though config/policy bytes did not + change. Core also carries a nonzero + `HOOK_INVARIANT_REVISION: u32`, bumped for every parser, merge, budget, + cache-artifact, or invariant semantic change. The cache tuple stores all + fields above, so a deploy cannot silently reuse old finals while a new + restriction waits for cache expiry. Until then the + operation set is headers-only, and `Set-Cookie` is fully reserved. + Violations are rejected at the operation layer (§2) and + logged at `warn` with the integration id. The reserved lists are single + constants next to the definitions they protect, not duplicated in the + hook. +- For non-reserved headers, the mutator API distinguishes **append** from + **replace** explicitly; append/replace legality comes from a **core-owned field registry**, + not adapter judgment, and the v1 registry admits **inert fields + only**: "headers-only" is not automatically permission-neutral, since + `Link` (preload/prefetch), `Reporting-Endpoints`/NEL, CSP report + directives, and `Refresh` cause browser-initiated vendor contact on + requests that granted nothing. Fields with active egress side effects + are **rejected in v1**; a follow-up may admit them behind declared + required permissions gated at mutation time. Within the inert set, + each field is classified append-legal (genuinely list-valued), + replace-only (true singletons, e.g. `Content-Location`, `Retry-After`; + an earlier draft miscited `Content-Language`, which is list-valued), + or rejected; **unknown extension fields are rejected entirely in v1** + (neither append nor replace, their side-effect class is unknowable). + The v1 registry is enumerated here, not delegated: **admitted**, + `Cache-Control` (monotonic merge per this section), `Vary` (union + merge), `Content-Language` (append), `X-Robots-Tag` (append), + `Retry-After` (replace-only), `Content-Location` (replace-only); + everything else known is classified reserved or rejected by the rules + above, and growing the admitted set is a spec change to this list (cookies: see the §3 deferral above). Replacing a + header the origin set is a deliberate act, visible in the mutator's code. +- Later registrations see earlier mutations (order = registration order, + which is deterministic). +- **Operation-layer hygiene:** generic `append`/`replace` reject the + `Set-Cookie` header name outright, cookies go only through + the deferred cookie builder (when it exists), so cookie validation + cannot be bypassed by spelling + the header name in a generic op. Per-integration limits bound total + operations (≤ 32), added headers (≤ 16), and added bytes (≤ 8 KiB), and + a **cumulative final-response budget** (≤ 128 headers / ≤ 32 KiB total, + counting `name: value` plus separators, within any lower adapter + ceiling, and those ceilings are the enumerated capability cells **below**, + with the counting rule fixed exactly: counted bytes = Σ over emitted + fields of `len(name) + 2 + len(value) + 2` (the `": "` and CRLF + separators), validated against core's budget at startup so a batch + that passes core can never fail only on one adapter; a snapshot + already **over** the core budget before any mutation rejects every + ordinary batch, the budget bounds additions and never bricks an + over-budget origin response, which passes through and is counted; + security follows the replacement/reserve rule below) + bounds the sum across integrations. The budget has normative priority + partitions: ordinary mutators may consume at most **112 headers / 24 KiB**, + reserving 16 headers / 8 KiB for the core-owned security channel. Ordinary + batches remain registration-ordered within their partition. A security + `Continue` batch uses the reserve and, if necessary, evicts whole accepted + ordinary batches in reverse registration order until it fits; it never + removes half a batch and never drops origin fields. Ordinary output can + therefore never crowd out security effects. If the immutable origin head + plus the security batch alone exceeds the full budget, the security batch + is rejected atomically and the request follows the documented security + fail-open path with a dedicated metric, origin fields are not silently + sacrificed. A security `Respond` owns a replacement + response: all ordinary mutation batches are discarded and the challenge + is validated against the full 128-header / 32-KiB budget. A base publisher + response already over the full budget still passes unchanged, but no + ordinary mutation applies; security `Continue` applies only if the final + response fits after all ordinary batches are removed. These are separate + outcomes and metrics. + + | Adapter | Header-count / total-bytes ceiling (capability cell) | + | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | + | Axum | no platform ceiling below the core budget (native HTTP stack), cell fixed at ≥ 128 headers / ≥ 32 KiB | + | Fastly | **qualification-pending**: the measured platform ceiling is recorded in this cell by the adapter-qualification commit; unrecorded ⇒ hook startup fails | + | Cloudflare | **qualification-pending**: same rule | + | Spin | **qualification-pending**: same rule | + + A recorded cell below core's 128-header / 32 KiB budget is a startup + error (shrink the core budget or raise the ceiling, never a silent + per-adapter divergence). + + Hook/cache eligibility has the following concrete adapter cells; any + `qualification-pending` cell fails startup when the depending feature is + selected: + + | Capability | Fastly | Axum (dev) | Cloudflare | Spin | + | -------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- | --------------------------------------------------- | ------------------------------ | -------------------------------- | + | Runtime secret lookup for Vary-HMAC/DataDome | wired secret store; qualify key-rotation behavior | dev secret binding required | qualify Workers secret binding | qualify component secret binding | + | Persisted processed artifact + mutation IR/read sets | qualification-pending | in-process dev implementation required; non-durable | qualification-pending | qualification-pending | + | Atomic artifact/metadata entry commit | qualification-pending | implementation required | qualification-pending | qualification-pending | + | `Vary` variant index + insert-new-then-index-update rekey | qualification-pending | implementation required | qualification-pending | qualification-pending | + | DataDome field-line order, trusted IP/port, fixed HTTPS backend/no-redirect, and exact form limits | qualification-pending | qualification-pending | qualification-pending | qualification-pending | + | SecurityUse JA4 request evidence | platform value available; exact-field/payload qualification and sign-offs 23/28 pending | unavailable | unavailable | unavailable | + + The qualification commit records storage lifetime, maximum object size, + concurrency semantics, torn-write behavior, and fault-injection evidence; + “platform has KV” is not a qualifying cell. + `expose_host_fingerprints_to_vendor = true` also requires a qualified + SecurityUse JA4 cell; unsupported or pending is a startup error, while the + default `false` remains portable. + + Each mutator receives an **immutable, redacted snapshot of the + response head** (status and headers as of its turn, prior integrations' + accepted operations applied) as its read context; it never holds a + mutable reference (§2). Redaction is a security boundary, not + tidiness: the hook runs after core queues the EC `Set-Cookie`, so an + unredacted view would hand a mutator the raw EC to copy into + `X-Vendor-Identity` or its own cookie, walking around + `AuthorizedIdentity` entirely. The snapshot therefore + **excludes every `Set-Cookie` value and every reserved identity, + consent, and privacy header value** (names may be listed as present; + values are withheld). + Each registration also declares the complete set of response fields its + decision may read, including status as a distinguished input. Core records + the union with the accepted operation batch. Undeclared reads are a hard + conformance failure in tests; an integration unable to declare a complete + read set marks itself `revalidation = "refetch"`, which forbids IR replay + after any origin metadata change. + Operations arrive as **attributed batches bound to a registration + ID**, one batch per integration per response, ordered by + registration, with the security channel's batch (§4a) ordered **after** + ordinary response mutators, one global order, core finalization → + ordinary mutators → security effects → invariant pass, so the + security layer's precedence over publisher-facing mutations holds through + both position and its reserved/response-owning budget rule; the current flat effects vector satisfies neither + attribution nor budgets and will be restructured accordingly. Validation + and budgeting are **atomic per batch**: a batch that exceeds its + budget is rejected whole (logged, attributed), never partially + applied, since item-by-item rejection could apply a security 302's + `Set-Cookie` while dropping its `Location`. The response itself is + never rejected. A mutator that returns an error is skipped in full, its + operations are all-or-nothing, and the response proceeds without it. + **Panics are forbidden and fatal, not recoverable**: the primary target + (`wasm32-wasip1`) has no unwinding support, so there is no unwind + boundary to catch at, a spec that promised panic recovery would be + unimplementable there. Mutators are infallible-by-construction or + return `Result`; a panic is a bug that takes the instance down, same as + anywhere else in the request path. + +## 3a. Response eligibility, normative + +Which responses the hook runs on, enumerated so two implementations cannot +diverge silently: + +In this table, **persisted post-hook finals** means the cache-safe ordinary +artifact only: origin metadata plus accepted ordinary mutation IR and the +cache/privacy invariant result, with `Set-Cookie`, core request-specific +identity fields, security-channel effects, and origin validators excluded. +The security request filter evaluates every request before cache selection; a +fresh `Respond` bypasses the artifact, while a fresh `Continue` batch is +applied to the persisted ordinary artifact and the invariant pass reruns before +emission. This applies equally to ordinary hits, local conditionals, +origin-revalidation 304s, and HEAD. Therefore "`Set-Cookie` is never replayed" +means never replayed from storage; a freshly validated per-request typed cookie +operation may still emit on that response. Security `Respond` outputs are +always `private, no-store` and never become artifacts. + +An **unconditional recovery fetch** first saves the client's preconditions for +later local evaluation, then removes every upstream conditional/range field +(`If-Match`, `If-None-Match`, `If-Modified-Since`, `If-Unmodified-Since`, +`If-Range`, and `Range`) so origin must return full bytes. TS processes that +200 under current revisions, constructs the processed validator, and only then +evaluates the saved client preconditions as the authoritative server for the +transformed representation. Another bodyless origin 304 can never satisfy +recovery. + +The 304 **safe-update set** is exactly `Cache-Control`, `Expires`, `Date`, +`Age`, `Vary`, `Surrogate-Control`, `CDN-Cache-Control`, +`Cloudflare-CDN-Cache-Control`, and `Edge-Control`. The phrase +"registry-admitted mutable fields" in the matrix denotes an empty set in v1; +adding any name requires a reviewed spec/registry revision and conformance +fixture, never a runtime wildcard. + +| Response | Hook runs? | +| ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Processed HTML document (rewritten by TS) | Yes | +| Streamed processed document | Yes, operations apply to the header block before first byte | +| Pass-through proxy response (not processed) | No, TS is a transparent proxy for it | +| Served-from-cache processed document | Serves the **persisted post-hook finals** captured at cache fill (revision-versioned; mismatch = miss), mutators run at fill/processing time only, so a normal hit and a conditional hit return identical policy metadata **by construction**, with no purity or determinism assumption about mutators | +| Redirect (3xx) | No | +| Error responses TS itself generates (4xx/5xx) | No | +| `304 Not Modified` for a processed representation | **Persisted-metadata pass**: a cached processed 200 stores its final post-hook headers, accepted mutation-operation batches, and the union of every mutator's declared response-field read set (the persisted mutation IR), versioned by §3's complete cache revision tuple, including model epoch and logical activation generation. A local conditional hit re-emits persisted finals only when the complete tuple matches current active. An origin-revalidation 304 is staged and diffed against separately stored origin-side metadata. (a) Any byte-coupled field changed (`Content-Encoding`, `Content-Type`, validators, digests) → invalidate and fetch/process a full 200. (b) A changed metadata field that intersects any persisted mutator read set, or an artifact/mutator lacking a complete read-set declaration, is also unsafe → full 200 refetch and ordinary hook execution; deterministic replay of old operations cannot stand in for re-evaluating a decision made from changed inputs. (c) If every changed field is outside every declared read set and belongs to the enumerated safe-update set (`Cache-Control`, reserved CDN cache fields, `Expires`, `Date`, `Age`, `Vary`, registry-admitted mutable fields), replay the persisted deterministic operations over updated origin metadata and rerun invariants. Updated origin metadata, finals, IR, read sets, and complete revision tuple publish in one atomic entry commit; changed `Vary` uses insert-new-entry-then-index-update ordering. (d) No change → re-emit persisted finals. Artifact absence or any tuple mismatch triggers an unconditional recovery fetch so TS obtains bytes. For processed-document GET/HEAD routes, TS is explicitly the authoritative server for the transformed representation: after processing the full 200 it evaluates RFC 9110 §13 preconditions against **processed** validators; this is not evaluation of origin validators by an intermediary cache. Other methods are never eligible for this recovery path. `Set-Cookie` and origin validators are never replayed. Absent metadata → cache miss | +| `HEAD` of a processed document | **Serves the persisted GET artifact when one exists**, parity with GET/304 by construction, since RFC 9111 §4.3.5 lets HEAD metadata update a stored GET response and mutators are not required to be deterministic; with no persisted artifact, HEAD is processed like a GET (headers only) and its finals stored as a **distinct head-only artifact type that never satisfies a later GET**, when a stored GET artifact exists a HEAD may **update** it only when the comparison, made against the separately stored **origin-side** metadata, never the processed artifact's (origin and rewritten lengths differ by construction, so the processed length is never the comparand and HEAD metadata never touches processed-side headers), finds validators and origin `Content-Length` matching and no byte-coupled representation field changed (RFC 9111 §4.3.5); qualifying updates land origin-side under the same atomic-commit discipline as the 304 rules, and any mismatch **invalidates** the stored GET artifact rather than updating it | +| Informational `1xx`, `204`, `205`, `206` | No, enumerated so adapters do not infer independently | + +This deliberately narrows #782's general "outbound response" phrasing to +processed documents (§6). + +## 4a. The security channel, normative closed boundary + +The security channel (today: DataDome) runs under a distinct, typed +`SecurityUse` authority rather than the advertising permissions P1/P4. +`SecurityUse` permits bot/fraud evaluation and challenge continuity only; +it never authorizes TS-controlled advertising identity, graph linkage, partner egress, +other integrations, or general raw-value observability. Request-scoped raw +security evidence may be disclosed only to the fixed DataDome Protection API +endpoint and only from that integration's explicit field allowlist; it is not +persisted in the identity graph, exposed to publisher origin or other +integrations, or emitted in logs. It carries its own configured retention and +deletion path. An advertising opt-out does not erase a +strictly security-scoped identifier, while an authenticated deletion request +or expiry under the security retention policy does. This is not a general +exception. The path-only `Request` remains publisher-originated data and is +explicitly covered by vendor retention/DSR sign-off; query strings and full +referrers are never in the security view. Every degree of freedom is closed: + +- **Host evidence is not a back door to the device provider.** + `[device] provider = "fastly"` is an explicit opt-in selection (#1044) and + the hook does not widen it: no JA4-derived classification is stored by a + mutator. If DataDome's Protection API is allowed to receive + request-scoped `TlsProtocol`/`JA4` evidence, its registration enumerates each field, + proves vendor necessity and payload bounds, and keeps it ephemeral under + `SecurityUse`; those exact consumers and fields are part of product/vendor + sign-offs 23/28. `TlsCipher` is omitted because the platform value has + different semantics from the vendor field, `H2Fingerprint` is not in the + vendor contract, and all other host evidence is omitted. +- **Deletion and retention name the system boundary honestly.** TS stores no + server-side DataDome identifier mapping. On an authenticated TS deletion + request it excludes the route from vendor validation, emits a typed + `datadome` cookie deletion, and sends no ClientID to DataDome for that + request; re-presentation retries deletion. A lost browser response can leave + the cookie until its configured `security_cookie_max_age`, which is the + bounded residual sign-off 23 accepts. This operation does **not** claim to + erase data already held by DataDome: vendor-side retention and data-subject + deletion require a named contractual/API procedure in the decision record. + If no such vendor procedure exists, operator documentation says so and may + not describe TS cookie deletion as vendor-data deletion. + +- **Typed security-cookie operation with a concrete lifecycle, not + header strings.** The channel emits cookies only through a typed + operation, and the registration is not a placeholder, for DataDome + it pins, **aligned to documented vendor behavior where hardening was + not intended**: cookie name exactly `datadome`; one configured ownership + tuple for every set and deletion: path exactly `/` and + `security_cookie_domain = "host-only"` (default) or one explicit normalized + ASCII domain. Host-only mode requires the vendor `Domain` attribute to be + absent; explicit-domain mode requires it to equal the configured domain + exactly, TS never accepts a different domain and never rewrites one scope + into another. The explicit value cannot exceed the registrable domain, + computed against the **vendored + Mozilla PSL snapshot in §4a.1**, + ICANN + private sections, IDNA-mapped; IP-literal or single-label + hosts fall back to host-only), path `/`; `Secure` mandatory; + `SameSite` configurable `Lax` (default) / `Strict` / `None` + (`None` requires `Secure`), matching the vendor's endpoint options; + the configured/returned `Domain` must additionally **domain-match the current + request host** per RFC 10025, the current cookie specification obsoleting RFC 6265 (domain-match and the PSL boundary check + are separate requirements) and a vendor cookie using `Expires` is + normalized to its Max-Age equivalent (both present → `Max-Age` wins, + per RFC 10025); a normalized lifetime exceeding the ceiling rejects + the whole operation batch; and the parser is total: repeated `Cookie` + request fields are joined with `"; "` (semicolon-space, the + order-preserving join of RFC 10025) before parsing, and + **duplicate `datadome` pairs after the join make the request-side + identity ambiguous: treated as cookie-absent for the vendor call and + counted**, while cookies under other names pass through untouched; + `Set-Cookie` fields are **never combined**, and a vendor response + carrying more than one `datadome` `Set-Cookie` field, a `Set-Cookie` + for any other name (the typed operation pins the name exactly), an + **`HttpOnly` attribute** (the vendor requires script readability, so + its presence is a protocol anomaly), duplicate attributes, duplicate + cookies in one field, an unparseable `Expires`, or unknown attributes + each reject the operation batch (the vendor cookie is well-formed; + strictness is safe). `Expires` normalizes as + `Max-Age = max(0, floor(expires − now))` whole seconds on the server + wall clock at parse time (the shared skew-bounded basis; a result of + 0 is a deletion), and the 512-byte limit measures the **normalized** + serialized `name=value` plus attributes in bytes; + `Max-Age` is capped by required operator configuration + `security_cookie_max_age` in the vendor-supported range 7 days through + **31,536,000 seconds** (one year); the returned cookie may be shorter but + never longer. There is no silent one-year default. Size ≤ **512 bytes** + (DataDome's current Fastly-module limit; 4 KiB was ours, not theirs). + Where the contract **is** deliberately narrower than the vendor, the + spec-pinned pointer allowlist starting at ClientID-only against + DataDome's mandatory response-directed mapping set, that reduction + needs explicit product **and vendor** acceptance: sign-off item 28; + a violating operation is rejected whole (the batch rule). **Both + sessionByHeader is startup-rejected in v1, one state, not three**: + TS never sends `X-DataDome-X-Set-Cookie: true`; a vendor + `X-Set-Cookie` **invalidates the batch → Continue** (its matrix cell, since + a session mode the fleet never requested must not half-apply); and an incoming + browser `X-DataDome-ClientID` is **not forwarded to the vendor** + (cookie-only session identity, an earlier revision translated + `X-Set-Cookie` into an ordinary cookie, which is not equivalent: + header-session clients expect JavaScript to receive `X-Set-Cookie` + and `X-DD-B`, and the higher-priority header session would never see + a cookie update; the older DataDome spec's "always send + X-DataDome-X-Set-Cookie when the header ID is used" is superseded for + v1 by this section). Supporting header mode later means the full vendor + protocol, typed owner-scoped `X-Set-Cookie`/`X-DD-B` forwarding, + CORS exposure, and a JavaScript/local-storage identifier observer, + as an explicit opt-in under **sign-off 23**. Every `ts-*` + name is rejected. **Read is owner-only across every _server-side_ surface, and the + browser side is explicitly not ownable**: DataDome requires the + cookie to be readable by its JavaScript and warns against `HttpOnly`, + so **every same-origin page script can observe it**, and a Respond + serves vendor-owned HTML under the publisher origin (vendor scripts + with same-origin access to cookies, storage, and APIs; publisher CSP + can conversely break the challenge). Those browser-side observers, + same-origin vendor code, CSP interaction, and challenge redirects + enter **sign-offs 23/28** for ratification (both decision records + still open), not implied. The server-side strip + inventory is exhaustive, not integration-scoped, because the browser sends `datadome` in the ordinary + `Cookie` header, so it is removed from **every non-DataDome surface**: + other integrations' request views, publisher-origin proxy forwarding, + proxy/click/Testlight upstreams, auction/page-bids request + serialization, and logs (redaction list), each surface a tested row + of the inventory; only the security channel itself observes it; vendor + egress goes only to the fixed DataDome Protection API authority and path; + redirects are not followed; deletion is always possible + through the `SecurityUse` lifecycle; and advertising withdrawal never + grants access to or reuses the identifier. No other request filter + inherits the cookie capability. +- **Cookie ownership makes deletion total for the scope TS creates.** While + DataDome is enabled, `datadome` is a security-owned name across the final + response: before the security batch applies, core removes and meters every + origin, core, cached, or ordinary-mutator `Set-Cookie` for that name; + unrelated cookie names remain separate field lines. Only the typed security + operation may emit it. Authenticated deletion emits the same configured + `(name, domain mode/domain, path)` tuple with `Max-Age=0`; it does not guess a + scope from the request cookie, whose wire form carries no Domain or Path. + Candidate validation rejects a change of domain mode/domain while the + previous active DataDome configuration can still have a live cookie. The + supported migration is disable + wait at least the previous + `security_cookie_max_age` + activate the new scope; a faster scope change + requires a separate bounded deletion-fan-out design. The permission spec + §5.5 whole-settings serve fence applies before cookie processing. A bounded + old-generation admission validation may survive only during the pre-promotion + drain; the register's promotion-not-before plus member quiescence proves it + and every admitted effect ended before the activation CAS. After that CAS, + no instance may emit, refresh, or delete a `datadome` cookie until it has + loaded and leased the exact new active tuple. A stale instance stops at serve + admission rather than extending the old scope. Fixtures cover origin + collision, host-only vs explicit-domain set/delete, attempted domain change, + and duplicate request cookies. The deletion claim therefore covers every + cookie this contract can create, not arbitrary pre-contract scopes. +- **The incoming `X-DataDome-ClientID` request header is owner-only, + like the cookie.** DataDome prioritizes the header over the cookie, + so leaving it in the shared request would hand other integrations and + upstream routing the same identifier the cookie boundary strips: core + **removes it from the shared request** before integrations and + upstream routing run, it joins `RedactedRequestView`'s enumerated + strip set (providers spec), **and in v1 it is stripped for the + vendor too: the Protection API request's ClientID derives only from + the `datadome` cookie, never from the incoming header** (DataDome's + contract requires `X-DataDome-X-Set-Cookie: true` whenever a + header-supplied ClientID is forwarded, so forwarding the header under + cookie-only mode would misdeclare the session mode); observed header + occurrences are counted, and a **fixture pins the path**: a request + carrying both cookie and header produces a vendor payload whose + ClientID equals the cookie value, with no header-derived identity + sent. Only DataDome-returned overlay data reaches the publisher, + never the raw browser-supplied header. +- **The pointer protocol has a total parser contract**, adapters + cannot differ where malformed batches fail open: the pointer list is + tokenized by the vendor's documented space separation, repeated + pointer header fields are concatenated with a single SP before + tokenizing, tokens split on runs of SP/HTAB, empty tokens ignored, + then names are ASCII-lowercased before duplicate detection; duplicate + names after normalization, invalid names, more than 16 pointers, or + more than 4 KiB of pointer payload render the batch invalid + (→ Continue, the vendor's fail-open). **Pointed-field multiplicity is + closed**: for singleton fields (`Location`, `Content-Type`, + `X-DataDome`, `X-DD-B`, `X-Set-Cookie`) more than one instance in the + vendor response invalidates the batch atomically, never a + first/last/join choice an adapter makes; list-valued fields + (`Cache-Control`, `Pragma`) are joined per RFC 9110 §5.3 before their + matrix outcome applies; `Set-Cookie` multiplicity follows the typed + cookie rule (exactly one `datadome` field, above). No both-source + priority rule exists in v1: the header session form (`X-Set-Cookie`) + is matrix-governed as batch-invalid, so "header form wins" is + unreachable and deleted. +- **Request-header pointers are a positive, enumerated allowlist with no + default publisher-origin identifier exposure.** + "Documented enrichment headers" is not enforceable; the registration + enumerates the exact names from the **inline DataDome field contract in + §4a.2**, spec-pinned + today to exactly **`X-DataDome-ClientID`**, admitted only when the + operator explicitly sets + `[integrations.datadome] expose_client_id_to_origin = true` (default + `false`); every other `X-DataDome-*` + field is rejected until a reviewed commit adds it to §4a.2 + ("documented enrichment set, listed one by one" without an actual list + was a wildcard whose contents could change outside the spec), + resolving what was a contradiction. When the opt-in is false, the + vendor-returned ClientID is discarded and the publisher origin is not + an identifier observer. When true, it applies only to an owner-scoped + publisher-upstream overlay, never the shared request; startup logs the + additional consumer, operator documentation must disclose its purpose + and retention, and a fixture proves no other surface can read it. + Everything else, authentication, + `Cookie`, `Forwarded`/`X-Forwarded-*`, other identity, consent, and + routing-authority fields, is rejected by name and by class: a + compromised endpoint must not replace origin credentials, inject + `ts-ec`, or spoof client location. +- **Browser-response headers are decision-scoped through the single + matrix, and only there.** This spec carries **no per-decision field + list**: every pointer's outcome per decision and session mode, + `Location`'s Respond-only 3xx admission, `Pragma`'s + drop-individually middle path (response `Pragma: no-cache` has no + standardized meaning, RFC 9111 §5.4), and the batch-invalidation + default for unlisted names, is exactly one cell of the matrix in + §4a.2.3. The fail-open consequence of batch + invalidation stays within sign-off 28's scope, and both documented + vendor responses are verbatim fixtures at the matrix. +- **Representation rules are decision-scoped and narrow.** A _Respond_ + decision (challenge/deny) owns its body but may describe it with + **`Content-Type` only**, encoding and validator fields + (`Content-Encoding`, `ETag`, `Last-Modified`, digests) stay reserved + even for Respond: challenge bodies are simple and uncacheable, the + allowlist does not admit those fields, and ambiguity here decides + whether a challenge enforces or silently fails open (batch rejection → + Continue). If the vendor ever requires more, it arrives as a reviewed + §4a.2 field-contract addition. A _Continue_ decision may not touch + representation metadata of publisher bytes. +- **Respond transport is bounded, with exact measurement points.** The + challenge body has a maximum size (64 KiB) and a **complete-response + deadline of 3000 ms on the instance's monotonic clock, measured from + immediately before vendor-backend acquisition/dispatch to the final + body byte**, meaning connection setup and request send are inside the + window; the 1500 ms first-byte bound (the older spec's figure, now + first-byte only) runs from the same origin on the same clock. At + expiry the decision is final (batch fails → Continue) and the vendor + request is canceled; cancellation and resource cleanup complete + asynchronously and never delay the response. TS sends + `Accept-Encoding: identity`, and because that does not _guarantee_ + identity coding, a response arriving with any `Content-Encoding` is + itself batch-invalid; `Content-Length` is recomputed from the actual + bytes before Respond commits. **On a HEAD request, Respond validates + the challenge body exactly as for GET (size, deadline, encoding) but + emits no body**: the outward response **omits `Content-Length`** and + carries no content, RFC 9110 §9.3.2 permits `Content-Length` on + HEAD only when it equals the equivalent GET body's length, and the + vendor does not guarantee method-invariant challenge bodies, so the + validated HEAD bytes cannot establish the GET length (a + vendor-guaranteed equivalence, if ever ratified, may restore the + field as a reviewed change; the older DataDome spec's HEAD handling + remains superseded). Exceeding + size, first-byte, or total deadline fails the batch → Continue. +- **One pointer contract, one place.** The single normative + decision × session-mode × pointer matrix lives in **§4a.2.3**, this + spec's earlier inline + decision-scoped list and outcome list are deleted in its favor + (duplicated lists disagreed about `X-Set-Cookie`, `X-DataDome`, + `X-DD-*`, and `Pragma`, letting one conforming implementation accept + the vendor's documented `Set-Cookie X-DD-B` allow-example while + another invalidated the whole batch). No `X-DD-*` wildcard exists: + every name is enumerated, `X-DD-B` included and forwarded exactly once + as the vendor's documented cookie-mode browser-response signal; it is + never copied into publisher-upstream or another integration. The + documented vendor responses (both the challenge example and the + `Set-Cookie X-DD-B` allow example) are **verbatim fixtures asserting + the decision survives** and exactly the mapped fields emit. +- **Every security Respond ends uncacheable, unconditionally.** After + the decision's fields are applied, the invariant pass forces + `Cache-Control: private, no-store` and strips all CDN cache fields on + **every** Respond regardless of status, vendor pointers, or cookie + emission, a cookie-less `301` challenge with no effective vendor + cache header could otherwise be heuristically stored and served to + unrelated clients. +- **One global order:** core finalization → ordinary mutators → + security effects → **final cache/privacy invariant pass, + unconditionally last**. Security precedence over publisher-facing + mutations comes from its position, not a "wins" rule; nothing outranks + the invariant pass, or a challenge could combine `Set-Cookie` with + public caching. The older DataDome spec's "applies last, after + finalization" wording is **superseded by this order**. That baseline remains + unchanged; this PR-specific section prevents an implementation from placing + DataDome after the invariant pass and reopening the + public-cache-plus-cookie bug. +- The channel adopts the shared layers: structured attributed batches + (§3, atomic per batch, a 302 must never lose `Location` to a budget + while keeping its cookie), reserved header names, budgets, and the + invariant pass, with one sequencing rule fail-open depends on: the + complete challenge batch is **validated and budgeted before the + Respond decision commits**, so a rejection converts to Continue while + the publisher route is still available; discovering the rejection + after Respond has short-circuited routing would leave nothing to fail + open _to_. + +### 4a.1 Public Suffix List snapshot (normative) + +The vendored Mozilla PSL revision used for registrable-domain +computation in §4a: the implementation PR vendors the list file +and records its upstream commit hash here. Rules: ICANN **and** private +sections apply; hostnames are IDNA-mapped before matching; IP literals +and single-label hosts have no registrable domain (cookie falls back to +host-only). Updating the snapshot is a reviewed spec change. + +| Field | Value | +| ---------------------- | -------------------------------------------------------------------- | +| Upstream repository | `publicsuffix/list` | +| Upstream commit | `e1b8015c3b2f0f4f8c18659c2480fc1a22c07b20` | +| Upstream source path | `public_suffix_list.dat` | +| Required vendored path | `crates/trusted-server-core/data/public_suffix_list.dat` | +| Required hash path | `crates/trusted-server-core/data/public_suffix_list.sha256` | +| Required provenance | `crates/trusted-server-core/data/public_suffix_list.provenance.json` | + +The implementation copies the source bytes at that commit without editing +and writes the lowercase 64-hex SHA-256 plus one trailing LF (no filename or +other fields) to the required hash path. CI verifies the bytes, hash, and +commit reference together; updating any one without the others fails. The +provenance file is canonical JSON with exactly +`{upstream_repository, upstream_commit_oid, upstream_commit_tree_oid, +source_path, source_blob_oid, source_sha256_hex}`. The vendoring PR description +quotes the same commit/tree/blob values and the independent command output +that verified the raw upstream SHA-256 and byte-for-byte vendored copy. A +commit OID without its tree and source-blob witness does not satisfy the +release gate. + +### 4a.2 DataDome field contract (normative) + +Adding or changing any name here is a reviewed spec change. This subsection +holds the **only** normative Protection API request-field and response-pointer +lists; §4a carries no duplicate or per-decision lists of its own. + +#### 4a.2.1 Protection API request fields (browser request → DataDome only) + +`SecurityUse` admits only the fields below to the configured DataDome +Protection API endpoint. They are request-scoped and are never persisted in +the identity graph, copied to publisher upstream or another integration, or +logged as raw values. + +The endpoint is the fixed core-owned +`https://api-fastly.datadome.co/validate-request`; “configured endpoint” in +this subsection means that DataDome protection is enabled, not that an operator may +supply an authority. Redirect following is disabled. No TS-controlled +advertising identifier, consent-store key, graph value, request query, or full +referrer is admitted. The normalized publisher URL path remains disclosed and +may itself contain publisher-chosen data; sign-offs 23/28 must classify that +surface, its retention, and DSR handling rather than calling the entire URL +identity-free. + +Core-derived fields: + +- `Key`, `IP`, `Method`, `Protocol`, `Host`, `ServerHostname`, `Request` +- `RequestModuleName`, `ModuleVersion`, `TimeRequest`, `Port` +- `ServerName`, `ServerRegion` +- `ClientID` from the single unambiguous `datadome` cookie only. The form key + is always present because the Protection API declares it mandatory; its + value is the empty string when no unambiguous cookie exists +- `CookiesLen`, `AuthorizationLen`, and `PostParamLen` as lengths only +- `HeadersList`, containing only the source header names admitted by the + next list plus `authorization`, `content-length`, and `cookie` (whose values + remain length-only/ClientID-only above). Names are lowercased, comma-separated, + and retain received field-line order, including repeated admitted names; + arbitrary/custom header names are excluded. An adapter that cannot preserve + received header order does not qualify this integration until DataDome + approves a canonical replacement order in sign-off 28 + +Core derives those fields identically on every qualified adapter: + +- `Key` is the resolved DataDome server secret and is never obtained from + request/config text; `IP` and `Port` are the remote address and TCP source + port from trusted connection metadata. Missing `Key`, `IP`, or `Port` skips + the call through the metered fail-open path; no sentinel is synthesized +- `Method` is the validated HTTP method token; `Protocol` is exactly `http` or + `https` from the adapter request URI; `Host` is the normalized ASCII request + authority with a non-default port retained; `ServerHostname` is trusted TLS + SNI/local-host metadata, omitted when unavailable +- `Request` is only the URL path. Empty path becomes `/`; dot segments are + removed, percent escapes are preserved without percent-decoding and + normalized to uppercase hex, and the complete query and fragment are + discarded before the security view exists +- `RequestModuleName` is the literal `trusted-server`; `ModuleVersion` is the + build's checked-in Trusted Server version; `TimeRequest` is the request-ingress + Unix timestamp in decimal microseconds, captured once before integration + processing and constrained to `0..=2^53-1` +- `ServerName` is the adapter-qualified deployment/service name and + `ServerRegion` is its adapter-qualified region code; either is omitted when + the platform cannot supply it without request input +- `CookiesLen`, `AuthorizationLen`, and `PostParamLen` are decimal byte counts + of the received field/body surfaces before redaction. Overflow beyond an + unsigned 64-bit count skips the call; it never wraps or truncates + +Exact request-header value mappings: + +- `Accept` ← `accept`; `AcceptCharset` ← `accept-charset`; + `AcceptEncoding` ← `accept-encoding`; `AcceptLanguage` ← `accept-language` +- `CacheControl` ← `cache-control`; `Connection` ← `connection`; + `ContentType` ← `content-type`; `From` ← `from`; `Origin` ← a successfully + parsed `origin` serialized as scheme + ASCII host + non-default port only; + `Pragma` ← `pragma`; `Referer` ← a successfully parsed `referer` reduced to + scheme + ASCII host + non-default port only; `UserAgent` ← `user-agent`; + `Via` ← `via` +- `SecCHDeviceMemory` ← `sec-ch-device-memory`; `SecCHUA` ← `sec-ch-ua`; + `SecCHUAArch` ← `sec-ch-ua-arch`; `SecCHUAFullVersionList` ← + `sec-ch-ua-full-version-list`; `SecCHUAMobile` ← `sec-ch-ua-mobile`; + `SecCHUAModel` ← `sec-ch-ua-model`; `SecCHUAPlatform` ← + `sec-ch-ua-platform` +- `SecFetchDest` ← `sec-fetch-dest`; `SecFetchMode` ← `sec-fetch-mode`; + `SecFetchSite` ← `sec-fetch-site`; `SecFetchStorageAccess` ← + `sec-fetch-storage-access`; `SecFetchUser` ← `sec-fetch-user` +- `X-Requested-With` ← `x-requested-with` + +Request-field multiplicity is normalized **before** parsing, truncation, and +form encoding, and adapters expose every received field line rather than a +preselected first/last value. Every admitted value line must contain valid HTTP +field-value octets **and** valid UTF-8 after OWS removal; otherwise the vendor +call is skipped through the metered fail-open path, because adapter-specific +byte-to-string replacement is forbidden: + +- The list-valued source fields are exactly `accept`, `accept-charset`, + `accept-encoding`, `accept-language`, `cache-control`, `connection`, + `pragma`, `via`, `sec-ch-ua`, and `sec-ch-ua-full-version-list`. Core removes + leading and trailing optional whitespace from each field value, rejects a + value containing invalid field-value octets, and combines all field lines + (including empty values) in received order with the two literal bytes `, `. + This one normalized value is then parsed where the mapping above requires + parsing and is then bounded. Commas inside an individual value are not split + and reserialized. +- Every other admitted value-bearing source header in the exact mapping above + is singleton. Zero lines means omit the DataDome field. Exactly one valid + line is OWS-normalized and processed. Two or more lines, even identical, + are ambiguous and skip the vendor call through the metered fail-open path; + core never chooses first, last, or comma-joined. In particular this applies + to `origin`, `referer`, `user-agent`, `content-type`, `from`, every remaining + `sec-ch-*`/`sec-fetch-*` field, and `x-requested-with`. +- `authorization` and `content-length` are security singletons for this view. + Repetition skips the vendor call before either length or `HeadersList` is + constructed. `AuthorizationLen` is the byte length of the one + OWS-normalized value. `PostParamLen` is always the byte length of the body + actually presented to core, not the numeric `content-length` value; a + malformed or body-inconsistent `content-length` is rejected by the shared + HTTP request boundary before integrations run. +- Multiple `cookie` field lines are permitted. Core OWS-normalizes them and + joins them in received order with the literal bytes `; ` for the shared RFC + cookie parser. `CookiesLen` is the byte length of that canonical joined + value. `ClientID` is populated only when the parsed result contains exactly + one syntactically valid `datadome` pair; malformed cookie syntax or duplicate + `datadome` pairs produces the required empty `ClientID` value without + exposing another cookie. The original cookie values never enter the vendor + payload. +- After successful normalization, `HeadersList` records the lowercased name of + every admitted received field line in original line order, so repeated list + fields and cookie lines remain repeated. A rejected request produces no + `HeadersList` and no vendor call. Per-field caps apply to the single + normalized value; the 24,576-byte cap applies after complete form encoding. + +For an optional mapped value, zero received lines omits both source and mapped +field; one or more lines whose OWS-normalized values are all empty omits the +mapped form field but retains each received source name in `HeadersList`. If at +least one list-valued line is nonempty, empty siblings remain represented in +the exact received-order `, ` join. Mandatory `ClientID` and the three length +fields follow their explicit rules instead of this optional-field omission. + +Adapter qualification fixtures feed the same ordered repeated-field corpus to +every host and assert byte-identical form fields, lengths, `HeadersList`, and +reject/omit outcomes. The corpus includes repeated list fields, identical and +different singleton duplicates, multiple cookies, duplicate `datadome` +cookies, empty values, invalid octets, and headers whose individual values +contain commas; invalid UTF-8 is a skip, never replacement decoding. + +`true-client-ip`, `x-forwarded-for`, and `x-real-ip` are not admitted in v1. +The trusted `IP` field already supplies connection provenance; copying raw +forwarding headers would let a client or unqualified proxy manufacture vendor +evidence. A future adapter-normalized forwarding chain requires a separately +named typed field and vendor sign-off, never reuse of the raw header mapping. + +Platform host evidence: + +- `TlsProtocol`, capped by TS at 32 bytes +- `JA4`, capped by TS at 128 bytes, only when the operator explicitly sets + `[integrations.datadome] expose_host_fingerprints_to_vendor = true`; + the default is `false`, omission is represented by absence rather than an + empty field, and startup logs the additional vendor disclosure +- `TlsCipher` is omitted in v1: DataDome defines it as the ordered list of + cipher suites offered by the client, while `RuntimeServices::client_info()` + exposes only the negotiated cipher. Substituting that value would silently + change the field's meaning +- `H2Fingerprint` is omitted in v1 because the current Protection API contract + does not define such a request field + +`X-DataDome-ClientID` is never a Protection API source in cookie-mode v1. +No wildcard (`Sec-CH-*`, `Sec-Fetch-*`, `X-*`, or otherwise) expands this +list. + +The following limits are bytes of the decoded field value before form +encoding. Truncation is UTF-8-boundary-safe. `XForwardedForIP` alone truncates +from the end; every other bounded field retains its prefix: + +- 8 bytes: `SecCHDeviceMemory`, `SecCHUAMobile`, + `SecFetchStorageAccess`, `SecFetchUser` +- 16 bytes: `SecCHUAArch` +- 32 bytes: `SecCHUAPlatform`, `SecFetchDest`, `SecFetchMode`, and the TS cap + on `TlsProtocol` +- 64 bytes: `ContentType`, `SecFetchSite`, and the TS cap on `ServerRegion` +- 128 bytes: `AcceptCharset`, `AcceptEncoding`, `CacheControl`, `Connection`, + `From`, `Pragma`, `SecCHUA`, `SecCHUAModel`, `X-Requested-With`, and the TS + cap on opt-in `JA4` +- 256 bytes: `AcceptLanguage`, `SecCHUAFullVersionList`, `Via` +- 512 bytes: `Accept`, `ClientID`, `HeadersList`, `Host`, `Origin`, + origin-only `Referer`, `ServerHostname`, and `ServerName` +- 768 bytes: `UserAgent` +- 2,048 bytes: path-only `Request` + +`Key`, `AuthorizationLen`, `CookiesLen`, `IP`, `Method`, `ModuleVersion`, +`Port`, `PostParamLen`, `Protocol`, `RequestModuleName`, and `TimeRequest` are +unbounded per-field by the vendor table but remain subject to the total bound. +The complete `application/x-www-form-urlencoded` body, including field names, +`=`/`&` separators, and percent-encoding expansion, must be at most **24,576 +bytes**. Core constructs and measures the whole payload before issuing the +request. It does not silently drop optional fields to fit: overflow skips the +vendor call and takes the same metered fail-open `Continue` path as a transport +failure. + +This is deliberately narrower than DataDome's currently documented required +surface: notably, it withholds `CookiesList` and omits empty source-header +fields. Product/vendor sign-off 28 therefore requires written confirmation +that this exact reduced profile is supported. Until that confirmation and +adapter conformance fixtures exist, the DataDome integration is not +release-qualified. + +#### 4a.2.2 Request-direction pointer (vendor response → publisher-upstream overlay) + +The complete set of vendor-response header pointers the security +channel (hook spec §4a) may copy into the owner-scoped +publisher-upstream overlay. Every `X-DataDome-*` name not listed here +is rejected. + +| Header | Direction | Scope | +| --------------------- | --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `X-DataDome-ClientID` | response → upstream overlay | Disabled by default; admitted only with `expose_client_id_to_origin = true`. Owner-scoped publisher overlay only, never the shared request view or another integration | + +#### 4a.2.3 The single pointer matrix (normative, decision × session mode × pointer) + +This is the one authoritative browser-response contract. Session mode +is **cookie** in v1 (sessionByHeader is startup-rejected; a header-mode +column is added by the sign-off-23 opt-in, never implicitly). No +wildcard rows exist, every accepted name is enumerated, and **every +cell terminates in exactly one outcome**. + +| Pointer | Respond (cookie mode) | Continue (cookie mode) | +| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------- | +| `Set-Cookie` | typed `datadome` cookie operation (hook spec §4a parser; exactly one `datadome` field) | typed `datadome` cookie operation | +| `X-Set-Cookie` | **invalidate the batch → Continue** (mode mismatch: TS never requests header sessions in v1; a vendor response asserting one must not half-apply) | **invalidate the batch** → effects dropped | +| `Location` | forward on a 3xx Respond, replace; **on a non-3xx Respond → invalidate the batch → Continue** (a redirect field without a redirect status is a malformed decision) | **invalidate the batch** (effects dropped) | +| `Content-Type` | forward (owns its body), replace | **invalidate the batch** (effects dropped) | +| `Cache-Control` | restricted merge; invariant pass last | **invalidate the batch** (effects dropped) | +| `Pragma` | drop-individually, logged (response `Pragma` has no standardized meaning, RFC 9111 §5.4) | drop-individually, logged | +| `X-DataDome` | forward, owner-scoped typed telemetry | forward, owner-scoped typed telemetry | +| `X-DD-B` | forward as a browser-response security signal; never copy to publisher-upstream or another integration | forward as a browser-response security signal | +| anything not listed | invalidate the batch → Continue | invalidate → effects dropped | + +Singleton-field multiplicity (`Location`, `Content-Type`, `X-DataDome`, +`X-DD-B`, `X-Set-Cookie` appearing more than once) invalidates the +batch atomically; list-valued fields (`Cache-Control`, `Pragma`) join +per RFC 9110 §5.3 before their cell applies (hook spec §4a). + +`X-DD-B` is security-owned when DataDome is enabled. Before applying the fresh +security batch, core removes every pre-existing instance from the origin, +cached ordinary artifact, 304 metadata update, core response, or ordinary +mutator. A valid pointed vendor value then uses **replace-all** and the final +response cardinality must be exactly one; if the fresh vendor batch does not +point to it, final cardinality is zero. Append is never allowed. Fixtures cover +origin collision, cache-hit collision, 304 collision, repeated vendor fields, +and one valid fresh value, proving “exactly once” at final emission rather than +merely inside the vendor batch. + +**Fixtures**: DataDome's documented challenge response (`Set-Cookie`, +`Pragma`, `X-DataDome`, `Cache-Control`) asserts the decision stays +**Respond** with exactly the mapped fields; the documented allow example +(`Set-Cookie X-DD-B`) asserts Continue proceeds with the cookie applied +and `X-DD-B` forwarded exactly once, neither fixture may fail open. + +### 4a.3 DataDome configuration delta (normative) + +The canonical v1 effective configuration is the pre-existing DataDome schema +plus this exact PR-specific delta. The configuration parser rejects unknown +security/session fields rather than ignoring them as compatibility toggles, +and the materialized values below participate in permission §5.5's complete +effective-config digest. + +```toml +[integrations.datadome] + +# Existing timeout_ms remains the 1500 ms first-byte bound. +complete_response_timeout_ms = 3000 +challenge_body_max_bytes = 65536 + +# Required when enable_protection = true; there is no silent lifetime default. +security_cookie_max_age = 2592000 # example; allowed 604800..=31536000 +security_cookie_domain = "host-only" # or one exact normalized ASCII domain +security_cookie_same_site = "Lax" # Lax | Strict | None + +expose_client_id_to_origin = false +expose_host_fingerprints_to_vendor = false +``` + +The Protection API authority is not configurable: it is the core-owned +`https://api-fastly.datadome.co/validate-request` endpoint defined in §4a.2.1, +with redirects disabled. The baseline `protection_api_origin` field is a +startup error under this delta, as are `sessionByHeader`, `session_by_header`, +any equivalent session-mode spelling, and every other unknown +security/session field. Supporting another region or a publisher proxy is a +reviewed endpoint-registry and product/security decision, never a free-form +URL setting. + +`complete_response_timeout_ms` defaults to and may not exceed 3000; +`challenge_body_max_bytes` defaults to and may not exceed 65,536. +`security_cookie_max_age` is mandatory when protection is enabled and must be +in `604800..=31536000` seconds. `security_cookie_domain` defaults to +`host-only`; an explicit value must pass §4a's exact-domain, domain-match, PSL, +and active-scope-change rules. `security_cookie_same_site` accepts exactly +`Lax`, `Strict`, or `None`; `None` is valid only with the unconditionally +emitted `Secure` attribute, and the cookie never carries `HttpOnly`. + +Both exposure booleans default to `false`. ClientID-to-origin additionally +requires the owner-scoped overlay capability. Host fingerprints additionally +require qualified JA4 availability and sign-offs 23/28. A selected adapter +that cannot preserve admitted request-header field-line order or enforce the +request/body limits fails startup for protection rather than synthesizing +different evidence. + +## 4. Done-when (from #782, sharpened) + +1. Trait + builder + registry application, each public item documented. +2. **The old generic `RequestFilterEffects.response_headers` channel is + removed.** DataDome uses the separate sealed, core-registered + `DataDomeSecurityRequestFilter` and typed `DataDomeSecurityEffects` defined + by §4a's PR-specific delta. Generic request filters + receive only `RedactedRequestView` and ordinary attributed effects and + cannot express the security view, owner overlay, cookie operation, or + reserved security header. The dedicated channel is necessary because + DataDome sets headers **and cookies** on 200, 301/302, 401, 403, and 429 + responses, response classes (§3a) the ordinary response hook never runs + on, with cookie emission v1 reserves. +3. **At least one real consumer ships in the same PR**, an existing + integration registering a mutator for a real need (or, failing a real + need, the feature waits; scaffolding with only self-referential tests is + dead code and will be removed). +4. Every adapter applies mutations on its outbound path, with a per-adapter + route test asserting an integration-set header appears in the response. +5. A parity-suite case asserts identical mutation behavior across adapters. +6. Reserved-surface, append/replace, operation-limit, and erroring-mutator + semantics covered by unit tests. +7. **Every row of the §3a eligibility matrix has a test**, streaming, + cache-hit, pass-through, redirect, error, and 304 each proven to run or + not run the hook, not merely one positive header test per adapter. +8. Cache/privacy invariant tests, one per restriction source and shape: + a **core-owned** cookie already queued before the hook + an + integration's public `Cache-Control` replacement → private/no-store, + surrogate stripped; **core-private cookieless** processed HTML + + public replacement → restriction preserved; **origin-private cookieless** processed HTML that retained the + origin's cache restrictions + public replacement → restriction + preserved (pass-through responses never run the hook, §3a); a cache-hit serve re-applying mutations without + weakening the stored classification; a `Vary` mutation neither + dropping core-required values nor bypassing the snapshot; the §3 + normalization fixture proves field-line grouping/case/order collapse to one + sorted lowercase descriptor, repeated names digest once, request value + instances retain octet/order boundaries, malformed/empty mutation members + reject, a malformed snapshot becomes `Vary: *` plus `no-store`, and `*` + writes no artifact; a `pre_epic_v1` artifact/index descriptor is a miss + immediately after the model-only `permissions_v2` activation CAS even when + config and policy digests are unchanged; each of the four enumerated CDN fields (`Surrogate-Control`, + `CDN-Cache-Control`, `Cloudflare-CDN-Cache-Control`, `Edge-Control`) + individually stripped; and a rejected `Content-Encoding` mutation. + +## 5. Size and sequencing + +The ordinary mutator API in §§2–3 is a modest headers-only feature with no +provider or permission-model coupling. It lands only with the real consumer +required by §4 item 3; without one, scaffolding does not ship. The separately +typed security channel in §4a is already that consumer's PR-specific contract: +it owns DataDome cookie operations and intentionally couples configuration +activation to permission §5.5. Those capabilities never become part of the +ordinary mutator API. + +## 6. Divergences from issue #782 + +This spec supersedes #782 on the following points; the issue is updated to +reference this spec when the implementing PR (the hook's return with its +first consumer, §7) merges: + +| #782 says | This spec says | Why | +| -------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | +| Mutations apply to the outbound response generally | Eligibility is the explicit §3a matrix, centered on processed documents | Pass-through/error/304 mutation has different semantics per adapter; enumerating beats implying | +| Ship the trait + registry + adapter application | Additionally: structured operations API (§2), reserved surface (§3), and a real consumer in the same PR (§4, item 3) | PR #838 shipped the trait with zero call sites; an unrestricted `&mut HeaderMap` cannot enforce any collision policy | + +## 7. Disposition (2026-08-25) + +- The earlier draft of PR #1047 carried an `IntegrationResponseMutator` + response-header hook. The hook was removed from that PR before the series + was presented, so PR #1047 ships the documentation set only and no hook + trait, registration builder, or adapter call site exists in the series' + tree. This is exactly the outcome §4 item 3 and §5 require for a feature + with no consumer, applied to the feature's own spec. +- The hook returns together with its first consumer, meaning an integration + that must set response headers, for example `Accept-CH` client-hint + requests or detection results such as §4a's security channel. The + returning PR implements this spec, not PR #838's shape. +- Recorded for that future design, from review of the earlier draft: the + `&mut HeaderMap` shape concern stands. Handing a mutator a mutable + header map makes §3's collision policy unenforceable by construction, + because core cannot validate or attribute writes it never sees, so the + structured, attributed operations API of §2 is the required shape and the + `&mut HeaderMap` API must not return with the hook. diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md new file mode 100644 index 000000000..ec98c2844 --- /dev/null +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -0,0 +1,826 @@ +# Design Spec: Jurisdiction Permission Model + +**Status:** Implemented in PR #1045; revised against the implementation, 2026-08-25. +**Author:** Engineering +**Issue references:** #779 +**Related specs:** `2026-07-30-pluggable-providers-design.md`, +`2026-07-30-provider-migration-rollout-design.md` +**Last updated:** 2026-08-25 + +> **Context.** PR #838 proposed a permission model whose review surfaced two +> classes of defect this spec exists to prevent: (1) silent behavioral +> inversions of consent-signal precedence, most seriously a present TCF string +> short-circuiting GPC/GPP/US-Privacy opt-outs, and (2) fail-open jurisdiction +> resolution when geolocation is disabled. Both defect classes are closed in +> the implementation. The precedence rules (§4) and the failure-mode matrix +> (§6) are normative and are now backed by pinning tests in +> `crates/trusted-server-core/src/ec/consent.rs`. One structural position of +> the 2026-07-31 draft was not adopted: policy remains a build-time-embedded +> `permissions.yaml`, not a `[permissions]` section of `trusted-server.toml`, +> because the runtime config push and activation apparatus the draft assumed +> does not exist yet (§3.1). Every other draft position that was narrowed, +> simplified, or deferred is recorded in §11. + +--- + +## 1. Overview + +The permission model replaces the hard-wired jurisdiction gate +(`allows_ec_creation` and its companions, now removed) with a single resolved +**permission set** per request. The data decisions Trusted Server itself makes +through this model are EC provider execution, EC creation and withdrawal, EID +transmission into the bidstream, and sharing of the EC identifier beyond the +edge (§7). Server-side auction dispatch is not yet a consumer (§7.4). + +The set is resolved from three inputs: + +1. **Jurisdiction**, the country and optional region the request resolves to + (§5). +2. **Policy**, a declarative map from jurisdiction to a baseline acquisition + rule per permission, plus a declared signal policy (§3). +3. **Signals**, the request's privacy signals, being TCF, GPP, GPC, and US + Privacy (§4). + +These are the initial sources. Issues #777 and #779 also envision publisher +interaction and external services as permission sources. That source interface +remains **explicitly deferred**, not silently dropped. §10 records the +divergence, and the documentation (`docs/guide/permission-model.md`) already +frames consent as one source among several so a later source plugs into the +same mechanism. Core code resolves permissions through a per-permission +`ConsentSignal` closure (`Grant`, `Revoke`, `Neutral`), so a new source is a +new producer of that signal, not a new resolution algorithm. + +Scope: the model governs decisions Trusted Server makes. A downstream protocol +receives the full regulatory context only where that protocol defines fields +for it (OpenRTB consent fields, and proxy-mode forwarding of raw strings). +The draft's stronger rule, that identity rows carry normalized per-permission +provenance and a digest and never a raw string, is **not yet true**: the +identity-graph entry (`KvEntry` in `ec/kv_types.rs`) stores the raw TCF and +GPP strings alongside the row today. The normalized provenance model travels +with the providers-spec storage work and is recorded as deferred (§11). + +## 2. Vocabulary: the IAB Privacy Taxonomy Data Uses + +Permissions are named by **IAB Privacy Taxonomy Data Uses**, mapped from the +IAB TCF Europe purposes and used strictly as technical identifiers. No CMP or +TCF policy is implemented by naming them. This replaces the draft's +TCF-purpose-identifier vocabulary (`store-on-device`, +`select-personalised-ads`). The taxonomy adoption postdates the 2026-07-31 +draft and follows the joint taxonomy work with the IAB Tech Lab. + +The implementation (`crates/trusted-server-core/src/permissions.rs`) models +the vocabulary in two tiers: + +- **Eleven named permissions**, one per TCF purpose 1 through 11, each with a + Data Use identifier. All eleven are resolved against the session signal + (§4): a present TCF record grants or revokes each mapped purpose. Two + purposes have no published Data Use yet, so purpose 1 uses a proposed + `necessary.operations.storage` key and purpose 11 keeps its TCF identifier + `select-basic-content`, both flagged for an upstream taxonomy addition. +- **Fifty-three additional taxonomy Data Uses**, carried so `permissions.yaml` + can declare a policy flag for the whole taxonomy. No provider gates on them + today and no signal maps to them, so their configured baseline stands. + They exist for completeness, testing, and demonstration, and where no + informed policy decision has been made the shipped file sets them `denied`. + +The two Data Uses that carry enforcement weight today are +`necessary.operations.storage` (TCF Purpose 1, storage) and +`advertising_marketing.first_party.targeted` (TCF Purpose 4, personalized-ad +selection). Provider execution gates on whatever a provider declares (the +built-in Edge Cookie providers declare storage), and sharing beyond the edge +gates on the storage plus personalized-ad pair (§7). + +This is a deliberate departure from the draft's rule that a permission appears +only when it has both a signal mapping and an enforcement point. The eleven +named Data Uses all have the signal mapping, and the fifty-three baseline-only +Data Uses are declared policy rather than enforced policy. The file header of +`permissions.yaml` states this plainly, and the `denied` default means an +undeployed flag cannot silently authorize anything. Policy validation still +**rejects** any rule or flag that references an identifier outside the modeled +vocabulary, so a policy cannot name a Data Use the code does not know. + +The eleven named Data Uses, with the TCF purpose each maps from: + +| # | Data Use identifier | TCF purpose | +| --- | ----------------------------------------------- | ----------------------------------------------- | +| 1 | `necessary.operations.storage` | Store and/or access information on a device | +| 2 | `advertising_marketing.first_party.contextual` | Use limited data to select advertising | +| 3 | `advertising_marketing.profiling` | Create profiles for personalised advertising | +| 4 | `advertising_marketing.first_party.targeted` | Use profiles to select personalised advertising | +| 5 | `advertising_marketing.personalize.profiling` | Create profiles to personalise content | +| 6 | `advertising_marketing.personalize.content` | Use profiles to select personalised content | +| 7 | `analytics.ad_reporting.measure_ad_performance` | Measure advertising performance | +| 8 | `analytics.ad_reporting.content_performance` | Measure content performance | +| 9 | `analytics.ad_reporting.market_research` | Understand audiences through statistics | +| 10 | `necessary.operations.improve` | Develop and improve services | +| 11 | `select-basic-content` | Use limited data to select content | + +(The purpose names are the IAB names verbatim, including their original +spelling.) + +## 3. Policy + +### 3.1 Location: `permissions.yaml`, compiled into the build + +Policy lives in a human-editable `permissions.yaml` at the repository root, +compiled into the binary with `include_str!` and parsed once per instance +(cached behind a `OnceLock` in `PermissionMaps::standard`). A deployer edits +or replaces the file and rebuilds to change policy. The file is not read at +runtime. + +This keeps the mechanism the draft rejected, for a reason the draft's own +premise no longer supports: the draft required policy to flow through the +runtime config pipeline (`ts config push`, staged activation, §5.5), and that +activation apparatus does not exist. Publishing a `[permissions]` TOML section +with no activation protocol would reintroduce exactly the mixed-revision and +lazy-validation hazards the draft cataloged. Until the runtime pipeline +exists, the embedded file is the safer home, and moving policy to runtime +configuration is recorded as deferred follow-up (§11), not abandoned. + +Within the embedded model, the draft's specific complaints are answered: + +- The parse runs once per instance, and the embedded file is a build-time + constant covered by unit tests, so a malformed committed file fails the + test suite rather than surfacing as a per-request failure. The documented + panic on a malformed embedded file is a build defect signal, not a runtime + condition. +- Unknown fields on a detailed rule are rejected (`deny_unknown_fields`), so + a misspelled override key fails loudly instead of being swallowed. +- Two rule keys naming the same location in different case are rejected at + parse, so one spelling cannot silently overwrite another. +- Auditability lives where the draft placed it, in version control: the file + ships in the repository, and its history is the change log. + +The `include_str!` path still reaches above the crate root, so the crate +packaging concern the draft raised remains open and moves with the runtime +follow-up. + +**Fallback posture.** The file is always present, so there is no "no policy" +state. A location that resolves no rule and has no configured default, and a +failed geo lookup, both resolve every permission to the **requires-signal +floor**: nothing is set without a signal that grants it. Absence of an +applicable rule is always safe, and there is no fail-open default. The +draft's `regime = "gdpr"` component of the protective profile has no +implemented counterpart because no regime concept exists (§3.2). + +### 3.2 Format + +Named **groups** (baselines) and **rules** mapping a country (`FR`) or +country/region pair (`US/CA`) to a group, plus a **signals** section that +declares how each session signal maps onto Data Uses. Each permission resolves +to an **acquisition rule**: + +- `granted`, set without any signal, +- `requires_signal`, set only when a signal grants it (opt-in), +- `denied`, never set, even when a signal grants it. + +```yaml +# permissions.yaml (abbreviated). The shipped file lists every Data Use in +# every group so each group's meaning is fully explicit. +groups: + gdpr-eu: + necessary.operations.storage: requires_signal + advertising_marketing.first_party.targeted: requires_signal + # ... every remaining Data Use, requires_signal or denied + gdpr-uk: + necessary.operations.storage: granted + # ... the other mapped purposes requires_signal, the rest denied + us-opt-out: + necessary.operations.storage: granted + advertising_marketing.first_party.targeted: granted + # ... the other mapped purposes granted, the rest denied + +rules: + FR: gdpr-eu + GB: gdpr-uk + US: us-opt-out + AU: us-opt-out + # A country/region key takes precedence over its country. A detailed rule + # applies explicit per-permission acquisitions on top of its group: + # US/CA: + # group: us-opt-out + # permissions: + # advertising_marketing.first_party.targeted: requires_signal + +signals: + tcf: + authoritative: true + purposes: + 1: necessary.operations.storage + 4: advertising_marketing.first_party.targeted + # ... purposes 2, 3, 5..11 likewise + us_opt_out: + sources: [gpc, gpp_sale_opt_out, us_privacy_opt_out] + revokes: all +``` + +Format rules, as implemented: + +- A group is a flat map of Data Use to acquisition flag, with an optional + `default` key covering any permission the group omits. A group without + `default` must list **every** modeled permission exactly once, or the + parse fails (`IncompleteGroup`). The shipped groups list every Data Use. +- A detailed rule is `{group, permissions}` where `permissions` maps a Data + Use to an explicit acquisition (`granted`, `requires_signal`, or + `denied`), overriding the group baseline for exactly that Data Use. This + adopts the draft's requirement that overrides name explicit target states. + The earlier `+`/`-` sigil scheme, which could not express + `requires_signal`, is gone. +- A rule key is a bare country or a `country/region` pair. Keys are matched + case-insensitively, and a region entry takes precedence over its country + entry. +- The **signals** section is new relative to the draft: the TCF purpose to + Data Use mapping, the opt-out source list, and the opt-out revoke set are + data in the file, so no signal-to-permission policy lives in the code. The + `signals.tcf.authoritative` flag governs only whether a present TCF + record's own grants and revokes apply. It never lets a TCF record override + an opt-out (§4). The `us_opt_out.revokes` value is `all` or an explicit + list of Data Uses, so a deployer bounds what an opt-out drops. + +The draft's required per-group **`regime`** class (`gdpr`, `us-privacy`, +`none`) is **not implemented**. Its intended consumer, server-side auction +dispatch, was not migrated to the permission model (§7.4), so the field would +be inert today. It returns with the dispatch migration. + +### 3.3 Validation + +Validation runs where the policy actually enters the system: + +- **At parse**, meaning the unit tests and any `PermissionMaps::from_yaml` + caller, the file is rejected for: malformed YAML, a rule referencing an + undefined group, an unknown Data Use identifier anywhere (group flag, + detailed-rule entry, signals purpose map, or revoke list), an acquisition + value outside `granted | requires_signal | denied`, a group without + `default` that does not list every permission, duplicate rule keys under + case-insensitive comparison (`us` and `US`), unknown fields on a detailed + rule, and a `revokes` keyword other than `all`. +- **At startup**, `[geo] default_country` must be set and must resolve to a + rule in the compiled `permissions.yaml` + (`GeoConfig::validate_default_country`), and the no-geo acknowledgment + must be present where required (§5.3). Both are settings-construction + failures, never per-request failures. + +Not implemented from the draft's list, and recorded as future hardening +(§11): checking rule-key country parts against the assigned ISO 3166-1 list, +checking region parts against assigned ISO 3166-2 subdivisions, and the group +identifier grammar. A mistyped country key (`DL` for `DK`) therefore still +parses. For the shipped table the EU and EEA coverage test (§3.5) closes the +consequence the draft cared about, a member state silently dropping to the +fallback. + +### 3.4 One source of jurisdiction truth (deferred) + +Not implemented. `detect_jurisdiction`, driven by the runtime lists +`consent.gdpr.applies_in` and `consent.us_states.privacy_states`, remains the +jurisdiction source for the auction consent gate and for +`ConsentContext.jurisdiction`, while the permission model resolves against +`permissions.yaml` independently. The drift risk the draft named is real and +stands recorded: adding a country to one source has no effect on the other, +and no CI test asserts consistency between the legacy lists and the policy +table. Unifying the two, with the auction gate reading a policy regime class, +travels with the dispatch migration (§7.4) as deferred follow-up. + +### 3.5 Shipped-table coverage + +Implemented as a unit test +(`every_eu_and_eea_member_requires_a_signal_for_storage` in +`permissions.rs`): every one of the 27 EU member states plus the three EEA +members (IS, LI, NO), 30 codes in all, must have a rule, and each must +resolve `necessary.operations.storage` as `requires_signal`. A mistyped +member-state key fails this test rather than silently diverting the country +to the deployer default. The shipped table maps the EU 27 and EEA to +`gdpr-eu`, the UK to `gdpr-uk` (storage `granted`, a baseline the rollout +ledger row 4 asks the task force to confirm with its citation, everything +else opt-in), and the US and Australia to +`us-opt-out`. Countries with no rule fall to the deployer's +`[geo] default_country` (§5.4). + +## 4. Signal precedence (normative, implemented) + +Precedence is **fixed in code** +(`permission_signal` in `crates/trusted-server-core/src/ec/consent.rs`), not +in policy, and runs most restrictive first. The policy file decides which +sources count and what they revoke or grant. The code decides only the order. + +1. **Policy `denied`** is never set, regardless of any signal. (Enforced in + the resolver, `PermissionMaps::resolve_with`.) +2. **A US-style opt-out always suppresses the Data Uses the policy revokes**, + regardless of any consent record present. The opt-out sources are the + `Sec-GPC` header, a GPP US sale opt-out, and a US Privacy sale opt-out, + as declared in `signals.us_opt_out.sources`. A GPC header suppresses the + revoked Data Uses even when an accompanying TCF string consents to them. + An explicit opt-out is never overridden by another signal, and the + `signals.tcf.authoritative` flag cannot change that. (This is the rule + PR #838 inverted. Three pinning tests now hold it in place, one per + opt-out source against a consenting TCF record.) +3. **A consent record present but undecodable revokes everything.** A + malformed record is a preference that could not be read, so it fails + closed rather than degrading to the no-signal baseline, which under a + `granted` baseline would turn garbage into a grant. It never withdraws + (§4.2). An **expired** TCF record is deliberately a distinct state, not + malformed: the decoded record is cleared, the raw string is kept for + proxy forwarding, and acquisition proceeds as if the record were absent, + so the baseline applies. +4. **Only then does a present TCF record decide the mapped Data Uses**, when + `signals.tcf.authoritative` is true: granted where the record consents to + the mapped purpose, revoked where it does not, neutral where no purpose + maps. The effective record is the standalone TC string or the EU TCF + section of a GPP string (`effective_tcf`). A TCF refusal of a mapped + purpose is a revoke at this step, which drops a `granted` baseline and + leaves a `requires_signal` baseline unset. +5. **No signal leaves the baseline standing**: `granted` sets the + permission, `requires_signal` leaves it unset. + +Two simplifications against the draft's taxonomy, both recorded in §11: + +- **TCF is the only grant-class signal.** The draft's grant class also + admitted explicit GPP/USP non-opt-out values, regime-scoped, so a US rule + could be `requires_signal` yet grant on signal-carrying traffic. + The implementation instead expresses the US posture as a `granted` + baseline that opt-outs revoke, so no-signal US traffic is allowed rather + than blocked pending a signal. Explicit non-opt-out GPP/USP values grant + nothing on their own. +- **Malformed-present blocks everything, not per family.** Any present but + undecodable record (TCF, GPP, or US Privacy) revokes every Data Use for + the request, rather than blocking only the permissions mapped to the + malformed source. This is strictly more restrictive than the draft's + per-family rule. + +### 4.1 Decision matrix + +For each permission, with baseline _B_ from the resolved rule: + +| Signal state (per §4 order) | B = granted | B = requires_signal | B = denied | +| ------------------------------------------- | ----------- | ------------------- | ---------- | +| Opt-out present, Data Use in the revoke set | unset | unset | unset | +| Any record present but undecodable | unset | unset | unset | +| TCF present, consents to the mapped purpose | set | set | unset | +| TCF present, refuses the mapped purpose | unset | unset | unset | +| No signal (or neutral for this Data Use) | set | unset | unset | + +An expired TCF record resolves as the "no signal" row. Whether an unset +outcome is also a **withdrawal** is a separate, narrower question (§4.2). + +### 4.2 Withdrawal vs. absence + +Withdrawal (destructive: expire the `ts-ec` cookie, write the identity-graph +tombstone) and non-grant (the permission is simply unset, EC response +headers stripped, nothing egressed) are distinct outcomes, never conflated. +"Baseline" below means the resolved acquisition rule for +`necessary.operations.storage` in the request's jurisdiction, resolved once +at `EcContext` construction (`storage_acquisition`), never a group label. + +The implemented trigger, exhaustively (nothing else withdraws): + +1. **A TCF record refusing storage (Purpose 1) withdraws iff the baseline is + not `granted`, and only when the refusal is carried by the live + request.** Under a `requires_signal` (or `denied`) baseline the refusal + is the visitor declining the very signal storage depends on, so it + withdraws. Where the baseline is `granted`, storage never depended on + the record, so the refusal suppresses use without destroying the + identifier. Tombstones are irreversible, and PR #838 wrote them for + visitors in unregulated jurisdictions whose global CMP emitted a + purpose-refusing string. The `EcContext` consent pipeline runs without + the persisted-KV inputs, so the withdrawal decision sees live request + signals only, satisfying the draft's live-request constraint by + construction. +2. **US-style opt-outs never withdraw.** GPC and sale opt-outs are use + restrictions: they suppress the permissions the policy revokes (EC + headers stripped, nothing egressed) but never trigger destruction, so + lifting the opt-out restores the identity. +3. **A malformed record never withdraws.** It suppresses only (§4, step 3). + Destruction requires an affirmative, decodable signal. +4. **Absence of signal never destroys identity.** A visitor who has not yet + made a choice is never stripped of an existing identity. +5. **A policy change is not a user signal.** There are no runtime policy + edits (§3.1), and a rebuild that tightens a baseline does not itself + tombstone: withdrawal still requires the affirmative refusal above on a + live request. + +The draft's additional trigger, an explicit storage-withdrawal or +authenticated deletion request honored in every jurisdiction, has no +implemented carrier: no such endpoint exists. It is recorded as deferred +(§11), and when it arrives it joins this list as a global trigger. + +`ec_storage_withdrawn` (in `ec/consent.rs`, surfaced as +`EcContext::storage_withdrawn`) has direct unit coverage for every arm +above: refusal under `requires_signal` withdraws, refusal under `granted` +does not, consent does not, GPC alone does not, sale opt-outs do not, no +signal does not, malformed does not. + +### 4.3 Withdrawal durability (largely deferred) + +Implemented behavior (`ec/finalize.rs`): when the request carries the +withdrawal signal and the client presented a cookie, the response expires +the EC cookie, and the identity-graph tombstone is written for each +presented identifier the provider accepts (the incoming cookie value and +the active value). The tombstone is the authoritative revocation marker for +subsequent EC behavior. A tombstone write failure is logged at error level +and the request completes, so the write is best effort. + +The draft's durability protocol is **not implemented** and is recorded as +deferred follow-up in full: the family ID with deterministic derivation for +legacy rows, the family revocation record written before the cookie +expires, the permission-exempt suppression and authority-state records with +CAS fencing, evidence-recency comparison and anti-replay pinning, the +durable negative-intent outbox, the global identity safety breaker, and the +associated consistency and retention contracts. That machinery depends on +storage primitives (linearizable per-key CAS, independent durability +domains) the current adapters do not qualify. The 2026-07-31 draft remains +the reference design for that work. Until it lands, the known gaps the +draft called out stand: cookie expiry is not fenced on the tombstone +commit, and revocation durability is bounded by the KV store's behavior. + +### 4.4 Signal normalization + +The consent subsystem (`consent/mod.rs`) remains the decoder and +normalizer. The permission layer consumes its output only through the +per-permission `ConsentSignal` closure. The implemented pipeline: + +1. Extract raw signals from cookies and headers, and decode TCF v2, GPP, + and US Privacy. A decode failure keeps the raw string and leaves the + decoded field empty, which the permission layer reads as + malformed-present (§4, step 3). +2. Resolve standalone-TCF vs GPP-embedded-TCF conflicts per the configured + mode (`restrictive`, `permissive`, `newest`), preserving the pre-epic + selection algorithm. +3. Apply the expiry check: a TCF record older than the configured maximum + age has its decoded form cleared, the `expired` flag set, and its raw + string preserved. Expiry is its own state, excluded from + malformed-present, and resolves as absent for acquisition. +4. Construct a US Privacy string from GPC for US privacy states with no + explicit USP cookie, so the opt-out also travels in transport fields. + +The draft's declared reordering, expiry filtering **before** conflict +resolution, was **not implemented**: conflict resolution still runs first, +so the pre-epic order stands. Recorded in §11. + +**Persisted-KV consent.** When the pipeline runs with an EC ID and a KV +store (not on the `EcContext` construction path), a request carrying no +consent signals falls back to the consent persisted for that EC ID, with +the jurisdiction re-derived from the current request's geo. Staleness is +enforced by the store: entries are written with a TTL equal to +`max_consent_age_days`, so an entry older than a live record's allowed age +has expired out of the store. A live signal always wins because the +fallback is consulted only when the request carries none. The draft's +declared change, running the loaded record through the full normalization +pipeline, is not implemented: the loaded record substitutes directly. The +narrow read is permission-exempt by construction, since determining +storage cannot itself require storage. + +**Proxy mode.** Proxy mode still skips semantic decoding entirely. The +draft's minimal opt-out extraction was not implemented, but the fail-open +consequence the draft feared does not arise under the permission model: a +present record in proxy mode is present-but-undecoded, which blocks every +baseline grant (§4, step 3), and the GPC header needs no decoding, so the +GPC opt-out is honored directly. No grants are ever derived in proxy mode. +The net posture is equal to or more restrictive than the draft's row. +Absent records resolve to the baseline. + +### 4.5 US signal field mapping + +Implemented sources, as declared in the shipped `signals` section: + +| Source | Effect | +| ---------------------------------- | --------------------------------------- | +| `Sec-GPC` request header | US-style opt-out | +| GPP US section sale opt-out | US-style opt-out | +| US Privacy `opt_out_sale = Y` | US-style opt-out | +| US Privacy `opt_out_sale = N` | Nothing (no grant class exists for USP) | +| Any explicit Not Applicable value | Nothing | +| Absent / unknown / reserved values | Nothing | + +An opt-out revokes the Data Uses the policy's `revokes` value names. The +shipped file says `revokes: all`, so an opt-out drops **every** granted +Data Use, including storage. That is deliberately broader than the draft's +mapping, which scoped sale opt-outs to personalized-ad selection only, and +a deployer narrows it by listing specific Data Uses instead. No +sale-family opt-out is destructive (§4.2). + +The remainder of the draft's §4.5 is **not implemented** and is recorded +as deferred: `SharingOptOut` and `TargetedAdvertisingOptOut` as distinct +inputs, grant-class non-opt-out values, embedded-GPC detection inside GPP +sections, the per-section applicability and aggregation algorithm with its +state-over-national precedence, the mapped-section malformed blocker at +per-section granularity, the derived OpenRTB `gpp_sid` construction with +the `__gpp_sid` consistency companion, and the complete pinned section +map. The current GPP decoder surfaces the EU TCF section, the section ID +list, and a US sale opt-out. Extending it to the full pinned registry is +its own project. + +#### 4.5.1 GPP registry snapshot (deferred) + +Not implemented. The vendored registry snapshot, the pinned per-section +accepted versions, the provenance manifest, and the fixture corpus travel +with the §4.5 decoder work. The 2026-07-31 draft's §4.5.1, including the +pinned upstream commit, remains the reference for that effort. + +## 5. Jurisdiction resolution + +### 5.1 Order + +Geo resolution runs **before** permission resolution. Jurisdiction is an +input to the permission set, which is why geo providers cannot themselves +be gated on it (providers spec §5). The selected geo provider resolves a +country and optional region, and rules match `country/region` first, then +`country`, case-insensitively. Implemented in +`EcContext::read_from_request_resolving_geo` and +`PermissionMaps::rules_for`. + +### 5.2 Lookup failure + +Implemented, with the failure state carried explicitly: +`GeoStatus { Located, NoLocation, Failed }` (in `ec/consent.rs`) separates +"the provider resolved no location" from "the lookup errored". A **failed** +lookup resolves every permission to the **requires-signal floor**, never +the deployer's `[geo] default_country`. The failure is logged at error +level so it is visible. The storage-withdrawal baseline follows the same +floor, so a failed lookup cannot widen destructive withdrawal either. The +rule is proven through the seam by +`a_geo_provider_failure_resolves_permissions_at_the_requires_signal_floor` +in `ec/mod.rs`, which drives a `PlatformGeo` that returns an error rather +than constructing the status by hand. + +**Which providers can reach it.** The floor is the contract for a geo +provider that performs its own fallible lookup, and none of the providers +shipped in this workspace is one. Fastly's `geo_lookup` returns an +`Option` and the SDK collapses every hostcall, buffer and parse failure +into `None` before it reaches the caller; the Cloudflare provider reads +request headers, which cannot error; the Axum and Spin providers resolve +nothing at all; and `DisabledGeo`, the default whenever +`[geo] provider` is not `"platform"`, returns nothing by construction. So +**a host geo outage today does not reach this floor.** It surfaces as +`Ok(None)`, which is `NoLocation`, and falls back to the deployer's +`[geo] default_country` like any other unmatched request. A deployer +choosing a permissive default country is therefore choosing what a geo +outage does, and should read §5.3 and the default-country guidance with +that in mind. The floor becomes reachable when a vendor geo crate under +`crates/geo/` does a real lookup that can fail. + +At the floor, an explicit valid grant still counts: a TCF record consenting +to a mapped purpose sets that permission under `requires_signal`, exactly +the divergence-from-deny-all the draft declared for this row. Absent, +malformed, or refusing evidence sets nothing. + +Not implemented: a lookup-failure metric (the error log is the signal +today) and the draft's `regime = "gdpr"` component of the failure profile, +since no regime concept exists. The draft's capability check, that an +adapter whose geo implementation can never resolve anything must not accept +the selection, is part of the providers-spec provider qualification rather +than this model. + +### 5.3 No geo provider selected + +Every request resolves to `[geo] default_country`, so jurisdiction becomes +a static constant, which is only honest when the operator can genuinely +assert single-jurisdiction traffic. + +Constraint, implemented in +`GeoConfig::validate_jurisdiction_acknowledgment`: **startup fails** when +an Edge Cookie provider is configured and no geo provider is selected, +unless the operator sets `[geo] assume_single_jurisdiction = true`. This +makes the dangerous migration config (a permissive `default_country` with +geo unset) an explicit operator decision rather than an accident, closing +the highest-severity finding of the PR #838 review. + +The guard's consumer list is narrower than the draft's: the draft enumerated +every jurisdiction consumer (EC provider, regime-gated auction dispatch, +raw-EC and EID egress). In the implementation the EC provider is the only +consumer whose behavior the policy gates, because auction dispatch was not +migrated (§7.4), so the guard fires on the EC provider selection alone. When +dispatch joins the model, the guard's trigger list grows with it. + +### 5.4 Defaults: one deployer fallback plus a protective floor + +`[geo] default_country` is **required in every mode** and is validated at +startup: it must be set, and it must resolve to a rule in +`permissions.yaml`. It accepts a country (`FR`) or a country/region key +(`US/CA`), matched case-insensitively, so a no-geo single-state deployment +can select its state rule. It covers two states the draft kept separate: + +- the geo provider resolved no location (or none is configured), and +- the resolved country or country/region matches no rule. + +The draft's `rules.default` policy entry does not exist, so +"resolved-but-unmatched" falls to the same deployer default as +"unresolved". The separation the draft treated as safety-critical is the +one the implementation does keep: a **failed** lookup never reaches the +deployer default and resolves at the requires-signal floor instead (§5.2). +With no default configured startup fails, and in the unreachable +belt-and-braces case where resolution still finds no rule, the floor +applies. + +### 5.5 Policy revision activation (deferred) + +Not implemented, and currently moot: policy is compiled into the binary +(§3.1), so the deployed artifact is the policy identity and there is no +runtime activation to coordinate. The draft's activation design, covering +the JCS-canonical policy digest and ordinal pair, the activation register +and candidate protocol, fleet readiness and quiescence, admission leases, +the hash-linked activation journal with store-clock retention, and the +model-epoch transition, is the reference design for the runtime-policy +follow-up recorded in §11. Its guarantee that mixed-revision irreversible +behavior is prohibited is honored today by construction, because a +deployment runs exactly one embedded policy and destructive withdrawal +requires a live user signal (§4.2), never a policy change. + +## 6. Failure-mode matrix (normative, implemented) + +| Condition | Resolution behavior | +| ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Geo lookup reports a failure | Requires-signal floor for every permission, never the deployer default; error logged. No provider shipped today can report one, so a host geo outage lands on the row above instead (§5.2) | +| No geo provider configured | `default_country` baseline, guarded by `assume_single_jurisdiction` (§5.3) | +| Country resolved, no matching rule | `default_country` baseline (§5.4) | +| Region resolved, no region rule | Country rule | +| `default_country` unset or names no rule | Startup failure (§3.3) | +| EC provider configured, no geo, no acknowledgment | Startup failure (§5.3) | +| Malformed `permissions.yaml` | Parse error at settings load, once per instance; the embedded file is a build-time constant, never per-request | +| Undecodable record present (TCF, GPP, or USP) | Revokes every Data Use (fail-closed acquisition); never withdraws; opt-outs still honored | +| Expired TCF record | Distinct state, not malformed; treated as absent, so the baseline applies | +| Signals contradict (opt-out plus consent) | Opt-out wins (§4) | +| No EC provider selected | Identity fails closed: nothing minted, an incoming cookie value never used or egressed (§7) | + +The posture is fail-closed. Every ambiguous state resolves to the +configured baseline or more restrictive. + +## 7. Enforcement points + +Consumers of the resolved set in the implementation: + +1. **EC provider execution.** The provider declares + `required_permissions()`, core resolves a `PermissionState` once per + request at `EcContext` construction, and the provider executes only when + every declared permission is set (`ec_allowed`). A provider that + requires nothing always runs. **Geo** is ungated because gating it is + circular, jurisdiction being an input to permission resolution. + **Device** is ungated by a separate, deliberate decision: its + security-classification role must run for traffic that has granted + nothing, and operator selection is the recorded authorization (providers + spec §5). The built-in Edge Cookie providers declare + `necessary.operations.storage`. + +2. **EC lifecycle.** Creation requires the provider's declared permissions + through the gate above. Withdrawal follows §4.2. Recognition and + revocation of an existing identifier are never permission-gated: the + withdrawal path runs precisely when `ec_allowed` is false, reading the + raw cookie value kept for that purpose. + +3. **Sharing beyond the edge.** One predicate, + `EcContext::ec_sharing_allowed`, requires the provider gate plus + **both** `necessary.operations.storage` and + `advertising_marketing.first_party.targeted`. A storage-only grant + therefore keeps first-party use while withholding partner sharing. The + implemented inventory: + + | Path | Gate | + | -------------------------------------------- | ------------------------------------------------------------- | + | Bidstream EIDs (every auction path) | `gate_eids_by_permissions`, storage plus personalized ads | + | OpenRTB `user.id` on the `/auction` endpoint | `ec_sharing_allowed` | + | Identify endpoint (partner-facing) | `ec_sharing_allowed` | + | Pull sync (browser-request-scoped) | `ec_sharing_allowed`, from the live request resolution | + | Batch sync (context-free S2S) | Authenticated; withdrawn or missing rows are ineligible | + | KV EID resolution for auctions | `ec_allowed`, then the EID pair gate on the result | + | Publisher navigation and page-bids `user.id` | `ec_sharing_allowed` (the storage plus personalised-ads pair) | + + With **no EC provider configured**, identity fails closed: the gate is + closed rather than open by default (`ec_allowed` is false), so a cookie + value present on the request is treated as absent and never used or + egressed. This replaces PR #838's vacuously-true `is_none_or` check. + + **Follow-up note (recorded, not silent).** The publisher navigation and + page-bids paths attach the EC-derived request ID and `user.id` under + the provider gate (`ec_allowed`) rather than the sharing pair, while + their EIDs are pair-gated. With the built-in providers (storage-only + requirement), a storage-only grant can therefore still place the EC in + `user.id` on those paths. Aligning them with the `/auction` endpoint's + pair gate is recorded follow-up (§11). The draft's fuller inventory + (proxy/click/Testlight forwarding gates, the observability denylist + with typed redaction boundaries, the raw-regulatory-transport + destination allowlist, integration response cookies) is deferred with + it. Today EC values are truncated (`log_id`) before logging as the + observability mitigation. + +4. **Server-side auction dispatch (not migrated).** Dispatch is still gated + by the consent subsystem (`consent_allows_server_side_auction`), not by + the permission model: when the jurisdiction is GDPR or unknown, or an EU + TCF signal is present, dispatch requires an effective TCF record + consenting to Purpose 1, and otherwise no bid request leaves (a no-bid + response, with no PBS/APS call and no UA/IP/geo forwarding). Known + non-GDPR jurisdictions without an EU TCF signal dispatch freely. The + draft's regime-keyed dispatch table, and the `ContextualAuctionView` + positive projection for dispatch with personalized-ad selection unset, + are **not implemented**. When personalized-ad selection is unset today, + EIDs and the pair-gated identifiers are stripped but dispatch is the + ordinary request, not a contextual projection. Recorded as deferred + (§11) together with §3.4. + +The client-cycle resolve endpoint (`/_ts/api/v1/ec/resolve`) is a further +consumer: a provider-derived identifier posted by the page is accepted only +through the same provider and permission gates. + +### 7.1 Contextual OpenRTB v1 allowlist (deferred) + +Not implemented. The machine-readable projection manifest, its path +grammar, cardinalities, cross-field rules, and derivation vocabulary, and +the conformance walker over final encoded bytes, belong to the dispatch +migration (§7.4) and remain specified by the 2026-07-31 draft for that +work. + +## 8. Testing strategy + +Implemented, in `permissions.rs`, `ec/consent.rs`, `ec/mod.rs`, +`ec/finalize.rs`, and `consent/mod.rs`: + +- **Signal precedence pinning.** Three opt-out-beats-TCF tests, one per + opt-out source against a consenting TCF record + (`gpc_suppresses_storage_even_with_a_consenting_tcf_record` and + companions). These reinstate the + behavior PR #838 inverted. +- **Withdrawal scoping.** One test per §4.2 arm: refusal under + `requires_signal` withdraws, refusal under `granted` suppresses without + destroying, consent is not a withdrawal, GPC alone never withdraws, sale + opt-outs never withdraw, no signal never withdraws, malformed never + withdraws. +- **Fail-closed acquisition.** Malformed records block baseline grants; + each undecodable record family is detected; an expired TCF record is not + treated as malformed and resolves at the baseline. +- **Geo status.** A failed lookup resolves at the requires-signal floor + (permissions and the storage baseline both), driven through a + `PlatformGeo` that returns an error rather than from a hand-built status, + and no-location falls back to the configured default. +- **Policy parsing and validation.** Groups, rules, detailed-rule + acquisition maps (including `requires_signal` per rule), rejection of + unknown groups, unknown permissions, unknown acquisitions, incomplete + groups, case-insensitive duplicate rule keys, unknown revoke keywords, + and the signals-section round trip. +- **Shipped-table coverage.** All 30 EU/EEA codes lock storage to + requires-signal (§3.5). +- **Vocabulary breadth.** A TCF record grants or revokes every one of the + eleven mapped purposes, not only storage and personalized ads. +- **Gates.** Provider execution blocked without its required permissions, + the sharing pair withheld on a storage-only grant, EIDs stripped when + either permission of the pair is unset, and the no-provider stateless + posture. + +The draft's fuller matrices (the complete normalization matrix as +table-driven tests, the per-row egress inventory with a denylist check, the +dispatch regime-by-signal matrix, contextual-serializer poisoning, S2S +authority denial reasons, and the §4.3 fault-injection suite) travel with +their deferred features. + +## 9. Out of scope + +- The §4.3 durability protocol, §4.5 full field mapping and registry + snapshot, §5.5 activation apparatus, §7.4 dispatch migration, and §7.1 + contextual projection: deferred follow-ups, recorded in §11 with the + 2026-07-31 draft as their reference design. Deferred, not rejected. +- Runtime policy configuration (a `[permissions]` config section): + deferred until the config push and activation pipeline exists (§3.1). +- Per-signal jurisdiction scoping (honoring GPC only where a law defines + it): rejected, as in the draft. The opt-out signal layer is + jurisdiction-free in the implementation, and the country baseline decides + only what a revoke has to drop. +- An authenticated deletion or explicit storage-withdrawal endpoint: + deferred (§4.2). + +## 10. Divergences from issue #779 + +This spec supersedes #779 on the following points, so there is one +acceptance contract, not two: + +| #779 says | This spec says | Why | +| -------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | +| Unmatched countries fall to `default_country` | Adopted: unmatched and unresolved requests both fall to the required `[geo] default_country`; a **failed** lookup floors instead (§5) | The failure state is the one that must never reach a permissive default; the draft's `rules.default` split was not kept | +| The full TCF purpose vocabulary is modeled | Adopted and extended: all eleven purposes are signal-resolved, and the full Privacy Taxonomy is carried as declared baseline (§2) | The joint taxonomy work made whole-taxonomy declaration the goal; `denied` defaults keep undeclared uses inert | +| Policy is an embedded file | Adopted: `permissions.yaml` is compiled into the build (§3.1); runtime configuration is deferred follow-up | The runtime push and activation pipeline does not exist; version control is the audit trail meanwhile | +| Permission sources are open-ended (#777: publisher interaction, external services may grant) | Sources are jurisdiction, policy, and the §4 signals; further sources are deferred, and the `ConsentSignal` closure is their seam (§1) | Shipping an interface with no second source repeats the inert-surface mistake; the extension seam is defined | + +## 11. Revision record vs the 2026-07-31 draft + +One row per divergence between the draft and the implementation this +revision was verified against (branch `split/5-response-hook-docs`, +PR #1045). + +| Draft position | Implemented position | Why | +| --------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Vocabulary is two TCF-purpose identifiers, enforced permissions only (§2) | IAB Privacy Taxonomy Data Uses: eleven named purposes all signal-resolved, plus 53 taxonomy Data Uses carried as declared but unenforced baseline flags | The joint taxonomy adoption postdates the draft; whole-taxonomy declaration serves completeness and demonstration, with `denied` defaults keeping unenforced flags inert | +| Policy lives in `[permissions]` in `trusted-server.toml`, published via `ts config push` (§3.1) | Policy is `permissions.yaml`, compiled into the build with `include_str!`, parsed once and covered by tests | The runtime config push and activation apparatus does not exist; publishing runtime policy without it would recreate the hazards the draft cataloged; runtime policy is deferred follow-up | +| Every group carries a required `regime` class read by auction dispatch (§3.2) | No regime field exists | Its only consumer, regime-gated dispatch, was not migrated; the field returns with that work | +| Overrides name explicit acquisition rules, replacing the `+`/`-` sigils (§3.2) | Adopted: a detailed rule's `permissions` map assigns `granted`, `requires_signal`, or `denied` per Data Use | The draft's requirement, expressed in the YAML schema; `requires_signal` is now expressible per rule | +| Two fallbacks: policy `rules.default` for unmatched countries, `default_country` only for the static no-geo mode (§5.4) | One required `[geo] default_country` covers unmatched and unresolved requests in every mode, startup-validated; a failed lookup floors separately | One deployer knob is simpler; the safety-critical separation kept is failure vs absence, carried by `GeoStatus` | +| Validation checks assigned ISO 3166-1 codes, assigned subdivisions, and a group identifier grammar (§3.3) | Validation covers unknown groups, permissions, acquisitions, revoke rules, incomplete groups, case-duplicate rule keys, and unknown fields on detailed rules | Smaller surface shipped first; the EU/EEA coverage test guards the shipped table against the typo class; ISO-assignment checks are future hardening | +| Three-class signal taxonomy with regime-scoped grant acceptance; the US posture is `requires_signal` with GPP/USP non-opt-out values as grants (§4) | TCF is the only grant source; the US posture is a `granted` baseline that opt-outs revoke; explicit non-opt-out values grant nothing | A simpler two-signal model without regimes; the cost, no-signal US traffic is allowed by baseline rather than blocked pending a signal, is a deliberate policy choice in the shipped file | +| Malformed-present blocks grants per record family and mapped section (§4.4, §4.5) | Any present-but-undecodable record revokes every Data Use for the request | Strictly more restrictive simplification; per-family scoping needs the full §4.5 decoder work | +| Normalization runs expiry before conflict resolution, a declared change (§4.4) | Conflict resolution still runs before the expiry check | The reordering was not implemented; the expired state itself (distinct from malformed, absent for acquisition) was adopted | +| Persisted-KV consent flows through the full normalization pipeline with an explicit TTL comparison (§4.4) | The loaded record substitutes directly when the request carries no signals, jurisdiction re-derived; staleness is enforced by the store TTL (`max_consent_age_days`) | The store-level TTL delivers the staleness bound without a second normalization pass | +| Proxy mode gains minimal opt-out extraction (§4.4) | Proxy mode still skips decoding; a present record blocks all grants via the malformed-present rule and the GPC header opt-out is honored without decoding | The permission-layer outcome is equally or more restrictive with no new decode paths; revisit with the §4.5 decoder work | +| Withdrawal has four triggers including an explicit storage-withdrawal or authenticated deletion request (§4.2) | The TCF Purpose 1 refusal under a non-granted baseline is the only trigger; opt-outs, malformed records, absence, and policy changes never withdraw (adopted) | No deletion endpoint exists to carry the extra trigger; the narrowest destructive surface shipped first | +| §4.3 durability protocol: family records first, suppression and authority-state records, outbox, breaker, strong reads | Cookie expiry plus best-effort identity-graph tombstones per presented identifier, with failures logged | The protocol requires storage primitives (linearizable CAS, independent durability domains) the adapters do not yet qualify; deferred with the providers-spec storage work | +| §4.5 field mapping and §4.5.1 vendored registry snapshot (sharing/targeted opt-outs, embedded GPC, applicability, derived `gpp_sid`) | Opt-out sources are the GPC header, a GPP sale opt-out, and a USP sale opt-out; the revoke set is policy-declared, shipped as `all` (which also drops storage) | The full decoder and registry vendoring are their own project; the policy-declared revoke set gives deployers the scoping lever meanwhile | +| §5.5 activation: JCS policy digests, ordinals, activation register, journal, drains, admission leases | None of it exists; the built binary is the policy identity | With no runtime policy there is nothing to activate; the draft remains the reference design for the runtime-config follow-up | +| §3.4 single jurisdiction truth, and §7 dispatch gated on the policy regime with a contextual projection | Auction dispatch keeps the consent-subsystem gate (effective TCF Purpose 1 for GDPR or unknown jurisdictions); `detect_jurisdiction` and its lists remain; no contextual view | Dispatch migration is follow-up; the legacy-list drift risk the draft named still stands and is recorded rather than resolved | +| Every raw-EC egress path is pair-gated, with per-row tests and a denylist check (§7) | Pair gating is centralized in `ec_sharing_allowed` (auction endpoint `user.id`, publisher navigation and page-bids `user.id`, identify, pull sync) and `gate_eids_by_permissions` (EIDs everywhere); batch sync checks row state only | Partial adoption; aligning the remaining paths, the S2S stored-provenance authority, and the inventory tests is recorded follow-up | +| Identity rows never store raw consent strings, only normalized provenance and a digest (§1) | The identity-graph entry stores the raw TCF and GPP strings with the row | The normalized provenance schema belongs to the providers-spec storage work; until then rows carry the raw strings | +| No signals block in policy; the signal mapping is fixed in the spec | New: a `signals` section in `permissions.yaml` declares the TCF purpose map, opt-out sources, and revoke set, with `tcf.authoritative` governing only TCF's own effect | Moves signal policy from code into deployer-editable data; the flag can never let a TCF record override an opt-out, preserving §4 precedence | +| The §5.3 no-geo guard covers every jurisdiction consumer | The guard fires when an Edge Cookie provider is configured with no geo provider | The EC provider is the only policy-gated consumer today; the trigger list grows when dispatch and further egress paths join the model | +| `default_country` is required only in the acknowledged static no-geo mode (§5.4) | Required always and startup-validated against `permissions.yaml` | It is the baseline for unmatched requests in every mode, so it must always exist | diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md new file mode 100644 index 000000000..78aca0684 --- /dev/null +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -0,0 +1,641 @@ +# Design Spec: Pluggable Edge Cookie, Device, and Geo Providers + +**Status:** Implemented in PR #1043 (Edge Cookie provider seam) and PR #1044 +(device and geo selection); revised against the implementation, 2026-08-25. +**Author:** Engineering +**Issue references:** #777, #778, #780, #781 +**Related specs:** `2026-07-30-permission-model-design.md`, +`2026-07-30-provider-migration-rollout-design.md`, +`2026-07-30-client-cycle-ec-resolve-design.md` +**Last updated:** 2026-08-25 + +> **Context.** PR #838 proposed a first implementation of this epic in a single +> change. Review of that PR surfaced design gaps this spec exists to close +> before a second implementation pass: an identity abstraction that owned +> minting but not recognition, per-adapter divergence in provider selection, +> silent misconfiguration modes, and speculative trait surface with no +> production caller. This spec is the authoritative statement of what the +> provider architecture must do; where it contradicts PR #838, this spec wins. +> The second pass has now landed (PR #1043 and PR #1044, with permission +> enforcement in PR #1045), and this revision restates the spec to match the +> implemented code. A final section records every divergence from the +> 2026-07-31 draft. + +--- + +## 1. Overview and goals + +Trusted Server makes three per-request data decisions that were previously +hard-wired: whether to create or keep an Edge Cookie (EC) identity, how to +classify the requesting device, and whether to resolve geolocation. Each is +now a **provider**, a selectable component chosen in operator configuration, +with a deliberately neutral default. + +Goals, as implemented: + +- A deployment picks an implementation per concern (including none) without a + code change to Trusted Server core. +- Defaults are neutral. With no configuration, no EC is created, device + classification uses only the User-Agent, and no geolocation is performed. A + default deployment makes no third-party or host-specific call. +- An **EC provider declares** the permissions its data use requires + (`required_permissions` on the trait), and **core enforces** that + declaration before minting or using an identity. A provider cannot + authorize itself. The enforcement machinery is the permission model's + subject and lands with it in PR #1045 (see the permission model spec). + Geo and device carry the same declaration method with an empty default, + for the reasons spelled out in section 5. +- All adapters (Fastly, Axum, Cloudflare, Spin) route selection through the + same core builders, so identical configuration selects identical providers + everywhere. A selection the deployment cannot satisfy fails loudly rather + than degrading. The EC API routes (identify, batch-sync, ec/resolve) are + registered by the Fastly entry point only today, because the portability + adapters do not yet wire a platform KV store. The Spin adapter's route + list documents that gap explicitly rather than leaving those paths silent. + +Non-goals: + +- No vendor provider ships in this epic beyond the host-platform + implementations named below. The `crates/edgecookie/` directory holds a + README describing where vendor EC crates will live. +- The client-cycle (browser round-trip) provider type has its own spec. The + trait ships the seam for it (`resolve_from_client`, a no-op by default) + and a demonstration provider (`client-fixed`) compiled only into test and + demonstration builds. Production selection of the demo provider is a + startup error. + +## 2. Provider taxonomy + +| Concern | Trait | Built-in default | Opt-in implementations | +| ----------- | -------------------- | --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | +| EC identity | `EdgeCookieProvider` | none (stateless) | `hmac` (in core, HMAC over client IP, preserves today's identity), `host-signals` (in core, see below), and `client-fixed` (demo builds only) | +| Device | `DeviceProvider` | `builtin` (User-Agent only) | `fastly` (TLS JA4 and HTTP/2 signals through an injected `HostSignals` service) | +| Geo | `PlatformGeo` | none (no location) | `platform` (host geo lookup) | + +The geo trait is the existing `PlatformGeo` in `platform/traits.rs` rather +than a new `GeoProvider` name. The EC trait lives in `ec/provider.rs` and the +device trait in `ec/device.rs`. + +Selection keys are strings in operator configuration +(`trusted-server.example.toml` carries the commented template): + +```toml +[ec] +provider = "hmac" + +[ec.providers.hmac] +passphrase = "replace-with-32-plus-byte-random-secret" + +[device] +provider = "builtin" # default. "fastly" opts into TLS/H2 signal evidence + +[geo] +provider = "platform" # default is none (no location, no host call) +default_country = "FR" # required (section 6) +# assume_single_jurisdiction = true # required when EC runs with no geo +``` + +**The `host-signals` EC provider** (identity from HMAC over the host TLS JA4 +and HTTP/2 fingerprints plus the client IP) was deliberately dropped from the +2026-07-31 draft. It has since shipped in PR #1044 as an opt-in built-in +(`[ec.providers.host-signals]`), implemented against the host-agnostic +`HostSignals` capability rather than a Fastly API, so any host that supplies +the fingerprints can run it and a host that supplies none cannot build it. +When the host supplies no fingerprint at all the provider defers with a +warning instead of degrading to an IP-only identifier under the host-signals +name. **An open review question stands on whether this provider should ship +in the series at all**, because its identifier shape shares the built-in +HMAC grammar and a sign-off row defers host fingerprint processing. The +question is flagged for the series review and this spec does not present +the provider as settled either way. + +## 3. The identity lifecycle contract + +This is the section PR #838 lacked. Its trait abstracted **minting** an +identifier but left **recognition** and **KV key normalization** hard-coded +to the built-in HMAC shape, so a provider whose identifiers did not match +`{64hex}.{6alnum}` minted cookies that the very next request discarded. + +The implemented contract routes every lifecycle operation core performs on +an EC value through the selected provider: + +| Lifecycle operation | Where core uses it | Contract | +| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Mint** | EC generation on first eligible request, and the client-cycle resolve endpoint | The provider returns the identifier (`generate` server-side, `resolve_from_client` for the client cycle) and only core writes the cookie, after enforcing the global bounds below. | +| **Recognize** | Reading `ts-ec` back from the request, deciding `ec_was_present`, withdrawal checks, and every path that hands the value onward: the origin URL in `append_ec_id`, the click-target URL in `handle_first_party_click`, and the proxied body an integration builds | `accepts_id` answers whether a value is a well-formed identifier the provider issues. A value the selected provider does not recognize is treated as absent, so it is never used or egressed, while the raw cookie value stays visible to withdrawal handling. The egress paths reach the same answer through `edge_cookie::recognized_ec_id`, and a deployment with no provider selected recognizes nothing and so egresses nothing. | +| **KV key** | Identity-graph row reads and writes | `normalize_id_for_kv` returns the key form. The default lowercases the built-in HMAC hash segment and preserves the suffix, keeping today's keys. An opaque or case-sensitive provider overrides to the identity function so distinct identifiers never collapse into one row. | +| **Withdraw** | Expiring the cookie and writing revocation markers | The identifiers eligible for a **graph tombstone** are exactly those the selected provider owns, dispatched on the `{code}~` prefix first and then `accepts_id`, never a shape check the provider cannot influence. Expiring the **cookie** is broader: it keys off the raw cookie being present, so it still fires for an identifier the selected provider does not own (see the switching case, §6.1). | + +**Invariant:** for every provider `P` and every identifier `id` minted by +`P`, `id` round-trips read-back byte for byte. A test in `ec/mod.rs` proves +the round-trip with a non-default provider whose identifiers are opaque, and +a second test in `ec/resolve.rs` proves the client-cycle value survives the +full scenario verbatim. + +The draft's richer lifecycle surface, a canonicalizing `parse` with +per-provider equivalence fixtures, a core-constructed graph key built from a +provider `graph_key_suffix`, a declared cluster-prefix capability, declared +namespace descriptors with a startup disjointness proof, and a reusable +conformance suite driven by fixtures, is **not implemented in these PRs**. +Recognition plus KV normalization proved sufficient for the operations core +actually performs today, and each deferred piece is tracked as follow-up +work rather than silently dropped (see the revision record). Until the key +grammar lands, the KV key is the provider's normalized identifier verbatim, +which keeps every pre-epic HMAC row reachable. + +The pre-epic IP-cluster prefix listing runs unchanged, but the key space it +lists over does not. A fresh mint is keyed `hmac~.`, so the +prefix `evaluate_cluster` derives is `hmac~` for a coded row while a +legacy bare row still lists under `` on its own. Prefix matching is +anchored at the start of the key, so two rows for the same client IP that +straddle the envelope never count each other, and `cluster_size` under-reports +for as long as both populations coexist. + +That undercount is accepted rather than bridged, for three reasons. +`cluster_size` is reported in the identify response and gates nothing: the +only place its value is read at all is a cache short circuit in +`evaluate_cluster` that tests whether a value is stored, not what it is, and +the `cluster_trust_threshold` and `cluster_recheck_secs` settings that a +reader might expect to gate on it have no readers in the code. The undercount +is bounded by the legacy bare-identifier read window in section 6.1 and ends +when the last pre-epic cookie expires. And bridging it would mean a second +prefix scan on every identify request for the whole of that window. It becomes +a real fault only if a later change makes the count gate something, and that +is the change that has to build the bridge. + +One global rule sits above every provider, and it is implemented: + +- **Identifier bounds.** A minted identifier obeys a global cookie-safe + alphabet (normatively `[A-Za-z0-9._~-]`, valid cookie octets with no + separators, whitespace, or control characters) and a global maximum of + **256 bytes**, stated here so dependent documents reference one number. + The bound applies to the identifier itself, not only its key form. Core + enforces the bound wherever an identifier enters the system, at mint + (both `generate` and the resolve endpoint), at cookie read-back, and at + cookie write. The constant is `MAX_EC_ID_LEN` in `ec/cookies.rs`. A + violating value is rejected outright and logged. No sanitizing rewrite + exists anywhere on the path, so an identifier survives byte for byte or + not at all, and the cookie value and the identity-graph key can never + silently diverge. + +## 4. Trait surface: minimalism rule + +Every trait method must have at least one production (non-test) caller in +the same change that introduces it. How the surface observed in PR #838 +resolved in the implementation: + +- `keys_equal`: **not shipped.** Its legitimate purpose (equivalent-envelope + comparison, #778) is served structurally, because read-back acceptance and + KV normalization both route through the provider, so no comparison method + exists to leave uncalled. +- `GeneratedEdgeCookie::response_headers`: **shipped, with a production + caller.** EC finalization applies provider-requested headers to the + outbound response, and the client-cycle resolve path returns them, which + is how a client-side provider requests further evidence from the page. + The draft banned the field when nothing consumed it. The consumer landed + in the same series, satisfying the rule the ban enforced. +- `IdentityInput.permissions` / `IdentityInput.consent`: **shipped and + populated.** The organic mint path passes the request's resolved + permission state and consent context so a provider can read them for + behavior beyond gating. The gate itself has already run before `generate` + is called, so a provider cannot use the fields to authorize itself. +- `required_permissions` on `DeviceProvider` and `PlatformGeo`: **present, + with an empty default and no enforcement point.** The draft removed the + method from both traits because PR #838's copies were decorative. The + implementation keeps one uniform declaration seam across all three traits + instead. The built-in device and geo providers declare empty sets, and + core enforces the declaration only for the EC provider (section 5), so + nothing reads as a gate that is not one. The geo circularity argument + stands unchanged and is restated in section 5. + +The implemented `EdgeCookieProvider` surface (`ec/provider.rs`): + +```rust +pub trait EdgeCookieProvider: Send + Sync + core::fmt::Debug { + /// Stable configuration key ("hmac"). + fn id(&self) -> &'static str; + /// Registered four-character code (provider-code-registry.md), the + /// `{code}~` namespace of every identifier the provider mints. + /// Mandatory, no default: a provider cannot exist without a unique + /// code, so identifiers from different providers can never collide. + fn code(&self) -> ProviderCode; + /// Derives an identifier from the provider's injected services and the + /// request evidence passed at call time. A client-side provider defers + /// here (returns no id) and mints later in resolve_from_client. + fn generate( + &self, + request_info: &dyn RequestInfo, + input: &IdentityInput<'_>, + ) -> Result>; + /// Whether `value` is a well-formed identifier this provider issues. + /// Default: the built-in HMAC shape (`<64 hex>.<6 alphanumeric>`). + fn accepts_id(&self, value: &str) -> bool { /* built-in shape */ } + /// The KV-key form of `value`. Default: lowercase the HMAC hash + /// segment, preserve the suffix. Opaque providers return the value + /// unchanged. + fn normalize_id_for_kv(&self, value: &str) -> String { /* ... */ } + /// Permissions this provider's data use requires. Default: none, so a + /// vendor-neutral provider requires no permission. + fn required_permissions(&self) -> PermissionSet { /* none */ } + /// Client-cycle counterpart to generate: mints from a value the page + /// posted to the resolve endpoint, after verifying it. Default: no-op, + /// so a server-side provider does not participate. See the + /// client-cycle spec. + fn resolve_from_client( + &self, + input: &ClientResolveInput<'_>, + ) -> Result> { /* ... */ } +} +``` + +Core owns the code envelope. At mint it prefixes the provider's value with +`{code}~`, at read-back it strips and checks the code before the provider's +`accepts_id` sees the value part, and the identity graph key preserves the +code verbatim around the provider's canonical form. A cookie carrying +another provider's code is treated as absent, never adopted, so switching +providers cannot silently mix identity populations, and a withdrawal always +acts on a key that can only belong to one provider. The built-in HMAC +provider mints `hmac~<64 hex>.<6 alphanumeric>` and dual-reads its +pre-envelope bare form for one release cycle so deployed cookies keep +working; the bare form belongs to hmac alone. Codes are allocated +append-only in `provider-code-registry.md`, and a leading digit is valid +(`51dd`). + +The draft's alternative shape (`parse` returning a typed `EcId`, +`graph_key_suffix`, `cluster_prefix`, `verify`, and a version-carrying +`GeneratedIdentity`) was not adopted. `verify`, provider versions, and +`mint_version` are tracked follow-up work with the migration spec. +Request data reaches a provider through injected services and the +`RequestInfo` passed at call time, not through a fixed parameter struct. +`RequestInfo` carries the evidence a provider in this workspace reads today, +which is the normalized client IP. Further evidence (headers, cookies, client +hints, the URL) is added to it as a defaulted accessor in the change that first +reads it, so an existing implementation keeps compiling and no accessor lands +ahead of the caller that consumes it. + +## 5. Permission enforcement is core's job, for EC providers + +Before minting through an EC provider, core resolves the request's +permission state and refuses when the provider's `required_permissions()` +are not all set. The gate is implemented in `EcContext`. The selected +provider is built once at request read time, its declaration is checked +against the resolved state, and generation is skipped (with a log line +naming the jurisdiction) when the requirement is not met. With no provider +selected, nothing may mint or use an identifier, so the gate is closed +rather than open by default. The enforcement point lands with the +permission model in PR #1045, and the permission model spec governs the +resolution machinery (country and region baselines, signals, and the +requires-signal floor). + +**Recognition and withdrawal always run**, permissions or not. Read-back +acceptance and withdrawal eligibility go through `accepts_id` with no +permission check, and withdrawal handling keeps the raw cookie value even +when the identifier is treated as absent, so an opt-out can always reach +the identity it revokes. A blanket execution gate would refuse to run the +provider in exactly the state an opt-out produces. + +The draft additionally specified an identity activation protocol (a +two-record commit point before any egress), rowless-cookie classification +and per-prefix withdrawal records, negative-record admission rules, and a +typed egress boundary (`AuthorizedIdentity`, +`RedactedRequestView`). **None of that is implemented in these PRs.** +Those positions remain recorded in the draft and are tracked as follow-up +work with the permission model spec, which owns identity-state persistence +and egress typing. The revision record lists them as deferred. + +The gate applies to EC providers **only**. Geo and device are ungated for +two different reasons, stated separately because only one of them is +structural: + +- **Geo: circularity.** The permission set is resolved from jurisdiction, + which is resolved by the geo provider. Gating geo on the resolved set is + unsatisfiable. `PlatformGeo::required_permissions` exists with an empty + default for interface uniformity, and nothing consults it on the lookup + path. +- **Device: host evidence is an explicit opt-in, not authorized by + selection defaults.** Device classification is not an input to permission + resolution. The neutral `builtin` classifier reads only the User-Agent + and makes no host call. The draft went further and made selecting a + fingerprint-reading device provider a startup error pending a separate + security design. The implementation instead ships `[device] provider = +"fastly"` as a selectable opt-in. The Fastly adapter injects a + `HostSignals` service carrying the TLS JA4 and HTTP/2 fingerprints, and + the provider uses them to strengthen the browser/bot gate that guards EC + writes. Identity rows persist the derived classification fields (the JA4 + class segment and a 12-hex-character hash prefix of the HTTP/2 SETTINGS + fingerprint), not raw fingerprints, and the neutral default persists + neither because the builtin provider produces no such fields. + +## 6. Selection, validation, and failure modes + +All configuration validation happens at **settings construction** +(`Settings::finalize_deserialized` runs every check below), so a +misconfiguration expressible in configuration alone is a startup error, +never a silent behavior change. A selection that only the running host can +satisfy (an injected vendor provider, or host fingerprints) fails loudly +when the provider is built, stopping the request rather than degrading. + +| Configuration state | Behavior | +| ------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `[ec] provider` set, its `[ec.providers.]` block missing | Startup error naming the missing block. There is no closed key list in core for EC, because a vendor key is legitimate when its block is present, so an unknown key with no block fails this same check. | +| `[ec.providers.]` block present, `provider` unset | **Startup error.** (In PR #838 this silently ran stateless. The half-migrated config becomes a production identity outage detected by revenue drop. Rejecting it is the fix.) An operator who genuinely wants stateless deletes the block. | +| `provider = "none"` (explicit stateless) | Valid, and means exactly what omitting the selector means. Any configured provider block alongside it is a startup error, the same stray-block rule as below. | +| A configured `[ec.providers.]` block that is not the selected one | **Startup error** (checked for the `hmac` block and every vendor block). An unreferenced block is almost always a mistyped selector or a stale block, and accepting it silently invites configuration drift. | +| A selected vendor key whose provider the adapter did not inject | Loud failure when the provider is built, naming the key, so the deployment never silently runs stateless. | +| `provider = "host-signals"` on a host that supplies no fingerprints | Loud failure when the provider is built. A host that cannot produce `HostSignals` cannot run the provider. | +| `provider = "client-fixed"` in a production build | Startup error. The demonstration provider is compiled only behind the `client-fixed-demo` cargo feature. | +| No `provider`, no providers block | Valid, the neutral default for that concern. | +| Deprecated `[ec] passphrase` | Migrated to `provider = "hmac"` with the passphrase in `[ec.providers.hmac]`, with a deprecation warning naming the new location. Both forms together are rejected so a half-edited file fails loudly instead of one form silently winning. | +| Any unknown key in `[ec]`, `[device]`, `[geo]`, or a built-in provider block | Startup error. `deny_unknown_fields` is on `Ec`, `DeviceConfig`, `GeoConfig`, and both built-in provider config structs, so a typo like `providr`, or a key from a deferred feature (`legacy_providers`, `rewrite_legacy`, `versions`), fails loudly. | +| `[device] provider` names an unknown key | Startup error. Valid keys are `builtin` (default) and `fastly`. | +| `[geo] provider` names an unknown key | Startup error. Valid states are unset (default, no geolocation), `none` (the same, spelled out), and `platform`. | +| `[geo] default_country` unset, or matching no `permissions.yaml` rule | **Startup error.** The value is the permission baseline for a request the geo provider leaves unmatched, so there must always be one and the value must resolve to a real rule. | +| An EC provider configured, no geo provider, `assume_single_jurisdiction` unset | **Startup error.** With geolocation off, every request resolves as `default_country`, so a visitor from any other jurisdiction silently receives the default jurisdiction's rules. That is acceptable only as an explicit operator decision. | + +One draft row was not adopted, the startup error for a minting provider +with no identity-graph store. `[ec] ec_store` remains optional, because the +portability adapters run without platform KV. The client-cycle resolve +endpoint refuses to mint when no graph is available (a cookie without a row +could never be withdrawn through the graph), and the organic path persists +the row whenever the graph is configured. Whether configuration should +force the pairing is follow-up work with the migration spec. + +Vendor provider blocks deserve their own note. Any `[ec.providers.]` +block whose key is not a built-in is captured in core as raw values (a +flattened map), and the adapter that injects the vendor provider +deserializes its own block into the vendor crate's config type. Core never +names a vendor, so a new provider adds nothing to core. The vendor crate +applies its own `deny_unknown_fields` when it deserializes. + +### 6.1 Provider switching: what a switch actually does + +Switching `[ec] provider` **retires every identity the previous provider +minted**. This section says exactly what that means, because a deployer has +to plan around it rather than discover it. + +The draft specified an ordered `legacy_providers` reader list, provider +`versions` with `mint_version` rotation, provenance tagging, and retirement +evidence rules, as the mechanism that would carry identities across a +switch. **None of that is implemented in these PRs.** The keys are rejected +as unknown, and the design is tracked follow-up work with the migration +spec. Until it lands there is no continuity across a switch of any kind. + +An earlier version of this section claimed shape-based continuity, that old +cookies stay recognized when the newly selected provider accepts their +shape. That is not what the code does and never was once core took +ownership of the `{code}~` envelope (§5). Ownership is decided on the code +prefix **before** any provider is asked about the shape, so a newly selected +provider rejects every identifier the previous one minted, whatever its +shape, because the code differs. + +What a switch does, precisely: + +- **Read-back.** Every identifier carrying the retired provider's code is + treated as absent. It never becomes the request's active identity, never + egresses to a partner, and is rejected on the pull-sync, batch-sync and + admin paths too. This is the §5 guarantee and it is the half of the + behavior that matters most: two providers' identity populations can never + mix. +- **The browser cookie.** A later withdrawal still expires the `ts-ec` + cookie, because that path keys off the raw cookie being present rather + than off who owns it. The browser stops carrying the retired identifier. +- **The identity-graph rows.** A later withdrawal does **not** tombstone the + retired provider's rows. Core cannot derive their canonical keys, because + the canonical form is the owning provider's own normalization and the + owning provider is no longer configured. Those rows stay as they are until + their one-year entry TTL expires. + +**What a deployer must do about revocation.** Treat a provider switch as a +one-way retirement of the identity population, and deal with the previous +provider's rows before or alongside the switch, not after. A withdrawal +that arrives after the switch clears the browser but leaves the row. Every +row a retired provider wrote shares that provider's `{code}~` key prefix, so +the set is identifiable and can be listed and cleared with the platform's +own KV tooling. Either clear it at the switch, or accept that the rows +persist until the one-year TTL expires and that withdrawals arriving in the +meantime are recorded only in the browser. Do not switch providers while +identities are live that the deployment may still be obliged to revoke in +the identity graph. + +The `cluster_fallback` degradation policy from the draft is likewise +deferred with the cluster capability itself. + +### 6.2 Runtime failure modes + +Startup validation covers configuration. This covers a healthy +configuration meeting an unhealthy runtime. Implemented behavior, each row +logged, none silent: + +| Failure | Behavior | +| -------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `generate` returns an error | No identity this request. The organic caller logs at error level and the request proceeds stateless. No cookie is written. | +| A provider mints an identifier outside the global bounds | Rejected at mint, never rewritten. The organic path yields no identity. The resolve endpoint returns 400. | +| Identity-graph write fails at mint | The mint is undone (no identifier, no cookie), with the error logged. The resolve endpoint returns 503. The next eligible request retries. | +| The host-signals provider finds no TLS/HTTP-2 fingerprints | Defers with a warning. No identity this request, and no degraded IP-only identifier is minted under the host-signals name. | +| Geo lookup **fails** (the provider errors) | Every permission resolves to the requires-signal floor, and the failure is logged at error level. The failure is **not** papered over with the `default_country` baseline. | +| Geo resolves **no location**, or a country/region with no rule | The `[geo] default_country` baseline applies. This is the configured-default case, deliberately distinct from the failure row above (`GeoStatus` in `ec/consent.rs`). | +| An incoming cookie value fails the bounds at read-back | Treated as absent, with a warning naming the source. | + +The distinction between a failed lookup and no location is resolved in +core, where `EcContext::read_from_request_resolving_geo` runs the +configured geo provider itself and classifies the outcome, so every +adapter reports the two states identically. The draft's remaining matrix rows (rowless +withdrawal records, promotion, the negative-intent outbox, the identity +safety breaker, cluster-listing degradation) belong to the deferred +material of sections 5 and 6.3. + +### 6.3 Storage contract + +The draft specified a delimiter-free physical key grammar with fixed-width +segments, a provider-code registry, record classes for family revocation, +authority state, negative-intent outbox, rowless withdrawal, and deployment +metadata, wire schemas with known-answer vectors, and a per-field graph-row +contract. The provider-code registry is now implemented: codes are +allocated in `provider-code-registry.md`, carried as the `{code}~` prefix +of every minted identifier, and therefore present in every graph key. The +key grammar differs from the draft in one deliberate way, a tilde separator +instead of delimiter-free fixed width, because pre-envelope bare +identifiers remain deployed and a code such as `51dd` is valid hex, so +delimiter-free parsing could misread a legacy identifier during the +migration window. The remainder (record classes, family revocation, +authority state, outbox, rowless withdrawal, wire schemas, per-field +contract) is not implemented in these PRs and stands as recorded design for +the follow-ups. + +The implemented storage today keys the identity graph by the selected +provider's `normalize_id_for_kv` output verbatim. For the built-in HMAC +provider that is the identifier with the hash segment lowercased, which is +today's key, so every pre-epic row stays reachable and the pre-epic +cluster prefix listing stays intact. For an opaque provider the identifier +itself is the key. Rows carry the same JSON envelope as before the epic, +extended with the derived device-classification fields noted in section 5. + +## 7. Composition root and adapter parity + +Provider construction happens in one place per concern, in core, called by +every adapter. No adapter wires a concrete implementation directly into the +request path: + +- `build_provider` (`ec/provider.rs`) constructs the selected EC provider, + injecting the host's `HostSignals` when supplied and matching an + adapter-injected vendor provider by its `id()`. The provider is built + once per request during `EcContext` construction and reused for + read-back, the permission gate, and minting, so the per-request + triple-build observed in PR #838 (cloning the secret into a fresh box up + to three times per request) is gone. +- `build_device_provider` (`ec/device.rs`) returns the builtin classifier + unless `fastly` is selected, in which case the adapter's closure builds + the host-evidence provider. +- `build_geo_provider` (`platform/mod.rs`) returns `DisabledGeo` unless + `platform` is selected, in which case the adapter's host geo + implementation is used. All four adapters (Fastly, Axum, Cloudflare, + Spin) route their host geo through this selector when they assemble + their runtime services, verified in each adapter's platform wiring. + +All four adapters construct the EC request state through the same core +constructors (`EcContext::read_from_request_resolving_geo` and its +variants), so selector behavior and the geo failure classification are +identical everywhere. The cross-adapter parity suite +(`trusted-server-integration-tests`) asserts geo response parity across +adapters. The EC API routes are Fastly-only today, as section 1 notes, and +the Spin adapter's route list records why. + +The draft's adapter capability matrix (declared per-record-class +consistency semantics, durability and retention proofs, activation and +lease qualification) is **not implemented in these PRs** and is tracked +follow-up work. The matrix's motivating rule is preserved for that +follow-up, which is that "has KV" says nothing about whether a revocation +is observable, so eligibility for identity features must eventually be +declared and checked, not assumed. + +## 8. Crate layout and CI + +Host and vendor provider crates live in nested directories grouped by +capability, with flat package names following the existing convention: + +- `crates/device/fastly` is package `trusted-server-device-fastly`. +- `crates/geo/fastly` is package `trusted-server-geo-fastly`. +- `crates/edgecookie/` is the documented home for vendor EC + crates. The directory currently holds only a README, because the + built-in providers live in core and no vendor crate exists yet. + +The draft mandated flat directories (`crates/trusted-server-geo-fastly`) +and banned placeholder directories. The implementation diverges on both +points. Nested directories scale per vendor as providers multiply, package +names already carry the flat convention, and the README stakes out the +location before the first vendor crate lands. Both divergences are +recorded in the revision table. + +Every new crate is in the `.cargo/config.toml` aliases (`check-fastly`, +`clippy-fastly`, `test-fastly`, `build-fastly`), so the provider crates are +linted with `-D warnings` and tested by the same gates as every other +workspace member, closing the PR #838 gap where new crates compiled only +transitively. + +## 9. Behavior preservation notes + +Two defaults chosen for neutrality change effective behavior on existing +Fastly deployments. Both are called out in the migration spec and must be +prominent in release notes: + +- **Bot gate.** The pre-provider EC bot gate required JA4 and platform + class. The default `builtin` classifier is User-Agent only, so the gate + is weaker by default. The stronger gate is available as `[device] +provider = "fastly"` rather than being startup-rejected as the draft + specified. Release notes call out the weaker default rather than + presenting selection alone as authorization. +- **Geo.** With no geo provider, jurisdiction resolution falls to the + required `[geo] default_country`. The permission model constrains the + combination so it cannot silently grant permissions to mis-attributed + traffic. The default must resolve to a real `permissions.yaml` rule, a + deployment running an EC provider without geo must set + `assume_single_jurisdiction = true`, and a failed lookup resolves to the + requires-signal floor instead of the default. The default flip landed in + the same series as those constraints, honoring the draft's sequencing + requirement that the constraint exist before the flip. + +## 10. Testing strategy + +Implemented, in the crates named: + +- Round-trip tests with a non-default provider, proving an opaque + identifier survives read-back byte for byte (`ec/mod.rs`) and the + client-cycle value survives the full scenario as cookie and KV key + (`ec/resolve.rs`). +- Delegation tests proving the injected-provider wrapper forwards + `accepts_id` and `normalize_id_for_kv` to the inner provider, so a + vendor identifier is never dropped by the built-in defaults. +- Gate tests proving the HMAC provider's declared requirement blocks + generation until the permission is set, and that a provider declaring + nothing requires nothing. +- Settings validation tests covering the section 6 table, including the + missing block, the block without a selector, explicit `none`, the stray + block, unknown selector keys for all three concerns, unknown fields in + every section, the deprecated passphrase migration with its both-forms + rejection, `default_country` validation, and the jurisdiction + acknowledgment. +- Geo builder tests showing the default selects no geo, `none` selects no + geo explicitly, and `platform` selects the host implementation. +- Host-signals provider tests covering minting from fingerprints, + deferring without them, and the loud failure of a selected but + uninjected vendor provider. + +Deferred with their features are the fixture-driven provider conformance +suite, legacy-reader tests, and the parity cases for capability-mismatch +startup failures. + +## 11. Implementation order + +As landed: + +1. **PR #1043, the Edge Cookie provider seam.** The trait with recognition + and KV normalization, the global identifier bounds, selection and + validation, the vendor block capture, the deprecated-passphrase + migration, and the round-trip proof with a non-default provider. +2. **PR #1044, device and geo selection.** `DeviceProvider` with the + builtin default and the opt-in Fastly host-evidence provider, + `PlatformGeo` selection with the no-geo default, all four adapters + routed through the shared builders, and the opt-in host-signals EC + provider (carrying the open review question of section 2). +3. **PR #1045, the permission model.** The enforcement point for + `required_permissions`, the `default_country` requirement and + jurisdiction acknowledgment, and the failed-lookup floor. That change + has its own spec, which this document cross-references rather than + restates. + +The draft's step 4 warning (do not flip the geo neutral default before the +permission model exists) was honored. The flip and its constraints landed +together in the permission model change. + +## 12. Divergences from issue #778 + +This spec supersedes #778 on the following points, so implementation has +one acceptance contract: + +| #778 says | This spec says | Why | +| ------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Identifier comparison is a provider operation (`keys_equal`) | Comparison is structural. Read-back acceptance and KV normalization route through the provider, so no comparison method exists (§3, §4) | Satisfies the same requirement with no method to leave uncalled | +| A provider can return response headers | Kept, with a production consumer. EC finalization applies them, and the client-cycle path uses them (§4) | The caller the minimalism rule demands landed in the same series | +| One built-in provider (HMAC) preserving today's behavior | HMAC preserved verbatim, plus the opt-in host-signals built-in (open question, §2) and the demo client-cycle provider | A switch retires the previous provider's identity population outright (§6.1); carrying identities across a switch (`legacy_providers`) remains follow-up work with the migration spec | + +## 13. Revision record vs the 2026-07-31 draft + +One row per divergence between the 2026-07-31 draft and the implementation +this revision describes. + +| Draft position | Implemented position | Why | +| ----------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Trait surface is a canonicalizing `parse` returning a typed id, plus `graph_key_suffix`, `cluster_prefix`, and `verify` | `accepts_id` (recognition) plus `normalize_id_for_kv` (KV key form), defaults matching the built-in shape. No typed id, key suffix, cluster capability, or `verify`. `keys_equal` stays out, as the draft required. | Recognition and KV keying are the two operations core performs today. A byte-for-byte round-trip test with a non-default provider pins the contract. | +| `GeneratedEdgeCookie::response_headers` and `IdentityInput.permissions` / `.consent` banned as speculative surface | Shipped with production consumers. Finalization applies provider headers, the resolve path returns them, and the organic mint path populates the input fields. | The client-cycle resolve path landed in the same series and is their caller, satisfying the minimalism rule the ban enforced. | +| Identifier bounds enforced at mint and parse | Enforced at mint (`generate` and the resolve endpoint), cookie read-back, and cookie write. Violations rejected outright, never rewritten. `MAX_EC_ID_LEN` in `ec/cookies.rs`. | Every identifier entry point is covered, and the pre-epic sanitizing rewrite was removed as a silent-divergence hazard. | +| `provider = "none"` is valid alongside `legacy_providers` blocks | `none` (or an omitted selector) with any configured provider block is a startup error. | No `legacy_providers` exists in these PRs, so a block alongside statelessness can only be a mistake. | +| Every selection key is closed and unknown keys are startup errors | Device and geo keys are closed. EC vendor keys are open. Unknown blocks are captured as raw values in core, the adapter deserializes its own block, and a selected key with no injected provider fails loudly. | Core never names a vendor, so a vendor provider adds no core change. | +| Capability mismatch is a startup error at adapter wiring time | Configuration coherence fails at startup. A host-capability mismatch (missing `HostSignals`, uninjected vendor) fails loudly when the provider is built, stopping the request. | The adapter capability declaration that would move the check to startup is deferred with the capability matrix. | +| A minting provider with no identity-graph store is a startup error | Not implemented. `ec_store` stays optional. The resolve endpoint refuses to mint without a graph. The organic path persists rows whenever the graph is configured. | Portability adapters run without platform KV. Whether configuration should force the pairing is follow-up work. | +| `[device] provider = "fastly"` is startup-rejected pending a separate security design | Shipped as a selectable opt-in. The Fastly adapter injects `HostSignals`, the provider strengthens the browser/bot gate, and rows persist derived classes, not raw fingerprints. | Selection is an explicit operator opt-in and the neutral default makes no host fingerprint call. | +| The `host-signals` EC provider is deliberately dropped and its selection rejected | Shipped in PR #1044 as an opt-in built-in that defers with a warning when the host supplies no signals. **Open, flagged for the series review**, not settled either way. | Its identifier shape shares the HMAC grammar, and a sign-off row defers host fingerprint processing, so the review decides whether the provider ships in the series. | +| Geo default flip sequenced into the later permission-model step, with an acknowledgment guard | Landed as specified in the same series, with the default of none, `default_country` required and validated against `permissions.yaml`, the `assume_single_jurisdiction` acknowledgment, and a failed lookup resolving to the requires-signal floor with error logging (`GeoStatus`, resolved in core so all adapters agree). | The permission model shipped in PR #1045, so the constraints exist where the draft required them. | +| All adapters serve the full EC feature set identically | Selector behavior is identical through the shared builders and core constructors. The EC API routes (identify, batch-sync, ec/resolve) are Fastly-only, documented in the Spin route list. | The portability adapters do not yet wire platform KV, and the gap is documented rather than silent. | +| Conformance suite, adapter capability matrix, delimiter-free key grammar, `verify`, `legacy_providers`, `versions` / `mint_version` | None of these are in PR #1043 or #1044. All are tracked follow-up work, deferred, not silently dropped. | The shipped seam did not need them, and each returns with the feature that gives it a production caller, per the spec's own minimalism rule. | +| `required_permissions` removed from the device and geo traits, added to the EC trait only at the permission-model step | Present on all three traits from the start, with empty defaults. Core enforces the EC declaration (gate in `EcContext`, landing in PR #1045). No device or geo enforcement point exists. | One uniform declaration seam, with an empty default that gates nothing, avoids the decorative-gate hazard while keeping the interface stable. The geo circularity stands. | +| Flat crate directories (`crates/trusted-server-geo-fastly`), no placeholder directories | Nested directories per capability (`crates/device/fastly`, `crates/geo/fastly`, `crates/edgecookie/`), flat package names. `crates/edgecookie` ships a README before its first crate. | Nested directories scale per vendor, package names already carry the naming convention, and the README stakes out the vendor location. | diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md new file mode 100644 index 000000000..efe904a96 --- /dev/null +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -0,0 +1,516 @@ +# Design Spec: Provider and Permission Model, Migration and Rollout + +**Status:** Revised against the implemented series (PRs #1043-#1047), +2026-08-25. The §8 sign-off rows remain the series' decision ledger; rows the +implementation now satisfies are marked with their PR so the task force can +ratify rather than re-litigate. +**Author:** Engineering +**Issue references:** #777-#781 (epic) +**Related specs:** `2026-07-30-pluggable-providers-design.md`, +`2026-07-30-permission-model-design.md`, +`2026-07-30-integration-response-header-hook-design.md` (not implemented in +the series, see its status) +**Last updated:** 2026-08-25 + +> **Context.** The provider/permission epic is a breaking change to a live +> identity system. PR #838's review showed that the riskiest part of such a +> change is not the new code but the transition, meaning silent +> misconfiguration modes, undeclared behavior changes discovered by deleted +> tests, and no written statement of which pre-change behaviors were +> guaranteed to survive. This spec is that statement. The epic has now been +> implemented as five stacked PRs (#1043-#1047, the "series" below), and this +> revision reconciles the spec against that series, keeping §2's matrix and +> §8's ledger as the record the task force ratifies. Any further +> implementation PR must reconcile its diff against §2's matrix and list +> every deliberate divergence in its description. + +## The implemented series + +Five stacked PRs, verified against the tree at +`split/5-response-hook-docs` (the head of PR #1047; the branch is rebuilt on +each rebase, so the PR is the stable reference). A sixth PR, #1084, adds the +integration provider seam spec on top of the series and changes no code: + +1. **PR #1043, the Edge Cookie provider seam.** The `EdgeCookieProvider` + trait (`id`, `generate`, `accepts_id`, `normalize_id_for_kv`, + `required_permissions`, `resolve_from_client` in + `crates/trusted-server-core/src/ec/provider.rs`) routes identifier + creation, cookie read-back, and identity-graph keying through the + selected provider. Global identifier bounds are enforced by core at every + entry point (`MAX_EC_ID_LEN = 256` bytes and the cookie-safe alphabet + `[A-Za-z0-9._~-]` in `ec/cookies.rs`), rejecting loudly and never + rewriting, so the cookie value and the graph key cannot silently + diverge. `[ec] provider = "none"` spells explicit statelessness. A + configured `[ec.providers.*]` block with no selector, an unreferenced + block, and a selector with no block are each startup errors. The + deprecated `[ec] passphrase` form still starts for one release cycle, + mapping to `provider = "hmac"` with a deprecation warning, and a + configuration carrying both forms is rejected. +2. **PR #1044, device and geo selection.** `[device] provider` selects + `builtin` (User-Agent only, the default) or the opt-in `fastly` + classifier (`crates/device/fastly`, TLS JA4 and HTTP/2 signals). + `[geo] provider` defaults to no geolocation, with `"platform"` opting in + to the host lookup (`crates/geo/fastly`) and `"none"` spelling the + opt-out. All four adapters route geo through the one + `build_geo_provider` selector. The provider configuration structs carry + `deny_unknown_fields`, so a mistyped key fails startup. The host-signal + Edge Cookie provider (`[ec.providers.host-signals]`) ships opt-in. + Whether the host-signal surface stays is an **open review question** + (sign-off row 22), not a settled decision. +3. **PR #1045, the permission model.** Permission names follow the IAB + Privacy Taxonomy Data Uses (`permissions.yaml`, compiled into the + build). Signal precedence is fixed in code, most restrictive first, + meaning an opt-out (Sec-GPC, GPP sale opt-out, US Privacy) suppresses + the Data Uses the policy revokes even against a consenting TCF record, + a present but undecodable record blocks baseline grants (fail-closed), + and only then does a TCF record decide its mapped Data Uses. + Destructive withdrawal is narrow, where only a TCF record refusing + storage in a jurisdiction whose baseline did not grant storage expires + the cookie and writes the identity-graph tombstone, and opt-outs never + destroy an issued identifier. Sharing beyond the edge (bidstream + `user.id`, the identify response, partner pull sync) requires storage + plus personalised-ad selection, the same pair that gates bidstream + EIDs. `[geo] default_country` becomes required, a failed geo lookup + resolves at the requires-signal floor instead of the default, and a + no-geo deployment running an Edge Cookie provider must set + `[geo] assume_single_jurisdiction = true`. +4. **PR #1046, the hardened client-cycle resolve endpoint.** + `POST /_ts/api/v1/ec/resolve` enforces publisher-origin `Origin` (403), + a `text/plain` or `application/json` content-type allowlist (415), a + body size cap (413), the global identifier bounds on the created + identifier (400), and refusal to replace a different identity already on + the request (409). The identity-graph row is written before the cookie + is set, with no graph meaning no identifier is created and a graph + write failure returning 503. Every response carries + `Cache-Control: no-store`. A non-HttpOnly `ts-ecr` marker cookie tells + the page script a resolve succeeded, fixing the re-post loop the + HttpOnly Edge Cookie would otherwise cause, and a Rust test pins the + marker name and the demo's fixed word against the page script source. + The client-fixed demonstration provider compiles only under the + `client-fixed-demo` cargo feature and production builds reject + selecting the demo at startup. +5. **PR #1047, the documentation set.** Configuration reference for + `[ec]`, `[device]`, and `[geo]`, the Edge Cookie guide rewritten around + providers and the permission model, and the example configuration + documenting every selector. The integration response-header hook was + **stripped from the series** because the hook had no consumer, which is + that spec's own rule for speculative surface. The hook spec is retained + as the design bar for when its first consumer arrives. + +--- + +## 1. Scope + +Covers the transition of existing deployments from the hard-wired EC / +device / geo behavior to the provider architecture and permission model. +Applies to every implementation PR in the epic (now the implemented series +PRs #1043-#1047 and any follow-up), and to the operator-facing migration +guide that ships with the last of them (still outstanding, §7). + +## 2. Behavior-preservation matrix + +For each decision the system makes today, the target behavior after the +epic, and whether that is a preservation or a declared change. **Silent +changes are defects.** PR #838 changed six of these without declaring any. +The Status column now also records where the implemented series stands, +naming the PR. Rows the series did not touch keep their design-target text +for the follow-up work that will implement them. + +| # | Decision (today) | After epic | Status | +| --- | -------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 1 | EU/GDPR request, no TCF consent → no EC | Same (opt-in baseline) | Preserved. Implemented (#1045), where `gdpr-eu` requires a signal for storage, with an EU-27 plus EEA coverage test | +| 2 | US-state request, sale opt-out → no EC, existing EC expired and tombstoned even beside a consenting TCF string | Opt-out suppresses use but does not delete the identity. Lifting the opt-out restores the identity. The positive contextual-auction projection (permission §7's `ContextualAuctionView`) remains the design for what may still dispatch | **Changed deliberately.** The separation is implemented (#1045), with opt-outs stripping egress and never tombstoning. The contextual projection is not implemented (sign-off 4 open), and today EIDs and `user.id` are withheld | +| 3 | US-state request, no signals at all → no EC (fail-closed) | The draft's recipe kept this via a `requires_signal` US rule with an extended grant-signal class | **Changed by the shipped default (#1045), flagged.** `permissions.yaml` ships US as a `granted` baseline whose uses drop on any opt-out, so a no-signal US request receives an EC. A deployer edits the yaml to differ | +| 3a | US-state request, explicit not-opted-out GPP/USP value → EC allowed | No US signal grants anything. N/A, absent, reserved, and unknown values grant nothing | Implemented (#1045) in a stricter form than drafted, where signals only revoke, so the drafted "explicit not-opted-out may grant" class does not exist (sign-offs 3, 17) | +| 3b | US-state request, TCF record refusing Purpose 1, no US opt-out → no EC | Refusal beats coexisting non-TCF grant signals | Implemented (#1045), where an authoritative TCF refusal revokes its mapped uses. Under the shipped `granted` US baseline the refusal suppresses without tombstoning | +| 3c | Consent-record conflict modes, expiry, KV fallback, proxy mode | Per the permission spec §4.4 matrix, whose changed row is that malformed-present blocks acquisition | Malformed-present fail-closed is implemented (#1045). The rest of the consent normalization pipeline is carried forward unchanged by the series | +| 3d | Valid plus expired consent records: conflict resolution can select the expired record | Expired sources drop before conflict resolution | Open. Not addressed by the series | +| 3e | Only the GPP sale field (and USP) is consulted, with sharing/targeted opt-outs ignored | Sale, sharing, and targeted-advertising opt-outs deny the personalised-ads uses, and none affects storage or destroys identity | Partially implemented (#1045), where sale, USP, and Sec-GPC are honored and never destructive. The sharing and targeted-advertising GPP fields are not yet decoded (sign-off 3 open) | +| 3f | Non-privacy-state US traffic (for example Wyoming) is non-regulated → EC allowed | Country-level `US` is a protective floor, region rules may be stricter, and regionless traffic never degrades to non-regulated | Implemented (#1045), where `permissions.yaml` maps country `US` to `us-opt-out`, with state rows able to override | +| 3g | Graph rows persist JA4 class, H2 fingerprint hash, and buyer-facing quality metadata | The draft discontinued these for new rows | **Contradicted by the series, flagged.** Rows still persist device signals, including fingerprint hash prefixes, when the opt-in providers run (#1044/#1046). Tied to the open host-signal question (sign-off 22) | +| 4 | UK request, no TCF record → no EC | Same, unless the policy deliberately adopts a `granted` storage baseline for GB with citation and sign-off | **The shipped default adopts `granted` storage for GB (#1045), flagged.** The citation and sign-off the draft required do not exist yet. The task force owns this row | +| 5 | No country resolvable (geo failure) → no EC (fail-closed) | Protective failure profile, where permissions resolve at the requires-signal floor and `default_country` is reserved for unmatched requests in acknowledged static-jurisdiction mode | Implemented (#1045), where a failed lookup resolves at the requires-signal floor, logged at error level, and never falls back to `default_country` (sign-off 18) | +| 6 | Non-regulated country, TCF record refusing Purpose 1 → EC still created, identity never tombstoned | Refusal blocks new grants everywhere, and existing identity is never tombstoned where the baseline is `granted` | Implemented (#1045), where an authoritative refusal revokes its mapped uses everywhere and withdrawal is scoped to non-granted baselines | +| 7 | Country resolved but in no regulation list → EC created, EIDs pass through | Governed by the deployment's default rule. The implementation expresses this as the required `[geo] default_country`, naming the `permissions.yaml` rule for unmatched requests | Implemented (#1045) with a changed mechanism, since no `rules.default` entry exists and `default_country` is required and validated at startup | +| 8 | Opt-out signal outside US states → ignored today | Mapped use restrictions are honored globally, and opt-outs never tombstone identity | Implemented (#1045), where the signal mapping is jurisdiction-free and suppresses even TCF-consented uses, without destruction (sign-off 1) | +| 9 | Fastly bot gate requires JA4 plus platform class before KV-backed EC writes | The draft deferred host fingerprinting and startup-failed `[device] provider = "fastly"` | **Contradicted by the series, flagged.** The `fastly` device provider ships opt-in with `builtin` (UA-only) as the default (#1044). Whether the host-signal surface stays is sign-off 22, open | +| 10 | Fastly always resolves geo per request | Only with `[geo] provider = "platform"`. The neutral default flips only together with the permission model's jurisdiction guard, never in an intermediate step | Implemented as sequenced (#1044 kept the platform default, and #1045 flipped geo off by default together with `default_country` and the acknowledgment guard) | +| 11a | Raw EC egress on jurisdiction-gated paths today (`user.id`, EIDs, identify, pull sync) | Gated by the sharing pair (storage plus personalised-ads), at least as strict as today for every path | Implemented (#1045), where `ec_sharing_allowed` gates `user.id`, the identify response, and pull sync, and `gate_eids_by_permissions` gates EIDs, all on the same pair | +| 11b | Proxy / click / Testlight forwarding extract the raw EC cookie/header without today's jurisdiction gate | Gated by the egress inventory (both purposes) | Open. Not implemented by the series (sign-off 8) | +| 11c | Batch sync only authenticates the S2S caller and checks row state | Gated by stored-provenance recompute, with legacy rows failing closed until backfilled | Open. Not implemented. Batch sync still checks caller and row state, with missing or withdrawn rows collapsing to ineligible (sign-offs 7, 13, 25) | +| 12 | EC generation succeeds without a configured identity-graph store | The draft required an openable graph store at startup for a creating provider | **Changed from the draft (#1043/#1046).** No startup requirement ships. The implemented rule is per request, where no available graph means no identifier is created (no phantom cookies), organic and resolve paths alike | +| 13 | Cookies created by graphless deployments have no graph row | Rowless proof under the graphless-migration flag, capped `w` records, disclosed cookie-only expiry | Open. None of the rowless machinery is implemented. A cookie without a row is simply never shared (sign-offs 21, 29, 30) | + +Rows 3, 4, and 7 are policy decisions, not code decisions. In the +implemented series they live in `permissions.yaml`, which is compiled into +the build precisely so the policy stays visible and reviewable in version +control, made explicitly by maintainers rather than implied by an +implementation. The shipped defaults for rows 3 and 4 are flagged above for +the task force. + +## 3. Identity stability guarantee + +Today's EC identifier is `{64-hex}.{6-char}` where the 64-hex part is +deterministic, `HMAC-SHA256(passphrase, normalized_ip)`, and the 6-char +suffix is random per creation (an existing test asserts two creations +differ). Full identifiers are therefore not reproducible by design, and no +test may pretend otherwise. What stability means, precisely, for a +deployment that selects `provider = "hmac"` and carries its passphrase over +verbatim, with where the series stands on each: + +- **The deterministic prefix is stable per inputs.** Implemented and + tested (#1043, + `generate_hmac_ec_id_is_stable_per_parts_and_collision_resistant`). + The committed known-answer vector the draft required (fixed passphrase + plus IP → exact expected 64-hex prefix, failing CI on divergence) is + **not yet committed** and remains open work. +- **Existing cookies stay parseable.** The `is_valid_ec_id` shape check is + unchanged and the HMAC provider's `accepts_id` delegates to that check, + so pre-series `ts-ec` values keep resolving and their graph rows (keyed + through `normalize_id_for_kv`, which lowercases the hash exactly as the + pre-series normalization did) remain reachable. +- **The hash prefix keeps its semantics.** `ec_hash` remains the 64-hex + prefix, preserving both its stability and its deliberate collision + across identifiers created from the same IP. +- **Cookie name, attributes, and max-age are unchanged.** `ts-ec`, + `Domain` derived from the publisher domain, `Path=/`, `Secure`, + `SameSite=Lax`, `HttpOnly`, one-year `Max-Age` (`ec/cookies.rs`). +- **New global bounds (#1043), a declared addition.** Every identifier, + whatever provider created the value, must fit 256 bytes and + `[A-Za-z0-9._~-]`. Out-of-bounds identifiers are rejected loudly and + never rewritten. + +## 4. Configuration migration + +Old shape: + +```toml +[ec] +passphrase = "replace-with-32-plus-byte-random-secret" +``` + +New shape: + +```toml +[ec] +provider = "hmac" + +[ec.providers.hmac] +passphrase = "replace-with-32-plus-byte-random-secret" +``` + +Requirements, each marked with its implementation state: + +1. **The transition has a dual-read release; loud rejection comes one + release later.** Implemented (#1043) in the simple form, where the + current release accepts the old shape, mapping `[ec] passphrase` to the + `hmac` provider internally and logging a deprecation warning per + startup, and accepts the new shape. A configuration mixing the two + forms is rejected, not reconciled. Ordering remains strictly + reader-first, because pre-series binaries reject the new `[ec]` keys + and the new `[device]` / `[geo]` sections as unknown fields, so a fleet + converges binaries first and flips configuration second. The follow-on + release that rejects `[ec] passphrase` outright with a message naming + the new location is scheduled, not yet in the tree. + + The draft's full N+1/N+2 interim (the negative-record read/write + matrix, the `permissions_v2` model promotion, the `m00` mirror, and the + rollback-floor machinery) is **deferred** together with the durable + suppression design that motivates all of that machinery (sign-off rows + 11, 16, 19, 20). The series needs none of that machinery because the + only durable negative artifact today is the identity-graph withdrawal + tombstone the pre-series lifecycle already wrote, so a binary rollback + strands no new state. The interim matrix stays recorded in row 20 as + the design bar for when durable suppression ships. + +2. **Adapter qualification is a pre-ratification prerequisite.** + **Deferred.** No machine-readable capability matrix exists. The + series' de facto posture is that the identity endpoints (resolve, + identify, batch sync) route only on the Fastly adapter, and the other + adapters run without them, which is the stateless-identity path of + requirement 3. PSL vendoring and the pinned GPP corpus (draft + requirement text) remain prerequisites of the response-hook and GPP + follow-ups respectively (sign-offs 23, 28, 32). +3. **Revocation-eligible storage is a per-adapter gate, and ungated + adapters migrate stateless.** Direction implemented, since + `provider = "none"` spells explicit statelessness (#1043) and + non-Fastly adapters deliberately do not route the identity endpoints + (#1046), matching identify and batch sync. The formal per-adapter + capability gate is deferred with requirement 2 (sign-off 12). +4. **The graphless migration.** **Superseded in the series.** No + graphless flag, stub backfill, or rowless classification exists, and + the draft's 4b startup requirement (a creating provider must have an + openable graph store at boot) did not ship. The implemented rule is + request-scoped, where no available identity graph means no identifier + is created, on the organic path and on resolve alike, so a graphless + deployment runs identity-less rather than failing startup (§2 row 12). + The rowless design returns, if at all, with sign-offs 21, 29, and 30. +5. **The graph schema change is expand-contract.** **Deferred.** Rows + carry the existing schema. No provider/version field, per-permission + provenance, policy revision, family ID, model epoch, or rollback floor + ships in the series. This work belongs to the durable-suppression and + provenance follow-up (sign-offs 11, 16, 19, 20, 25). +6. **Half-migrated fails loud.** Implemented (#1043). An + `[ec.providers.hmac]` block with no `provider = "hmac"` selector is a + startup error, as is a selector whose block is absent and an + unreferenced block alongside a different selection. The exact state + that validated green and silently created zero ECs in PR #838 now + refuses to start. +7. **PR #838-era keys.** **Revised.** The draft required rejecting + `provider = "host-signals"` and `provider = "client-fixed"` as unknown + keys. The series instead ships both deliberately. `host-signals` is a + supported opt-in selection (#1044) pending the sign-off 22 review, and + `client-fixed` exists only under the `client-fixed-demo` cargo feature + with production builds rejecting the selection at startup (#1046). + Genuinely unknown keys still fail loud through the unknown-selector + error and `deny_unknown_fields`. +8. **Provider switches go through legacy readers.** **Deferred.** No + `legacy_providers` mechanism exists. Today a provider switch on a + deployment with live identities strands the outgoing provider's + cookies (the new provider's `accepts_id` rejects them and identity + restarts). The reader-chain design remains the bar for when a second + server-side provider makes switching real. +9. **The example config ships the migrated shape.** **Revised.** The + example ships Edge Cookie identity off by default, with each selector + and its block documented together and commented together, and + `default_country = "FR"` uncommented. The draft wanted the happy path + uncommented. The series instead closes PR #838's silent-stateless trap + by validation (requirement 6), so a half-uncommented configuration + refuses to start rather than running stateless. Static-geo examples + pair `default_country` with the commented + `assume_single_jurisdiction` acknowledgment, which startup enforces + whenever an EC provider runs with no geo provider (#1045). +10. Every misconfiguration in the providers spec §6 table fails at + **startup**. Implemented for configuration errors (#1043-#1045: + selector/block mismatches, unknown keys, unknown selector values, + missing `default_country`, missing acknowledgment, demo provider in a + production build). The pre-series passphrase minimum and placeholder + rejection carry over unchanged. One residual is declared. A + selected vendor or host provider the running adapter does not inject + fails per request, loudly, because adapter injection is a build fact + the settings layer cannot see. Closing that residual belongs to the + capability-matrix follow-up (requirement 2). +11. Validation is split into two named layers. Partially implemented. + Structural validation runs at `ts config push` / `ts config validate` + (the CLI deserializes and validates the typed settings, with a + regression test covering environment overlays) and again at startup, + and startup additionally validates deployment facts only startup can + see (provider availability in the build, `default_country` against + the compiled `permissions.yaml`, the acknowledgment rule). The + machine-readable adapter capability profile for push-time deployment + pre-checks is deferred with requirement 2. + +## 5. Minimal-divergence migration recipe (operator-facing) + +"Keep exactly today's behavior" is not fully achievable, and the recipe's +name says so. The divergences that actually shipped in the series, each a +matrix row: global honoring of mapped opt-outs (row 8), an authoritative +TCF refusal blocking new grants everywhere (row 6), the protective +geo-failure floor (row 5), country-wide protective US handling (row 3f), +malformed-present blocking acquisition (row 3c), and the shipped `granted` +US and GB storage baselines (rows 3 and 4, flagged for ratification). +Divergences the draft listed that have **not** shipped and remain deferred: +the sharing/targeted GPP fields (row 3e), the grant-signal class (row 3a, +now stricter instead), and the batch-sync provenance gate (row 11c). + +Policy now lives in `permissions.yaml`, compiled into the build. There is +no `[permissions]` block in `trusted-server.toml`, so the draft's +partial-policy trap (a TOML table containing only a permissive default) +cannot be written at all. A deployer edits the yaml and rebuilds, which +keeps every policy change reviewable in version control. The shipped +default policy maps the EU-27 plus the EEA states to `gdpr-eu` (a signal +required for every modeled use), GB to `gdpr-uk` (storage `granted`, +flagged in row 4), and US and AU to `us-opt-out` (a `granted` baseline +where all granted uses drop on any opt-out signal), leaves every unmodeled +Data Use `denied`, and sends unmatched requests to the rule named by the +required `[geo] default_country`. + +The migrated operator configuration is: + +- `[ec] provider = "hmac"` with its `[ec.providers.hmac]` block (32-plus + character passphrase), carried over verbatim for identity stability + (§3); +- `[device]` left at the `builtin` default, with `fastly` as the opt-in + documented alongside the open sign-off 22 question; +- `[geo] provider = "platform"` where the adapter supplies a host lookup + (Fastly and Cloudflare in production, the Axum dev server in + development, while Spin resolves nothing either way). A selected + provider's lookup failure resolves at the requires-signal floor and + never at `default_country`. A static deployment instead sets + `default_country` together with `assume_single_jurisdiction = true`; +- `[geo] default_country` naming the fallback rule (required). + +The draft's committed per-adapter fixture files +(`docs/guide/fixtures/migration-preserving-.toml`, CI-validated +and pinned against the §2 preservation rows) are **not yet written** and +remain the bar for the migration guide (§7). The guide must also state +that no recipe preserves row 8, because the global honoring of opt-out +signals is unconditional in the shipped model. + +## 6. Rollout sequence and observability + +1. Implementation PRs land in the epic's order. **Done as specified.** + The series landed providers first with the geo default held at today's + behavior (#1044), and the permission model PR flipped the default + together with its jurisdiction guard (#1045). Each PR's description + states what that PR changes. +2. **Adapter qualification as a release gate.** **Deferred** with §4 + requirement 2. The series' posture is the declared stateless-identity + path for adapters without the identity endpoints. The response hook + remains outside the series entirely (#1047). +3. **Staged activation for policy publication.** **Deferred** (sign-off + 19). `ts config push` today publishes the operator configuration as a + blob envelope and validates the configuration. The immutable + `push_sequence` envelope, prepare/commit activation, admission lease, + quiescence barrier, and scheduled-unavailability protocol are not + implemented. Policy in the shipped series changes by rebuilding + (`permissions.yaml`) or by pushing configuration, both taking effect + on restart/reload rather than through a fleet-wide activation CAS. +4. Before/after deploy, operators watch **EC issuance rate** and EID + attachment rate as the canary metrics, because the failure mode of a + bad migration is a silent drop to zero (or a silent grant to + everyone), not an error rate. **The named metric set with thresholds, + windows, and actions is not yet built.** The requirement stands for + the migration guide. The retirement-readiness bar for a legacy + provider (legacy-reader hits at zero for a quiet period no shorter + than the maximum cookie/row lifetime plus rollout skew) transfers to + the deferred `legacy_providers` design (§4 requirement 8). +5. Startup logs. Partially implemented. Startup logs the effective + default baseline and the exact list of permissions granted without a + signal (`Permission baseline: [geo] default_country = ...; granted +without a signal: [...]`), plus the passphrase deprecation warning. + The single greppable line naming the selected provider per concern and + whether geo is live remains to add. +6. **The batch-sync coverage dip.** **Deferred** with the provenance + gate (row 11c, sign-offs 7, 13, 25). Today batch sync authenticates + the caller and checks row state, and rows that are missing or + withdrawn collapse to ineligible. There is still no fail-open + shortcut, and grandfathering pre-epic identities past the permission + model remains rejected. +7. Rollback is config-only where possible. Implemented in the simple + sense, where selection is configuration, so reverting to the previous + configuration on the previous binary restores the previous behavior, + and the same compiled binary switches providers through the + `TRUSTED_SERVER__ec__provider` / `TRUSTED_SERVER__device__provider` / + `TRUSTED_SERVER__geo__provider` overrides applied when the operator + publishes configuration through `ts config push`. The durable + artifacts today are exactly the **identity-graph withdrawal + tombstones** written by explicit storage withdrawal (no recovery, that + is their purpose). The draft's longer durable list (model epoch tuple, + use-opt-out suppressions, negative outbox, safety breaker) describes + the deferred durable-suppression design (sign-offs 11, 16, 19, 20). + The two documented-not-automated procedures (cleanup after a policy + tightening, legacy-reader retirement) remain for the migration guide. + +### 6.1 Operator CLI delta for this epic + +The pre-existing CLI design remains unchanged and is what the series +uses. `ts config push` validates and publishes the operator configuration +(with typed environment overlays covered by a CLI regression test), and +`ts config validate` runs the same structural validation standalone. + +The draft's normative delta to `ts config push` (never-reused +`push_sequence`, immutable envelope identity, candidate CAS, §5.5 +promotion protocol) and the `ts config gc` command are **not +implemented**. They are deferred together with sign-off 19 and remain the +design bar for the durable-suppression and staged-activation follow-up. +Nothing in the series authorizes a generic raw metadata command or a +model-transition command. + +## 7. Documentation deliverables + +- Migration guide page (§5), linked from `CHANGELOG.md` and the release + notes. **Outstanding.** The series updated `configuration.md`, + `edge-cookies.md`, `ec-setup-guide.md`, and `error-reference.md` + (#1047), and the example configuration documents every selector, but + the dedicated migration page with per-adapter fixtures and the + CHANGELOG link do not exist yet. +- `configuration.md` documents **every** valid `provider` value for all + three concerns. **Done** (#1047), including the required + `default_country`, the acknowledgment flag, and the requires-signal + floor on a failed lookup. The environment-variable overrides are now + real in production, applied as typed EdgeZero app-config overlays when + the operator publishes through `ts config push`, with a CLI regression + test. (In PR #838 the documented override existed only under + `#[cfg(test)]`. The core test-only helper survives as legacy, and the + runtime path does not use that helper.) +- The permission model page states the precedence rules the code + implements. **Done** (#1045/#1047). The permission-model guide is in + the docs navigation, and the precedence is fixed in code (opt-out over + TCF, malformed-present fail-closed, then TCF), with pinning tests per + opt-out source against a consenting TCF record. Operator docs and + normative spec must not diverge on precedence. + +## 8. Product decisions requiring explicit sign-off + +These are product decisions this spec set needs that #838 had not already +made (or made differently). The table records the recommended resolution +approved for this spec revision, not a final product decision. +**Implementation is blocked while any row is `open`.** Each row is a +decision, not an assignment, since who decided is captured inside the +record itself (`docs/superpowers/specs/decisions/NN-title.md`, the +decision, the deciders, the date). The Decision-record column holds the +link (`(none)` while open). An unratified row reverts to open, not to +silently implemented. **The implemented series (PRs #1043-#1047) is +presented to the task force for ratification of the rows marked below as +implemented or bearing on the series**, so the task force can ratify +rather than re-litigate. Ratifying such a row creates its decision record +and closes the row. Rejecting one reverts the implementation, never keeps +the code with the row open. + +| # | Recommended resolution | Where | Decision record | Status | +| --- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 1 | Honor mapped use opt-outs globally. Destructive identity effects are limited to explicit storage withdrawal, authenticated deletion, or a qualifying live TCF Purpose 1 refusal. | permission §4, §4.2 | (none) | open. Implemented by #1045 (jurisdiction-free suppression, withdrawal only on a live TCF storage refusal under a non-granted baseline). Ratify | +| 2 | GPP/USP sale opt-outs suppress the personalised-ads uses only; they neither revoke storage nor delete the identity. | permission §4.5 | (none) | open. Partially implemented by #1045 (no opt-out deletes or destructively revokes). The shipped default maps opt-outs to `revokes: all` rather than personalised-ads only, a `permissions.yaml` choice to ratify | +| 3 | Sharing/targeted-advertising opt-outs suppress the personalised-ads uses; an explicit applicable not-opted-out value may grant them. Neither affects storage. | permission §4.5 | (none) | open. Not implemented (the sharing and targeted-advertising GPP fields are not decoded, and no US signal grants anything in the shipped model) | +| 4 | US auction dispatch may continue while personalised-ads is unset only through permission §7's positive `ContextualAuctionView` and its sole normative inline manifest. | permission §7.1 | (none) | open. The strip half is implemented by #1045 (EIDs and `user.id` withheld when the pair is unset). The positive contextual projection is not implemented | +| 5 | Country-only and regionless US traffic use a protective country-wide `us-opt-out` floor; state rules may be stricter. | permission §3.4 | (none) | open. Implemented by #1045 (`permissions.yaml` maps country `US` to `us-opt-out` with state overrides available). Ratify | +| 6 | Raw regulatory strings reach only the positively registered OpenRTB field that requires each source; all other destinations default deny. Identity rows retain normalized provenance/digests, not raw consent snapshots. | permission §7; providers §6.3 | (none) | open. Not addressed by the series | +| 7 | Reject legacy batch-sync traffic until live-browser provenance backfill makes the row re-evaluable. | rollout §6 item 6; permission §7 | (none) | open. Not implemented (no provenance exists to recompute) | +| 8 | Gate proxy, click, and Testlight identity forwarding on the sharing pair (storage plus personalised-ads). | §2 row 11b | (none) | open. Not implemented by the series | +| 9 | Defer integration-owned cookie operations from the v1 response hook; require a complete read/use/withdraw lifecycle before admission. | hook §3 | (none) | open. Overtaken (#1047 removed the whole hook from the series, so no cookie surface shipped). The deferral returns with the hook's first consumer | +| 10 | Do not create a blanket session-cookie exemption; every cookie must be covered by an approved permission or narrowly defined security-use authority. | hook §3 | (none) | open. Hook not shipped, unaffected | +| 11 | Require a durable per-family negative-intent outbox in a failure domain independent of its strong target and checked freshly by every identity consumer, with a globally visible breaker over positive identity operations when neither can commit. | permission §4.3 | (none) | open. Not implemented (part of the durable-suppression follow-up) | +| 12 | Adapters that cannot meet the revocation-storage contract migrate stateless rather than weakening the contract. | rollout §6 item 2; recipe §5 | (none) | open. Direction implemented (`provider = "none"` spells stateless in #1043; the identity endpoints were already Fastly-only before the series, and #1046 keeps `resolve` on the same footing). The formal capability gate is deferred. Ratify the direction | +| 13 | Keep batch sync fail-closed at cutover; stage partner communication and cleanup using explicit coverage thresholds, windows, and pause actions. | rollout §6 item 6 | (none) | open. Not implemented (no provenance cutover exists yet) | +| 14 | Policy tightening does not reinterpret historical refusal as a destructive event; destructive withdrawal requires fresh, live qualifying evidence. | permission §4.2 trigger 2 | (none) | open. Implemented by #1045 (withdrawal evaluates the live request's TCF record only, and historical records are never reinterpreted). Ratify | +| 15 | Descope the client cycle and `rewrite_legacy`; ship the v1 integration hook as headers-only. | client spec status; providers §6.1; hook §3 | (none) | open. Overtaken (the client cycle shipped hardened as #1046 instead of descoped, `rewrite_legacy` does not exist, and the hook shipped not at all per #1047). Re-decide against the shipped shape | +| 16 | Persist use-opt-out suppression until ordered explicit authorization for that use or identity deletion, with TCF `LastUpdated` or an authenticated monotonic revision proving order. | permission §4.3 | (none) | open. Not implemented (suppression in the series is request-scoped with no durable record) | +| 17 | N/A, absent, reserved, unknown, and unsupported values never grant processing. | permission §4.5 | (none) | open. Implemented by #1045 in a stricter form (signals only revoke, so no US-signal value grants anything). Ratify | +| 18 | A selected geo provider's lookup failure uses the compiled-in protective profile; `default_country` is only for acknowledged static-jurisdiction mode. | permission §5.2 | (none) | open. Implemented by #1045 (failed lookups resolve at the requires-signal floor, logged at error level, never `default_country`). Ratify | +| 19 | Use immutable version-addressed whole-config publication plus prepare/commit activation of the complete tuple, with authenticated fleet membership, bounded admission lease, quiescence, and an activation journal; a second unanimous model transition advances model epoch, minimum binary generation, and row schema floor atomically. | permission §5.5; rollout §6.1 | (none) | open. Not implemented (`ts config push` publishes and validates without the activation protocol) | +| 20 | N+1 keeps v1 creation and pre-epic live gating, reads/enforces N+2 negative state for rollback safety, and never originates durable use suppression; new-shape settings alone do not activate the new writer/model. | migration §4.4 | (none) | open. Overtaken in part (the shipped migration is a one-release dual-read of `[ec] passphrase` in #1043 with mixed forms rejected and nothing durable added, so the full interim waits for the durable design) | +| 21 | Expire and re-create rowless legacy cookies without continuity; a prefix match cannot authenticate the cookie suffix. | providers §5 | (none) | open. Not implemented (no rowless classification exists, and a cookie with no row is never shared) | +| 22 | Defer host JA4/H2 fingerprint processing to a separate approved design; reject `[device] provider = "fastly"` at startup and do not persist fingerprint-derived classifications. | providers §5 | (none) | open. **Contradicted by the series and flagged for review** (#1044 ships the `fastly` device provider and the host-signal EC provider opt-in, and device signals including fingerprint hash prefixes persist in rows) | +| 23 | Permit a narrow `SecurityUse` authority for DataDome only, with the exact bounded surface the hook spec defines. | hook §4a; permission §7 | (none) | open. Hook not shipped, unaffected | +| 24 | Malformed/absence suppression overrides a permissive baseline but clears on newer valid evidence; it is not sticky like an explicit use opt-out. | permission §4.3, §4.1 | (none) | open. Implemented by #1045 by construction (the fail-closed block is request-scoped, so newer valid evidence re-resolves). Ratify together with row 16's durable design | +| 25 | Enforce a stored-jurisdiction/provenance horizon for batch sync: moves into stricter regimes fail closed by the horizon, and moves out require a live visit. | permission §7 | (none) | open. Not implemented | +| 26 | Aggregate embedded GPP GPC with `Sec-GPC` by OR as a global, non-destructive use opt-out. | permission §4.5 | (none) | open. Partially implemented by #1045 (`Sec-GPC` is an honored non-destructive source). The embedded GPP GPC subfield is not read | +| 27 | In proxy mode, decode only mapped opt-out fields and derive no grants. | permission §4.4 | (none) | open. Partially (the shipped model derives no grants from any US signal anywhere). The proxy-mode decode restriction is not separately implemented | +| 28 | Require product and written vendor conformance approval for the reduced DataDome surface the hook spec pins. | hook §4a.2 | (none) | open. Hook not shipped, unaffected | +| 29 | Accept rowless roaming-cookie expiry as a bounded residual only with telemetry, an explicit maximum lifetime, operator documentation, and a removal/sunset criterion. | providers §5 | (none) | open. Not implemented | +| 30 | Saturation blocks rowless admission for that prefix but never revokes an authenticated real row without its exact suffix; monitor NAT-cohort pressure. | providers §5 | (none) | open. Not implemented | +| 31 | Keep replay history bounded by evicting expired/grant entries first and retaining restrictive state for its full horizon; saturation never shortens a later opt-out. | permission §4.3; providers wire schema | (none) | open. Not implemented | +| 32 | Accept official GPP sections 24-27 version 1, pin their layouts to the vendored IAB commit, and treat complete decoder/fixture support as a release prerequisite, with full vendoring evidence. | permission §4.5.1 | (none) | open. Not implemented (the shipped decoder consults the GPP sale field, with no vendored IAB corpus) | +| 33 | Treat any malformed or unsupported-version **mapped** GPP section as a global blocker for grants to the permissions its schema maps, while still honoring decodable opt-outs elsewhere and never deriving withdrawal from malformed bytes; unknown unmapped section IDs remain non-contributing. | permission §4.5 | (none) | open. Partially implemented by #1045 (a present undecodable record blocks baseline grants). The per-section mapped-blocker rule is not implemented | +| 34 | Permit providers whose canonical identifiers cannot fit an injective graph suffix to use the `sha256-detect` mode: domain-separated collision resistance plus stored canonical-identifier comparison, fail-closed collision handling, no overwrite/join. | providers §2, §6.3 | (none) | open. Premise revised by #1043 (identifiers are globally bounded at 256 bytes and the graph is keyed by `normalize_id_for_kv`). No `sha256-detect` mode exists | + +## Revision record vs the 2026-07-31 draft + +| Draft position | Revised position | Why | +| -------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Epic #777-#781 pending sign-off, no implementation | Implemented as five stacked PRs #1043-#1047, with §8 staying the ledger and rows marked for ratification | The series shipped the seam, selection, permission model, resolve endpoint, and docs, so the task force ratifies rather than re-litigates | +| `[device] provider = "fastly"` startup-fails pending a separate fingerprinting design (§2 row 9, row 22) | The `fastly` device provider and host-signal EC provider ship opt-in, and device signals persist in graph rows. The identifier-collision defect the review found (host-signal identifiers shared the HMAC grammar and keyspace) is fixed by the mandatory provider-code envelope, so host-signal identifiers are `hs00~` namespaced; the policy question of row 22 is unchanged by the collision fix | Implementation choice, deliberately flagged as the open review question rather than presented as settled | +| US recipe: `requires_signal` with an extended grant-signal class (§2 rows 3/3a) | Shipped default: `granted` US baseline, `revokes: all` on any opt-out, and no US signal grants anything | The shipped signal model is revoke-only, which is simpler and stricter on grants, and the baseline choice is deployer-editable yaml, flagged in rows 3 and 5 | +| `[permissions]` TOML policy published at runtime | `permissions.yaml` compiled into the build, no `[permissions]` block | Policy stays reviewable in version control and the partial-policy trap cannot be written | +| `rules.default` worldwide default entry (§2 row 7) | Required `[geo] default_country` naming the fallback rule, validated at startup | Same role, one mechanism, loud when missing | +| Graph store mandatory at startup for creating providers (§2 row 12, §4.4b) | No startup requirement, and with no graph no identifier is created, per request | The phantom-cookie rule holds without a breaking startup change, and graphless deployments run identity-less | +| `legacy_providers` reader chain for provider switches (§4.8) | Not implemented, so a switch restarts identity | Deferred until a second server-side provider makes switching real | +| N+1/N+2 negative-record machinery, model epochs, `m00` mirror (§4.1, row 20) | Deferred with rows 11, 16, 19, 20. The shipped dual-read is one release of `[ec] passphrase` mapping with mixed forms rejected | Nothing durable beyond pre-existing withdrawal tombstones ships, so binary rollback strands no new state | +| Staged activation, `push_sequence`, quiescence, `ts config gc` (§6.3, §6.1) | Basic `ts config push` / `ts config validate` only | The activation protocol belongs to the deferred durable-suppression rollout (row 19) | +| Migration guide with committed per-adapter fixtures and a full gated metric set (§5, §6.4) | `configuration.md` / `edge-cookies.md` document the migrated shape, while fixtures, the guide page, and metrics remain outstanding | Documentation shipped for configuration, and the operational guide is the remaining deliverable before a release | +| Client cycle descoped and hook shipped headers-only (row 15) | The client cycle shipped hardened (#1046) and the hook shipped not at all (#1047) | Hardening replaced descoping, and the hook had no consumer, which is the hook spec's own admission rule | +| Example config ships the migrated happy path uncommented (§4.9) | Identity off by default with selector and block commented together, and validation closes the silent-stateless trap | Loud startup validation, not an uncommented default, is what prevents PR #838's silent-stateless state | +| Environment override documented but `#[cfg(test)]`-only in PR #838 (§7) | Override applied as typed EdgeZero app-config overlays at `ts config push`; the existing CLI overlay test covers the mechanism, and a provider-specific override test is still to write | The same compiled binary switches providers at deployment through the published configuration | +| Pinned known-answer HMAC vectors committed (§3) | Stability tested per inputs, and the pinned vector is still to commit | The cross-version CI pin remains open work under §3 | +| GB storage baseline change only with citation and sign-off (§2 row 4) | The shipped `permissions.yaml` adopts `granted` storage for GB without a recorded citation | Flagged in row 4, since the task force owns the decision and its record | diff --git a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md index 373627392..cc53032a0 100644 --- a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md +++ b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md @@ -1,12 +1,13 @@ # Design Spec: The Integration Provider Seam -**Status:** Proposed, 2026-08-27, revised 2026-08-28. This PR adds this -document only and targets `main` directly. Following the review of #1043 +**Status:** Proposed, 2026-08-27, revised 2026-08-28. This PR adds design +documents only and targets `main` directly. Following the review of #1043 (27 August) the seam it defines is a precondition for the provider series rather than a follow-up to it, so the order is now this spec, then its implementation in a seventh PR against `main` (51Degrees), then PRs #1043 -to #1047 reworked onto it. It reads alongside the series' specs, which land -with PR #1047. +to #1047 reworked onto it. It reads alongside the series' specs, which this +PR now carries too, so the whole normative set is reviewable before any of +the code lands. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-07-30-pluggable-providers-design.md`, `2026-07-30-provider-migration-rollout-design.md`, @@ -418,3 +419,4 @@ defines, and both should land before the first vendor is asked to use it. | 2026-08-28 | Corrected line references to `main` at b7fcb5d4c and added what mapping `main` found: composition of the served script moves into core (§3.2), registration enumeration and the auction-only `adserver_mock` case (§3.3), the duplicate-id gap (§3.1), the source-file guard and the `ts audit` vendor table (§4), the renderer risk (§7). | | 2026-08-28 | Recorded what implementing the seam found (§8): the operator CLI skips a vendor's deploy rules, a carried module's hash literal is fragile, providers resolve more than once per request, and one core reader still reads an APS payload. Recorded the construction-time hash check in §3.2. | | 2026-08-28 | Adopted the #1043 review's registration shape for identity and applied its rule to geo and device, with no provider built into core (§3.6, §6 item 2, §8 rows 7 to 9). Recorded the relationship to #986 and reordered the series so this spec and its implementation come first. | +| 2026-08-30 | Moved the five series design specs and the provider-code registry into this PR from PRs #1043 to #1047, so every normative document is reviewed before the code that implements it. Document content is unchanged; only this status line and this row are new. | diff --git a/docs/superpowers/specs/provider-code-registry.md b/docs/superpowers/specs/provider-code-registry.md new file mode 100644 index 000000000..0a2014c54 --- /dev/null +++ b/docs/superpowers/specs/provider-code-registry.md @@ -0,0 +1,30 @@ +# Provider-code registry (normative, append-only, never reused) + +Four-character codes (`[a-z0-9]`, zero-padded) that namespace Edge Cookie +identifiers. Every EC provider MUST allocate a code here before it can +exist: the `EdgeCookieProvider::code()` trait method is mandatory, and core +applies the code as the `{code}~` prefix of every identifier the provider +mints, checks it at read-back, and keys the identity graph with it. A +provider only ever sees its own value part, so identifiers from different +providers can never collide in the cookie, the graph, or a withdrawal, and +every identifier records which provider created it. + +Allocation is a reviewed commit to this file; codes are immutable and never +recycled, including for retired providers. A leading digit is valid. The +tilde separator keeps parsing exact while pre-envelope identifiers remain +deployed: a legacy bare identifier contains no tilde and dual-reads under +the built-in HMAC provider only. + +The class of provider expected to grow this table is one that consumes a +web-browser-supplied unique identifier, arriving either as a new web +platform feature or from a user-installed extension, delivered to the edge +through the client-cycle resolve path and verified by the provider before +minting. + +| Code | Provider | Allocated | Status | +| ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | -------------------------- | +| `hmac` | Built-in HMAC EC provider. Mints `hmac~<64 hex>.<6 alnum>`; dual-reads its pre-envelope bare form so deployed cookies keep working (retirement condition in `provider_owns_id`) | 2026-08-02 | active | +| `hs00` | Built-in host-signal EC provider (opt-in; TLS JA4 plus HTTP/2 signals plus client IP) | 2026-08-25 | active | +| `cfix` | Client-fixed demonstration provider (compiled only behind the `client-fixed-demo` cargo feature) | 2026-08-25 | active, test and demo only | +| `51dd` | 51Degrees Identifier (51Did) vendor provider | 2026-08-25 | reserved | +| `t0..` | Prefix family reserved for in-tree test providers (`t0cc`, `t0op`, and similar); never valid in configuration | 2026-08-25 | reserved | From 92f48b03690b495df2c00e4b154195bb3cdd5fd6 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 30 Aug 2026 22:27:03 +0100 Subject: [PATCH 05/17] Carve evidence interfaces out of the trait minimalism rule The rule that every trait method needs a production caller in the change that introduces it is right for a behavioural trait, where a method nothing calls is dead weight. It is wrong for an evidence interface, and we applied it there and should not have. An evidence interface describes what a request carries, not what today's code happens to read. Held to the caller rule it grows a method each time a vendor arrives, so a vendor cannot write against it and it cannot stay stable across a release. It also puts the boundary in the wrong place. What a provider may see is not the control, because what a provider may do with what it sees is the control, and that is the permission model's job. So RequestInfo carries what the request carries. The rule stands everywhere else. Addresses: docs/superpowers/specs/2026-07-30-pluggable-providers-design.md section 4, a rule applied to an interface it does not suit --- .../2026-07-30-pluggable-providers-design.md | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index 78aca0684..4640d1fc0 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -178,10 +178,22 @@ One global rule sits above every provider, and it is implemented: not at all, and the cookie value and the identity-graph key can never silently diverge. -## 4. Trait surface: minimalism rule +## 4. Trait surface: minimalism rule, and where it does not apply Every trait method must have at least one production (non-test) caller in -the same change that introduces it. How the surface observed in PR #838 +the same change that introduces it. **This rule does not apply to an evidence +interface, and applying it there was a mistake we made and are correcting.** + +An evidence interface describes what a request carries, not what today's code +happens to read. Holding it to the caller rule produces an interface that grows +a method each time a vendor arrives, which is not something a vendor can write +against and cannot be stable across a release. It also puts the boundary in the +wrong place, because what a provider may see is not the control. What a provider +may do with what it sees is the control, and that is the permission model. + +So `RequestInfo` carries everything the request carries, whether or not code in +this repository reads it yet. The rule stands for behavioural traits, where a +method with no caller really is dead weight. How the surface observed in PR #838 resolved in the implementation: - `keys_equal`: **not shipped.** Its legitimate purpose (equivalent-envelope From 7c4f0b789cbbf714f3900871610569e19e72dbb1 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 30 Aug 2026 22:30:08 +0100 Subject: [PATCH 06/17] Say create rather than mint throughout the provider specs "Mint" is jargon for something ordinary, which is making an identifier. It was defined in the glossary, so it was at least explained, but a reader should not need a glossary for the word that describes the main operation the specs are about. Fifty-one uses across five files, including the glossary row itself, now read create, creates, created or creating. Nothing else changed, so a reader comparing against the previous revision sees one word substituted and no argument altered. Addresses: plain language in documents others read --- ...26-07-30-client-cycle-ec-resolve-design.md | 24 ++++---- .../2026-07-30-permission-model-design.md | 2 +- .../2026-07-30-pluggable-providers-design.md | 60 +++++++++---------- ...-08-27-integration-provider-seam-design.md | 2 +- .../specs/provider-code-registry.md | 18 +++--- 5 files changed, 53 insertions(+), 53 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index 6949889a3..4abab609e 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -16,7 +16,7 @@ defers, and why the feature is normative rather than deferred. > browser POST to a new public endpoint (`POST /_ts/api/v1/ec/resolve`), > plus a demo provider (`client-fixed`) and a JS bundle. Review found the > endpoint accepted cross-origin identity-setting posts with no origin -> check, minted cookies with no identity-graph row (violating an invariant +> check, created cookies with no identity-graph row (violating an invariant > the organic path enforces explicitly), was registered on only one of four > adapters, and could never round-trip because the core did not recognize > non-HMAC identifiers. None of that is an argument the feature is a bad @@ -54,7 +54,7 @@ verify against. | Threat | Vector | Consequence if unmitigated | | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Cross-site identity fixation** | `text/plain` POST is a CORS-simple request: any page on the web can `fetch(resolveUrl, {method: "POST", credentials: "include", body: payload})` with no preflight | An attacker pins a chosen identity onto a victim's first-party cookie jar, a login-CSRF for the ad-identity layer, and the victim's activity accretes to an attacker-controlled ID | -| **Replay** | A captured valid payload (from the attacker's own session or a leak) replayed against another browser | Same as fixation, without needing to mint payloads | +| **Replay** | A captured valid payload (from the attacker's own session or a leak) replayed against another browser | Same as fixation, without needing to create payloads | | **Phantom identity** | Endpoint sets the cookie without an identity-graph row | Later requests carry an EC that the KV graph has never seen; downstream sync and withdrawal logic operate on an identity that half-exists (the organic generation path explicitly refuses to write a cookie when the graph write fails, for exactly this reason) | | **Un-tombstoneable identity** | Core does not recognize the provider's identifier shape | Withdrawal cannot expire or tombstone the identity, which is a compliance failure and not only a bug | | **Amplification** | The page script cannot observe an HttpOnly cookie, so it cannot know the cookie is already set | A POST on every page view of every session (PR #838's JS gated on reading a cookie its own server marked HttpOnly, making the guard permanently false) | @@ -79,7 +79,7 @@ same-site` (which v1 does not consult at all), not to a suffix match on open (§7.5) for publishers whose pages run on domains other than the configured apex. 2. **Verify the payload per provider.** The endpoint hands the posted - payload to the selected provider's `resolve_from_client` and mints only + payload to the selected provider's `resolve_from_client` and creates only what the provider returns; whether the payload is trustworthy is the provider's responsibility, stated on the trait. A real vendor provider verifies a signed, audience-bound, expiring envelope; the draft's @@ -90,8 +90,8 @@ same-site` (which v1 does not consult at all), not to a suffix match on server-side providers are untouched. 3. **Preserve the identity-graph invariant.** Implemented: the graph row is written before the cookie is set, keyed by the provider's - `normalize_id_for_kv` canonical form, exactly like the organic mint - path. No graph available → no mint (`204`), same as organic generation; + `normalize_id_for_kv` canonical form, exactly like the organic create + path. No graph available → no create (`204`), same as organic generation; a graph write failure → `503`, no cookie. 4. **Round-trip through the lifecycle contract.** Implemented: read-back goes through the selected provider's `accepts_id`, the KV key through @@ -101,7 +101,7 @@ same-site` (which v1 does not consult at all), not to a suffix match on 5. **Exist on every adapter, where parity means identical behavior, including identical refusal.** Partially implemented, documented: the Fastly adapter routes the endpoint (passing the same bot-gated identity - graph as organic generation, so unrecognized clients cannot mint + graph as organic generation, so unrecognized clients cannot create through resolve either). The Axum, Cloudflare, and Spin adapters deliberately do not route it, matching `identify` and `batch-sync`, which need the same platform KV wiring those adapters do not have; the @@ -113,20 +113,20 @@ same-site` (which v1 does not consult at all), not to a suffix match on declaration.** Implemented: every response carries `Cache-Control: no-store`, and the gate is the selected provider's complete `required_permissions()` through the same resolved permission state as - organic minting, not a hard-coded storage check. + organic creating, not a hard-coded storage check. 7. **Bound every input.** Implemented: request body at most 65,536 bytes, where an advertised `Content-Length` over the limit answers `413` before the read, and the read body is re-checked so a missing or false length does not bypass the bound. `Content-Type` allowlist: `text/plain` and `application/json`, matched on the media type alone, case-insensitively, ignoring parameters (the browser's default `text/plain;charset=UTF-8` - passes); anything else → `415`. The minted identifier must fit the + passes); anything else → `415`. The created identifier must fit the global identifier bounds (at most 256 bytes, cookie-safe alphabet, - shared with every other mint path); violation → `400`, never a rewrite. + shared with every other create path); violation → `400`, never a rewrite. Core then applies the provider's registered code envelope (`provider-code-registry.md`): the cookie and the identity-graph key carry `{code}~value`, so a client-set identity is namespaced to its - provider exactly like an edge-minted one (the demo's cookie value is + provider exactly like an edge-created one (the demo's cookie value is `cfix~an-ec`). Status codes are part of the contract: `400` out-of-bounds identifier, `403` origin rejection, `409` different-identity conflict, `413` body, `415` content type, `503` graph-write failure, `204` closed gate / no @@ -147,7 +147,7 @@ no-store`, and the gate is the selected provider's complete primitive no production adapter exposes today. Designing the reservation against a hypothetical envelope would repeat the mistake this series exists to fix. v1's stance: the endpoint is safe without it - for the providers v1 ships (the demo mints a constant, feature-gated + for the providers v1 ships (the demo creates a constant, feature-gated out of production), and the reservation lands with the first vendor scheme, designed against its real envelope, with the draft's §3.9 as the starting bar. Until then the draft's text is preserved below as the @@ -208,7 +208,7 @@ identifier, constant-equality verification) is compiled only behind the `client-fixed-demo` cargo feature. In a production build the settings validator rejects the selection at startup with a direct message, and the provider builder rejects it again as defense in depth. A fixed shared word -is not an identity; the demo exists to exercise verify-before-mint end to +is not an identity; the demo exists to exercise verify-before-create end to end in tests and demonstrations. ## 6. Testing diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index ec98c2844..0cedf6f63 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -638,7 +638,7 @@ requires a live user signal (§4.2), never a policy change. | Undecodable record present (TCF, GPP, or USP) | Revokes every Data Use (fail-closed acquisition); never withdraws; opt-outs still honored | | Expired TCF record | Distinct state, not malformed; treated as absent, so the baseline applies | | Signals contradict (opt-out plus consent) | Opt-out wins (§4) | -| No EC provider selected | Identity fails closed: nothing minted, an incoming cookie value never used or egressed (§7) | +| No EC provider selected | Identity fails closed: nothing created, an incoming cookie value never used or egressed (§7) | The posture is fail-closed. Every ambiguous state resolves to the configured baseline or more restrictive. diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index 4640d1fc0..b8000564e 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -12,7 +12,7 @@ > **Context.** PR #838 proposed a first implementation of this epic in a single > change. Review of that PR surfaced design gaps this spec exists to close > before a second implementation pass: an identity abstraction that owned -> minting but not recognition, per-adapter divergence in provider selection, +> creating but not recognition, per-adapter divergence in provider selection, > silent misconfiguration modes, and speculative trait surface with no > production caller. This spec is the authoritative statement of what the > provider architecture must do; where it contradicts PR #838, this spec wins. @@ -40,7 +40,7 @@ Goals, as implemented: default deployment makes no third-party or host-specific call. - An **EC provider declares** the permissions its data use requires (`required_permissions` on the trait), and **core enforces** that - declaration before minting or using an identity. A provider cannot + declaration before creating or using an identity. A provider cannot authorize itself. The enforcement machinery is the permission model's subject and lands with it in PR #1045 (see the permission model spec). Geo and device carry the same declaration method with an empty default, @@ -111,22 +111,22 @@ the provider as settled either way. ## 3. The identity lifecycle contract -This is the section PR #838 lacked. Its trait abstracted **minting** an +This is the section PR #838 lacked. Its trait abstracted **creating** an identifier but left **recognition** and **KV key normalization** hard-coded to the built-in HMAC shape, so a provider whose identifiers did not match -`{64hex}.{6alnum}` minted cookies that the very next request discarded. +`{64hex}.{6alnum}` created cookies that the very next request discarded. The implemented contract routes every lifecycle operation core performs on an EC value through the selected provider: | Lifecycle operation | Where core uses it | Contract | | ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **Mint** | EC generation on first eligible request, and the client-cycle resolve endpoint | The provider returns the identifier (`generate` server-side, `resolve_from_client` for the client cycle) and only core writes the cookie, after enforcing the global bounds below. | +| **Create** | EC generation on first eligible request, and the client-cycle resolve endpoint | The provider returns the identifier (`generate` server-side, `resolve_from_client` for the client cycle) and only core writes the cookie, after enforcing the global bounds below. | | **Recognize** | Reading `ts-ec` back from the request, deciding `ec_was_present`, withdrawal checks, and every path that hands the value onward: the origin URL in `append_ec_id`, the click-target URL in `handle_first_party_click`, and the proxied body an integration builds | `accepts_id` answers whether a value is a well-formed identifier the provider issues. A value the selected provider does not recognize is treated as absent, so it is never used or egressed, while the raw cookie value stays visible to withdrawal handling. The egress paths reach the same answer through `edge_cookie::recognized_ec_id`, and a deployment with no provider selected recognizes nothing and so egresses nothing. | | **KV key** | Identity-graph row reads and writes | `normalize_id_for_kv` returns the key form. The default lowercases the built-in HMAC hash segment and preserves the suffix, keeping today's keys. An opaque or case-sensitive provider overrides to the identity function so distinct identifiers never collapse into one row. | | **Withdraw** | Expiring the cookie and writing revocation markers | The identifiers eligible for a **graph tombstone** are exactly those the selected provider owns, dispatched on the `{code}~` prefix first and then `accepts_id`, never a shape check the provider cannot influence. Expiring the **cookie** is broader: it keys off the raw cookie being present, so it still fires for an identifier the selected provider does not own (see the switching case, §6.1). | -**Invariant:** for every provider `P` and every identifier `id` minted by +**Invariant:** for every provider `P` and every identifier `id` created by `P`, `id` round-trips read-back byte for byte. A test in `ec/mod.rs` proves the round-trip with a non-default provider whose identifiers are opaque, and a second test in `ec/resolve.rs` proves the client-cycle value survives the @@ -144,7 +144,7 @@ grammar lands, the KV key is the provider's normalized identifier verbatim, which keeps every pre-epic HMAC row reachable. The pre-epic IP-cluster prefix listing runs unchanged, but the key space it -lists over does not. A fresh mint is keyed `hmac~.`, so the +lists over does not. A fresh create is keyed `hmac~.`, so the prefix `evaluate_cluster` derives is `hmac~` for a coded row while a legacy bare row still lists under `` on its own. Prefix matching is anchored at the start of the key, so two rows for the same client IP that @@ -165,12 +165,12 @@ is the change that has to build the bridge. One global rule sits above every provider, and it is implemented: -- **Identifier bounds.** A minted identifier obeys a global cookie-safe +- **Identifier bounds.** A created identifier obeys a global cookie-safe alphabet (normatively `[A-Za-z0-9._~-]`, valid cookie octets with no separators, whitespace, or control characters) and a global maximum of **256 bytes**, stated here so dependent documents reference one number. The bound applies to the identifier itself, not only its key form. Core - enforces the bound wherever an identifier enters the system, at mint + enforces the bound wherever an identifier enters the system, at create (both `generate` and the resolve endpoint), at cookie read-back, and at cookie write. The constant is `MAX_EC_ID_LEN` in `ec/cookies.rs`. A violating value is rejected outright and logged. No sanitizing rewrite @@ -207,7 +207,7 @@ resolved in the implementation: The draft banned the field when nothing consumed it. The consumer landed in the same series, satisfying the rule the ban enforced. - `IdentityInput.permissions` / `IdentityInput.consent`: **shipped and - populated.** The organic mint path passes the request's resolved + populated.** The organic create path passes the request's resolved permission state and consent context so a provider can read them for behavior beyond gating. The gate itself has already run before `generate` is called, so a provider cannot use the fields to authorize itself. @@ -227,13 +227,13 @@ pub trait EdgeCookieProvider: Send + Sync + core::fmt::Debug { /// Stable configuration key ("hmac"). fn id(&self) -> &'static str; /// Registered four-character code (provider-code-registry.md), the - /// `{code}~` namespace of every identifier the provider mints. + /// `{code}~` namespace of every identifier the provider creates. /// Mandatory, no default: a provider cannot exist without a unique /// code, so identifiers from different providers can never collide. fn code(&self) -> ProviderCode; /// Derives an identifier from the provider's injected services and the /// request evidence passed at call time. A client-side provider defers - /// here (returns no id) and mints later in resolve_from_client. + /// here (returns no id) and creates later in resolve_from_client. fn generate( &self, request_info: &dyn RequestInfo, @@ -249,7 +249,7 @@ pub trait EdgeCookieProvider: Send + Sync + core::fmt::Debug { /// Permissions this provider's data use requires. Default: none, so a /// vendor-neutral provider requires no permission. fn required_permissions(&self) -> PermissionSet { /* none */ } - /// Client-cycle counterpart to generate: mints from a value the page + /// Client-cycle counterpart to generate: creates from a value the page /// posted to the resolve endpoint, after verifying it. Default: no-op, /// so a server-side provider does not participate. See the /// client-cycle spec. @@ -260,14 +260,14 @@ pub trait EdgeCookieProvider: Send + Sync + core::fmt::Debug { } ``` -Core owns the code envelope. At mint it prefixes the provider's value with +Core owns the code envelope. At create it prefixes the provider's value with `{code}~`, at read-back it strips and checks the code before the provider's `accepts_id` sees the value part, and the identity graph key preserves the code verbatim around the provider's canonical form. A cookie carrying another provider's code is treated as absent, never adopted, so switching providers cannot silently mix identity populations, and a withdrawal always acts on a key that can only belong to one provider. The built-in HMAC -provider mints `hmac~<64 hex>.<6 alphanumeric>` and dual-reads its +provider creates `hmac~<64 hex>.<6 alphanumeric>` and dual-reads its pre-envelope bare form for one release cycle so deployed cookies keep working; the bare form belongs to hmac alone. Codes are allocated append-only in `provider-code-registry.md`, and a leading digit is valid @@ -287,13 +287,13 @@ ahead of the caller that consumes it. ## 5. Permission enforcement is core's job, for EC providers -Before minting through an EC provider, core resolves the request's +Before creating through an EC provider, core resolves the request's permission state and refuses when the provider's `required_permissions()` are not all set. The gate is implemented in `EcContext`. The selected provider is built once at request read time, its declaration is checked against the resolved state, and generation is skipped (with a log line naming the jurisdiction) when the requirement is not met. With no provider -selected, nothing may mint or use an identifier, so the gate is closed +selected, nothing may create or use an identifier, so the gate is closed rather than open by default. The enforcement point lands with the permission model in PR #1045, and the permission model spec governs the resolution machinery (country and region baselines, signals, and the @@ -364,10 +364,10 @@ when the provider is built, stopping the request rather than degrading. | `[geo] default_country` unset, or matching no `permissions.yaml` rule | **Startup error.** The value is the permission baseline for a request the geo provider leaves unmatched, so there must always be one and the value must resolve to a real rule. | | An EC provider configured, no geo provider, `assume_single_jurisdiction` unset | **Startup error.** With geolocation off, every request resolves as `default_country`, so a visitor from any other jurisdiction silently receives the default jurisdiction's rules. That is acceptable only as an explicit operator decision. | -One draft row was not adopted, the startup error for a minting provider +One draft row was not adopted, the startup error for a creating provider with no identity-graph store. `[ec] ec_store` remains optional, because the portability adapters run without platform KV. The client-cycle resolve -endpoint refuses to mint when no graph is available (a cookie without a row +endpoint refuses to create when no graph is available (a cookie without a row could never be withdrawn through the graph), and the organic path persists the row whenever the graph is configured. Whether configuration should force the pairing is follow-up work with the migration spec. @@ -382,7 +382,7 @@ applies its own `deny_unknown_fields` when it deserializes. ### 6.1 Provider switching: what a switch actually does Switching `[ec] provider` **retires every identity the previous provider -minted**. This section says exactly what that means, because a deployer has +created**. This section says exactly what that means, because a deployer has to plan around it rather than discover it. The draft specified an ordered `legacy_providers` reader list, provider @@ -397,7 +397,7 @@ cookies stay recognized when the newly selected provider accepts their shape. That is not what the code does and never was once core took ownership of the `{code}~` envelope (§5). Ownership is decided on the code prefix **before** any provider is asked about the shape, so a newly selected -provider rejects every identifier the previous one minted, whatever its +provider rejects every identifier the previous one created, whatever its shape, because the code differs. What a switch does, precisely: @@ -441,9 +441,9 @@ logged, none silent: | Failure | Behavior | | -------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `generate` returns an error | No identity this request. The organic caller logs at error level and the request proceeds stateless. No cookie is written. | -| A provider mints an identifier outside the global bounds | Rejected at mint, never rewritten. The organic path yields no identity. The resolve endpoint returns 400. | -| Identity-graph write fails at mint | The mint is undone (no identifier, no cookie), with the error logged. The resolve endpoint returns 503. The next eligible request retries. | -| The host-signals provider finds no TLS/HTTP-2 fingerprints | Defers with a warning. No identity this request, and no degraded IP-only identifier is minted under the host-signals name. | +| A provider creates an identifier outside the global bounds | Rejected at create, never rewritten. The organic path yields no identity. The resolve endpoint returns 400. | +| Identity-graph write fails at create | The create is undone (no identifier, no cookie), with the error logged. The resolve endpoint returns 503. The next eligible request retries. | +| The host-signals provider finds no TLS/HTTP-2 fingerprints | Defers with a warning. No identity this request, and no degraded IP-only identifier is created under the host-signals name. | | Geo lookup **fails** (the provider errors) | Every permission resolves to the requires-signal floor, and the failure is logged at error level. The failure is **not** papered over with the `default_country` baseline. | | Geo resolves **no location**, or a country/region with no rule | The `[geo] default_country` baseline applies. This is the configured-default case, deliberately distinct from the failure row above (`GeoStatus` in `ec/consent.rs`). | | An incoming cookie value fails the bounds at read-back | Treated as absent, with a warning naming the source. | @@ -464,7 +464,7 @@ authority state, negative-intent outbox, rowless withdrawal, and deployment metadata, wire schemas with known-answer vectors, and a per-field graph-row contract. The provider-code registry is now implemented: codes are allocated in `provider-code-registry.md`, carried as the `{code}~` prefix -of every minted identifier, and therefore present in every graph key. The +of every created identifier, and therefore present in every graph key. The key grammar differs from the draft in one deliberate way, a tilde separator instead of delimiter-free fixed width, because pre-envelope bare identifiers remain deployed and a code such as `51dd` is valid hex, so @@ -492,7 +492,7 @@ request path: injecting the host's `HostSignals` when supplied and matching an adapter-injected vendor provider by its `id()`. The provider is built once per request during `EcContext` construction and reused for - read-back, the permission gate, and minting, so the per-request + read-back, the permission gate, and creating, so the per-request triple-build observed in PR #838 (cloning the secret into a fresh box up to three times per request) is gone. - `build_device_provider` (`ec/device.rs`) returns the builtin classifier @@ -588,7 +588,7 @@ Implemented, in the crates named: acknowledgment. - Geo builder tests showing the default selects no geo, `none` selects no geo explicitly, and `platform` selects the host implementation. -- Host-signals provider tests covering minting from fingerprints, +- Host-signals provider tests covering creating from fingerprints, deferring without them, and the loud failure of a selected but uninjected vendor provider. @@ -638,12 +638,12 @@ this revision describes. | Draft position | Implemented position | Why | | ----------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Trait surface is a canonicalizing `parse` returning a typed id, plus `graph_key_suffix`, `cluster_prefix`, and `verify` | `accepts_id` (recognition) plus `normalize_id_for_kv` (KV key form), defaults matching the built-in shape. No typed id, key suffix, cluster capability, or `verify`. `keys_equal` stays out, as the draft required. | Recognition and KV keying are the two operations core performs today. A byte-for-byte round-trip test with a non-default provider pins the contract. | -| `GeneratedEdgeCookie::response_headers` and `IdentityInput.permissions` / `.consent` banned as speculative surface | Shipped with production consumers. Finalization applies provider headers, the resolve path returns them, and the organic mint path populates the input fields. | The client-cycle resolve path landed in the same series and is their caller, satisfying the minimalism rule the ban enforced. | -| Identifier bounds enforced at mint and parse | Enforced at mint (`generate` and the resolve endpoint), cookie read-back, and cookie write. Violations rejected outright, never rewritten. `MAX_EC_ID_LEN` in `ec/cookies.rs`. | Every identifier entry point is covered, and the pre-epic sanitizing rewrite was removed as a silent-divergence hazard. | +| `GeneratedEdgeCookie::response_headers` and `IdentityInput.permissions` / `.consent` banned as speculative surface | Shipped with production consumers. Finalization applies provider headers, the resolve path returns them, and the organic create path populates the input fields. | The client-cycle resolve path landed in the same series and is their caller, satisfying the minimalism rule the ban enforced. | +| Identifier bounds enforced at create and parse | Enforced at create (`generate` and the resolve endpoint), cookie read-back, and cookie write. Violations rejected outright, never rewritten. `MAX_EC_ID_LEN` in `ec/cookies.rs`. | Every identifier entry point is covered, and the pre-epic sanitizing rewrite was removed as a silent-divergence hazard. | | `provider = "none"` is valid alongside `legacy_providers` blocks | `none` (or an omitted selector) with any configured provider block is a startup error. | No `legacy_providers` exists in these PRs, so a block alongside statelessness can only be a mistake. | | Every selection key is closed and unknown keys are startup errors | Device and geo keys are closed. EC vendor keys are open. Unknown blocks are captured as raw values in core, the adapter deserializes its own block, and a selected key with no injected provider fails loudly. | Core never names a vendor, so a vendor provider adds no core change. | | Capability mismatch is a startup error at adapter wiring time | Configuration coherence fails at startup. A host-capability mismatch (missing `HostSignals`, uninjected vendor) fails loudly when the provider is built, stopping the request. | The adapter capability declaration that would move the check to startup is deferred with the capability matrix. | -| A minting provider with no identity-graph store is a startup error | Not implemented. `ec_store` stays optional. The resolve endpoint refuses to mint without a graph. The organic path persists rows whenever the graph is configured. | Portability adapters run without platform KV. Whether configuration should force the pairing is follow-up work. | +| A creating provider with no identity-graph store is a startup error | Not implemented. `ec_store` stays optional. The resolve endpoint refuses to create without a graph. The organic path persists rows whenever the graph is configured. | Portability adapters run without platform KV. Whether configuration should force the pairing is follow-up work. | | `[device] provider = "fastly"` is startup-rejected pending a separate security design | Shipped as a selectable opt-in. The Fastly adapter injects `HostSignals`, the provider strengthens the browser/bot gate, and rows persist derived classes, not raw fingerprints. | Selection is an explicit operator opt-in and the neutral default makes no host fingerprint call. | | The `host-signals` EC provider is deliberately dropped and its selection rejected | Shipped in PR #1044 as an opt-in built-in that defers with a warning when the host supplies no signals. **Open, flagged for the series review**, not settled either way. | Its identifier shape shares the HMAC grammar, and a sign-off row defers host fingerprint processing, so the review decides whether the provider ships in the series. | | Geo default flip sequenced into the later permission-model step, with an acknowledgment guard | Landed as specified in the same series, with the default of none, `default_country` required and validated against `permissions.yaml`, the `assume_single_jurisdiction` acknowledgment, and a failed lookup resolving to the requires-signal floor with error logging (`GeoStatus`, resolved in core so all adapters agree). | The permission model shipped in PR #1045, so the constraints exist where the draft required them. | diff --git a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md index cc53032a0..7e35617bc 100644 --- a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md +++ b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md @@ -300,7 +300,7 @@ deployment that lists the same integrations gets the same responses. built-in one. 2. **Capabilities round trip.** The same test integration declares an identity, a geo and a device provider. With the three selectors naming - it, a request is served by all three (the minted identifier carries its + it, a request is served by all three (the created identifier carries its code, the resolved country and the device signals are its). With a selector naming a module that lacks the capability, startup fails with an error that names the module and the capability. diff --git a/docs/superpowers/specs/provider-code-registry.md b/docs/superpowers/specs/provider-code-registry.md index 0a2014c54..e6771337e 100644 --- a/docs/superpowers/specs/provider-code-registry.md +++ b/docs/superpowers/specs/provider-code-registry.md @@ -4,7 +4,7 @@ Four-character codes (`[a-z0-9]`, zero-padded) that namespace Edge Cookie identifiers. Every EC provider MUST allocate a code here before it can exist: the `EdgeCookieProvider::code()` trait method is mandatory, and core applies the code as the `{code}~` prefix of every identifier the provider -mints, checks it at read-back, and keys the identity graph with it. A +creates, checks it at read-back, and keys the identity graph with it. A provider only ever sees its own value part, so identifiers from different providers can never collide in the cookie, the graph, or a withdrawal, and every identifier records which provider created it. @@ -19,12 +19,12 @@ The class of provider expected to grow this table is one that consumes a web-browser-supplied unique identifier, arriving either as a new web platform feature or from a user-installed extension, delivered to the edge through the client-cycle resolve path and verified by the provider before -minting. +creating. -| Code | Provider | Allocated | Status | -| ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | -------------------------- | -| `hmac` | Built-in HMAC EC provider. Mints `hmac~<64 hex>.<6 alnum>`; dual-reads its pre-envelope bare form so deployed cookies keep working (retirement condition in `provider_owns_id`) | 2026-08-02 | active | -| `hs00` | Built-in host-signal EC provider (opt-in; TLS JA4 plus HTTP/2 signals plus client IP) | 2026-08-25 | active | -| `cfix` | Client-fixed demonstration provider (compiled only behind the `client-fixed-demo` cargo feature) | 2026-08-25 | active, test and demo only | -| `51dd` | 51Degrees Identifier (51Did) vendor provider | 2026-08-25 | reserved | -| `t0..` | Prefix family reserved for in-tree test providers (`t0cc`, `t0op`, and similar); never valid in configuration | 2026-08-25 | reserved | +| Code | Provider | Allocated | Status | +| ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | -------------------------- | +| `hmac` | Built-in HMAC EC provider. Creates `hmac~<64 hex>.<6 alnum>`; dual-reads its pre-envelope bare form so deployed cookies keep working (retirement condition in `provider_owns_id`) | 2026-08-02 | active | +| `hs00` | Built-in host-signal EC provider (opt-in; TLS JA4 plus HTTP/2 signals plus client IP) | 2026-08-25 | active | +| `cfix` | Client-fixed demonstration provider (compiled only behind the `client-fixed-demo` cargo feature) | 2026-08-25 | active, test and demo only | +| `51dd` | 51Degrees Identifier (51Did) vendor provider | 2026-08-25 | reserved | +| `t0..` | Prefix family reserved for in-tree test providers (`t0cc`, `t0op`, and similar); never valid in configuration | 2026-08-25 | reserved | From b69858df241899ee11de2a110ef25f909c09e68b Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Mon, 31 Aug 2026 10:22:10 +0100 Subject: [PATCH 07/17] Remove the reserved 51Degrees provider code row and apply spec house-style fixes Delete the reserved 51dd provider-code registry row so the unpublished vendor provider is not named in the public spec. Correct section 8's count from four to seven, replace several with many, and restructure clause-joining colons into separate sentences across the provider specs. The 51Degrees contribution attribution lines are kept deliberately. --- ...26-07-30-client-cycle-ec-resolve-design.md | 52 +++++----- ...integration-response-header-hook-design.md | 2 +- .../2026-07-30-permission-model-design.md | 94 +++++++++---------- .../2026-07-30-pluggable-providers-design.md | 42 ++++----- ...07-30-provider-migration-rollout-design.md | 48 +++++----- ...-08-27-integration-provider-seam-design.md | 14 +-- .../specs/provider-code-registry.md | 7 +- 7 files changed, 129 insertions(+), 130 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index 4abab609e..63e9226d7 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -32,7 +32,7 @@ defers, and why the feature is normative rather than deferred. ## 1. Overview A **client-cycle** EC provider establishes the identifier via a browser -round trip: server-injected first-party JS obtains or derives a value in the +round trip, where server-injected first-party JS obtains or derives a value in the page (typically a signed envelope from a vendor identity system), posts it to a Trusted Server endpoint, and the endpoint, after provider-specific verification, sets the first-party `ts-ec` cookie. @@ -41,11 +41,11 @@ This differs from server-side providers in one security-critical way: **the identifier is attacker-influenceable input**, not server-derived evidence. Everything in this spec follows from that. -Why this feature is normative in the series rather than deferred: the first +This feature is normative in the series rather than deferred because the first vendor integration this project targets works client-side by design, since the page script talks to the vendor's identity system and hands the result to the edge, so the server-side path alone cannot carry it. The 2026-07-31 -draft deferred the feature for lack of a consumer; the consumer now exists +draft deferred the feature for lack of a consumer. The consumer now exists as a planned vendor provider, and v1 builds the endpoint that provider will verify against. @@ -55,7 +55,7 @@ verify against. | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Cross-site identity fixation** | `text/plain` POST is a CORS-simple request: any page on the web can `fetch(resolveUrl, {method: "POST", credentials: "include", body: payload})` with no preflight | An attacker pins a chosen identity onto a victim's first-party cookie jar, a login-CSRF for the ad-identity layer, and the victim's activity accretes to an attacker-controlled ID | | **Replay** | A captured valid payload (from the attacker's own session or a leak) replayed against another browser | Same as fixation, without needing to create payloads | -| **Phantom identity** | Endpoint sets the cookie without an identity-graph row | Later requests carry an EC that the KV graph has never seen; downstream sync and withdrawal logic operate on an identity that half-exists (the organic generation path explicitly refuses to write a cookie when the graph write fails, for exactly this reason) | +| **Phantom identity** | Endpoint sets the cookie without an identity-graph row | Later requests carry an EC that the KV graph has never seen. Downstream sync and withdrawal logic operate on an identity that half-exists (the organic generation path explicitly refuses to write a cookie when the graph write fails, for exactly this reason) | | **Un-tombstoneable identity** | Core does not recognize the provider's identifier shape | Withdrawal cannot expire or tombstone the identity, which is a compliance failure and not only a bug | | **Amplification** | The page script cannot observe an HttpOnly cookie, so it cannot know the cookie is already set | A POST on every page view of every session (PR #838's JS gated on reading a cookie its own server marked HttpOnly, making the guard permanently false) | @@ -65,12 +65,12 @@ verify against. 1. **Reject cross-site requests with an origin check.** v1 authorizes a request only when its `Origin` header names the publisher's configured - domain or a subdomain of it; a missing or foreign `Origin` is rejected + domain or a subdomain of it. A missing or foreign `Origin` is rejected with `403`. Browsers always send `Origin` on POST `fetch`, so its absence means a non-browser caller, which has no business on a page-script endpoint. The 2026-07-31 draft asked for an exact origin allowlist as new configuration plus an optional session-bound CSRF - token; v1 derives the allowed set from `publisher.domain` (existing, + token. v1 derives the allowed set from `publisher.domain` (existing, operator-trusted configuration) and admits the publisher's own subdomains, because the publisher controls their subdomain namespace, and the draft's sibling-subdomain concern applies to `Sec-Fetch-Site: @@ -80,41 +80,41 @@ same-site` (which v1 does not consult at all), not to a suffix match on configured apex. 2. **Verify the payload per provider.** The endpoint hands the posted payload to the selected provider's `resolve_from_client` and creates only - what the provider returns; whether the payload is trustworthy is the + what the provider returns. Whether the payload is trustworthy is the provider's responsibility, stated on the trait. A real vendor provider - verifies a signed, audience-bound, expiring envelope; the draft's + verifies a signed, audience-bound, expiring envelope. The draft's session-binding and replay analysis (see §3.9) is the bar that verification must clear when the vendor scheme lands. The demo provider verifies a fixed constant and is compiled out of production builds (§5). `resolve_from_client` is normative in v1, with a no-op default so server-side providers are untouched. -3. **Preserve the identity-graph invariant.** Implemented: the graph row is +3. **Preserve the identity-graph invariant.** Implemented. The graph row is written before the cookie is set, keyed by the provider's `normalize_id_for_kv` canonical form, exactly like the organic create path. No graph available → no create (`204`), same as organic generation; a graph write failure → `503`, no cookie. -4. **Round-trip through the lifecycle contract.** Implemented: read-back +4. **Round-trip through the lifecycle contract.** Implemented. Read-back goes through the selected provider's `accepts_id`, the KV key through `normalize_id_for_kv`, and withdrawal reaches the row like any other identity. A round-trip test drives an opaque client identifier through organic deferral, resolve, cookie set, and verbatim read-back. 5. **Exist on every adapter, where parity means identical behavior, - including identical refusal.** Partially implemented, documented: the + including identical refusal.** Partially implemented, documented. The Fastly adapter routes the endpoint (passing the same bot-gated identity graph as organic generation, so unrecognized clients cannot create through resolve either). The Axum, Cloudflare, and Spin adapters deliberately do not route it, matching `identify` and `batch-sync`, - which need the same platform KV wiring those adapters do not have; the + which need the same platform KV wiring those adapters do not have. The route list in the Spin adapter documents all three together. The draft's stronger ask, identical startup rejection of the client-cycle selection on adapters that cannot serve it, is the agreed follow-up when the portability adapters gain KV (§7.4). 6. **Be uncacheable and permission-gated on the provider's full - declaration.** Implemented: every response carries `Cache-Control: + declaration.** Implemented. Every response carries `Cache-Control: no-store`, and the gate is the selected provider's complete `required_permissions()` through the same resolved permission state as organic creating, not a hard-coded storage check. -7. **Bound every input.** Implemented: request body at most 65,536 bytes, where +7. **Bound every input.** Implemented. Request body at most 65,536 bytes, where an advertised `Content-Length` over the limit answers `413` before the read, and the read body is re-checked so a missing or false length does not bypass the bound. `Content-Type` allowlist: `text/plain` and @@ -132,8 +132,8 @@ no-store`, and the gate is the selected provider's complete `415` content type, `503` graph-write failure, `204` closed gate / no provider / no graph / unverified payload. Tests exercise each rejection. 8. **Define behavior against an existing identity, with no silent - replacement.** Implemented: resolving to the same identity refreshes - idempotently; resolving to a different identity while the request + replacement.** Implemented. Resolving to the same identity refreshes + idempotently. Resolving to a different identity while the request carries a recognized EC is rejected with `409`. Any legitimate re-identification flow (account link, vendor migration) is an explicit linking design this spec does not authorize (§7). @@ -146,7 +146,7 @@ no-store`, and the gate is the selected provider's complete envelope format that does not exist yet, and a CAS-class storage primitive no production adapter exposes today. Designing the reservation against a hypothetical envelope would repeat the mistake - this series exists to fix. v1's stance: the endpoint is safe without it + this series exists to fix. v1's stance is that the endpoint is safe without it for the providers v1 ships (the demo creates a constant, feature-gated out of production), and the reservation lands with the first vendor scheme, designed against its real envelope, with the draft's §3.9 as @@ -176,7 +176,7 @@ the bar for the vendor-scheme implementation: ## 4. Requirements on the page script - **The re-post guard must not depend on reading an HttpOnly cookie.** - Implemented as the draft's first option: the resolve response sets a + Implemented as the draft's first option. The resolve response sets a non-HttpOnly companion marker cookie (`ts-ecr=1`) carrying no identity, which is the only signal the page has that a resolve succeeded. The marker shares the Edge Cookie's scope and lifetime and is expired @@ -196,19 +196,19 @@ the bar for the vendor-scheme implementation: The interaction between provider-keyed bundle content and content-hash / SRI pinning remains open (§7.6). - **Any constant shared between Rust and TS is asserted equal by a test.** - Implemented: a Rust test reads the page-script source and asserts the + Implemented. A Rust test reads the page-script source and asserts the fixed word and the marker cookie name match their Rust constants, so a rename on either side fails the build instead of silently breaking the round trip. ## 5. Demo providers -Implemented as required: the `client-fixed` demonstration provider (fixed +Implemented as required. The `client-fixed` demonstration provider (fixed identifier, constant-equality verification) is compiled only behind the `client-fixed-demo` cargo feature. In a production build the settings validator rejects the selection at startup with a direct message, and the provider builder rejects it again as defense in depth. A fixed shared word -is not an identity; the demo exists to exercise verify-before-create end to +is not an identity. The demo exists to exercise verify-before-create end to end in tests and demonstrations. ## 6. Testing @@ -251,10 +251,10 @@ end in tests and demonstrations. | Draft position | v1 (PR #1046) | Why | | ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | -| Feature deferred, `resolve_from_client` de-normalized | Feature normative; trait method kept with a no-op default | The first vendor integration works client-side by design, so the endpoint is on the critical path for the series' first real provider | -| Origin check via new allowlist config + CSRF token | Publisher apex + subdomains from existing `publisher.domain`; missing/foreign `Origin` → `403` | Uses existing operator-trusted configuration; explicit allowlist stays open for multi-domain publishers (§7.5) | -| §3.9 reservation/replay machinery required before code | Deferred to the vendor scheme; draft text retained verbatim as the bar | The design needs the real envelope's unique id and session binding, and a CAS-class primitive no production adapter exposes today | -| Identical behavior or identical startup refusal, 4 ways | Fastly routes it (bot-gated graph); portability adapters documented as deliberately not routing, like identify | Same platform-KV constraint as the existing EC API routes; startup rejection follow-up recorded (§7.4) | -| Marker cookie or injected variable (design must pick) | Marker cookie (`ts-ecr`), expired with the EC cookie | The draft's own first option; testable and observable | +| Feature deferred, `resolve_from_client` de-normalized | Feature normative. Trait method kept with a no-op default | The first vendor integration works client-side by design, so the endpoint is on the critical path for the series' first real provider | +| Origin check via new allowlist config + CSRF token | Publisher apex + subdomains from existing `publisher.domain`. Missing/foreign `Origin` → `403` | Uses existing operator-trusted configuration. Explicit allowlist stays open for multi-domain publishers (§7.5) | +| §3.9 reservation/replay machinery required before code | Deferred to the vendor scheme. Draft text retained verbatim as the bar | The design needs the real envelope's unique id and session binding, and a CAS-class primitive no production adapter exposes today | +| Identical behavior or identical startup refusal, 4 ways | Fastly routes it (bot-gated graph). Portability adapters documented as deliberately not routing, like identify | Same platform-KV constraint as the existing EC API routes. Startup rejection follow-up recorded (§7.4) | +| Marker cookie or injected variable (design must pick) | Marker cookie (`ts-ecr`), expired with the EC cookie | The draft's own first option. Testable and observable | | Demo gated by cargo feature or `#[cfg(test)]` | Both: `client-fixed-demo` feature + startup rejection in the settings validator | Defense in depth | | Everything else in §3 (graph row, bounds, 409, no-store, full-declaration gate) | Implemented as specified | (none) | diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index 0182495fb..ce1631ce1 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -1128,7 +1128,7 @@ and active-scope-change rules. `security_cookie_same_site` accepts exactly emitted `Secure` attribute, and the cookie never carries `HttpOnly`. Both exposure booleans default to `false`. ClientID-to-origin additionally -requires the owner-scoped overlay capability. Host fingerprints additionally +requires the owner-scoped overlay capability. Host signals additionally require qualified JA4 availability and sign-offs 23/28. A selected adapter that cannot preserve admitted request-header field-line order or enforce the request/body limits fails startup for protection rather than synthesizing diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index 0cedf6f63..d0252acf2 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -45,12 +45,12 @@ These are the initial sources. Issues #777 and #779 also envision publisher interaction and external services as permission sources. That source interface remains **explicitly deferred**, not silently dropped. §10 records the divergence, and the documentation (`docs/guide/permission-model.md`) already -frames consent as one source among several so a later source plugs into the +frames consent as one source among many so a later source plugs into the same mechanism. Core code resolves permissions through a per-permission `ConsentSignal` closure (`Grant`, `Revoke`, `Neutral`), so a new source is a new producer of that signal, not a new resolution algorithm. -Scope: the model governs decisions Trusted Server makes. A downstream protocol +Scope. The model governs decisions Trusted Server makes. A downstream protocol receives the full regulatory context only where that protocol defines fields for it (OpenRTB consent fields, and proxy-mode forwarding of raw strings). The draft's stronger rule, that identity rows carry normalized per-permission @@ -129,7 +129,7 @@ or replaces the file and rebuilds to change policy. The file is not read at runtime. This keeps the mechanism the draft rejected, for a reason the draft's own -premise no longer supports: the draft required policy to flow through the +premise no longer supports. The draft required policy to flow through the runtime config pipeline (`ts config push`, staged activation, §5.5), and that activation apparatus does not exist. Publishing a `[permissions]` TOML section with no activation protocol would reintroduce exactly the mixed-revision and @@ -148,7 +148,7 @@ Within the embedded model, the draft's specific complaints are answered: a misspelled override key fails loudly instead of being swallowed. - Two rule keys naming the same location in different case are rejected at parse, so one spelling cannot silently overwrite another. -- Auditability lives where the draft placed it, in version control: the file +- Auditability lives where the draft placed it, in version control. The file ships in the repository, and its history is the change log. The `include_str!` path still reaches above the crate root, so the crate @@ -229,7 +229,7 @@ Format rules, as implemented: - A rule key is a bare country or a `country/region` pair. Keys are matched case-insensitively, and a region entry takes precedence over its country entry. -- The **signals** section is new relative to the draft: the TCF purpose to +- The **signals** section is new relative to the draft. The TCF purpose to Data Use mapping, the opt-out source list, and the opt-out revoke set are data in the file, so no signal-to-permission policy lives in the code. The `signals.tcf.authoritative` flag governs only whether a present TCF @@ -275,7 +275,7 @@ Not implemented. `detect_jurisdiction`, driven by the runtime lists jurisdiction source for the auction consent gate and for `ConsentContext.jurisdiction`, while the permission model resolves against `permissions.yaml` independently. The drift risk the draft named is real and -stands recorded: adding a country to one source has no effect on the other, +stands recorded. Adding a country to one source has no effect on the other, and no CI test asserts consistency between the legacy lists and the policy table. Unifying the two, with the auction gate reading a policy regime class, travels with the dispatch migration (§7.4) as deferred follow-up. @@ -318,7 +318,7 @@ sources count and what they revoke or grant. The code decides only the order. closed rather than degrading to the no-signal baseline, which under a `granted` baseline would turn garbage into a grant. It never withdraws (§4.2). An **expired** TCF record is deliberately a distinct state, not - malformed: the decoded record is cleared, the raw string is kept for + malformed. The decoded record is cleared, the raw string is kept for proxy forwarding, and acquisition proceeds as if the record were absent, so the baseline applies. 4. **Only then does a present TCF record decide the mapped Data Uses**, when @@ -363,8 +363,8 @@ outcome is also a **withdrawal** is a separate, narrower question (§4.2). ### 4.2 Withdrawal vs. absence -Withdrawal (destructive: expire the `ts-ec` cookie, write the identity-graph -tombstone) and non-grant (the permission is simply unset, EC response +Withdrawal (destructive, expiring the `ts-ec` cookie and writing the +identity-graph tombstone) and non-grant (the permission is simply unset, EC response headers stripped, nothing egressed) are distinct outcomes, never conflated. "Baseline" below means the resolved acquisition rule for `necessary.operations.storage` in the request's jurisdiction, resolved once @@ -385,7 +385,7 @@ The implemented trigger, exhaustively (nothing else withdraws): signals only, satisfying the draft's live-request constraint by construction. 2. **US-style opt-outs never withdraw.** GPC and sale opt-outs are use - restrictions: they suppress the permissions the policy revokes (EC + restrictions, which suppress the permissions the policy revokes (EC headers stripped, nothing egressed) but never trigger destruction, so lifting the opt-out restores the identity. 3. **A malformed record never withdraws.** It suppresses only (§4, step 3). @@ -394,12 +394,12 @@ The implemented trigger, exhaustively (nothing else withdraws): made a choice is never stripped of an existing identity. 5. **A policy change is not a user signal.** There are no runtime policy edits (§3.1), and a rebuild that tightens a baseline does not itself - tombstone: withdrawal still requires the affirmative refusal above on a + tombstone, because withdrawal still requires the affirmative refusal above on a live request. The draft's additional trigger, an explicit storage-withdrawal or authenticated deletion request honored in every jurisdiction, has no -implemented carrier: no such endpoint exists. It is recorded as deferred +implemented carrier, as no such endpoint exists. It is recorded as deferred (§11), and when it arrives it joins this list as a global trigger. `ec_storage_withdrawn` (in `ec/consent.rs`, surfaced as @@ -428,7 +428,7 @@ associated consistency and retention contracts. That machinery depends on storage primitives (linearizable per-key CAS, independent durability domains) the current adapters do not qualify. The 2026-07-31 draft remains the reference design for that work. Until it lands, the known gaps the -draft called out stand: cookie expiry is not fenced on the tombstone +draft called out stand. Cookie expiry is not fenced on the tombstone commit, and revocation durability is bounded by the KV store's behavior. ### 4.4 Signal normalization @@ -444,7 +444,7 @@ per-permission `ConsentSignal` closure. The implemented pipeline: 2. Resolve standalone-TCF vs GPP-embedded-TCF conflicts per the configured mode (`restrictive`, `permissive`, `newest`), preserving the pre-epic selection algorithm. -3. Apply the expiry check: a TCF record older than the configured maximum +3. Apply the expiry check, where a TCF record older than the configured maximum age has its decoded form cleared, the `expired` flag set, and its raw string preserved. Expiry is its own state, excluded from malformed-present, and resolves as absent for acquisition. @@ -459,18 +459,18 @@ so the pre-epic order stands. Recorded in §11. store (not on the `EcContext` construction path), a request carrying no consent signals falls back to the consent persisted for that EC ID, with the jurisdiction re-derived from the current request's geo. Staleness is -enforced by the store: entries are written with a TTL equal to +enforced by the store, where entries are written with a TTL equal to `max_consent_age_days`, so an entry older than a live record's allowed age has expired out of the store. A live signal always wins because the fallback is consulted only when the request carries none. The draft's declared change, running the loaded record through the full normalization -pipeline, is not implemented: the loaded record substitutes directly. The +pipeline, is not implemented. The loaded record substitutes directly. The narrow read is permission-exempt by construction, since determining storage cannot itself require storage. **Proxy mode.** Proxy mode still skips semantic decoding entirely. The draft's minimal opt-out extraction was not implemented, but the fail-open -consequence the draft feared does not arise under the permission model: a +consequence the draft feared does not arise under the permission model. A present record in proxy mode is present-but-undecoded, which blocks every baseline grant (§4, step 3), and the GPC header needs no decoding, so the GPC opt-out is honored directly. No grants are ever derived in proxy mode. @@ -545,9 +545,9 @@ than constructing the status by hand. provider that performs its own fallible lookup, and none of the providers shipped in this workspace is one. Fastly's `geo_lookup` returns an `Option` and the SDK collapses every hostcall, buffer and parse failure -into `None` before it reaches the caller; the Cloudflare provider reads -request headers, which cannot error; the Axum and Spin providers resolve -nothing at all; and `DisabledGeo`, the default whenever +into `None` before it reaches the caller. The Cloudflare provider reads +request headers, which cannot error. The Axum and Spin providers resolve +nothing at all. `DisabledGeo`, the default whenever `[geo] provider` is not `"platform"`, returns nothing by construction. So **a host geo outage today does not reach this floor.** It surfaces as `Ok(None)`, which is `NoLocation`, and falls back to the deployer's @@ -557,7 +557,7 @@ outage does, and should read §5.3 and the default-country guidance with that in mind. The floor becomes reachable when a vendor geo crate under `crates/geo/` does a real lookup that can fail. -At the floor, an explicit valid grant still counts: a TCF record consenting +At the floor, an explicit valid grant still counts. A TCF record consenting to a mapped purpose sets that permission under `requires_signal`, exactly the divergence-from-deny-all the draft declared for this row. Absent, malformed, or refusing evidence sets nothing. @@ -583,7 +583,7 @@ makes the dangerous migration config (a permissive `default_country` with geo unset) an explicit operator decision rather than an accident, closing the highest-severity finding of the PR #838 review. -The guard's consumer list is narrower than the draft's: the draft enumerated +The guard's consumer list is narrower than the draft's. The draft enumerated every jurisdiction consumer (EC provider, regime-gated auction dispatch, raw-EC and EID egress). In the implementation the EC provider is the only consumer whose behavior the policy gates, because auction dispatch was not @@ -593,7 +593,7 @@ dispatch joins the model, the guard's trigger list grows with it. ### 5.4 Defaults: one deployer fallback plus a protective floor `[geo] default_country` is **required in every mode** and is validated at -startup: it must be set, and it must resolve to a rule in +startup, meaning it must be set, and it must resolve to a rule in `permissions.yaml`. It accepts a country (`FR`) or a country/region key (`US/CA`), matched case-insensitively, so a no-geo single-state deployment can select its state rule. It covers two states the draft kept separate: @@ -604,7 +604,7 @@ can select its state rule. It covers two states the draft kept separate: The draft's `rules.default` policy entry does not exist, so "resolved-but-unmatched" falls to the same deployer default as "unresolved". The separation the draft treated as safety-critical is the -one the implementation does keep: a **failed** lookup never reaches the +one the implementation does keep, where a **failed** lookup never reaches the deployer default and resolves at the requires-signal floor instead (§5.2). With no default configured startup fails, and in the unreachable belt-and-braces case where resolution still finds no rule, the floor @@ -612,7 +612,7 @@ applies. ### 5.5 Policy revision activation (deferred) -Not implemented, and currently moot: policy is compiled into the binary +Not implemented, and currently moot. Policy is compiled into the binary (§3.1), so the deployed artifact is the policy identity and there is no runtime activation to coordinate. The draft's activation design, covering the JCS-canonical policy digest and ordinal pair, the activation register @@ -628,17 +628,17 @@ requires a live user signal (§4.2), never a policy change. | Condition | Resolution behavior | | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Geo lookup reports a failure | Requires-signal floor for every permission, never the deployer default; error logged. No provider shipped today can report one, so a host geo outage lands on the row above instead (§5.2) | +| Geo lookup reports a failure | Requires-signal floor for every permission, never the deployer default, with the error logged. No provider shipped today can report one, so a host geo outage lands on the row above instead (§5.2) | | No geo provider configured | `default_country` baseline, guarded by `assume_single_jurisdiction` (§5.3) | | Country resolved, no matching rule | `default_country` baseline (§5.4) | | Region resolved, no region rule | Country rule | | `default_country` unset or names no rule | Startup failure (§3.3) | | EC provider configured, no geo, no acknowledgment | Startup failure (§5.3) | -| Malformed `permissions.yaml` | Parse error at settings load, once per instance; the embedded file is a build-time constant, never per-request | -| Undecodable record present (TCF, GPP, or USP) | Revokes every Data Use (fail-closed acquisition); never withdraws; opt-outs still honored | -| Expired TCF record | Distinct state, not malformed; treated as absent, so the baseline applies | +| Malformed `permissions.yaml` | Parse error at settings load, once per instance, because the embedded file is a build-time constant, never per-request | +| Undecodable record present (TCF, GPP, or USP) | Revokes every Data Use (fail-closed acquisition), never withdraws, and still honors opt-outs | +| Expired TCF record | Distinct state, not malformed, and treated as absent, so the baseline applies | | Signals contradict (opt-out plus consent) | Opt-out wins (§4) | -| No EC provider selected | Identity fails closed: nothing created, an incoming cookie value never used or egressed (§7) | +| No EC provider selected | Identity fails closed, with nothing created and an incoming cookie value never used or egressed (§7) | The posture is fail-closed. Every ambiguous state resolves to the configured baseline or more restrictive. @@ -653,7 +653,7 @@ Consumers of the resolved set in the implementation: every declared permission is set (`ec_allowed`). A provider that requires nothing always runs. **Geo** is ungated because gating it is circular, jurisdiction being an input to permission resolution. - **Device** is ungated by a separate, deliberate decision: its + **Device** is ungated by a separate, deliberate decision, because its security-classification role must run for traffic that has granted nothing, and operator selection is the recorded authorization (providers spec §5). The built-in Edge Cookie providers declare @@ -661,7 +661,7 @@ Consumers of the resolved set in the implementation: 2. **EC lifecycle.** Creation requires the provider's declared permissions through the gate above. Withdrawal follows §4.2. Recognition and - revocation of an existing identifier are never permission-gated: the + revocation of an existing identifier are never permission-gated, and the withdrawal path runs precisely when `ec_allowed` is false, reading the raw cookie value kept for that purpose. @@ -678,11 +678,11 @@ Consumers of the resolved set in the implementation: | OpenRTB `user.id` on the `/auction` endpoint | `ec_sharing_allowed` | | Identify endpoint (partner-facing) | `ec_sharing_allowed` | | Pull sync (browser-request-scoped) | `ec_sharing_allowed`, from the live request resolution | - | Batch sync (context-free S2S) | Authenticated; withdrawn or missing rows are ineligible | + | Batch sync (context-free S2S) | Authenticated, and withdrawn or missing rows are ineligible | | KV EID resolution for auctions | `ec_allowed`, then the EID pair gate on the result | | Publisher navigation and page-bids `user.id` | `ec_sharing_allowed` (the storage plus personalised-ads pair) | - With **no EC provider configured**, identity fails closed: the gate is + With **no EC provider configured**, identity fails closed, meaning the gate is closed rather than open by default (`ec_allowed` is false), so a cookie value present on the request is treated as absent and never used or egressed. This replaces PR #838's vacuously-true `is_none_or` check. @@ -702,7 +702,7 @@ Consumers of the resolved set in the implementation: 4. **Server-side auction dispatch (not migrated).** Dispatch is still gated by the consent subsystem (`consent_allows_server_side_auction`), not by - the permission model: when the jurisdiction is GDPR or unknown, or an EU + the permission model. When the jurisdiction is GDPR or unknown, or an EU TCF signal is present, dispatch requires an effective TCF record consenting to Purpose 1, and otherwise no bid request leaves (a no-bid response, with no PBS/APS call and no UA/IP/geo forwarding). Known @@ -715,7 +715,7 @@ Consumers of the resolved set in the implementation: (§11) together with §3.4. The client-cycle resolve endpoint (`/_ts/api/v1/ec/resolve`) is a further -consumer: a provider-derived identifier posted by the page is accepted only +consumer, where a provider-derived identifier posted by the page is accepted only through the same provider and permission gates. ### 7.1 Contextual OpenRTB v1 allowlist (deferred) @@ -741,8 +741,8 @@ Implemented, in `permissions.rs`, `ec/consent.rs`, `ec/mod.rs`, destroying, consent is not a withdrawal, GPC alone never withdraws, sale opt-outs never withdraw, no signal never withdraws, malformed never withdraws. -- **Fail-closed acquisition.** Malformed records block baseline grants; - each undecodable record family is detected; an expired TCF record is not +- **Fail-closed acquisition.** Malformed records block baseline grants, + each undecodable record family is detected, and an expired TCF record is not treated as malformed and resolves at the baseline. - **Geo status.** A failed lookup resolves at the requires-signal floor (permissions and the storage baseline both), driven through a @@ -790,10 +790,10 @@ acceptance contract, not two: | #779 says | This spec says | Why | | -------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | -| Unmatched countries fall to `default_country` | Adopted: unmatched and unresolved requests both fall to the required `[geo] default_country`; a **failed** lookup floors instead (§5) | The failure state is the one that must never reach a permissive default; the draft's `rules.default` split was not kept | -| The full TCF purpose vocabulary is modeled | Adopted and extended: all eleven purposes are signal-resolved, and the full Privacy Taxonomy is carried as declared baseline (§2) | The joint taxonomy work made whole-taxonomy declaration the goal; `denied` defaults keep undeclared uses inert | -| Policy is an embedded file | Adopted: `permissions.yaml` is compiled into the build (§3.1); runtime configuration is deferred follow-up | The runtime push and activation pipeline does not exist; version control is the audit trail meanwhile | -| Permission sources are open-ended (#777: publisher interaction, external services may grant) | Sources are jurisdiction, policy, and the §4 signals; further sources are deferred, and the `ConsentSignal` closure is their seam (§1) | Shipping an interface with no second source repeats the inert-surface mistake; the extension seam is defined | +| Unmatched countries fall to `default_country` | Adopted. Unmatched and unresolved requests both fall to the required `[geo] default_country`, and a **failed** lookup floors instead (§5) | The failure state is the one that must never reach a permissive default, and the draft's `rules.default` split was not kept | +| The full TCF purpose vocabulary is modeled | Adopted and extended. All eleven purposes are signal-resolved, and the full Privacy Taxonomy is carried as declared baseline (§2) | The joint taxonomy work made whole-taxonomy declaration the goal, and `denied` defaults keep undeclared uses inert | +| Policy is an embedded file | Adopted. `permissions.yaml` is compiled into the build (§3.1), and runtime configuration is deferred follow-up | The runtime push and activation pipeline does not exist, and version control is the audit trail meanwhile | +| Permission sources are open-ended (#777: publisher interaction, external services may grant) | Sources are jurisdiction, policy, and the §4 signals. Further sources are deferred, and the `ConsentSignal` closure is their seam (§1) | Shipping an interface with no second source repeats the inert-surface mistake, and the extension seam is defined | ## 11. Revision record vs the 2026-07-31 draft @@ -803,11 +803,11 @@ PR #1045). | Draft position | Implemented position | Why | | --------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Vocabulary is two TCF-purpose identifiers, enforced permissions only (§2) | IAB Privacy Taxonomy Data Uses: eleven named purposes all signal-resolved, plus 53 taxonomy Data Uses carried as declared but unenforced baseline flags | The joint taxonomy adoption postdates the draft; whole-taxonomy declaration serves completeness and demonstration, with `denied` defaults keeping unenforced flags inert | -| Policy lives in `[permissions]` in `trusted-server.toml`, published via `ts config push` (§3.1) | Policy is `permissions.yaml`, compiled into the build with `include_str!`, parsed once and covered by tests | The runtime config push and activation apparatus does not exist; publishing runtime policy without it would recreate the hazards the draft cataloged; runtime policy is deferred follow-up | -| Every group carries a required `regime` class read by auction dispatch (§3.2) | No regime field exists | Its only consumer, regime-gated dispatch, was not migrated; the field returns with that work | -| Overrides name explicit acquisition rules, replacing the `+`/`-` sigils (§3.2) | Adopted: a detailed rule's `permissions` map assigns `granted`, `requires_signal`, or `denied` per Data Use | The draft's requirement, expressed in the YAML schema; `requires_signal` is now expressible per rule | -| Two fallbacks: policy `rules.default` for unmatched countries, `default_country` only for the static no-geo mode (§5.4) | One required `[geo] default_country` covers unmatched and unresolved requests in every mode, startup-validated; a failed lookup floors separately | One deployer knob is simpler; the safety-critical separation kept is failure vs absence, carried by `GeoStatus` | +| Vocabulary is two TCF-purpose identifiers, enforced permissions only (§2) | IAB Privacy Taxonomy Data Uses, with eleven named purposes all signal-resolved, plus 53 taxonomy Data Uses carried as declared but unenforced baseline flags | The joint taxonomy adoption postdates the draft, and whole-taxonomy declaration serves completeness and demonstration, with `denied` defaults keeping unenforced flags inert | +| Policy lives in `[permissions]` in `trusted-server.toml`, published via `ts config push` (§3.1) | Policy is `permissions.yaml`, compiled into the build with `include_str!`, parsed once and covered by tests | The runtime config push and activation apparatus does not exist, so publishing runtime policy without it would recreate the hazards the draft cataloged. Runtime policy is deferred follow-up | +| Every group carries a required `regime` class read by auction dispatch (§3.2) | No regime field exists | Its only consumer, regime-gated dispatch, was not migrated, and the field returns with that work | +| Overrides name explicit acquisition rules, replacing the `+`/`-` sigils (§3.2) | Adopted. A detailed rule's `permissions` map assigns `granted`, `requires_signal`, or `denied` per Data Use | The draft's requirement, expressed in the YAML schema. `requires_signal` is now expressible per rule | +| Two fallbacks: policy `rules.default` for unmatched countries, `default_country` only for the static no-geo mode (§5.4) | One required `[geo] default_country` covers unmatched and unresolved requests in every mode, startup-validated, and a failed lookup floors separately | One deployer knob is simpler, and the safety-critical separation kept is failure vs absence, carried by `GeoStatus` | | Validation checks assigned ISO 3166-1 codes, assigned subdivisions, and a group identifier grammar (§3.3) | Validation covers unknown groups, permissions, acquisitions, revoke rules, incomplete groups, case-duplicate rule keys, and unknown fields on detailed rules | Smaller surface shipped first; the EU/EEA coverage test guards the shipped table against the typo class; ISO-assignment checks are future hardening | | Three-class signal taxonomy with regime-scoped grant acceptance; the US posture is `requires_signal` with GPP/USP non-opt-out values as grants (§4) | TCF is the only grant source; the US posture is a `granted` baseline that opt-outs revoke; explicit non-opt-out values grant nothing | A simpler two-signal model without regimes; the cost, no-signal US traffic is allowed by baseline rather than blocked pending a signal, is a deliberate policy choice in the shipped file | | Malformed-present blocks grants per record family and mapped section (§4.4, §4.5) | Any present-but-undecodable record revokes every Data Use for the request | Strictly more restrictive simplification; per-family scoping needs the full §4.5 decoder work | diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index b8000564e..393293873 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -96,16 +96,16 @@ default_country = "FR" # required (section 6) ``` **The `host-signals` EC provider** (identity from HMAC over the host TLS JA4 -and HTTP/2 fingerprints plus the client IP) was deliberately dropped from the +and HTTP/2 signals plus the client IP) was deliberately dropped from the 2026-07-31 draft. It has since shipped in PR #1044 as an opt-in built-in (`[ec.providers.host-signals]`), implemented against the host-agnostic `HostSignals` capability rather than a Fastly API, so any host that supplies -the fingerprints can run it and a host that supplies none cannot build it. -When the host supplies no fingerprint at all the provider defers with a +the signals can run it and a host that supplies none cannot build it. +When the host supplies no signal at all the provider defers with a warning instead of degrading to an IP-only identifier under the host-signals name. **An open review question stands on whether this provider should ship in the series at all**, because its identifier shape shares the built-in -HMAC grammar and a sign-off row defers host fingerprint processing. The +HMAC grammar and a sign-off row defers host signal processing. The question is flagged for the series review and this spec does not present the provider as settled either way. @@ -124,7 +124,7 @@ an EC value through the selected provider: | **Create** | EC generation on first eligible request, and the client-cycle resolve endpoint | The provider returns the identifier (`generate` server-side, `resolve_from_client` for the client cycle) and only core writes the cookie, after enforcing the global bounds below. | | **Recognize** | Reading `ts-ec` back from the request, deciding `ec_was_present`, withdrawal checks, and every path that hands the value onward: the origin URL in `append_ec_id`, the click-target URL in `handle_first_party_click`, and the proxied body an integration builds | `accepts_id` answers whether a value is a well-formed identifier the provider issues. A value the selected provider does not recognize is treated as absent, so it is never used or egressed, while the raw cookie value stays visible to withdrawal handling. The egress paths reach the same answer through `edge_cookie::recognized_ec_id`, and a deployment with no provider selected recognizes nothing and so egresses nothing. | | **KV key** | Identity-graph row reads and writes | `normalize_id_for_kv` returns the key form. The default lowercases the built-in HMAC hash segment and preserves the suffix, keeping today's keys. An opaque or case-sensitive provider overrides to the identity function so distinct identifiers never collapse into one row. | -| **Withdraw** | Expiring the cookie and writing revocation markers | The identifiers eligible for a **graph tombstone** are exactly those the selected provider owns, dispatched on the `{code}~` prefix first and then `accepts_id`, never a shape check the provider cannot influence. Expiring the **cookie** is broader: it keys off the raw cookie being present, so it still fires for an identifier the selected provider does not own (see the switching case, §6.1). | +| **Withdraw** | Expiring the cookie and writing revocation markers | The identifiers eligible for a **graph tombstone** are exactly those the selected provider owns, dispatched on the `{code}~` prefix first and then `accepts_id`, never a shape check the provider cannot influence. Expiring the **cookie** is broader, because it keys off the raw cookie being present, so it still fires for an identifier the selected provider does not own (see the switching case, §6.1). | **Invariant:** for every provider `P` and every identifier `id` created by `P`, `id` round-trips read-back byte for byte. A test in `ec/mod.rs` proves @@ -152,7 +152,7 @@ straddle the envelope never count each other, and `cluster_size` under-reports for as long as both populations coexist. That undercount is accepted rather than bridged, for three reasons. -`cluster_size` is reported in the identify response and gates nothing: the +`cluster_size` is reported in the identify response and gates nothing, and the only place its value is read at all is a cache short circuit in `evaluate_cluster` that tests whether a value is stored, not what it is, and the `cluster_trust_threshold` and `cluster_recheck_secs` settings that a @@ -192,7 +192,7 @@ wrong place, because what a provider may see is not the control. What a provider may do with what it sees is the control, and that is the permission model. So `RequestInfo` carries everything the request carries, whether or not code in -this repository reads it yet. The rule stands for behavioural traits, where a +this repository reads it yet. The rule stands for behavioral traits, where a method with no caller really is dead weight. How the surface observed in PR #838 resolved in the implementation: @@ -269,9 +269,9 @@ providers cannot silently mix identity populations, and a withdrawal always acts on a key that can only belong to one provider. The built-in HMAC provider creates `hmac~<64 hex>.<6 alphanumeric>` and dual-reads its pre-envelope bare form for one release cycle so deployed cookies keep -working; the bare form belongs to hmac alone. Codes are allocated +working, and the bare form belongs to hmac alone. Codes are allocated append-only in `provider-code-registry.md`, and a leading digit is valid -(`51dd`). +(`1a2b`). The draft's alternative shape (`parse` returning a typed `EcId`, `graph_key_suffix`, `cluster_prefix`, `verify`, and a version-carrying @@ -328,14 +328,14 @@ structural: selection defaults.** Device classification is not an input to permission resolution. The neutral `builtin` classifier reads only the User-Agent and makes no host call. The draft went further and made selecting a - fingerprint-reading device provider a startup error pending a separate + host-signal-reading device provider a startup error pending a separate security design. The implementation instead ships `[device] provider = "fastly"` as a selectable opt-in. The Fastly adapter injects a - `HostSignals` service carrying the TLS JA4 and HTTP/2 fingerprints, and + `HostSignals` service carrying the TLS JA4 and HTTP/2 signals, and the provider uses them to strengthen the browser/bot gate that guards EC writes. Identity rows persist the derived classification fields (the JA4 class segment and a 12-hex-character hash prefix of the HTTP/2 SETTINGS - fingerprint), not raw fingerprints, and the neutral default persists + signal), not raw signals, and the neutral default persists neither because the builtin provider produces no such fields. ## 6. Selection, validation, and failure modes @@ -344,7 +344,7 @@ All configuration validation happens at **settings construction** (`Settings::finalize_deserialized` runs every check below), so a misconfiguration expressible in configuration alone is a startup error, never a silent behavior change. A selection that only the running host can -satisfy (an injected vendor provider, or host fingerprints) fails loudly +satisfy (an injected vendor provider, or host signals) fails loudly when the provider is built, stopping the request rather than degrading. | Configuration state | Behavior | @@ -354,7 +354,7 @@ when the provider is built, stopping the request rather than degrading. | `provider = "none"` (explicit stateless) | Valid, and means exactly what omitting the selector means. Any configured provider block alongside it is a startup error, the same stray-block rule as below. | | A configured `[ec.providers.]` block that is not the selected one | **Startup error** (checked for the `hmac` block and every vendor block). An unreferenced block is almost always a mistyped selector or a stale block, and accepting it silently invites configuration drift. | | A selected vendor key whose provider the adapter did not inject | Loud failure when the provider is built, naming the key, so the deployment never silently runs stateless. | -| `provider = "host-signals"` on a host that supplies no fingerprints | Loud failure when the provider is built. A host that cannot produce `HostSignals` cannot run the provider. | +| `provider = "host-signals"` on a host that supplies no signals | Loud failure when the provider is built. A host that cannot produce `HostSignals` cannot run the provider. | | `provider = "client-fixed"` in a production build | Startup error. The demonstration provider is compiled only behind the `client-fixed-demo` cargo feature. | | No `provider`, no providers block | Valid, the neutral default for that concern. | | Deprecated `[ec] passphrase` | Migrated to `provider = "hmac"` with the passphrase in `[ec.providers.hmac]`, with a deprecation warning naming the new location. Both forms together are rejected so a half-edited file fails loudly instead of one form silently winning. | @@ -406,7 +406,7 @@ What a switch does, precisely: treated as absent. It never becomes the request's active identity, never egresses to a partner, and is rejected on the pull-sync, batch-sync and admin paths too. This is the §5 guarantee and it is the half of the - behavior that matters most: two providers' identity populations can never + behavior that matters most, which is that two providers' identity populations can never mix. - **The browser cookie.** A later withdrawal still expires the `ts-ec` cookie, because that path keys off the raw cookie being present rather @@ -443,7 +443,7 @@ logged, none silent: | `generate` returns an error | No identity this request. The organic caller logs at error level and the request proceeds stateless. No cookie is written. | | A provider creates an identifier outside the global bounds | Rejected at create, never rewritten. The organic path yields no identity. The resolve endpoint returns 400. | | Identity-graph write fails at create | The create is undone (no identifier, no cookie), with the error logged. The resolve endpoint returns 503. The next eligible request retries. | -| The host-signals provider finds no TLS/HTTP-2 fingerprints | Defers with a warning. No identity this request, and no degraded IP-only identifier is created under the host-signals name. | +| The host-signals provider finds no TLS/HTTP-2 signals | Defers with a warning. No identity this request, and no degraded IP-only identifier is created under the host-signals name. | | Geo lookup **fails** (the provider errors) | Every permission resolves to the requires-signal floor, and the failure is logged at error level. The failure is **not** papered over with the `default_country` baseline. | | Geo resolves **no location**, or a country/region with no rule | The `[geo] default_country` baseline applies. This is the configured-default case, deliberately distinct from the failure row above (`GeoStatus` in `ec/consent.rs`). | | An incoming cookie value fails the bounds at read-back | Treated as absent, with a warning naming the source. | @@ -462,12 +462,12 @@ The draft specified a delimiter-free physical key grammar with fixed-width segments, a provider-code registry, record classes for family revocation, authority state, negative-intent outbox, rowless withdrawal, and deployment metadata, wire schemas with known-answer vectors, and a per-field graph-row -contract. The provider-code registry is now implemented: codes are +contract. The provider-code registry is now implemented, with codes allocated in `provider-code-registry.md`, carried as the `{code}~` prefix of every created identifier, and therefore present in every graph key. The key grammar differs from the draft in one deliberate way, a tilde separator instead of delimiter-free fixed width, because pre-envelope bare -identifiers remain deployed and a code such as `51dd` is valid hex, so +identifiers remain deployed and a code such as `1a2b` is valid hex, so delimiter-free parsing could misread a legacy identifier during the migration window. The remainder (record classes, family revocation, authority state, outbox, rowless withdrawal, wire schemas, per-field @@ -588,7 +588,7 @@ Implemented, in the crates named: acknowledgment. - Geo builder tests showing the default selects no geo, `none` selects no geo explicitly, and `platform` selects the host implementation. -- Host-signals provider tests covering creating from fingerprints, +- Host-signals provider tests covering creating from host signals, deferring without them, and the loud failure of a selected but uninjected vendor provider. @@ -644,8 +644,8 @@ this revision describes. | Every selection key is closed and unknown keys are startup errors | Device and geo keys are closed. EC vendor keys are open. Unknown blocks are captured as raw values in core, the adapter deserializes its own block, and a selected key with no injected provider fails loudly. | Core never names a vendor, so a vendor provider adds no core change. | | Capability mismatch is a startup error at adapter wiring time | Configuration coherence fails at startup. A host-capability mismatch (missing `HostSignals`, uninjected vendor) fails loudly when the provider is built, stopping the request. | The adapter capability declaration that would move the check to startup is deferred with the capability matrix. | | A creating provider with no identity-graph store is a startup error | Not implemented. `ec_store` stays optional. The resolve endpoint refuses to create without a graph. The organic path persists rows whenever the graph is configured. | Portability adapters run without platform KV. Whether configuration should force the pairing is follow-up work. | -| `[device] provider = "fastly"` is startup-rejected pending a separate security design | Shipped as a selectable opt-in. The Fastly adapter injects `HostSignals`, the provider strengthens the browser/bot gate, and rows persist derived classes, not raw fingerprints. | Selection is an explicit operator opt-in and the neutral default makes no host fingerprint call. | -| The `host-signals` EC provider is deliberately dropped and its selection rejected | Shipped in PR #1044 as an opt-in built-in that defers with a warning when the host supplies no signals. **Open, flagged for the series review**, not settled either way. | Its identifier shape shares the HMAC grammar, and a sign-off row defers host fingerprint processing, so the review decides whether the provider ships in the series. | +| `[device] provider = "fastly"` is startup-rejected pending a separate security design | Shipped as a selectable opt-in. The Fastly adapter injects `HostSignals`, the provider strengthens the browser/bot gate, and rows persist derived classes, not raw signals. | Selection is an explicit operator opt-in and the neutral default makes no host signal call. | +| The `host-signals` EC provider is deliberately dropped and its selection rejected | Shipped in PR #1044 as an opt-in built-in that defers with a warning when the host supplies no signals. **Open, flagged for the series review**, not settled either way. | Its identifier shape shares the HMAC grammar, and a sign-off row defers host signal processing, so the review decides whether the provider ships in the series. | | Geo default flip sequenced into the later permission-model step, with an acknowledgment guard | Landed as specified in the same series, with the default of none, `default_country` required and validated against `permissions.yaml`, the `assume_single_jurisdiction` acknowledgment, and a failed lookup resolving to the requires-signal floor with error logging (`GeoStatus`, resolved in core so all adapters agree). | The permission model shipped in PR #1045, so the constraints exist where the draft required them. | | All adapters serve the full EC feature set identically | Selector behavior is identical through the shared builders and core constructors. The EC API routes (identify, batch-sync, ec/resolve) are Fastly-only, documented in the Spin route list. | The portability adapters do not yet wire platform KV, and the gap is documented rather than silent. | | Conformance suite, adapter capability matrix, delimiter-free key grammar, `verify`, `legacy_providers`, `versions` / `mint_version` | None of these are in PR #1043 or #1044. All are tracked follow-up work, deferred, not silently dropped. | The shipped seam did not need them, and each returns with the feature that gives it a production caller, per the spec's own minimalism rule. | diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md index efe904a96..5886f7f71 100644 --- a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -1,7 +1,7 @@ # Design Spec: Provider and Permission Model, Migration and Rollout **Status:** Revised against the implemented series (PRs #1043-#1047), -2026-08-25. The §8 sign-off rows remain the series' decision ledger; rows the +2026-08-25. The §8 sign-off rows remain the series' decision ledger, and rows the implementation now satisfies are marked with their PR so the task force can ratify rather than re-litigate. **Author:** Engineering @@ -124,16 +124,16 @@ for the follow-up work that will implement them. | 3a | US-state request, explicit not-opted-out GPP/USP value → EC allowed | No US signal grants anything. N/A, absent, reserved, and unknown values grant nothing | Implemented (#1045) in a stricter form than drafted, where signals only revoke, so the drafted "explicit not-opted-out may grant" class does not exist (sign-offs 3, 17) | | 3b | US-state request, TCF record refusing Purpose 1, no US opt-out → no EC | Refusal beats coexisting non-TCF grant signals | Implemented (#1045), where an authoritative TCF refusal revokes its mapped uses. Under the shipped `granted` US baseline the refusal suppresses without tombstoning | | 3c | Consent-record conflict modes, expiry, KV fallback, proxy mode | Per the permission spec §4.4 matrix, whose changed row is that malformed-present blocks acquisition | Malformed-present fail-closed is implemented (#1045). The rest of the consent normalization pipeline is carried forward unchanged by the series | -| 3d | Valid plus expired consent records: conflict resolution can select the expired record | Expired sources drop before conflict resolution | Open. Not addressed by the series | +| 3d | Valid plus expired consent records, where conflict resolution can select the expired record | Expired sources drop before conflict resolution | Open. Not addressed by the series | | 3e | Only the GPP sale field (and USP) is consulted, with sharing/targeted opt-outs ignored | Sale, sharing, and targeted-advertising opt-outs deny the personalised-ads uses, and none affects storage or destroys identity | Partially implemented (#1045), where sale, USP, and Sec-GPC are honored and never destructive. The sharing and targeted-advertising GPP fields are not yet decoded (sign-off 3 open) | | 3f | Non-privacy-state US traffic (for example Wyoming) is non-regulated → EC allowed | Country-level `US` is a protective floor, region rules may be stricter, and regionless traffic never degrades to non-regulated | Implemented (#1045), where `permissions.yaml` maps country `US` to `us-opt-out`, with state rows able to override | -| 3g | Graph rows persist JA4 class, H2 fingerprint hash, and buyer-facing quality metadata | The draft discontinued these for new rows | **Contradicted by the series, flagged.** Rows still persist device signals, including fingerprint hash prefixes, when the opt-in providers run (#1044/#1046). Tied to the open host-signal question (sign-off 22) | +| 3g | Graph rows persist JA4 class, H2 signal hash, and buyer-facing quality metadata | The draft discontinued these for new rows | **Contradicted by the series, flagged.** Rows still persist device signals, including signal hash prefixes, when the opt-in providers run (#1044/#1046). Tied to the open host-signal question (sign-off 22) | | 4 | UK request, no TCF record → no EC | Same, unless the policy deliberately adopts a `granted` storage baseline for GB with citation and sign-off | **The shipped default adopts `granted` storage for GB (#1045), flagged.** The citation and sign-off the draft required do not exist yet. The task force owns this row | | 5 | No country resolvable (geo failure) → no EC (fail-closed) | Protective failure profile, where permissions resolve at the requires-signal floor and `default_country` is reserved for unmatched requests in acknowledged static-jurisdiction mode | Implemented (#1045), where a failed lookup resolves at the requires-signal floor, logged at error level, and never falls back to `default_country` (sign-off 18) | | 6 | Non-regulated country, TCF record refusing Purpose 1 → EC still created, identity never tombstoned | Refusal blocks new grants everywhere, and existing identity is never tombstoned where the baseline is `granted` | Implemented (#1045), where an authoritative refusal revokes its mapped uses everywhere and withdrawal is scoped to non-granted baselines | | 7 | Country resolved but in no regulation list → EC created, EIDs pass through | Governed by the deployment's default rule. The implementation expresses this as the required `[geo] default_country`, naming the `permissions.yaml` rule for unmatched requests | Implemented (#1045) with a changed mechanism, since no `rules.default` entry exists and `default_country` is required and validated at startup | | 8 | Opt-out signal outside US states → ignored today | Mapped use restrictions are honored globally, and opt-outs never tombstone identity | Implemented (#1045), where the signal mapping is jurisdiction-free and suppresses even TCF-consented uses, without destruction (sign-off 1) | -| 9 | Fastly bot gate requires JA4 plus platform class before KV-backed EC writes | The draft deferred host fingerprinting and startup-failed `[device] provider = "fastly"` | **Contradicted by the series, flagged.** The `fastly` device provider ships opt-in with `builtin` (UA-only) as the default (#1044). Whether the host-signal surface stays is sign-off 22, open | +| 9 | Fastly bot gate requires JA4 plus platform class before KV-backed EC writes | The draft deferred host signal processing and startup-failed `[device] provider = "fastly"` | **Contradicted by the series, flagged.** The `fastly` device provider ships opt-in with `builtin` (UA-only) as the default (#1044). Whether the host-signal surface stays is sign-off 22, open | | 10 | Fastly always resolves geo per request | Only with `[geo] provider = "platform"`. The neutral default flips only together with the permission model's jurisdiction guard, never in an intermediate step | Implemented as sequenced (#1044 kept the platform default, and #1045 flipped geo off by default together with `default_country` and the acknowledgment guard) | | 11a | Raw EC egress on jurisdiction-gated paths today (`user.id`, EIDs, identify, pull sync) | Gated by the sharing pair (storage plus personalised-ads), at least as strict as today for every path | Implemented (#1045), where `ec_sharing_allowed` gates `user.id`, the identify response, and pull sync, and `gate_eids_by_permissions` gates EIDs, all on the same pair | | 11b | Proxy / click / Testlight forwarding extract the raw EC cookie/header without today's jurisdiction gate | Gated by the egress inventory (both purposes) | Open. Not implemented by the series (sign-off 8) | @@ -201,7 +201,7 @@ passphrase = "replace-with-32-plus-byte-random-secret" Requirements, each marked with its implementation state: -1. **The transition has a dual-read release; loud rejection comes one +1. **The transition has a dual-read release, and loud rejection comes one release later.** Implemented (#1043) in the simple form, where the current release accepts the old shape, mapping `[ec] passphrase` to the `hmac` provider internally and logging a deprecation warning per @@ -461,38 +461,38 @@ the code with the row open. | # | Recommended resolution | Where | Decision record | Status | | --- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 1 | Honor mapped use opt-outs globally. Destructive identity effects are limited to explicit storage withdrawal, authenticated deletion, or a qualifying live TCF Purpose 1 refusal. | permission §4, §4.2 | (none) | open. Implemented by #1045 (jurisdiction-free suppression, withdrawal only on a live TCF storage refusal under a non-granted baseline). Ratify | -| 2 | GPP/USP sale opt-outs suppress the personalised-ads uses only; they neither revoke storage nor delete the identity. | permission §4.5 | (none) | open. Partially implemented by #1045 (no opt-out deletes or destructively revokes). The shipped default maps opt-outs to `revokes: all` rather than personalised-ads only, a `permissions.yaml` choice to ratify | -| 3 | Sharing/targeted-advertising opt-outs suppress the personalised-ads uses; an explicit applicable not-opted-out value may grant them. Neither affects storage. | permission §4.5 | (none) | open. Not implemented (the sharing and targeted-advertising GPP fields are not decoded, and no US signal grants anything in the shipped model) | +| 2 | GPP/USP sale opt-outs suppress the personalised-ads uses only. They neither revoke storage nor delete the identity. | permission §4.5 | (none) | open. Partially implemented by #1045 (no opt-out deletes or destructively revokes). The shipped default maps opt-outs to `revokes: all` rather than personalised-ads only, a `permissions.yaml` choice to ratify | +| 3 | Sharing/targeted-advertising opt-outs suppress the personalised-ads uses. An explicit applicable not-opted-out value may grant them. Neither affects storage. | permission §4.5 | (none) | open. Not implemented (the sharing and targeted-advertising GPP fields are not decoded, and no US signal grants anything in the shipped model) | | 4 | US auction dispatch may continue while personalised-ads is unset only through permission §7's positive `ContextualAuctionView` and its sole normative inline manifest. | permission §7.1 | (none) | open. The strip half is implemented by #1045 (EIDs and `user.id` withheld when the pair is unset). The positive contextual projection is not implemented | -| 5 | Country-only and regionless US traffic use a protective country-wide `us-opt-out` floor; state rules may be stricter. | permission §3.4 | (none) | open. Implemented by #1045 (`permissions.yaml` maps country `US` to `us-opt-out` with state overrides available). Ratify | -| 6 | Raw regulatory strings reach only the positively registered OpenRTB field that requires each source; all other destinations default deny. Identity rows retain normalized provenance/digests, not raw consent snapshots. | permission §7; providers §6.3 | (none) | open. Not addressed by the series | +| 5 | Country-only and regionless US traffic use a protective country-wide `us-opt-out` floor. State rules may be stricter. | permission §3.4 | (none) | open. Implemented by #1045 (`permissions.yaml` maps country `US` to `us-opt-out` with state overrides available). Ratify | +| 6 | Raw regulatory strings reach only the positively registered OpenRTB field that requires each source. All other destinations default deny. Identity rows retain normalized provenance/digests, not raw consent snapshots. | permission §7; providers §6.3 | (none) | open. Not addressed by the series | | 7 | Reject legacy batch-sync traffic until live-browser provenance backfill makes the row re-evaluable. | rollout §6 item 6; permission §7 | (none) | open. Not implemented (no provenance exists to recompute) | | 8 | Gate proxy, click, and Testlight identity forwarding on the sharing pair (storage plus personalised-ads). | §2 row 11b | (none) | open. Not implemented by the series | -| 9 | Defer integration-owned cookie operations from the v1 response hook; require a complete read/use/withdraw lifecycle before admission. | hook §3 | (none) | open. Overtaken (#1047 removed the whole hook from the series, so no cookie surface shipped). The deferral returns with the hook's first consumer | -| 10 | Do not create a blanket session-cookie exemption; every cookie must be covered by an approved permission or narrowly defined security-use authority. | hook §3 | (none) | open. Hook not shipped, unaffected | +| 9 | Defer integration-owned cookie operations from the v1 response hook, and require a complete read/use/withdraw lifecycle before admission. | hook §3 | (none) | open. Overtaken (#1047 removed the whole hook from the series, so no cookie surface shipped). The deferral returns with the hook's first consumer | +| 10 | Do not create a blanket session-cookie exemption. Every cookie must be covered by an approved permission or narrowly defined security-use authority. | hook §3 | (none) | open. Hook not shipped, unaffected | | 11 | Require a durable per-family negative-intent outbox in a failure domain independent of its strong target and checked freshly by every identity consumer, with a globally visible breaker over positive identity operations when neither can commit. | permission §4.3 | (none) | open. Not implemented (part of the durable-suppression follow-up) | | 12 | Adapters that cannot meet the revocation-storage contract migrate stateless rather than weakening the contract. | rollout §6 item 2; recipe §5 | (none) | open. Direction implemented (`provider = "none"` spells stateless in #1043; the identity endpoints were already Fastly-only before the series, and #1046 keeps `resolve` on the same footing). The formal capability gate is deferred. Ratify the direction | -| 13 | Keep batch sync fail-closed at cutover; stage partner communication and cleanup using explicit coverage thresholds, windows, and pause actions. | rollout §6 item 6 | (none) | open. Not implemented (no provenance cutover exists yet) | -| 14 | Policy tightening does not reinterpret historical refusal as a destructive event; destructive withdrawal requires fresh, live qualifying evidence. | permission §4.2 trigger 2 | (none) | open. Implemented by #1045 (withdrawal evaluates the live request's TCF record only, and historical records are never reinterpreted). Ratify | -| 15 | Descope the client cycle and `rewrite_legacy`; ship the v1 integration hook as headers-only. | client spec status; providers §6.1; hook §3 | (none) | open. Overtaken (the client cycle shipped hardened as #1046 instead of descoped, `rewrite_legacy` does not exist, and the hook shipped not at all per #1047). Re-decide against the shipped shape | +| 13 | Keep batch sync fail-closed at cutover, and stage partner communication and cleanup using explicit coverage thresholds, windows, and pause actions. | rollout §6 item 6 | (none) | open. Not implemented (no provenance cutover exists yet) | +| 14 | Policy tightening does not reinterpret historical refusal as a destructive event. Destructive withdrawal requires fresh, live qualifying evidence. | permission §4.2 trigger 2 | (none) | open. Implemented by #1045 (withdrawal evaluates the live request's TCF record only, and historical records are never reinterpreted). Ratify | +| 15 | Descope the client cycle and `rewrite_legacy`, and ship the v1 integration hook as headers-only. | client spec status; providers §6.1; hook §3 | (none) | open. Overtaken (the client cycle shipped hardened as #1046 instead of descoped, `rewrite_legacy` does not exist, and the hook shipped not at all per #1047). Re-decide against the shipped shape | | 16 | Persist use-opt-out suppression until ordered explicit authorization for that use or identity deletion, with TCF `LastUpdated` or an authenticated monotonic revision proving order. | permission §4.3 | (none) | open. Not implemented (suppression in the series is request-scoped with no durable record) | | 17 | N/A, absent, reserved, unknown, and unsupported values never grant processing. | permission §4.5 | (none) | open. Implemented by #1045 in a stricter form (signals only revoke, so no US-signal value grants anything). Ratify | | 18 | A selected geo provider's lookup failure uses the compiled-in protective profile; `default_country` is only for acknowledged static-jurisdiction mode. | permission §5.2 | (none) | open. Implemented by #1045 (failed lookups resolve at the requires-signal floor, logged at error level, never `default_country`). Ratify | -| 19 | Use immutable version-addressed whole-config publication plus prepare/commit activation of the complete tuple, with authenticated fleet membership, bounded admission lease, quiescence, and an activation journal; a second unanimous model transition advances model epoch, minimum binary generation, and row schema floor atomically. | permission §5.5; rollout §6.1 | (none) | open. Not implemented (`ts config push` publishes and validates without the activation protocol) | -| 20 | N+1 keeps v1 creation and pre-epic live gating, reads/enforces N+2 negative state for rollback safety, and never originates durable use suppression; new-shape settings alone do not activate the new writer/model. | migration §4.4 | (none) | open. Overtaken in part (the shipped migration is a one-release dual-read of `[ec] passphrase` in #1043 with mixed forms rejected and nothing durable added, so the full interim waits for the durable design) | -| 21 | Expire and re-create rowless legacy cookies without continuity; a prefix match cannot authenticate the cookie suffix. | providers §5 | (none) | open. Not implemented (no rowless classification exists, and a cookie with no row is never shared) | -| 22 | Defer host JA4/H2 fingerprint processing to a separate approved design; reject `[device] provider = "fastly"` at startup and do not persist fingerprint-derived classifications. | providers §5 | (none) | open. **Contradicted by the series and flagged for review** (#1044 ships the `fastly` device provider and the host-signal EC provider opt-in, and device signals including fingerprint hash prefixes persist in rows) | +| 19 | Use immutable version-addressed whole-config publication plus prepare/commit activation of the complete tuple, with authenticated fleet membership, bounded admission lease, quiescence, and an activation journal. A second unanimous model transition advances model epoch, minimum binary generation, and row schema floor atomically. | permission §5.5; rollout §6.1 | (none) | open. Not implemented (`ts config push` publishes and validates without the activation protocol) | +| 20 | N+1 keeps v1 creation and pre-epic live gating, reads/enforces N+2 negative state for rollback safety, and never originates durable use suppression. New-shape settings alone do not activate the new writer/model. | migration §4.4 | (none) | open. Overtaken in part (the shipped migration is a one-release dual-read of `[ec] passphrase` in #1043 with mixed forms rejected and nothing durable added, so the full interim waits for the durable design) | +| 21 | Expire and re-create rowless legacy cookies without continuity. A prefix match cannot authenticate the cookie suffix. | providers §5 | (none) | open. Not implemented (no rowless classification exists, and a cookie with no row is never shared) | +| 22 | Defer host JA4/H2 signal processing to a separate approved design. Reject `[device] provider = "fastly"` at startup and do not persist signal-derived classifications. | providers §5 | (none) | open. **Contradicted by the series and flagged for review** (#1044 ships the `fastly` device provider and the host-signal EC provider opt-in, and device signals including signal hash prefixes persist in rows) | | 23 | Permit a narrow `SecurityUse` authority for DataDome only, with the exact bounded surface the hook spec defines. | hook §4a; permission §7 | (none) | open. Hook not shipped, unaffected | -| 24 | Malformed/absence suppression overrides a permissive baseline but clears on newer valid evidence; it is not sticky like an explicit use opt-out. | permission §4.3, §4.1 | (none) | open. Implemented by #1045 by construction (the fail-closed block is request-scoped, so newer valid evidence re-resolves). Ratify together with row 16's durable design | -| 25 | Enforce a stored-jurisdiction/provenance horizon for batch sync: moves into stricter regimes fail closed by the horizon, and moves out require a live visit. | permission §7 | (none) | open. Not implemented | +| 24 | Malformed/absence suppression overrides a permissive baseline but clears on newer valid evidence. It is not sticky like an explicit use opt-out. | permission §4.3, §4.1 | (none) | open. Implemented by #1045 by construction (the fail-closed block is request-scoped, so newer valid evidence re-resolves). Ratify together with row 16's durable design | +| 25 | Enforce a stored-jurisdiction/provenance horizon for batch sync, where moves into stricter regimes fail closed by the horizon, and moves out require a live visit. | permission §7 | (none) | open. Not implemented | | 26 | Aggregate embedded GPP GPC with `Sec-GPC` by OR as a global, non-destructive use opt-out. | permission §4.5 | (none) | open. Partially implemented by #1045 (`Sec-GPC` is an honored non-destructive source). The embedded GPP GPC subfield is not read | | 27 | In proxy mode, decode only mapped opt-out fields and derive no grants. | permission §4.4 | (none) | open. Partially (the shipped model derives no grants from any US signal anywhere). The proxy-mode decode restriction is not separately implemented | | 28 | Require product and written vendor conformance approval for the reduced DataDome surface the hook spec pins. | hook §4a.2 | (none) | open. Hook not shipped, unaffected | | 29 | Accept rowless roaming-cookie expiry as a bounded residual only with telemetry, an explicit maximum lifetime, operator documentation, and a removal/sunset criterion. | providers §5 | (none) | open. Not implemented | -| 30 | Saturation blocks rowless admission for that prefix but never revokes an authenticated real row without its exact suffix; monitor NAT-cohort pressure. | providers §5 | (none) | open. Not implemented | -| 31 | Keep replay history bounded by evicting expired/grant entries first and retaining restrictive state for its full horizon; saturation never shortens a later opt-out. | permission §4.3; providers wire schema | (none) | open. Not implemented | +| 30 | Saturation blocks rowless admission for that prefix but never revokes an authenticated real row without its exact suffix. Monitor NAT-cohort pressure. | providers §5 | (none) | open. Not implemented | +| 31 | Keep replay history bounded by evicting expired/grant entries first and retaining restrictive state for its full horizon. Saturation never shortens a later opt-out. | permission §4.3; providers wire schema | (none) | open. Not implemented | | 32 | Accept official GPP sections 24-27 version 1, pin their layouts to the vendored IAB commit, and treat complete decoder/fixture support as a release prerequisite, with full vendoring evidence. | permission §4.5.1 | (none) | open. Not implemented (the shipped decoder consults the GPP sale field, with no vendored IAB corpus) | -| 33 | Treat any malformed or unsupported-version **mapped** GPP section as a global blocker for grants to the permissions its schema maps, while still honoring decodable opt-outs elsewhere and never deriving withdrawal from malformed bytes; unknown unmapped section IDs remain non-contributing. | permission §4.5 | (none) | open. Partially implemented by #1045 (a present undecodable record blocks baseline grants). The per-section mapped-blocker rule is not implemented | +| 33 | Treat any malformed or unsupported-version **mapped** GPP section as a global blocker for grants to the permissions its schema maps, while still honoring decodable opt-outs elsewhere and never deriving withdrawal from malformed bytes. Unknown unmapped section IDs remain non-contributing. | permission §4.5 | (none) | open. Partially implemented by #1045 (a present undecodable record blocks baseline grants). The per-section mapped-blocker rule is not implemented | | 34 | Permit providers whose canonical identifiers cannot fit an injective graph suffix to use the `sha256-detect` mode: domain-separated collision resistance plus stored canonical-identifier comparison, fail-closed collision handling, no overwrite/join. | providers §2, §6.3 | (none) | open. Premise revised by #1043 (identifiers are globally bounded at 256 bytes and the graph is keyed by `normalize_id_for_kv`). No `sha256-detect` mode exists | ## Revision record vs the 2026-07-31 draft @@ -500,7 +500,7 @@ the code with the row open. | Draft position | Revised position | Why | | -------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Epic #777-#781 pending sign-off, no implementation | Implemented as five stacked PRs #1043-#1047, with §8 staying the ledger and rows marked for ratification | The series shipped the seam, selection, permission model, resolve endpoint, and docs, so the task force ratifies rather than re-litigates | -| `[device] provider = "fastly"` startup-fails pending a separate fingerprinting design (§2 row 9, row 22) | The `fastly` device provider and host-signal EC provider ship opt-in, and device signals persist in graph rows. The identifier-collision defect the review found (host-signal identifiers shared the HMAC grammar and keyspace) is fixed by the mandatory provider-code envelope, so host-signal identifiers are `hs00~` namespaced; the policy question of row 22 is unchanged by the collision fix | Implementation choice, deliberately flagged as the open review question rather than presented as settled | +| `[device] provider = "fastly"` startup-fails pending a separate host-signal design (§2 row 9, row 22) | The `fastly` device provider and host-signal EC provider ship opt-in, and device signals persist in graph rows. The identifier-collision defect the review found (host-signal identifiers shared the HMAC grammar and keyspace) is fixed by the mandatory provider-code envelope, so host-signal identifiers are `hs00~` namespaced. The policy question of row 22 is unchanged by the collision fix | Implementation choice, deliberately flagged as the open review question rather than presented as settled | | US recipe: `requires_signal` with an extended grant-signal class (§2 rows 3/3a) | Shipped default: `granted` US baseline, `revokes: all` on any opt-out, and no US signal grants anything | The shipped signal model is revoke-only, which is simpler and stricter on grants, and the baseline choice is deployer-editable yaml, flagged in rows 3 and 5 | | `[permissions]` TOML policy published at runtime | `permissions.yaml` compiled into the build, no `[permissions]` block | Policy stays reviewable in version control and the partial-policy trap cannot be written | | `rules.default` worldwide default entry (§2 row 7) | Required `[geo] default_country` naming the fallback rule, validated at startup | Same role, one mechanism, loud when missing | diff --git a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md index 7e35617bc..3fbb27b70 100644 --- a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md +++ b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md @@ -44,7 +44,7 @@ the code lands. ## 1. The problem, with the code that causes it Every claim here was read from `main` at b7fcb5d4c (28 August), which the -seventh PR targets; the five series PRs do not touch these files. +seventh PR targets, and the five series PRs do not touch these files. 1. **The registry is closed.** `IntegrationRegistry::new` takes only `&Settings` and iterates a fixed table @@ -137,7 +137,7 @@ modules so a vendor crate rebuild invalidates the server-side template cache. The registry verifies a carried module's declared hash against its source when it is built, so a stale literal is a startup error rather than a stale script served under a valid-looking URL. Covering carried modules in -the template cache fingerprint means the publisher entry point needs the +the template cache hash means the publisher entry point needs the registry, so it takes the configuration and the registry as one argument rather than two. The served script keeps its cache rule, being the `?v=` query matched at serve time (there is no integrity attribute on the tag today, and @@ -271,7 +271,7 @@ Two more places every move must touch, found by mapping `main`: files included, so a vendor move that leaves its entry behind breaks the build rather than a test. The guard cannot derive its list from the registrations, because `include_str!` paths are fixed at compile time, so - this change drops the nine vendors' files from the guard instead: a module + this change drops the nine vendors' files from the guard instead, because a module crate is outside the core neutrality guarantee, and a move then deletes nothing there. - The `ts audit` command carries its own vendor table (detection patterns @@ -326,7 +326,7 @@ that the project pays for today, most recently in PR #1054. ## 8. What implementing this found A probe integration built outside `trusted-server-core` and registered -through an adapter exercised every seam end to end. Four things surfaced +through an adapter exercised every seam end to end. Seven things surfaced that reading the code did not, and they are recorded here rather than left for each vendor to rediscover. @@ -343,7 +343,7 @@ for each vendor to rediscover. crate keeps a literal beside its `include_str!`, and on a checkout that rewrites line endings the embedded file changes and the literal stops matching, which fails startup on that machine only. A generated helper - or a documented build-script recipe removes the trap; the probe pins the + or a documented build-script recipe removes the trap, and the probe pins the file's line endings and tests the literal, which every vendor would otherwise have to reinvent. 3. **A provider is resolved more than once per request.** A proxy that @@ -405,7 +405,7 @@ defines, and both should land before the first vendor is asked to use it. | 3 | A registration may carry its own browser JavaScript | Proposed | | 4 | Deploy validation moves onto the registration | Proposed | | 5 | The nine existing integrations migrate one PR each, on the schedule in §4 | Proposed | -| 6 | This change is complete in itself: after it, no vendor move needs a core change | Proposed | +| 6 | This change is complete in itself, so after it no vendor move needs a core change | Proposed | | 7 | Identity, geo and device providers are capabilities of a module registration (§3.6), the #1043 review's rule applied to all three | Proposed | | 8 | No provider is built into core: HMAC and the User-Agent-only device provider are Tech Lab-owned modules configured under `[integrations.]`, and core keeps only `none` | Proposed | | 9 | This spec and its core implementation precede #1043; 51Degrees implements the core seam, the nine vendor moves in §4 stay one PR each | Proposed | @@ -419,4 +419,4 @@ defines, and both should land before the first vendor is asked to use it. | 2026-08-28 | Corrected line references to `main` at b7fcb5d4c and added what mapping `main` found: composition of the served script moves into core (§3.2), registration enumeration and the auction-only `adserver_mock` case (§3.3), the duplicate-id gap (§3.1), the source-file guard and the `ts audit` vendor table (§4), the renderer risk (§7). | | 2026-08-28 | Recorded what implementing the seam found (§8): the operator CLI skips a vendor's deploy rules, a carried module's hash literal is fragile, providers resolve more than once per request, and one core reader still reads an APS payload. Recorded the construction-time hash check in §3.2. | | 2026-08-28 | Adopted the #1043 review's registration shape for identity and applied its rule to geo and device, with no provider built into core (§3.6, §6 item 2, §8 rows 7 to 9). Recorded the relationship to #986 and reordered the series so this spec and its implementation come first. | -| 2026-08-30 | Moved the five series design specs and the provider-code registry into this PR from PRs #1043 to #1047, so every normative document is reviewed before the code that implements it. Document content is unchanged; only this status line and this row are new. | +| 2026-08-30 | Moved the five series design specs and the provider-code registry into this PR from PRs #1043 to #1047, so every normative document is reviewed before the code that implements it. Document content is unchanged, and only this status line and this row are new. | diff --git a/docs/superpowers/specs/provider-code-registry.md b/docs/superpowers/specs/provider-code-registry.md index e6771337e..475237b20 100644 --- a/docs/superpowers/specs/provider-code-registry.md +++ b/docs/superpowers/specs/provider-code-registry.md @@ -2,17 +2,17 @@ Four-character codes (`[a-z0-9]`, zero-padded) that namespace Edge Cookie identifiers. Every EC provider MUST allocate a code here before it can -exist: the `EdgeCookieProvider::code()` trait method is mandatory, and core +exist, because the `EdgeCookieProvider::code()` trait method is mandatory, and core applies the code as the `{code}~` prefix of every identifier the provider creates, checks it at read-back, and keys the identity graph with it. A provider only ever sees its own value part, so identifiers from different providers can never collide in the cookie, the graph, or a withdrawal, and every identifier records which provider created it. -Allocation is a reviewed commit to this file; codes are immutable and never +Allocation is a reviewed commit to this file. Codes are immutable and never recycled, including for retired providers. A leading digit is valid. The tilde separator keeps parsing exact while pre-envelope identifiers remain -deployed: a legacy bare identifier contains no tilde and dual-reads under +deployed, where a legacy bare identifier contains no tilde and dual-reads under the built-in HMAC provider only. The class of provider expected to grow this table is one that consumes a @@ -26,5 +26,4 @@ creating. | `hmac` | Built-in HMAC EC provider. Creates `hmac~<64 hex>.<6 alnum>`; dual-reads its pre-envelope bare form so deployed cookies keep working (retirement condition in `provider_owns_id`) | 2026-08-02 | active | | `hs00` | Built-in host-signal EC provider (opt-in; TLS JA4 plus HTTP/2 signals plus client IP) | 2026-08-25 | active | | `cfix` | Client-fixed demonstration provider (compiled only behind the `client-fixed-demo` cargo feature) | 2026-08-25 | active, test and demo only | -| `51dd` | 51Degrees Identifier (51Did) vendor provider | 2026-08-25 | reserved | | `t0..` | Prefix family reserved for in-tree test providers (`t0cc`, `t0op`, and similar); never valid in configuration | 2026-08-25 | reserved | From 8e23e1eafc66df0d6e3d56f4c1943fc6df199d03 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Mon, 31 Aug 2026 10:27:13 +0100 Subject: [PATCH 08/17] Finish the clause-punctuation sweep on the permission model spec --- .../2026-07-30-permission-model-design.md | 30 +++++++++---------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index d0252acf2..531a38243 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -808,19 +808,19 @@ PR #1045). | Every group carries a required `regime` class read by auction dispatch (§3.2) | No regime field exists | Its only consumer, regime-gated dispatch, was not migrated, and the field returns with that work | | Overrides name explicit acquisition rules, replacing the `+`/`-` sigils (§3.2) | Adopted. A detailed rule's `permissions` map assigns `granted`, `requires_signal`, or `denied` per Data Use | The draft's requirement, expressed in the YAML schema. `requires_signal` is now expressible per rule | | Two fallbacks: policy `rules.default` for unmatched countries, `default_country` only for the static no-geo mode (§5.4) | One required `[geo] default_country` covers unmatched and unresolved requests in every mode, startup-validated, and a failed lookup floors separately | One deployer knob is simpler, and the safety-critical separation kept is failure vs absence, carried by `GeoStatus` | -| Validation checks assigned ISO 3166-1 codes, assigned subdivisions, and a group identifier grammar (§3.3) | Validation covers unknown groups, permissions, acquisitions, revoke rules, incomplete groups, case-duplicate rule keys, and unknown fields on detailed rules | Smaller surface shipped first; the EU/EEA coverage test guards the shipped table against the typo class; ISO-assignment checks are future hardening | -| Three-class signal taxonomy with regime-scoped grant acceptance; the US posture is `requires_signal` with GPP/USP non-opt-out values as grants (§4) | TCF is the only grant source; the US posture is a `granted` baseline that opt-outs revoke; explicit non-opt-out values grant nothing | A simpler two-signal model without regimes; the cost, no-signal US traffic is allowed by baseline rather than blocked pending a signal, is a deliberate policy choice in the shipped file | -| Malformed-present blocks grants per record family and mapped section (§4.4, §4.5) | Any present-but-undecodable record revokes every Data Use for the request | Strictly more restrictive simplification; per-family scoping needs the full §4.5 decoder work | -| Normalization runs expiry before conflict resolution, a declared change (§4.4) | Conflict resolution still runs before the expiry check | The reordering was not implemented; the expired state itself (distinct from malformed, absent for acquisition) was adopted | -| Persisted-KV consent flows through the full normalization pipeline with an explicit TTL comparison (§4.4) | The loaded record substitutes directly when the request carries no signals, jurisdiction re-derived; staleness is enforced by the store TTL (`max_consent_age_days`) | The store-level TTL delivers the staleness bound without a second normalization pass | -| Proxy mode gains minimal opt-out extraction (§4.4) | Proxy mode still skips decoding; a present record blocks all grants via the malformed-present rule and the GPC header opt-out is honored without decoding | The permission-layer outcome is equally or more restrictive with no new decode paths; revisit with the §4.5 decoder work | -| Withdrawal has four triggers including an explicit storage-withdrawal or authenticated deletion request (§4.2) | The TCF Purpose 1 refusal under a non-granted baseline is the only trigger; opt-outs, malformed records, absence, and policy changes never withdraw (adopted) | No deletion endpoint exists to carry the extra trigger; the narrowest destructive surface shipped first | -| §4.3 durability protocol: family records first, suppression and authority-state records, outbox, breaker, strong reads | Cookie expiry plus best-effort identity-graph tombstones per presented identifier, with failures logged | The protocol requires storage primitives (linearizable CAS, independent durability domains) the adapters do not yet qualify; deferred with the providers-spec storage work | -| §4.5 field mapping and §4.5.1 vendored registry snapshot (sharing/targeted opt-outs, embedded GPC, applicability, derived `gpp_sid`) | Opt-out sources are the GPC header, a GPP sale opt-out, and a USP sale opt-out; the revoke set is policy-declared, shipped as `all` (which also drops storage) | The full decoder and registry vendoring are their own project; the policy-declared revoke set gives deployers the scoping lever meanwhile | -| §5.5 activation: JCS policy digests, ordinals, activation register, journal, drains, admission leases | None of it exists; the built binary is the policy identity | With no runtime policy there is nothing to activate; the draft remains the reference design for the runtime-config follow-up | -| §3.4 single jurisdiction truth, and §7 dispatch gated on the policy regime with a contextual projection | Auction dispatch keeps the consent-subsystem gate (effective TCF Purpose 1 for GDPR or unknown jurisdictions); `detect_jurisdiction` and its lists remain; no contextual view | Dispatch migration is follow-up; the legacy-list drift risk the draft named still stands and is recorded rather than resolved | -| Every raw-EC egress path is pair-gated, with per-row tests and a denylist check (§7) | Pair gating is centralized in `ec_sharing_allowed` (auction endpoint `user.id`, publisher navigation and page-bids `user.id`, identify, pull sync) and `gate_eids_by_permissions` (EIDs everywhere); batch sync checks row state only | Partial adoption; aligning the remaining paths, the S2S stored-provenance authority, and the inventory tests is recorded follow-up | -| Identity rows never store raw consent strings, only normalized provenance and a digest (§1) | The identity-graph entry stores the raw TCF and GPP strings with the row | The normalized provenance schema belongs to the providers-spec storage work; until then rows carry the raw strings | -| No signals block in policy; the signal mapping is fixed in the spec | New: a `signals` section in `permissions.yaml` declares the TCF purpose map, opt-out sources, and revoke set, with `tcf.authoritative` governing only TCF's own effect | Moves signal policy from code into deployer-editable data; the flag can never let a TCF record override an opt-out, preserving §4 precedence | -| The §5.3 no-geo guard covers every jurisdiction consumer | The guard fires when an Edge Cookie provider is configured with no geo provider | The EC provider is the only policy-gated consumer today; the trigger list grows when dispatch and further egress paths join the model | +| Validation checks assigned ISO 3166-1 codes, assigned subdivisions, and a group identifier grammar (§3.3) | Validation covers unknown groups, permissions, acquisitions, revoke rules, incomplete groups, case-duplicate rule keys, and unknown fields on detailed rules | Smaller surface shipped first, and the EU/EEA coverage test guards the shipped table against the typo class. ISO-assignment checks are future hardening | +| Three-class signal taxonomy with regime-scoped grant acceptance, where the US posture is `requires_signal` with GPP/USP non-opt-out values as grants (§4) | TCF is the only grant source, the US posture is a `granted` baseline that opt-outs revoke, and explicit non-opt-out values grant nothing | A simpler two-signal model without regimes. The cost, no-signal US traffic is allowed by baseline rather than blocked pending a signal, is a deliberate policy choice in the shipped file | +| Malformed-present blocks grants per record family and mapped section (§4.4, §4.5) | Any present-but-undecodable record revokes every Data Use for the request | Strictly more restrictive simplification, and per-family scoping needs the full §4.5 decoder work | +| Normalization runs expiry before conflict resolution, a declared change (§4.4) | Conflict resolution still runs before the expiry check | The reordering was not implemented, though the expired state itself (distinct from malformed, absent for acquisition) was adopted | +| Persisted-KV consent flows through the full normalization pipeline with an explicit TTL comparison (§4.4) | The loaded record substitutes directly when the request carries no signals, jurisdiction re-derived, and staleness is enforced by the store TTL (`max_consent_age_days`) | The store-level TTL delivers the staleness bound without a second normalization pass | +| Proxy mode gains minimal opt-out extraction (§4.4) | Proxy mode still skips decoding, so a present record blocks all grants via the malformed-present rule and the GPC header opt-out is honored without decoding | The permission-layer outcome is equally or more restrictive with no new decode paths, and this is revisited with the §4.5 decoder work | +| Withdrawal has four triggers including an explicit storage-withdrawal or authenticated deletion request (§4.2) | The TCF Purpose 1 refusal under a non-granted baseline is the only trigger, and opt-outs, malformed records, absence, and policy changes never withdraw (adopted) | No deletion endpoint exists to carry the extra trigger, so the narrowest destructive surface shipped first | +| §4.3 durability protocol: family records first, suppression and authority-state records, outbox, breaker, strong reads | Cookie expiry plus best-effort identity-graph tombstones per presented identifier, with failures logged | The protocol requires storage primitives (linearizable CAS, independent durability domains) the adapters do not yet qualify, so it is deferred with the providers-spec storage work | +| §4.5 field mapping and §4.5.1 vendored registry snapshot (sharing/targeted opt-outs, embedded GPC, applicability, derived `gpp_sid`) | Opt-out sources are the GPC header, a GPP sale opt-out, and a USP sale opt-out. The revoke set is policy-declared, shipped as `all` (which also drops storage) | The full decoder and registry vendoring are their own project, and the policy-declared revoke set gives deployers the scoping lever meanwhile | +| §5.5 activation: JCS policy digests, ordinals, activation register, journal, drains, admission leases | None of it exists, and the built binary is the policy identity | With no runtime policy there is nothing to activate, and the draft remains the reference design for the runtime-config follow-up | +| §3.4 single jurisdiction truth, and §7 dispatch gated on the policy regime with a contextual projection | Auction dispatch keeps the consent-subsystem gate (effective TCF Purpose 1 for GDPR or unknown jurisdictions). `detect_jurisdiction` and its lists remain, with no contextual view | Dispatch migration is follow-up. The legacy-list drift risk the draft named still stands and is recorded rather than resolved | +| Every raw-EC egress path is pair-gated, with per-row tests and a denylist check (§7) | Pair gating is centralized in `ec_sharing_allowed` (auction endpoint `user.id`, publisher navigation and page-bids `user.id`, identify, pull sync) and `gate_eids_by_permissions` (EIDs everywhere). Batch sync checks row state only | Partial adoption. Aligning the remaining paths, the S2S stored-provenance authority, and the inventory tests is recorded follow-up | +| Identity rows never store raw consent strings, only normalized provenance and a digest (§1) | The identity-graph entry stores the raw TCF and GPP strings with the row | The normalized provenance schema belongs to the providers-spec storage work, and until then rows carry the raw strings | +| No signals block in policy, and the signal mapping is fixed in the spec | New. A `signals` section in `permissions.yaml` declares the TCF purpose map, opt-out sources, and revoke set, with `tcf.authoritative` governing only TCF's own effect | Moves signal policy from code into deployer-editable data, and the flag can never let a TCF record override an opt-out, preserving §4 precedence | +| The §5.3 no-geo guard covers every jurisdiction consumer | The guard fires when an Edge Cookie provider is configured with no geo provider | The EC provider is the only policy-gated consumer today, and the trigger list grows when dispatch and further egress paths join the model | | `default_country` is required only in the acknowledged static no-geo mode (§5.4) | Required always and startup-validated against `permissions.yaml` | It is the baseline for unmatched requests in every mode, so it must always exist | From 180060402f91b4efa96f5823da8934d19c1c78f8 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Mon, 31 Aug 2026 11:40:40 +0100 Subject: [PATCH 09/17] Finish the clause-punctuation sweep on the response-header-hook spec --- ...integration-response-header-hook-design.md | 256 +++++++++--------- 1 file changed, 128 insertions(+), 128 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index ce1631ce1..53d886923 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -1,7 +1,7 @@ # Design Spec: Integration Response-Header Hook **Status:** Not implemented in the series. PR #1047 carries the -documentation set only; the hook was removed from the earlier draft of that +documentation set only. The hook was removed from the earlier draft of that PR because it has no consumer, which is this spec's own §-rule for speculative surface. The spec is retained as the design bar for the hook when its first consumer arrives (an integration that must set response @@ -32,7 +32,7 @@ governed by this spec, §4a is the normative PR-specific delta and supersedes that baseline wherever its generic request/effect API, header ordering, session-by-header behavior, endpoint/request surface, cookie lifecycle, challenge transport, or response-pointer behavior conflicts. The baseline -continues to provide historical context; it is not a second normative source +continues to provide historical context. It is not a second normative source for those surfaces. Integrations can today rewrite request-path behavior (proxies, attribute @@ -61,18 +61,18 @@ processed. which **core validates and applies**, attributing each to its integration id. PR #838's shape handed the integration an unrestricted `&mut HeaderMap`, which makes §3's collision - policy unenforceable by construction: core cannot validate or attribute + policy unenforceable by construction, because core cannot validate or attribute writes it never sees. An API that cannot express a violation beats one that promises to catch it. - **Every adapter calls the apply point** on its outbound-response path for processed documents. The call site lives in shared response-finalization - code where one exists; where adapters finalize independently, each adapter + code where one exists. Where adapters finalize independently, each adapter gains the call and a test proving it. - **Ordering is three stages, and the last one is inviolable:** core response-header handling (EC Set-Cookie emission, EC header clearing, privacy headers) → integration operations → **final cache/privacy invariant enforcement**, which no integration operation can override. - Running the hook dead-last would be wrong: current `main` deliberately + Running the hook dead-last would be wrong, because current `main` deliberately runs cookie-cache protection _after_ arbitrary header changes, stripping surrogate caching and forcing private/no-store on any response that sets a cookie, a hook applied after that recheck could combine an appended @@ -82,7 +82,7 @@ processed. cookie rule. Core **snapshots the complete pre-hook cache restriction state**, whether the restriction came from core's own classification (processed auction HTML is marked private even when no cookie is - emitted; today's final helper returns early without `Set-Cookie`) **or + emitted, and today's final helper returns early without `Set-Cookie`) **or from the origin** (an origin-supplied `private, no-store` that core merely passed through), and the post-hook response may only be **equal or stronger** on the privacy axis: integrations can tighten @@ -92,7 +92,7 @@ processed. are orthogonal constraints (RFC 9111: `no-cache` permits shared storage subject to revalidation; `private` forbids shared storage), so "replace `private` with the stronger `no-cache`" would make a - personalized response shared-storable. The merge: each of the **six sticky directives**, `no-store`, `no-cache`, + personalized response shared-storable. Under the merge, each of the **six sticky directives**, `no-store`, `no-cache`, `private`, `must-revalidate`, `proxy-revalidate`, `no-transform`, is present-in-snapshot-or-mutation ⇒ present-in-final (this "snapshot or mutation ⇒ final" rule scopes to exactly these six), with two refinements. First, `must-understand` is the deliberate @@ -127,14 +127,14 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may a matching rule four adapters would implement identically). Merging them per-directive on unrestricted responses was a hole (an unrestricted `CDN-Cache-Control: max-age=60` could become a year), and - they are additionally stripped from any restricted response; and the + they are additionally stripped from any restricted response, and the final `Vary` is the **union of the complete snapshot `Vary` set**, origin-supplied members included, not only core-required ones, and the mutation. Ordering is normative so the final `Vary` reaches TS's own cache key, not just the wire: **mutation/invariant → final `Vary` computation → cache-key construction → body/metadata commit**, with three identity rules. Cache matching uses the **exact final publisher - request** (post-overlay view; keying from the redacted view would + request** (post-overlay view, because keying from the redacted view would collapse personalized variants). The final `Vary` name list has one cross-adapter grammar. Core collects every @@ -145,7 +145,7 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may empty member or invalid token introduced by a mutation invalidates that batch. If the reverted snapshot itself is malformed, the invariant replaces the final value with `Vary: *`, forces `no-store`, and writes no cache - artifact. `*` in either source likewise dominates every other member: the + artifact. `*` in either source likewise dominates every other member, so the normalized result is the single `*` and is uncacheable. For an ordinary list, the wire response emits one lowercase `, `-joined value, while the variant descriptor stores `vary_names` as the exact sorted JSON array of @@ -154,11 +154,11 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may A normalized name nominates one request header. Digest construction obtains **all** values for that header from the exact final publisher request after - overlay, preserving received field-line order and value octets; it does not + overlay, preserving received field-line order and value octets. It does not comma-fold, trim, or split those request values. The `` bytes in the HMAC input below are always the normalized lowercase ASCII name, and a name - is digested once even if it appeared repeatedly in `Vary`. Known-answer - normalization fixture: snapshot/mutation lines `Vary: X-Tenant , + is digested once even if it appeared repeatedly in `Vary`. In the known-answer + normalization fixture, snapshot/mutation lines `Vary: X-Tenant , Accept-Encoding` and `Vary: accept-encoding` produce wire value `accept-encoding, x-tenant` and descriptor value `["accept-encoding", "x-tenant"]`, then digest each nominated request @@ -187,10 +187,10 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may [{ "id": "", "key_base64url": "" }] }`. `keys` is sorted by `id`, contains 1..4 unique entries, rejects unknown fields, and every value decodes to exactly 32 CSPRNG bytes. An id is derived, - not operator-invented: the first 16 lowercase hex characters of - SHA-256 over the raw key bytes; a supplied mismatch or one id bound to + not operator-invented, and is the first 16 lowercase hex characters of + SHA-256 over the raw key bytes. A supplied mismatch or one id bound to different bytes is fatal. The current id must exist in the array. - Every cache entry stores its id; raw keys never enter config, cache + Every cache entry stores its id. Raw keys never enter config, cache artifacts, logs, or metrics. Startup **fails** when response caching is enabled and the keyring/current key does not resolve (digests are never computed unkeyed). @@ -207,22 +207,22 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may still-unknown ID, malformed descriptor, missing artifact, expiry, or revision mismatch is a miss for that descriptor, never a comparison under another key. Multiple matching descriptors are corruption and make the entire base - lookup a miss with a metric; index order never chooses a winner. + lookup a miss with a metric. Index order never chooses a winner. Publication writes and verifies the immutable artifact first, then CAS-adds or replaces its complete descriptor in the index. Rekeying or a changed `Vary` set inserts the new artifact/descriptor before removing the old - descriptor; a crash may leave a safely unreachable artifact or two + descriptor. A crash may leave a safely unreachable artifact or two nonmatching descriptors but cannot point to a partial artifact. Index capacity eviction removes expired descriptors first and otherwise the - least-recently-used complete descriptor; it never rewrites a digest under a + least-recently-used complete descriptor. It never rewrites a digest under a new key ID. This is the one meaning of “insert-new-then-index-update rekey” in the capability matrix and 304 rules. Rotation atomically replaces the secret entry with a new valid keyring containing the new current key **and all still-live previous keys**. Fleet - propagation may be mixed only in the safe direction: a process with the old - keyring can write/read the old id; the variant index exposes that id before + propagation may be mixed only in the safe direction, where a process with the old + keyring can write/read the old id. The variant index exposes that id before lookup, so a process seeing an unknown descriptor id refreshes the keyring once, then treats a still-unknown descriptor as a cache miss. It never guesses, probes with the current key, or computes unkeyed. A previous key may @@ -250,13 +250,13 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may are independent axes, so no single most-restrictive point exists): the response is treated as **uncacheable for the storage decision** (`no-store`-equivalent in the invariant) and any mutation batch - merging against the malformed value is rejected whole; among - well-formed values, duplicate directives keep the strongest; quoted and unquoted - forms are equivalent; conflicting `max-age` values keep the smallest; + merging against the malformed value is rejected whole. Among + well-formed values, duplicate directives keep the strongest, quoted and unquoted + forms are equivalent, conflicting `max-age` values keep the smallest, and unknown extension directives are dropped **from mutations only, while unknown directives already in the snapshot are preserved verbatim** (a downstream cache may honor a restrictive extension TS does not - recognize; dropping it would weaken origin policy, RFC 9111 §5.2.3); + recognize, so dropping it would weaken origin policy, RFC 9111 §5.2.3); `Expires` participates in the freshness bound via **RFC 9111 §4.2.1's freshness-lifetime algorithm, referenced directly**: the `Expires`-derived lifetime is `Expires − Date` (absent `Date` → @@ -270,7 +270,7 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may origin's shorter or already-expired `Expires`; and `Vary: *` is treated as uncacheable-by-shared-caches (no-store-equivalent for the invariant). Conformance fixtures cover each rule. Middle-stage placement also keeps - the earlier property: an integration mutation is not silently stripped + the earlier property. An integration mutation is not silently stripped by ordinary core handling, only by the invariant pass, which logs the downgrade it applies. @@ -294,14 +294,14 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may consent/privacy headers core emits; (b) reserved cookie _names_ within `Set-Cookie`, `ts-ec`, `ts-eids`, and the other `ts-*` cookies core owns. **Cookie operations are deferred out of the v1 hook, headers only.** The write-side gate alone ("persistent cookies require P1") was shown - insufficient: it never modeled reading, using, forwarding, or + insufficient, because it never modeled reading, using, forwarding, or withdrawing the cookie, so a P1-granted-then-withdrawn integration cookie would keep arriving on every request with nothing required to expire, hide, or stop egressing it, and an advertising-identifier cookie needs P4 the contract never expressed. Rather than ship "inside the permission model" as a claim the model does not back, `append_set_cookie` and the typed cookie builder are **deferred** to a - follow-up spec whose entry bar is: declared per-cookie required + follow-up spec whose entry bar comprises declared per-cookie required permissions, a typed authorized request-side view, stripping from unauthorized integration/proxy inputs, mandatory expiry on destructive P1 withdrawal, and startup-unique (name, domain, path) ownership. @@ -309,14 +309,14 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may construction rejects a duplicate ID (current code silently coalesces, which corrupts attribution and budgets), with a duplicate-ID test in the done-when. The registration's `behavior_revision` follows §2's bump - contract; configuration-dependent behavior is captured separately by the + contract. Configuration-dependent behavior is captured separately by the effective-config digest. Model-only activation is separate from both, so the **one cache revision tuple** contains exactly `integration_registry_revision`, `effective_config_revision`, `active_policy_digest`, `active_policy_ordinal`, `model_epoch`, `activation_generation`, and `hook_invariant_revision` from the strong active tuple at publication. Every processed artifact, mutation IR/read-set bundle, - variant descriptor, and variant-index update stores that complete tuple; a + variant descriptor, and variant-index update stores that complete tuple. A lookup, local conditional, HEAD update, or 304 replay requires byte-for-byte equality with current active. In particular, the `permissions_v2` model CAS misses every `pre_epic_v1` artifact even though config/policy bytes did not @@ -331,7 +331,7 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may constants next to the definitions they protect, not duplicated in the hook. - For non-reserved headers, the mutator API distinguishes **append** from - **replace** explicitly; append/replace legality comes from a **core-owned field registry**, + **replace** explicitly. Append/replace legality comes from a **core-owned field registry**, not adapter judgment, and the v1 registry admits **inert fields only**: "headers-only" is not automatically permission-neutral, since `Link` (preload/prefetch), `Reporting-Endpoints`/NEL, CSP report @@ -349,7 +349,7 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may merge), `Content-Language` (append), `X-Robots-Tag` (append), `Retry-After` (replace-only), `Content-Location` (replace-only); everything else known is classified reserved or rejected by the rules - above, and growing the admitted set is a spec change to this list (cookies: see the §3 deferral above). Replacing a + above, and growing the admitted set is a spec change to this list (for cookies, see the §3 deferral above). Replacing a header the origin set is a deliberate act, visible in the mutator's code. - Later registrations see earlier mutations (order = registration order, which is deterministic). @@ -362,37 +362,37 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may a **cumulative final-response budget** (≤ 128 headers / ≤ 32 KiB total, counting `name: value` plus separators, within any lower adapter ceiling, and those ceilings are the enumerated capability cells **below**, - with the counting rule fixed exactly: counted bytes = Σ over emitted + with the counting rule fixed exactly, where counted bytes = Σ over emitted fields of `len(name) + 2 + len(value) + 2` (the `": "` and CRLF separators), validated against core's budget at startup so a batch - that passes core can never fail only on one adapter; a snapshot + that passes core can never fail only on one adapter. A snapshot already **over** the core budget before any mutation rejects every ordinary batch, the budget bounds additions and never bricks an over-budget origin response, which passes through and is counted; security follows the replacement/reserve rule below) bounds the sum across integrations. The budget has normative priority - partitions: ordinary mutators may consume at most **112 headers / 24 KiB**, + partitions, where ordinary mutators may consume at most **112 headers / 24 KiB**, reserving 16 headers / 8 KiB for the core-owned security channel. Ordinary batches remain registration-ordered within their partition. A security `Continue` batch uses the reserve and, if necessary, evicts whole accepted - ordinary batches in reverse registration order until it fits; it never + ordinary batches in reverse registration order until it fits. It never removes half a batch and never drops origin fields. Ordinary output can therefore never crowd out security effects. If the immutable origin head plus the security batch alone exceeds the full budget, the security batch is rejected atomically and the request follows the documented security fail-open path with a dedicated metric, origin fields are not silently sacrificed. A security `Respond` owns a replacement - response: all ordinary mutation batches are discarded and the challenge + response. All ordinary mutation batches are discarded and the challenge is validated against the full 128-header / 32-KiB budget. A base publisher response already over the full budget still passes unchanged, but no - ordinary mutation applies; security `Continue` applies only if the final + ordinary mutation applies. Security `Continue` applies only if the final response fits after all ordinary batches are removed. These are separate outcomes and metrics. | Adapter | Header-count / total-bytes ceiling (capability cell) | | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | | Axum | no platform ceiling below the core budget (native HTTP stack), cell fixed at ≥ 128 headers / ≥ 32 KiB | - | Fastly | **qualification-pending**: the measured platform ceiling is recorded in this cell by the adapter-qualification commit; unrecorded ⇒ hook startup fails | + | Fastly | **qualification-pending**: the measured platform ceiling is recorded in this cell by the adapter-qualification commit, and unrecorded ⇒ hook startup fails | | Cloudflare | **qualification-pending**: same rule | | Spin | **qualification-pending**: same rule | @@ -400,31 +400,31 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may error (shrink the core budget or raise the ceiling, never a silent per-adapter divergence). - Hook/cache eligibility has the following concrete adapter cells; any + Hook/cache eligibility has the following concrete adapter cells, and any `qualification-pending` cell fails startup when the depending feature is selected: | Capability | Fastly | Axum (dev) | Cloudflare | Spin | | -------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- | --------------------------------------------------- | ------------------------------ | -------------------------------- | - | Runtime secret lookup for Vary-HMAC/DataDome | wired secret store; qualify key-rotation behavior | dev secret binding required | qualify Workers secret binding | qualify component secret binding | - | Persisted processed artifact + mutation IR/read sets | qualification-pending | in-process dev implementation required; non-durable | qualification-pending | qualification-pending | + | Runtime secret lookup for Vary-HMAC/DataDome | wired secret store, and qualify key-rotation behavior | dev secret binding required | qualify Workers secret binding | qualify component secret binding | + | Persisted processed artifact + mutation IR/read sets | qualification-pending | in-process dev implementation required, non-durable | qualification-pending | qualification-pending | | Atomic artifact/metadata entry commit | qualification-pending | implementation required | qualification-pending | qualification-pending | | `Vary` variant index + insert-new-then-index-update rekey | qualification-pending | implementation required | qualification-pending | qualification-pending | | DataDome field-line order, trusted IP/port, fixed HTTPS backend/no-redirect, and exact form limits | qualification-pending | qualification-pending | qualification-pending | qualification-pending | - | SecurityUse JA4 request evidence | platform value available; exact-field/payload qualification and sign-offs 23/28 pending | unavailable | unavailable | unavailable | + | SecurityUse JA4 request evidence | platform value available, with exact-field/payload qualification and sign-offs 23/28 pending | unavailable | unavailable | unavailable | The qualification commit records storage lifetime, maximum object size, concurrency semantics, torn-write behavior, and fault-injection evidence; “platform has KV” is not a qualifying cell. `expose_host_fingerprints_to_vendor = true` also requires a qualified - SecurityUse JA4 cell; unsupported or pending is a startup error, while the + SecurityUse JA4 cell. Unsupported or pending is a startup error, while the default `false` remains portable. Each mutator receives an **immutable, redacted snapshot of the response head** (status and headers as of its turn, prior integrations' - accepted operations applied) as its read context; it never holds a + accepted operations applied) as its read context. It never holds a mutable reference (§2). Redaction is a security boundary, not - tidiness: the hook runs after core queues the EC `Set-Cookie`, so an + tidiness, because the hook runs after core queues the EC `Set-Cookie`, so an unredacted view would hand a mutator the raw EC to copy into `X-Vendor-Identity` or its own cookie, walking around `AuthorizedIdentity` entirely. The snapshot therefore @@ -434,7 +434,7 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may Each registration also declares the complete set of response fields its decision may read, including status as a distinguished input. Core records the union with the accepted operation batch. Undeclared reads are a hard - conformance failure in tests; an integration unable to declare a complete + conformance failure in tests. An integration unable to declare a complete read set marks itself `revalidation = "refetch"`, which forbids IR replay after any origin metadata change. Operations arrive as **attributed batches bound to a registration @@ -443,7 +443,7 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may ordinary response mutators, one global order, core finalization → ordinary mutators → security effects → invariant pass, so the security layer's precedence over publisher-facing mutations holds through - both position and its reserved/response-owning budget rule; the current flat effects vector satisfies neither + both position and its reserved/response-owning budget rule. The current flat effects vector satisfies neither attribution nor budgets and will be restructured accordingly. Validation and budgeting are **atomic per batch**: a batch that exceeds its budget is rejected whole (logged, attributed), never partially @@ -464,15 +464,15 @@ Which responses the hook runs on, enumerated so two implementations cannot diverge silently: In this table, **persisted post-hook finals** means the cache-safe ordinary -artifact only: origin metadata plus accepted ordinary mutation IR and the +artifact only, comprising origin metadata plus accepted ordinary mutation IR and the cache/privacy invariant result, with `Set-Cookie`, core request-specific identity fields, security-channel effects, and origin validators excluded. -The security request filter evaluates every request before cache selection; a +The security request filter evaluates every request before cache selection. A fresh `Respond` bypasses the artifact, while a fresh `Continue` batch is applied to the persisted ordinary artifact and the invariant pass reruns before emission. This applies equally to ordinary hits, local conditionals, origin-revalidation 304s, and HEAD. Therefore "`Set-Cookie` is never replayed" -means never replayed from storage; a freshly validated per-request typed cookie +means never replayed from storage. A freshly validated per-request typed cookie operation may still emit on that response. Security `Respond` outputs are always `private, no-store` and never become artifacts. @@ -497,11 +497,11 @@ fixture, never a runtime wildcard. | Processed HTML document (rewritten by TS) | Yes | | Streamed processed document | Yes, operations apply to the header block before first byte | | Pass-through proxy response (not processed) | No, TS is a transparent proxy for it | -| Served-from-cache processed document | Serves the **persisted post-hook finals** captured at cache fill (revision-versioned; mismatch = miss), mutators run at fill/processing time only, so a normal hit and a conditional hit return identical policy metadata **by construction**, with no purity or determinism assumption about mutators | +| Served-from-cache processed document | Serves the **persisted post-hook finals** captured at cache fill (revision-versioned, mismatch = miss), mutators run at fill/processing time only, so a normal hit and a conditional hit return identical policy metadata **by construction**, with no purity or determinism assumption about mutators | | Redirect (3xx) | No | | Error responses TS itself generates (4xx/5xx) | No | -| `304 Not Modified` for a processed representation | **Persisted-metadata pass**: a cached processed 200 stores its final post-hook headers, accepted mutation-operation batches, and the union of every mutator's declared response-field read set (the persisted mutation IR), versioned by §3's complete cache revision tuple, including model epoch and logical activation generation. A local conditional hit re-emits persisted finals only when the complete tuple matches current active. An origin-revalidation 304 is staged and diffed against separately stored origin-side metadata. (a) Any byte-coupled field changed (`Content-Encoding`, `Content-Type`, validators, digests) → invalidate and fetch/process a full 200. (b) A changed metadata field that intersects any persisted mutator read set, or an artifact/mutator lacking a complete read-set declaration, is also unsafe → full 200 refetch and ordinary hook execution; deterministic replay of old operations cannot stand in for re-evaluating a decision made from changed inputs. (c) If every changed field is outside every declared read set and belongs to the enumerated safe-update set (`Cache-Control`, reserved CDN cache fields, `Expires`, `Date`, `Age`, `Vary`, registry-admitted mutable fields), replay the persisted deterministic operations over updated origin metadata and rerun invariants. Updated origin metadata, finals, IR, read sets, and complete revision tuple publish in one atomic entry commit; changed `Vary` uses insert-new-entry-then-index-update ordering. (d) No change → re-emit persisted finals. Artifact absence or any tuple mismatch triggers an unconditional recovery fetch so TS obtains bytes. For processed-document GET/HEAD routes, TS is explicitly the authoritative server for the transformed representation: after processing the full 200 it evaluates RFC 9110 §13 preconditions against **processed** validators; this is not evaluation of origin validators by an intermediary cache. Other methods are never eligible for this recovery path. `Set-Cookie` and origin validators are never replayed. Absent metadata → cache miss | -| `HEAD` of a processed document | **Serves the persisted GET artifact when one exists**, parity with GET/304 by construction, since RFC 9111 §4.3.5 lets HEAD metadata update a stored GET response and mutators are not required to be deterministic; with no persisted artifact, HEAD is processed like a GET (headers only) and its finals stored as a **distinct head-only artifact type that never satisfies a later GET**, when a stored GET artifact exists a HEAD may **update** it only when the comparison, made against the separately stored **origin-side** metadata, never the processed artifact's (origin and rewritten lengths differ by construction, so the processed length is never the comparand and HEAD metadata never touches processed-side headers), finds validators and origin `Content-Length` matching and no byte-coupled representation field changed (RFC 9111 §4.3.5); qualifying updates land origin-side under the same atomic-commit discipline as the 304 rules, and any mismatch **invalidates** the stored GET artifact rather than updating it | +| `304 Not Modified` for a processed representation | **Persisted-metadata pass**: a cached processed 200 stores its final post-hook headers, accepted mutation-operation batches, and the union of every mutator's declared response-field read set (the persisted mutation IR), versioned by §3's complete cache revision tuple, including model epoch and logical activation generation. A local conditional hit re-emits persisted finals only when the complete tuple matches current active. An origin-revalidation 304 is staged and diffed against separately stored origin-side metadata. (a) Any byte-coupled field changed (`Content-Encoding`, `Content-Type`, validators, digests) → invalidate and fetch/process a full 200. (b) A changed metadata field that intersects any persisted mutator read set, or an artifact/mutator lacking a complete read-set declaration, is also unsafe → full 200 refetch and ordinary hook execution, because deterministic replay of old operations cannot stand in for re-evaluating a decision made from changed inputs. (c) If every changed field is outside every declared read set and belongs to the enumerated safe-update set (`Cache-Control`, reserved CDN cache fields, `Expires`, `Date`, `Age`, `Vary`, registry-admitted mutable fields), replay the persisted deterministic operations over updated origin metadata and rerun invariants. Updated origin metadata, finals, IR, read sets, and complete revision tuple publish in one atomic entry commit, and changed `Vary` uses insert-new-entry-then-index-update ordering. (d) No change → re-emit persisted finals. Artifact absence or any tuple mismatch triggers an unconditional recovery fetch so TS obtains bytes. For processed-document GET/HEAD routes, TS is explicitly the authoritative server for the transformed representation, and after processing the full 200 it evaluates RFC 9110 §13 preconditions against **processed** validators. This is not evaluation of origin validators by an intermediary cache. Other methods are never eligible for this recovery path. `Set-Cookie` and origin validators are never replayed. Absent metadata → cache miss | +| `HEAD` of a processed document | **Serves the persisted GET artifact when one exists**, parity with GET/304 by construction, since RFC 9111 §4.3.5 lets HEAD metadata update a stored GET response and mutators are not required to be deterministic. With no persisted artifact, HEAD is processed like a GET (headers only) and its finals stored as a **distinct head-only artifact type that never satisfies a later GET**, when a stored GET artifact exists a HEAD may **update** it only when the comparison, made against the separately stored **origin-side** metadata, never the processed artifact's (origin and rewritten lengths differ by construction, so the processed length is never the comparand and HEAD metadata never touches processed-side headers), finds validators and origin `Content-Length` matching and no byte-coupled representation field changed (RFC 9111 §4.3.5); qualifying updates land origin-side under the same atomic-commit discipline as the 304 rules, and any mismatch **invalidates** the stored GET artifact rather than updating it | | Informational `1xx`, `204`, `205`, `206` | No, enumerated so adapters do not infer independently | This deliberately narrows #782's general "outbound response" phrasing to @@ -515,19 +515,19 @@ The security channel (today: DataDome) runs under a distinct, typed it never authorizes TS-controlled advertising identity, graph linkage, partner egress, other integrations, or general raw-value observability. Request-scoped raw security evidence may be disclosed only to the fixed DataDome Protection API -endpoint and only from that integration's explicit field allowlist; it is not +endpoint and only from that integration's explicit field allowlist. It is not persisted in the identity graph, exposed to publisher origin or other integrations, or emitted in logs. It carries its own configured retention and deletion path. An advertising opt-out does not erase a strictly security-scoped identifier, while an authenticated deletion request or expiry under the security retention policy does. This is not a general exception. The path-only `Request` remains publisher-originated data and is -explicitly covered by vendor retention/DSR sign-off; query strings and full +explicitly covered by vendor retention/DSR sign-off. Query strings and full referrers are never in the security view. Every degree of freedom is closed: - **Host evidence is not a back door to the device provider.** `[device] provider = "fastly"` is an explicit opt-in selection (#1044) and - the hook does not widen it: no JA4-derived classification is stored by a + the hook does not widen it, and no JA4-derived classification is stored by a mutator. If DataDome's Protection API is allowed to receive request-scoped `TlsProtocol`/`JA4` evidence, its registration enumerates each field, proves vendor necessity and payload bounds, and keeps it ephemeral under @@ -539,10 +539,10 @@ referrers are never in the security view. Every degree of freedom is closed: server-side DataDome identifier mapping. On an authenticated TS deletion request it excludes the route from vendor validation, emits a typed `datadome` cookie deletion, and sends no ClientID to DataDome for that - request; re-presentation retries deletion. A lost browser response can leave + request, and re-presentation retries deletion. A lost browser response can leave the cookie until its configured `security_cookie_max_age`, which is the bounded residual sign-off 23 accepts. This operation does **not** claim to - erase data already held by DataDome: vendor-side retention and data-subject + erase data already held by DataDome, because vendor-side retention and data-subject deletion require a named contractual/API procedure in the decision record. If no such vendor procedure exists, operator documentation says so and may not describe TS cookie deletion as vendor-data deletion. @@ -552,10 +552,10 @@ referrers are never in the security view. Every degree of freedom is closed: operation, and the registration is not a placeholder, for DataDome it pins, **aligned to documented vendor behavior where hardening was not intended**: cookie name exactly `datadome`; one configured ownership - tuple for every set and deletion: path exactly `/` and + tuple for every set and deletion, with path exactly `/` and `security_cookie_domain = "host-only"` (default) or one explicit normalized ASCII domain. Host-only mode requires the vendor `Domain` attribute to be - absent; explicit-domain mode requires it to equal the configured domain + absent. Explicit-domain mode requires it to equal the configured domain exactly, TS never accepts a different domain and never rewrites one scope into another. The explicit value cannot exceed the registrable domain, computed against the **vendored @@ -569,11 +569,11 @@ referrers are never in the security view. Every degree of freedom is closed: are separate requirements) and a vendor cookie using `Expires` is normalized to its Max-Age equivalent (both present → `Max-Age` wins, per RFC 10025); a normalized lifetime exceeding the ceiling rejects - the whole operation batch; and the parser is total: repeated `Cookie` + the whole operation batch. The parser is total, where repeated `Cookie` request fields are joined with `"; "` (semicolon-space, the order-preserving join of RFC 10025) before parsing, and **duplicate `datadome` pairs after the join make the request-side - identity ambiguous: treated as cookie-absent for the vendor call and + identity ambiguous, so they are treated as cookie-absent for the vendor call and counted**, while cookies under other names pass through untouched; `Set-Cookie` fields are **never combined**, and a vendor response carrying more than one `datadome` `Set-Cookie` field, a `Set-Cookie` @@ -584,7 +584,7 @@ referrers are never in the security view. Every degree of freedom is closed: each reject the operation batch (the vendor cookie is well-formed; strictness is safe). `Expires` normalizes as `Max-Age = max(0, floor(expires − now))` whole seconds on the server - wall clock at parse time (the shared skew-bounded basis; a result of + wall clock at parse time (the shared skew-bounded basis, where a result of 0 is a deletion), and the 512-byte limit measures the **normalized** serialized `name=value` plus attributes in bytes; `Max-Age` is capped by required operator configuration @@ -595,18 +595,18 @@ referrers are never in the security view. Every degree of freedom is closed: Where the contract **is** deliberately narrower than the vendor, the spec-pinned pointer allowlist starting at ClientID-only against DataDome's mandatory response-directed mapping set, that reduction - needs explicit product **and vendor** acceptance: sign-off item 28; - a violating operation is rejected whole (the batch rule). **Both + needs explicit product **and vendor** acceptance under sign-off item 28. A + violating operation is rejected whole (the batch rule). **Both sessionByHeader is startup-rejected in v1, one state, not three**: TS never sends `X-DataDome-X-Set-Cookie: true`; a vendor `X-Set-Cookie` **invalidates the batch → Continue** (its matrix cell, since a session mode the fleet never requested must not half-apply); and an incoming browser `X-DataDome-ClientID` is **not forwarded to the vendor** (cookie-only session identity, an earlier revision translated - `X-Set-Cookie` into an ordinary cookie, which is not equivalent: + `X-Set-Cookie` into an ordinary cookie, which is not equivalent, because header-session clients expect JavaScript to receive `X-Set-Cookie` and `X-DD-B`, and the higher-priority header session would never see - a cookie update; the older DataDome spec's "always send + a cookie update. The older DataDome spec's "always send X-DataDome-X-Set-Cookie when the header ID is used" is superseded for v1 by this section). Supporting header mode later means the full vendor protocol, typed owner-scoped `X-Set-Cookie`/`X-DD-B` forwarding, @@ -617,7 +617,7 @@ referrers are never in the security view. Every degree of freedom is closed: cookie to be readable by its JavaScript and warns against `HttpOnly`, so **every same-origin page script can observe it**, and a Respond serves vendor-owned HTML under the publisher origin (vendor scripts - with same-origin access to cookies, storage, and APIs; publisher CSP + with same-origin access to cookies, storage, and APIs, while publisher CSP can conversely break the challenge). Those browser-side observers, same-origin vendor code, CSP interaction, and challenge redirects enter **sign-offs 23/28** for ratification (both decision records @@ -627,28 +627,28 @@ referrers are never in the security view. Every degree of freedom is closed: other integrations' request views, publisher-origin proxy forwarding, proxy/click/Testlight upstreams, auction/page-bids request serialization, and logs (redaction list), each surface a tested row - of the inventory; only the security channel itself observes it; vendor - egress goes only to the fixed DataDome Protection API authority and path; - redirects are not followed; deletion is always possible - through the `SecurityUse` lifecycle; and advertising withdrawal never + of the inventory. Only the security channel itself observes it. Vendor + egress goes only to the fixed DataDome Protection API authority and path. + Redirects are not followed. Deletion is always possible + through the `SecurityUse` lifecycle, and advertising withdrawal never grants access to or reuses the identifier. No other request filter inherits the cookie capability. - **Cookie ownership makes deletion total for the scope TS creates.** While DataDome is enabled, `datadome` is a security-owned name across the final - response: before the security batch applies, core removes and meters every - origin, core, cached, or ordinary-mutator `Set-Cookie` for that name; - unrelated cookie names remain separate field lines. Only the typed security + response, and before the security batch applies, core removes and meters every + origin, core, cached, or ordinary-mutator `Set-Cookie` for that name. + Unrelated cookie names remain separate field lines. Only the typed security operation may emit it. Authenticated deletion emits the same configured `(name, domain mode/domain, path)` tuple with `Max-Age=0`; it does not guess a scope from the request cookie, whose wire form carries no Domain or Path. Candidate validation rejects a change of domain mode/domain while the previous active DataDome configuration can still have a live cookie. The supported migration is disable + wait at least the previous - `security_cookie_max_age` + activate the new scope; a faster scope change + `security_cookie_max_age` + activate the new scope. A faster scope change requires a separate bounded deletion-fan-out design. The permission spec §5.5 whole-settings serve fence applies before cookie processing. A bounded old-generation admission validation may survive only during the pre-promotion - drain; the register's promotion-not-before plus member quiescence proves it + drain. The register's promotion-not-before plus member quiescence proves it and every admitted effect ended before the activation CAS. After that CAS, no instance may emit, refresh, or delete a `datadome` cookie until it has loaded and leased the exact new active tuple. A stale instance stops at serve @@ -659,11 +659,11 @@ referrers are never in the security view. Every degree of freedom is closed: - **The incoming `X-DataDome-ClientID` request header is owner-only, like the cookie.** DataDome prioritizes the header over the cookie, so leaving it in the shared request would hand other integrations and - upstream routing the same identifier the cookie boundary strips: core + upstream routing the same identifier the cookie boundary strips, so core **removes it from the shared request** before integrations and upstream routing run, it joins `RedactedRequestView`'s enumerated strip set (providers spec), **and in v1 it is stripped for the - vendor too: the Protection API request's ClientID derives only from + vendor too, because the Protection API request's ClientID derives only from the `datadome` cookie, never from the incoming header** (DataDome's contract requires `X-DataDome-X-Set-Cookie: true` whenever a header-supplied ClientID is forwarded, so forwarding the header under @@ -674,27 +674,27 @@ referrers are never in the security view. Every degree of freedom is closed: sent. Only DataDome-returned overlay data reaches the publisher, never the raw browser-supplied header. - **The pointer protocol has a total parser contract**, adapters - cannot differ where malformed batches fail open: the pointer list is + cannot differ where malformed batches fail open, where the pointer list is tokenized by the vendor's documented space separation, repeated pointer header fields are concatenated with a single SP before tokenizing, tokens split on runs of SP/HTAB, empty tokens ignored, - then names are ASCII-lowercased before duplicate detection; duplicate + then names are ASCII-lowercased before duplicate detection. Duplicate names after normalization, invalid names, more than 16 pointers, or more than 4 KiB of pointer payload render the batch invalid (→ Continue, the vendor's fail-open). **Pointed-field multiplicity is closed**: for singleton fields (`Location`, `Content-Type`, `X-DataDome`, `X-DD-B`, `X-Set-Cookie`) more than one instance in the vendor response invalidates the batch atomically, never a - first/last/join choice an adapter makes; list-valued fields + first/last/join choice an adapter makes. List-valued fields (`Cache-Control`, `Pragma`) are joined per RFC 9110 §5.3 before their - matrix outcome applies; `Set-Cookie` multiplicity follows the typed + matrix outcome applies. `Set-Cookie` multiplicity follows the typed cookie rule (exactly one `datadome` field, above). No both-source priority rule exists in v1: the header session form (`X-Set-Cookie`) is matrix-governed as batch-invalid, so "header form wins" is unreachable and deleted. - **Request-header pointers are a positive, enumerated allowlist with no default publisher-origin identifier exposure.** - "Documented enrichment headers" is not enforceable; the registration + "Documented enrichment headers" is not enforceable. The registration enumerates the exact names from the **inline DataDome field contract in §4a.2**, spec-pinned today to exactly **`X-DataDome-ClientID`**, admitted only when the @@ -707,12 +707,12 @@ referrers are never in the security view. Every degree of freedom is closed: resolving what was a contradiction. When the opt-in is false, the vendor-returned ClientID is discarded and the publisher origin is not an identifier observer. When true, it applies only to an owner-scoped - publisher-upstream overlay, never the shared request; startup logs the + publisher-upstream overlay, never the shared request. Startup logs the additional consumer, operator documentation must disclose its purpose and retention, and a fixture proves no other surface can read it. Everything else, authentication, `Cookie`, `Forwarded`/`X-Forwarded-*`, other identity, consent, and - routing-authority fields, is rejected by name and by class: a + routing-authority fields, is rejected by name and by class, because a compromised endpoint must not replace origin credentials, inject `ts-ec`, or spoof client location. - **Browser-response headers are decision-scoped through the single @@ -729,7 +729,7 @@ referrers are never in the security view. Every degree of freedom is closed: decision (challenge/deny) owns its body but may describe it with **`Content-Type` only**, encoding and validator fields (`Content-Encoding`, `ETag`, `Last-Modified`, digests) stay reserved - even for Respond: challenge bodies are simple and uncacheable, the + even for Respond, because challenge bodies are simple and uncacheable, the allowlist does not admit those fields, and ambiguity here decides whether a challenge enforces or silently fails open (batch rejection → Continue). If the vendor ever requires more, it arrives as a reviewed @@ -740,10 +740,10 @@ referrers are never in the security view. Every degree of freedom is closed: deadline of 3000 ms on the instance's monotonic clock, measured from immediately before vendor-backend acquisition/dispatch to the final body byte**, meaning connection setup and request send are inside the - window; the 1500 ms first-byte bound (the older spec's figure, now + window, and the 1500 ms first-byte bound (the older spec's figure, now first-byte only) runs from the same origin on the same clock. At expiry the decision is final (batch fails → Continue) and the vendor - request is canceled; cancellation and resource cleanup complete + request is canceled. Cancellation and resource cleanup complete asynchronously and never delay the response. TS sends `Accept-Encoding: identity`, and because that does not _guarantee_ identity coding, a response arriving with any `Content-Encoding` is @@ -756,7 +756,7 @@ referrers are never in the security view. Every degree of freedom is closed: vendor does not guarantee method-invariant challenge bodies, so the validated HEAD bytes cannot establish the GET length (a vendor-guaranteed equivalence, if ever ratified, may restore the - field as a reviewed change; the older DataDome spec's HEAD handling + field as a reviewed change, and the older DataDome spec's HEAD handling remains superseded). Exceeding size, first-byte, or total deadline fails the batch → Continue. - **One pointer contract, one place.** The single normative @@ -768,7 +768,7 @@ referrers are never in the security view. Every degree of freedom is closed: the vendor's documented `Set-Cookie X-DD-B` allow-example while another invalidated the whole batch). No `X-DD-*` wildcard exists: every name is enumerated, `X-DD-B` included and forwarded exactly once - as the vendor's documented cookie-mode browser-response signal; it is + as the vendor's documented cookie-mode browser-response signal. It is never copied into publisher-upstream or another integration. The documented vendor responses (both the challenge example and the `Set-Cookie X-DD-B` allow example) are **verbatim fixtures asserting @@ -783,29 +783,29 @@ referrers are never in the security view. Every degree of freedom is closed: - **One global order:** core finalization → ordinary mutators → security effects → **final cache/privacy invariant pass, unconditionally last**. Security precedence over publisher-facing - mutations comes from its position, not a "wins" rule; nothing outranks + mutations comes from its position, not a "wins" rule. Nothing outranks the invariant pass, or a challenge could combine `Set-Cookie` with public caching. The older DataDome spec's "applies last, after finalization" wording is **superseded by this order**. That baseline remains - unchanged; this PR-specific section prevents an implementation from placing + unchanged. This PR-specific section prevents an implementation from placing DataDome after the invariant pass and reopening the public-cache-plus-cookie bug. - The channel adopts the shared layers: structured attributed batches (§3, atomic per batch, a 302 must never lose `Location` to a budget while keeping its cookie), reserved header names, budgets, and the - invariant pass, with one sequencing rule fail-open depends on: the + invariant pass, with one sequencing rule fail-open depends on, where the complete challenge batch is **validated and budgeted before the Respond decision commits**, so a rejection converts to Continue while - the publisher route is still available; discovering the rejection + the publisher route is still available, because discovering the rejection after Respond has short-circuited routing would leave nothing to fail open _to_. ### 4a.1 Public Suffix List snapshot (normative) The vendored Mozilla PSL revision used for registrable-domain -computation in §4a: the implementation PR vendors the list file +computation in §4a, and the implementation PR vendors the list file and records its upstream commit hash here. Rules: ICANN **and** private -sections apply; hostnames are IDNA-mapped before matching; IP literals +sections apply, hostnames are IDNA-mapped before matching, and IP literals and single-label hosts have no registrable domain (cookie falls back to host-only). Updating the snapshot is a reviewed spec change. @@ -821,7 +821,7 @@ host-only). Updating the snapshot is a reviewed spec change. The implementation copies the source bytes at that commit without editing and writes the lowercase 64-hex SHA-256 plus one trailing LF (no filename or other fields) to the required hash path. CI verifies the bytes, hash, and -commit reference together; updating any one without the others fails. The +commit reference together. Updating any one without the others fails. The provenance file is canonical JSON with exactly `{upstream_repository, upstream_commit_oid, upstream_commit_tree_oid, source_path, source_blob_oid, source_sha256_hex}`. The vendoring PR description @@ -849,7 +849,7 @@ this subsection means that DataDome protection is enabled, not that an operator supply an authority. Redirect following is disabled. No TS-controlled advertising identifier, consent-store key, graph value, request query, or full referrer is admitted. The normalized publisher URL path remains disclosed and -may itself contain publisher-chosen data; sign-offs 23/28 must classify that +may itself contain publisher-chosen data. Sign-offs 23/28 must classify that surface, its retention, and DSR handling rather than calling the entire URL identity-free. @@ -859,7 +859,7 @@ Core-derived fields: - `RequestModuleName`, `ModuleVersion`, `TimeRequest`, `Port` - `ServerName`, `ServerRegion` - `ClientID` from the single unambiguous `datadome` cookie only. The form key - is always present because the Protection API declares it mandatory; its + is always present because the Protection API declares it mandatory. Its value is the empty string when no unambiguous cookie exists - `CookiesLen`, `AuthorizationLen`, and `PostParamLen` as lengths only - `HeadersList`, containing only the source header names admitted by the @@ -875,7 +875,7 @@ Core derives those fields identically on every qualified adapter: - `Key` is the resolved DataDome server secret and is never obtained from request/config text; `IP` and `Port` are the remote address and TCP source port from trusted connection metadata. Missing `Key`, `IP`, or `Port` skips - the call through the metered fail-open path; no sentinel is synthesized + the call through the metered fail-open path. No sentinel is synthesized - `Method` is the validated HTTP method token; `Protocol` is exactly `http` or `https` from the adapter request URI; `Host` is the normalized ASCII request authority with a non-default port retained; `ServerHostname` is trusted TLS @@ -889,11 +889,11 @@ Core derives those fields identically on every qualified adapter: Unix timestamp in decimal microseconds, captured once before integration processing and constrained to `0..=2^53-1` - `ServerName` is the adapter-qualified deployment/service name and - `ServerRegion` is its adapter-qualified region code; either is omitted when + `ServerRegion` is its adapter-qualified region code. Either is omitted when the platform cannot supply it without request input - `CookiesLen`, `AuthorizationLen`, and `PostParamLen` are decimal byte counts of the received field/body surfaces before redaction. Overflow beyond an - unsigned 64-bit count skips the call; it never wraps or truncates + unsigned 64-bit count skips the call. It never wraps or truncates Exact request-header value mappings: @@ -918,7 +918,7 @@ Exact request-header value mappings: Request-field multiplicity is normalized **before** parsing, truncation, and form encoding, and adapters expose every received field line rather than a preselected first/last value. Every admitted value line must contain valid HTTP -field-value octets **and** valid UTF-8 after OWS removal; otherwise the vendor +field-value octets **and** valid UTF-8 after OWS removal, otherwise the vendor call is skipped through the metered fail-open path, because adapter-specific byte-to-string replacement is forbidden: @@ -942,14 +942,14 @@ byte-to-string replacement is forbidden: Repetition skips the vendor call before either length or `HeadersList` is constructed. `AuthorizationLen` is the byte length of the one OWS-normalized value. `PostParamLen` is always the byte length of the body - actually presented to core, not the numeric `content-length` value; a + actually presented to core, not the numeric `content-length` value. A malformed or body-inconsistent `content-length` is rejected by the shared HTTP request boundary before integrations run. - Multiple `cookie` field lines are permitted. Core OWS-normalizes them and joins them in received order with the literal bytes `; ` for the shared RFC cookie parser. `CookiesLen` is the byte length of that canonical joined value. `ClientID` is populated only when the parsed result contains exactly - one syntactically valid `datadome` pair; malformed cookie syntax or duplicate + one syntactically valid `datadome` pair. Malformed cookie syntax or duplicate `datadome` pairs produces the required empty `ClientID` value without exposing another cookie. The original cookie values never enter the vendor payload. @@ -957,10 +957,10 @@ byte-to-string replacement is forbidden: every admitted received field line in original line order, so repeated list fields and cookie lines remain repeated. A rejected request produces no `HeadersList` and no vendor call. Per-field caps apply to the single - normalized value; the 24,576-byte cap applies after complete form encoding. + normalized value. The 24,576-byte cap applies after complete form encoding. For an optional mapped value, zero received lines omits both source and mapped -field; one or more lines whose OWS-normalized values are all empty omits the +field. One or more lines whose OWS-normalized values are all empty omits the mapped form field but retains each received source name in `HeadersList`. If at least one list-valued line is nonempty, empty siblings remain represented in the exact received-order `, ` join. Mandatory `ClientID` and the three length @@ -971,10 +971,10 @@ every host and assert byte-identical form fields, lengths, `HeadersList`, and reject/omit outcomes. The corpus includes repeated list fields, identical and different singleton duplicates, multiple cookies, duplicate `datadome` cookies, empty values, invalid octets, and headers whose individual values -contain commas; invalid UTF-8 is a skip, never replacement decoding. +contain commas. Invalid UTF-8 is a skip, never replacement decoding. `true-client-ip`, `x-forwarded-for`, and `x-real-ip` are not admitted in v1. -The trusted `IP` field already supplies connection provenance; copying raw +The trusted `IP` field already supplies connection provenance, and copying raw forwarding headers would let a client or unqualified proxy manufacture vendor evidence. A future adapter-normalized forwarding chain requires a separately named typed field and vendor sign-off, never reuse of the raw header mapping. @@ -999,7 +999,7 @@ list. The following limits are bytes of the decoded field value before form encoding. Truncation is UTF-8-boundary-safe. `XForwardedForIP` alone truncates -from the end; every other bounded field retains its prefix: +from the end, while every other bounded field retains its prefix: - 8 bytes: `SecCHDeviceMemory`, `SecCHUAMobile`, `SecFetchStorageAccess`, `SecFetchUser` @@ -1022,12 +1022,12 @@ unbounded per-field by the vendor table but remain subject to the total bound. The complete `application/x-www-form-urlencoded` body, including field names, `=`/`&` separators, and percent-encoding expansion, must be at most **24,576 bytes**. Core constructs and measures the whole payload before issuing the -request. It does not silently drop optional fields to fit: overflow skips the +request. It does not silently drop optional fields to fit. Overflow skips the vendor call and takes the same metered fail-open `Continue` path as a transport failure. This is deliberately narrower than DataDome's currently documented required -surface: notably, it withholds `CookiesList` and omits empty source-header +surface. Notably, it withholds `CookiesList` and omits empty source-header fields. Product/vendor sign-off 28 therefore requires written confirmation that this exact reduced profile is supported. Until that confirmation and adapter conformance fixtures exist, the DataDome integration is not @@ -1042,38 +1042,38 @@ is rejected. | Header | Direction | Scope | | --------------------- | --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `X-DataDome-ClientID` | response → upstream overlay | Disabled by default; admitted only with `expose_client_id_to_origin = true`. Owner-scoped publisher overlay only, never the shared request view or another integration | +| `X-DataDome-ClientID` | response → upstream overlay | Disabled by default. Admitted only with `expose_client_id_to_origin = true`. Owner-scoped publisher overlay only, never the shared request view or another integration | #### 4a.2.3 The single pointer matrix (normative, decision × session mode × pointer) This is the one authoritative browser-response contract. Session mode -is **cookie** in v1 (sessionByHeader is startup-rejected; a header-mode +is **cookie** in v1 (sessionByHeader is startup-rejected, and a header-mode column is added by the sign-off-23 opt-in, never implicitly). No wildcard rows exist, every accepted name is enumerated, and **every cell terminates in exactly one outcome**. | Pointer | Respond (cookie mode) | Continue (cookie mode) | | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------- | -| `Set-Cookie` | typed `datadome` cookie operation (hook spec §4a parser; exactly one `datadome` field) | typed `datadome` cookie operation | +| `Set-Cookie` | typed `datadome` cookie operation (hook spec §4a parser, exactly one `datadome` field) | typed `datadome` cookie operation | | `X-Set-Cookie` | **invalidate the batch → Continue** (mode mismatch: TS never requests header sessions in v1; a vendor response asserting one must not half-apply) | **invalidate the batch** → effects dropped | | `Location` | forward on a 3xx Respond, replace; **on a non-3xx Respond → invalidate the batch → Continue** (a redirect field without a redirect status is a malformed decision) | **invalidate the batch** (effects dropped) | | `Content-Type` | forward (owns its body), replace | **invalidate the batch** (effects dropped) | -| `Cache-Control` | restricted merge; invariant pass last | **invalidate the batch** (effects dropped) | +| `Cache-Control` | restricted merge, invariant pass last | **invalidate the batch** (effects dropped) | | `Pragma` | drop-individually, logged (response `Pragma` has no standardized meaning, RFC 9111 §5.4) | drop-individually, logged | | `X-DataDome` | forward, owner-scoped typed telemetry | forward, owner-scoped typed telemetry | -| `X-DD-B` | forward as a browser-response security signal; never copy to publisher-upstream or another integration | forward as a browser-response security signal | +| `X-DD-B` | forward as a browser-response security signal, never copy to publisher-upstream or another integration | forward as a browser-response security signal | | anything not listed | invalidate the batch → Continue | invalidate → effects dropped | Singleton-field multiplicity (`Location`, `Content-Type`, `X-DataDome`, `X-DD-B`, `X-Set-Cookie` appearing more than once) invalidates the -batch atomically; list-valued fields (`Cache-Control`, `Pragma`) join +batch atomically. List-valued fields (`Cache-Control`, `Pragma`) join per RFC 9110 §5.3 before their cell applies (hook spec §4a). `X-DD-B` is security-owned when DataDome is enabled. Before applying the fresh security batch, core removes every pre-existing instance from the origin, cached ordinary artifact, 304 metadata update, core response, or ordinary mutator. A valid pointed vendor value then uses **replace-all** and the final -response cardinality must be exactly one; if the fresh vendor batch does not +response cardinality must be exactly one. If the fresh vendor batch does not point to it, final cardinality is zero. Append is never allowed. Fixtures cover origin collision, cache-hit collision, 304 collision, repeated vendor fields, and one valid fresh value, proving “exactly once” at final emission rather than @@ -1081,7 +1081,7 @@ merely inside the vendor batch. **Fixtures**: DataDome's documented challenge response (`Set-Cookie`, `Pragma`, `X-DataDome`, `Cache-Control`) asserts the decision stays -**Respond** with exactly the mapped fields; the documented allow example +**Respond** with exactly the mapped fields. The documented allow example (`Set-Cookie X-DD-B`) asserts Continue proceeds with the cookie applied and `X-DD-B` forwarded exactly once, neither fixture may fail open. @@ -1109,7 +1109,7 @@ expose_client_id_to_origin = false expose_host_fingerprints_to_vendor = false ``` -The Protection API authority is not configurable: it is the core-owned +The Protection API authority is not configurable. It is the core-owned `https://api-fastly.datadome.co/validate-request` endpoint defined in §4a.2.1, with redirects disabled. The baseline `protection_api_origin` field is a startup error under this delta, as are `sessionByHeader`, `session_by_header`, @@ -1149,7 +1149,7 @@ different evidence. on, with cookie emission v1 reserves. 3. **At least one real consumer ships in the same PR**, an existing integration registering a mutator for a real need (or, failing a real - need, the feature waits; scaffolding with only self-referential tests is + need, the feature waits, and scaffolding with only self-referential tests is dead code and will be removed). 4. Every adapter applies mutations on its outbound path, with a per-adapter route test asserting an integration-set header appears in the response. @@ -1190,14 +1190,14 @@ ordinary mutator API. ## 6. Divergences from issue #782 -This spec supersedes #782 on the following points; the issue is updated to +This spec supersedes #782 on the following points. The issue is updated to reference this spec when the implementing PR (the hook's return with its first consumer, §7) merges: | #782 says | This spec says | Why | | -------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | -| Mutations apply to the outbound response generally | Eligibility is the explicit §3a matrix, centered on processed documents | Pass-through/error/304 mutation has different semantics per adapter; enumerating beats implying | -| Ship the trait + registry + adapter application | Additionally: structured operations API (§2), reserved surface (§3), and a real consumer in the same PR (§4, item 3) | PR #838 shipped the trait with zero call sites; an unrestricted `&mut HeaderMap` cannot enforce any collision policy | +| Mutations apply to the outbound response generally | Eligibility is the explicit §3a matrix, centered on processed documents | Pass-through/error/304 mutation has different semantics per adapter, and enumerating beats implying | +| Ship the trait + registry + adapter application | Additionally: structured operations API (§2), reserved surface (§3), and a real consumer in the same PR (§4, item 3) | PR #838 shipped the trait with zero call sites, and an unrestricted `&mut HeaderMap` cannot enforce any collision policy | ## 7. Disposition (2026-08-25) @@ -1211,7 +1211,7 @@ first consumer, §7) merges: that must set response headers, for example `Accept-CH` client-hint requests or detection results such as §4a's security channel. The returning PR implements this spec, not PR #838's shape. -- Recorded for that future design, from review of the earlier draft: the +- Recorded for that future design, from review of the earlier draft. The `&mut HeaderMap` shape concern stands. Handing a mutator a mutable header map makes §3's collision policy unenforceable by construction, because core cannot validate or attribute writes it never sees, so the From bb2bb0764658edbf17b268505ebeb70941db2c8b Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Mon, 31 Aug 2026 23:22:51 +0100 Subject: [PATCH 10/17] Restatus the unmerged provider specs and correct the seam spec counts The four series specs (client-cycle EC resolve, permission model, pluggable providers, migration and rollout) each carried a Status line saying they were implemented. The code they describe is only in PRs #1043 to #1047 and none of those is merged, so the line read as shipped behavior. Each now says Proposed, names the PR that carries the implementation and states that it is not yet on main, keeping the existing revision dates and notes. The integration provider seam spec carried counts and line references that do not hold on main at d516a9e94. Corrected against that commit: - Section 4 said migration_guards.rs embeds "the thirteen vendor files". The directory holds 23 .rs files (2 infrastructure, 6 in nextjs/, 2 in datadome/, 13 top-level integration modules), the guard embeds 20 of them and 9 of those 20 belong to the nine vendors, with osano.rs and the two datadome/ files absent. builders() registers 13 integrations, which is a different 13 from the file count. - Section 3.5 gave no counts for the prepare and finalize calls. There are nine production prepare_request call sites across the four adapters and a tenth in core, and the single production finalize_response call site is in core rather than in any adapter. - Section 8 item 3 described a proxy resolving geo twice, which does not happen on main. The real double resolution is the adapter EC context build against handle_auction on POST /auction. - Section 8 item 5 understated the Spin gap and misdescribed Cloudflare. Cloudflare covers every route it registers and has no health route, while Spin skips its first-party bindings as well as its inline admin stubs. - Line references: settings.rs:166 to :215, auction/mod.rs:49 to the list at :51 to :53, publisher.rs:4361 to :4369. Section 6 now requires the round trip to be proven on the Fastly adapter, the primary deployment target, rather than on any adapter, because Fastly has no library target and the round trip otherwise only runs on the Axum dev server. --- ...26-07-30-client-cycle-ec-resolve-design.md | 3 +- .../2026-07-30-permission-model-design.md | 3 +- .../2026-07-30-pluggable-providers-design.md | 5 +- ...07-30-provider-migration-rollout-design.md | 9 +- ...-08-27-integration-provider-seam-design.md | 123 ++++++++++++------ 5 files changed, 95 insertions(+), 48 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index 63e9226d7..d8b161ae8 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -1,6 +1,7 @@ # Design Spec: Client-Cycle Edge Cookie Providers and the Resolve Endpoint -**Status:** Implemented (hardened v1) in PR #1046, on the threat model below. +**Status:** Proposed. PR #1046 carries the implementation (hardened v1, on +the threat model below) and is not yet merged to main. The full anti-replay reservation machinery (§3.9) and the vendor envelope verification it serves are deliberately not in v1: they land with the first real vendor scheme, which brings the concrete envelope format the diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index 531a38243..6e9438a4b 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -1,6 +1,7 @@ # Design Spec: Jurisdiction Permission Model -**Status:** Implemented in PR #1045; revised against the implementation, 2026-08-25. +**Status:** Proposed. PR #1045 carries the implementation and is not yet +merged to main. Revised against that implementation, 2026-08-25. **Author:** Engineering **Issue references:** #779 **Related specs:** `2026-07-30-pluggable-providers-design.md`, diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index 393293873..e7fd775c9 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -1,7 +1,8 @@ # Design Spec: Pluggable Edge Cookie, Device, and Geo Providers -**Status:** Implemented in PR #1043 (Edge Cookie provider seam) and PR #1044 -(device and geo selection); revised against the implementation, 2026-08-25. +**Status:** Proposed. The implementation is carried by PR #1043 (Edge Cookie +provider seam) and PR #1044 (device and geo selection), neither of which is +yet merged to main. Revised against that implementation, 2026-08-25. **Author:** Engineering **Issue references:** #777, #778, #780, #781 **Related specs:** `2026-07-30-permission-model-design.md`, diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md index 5886f7f71..bea38238c 100644 --- a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -1,9 +1,10 @@ # Design Spec: Provider and Permission Model, Migration and Rollout -**Status:** Revised against the implemented series (PRs #1043-#1047), -2026-08-25. The §8 sign-off rows remain the series' decision ledger, and rows the -implementation now satisfies are marked with their PR so the task force can -ratify rather than re-litigate. +**Status:** Proposed. The implementation is carried by the series PRs #1043 to +#1047, none of which is yet merged to main, and this document was revised +against that series on 2026-08-25. The §8 sign-off rows remain the series' +decision ledger, and rows the implementation now satisfies are marked with +their PR so the task force can ratify rather than re-litigate. **Author:** Engineering **Issue references:** #777-#781 (epic) **Related specs:** `2026-07-30-pluggable-providers-design.md`, diff --git a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md index 3fbb27b70..e61e3525c 100644 --- a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md +++ b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md @@ -67,11 +67,12 @@ seventh PR targets, and the five series PRs do not touch these files. imports and calls each vendor's config type by name (`crates/trusted-server-core/src/config.rs:136` to `:166`). 4. **Auction providers are a second closed table.** - `crates/trusted-server-core/src/auction/mod.rs:49` lists Prebid, APS and - the ad server mock. + The list of Prebid, APS and the ad server mock is at + `crates/trusted-server-core/src/auction/mod.rs:51` to `:53`, inside + `provider_builders()` at `:49`. 5. **Two vendors reach further into core.** DataDome drives cache privacy and the origin fetch decision through a marker type - (`html_processor.rs:303`, `publisher.rs:4361` to `:4381`, + (`html_processor.rs:303`, `publisher.rs:4369` to `:4381`, `publisher.rs:2653`), and GPT diagnostics is called by name from all four adapters (for example `crates/trusted-server-adapter-fastly/src/app.rs:564`). @@ -182,9 +183,16 @@ the neutral form. extension meaning "this response is personalized to the request, do not share it", set by any integration. Core keeps the behavior, being the full body buffer and private cache, and stops naming a vendor. -- **Request prepare and finalize.** The direct GPT diagnostics calls in the - four adapters move behind hooks on the registration, so an adapter runs - whatever its registrations declare. +- **Request prepare and finalize.** The direct GPT diagnostics calls move + behind hooks on the registration, so an adapter runs whatever its + registrations declare. Those calls reach beyond the adapter edge. On + `main` nine production `prepare_request` call sites sit in the four + adapters (two in Fastly, two in Axum, two in Cloudflare and three in + Spin) and a tenth sits in core's `handle_publisher_request` + (`publisher.rs:4050`). `finalize_response` has one production call site + and it is in core rather than in any adapter (`publisher.rs:4783`), so + the finalize hook has to run on the core response path and not only at + the adapter edge. ### 3.6 Identity, geo and device as registration capabilities @@ -262,18 +270,27 @@ TypeScript, its config type and its tests into `[integrations.]` configuration tables need no change, because `IntegrationSettings` is a flattened map that already accepts unknown vendor -keys (`crates/trusted-server-core/src/settings.rs:166`). +keys (`crates/trusted-server-core/src/settings.rs:215`). Two more places every move must touch, found by mapping `main`: -- `crates/trusted-server-core/src/migration_guards.rs` embeds every core - source file by relative path with `include_str!`, the thirteen vendor - files included, so a vendor move that leaves its entry behind breaks the - build rather than a test. The guard cannot derive its list from the - registrations, because `include_str!` paths are fixed at compile time, so - this change drops the nine vendors' files from the guard instead, because a module - crate is outside the core neutrality guarantee, and a move then deletes - nothing there. +- `crates/trusted-server-core/src/migration_guards.rs` embeds core source + files by relative path with `include_str!`, so a vendor move that leaves + an embedded entry behind breaks the build rather than a test. On `main` + the directory `crates/trusted-server-core/src/integrations/` holds 23 + `.rs` files, being 2 infrastructure files (`mod.rs` and `registry.rs`), + 6 files in the `nextjs/` subdirectory, 2 in the `datadome/` submodule and + 13 top-level integration modules. The guard embeds 20 of those 23, and 9 + of the 20 are files of the nine vendors in the table above. The three it + does not embed are `osano.rs` and the two `datadome/` files, so the guard + is already incomplete and the Osano move has no guard entry to delete. + Separately, `builders()` registers 13 integrations, which is not the same + 13 as the file count, because `adserver_mock` is a file with no + registration while `nextjs` is a registration held in a subdirectory. The + guard cannot derive its list from the registrations, because + `include_str!` paths are fixed at compile time, so this change drops the + vendors' files from the guard instead, because a module crate is outside + the core neutrality guarantee, and a move then deletes nothing there. - The `ts audit` command carries its own vendor table (detection patterns and configuration section names in `crates/trusted-server-cli/src/commands/audit/analyzer.rs` and @@ -292,18 +309,24 @@ deployment that lists the same integrations gets the same responses. ## 6. Acceptance -1. **A round trip with a non-default implementation.** A test integration - defined outside `trusted-server-core`, carrying its own JavaScript, - registers through an adapter, appears in the served bundle with the - right hash, runs its hooks in the right order, and is rejected on a - duplicate id. A seam is only proven by an implementation that is not the - built-in one. -2. **Capabilities round trip.** The same test integration declares an - identity, a geo and a device provider. With the three selectors naming - it, a request is served by all three (the created identifier carries its - code, the resolved country and the device signals are its). With a - selector naming a module that lacks the capability, startup fails with - an error that names the module and the capability. +1. **A round trip with a non-default implementation, on Fastly.** A test + integration defined outside `trusted-server-core`, carrying its own + JavaScript, registers through an adapter, appears in the served bundle + with the right hash, runs its hooks in the right order, and is rejected + on a duplicate id. A seam is only proven by an implementation that is + not the built-in one. The round trip has to run on the Fastly adapter, + which is the primary deployment target, and not only on the Axum dev + server, because a seam proven on the dev server alone is not a seam any + production deployment can use. The Fastly adapter carries only + `src/main.rs` and so has no library target for an external crate to + register through (§8 item 7), so meeting this criterion means giving + that adapter a composition entry point a vendor crate can reach. +2. **Capabilities round trip, on the same adapter.** The same test + integration declares an identity, a geo and a device provider. With the + three selectors naming it, a request is served by all three (the created + identifier carries its code, the resolved country and the device signals + are its). With a selector naming a module that lacks the capability, + startup fails with an error that names the module and the capability. 3. **Parity.** The existing integration and parity suites pass unchanged, because the built-in set still registers through the same path. 4. **No vendor left behind.** The rewritten deploy-validation test shows @@ -346,12 +369,22 @@ for each vendor to rediscover. or a documented build-script recipe removes the trap, and the probe pins the file's line endings and tests the literal, which every vendor would otherwise have to reinvent. -3. **A provider is resolved more than once per request.** A proxy that - wants the resolved location calls the geo provider itself while the - request path has already called it. `CLAUDE.md`'s principle that a - vendor sharing one backend makes a single call per request needs a - per-request provider context to hang that on, which this change does not - introduce. +3. **A provider is resolved more than once per request.** On `main` this is + core against core rather than a proxy against the request path. Every + adapter resolves geo once to build the EC context (`build_ec_context` at + `crates/trusted-server-adapter-axum/src/app.rs:162`, + `crates/trusted-server-adapter-cloudflare/src/app.rs:142` and + `crates/trusted-server-adapter-spin/src/app.rs:346`, and + `build_ec_request_state` at + `crates/trusted-server-adapter-fastly/src/app.rs:394`). On + `POST /auction` the same request then reaches `handle_auction`, which + looks the same client IP up a second time to fill in the auction's + device info (`crates/trusted-server-core/src/auction/endpoints.rs:262`). + Those two are the only production geo call sites in the tree, so the + auction route is where the duplication shows. `CLAUDE.md`'s principle + that a vendor sharing one backend makes a single call per request needs + a per-request provider context to hang that on, which this change does + not introduce. 4. **One core reader still reaches into a vendor's payload.** The `hb_adid` fallback in the publisher reads the APS renderer's fields, so the APS migration needs a neutral answer for it rather than only the @@ -359,13 +392,22 @@ for each vendor to rediscover. 5. **Request preparation covers different routes on each host.** Every adapter runs preparers before routing, but not on the same set of - routes: one runs them on every route but the health check, one skips a - batch endpoint and its admin diagnostics deliberately, one covers three - of its paths, and one skips its inline admin stubs. A module that strips - its own reserved query or cookie is therefore protected on a different - set of routes depending on the host it is deployed to. Making that - uniform means routing each adapter's hand-written handlers through one - wrapper, which is worth doing before a vendor depends on it. + routes. Cloudflare covers everything it registers, because every route + binding goes through one wrapper (`make_handler`) and the adapter has no + health route at all. Axum covers every route but `GET /health`, which is + bound to an inline closure. Fastly covers every route but `GET /health`, + which short-circuits before the app is built, plus the S2S batch sync + and the two admin lookup routes, which return deliberately before the + preparer runs. Spin is the widest gap, covering only `POST /auction`, + the two page-bids GET routes and the publisher fallback, so its + `/health`, its discovery and signature-verification routes, its inline + admin stubs and all six of its first-party bindings (`proxy`, `click`, + `sign` on GET and POST, and `proxy-rebuild` on GET and POST) run with no + preparer at all. A module that strips its own reserved query or cookie + is therefore protected on a different set of routes depending on the + host it is deployed to. Making that uniform means routing each adapter's + hand-written handlers through one wrapper, which is worth doing before a + vendor depends on it. 6. **A module's validate function runs nowhere in a real deployment.** Building the registry calls only a builder's build function, and the @@ -420,3 +462,4 @@ defines, and both should land before the first vendor is asked to use it. | 2026-08-28 | Recorded what implementing the seam found (§8): the operator CLI skips a vendor's deploy rules, a carried module's hash literal is fragile, providers resolve more than once per request, and one core reader still reads an APS payload. Recorded the construction-time hash check in §3.2. | | 2026-08-28 | Adopted the #1043 review's registration shape for identity and applied its rule to geo and device, with no provider built into core (§3.6, §6 item 2, §8 rows 7 to 9). Recorded the relationship to #986 and reordered the series so this spec and its implementation come first. | | 2026-08-30 | Moved the five series design specs and the provider-code registry into this PR from PRs #1043 to #1047, so every normative document is reviewed before the code that implements it. Document content is unchanged, and only this status line and this row are new. | +| 2026-08-31 | Corrected the counts and line references the review found, against `main` at d516a9e94: the source-file guard counts (§4), the prepare and finalize call-site counts (§3.5), the real double geo resolution on `POST /auction` (§8 item 3), the per-adapter preparer coverage (§8 item 5), and the `settings.rs`, `auction/mod.rs` and `publisher.rs` line references. §6 now requires the round trip on Fastly rather than on any adapter. | From ee03e8a9051b73c8fd11adcd1f21059c526b3cd5 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Tue, 1 Sep 2026 04:26:59 +0100 Subject: [PATCH 11/17] Answer the PR #1084 review findings in the series specs Rewrite the resolve endpoint's origin check as an exact serialized-origin comparison, defaulting to the single origin https://{publisher.domain} with an optional operator-configured list, and delete the sibling subdomain justification that did not hold. Record that the origin check is defense in depth under the provider's envelope verification, require the sibling subdomain, wrong scheme and non-default port rejection cases, and close open question 7.5. Require core to expire the ts-ecr marker on any request carrying a ts-ec the selected provider does not own, in both the client-cycle and pluggable provider specs, so a provider switch restarts the client cycle instead of leaving a visitor with a marker and no identity. Record the constraints on carrying resolved permissions to the browser without designing the mechanism, which belongs with the first vendor module. Require a nonempty DeviceProvider::required_permissions from a module-supplied provider to be refused at registry resolution rather than described as a documented no-op. Replace the DataDome singleton-header rule that skipped the vendor call on a repeated field, which would have created an attacker-controlled bot detection bypass and described no shipped behavior. Core now passes repeated field lines through as received and leaves interpretation to the vendor integration, keeping only the protocol-level content-length rule. Correct the APS migration row and add a seam spec finding for the pre-existing browser-side APS coupling in core TypeScript. --- ...26-07-30-client-cycle-ec-resolve-design.md | 116 ++++++++++++------ ...integration-response-header-hook-design.md | 77 +++++++++--- .../2026-07-30-pluggable-providers-design.md | 77 ++++++++---- ...-08-27-integration-provider-seam-design.md | 48 +++++--- 4 files changed, 227 insertions(+), 91 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index d8b161ae8..8b8b0a991 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -65,20 +65,23 @@ verify against. `POST /_ts/api/v1/ec/resolve` MUST: 1. **Reject cross-site requests with an origin check.** v1 authorizes a - request only when its `Origin` header names the publisher's configured - domain or a subdomain of it. A missing or foreign `Origin` is rejected - with `403`. Browsers always send `Origin` on POST `fetch`, so its - absence means a non-browser caller, which has no business on a - page-script endpoint. The 2026-07-31 draft asked for an exact origin - allowlist as new configuration plus an optional session-bound CSRF - token. v1 derives the allowed set from `publisher.domain` (existing, - operator-trusted configuration) and admits the publisher's own - subdomains, because the publisher controls their subdomain namespace, and the - draft's sibling-subdomain concern applies to `Sec-Fetch-Site: -same-site` (which v1 does not consult at all), not to a suffix match on - the publisher's own apex. An explicit multi-origin allowlist remains - open (§7.5) for publishers whose pages run on domains other than the - configured apex. + request only when its `Origin` header, serialized as scheme, lowercased + host and effective port, is byte-equal to an accepted origin. The + default accepted set is the single origin `https://{publisher.domain}` + and nothing else, so a sibling subdomain, the `http://` scheme and a + non-default port are all refused. A missing or foreign `Origin` is + rejected with `403`. Browsers always send `Origin` on POST `fetch`, so + its absence means a non-browser caller, which has no business on a + page-script endpoint. An operator may configure further accepted + origins, each of them compared the same exact way, for a publisher + whose pages are served from `www` or from another domain. This is the + exact allowlist the 2026-07-31 draft asked for, and it replaces the + suffix match on `publisher.domain` an earlier revision of this spec + described, which admitted every subdomain of the apex including ones + the publisher may not control. The origin check is defense in depth + and not the primary control. The primary control is the provider's + envelope verification with audience and session binding, which §3.9 + parks and which still has to land with the vendor scheme. 2. **Verify the payload per provider.** The endpoint hands the posted payload to the selected provider's `resolve_from_client` and creates only what the provider returns. Whether the payload is trustworthy is the @@ -183,15 +186,45 @@ the bar for the vendor-scheme implementation: marker shares the Edge Cookie's scope and lifetime and is expired together with it on withdrawal, so a visitor who later re-establishes the permission can resolve again. +- **The marker must not outlive the provider that set it.** The marker + carries no identity, so unlike the `ts-ec` cookie it is not namespaced + by the provider code envelope, and a long `Max-Age` that only + withdrawal expires would leave it standing across a provider switch. + The visitor would then hold a marker saying a resolve has already + succeeded with no identity behind it, and the page script would + suppress the re-post that would start a new one, so the visitor sits + with no identity rather than a restarted one. Core therefore expires + the marker on any request that carries a `ts-ec` the selected provider + does not own, which is the same `{code}~` ownership test core already + applies to the identifier itself. A switch then restarts the client + cycle instead of stalling it. - **The page leg is permission-gated before vendor contact.** The demo module contacts no vendor (it posts a constant), so the draft's injection-time and live-CMP gating requirements bind the first vendor - module, not v1: a vendor module must not derive identity or contact the - vendor for a visitor whose resolved permissions do not satisfy the - provider's declaration, must re-check immediately before vendor contact - (consent can change between document delivery and asynchronous vendor - contact, including BFCache restoration), and its injection is keyed off - the provider selection exactly as the demo's is today. + module rather than v1. A vendor module must not derive identity or + contact the vendor for a visitor whose resolved permissions do not + satisfy the provider's declaration, must re-check immediately before + vendor contact (consent can change between document delivery and + asynchronous vendor contact, including BFCache restoration), and its + injection is keyed off the provider selection exactly as the demo's is + today. +- **How the resolved permissions reach the browser is not designed here.** + The requirement above assumes the page can read the server's decision, + and nothing in v1 carries that decision to the page. The binding + constraint is that the JavaScript bundle is composed at startup and + served under a content hash, so one body is shared across every visitor + and cannot carry per-visitor permission state, which means the signal + has to be per request. The candidate carriers are a response header on + the document, a value injected into the document during HTML + processing, a first-party endpoint the page fetches, and a + non-HttpOnly cookie set alongside the marker. This spec names them and + deliberately does not choose between them. Whichever is chosen, the + server's resolved permission decision is the authority, and an in-page + CMP read is only a withdrawal re-check layered under it and never a + substitute for it, because a page-side read can narrow what the server + resolved and must never widen it. Designing the mechanism is out of + scope for this set of PRs and belongs with the first vendor module, + which is the first consumer that needs it. - The JS module ships through the standard integration bundle mechanism, loaded only when a client-cycle provider is the selected EC provider. The interaction between provider-keyed bundle content and content-hash / @@ -214,13 +247,17 @@ end in tests and demonstrations. ## 6. Testing -- Endpoint unit tests cover: origin rejection (missing and foreign), - subdomain acceptance, content-type rejection, the body bound, the - identifier bound, the different-identity conflict, the no-graph refusal, - the closed permission gate, the unverified payload, and the marker, - cache-control, and graph-row effects of a success. +- Endpoint unit tests cover origin rejection (missing, foreign, a sibling + subdomain of the publisher apex, the `http://` scheme, and a non-default + port), acceptance of an operator-configured extra origin, content-type + rejection, the body bound, the identifier bound, the different-identity + conflict, the no-graph refusal, the closed permission gate, the + unverified payload, and the marker, cache-control, and graph-row effects + of a success. - A round-trip test drives a client identifier through organic deferral, resolve, cookie set, and verbatim read-back recognition. +- A test asserts that a request carrying a `ts-ec` the selected provider + does not own expires the `ts-ecr` marker. - The cross-language constant test pins the endpoint's shared constants to the page-script source. - Remaining for the vendor scheme: a real-browser integration round trip @@ -241,21 +278,26 @@ end in tests and demonstrations. 3. Rate limiting / abuse posture at the edge for an unauthenticated POST. 4. Startup rejection of the client-cycle selection on adapters that cannot route the endpoint, once the portability adapters gain platform KV. -5. An explicit multi-origin allowlist for publishers whose pages run on - domains other than the configured apex (v1 authorizes the apex and its - subdomains). +5. **Answered, and kept here for the record.** An explicit multi-origin + allowlist for publishers whose pages run on domains other than the + configured apex. §3.1 now requires exact serialized-origin comparison, + with a default accepted set of the single origin + `https://{publisher.domain}` and an optional operator-configured list + of further exact origins for the `www` or other-domain case, so the + question is settled rather than carried. 6. How JS module selection keyed off EC provider configuration coexists with content-hashed/SRI-pinned bundles, meaning per-config hashes, cache keying, and the config-push story for them. ## 8. Revision record vs the 2026-07-31 draft -| Draft position | v1 (PR #1046) | Why | -| ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | -| Feature deferred, `resolve_from_client` de-normalized | Feature normative. Trait method kept with a no-op default | The first vendor integration works client-side by design, so the endpoint is on the critical path for the series' first real provider | -| Origin check via new allowlist config + CSRF token | Publisher apex + subdomains from existing `publisher.domain`. Missing/foreign `Origin` → `403` | Uses existing operator-trusted configuration. Explicit allowlist stays open for multi-domain publishers (§7.5) | -| §3.9 reservation/replay machinery required before code | Deferred to the vendor scheme. Draft text retained verbatim as the bar | The design needs the real envelope's unique id and session binding, and a CAS-class primitive no production adapter exposes today | -| Identical behavior or identical startup refusal, 4 ways | Fastly routes it (bot-gated graph). Portability adapters documented as deliberately not routing, like identify | Same platform-KV constraint as the existing EC API routes. Startup rejection follow-up recorded (§7.4) | -| Marker cookie or injected variable (design must pick) | Marker cookie (`ts-ecr`), expired with the EC cookie | The draft's own first option. Testable and observable | -| Demo gated by cargo feature or `#[cfg(test)]` | Both: `client-fixed-demo` feature + startup rejection in the settings validator | Defense in depth | -| Everything else in §3 (graph row, bounds, 409, no-store, full-declaration gate) | Implemented as specified | (none) | +| Draft position | v1 (PR #1046) | Why | +| ------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Feature deferred, `resolve_from_client` de-normalized | Feature normative. Trait method kept with a no-op default | The first vendor integration works client-side by design, so the endpoint is on the critical path for the series' first real provider | +| Origin check via new allowlist config + CSRF token | Exact serialized-origin comparison (scheme, lowercased host, effective port). Default accepted set is the single origin `https://{publisher.domain}`, plus an optional operator-configured list of further exact origins. Missing/foreign `Origin` → `403` | The draft's exact allowlist is adopted. An earlier revision of this spec allowed any suffix match on the apex, which admitted every subdomain including ones the publisher may not control, and its justification did not hold. This closes §7.5 | +| Marker cookie survives whatever happens to the identity it marks | Core expires the `ts-ecr` marker on any request carrying a `ts-ec` the selected provider does not own | The marker is not namespaced by the provider code envelope, so without this a provider switch leaves a visitor with a marker, no identity, and a page script that will not re-post | +| §3.9 reservation/replay machinery required before code | Deferred to the vendor scheme. Draft text retained verbatim as the bar | The design needs the real envelope's unique id and session binding, and a CAS-class primitive no production adapter exposes today | +| Identical behavior or identical startup refusal, 4 ways | Fastly routes it (bot-gated graph). Portability adapters documented as deliberately not routing, like identify | Same platform-KV constraint as the existing EC API routes. Startup rejection follow-up recorded (§7.4) | +| Marker cookie or injected variable (design must pick) | Marker cookie (`ts-ecr`), expired with the EC cookie | The draft's own first option. Testable and observable | +| Demo gated by cargo feature or `#[cfg(test)]` | Both: `client-fixed-demo` feature + startup rejection in the settings validator | Defense in depth | +| Everything else in §3 (graph row, bounds, 409, no-store, full-declaration gate) | Implemented as specified | (none) | diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index 53d886923..c297d95c9 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -932,19 +932,52 @@ byte-to-string replacement is forbidden: parsing and is then bounded. Commas inside an individual value are not split and reserialized. - Every other admitted value-bearing source header in the exact mapping above - is singleton. Zero lines means omit the DataDome field. Exactly one valid - line is OWS-normalized and processed. Two or more lines, even identical, - are ambiguous and skip the vendor call through the metered fail-open path; - core never chooses first, last, or comma-joined. In particular this applies - to `origin`, `referer`, `user-agent`, `content-type`, `from`, every remaining + is presented as received. Zero lines means omit the DataDome field. Each + received line has its leading and trailing optional whitespace removed and + is then checked for valid field-value octets and valid UTF-8, exactly as + above. Two or more lines, identical or not, are preserved as separate lines + in received order and reach the integration that way. Core does not drop a + line, comma-join the lines, choose the first or the last, or skip the + vendor call because a field arrived more than once. This covers `origin`, + `referer`, `user-agent`, `content-type`, `from`, every remaining `sec-ch-*`/`sec-fetch-*` field, and `x-requested-with`. -- `authorization` and `content-length` are security singletons for this view. - Repetition skips the vendor call before either length or `HeadersList` is - constructed. `AuthorizationLen` is the byte length of the one - OWS-normalized value. `PostParamLen` is always the byte length of the body - actually presented to core, not the numeric `content-length` value. A - malformed or body-inconsistent `content-length` is rejected by the shared - HTTP request boundary before integrations run. +- **Why core does not decide this.** Header multiplicity is attacker + controlled, so a core rule that skipped the vendor call whenever an + admitted field arrived twice would let any client turn bot detection off + by sending one header twice. That is a bypass this document would have + created rather than described, because nothing in the shipped code behaves + that way. The protection path reads each mapped field through a single + `headers().get()` in `header_value` + (`crates/trusted-server-core/src/integrations/datadome/protection.rs`) and + calls the Protection API regardless of how many lines arrived. The + principle behind the rewrite is that core must not invent vendor-specific + header normalization and must not decide on a vendor's behalf to skip a + vendor call. How a repeated or otherwise ambiguous field is interpreted, + and how it is encoded into the vendor's payload, is the vendor's detection + logic, so it belongs to the vendor integration, which sees every received + line in received order through `HeadersList` and the presented field + lines. +- **Where that logic belongs.** The DataDome integration is core code today, + in `crates/trusted-server-core/src/integrations/datadome.rs` and the + `crates/trusted-server-core/src/integrations/datadome/` directory. Vendor + detection logic of this kind belongs in the vendor's own module rather + than in core, and the integration provider seam + (`2026-08-27-integration-provider-seam-design.md`) is where that move is + designed. This is a direction for that migration rather than a complaint + about the code as it stands. +- `content-length` stays a protocol-level singleton. A repeated, malformed, + or body-inconsistent `content-length` is rejected by the shared HTTP + request boundary before any integration runs, which is an HTTP + correctness rule and not a vendor decision, so it is unchanged. + `PostParamLen` is always the byte length of the body actually presented to + core, not the numeric `content-length` value. +- `authorization` never reaches the vendor payload as a value. + `AuthorizationLen` is the byte length of the OWS-normalized value. When + more than one `authorization` line arrives, every line is presented in + received order and how they are counted is the integration's decision, + as it is for any other repeated field. Repetition no longer skips the + vendor call, for the same reason as above, because that let the client + decide whether the vendor is consulted at all. - Multiple `cookie` field lines are permitted. Core OWS-normalizes them and joins them in received order with the literal bytes `; ` for the shared RFC cookie parser. `CookiesLen` is the byte length of that canonical joined @@ -969,9 +1002,12 @@ fields follow their explicit rules instead of this optional-field omission. Adapter qualification fixtures feed the same ordered repeated-field corpus to every host and assert byte-identical form fields, lengths, `HeadersList`, and reject/omit outcomes. The corpus includes repeated list fields, identical and -different singleton duplicates, multiple cookies, duplicate `datadome` -cookies, empty values, invalid octets, and headers whose individual values -contain commas. Invalid UTF-8 is a skip, never replacement decoding. +different duplicates of the pass-through fields, multiple cookies, duplicate +`datadome` cookies, empty values, invalid octets, and headers whose individual +values contain commas. For a duplicated pass-through field the asserted +outcome is that every received line survives in received order and that the +vendor call still happens, never a skip. Invalid UTF-8 is a skip, never +replacement decoding. `true-client-ip`, `x-forwarded-for`, and `x-real-ip` are not admitted in v1. The trusted `IP` field already supplies connection provenance, and copying raw @@ -1211,6 +1247,17 @@ first consumer, §7) merges: that must set response headers, for example `Accept-CH` client-hint requests or detection results such as §4a's security channel. The returning PR implements this spec, not PR #838's shape. +- Corrected on 2026-09-01, from review of PR #1084. §4a.2 previously + required core to treat two or more lines of an admitted singleton + request header as ambiguous and to skip the vendor call. Header + multiplicity is attacker controlled, so that rule would have given any + client a way to turn bot detection off by repeating a header, and it + described no shipped behavior, because `header_value` in + `crates/trusted-server-core/src/integrations/datadome/protection.rs` + reads each mapped field with a single `headers().get()` and the + Protection API is called regardless. Core now passes repeated lines + through as received and leaves the interpretation to the vendor + integration. - Recorded for that future design, from review of the earlier draft. The `&mut HeaderMap` shape concern stands. Handing a mutator a mutable header map makes §3's collision policy unenforceable by construction, diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index e7fd775c9..8bfa1840a 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -213,13 +213,18 @@ resolved in the implementation: behavior beyond gating. The gate itself has already run before `generate` is called, so a provider cannot use the fields to authorize itself. - `required_permissions` on `DeviceProvider` and `PlatformGeo`: **present, - with an empty default and no enforcement point.** The draft removed the - method from both traits because PR #838's copies were decorative. The - implementation keeps one uniform declaration seam across all three traits - instead. The built-in device and geo providers declare empty sets, and - core enforces the declaration only for the EC provider (section 5), so - nothing reads as a gate that is not one. The geo circularity argument - stands unchanged and is restated in section 5. + with an empty default, and refused rather than ignored when a module + declares one.** The draft removed the method from both traits because PR + #838's copies were decorative. The implementation keeps one uniform + declaration seam across all three traits instead. The built-in device + and geo providers declare empty sets, and core enforces the declaration + as a request gate only for the EC provider (section 5). Because no + per-request device gate exists to honor a device declaration, a nonempty + `required_permissions` from a module-supplied device provider is + rejected when the registry resolves the selection and the deployment + fails to start, so nothing reads as a gate that is not one. Section 5 + gives the reasoning. The geo circularity argument stands unchanged and + is restated in section 5. The implemented `EdgeCookieProvider` surface (`ec/provider.rs`): @@ -317,7 +322,7 @@ work with the permission model spec, which owns identity-state persistence and egress typing. The revision record lists them as deferred. The gate applies to EC providers **only**. Geo and device are ungated for -two different reasons, stated separately because only one of them is +different reasons, stated separately because only one of them is structural: - **Geo: circularity.** The permission set is resolved from jurisdiction, @@ -338,6 +343,21 @@ structural: class segment and a 12-hex-character hash prefix of the HTTP/2 SETTINGS signal), not raw signals, and the neutral default persists neither because the builtin provider produces no such fields. +- **Device: a declared permission is refused at startup, not ignored at + request time.** `DeviceProvider::required_permissions` has no + enforcement point on the device path, so a declaration cannot be + honored. Before the module seam that was inert, because core and the + host supplied the only two device providers and both declare the empty + set. With a vendor seam the method reads as a promise a vendor could + build on, and a vendor device provider declaring a permission would + still run on every request while appearing to be gated. Core therefore + rejects a nonempty `required_permissions` from a module-supplied device + provider when the registry resolves the selection, failing the + deployment at startup with a message naming the provider and the + permissions it declared. That refusal is lifted only when a real + per-request device gate exists, which is separate design work. The + built-in and host device providers are unaffected, because both declare + the empty set. ## 6. Selection, validation, and failure modes @@ -417,6 +437,15 @@ What a switch does, precisely: the canonical form is the owning provider's own normalization and the owning provider is no longer configured. Those rows stay as they are until their one-year entry TTL expires. +- **The `ts-ecr` client-cycle marker.** The marker carries no identity, so + it is not namespaced by the `{code}~` envelope and a switch would + otherwise leave it standing with its long `Max-Age` intact. Core expires + it on any request carrying a `ts-ec` the selected provider does not own, + using the same ownership test as read-back above, so a visitor whose + identifier has just become unrecognized is not left behind a marker that + tells the page script a resolve has already succeeded. Without that the + visitor would sit with no identity instead of a restarted one. The + client-cycle spec states the rule in full. **What a deployer must do about revocation.** Treat a provider switch as a one-way retirement of the identity population, and deal with the previous @@ -636,19 +665,19 @@ one acceptance contract: One row per divergence between the 2026-07-31 draft and the implementation this revision describes. -| Draft position | Implemented position | Why | -| ----------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Trait surface is a canonicalizing `parse` returning a typed id, plus `graph_key_suffix`, `cluster_prefix`, and `verify` | `accepts_id` (recognition) plus `normalize_id_for_kv` (KV key form), defaults matching the built-in shape. No typed id, key suffix, cluster capability, or `verify`. `keys_equal` stays out, as the draft required. | Recognition and KV keying are the two operations core performs today. A byte-for-byte round-trip test with a non-default provider pins the contract. | -| `GeneratedEdgeCookie::response_headers` and `IdentityInput.permissions` / `.consent` banned as speculative surface | Shipped with production consumers. Finalization applies provider headers, the resolve path returns them, and the organic create path populates the input fields. | The client-cycle resolve path landed in the same series and is their caller, satisfying the minimalism rule the ban enforced. | -| Identifier bounds enforced at create and parse | Enforced at create (`generate` and the resolve endpoint), cookie read-back, and cookie write. Violations rejected outright, never rewritten. `MAX_EC_ID_LEN` in `ec/cookies.rs`. | Every identifier entry point is covered, and the pre-epic sanitizing rewrite was removed as a silent-divergence hazard. | -| `provider = "none"` is valid alongside `legacy_providers` blocks | `none` (or an omitted selector) with any configured provider block is a startup error. | No `legacy_providers` exists in these PRs, so a block alongside statelessness can only be a mistake. | -| Every selection key is closed and unknown keys are startup errors | Device and geo keys are closed. EC vendor keys are open. Unknown blocks are captured as raw values in core, the adapter deserializes its own block, and a selected key with no injected provider fails loudly. | Core never names a vendor, so a vendor provider adds no core change. | -| Capability mismatch is a startup error at adapter wiring time | Configuration coherence fails at startup. A host-capability mismatch (missing `HostSignals`, uninjected vendor) fails loudly when the provider is built, stopping the request. | The adapter capability declaration that would move the check to startup is deferred with the capability matrix. | -| A creating provider with no identity-graph store is a startup error | Not implemented. `ec_store` stays optional. The resolve endpoint refuses to create without a graph. The organic path persists rows whenever the graph is configured. | Portability adapters run without platform KV. Whether configuration should force the pairing is follow-up work. | -| `[device] provider = "fastly"` is startup-rejected pending a separate security design | Shipped as a selectable opt-in. The Fastly adapter injects `HostSignals`, the provider strengthens the browser/bot gate, and rows persist derived classes, not raw signals. | Selection is an explicit operator opt-in and the neutral default makes no host signal call. | -| The `host-signals` EC provider is deliberately dropped and its selection rejected | Shipped in PR #1044 as an opt-in built-in that defers with a warning when the host supplies no signals. **Open, flagged for the series review**, not settled either way. | Its identifier shape shares the HMAC grammar, and a sign-off row defers host signal processing, so the review decides whether the provider ships in the series. | -| Geo default flip sequenced into the later permission-model step, with an acknowledgment guard | Landed as specified in the same series, with the default of none, `default_country` required and validated against `permissions.yaml`, the `assume_single_jurisdiction` acknowledgment, and a failed lookup resolving to the requires-signal floor with error logging (`GeoStatus`, resolved in core so all adapters agree). | The permission model shipped in PR #1045, so the constraints exist where the draft required them. | -| All adapters serve the full EC feature set identically | Selector behavior is identical through the shared builders and core constructors. The EC API routes (identify, batch-sync, ec/resolve) are Fastly-only, documented in the Spin route list. | The portability adapters do not yet wire platform KV, and the gap is documented rather than silent. | -| Conformance suite, adapter capability matrix, delimiter-free key grammar, `verify`, `legacy_providers`, `versions` / `mint_version` | None of these are in PR #1043 or #1044. All are tracked follow-up work, deferred, not silently dropped. | The shipped seam did not need them, and each returns with the feature that gives it a production caller, per the spec's own minimalism rule. | -| `required_permissions` removed from the device and geo traits, added to the EC trait only at the permission-model step | Present on all three traits from the start, with empty defaults. Core enforces the EC declaration (gate in `EcContext`, landing in PR #1045). No device or geo enforcement point exists. | One uniform declaration seam, with an empty default that gates nothing, avoids the decorative-gate hazard while keeping the interface stable. The geo circularity stands. | -| Flat crate directories (`crates/trusted-server-geo-fastly`), no placeholder directories | Nested directories per capability (`crates/device/fastly`, `crates/geo/fastly`, `crates/edgecookie/`), flat package names. `crates/edgecookie` ships a README before its first crate. | Nested directories scale per vendor, package names already carry the naming convention, and the README stakes out the vendor location. | +| Draft position | Implemented position | Why | +| ----------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Trait surface is a canonicalizing `parse` returning a typed id, plus `graph_key_suffix`, `cluster_prefix`, and `verify` | `accepts_id` (recognition) plus `normalize_id_for_kv` (KV key form), defaults matching the built-in shape. No typed id, key suffix, cluster capability, or `verify`. `keys_equal` stays out, as the draft required. | Recognition and KV keying are the two operations core performs today. A byte-for-byte round-trip test with a non-default provider pins the contract. | +| `GeneratedEdgeCookie::response_headers` and `IdentityInput.permissions` / `.consent` banned as speculative surface | Shipped with production consumers. Finalization applies provider headers, the resolve path returns them, and the organic create path populates the input fields. | The client-cycle resolve path landed in the same series and is their caller, satisfying the minimalism rule the ban enforced. | +| Identifier bounds enforced at create and parse | Enforced at create (`generate` and the resolve endpoint), cookie read-back, and cookie write. Violations rejected outright, never rewritten. `MAX_EC_ID_LEN` in `ec/cookies.rs`. | Every identifier entry point is covered, and the pre-epic sanitizing rewrite was removed as a silent-divergence hazard. | +| `provider = "none"` is valid alongside `legacy_providers` blocks | `none` (or an omitted selector) with any configured provider block is a startup error. | No `legacy_providers` exists in these PRs, so a block alongside statelessness can only be a mistake. | +| Every selection key is closed and unknown keys are startup errors | Device and geo keys are closed. EC vendor keys are open. Unknown blocks are captured as raw values in core, the adapter deserializes its own block, and a selected key with no injected provider fails loudly. | Core never names a vendor, so a vendor provider adds no core change. | +| Capability mismatch is a startup error at adapter wiring time | Configuration coherence fails at startup. A host-capability mismatch (missing `HostSignals`, uninjected vendor) fails loudly when the provider is built, stopping the request. | The adapter capability declaration that would move the check to startup is deferred with the capability matrix. | +| A creating provider with no identity-graph store is a startup error | Not implemented. `ec_store` stays optional. The resolve endpoint refuses to create without a graph. The organic path persists rows whenever the graph is configured. | Portability adapters run without platform KV. Whether configuration should force the pairing is follow-up work. | +| `[device] provider = "fastly"` is startup-rejected pending a separate security design | Shipped as a selectable opt-in. The Fastly adapter injects `HostSignals`, the provider strengthens the browser/bot gate, and rows persist derived classes, not raw signals. | Selection is an explicit operator opt-in and the neutral default makes no host signal call. | +| The `host-signals` EC provider is deliberately dropped and its selection rejected | Shipped in PR #1044 as an opt-in built-in that defers with a warning when the host supplies no signals. **Open, flagged for the series review**, not settled either way. | Its identifier shape shares the HMAC grammar, and a sign-off row defers host signal processing, so the review decides whether the provider ships in the series. | +| Geo default flip sequenced into the later permission-model step, with an acknowledgment guard | Landed as specified in the same series, with the default of none, `default_country` required and validated against `permissions.yaml`, the `assume_single_jurisdiction` acknowledgment, and a failed lookup resolving to the requires-signal floor with error logging (`GeoStatus`, resolved in core so all adapters agree). | The permission model shipped in PR #1045, so the constraints exist where the draft required them. | +| All adapters serve the full EC feature set identically | Selector behavior is identical through the shared builders and core constructors. The EC API routes (identify, batch-sync, ec/resolve) are Fastly-only, documented in the Spin route list. | The portability adapters do not yet wire platform KV, and the gap is documented rather than silent. | +| Conformance suite, adapter capability matrix, delimiter-free key grammar, `verify`, `legacy_providers`, `versions` / `mint_version` | None of these are in PR #1043 or #1044. All are tracked follow-up work, deferred, not silently dropped. | The shipped seam did not need them, and each returns with the feature that gives it a production caller, per the spec's own minimalism rule. | +| `required_permissions` removed from the device and geo traits, added to the EC trait only at the permission-model step | Present on all three traits from the start, with empty defaults. Core enforces the EC declaration (gate in `EcContext`, landing in PR #1045). No device or geo per-request enforcement point exists, so a nonempty device declaration from a module-supplied provider is refused when the registry resolves the selection and the deployment fails to start. | One uniform declaration seam keeps the interface stable, and refusing at startup what core cannot honor per request avoids the decorative-gate hazard the draft aimed at. The geo circularity stands. | +| Flat crate directories (`crates/trusted-server-geo-fastly`), no placeholder directories | Nested directories per capability (`crates/device/fastly`, `crates/geo/fastly`, `crates/edgecookie/`), flat package names. `crates/edgecookie` ships a README before its first crate. | Nested directories scale per vendor, package names already carry the naming convention, and the README stakes out the vendor location. | diff --git a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md index e61e3525c..53e9e9d9a 100644 --- a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md +++ b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md @@ -261,12 +261,12 @@ One vendor per PR, after this change lands. Each moves its Rust, its TypeScript, its config type and its tests into `crates/integrations/`, and the adapter that wants it depends on it. -| Vendor | What it needs | -| ---------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | -| Didomi, Google Tag Manager, Lockr, Osano, Permutive, Sourcepoint | Move as they are. Coupled only through the builder table, deploy validation and the JS map. | -| APS | Also needs the auction provider seam and the generalized renderer contract in §3.4, both of which this change delivers. | -| GPT (the `gpt` proxy and `gpt_diagnostics`) | The proxy moves as it is. The diagnostics half needs the prepare and finalize hooks in §3.5. | -| DataDome | Needs the neutral response-shaping hook in §3.5, and about forty test literals move with it. | +| Vendor | What it needs | +| ---------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Didomi, Google Tag Manager, Lockr, Osano, Permutive, Sourcepoint | Move as they are. Coupled only through the builder table, deploy validation and the JS map. | +| APS | Needs the auction provider seam and the generalized renderer contract in §3.4, both of which this change delivers, and browser-side work it does not deliver, because core TypeScript imports APS directly (§8 item 8). | +| GPT (the `gpt` proxy and `gpt_diagnostics`) | The proxy moves as it is. The diagnostics half needs the prepare and finalize hooks in §3.5. | +| DataDome | Needs the neutral response-shaping hook in §3.5, and about forty test literals move with it. | `[integrations.]` configuration tables need no change, because `IntegrationSettings` is a flattened map that already accepts unknown vendor @@ -349,7 +349,7 @@ that the project pays for today, most recently in PR #1054. ## 8. What implementing this found A probe integration built outside `trusted-server-core` and registered -through an adapter exercised every seam end to end. Seven things surfaced +through an adapter exercised every seam end to end. Eight things surfaced that reading the code did not, and they are recorded here rather than left for each vendor to rediscover. @@ -425,6 +425,23 @@ for each vendor to rediscover. adapter. The other three adapters expose both entry points. Fastly is the primary deployment target, so this one decides whether the seam is usable in production or only in the dev server. +8. **Core TypeScript imports APS directly, so generalizing the Rust + renderer alone does not move APS out.** On `main` at d516a9e94, + `crates/trusted-server-js/lib/src/core/auction.ts:5` imports + `parseApsRendererDescriptor` from `../integrations/aps/render` and calls + it at line 139, `crates/trusted-server-js/lib/src/core/request.ts:2` + imports `dispatchApsRendering` and `renderApsCreative` from the same + module and calls both at lines 56 to 59, and + `crates/trusted-server-js/lib/src/core/types.ts:69` fixes the shared + renderer type with `export type AuctionBidRenderer = ApsRendererV1`. + That coupling is pre-existing on `main` and is introduced by no PR in + this stack. §3.4 does not reach it either, because §3.4 generalizes the + Rust `BidRenderer` enum and the serialized descriptor, not the browser + code that consumes them. Moving APS therefore needs the browser side + generalized too, so that core TypeScript names no vendor. Designing + that, whether as a browser-side renderer registry or in some other + shape, is out of scope for this stack and belongs with the APS + migration in §4. Items 1, 6 and 7 are the ones a vendor meets on its first day, and item 7 decides whether any of this is reachable on the platform most deployments @@ -454,12 +471,13 @@ defines, and both should land before the first vendor is asked to use it. ## Revision record -| Date | Change | -| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| 2026-08-27 | First draft, written against `split/5-response-hook-docs`. | -| 2026-08-27 | Brought the bid renderer contract into scope (§3.4), so that after this change no vendor move needs a core change (§8 row 6). | -| 2026-08-28 | Corrected line references to `main` at b7fcb5d4c and added what mapping `main` found: composition of the served script moves into core (§3.2), registration enumeration and the auction-only `adserver_mock` case (§3.3), the duplicate-id gap (§3.1), the source-file guard and the `ts audit` vendor table (§4), the renderer risk (§7). | -| 2026-08-28 | Recorded what implementing the seam found (§8): the operator CLI skips a vendor's deploy rules, a carried module's hash literal is fragile, providers resolve more than once per request, and one core reader still reads an APS payload. Recorded the construction-time hash check in §3.2. | -| 2026-08-28 | Adopted the #1043 review's registration shape for identity and applied its rule to geo and device, with no provider built into core (§3.6, §6 item 2, §8 rows 7 to 9). Recorded the relationship to #986 and reordered the series so this spec and its implementation come first. | -| 2026-08-30 | Moved the five series design specs and the provider-code registry into this PR from PRs #1043 to #1047, so every normative document is reviewed before the code that implements it. Document content is unchanged, and only this status line and this row are new. | +| Date | Change | +| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-08-27 | First draft, written against `split/5-response-hook-docs`. | +| 2026-08-27 | Brought the bid renderer contract into scope (§3.4), so that after this change no vendor move needs a core change (§8 row 6). | +| 2026-08-28 | Corrected line references to `main` at b7fcb5d4c and added what mapping `main` found: composition of the served script moves into core (§3.2), registration enumeration and the auction-only `adserver_mock` case (§3.3), the duplicate-id gap (§3.1), the source-file guard and the `ts audit` vendor table (§4), the renderer risk (§7). | +| 2026-08-28 | Recorded what implementing the seam found (§8): the operator CLI skips a vendor's deploy rules, a carried module's hash literal is fragile, providers resolve more than once per request, and one core reader still reads an APS payload. Recorded the construction-time hash check in §3.2. | +| 2026-08-28 | Adopted the #1043 review's registration shape for identity and applied its rule to geo and device, with no provider built into core (§3.6, §6 item 2, §8 rows 7 to 9). Recorded the relationship to #986 and reordered the series so this spec and its implementation come first. | +| 2026-08-30 | Moved the five series design specs and the provider-code registry into this PR from PRs #1043 to #1047, so every normative document is reviewed before the code that implements it. Document content is unchanged, and only this status line and this row are new. | | 2026-08-31 | Corrected the counts and line references the review found, against `main` at d516a9e94: the source-file guard counts (§4), the prepare and finalize call-site counts (§3.5), the real double geo resolution on `POST /auction` (§8 item 3), the per-adapter preparer coverage (§8 item 5), and the `settings.rs`, `auction/mod.rs` and `publisher.rs` line references. §6 now requires the round trip on Fastly rather than on any adapter. | +| 2026-09-01 | Answered the review finding that generalizing the Rust `BidRenderer` does not move APS out. Recorded the pre-existing browser-side coupling in core TypeScript as §8 item 8, and corrected the APS migration row in §4 to name the browser-side work. Designing a browser-side renderer contract stays out of scope for this stack. | From 31b8f45c4c0a3f96b72dcb6bcc8dd7a384bbae93 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Tue, 1 Sep 2026 05:01:02 +0100 Subject: [PATCH 12/17] Qualify the completeness claim against the browser-side APS coupling Sign-off row 6 said no vendor move needs a core change after this design. Section 8 item 8 now records that TSJS core still imports the APS renderer directly, so an APS move needs a browser renderer contract that this design does not provide. Row 6 now claims only the Rust side, which is what the design actually delivers. --- .../specs/2026-08-27-integration-provider-seam-design.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md index 53e9e9d9a..d9eb5692a 100644 --- a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md +++ b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md @@ -464,7 +464,7 @@ defines, and both should land before the first vendor is asked to use it. | 3 | A registration may carry its own browser JavaScript | Proposed | | 4 | Deploy validation moves onto the registration | Proposed | | 5 | The nine existing integrations migrate one PR each, on the schedule in §4 | Proposed | -| 6 | This change is complete in itself, so after it no vendor move needs a core change | Proposed | +| 6 | This change completes the Rust side, so after it no vendor move needs a Rust core change. The browser side is not complete, because TSJS core still imports the APS renderer directly (§8 item 8), so an APS move still needs a browser renderer contract | Proposed | | 7 | Identity, geo and device providers are capabilities of a module registration (§3.6), the #1043 review's rule applied to all three | Proposed | | 8 | No provider is built into core: HMAC and the User-Agent-only device provider are Tech Lab-owned modules configured under `[integrations.]`, and core keeps only `none` | Proposed | | 9 | This spec and its core implementation precede #1043; 51Degrees implements the core seam, the nine vendor moves in §4 stay one PR each | Proposed | @@ -474,7 +474,7 @@ defines, and both should land before the first vendor is asked to use it. | Date | Change | | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 2026-08-27 | First draft, written against `split/5-response-hook-docs`. | -| 2026-08-27 | Brought the bid renderer contract into scope (§3.4), so that after this change no vendor move needs a core change (§8 row 6). | +| 2026-08-27 | Brought the bid renderer contract into scope (§3.4), so that after this change no vendor move needs a Rust core change (§8 row 6). The browser side is recorded as outstanding in §8 item 8. | | 2026-08-28 | Corrected line references to `main` at b7fcb5d4c and added what mapping `main` found: composition of the served script moves into core (§3.2), registration enumeration and the auction-only `adserver_mock` case (§3.3), the duplicate-id gap (§3.1), the source-file guard and the `ts audit` vendor table (§4), the renderer risk (§7). | | 2026-08-28 | Recorded what implementing the seam found (§8): the operator CLI skips a vendor's deploy rules, a carried module's hash literal is fragile, providers resolve more than once per request, and one core reader still reads an APS payload. Recorded the construction-time hash check in §3.2. | | 2026-08-28 | Adopted the #1043 review's registration shape for identity and applied its rule to geo and device, with no provider built into core (§3.6, §6 item 2, §8 rows 7 to 9). Recorded the relationship to #986 and reordered the series so this spec and its implementation come first. | From 4d1f4e0606568a716ef4fc32ea0515c16f820435 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Tue, 1 Sep 2026 14:05:42 +0100 Subject: [PATCH 13/17] Format the provider specs for the docs Prettier gate and correct two wording claims Prettier realigns table columns in six specs that the 1 September review edits widened, which the format-docs workflow checks. The client-cycle spec no longer says the origin check normalizes to an effective port, because origins_match compares the port as the browser wrote it and adds or removes nothing. The response-header-hook spec no longer claims Trusted Server rejects a malformed or body-inconsistent content-length, because no core code does that. It is HTTP framing handled by the host HTTP layer, and the point that matters is that no integration reads the value. --- ...26-07-30-client-cycle-ec-resolve-design.md | 23 +++--- ...integration-response-header-hook-design.md | 59 +++++++------ .../2026-07-30-permission-model-design.md | 82 +++++++++---------- .../2026-07-30-pluggable-providers-design.md | 6 +- ...07-30-provider-migration-rollout-design.md | 48 +++++------ ...-08-27-integration-provider-seam-design.md | 24 +++--- 6 files changed, 121 insertions(+), 121 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index 8b8b0a991..1ffde33ba 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -66,7 +66,8 @@ verify against. 1. **Reject cross-site requests with an origin check.** v1 authorizes a request only when its `Origin` header, serialized as scheme, lowercased - host and effective port, is byte-equal to an accepted origin. The + host and the port as the browser wrote it, is byte-equal to an accepted + origin. No port is added or removed on either side. The default accepted set is the single origin `https://{publisher.domain}` and nothing else, so a sibling subdomain, the `http://` scheme and a non-default port are all refused. A missing or foreign `Origin` is @@ -291,13 +292,13 @@ end in tests and demonstrations. ## 8. Revision record vs the 2026-07-31 draft -| Draft position | v1 (PR #1046) | Why | -| ------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Feature deferred, `resolve_from_client` de-normalized | Feature normative. Trait method kept with a no-op default | The first vendor integration works client-side by design, so the endpoint is on the critical path for the series' first real provider | -| Origin check via new allowlist config + CSRF token | Exact serialized-origin comparison (scheme, lowercased host, effective port). Default accepted set is the single origin `https://{publisher.domain}`, plus an optional operator-configured list of further exact origins. Missing/foreign `Origin` → `403` | The draft's exact allowlist is adopted. An earlier revision of this spec allowed any suffix match on the apex, which admitted every subdomain including ones the publisher may not control, and its justification did not hold. This closes §7.5 | -| Marker cookie survives whatever happens to the identity it marks | Core expires the `ts-ecr` marker on any request carrying a `ts-ec` the selected provider does not own | The marker is not namespaced by the provider code envelope, so without this a provider switch leaves a visitor with a marker, no identity, and a page script that will not re-post | -| §3.9 reservation/replay machinery required before code | Deferred to the vendor scheme. Draft text retained verbatim as the bar | The design needs the real envelope's unique id and session binding, and a CAS-class primitive no production adapter exposes today | -| Identical behavior or identical startup refusal, 4 ways | Fastly routes it (bot-gated graph). Portability adapters documented as deliberately not routing, like identify | Same platform-KV constraint as the existing EC API routes. Startup rejection follow-up recorded (§7.4) | -| Marker cookie or injected variable (design must pick) | Marker cookie (`ts-ecr`), expired with the EC cookie | The draft's own first option. Testable and observable | -| Demo gated by cargo feature or `#[cfg(test)]` | Both: `client-fixed-demo` feature + startup rejection in the settings validator | Defense in depth | -| Everything else in §3 (graph row, bounds, 409, no-store, full-declaration gate) | Implemented as specified | (none) | +| Draft position | v1 (PR #1046) | Why | +| ------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Feature deferred, `resolve_from_client` de-normalized | Feature normative. Trait method kept with a no-op default | The first vendor integration works client-side by design, so the endpoint is on the critical path for the series' first real provider | +| Origin check via new allowlist config + CSRF token | Exact serialized-origin comparison (scheme, lowercased host, port as written). Default accepted set is the single origin `https://{publisher.domain}`, plus an optional operator-configured list of further exact origins. Missing/foreign `Origin` → `403` | The draft's exact allowlist is adopted. An earlier revision of this spec allowed any suffix match on the apex, which admitted every subdomain including ones the publisher may not control, and its justification did not hold. This closes §7.5 | +| Marker cookie survives whatever happens to the identity it marks | Core expires the `ts-ecr` marker on any request carrying a `ts-ec` the selected provider does not own | The marker is not namespaced by the provider code envelope, so without this a provider switch leaves a visitor with a marker, no identity, and a page script that will not re-post | +| §3.9 reservation/replay machinery required before code | Deferred to the vendor scheme. Draft text retained verbatim as the bar | The design needs the real envelope's unique id and session binding, and a CAS-class primitive no production adapter exposes today | +| Identical behavior or identical startup refusal, 4 ways | Fastly routes it (bot-gated graph). Portability adapters documented as deliberately not routing, like identify | Same platform-KV constraint as the existing EC API routes. Startup rejection follow-up recorded (§7.4) | +| Marker cookie or injected variable (design must pick) | Marker cookie (`ts-ecr`), expired with the EC cookie | The draft's own first option. Testable and observable | +| Demo gated by cargo feature or `#[cfg(test)]` | Both: `client-fixed-demo` feature + startup rejection in the settings validator | Defense in depth | +| Everything else in §3 (graph row, bounds, 409, no-store, full-declaration gate) | Implemented as specified | (none) | diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index c297d95c9..dabfb2aa2 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -389,12 +389,12 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may response fits after all ordinary batches are removed. These are separate outcomes and metrics. - | Adapter | Header-count / total-bytes ceiling (capability cell) | - | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | - | Axum | no platform ceiling below the core budget (native HTTP stack), cell fixed at ≥ 128 headers / ≥ 32 KiB | + | Adapter | Header-count / total-bytes ceiling (capability cell) | + | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | + | Axum | no platform ceiling below the core budget (native HTTP stack), cell fixed at ≥ 128 headers / ≥ 32 KiB | | Fastly | **qualification-pending**: the measured platform ceiling is recorded in this cell by the adapter-qualification commit, and unrecorded ⇒ hook startup fails | - | Cloudflare | **qualification-pending**: same rule | - | Spin | **qualification-pending**: same rule | + | Cloudflare | **qualification-pending**: same rule | + | Spin | **qualification-pending**: same rule | A recorded cell below core's 128-header / 32 KiB budget is a startup error (shrink the core budget or raise the ceiling, never a silent @@ -404,13 +404,13 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may `qualification-pending` cell fails startup when the depending feature is selected: - | Capability | Fastly | Axum (dev) | Cloudflare | Spin | - | -------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- | --------------------------------------------------- | ------------------------------ | -------------------------------- | - | Runtime secret lookup for Vary-HMAC/DataDome | wired secret store, and qualify key-rotation behavior | dev secret binding required | qualify Workers secret binding | qualify component secret binding | - | Persisted processed artifact + mutation IR/read sets | qualification-pending | in-process dev implementation required, non-durable | qualification-pending | qualification-pending | - | Atomic artifact/metadata entry commit | qualification-pending | implementation required | qualification-pending | qualification-pending | - | `Vary` variant index + insert-new-then-index-update rekey | qualification-pending | implementation required | qualification-pending | qualification-pending | - | DataDome field-line order, trusted IP/port, fixed HTTPS backend/no-redirect, and exact form limits | qualification-pending | qualification-pending | qualification-pending | qualification-pending | + | Capability | Fastly | Axum (dev) | Cloudflare | Spin | + | -------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | --------------------------------------------------- | ------------------------------ | -------------------------------- | + | Runtime secret lookup for Vary-HMAC/DataDome | wired secret store, and qualify key-rotation behavior | dev secret binding required | qualify Workers secret binding | qualify component secret binding | + | Persisted processed artifact + mutation IR/read sets | qualification-pending | in-process dev implementation required, non-durable | qualification-pending | qualification-pending | + | Atomic artifact/metadata entry commit | qualification-pending | implementation required | qualification-pending | qualification-pending | + | `Vary` variant index + insert-new-then-index-update rekey | qualification-pending | implementation required | qualification-pending | qualification-pending | + | DataDome field-line order, trusted IP/port, fixed HTTPS backend/no-redirect, and exact form limits | qualification-pending | qualification-pending | qualification-pending | qualification-pending | | SecurityUse JA4 request evidence | platform value available, with exact-field/payload qualification and sign-offs 23/28 pending | unavailable | unavailable | unavailable | The qualification commit records storage lifetime, maximum object size, @@ -492,17 +492,17 @@ The 304 **safe-update set** is exactly `Cache-Control`, `Expires`, `Date`, adding any name requires a reviewed spec/registry revision and conformance fixture, never a runtime wildcard. -| Response | Hook runs? | -| ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Processed HTML document (rewritten by TS) | Yes | -| Streamed processed document | Yes, operations apply to the header block before first byte | -| Pass-through proxy response (not processed) | No, TS is a transparent proxy for it | -| Served-from-cache processed document | Serves the **persisted post-hook finals** captured at cache fill (revision-versioned, mismatch = miss), mutators run at fill/processing time only, so a normal hit and a conditional hit return identical policy metadata **by construction**, with no purity or determinism assumption about mutators | -| Redirect (3xx) | No | -| Error responses TS itself generates (4xx/5xx) | No | +| Response | Hook runs? | +| ------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Processed HTML document (rewritten by TS) | Yes | +| Streamed processed document | Yes, operations apply to the header block before first byte | +| Pass-through proxy response (not processed) | No, TS is a transparent proxy for it | +| Served-from-cache processed document | Serves the **persisted post-hook finals** captured at cache fill (revision-versioned, mismatch = miss), mutators run at fill/processing time only, so a normal hit and a conditional hit return identical policy metadata **by construction**, with no purity or determinism assumption about mutators | +| Redirect (3xx) | No | +| Error responses TS itself generates (4xx/5xx) | No | | `304 Not Modified` for a processed representation | **Persisted-metadata pass**: a cached processed 200 stores its final post-hook headers, accepted mutation-operation batches, and the union of every mutator's declared response-field read set (the persisted mutation IR), versioned by §3's complete cache revision tuple, including model epoch and logical activation generation. A local conditional hit re-emits persisted finals only when the complete tuple matches current active. An origin-revalidation 304 is staged and diffed against separately stored origin-side metadata. (a) Any byte-coupled field changed (`Content-Encoding`, `Content-Type`, validators, digests) → invalidate and fetch/process a full 200. (b) A changed metadata field that intersects any persisted mutator read set, or an artifact/mutator lacking a complete read-set declaration, is also unsafe → full 200 refetch and ordinary hook execution, because deterministic replay of old operations cannot stand in for re-evaluating a decision made from changed inputs. (c) If every changed field is outside every declared read set and belongs to the enumerated safe-update set (`Cache-Control`, reserved CDN cache fields, `Expires`, `Date`, `Age`, `Vary`, registry-admitted mutable fields), replay the persisted deterministic operations over updated origin metadata and rerun invariants. Updated origin metadata, finals, IR, read sets, and complete revision tuple publish in one atomic entry commit, and changed `Vary` uses insert-new-entry-then-index-update ordering. (d) No change → re-emit persisted finals. Artifact absence or any tuple mismatch triggers an unconditional recovery fetch so TS obtains bytes. For processed-document GET/HEAD routes, TS is explicitly the authoritative server for the transformed representation, and after processing the full 200 it evaluates RFC 9110 §13 preconditions against **processed** validators. This is not evaluation of origin validators by an intermediary cache. Other methods are never eligible for this recovery path. `Set-Cookie` and origin validators are never replayed. Absent metadata → cache miss | -| `HEAD` of a processed document | **Serves the persisted GET artifact when one exists**, parity with GET/304 by construction, since RFC 9111 §4.3.5 lets HEAD metadata update a stored GET response and mutators are not required to be deterministic. With no persisted artifact, HEAD is processed like a GET (headers only) and its finals stored as a **distinct head-only artifact type that never satisfies a later GET**, when a stored GET artifact exists a HEAD may **update** it only when the comparison, made against the separately stored **origin-side** metadata, never the processed artifact's (origin and rewritten lengths differ by construction, so the processed length is never the comparand and HEAD metadata never touches processed-side headers), finds validators and origin `Content-Length` matching and no byte-coupled representation field changed (RFC 9111 §4.3.5); qualifying updates land origin-side under the same atomic-commit discipline as the 304 rules, and any mismatch **invalidates** the stored GET artifact rather than updating it | -| Informational `1xx`, `204`, `205`, `206` | No, enumerated so adapters do not infer independently | +| `HEAD` of a processed document | **Serves the persisted GET artifact when one exists**, parity with GET/304 by construction, since RFC 9111 §4.3.5 lets HEAD metadata update a stored GET response and mutators are not required to be deterministic. With no persisted artifact, HEAD is processed like a GET (headers only) and its finals stored as a **distinct head-only artifact type that never satisfies a later GET**, when a stored GET artifact exists a HEAD may **update** it only when the comparison, made against the separately stored **origin-side** metadata, never the processed artifact's (origin and rewritten lengths differ by construction, so the processed length is never the comparand and HEAD metadata never touches processed-side headers), finds validators and origin `Content-Length` matching and no byte-coupled representation field changed (RFC 9111 §4.3.5); qualifying updates land origin-side under the same atomic-commit discipline as the 304 rules, and any mismatch **invalidates** the stored GET artifact rather than updating it | +| Informational `1xx`, `204`, `205`, `206` | No, enumerated so adapters do not infer independently | This deliberately narrows #782's general "outbound response" phrasing to processed documents (§6). @@ -965,12 +965,11 @@ byte-to-string replacement is forbidden: (`2026-08-27-integration-provider-seam-design.md`) is where that move is designed. This is a direction for that migration rather than a complaint about the code as it stands. -- `content-length` stays a protocol-level singleton. A repeated, malformed, - or body-inconsistent `content-length` is rejected by the shared HTTP - request boundary before any integration runs, which is an HTTP - correctness rule and not a vendor decision, so it is unchanged. - `PostParamLen` is always the byte length of the body actually presented to - core, not the numeric `content-length` value. +- `content-length` is not a vendor decision. It is HTTP framing, parsed by + the host HTTP layer beneath Trusted Server before any integration runs, + and no integration reads it as a value. `PostParamLen` is always the byte + length of the body actually presented to core, not the numeric + `content-length` value, so the header never reaches the vendor payload. - `authorization` never reaches the vendor payload as a value. `AuthorizationLen` is the byte length of the OWS-normalized value. When more than one `authorization` line arrives, every line is presented in @@ -1230,8 +1229,8 @@ This spec supersedes #782 on the following points. The issue is updated to reference this spec when the implementing PR (the hook's return with its first consumer, §7) merges: -| #782 says | This spec says | Why | -| -------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | +| #782 says | This spec says | Why | +| -------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | | Mutations apply to the outbound response generally | Eligibility is the explicit §3a matrix, centered on processed documents | Pass-through/error/304 mutation has different semantics per adapter, and enumerating beats implying | | Ship the trait + registry + adapter application | Additionally: structured operations API (§2), reserved surface (§3), and a real consumer in the same PR (§4, item 3) | PR #838 shipped the trait with zero call sites, and an unrestricted `&mut HeaderMap` cannot enforce any collision policy | diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index 6e9438a4b..fe806cbe2 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -627,19 +627,19 @@ requires a live user signal (§4.2), never a policy change. ## 6. Failure-mode matrix (normative, implemented) -| Condition | Resolution behavior | -| ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Condition | Resolution behavior | +| ------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Geo lookup reports a failure | Requires-signal floor for every permission, never the deployer default, with the error logged. No provider shipped today can report one, so a host geo outage lands on the row above instead (§5.2) | -| No geo provider configured | `default_country` baseline, guarded by `assume_single_jurisdiction` (§5.3) | -| Country resolved, no matching rule | `default_country` baseline (§5.4) | -| Region resolved, no region rule | Country rule | -| `default_country` unset or names no rule | Startup failure (§3.3) | -| EC provider configured, no geo, no acknowledgment | Startup failure (§5.3) | -| Malformed `permissions.yaml` | Parse error at settings load, once per instance, because the embedded file is a build-time constant, never per-request | -| Undecodable record present (TCF, GPP, or USP) | Revokes every Data Use (fail-closed acquisition), never withdraws, and still honors opt-outs | -| Expired TCF record | Distinct state, not malformed, and treated as absent, so the baseline applies | -| Signals contradict (opt-out plus consent) | Opt-out wins (§4) | -| No EC provider selected | Identity fails closed, with nothing created and an incoming cookie value never used or egressed (§7) | +| No geo provider configured | `default_country` baseline, guarded by `assume_single_jurisdiction` (§5.3) | +| Country resolved, no matching rule | `default_country` baseline (§5.4) | +| Region resolved, no region rule | Country rule | +| `default_country` unset or names no rule | Startup failure (§3.3) | +| EC provider configured, no geo, no acknowledgment | Startup failure (§5.3) | +| Malformed `permissions.yaml` | Parse error at settings load, once per instance, because the embedded file is a build-time constant, never per-request | +| Undecodable record present (TCF, GPP, or USP) | Revokes every Data Use (fail-closed acquisition), never withdraws, and still honors opt-outs | +| Expired TCF record | Distinct state, not malformed, and treated as absent, so the baseline applies | +| Signals contradict (opt-out plus consent) | Opt-out wins (§4) | +| No EC provider selected | Identity fails closed, with nothing created and an incoming cookie value never used or egressed (§7) | The posture is fail-closed. Every ambiguous state resolves to the configured baseline or more restrictive. @@ -789,12 +789,12 @@ their deferred features. This spec supersedes #779 on the following points, so there is one acceptance contract, not two: -| #779 says | This spec says | Why | -| -------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | -| Unmatched countries fall to `default_country` | Adopted. Unmatched and unresolved requests both fall to the required `[geo] default_country`, and a **failed** lookup floors instead (§5) | The failure state is the one that must never reach a permissive default, and the draft's `rules.default` split was not kept | -| The full TCF purpose vocabulary is modeled | Adopted and extended. All eleven purposes are signal-resolved, and the full Privacy Taxonomy is carried as declared baseline (§2) | The joint taxonomy work made whole-taxonomy declaration the goal, and `denied` defaults keep undeclared uses inert | -| Policy is an embedded file | Adopted. `permissions.yaml` is compiled into the build (§3.1), and runtime configuration is deferred follow-up | The runtime push and activation pipeline does not exist, and version control is the audit trail meanwhile | -| Permission sources are open-ended (#777: publisher interaction, external services may grant) | Sources are jurisdiction, policy, and the §4 signals. Further sources are deferred, and the `ConsentSignal` closure is their seam (§1) | Shipping an interface with no second source repeats the inert-surface mistake, and the extension seam is defined | +| #779 says | This spec says | Why | +| -------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | +| Unmatched countries fall to `default_country` | Adopted. Unmatched and unresolved requests both fall to the required `[geo] default_country`, and a **failed** lookup floors instead (§5) | The failure state is the one that must never reach a permissive default, and the draft's `rules.default` split was not kept | +| The full TCF purpose vocabulary is modeled | Adopted and extended. All eleven purposes are signal-resolved, and the full Privacy Taxonomy is carried as declared baseline (§2) | The joint taxonomy work made whole-taxonomy declaration the goal, and `denied` defaults keep undeclared uses inert | +| Policy is an embedded file | Adopted. `permissions.yaml` is compiled into the build (§3.1), and runtime configuration is deferred follow-up | The runtime push and activation pipeline does not exist, and version control is the audit trail meanwhile | +| Permission sources are open-ended (#777: publisher interaction, external services may grant) | Sources are jurisdiction, policy, and the §4 signals. Further sources are deferred, and the `ConsentSignal` closure is their seam (§1) | Shipping an interface with no second source repeats the inert-surface mistake, and the extension seam is defined | ## 11. Revision record vs the 2026-07-31 draft @@ -802,26 +802,26 @@ One row per divergence between the draft and the implementation this revision was verified against (branch `split/5-response-hook-docs`, PR #1045). -| Draft position | Implemented position | Why | -| --------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Vocabulary is two TCF-purpose identifiers, enforced permissions only (§2) | IAB Privacy Taxonomy Data Uses, with eleven named purposes all signal-resolved, plus 53 taxonomy Data Uses carried as declared but unenforced baseline flags | The joint taxonomy adoption postdates the draft, and whole-taxonomy declaration serves completeness and demonstration, with `denied` defaults keeping unenforced flags inert | -| Policy lives in `[permissions]` in `trusted-server.toml`, published via `ts config push` (§3.1) | Policy is `permissions.yaml`, compiled into the build with `include_str!`, parsed once and covered by tests | The runtime config push and activation apparatus does not exist, so publishing runtime policy without it would recreate the hazards the draft cataloged. Runtime policy is deferred follow-up | -| Every group carries a required `regime` class read by auction dispatch (§3.2) | No regime field exists | Its only consumer, regime-gated dispatch, was not migrated, and the field returns with that work | -| Overrides name explicit acquisition rules, replacing the `+`/`-` sigils (§3.2) | Adopted. A detailed rule's `permissions` map assigns `granted`, `requires_signal`, or `denied` per Data Use | The draft's requirement, expressed in the YAML schema. `requires_signal` is now expressible per rule | -| Two fallbacks: policy `rules.default` for unmatched countries, `default_country` only for the static no-geo mode (§5.4) | One required `[geo] default_country` covers unmatched and unresolved requests in every mode, startup-validated, and a failed lookup floors separately | One deployer knob is simpler, and the safety-critical separation kept is failure vs absence, carried by `GeoStatus` | -| Validation checks assigned ISO 3166-1 codes, assigned subdivisions, and a group identifier grammar (§3.3) | Validation covers unknown groups, permissions, acquisitions, revoke rules, incomplete groups, case-duplicate rule keys, and unknown fields on detailed rules | Smaller surface shipped first, and the EU/EEA coverage test guards the shipped table against the typo class. ISO-assignment checks are future hardening | -| Three-class signal taxonomy with regime-scoped grant acceptance, where the US posture is `requires_signal` with GPP/USP non-opt-out values as grants (§4) | TCF is the only grant source, the US posture is a `granted` baseline that opt-outs revoke, and explicit non-opt-out values grant nothing | A simpler two-signal model without regimes. The cost, no-signal US traffic is allowed by baseline rather than blocked pending a signal, is a deliberate policy choice in the shipped file | -| Malformed-present blocks grants per record family and mapped section (§4.4, §4.5) | Any present-but-undecodable record revokes every Data Use for the request | Strictly more restrictive simplification, and per-family scoping needs the full §4.5 decoder work | -| Normalization runs expiry before conflict resolution, a declared change (§4.4) | Conflict resolution still runs before the expiry check | The reordering was not implemented, though the expired state itself (distinct from malformed, absent for acquisition) was adopted | -| Persisted-KV consent flows through the full normalization pipeline with an explicit TTL comparison (§4.4) | The loaded record substitutes directly when the request carries no signals, jurisdiction re-derived, and staleness is enforced by the store TTL (`max_consent_age_days`) | The store-level TTL delivers the staleness bound without a second normalization pass | -| Proxy mode gains minimal opt-out extraction (§4.4) | Proxy mode still skips decoding, so a present record blocks all grants via the malformed-present rule and the GPC header opt-out is honored without decoding | The permission-layer outcome is equally or more restrictive with no new decode paths, and this is revisited with the §4.5 decoder work | -| Withdrawal has four triggers including an explicit storage-withdrawal or authenticated deletion request (§4.2) | The TCF Purpose 1 refusal under a non-granted baseline is the only trigger, and opt-outs, malformed records, absence, and policy changes never withdraw (adopted) | No deletion endpoint exists to carry the extra trigger, so the narrowest destructive surface shipped first | -| §4.3 durability protocol: family records first, suppression and authority-state records, outbox, breaker, strong reads | Cookie expiry plus best-effort identity-graph tombstones per presented identifier, with failures logged | The protocol requires storage primitives (linearizable CAS, independent durability domains) the adapters do not yet qualify, so it is deferred with the providers-spec storage work | -| §4.5 field mapping and §4.5.1 vendored registry snapshot (sharing/targeted opt-outs, embedded GPC, applicability, derived `gpp_sid`) | Opt-out sources are the GPC header, a GPP sale opt-out, and a USP sale opt-out. The revoke set is policy-declared, shipped as `all` (which also drops storage) | The full decoder and registry vendoring are their own project, and the policy-declared revoke set gives deployers the scoping lever meanwhile | -| §5.5 activation: JCS policy digests, ordinals, activation register, journal, drains, admission leases | None of it exists, and the built binary is the policy identity | With no runtime policy there is nothing to activate, and the draft remains the reference design for the runtime-config follow-up | -| §3.4 single jurisdiction truth, and §7 dispatch gated on the policy regime with a contextual projection | Auction dispatch keeps the consent-subsystem gate (effective TCF Purpose 1 for GDPR or unknown jurisdictions). `detect_jurisdiction` and its lists remain, with no contextual view | Dispatch migration is follow-up. The legacy-list drift risk the draft named still stands and is recorded rather than resolved | -| Every raw-EC egress path is pair-gated, with per-row tests and a denylist check (§7) | Pair gating is centralized in `ec_sharing_allowed` (auction endpoint `user.id`, publisher navigation and page-bids `user.id`, identify, pull sync) and `gate_eids_by_permissions` (EIDs everywhere). Batch sync checks row state only | Partial adoption. Aligning the remaining paths, the S2S stored-provenance authority, and the inventory tests is recorded follow-up | -| Identity rows never store raw consent strings, only normalized provenance and a digest (§1) | The identity-graph entry stores the raw TCF and GPP strings with the row | The normalized provenance schema belongs to the providers-spec storage work, and until then rows carry the raw strings | -| No signals block in policy, and the signal mapping is fixed in the spec | New. A `signals` section in `permissions.yaml` declares the TCF purpose map, opt-out sources, and revoke set, with `tcf.authoritative` governing only TCF's own effect | Moves signal policy from code into deployer-editable data, and the flag can never let a TCF record override an opt-out, preserving §4 precedence | -| The §5.3 no-geo guard covers every jurisdiction consumer | The guard fires when an Edge Cookie provider is configured with no geo provider | The EC provider is the only policy-gated consumer today, and the trigger list grows when dispatch and further egress paths join the model | -| `default_country` is required only in the acknowledged static no-geo mode (§5.4) | Required always and startup-validated against `permissions.yaml` | It is the baseline for unmatched requests in every mode, so it must always exist | +| Draft position | Implemented position | Why | +| --------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Vocabulary is two TCF-purpose identifiers, enforced permissions only (§2) | IAB Privacy Taxonomy Data Uses, with eleven named purposes all signal-resolved, plus 53 taxonomy Data Uses carried as declared but unenforced baseline flags | The joint taxonomy adoption postdates the draft, and whole-taxonomy declaration serves completeness and demonstration, with `denied` defaults keeping unenforced flags inert | +| Policy lives in `[permissions]` in `trusted-server.toml`, published via `ts config push` (§3.1) | Policy is `permissions.yaml`, compiled into the build with `include_str!`, parsed once and covered by tests | The runtime config push and activation apparatus does not exist, so publishing runtime policy without it would recreate the hazards the draft cataloged. Runtime policy is deferred follow-up | +| Every group carries a required `regime` class read by auction dispatch (§3.2) | No regime field exists | Its only consumer, regime-gated dispatch, was not migrated, and the field returns with that work | +| Overrides name explicit acquisition rules, replacing the `+`/`-` sigils (§3.2) | Adopted. A detailed rule's `permissions` map assigns `granted`, `requires_signal`, or `denied` per Data Use | The draft's requirement, expressed in the YAML schema. `requires_signal` is now expressible per rule | +| Two fallbacks: policy `rules.default` for unmatched countries, `default_country` only for the static no-geo mode (§5.4) | One required `[geo] default_country` covers unmatched and unresolved requests in every mode, startup-validated, and a failed lookup floors separately | One deployer knob is simpler, and the safety-critical separation kept is failure vs absence, carried by `GeoStatus` | +| Validation checks assigned ISO 3166-1 codes, assigned subdivisions, and a group identifier grammar (§3.3) | Validation covers unknown groups, permissions, acquisitions, revoke rules, incomplete groups, case-duplicate rule keys, and unknown fields on detailed rules | Smaller surface shipped first, and the EU/EEA coverage test guards the shipped table against the typo class. ISO-assignment checks are future hardening | +| Three-class signal taxonomy with regime-scoped grant acceptance, where the US posture is `requires_signal` with GPP/USP non-opt-out values as grants (§4) | TCF is the only grant source, the US posture is a `granted` baseline that opt-outs revoke, and explicit non-opt-out values grant nothing | A simpler two-signal model without regimes. The cost, no-signal US traffic is allowed by baseline rather than blocked pending a signal, is a deliberate policy choice in the shipped file | +| Malformed-present blocks grants per record family and mapped section (§4.4, §4.5) | Any present-but-undecodable record revokes every Data Use for the request | Strictly more restrictive simplification, and per-family scoping needs the full §4.5 decoder work | +| Normalization runs expiry before conflict resolution, a declared change (§4.4) | Conflict resolution still runs before the expiry check | The reordering was not implemented, though the expired state itself (distinct from malformed, absent for acquisition) was adopted | +| Persisted-KV consent flows through the full normalization pipeline with an explicit TTL comparison (§4.4) | The loaded record substitutes directly when the request carries no signals, jurisdiction re-derived, and staleness is enforced by the store TTL (`max_consent_age_days`) | The store-level TTL delivers the staleness bound without a second normalization pass | +| Proxy mode gains minimal opt-out extraction (§4.4) | Proxy mode still skips decoding, so a present record blocks all grants via the malformed-present rule and the GPC header opt-out is honored without decoding | The permission-layer outcome is equally or more restrictive with no new decode paths, and this is revisited with the §4.5 decoder work | +| Withdrawal has four triggers including an explicit storage-withdrawal or authenticated deletion request (§4.2) | The TCF Purpose 1 refusal under a non-granted baseline is the only trigger, and opt-outs, malformed records, absence, and policy changes never withdraw (adopted) | No deletion endpoint exists to carry the extra trigger, so the narrowest destructive surface shipped first | +| §4.3 durability protocol: family records first, suppression and authority-state records, outbox, breaker, strong reads | Cookie expiry plus best-effort identity-graph tombstones per presented identifier, with failures logged | The protocol requires storage primitives (linearizable CAS, independent durability domains) the adapters do not yet qualify, so it is deferred with the providers-spec storage work | +| §4.5 field mapping and §4.5.1 vendored registry snapshot (sharing/targeted opt-outs, embedded GPC, applicability, derived `gpp_sid`) | Opt-out sources are the GPC header, a GPP sale opt-out, and a USP sale opt-out. The revoke set is policy-declared, shipped as `all` (which also drops storage) | The full decoder and registry vendoring are their own project, and the policy-declared revoke set gives deployers the scoping lever meanwhile | +| §5.5 activation: JCS policy digests, ordinals, activation register, journal, drains, admission leases | None of it exists, and the built binary is the policy identity | With no runtime policy there is nothing to activate, and the draft remains the reference design for the runtime-config follow-up | +| §3.4 single jurisdiction truth, and §7 dispatch gated on the policy regime with a contextual projection | Auction dispatch keeps the consent-subsystem gate (effective TCF Purpose 1 for GDPR or unknown jurisdictions). `detect_jurisdiction` and its lists remain, with no contextual view | Dispatch migration is follow-up. The legacy-list drift risk the draft named still stands and is recorded rather than resolved | +| Every raw-EC egress path is pair-gated, with per-row tests and a denylist check (§7) | Pair gating is centralized in `ec_sharing_allowed` (auction endpoint `user.id`, publisher navigation and page-bids `user.id`, identify, pull sync) and `gate_eids_by_permissions` (EIDs everywhere). Batch sync checks row state only | Partial adoption. Aligning the remaining paths, the S2S stored-provenance authority, and the inventory tests is recorded follow-up | +| Identity rows never store raw consent strings, only normalized provenance and a digest (§1) | The identity-graph entry stores the raw TCF and GPP strings with the row | The normalized provenance schema belongs to the providers-spec storage work, and until then rows carry the raw strings | +| No signals block in policy, and the signal mapping is fixed in the spec | New. A `signals` section in `permissions.yaml` declares the TCF purpose map, opt-out sources, and revoke set, with `tcf.authoritative` governing only TCF's own effect | Moves signal policy from code into deployer-editable data, and the flag can never let a TCF record override an opt-out, preserving §4 precedence | +| The §5.3 no-geo guard covers every jurisdiction consumer | The guard fires when an Edge Cookie provider is configured with no geo provider | The EC provider is the only policy-gated consumer today, and the trigger list grows when dispatch and further egress paths join the model | +| `default_country` is required only in the acknowledged static no-geo mode (§5.4) | Required always and startup-validated against `permissions.yaml` | It is the baseline for unmatched requests in every mode, so it must always exist | diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index 8bfa1840a..3585f5833 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -125,7 +125,7 @@ an EC value through the selected provider: | **Create** | EC generation on first eligible request, and the client-cycle resolve endpoint | The provider returns the identifier (`generate` server-side, `resolve_from_client` for the client cycle) and only core writes the cookie, after enforcing the global bounds below. | | **Recognize** | Reading `ts-ec` back from the request, deciding `ec_was_present`, withdrawal checks, and every path that hands the value onward: the origin URL in `append_ec_id`, the click-target URL in `handle_first_party_click`, and the proxied body an integration builds | `accepts_id` answers whether a value is a well-formed identifier the provider issues. A value the selected provider does not recognize is treated as absent, so it is never used or egressed, while the raw cookie value stays visible to withdrawal handling. The egress paths reach the same answer through `edge_cookie::recognized_ec_id`, and a deployment with no provider selected recognizes nothing and so egresses nothing. | | **KV key** | Identity-graph row reads and writes | `normalize_id_for_kv` returns the key form. The default lowercases the built-in HMAC hash segment and preserves the suffix, keeping today's keys. An opaque or case-sensitive provider overrides to the identity function so distinct identifiers never collapse into one row. | -| **Withdraw** | Expiring the cookie and writing revocation markers | The identifiers eligible for a **graph tombstone** are exactly those the selected provider owns, dispatched on the `{code}~` prefix first and then `accepts_id`, never a shape check the provider cannot influence. Expiring the **cookie** is broader, because it keys off the raw cookie being present, so it still fires for an identifier the selected provider does not own (see the switching case, §6.1). | +| **Withdraw** | Expiring the cookie and writing revocation markers | The identifiers eligible for a **graph tombstone** are exactly those the selected provider owns, dispatched on the `{code}~` prefix first and then `accepts_id`, never a shape check the provider cannot influence. Expiring the **cookie** is broader, because it keys off the raw cookie being present, so it still fires for an identifier the selected provider does not own (see the switching case, §6.1). | **Invariant:** for every provider `P` and every identifier `id` created by `P`, `id` round-trips read-back byte for byte. A test in `ec/mod.rs` proves @@ -375,7 +375,7 @@ when the provider is built, stopping the request rather than degrading. | `provider = "none"` (explicit stateless) | Valid, and means exactly what omitting the selector means. Any configured provider block alongside it is a startup error, the same stray-block rule as below. | | A configured `[ec.providers.]` block that is not the selected one | **Startup error** (checked for the `hmac` block and every vendor block). An unreferenced block is almost always a mistyped selector or a stale block, and accepting it silently invites configuration drift. | | A selected vendor key whose provider the adapter did not inject | Loud failure when the provider is built, naming the key, so the deployment never silently runs stateless. | -| `provider = "host-signals"` on a host that supplies no signals | Loud failure when the provider is built. A host that cannot produce `HostSignals` cannot run the provider. | +| `provider = "host-signals"` on a host that supplies no signals | Loud failure when the provider is built. A host that cannot produce `HostSignals` cannot run the provider. | | `provider = "client-fixed"` in a production build | Startup error. The demonstration provider is compiled only behind the `client-fixed-demo` cargo feature. | | No `provider`, no providers block | Valid, the neutral default for that concern. | | Deprecated `[ec] passphrase` | Migrated to `provider = "hmac"` with the passphrase in `[ec.providers.hmac]`, with a deprecation warning naming the new location. Both forms together are rejected so a half-edited file fails loudly instead of one form silently winning. | @@ -473,7 +473,7 @@ logged, none silent: | `generate` returns an error | No identity this request. The organic caller logs at error level and the request proceeds stateless. No cookie is written. | | A provider creates an identifier outside the global bounds | Rejected at create, never rewritten. The organic path yields no identity. The resolve endpoint returns 400. | | Identity-graph write fails at create | The create is undone (no identifier, no cookie), with the error logged. The resolve endpoint returns 503. The next eligible request retries. | -| The host-signals provider finds no TLS/HTTP-2 signals | Defers with a warning. No identity this request, and no degraded IP-only identifier is created under the host-signals name. | +| The host-signals provider finds no TLS/HTTP-2 signals | Defers with a warning. No identity this request, and no degraded IP-only identifier is created under the host-signals name. | | Geo lookup **fails** (the provider errors) | Every permission resolves to the requires-signal floor, and the failure is logged at error level. The failure is **not** papered over with the `default_country` baseline. | | Geo resolves **no location**, or a country/region with no rule | The `[geo] default_country` baseline applies. This is the configured-default case, deliberately distinct from the failure row above (`GeoStatus` in `ec/consent.rs`). | | An incoming cookie value fails the bounds at read-back | Treated as absent, with a warning naming the source. | diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md index bea38238c..8d8d4c3ce 100644 --- a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -125,16 +125,16 @@ for the follow-up work that will implement them. | 3a | US-state request, explicit not-opted-out GPP/USP value → EC allowed | No US signal grants anything. N/A, absent, reserved, and unknown values grant nothing | Implemented (#1045) in a stricter form than drafted, where signals only revoke, so the drafted "explicit not-opted-out may grant" class does not exist (sign-offs 3, 17) | | 3b | US-state request, TCF record refusing Purpose 1, no US opt-out → no EC | Refusal beats coexisting non-TCF grant signals | Implemented (#1045), where an authoritative TCF refusal revokes its mapped uses. Under the shipped `granted` US baseline the refusal suppresses without tombstoning | | 3c | Consent-record conflict modes, expiry, KV fallback, proxy mode | Per the permission spec §4.4 matrix, whose changed row is that malformed-present blocks acquisition | Malformed-present fail-closed is implemented (#1045). The rest of the consent normalization pipeline is carried forward unchanged by the series | -| 3d | Valid plus expired consent records, where conflict resolution can select the expired record | Expired sources drop before conflict resolution | Open. Not addressed by the series | +| 3d | Valid plus expired consent records, where conflict resolution can select the expired record | Expired sources drop before conflict resolution | Open. Not addressed by the series | | 3e | Only the GPP sale field (and USP) is consulted, with sharing/targeted opt-outs ignored | Sale, sharing, and targeted-advertising opt-outs deny the personalised-ads uses, and none affects storage or destroys identity | Partially implemented (#1045), where sale, USP, and Sec-GPC are honored and never destructive. The sharing and targeted-advertising GPP fields are not yet decoded (sign-off 3 open) | | 3f | Non-privacy-state US traffic (for example Wyoming) is non-regulated → EC allowed | Country-level `US` is a protective floor, region rules may be stricter, and regionless traffic never degrades to non-regulated | Implemented (#1045), where `permissions.yaml` maps country `US` to `us-opt-out`, with state rows able to override | -| 3g | Graph rows persist JA4 class, H2 signal hash, and buyer-facing quality metadata | The draft discontinued these for new rows | **Contradicted by the series, flagged.** Rows still persist device signals, including signal hash prefixes, when the opt-in providers run (#1044/#1046). Tied to the open host-signal question (sign-off 22) | +| 3g | Graph rows persist JA4 class, H2 signal hash, and buyer-facing quality metadata | The draft discontinued these for new rows | **Contradicted by the series, flagged.** Rows still persist device signals, including signal hash prefixes, when the opt-in providers run (#1044/#1046). Tied to the open host-signal question (sign-off 22) | | 4 | UK request, no TCF record → no EC | Same, unless the policy deliberately adopts a `granted` storage baseline for GB with citation and sign-off | **The shipped default adopts `granted` storage for GB (#1045), flagged.** The citation and sign-off the draft required do not exist yet. The task force owns this row | | 5 | No country resolvable (geo failure) → no EC (fail-closed) | Protective failure profile, where permissions resolve at the requires-signal floor and `default_country` is reserved for unmatched requests in acknowledged static-jurisdiction mode | Implemented (#1045), where a failed lookup resolves at the requires-signal floor, logged at error level, and never falls back to `default_country` (sign-off 18) | | 6 | Non-regulated country, TCF record refusing Purpose 1 → EC still created, identity never tombstoned | Refusal blocks new grants everywhere, and existing identity is never tombstoned where the baseline is `granted` | Implemented (#1045), where an authoritative refusal revokes its mapped uses everywhere and withdrawal is scoped to non-granted baselines | | 7 | Country resolved but in no regulation list → EC created, EIDs pass through | Governed by the deployment's default rule. The implementation expresses this as the required `[geo] default_country`, naming the `permissions.yaml` rule for unmatched requests | Implemented (#1045) with a changed mechanism, since no `rules.default` entry exists and `default_country` is required and validated at startup | | 8 | Opt-out signal outside US states → ignored today | Mapped use restrictions are honored globally, and opt-outs never tombstone identity | Implemented (#1045), where the signal mapping is jurisdiction-free and suppresses even TCF-consented uses, without destruction (sign-off 1) | -| 9 | Fastly bot gate requires JA4 plus platform class before KV-backed EC writes | The draft deferred host signal processing and startup-failed `[device] provider = "fastly"` | **Contradicted by the series, flagged.** The `fastly` device provider ships opt-in with `builtin` (UA-only) as the default (#1044). Whether the host-signal surface stays is sign-off 22, open | +| 9 | Fastly bot gate requires JA4 plus platform class before KV-backed EC writes | The draft deferred host signal processing and startup-failed `[device] provider = "fastly"` | **Contradicted by the series, flagged.** The `fastly` device provider ships opt-in with `builtin` (UA-only) as the default (#1044). Whether the host-signal surface stays is sign-off 22, open | | 10 | Fastly always resolves geo per request | Only with `[geo] provider = "platform"`. The neutral default flips only together with the permission model's jurisdiction guard, never in an intermediate step | Implemented as sequenced (#1044 kept the platform default, and #1045 flipped geo off by default together with `default_country` and the acknowledgment guard) | | 11a | Raw EC egress on jurisdiction-gated paths today (`user.id`, EIDs, identify, pull sync) | Gated by the sharing pair (storage plus personalised-ads), at least as strict as today for every path | Implemented (#1045), where `ec_sharing_allowed` gates `user.id`, the identify response, and pull sync, and `gate_eids_by_permissions` gates EIDs, all on the same pair | | 11b | Proxy / click / Testlight forwarding extract the raw EC cookie/header without today's jurisdiction gate | Gated by the egress inventory (both purposes) | Open. Not implemented by the series (sign-off 8) | @@ -469,23 +469,23 @@ the code with the row open. | 6 | Raw regulatory strings reach only the positively registered OpenRTB field that requires each source. All other destinations default deny. Identity rows retain normalized provenance/digests, not raw consent snapshots. | permission §7; providers §6.3 | (none) | open. Not addressed by the series | | 7 | Reject legacy batch-sync traffic until live-browser provenance backfill makes the row re-evaluable. | rollout §6 item 6; permission §7 | (none) | open. Not implemented (no provenance exists to recompute) | | 8 | Gate proxy, click, and Testlight identity forwarding on the sharing pair (storage plus personalised-ads). | §2 row 11b | (none) | open. Not implemented by the series | -| 9 | Defer integration-owned cookie operations from the v1 response hook, and require a complete read/use/withdraw lifecycle before admission. | hook §3 | (none) | open. Overtaken (#1047 removed the whole hook from the series, so no cookie surface shipped). The deferral returns with the hook's first consumer | +| 9 | Defer integration-owned cookie operations from the v1 response hook, and require a complete read/use/withdraw lifecycle before admission. | hook §3 | (none) | open. Overtaken (#1047 removed the whole hook from the series, so no cookie surface shipped). The deferral returns with the hook's first consumer | | 10 | Do not create a blanket session-cookie exemption. Every cookie must be covered by an approved permission or narrowly defined security-use authority. | hook §3 | (none) | open. Hook not shipped, unaffected | | 11 | Require a durable per-family negative-intent outbox in a failure domain independent of its strong target and checked freshly by every identity consumer, with a globally visible breaker over positive identity operations when neither can commit. | permission §4.3 | (none) | open. Not implemented (part of the durable-suppression follow-up) | | 12 | Adapters that cannot meet the revocation-storage contract migrate stateless rather than weakening the contract. | rollout §6 item 2; recipe §5 | (none) | open. Direction implemented (`provider = "none"` spells stateless in #1043; the identity endpoints were already Fastly-only before the series, and #1046 keeps `resolve` on the same footing). The formal capability gate is deferred. Ratify the direction | -| 13 | Keep batch sync fail-closed at cutover, and stage partner communication and cleanup using explicit coverage thresholds, windows, and pause actions. | rollout §6 item 6 | (none) | open. Not implemented (no provenance cutover exists yet) | +| 13 | Keep batch sync fail-closed at cutover, and stage partner communication and cleanup using explicit coverage thresholds, windows, and pause actions. | rollout §6 item 6 | (none) | open. Not implemented (no provenance cutover exists yet) | | 14 | Policy tightening does not reinterpret historical refusal as a destructive event. Destructive withdrawal requires fresh, live qualifying evidence. | permission §4.2 trigger 2 | (none) | open. Implemented by #1045 (withdrawal evaluates the live request's TCF record only, and historical records are never reinterpreted). Ratify | -| 15 | Descope the client cycle and `rewrite_legacy`, and ship the v1 integration hook as headers-only. | client spec status; providers §6.1; hook §3 | (none) | open. Overtaken (the client cycle shipped hardened as #1046 instead of descoped, `rewrite_legacy` does not exist, and the hook shipped not at all per #1047). Re-decide against the shipped shape | +| 15 | Descope the client cycle and `rewrite_legacy`, and ship the v1 integration hook as headers-only. | client spec status; providers §6.1; hook §3 | (none) | open. Overtaken (the client cycle shipped hardened as #1046 instead of descoped, `rewrite_legacy` does not exist, and the hook shipped not at all per #1047). Re-decide against the shipped shape | | 16 | Persist use-opt-out suppression until ordered explicit authorization for that use or identity deletion, with TCF `LastUpdated` or an authenticated monotonic revision proving order. | permission §4.3 | (none) | open. Not implemented (suppression in the series is request-scoped with no durable record) | | 17 | N/A, absent, reserved, unknown, and unsupported values never grant processing. | permission §4.5 | (none) | open. Implemented by #1045 in a stricter form (signals only revoke, so no US-signal value grants anything). Ratify | | 18 | A selected geo provider's lookup failure uses the compiled-in protective profile; `default_country` is only for acknowledged static-jurisdiction mode. | permission §5.2 | (none) | open. Implemented by #1045 (failed lookups resolve at the requires-signal floor, logged at error level, never `default_country`). Ratify | | 19 | Use immutable version-addressed whole-config publication plus prepare/commit activation of the complete tuple, with authenticated fleet membership, bounded admission lease, quiescence, and an activation journal. A second unanimous model transition advances model epoch, minimum binary generation, and row schema floor atomically. | permission §5.5; rollout §6.1 | (none) | open. Not implemented (`ts config push` publishes and validates without the activation protocol) | | 20 | N+1 keeps v1 creation and pre-epic live gating, reads/enforces N+2 negative state for rollback safety, and never originates durable use suppression. New-shape settings alone do not activate the new writer/model. | migration §4.4 | (none) | open. Overtaken in part (the shipped migration is a one-release dual-read of `[ec] passphrase` in #1043 with mixed forms rejected and nothing durable added, so the full interim waits for the durable design) | | 21 | Expire and re-create rowless legacy cookies without continuity. A prefix match cannot authenticate the cookie suffix. | providers §5 | (none) | open. Not implemented (no rowless classification exists, and a cookie with no row is never shared) | -| 22 | Defer host JA4/H2 signal processing to a separate approved design. Reject `[device] provider = "fastly"` at startup and do not persist signal-derived classifications. | providers §5 | (none) | open. **Contradicted by the series and flagged for review** (#1044 ships the `fastly` device provider and the host-signal EC provider opt-in, and device signals including signal hash prefixes persist in rows) | +| 22 | Defer host JA4/H2 signal processing to a separate approved design. Reject `[device] provider = "fastly"` at startup and do not persist signal-derived classifications. | providers §5 | (none) | open. **Contradicted by the series and flagged for review** (#1044 ships the `fastly` device provider and the host-signal EC provider opt-in, and device signals including signal hash prefixes persist in rows) | | 23 | Permit a narrow `SecurityUse` authority for DataDome only, with the exact bounded surface the hook spec defines. | hook §4a; permission §7 | (none) | open. Hook not shipped, unaffected | | 24 | Malformed/absence suppression overrides a permissive baseline but clears on newer valid evidence. It is not sticky like an explicit use opt-out. | permission §4.3, §4.1 | (none) | open. Implemented by #1045 by construction (the fail-closed block is request-scoped, so newer valid evidence re-resolves). Ratify together with row 16's durable design | -| 25 | Enforce a stored-jurisdiction/provenance horizon for batch sync, where moves into stricter regimes fail closed by the horizon, and moves out require a live visit. | permission §7 | (none) | open. Not implemented | +| 25 | Enforce a stored-jurisdiction/provenance horizon for batch sync, where moves into stricter regimes fail closed by the horizon, and moves out require a live visit. | permission §7 | (none) | open. Not implemented | | 26 | Aggregate embedded GPP GPC with `Sec-GPC` by OR as a global, non-destructive use opt-out. | permission §4.5 | (none) | open. Partially implemented by #1045 (`Sec-GPC` is an honored non-destructive source). The embedded GPP GPC subfield is not read | | 27 | In proxy mode, decode only mapped opt-out fields and derive no grants. | permission §4.4 | (none) | open. Partially (the shipped model derives no grants from any US signal anywhere). The proxy-mode decode restriction is not separately implemented | | 28 | Require product and written vendor conformance approval for the reduced DataDome surface the hook spec pins. | hook §4a.2 | (none) | open. Hook not shipped, unaffected | @@ -498,20 +498,20 @@ the code with the row open. ## Revision record vs the 2026-07-31 draft -| Draft position | Revised position | Why | -| -------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Epic #777-#781 pending sign-off, no implementation | Implemented as five stacked PRs #1043-#1047, with §8 staying the ledger and rows marked for ratification | The series shipped the seam, selection, permission model, resolve endpoint, and docs, so the task force ratifies rather than re-litigates | +| Draft position | Revised position | Why | +| ----------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Epic #777-#781 pending sign-off, no implementation | Implemented as five stacked PRs #1043-#1047, with §8 staying the ledger and rows marked for ratification | The series shipped the seam, selection, permission model, resolve endpoint, and docs, so the task force ratifies rather than re-litigates | | `[device] provider = "fastly"` startup-fails pending a separate host-signal design (§2 row 9, row 22) | The `fastly` device provider and host-signal EC provider ship opt-in, and device signals persist in graph rows. The identifier-collision defect the review found (host-signal identifiers shared the HMAC grammar and keyspace) is fixed by the mandatory provider-code envelope, so host-signal identifiers are `hs00~` namespaced. The policy question of row 22 is unchanged by the collision fix | Implementation choice, deliberately flagged as the open review question rather than presented as settled | -| US recipe: `requires_signal` with an extended grant-signal class (§2 rows 3/3a) | Shipped default: `granted` US baseline, `revokes: all` on any opt-out, and no US signal grants anything | The shipped signal model is revoke-only, which is simpler and stricter on grants, and the baseline choice is deployer-editable yaml, flagged in rows 3 and 5 | -| `[permissions]` TOML policy published at runtime | `permissions.yaml` compiled into the build, no `[permissions]` block | Policy stays reviewable in version control and the partial-policy trap cannot be written | -| `rules.default` worldwide default entry (§2 row 7) | Required `[geo] default_country` naming the fallback rule, validated at startup | Same role, one mechanism, loud when missing | -| Graph store mandatory at startup for creating providers (§2 row 12, §4.4b) | No startup requirement, and with no graph no identifier is created, per request | The phantom-cookie rule holds without a breaking startup change, and graphless deployments run identity-less | -| `legacy_providers` reader chain for provider switches (§4.8) | Not implemented, so a switch restarts identity | Deferred until a second server-side provider makes switching real | -| N+1/N+2 negative-record machinery, model epochs, `m00` mirror (§4.1, row 20) | Deferred with rows 11, 16, 19, 20. The shipped dual-read is one release of `[ec] passphrase` mapping with mixed forms rejected | Nothing durable beyond pre-existing withdrawal tombstones ships, so binary rollback strands no new state | -| Staged activation, `push_sequence`, quiescence, `ts config gc` (§6.3, §6.1) | Basic `ts config push` / `ts config validate` only | The activation protocol belongs to the deferred durable-suppression rollout (row 19) | -| Migration guide with committed per-adapter fixtures and a full gated metric set (§5, §6.4) | `configuration.md` / `edge-cookies.md` document the migrated shape, while fixtures, the guide page, and metrics remain outstanding | Documentation shipped for configuration, and the operational guide is the remaining deliverable before a release | -| Client cycle descoped and hook shipped headers-only (row 15) | The client cycle shipped hardened (#1046) and the hook shipped not at all (#1047) | Hardening replaced descoping, and the hook had no consumer, which is the hook spec's own admission rule | -| Example config ships the migrated happy path uncommented (§4.9) | Identity off by default with selector and block commented together, and validation closes the silent-stateless trap | Loud startup validation, not an uncommented default, is what prevents PR #838's silent-stateless state | -| Environment override documented but `#[cfg(test)]`-only in PR #838 (§7) | Override applied as typed EdgeZero app-config overlays at `ts config push`; the existing CLI overlay test covers the mechanism, and a provider-specific override test is still to write | The same compiled binary switches providers at deployment through the published configuration | -| Pinned known-answer HMAC vectors committed (§3) | Stability tested per inputs, and the pinned vector is still to commit | The cross-version CI pin remains open work under §3 | -| GB storage baseline change only with citation and sign-off (§2 row 4) | The shipped `permissions.yaml` adopts `granted` storage for GB without a recorded citation | Flagged in row 4, since the task force owns the decision and its record | +| US recipe: `requires_signal` with an extended grant-signal class (§2 rows 3/3a) | Shipped default: `granted` US baseline, `revokes: all` on any opt-out, and no US signal grants anything | The shipped signal model is revoke-only, which is simpler and stricter on grants, and the baseline choice is deployer-editable yaml, flagged in rows 3 and 5 | +| `[permissions]` TOML policy published at runtime | `permissions.yaml` compiled into the build, no `[permissions]` block | Policy stays reviewable in version control and the partial-policy trap cannot be written | +| `rules.default` worldwide default entry (§2 row 7) | Required `[geo] default_country` naming the fallback rule, validated at startup | Same role, one mechanism, loud when missing | +| Graph store mandatory at startup for creating providers (§2 row 12, §4.4b) | No startup requirement, and with no graph no identifier is created, per request | The phantom-cookie rule holds without a breaking startup change, and graphless deployments run identity-less | +| `legacy_providers` reader chain for provider switches (§4.8) | Not implemented, so a switch restarts identity | Deferred until a second server-side provider makes switching real | +| N+1/N+2 negative-record machinery, model epochs, `m00` mirror (§4.1, row 20) | Deferred with rows 11, 16, 19, 20. The shipped dual-read is one release of `[ec] passphrase` mapping with mixed forms rejected | Nothing durable beyond pre-existing withdrawal tombstones ships, so binary rollback strands no new state | +| Staged activation, `push_sequence`, quiescence, `ts config gc` (§6.3, §6.1) | Basic `ts config push` / `ts config validate` only | The activation protocol belongs to the deferred durable-suppression rollout (row 19) | +| Migration guide with committed per-adapter fixtures and a full gated metric set (§5, §6.4) | `configuration.md` / `edge-cookies.md` document the migrated shape, while fixtures, the guide page, and metrics remain outstanding | Documentation shipped for configuration, and the operational guide is the remaining deliverable before a release | +| Client cycle descoped and hook shipped headers-only (row 15) | The client cycle shipped hardened (#1046) and the hook shipped not at all (#1047) | Hardening replaced descoping, and the hook had no consumer, which is the hook spec's own admission rule | +| Example config ships the migrated happy path uncommented (§4.9) | Identity off by default with selector and block commented together, and validation closes the silent-stateless trap | Loud startup validation, not an uncommented default, is what prevents PR #838's silent-stateless state | +| Environment override documented but `#[cfg(test)]`-only in PR #838 (§7) | Override applied as typed EdgeZero app-config overlays at `ts config push`; the existing CLI overlay test covers the mechanism, and a provider-specific override test is still to write | The same compiled binary switches providers at deployment through the published configuration | +| Pinned known-answer HMAC vectors committed (§3) | Stability tested per inputs, and the pinned vector is still to commit | The cross-version CI pin remains open work under §3 | +| GB storage baseline change only with citation and sign-off (§2 row 4) | The shipped `permissions.yaml` adopts `granted` storage for GB without a recorded citation | Flagged in row 4, since the task force owns the decision and its record | diff --git a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md index d9eb5692a..cec7ca0ff 100644 --- a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md +++ b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md @@ -457,24 +457,24 @@ defines, and both should land before the first vendor is asked to use it. ## 9. Sign-off -| # | Decision | Status | -| --- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- | -| 1 | Vendor integrations belong outside core, behind the registration contract | Proposed | -| 2 | Tech Lab engineering reviews vendor crates, and does not maintain them | Proposed, governance | -| 3 | A registration may carry its own browser JavaScript | Proposed | -| 4 | Deploy validation moves onto the registration | Proposed | -| 5 | The nine existing integrations migrate one PR each, on the schedule in §4 | Proposed | -| 6 | This change completes the Rust side, so after it no vendor move needs a Rust core change. The browser side is not complete, because TSJS core still imports the APS renderer directly (§8 item 8), so an APS move still needs a browser renderer contract | Proposed | -| 7 | Identity, geo and device providers are capabilities of a module registration (§3.6), the #1043 review's rule applied to all three | Proposed | -| 8 | No provider is built into core: HMAC and the User-Agent-only device provider are Tech Lab-owned modules configured under `[integrations.]`, and core keeps only `none` | Proposed | -| 9 | This spec and its core implementation precede #1043; 51Degrees implements the core seam, the nine vendor moves in §4 stay one PR each | Proposed | +| # | Decision | Status | +| --- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- | +| 1 | Vendor integrations belong outside core, behind the registration contract | Proposed | +| 2 | Tech Lab engineering reviews vendor crates, and does not maintain them | Proposed, governance | +| 3 | A registration may carry its own browser JavaScript | Proposed | +| 4 | Deploy validation moves onto the registration | Proposed | +| 5 | The nine existing integrations migrate one PR each, on the schedule in §4 | Proposed | +| 6 | This change completes the Rust side, so after it no vendor move needs a Rust core change. The browser side is not complete, because TSJS core still imports the APS renderer directly (§8 item 8), so an APS move still needs a browser renderer contract | Proposed | +| 7 | Identity, geo and device providers are capabilities of a module registration (§3.6), the #1043 review's rule applied to all three | Proposed | +| 8 | No provider is built into core: HMAC and the User-Agent-only device provider are Tech Lab-owned modules configured under `[integrations.]`, and core keeps only `none` | Proposed | +| 9 | This spec and its core implementation precede #1043; 51Degrees implements the core seam, the nine vendor moves in §4 stay one PR each | Proposed | ## Revision record | Date | Change | | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 2026-08-27 | First draft, written against `split/5-response-hook-docs`. | -| 2026-08-27 | Brought the bid renderer contract into scope (§3.4), so that after this change no vendor move needs a Rust core change (§8 row 6). The browser side is recorded as outstanding in §8 item 8. | +| 2026-08-27 | Brought the bid renderer contract into scope (§3.4), so that after this change no vendor move needs a Rust core change (§8 row 6). The browser side is recorded as outstanding in §8 item 8. | | 2026-08-28 | Corrected line references to `main` at b7fcb5d4c and added what mapping `main` found: composition of the served script moves into core (§3.2), registration enumeration and the auction-only `adserver_mock` case (§3.3), the duplicate-id gap (§3.1), the source-file guard and the `ts audit` vendor table (§4), the renderer risk (§7). | | 2026-08-28 | Recorded what implementing the seam found (§8): the operator CLI skips a vendor's deploy rules, a carried module's hash literal is fragile, providers resolve more than once per request, and one core reader still reads an APS payload. Recorded the construction-time hash check in §3.2. | | 2026-08-28 | Adopted the #1043 review's registration shape for identity and applied its rule to geo and device, with no provider built into core (§3.6, §6 item 2, §8 rows 7 to 9). Recorded the relationship to #986 and reordered the series so this spec and its implementation come first. | From bc20bc5a30a78bd7e59ca7adbef17ee0fc4bdc06 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Tue, 1 Sep 2026 15:02:18 +0100 Subject: [PATCH 14/17] Cite RFC 6454 for the resolve endpoint's origin comparison The client-cycle spec described the origin check as a string comparison of the serialized origin. The implementation on the client-resolve branch now applies the RFC 6454 same-origin test, comparing the scheme, host and port triple with a missing port meaning the scheme's default, so the spec names that rule and the RFC rather than describing a comparison of its own. --- ...26-07-30-client-cycle-ec-resolve-design.md | 30 ++++++++++--------- 1 file changed, 16 insertions(+), 14 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index 1ffde33ba..c793df81f 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -65,16 +65,18 @@ verify against. `POST /_ts/api/v1/ec/resolve` MUST: 1. **Reject cross-site requests with an origin check.** v1 authorizes a - request only when its `Origin` header, serialized as scheme, lowercased - host and the port as the browser wrote it, is byte-equal to an accepted - origin. No port is added or removed on either side. The + request only when its `Origin` header is the same origin as an accepted + origin under RFC 6454, meaning the scheme, host and port triple of §4 + compared by the §5 rule, with a missing port standing for the scheme's + default, and with a value that is not a serialized origin under §6.1 + never matching. The default accepted set is the single origin `https://{publisher.domain}` and nothing else, so a sibling subdomain, the `http://` scheme and a non-default port are all refused. A missing or foreign `Origin` is rejected with `403`. Browsers always send `Origin` on POST `fetch`, so its absence means a non-browser caller, which has no business on a page-script endpoint. An operator may configure further accepted - origins, each of them compared the same exact way, for a publisher + origins, each of them compared by the same RFC 6454 rule, for a publisher whose pages are served from `www` or from another domain. This is the exact allowlist the 2026-07-31 draft asked for, and it replaces the suffix match on `publisher.domain` an earlier revision of this spec @@ -292,13 +294,13 @@ end in tests and demonstrations. ## 8. Revision record vs the 2026-07-31 draft -| Draft position | v1 (PR #1046) | Why | -| ------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Feature deferred, `resolve_from_client` de-normalized | Feature normative. Trait method kept with a no-op default | The first vendor integration works client-side by design, so the endpoint is on the critical path for the series' first real provider | -| Origin check via new allowlist config + CSRF token | Exact serialized-origin comparison (scheme, lowercased host, port as written). Default accepted set is the single origin `https://{publisher.domain}`, plus an optional operator-configured list of further exact origins. Missing/foreign `Origin` → `403` | The draft's exact allowlist is adopted. An earlier revision of this spec allowed any suffix match on the apex, which admitted every subdomain including ones the publisher may not control, and its justification did not hold. This closes §7.5 | -| Marker cookie survives whatever happens to the identity it marks | Core expires the `ts-ecr` marker on any request carrying a `ts-ec` the selected provider does not own | The marker is not namespaced by the provider code envelope, so without this a provider switch leaves a visitor with a marker, no identity, and a page script that will not re-post | -| §3.9 reservation/replay machinery required before code | Deferred to the vendor scheme. Draft text retained verbatim as the bar | The design needs the real envelope's unique id and session binding, and a CAS-class primitive no production adapter exposes today | -| Identical behavior or identical startup refusal, 4 ways | Fastly routes it (bot-gated graph). Portability adapters documented as deliberately not routing, like identify | Same platform-KV constraint as the existing EC API routes. Startup rejection follow-up recorded (§7.4) | -| Marker cookie or injected variable (design must pick) | Marker cookie (`ts-ecr`), expired with the EC cookie | The draft's own first option. Testable and observable | -| Demo gated by cargo feature or `#[cfg(test)]` | Both: `client-fixed-demo` feature + startup rejection in the settings validator | Defense in depth | -| Everything else in §3 (graph row, bounds, 409, no-store, full-declaration gate) | Implemented as specified | (none) | +| Draft position | v1 (PR #1046) | Why | +| ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Feature deferred, `resolve_from_client` de-normalized | Feature normative. Trait method kept with a no-op default | The first vendor integration works client-side by design, so the endpoint is on the critical path for the series' first real provider | +| Origin check via new allowlist config + CSRF token | RFC 6454 same-origin comparison of the scheme, host and port triple. Default accepted set is the single origin `https://{publisher.domain}`, plus an optional operator-configured list of further exact origins. Missing/foreign `Origin` → `403` | The draft's exact allowlist is adopted. An earlier revision of this spec allowed any suffix match on the apex, which admitted every subdomain including ones the publisher may not control, and its justification did not hold. This closes §7.5 | +| Marker cookie survives whatever happens to the identity it marks | Core expires the `ts-ecr` marker on any request carrying a `ts-ec` the selected provider does not own | The marker is not namespaced by the provider code envelope, so without this a provider switch leaves a visitor with a marker, no identity, and a page script that will not re-post | +| §3.9 reservation/replay machinery required before code | Deferred to the vendor scheme. Draft text retained verbatim as the bar | The design needs the real envelope's unique id and session binding, and a CAS-class primitive no production adapter exposes today | +| Identical behavior or identical startup refusal, 4 ways | Fastly routes it (bot-gated graph). Portability adapters documented as deliberately not routing, like identify | Same platform-KV constraint as the existing EC API routes. Startup rejection follow-up recorded (§7.4) | +| Marker cookie or injected variable (design must pick) | Marker cookie (`ts-ecr`), expired with the EC cookie | The draft's own first option. Testable and observable | +| Demo gated by cargo feature or `#[cfg(test)]` | Both: `client-fixed-demo` feature + startup rejection in the settings validator | Defense in depth | +| Everything else in §3 (graph row, bounds, 409, no-store, full-declaration gate) | Implemented as specified | (none) | From 895b99bb42e90c61964897ea0b341faba2c2860f Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Tue, 1 Sep 2026 15:32:46 +0100 Subject: [PATCH 15/17] Record the chosen page carrier for resolved permissions --- ...26-07-30-client-cycle-ec-resolve-design.md | 46 +++++++++++-------- 1 file changed, 28 insertions(+), 18 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index c793df81f..a202fc3ff 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -10,7 +10,7 @@ defers, and why the feature is normative rather than deferred. **Author:** Engineering (revised against the implementation, 2026-08-25) **Issue references:** #778 (series), successor spec of the 2026-07-31 draft **Related specs:** `2026-07-30-pluggable-providers-design.md` -**Last updated:** 2026-08-25 +**Last updated:** 2026-09-01 > **Context.** PR #838 shipped, undeclared and unspec'd, a second provider > _type_: a "client-cycle" EC provider whose identifier is established by a @@ -211,23 +211,33 @@ the bar for the vendor-scheme implementation: asynchronous vendor contact, including BFCache restoration), and its injection is keyed off the provider selection exactly as the demo's is today. -- **How the resolved permissions reach the browser is not designed here.** - The requirement above assumes the page can read the server's decision, - and nothing in v1 carries that decision to the page. The binding - constraint is that the JavaScript bundle is composed at startup and - served under a content hash, so one body is shared across every visitor - and cannot carry per-visitor permission state, which means the signal - has to be per request. The candidate carriers are a response header on - the document, a value injected into the document during HTML - processing, a first-party endpoint the page fetches, and a - non-HttpOnly cookie set alongside the marker. This spec names them and - deliberately does not choose between them. Whichever is chosen, the - server's resolved permission decision is the authority, and an in-page - CMP read is only a withdrawal re-check layered under it and never a - substitute for it, because a page-side read can narrow what the server - resolved and must never widen it. Designing the mechanism is out of - scope for this set of PRs and belongs with the first vendor module, - which is the first consumer that needs it. +- **How the resolved permissions reach the browser is now chosen.** The + requirement above assumes the page can read the server's decision, and + v1 had no carrier for it. The binding constraint is that the JavaScript + bundle is composed at startup and served under a content hash, so one + body is shared across every visitor and cannot carry per-visitor + permission state, which means the signal has to be per request. The + permission-model PR (#1045) picks the injected-value carrier and + delivers it. The page receives `window.tsjs.permissions`, an object + `{"set": ["necessary.operations.storage", "..."]}` naming the Data Uses + set for the request, using the same keys as `permissions.yaml` and + `Permission::as_str()`. Under inline assembly it is injected as a + `