Skip to content

Reader: anonymous event counts through POST /api/count (draft: needs Mateo's privacy decision) - #763

Draft
sdcarlson wants to merge 4 commits into
mainfrom
telemetry/browser-counts
Draft

sdcarlson wants to merge 4 commits into
mainfrom
telemetry/browser-counts

Conversation

@sdcarlson

@sdcarlson sdcarlson commented Oct 31, 2026 •

Copy link
Copy Markdown
Collaborator

Draft until Mateo's privacy decision, Seth's handle and Mateo's sign-off. RISE's codex-feedback workflow would otherwise auto-merge it.

This builds on #762 (telemetry/composer-counts), because it reuses that branch's worker/telemetry.mjs writer and telemetry/events.json allowlist. Until #762 merges, this diff also contains its commit c64f1bb. This packet is the second commit.

Update 2026-11-04: main merged in (00ef244), through the updated #762 branch. Main deleted worker/live-realtime.mjs, which worker/count.mjs imported readText from, and moved readText unchanged into worker/mcp-server.mjs as an unexported function. mcp-server.mjs now exports it and count.mjs imports it from there; without that the Worker does not bundle. (A small shared module would do as well; this is the smallest change.) FAST_TEST_FILES is now 162, and the architecture diagram was regenerated with npm run docs:diagram.

Checks on 00ef244: fast suite 162 files, 3821 tests (3817 passed, 4 skipped, 0 failed); full unit suite 608 files, 9301 tests (9293 passed, 8 skipped, 0 failed); vite build; first load 62.8 of 64.0 KB brotli; e2e gate 72 passed, 3 skipped; both wrangler dry runs, production listing EVENTS and COUNT_LIMITER; CI green on GitHub. PRIVACY.md's "Last updated" is 2 November 2026 on main and still needs the date these sentences publish. The test counts below describe the PR as first opened.

What changed

Server

  • worker/count.mjs (new): POST /api/count, dispatched from worker/index.mjs. A beacon is written only when every one of these holds:

    • it is a POST whose Origin equals the request's own origin;
    • its client address has a CF-Connecting-IP and is within COUNT_LIMITER, which is charged before the body is read;
    • its body is at most 512 bytes of UTF-8 JSON, exactly { "v": 1, "e": <event>, "f": { ... } };
    • the event is one of the reader's (its derived fields are exactly browser and device), so the connector's events are refused;
    • f holds exactly that event's other fields.

    The server derives browser (chrome|safari|firefox|edge|other) and device (mobile|desktop) from the User-Agent and keeps nothing else of it. Numbers are fitted:

    • minutes goes to the least of 1, 2, 5, 10, 20, 30, 60 or 120 at or above it, and 120 for anything longer;
    • ms is rounded to 50 and held within 0..60000.

    Anything else is dropped. Every request is answered 204 with no body, and nothing is logged. With no limiter or a failing limiter, nothing is written.

  • worker/telemetry.mjs: the writer now holds a number to what the allowlist gives its field (one of its values, or a multiple of step up to max), so a fitted number off its list cannot be written either.

  • telemetry/events.json: adds rise.reading.begun {browser, device, source, mode}, rise.reading.ended {browser, device, end; minutes}, rise.app.failed {browser, device, class} and rise.reading.first_word {browser, device; ms}, all owned by Reader. browser and device are marked derived.

  • wrangler.production.jsonc: a ratelimits binding COUNT_LIMITER, 60 per 60 seconds, namespace 2026103102. No other configuration gets it; worker/production-config.test.js pins that.

Client

  • src/core/telemetry.js (new): count(event, fields) calls navigator.sendBeacon('/api/count', new Blob([json], { type: 'application/json' })). It sends only when location.hostname === 'rise.syberlabs.io' and top === globalThis, which rules out frames, so nothing is sent from inside ChatGPT, Claude or any embed. It uses no storage, cookie or queue, and never throws.
  • src/core/begin-steps.js: markBegin() and takeBegin(), a performance mark that Home's Begin leaves for the first word to be measured from. It is read once and then cleared.

Call sites wired

  • Reading begin and end, in src/components/read/Chamber.js:
    • rise.reading.begun at the first playing state, which covers autostart, the Chamber's own Begin and a host that plays. A reading that opens as a page is counted as it opens.
    • source:
      • import when the provenance is local-text;
      • otherwise public when the session keeps a publicPath (route-url.js's own definition);
      • otherwise library for library-work, library-division, school, chapel-book or keystone;
      • otherwise other.
    • rise.reading.ended: yes when the Player completes; no when the Chamber is destroyed or the page is hidden for good (pagehide, added at begin and removed on destroy). Counted once per reading. minutes is wall time since begin.
  • First word, rise.reading.first_word: from the press of Home's Begin ([data-home="enter"]) to the first atom spoken or shown, meaning not concealed, not an authored boundary, and with text. This is R2's interval, as e2e/home-begin.spec.js measures it. The mark is cleared when Home's launch fails, when a Home reading is left before its first word, and when the reading waits for a second press (no audio clock).
  • unhandled: the error boundary's window error and unhandledrejection listeners, counted once per page load. An error that recurs every frame would otherwise be a Worker request every frame, which the per-address limiter drops but still pays for.
  • audio_failed: the audio engine's init() failure, beside its existing onUnavailable call.
  • speech_unavailable and webgl_unavailable: the Live venue's capability record, read once per LiveHost (speechOutput === 'none', webgl2 === false).

Call sites left out, and why

  • kev_load_failed: Kev loads only on the EnterpRise page. src/enterprise/boundary.test.js holds that EnterpRise imports nothing outside src/enterprise. Wiring it needs either that boundary opened to src/core/telemetry.js or a second copy of the sender inside EnterpRise, and that is the owner's decision. The value stays in the allowlist; nothing sends it, and the policy does not list it.
  • WebGL in the main reader: no single probe exists. The living flame falls back to CPU on its own WEBGL2_UNAVAILABLE, the Home atmosphere's hardwareWebGL2() also rejects software renderers and runs on every Home visit, and the wormhole drops to 2D. Counting those would count engines rather than browsers.
  • Speech in the main reader: Chamber recitation uses static voice packs, not the browser's speech, so only the Live venue can lack a speech voice.
  • The recitation voice failing to prepare (the factory's toast) is a content or network failure of the voice pack, not the audio system, so it is not counted as audio_failed.
  • First word for readings not begun by Home's Begin: Library's Enter, Keystones, Try, Continue, Scriptorium and Journeys each have their own Begin. There is no single press that can be carried to the Chamber without threading a timestamp through each launch path.
  • Live readings' own lifecycle in the Live venue: they are counted through the Chamber when one shows them, not through LiveHost.

Policy

  • PRIVACY.md: one paragraph at the end of section 4's "Page-view counts", naming the events, the derived browser and device, no cookie, no storage, no identifier, no reading text or typing, the per-address limit, nothing sent from frames, and three months' retention.
  • Section 6: one bullet, "Anonymous event counts", directly after "Page-view counts only".
  • Section 10: a legitimate-interest bullet for the counts. The policy's existing consent paragraph covers them unchanged: no cookie, no storage, no identification.
  • Section 11: three months.
  • public/PRIVACY.md and public/privacy.html were regenerated with npm run build:legal, and docs/specs/ARCHITECTURE.md's generated diagram with npm run docs:diagram.

Why

This is the approved SyberLabs telemetry design (telemetry.v1): reader-side counts as closed enums and bucketed numbers, never text or identifiers. They show where readings start, whether they finish, how long Begin takes and what fails.

How release, browser and device are derived

  • release is 'unknown', as in Composer: count each Current the door admits or refuses, by host app and refusal code #762. CI writes the commit into dist/release.txt after wrangler bundles the Worker, so naming it needs a deploy-time variable, which is a workflow change.
  • browser and device come from the User-Agent, read in worker/count.mjs; the string is never written.
    • Browser: the first match wins, in this order: Edge (Edg/, EdgA/, EdgiOS/); Opera, Samsung, Yandex or UC as other; Firefox (Firefox/, FxiOS/); Chrome (Chrome/, CriOS/); then Safari (Version/…Safari/, not Android).
    • Device: mobile for Mobi, Android, iPhone, iPad or iPod; otherwise desktop.

Tests

  • Fast suite, the list the required CI check runs (npx vitest run --config vitest.fast.config.js): 174 files collected, equal to FAST_TEST_FILES. 3958 tests: 3954 passed, 4 skipped, 0 failed. On Composer: count each Current the door admits or refuses, by host app and refusal code #762's branch the same suite was 170 files and 3927 tests (3923 passed, 4 skipped).
  • New:
    • worker/count.test.js (11 tests): written fields and order; reader events only; User-Agent table; number fitting; malformed and unlisted bodies dropped; foreign, absent and near Origins dropped; POST only; 512-byte limit and non-JSON or non-UTF-8 bodies; the limiter charged before the body, over-limit, missing, throwing and no-address cases; a byte-identical 204 with the binding present, absent and throwing, with nothing logged; the route reached through the Worker.
    • src/core/telemetry.test.js (7 tests): the beacon's shape; no send off-host or in a frame; no storage, cookie or queue; never throws; the Begin mark taken once, and absent without a timeline.
    • src/components/read/Chamber.counts.test.js (9 tests): begin once, with source and mode; end yes, no and on pagehide, once each; no end without a begin; first word only after Home's Begin; no stale interval left behind.
    • src/core/error-boundary.test.js (1 test): unhandled errors counted once per page.
  • Extended:
    • worker/telemetry.test.js: the allowlist's six events; numbers with values or a step; numbers off their list refused (24 tests now).
    • src/audio/engine.lifecycle.test.js: the init failure is counted.
    • src/live/host/LiveHost.test.js: the capability counts.
    • worker/production-config.test.js: COUNT_LIMITER in production only.
  • Mutation checks, each reverted. Each of these failed its test: dropping the Origin check, a limiter that fails open, a 1 KB body limit (this one first passed, and the test was fixed to pin the literal 512 bytes), a body on the 204, no frame check, a look-alike host match, begin counted on every play, the first word counted for any origin, and unhandled errors counted every time.
  • Also run locally, all passing:
    • node scripts/ci-hygiene.mjs, npm run roadmap:check, npm run test:roadmap, npm run security:compat;
    • npm run docs:diagram, unchanged on a re-run;
    • npm run verify:legal, npx vite build;
    • npm run measure:first-load: 63.5 of 64.0 KB brotli, from 63.4;
    • both wrangler deploy --dry-run bundles. Production lists env.COUNT_LIMITER (60 requests/60s) beside env.EVENTS.
    • A local wrangler dev --local of the production configuration answered a valid beacon, a foreign-origin one, a 2 KB one and a GET with 204 and an empty body, with no runtime error.
  • Browser gate (npm run test:e2e:gate), run locally on this tree, which includes Composer: count each Current the door admits or refuses, by host app and refusal code #762: 72 passed, 3 skipped, 0 failed, in 3.6 minutes. This session's image ships Chromium build 1243, while Playwright 1.61.1 looks for build 1228. The run therefore pointed PLAYWRIGHT_BROWSERS_PATH at a scratch folder that links 1228 to the installed 1243; nothing in the repository changed for it. CI runs the gate with its own browser.
  • Not run locally: npm run security:audit.
  • The private dependency-evidence server (GitHits) failed to connect in this session, so no version-grounded lookup ran. The ratelimits shape (period 10 or 60) was checked against the installed wrangler 4.144.0 schema and its dry run instead.

Needs a human key

Deliberately not done, and limitations

  • No consent prompt. By the policy's own rule none is needed: nothing sets a cookie, uses storage or identifies anyone.
  • The policy is otherwise unchanged. Mateo may also want these to name the counts: the short version, section 6's bullet title "Page-view counts only" (followed by "no other analytics"), section 7 and section 9's list of server processing.
  • Analytics Engine keeps each count as its own anonymous row, with a timestamp, for three months. Section 10's new bullet says "Each count is one anonymous event" rather than the page-view bullet's "totals".
  • Anyone can forge Origin from outside a browser and write allowlisted values. No personal data is at stake, and the per-address limit bounds it. Every beacon is a Worker request, even when it is dropped.
  • Measurement:
    • minutes includes time the tab was hidden.
    • end: yes means the Player reached the last word, so a reading finished in the Page view is counted as no.
    • The first word includes the photosensitivity consent's wait when that consent is shown.
    • unhandled counts page loads that met an error, not errors. It includes benign browser errors (a ResizeObserver loop, for example) and extensions' errors.
  • Classification is approximate: iPadOS in desktop mode counts as desktop; Android tablets count as mobile; Brave and webviews count as chrome.
  • /api/count exists on every configuration, but writes only where EVENTS and COUNT_LIMITER are both bound (production).
  • No task record: the tracker has no task for this work, and this PR does not create one.

Generated by Claude Code

…and refusal code

One Analytics Engine data point per rise_present call: rise.current.admitted
with its beats and scenes, rise.current.refused with the refusal's code from
the door's closed set. Only what telemetry/events.json names is written, and
every answer is byte-identical with the binding absent, present or throwing.
EVENTS is bound in wrangler.production.jsonc alone. PRIVACY.md says so in the
ChatGPT and Claude list and in section 11.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VHrJ4hmTtR1RFNpc215EK7
A reading's begin and end, the first word after Home's Begin, and four
failure classes, each one beacon from rise.syberlabs.io (never from a frame)
to worker/count.mjs. It takes only same-origin JSON of at most 512 bytes
within COUNT_LIMITER, derives the browser and the device from the User-Agent,
fits the numbers, writes only what telemetry/events.json names, and always
answers 204. PRIVACY.md names the counts in sections 4, 6, 10 and 11.
Builds on #762's writer and allowlist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VHrJ4hmTtR1RFNpc215EK7
@netlify

netlify Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for rise-v2-symbolic-experience ready!

Name Link
🔨 Latest commit 00ef244
🔍 Latest deploy log https://app.netlify.com/projects/rise-v2-symbolic-experience/deploys/6aeaf9113a298d85982e9edf
😎 Deploy Preview https://deploy-preview-763--rise-v2-symbolic-experience.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Bring this branch up to date with main's dead-code sweeps (#765, #766,
#767) so it can merge. Two files conflicted:

- worker/mcp-server.mjs: main deleted Gate 0 and the witness log, made
  dispatch take the card as its third argument
  (dispatch(message, origin, card = null)) and deleted handleLive. All
  of that is kept as main has it. This branch's count callback becomes
  dispatch's fourth argument (count = () => {}): dispatch passes it to
  call() for rise_present, and handleMcp still derives the host with
  hostOf and writes through countEvent. No response changes.
- vitest.fast.config.js: main lowered FAST_TEST_FILES from 169 to 157
  after deleting test files, and this branch adds
  worker/telemetry.test.js. The merged list collects 158 files
  (vitest list --filesOnly, and the fast run's JSON report), so
  FAST_TEST_FILES is 158.

PRIVACY.md, public/PRIVACY.md, public/privacy.html,
worker/production-config.test.js and wrangler.production.jsonc merged
without conflict; npm run verify:legal confirms the generated copies
still match PRIVACY.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VHrJ4hmTtR1RFNpc215EK7
Bring this branch up to date with main's dead-code sweeps (#765, #766,
#767). It merges the updated telemetry/composer-counts (8f45346), which
already merges main at bce75dc, so this one merge brings in both.

Two files conflicted:

- vitest.fast.config.js: #762's merge set FAST_TEST_FILES to 158, and
  this branch had raised it from 170 to 174 for its four test files.
  The merged list collects 162 files (vitest list --filesOnly, and the
  fast run's JSON report), so FAST_TEST_FILES is 162.
- docs/specs/ARCHITECTURE.md: only the generated diagram conflicted.
  Took main's and regenerated it with npm run docs:diagram, which adds
  this branch's src/core/telemetry.js to core's module count and its
  imports to the audio, components and live edges into core.

One break git could not see: main (#767) deleted
worker/live-realtime.mjs, whose readText worker/count.mjs imports, and
moved readText unchanged into worker/mcp-server.mjs as a private
function. worker/mcp-server.mjs now exports it and worker/count.mjs
imports it from there. It is the same function count.mjs used before,
so the 512-byte and UTF-8 limits on a beacon are unchanged.

PRIVACY.md, public/PRIVACY.md, public/privacy.html, src/audio/engine.js,
src/audio/engine.lifecycle.test.js, src/components/Home.js,
src/components/read/Chamber.js, src/core/error-boundary.js,
src/live/host/LiveHost.js, src/live/host/LiveHost.test.js,
worker/index.mjs, worker/production-config.test.js and
wrangler.production.jsonc merged without conflict; npm run verify:legal
confirms the generated legal copies still match PRIVACY.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VHrJ4hmTtR1RFNpc215EK7

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants