Skip to content

[posthog-monitor] Telemetry: report when the consent browser actually opens - #15

Open
Automage wants to merge 1 commit into
mainfrom
claude/laughing-bardeen-z3t1hc
Open

[posthog-monitor] Telemetry: report when the consent browser actually opens#15
Automage wants to merge 1 commit into
mainfrom
claude/laughing-bardeen-z3t1hc

Conversation

@Automage

Copy link
Copy Markdown
Contributor

Evidence

Over the last 21 days, six installs hit broker_connect_failed with error_code: "OAUTH_TIMEOUT" — each exactly once, six different installs. Outcomes:

installs
hit OAUTH_TIMEOUT, later reached broker_connected 2
hit it, never produced a broker_connected 4

Two of those four emitted nothing at all after ~15 minutes from first launch (6 and 8 lifetime events respectively) — a fresh install, one connect attempt, gone. That is a drop-off at the broker-connect step, but the telemetry cannot say which kind.

One of them also showed broker_connect_startedbroker_connect_failed 16m34s apart against a 10-minute budget, which means the timeout had already fired and its rejection sat unobserved while doConnect() was still in flight.

What can't be answered today

OAUTH_TIMEOUT is ours, not Robinhood's: openLoopback arms a CONSENT_TIMEOUT_MS = 10-minute timer (app/src/main/services/broker/robinhood/client.ts:26,140) and rejects the pending consent if no callback arrives. Two very different situations produce the identical event:

  1. The user saw the consent page and abandoned it → a copy/flow problem.
  2. The user never got a usable page → a defect.

Nothing distinguishes them, so neither can be acted on.

There is also a concrete reason to suspect (2). The timer arms at loopback bind (client.ts:140), which happens before beginAuthorization, before the SDK registers the client, and before the browser opens (client.ts:259-264). Whatever that round-trip costs is deducted from the user's consent window, and nothing measures it.

The change

Adds one allowlisted event, broker_consent_opened, emitted from BrokerOAuthProvider.redirectToAuthorization (oauth.ts) — the single point where the consent browser is actually launched:

  • Emitted after openBrowser, so a launch that throws isn't recorded as a page the user saw.
  • A silent connect throws ConsentRequired at the browser gate before openBrowser, so it reports nothing — by construction.
  • armed_ms: milliseconds from loopback bind to browser open. The provider stamps the time in beginAuthorization (called immediately after the bind) and reports the delta at open; cleared in endAuthorization.

That gives the funnel two readings it doesn't have:

  • OAUTH_TIMEOUT with a preceding broker_consent_opened → the user saw the page and walked away.
  • OAUTH_TIMEOUT without one → the user never got a page.
  • A routinely large armed_ms → the deadline is armed in the wrong place and should start at browser-open; a small one → the timeout is doing its job and the drop-off is behavioural.

Allowlist invariants

Held. One z.strictObject whose only prop is a non-negative integer duration — no free-form string, no identifier, nothing about the account or the authorization URL (the URL argument is never read into the event). Unknown props still drop the whole event, and an out-of-shape armed_ms would fail validation rather than leak.

Verification

  • bun install --frozen-lockfile — clean.
  • bun run typecheck — clean.
  • bunx @biomejs/biome check on the three changed files — clean.
  • bun test from app/: 303 pass / 7 fail with this change vs 301 pass / 7 fail on the same base without it. The +2 are the new tests; the 7 failures (agent CLAUDE.md composition, Codex app-server WebSocket) are identical with the change stashed and unrelated to this diff.

Two tests added to oauth.test.ts: opening the browser reports the event once with an integer, non-negative armed_ms; the gate-throw path reports nothing. They assert via spyOn(analytics, "track") rather than starting the singleton with a fake client — AnalyticsService.start is idempotent (analytics/index.ts:105), so in a full-suite run whichever file starts it first owns it and a second fake client would silently receive nothing. That is exactly how the first draft of this test passed alone and failed in the suite.

Risk

Low, and additive. One new event on a path that only runs during interactive consent; no behaviour changes, and openBrowser is still called exactly as before. Worst case the event is emitted and nobody looks at it.


Generated by Claude Code

`broker_connect_failed` with `OAUTH_TIMEOUT` currently covers two different
outcomes that call for opposite responses: the user saw the Robinhood consent
page and walked away, or never got a usable page at all. The events are
identical, so the funnel can't tell a copy/UX problem from a defect.

Adds `broker_consent_opened`, emitted from `redirectToAuthorization` after the
browser is actually launched — so a launch that throws is not recorded as a page
the user saw, and a silent connect (which throws `ConsentRequired` at the gate,
before `openBrowser`) reports nothing. An `OAUTH_TIMEOUT` preceded by this event
means abandonment; one without it means the user never saw a page.

Its `armed_ms` measures the second thing worth knowing: the loopback's timeout
starts at bind, before the SDK registers the client and opens the page, so a slow
round-trip silently eats the consent window. The provider stamps the arming time
in `beginAuthorization` (called immediately after the bind) and reports the delta
at open. If that number is routinely large, the deadline is armed in the wrong
place; if it's small, the timeout is doing its job and the drop-off is behavioural.

Allowlist invariants hold: one `z.strictObject` with a single non-negative integer
duration — no free-form string, nothing identifying, and nothing about the user's
account or the authorization URL.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0128hXeESThVH1BrjUp2MbUP
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying opentradeoss with  Cloudflare Pages  Cloudflare Pages

Latest commit: a17c554
Status: ✅  Deploy successful!
Preview URL: https://3bb4e661.opentradeoss.pages.dev
Branch Preview URL: https://claude-laughing-bardeen-z3t1.opentradeoss.pages.dev

View logs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants