[posthog-monitor] Telemetry: report when the consent browser actually opens - #15
Open
Automage wants to merge 1 commit into
Open
[posthog-monitor] Telemetry: report when the consent browser actually opens#15Automage wants to merge 1 commit into
Automage wants to merge 1 commit into
Conversation
`broker_connect_failed` with `OAUTH_TIMEOUT` currently covers two different outcomes that call for opposite responses: the user saw the Robinhood consent page and walked away, or never got a usable page at all. The events are identical, so the funnel can't tell a copy/UX problem from a defect. Adds `broker_consent_opened`, emitted from `redirectToAuthorization` after the browser is actually launched — so a launch that throws is not recorded as a page the user saw, and a silent connect (which throws `ConsentRequired` at the gate, before `openBrowser`) reports nothing. An `OAUTH_TIMEOUT` preceded by this event means abandonment; one without it means the user never saw a page. Its `armed_ms` measures the second thing worth knowing: the loopback's timeout starts at bind, before the SDK registers the client and opens the page, so a slow round-trip silently eats the consent window. The provider stamps the arming time in `beginAuthorization` (called immediately after the bind) and reports the delta at open. If that number is routinely large, the deadline is armed in the wrong place; if it's small, the timeout is doing its job and the drop-off is behavioural. Allowlist invariants hold: one `z.strictObject` with a single non-negative integer duration — no free-form string, nothing identifying, and nothing about the user's account or the authorization URL. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0128hXeESThVH1BrjUp2MbUP
Deploying opentradeoss with
|
| Latest commit: |
a17c554
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://3bb4e661.opentradeoss.pages.dev |
| Branch Preview URL: | https://claude-laughing-bardeen-z3t1.opentradeoss.pages.dev |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Evidence
Over the last 21 days, six installs hit
broker_connect_failedwitherror_code: "OAUTH_TIMEOUT"— each exactly once, six different installs. Outcomes:OAUTH_TIMEOUT, later reachedbroker_connectedbroker_connectedTwo of those four emitted nothing at all after ~15 minutes from first launch (6 and 8 lifetime events respectively) — a fresh install, one connect attempt, gone. That is a drop-off at the broker-connect step, but the telemetry cannot say which kind.
One of them also showed
broker_connect_started→broker_connect_failed16m34s apart against a 10-minute budget, which means the timeout had already fired and its rejection sat unobserved whiledoConnect()was still in flight.What can't be answered today
OAUTH_TIMEOUTis ours, not Robinhood's:openLoopbackarms aCONSENT_TIMEOUT_MS= 10-minute timer (app/src/main/services/broker/robinhood/client.ts:26,140) and rejects the pending consent if no callback arrives. Two very different situations produce the identical event:Nothing distinguishes them, so neither can be acted on.
There is also a concrete reason to suspect (2). The timer arms at loopback bind (
client.ts:140), which happens beforebeginAuthorization, before the SDK registers the client, and before the browser opens (client.ts:259-264). Whatever that round-trip costs is deducted from the user's consent window, and nothing measures it.The change
Adds one allowlisted event,
broker_consent_opened, emitted fromBrokerOAuthProvider.redirectToAuthorization(oauth.ts) — the single point where the consent browser is actually launched:openBrowser, so a launch that throws isn't recorded as a page the user saw.ConsentRequiredat the browser gate beforeopenBrowser, so it reports nothing — by construction.armed_ms: milliseconds from loopback bind to browser open. The provider stamps the time inbeginAuthorization(called immediately after the bind) and reports the delta at open; cleared inendAuthorization.That gives the funnel two readings it doesn't have:
OAUTH_TIMEOUTwith a precedingbroker_consent_opened→ the user saw the page and walked away.OAUTH_TIMEOUTwithout one → the user never got a page.armed_ms→ the deadline is armed in the wrong place and should start at browser-open; a small one → the timeout is doing its job and the drop-off is behavioural.Allowlist invariants
Held. One
z.strictObjectwhose only prop is a non-negative integer duration — no free-form string, no identifier, nothing about the account or the authorization URL (theURLargument is never read into the event). Unknown props still drop the whole event, and an out-of-shapearmed_mswould fail validation rather than leak.Verification
bun install --frozen-lockfile— clean.bun run typecheck— clean.bunx @biomejs/biome checkon the three changed files — clean.bun testfromapp/: 303 pass / 7 fail with this change vs 301 pass / 7 fail on the same base without it. The +2 are the new tests; the 7 failures (agent CLAUDE.md composition, Codex app-server WebSocket) are identical with the change stashed and unrelated to this diff.Two tests added to
oauth.test.ts: opening the browser reports the event once with an integer, non-negativearmed_ms; the gate-throw path reports nothing. They assert viaspyOn(analytics, "track")rather than starting the singleton with a fake client —AnalyticsService.startis idempotent (analytics/index.ts:105), so in a full-suite run whichever file starts it first owns it and a second fake client would silently receive nothing. That is exactly how the first draft of this test passed alone and failed in the suite.Risk
Low, and additive. One new event on a path that only runs during interactive consent; no behaviour changes, and
openBrowseris still called exactly as before. Worst case the event is emitted and nobody looks at it.Generated by Claude Code