Skip to content

telemetry: emit standard toolkit auth metrics for parity with other IDE extensions - #577

Draft
laileni-aws wants to merge 1 commit into
mainfrom
telemetry/auth-metrics
Draft

telemetry: emit standard toolkit auth metrics for parity with other IDE extensions#577
laileni-aws wants to merge 1 commit into
mainfrom
telemetry/auth-metrics

Conversation

@laileni-aws

Copy link
Copy Markdown
Contributor

Problem

The Eclipse plugin does not emit the standard toolkit authentication telemetry that the VS Code and JetBrains extensions emit, so its authentication experience cannot be compared with theirs — in particular how long an authentication session lives before the user has to sign in again, and whether the login UI loads.

Solution

Emit the same events the other extensions do:

  • aws_loginWithBrowser on every interactive login and re-authentication, with result, credentialStartUrl, credentialType, isReAuth, and sessionDuration. The instant of the latest successful browser login is kept in the plugin store per start URL, so the next login reports how long the previous session lived. A first login, and the first login after a sign-out, report no duration because there is no previous session to measure; a sign-out clears the recorded login. A failed login reports a short reason code (the exception type name) — never an error message or a file path.
    • The silent token refresh performed on IDE start-up (loginOnInvalidToken=false, no browser involved) does not emit this event and does not reset the recorded login, so start-ups are not counted as browser logins.
  • auth_userState with source=startup, reporting whether the plugin started connected, expired, or notConnected. The state restored from the plugin store is optimistic, so the metric is reported for the first state that follows it — the outcome of the re-authentication performed on start-up. Emitted once per plugin session.
  • toolkit_didLoadModule with module=login, reporting that the login view loaded (Succeeded, on the webview's onLoad callback) or the reason code of the failure that prevented it (DependencyMissing, AssetLoadFailed).

The re-authentication view in this plugin is built from native SWT widgets rather than a webview, so it has no equivalent load step and emits no toolkit_didLoadModule event.

Telemetry definitions

The bundled commonDefinitions.json predated toolkit_didLoadModule and the sessionDuration field of aws_loginWithBrowser, so both are added from the shared definitions, along with an optional reason on both metrics to match what the other extensions report.

The generated builder types sessionDuration as a primitive int, which cannot hold a duration longer than ~25 days in milliseconds and serializes to 0 when unset. Sessions are expected to live for months, so the entry is written with the exact millisecond value and omitted entirely when no previous session is known.

Testing

mvn package — BUILD SUCCESS, 524 tests, 0 failures (checkstyle and coverage checks included).

New unit tests cover:

  • sessionDuration reported only when a previous login exists for the same start URL; omitted on first login; not overwritten by a failed login
  • aws_loginWithBrowser Succeeded / Failed (with exception-type reason) / isReAuth
  • no aws_loginWithBrowser and no timestamp update on the silent start-up token refresh
  • auth_userState emitted once on start-up with the correct status (notConnected immediately when nothing is stored; connected/expired after the start-up re-authentication resolves)
  • toolkit_didLoadModule login failure paths (assets missing / cannot be served)
  • AuthPluginStore timestamp persistence, start-URL matching, and clearing on sign-out

Not covered by an automated test: the onLoad success emission in LoginViewActionHandler, because that handler depends on Eclipse preference services that are unavailable under plain surefire. It was verified by inspection.

…uration, didLoadModule login)

The plugin did not report the standard toolkit auth telemetry that the VS Code
and JetBrains extensions report, so the authentication experience of the Eclipse
plugin could not be compared with theirs. Emit the same events:

- aws_loginWithBrowser on every login and re-authentication, with the result,
  the start url, the credential type, whether it renewed an existing connection,
  and the duration of the previous session for that start url. The latest
  successful login is recorded in the plugin store so the next login can report
  how long the session lived for; a first login, and the first login after a sign
  out, report no duration because there is no previous session to measure. A
  failed login reports a short reason code, never an error message or a path.
- auth_userState with source startup, reporting whether the plugin started
  connected, expired, or not connected. The state restored from the plugin store
  is optimistic, so the metric is reported for the first state that follows it,
  which is the outcome of the re-authentication performed on start up.
- toolkit_didLoadModule with module login, reporting that the login view loaded
  or the reason code of the failure that stopped it from loading.

The re-authentication view of this plugin is built from native widgets rather
than a web view, so it has no equivalent load step and reports no
toolkit_didLoadModule event.

The telemetry definitions this plugin generates its metric builders from
predated toolkit_didLoadModule and the session duration of aws_loginWithBrowser,
so both are added from the shared definitions. An optional reason is added to
both metrics as well, matching what the other extensions report on them.

The generated builder types session duration as a primitive int, which cannot
hold a duration of more than roughly twenty five days in milliseconds and which
serializes to zero when it is not set. Sessions are expected to live for months,
so the entry is written with the exact millisecond value, and left out entirely
when no previous session is known.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant