Skip to content

feat: add workflow telemetry traces#995

Merged
EhabY merged 6 commits into
mainfrom
feat/issue-986-telemetry-traces
Jun 10, 2026
Merged

feat: add workflow telemetry traces#995
EhabY merged 6 commits into
mainfrom
feat/issue-986-telemetry-traces

Conversation

@EhabY

@EhabY EhabY commented Jun 8, 2026

Copy link
Copy Markdown
Collaborator

Generated by Coder Agents.

Fixes #986.

Summary

  • add privacy-safe workflow traces for workspace picker/open/devcontainer handoffs, start prompts, and diagnostic command completions
  • preserve existing command.invoked coverage while marking normally-returned cancellation/failure paths with bounded categories
  • avoid logging raw workspace names, picker queries, local/export/devcontainer paths, or raw command output
  • return structured telemetry export outcomes so caller telemetry can distinguish success, cancellation, and failure

Validation

  • pnpm test:extension ./test/unit/instrumentation/commands.test.ts ./test/unit/instrumentation/workspace.test.ts ./test/unit/telemetry/export/command.test.ts ./test/unit/uri/uriHandler.test.ts
  • pnpm format:check
  • pnpm lint
  • pnpm typecheck
  • git diff --check
Implementation plan

Issue 986 telemetry implementation plan

Goal

Add the missing low-volume, privacy-safe telemetry for issue #986 without duplicating existing command.invoked, workspace state transition, or start/update operation telemetry.

Principles

  • Keep command.invoked as the broad command coverage signal.
  • Add specific traces only where command handlers return normally on cancellation/failure or where command-specific context is otherwise invisible.
  • Use existing result semantics from TelemetryService.trace:
    • result=success for completed/accepted/selected paths.
    • result=aborted for user cancellation/dismissal.
    • result=error for thrown or handled failures, using span.markFailure() when the command handles the error and returns.
  • Avoid raw workspace names, search queries, local/export/devcontainer paths, and raw command output.
  • Match existing dot-separated telemetry naming patterns such as command.invoked, workspace.update.prompted, and workspace.start.triggered.

Event plan

Existing event retained

  • command.invoked
    • No schema expansion planned.
    • Still records command ID, duration, thrown errors, and top-level command success/error.

Workspace picker

  • workspace.picker.prompted
    • One trace per picker opening.
    • Properties: picker source/category where useful, selected workspace state bucket when selected.
    • Measurements: final result count and selected workspace agent counts.
    • Result: success when a workspace is selected, aborted when dismissed, error when fetching/searching fails.

Workspace open handoff

  • workspace.open
    • One trace covering open and openFromSidebar handoff to VS Code.
    • Properties: source, workspace/build/agent state buckets, handoff category.
    • Measurements: agent counts.
    • Result: success when VS Code open handoff succeeds, aborted when the user cancels workspace/agent/recent-folder selection, error when the handoff fails.

Devcontainer open handoff

  • workspace.devcontainer.open
    • One trace around devcontainer handoff.
    • Properties: mode/category and bounded failure category on handled/known errors.
    • Result: success or error.

Start/update prompt decisions

  • workspace.start.prompted

    • For the stopped-workspace modal that asks whether to start/update.
    • Properties: whether an update was offered and the accepted action when present.
    • Result: success when a choice is accepted, aborted when dismissed.
  • workspace.update.prompted

    • Reuse the existing event name.
    • Add a compact prompt-kind/action property where practical so parameter prompting and confirmation prompting remain distinguishable without adding another update event.

Diagnostic command outcomes

  • command.diagnostic.completed
    • One event name for coder.speedTest, coder.supportBundle, and coder.exportTelemetry.
    • Property: commandId plus bounded outcome/failure category where useful.
    • Measurements: bounded summaries only, e.g. requested speedtest seconds, parsed interval count/throughput, exported event count. No paths or raw output.
    • Result: success, aborted, or error using span.markAborted() / span.markFailure() for handled outcomes.

Test plan

  • Update test/unit/instrumentation/workspace.test.ts for prompt telemetry behavior.
  • Update test/unit/telemetry/export/command.test.ts for export diagnostic outcomes.
  • Add focused command tests only where the behavior cannot be validated through smaller exported helpers.
  • Run targeted unit tests first, then pnpm typecheck or targeted typechecking if needed.

@EhabY EhabY self-assigned this Jun 8, 2026
@EhabY EhabY force-pushed the feat/issue-986-telemetry-traces branch 3 times, most recently from 13ac369 to fc8b81d Compare June 10, 2026 12:08
@EhabY EhabY requested a review from mtojek June 10, 2026 12:13
Comment thread src/instrumentation/outcomes.ts Outdated
Comment thread src/instrumentation/outcomes.ts Outdated
Comment thread src/instrumentation/workspace.ts
Comment thread src/instrumentation/workspace.ts
Comment thread src/instrumentation/workspaceOpen.ts Outdated
Comment thread test/unit/uri/uriHandler.test.ts Outdated
Comment thread test/unit/instrumentation/workspaceOpen.test.ts
Comment thread src/commands.ts
Comment thread src/commands.ts
Comment thread src/instrumentation/diagnostics.ts
@EhabY EhabY force-pushed the feat/issue-986-telemetry-traces branch from e825971 to 78285e9 Compare June 10, 2026 14:36
@EhabY EhabY requested a review from mtojek June 10, 2026 14:39

@mtojek mtojek left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ship it!

EhabY and others added 6 commits June 10, 2026 18:04
Split instrumentation/commands.ts into workspaceOpen.ts, diagnostics.ts,
and shared outcomes.ts to mirror a future commands.ts split. Unify trace
methods on the cancel/fail/succeed* trio, shorten trace<Noun> names, and
make ExportTelemetryObserver a required structural subset of
DiagnosticTrace so callers pass their trace directly.

Commands now keep business bodies in named run* methods instead of
inline closures, the open source defaults to "command" rather than
being inferred from argument presence, and the dead openWorkspace
wrapper is gone. Restore workspaceName on workspace.update.prompted and
workspace.start.prompted to match the other operation traces. Rename
the telemetry-only test to updateWorkspace.telemetry.test.ts.
Rename the Commands openTelemetry field to workspaceOpenTelemetry so it
mirrors its class like the sibling fields (and stops reading like the
OpenTelemetry framework), group the three telemetry helpers together in
the constructor, and rename AuthTelemetry.traceAuthRecovery to
traceRecovery to drop the qualifier the class already implies.
Address review feedback and standardize span outcome vocabulary:

- Rename failure -> error (markError, error.type, *ErrorCategory) to match
  the OTel span status and the error.type semantic convention.
- Unify cancellation on "abort" across the telemetry layer; the VS Code
  cancellation mirrors (CancellationToken, ProgressResult) stay as-is.
- OTLP export: map aborted spans to UNSET and backfill error.type on error
  spans from the exception type/code, falling back to OTel's _OTHER.
- DevContainer casing, type-safe start/update button captions, and test
  cleanups (shared open defaults, multi-connected-agent coverage).
@EhabY EhabY force-pushed the feat/issue-986-telemetry-traces branch from 78285e9 to c0057d7 Compare June 10, 2026 15:04
@EhabY EhabY merged commit 0f29224 into main Jun 10, 2026
12 of 13 checks passed
@EhabY EhabY deleted the feat/issue-986-telemetry-traces branch June 10, 2026 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Telemetry: instrument workspace and diagnostic command workflows

2 participants