Skip to content

WIP: Record one cli.command telemetry event per run - #46

Draft
zmofei wants to merge 4 commits into
run-recordfrom
telemetry-events
Draft

zmofei wants to merge 4 commits into
run-recordfrom
telemetry-events

Conversation

@zmofei

@zmofei zmofei commented Sep 24, 2026 •

Copy link
Copy Markdown
Member

WIP — not ready for review. Stacked on #56, for its dated_jsonl; #57 below it owns collection: every call site reports to run_record. This PR changes no call site; it adds the telemetry projection of that record, the sink, and one line in run_record::finish that hands the record to telemetry.

Each run now records one cli.command telemetry event, following the draft schema in the Telemetry proposal. By default the event is appended to ~/.mapbox/.telemetry/<UTC date>.jsonl (kept 7 days) and nothing leaves the machine.

What's in it

  • src/telemetry_event.rs: decides what the event contains. At exit it chooses each field from the run's run_record::Record (command path, invocation, params, options, token prefix and account, exit and error codes, stdout bytes, duration, requests) under the rules below. The record holds raw facts, so this file is the whole of what can leave the machine.
  • src/telemetry_sink.rs: where the event goes. A Sink trait with two implementations; selected() is the one place that picks one:
    • FileSink (used today) appends one JSON line per run. It's for local testing.
    • HttpSink is the interface for Mapbox Events, with its contract written down. It's not implemented, so it isn't selected.
  • Parameters: a value is sent only for enums, booleans, spec-typed numbers and language/country/types. Free strings send their length, --file its size, --data its size and top-level keys. lon/lat send their name only.
  • Requests are recorded in http::send, the single request path from Route every request through http::send and collect each run in run_record #57 (held by only_http_sends_requests). Event files are written through Route every request through http::send and collect each run in run_record #57's dated_jsonl.
  • Opt-out: MAPBOX_CLI_NO_TELEMETRY=1. A persisted mapbox config set telemetry off is in the stacked PR WIP: Add a persisted telemetry setting to mapbox config #47.
  • Groundwork for workflows (no workflow command exists yet):
    • parentEventId is read from MAPBOX_CLI_PARENT_EVENT, and only when it is a UUID. A workflow will set it on the steps it launches, so a mapbox run inside a user's script is tied to its workflow instead of counted as direct use. For that, eventId is fixed on first use (telemetry_event::event_id()), so a workflow can hand it to its steps before the event is built.
    • workflow records the source (builtin, marketplace or custom), and the name only for workflows Mapbox names. set_workflow drops the name for custom, whatever the caller passes.
    • For a workflow run, stepCount and up to 20 steps. A step is built only by cli_step (command names checked against the command tree, plus exit and error code) or script_step (exit code and duration only), so a script step can't carry a command name or an error category.
  • A panic is recorded as errorCode: "panic". completion records nothing, since it runs at every shell startup.

Behavior changes to confirm

  • The event is written only when ~/.mapbox already exists; recording never creates or chmods it. Command history (Record command history and add mapbox history #56) is on by default and does create a missing ~/.mapbox, so once both land, telemetry on a fresh machine starts recording from the second run.
  • Whichever of this and Record command history and add mapbox history #56 lands second needs a rebase: both add a consumer in run_record::finish, and tests/telemetry_events.rs's without_a_config_directory_nothing_is_created will need MAPBOX_HISTORY=0 once history can create the directory.

Before HttpSink is implemented

  • cli.command has to be registered in mapbox/event-schema and get a warehouse table.
  • The README privacy disclosure has to be rewritten: it still says "never the operation or its arguments". This PR only adds a short note about the local file.
  • Legal review of the collected fields.

Not checked

  • The Windows tilesets-cli exit path hasn't been compiled locally.
  • Nothing is sent anywhere yet. HttpSink is an interface only.
  • installMethod is inferred from the binary's location, so it reads other when MAPBOX_INSTALL_DIR was used.

Each run appends one line of execution metadata to
~/.mapbox/history/<UTC date>.jsonl, kept 30 days: the command path from
the command tree, invocation, exit code, error code, duration, request
count and the last five request ids. Never an argument value, URL, error
message or account: arguments carry search terms, file paths and ids.
The file is written through dated_jsonl: private, one file per UTC day,
pruned to the retention window.

On by default. `mapbox config set history off` turns it off for good,
MAPBOX_HISTORY=0 or =1 for a session over the setting. With it off,
nothing is written and no directory is created. With it on, the config
directory is created when missing, so history works the same for someone
who only ever set MAPBOX_ACCESS_TOKEN. Not recorded: --help, --version,
completion, history itself, and runs under sudo, whose root-owned files
would stop the user's own runs appending (aws/aws-cli#10031).

`mapbox history list` shows the most recent runs, newest first, and
`mapbox history show [id]` one run, the newest by default, found by any
prefix of its id.

The read-only-command tests that held ~/.mapbox untouched now check both
sides: with history on only `history/` may appear, with it off nothing.
- `history list` text output gets a header row.
- Tests that hold a run to leaving nothing on disk turn history off in
  their helper instead of looping over both states; tests/history.rs
  checks what history leaves, and that skipped runs create nothing.
- README says turning history off keeps what was already recorded.
- Drop the setting count from config.rs's module doc, which every new key
  would have to edit.
Moves dated_jsonl::shed here from the diagnostic-log PR so history, on by
default, is bounded by size as well as by age. trim now keeps exactly the
bytes shed hands it; the margin was applied twice before.
@zmofei
zmofei added this pull request to stack #60 September 28, 2026 12:18
Built from the run_record the foundation change already collects, so no
call site changes here: telemetry_event chooses each field from the
record under its privacy rules, and telemetry_sink delivers the line (to
a local file for now, through dated_jsonl). run_record::finish hands the
record to telemetry.
@zmofei zmofei self-assigned this Sep 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant