feat(telemetry): add anonymous opt-out OpenShell usage telemetry#1433
feat(telemetry): add anonymous opt-out OpenShell usage telemetry#1433kirit93 wants to merge 2 commits into
Conversation
|
🌿 Preview your docs: https://nvidia-preview-pr-1433.docs.buildwithfern.com/openshell |
4aa8e3c to
fab5164
Compare
|
Looks like the Rust telemetry helper shells out to Suggestion: we might want to implement the telemetry publisher fully in Rust for better portability. That would keep the runtime behavior inside the shipped binaries, avoid depending on Python/script packaging, and make the opt-out/publish path easier to test across gateway, supervisor, and package installs. |
|
@kirit93 Can you add a short telemetry section to the primary README as part of this PR? |
20df3af to
96c8730
Compare
dc0aa8a to
dbab5e8
Compare
russellb
left a comment
There was a problem hiding this comment.
Something seems to be wrong with your branch. It appears to be reverting a lot of changes from main.
|
@russellb yeah i'm looking at this as well |
| const TELEMETRY_EVENT_QUEUE_CAPACITY: usize = 1024; | ||
| const MAX_TELEMETRY_INTEGER: u64 = 9_223_372_036_854_775_807; | ||
| const CLIENT_ID: &str = "415437562476676"; | ||
| const DEFAULT_ENDPOINT: &str = "https://events.telemetry.data-uat.nvidia.com/v1.1/events/json"; |
There was a problem hiding this comment.
Is this something that could ever be visible to people outside of NVIDIA?
dbab5e8 to
5b2e26f
Compare
|
|
||
| let effectively_denied = force_deny | ||
| || (!allowed && l7_config.config.enforcement == crate::l7::EnforcementMode::Enforce); | ||
| emit_activity_simple(activity_tx, effectively_denied, "l7_policy"); |
There was a problem hiding this comment.
[P2] This records an L7 activity event before the forward-proxy SSRF checks. If L7 allows but the later SSRF check denies the same request, the same request gets counted once as allowed L7 activity here and again as denied SSRF activity below. That inflates networkActivityCount and understates denialRatePct. Please defer the L7 allowed activity record until after SSRF succeeds, or otherwise ensure each forward request contributes exactly one activity event.
Signed-off-by: Kirit93 <kthadaka@nvidia.com>
5b2e26f to
bdc660f
Compare
Signed-off-by: Kirit Thadaka <kthadaka@nvidia.com>
Summary
Add lightweight, opt-out telemetry for anonymous OpenShell usage metrics. This gives the project aggregate visibility into sandbox lifecycle activity, policy updates, provider operations, sandbox creation shape, and sandbox network-denial summaries.
Telemetry does not track individual users and does not collect personal data, sandbox IDs, sandbox names, hosts, paths, model names, provider names, credentials, prompts, request payloads, or user content.
The telemetry events captured are:
Sandbox creation: records whether a sandbox create request succeeded or failed. It also records whether GPU was requested as true/false, the number of providers attached as a count only, whether a custom policy was provided as true/false with no policy details collected, whether the template source was default or image-based without collecting the image name, and which compute driver was used (docker, kubernetes, podman, vm, or unknown).
Sandbox deletion: records whether a sandbox delete request succeeded or failed.
Sandbox policy updates: records whether a gateway-level sandbox policy update succeeded or failed. Policy contents and rule details are not collected.
Provider lifecycle: records provider create, update, and delete outcomes. Provider type is mapped to a broad profile bucket such as anthropic, claude, codex, github, gitlab, nvidia, openai, outlook, or custom. Provider names, credentials, endpoints, and configuration details are not collected.
Policy draft decisions: records whether approve, reject, approve-all, or undo actions succeeded or failed. It also records the number of rules affected as a count only. Rule contents are not collected.
Sandbox network activity summaries: records aggregate network activity counts, denied action counts, denial rate percentage, and denial counts grouped into coarse categories such as connect policy, forward policy, L7 policy, SSRF, bypass, stale policy, or unknown. Destinations, hostnames, URLs, request paths, payloads, binaries, and raw deny messages are not collected.
Telemetry can be disabled at deployment time by setting OPENSHELL_TELEMETRY_ENABLED=false on the gateway.
Related Issue
Fixes #1054
Testing
mise run pre-commitpassesChecklist