feat(analytics): add Sentry, MetaMetrics, Segment, and Tempo skills

domains/analytics/knowledge/metrametrics-identity.md

@@ -0,0 +1,41 @@
+---
+name: metrametrics-identity
+domain: analytics
+description: isOptIn:true unconditionally strips user identity in MetaMetricsController — always sends as anonymous ID
+---
+
+# MetaMetrics Identity Stripping
+
+## The Mechanism
+
+In `MetaMetricsController` (`app/scripts/controllers/metametrics-controller.ts`):
+
+```typescript
+if (excludeMetaMetricsId || (isOptIn && !metaMetricsIdOverride)) {
+  idType = 'anonymousId';
+  idValue = METAMETRICS_ANONYMOUS_ID; // 0x0000000000000000
+}
+```
+
+When `isOptIn: true` with no `metaMetricsIdOverride`:
+- The user's real `metaMetricsId` is discarded
+- ALL such events share a single anonymous ID (`0x0000000000000000`) in Segment
+- User-level attribution is completely lost
+
+This is **unconditional** — it applies to fully opted-in users with valid IDs, not just anonymous users.
+
+## Intended Use
+
+The onboarding opt-in flow (`creation-successful.tsx`) — where the user hasn't committed to MetaMetrics yet and no `metaMetricsId` has been persisted. The event must fire regardless of opt-in state.
+
+## The Misuse Pattern
+
+Post-opt-in `trackEvent` calls with `{ isOptIn: true }` without `metaMetricsIdOverride`. Defeats the purpose of Segment user-level dimensions (account types, feature flags).
+
+## Detection
+
+```bash
+grep -r "isOptIn: true" app/scripts/ ui/ --include="*.ts" --include="*.tsx"
+```
+
+Any occurrence outside `creation-successful.tsx` (or the onboarding flow) is suspect.

domains/analytics/knowledge/segment-governance.md

@@ -0,0 +1,40 @@
+---
+name: segment-governance
+domain: analytics
+description: Segment event governance via segment-schema is advisory — no CI enforcement prevents unregistered events from shipping
+---
+
+# Segment Event Governance
+
+## Architecture
+
+| Component | Location |
+|-----------|----------|
+| Tracking plan | `Consensys/segment-schema` → `tracking-plans/metamask-extension.yaml` |
+| Event registry | `shared/constants/metametrics.ts` → `MetaMetricsEventName` enum (300+ entries) |
+| Review process | `CONTRIBUTING.md` in segment-schema; Data Council review |
+| Governance channel | `#metamask-metametrics`, `@consensys/data-council` |
+
+## The Gap
+
+There is **no CI enforcement** in the extension repo. A developer can:
+
+1. Add entry to `MetaMetricsEventName` enum
+2. Call `trackEvent` with it
+3. Merge and ship to production
+
+...without registering in segment-schema or going through Data Council review.
+
+## Implications
+
+- Schema drift between tracking plan and production events
+- No property schema validation for unregistered events
+- Billing impact goes unreviewed
+- Data Council review is bypassable by omission
+
+## Recommended Fix
+
+CI check that:
+1. Parses `MetaMetricsEventName` entries
+2. Validates each against `tracking-plans/metamask-extension.yaml`
+3. Fails build if event is missing from the plan

domains/analytics/knowledge/span-sub-sampling.md

@@ -0,0 +1,72 @@
+---
+name: span-sub-sampling
+domain: analytics
+description: Deterministic per-trace sub-sampling for high-frequency custom spans — global tracesSampleRate × span sub-rate, traceId-hash bucketed
+---
+
+# Span Sub-Sampling
+
+Durable fix for a custom span that fans out and eats the span budget. Layer a per-trace sub-rate **under** the global `tracesSampleRate`, keyed on the trace id so every span in a trace is kept-or-dropped together. Source: [PR #39891](https://github.com/MetaMask/metamask-extension/pull/39891) (`shared/lib/wrapper-sampling.ts`).
+
+## Rate Math
+
+```
+effective rate = global tracesSampleRate × span sub-rate
+```
+
+- Global `tracesSampleRate` is already small (extension prod: 0.75%).
+- The sub-rate cuts the custom span on top: `0.75% × 1% = 0.0075%`.
+- PR #39891 ships a sub-rate of 0.5% (`WRAPPER_SAMPLE_RATE = 0.005`) — a conservative pilot — and names 5% as the step-up once the denylist is confirmed effective in production.
+
+Pick the sub-rate from how many sampled traces the metric needs to stay useful — not from the quota alone. Too low and the metric goes dark.
+
+## Pattern
+
+```ts
+const WRAPPER_SAMPLE_RATE = 0.005;
+
+// Deterministic: same answer for the same traceId, so all spans in a trace
+// are kept or dropped together — clean waterfalls, no partial gaps.
+export function shouldSampleWrappers(traceId: string | undefined): boolean {
+  if (!traceId || traceId.length < 8) {
+    return false;
+  }
+  const hashBucket = parseInt(traceId.slice(0, 8), 16) % 10000;
+  return hashBucket < WRAPPER_SAMPLE_RATE * 10000;
+}
+```
+
+**Why deterministic, not `Math.random()` per call:** independent per-span sampling shreds a trace into partial waterfalls (some spans present, siblings missing) — useless for attribution. Hashing the trace id makes keep/drop a property of the whole trace.
+
+## Gate Order (cheapest check first)
+
+```ts
+const traceId = sentryGetActiveSpan()?.spanContext().traceId;
+if (!traceId || isReadOnlyAction(action) || !shouldSampleWrappers(traceId)) {
+  return doWorkWithoutSpan();
+}
+return trace({ name, op, data }, doWorkWithSpan);
+```
+
+1. No active trace → no span.
+2. Denylist → skip noise (below).
+3. Sub-sample miss → skip this trace's spans.
+
+## Denylist: cut before you sample
+
+Drop spans with no timing/attribution signal before sub-sampling. In PR #39891, read-only verbs are ~90% of `messenger.call` volume:
+
+```ts
+const READ_ONLY_VERB = /^(?:get|has|find|is|peek)(?:[A-Z]|$)/u;
+```
+
+Removing ~90% of volume before the sample multiplies headroom — a higher sub-rate then yields the same span budget, so kept traces are denser and more useful.
+
+## Where the Gate Goes
+
+- **Consumer (extension):** spans go through `trace()`. Gate at the call site, or for a whole span family inside the wrapper. `traceId` from `sentryGetActiveSpan()?.spanContext().traceId`.
+- **Controller package (core):** controllers call an injected `trace` callback. Gate in the package's trace util or the callback so every consumer inherits the cap. Pull the trace id from the controller's tracing context, not a fresh Sentry import.
+
+## Kill Switch
+
+Ship every always-on span family with an env disable flag (PR #39891: `SENTRY_DISTRIBUTED_TRACING_DISABLED` returns the messenger un-wrapped). It turns a future emergency cut into a config flip instead of a cherry-pick.

domains/analytics/skills/grafana-tempo-queries/skill.md

@@ -0,0 +1,125 @@
+---
+name: grafana-tempo-queries
+description: Query backend traces in Grafana Tempo with TraceQL — find traces by service or span attribute, fetch a trace by id, inspect its span tree, and enumerate tag values. Covers the datasource-proxy access path, the credential-expiry failure that returns empty results indistinguishable from "no data", the negative control that proves a filter actually applied, and the id/kind/base64 decoding quirks in the response. Use when investigating backend latency, checking what the backend recorded for a request, or establishing which infrastructure tiers a trace reaches. Triggers on Tempo, TraceQL, Grafana traces, backend span inspection, "does the backend have this trace", or tracing a request past the API boundary.
+maturity: experimental
+---
+
+# grafana-tempo-queries
+
+Tempo holds **backend** spans. Client spans from the extension and mobile go to Sentry via the SDK's own transport and never appear here — so a Tempo trace normally starts at an inbound server span, and a missing root is expected rather than broken. To join the two halves, see `sentry-grafana-correlation`.
+
+## Setup
+
+Everything goes through Grafana's datasource proxy, so a Grafana session is the only credential needed. Keep the host, datasource uid, org id, and session in your environment — this repository is public, so never commit them.
+
+```bash
+# Set these once per shell, from your own Grafana instance:
+#   GRAFANA_HOST   e.g. https://grafana.<your-org-domain>
+#   TEMPO_UID      the Tempo datasource uid (see discovery below)
+#   GRAFANA_ORG    the numeric org id the datasource belongs to
+#   GRAFANA_SESSION  value of the grafana_session cookie from an authenticated browser
+BASE="$GRAFANA_HOST/api/datasources/proxy/uid/$TEMPO_UID"
+AUTH=(-H "Cookie: grafana_session=$GRAFANA_SESSION" -H "X-Grafana-Org-Id: $GRAFANA_ORG")
+```
+
+Discover the datasource uid rather than guessing it:
+
+```bash
+curl -s "$GRAFANA_HOST/api/datasources" "${AUTH[@]}" \
+  | node -e 'JSON.parse(require("fs").readFileSync(0)).filter(d=>d.type==="tempo").forEach(d=>console.log(d.uid,d.name))'
+```
+
+## Check the instrument before believing a result
+
+**A stale session returns HTTP 401 with an empty body, and a naive parser reports that as zero results** — indistinguishable from "this data does not exist". This is the single most expensive failure mode here: it produces confident negative conclusions about instrumentation coverage.
+
+```bash
+# 1. Prove you are authenticated. Do this first, every session.
+curl -s -o /dev/null -w 'grafana auth: HTTP %{http_code}\n' "$GRAFANA_HOST/api/user" "${AUTH[@]}"
+
+# 2. Prove the filter is actually being applied, with a query that must match nothing.
+curl -s -G "$BASE/api/search" "${AUTH[@]}" \
+  --data-urlencode 'q={span.db.system = "not-a-real-db-xyz"}' \
+  --data-urlencode "start=$START" --data-urlencode "end=$NOW" \
+  | node -e 'const j=JSON.parse(require("fs").readFileSync(0));console.log("control traces:",(j.traces||[]).length,"(must be 0)")'
+```
+
+If several different filters all return exactly your `limit`, the filter is not being applied — treat the results as unfiltered until the negative control returns 0.
+
+## Core queries
+
+Every endpoint wants an explicit epoch-seconds window. Omitting it on a by-id lookup makes the request hunt across all blocks and hit a context deadline.
+
+```bash
+NOW=$(date +%s); START=$((NOW-3600))
+```
+
+**Search by TraceQL.** Returns trace summaries plus the spans that matched.
+
+```bash
+curl -s -G "$BASE/api/search" "${AUTH[@]}" \
+  --data-urlencode 'q={resource.service.name="my-service"}' \
+  --data-urlencode "start=$START" --data-urlencode "end=$NOW" \
+  --data-urlencode "limit=20"
+```
+
+**Fetch one trace in full** (OTLP JSON: resource batches → scope spans → spans).
+
+```bash
+curl -s "$BASE/api/traces/$TRACE_ID?start=$START&end=$NOW" "${AUTH[@]}"
+```
+
+**Enumerate values for a tag** — useful for inventorying what a fleet emits. Expect a `502` on high-cardinality tags; fall back to inspecting individual traces rather than concluding the tag is unused.
+
+```bash
+curl -s -G "$BASE/api/v2/search/tag/span.db.system/values" "${AUTH[@]}" \
+  --data-urlencode "start=$START" --data-urlencode "end=$NOW"
+```
+
+## TraceQL patterns worth knowing
+
+| Goal | Query |
+| --- | --- |
+| One service | `{resource.service.name="svc-name"}` |
+| Several services | `{resource.service.name=~"(svc-a|svc-b)-prd"}` |
+| Attribute present at all | `{span.db.system != nil}` |
+| Span kind | `{kind=server}`, `{kind=client}` |
+| Slow spans | `{duration > 1s}` |
+| **Two conditions anywhere in the same trace** | `{resource.service.name="svc-a"} && {span.db.system != nil}` |
+
+The last one is the important one: `&&` between two brace groups is a **trace-level** conjunction, not a single-span filter. It answers "does a request into this service reach a database at all", which is how you map how deep a trace goes without reading traces one at a time.
+
+## Reading the response
+
+- **Span and trace ids are base64**, not hex. Decode before comparing them to anything from a header or from Sentry: `Buffer.from(id,"base64").toString("hex")`.
+- **`kind` is a string** (`SPAN_KIND_SERVER`, `SPAN_KIND_CLIENT`, `SPAN_KIND_INTERNAL`), not the numeric enum. Filtering on `sp.kind === 2` silently matches nothing.
+- **Search results drop leading zeros from trace ids.** A 31-character id is a 32-character id with a leading zero; zero-pad before using it anywhere else, or the lookup fails for a reason that looks like absence.
+- **`rootServiceName: "<root span not yet received>"`** means the trace's root is not in Tempo. For client-originated requests that is the normal case — the root is a client span living in Sentry — and it is the marker for finding them.
+- Resource attributes carry deployment context (`service.name`, kubernetes pod/namespace/cluster, region); span attributes carry the request (`http.*`, `net.*`, `db.*`).
+
+## Deep links for sharing
+
+A link is more useful than a pasted id. Build a Grafana Explore URL with the query pre-filled:
+
+```bash
+node -e '
+const left={datasource:process.env.TEMPO_UID,
+  queries:[{refId:"A",datasource:{type:"tempo",uid:process.env.TEMPO_UID},queryType:"traceql",query:process.argv[1]}],
+  range:{from:"now-6h",to:"now"}};
+console.log(`${process.env.GRAFANA_HOST}/explore?orgId=${process.env.GRAFANA_ORG}&left=${encodeURIComponent(JSON.stringify(left))}`);
+' '<trace-id-or-traceql>'
+```
+
+Prefer an absolute `from`/`to` when the link needs to outlive the event; a relative window slides off it and the reader opens an empty result.
+
+## Failure modes
+
+| Symptom | Cause | Response |
+| --- | --- | --- |
+| All queries return 0 | Session expired (401, empty body) | Check `/api/user` first |
+| Every filter returns exactly `limit` | Filter not applied | Run the negative control |
+| By-id lookup times out | No time window | Pass `start`/`end` |
+| Tag-values returns 502 | High cardinality | Inspect traces directly |
+| Id from search not found elsewhere | Leading zeros stripped | Zero-pad to 32 chars |
+| Kind filter matches nothing | Comparing to a number | Compare to `SPAN_KIND_*` |
+| Trace has no root | Root is a client span | Expected; see `sentry-grafana-correlation` |

domains/analytics/skills/instrumentation/repos/metamask-extension.md

@@ -0,0 +1,61 @@
+---
+repo: metamask-extension
+parent: instrumentation
+---
+
+## Key Files
+
+| Content | Path |
+|---------|------|
+| Sentry trace wrapper | `shared/lib/trace.ts` |
+| Trace name enum | `shared/lib/trace.ts` → `TraceName` |
+| MetaMetrics controller | `app/scripts/controllers/metametrics-controller.ts` |
+| Event enum | `shared/constants/metametrics.ts` → `MetaMetricsEventName` |
+| Sentry setup + sample rate | `app/scripts/lib/setupSentry.js` → `getTracesSampleRate()` |
+| Segment tracking plan | `Consensys/segment-schema` → `tracking-plans/metamask-extension.yaml` |
+
+## Cross-Process Context (UI → Background)
+
+The extension has two Sentry hubs — one in the UI process and one in the background service worker. A trace starting in UI and continuing in background requires explicit context propagation across the RPC boundary:
+
+```typescript
+// Serialize at UI call site
+const context: SerializedTraceContext = {
+  _name: TraceName.MyOperation,
+  _traceId: span.spanContext().traceId,
+  _spanId: span.spanContext().spanId,
+}
+
+// Background receives context, creates child span
+trace({ name: TraceName.MyOperation, parentContext: context }, async () => { ... })
+```
+
+Without propagation: Sentry shows two disconnected operations. With propagation: complete tree from user action to RPC call.
+
+## Sentry Sample Rate
+
+```bash
+grep -n "tracesSampleRate" app/scripts/lib/setupSentry.js
+# Verify current value before calculating — it has changed between releases
+```
+
+## Sentry Traces Explorer Query (Volume Estimation)
+
+```
+Environment: production | Time range: 30 days | Mode: aggregate
+Query: span.op:http.client span.description:*{endpoint}*
+Group by: span.description, transaction
+Sort: -count(span.duration)
+```
+
+## Detect `isOptIn` Misuse
+
+```bash
+grep -rn "isOptIn: true" app/scripts/ ui/ --include="*.ts" --include="*.tsx"
+# Any occurrence outside the onboarding opt-in flow is suspect
+```
+
+## Data Council Contact
+
+- Slack: `#metamask-metametrics`
+- Team: `@consensys/data-council`