feat(tracing): suppress conversation payloads in traces for private channels and DMs

[sentry-junior]

Problem

Conversations in private Slack channels (G-prefix) and DMs (D-prefix) may contain sensitive content. Today, Junior captures full conversation payloads into Sentry traces and logs regardless of channel type. This includes user messages, assistant responses, system prompts, tool arguments, and tool results — all of which can contain private information.

Current State

Payload capture is pervasive

There are 6 capture layers that record conversation content into Sentry:

Layer	File	What it captures
Vercel AI integration	instrumentation.ts	Auto-captures all AI request/response payloads (recordInputs: true, recordOutputs: true)
pi-ai traced stream	chat/pi/traced-stream.ts	gen_ai.input.messages, gen_ai.output.messages, gen_ai.system_instructions
Direct completions	chat/pi/client.ts	Same gen_ai attributes for completeText calls
Agent turn span	chat/respond.ts	app.message.input (summarized), gen_ai.input.messages, gen_ai.output.messages
Tool call spans	chat/tools/agent-tools.ts	gen_ai.tool.call.arguments, gen_ai.tool.call.result
Turn result	chat/services/turn-result.ts	app.message.output (summarized)

Additionally:

• sendDefaultPii: true in SDK init
• Sentry structured logs (logging.ts) emit attributes that may contain message content
• streamGenAiSpans: true enables automatic gen_ai span streaming

Channel type detection already exists

// packages/junior/src/chat/slack/client.ts
// C: public channel — payload capture OK
// G: private channel / group DM — suppress payloads
// D: direct message (1:1) — suppress payloads
export function isDmChannel(channelId: string): boolean { ... }

The channel ID is available throughout the turn via ReplyRequestContext.correlation.channelId / spanContext.slackChannelId.

Proposed Approach

Design principle: allowlist-based, defense-in-depth

Only public channels (C*) should be eligible for payload capture. Private channels (G*), DMs (D*), and unknown/missing channel IDs must suppress all conversation payloads. Default to suppression.

Layer 1: Capture-time prevention (primary)

1. Add a privacy decision helper near existing channel utilities:

export function shouldCaptureConversationPayloads(channelId: string | undefined | null): boolean {
  const normalized = channelId?.trim();
  return Boolean(normalized && normalized.startsWith("C"));
}

2. Compute once per turn, carry through ReplyRequestContext and span context:

const capturePayloads = shouldCaptureConversationPayloads(channelId);

3. Disable vercelAIIntegration auto-capture globally:

Sentry.vercelAIIntegration({
  recordInputs: false,
  recordOutputs: false,
})

Then manually record payload attributes only when capturePayloads === true.

4. Guard every manual payload write across the 6 capture sites. Example pattern:

if (capturePayloads) {
  span.setAttribute("gen_ai.input.messages", serialized);
} else {
  span.setAttribute("gen_ai.input.message_count", messages.length);
}

5. Guard Sentry log emissions in logging.ts to omit content attributes for private turns.

Layer 2: Send-time scrubbing (defense-in-depth)

Add Sentry SDK hooks as a backstop against regressions or third-party integrations that bypass the guards:

• beforeSendTransaction — scrub payload attributes from transaction and child spans
• beforeSendSpan — scrub payload attributes from individual spans
• beforeSendLog — scrub payload attributes from structured logs
• beforeSend — scrub from error events, extras, breadcrumbs

Use a denylisted attribute set:

gen_ai.input.messages, gen_ai.output.messages, gen_ai.system_instructions,
app.message.input, app.message.output,
gen_ai.tool.call.arguments, gen_ai.tool.call.result

Gate scrubbing on a per-turn app.payload_capture attribute (true = public channel, keep payloads).

What to keep for all channel types

Operational metadata must remain to preserve debuggability:

• Token counts, model name, provider name, latency
• Tool names (not arguments/results)
• Status codes, error categories
• Message count, output length
• app.slack.channel_kind, app.payload_capture

Also consider

• Set sendDefaultPii: false globally and explicitly add only approved identifiers
• Add app.slack.channel_kind attribute (public_channel / private_channel_or_mpim / dm / unknown) for observability

Files to modify

• packages/junior/src/instrumentation.ts — SDK init, hooks, integration config
• packages/junior/src/chat/slack/client.ts — add shouldCaptureConversationPayloads / getSlackChannelKind
• packages/junior/src/chat/pi/traced-stream.ts — guard payload attributes
• packages/junior/src/chat/pi/client.ts — guard payload attributes
• packages/junior/src/chat/respond.ts — guard payload attributes, carry flag
• packages/junior/src/chat/tools/agent-tools.ts — guard tool call/result attributes
• packages/junior/src/chat/services/turn-result.ts — guard output attributes
• packages/junior/src/chat/logging.ts — guard log content attributes

Verification

• Unit tests for shouldCaptureConversationPayloads (C=true, G/D/undefined/unknown=false)
• Unit tests for sanitizer removing denylisted keys
• Integration-style test: simulated G/D turn produces no payload attributes in spans or logs
• Regression test: C turn still captures payloads normally
• Verify against @sentry/node@10.53.1 hook support (beforeSendSpan, beforeSendLog)

Risks

• Highest priority: vercelAIIntegration({ recordInputs: true }) auto-captures payloads independent of any guard. Must be disabled globally or proven scrubbed at send time.
• Tool results can contain full Slack thread content, file contents, or code — must be treated as payload, not metadata.
• Unknown channel IDs must default to suppression.
• Sentry logs are a separate data path from spans — require independent handling.

---

Action taken on behalf of David Cramer.