feat(profiling): Route raw profiles through objectstore service by markushi · Pull Request #6025 · getsentry/relay

markushi · 2026-05-28T07:50:23Z

Summary

Extract StoreRawProfile into the objectstore service to upload raw profile blobs (e.g. Perfetto traces) before forwarding to Kafka
Replace inline raw profile blob in the Kafka message with an objectstore key reference
Objectstore service always forwards the StoreProfileChunk to the Store service, even if the upload fails, ensuring the Kafka message is produced

Test plan

Verify profile chunks without raw profiles are sent directly to Store
Verify profile chunks with raw profiles are routed through Objectstore
Verify Kafka message contains objectstore key reference instead of raw blob
Verify Kafka message is produced even when objectstore upload fails
Run integration tests: pytest tests/integration/test_profile_chunks_perfetto.py

🤖 Generated with Claude Code

This PR is part of a stack:

feat(profiling): Add Perfetto trace format support #5659
feat(profiling): Add pipeline workflow to perfetto profiling #5932
feat(profiling): Route raw profiles through objectstore service (this PR)

Extract StoreRawProfile into the objectstore service and replace inline raw profile blob with objectstore key reference in the Kafka message. Co-Authored-By: Claude <noreply@anthropic.com>

Align the objectstore usecase identifier with the naming convention so it can be easily distinguished from a future profiles_v2 usecase. Co-Authored-By: Claude <noreply@anthropic.com>

Add try_send_to_objectstore to StoreHandle, which returns the message when the objectstore service is not configured. Profile chunks fall back to sending the StoreProfileChunk directly to Store, preventing data loss. Also log session creation errors matching the pattern in other objectstore handlers. Co-Authored-By: Claude <noreply@anthropic.com>

The rename in 3a83bc5 only updated MessageKind::as_str() but missed the Usecase::new() call, causing raw profiles to be stored under the generic "profiles" namespace instead of "profiles_raw". Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

lcian · 2026-05-28T11:49:31Z

        let trace_attachments = Usecase::new("trace_attachments")
            .with_expiration_policy(ExpirationPolicy::TimeToLive(DEFAULT_ATTACHMENT_RETENTION));
+        let profiles = Usecase::new("profiles")
+            .with_expiration_policy(ExpirationPolicy::TimeToLive(DEFAULT_ATTACHMENT_RETENTION));


Is this what we want?

Note that DEFAULT_ATTACHMENT_RETENTION means event attachments here. Even if that happens to be the correct retention you actually want, you likely want to use a different constant here to not mix things up.
Here you likely want to use the profiles retention instead.

Good point, I was wondering about this as well. Some quick research showed me that the monolith builds a project config, which then gets propagate to relay. And that config is coming from getsentry. At this point I feel like pulling on a thread a bit 😅 but I guess we need this, maybe @jjbayer could chime in here as well, before I start adding support for this.

We would need to

[getsentry] Propagate the config based on the plan, looks like this already happens automatically as it iterates over all data categories

[sentry] Extend the RETENTIONS_CONFIG_MAPPING to also include DataCategory.PROFILE_DURATION and DataCategory.PROFILE_DURATION_UI, so those get propagated to relay

[relay] Extend RetentionsConfig and use that config here

That sounds sensible, yes.

The retention is already read from passed in by forward_store via StoreRawProfile::retention, and used in handle_raw_profile.

The fact that profiles don't have a custom retention config is out of scope of this PR IMO.

TL;DR: A hard-coded constant is fine for the fallback, but I would define it in a custom constant and set it to 90 days to match the default event retention.

Co-Authored-By: Claude <noreply@anthropic.com>

… feat/markushi/perfetto-profiling-support-pipeline-storage-mechanism

loewenheim · 2026-06-03T14:56:47Z

        let trace_attachments = Usecase::new("trace_attachments")
            .with_expiration_policy(ExpirationPolicy::TimeToLive(DEFAULT_ATTACHMENT_RETENTION));
+        let profiles = Usecase::new("profiles")
+            .with_expiration_policy(ExpirationPolicy::TimeToLive(DEFAULT_ATTACHMENT_RETENTION));


That sounds sensible, yes.

jjbayer · 2026-06-03T15:14:38Z

            Self::Envelope => "envelope",
            Self::EventAttachment => "attachment",
            Self::TraceAttachment => "attachment_v2",
+            Self::RawProfile => "profiles_raw",


Other variants use singular:

Suggested change

Self::RawProfile => "profiles_raw",

Self::RawProfile => "raw_profile",

jjbayer · 2026-06-03T15:29:30Z

+                if let Some(unsent) = s.try_send_to_objectstore(msg) {
+                    s.send_to_store(unsent.map(|profile, _| profile.store_message));
+                }


We only need this branch if we want to support perfetto profiles in self-hosted, which has no objectstore yet by default. In that case, we also need the kafka consumer to handler the raw_profile_object_store_key correctly.

So the simpler approach would be to never send raw profiles via kafka, and accept that perfetto is not enabled in self-hosted.

As relay takes care of parsing the perfetto profile into a sample v2 format I think it's still worth sending the profile chunk kafka message, no matter if object store is available or not. If it's not available, the kafka message would still be sent, just without a raw_profile_object_store_key set.

So the simpler approach would be to never send raw profiles via kafka

Maybe I'm misunderstanding something here, but that's the intention of this PR. Instead of packing the raw profile into the kafka message, we reference using raw_profile_object_store_key instead, which is only set when objectstore is available and successfully stored the raw profile.

jjbayer · 2026-06-03T15:33:44Z

-                raw_profile: chunk.raw_profile,
-            }));
+            if chunk.raw_profile.is_some() {
+                let msg = chunk.map(|chunk, _| {


It should be possible to move the if into the map and then use if let Some(raw_profile) = chunk.raw_profile.

jjbayer · 2026-06-03T15:43:58Z

+    /// Objectstore key where the raw profile blob is stored.
+    pub raw_profile_object_store_key: Option<String>,
+    /// Content type of the raw profile (e.g. Perfetto trace).
+    pub raw_profile_content_type: Option<ContentType>,


Since a StoreProfileChunk can only have either a byte payload or an objectstore key, I would encode this into the type system as something like

pub payload: StoreProfileChunkPayload

where

enum StoreProfileChunkPayload { Bytes(Bytes), ObjectstoreKey(String) }

jjbayer · 2026-06-03T15:46:50Z

-                raw_profile: message.raw_profile.as_ref().map(|r| r.payload.clone()),
-                raw_profile_content_type: message.raw_profile.map(|r| r.content_type),
+                raw_profile_object_store_key: message.raw_profile_object_store_key,
+                raw_profile_content_type: message.raw_profile_content_type,


Why do we need to provide the content type in the kafka message in the first place? Isn't it always ContentType::PerfettoTrace?

Yes, as of now it can only be perfetto. I added it nevertheless to make it easier to add other profiles formats in future. Also it will be helpful downstream to know what format raw_profile_object_store_key refers to, otherwise we'd need to make at least a HEAD request towards objectstore every time we want to process / surface the data (e.g. a "Download Perfetto Trace" button on the frontend).

An alternative would be to simply rename raw_profile_object_store_key to perfetto_profile_object_store_key.

No strong opinions on my end, happy for any directions here as I'm still new to the codebase.

jjbayer · 2026-06-03T15:50:45Z

+    /// Content type of the raw profile.
+    pub content_type: ContentType,
+    /// The profile chunk message to forward to Kafka after upload.
+    pub store_message: StoreProfileChunk,


This looks unnecessarily nested. The objectstore service should be able to construct the StoreProfileChunk message from scratch when it sends the data to the store service.

jjbayer · 2026-06-03T16:00:59Z

        let trace_attachments = Usecase::new("trace_attachments")
            .with_expiration_policy(ExpirationPolicy::TimeToLive(DEFAULT_ATTACHMENT_RETENTION));
+        let profiles = Usecase::new("profiles")
+            .with_expiration_policy(ExpirationPolicy::TimeToLive(DEFAULT_ATTACHMENT_RETENTION));


The retention is already read from passed in by forward_store via StoreRawProfile::retention, and used in handle_raw_profile.

The fact that profiles don't have a custom retention config is out of scope of this PR IMO.

TL;DR: A hard-coded constant is fine for the fallback, but I would define it in a custom constant and set it to 90 days to match the default event retention.

Co-authored-by: Sebastian Zivota <loewenheim@users.noreply.github.com>

…tto-profiling-support-pipeline-storage-mechanism' into feat/markushi/perfetto-profiling-support-pipeline-storage-mechanism

cursor

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit dc7c95b. Configure here.}

cursor · 2026-06-08T07:29:48Z

+    object_store_key = profile["raw_profile_object_store_key"]
+    assert object_store_key, "expected raw_profile_object_store_key in Kafka message"
+
+    stored = objectstore("profiles_raw", project_id).get(object_store_key)


Wrong objectstore usecase in test

Medium Severity

The new integration test fetches the uploaded Perfetto blob with objectstore usecase profiles_raw, but Relay uploads via profile_raw. Objectstore keys are scoped by usecase (same pattern as attachments), so the GET likely misses the object the test claims to verify.

^{Reviewed by Cursor Bugbot for commit dc7c95b. Configure here.}

cursor · 2026-06-08T07:29:48Z

+                            payload: chunk.payload,
+                            quantities: chunk.quantities,
+                            raw_profile_object_store_key: None,
+                            raw_profile_content_type: None,


Content type omitted after failures

Medium Severity

When a chunk has a raw profile, StoreProfileChunk is built with raw_profile_content_type set to None, and objectstore only fills it after a successful upload. Previously, Kafka always included the content type whenever raw_profile was present. Failed uploads, session errors, objectstore fallback, and load-shed paths now emit messages without that field despite an ingested Perfetto trace.

Additional Locations (1)

relay-server/src/services/objectstore.rs#L627-L637

^{Reviewed by Cursor Bugbot for commit dc7c95b. Configure here.}

ref: Route raw profiles through objectstore service

de9d00c

Extract StoreRawProfile into the objectstore service and replace inline raw profile blob with objectstore key reference in the Kafka message. Co-Authored-By: Claude <noreply@anthropic.com>

markushi changed the base branch from master to feat/markushi/perfetto-profiling-support-pipeline May 28, 2026 07:51

ref(objectstore): Rename raw profile usecase to profiles_raw

3a83bc5

Align the objectstore usecase identifier with the naming convention so it can be easily distinguished from a future profiles_v2 usecase. Co-Authored-By: Claude <noreply@anthropic.com>

markushi commented May 28, 2026

View reviewed changes

Comment thread relay-server/src/services/objectstore.rs

markushi marked this pull request as ready for review May 28, 2026 08:07

markushi requested a review from a team as a code owner May 28, 2026 08:07

sentry Bot reviewed May 28, 2026

View reviewed changes

Comment thread relay-server/src/processing/profile_chunks/mod.rs

cursor Bot reviewed May 28, 2026

View reviewed changes

Comment thread relay-server/src/processing/profile_chunks/mod.rs Outdated

Comment thread relay-server/src/services/objectstore.rs Outdated

markushi added the skip-changelog label May 28, 2026

sentry Bot reviewed May 28, 2026

View reviewed changes

Comment thread relay-server/src/services/objectstore.rs Outdated

lcian reviewed May 28, 2026

View reviewed changes

Comment thread relay-server/src/services/objectstore.rs Outdated

lcian reviewed May 28, 2026

View reviewed changes

Comment thread relay-server/src/services/objectstore.rs

feat(profiling): Store content type on raw profile objectstore upload

907aa48

Co-Authored-By: Claude <noreply@anthropic.com>

cursor Bot reviewed May 29, 2026

View reviewed changes

Comment thread relay-server/src/services/objectstore.rs Outdated

Merge branch 'feat/markushi/perfetto-profiling-support-pipeline' into…

ffa1f8e

… feat/markushi/perfetto-profiling-support-pipeline-storage-mechanism

This was referenced Jun 3, 2026

feat(profiling): Add Perfetto trace format support #5659

Open

feat(profiling): Add pipeline workflow to perfetto profiling #5932

Open

loewenheim requested changes Jun 3, 2026

View reviewed changes

jjbayer requested changes Jun 3, 2026

View reviewed changes

markushi and others added 3 commits June 8, 2026 08:19

Update relay-server/src/services/objectstore.rs

22f3895

Co-authored-by: Sebastian Zivota <loewenheim@users.noreply.github.com>

Merge remote-tracking branch 'refs/remotes/origin/feat/markushi/perfe…

08a863b

…tto-profiling-support-pipeline-storage-mechanism' into feat/markushi/perfetto-profiling-support-pipeline-storage-mechanism

Add custom retention constant for raw profiles

dc7c95b

cursor Bot reviewed Jun 8, 2026

View reviewed changes

	Self::RawProfile => "profiles_raw",
	Self::RawProfile => "raw_profile",

Conversation

markushi commented May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lcian May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot Jun 8, 2026

Choose a reason for hiding this comment

Wrong objectstore usecase in test

Uh oh!

cursor Bot Jun 8, 2026

Choose a reason for hiding this comment

Content type omitted after failures

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

markushi commented May 28, 2026 •

edited

Loading

lcian May 28, 2026 •

edited

Loading