ref(profiling): Remove base64 encoding from profile task payload#115069
ref(profiling): Remove base64 encoding from profile task payload#115069
Conversation
Pass raw bytes directly to process_profile_task instead of base64 encoding. The task now accepts both bytes (new) and base64 string (legacy) for backwards compatibility during deployment. STREAM-882
github-actions
Bot
added
the
Scope: Backend
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
| if sampled or options.get("profiling.profile_metrics.unsampled_profiles.enabled"): | ||
| b64encoded = b64encode(message.payload.value).decode("utf-8") | ||
| process_profile_task.delay(payload=b64encoded, sampled=sampled) | ||
| process_profile_task.delay(payload=message.payload.value, sampled=sampled) |
There was a problem hiding this comment.
I don't think we have yet rolled out any scenario where we send raw bytes payload to a task, is that correct ?
If I am right this should be rolled out behind a feature flag.
If something goes wrong the only option is to rollback the deployment or revert the commit if we start seeing issues later, which means outage.
I know that is friction but that is all we have to reduce risk when shipping the monolith to prod.
Add profiling.process.raw_bytes_payload.enabled option to control whether the profile consumer sends raw bytes or base64-encoded payload to taskbroker. Defaults to off (base64) for safe rollout. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 9465fda. Configure here.
| processing_strategy.terminate() | ||
|
|
||
| process_profile_task.assert_called_with( | ||
| payload=b64encode(payload).decode("utf-8"), |
There was a problem hiding this comment.
Test expects raw bytes but feature flag is disabled
High Severity
The test assertion was updated to expect raw bytes (payload=payload), but the profiling.process.raw_bytes_payload.enabled option defaults to False and is never overridden in this test. With the flag disabled, factory.py takes the else branch and base64-encodes the payload. The test will fail because the actual call uses a base64 string while the assertion expects raw bytes. The test needs @override_options({"profiling.process.raw_bytes_payload.enabled": True}) to match the expected behavior, or the assertion needs to keep using the base64-encoded form.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 9465fda. Configure here.
- Fix typing issue by avoiding reassignment with different types - Update test_basic_profile_to_task to expect base64 (default behavior) - Add profiling.process.raw_bytes_payload.enabled to backpressure test overrides Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2 participants
Summary
Pass raw bytes directly to
process_profile_taskinstead of base64 encoding. The task now accepts bothbytes(new) and base64str(legacy) for backwards compatibility during deployment.This is possible because taskbroker supports bytes in task parameters via msgpack serialization (getsentry/taskbroker#602).
Split out from #115065 as an independent change.
We should be able to ship this now, because
msgpackhas been turned on in prod for many days now, and self-hosted has been released since then here: https://github.com/getsentry/taskbroker/releases/tag/26.4.2Test plan
Refs STREAM-882