Skip to content

fix(asyncpg): Use distinct span ops for cursor iteration and fetch to prevent N+1 false positives#6609

Merged
ericapisani merged 5 commits into
masterfrom
py-2534-identify-db-ops-related-to-cursor-iteration
Jun 22, 2026
Merged

fix(asyncpg): Use distinct span ops for cursor iteration and fetch to prevent N+1 false positives#6609
ericapisani merged 5 commits into
masterfrom
py-2534-identify-db-ops-related-to-cursor-iteration

Conversation

@ericapisani

Copy link
Copy Markdown
Member

When asyncpg iterates a server-side cursor (via CursorIterator.anext) or fetches rows in batches (via Cursor.fetch), each batch triggers BaseCursor._bind_exec or BaseCursor._exec, producing a burst of identical spans. Sentry's N+1 detector incorrectly flags these as N+1 queries.

Set sentry.op to db.cursor.iter for CursorIterator-driven spans and db.cursor.fetch for Cursor.fetch-driven spans so the backend can exclude them from N+1 detection.

Adds span_op_override_value parameter to record_sql_queries to propagate the override without changing existing callers.

This will require changes on the backend in order to not trigger n+1 alerts on these ops.

Fixes #6576
Fixes PY-2534

… prevent N+1 false positives

When asyncpg iterates a server-side cursor (via CursorIterator.__anext__) or
fetches rows in batches (via Cursor.fetch), each batch triggers BaseCursor._bind_exec
or BaseCursor._exec, producing a burst of identical spans. Sentry's N+1 detector
incorrectly flags these as N+1 queries.

Set `sentry.op` to `db.cursor.iter` for CursorIterator-driven spans and `db.cursor.fetch`
for Cursor.fetch-driven spans so the backend can exclude them from N+1 detection.

Adds span_op_override_value parameter to record_sql_queries to propagate the
override without changing existing callers.

This will require changes on the backend in order to not trigger n+1 alerts on these
ops.

Fixes #6576
Fixes PY-2534
@linear-code

linear-code Bot commented Jun 19, 2026

Copy link
Copy Markdown

PY-2534

@github-actions

github-actions Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Codecov Results 📊

91382 passed | ⏭️ 6136 skipped | Total: 97518 | Pass Rate: 93.71% | Execution Time: 331m 35s

📊 Comparison with Base Branch

Metric Change
Total Tests
Passed Tests
Failed Tests
Skipped Tests

✨ No test changes detected

All tests are passing successfully.

✅ Patch coverage is 88.89%. Project has 2412 uncovered lines.
✅ Project coverage is 89.85%. Comparing base (base) to head (head).

Files with missing lines (1)
File Patch % Lines
sentry_sdk/integrations/asyncpg.py 85.71% ⚠️ 1 Missing and 1 partials
Coverage diff
@@            Coverage Diff             @@
##          main       #PR       +/-##
==========================================
+ Coverage    89.85%    89.85%        —%
==========================================
  Files          192       192         —
  Lines        23745     23752        +7
  Branches      8198      8202        +4
==========================================
+ Hits         21334     21340        +6
- Misses        2411      2412        +1
- Partials      1344      1345        +1

Generated by Codecov Action

ericapisani added a commit to getsentry/sentry that referenced this pull request Jun 19, 2026
@ericapisani ericapisani marked this pull request as ready for review June 19, 2026 17:45
@ericapisani ericapisani requested a review from a team as a code owner June 19, 2026 17:45
wahajahmed010

This comment was marked as spam.

Comment thread sentry_sdk/integrations/asyncpg.py Outdated
if sentry_sdk.get_client().get_integration(AsyncPGIntegration) is None:
return await f(*args, **kwargs)

if _asyncpg_cursor_iterator_is_invoked.get():

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

preferable to set this as state on the cursor imo

cursor._iterator_is_invoked = True

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The various Cursor classes in the asyncpg integration prevent us from doing this through their use of __slots__ (as their presence removes the .__dict__ for setting dynamic values on each instance created).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok in that case, we need to be sure that each context var maps cleanly to only one running cursor instance at a time.

@alexander-alderman-webb alexander-alderman-webb Jun 22, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did a little snooping around in the library, both Cursor and CursorIterator inherit from BaseCursor.

So you should be able to tell which case you're in by checking isinstance(self, ...) in BaseCursor._exec, etc ...

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a great idea 🚀 Give me a few to make that change

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in cd25110

Comment on lines +171 to +173
"sentry.op": span_op_override_value
if span_op_override_value
else OP.DB,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The check for span_op_override_value is inconsistent; one path uses a truthiness check while another uses is not None, causing different behavior for empty strings.
Severity: LOW

Suggested Fix

Standardize the conditional check for span_op_override_value in both the streaming and non-streaming code paths. Use the more explicit is not None check in both locations to ensure consistent behavior for all possible string values.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: sentry_sdk/tracing_utils.py#L171-L173

Potential issue: The function `record_sql_queries` checks the `span_op_override_value`
parameter inconsistently across two code paths. The streaming path uses a truthiness
check (`if span_op_override_value`), which would cause an empty string to be evaluated
as false, falling back to the default `OP.DB`. The non-streaming path uses an explicit
`is not None` check, which would correctly use an empty string as the operation name.
While no current callers pass an empty string, this inconsistency creates a latent bug
that could be triggered by future code changes, leading to incorrect span operation
names.

Also affects:

  • sentry_sdk/tracing_utils.py:179~179

Did we get this right? 👍 / 👎 to inform future reviews.

@ericapisani ericapisani merged commit dafa2bd into master Jun 22, 2026
144 checks passed
@ericapisani ericapisani deleted the py-2534-identify-db-ops-related-to-cursor-iteration branch June 22, 2026 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Sentry detects N+1 queries in server side cursor

4 participants