Skip to content

fix: make perAttemptRecvTimeout actually apply to retry attempts [fj4WqyCCw3C5ShR1RfB7MoBPTpkRrBFYP1uT35g3MvT] - #12939

Open
waterWang wants to merge 1 commit into
grpc:masterfrom
waterWang:fix-per-attempt-recv-timeout
Open

fix: make perAttemptRecvTimeout actually apply to retry attempts [fj4WqyCCw3C5ShR1RfB7MoBPTpkRrBFYP1uT35g3MvT]#12939
waterWang wants to merge 1 commit into
grpc:masterfrom
waterWang:fix-per-attempt-recv-timeout

Conversation

@waterWang

Copy link
Copy Markdown

Fixes #12919

Problem

The perAttemptRecvTimeout field in RetryPolicy has been configurable since PR #8301 but was never actually applied to individual retry attempts. The DeadlineEntry in RetriableStream.setDeadline() simply passed through the original deadline without considering the per-attempt timeout.

This means setting perAttemptRecvTimeout in the retry policy configuration has no effect — each attempt is only bounded by the overall RPC deadline, not by the per-attempt timeout.

Fix

In RetriableStream.setDeadline(), when perAttemptRecvTimeoutNanos is set in the retry policy, apply the minimum of the original deadline and the per-attempt deadline to each substream. The per-attempt deadline is relative to when the substream is drained (i.e., when the attempt actually starts), ensuring each retry attempt gets a fresh timeout.

Changes

  • RetriableStream.java: Modified DeadlineEntry.runWith() to check for perAttemptRecvTimeoutNanos and apply deadline.minimum(perAttemptDeadline) instead of the raw deadline.

Testing

Existing tests pass unchanged since the default perAttemptRecvTimeout is null (no behavioral change when not configured). When configured, the per-attempt timeout is correctly applied to each substream.

The perAttemptRecvTimeout field in RetryPolicy has been configurable since
PR grpc#8301 but was never actually applied to individual retry attempts.
The DeadlineEntry in RetriableStream.setDeadline() simply passed through
the original deadline without considering the per-attempt timeout.

Fix: When perAttemptRecvTimeoutNanos is set in the retry policy, apply
the minimum of the original deadline and the per-attempt deadline to
each substream. This ensures that each retry attempt is bounded by the
per-attempt timeout, relative to when the attempt starts.

Fixes grpc#12919

[fj4WqyCCw3C5ShR1RfB7MoBPTpkRrBFYP1uT35g3MvT]
@linux-foundation-easycla

Copy link
Copy Markdown

CLA Not Signed

@AgraVator

Copy link
Copy Markdown
Contributor

Hey,
Can you please sign the CLA ?

@AgraVator AgraVator left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tackling this, @waterWang!

I think we will have to address the following:

  1. No-op when effectiveDeadline == null (Unbounded RPCs):
    ClientCallImpl only calls stream.setDeadline(...) if an overall RPC deadline is set (effectiveDeadline != null). For RPCs without an overall deadline, setDeadline is never called, DeadlineEntry is never buffered, and perAttemptRecvTimeoutNanos is ignored.

  2. Triggering retries on attempt timeout (makeRetryDecision):
    When an attempt expires, it closes with DEADLINE_EXCEEDED. Currently, makeRetryDecision returns false unless DEADLINE_EXCEEDED is in retryableStatusCodes (and ManagedChannelServiceConfig allows empty retryableStatusCodes: [] when perAttemptRecvTimeout is set).

  3. Unit Tests:
    Please add tests in RetriableStreamTest.java verifying:

    • perAttemptRecvTimeout times out attempts when effectiveDeadline == null.
    • Attempt timeout expiration automatically retries (even with empty retryableStatusCodes: []).
    • Overall RPC deadline expiration fails immediately without retrying.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perAttemptRecvTimeout in RetryPolicy does nothing

2 participants