Skip to content

fix(azure): surface the read error that decides purchase retry-vs-abort - #1767

Merged
cristim merged 3 commits into
mainfrom
fix/1766-purchase-retry-classification
Aug 8, 2026
Merged

cristim merged 3 commits into
mainfrom
fix/1766-purchase-retry-classification

Conversation

@cristim

@cristim cristim commented Aug 8, 2026 •

Copy link
Copy Markdown
Member

Closes #1766

The defect

doPurchase discarded the io.ReadAll error, and its error string is the only input to the retry decision:

body, _ := io.ReadAll(resp.Body)
...
return fmt.Errorf("reservation purchase failed with status %d: %s", resp.StatusCode, string(body))
func IsSessionTimeout(err error) bool {
    return strings.Contains(err.Error(), sessionTimeoutFragment)   // "Session timed out"
}
if IsSessionTimeout(purchaseErr) && attempt < purchaseMaxAttempts {
    continue   // re-run calculatePrice and retry
}
return "", purchaseErr

So a 400 whose body was truncated before the fragment produced a bare status 400: <partial>, IsSessionTimeout returned false, and the purchase was abandoned on its first attempt — the exact condition purchaseMaxAttempts exists to survive. Nothing in the message indicated the response had not been fully read.

Direction: fails closed. It abandons a purchase, it does not double-buy — hence severity/medium. But a designed-retryable failure became permanent silently, and the diagnostic that would have explained it is the thing that went missing.

Found while triaging LeanerCloud/cloud-commitments-platform#174's 49 production findings. It is the only one of them where a discarded error feeds control flow rather than a message — full triage here.

The fix

Correction (head c4d2d85b1). The first revision of this PR surfaced the read error in the message but did not change the retry decision — IsSessionTimeout still ran strings.Contains over a string built from a truncated body, so the purchase was still abandoned on its first attempt. The message improved; the behaviour did not, and Closes LeanerCloud/cloud-commitments-cli#1766 overstated the diff. Measured on the real loop with a counting client (fresh response per attempt, since the truncating body is stateful):

scenario purchase attempts, first revision
readable session-timeout 400 (control) 3
truncated session-timeout 400 (defect) 1

The control moves, so the probe measures the loop rather than a constant.

Retryability is now carried out of band. doPurchase wraps a new errRetryabilityUnknown sentinel when a non-2xx body fails to read, and DoPurchaseTwoStep treats errors.Is(err, errRetryabilityUnknown) as retryable alongside IsSessionTimeout. The retry decision no longer depends on text parsed from a body that never fully arrived, so Closes LeanerCloud/cloud-commitments-cli#1766 now matches the diff.

Scoped to 400 — matching IsSessionTimeout's own status scope and the only scenario #1766 reports.

That scoping is not cosmetic. The safety review established that the idempotency guard does not cover in-loop retries: DoIdempotentPurchaseTwoStep calls FindReservationOrderByIdempotencyToken exactly once, before DoPurchaseTwoStep is entered, and there is no recheck between attempts — each retry mints a fresh reservationOrderId via doCalculatePrice. Verified here: zero lookups occur inside the loop. So the guard covers a re-drive across invocations and nothing within one, and the bound is purchaseMaxAttempts, not the guard.

That makes "a non-2xx means the order did not commit" the only thing between a truncated read and a double purchase — and that premise is unverified for this API on 409 and 5xx. A truncated response is also mechanistically different from a cleanly read one: truncation is the transport-failure class associated with genuine commit ambiguity, since the server may have finished processing and begun writing when the connection died. A cleanly read 500 is a deliberate synchronous statement. An earlier revision of this PR treated the two as equivalent, and its safety comment asserted the non-commit premise outright; both are corrected.

Narrowing costs no coverage — #1766's defect is a 400 with a truncated body, the only case in the issue and in every regression test here — while removing the unverified commit-ambiguous statuses from the retry surface entirely.

The missing in-loop recheck is pre-existing on the IsSessionTimeout path since #677/#1550. This PR widens what reaches it; the 400 scoping is the mitigation. Tracked separately as #1774 and deliberately not addressed here.

Success still depends on the status code alone, so an unreadable body cannot turn a completed purchase into a reported failure — verified by mutation (hoisting the readErr check above the 2xx short-circuit makes the guard fire).

Regression tests, both directions

Driven by a body that yields 12 bytes then fails, modeling a response cut short mid-transfer.

TestDoPurchaseTwoStep_TruncatedSessionTimeoutStillRetries is the regression test for the retry decision, and it is deliberately an attempt-count assertion rather than a message assertion. It fails against the message-only first revision with expected: 3, actual: 1.

This matters beyond this PR. The first revision's test asserted on the error text and passed while the purchase was still abandoned — the assertion was tautological once the message changed, while being labelled as pinning retryability, a property that did not hold. That is the same defect class as #1595 and #1740: an assertion reading as a guarantee it does not provide, written during the very work that was removing them. Only counting attempts distinguishes a changed message from changed behaviour.

The test carries the readable control (which must reach purchaseMaxAttempts) so it measures the loop rather than a constant; if the control ever stops retrying, the test stops being evidence.

TestDoPurchase_UnreadableBodyDoesNotSilentlyLookPermanent now asserts errors.Is(err, errRetryabilityUnknown) — the thing the loop actually consults — rather than substrings of the message.

Two further tests pin the 400 scoping so it cannot be silently widened again, since nothing else in the code would notice:

  • TestDoPurchase_TruncatedNon400IsNotRetryable — 409, 500, 502 and 504 with truncated bodies must not carry the sentinel.
  • TestDoPurchaseTwoStep_TruncatedNon400DoesNotRetry — a truncated 500 must be attempted exactly once.

Both fail when the condition is widened back to any non-2xx, with the message naming the double-buy risk and #1774.

TestDoPurchase_UnreadableBodyOnSuccessStillSucceeds — guards the opposite direction: a truncated body on a 200 must still succeed.

Siblings checked, not assumed

The two other io.ReadAll discards in this file are not affected, verified individually:

  • :287 doCalculatePrice — truncation fails json.Unmarshal, and an empty ReservationOrderID is separately guarded.
  • :390 fetchReservationOrdersPage — backs the idempotency-token lookup. Truncation fails json.Unmarshal, and the caller DoIdempotentPurchaseTwoStep explicitly refuses to purchase on a lookup error rather than falling through, so there is no double-buy path. That code is well guarded.

Gates

providers/azure: gofmt clean, go build ./..., go vet ./..., go test -race -count=1 ./... 851 passed / 14 packages. Root module: build, vet, gocyclo -over 10 -ignore "_test\.go" . exit 0 empty, golangci-lint v2.10.1 exit 0 with a genuine 0 issues. line (both asserted, since a missing --out-format or a concurrent invocation exits 3 with zero findings and reads like a clean run).

Rebased onto 6d5275c85. golangci-lint on providers/azure compared as a set diff against origin/main, not by totals:

base (main): 592    head: 589
only in HEAD (new):    (none)
only in BASE (fixed):  purchase.go:: Error return value of `io.ReadAll` is not checked (errcheck)
                       purchase_test.go:: `behaviour` is a misspelling of `behavior` (misspell)  x2

Drive-by, disclosed: a global spelling fix in this test file also corrected two pre-existing behaviour misspellings that main already carried. Two words in a file this PR already rewrites; flagged rather than reverted, since reverting means deliberately restoring a misspelling.

The set-diff method earned itself twice here. An intermediate revision removed one finding and added two misspell in new comments (net 592→593); a later one added one behaviour that exactly offset the removed errcheck, leaving the total at 592 — unchanged, and wrong. Totals alone would have passed both times. I also had to re-derive the baseline twice: git checkout -- <path> restores from the index, i.e. my own commit, not main. The comparison above uses git checkout origin/main -- providers/azure/.

providers/azure carries 589 pre-existing findings tracked in LeanerCloud/cloud-commitments-platform#174 and LeanerCloud/cloud-commitments-go#63; this PR neither adds to them nor masks them.

Known-red inherited from main: TestGrantCeiling_ConstraintContainment in internal/auth fails on main today (#1737 and #1758 colliding on execute:ri-exchange). This branch changes only providers/azure/services/internal/reservations/ — confirmed by git diff origin/main --name-only — so that failure is inherited, not caused, and a separate fix is in flight.

Summary by CodeRabbit

  • Bug Fixes
    • Improved handling of incomplete Azure reservation purchase responses.
    • Automatically retries purchases when response data is truncated or a session timeout occurs.
    • Preserves successful status handling even when response bodies cannot be fully read.
    • Avoids unnecessary retries for other non-retryable response errors.
    • Ensures retry attempts stop at the configured limit while returning the appropriate error.

@cristim cristim added triaged Item has been triaged priority/p2 Backlog-worthy severity/medium Moderate harm urgency/this-sprint Within the current sprint impact/few Limited audience effort/xs Trivial / one-liner type/bug Defect labels Aug 8, 2026
@coderabbitai

coderabbitai Bot commented Aug 8, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The Azure reservation purchase path now preserves response-body read errors, classifies retryability as unknown for truncated 400 responses, and retries unreadable responses. Tests cover error propagation, retry limits, status handling, and non-retryable statuses.

Changes

Azure purchase retryability

Layer / File(s) Summary
Purchase error classification and retry flow
providers/azure/services/internal/reservations/purchase.go
doPurchase now returns response-body read failures with errRetryabilityUnknown for non-success 400 responses. Successful status codes remain successful. DoPurchaseTwoStep retries unreadable responses.
Purchase retry regression coverage
providers/azure/services/internal/reservations/purchase_test.go
Tests cover truncated bodies, retry-to-cap behavior, successful status handling, status-specific retryability, single-attempt handling, and corrected comment spellings.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related issues

Possibly related PRs

  • LeanerCloud/CUDly#680: Modifies the same Azure reservation purchase implementation and tests, including retry and truncated-response handling.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes satisfy #1766 by preserving read errors, propagating unknown retryability, retrying affected purchases, and adding regression tests.
Out of Scope Changes check ✅ Passed The changes remain within the purchase error-handling fix and its regression coverage; comment spelling corrections do not add unrelated code scope.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main fix: surfacing response read errors to determine whether purchase processing retries or aborts.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/1766-purchase-retry-classification

Comment @coderabbitai help to get the list of available commands.

@cristim

cristim commented Aug 8, 2026

Copy link
Copy Markdown
Member Author

Independent review at 182f242a9 — 1 blocking framing defect, 1 minor

Fresh worktree at the PR head plus a second at origin/main, every claim re-derived rather than taken from the body.

The two "opposite direction" checks hold up well. The central claim does not.


F1 (blocking as framed) — the fix does not change the retry decision. Only the message.

The PR body says:

Retryability is no longer classified from a body that was never fully received.

It still is. The new error text does not contain Session timed out, so IsSessionTimeout still returns false, and DoPurchaseTwoStep still takes return "", purchaseErr on the first attempt. The purchase is abandoned in exactly the scenario #1766 describes, exactly as before.

I drove the real retry loop with a counting HTTPClient (the PR's own truncatingBody is stateful and cannot be reused across attempts, so the probe mints a fresh one per response) — identical probe, both sides:

scenario origin/main (pre-fix) 182f242a9 (post-fix)
400, body cut short before the fragment purchaseAttempts=1, IsSessionTimeout=false purchaseAttempts=1, IsSessionTimeout=false
400, full body, fragment intact (control) purchaseAttempts=3, IsSessionTimeout=true purchaseAttempts=3, IsSessionTimeout=true

The control moves, so the probe is measuring the retry loop and not a constant. The defect row does not move.

Pre-fix error:

reservation purchase failed with status 400: {"error":{"c

Post-fix error:

reservation purchase failed with status 400 and its response body could not be read,
so retryability is unknown (partial body: "{\"error\":{\"c"): unexpected EOF reading response body

Same decision, richer text.

This is a genuine and worthwhile improvement — the issue has two halves ("the retry is abandoned" and "nothing in the message indicated the response had not been fully read"), and this closes the second one cleanly. But it is an observability fix, not a retry-classification fix, and the body and the code comment both describe it as the latter. The comment at purchase.go:484-490 ends "Surface the read failure rather than classifying retryability from a body that was never fully received" — the surfacing happens, the classification still does too.

Concretely: Closes LeanerCloud/cloud-commitments-cli#1766 overstates it. #1766's core sentence — "the purchase is abandoned — the exact condition the retry exists for" — is still true at this head.

Making the behaviour match the claim needs the unknown-retryability state carried out of band rather than in the string: a sentinel or typed error from doPurchase, and a branch in DoPurchaseTwoStep that treats errors.Is(err, errRetryabilityUnknown) as retryable alongside IsSessionTimeout. Worth noting that is safe here — a non-2xx means the purchase did not commit, and DoIdempotentPurchaseTwoStep already refuses to fall through on a lookup error — but it is a design call, not a one-liner, and it deserves its own change.

Either extend the PR to change the classification, or keep the scope and adjust the framing (retitle to the diagnostic it delivers, drop Closes to Refs, and leave #1766 open for the behavioural half). I would not close #1766 on this diff.


F2 (minor) — the "essential property" assertion is tautological post-fix

purchase_test.go:863 and :872:

assert.ErrorContains(t, err, "could not be read", ...)                    // :863
...
assert.True(t,
    IsSessionTimeout(err) || strings.Contains(err.Error(), "could not be read"),
    "an unreadable body must not silently produce a non-retryable classification")   // :872

Once :863 passes, the second disjunct of :872 is necessarily true. :872 cannot fail independently of :863 — it is asserting the same substring twice, the second time behind an || that can never be reached.

It is also the one assertion labelled as pinning retryability, and per F1 that property does not hold. A reader scanning the test comes away believing the retry classification is guarded. Given this repo just spent #1740/#1750 on assertions that read as guarantees they do not provide, this one is worth removing or replacing — the honest version drives DoPurchaseTwoStep and asserts the attempt count, which is exactly the assertion that would have caught F1.

Also minor: the body heads the section "Regression tests, both directions", but TestDoPurchase_UnreadableBodyOnSuccessStillSucceeds passes against pre-fix code (verified). It is a guard, not a regression test — correctly so, since that direction was never broken. Worth saying that rather than implying both fail pre-fix.


Verified clean

The success path is unchanged, and its guard genuinely bites. resp.StatusCode is checked before readErr, so an unreadable body cannot fail a completed purchase. Confirmed by mutation — hoisting the readErr check above the 2xx short-circuit:

--- FAIL: TestDoPurchase_UnreadableBodyOnSuccessStillSucceeds
    MUTANT: read failed, status 200: unexpected EOF reading response body

This is the dangerous direction and it is properly nailed down.

The pre-fix failure re-derives verbatim. Checked out the PR's purchase_test.go onto origin/main production code:

purchase_test.go:863: Error "reservation purchase failed with status 400: {\"error\":{\"c" does not contain "could not be read"
purchase_test.go:865: Error "reservation purchase failed with status 400: {\"error\":{\"c" does not contain "unexpected EOF reading response body"
purchase_test.go:872: Should be true
--- FAIL: TestDoPurchase_UnreadableBodyDoesNotSilentlyLookPermanent
--- PASS: TestDoPurchase_UnreadableBodyOnSuccessStillSucceeds

Quoted text matches the body exactly.

The sibling assessment holds, and I widened it past the file. Swept every non-test discarded io.ReadAll error in providers/azure (5 sites; ERE with [[:space:]], sanity-checked against a line known to match):

site feeds verdict
purchase.go:287 doCalculatePrice message; truncation fails json.Unmarshal, empty ReservationOrderID separately guarded; caller returns immediately without classifying safe
purchase.go:390 fetchReservationOrdersPage message; truncation fails json.Unmarshal; DoIdempotentPurchaseTwoStep:448 refuses to purchase on a lookup error safe, fails closed
internal/pricing/retail_prices.go:110 message only, inside the non-200 branch safe
compute/client.go:406 triggerCapacityProviderRegistration message only safe
compute/client.go:372 fetchCapacityProviderState its result does feed control flow (state.RegistrationState == "Registered"), but a truncated 2xx fails json.Unmarshal → early return before the branch, and ensureCapacityProviderRegistered only logs a warning either way safe

So :475 really was the only one where a discarded error reached a decision. That distinguishing property survives the change.

The golangci set diff reproduces exactly. 592 on main, 591 on head; the sole real delta is purchase.go:475:8: Error return value of io.ReadAll is not checked (errcheck), removed. A raw comm also surfaces 41 purchase_test.go entries on each side — every one is a +1 line shift from the added "strings" import, each NEW matching a REMOVED one line lower. Zero genuinely new findings; the new test code adds none of its own.


Gates at 182f242a9

gate result
gofmt -l . clean (0 files)
providers/azure: go build ./... + go vet ./... + go test -race -count=1 ./... exit 0 — 12 ok + 2 no-test-files = 14 packages, 0 FAIL
gocyclo -over 10 -ignore "_test\.go" (root and providers/azure) exit 0, empty
golangci-lint run @ CI-pinned v2.10.1, providers/azure exit 1 / 591 issues both sides bar the one removed — pre-existing debt per LeanerCloud/cloud-commitments-platform#174/#1759, neither added to nor masked

Everything green. F1 is not a test or build failure — it is that the diff does less than it says, which no gate can see.

Verdict

Mergeable on its merits, not on its framing. The code is correct, narrowly scoped, and the risky direction is properly guarded. Before merge I would fix F2 and settle F1 one way or the other: either carry the unknown-retryability state out of band so the retry loop actually changes, or keep this scope and stop it closing #1766.

@cristim
cristim force-pushed the fix/1766-purchase-retry-classification branch from 182f242 to c4d2d85 Compare August 8, 2026 07:31

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
providers/azure/services/internal/reservations/purchase.go (1)

494-509: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Split these files to meet the 500-line limit.

Both final files exceed 500 lines. Keep the extraction within the reservations bounded context.

  • providers/azure/services/internal/reservations/purchase.go#L494-L509: extract a cohesive private purchase-response classification unit.
  • providers/azure/services/internal/reservations/purchase_test.go#L813-L948: move the truncation helpers and unreadable-body regression tests into a focused test file.

As per coding guidelines, “keep files under 500 lines.”

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@providers/azure/services/internal/reservations/purchase.go` around lines 494
- 509, Split the reservations purchase code and tests so every file stays under
500 lines. In providers/azure/services/internal/reservations/purchase.go lines
494-509, extract the cohesive private purchase-response classification logic
into a bounded-context helper while preserving status handling and
unreadable-body retryability behavior. In
providers/azure/services/internal/reservations/purchase_test.go lines 813-948,
move the truncation helpers and unreadable-body regression tests into a focused
reservations test file, updating references as needed without changing coverage.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@providers/azure/services/internal/reservations/purchase_test.go`:
- Around line 836-837: Update the comment near DoPurchaseTwoStep to describe the
retry condition as pre-fix behavior, using past tense and acknowledging that
retries were previously limited to IsSessionTimeout matches; do not alter the
purchase logic.

---

Nitpick comments:
In `@providers/azure/services/internal/reservations/purchase.go`:
- Around line 494-509: Split the reservations purchase code and tests so every
file stays under 500 lines. In
providers/azure/services/internal/reservations/purchase.go lines 494-509,
extract the cohesive private purchase-response classification logic into a
bounded-context helper while preserving status handling and unreadable-body
retryability behavior. In
providers/azure/services/internal/reservations/purchase_test.go lines 813-948,
move the truncation helpers and unreadable-body regression tests into a focused
reservations test file, updating references as needed without changing coverage.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 70d21cf1-9522-49de-8e20-20794b39ab83

📥 Commits

Reviewing files that changed from the base of the PR and between 6d5275c and c4d2d85.

📒 Files selected for processing (2)
  • providers/azure/services/internal/reservations/purchase.go
  • providers/azure/services/internal/reservations/purchase_test.go

Comment thread providers/azure/services/internal/reservations/purchase_test.go
cristim and others added 3 commits August 8, 2026 16:59
doPurchase discarded the io.ReadAll error, and its error string is the only
input to IsSessionTimeout, which is the only thing DoPurchaseTwoStep consults
to decide whether to retry. A 400 whose body was truncated before the
"Session timed out" fragment therefore produced a bare `status 400: <partial>`
error, IsSessionTimeout returned false, and the purchase was abandoned on its
first attempt -- the exact condition purchaseMaxAttempts exists to survive --
with nothing in the message to say the response had not been fully read.

The failure direction is closed: it abandons a purchase rather than
double-buying. But a designed-retryable failure became permanent silently, and
the diagnostic that would explain it was the thing that went missing.

Retryability is no longer classified from a body that was never fully
received. On a non-2xx response with a failed read, the read error is wrapped
and surfaced along with the partial body. Success is still decided by the
status code alone, so an unreadable body cannot turn a completed purchase into
a reported failure.

Two regression tests, both driven by a body that yields 12 bytes and then
fails. The first pins that an unreadable body cannot silently produce a
non-retryable classification; it fails against the pre-fix code, where the
error read `reservation purchase failed with status 400: {"error":{"c` and
carried neither the read failure nor the session-timeout fragment. The second
guards the opposite direction, that a truncated body on a 200 still succeeds.

The two sibling io.ReadAll discards in this file were checked rather than
assumed and are not affected: :287 in doCalculatePrice and :390 in
fetchReservationOrdersPage both feed a json.Unmarshal that fails on
truncation, and :390's caller DoIdempotentPurchaseTwoStep explicitly refuses
to purchase on a lookup error rather than falling through.

golangci-lint on providers/azure, compared as a set diff rather than by
totals: zero new findings, one removed (the errcheck this fixes), 592 -> 591.
The totals alone would have hidden it -- the first attempt at this change
removed one finding and added two misspellings in new comments.

Closes #1766

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The previous commit surfaced the discarded read error in the message but did
not change the retry decision. IsSessionTimeout still did strings.Contains on
a string built from a truncated body, so a session-timeout 400 whose body did
not read completely was still abandoned on its first attempt -- the exact
condition purchaseMaxAttempts exists to survive. The message improved; the
behaviour did not.

Measured on the real loop with a counting client, minting a fresh response per
attempt because the truncating body is stateful:

  readable session-timeout 400   3 purchase attempts   (control)
  truncated session-timeout 400  1 purchase attempt    (defect)

The control moves, so the probe measures the loop rather than a constant.

Retryability is now carried out of band. doPurchase wraps
errRetryabilityUnknown when a non-2xx body fails to read, and
DoPurchaseTwoStep treats errors.Is(err, errRetryabilityUnknown) as retryable
alongside IsSessionTimeout. Retrying is safe here: a non-2xx means the order
did not commit, the loop is bounded by purchaseMaxAttempts, and re-drives
across invocations are guarded by DoIdempotentPurchaseTwoStep, which refuses
to purchase when its idempotency lookup fails.

The regression test is an ATTEMPT-COUNT assertion, not a message assertion.
The previous test asserted on the error text and passed while the purchase was
still abandoned, which is how the gap survived review; only counting attempts
tells a changed message from changed behaviour. It fails against the
message-only fix with expected 3, actual 1, and carries the readable control
so it measures the loop rather than a constant.

The "essential property" assertion it replaces was tautological once the
message changed, while being labelled as pinning retryability -- a property
that did not hold. That is the same defect class as #1595 and #1740: an
assertion reading as a guarantee it does not provide.

Drive-by, disclosed: a global spelling fix in this test file also corrected two
pre-existing `behaviour` misspellings that main already carried. Set diff
against main: zero new findings, three removed.

Closes #1766

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
errRetryabilityUnknown fired on ANY non-2xx whose body failed to read -- 409,
500, 502, 504 alike -- which is far broader than the IsSessionTimeout path it
sits beside, and broader than the defect it was written for.

That breadth is load-bearing because the idempotency guard does not cover
in-loop retries. DoIdempotentPurchaseTwoStep calls
FindReservationOrderByIdempotencyToken exactly once, before DoPurchaseTwoStep
is entered; the loop then runs up to purchaseMaxAttempts with no recheck
between attempts, and each retry mints a fresh reservationOrderId via
doCalculatePrice. Verified here: zero lookups occur inside the loop. So the
guard catches a re-drive ACROSS invocations and nothing within one, which
makes "a non-2xx means the order did not commit" the only thing standing
between a truncated read and a double purchase -- and that premise is
unverified for this API on 409 and 5xx.

A truncated response is also mechanistically different from a cleanly read
one: truncation is the transport-failure class associated with genuine commit
ambiguity, since the server may have finished processing and begun writing
when the connection died. A cleanly read 500 is a deliberate synchronous
statement. The previous comment treated the two as equivalent.

Issue #1766's defect is a 400 "Session timed out" with a truncated body, the
only scenario in the issue, the PR description and both regression tests, so
scoping to 400 closes it with no loss of coverage while removing the
unverified commit-ambiguous statuses from the retry surface. Everything else
keeps its pre-existing behavior.

The sentinel's doc comment no longer asserts that a non-2xx means no commit.
It states the scope, why the wider surface is excluded, and that the bound is
purchaseMaxAttempts rather than the idempotency guard.

Two tests pin the narrowing so it cannot be silently widened again: 409, 500,
502 and 504 with truncated bodies must not carry the sentinel, and a truncated
500 must be attempted exactly once. Both fail when the condition is widened
back to any non-2xx. The attempt-count regression test still fails against the
message-only revision with expected 3, actual 1.

The missing in-loop idempotency recheck is pre-existing on the IsSessionTimeout
path and is tracked as #1774; it is deliberately not addressed here.

Refs #1766

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@cristim
cristim force-pushed the fix/1766-purchase-retry-classification branch from c4d2d85 to c3b629c Compare August 8, 2026 15:45
@cristim

cristim commented Aug 8, 2026

Copy link
Copy Markdown
Member Author

Merging. Closes #1766.

Merging on independent adversarial review. The one CodeRabbit thread is a tense nitpick, answered in writing above rather than resolved, and it is now stale in a second way: the head moved after it was posted.

The original defect: doPurchase did body, _ := io.ReadAll(resp.Body) and IsSessionTimeout does strings.Contains on the resulting string. A truncated read on a 400 "Session timed out" made the predicate false, so the purchase was abandoned rather than retried, and the operator saw status 400: with an empty body and no reason.

This PR took two rounds, and the first one did not work. The initial revision surfaced the read error in the message but left the decision unchanged. A reviewer drove the real retry loop with a counting client:

readable session-timeout 400   3 attempts   (control, moves)
truncated session-timeout 400  1 attempt    (defect, did not move)

The control moving is what proves the probe measures the loop rather than a constant. Closes LeanerCloud/cloud-commitments-cli#1766 overstated that diff, and the author confirmed it independently before re-fixing.

The author diagnosed its own error better than I did: it had asserted on the artifact it had just changed (the message) rather than the behaviour it claimed to fix (the retry). Counting attempts was the only thing that could distinguish them. The replacement test is an attempt-count assertion carrying the readable control, and it fails against the message-only revision with expected: 3, actual: 1.

Then the safety question changed the shape of the fix. Asked whether any non-2xx can follow a purchase that actually committed, the reviewer could not resolve Azures behaviour from this repo and said so, but resolved the thing that decides it, by probe:

DoIdempotentPurchaseTwoStep calls FindReservationOrderByIdempotencyToken exactly once, before DoPurchaseTwoStep is entered. The internal loop performs zero idempotency lookups between attempts, and each attempt calls doCalculatePrice fresh, minting a new distinct reservationOrderId. Demonstrated: attempt 1 retryable, attempt 2 returns 200 gives 2 distinct calculate-price calls (order-1, order-2), 2 purchase calls, 0 lookups between.

So the guard covers a re-drive across top-level invocations and nothing within one, and the bound is purchaseMaxAttempts rather than the guard. That makes "a non-2xx means the order did not commit" load-bearing rather than reassuring.

Hence the narrowing: errRetryabilityUnknown now requires resp.StatusCode == http.StatusBadRequest, matching IsSessionTimeouts existing scope, instead of firing on any non-2xx with a failed read. #1766s actual defect is a 400 "Session timed out" with a truncated body, the only scenario in the issue and both regression tests, so this closes it with zero coverage loss while removing exposure to the unverified 409/5xx partial-write class. The structural reason: a truncated response is the transport-failure class associated with genuine commit ambiguity, whereas a cleanly read 500 is a deliberate synchronous statement.

Two tests pin the narrowing, because nothing else in the code would notice a re-widening: TestDoPurchase_TruncatedNon400IsNotRetryable (409/500/502/504 truncated must not carry the sentinel) and TestDoPurchaseTwoStep_TruncatedNon400DoesNotRetry (a truncated 500 attempted exactly once). Both fail when widened back, with messages naming the double-buy risk and #1774.

The success path is unchanged and its guard bites — resp.StatusCode is checked before readErr, so an unreadable body cannot fail a completed purchase. Confirmed by mutation.

Flagged by the author, unprompted: the loops trailing "failed after %d attempts (session timeout)" would be inaccurate for a retryability-unknown exhaustion, but is unreachable (on the final attempt attempt < purchaseMaxAttempts is false, so it returns purchaseErr directly). Pre-existing dead code, no new inaccuracy. Worth recording because it is exactly the class of half-true message this PR spent two rounds removing.

The in-loop idempotency gap is pre-existing on the IsSessionTimeout path since #677/#1550. This PR widened what could reach it; the narrowing is the mitigation, not the fix. Tracked as #1774 (p1/severity/high).

Gates at c3b629c1c: 20/20 checks pass. providers/azure -race 863 passed / 14 packages, gofmt/build/vet clean, gocyclo 0, golangci-lint v2.10.1 exit 0 with a genuine 0 issues. line. Lint compared as a comm set diff against git checkout origin/main -- providers/azure/ rather than the index: 592 -> 589, zero only-in-HEAD, three only-in-BASE (the errcheck this fixes plus two disclosed spelling corrections). A root cmd timeout was investigated rather than assumed: panic: test timed out after 10m0s under concurrent-agent load, no hung test, and it passes alone at 448.9s.

@cristim
cristim merged commit d6e60f6 into main Aug 8, 2026
20 checks passed
cristim added a commit that referenced this pull request Aug 9, 2026
Comment-only. Both make exhaustiveness claims that stopped being true when
#1767 added a second retry trigger, and both now describe the current rule.

purchase_test.go said doPurchase's error text is "the ONLY input to
IsSessionTimeout, and DoPurchaseTwoStep retries only when that predicate
matches". Neither half holds: the loop also retries on
errors.Is(err, errRetryabilityUnknown). CodeRabbit flagged this on #1767; the
PR merged before the fix landed, so the claim is live on main.

DoPurchaseTwoStep's own doc had the same defect and nobody flagged it. It said
retry happens on a "Session timed out" 400 and that "Other 4xx/5xx errors are
returned immediately without retry", which omits the second trigger entirely.
Found by sweeping the whole #1767 diff for the same class rather than fixing
only the line that was reported. It now enumerates both cases and states that
everything else -- including a cleanly read 4xx and any 5xx -- returns without
retry.

Rather than past-tensing the old text, each comment leads with the current
rule and keeps the pre-fix explanation marked as the reason the test exists.
A reader who stops after the first paragraph should come away with something
true.

Exhaustiveness claims are what readers rely on to decide they need not check,
so a stale one is worse than none: #1757 (a live CSRF hole) survived behind a
parity claim that was not true, and #1764's getAccountScope godoc described
the inverse of its behavior after a change.

Refs #1767, #1766

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/xs Trivial / one-liner impact/few Limited audience priority/p2 Backlog-worthy severity/medium Moderate harm triaged Item has been triaged type/bug Defect urgency/this-sprint Within the current sprint

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(azure): discarded io.ReadAll error decides retry-vs-abort on the reservation purchase path

1 participant