Skip to content

The patch API client has no request timeout, so scan, get and apply hang forever on a stalled server #570

Description

[agent] Filed by the scheduled architecture audit routine (CLI and core). Register: discussion #560 register.

Kind: bug. Source: §1 #2; Part 7.2; register C02.

Problem

Both reqwest clients in ApiClient are built with no timeout and no connect_timeout:

None of these paths add a per-request bound:

  • the JSON retry loop send_json_request (#L475-L551), used by fetch_patch, the batch search and org resolution;
  • fetch_binary for blobs and diffs (#L1061-L1120).

The retry loop retries on status codes only, and a stalled connection never produces one.

Every other HTTP path in the crate is bounded, so the rule exists but is applied per call site:

Reproduced twice on main @ 1169ae6 with an integration test against ApiClient (not committed). The local TCP server accepts the connection, reads the request and never answers. Both fetch_blob(<hash>) and fetch_patch(<uuid>) were still pending when the test's 120 s outer guard fired, on both runs.

Symptoms

None filed. Several ci-janitor flake PRs (#419, #448, #565) handle API blips in tests, but a stall in production has no bound at all.

Impact

scan, get, apply (blob/diff fetch) and vex can hang a CI job until the job-level timeout on a stalled proxy, load balancer or half-open connection. This is P1 and the fix is small.

Proposed change

  • Give both clients a connect_timeout (e.g. 10 s) and a read/idle bound: reqwest's read_timeout, or a per-attempt timeout sized for the request kind. Keep blob/diff bodies on a longer bound than JSON.
  • Make the bound one named policy beside ApiRetryPolicy in api/retry.rs, overridable the same way. A stalled attempt then surfaces as ApiError::Network, which the existing proxy fallback already handles.
  • Out of scope here: adding retry to fetch_binary and merging the three retry systems (C15, a separate refactor).

Size and scope

api/client.rs and api/retry.rs, ~30–60 production lines plus tests.

Acceptance criteria

  • Regression tests: a server that accepts and never responds makes fetch_patch, the batch search and fetch_blob each return ApiError::Network within the configured bound, using a short test override rather than real minutes.
  • The timeout values are named constants in one place and documented in CLI_CONTRACT.md if an env override is added.
  • api_retry_e2e, binary_fetch_error_classification_e2e and blob_fetcher_edges_e2e stay green.

Dependencies

Blocks nothing. Related to C15 (one retry + timeout primitive) and to #571 (the uncapped blob/diff body in the same fetch_binary).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent:triagedarch-auditFiled by a scheduled architecture audit routine (see the architecture review discussion)bugSomething isn't workingpriority:p3

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions