fix(generator-cli): raise HTTP timeouts for hosted FAI analysis calls by willkendall01 · Pull Request #17613 · fern-api/fern

willkendall01 · 2026-09-01T20:09:09Z

Description

Linear ticket: Refs

AutoVersionStep calls the hosted FAI service at POST /sdks/analyze-commit-diff when no local BAML ai config is supplied (remote generation via fiddle). FAI chunks a multi-MB SDK diff and fans it out over 30-40 sequential LLM calls plus a changelog rollup, so a single request legitimately runs 2-5 minutes.

Node's global fetch gives up after undici's default 300s headers timeout, which is shorter than the FAI load balancer's idle timeout. On the slowest diffs the client would abort before the server did — and an aborted request is indistinguishable from a broken one, which is exactly the signal AutoVersionStep uses to choose between trusting the analysis and falling back to a blind PATCH bump.

Companion to fern-api/fern-platform#14376, which raises that load balancer's idle timeout from 60s to 900s. Without this change the client's 300s cap simply becomes the next binding constraint.

Changes Made

New packages/generator-cli/src/utils/faiFetch.ts: raises undici's header and body timeouts to 960s — deliberately just above the load balancer's 900s, so the server's timeout wins the race and the caller receives a loggable HTTP status instead of an opaque client abort.
AutoVersionStep.analyzeViaFaiService calls faiFetch instead of global fetch.
Added undici (already in the catalog at 6.27.0, and already bundled by the CLI build).
Updated README.md generator (if applicable) — n/a

Why a process-wide dispatcher rather than a per-request one

undici only honours a per-request dispatcher when called through its own fetch export. Verified empirically on Node 24.17.0:

new Agent({ headersTimeout: 50 }) passed as init.dispatcher
  -> GLOBAL FETCH IGNORED dispatcher -> status 200

So the helper installs a global dispatcher once per process, which is the same conclusion packages/cli/cli/src/cli.ts:167-172 already reached. Two consequences worth reviewing:

The install is skipped entirely when HTTP_PROXY is set, so it cannot clobber a ProxyAgent the process installed.
When AutoVersionStep runs inside the fern CLI process it was already covered, since cli.ts installs an effectively-unlimited dispatcher at startup. This change only affects the standalone generator-cli binary — which is the path fiddle-coordinator runs, and the one that was actually failing.

Testing

Unit tests added/updated — new src/__test__/faiFetch.test.ts: pass-through to fetch, timeouts exceed 900s, dispatcher installed at most once across many requests, and the HTTP_PROXY case.
Full generator-cli suite: 480 passed, 1 skipped. The existing vi.stubGlobal("fetch", ...) tests for the FAI path keep working untouched, because the helper calls the ambient fetch binding.
tsc --project tsconfig.build.json clean, biome format + biome lint --error-on-warnings clean, knip clean.

Context on the impact

In one week of production logs, AutoVersionStep logged 50 FAI analysis failed fallbacks across 107 version decisions (~47%), every one of them status 504 from the load balancer. 50 of the 55 PATCH bumps in that window were timeouts rather than analyzed decisions. Every failure was on a diff >= 1.2MB (median 2.8MB); nothing below that threshold ever timed out.

Generated with Claude Code

AutoVersionStep calls the hosted FAI service at `/sdks/analyze-commit-diff` for remote generation. FAI chunks a multi-MB SDK diff and fans it out over dozens of sequential LLM calls, so one request legitimately runs for minutes — longer than undici's default 300s headers timeout on Node's global fetch. Routes that call through a `faiFetch` helper that raises undici's header and body timeouts to 960s, just above the FAI load balancer's own idle timeout, so the server's timeout wins the race and the caller gets an HTTP status it can log rather than an opaque client-side abort. undici only honours a per-request `dispatcher` when called through its own `fetch` export — Node's global fetch silently ignores the option — so the helper installs a process-wide dispatcher once, mirroring what the CLI already does at startup in `packages/cli/cli/src/cli.ts`. An `HTTP_PROXY` dispatcher, if the process installed one, is left alone so the proxy is not dropped. Co-Authored-By: Claude <noreply@anthropic.com>

nitpickybot

AI Review Summary

Small, well-tested change that installs a process-wide undici dispatcher with long timeouts before FAI calls. Main concerns: the global dispatcher affects every fetch in the process (not just FAI), and the proxy guard only checks HTTP_PROXY while a rejected install promise would be cached forever.

🟡 1 warning(s)
🔵 2 suggestion(s)

To request another review, comment /ai-review on this pull request.

nitpickybot · 2026-09-01T20:09:44Z

+    if (process.env.HTTP_PROXY != null) {
+        return;
+    }


🟡 warning

Proxy detection only checks HTTP_PROXY. FAI is HTTPS, so a process configured via HTTPS_PROXY (or lowercase https_proxy/http_proxy, which undici's EnvHttpProxyAgent and most tooling honour) would have its ProxyAgent silently replaced. Widen the guard:

Suggested change

if (process.env.HTTP_PROXY != null) {

return;

}

const proxyEnvVars = ["HTTP_PROXY", "http_proxy", "HTTPS_PROXY", "https_proxy", "ALL_PROXY", "all_proxy"];

if (proxyEnvVars.some((name) => process.env[name] != null)) {

return;

}

nitpickybot · 2026-09-01T20:09:44Z

+    dispatcherInstallation ??= installLongRunningDispatcher();
+    await dispatcherInstallation;


🔵 suggestion

If installLongRunningDispatcher ever rejects (it currently can't, but the try/catch only wraps the import+construction — a future edit could change that), the rejected promise is memoized and every subsequent faiFetch call throws before the request is even attempted. Cheap insurance:

Suggested change

dispatcherInstallation ??= installLongRunningDispatcher();

await dispatcherInstallation;

dispatcherInstallation ??= installLongRunningDispatcher().catch(() => undefined);

await dispatcherInstallation;

nitpickybot · 2026-09-01T20:09:44Z

+        setGlobalDispatcher(
+            new Agent({
+                headersTimeout: LONG_RUNNING_TIMEOUT_MS,
+                bodyTimeout: LONG_RUNNING_TIMEOUT_MS
+            })
+        );


🔵 suggestion

setGlobalDispatcher with a fresh Agent replaces whatever dispatcher is currently installed, including any non-proxy customization (connect timeouts, custom CA, keep-alive tuning) another part of the process may have set. Worth noting the blast radius in the doc comment, or gating on getGlobalDispatcher() still being the stock default, since this changes timeouts for all fetches in the process, not just FAI.

devin-ai-integration

Devin Review found 1 potential issue.

1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

devin-ai-integration · 2026-09-01T20:11:11Z

+        setGlobalDispatcher(
+            new Agent({
+                headersTimeout: LONG_RUNNING_TIMEOUT_MS,
+                bodyTimeout: LONG_RUNNING_TIMEOUT_MS
+            })
+        );


🟡 CLI transport settings are overwritten

When the main CLI invokes faiFetch, it replaces the existing dispatcher and restores undici's default connection timeout. Slow connections can now fail.

Prompt for agents

Avoid replacing the process-wide dispatcher inside packages/generator-cli/src/utils/faiFetch.ts. The main Fern CLI installs an Agent with an effectively unlimited connect timeout before PostGenerationPipeline runs, but installLongRunningDispatcher replaces it with an Agent that only customizes header and body timeouts. Use an isolated undici fetch/dispatcher for the hosted FAI request, or preserve the existing dispatcher's transport behavior while extending these timeouts. Also cover execution after the main CLI dispatcher has already been installed.

Was this helpful? React with 👍 or 👎 to provide feedback.

willkendall01 self-assigned this Sep 1, 2026

nitpickybot Bot reviewed Sep 1, 2026

View reviewed changes

devin-ai-integration Bot reviewed Sep 1, 2026

View reviewed changes

Navigation Menu

Uh oh!

fix(generator-cli): raise HTTP timeouts for hosted FAI analysis calls - #17613

fix(generator-cli): raise HTTP timeouts for hosted FAI analysis calls#17613
willkendall01 wants to merge 1 commit into
mainfrom
fix/generator-cli-fai-http-timeouts

willkendall01 commented Sep 1, 2026 •

edited by devin-ai-integration Bot

Loading

Uh oh!

nitpickybot Bot left a comment

Uh oh!

nitpickybot Bot Sep 1, 2026

Uh oh!

nitpickybot Bot Sep 1, 2026

Uh oh!

nitpickybot Bot Sep 1, 2026

Uh oh!

devin-ai-integration Bot left a comment

Uh oh!

devin-ai-integration Bot Sep 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

-    if (process.env.HTTP_PROXY != null) {
-        return;
-    }
+    const proxyEnvVars = ["HTTP_PROXY", "http_proxy", "HTTPS_PROXY", "https_proxy", "ALL_PROXY", "all_proxy"];
+    if (proxyEnvVars.some((name) => process.env[name] != null)) {
+        return;
+    }

		dispatcherInstallation ??= installLongRunningDispatcher();
		await dispatcherInstallation;

Uh oh!

Conversation

willkendall01 commented Sep 1, 2026 • edited by devin-ai-integration Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Changes Made

Why a process-wide dispatcher rather than a per-request one

Testing

Context on the impact

Uh oh!

nitpickybot Bot left a comment

Choose a reason for hiding this comment

AI Review Summary

Uh oh!

nitpickybot Bot Sep 1, 2026

Choose a reason for hiding this comment

Uh oh!

nitpickybot Bot Sep 1, 2026

Choose a reason for hiding this comment

Uh oh!

nitpickybot Bot Sep 1, 2026

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot Sep 1, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

willkendall01 commented Sep 1, 2026 •

edited by devin-ai-integration Bot

Loading