fix(components): bound follow-up prompt generation latency

[auberginewly]

Problem

Follow-up prompt generation is an optional step, but provider calls are currently awaited directly in the main prediction flow. If a provider request hangs, fails slowly, or returns invalid output, the main response path can be delayed or interrupted even though follow-up prompts are not required for the chat response itself.

Solution

This PR makes follow-up prompt generation best-effort by wrapping provider calls with a shared timeout and retry helper. The helper uses a default 10s timeout and one retry, then gracefully returns undefined if generation still fails. This keeps the existing public API unchanged while bounding latency for the optional follow-up prompt step.

Changes

• Added an internal timeout/retry helper for follow-up prompt generation.
• Applied the helper to all follow-up prompt providers:
  • Anthropic
  • Azure OpenAI
  • Google GenAI
  • Mistral
  • OpenAI
  • Groq
  • Ollama
• Preserved the existing generateFollowUpPrompts(...) function signature.
• Added focused Jest coverage for best-effort behavior.

Testing

corepack pnpm@10.26.0 --filter flowise-components test -- --runInBand packages/components/src/followUpPrompts.test.ts
corepack pnpm@10.26.0 --filter flowise-components test -- --runInBand packages/components/src/utils.test.ts
corepack pnpm@10.26.0 --filter flowise-components exec tsc --noEmit --pretty false
corepack pnpm@10.26.0 quick
corepack pnpm@10.26.0 lint-staged

Test coverage includes:

• Successful provider response returns follow-up questions.
• First provider failure retries once and succeeds.
• Provider timeout returns undefined instead of blocking indefinitely.
• Disabled follow-up prompts do not read credentials or instantiate a provider.

Notes for Reviewer

• This intentionally does not expose timeout/retry settings as user-facing configuration to keep the PR small and avoid expanding the public config surface.
• Server call sites are unchanged; the behavior is contained inside generateFollowUpPrompts.
• Follow-up prompts remain optional. If generation fails, the main chat response can continue without follow-up prompts.

[gemini-code-assist]

Code Review

This pull request introduces a timeout and retry mechanism (invokeWithTimeoutAndRetry) for generating follow-up prompts across multiple LLM providers, along with comprehensive unit tests. The review feedback suggests improving this mechanism by integrating an AbortController to pass an AbortSignal to the LLM .invoke() calls, which prevents resource leaks from dangling requests. Additionally, it is recommended to log errors on the final retry attempt instead of swallowing them, and to refactor the internal helper function to use positional parameters instead of an options object.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

Issues Identified:

1. Resource Leak / Dangling Requests: When a timeout occurs, the in-flight LLM request is left running in the background, consuming server resources, connections, and potentially API tokens/billing. We should use an AbortController and pass its AbortSignal to the task so that the in-flight request can be aborted when a timeout or failure occurs.
2. Swallowed Errors: The catch block completely swallows all errors without any logging, making it extremely difficult to debug failures in follow-up prompt generation. We should log the error on the final attempt.
3. Internal Function Parameters: For internal functions like invokeWithTimeoutAndRetry, we should avoid using an options object and instead use positional parameters until the parameter list becomes complex or more parameters are needed.

Solution:

Update FollowUpPromptTask to accept an AbortSignal, and use AbortController to abort the in-flight request on timeout or failure. Also, log the error on the final attempt, and use positional parameters for invokeWithTimeoutAndRetry instead of an options object.

type FollowUpPromptTask<T> = (signal: AbortSignal) => Promise<T>

const DEFAULT_FOLLOW_UP_PROMPT_TIMEOUT_MS = 10000
const DEFAULT_FOLLOW_UP_PROMPT_MAX_RETRIES = 1

class FollowUpPromptTimeoutError extends Error {
    constructor(timeoutMs: number) {
        super(`Follow-up prompt generation timed out after ${timeoutMs}ms`)
        this.name = 'FollowUpPromptTimeoutError'
    }
}

const createTimeoutPromise = <T>(timeoutMs: number): { promise: Promise<T>; clear: () => void } => {
    let timeout: NodeJS.Timeout
    const promise = new Promise<T>((_, reject) => {
        timeout = setTimeout(() => reject(new FollowUpPromptTimeoutError(timeoutMs)), timeoutMs)
    })

    return {
        promise,
        clear: () => clearTimeout(timeout)
    }
}

const invokeWithTimeoutAndRetry = async <T>(
    task: FollowUpPromptTask<T>,
    timeoutMs: number = DEFAULT_FOLLOW_UP_PROMPT_TIMEOUT_MS,
    maxRetries: number = DEFAULT_FOLLOW_UP_PROMPT_MAX_RETRIES
): Promise<T | undefined> => {
    for (let attempt = 0; attempt <= maxRetries; attempt += 1) {
        const controller = new AbortController()
        const timeout = createTimeoutPromise<T>(timeoutMs)

        try {
            const taskPromise = task(controller.signal)
            taskPromise.catch(() => undefined)

            return await Promise.race([taskPromise, timeout.promise])
        } catch (error) {
            controller.abort()
            if (attempt === maxRetries) {
                console.error(`Follow-up prompt generation failed after ${maxRetries + 1} attempts:`, error)
                return undefined
            }
        } finally {
            timeout.clear()
        }
    }

    return undefined
}

References

1. For internal functions, defer refactoring to an options object until the parameter list becomes complex or more parameters are needed.

[gemini-code-assist]

Pass the AbortSignal to the Anthropic .invoke() call to ensure the request is aborted on timeout or failure.

Suggested change

	return await invokeWithTimeoutAndRetry(async () => {
	const structuredResponse = await structuredLLM.invoke(followUpPromptsPrompt)
	return structuredResponse as FollowUpPromptResult
	})
	return await invokeWithTimeoutAndRetry(async (signal) => {
	const structuredResponse = await structuredLLM.invoke(followUpPromptsPrompt, { signal })
	return structuredResponse as FollowUpPromptResult
	})

[gemini-code-assist]

Pass the AbortSignal to the Azure OpenAI .invoke() call to ensure the request is aborted on timeout or failure.

Suggested change

	return await invokeWithTimeoutAndRetry(async () => {
	const structuredResponse = await chain.invoke({
	history: apiMessageContent,
	format_instructions: formatInstructions
	})
	return structuredResponse as FollowUpPromptResult
	})
	return await invokeWithTimeoutAndRetry(async (signal) => {
	const structuredResponse = await chain.invoke({
	history: apiMessageContent,
	format_instructions: formatInstructions
	}, { signal })
	return structuredResponse as FollowUpPromptResult
	})

[gemini-code-assist]

Pass the AbortSignal to the Google GenAI .invoke() call to ensure the request is aborted on timeout or failure.

Suggested change

	return await invokeWithTimeoutAndRetry(async () => {
	const structuredResponse = await structuredLLM.invoke(followUpPromptsPrompt)
	return structuredResponse as FollowUpPromptResult
	})
	return await invokeWithTimeoutAndRetry(async (signal) => {
	const structuredResponse = await structuredLLM.invoke(followUpPromptsPrompt, { signal })
	return structuredResponse as FollowUpPromptResult
	})

[gemini-code-assist]

Pass the AbortSignal to the Mistral .invoke() call to ensure the request is aborted on timeout or failure.

Suggested change

	return await invokeWithTimeoutAndRetry(async () => {
	const structuredResponse = await structuredLLM.invoke(followUpPromptsPrompt)
	return structuredResponse as FollowUpPromptResult
	})
	return await invokeWithTimeoutAndRetry(async (signal) => {
	const structuredResponse = await structuredLLM.invoke(followUpPromptsPrompt, { signal })
	return structuredResponse as FollowUpPromptResult
	})

[gemini-code-assist]

Pass the AbortSignal to the OpenAI .invoke() call to ensure the request is aborted on timeout or failure.

Suggested change

	return await invokeWithTimeoutAndRetry(async () => {
	const structuredResponse = await structuredLLM.invoke(followUpPromptsPrompt)
	return structuredResponse as FollowUpPromptResult
	})
	return await invokeWithTimeoutAndRetry(async (signal) => {
	const structuredResponse = await structuredLLM.invoke(followUpPromptsPrompt, { signal })
	return structuredResponse as FollowUpPromptResult
	})

[gemini-code-assist]

Pass the AbortSignal to the Groq .invoke() call to ensure the request is aborted on timeout or failure.

Suggested change

	return await invokeWithTimeoutAndRetry(async () => {
	const structuredResponse = await structuredLLM.invoke(followUpPromptsPrompt)
	return structuredResponse as FollowUpPromptResult
	})
	return await invokeWithTimeoutAndRetry(async (signal) => {
	const structuredResponse = await structuredLLM.invoke(followUpPromptsPrompt, { signal })
	return structuredResponse as FollowUpPromptResult
	})