Skip to content

Fix LocalExecutor crash on non-picklable exceptions (e.g. httpx.HTTPStatusError)#64485

Open
Pranaykarvi wants to merge 1 commit intoapache:mainfrom
Pranaykarvi:fix/localexecutor-unserializable-exception-clean
Open

Fix LocalExecutor crash on non-picklable exceptions (e.g. httpx.HTTPStatusError)#64485
Pranaykarvi wants to merge 1 commit intoapache:mainfrom
Pranaykarvi:fix/localexecutor-unserializable-exception-clean

Conversation

@Pranaykarvi
Copy link
Copy Markdown
Contributor

Closes #64476

When LocalExecutor runs a task in a subprocess, exceptions are passed back via multiprocessing.Queue using pickle. Some exceptions like httpx.HTTPStatusError are not pickle-safe, causing:

TypeError: HTTPStatusError.__init__() missing 2 required keyword-only arguments: 'request' and 'response'

This crashes the scheduler loop and takes down the entire scheduler pod.

Root Cause

Raw exception objects were placed directly onto the result queue:

output.put((key, TaskInstanceState.FAILED, e))

Fix

Wrap the exception in a plain Exception before queuing:

safe_exc = Exception(f"{type(e).__name__}: {str(e)}\n{traceback.format_exc()}")
output.put((key, TaskInstanceState.FAILED, safe_exc))

Applied to both ExecuteTask and ExecuteCallback branches.

Impact

  • Scheduler no longer crashes on non-picklable exceptions
  • Full debugging info preserved
  • Minimal change, no other executor behaviour affected

@boring-cyborg boring-cyborg bot added the area:Executors-core LocalExecutor & SequentialExecutor label Mar 30, 2026
@eladkal
Copy link
Copy Markdown
Contributor

eladkal commented Mar 31, 2026

@Pranaykarvi please explain how you verified the issue present only in LocalExecutor?

@eladkal eladkal added this to the Airflow 3.2.1 milestone Mar 31, 2026
@eladkal eladkal added type:bug-fix Changelog: Bug Fixes backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch labels Mar 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Executors-core LocalExecutor & SequentialExecutor backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch type:bug-fix Changelog: Bug Fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Running Python DAG crashes Scheduler pod

2 participants