Fix: Use a short-live session instead of passing sessionDeps into DagRunWaiter#64520
Draft
sjyangkevin wants to merge 1 commit intoapache:mainfrom
Draft
Fix: Use a short-live session instead of passing sessionDeps into DagRunWaiter#64520sjyangkevin wants to merge 1 commit intoapache:mainfrom
sjyangkevin wants to merge 1 commit intoapache:mainfrom
Conversation
1 task
uranusjr
approved these changes
Mar 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
related: #64415
Why
The
SessionDepsession is held open from the moment the endpoint function starts until theStreamingResponsegenerator is fully consumed and FastAPI cleans up the dependency. During the polling loop (whiledag_run.statenot inState.finished_dr_states: await asyncio.sleep(self.interval)), the sync DB connection sits idle but checked out from the pool. If a DAG run takes minutes or hours to complete, that connection is locked for the entire duration. With many concurrent/waitrequests, this can exhaust the connection pool.This PR improves the above by opening a short-live session in
_serialize_xcoms, similar to what have been done in_get_dag_run.#64415 (comment)
Test Cases
Tests should have been handled in test_collect_task and test_should_respond_200_immediately_for_finished_run
Was generative AI tooling used to co-author this PR?
Generated-by: [Claude Code Opus 4.6] following the guidelines
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.