Fix missing _team_name on DagRun before some listener calls - #70760
Fix missing _team_name on DagRun before some listener calls#70760kacpermuda wants to merge 4 commits into
Conversation
|
cc @ferruzzi , FYI as I see that you worked on adding this before |
a0393d4 to
45cf7c3
Compare
45cf7c3 to
e83184a
Compare
|
I would introduce a helper (say |
e83184a to
9fe1f70
Compare
@uranusjr Extracted the logic to a separate helper, ptal |
| clear_db_teams() | ||
| clear_db_dag_bundles() |
There was a problem hiding this comment.
Clearing the db at the beginning of the tests should be a code smell. I believe this issue was fixed in #69093, if not, then we need to figure out what tests are not cleaning up after themselves and fix them.
There was a problem hiding this comment.
Correct, I should have just used the fixtures currently present. Adjusted the code.
| bundle = DagBundleModel(name="test_bundle") | ||
| bundle.teams.append(team) | ||
| session.add(bundle) | ||
| session.flush() |
There was a problem hiding this comment.
These tests all have a pretty big repetitive block at the top, maybe move the setup into a shared fixture if none of the existing fixtures do what you need? look at the testing_team fixture though, I think that covers most/all of this boilerplate.
There was a problem hiding this comment.
Correct, used testing_team and testing_dag_bundle fixtures instead, and linked them together in a new small fixture that I reused in all tests. Should be clean now, thanks for spotting that.
|
@kacpermuda can you resolve comments? |
9fdc186 to
21524ac
Compare
|
@vatsrahul1001 addressed all the comments |
|
LGTM!, can be merged after code owners review |
21524ac to
560334c
Compare
In multi-team deployments the scheduler fires
on_dag_run_running / success / failedlistener hooks from several code paths. Before this change,DagRun._team_namewas not guaranteed to be set when those hooks fired, so OpenLineage (and any other plugin reading_team_name) would miss the team on some events._team_nameis a private attribute — it is not part of any public API and is not guaranteed to be present on everyDagRunobject a listener receives. Since the value is already cached inself._dag_id_to_team_namefor metrics purposes, setting it consistently before each hook fires adds no cost. Listeners that wish to use it should treat it as best-effort and guard accordingly.This PR ensures
_team_nameis stamped on every ORMDagRunobject beforenotify_dagrun_state_changed()is called, covering all five listener code paths. The stamping is cheap:_get_team_names_for_dag_idscaches results inself._dag_id_to_team_namefor the lifetime of a scheduler loop, so most calls are a dict read with no DB query. I've also added comments to the existing stamping paths, just to make sure it's clear why it's done before listener call.Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Sonnet 4.6) following the guidelines
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.