Add indexes on span_status and dag_version_id in task_instance#53746
Add indexes on span_status and dag_version_id in task_instance#53746anandkaranubc wants to merge 2 commits intoapache:mainfrom
span_status and dag_version_id in task_instance#53746Conversation
span_status and dag_version_id in task_instancespan_status and dag_version_id in task_instance
|
Well. You also need to change definition of the models - migrations generally should be generated as result of changes in the models as far as I understand. |
|
Just linked the PR to the issue: closes: #53401 |
|
I am working on adding the indexes manually on DB in my Airflow 3 instance since it is affecting DP performance. I am curious why we are having |
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions. |
|
@anandkaranubc We are unassigning you from this issue as part of our updated assignment policy. This is not meant to discourage your contribution — quite the opposite! You are still very welcome to work on this issue and submit a PR for it. Simply comment that you are working on it and open a PR when ready. We found that formal assignments were not working well, as they often prevented others from contributing when the assignee was not actively working on the issue. |
closes: #53401
This PR adds two indexes to the
task_instancetable to improve query performance:idx_span_statuson thespan_statuscolumnidx_dag_version_idon thedag_version_idcolumnThese indexes help speed up scheduler and DAG processor operations when the
task_instancetable has a large number of records (~10M+), as noted in issue #53401.^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.