Standardize error handling in CLI connection import#64498
Standardize error handling in CLI connection import#64498Vegapunk-debug wants to merge 12 commits intoapache:mainfrom
Conversation
|
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
|
|
This one does not need newsfragment. |
|
@potiuk , I have completely removed the file now. |
|
Hi @potiuk , I resolved the issue you mentioned, Can you please review the PR. |
potiuk
left a comment
There was a problem hiding this comment.
Actually - some errors will ony be raised if we try to commit (and then individual variables are skipped), so I do not think that change is good -> if anything, connection import should be updated to also follow this pattern.
|
There are usually small number of connections or variables to import - so this optimisation is really worse than handling individual commit errors |
Handle exceptions during connection import and rollback on failure.
|
Hi @potiuk , Thank you for your guidance! I have updated the pull request based on your feedback. I completely reverted the optimization attempt in Please let me know if this looks good to go! |
|
It's different than variables. Can you use the same approach as there, please. |
|
Converted to draft for now. |
There was a problem hiding this comment.
Pull request overview
This PR updates the airflow connections import CLI flow to handle per-connection database/merge failures without aborting the entire import, aligning the behavior more closely with the variables import command.
Changes:
- Wraps
session.merge()+session.commit()in a per-itemtry/except, rolling back on failure and continuing to the next connection. - Adds failure messaging for individual connection import errors.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
@potiuk ,
Updated the implementation to follow variables_import approach.
closes: None (No related issue)
Description
This PR standardizes error handling for the
airflow connections importCLI operation, aligning its behavior withairflow variables import.Note: This PR was originally opened to address transaction overhead by batching commits, but based on maintainer feedback (@potiuk), it was pivoted to prioritize robust error handling over performance.
Previously, a single database or validation error during a bulk connection import would cause the entire command to fail.
This PR updates
connection_command.pyby wrapping the individualsession.merge()andsession.commit()operations inside atry...exceptblock within the_import_helperloop. If an individual connection fails to import, it now prints the specific error, executes asession.rollback(), and safely continues to the next connection in the file.connection_command.py: Added per-item try/except block withsession.rollback()to ensure single failures do not abort the entire batch import.variable_command.py: Reverted previous optimization attempts to maintain its existing, intentional per-item commit and error handling structure.Was generative AI tooling used to co-author this PR?
{pr_number}.significant.rst, in [airflow-core/newsfragments](https://github