Support intelligent commit retries

[Fokko]

Feature Request / Improvement

Within Iceberg, when a commit fails because of a concurrent operation, we can retry the operation by loading the latest version of the snapshot, and re-apply the operation.

[nicor88]

Few suggestions on this feature. It will be good to have control of the amount of retries and the retry strategy. After trying out a few retries libraries I found tenacity one of the most complete because it allows different options for retrying.

[github-actions]

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

[github-actions]

This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'

[sungwy]

mark not stale

[kevinjqliu]

As a workaround, to manually retry commits, update the table metadata by using

table = table.refresh()

before calling commit() again