Implement replace data files action

[Shekharrajak]

Which issue does this PR close?

• Ref Iceberg Table Maintenance: Acceleration Opportunities datafusion-comet#3371

• Compaction merges many small files → fewer large files
• Fewer files = less I/O overhead, faster scans
• Fewer manifest entries to track
• Change file format (e.g., Parquet v1 → v2) without changing data

What changes are included in this PR?

This PR adds ReplaceDataFilesAction - a new transaction action for replacing data files in Iceberg tables. This is essential for compaction operations where multiple small files are merged into larger ones.

In Java/Spark Iceberg: RewriteDataFilesSparkAction -> calls RewriteDataFiles API -> uses Operation.REPLACE

This Rust implementation provides the same primitive that Spark uses for:

• CALL system.rewrite_data_files('table')
• Compaction procedures
• Bin-packing optimizations

This allows maintenance tools to safely compact files without affecting ongoing reads or
concurrent writes - a key property for production table maintenance.

The plan is to have these features available so that datafusion-comet can use it for Comet Native execution.

Spark CALL system.rewrite_data_files('table') -> Spark Iceberg (Plan compaction) -> Comet Native (Execute in Rust) -> iceberg-rust (Read old files using ArrowReader, Write new files using ParquetWriter, Commit replace using ReplaceDataFilesAction)

Note: ArrowReader and ParquetWriter is already available.

Ref https://iceberg.apache.org/docs/latest/maintenance/

Are these changes tested?

Unit tests

[CTTY]

Hi @Shekharrajak , thanks for the contribution! However, even excluding the datafusion integration, I don't think just the transaction action alone is going to work as we wanted. We will have to add validation and delete processing.

Here is a stale draft that I haven't got a chance to work on, and I'm planning to pick it up soon

[Shekharrajak]

Spark CALL system.rewrite_data_files('table') -> Spark Iceberg (Plan compaction) -> Comet Native (Execute in Rust) -> iceberg-rust (Read old files using ArrowReader, Write new files using ParquetWriter, Commit replace using ReplaceDataFilesAction)

Yes, the plan is to use it in datafusion comet, this would be the flow :

Spark CALL system.rewrite_data_files('table') -> Spark Iceberg (Plan compaction) -> Comet Native (Execute in Rust) -> iceberg-rust (Read old files using ArrowReader, Write new files using ParquetWriter, Commit replace using ReplaceDataFilesAction)

ref https://iceberg.apache.org/docs/latest/maintenance/