[Feature] Add Java-compatible commit message serialization for cross-process C FFI writes

[suxiaogang223]

Search before asking

• I searched in the issues and found nothing similar.

Motivation

The C write API added by #520 supports writing and committing in one process. paimon_table_write_prepare_commit returns an opaque paimon_commit_messages* handle backed by Rust objects on the local heap. The handle can only be merged, committed, aborted, or freed in that process.

This is insufficient for a distributed write in which worker processes prepare files and a coordinator process performs the final commit. A worker cannot send the pointer to the coordinator; it needs a stable byte representation of the commit messages.

Apache Doris is one example of this architecture: BEs write data and prepare commit messages, and the FE collects the messages and commits once. Other native engines using the C FFI have the same requirement.

Paimon Java already has a versioned CommitMessageSerializer. Using that protocol as the compatibility reference would avoid introducing a separate wire format and would allow Java and Rust components to exchange commit messages.

Solution

Add a versioned serializer for Vec<CommitMessage> to the Rust core, then expose it through the C FFI.

Rust core

The serializer should:

• use the Java CommitMessageSerializer protocol as the compatibility reference;
• expose the serializer version instead of requiring FFI consumers to hard-code it;
• support the partition row, data files, deleted files, changelog files, and index files required by a commit;
• reject malformed input and unsupported versions with a clear error;
• define how the current Rust CommitMessage maps to Java's data and compaction increments.

The Rust model also contains state that may not map directly to the current Java protocol, such as conflict-check and fixed-bucket overwrite state. The implementation should either define a compatible representation or reject unsupported messages explicitly. This mapping needs agreement before implementation.

C FFI

Expose byte export/import APIs equivalent to:

paimon_result_bytes paimon_commit_messages_serialize(
    const paimon_commit_messages *messages);

paimon_result_commit_messages paimon_commit_messages_deserialize(
    const uint8_t *data,
    size_t data_len,
    /* commit context, or a handle that supplies it */);

void paimon_bytes_free(paimon_bytes bytes);

The exact deserialize signature must preserve the context currently stored in the opaque handle: table identity/location, commit_user, and overwrite mode. One option is to bind deserialized messages to an existing paimon_table_commit handle. Another is to pass an explicit, versioned context structure. The API should not trust table identity from an unvalidated remote payload.

Standard and postpone fixed-bucket commit messages should have an explicit supported behavior. They may use separate payload types or entry points if their models differ.

The serialized bytes should represent the Paimon commit message payload. An embedding engine can add its own transport envelope, such as a magic value, serializer version, and payload length.

Compatibility tests

Add Java-generated golden fixtures and verify both directions:

1. Java serializes and Rust deserializes equivalent messages.
2. Rust serializes and Java deserializes equivalent messages.
3. A coordinator can deserialize and merge payloads from multiple workers, then commit them once.
4. Truncated data, invalid lengths, and unsupported versions fail without leaks or panics.

The fixtures should cover partitioned data, new and deleted files, changelog files, index files, and supported fixed-bucket cases.

Anything else?

Related work found during the issue search:

• [Feature] Add write/commit C FFI bindings for bindings/c/ #520 added the write/commit C FFI. Its discussion explicitly deferred cross-process serialization to a later enhancement.
• Expose DataFrame-style write API (WriteBuilder / TableWrite / TableCommit) to Python #414 tracks serializable commit messages for Python/Ray, but does not provide a C FFI API.
• feat(go): add postpone fixed-bucket write bindings #722 added Go postpone fixed-bucket write bindings and states that cross-process commit-message serialization remains the integration's responsibility.

Open design questions:

1. Which Java serializer versions should Rust read and write initially?
2. Should the deserializer receive commit context from paimon_table_commit or from a separate versioned structure?
3. Should standard and postpone fixed-bucket messages share one wire type?

Java compatibility reference:

paimon-core/src/main/java/org/apache/paimon/table/sink/CommitMessageSerializer.java

Willingness to contribute

• I'm willing to submit a PR!