feat(hive): vendor Hive Metastore IDL and generated bindings - #749
Conversation
Generate the Hive Metastore C++ bindings once from the vendored Thrift IDL and check them into src/iceberg/catalog/hive/gen-cpp/, so a normal build needs no Thrift IDL compiler. dev/update_hive_thrift.sh regenerates them when the IDL changes (mirrors Arrow's build-support/update-thrift.sh). The vendored IDL is pinned to Hive 4.0.1: Hive 3.1 declares Table.creationMetadata before CreationMetadata is defined, so Thrift's C++ generator emits an incomplete type that fails under C++23 -Werror; 4.0 reorders the structs. The generated sources keep Thrift's own @generated header instead of an ASF license header, and are excluded from license-eye and the formatting hooks, matching how Apache Arrow vendors its Thrift-generated Parquet code.
MisterRaindrop
changed the title
|
Do we need to check in the generated code? It might be cleaner to have the build system generate these files on the fly instead, since the IDL is the source of truth and the generated code is just a derived artifact. |
I don't think so. These files are not changed constantly so it wastes resource and complicates the ci workflow if we built them dynamically. Arrow C++ does the same thing for parquet thrift source code. |
|
@MisterRaindrop Just curious, how long does it take to generate these files? |
The gen cpp speed is very fast, around one second. |
|
Thanks @MisterRaindrop for adding this and @zhjwpku for the review! |
3 participants
Summary