fix(hms): use get_table_req for Hive 4 metastore compatibility - #3187
Merged
Conversation
Hive 4.0.1 removed the `get_table` thrift method, so the HMS catalog fails with `Invalid method name: 'get_table'` against any Hive 4 metastore. Switch `load_table`, `table_exists`, and `rename_table` to `get_table_req`, which exists in every Hive release since 2.3 (the IDL the `hive_metastore` crate is generated from), so older metastores keep working. Move the integration test metastore from `apache/hive:3.1.3` (Debian Bullseye, EOL) to `apache/hive:4.2.1` so CI exercises a Hive 4 server, mirroring apache/iceberg-python#3924. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
kevinjqliu
changed the title
get_table_req for Hive 4 metastore compatibility
kevinjqliu
commented
Sep 10, 2026
|
|
||
| ENV AWSSDK_VERSION=2.20.18 | ||
| ENV HADOOP_VERSION=3.1.0 | ||
| FROM apache/hive:4.2.1 |
Contributor
Author
There was a problem hiding this comment.
mirrors change from pyiceberg
There was a problem hiding this comment.
🟢 Approval recommended
No unresolved blocking issues were identified.
Pull request overview
Updates HMS integration for Hive 4 compatibility.
Changes:
- Replaces
get_tablewithget_table_req. - Upgrades the test metastore to Hive 4.2.1.
File summaries
| File | Summary |
|---|---|
dev/hms/Dockerfile |
Updates the Hive test image and dependencies. |
crates/catalog/hms/src/catalog.rs |
Uses the compatible request-based table API. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
blackmwk
approved these changes
Sep 10, 2026
blackmwk
left a comment
blackmwk
left a comment
Contributor
There was a problem hiding this comment.
Thanks @kevinjqliu for this fix!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
What changes are included in this PR?
Follow up to #3174. Debian Bullseye reached EOL and broke
apt-getin theapache/hive:3.1.3image we use for HMS integration tests. #3174 worked around it by downloading the jars without apt, but the base image is still EOL. pyiceberg hit the same thing and moved toapache/hive:4.2.1in apache/iceberg-python#3924. This does the same here.Moving to Hive 4 exposed a real bug: Hive 4.0.1 dropped the
get_tablethrift method, so the HMS catalog fails withInvalid method name: 'get_table'against any Hive 4 metastore.load_table,table_exists, andrename_tabletoget_table_req. It exists since Hive 2.3, which is the IDLhive_metastoreis generated from, so nothing we could talk to before stops working.apache/hive:4.2.1, same Dockerfile as pyiceberg. 3.1.3 still servesget_table, so CI could never catch this.Are these changes tested?
Existing HMS cases in the catalog suite (
crates/catalog/loader/tests), now running against Hive 4.2.1.Locally, the 21 HMS cases pass against both 4.2.1 and 3.1.3. Without the code change they fail against 4.2.1 with the error above.
AI Disclosure
Written with Claude Code, reviewed by me.