HIVE-29301: Missing histogram in DESCRIBE FORMATTED after prior column DESCRIBE - #6670
Open
cyanzheng2926 wants to merge 2 commits into
Open
HIVE-29301: Missing histogram in DESCRIBE FORMATTED after prior column DESCRIBE#6670cyanzheng2926 wants to merge 2 commits into
cyanzheng2926 wants to merge 2 commits into
Conversation
…ng another DESCRIBE FORMATTED before setting metastore.stats.fetch.kll For embedded metastore, session level SET updates HiveConf in place, but embedded ObjectStore can keep a stale Configuration reference after an earlier column stats fetch, leaving describe formatted histogram values empty despite metastore.stats.fetch.kll=true. Rebind the embedded handler and ObjectStore when stats fetch keys change in embedded mode only. Add order-dependent qfile reproducers verified with TestMiniLlapCliDriver.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
For embedded metastore, session level SET updates HiveConf in place, but embedded ObjectStore can keep a stale Configuration reference after an earlier column stats fetch, leaving describe formatted histogram values empty despite metastore.stats.fetch.kll=true. Rebind the embedded handler and ObjectStore when
stats fetch keys change in embedded mode only. Add order-dependent qfile reproducers verified with TestMiniLlapCliDriver.
What changes were proposed in this pull request?
When
metastore.stats.fetch.kll=trueis set after an earlierDESCRIBE FORMATTEDon a column in the same session, histogram may be shown as empty on a subsequentDESCRIBE FORMATTEDeven when kll data exists in the metastore.This PR fixes that for embedded metastore by re-syncing the embedded handler and ObjectStore configuration after session level config set for stats-fetch flags.
Added
syncEmbeddedHandlerConfmethod underIMetaStoreClient(default no-op), delegate throughMetaStoreClientWrapper, implemented inThriftHiveMetaStoreClientto rebind embedded handler/ObjectStore when performing set on stats-fetch keys.Why are the changes needed?
For embedded metastore, session level set updates session HiveConf in place, but embedded ObjectStore can retain a stale config reference after an earlier column stats fetch, causing inconsistency between display and actual metadata storage.
Does this PR introduce any user-facing change?
Yes. For embedded metastore sessions, set metastore.stats.fetch.kll=true after an earlier column desc formatted statement now correctly populates histogram values on subsequent desc formatted commands.
How was this patch tested?
Added the qtest files using the examples from the jira https://issues.apache.org/jira/browse/HIVE-29301