feat(index): add vector index option surface and validation#19310
Open
chrevanthreddy wants to merge 1 commit into
Open
feat(index): add vector index option surface and validation#19310chrevanthreddy wants to merge 1 commit into
chrevanthreddy wants to merge 1 commit into
Conversation
Collaborator
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Describe the issue this Pull Request addresses
Resolves #19096 (umbrella: #19094, RFC-109)
First foundational slice of RFC-109 (Hudi Native Vector Index): the option surface and validation for vector indexes. This PR is intentionally scoped to the self-contained option/config layer so it compiles and tests standalone; the DDL command dispatch, MDT schema/payload, and bootstrap execution land in dependent follow-up PRs (#19097, #19099, …) where their partition-type and executor dependencies are introduced.
Summary and Changelog
org.apache.hudi.common.index.vector.VectorIndexOptions— the fullOPTIONS(...)key surface forCREATE INDEX ... USING VECTOR(dimension, metric, algorithm, quantizer, cluster/probe/refine counts, RaBitQ bits/seed/storage/posting layout, search-time flags) with typed accessors, defaults, and early validation.VectorIndexType— routing algorithm enum (IVFFLAT,IVFPQ_HNSW) with case/separator-insensitive parsing.VectorDistanceMetric—COSINE/L2/DOT_PRODUCTdistance enum (min-heap-consistent distances) with parsing.PARTITION_NAME_VECTOR_INDEX/PARTITION_NAME_VECTOR_INDEX_PREFIXconstants toHoodieTableMetadataUtil.TestVectorIndexOptions— 9 unit tests covering defaults, metric/algorithm parsing, and early-failure on invalid configs (missing dimension, invalid metric, out-of-range RaBitQ bits).Impact
No behavioral impact on existing tables or code paths. Adds new, unreferenced option/enum classes plus two constants. No schema, writer, reader, or DDL execution changes in this PR.
Risk Level
none
New self-contained classes + two constants + unit tests; nothing wires into existing runtime paths yet.
Documentation Update
Option semantics are documented as Javadoc on each option key. The user-facing RFC document is tracked separately (see #19309). No site docs change required until the feature is user-invokable.
Contributor's checklist