fix(parquet): address VECTOR review feedback on nested reads and shared helpers

[ChaomingZhangCN]

Purpose

Follow-up to the review feedback on apache/paimon-cpp#198 (VECTOR Parquet storage). It targets codex/vector-parquet-mvp so the PR branch can absorb the changes.

Addressed review points:

• ContainsVectorType / ContainsVectorField / ValidateVectorElements were duplicated in schema validation, the Parquet write converter and the VECTOR read wrapper. They now live in one place, VectorUtils (src/paimon/common/utils/arrow/vector_utils.{h,cpp}).
• The VECTOR element scan could read past the end of the values array. Arrow's C data interface importer does not check that a FixedSizeList child holds length * list_size values, so a caller passing a short child made the write-path nullability check index past the child's validity bitmap. VectorUtils::ValidateVectorElements now rejects that layout before scanning.
• HasSameNestedProjectionShape, CollectLeafIndices and SkipLeafIndices in the Parquet reader had no FIXED_SIZE_LIST branch, so reading a file that stores VECTOR as Arrow FixedSizeList (Paimon Rust and Python writers) failed with "partial projection inside list/map" when the vector was nested in a LIST, and a vector column was otherwise treated as a leaf.
• FieldMappingBuilder::CreateDataCastExecutors now exempts FIXED_SIZE_LIST from the scalar cast lookup, like the other nested types.
• NormalizeRecordBatchOffsets had no FIXED_SIZE_LIST branch, so sliced VECTOR columns were fully copied instead of having their buffers sliced.
• Rebuilding a nested type for VECTOR reads and writes no longer drops the name and metadata of a MAP entries field.
• Nits: needs_vector_conversion moved before the memory pool in the ParquetFormatWriter constructor, VECTOR test arrays built from JSON instead of Arrow builders, and the license header removed from the fixture notes.

Tests

• New src/paimon/common/utils/arrow/vector_utils_test.cpp: helper detection across nested types, null-element rejection for both LIST and FixedSizeList VECTOR values, slice handling, and the truncated-child layout.
• arrow_utils_test.cpp: regression test that the write-path nullability check rejects a FixedSizeList whose child is shorter than the declared dimension, plus a fixed_size_list case in the offset-normalization matrix so the zero-copy assertions cover it.
• parquet_vector_io_test.cpp: reading list<fixed_size_list<float, 3>> from a file written by the plain Arrow Parquet writer (the Rust/Python physical layout), and an end-to-end read with a predicate pushed down on a non-vector column next to a VECTOR column.

Validation status: the workspace had no configured build, so a Debug build with PAIMON_BUILD_TESTS=ON was configured from scratch (-DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++, since /usr/bin/c++ is a Clang that cannot link libstdc++ here). All third-party dependencies are bundled, so paimon-common-test, paimon-core-test and paimon-parquet-format-test are still building Arrow/Parquet/ORC from source on this 4-core VM. Test results will be reported once the build finishes.

API and Format

API: No change beyond PR apache#198. VectorUtils is an internal helper under src/paimon/common/utils/arrow/.

Schema protocol: No.

Storage format: No. VECTOR is still persisted as a standard Parquet LIST; the reader change only widens which physical layouts can be read back.

Documentation

No new documentation. The VECTOR section added by PR apache#198 already records the append-only Parquet-only scope.

Generative AI tooling

Generated-by: Cursor Cloud Agent (Claude Opus 5)