GH-50027: [Format][C++][Python] Add fixed and variable closedness range canonical extension types by Hoeze · Pull Request #50028 · apache/arrow

Hoeze · 2026-05-24T13:35:11Z

GitHub Issue: [Format] Add arrow.range canonical extension type for bounded ranges #50027

github-actions · 2026-05-24T13:35:36Z

⚠️ GitHub issue #50027 has been automatically assigned in GitHub to PR creator.

rok · 2026-05-24T19:50:45Z

Copilot

Copilot was unable to review this pull request because the user who requested the review is ineligible. To be eligible to request a review, you need a paid Copilot license, or your organization must enable Copilot code review.

rok · 2026-06-04T21:38:55Z

+}
+
+// ---------------------------------------------------------------------------
+// RangeIncType


Perhaps wwe can find a better name than RangeInc?

Absolutely 😄
Some ideas:

VarRangeType

VariableClosedRangeType

PerValueRangeType

What do you think?

One more idea:
FixedClosednessRangeType and VariableClosednessRangeType (similar to the tensor type naming

Range and GranularRange? Naming is hard.

Add a canonical extension type for bounded ranges (mathematical intervals), distinct from Arrow's calendar Interval (duration) type. - Spec: docs/source/format/CanonicalExtensions.rst adds the Range section. Storage is Struct<lower, upper> with both bounds nullable (null = +/-infinity, treated as exclusive). A closed parameter (left/right/both/neither, pandas vocabulary) is carried as JSON extension metadata; the subtype is read from storage. Disambiguates from the calendar Interval type per DB convention (INTERVAL = duration, RANGE/PERIOD = bounded set). - C++ reference impl: cpp/src/arrow/extension/range.{h,cc} (RangeType/RangeArray) with serialize/deserialize, storage validation, registration in the global registry, tests, and CMake/meson wiring.

The closedness is no longer defaulted on the wire: empty metadata or a JSON object without a "closed" key is now rejected by Deserialize, so a serialized arrow.range is always unambiguous. The C++ convenience default argument for constructing a RangeType in code is left-closed ([lower, upper)), matching the PostgreSQL/Rust/Python range convention. Spec and tests updated.

Verified by building the arrow-canonical-extensions-test target (50/50 pass, 10/10 RangeType). Two fixes to the previously-uncompiled test: - include arrow/array/array_nested.h for the full StructArray definition (it is only forward-declared in type_fwd.h). - wrap the CheckDeserialize helper in an anonymous namespace to avoid a link-time collision with the identically named helper in opaque_test.cc.

Add a sibling canonical extension type to arrow.range that stores bound inclusivity per value via non-nullable boolean lower_inc/upper_inc fields, storage Struct<lower:T, upper:T, lower_inc:bool, upper_inc:bool>. arrow.range carries a single type-level closed parameter, sufficient for discrete ranges that canonicalize to one closedness (int4range, int8range, daterange). Continuous ranges (numrange, tsrange, tstzrange) cannot be canonicalized, so closedness must travel with each value. arrow.range_inc mirrors PostgreSQL's internal range representation for that case; both types coexist. The type has no metadata parameters: inclusivity lives in storage, so Serialize emits {} and Deserialize accepts empty/{}/extra keys. A null (infinite) bound is always exclusive regardless of its flag. Covers C++ (type, array, registration, tests), pyarrow bindings and tests, and the format spec, status table, and C++/Python API docs.

Under CMAKE_UNITY_BUILD (Windows CI), range_test.cc and opaque_test.cc are merged into one translation unit. Both declared a CheckDeserialize helper (range's in an anonymous namespace, opaque's in namespace arrow), making the unqualified call ambiguous and failing the MSVC build with C2668. Rename the range helper to CheckRangeDeserialize to remove the collision.

Use JsonWriter for Serialize and the simdjson DOM helpers for Deserialize, matching the other canonical extension types.

Name the pair after arrow.fixed_shape_tensor and arrow.variable_shape_tensor: closedness is either one type parameter or stored per value. - arrow.range -> arrow.fixed_closedness_range - arrow.range_inc -> arrow.variable_closedness_range - C++: RangeType/RangeArray -> FixedClosednessRangeType/Array, RangeIncType/RangeIncArray -> VariableClosednessRangeType/Array, range()/range_inc() -> fixed_closedness_range()/variable_closedness_range() - pyarrow: range_/range_inc and the Range* classes follow the same names

Copilot

Copilot was unable to review this pull request because the user who requested the review is ineligible. To be eligible to request a review, you need a paid Copilot license, or your organization must enable Copilot code review.

github-actions Bot added Component: C++ Component: Python Component: Documentation awaiting review Awaiting review labels May 24, 2026

Hoeze marked this pull request as ready for review May 24, 2026 13:36

Hoeze requested review from AlenkaF, raulcd and rok as code owners May 24, 2026 13:36

rok mentioned this pull request May 24, 2026

[Format] Add arrow.range canonical extension type for bounded ranges #50027

Open

Copilot AI lite review requested due to automatic review settings June 4, 2026 19:52

Copilot AI reviewed Jun 4, 2026

rok reviewed Jun 4, 2026

View reviewed changes

github-actions Bot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels Jun 8, 2026

Hoeze added 13 commits September 12, 2026 14:19

style(format): align range extension type with neighbor C++ conventions

b30c001

docs(format): refine arrow.range spec and add status and C++ API entries

10d68d8

feat(python): add pyarrow bindings for arrow.range extension type

690fcf7

feat(format): allow non-nullable bounds in arrow.range

b3253df

feat(python): add allow_unbounded option to pyarrow range_

64e1088

style(format): apply clang-format and cmake-format to range extension

65e19d4

refactor(format): migrate arrow.range JSON metadata to simdjson

ad3168a

Use JsonWriter for Serialize and the simdjson DOM helpers for Deserialize, matching the other canonical extension types.

hoeze-minion force-pushed the feat/arrow-range-extension branch from f3ec0b6 to c414320 Compare September 13, 2026 00:24

hoeze-minion requested a review from pitrou as a code owner September 13, 2026 00:24

Copilot AI review requested due to automatic review settings September 13, 2026 00:24

Copilot AI reviewed Sep 13, 2026

github-actions Bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Sep 13, 2026

Hoeze changed the title ~~GH-50027: [Format][C++][Python] Add arrow.range canonical extension type~~ GH-50027: [Format][C++][Python] Add fixed and variable closedness range canonical extension types Sep 13, 2026

Navigation Menu

Uh oh!

GH-50027: [Format][C++][Python] Add fixed and variable closedness range canonical extension types - #50028

GH-50027: [Format][C++][Python] Add fixed and variable closedness range canonical extension types#50028
Hoeze wants to merge 13 commits into
apache:mainfrom
Hoeze:feat/arrow-range-extension

Hoeze commented May 24, 2026 •

edited by github-actions Bot

Loading

Uh oh!

github-actions Bot commented May 24, 2026

Uh oh!

rok commented May 24, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

rok Jun 4, 2026

Uh oh!

Hoeze Jun 4, 2026

Uh oh!

Hoeze Jun 4, 2026 •

edited

Loading

Uh oh!

rok Jun 4, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

Hoeze commented May 24, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented May 24, 2026

Uh oh!

rok commented May 24, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

rok Jun 4, 2026

Choose a reason for hiding this comment

Uh oh!

Hoeze Jun 4, 2026

Choose a reason for hiding this comment

Uh oh!

Hoeze Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rok Jun 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Hoeze commented May 24, 2026 •

edited by github-actions Bot

Loading

Hoeze Jun 4, 2026 •

edited

Loading