From 306608c607979a5001c43ab553bc6488b3f3cf88 Mon Sep 17 00:00:00 2001 From: Connor Tsui Date: Wed, 23 Sep 2026 14:37:12 -0400 Subject: [PATCH] docs: show decimal encoding evolution Signed-off-by: "Connor Tsui" --- docs/_static/versioning-flow.svg | 128 +++++++++++++++++-------------- docs/specs/versioning/design.md | 18 +++-- 2 files changed, 80 insertions(+), 66 deletions(-) diff --git a/docs/_static/versioning-flow.svg b/docs/_static/versioning-flow.svg index d3b26be8ffa..40129a2baa2 100644 --- a/docs/_static/versioning-flow.svg +++ b/docs/_static/versioning-flow.svg @@ -1,21 +1,18 @@ - - One decimal encoding, two wire contracts + Decimal encoding changes retain support for the v1 wire format - DecimalBytePartsArray holds either one signed integer child or a signed most-significant child - followed by up to three unsigned lower-part children. The plugin serializes the single-child - shape using vortex.decimal_byte_parts, called v1 here, and the multi-child shape using - vortex.decimal_byte_parts.v2. Both wire contracts deserialize into DecimalBytePartsArray. - v1 metadata records the signed child's type and a lower-part count of zero. v2 metadata records - the signed child's type and the ordered types of zero to three lower parts. Thus v2 also accepts - one child, although the serializer selects v1 for that shape. Both contracts require a decimal - data type and have no buffers of their own. Each child has the same length as the parent, and the - signed child carries the parent's nullability, while lower parts are non-nullable. The arrows show serialization - choices and the corresponding reads, not every valid v2 input. Values, data types, and nulls - are preserved. Frozen wire contracts are immutable. Edition permissions are checked separately, - and no declared edition currently permits v2. + The rows compare the decimal-byte-parts implementation before and after multi-child support. + Before the change, the array held one signed integer child and values had to fit in 64 bits. + It read and wrote vortex.decimal_byte_parts, called v1 here. After the change, the array holds + one signed child and up to three unsigned children, adding support for 128-bit and 256-bit + values. Its plugin still writes v1 for a single child, but writes vortex.decimal_byte_parts.v2 + for two to four children. Both wire formats deserialize into the updated array type. The v1 + contract still requires exactly one signed child. The v2 contract permits one to four children, + so it can be read with one child even though the serializer chooses v1 for that shape. + The vertical arrow marks a change to the implementation, not a conversion of stored data. - + - One decimal encoding, two wire contracts - The plugin converts between an array in memory and its serialized metadata and children. - IN MEMORY - ON WIRE - - DecimalBytePartsArray - - - - - - - - Decimals - Signed child - Decimals - Signed child - Unsigned child - One child - Multiple children (two shown) - - - - - serialize - deserialize - serialize - deserialize - - v1: exactly one signed child - vortex.decimal_byte_parts - Metadata: child type, lower-part count = 0 - Children: [signed] - - v2: signed child + 0 to 3 unsigned children - vortex.decimal_byte_parts.v2 - Metadata: signed type, ordered lower-part types - Children: [signed, lower parts, most significant first] - Although v2 accepts one child, the serializer chooses v1 when there are no lower parts. - Both contracts require a decimal dtype and have no own buffers. Each child has the same length as the parent. - The signed child carries the parent's nullability, whereas lower parts are non-nullable. - Conversions preserve values, types, and nulls. Frozen wire contracts remain fixed. - Edition permissions are checked separately, and no declared edition currently permits v2. + Decimal encoding changes retain support for the v1 wire format + IN-MEMORY IMPLEMENTATION + SERIALIZED DATA + + + Before multi-child support + DecimalBytePartsArray + One signed integer child + Values must fit in 64 bits. + + + v1 + vortex.decimal_byte_parts + Exactly one signed child + + + + + + write + read + + + + + + Add support for multiple children + + + After multi-child support + DecimalBytePartsArray + One signed child and up to three + unsigned children + Adds 128-bit and 256-bit values. + The plugin reads both v1 and v2. + + + v1: the wire contract is unchanged + vortex.decimal_byte_parts + Exactly one signed child + + + v2: an additional wire contract + vortex.decimal_byte_parts.v2 + One signed + 0 to 3 unsigned children + + + + + + + + + write: 1 child + read: 1 child + write: 2 to 4 children + read: 1 to 4 children + diff --git a/docs/specs/versioning/design.md b/docs/specs/versioning/design.md index d2502458124..0efb20b6f32 100644 --- a/docs/specs/versioning/design.md +++ b/docs/specs/versioning/design.md @@ -95,19 +95,21 @@ to the array structure, while still reading the same stored data under the same ## Example: decimal children -The decimal-byte-parts plugin can serialize the same in-memory array type using two wire formats. -Its encoding stores decimal values in integer child arrays, either in one child or split across -several children. The v1 wire contract permits only one child, while v2 adds support for multiple -children under a distinct wire ID.[^decimal-availability] +The decimal-byte-parts encoding initially stored each value in one signed integer child, so values +had to fit in 64 bits. Adding up to three unsigned children extended the encoding to values that +require 128 or 256 bits. The updated array type supports both single-child and multi-child arrays, +while its plugin continues to read and write the v1 wire format. Arrays with multiple children use +the additional v2 wire format.[^decimal-availability] ```{figure} ../../_static/versioning-flow.svg -:alt: One decimal encoding holds either one signed child or a signed child with unsigned lower parts. The serializer chooses v1 for one child and v2 for multiple children. Both wire formats deserialize into the same array type. +:alt: Before multi-child support, the decimal array held one signed child and read and wrote v1. After the change, the array supports one to four children and 128-bit and 256-bit values. Its plugin writes v1 for one child and v2 for multiple children, and reads both wire formats into the updated array type. :target: ../../_static/versioning-flow.svg :figclass: versioning-diagram -The arrows show the serializer's choices and the corresponding reads. Although v2 also accepts a -single child, the serializer chooses v1 for that shape. Each child has its own wire ID because it is -itself a serialized array. +The rows compare the implementations before and after multi-child support was added. The vertical +arrow marks that implementation change, while the horizontal arrows show writing and reading. +Although v2 also accepts a single child, the serializer chooses v1 for that shape. Each child is +itself a serialized array with its own wire ID. ``` For a single-child array, the serializer reuses the child and writes v1 metadata without