Skip to content

[C++] compute.cumulative_max initializes incorrectly for non-integer types #51194

Description

@adienes

Describe the bug, including details regarding any error messages, version, and platform.

the initial value is documented to be

the minimum value of input type (so that any other value will replace the start as the new maximum)

but since it uses std::numeric_limits<T>::min(), for floating-point types like double, this actually initializes with the smallest positive value of the input type. so any input with a prefix of values <=0 will yield incorrect results. this also impacts pandas.cummax()

GPT-6 Astra helped identify the bug

import pyarrow as pa
import pyarrow.compute as pc
import pandas as pd

values = [-2.5, 2.5]
arrow_result = pc.cumulative_max(pa.array(values, type=pa.float64())).to_pylist()
pandas_result = pd.Series(values, dtype="float64[pyarrow]").cummax().tolist()
expected = [-2.5, 2.5]

print("expected running maximum:", expected)
print("pyarrow cumulative_max:  ", arrow_result)
print("pandas cummax (arrow):   ", pandas_result)
print("pandas cummax (numpy):   ", pd.Series(values).cummax().tolist())
assert arrow_result == expected, "cumulative_max seeded with the smallest positive float"
>>> expected
[-2.5, 2.5]
>>> arrow_result
[2.2250738585072014e-308, 2.5]
>>> pandas_result
[2.2250738585072014e-308, 2.5]

Component(s)

C++

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions