SegFault with fill_null_forward() on Boolean ChunkedArray

[aiudirog]

Describe the bug, including details regarding any error messages, version, and platform.

I've encountered a consistent segfault when trying to use Pandas .ffill() with a boolean ChunkedArray on Windows (no problems on Linux):

import pandas as pd
import pyarrow as pa
print("Pandas:", pd.__version__)
print("PyArrow:", pa.__version__)

for i in range(1, 32):
    mx = 2 ** i
    print(i, mx)
    x = pd.Series(pd.NA, index=range(mx), dtype='boolean[pyarrow]')
    x[mx // 2] = True
    x.ffill()

This always crashes on the 12th iteration, when the array has 4096 values, across multiple computers with the following error:

Process finished with exit code -1073741819 (0xC0000005)

I dug under the hood to see what Pandas was doing and was able to create this pure PyArrow example (though I'm not sure if I'm using the API correctly):

import pyarrow as pa 
import pyarrow.compute as pc

pad = [None] * 4000
a = pa.chunked_array([pad, [True], pad], type=pa.bool_())
pc.fill_null_forward(a)

Additionally, I also tried to reproduce this with just pa.array() but that worked fine.

---

Operating System: Windows 10 (10.0.19045 Build 19045)
Processor: Intel Core i7-12850HX
Python Version: 3.12.7 (additionally confirmed by co-workers on 3.11 & 3.10)
PyArrow Versions Tested: 16.1, 17.0, 18.0, 18.1

Component(s)

Python

[aiudirog]

While running some other benchmarks, I ended up reproducing the issue on my personal Linux PC with Python 3.13.1 and PyArrow 18.1:

import pandas as pd
import numpy as np

data = pd.Series(np.random.random(100000) > 0.5, dtype='bool[pyarrow]')
data[0] = pd.NA
data[50000] = pd.NA

data.ffill()  # segfaults


import pyarrow.compute as pc

pc.fill_null_forward(data.values._pa_array)  # Also segfaults

System Info

OS: Arch Linux
Kernel: 6.12.8-zen1-1-zen
Python: 3.13.1
Installed Packages (venv):

numpy==2.2.1
pandas==2.2.3
pyarrow==18.1.0
python-dateutil==2.9.0.post0
pytz==2024.2
six==1.17.0
tzdata==2024.2

Additionally, I was able to reproduce it on a RHEL8 server in the current Docker python:3.12 image. On that same server, outside of the container, I was unable to reproduce the problem in the system Python 3.8 installation with PyArrow 14.0.1.

[aiudirog]

I found the bug! It appears allocations for new chunked arrays don't account for the BooleanType.byte_width() returning 0:

arrow/cpp/src/arrow/type.h

Lines 608 to 609 in 6a0414b

	// BooleanType within arrow use a single bit instead of the C 8-bits layout.
	int bit_width() const final { return 1; }

arrow/cpp/src/arrow/type.h

Lines 186 to 192 in 6a0414b

	/// \brief Returns the type's fixed byte width, if any. Returns -1
	/// for non-fixed-width types, and should only be used for
	/// subclasses of FixedWidthType
	virtual int32_t byte_width() const {
	int32_t num_bits = this->bit_width();
	return num_bits > 0 ? num_bits / 8 : -1;
	}

This results in the allocation of a chunk with 0 bytes in it, but with the length declared to that of the original chunk.

Example in ForwardFillChunked.Exec():

arrow/cpp/src/arrow/compute/kernels/vector_replace.cc

Lines 693 to 699 in 6a0414b

	if (is_fixed_width(out->type()->id())) {
	ArrayData* output = out->mutable_array();
	ARROW_ASSIGN_OR_RAISE(output->buffers[0], ctx->AllocateBitmap(chunk->length()));
	ARROW_ASSIGN_OR_RAISE(
	output->buffers[1],
	ctx->Allocate(out->type()->byte_width() * chunk->length()));
	}

[zanmato1984]

Issue resolved by pull request 50843
#50843