Segmentation fault upon multi-key sorting of concatenated tables

[mgorny]

Describe the bug, including details regarding any error messages, version, and platform.

Consider the following reduced example:

import pyarrow as pa
from datetime import datetime

tables = [
    pa.Table.from_pydict(
        {
            "date": [datetime(2020, 1, 1)],
            "variable": ["aapl"],
            "price": [110],
        }
    ),
    pa.Table.from_pydict(
        {
            "date": [datetime(2020, 1, 1)],
            "variable": ["tlsa"],
            "price": [220],
        }
    ),
]

unpivoted = pa.concat_tables(tables)
unpivoted.sort_by([("date", "ascending"), ("variable", "ascending")])

Upon running it on PyArrow 21.0.0 built from source, I'm getting a segmentation fault:

(gdb) bt
#0  0x00007fca89f77834 in arrow::NumericArray<arrow::Int64Type>::GetView (this=0x0, i=0)
    at /usr/src/debug/dev-libs/apache-arrow-21.0.0/apache-arrow-21.0.0/cpp/src/arrow/array/array_primitive.h:121
#1  arrow::compute::internal::ResolvedChunk::Value<arrow::Int64Type, arrow::compute::internal::GetViewType<arrow::Int64Type, void> > (
    this=this@entry=0x7ffc548ca930)
    at /usr/src/debug/dev-libs/apache-arrow-21.0.0/apache-arrow-21.0.0/cpp/src/arrow/compute/kernels/chunked_internal.h:56
#2  0x00007fca8a0bd3f5 in operator() (__closure=__closure@entry=0x7ffc548ca9c0, left=..., right=...)
    at /usr/src/debug/dev-libs/apache-arrow-21.0.0/apache-arrow-21.0.0/cpp/src/arrow/compute/kernels/vector_sort.cc:849
#3  0x00007fca8a0bd5b8 in __gnu_cxx::__ops::_Iter_comp_iter<arrow::compute::internal::(anonymous namespace)::TableSorter::MergeNonNulls<arrow::Int64Type>(arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*)::<lambda(arrow::compute::internal::CompressedChunkLocation, arrow::compute::internal::CompressedChunkLocation)> >::operator()<arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*> (this=0x7ffc548ca9c0, __it1=0x55dac2821588, __it2=0x55dac2821580)
    at /usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/predefined_ops.h:158
#4  std::__merge<arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, __gnu_cxx::__ops::_Iter_comp_iter<arrow::compute::internal::(anonymous namespace)::TableSorter::MergeNonNulls<arrow::Int64Type>(arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*)::<lambda(arrow::compute::internal::CompressedChunkLocation, arrow::compute::internal::CompressedChunkLocation)> > >(arrow::compute::internal::CompressedChunkLocation *, arrow::compute::internal::CompressedChunkLocation *, arrow::compute::internal::CompressedChunkLocation *, arrow::compute::internal::CompressedChunkLocation *, arrow::compute::internal::CompressedChunkLocation *, __gnu_cxx::__ops::_Iter_comp_iter<arrow::compute::internal::(anonymous namespace)::TableSorter::MergeNonNulls<arrow::Int64Type>(arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*)::<lambda(arrow::compute::internal::CompressedChunkLocation, arrow::compute::internal::CompressedChunkLocation)> >) (
    __first1=__first1@entry=0x55dac2821580, __last1=0x55dac2821588, __first2=0x55dac2821588, __last2=__last2@entry=0x55dac2821590, 
    __result=__result@entry=0x55dac2a6b4c0, __comp=...) at /usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_algo.h:4887
#5  0x00007fca8a0bd5ff in std::merge<arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::(anonymous namespace)::TableSorter::MergeNonNulls<arrow::Int64Type>(arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*)::<lambda(arrow::compute::internal::CompressedChunkLocation, arrow::compute::internal::CompressedChunkLocation)> >(arrow::compute::internal::CompressedChunkLocation *, arrow::compute::internal::CompressedChunkLocation *, arrow::compute::internal::CompressedChunkLocation *, arrow::compute::internal::CompressedChunkLocation *, arrow::compute::internal::CompressedChunkLocation *, struct {...}) (__first1=__first1@entry=0x55dac2821580, 
    __last1=<optimized out>, __first2=<optimized out>, __last2=__last2@entry=0x55dac2821590, __result=__result@entry=0x55dac2a6b4c0, 
    __comp=...) at /usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/predefined_ops.h:150
#6  0x00007fca8a0bd62c in arrow::compute::internal::(anonymous namespace)::TableSorter::MergeNonNulls<arrow::Int64Type> (
    this=<optimized out>, range_begin=0x55dac2821580, range_middle=<optimized out>, range_end=0x55dac2821590, 
    temp_indices=0x55dac2a6b4c0)
    at /usr/src/debug/dev-libs/apache-arrow-21.0.0/apache-arrow-21.0.0/cpp/src/arrow/compute/kernels/vector_sort.cc:840
#7  0x00007fca8a0bd664 in operator() (__closure=<optimized out>, range_begin=<optimized out>, range_middle=<optimized out>, 
    range_end=<optimized out>, temp_indices=<optimized out>)
    at /usr/src/debug/dev-libs/apache-arrow-21.0.0/apache-arrow-21.0.0/cpp/src/arrow/compute/kernels/vector_sort.cc:756
#8  std::__invoke_impl<void, arrow::compute::internal::(anonymous namespace)::TableSorter::MergeInternal<arrow::Int64Type>(std::vector<arrow::compute::internal::GenericNullPartitionResult<arrow::compute::internal::CompressedChunkLocation> >*, int64_t)::<lambda(arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*)>&, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*> (__f=...) at /usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63
#9  std::__invoke_r<void, arrow::compute::internal::(anonymous namespace)::TableSorter::MergeInternal<arrow::Int64Type>(std::vector<arrow::compute::internal::GenericNullPartitionResult<arrow::compute::internal::CompressedChunkLocation> >*, int64_t)::<lambda(arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*)>&, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*> (__fn=...) at /usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:113
#10 std::_Function_handler<void(arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*), arrow::compute::internal::(anonymous namespace)::TableSorter::MergeInternal<arrow::Int64Type>(std::vector<arrow::compute::internal::GenericNullPartitionResult<arrow::compute::internal::CompressedChunkLocation> >*, int64_t)::<lambda(arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*)> >::_M_invoke(const std::_Any_data &, arrow::compute::internal::CompressedChunkLocation *&&, arrow::compute::internal::CompressedChunkLocation *&&, arrow::compute::internal::CompressedChunkLocation *&&, arrow::compute::internal::CompressedChunkLocation *&&) (
    __functor=..., __args#0=<optimized out>, __args#1=<optimized out>, __args#2=<optimized out>, __args#3=<optimized out>)
    at /usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292
#11 0x00007fca8a0c43b3 in std::function<void(arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*, arrow::compute::internal::CompressedChunkLocation*)>::operator() (
    this=this@entry=0x7ffc548cab78, __args#0=<optimized out>, __args#1=<optimized out>, __args#2=<optimized out>, 
    __args#3=<optimized out>) at /usr/lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593
#12 0x00007fca8a0cb247 in arrow::compute::internal::GenericMergeImpl<arrow::compute::internal::CompressedChunkLocation, arrow::compute::internal::GenericNullPartitionResult<arrow::compute::internal::CompressedChunkLocation> >::MergeNullsAtEnd (this=0x7ffc548cab50, 
    left=..., right=..., null_count=0)
    at /usr/src/debug/dev-libs/apache-arrow-21.0.0/apache-arrow-21.0.0/cpp/src/arrow/compute/kernels/vector_sort_internal.h:441
#13 0x00007fca8a0cb53a in arrow::compute::internal::GenericMergeImpl<arrow::compute::internal::CompressedChunkLocation, arrow::compute:--Type <RET> for more, q to quit, c to continue without paging--
:internal::GenericNullPartitionResult<arrow::compute::internal::CompressedChunkLocation> >::Merge (this=this@entry=0x7ffc548cab50, 
    left=..., right=..., null_count=null_count@entry=0)
    at /usr/src/debug/dev-libs/apache-arrow-21.0.0/apache-arrow-21.0.0/cpp/src/arrow/compute/kernels/vector_sort_internal.h:373
#14 0x00007fca8a0bd296 in arrow::compute::internal::(anonymous namespace)::TableSorter::MergeInternal<arrow::Int64Type> (
    this=0x7ffc548cae70, sorted=0x7ffc548cac90, null_count=0)
    at /usr/src/debug/dev-libs/apache-arrow-21.0.0/apache-arrow-21.0.0/cpp/src/arrow/compute/kernels/vector_sort.cc:770

#15 0x00007fca8a0c0a61 in Visitor::Visit (this=0x7ffc548cacb0, type=...)
    at /usr/src/debug/dev-libs/apache-arrow-21.0.0/apache-arrow-21.0.0/cpp/src/arrow/compute/kernels/vector_sort.cc:721
#16 arrow::VisitTypeInline<arrow::compute::internal::(anonymous namespace)::TableSorter::SortInternal()::Visitor> (type=..., 
    visitor=visitor@entry=0x7ffc548cacb0)
    at /usr/src/debug/dev-libs/apache-arrow-21.0.0/apache-arrow-21.0.0/cpp/src/arrow/visit_type_inline.h:55
#17 0x00007fca8a0c1368 in arrow::compute::internal::(anonymous namespace)::TableSorter::SortInternal (this=this@entry=0x7ffc548cae70)
    at /usr/src/debug/dev-libs/apache-arrow-21.0.0/apache-arrow-21.0.0/cpp/src/arrow/compute/kernels/vector_sort.cc:731
#18 0x00007fca8a0c1755 in arrow::compute::internal::(anonymous namespace)::TableSorter::Sort (this=this@entry=0x7ffc548cae70)
    at /usr/src/debug/dev-libs/apache-arrow-21.0.0/apache-arrow-21.0.0/cpp/src/arrow/compute/kernels/vector_sort.cc:648

#19 0x00007fca8a0c1aa9 in arrow::compute::internal::(anonymous namespace)::SortIndicesMetaFunction::SortIndices (
    this=this@entry=0x55dac2665fc0, table=..., options=..., ctx=ctx@entry=0x7ffc548cb0a0)
    at /usr/src/debug/dev-libs/apache-arrow-21.0.0/apache-arrow-21.0.0/cpp/src/arrow/compute/kernels/vector_sort.cc:1070

#20 0x00007fca8a0c1daa in arrow::compute::internal::(anonymous namespace)::SortIndicesMetaFunction::ExecuteImpl (this=0x55dac2665fc0, 
    args=..., options=0x55dac2b9c0e0, ctx=0x7ffc548cb0a0)
    at /usr/src/debug/dev-libs/apache-arrow-21.0.0/apache-arrow-21.0.0/cpp/src/arrow/compute/kernels/vector_sort.cc:941
#21 0x00007fca8b5f3221 in arrow::compute::MetaFunction::Execute (this=0x55dac2665fc0, 
    args=std::vector of length 1, capacity 1 = {...}, options=0x55dac2b9c0e0, ctx=0x7ffc548cb0a0)
    at /usr/src/debug/dev-libs/apache-arrow-21.0.0/apache-arrow-21.0.0/cpp/src/arrow/compute/function.cc:483
#22 0x00007fca836abf45 in __pyx_pf_7pyarrow_8_compute_8Function_6call (__pyx_v_self=0x7fca83593670, __pyx_v_args=<optimized out>, 
    __pyx_v_options=<optimized out>, __pyx_v_memory_pool=<optimized out>, __pyx_v_length=<optimized out>)
    at /usr/src/debug/dev-python/pyarrow-21.0.0/apache-arrow-21.0.0/python-python3_12/build0/temp.linux-x86_64-cpython-312/_compute.cpp:19602

#23 __pyx_pw_7pyarrow_8_compute_8Function_7call (__pyx_v_self=<pyarrow._compute.MetaFunction at remote 0x7fca83593670>, 
    __pyx_args=<optimized out>, __pyx_nargs=<optimized out>, __pyx_kwds=<optimized out>)
    at /usr/src/debug/dev-python/pyarrow-21.0.0/apache-arrow-21.0.0/python-python3_12/build0/temp.linux-x86_64-cpython-312/_compute.cpp:19389
#24 0x00007fca8da1817e in PyObject_Vectorcall () from /usr/lib64/libpython3.12.so.1.0
#25 0x00007fca8da6dc26 in _PyEval_EvalFrameDefault () from /usr/lib64/libpython3.12.so.1.0
#26 0x00007fca8da1801b in PyObject_VectorcallMethod () from /usr/lib64/libpython3.12.so.1.0
#27 0x00007fca8c3e65f1 in __pyx_pf_7pyarrow_3lib_8_Tabular_33sort_by (__pyx_v_self=0x7fca7bfcc130, 
    __pyx_v_sorting=[('date', 'ascending'), ('variable', 'ascending')], __pyx_v_kwargs=Python Exception <class 'gdb.error'>: There is no member named dk_nentries.
)
    at /usr/src/debug/dev-python/pyarrow-21.0.0/apache-arrow-21.0.0/python-python3_12/build0/temp.linux-x86_64-cpython-312/lib.cpp:168993
#28 __pyx_pw_7pyarrow_3lib_8_Tabular_34sort_by (__pyx_v_self=<pyarrow.lib.Table at remote 0x7fca7bfcc130>, 
    __pyx_args=<optimized out>, __pyx_nargs=<optimized out>, __pyx_kwds=<optimized out>)
    at /usr/src/debug/dev-python/pyarrow-21.0.0/apache-arrow-21.0.0/python-python3_12/build0/temp.linux-x86_64-cpython-312/lib.cpp:168827
#29 0x00007fca8da1817e in PyObject_Vectorcall () from /usr/lib64/libpython3.12.so.1.0
#30 0x00007fca8da6dc26 in _PyEval_EvalFrameDefault () from /usr/lib64/libpython3.12.so.1.0
#31 0x00007fca8db1bb5a in PyEval_EvalCode () from /usr/lib64/libpython3.12.so.1.0
#32 0x00007fca8db40630 in ?? () from /usr/lib64/libpython3.12.so.1.0
#33 0x00007fca8db405a7 in ?? () from /usr/lib64/libpython3.12.so.1.0
#34 0x00007fca8db41d51 in ?? () from /usr/lib64/libpython3.12.so.1.0
#35 0x00007fca8db41af8 in _PyRun_SimpleFileObject () from /usr/lib64/libpython3.12.so.1.0
#36 0x00007fca8db41918 in _PyRun_AnyFileObject () from /usr/lib64/libpython3.12.so.1.0
#37 0x00007fca8db4da25 in Py_RunMain () from /usr/lib64/libpython3.12.so.1.0
#38 0x00007fca8db4d40a in Py_BytesMain () from /usr/lib64/libpython3.12.so.1.0
#39 0x00007fca8d6294ae in ?? () from /usr/lib64/libc.so.6
#40 0x00007fca8d629569 in __libc_start_main () from /usr/lib64/libc.so.6
#41 0x000055daabdc2095 in _start ()

This is Gentoo Linux amd64. Curious enough, I can't reproduce this with PyArrow installed from PyPI wheels.

Build logs for the build I've used to get backtraces:

• C++: dev-libs:apache-arrow-21.0.0:20250801-133657.log
• Python: dev-python:pyarrow-21.0.0:20250801-134715.log

Component(s)

Python, C++

[pitrou]

@mgorny Sorry that no-one answered here before. Can you still reproduce using a more up-to-date version of the git repo?

[mgorny]

I'm building the git versions right now, and I'll get back to you when I test them again.

[mgorny]

Yes, I can reproduce on top of 64cfa7c.

[pitrou]

I have a debug build of git main and I can't reproduce:

>>> import pyarrow as pa
...: from datetime import datetime
...: 
...: tables = [
...:     pa.Table.from_pydict(
...:         {
...:             "date": [datetime(2020, 1, 1)],
...:             "variable": ["aapl"],
...:             "price": [110],
...:         }
...:     ),
...:     pa.Table.from_pydict(
...:         {
...:             "date": [datetime(2020, 1, 1)],
...:             "variable": ["tlsa"],
...:             "price": [220],
...:         }
...:     ),
...: ]

>>> unpivoted = pa.concat_tables(tables)
>>> unpivoted
pyarrow.Table
date: timestamp[us]
variable: string
price: int64
----
date: [[2020-01-01 00:00:00.000000],[2020-01-01 00:00:00.000000]]
variable: [["aapl"],["tlsa"]]
price: [[110],[220]]

>>> unpivoted.sort_by([("date", "ascending"), ("variable", "ascending")])
pyarrow.Table
date: timestamp[us]
variable: string
price: int64
----
date: [[2020-01-01 00:00:00.000000,2020-01-01 00:00:00.000000]]
variable: [["aapl","tlsa"]]
price: [[110,220]]

[pitrou]

Perhaps you have another version of Arrow installed somewhere and it gets picked up by the dynamic linker?

[mgorny]

Ah, sorry, I should've probably linked the original misfiled bug report: narwhals-dev/narwhals#2921

It's definitely not trivial to reproduce, but it definitely happens on a clean Gentoo container. To simplify/update the instructions from the other bug:

docker run -it gentoo/stage3

Inside it:

# grab the today's repository snapshot
emerge-webrsync
# update post-snapshot
emerge --sync
# set up the gpg binpkg verification
getuto
# to test the git version (this will work once mirrors sync up)
printf 'dev-libs/apache-arrow **\ndev-python/pyarrow **\n' >> /etc/portage/package.accept_keywords/local.conf
# build and install it
emerge -1vg --jobs pyarrow

I've pushed the git ebuilds today, so they aren't on all mirrors yet — so this will use 21.0.0 if you're not lucky, but I can reliably reproduce it against all versions.

I can try digging deeper but I'd use some suggestions on what to look at.

[pitrou]

If possible, I would suggest finding an older version that works and then bisecting to find out if any particular commit on git main is responsible.

[thesamesam]

mgorny, could you get me a dockerfile which doesn't reproduce it (ideally some modern but non-Gentoo base, fedora rawhide or something) and I'll take a look?

[mgorny]

mgorny, could you get me a dockerfile which doesn't reproduce it (ideally some modern but non-Gentoo base, fedora rawhide or something) and I'll take a look?

As in docker run -it fedora, then dnf install python-pyarrow? (Yes, it doesn't happen there.)

[pitrou]

It's definitely not trivial to reproduce, but it definitely happens on a clean Gentoo container. To simplify/update the instructions from the other bug:

I've tried this and can reproduce the bug, the problem is that debug symbols are absent:

(gdb) bt
#0  0x00007ddf634af991 in ?? () from /usr/lib64/libarrow_compute.so.2200
#1  0x00007ddf634bb96e in arrow::compute::internal::GenericMergeImpl<arrow::compute::internal::CompressedChunkLocation, arrow::compute::internal::GenericNullPartitionResult<arrow::compute::internal::CompressedChunkLocation> >::MergeNullsAtEnd(arrow::compute::internal::GenericNullPartitionResult<arrow::compute::internal::CompressedChunkLocation> const&, arrow::compute::internal::GenericNullPartitionResult<arrow::compute::internal::CompressedChunkLocation> const&, long) const ()
   from /usr/lib64/libarrow_compute.so.2200
#2  0x00007ddf634a38c4 in ?? () from /usr/lib64/libarrow_compute.so.2200
#3  0x00007ddf634aa41c in ?? () from /usr/lib64/libarrow_compute.so.2200
#4  0x00007ddf634abcde in ?? () from /usr/lib64/libarrow_compute.so.2200
#5  0x00007ddf634ad302 in ?? () from /usr/lib64/libarrow_compute.so.2200
#6  0x00007ddf641a6588 in arrow::compute::MetaFunction::Execute(std::vector<arrow::Datum, std::allocator<arrow::Datum> > const&, arrow::compute::FunctionOptions const*, arrow::compute::ExecContext*) const () from /usr/lib64/libarrow.so.2200
#7  0x00007ddf6087c14d in ?? () from /usr/lib/python3.13/site-packages/pyarrow/_compute.cpython-313-x86_64-linux-gnu.so
#8  0x00007ddf65760053 in PyObject_Vectorcall () from /usr/lib64/libpython3.13.so.1.0
#9  0x00007ddf658826da in _PyEval_EvalFrameDefault () from /usr/lib64/libpython3.13.so.1.0
#10 0x00007ddf6576012e in PyObject_VectorcallMethod () from /usr/lib64/libpython3.13.so.1.0
#11 0x00007ddf64d1edb6 in ?? () from /usr/lib/python3.13/site-packages/pyarrow/lib.cpython-313-x86_64-linux-gnu.so
#12 0x00007ddf65760053 in PyObject_Vectorcall () from /usr/lib64/libpython3.13.so.1.0
#13 0x00007ddf658826da in _PyEval_EvalFrameDefault () from /usr/lib64/libpython3.13.so.1.0
#14 0x00007ddf6588b5b1 in PyEval_EvalCode () from /usr/lib64/libpython3.13.so.1.0
#15 0x00007ddf658e76fe in ?? () from /usr/lib64/libpython3.13.so.1.0
#16 0x00007ddf658e7d37 in ?? () from /usr/lib64/libpython3.13.so.1.0
#17 0x00007ddf658e815c in ?? () from /usr/lib64/libpython3.13.so.1.0
#18 0x00007ddf658ea7ee in ?? () from /usr/lib64/libpython3.13.so.1.0
#19 0x00007ddf6591110b in Py_RunMain () from /usr/lib64/libpython3.13.so.1.0
#20 0x00007ddf6591210d in Py_BytesMain () from /usr/lib64/libpython3.13.so.1.0
#21 0x00007ddf6550116e in ?? () from /usr/lib64/libc.so.6
#22 0x00007ddf65501229 in __libc_start_main () from /usr/lib64/libc.so.6
#23 0x000057bb00344095 in _start ()

[mgorny]

I've tried this and can reproduce the bug, the problem is that debug symbols are absent

Ah, sorry. You're going to need to install debugedit first:

emerge -1vg --jobs debugedit

You can set flags prior to compiling Arrow, e.g.:

export CFLAGS='-Og -g -DXXH_NO_INLINE_HINTS=1'
export CXXFLAGS='-Og -g -DXXH_NO_INLINE_HINTS=1'
# install split debug files and sources for gdb
export FEATURES="splitdebug installsources"

If you have them installed already, you can recompile by doing:

emerge -1v apache-arrow pyarrow

[mgorny]

If possible, I would suggest finding an older version that works and then bisecting to find out if any particular commit on git main is responsible.

I've been able to test back to apache-arrow-11.0.0.dev (7f6c5ae), and it also fails there. The older versions don't build cleanly for me, and I don't think there's a point in trying to fix that anyway, given we're already 3 years back — I think it's safe to say the problem always (potentially) existed.

We haven't hit it before, because a new test case in narwhals triggered it.