[C++][Python] Table.join_asof occasionally fails in doctest

[rok]

Describe the bug, including details regarding any error messages, version, and platform.

AMD64 Conda Python 3.11 Sphinx & Numpydoc job will occasionally fail with the following:

____________________ [doctest] pyarrow.lib.Table.join_asof _____________________
5784 --------
5785 >>> import pyarrow as pa
5786 >>> t1 = pa.table({'id': [1, 3, 2, 3, 3],
5787 ...                'year': [2020, 2021, 2022, 2022, 2023]})
5788 >>> t2 = pa.table({'id': [3, 4],
5789 ...                'year': [2020, 2021],
5790 ...                'n_legs': [5, 100],
5791 ...                'animal': ["Brittle stars", "Centipede"]})
5792 
5793 >>> t1.join_asof(t2, on='year', by='id', tolerance=-2)
Differences (unified diff with -expected +actual):
    @@ -5,6 +5,6 @@
     animal: string
     ----
    -id: [[1,3,2,3,3]]
    -year: [[2020,2021,2022,2022,2023]]
    -n_legs: [[null,5,null,5,null]]
    -animal: [[null,"Brittle stars",null,"Brittle stars",null]]
    +id: [[1,3],[2,3,3]]
    +year: [[2020,2021],[2022,2022,2023]]
    +n_legs: [[null,5],[null,5,null]]
    +animal: [[null,"Brittle stars"],[null,"Brittle stars",null]]

/opt/conda/envs/arrow/lib/python3.11/site-packages/pyarrow/lib.cpython-311-x86_64-linux-gnu.so:5793: DocTestFailure
__________________ [doctest] pyarrow.lib.default_memory_pool ___________________
147 default_memory_pool()
148 
149 Return the process-global memory pool.
150 
151 Examples
152 --------
153 >>> default_memory_pool()
Expected:
    <pyarrow.MemoryPool backend_name=... bytes_allocated=0 max_memory=...>
Got:
    <pyarrow.MemoryPool backend_name=mimalloc bytes_allocated=448 max_memory=2138560>

/opt/conda/envs/arrow/lib/python3.11/site-packages/pyarrow/lib.cpython-311-x86_64-linux-gnu.so:153: DocTestFailure

Component(s)

C++, Python

[rok]

@icexelloss any idea why this would happen?

[icexelloss]

Hi Rok,

I don’t why this happens but doesn’t look like an actual failure. Looks like the test condition is just too strict, it should ignore different batch slicing when comparing result.

[rok]

Oh, I was expecting the output would be a concated table. That makes a lot of sense, we should fix the test, thanks!

[tadeja]

A more recent test failure in CI job AMD64 Conda Python 3.11 Sphinx & Numpydoc

Details

=================================== FAILURES ===================================
____________________ [doctest] pyarrow.lib.Table.join_asof _____________________
5860 --------
5861 >>> import pyarrow as pa
5862 >>> t1 = pa.table({'id': [1, 3, 2, 3, 3],
5863 ...                'year': [2020, 2021, 2022, 2022, 2023]})
5864 >>> t2 = pa.table({'id': [3, 4],
5865 ...                'year': [2020, 2021],
5866 ...                'n_legs': [5, 100],
5867 ...                'animal': ["Brittle stars", "Centipede"]})
5868 
5869 >>> t1.join_asof(t2, on='year', by='id', tolerance=-2)
Differences (unified diff with -expected +actual):
    @@ -5,6 +5,6 @@
     animal: string
     ----
    -id: [[1,3,2,3,3]]
    -year: [[2020,2021,2022,2022,2023]]
    -n_legs: [[null,5,null,5,null]]
    -animal: [[null,"Brittle stars",null,"Brittle stars",null]]
    +id: [[1,3],[2,3,3]]
    +year: [[2020,2021],[2022,2022,2023]]
    +n_legs: [[null,5],[null,5,null]]
    +animal: [[null,"Brittle stars"],[null,"Brittle stars",null]]

/opt/conda/envs/arrow/lib/python3.11/site-packages/pyarrow/lib.cpython-311-x86_64-linux-gnu.so:5869: DocTestFailure
__________________ [doctest] pyarrow.lib.default_memory_pool ___________________
147 default_memory_pool()
148 
149 Return the process-global memory pool.
150 
151 Examples
152 --------
153 >>> default_memory_pool()
Expected:
    <pyarrow.MemoryPool backend_name=... bytes_allocated=0 max_memory=...>
Got:
    <pyarrow.MemoryPool backend_name=mimalloc bytes_allocated=448 max_memory=2138688>

/opt/conda/envs/arrow/lib/python3.11/site-packages/pyarrow/lib.cpython-311-x86_64-linux-gnu.so:153: DocTestFailure

[rok]

Issue resolved by pull request 50446
#50446