[Python] Conversion to/from numpy 2.0+ new StringDType

[jorisvandenbossche]

With the upcoming numpy 2.0 release, they introduce a new variable-length string dtype, StringDType (https://numpy.org/devdocs/release/2.0.0-notes.html#highlights).

Currently trying to convert that obviously gives an error:

>>> arr = np.array(["some", "strings"], dtype=np.dtypes.StringDType())
>>> arr
array(['some', 'strings'], dtype=StringDType())
>>> pa.array(arr)
...
ArrowNotImplementedError: Unsupported numpy type 2056

But ideally we should support this dtype both for conversion from numpy -> arrow, as in the arrow -> numpy conversion (although here it should probably be opt-in and by default still return object dtype)

Numpy provides a C API to access the individual string elements: https://numpy.org/devdocs/reference/c-api/strings.html

[github-actions]

This issue has been marked as stale because it has had no activity in the past 365 days. Please remove the stale label or comment below, or this issue will be closed in 14 days. If this improvement is still desired but has no current owner, please add the 'Status: needs champion' label.

[alippai]

I believe this is not stale, numpy is continuously improving the StringDType support.

@jorisvandenbossche would this be implemented on the C++ side or cython/python?

[thisisnic]

This issue has been closed because there has been no activity for the 14 days since the stale warning. Feel free to reopen if this has been closed in error.

[alippai]

@thisisnic there was clear activity (both comment and pending MR). Is the script broken?

[thisisnic]

Doing it manually due to GitHub API limits on the CI job, and it's missed the logic for where the label has been left but someone has commented. Will reopen!

[alippai]

Thanks for keeping the issue queue tidy

[rok]

cc @ngoldbaum