Map view types when converting arrow_expressions to vortex expressions

[paultiq]

Fixes #5759

This PR maps string_views to strings and binary_views to binary so that to_substrait will no longer raise ArrowNotImplementedError when constructing the substrait.

Vortex supports expressions over views and Arrow compute doesn't, but to_substrait raises ArrowNotImplementedError based on Arrow compute kernels... regardless of the backend.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.95%. Comparing base (41bb2bf) to head (efdf7fe).
⚠️ Report is 1 commits behind head on develop.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[danking]

This is great and we should merge! Can you follow up with a test that writes and reads through DuckDB a Vortex file containing strings? I think paultiq included one at one point in one of his GitHub issues.

[paultiq]

@danking Do you want that test case in this PR or a separate? Happy to do either.

[danking]

Separate is fine! Sorry I referred to you in the third person! GitHub mobile UI had me turned around haha

[joseph-isaacs]

@paultiq think we can get this over the line

[paultiq]

@paultiq think we can get this over the line

AFAIK, this is ready to merge, was there something else for this PR?

[joseph-isaacs]

The CI / Python lint is failing, we need to fix that

[joseph-isaacs]

Why did you ignore all the warnings?

[paultiq]

Why did you ignore all the warnings?

The root reason is lack of type stubs for pyarrow, which is being addressed in apache/arrow#32609

The ignores are consistent with the existing code:

vortex/vortex-python/python/vortex/arrow/expression.py

Lines 32 to 35 in 41bb2bf

	substrait_object = ExtendedExpression() # pyright: ignore[reportUnknownVariableType]
	substrait_object.ParseFromString(arrow_expression.to_substrait(schema)) # pyright: ignore[reportUnknownMemberType]
	expressions = extended_expression(substrait_object) # pyright: ignore[reportUnknownArgumentType]

And existing test cases:

vortex/vortex-python/benchmark/test_aggregation.py

Lines 23 to 24 in 41bb2bf

	for field in vxf.dtype.to_arrow_schema() # pyright: ignore[reportUnknownVariableType]
	if _has_mean(field.type) # pyright: ignore[reportUnknownMemberType, reportUnknownArgumentType]

(I'll fix that DCO in a sec...)

[paultiq]

The one question would be whether the direct test of _schema_for_substrait is OK (hence the reportPrivateUsage).
reportPrivateUsage is used in a few places in the project, although not in any test cases, so that'd be a judgement call.

test_expression.py tests both paths: the "private" _schema_for_substrait, as well as the "public" arrow_to_vortex path.

So, the choice would be either to allow the ignore: reportPrivateUsage or drop the direct test of _schema_for_substrait.

[paultiq]

(sorry for the multiple DCO commits, just not in my normal habits)

The remaining DCO warning is for 3107e20, @danking's commit.

[danking]

Thank you @paultiq! Sorry for the friction in our process currently. I'll keep an eye on the pyarrow types issue, thanks for flagging.