fix(datafusion): report an unconvertible column statistic as absent

[jackylee-ch]

stats_set_to_df (vortex-datafusion/src/convert/stats.rs:30-66) ran each statistic through two
vortex_expects. Both sides of that conversion come from the file — the stored value out of the
footer's stats set, the dtype off the column — so both can legitimately fail on a file we did not
write:

• Stat::dtype returns None where the statistic does not apply:
  vortex-array/src/expr/stats/mod.rs:173,175 return None for Max/Min of a DType::Null
  column, and Sum::return_dtype does the same for string, binary, list and struct columns. That hit
  vortex_expect("must have a valid dtype").
• Scalar::try_new rejects a stored value that does not match the column dtype, hitting
  vortex_expect("Stat::Minsomehow had an incompatibleDType").

Either one panicked inside DataFusion planning instead of returning an error or simply forgoing the
statistic.

Fix

Forgoing it is what the end of the same chain already chose: try_to_df().ok() drops a scalar
DataFusion cannot represent. This extends that to the whole conversion, which also lets the three
near-identical blocks collapse into one helper — and is what the TODO(connor) above them
("There's a lot that can go wrong here, should probably handle this more gracefully") asked for, so
that comment goes too.

The statistic's Precision is preserved rather than flattened: an exact statistic stays
Precision::Exact, and only the unconvertible ones become Absent. That distinction is easy to lose
here, since StatsSet::get returns Vortex's Precision and not an Option.

Tests

cargo test --release -p vortex-datafusion: 341 passed, 335 before. Four cases cover the
unconvertible shapes — Min/Max of a null column, Sum of a string column, and a stored value
that disagrees with the column dtype — and two more pin that a convertible statistic keeps its exact
or inexact precision.

Restoring the vortex_expects fails all four unconvertible cases, on both panic sites
(vortex-error/src/lib.rs:666 for the Option and :659 for the Result), and leaves the
precision and distinct_count cases green.

AI assistance

Written with agentic AI assistance; my first version returned Option instead of Precision and
would have silently flattened every statistic's precision — the compiler caught it, and the two
precision cases above exist so a reviewer does not have to.

[myrrc]

LGTM, let's fix overly verbose comments

[codspeed]

Merging this PR will degrade performance by 4.25%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 3 improved benchmarks
❌ 4 regressed benchmarks
✅ 2201 untouched benchmarks
⏩ 293 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Simulation	decode_primitives[f32, (1000, 512)]	42 µs	64.1 µs	-34.49%
❌	Simulation	take_fsl_u32_random[256, 10]	124.3 µs	159.5 µs	-22.05%
❌	WallTime	dbp_assemble_kernel_avx512[(I128, 1024)]	463 ns	551 ns	-15.97%
❌	WallTime	dbp_assemble_kernel_narrow_msp_avx2[(I128, 1024)]	629 ns	700 ns	-10.14%
⚡	Simulation	take_fsl_f16_random[16, 100]	179.5 µs	119.4 µs	+50.31%
⚡	Simulation	take_fsl_nullable_random[16, 100]	188 µs	165.7 µs	+13.43%
⚡	Simulation	take_fsl_random[64, 100]	227.6 µs	202.7 µs	+12.26%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing jackylee-ch:fix/df-stats-no-panic (dc840ae) with develop (a542cbd)}

Footnotes

1. 293 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[jackylee-ch]

Trimmed the "why" from both the helper doc and the test comments.