Skip to content

Optimizing a UTF-8 CASE WHEN into fill_null makes a valid expression fail execution #10008

Description

@mhk197

What happened?

A nullable UTF-8 array containing both a string and a null evaluates correctly with either of these expressions:

CASE WHEN is_null(x)     THEN 'fallback' ELSE x          END
CASE WHEN is_not_null(x) THEN x          ELSE 'fallback' END

optimize_recursive rewrites both to fill_null(x, 'fallback'). Binding succeeds, but execution fails because the canonical UTF-8 encoding has no fill-null kernel.

The CASE WHEN simplification checks that the fill expression is a literal, but does not check whether the input dtype has a fill-null implementation. Canonical fill-null dispatch handles bool, primitive, and decimal arrays and errors for VarBinView when the short circuits do not apply.

Steps to reproduce

use rstest::rstest;
use vortex_array::Canonical;
use vortex_array::IntoArray;
use vortex_array::VortexSessionExecute;
use vortex_array::array_session;
use vortex_array::arrays::VarBinViewArray;
use vortex_array::assert_arrays_eq;
use vortex_array::builtins::ArrayBuiltins;
use vortex_array::expr::case_when;
use vortex_array::expr::is_not_null;
use vortex_array::expr::is_null;
use vortex_array::expr::lit;
use vortex_array::expr::root;
use vortex_error::VortexResult;

#[rstest]
#[case::is_null(true)]
#[case::is_not_null(false)]
fn utf8_case_when_remains_executable_after_optimization(
    #[case] use_is_null: bool,
) -> VortexResult<()> {
    let mut ctx = array_session().create_execution_ctx();
    let input = VarBinViewArray::from_iter_nullable_str([Some("a"), None]).into_array();
    let expr = if use_is_null {
        case_when(is_null(root()), lit("fallback"), root())
    } else {
        case_when(is_not_null(root()), root(), lit("fallback"))
    };
    let original = input
        .clone()
        .apply_bound(&expr.bind(input.dtype())?)?
        .execute::<Canonical>(&mut ctx)?
        .into_array();
    assert_arrays_eq!(
        original,
        VarBinViewArray::from_iter_nullable_str([Some("a"), Some("fallback")]),
        &mut ctx
    );

    let optimized = expr.optimize_recursive(input.dtype())?;
    eprintln!("optimized: {optimized}");
    let result = input
        .clone()
        .apply_bound(&optimized.bind(input.dtype())?)?
        .execute::<Canonical>(&mut ctx)?
        .into_array();
    assert_arrays_eq!(
        result.cast(input.dtype().clone())?,
        VarBinViewArray::from_iter_nullable_str([Some("a"), Some("fallback")]),
        &mut ctx
    );
    Ok(())
}

Environment

  • Vortex Version: develop at 9e8abca
  • OS: macOS 26.6.1, Apple Silicon / aarch64-apple-darwin
  • Rust: 1.98.0

Additional context

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugA bug issue

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions