add ExecutionPlan::dynamic_filters() method

[jayshrivastava]

Which issue does this PR close?

• Closes: make dynamic filters in ExecutionPlan nodes discoverable #23814
• Closes: [dynamic filtering] 0. make dynamic filters discoverable in upstream datafusion datafusion-contrib/datafusion-distributed#584

Rationale for this change

External users who wish to propagate dynamic filter updates across network boundaries need to know in which direction updates need to flow. Thus ExecutionPlan::apply_expressions is not enough. The proposal in #23814 is to provide a separate API for dynamic filter producers.

What changes are included in this PR?

This change adds a new method ExecutionPlan::dynamic_expressions(&self) -> Vec<Arc<dyn PhysicalExpr>> which should return dynamic filters produced by the ExecutionPlan.

Are these changes tested?

Yes. Adds a new invariant that all expressions returned by ExecutionPlan::dynamic_expressions have an expression id.

Are there any user-facing changes?

There's a new API, ExecutionPlan::dynamic_expressions(&self) -> Vec<Arc<dyn PhysicalExpr>>

[github-actions]

Thank you for opening this pull request!

Reviewer note: cargo-semver-checks reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch).

Details

     Cloning apache/main
    Building datafusion v54.1.0 (current)
       Built [ 107.055s] (current)
     Parsing datafusion v54.1.0 (current)
      Parsed [   0.038s] (current)
    Building datafusion v54.1.0 (baseline)
       Built [ 106.741s] (baseline)
     Parsing datafusion v54.1.0 (baseline)
      Parsed [   0.037s] (baseline)
    Checking datafusion v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.933s] 223 checks: 223 pass, 30 skip
     Summary no semver update required
    Finished [ 216.683s] datafusion
    Building datafusion-ffi v54.1.0 (current)
       Built [  58.926s] (current)
     Parsing datafusion-ffi v54.1.0 (current)
      Parsed [   0.062s] (current)
    Building datafusion-ffi v54.1.0 (baseline)
       Built [  58.703s] (baseline)
     Parsing datafusion-ffi v54.1.0 (baseline)
      Parsed [   0.065s] (baseline)
    Checking datafusion-ffi v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.338s] 223 checks: 221 pass, 1 fail, 1 warn, 30 skip

--- failure constructible_struct_adds_field: externally-constructible struct adds field ---

Description:
A pub struct constructible with a struct literal has a new pub field. Existing struct literals must be updated to include the new field.
        ref: https://doc.rust-lang.org/reference/expressions/struct-expr.html
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/constructible_struct_adds_field.ron

Failed in:
  field FFI_ExecutionPlan.dynamic_expressions in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/execution_plan.rs:55
  field ForeignLibraryModule.create_exec_with_dynamic_expressions in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/tests/mod.rs:112

--- warning repr_c_plain_struct_fields_reordered: struct fields reordered in repr(C) struct ---

Description:
A public repr(C) struct had its fields reordered. This can change the struct's memory layout, possibly breaking FFI use cases that depend on field position and order.
        ref: https://doc.rust-lang.org/reference/type-layout.html#reprc-structs
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/repr_c_plain_struct_fields_reordered.ron

Failed in:
  FFI_ExecutionPlan.with_new_children moved from position 3 to 4, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/execution_plan.rs:57
  FFI_ExecutionPlan.name moved from position 4 to 5, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/execution_plan.rs:61
  FFI_ExecutionPlan.execute moved from position 5 to 6, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/execution_plan.rs:65
  FFI_ExecutionPlan.repartitioned moved from position 6 to 7, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/execution_plan.rs:71
  FFI_ExecutionPlan.metrics moved from position 7 to 8, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/execution_plan.rs:80
  FFI_ExecutionPlan.partition_statistics moved from position 8 to 9, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/execution_plan.rs:86
  FFI_ExecutionPlan.clone moved from position 9 to 10, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/execution_plan.rs:93
  FFI_ExecutionPlan.release moved from position 10 to 11, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/execution_plan.rs:96
  FFI_ExecutionPlan.private_data moved from position 11 to 12, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/execution_plan.rs:100
  FFI_ExecutionPlan.library_marker_id moved from position 12 to 13, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/execution_plan.rs:105
  ForeignLibraryModule.create_exec_with_statistics moved from position 15 to 16, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/tests/mod.rs:114
  ForeignLibraryModule.create_table_with_statistics moved from position 16 to 17, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/tests/mod.rs:116
  ForeignLibraryModule.create_physical_optimizer_rule moved from position 17 to 18, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/tests/mod.rs:119
  ForeignLibraryModule.create_context_aware_optimizer_rule moved from position 18 to 19, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/tests/mod.rs:121
  ForeignLibraryModule.version moved from position 19 to 20, in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/tests/mod.rs:123

     Summary semver requires new major version: 1 major and 0 minor checks failed
     Warning produced 1 major and 0 minor level warnings
    Finished [ 119.341s] datafusion-ffi
    Building datafusion-physical-plan v54.1.0 (current)
       Built [  36.179s] (current)
     Parsing datafusion-physical-plan v54.1.0 (current)
      Parsed [   0.144s] (current)
    Building datafusion-physical-plan v54.1.0 (baseline)
       Built [  36.112s] (baseline)
     Parsing datafusion-physical-plan v54.1.0 (baseline)
      Parsed [   0.148s] (baseline)
    Checking datafusion-physical-plan v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.956s] 223 checks: 223 pass, 30 skip
     Summary no semver update required
    Finished [  74.720s] datafusion-physical-plan