[HSTACK] feat: support custom logical schemas passed through DeltaCon…

[cdobre]

Summary

Adds automatic detection for virtual columns — logical fields present in a caller-supplied extended schema that do not exist in the table's physical Parquet files. This is needed when a higher-level engine augments the table schema (e.g. computed or injected columns) and wants to push a single unified schema through the scan layer.

Without this, callers had to strip virtual columns from the schema before handing it to DeltaScanBuilder, or accept a runtime "column not found" error.

DeltaScanBuilder::build() now automatically identifies virtual columns by diffing the caller-supplied config.schema against the table's physical snapshot schema (via read_schema()). Any field that appears in the extended schema but is absent from both the physical schema and the partition columns is treated as virtual:

• When virtual columns are detected at top level , a ProjectionExec is wrapped around the physical file scan. Physical columns are passed through directly; virtual columns are mapped to Literal(ScalarValue::Null) cast to the declared type.
• New nullable sub-field inside an existing struct are handled in a transparent way by Arrow parquet reader schema evolution

Concerns:

• EPIC: Remove DeltaTableProvider and complete next provider migration delta-io/delta-rs#4239 - EPIC: Remove DeltaTableProvider and complete next provider migration

[github-actions]

ACTION NEEDED

delta-rs follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.