Follow-up from review of #4952 (tracking comment, item 3).
What is pinned today
The contrib-delta Maven profile pins a delta-spark version per Spark profile, in spark/pom.xml:
| Spark profile |
delta.version |
| spark-3.5 |
3.3.2 |
| spark-4.0 |
4.0.0 |
| spark-4.1 |
4.1.0 |
with a top-level default of 4.1.0 in the root pom.xml. dev/verify-contrib-delta-gate.sh
asserts the mapping holds (spark-4.1 -> delta-spark 4.1.x, spark-3.5 -> 3.x, spark-4.0 -> 4.0.x)
and fails the build gate otherwise.
The gap
The pin is a real compatibility decision — the contrib reflects into delta-spark internals
(TahoeLogFileIndex, PreparedDeltaFileIndex, TahoeBatchFileIndex, DV plumbing), so the
supported range is narrower than "any delta-spark". But that decision is currently only implicit
in the poms plus an assertion in the gate script. There is no issue recording:
- which delta-spark versions the contrib claims to support, and why those;
- what a user should expect if they run against a different delta-spark than the pinned one;
- who updates the pin when a new delta-spark ships, and what has to be re-verified (the
reflection surface is the risky part, not the wire format);
- whether the per-Spark mapping should ever become a range rather than an exact pin.
Ask
Settle the policy and write it down — in the contrib user guide and/or the profile comments — and
keep the gate script assertion as the enforcement. This is a documentation/decision issue rather
than a code change; the current pins work.
🤖 Filed with Claude Code.
Follow-up from review of #4952 (tracking comment, item 3).
What is pinned today
The
contrib-deltaMaven profile pins adelta-sparkversion per Spark profile, inspark/pom.xml:delta.versionwith a top-level default of
4.1.0in the rootpom.xml.dev/verify-contrib-delta-gate.shasserts the mapping holds (spark-4.1 -> delta-spark 4.1.x, spark-3.5 -> 3.x, spark-4.0 -> 4.0.x)
and fails the build gate otherwise.
The gap
The pin is a real compatibility decision — the contrib reflects into delta-spark internals
(
TahoeLogFileIndex,PreparedDeltaFileIndex,TahoeBatchFileIndex, DV plumbing), so thesupported range is narrower than "any delta-spark". But that decision is currently only implicit
in the poms plus an assertion in the gate script. There is no issue recording:
reflection surface is the risky part, not the wire format);
Ask
Settle the policy and write it down — in the contrib user guide and/or the profile comments — and
keep the gate script assertion as the enforcement. This is a documentation/decision issue rather
than a code change; the current pins work.
🤖 Filed with Claude Code.