[SPARK-56413][UDF][BUILD][FOLLOWUP] Fix parent POM version in udf-worker-grpc module on branch-4.x#56325
Conversation
…ker-grpc module on branch-4.x ### What changes were proposed in this pull request? When the `udf-worker-grpc` module was backported to `branch-4.x` in SPARK-56413/SPARK-56661, its parent POM version was left at `5.0.0-SNAPSHOT` (the `master` version) instead of `4.2.0-SNAPSHOT`. This corrects the parent version to match the rest of `branch-4.x`. ### Why are the changes needed? `branch-4.x` is at `4.2.0-SNAPSHOT`, but `udf/worker/grpc/pom.xml` referenced parent `spark-parent_2.13:5.0.0-SNAPSHOT`, which cannot be resolved. Since the module is registered in the root POM `<modules>`, the build fails at POM model building for every job, breaking the entire `branch-4.x` build: ``` [FATAL] Non-resolvable parent POM for org.apache.spark:spark-udf-worker-grpc_2.13:5.0.0-SNAPSHOT: Could not find artifact org.apache.spark:spark-parent_2.13:pom:5.0.0-SNAPSHOT ``` The sibling modules `udf/worker/core` and `udf/worker/proto` already correctly use `4.2.0-SNAPSHOT`. Co-authored-by: Isaac
|
So we need to upgrade this to 4.3 after we branch 4.3? Or this should be 4.x and we change the cut branch to 4.3? |
|
This follows the same convention as every other POM - they all hardcode the version, and the sibling |
|
also cc @haiyangsun-db @HyukjinKwon |
| <groupId>org.apache.spark</groupId> | ||
| <artifactId>spark-parent_2.13</artifactId> | ||
| <version>5.0.0-SNAPSHOT</version> | ||
| <version>4.2.0-SNAPSHOT</version> |
There was a problem hiding this comment.
should be 4.3.0 in branch-4.x?
There was a problem hiding this comment.
thanks for fixing, I think @zhengruifeng is right that this was newly introduced recently in spark 4.x (after 4.2 cut).
There was a problem hiding this comment.
The parent version has to match the root pom.xml, and branch-4.x is currently on 4.2.0-SNAPSHOT (not 4.3.0 yet), so 4.2.0-SNAPSHOT is the correct value here. This was the actual bug — the module had 5.0.0-SNAPSHOT (master's version) leaking onto the branch.
That said, you have a point that the branch version looks stale — we should likely bump branch-4.x to 4.3.0-SNAPSHOT in a separate change. But that's a branch-wide version bump (root POM + all modules), so it's out of scope for this fix.
|
thanks for review, merging to 4.x! |
…ker-grpc module on branch-4.x ### What changes were proposed in this pull request? When the `udf-worker-grpc` module was backported to `branch-4.x` in [SPARK-56413](https://issues.apache.org/jira/browse/SPARK-56413)/SPARK-56661 (commit 7a70689), its parent POM version in `udf/worker/grpc/pom.xml` was left at `5.0.0-SNAPSHOT` (the `master` version) instead of `4.2.0-SNAPSHOT`. This one-line change corrects it. ### Why are the changes needed? `branch-4.x` is at `4.2.0-SNAPSHOT`, but `udf/worker/grpc/pom.xml` referenced parent `spark-parent_2.13:5.0.0-SNAPSHOT`, which cannot be resolved. Since the module is registered in the root POM `<modules>`, POM model building fails for every job, breaking the **entire** `branch-4.x` build: ``` [FATAL] Non-resolvable parent POM for org.apache.spark:spark-udf-worker-grpc_2.13:5.0.0-SNAPSHOT: Could not find artifact org.apache.spark:spark-parent_2.13:pom:5.0.0-SNAPSHOT in central ... and 'parent.relativePath' points at wrong local POM line 24, column 11 ``` The sibling modules `udf/worker/core` and `udf/worker/proto` already correctly use `4.2.0-SNAPSHOT`. ### Does this PR introduce _any_ user-facing change? No. Build-only fix on the unreleased `branch-4.x`. ### How was this patch tested? Existing CI. This unblocks all `branch-4.x` builds, which currently fail at the POM model building step before any compilation/tests run. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Opus 4.8) This pull request and its description were written by Isaac. Closes #56325 from cloud-fan/SPARK-56413-fix-grpc-pom-version-branch-4.x. Authored-by: Wenchen Fan <wenchen@databricks.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com>
What changes were proposed in this pull request?
When the
udf-worker-grpcmodule was backported tobranch-4.xin SPARK-56413/SPARK-56661 (commit 7a70689), its parent POM version inudf/worker/grpc/pom.xmlwas left at5.0.0-SNAPSHOT(themasterversion) instead of4.2.0-SNAPSHOT. This one-line change corrects it.Why are the changes needed?
branch-4.xis at4.2.0-SNAPSHOT, butudf/worker/grpc/pom.xmlreferenced parentspark-parent_2.13:5.0.0-SNAPSHOT, which cannot be resolved. Since the module is registered in the root POM<modules>, POM model building fails for every job, breaking the entirebranch-4.xbuild:The sibling modules
udf/worker/coreandudf/worker/protoalready correctly use4.2.0-SNAPSHOT.Does this PR introduce any user-facing change?
No. Build-only fix on the unreleased
branch-4.x.How was this patch tested?
Existing CI. This unblocks all
branch-4.xbuilds, which currently fail at the POM model building step before any compilation/tests run.Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.8)
This pull request and its description were written by Isaac.