Skip to content

[SPARK-56413][UDF][BUILD][FOLLOWUP] Fix parent POM version in udf-worker-grpc module on branch-4.x#56325

Closed
cloud-fan wants to merge 1 commit into
apache:branch-4.xfrom
cloud-fan:SPARK-56413-fix-grpc-pom-version-branch-4.x
Closed

[SPARK-56413][UDF][BUILD][FOLLOWUP] Fix parent POM version in udf-worker-grpc module on branch-4.x#56325
cloud-fan wants to merge 1 commit into
apache:branch-4.xfrom
cloud-fan:SPARK-56413-fix-grpc-pom-version-branch-4.x

Conversation

@cloud-fan
Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

When the udf-worker-grpc module was backported to branch-4.x in SPARK-56413/SPARK-56661 (commit 7a70689), its parent POM version in udf/worker/grpc/pom.xml was left at 5.0.0-SNAPSHOT (the master version) instead of 4.2.0-SNAPSHOT. This one-line change corrects it.

Why are the changes needed?

branch-4.x is at 4.2.0-SNAPSHOT, but udf/worker/grpc/pom.xml referenced parent spark-parent_2.13:5.0.0-SNAPSHOT, which cannot be resolved. Since the module is registered in the root POM <modules>, POM model building fails for every job, breaking the entire branch-4.x build:

[FATAL] Non-resolvable parent POM for org.apache.spark:spark-udf-worker-grpc_2.13:5.0.0-SNAPSHOT:
Could not find artifact org.apache.spark:spark-parent_2.13:pom:5.0.0-SNAPSHOT in central
... and 'parent.relativePath' points at wrong local POM @ line 24, column 11

The sibling modules udf/worker/core and udf/worker/proto already correctly use 4.2.0-SNAPSHOT.

Does this PR introduce any user-facing change?

No. Build-only fix on the unreleased branch-4.x.

How was this patch tested?

Existing CI. This unblocks all branch-4.x builds, which currently fail at the POM model building step before any compilation/tests run.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Opus 4.8)

This pull request and its description were written by Isaac.

…ker-grpc module on branch-4.x

### What changes were proposed in this pull request?

When the `udf-worker-grpc` module was backported to `branch-4.x` in SPARK-56413/SPARK-56661, its parent POM version was left at `5.0.0-SNAPSHOT` (the `master` version) instead of `4.2.0-SNAPSHOT`. This corrects the parent version to match the rest of `branch-4.x`.

### Why are the changes needed?

`branch-4.x` is at `4.2.0-SNAPSHOT`, but `udf/worker/grpc/pom.xml` referenced parent `spark-parent_2.13:5.0.0-SNAPSHOT`, which cannot be resolved. Since the module is registered in the root POM `<modules>`, the build fails at POM model building for every job, breaking the entire `branch-4.x` build:

```
[FATAL] Non-resolvable parent POM for org.apache.spark:spark-udf-worker-grpc_2.13:5.0.0-SNAPSHOT:
Could not find artifact org.apache.spark:spark-parent_2.13:pom:5.0.0-SNAPSHOT
```

The sibling modules `udf/worker/core` and `udf/worker/proto` already correctly use `4.2.0-SNAPSHOT`.

Co-authored-by: Isaac
@gaogaotiantian
Copy link
Copy Markdown
Contributor

So we need to upgrade this to 4.3 after we branch 4.3? Or this should be 4.x and we change the cut branch to 4.3?

@cloud-fan
Copy link
Copy Markdown
Contributor Author

This follows the same convention as every other POM - they all hardcode the version, and the sibling udf/worker/core and udf/worker/proto modules are already at 4.2.0-SNAPSHOT. When 4.3 is cut, the standard version-bump tooling rewrites all POMs together, so this gets updated automatically along with everything else. The issue here was just that the backport left this one file at the master version (5.0.0-SNAPSHOT) instead of matching the rest of the branch.

@cloud-fan
Copy link
Copy Markdown
Contributor Author

also cc @haiyangsun-db @HyukjinKwon

Comment thread udf/worker/grpc/pom.xml
<groupId>org.apache.spark</groupId>
<artifactId>spark-parent_2.13</artifactId>
<version>5.0.0-SNAPSHOT</version>
<version>4.2.0-SNAPSHOT</version>
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be 4.3.0 in branch-4.x?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for fixing, I think @zhengruifeng is right that this was newly introduced recently in spark 4.x (after 4.2 cut).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parent version has to match the root pom.xml, and branch-4.x is currently on 4.2.0-SNAPSHOT (not 4.3.0 yet), so 4.2.0-SNAPSHOT is the correct value here. This was the actual bug — the module had 5.0.0-SNAPSHOT (master's version) leaking onto the branch.

That said, you have a point that the branch version looks stale — we should likely bump branch-4.x to 4.3.0-SNAPSHOT in a separate change. But that's a branch-wide version bump (root POM + all modules), so it's out of scope for this fix.

Copy link
Copy Markdown
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM.

@cloud-fan
Copy link
Copy Markdown
Contributor Author

thanks for review, merging to 4.x!

cloud-fan added a commit that referenced this pull request Jun 5, 2026
…ker-grpc module on branch-4.x

### What changes were proposed in this pull request?

When the `udf-worker-grpc` module was backported to `branch-4.x` in [SPARK-56413](https://issues.apache.org/jira/browse/SPARK-56413)/SPARK-56661 (commit 7a70689), its parent POM version in `udf/worker/grpc/pom.xml` was left at `5.0.0-SNAPSHOT` (the `master` version) instead of `4.2.0-SNAPSHOT`. This one-line change corrects it.

### Why are the changes needed?

`branch-4.x` is at `4.2.0-SNAPSHOT`, but `udf/worker/grpc/pom.xml` referenced parent `spark-parent_2.13:5.0.0-SNAPSHOT`, which cannot be resolved. Since the module is registered in the root POM `<modules>`, POM model building fails for every job, breaking the **entire** `branch-4.x` build:

```
[FATAL] Non-resolvable parent POM for org.apache.spark:spark-udf-worker-grpc_2.13:5.0.0-SNAPSHOT:
Could not find artifact org.apache.spark:spark-parent_2.13:pom:5.0.0-SNAPSHOT in central
... and 'parent.relativePath' points at wrong local POM  line 24, column 11
```

The sibling modules `udf/worker/core` and `udf/worker/proto` already correctly use `4.2.0-SNAPSHOT`.

### Does this PR introduce _any_ user-facing change?

No. Build-only fix on the unreleased `branch-4.x`.

### How was this patch tested?

Existing CI. This unblocks all `branch-4.x` builds, which currently fail at the POM model building step before any compilation/tests run.

### Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Opus 4.8)

This pull request and its description were written by Isaac.

Closes #56325 from cloud-fan/SPARK-56413-fix-grpc-pom-version-branch-4.x.

Authored-by: Wenchen Fan <wenchen@databricks.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
@cloud-fan cloud-fan closed this Jun 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants