Skip to content

Add self-contained ai.chalk:chalk-java-shaded artifact (version-independent) - #200

Merged
karthikk-01 merged 2 commits into
mainfrom
feat/chalk-java-shaded
Aug 6, 2026
Merged

karthikk-01 merged 2 commits into
mainfrom
feat/chalk-java-shaded

Conversation

@karthikk-01

Copy link
Copy Markdown
Contributor

Summary

Adds a second published coordinate, ai.chalk:chalk-java-shaded — the same client + Flink connector as chalk-java, but with every third-party dependency relocated into ai.chalk.shaded.*. It's self-contained, so it works on any classpath regardless of what protobuf/grpc/jackson/arrow/netty versions a consumer (or their Flink/Spark runtime) pins.

Why

A consumer pinned below our dependency versions hits linkage errors — e.g. a client on protobuf 3.19 gets:

java.lang.NoClassDefFoundError: com/google/protobuf/MapFieldReflectionAccessor
    at ai.chalk.protos.chalk.server.v1.AuthServiceGrpc.getGetTokenMethod(...)

MapFieldReflectionAccessor is a protobuf 3.25 class our generated stubs need, but the older protobuf on their classpath wins. We can't ask every consumer to change their pinned versions, and we can't shade protobuf into the main chalk-java (its query API exposes ai.chalk.protos.* types). So this ships a separate, fully-relocated artifact for exactly these conflict-prone environments (Flink connector use is the driver — its public API exposes zero third-party types).

What changed

  • com.gradleup.shadow plugin + a shadowJar that relocates com.google (protobuf/guava/gson/flatbuffers/api-protos), io.grpc, io.netty, io.perfmark, com.fasterxml.jackson, org.apache.arrow, org.apache.commons, org.yaml.snakeyaml → ai.chalk.shaded.*, with mergeServiceFiles() so gRPC's ServiceLoader providers are rewritten too.
  • A chalk-java-shaded MavenPublication with a dependency-free POM (built manually, not from components.java) — critical: if the POM declared the deps, consumers would pull protobuf 3.25 transitively and re-hit the conflict.
  • Not relocated (intentional): org.slf4j (excluded — logging facade binds to the runtime), javax.annotation, com.github.luben (zstd-jni — native JNI, relocation breaks the .so symbol binding), org.apache.flink (compileOnly, provided by the cluster).

Verification (local)

Ran the exact failing call from the crash with protobuf 3.19.0 on the classpath:

Classpath Result
shaded jar + protobuf 3.19 ✅ PASS — getGetTokenMethod() returns ai.chalk.shaded.grpc.MethodDescriptor
unshaded chalk-java + protobuf 3.19 ❌ reproduces the MapFieldReflectionAccessor crash (proves the test is real)

Also verified: com/google/protobuf fully relocated (0 unshaded), ai.chalk.protos/ai.chalk.flink untouched, gRPC ServiceLoader files rewritten to shaded provider names, and the generated chalk-java-shaded POM is dependency-free with clean metadata.

Consume

implementation("ai.chalk:chalk-java-shaded:<version>")   // instead of chalk-java, for pinned-version envs

Needs a maintainer eye before release

I could verify the artifact + POM locally (shadowJar, generatePomFile, publishToMavenLocal), but not the actual Central Portal upload of this second coordinate — that runs in CI with signing secrets. Please confirm on first release that publishAllPublicationsToMavenCentral picks up the shaded publication (vanniktech is single-artifact-oriented; the manual publication is signed by signAllPublications() and should be bundled, but it's worth watching the first publish).

Relocates all third-party deps (protobuf, grpc, jackson, arrow, netty, ...)
into ai.chalk.shaded.* so the artifact is immune to whatever versions a
consumer or their runtime pins. Published as a separate coordinate with a
dependency-free POM (deps are bundled, not declared), so consumers never pull
a conflicting transitive. Fixes NoClassDefFoundError for clients pinned below
protobuf 3.25 (e.g. 3.19 lacking MapFieldReflectionAccessor).
@karthikk-01
karthikk-01 requested a review from bqin01 August 6, 2026 15:32
Bundled arrow/netty classes reference org.slf4j, which is not relocatable
(logging facade must bind to the runtime). Declaring slf4j-api guarantees the
class is present for any consumer, so the artifact is truly self-contained;
Flink parent-first-loads org.slf4j so its own version still wins on-cluster.
It is the only declared dep — everything else is bundled + relocated.
@karthikk-01
karthikk-01 requested a review from hugokitano August 6, 2026 17:08
@karthikk-01
karthikk-01 merged commit 9d47941 into main Aug 6, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants