Arrow fails with Netty 4.1.120.Final+

[wendigo]

Describe the bug, including details regarding any error messages, version, and platform.

Caused by: UnsupportedOperationException
	at EmptyByteBuf.memoryAddress(EmptyByteBuf.java:955)
	at DuplicatedByteBuf.memoryAddress(DuplicatedByteBuf.java:115)
	at UnsafeDirectLittleEndian.<init>(UnsafeDirectLittleEndian.java:45)
	at PooledByteBufAllocatorL.<init>(PooledByteBufAllocatorL.java:47)
	at NettyAllocationManager.<clinit>(NettyAllocationManager.java:54)
	... 128 more

[wendigo]

static final ArrowBuf EMPTY_BUFFER =
      new ArrowBuf(ReferenceManager.NO_OP, null, 0, NettyAllocationManager.EMPTY.memoryAddress());

this throws UnsupportedOperationException when memoryAddress() is called on EmptyByteBuf

[lidavidm]

Hmm, thanks. Hopefully this can be fixed upstream since trying to branch on every potential address check would be painful.

FWIW, we are tied to Unsafe/Netty until/unless someone can step up and rework things to work with the new native memory APIs (while figuring out how to preserve backwards compatibility somehow...)

[jbonofre]

Thanks for sharing. We should plan an effort about using new native memory APIs. I will take a cranck.

[lidavidm]

It sounds like the issue is just that we need to also require -Dio.netty.noUnsafe=false and/or --sun-misc-unsafe-memory-access=allow for users on JDK24 and above?

[wendigo]

@lidavidm correct, that works but this is not a long-term solution as the Unsafe will eventually go away

[jbonofre]

@lidavidm it should work. I'm fine to move forward like this, as soon as we put a TODO to use the new native APIs at some point 😄

[lidavidm]

We have a todo: #163

Let's use this issue to (1) test Java 24 in CI (2) update the documentation

[jbonofre]

I updated to Netty 4.2.1.Final and it seems to work. Can you please double check ?

[NathanEckert]

I encountered exactly the same issue when upgrading from arrow 18.3.0 to arrow 19.0.0 to connect to Dremio (which use as its JDBC driver arrow-flight-sql).

I am using Java 25

java.lang.NoClassDefFoundError: Could not initialize class org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.memory.RootAllocator

	at org.apache.arrow.driver.jdbc.ArrowFlightJdbcDriver.connect(ArrowFlightJdbcDriver.java:93)
	at org.apache.arrow.driver.jdbc.ArrowFlightJdbcDriver.connect(ArrowFlightJdbcDriver.java:46)
	at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:613)
	at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:160)
	at ...
Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.ExceptionInInitializerError [in thread "ForkJoinPool-1-worker-1"]
	at org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.memory.netty.DefaultAllocationManagerFactory.<clinit>(DefaultAllocationManagerFactory.java:26)
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:467)
	at java.base/java.lang.Class.forName(Class.java:458)
	at org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.memory.DefaultAllocationManagerOption.getFactory(DefaultAllocationManagerOption.java:105)
	at org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.memory.DefaultAllocationManagerOption.getDefaultAllocationManagerFactory(DefaultAllocationManagerOption.java:92)
	at org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.memory.BaseAllocator$Config.getAllocationManagerFactory(BaseAllocator.java:826)
	at org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.memory.ImmutableConfig.access$001(ImmutableConfig.java:20)