Classifier jars: separate Java library from native libs

[gotson]

Alternative to #1387

Not sure which approach we should go for.

The following classifiers are added:

• natives only per OS: natives-android, natives-linux, natives-freebsd, natives-mac, natives-windows. Not sure if splitting Linux and Linux-Musl makes sense or not.
• natives for desktop (all OSes except Android): natives-all. The name may not be the best, but at least it's better than all.
• the same default Jar but without any of the native libs: without-natives

This way of doing has much less friction on the overall project than the multi-modules approach.

[gotson]

I have been pondering whether instead of per OS, we should publish classifier jars per OS/Arch combination instead.

However i am not entirely sure of how the consumers of the library would be able to get all the native libs for a particular OS, without having to explicitly list all the archs…

[kkriske]

@gotson have you seen how javafx separates os-specific stuff in separate dependencies separated by classifiers?
It's pretty neat. A specific package like javafx-base has a single dependency on itself but with <classifier>${javafx.platform}</classifier>: https://central.sonatype.com/artifact/org.openjfx/javafx-base
That property is resolved by the parent, by letting maven do profile-based os-detection: https://central.sonatype.com/artifact/org.openjfx/javafx

[gotson]

@gotson have you seen how javafx separates os-specific stuff in separate dependencies separated by classifiers? It's pretty neat. A specific package like javafx-base has a single dependency on itself but with <classifier>${javafx.platform}</classifier>: central.sonatype.com/artifact/org.openjfx/javafx-base That property is resolved by the parent, by letting maven do profile-based os-detection: central.sonatype.com/artifact/org.openjfx/javafx

I was not aware, however i don't think we need OS detection on Maven side. The different libraries are more for packaging an app, is what i think, but again i may be wrong.

It seems those other projects support less OS/arch couples than we do too.

[gotson]

@pavel-yazev-km tagging you here as you seem to be one of the few people using Android. Given the android native libs need manual extraction anyway, how would you feel if the default package did not ship the Android native libs ?

[LibertyPaul]

@gotson significantly less comfortable to upgrade, but not a deal breaker

It would make us build it every time, which takes

• several hours of work if one knows how to build it
• a day of work if thats their 1st time

Compared to 5 mins of extracting the necessary .so into our project's native libs.

Since the build process requires some lengthy and resource heavy operations (even though it is neatly containerized), some folks would probably consider the alternatives.

[pavel-yazev-km]

accidentally used my personal account ^

[gotson]

@gotson significantly less comfortable to upgrade, but not a deal breaker

It would make us build it every time, which takes

* several hours of work if one knows how to build it

* a day of work if thats their 1st time

Compared to 5 mins of extracting the necessary .so into our project's native libs.

Since the build process requires some lengthy and resource heavy operations (even though it is neatly containerized), some folks would probably consider the alternatives.

@pavel-yazev-km Android libs would still be available in the specific jar with classifier natives-android. Since they cannot be loaded by the loader we use for desktop OSes, i don't see the point to package them in the default jar.

[pavel-yazev-km]

@gotson just to confirm my understanding

Currently we take the .so from an unzipped JAR:

> find . -name '*.so'
./org/sqlite/native/Linux-Android/x86/libsqlitejdbc.so
./org/sqlite/native/Linux-Android/arm/libsqlitejdbc.so
./org/sqlite/native/Linux-Android/aarch64/libsqlitejdbc.so   <--- This one
./org/sqlite/native/Linux-Android/x86_64/libsqlitejdbc.so
./org/sqlite/native/FreeBSD/x86/libsqlitejdbc.so
./org/sqlite/native/FreeBSD/aarch64/libsqlitejdbc.so
./org/sqlite/native/FreeBSD/x86_64/libsqlitejdbc.so
./org/sqlite/native/Linux/ppc64/libsqlitejdbc.so
./org/sqlite/native/Linux/riscv64/libsqlitejdbc.so
./org/sqlite/native/Linux/armv7/libsqlitejdbc.so
./org/sqlite/native/Linux/armv6/libsqlitejdbc.so
./org/sqlite/native/Linux/x86/libsqlitejdbc.so
./org/sqlite/native/Linux/arm/libsqlitejdbc.so
./org/sqlite/native/Linux/aarch64/libsqlitejdbc.so
./org/sqlite/native/Linux/x86_64/libsqlitejdbc.so
./org/sqlite/native/Linux-Musl/x86/libsqlitejdbc.so
./org/sqlite/native/Linux-Musl/aarch64/libsqlitejdbc.so
./org/sqlite/native/Linux-Musl/x86_64/libsqlitejdbc.so

Will it not be available anymore?
Is natives-android something that is published in a different place than Releases ?

[gotson]

@gotson just to confirm my understanding

Currently we take the .so from an unzipped JAR:

> find . -name '*.so'
./org/sqlite/native/Linux-Android/x86/libsqlitejdbc.so
./org/sqlite/native/Linux-Android/arm/libsqlitejdbc.so
./org/sqlite/native/Linux-Android/aarch64/libsqlitejdbc.so   <--- This one
./org/sqlite/native/Linux-Android/x86_64/libsqlitejdbc.so
./org/sqlite/native/FreeBSD/x86/libsqlitejdbc.so
./org/sqlite/native/FreeBSD/aarch64/libsqlitejdbc.so
./org/sqlite/native/FreeBSD/x86_64/libsqlitejdbc.so
./org/sqlite/native/Linux/ppc64/libsqlitejdbc.so
./org/sqlite/native/Linux/riscv64/libsqlitejdbc.so
./org/sqlite/native/Linux/armv7/libsqlitejdbc.so
./org/sqlite/native/Linux/armv6/libsqlitejdbc.so
./org/sqlite/native/Linux/x86/libsqlitejdbc.so
./org/sqlite/native/Linux/arm/libsqlitejdbc.so
./org/sqlite/native/Linux/aarch64/libsqlitejdbc.so
./org/sqlite/native/Linux/x86_64/libsqlitejdbc.so
./org/sqlite/native/Linux-Musl/x86/libsqlitejdbc.so
./org/sqlite/native/Linux-Musl/aarch64/libsqlitejdbc.so
./org/sqlite/native/Linux-Musl/x86_64/libsqlitejdbc.so

Will it not be available anymore? Is natives-android something that is published in a different place than Releases ?

That PR would result in multiple jars being published, both on Maven Central and in the Github Releases:

• sqlite-jdbc-X.Y.Z.jar: the existing jar, but i am pondering not bundling the Android natives in there.
• JARs with classifiers, for example sqlite-jdbc-X.Y.Z-natives-android.jar: this one would contain ONLY the native libs for Android. It would not contain the Java classes.

With classifiers, one could add multiple dependencies depending of what they want.

For Android, you would need to download the jar with thenatives-android classifer, unzip, and pick the library you would need.

For someone using Maven Central, for example if they are only interested in macOS libs, they could use the following:

<dependencies>
    <dependency>
      <groupId>org.xerial</groupId>
      <artifactId>sqlite-jdbc</artifactId>
      <version>3.51.3.0</version>
      <classifier>without-natives</classifier>
    </dependency>
    <dependency>
      <groupId>org.xerial</groupId>
      <artifactId>sqlite-jdbc</artifactId>
      <version>3.51.3.0</version>
      <classifier>natives-mac</classifier>
    </dependency>
</dependencies>

The first jar would have the classes, but no native libraries.
The second jar would have only the native libraries for Mac.

This would allow for composition, depending on what you need, without having to pull and bundle libs you won't be using.

[pavel-yazev-km]

@gotson thanks!

The new approach doesn't seem to create additional effort for our use-case.

[gotson]

@gotson thanks!

The new approach doesn't seem to create additional effort for our use-case.

Thanks for confirming.

It could shave off ~4.5Mb on the jar by not shipping Android libs, which anyhow have to be manually extracted.

I'll adjust the PR with those changes in mind.