Skip to content

Add hdf5-native Maven JARs and SciJava native-lib-loader integration#6356

Open
matteodg wants to merge 11 commits intoHDFGroup:developfrom
matteodg:6355-add-hdf5-native-maven-artifacts
Open

Add hdf5-native Maven JARs and SciJava native-lib-loader integration#6356
matteodg wants to merge 11 commits intoHDFGroup:developfrom
matteodg:6355-add-hdf5-native-maven-artifacts

Conversation

@matteodg
Copy link
Copy Markdown

@matteodg matteodg commented Apr 7, 2026

Fixes #6355

I created this with Cursor, as my knowledge of CMake is basically zero: I tried to do it myself, but I was not able. I tested this in Linux x86_64 and Windows x86_64 and it is creating the proper jar and pom.xml file, that I can install in my local Maven repository and depend on them from an example application. This was there is not need of installing HDF5 library or setting up LD_LIBRARY_PATH anymore.

This will be useful for the future of HDFView as well.

@brtnfld
Copy link
Copy Markdown
Collaborator

brtnfld commented Apr 8, 2026

Must Fix:
FFM H5.java static block lacks try-catch around Hdf5NativeLoader call -- NoClassDefFoundError will permanently break H5 if the native-lib-loader JAR is missing

The FFM [H5.java:314] calls Hdf5NativeLoader.loadBundledNativesIfPresent(false) with no try-catch in the static block. If native-lib-loader-2.5.0.jar is missing from the classpath, loading Hdf5NativeLoader will throw NoClassDefFoundError (because it imports org.scijava.nativelib.NativeLoader), which will cause an ExceptionInInitializerError that permanently prevents H5 from being used in that JVM.

The JNI [H5.java:366-374] correctly wraps the call in catch (Throwable err), which handles this. The FFM version does not. The FFM path should have the same protection.

Nice to have:

  • Partial-load when JNI bridge fails but hdf5 succeeds -- works correctly but worth a comment
  • [docs] Transitive native deps (zlib, szip) not bundled -- "zero-install" doesn't hold if HDF5 was built with external filters

Moderate:

  • Duplicate POM profiles across two pom.xml.in files -- The 5 OS-activated profiles (~70 lines each) in pom.xml.in and pom-jni.xml.in) are nearly identical (JNI version adds hdf5-jni-native dependency). This is a lot of duplicated XML that will need to be kept in sync. Not easily avoidable with Maven profiles though.
    HDF5JavaNativeBundles.cmake is include()'d, not add_subdirectory() -- This means all its variables leak into the parent scope. The leading underscores help, but it's still a code smell. Consider using add_subdirectory() or wrapping in a function.

Low Severity

  • Regex in validate-maven-artifacts.sh is getting unwieldy ([line 30]

grep -qE "<artifactId>hdf5-java(-ffm|-jni)?</artifactId>|<artifactId>hdf5-native</artifactId>|<artifactId>hdf5-jni-native</artifactId>|<artifactId>hdf5-java-examples</artifactId>"

Consider using a simpler pattern like hdf5-(java|native|jni-native) or an array-based check.

  • find with -exec cp {} may clobber files -- In ctest.yml and maven-staging.yml, multiple POM files with different names (pom.xml, pom-hdf5-native.xml, etc.) are copied to a flat directory. If there are multiple pom.xml files from different build directories, later ones overwrite earlier ones. This was a pre-existing issue but is now slightly more likely to bite.

  • The _np and _jp variable names in maven-deploy.yml ([lines 112-116] are cryptic. Minor, but NATIVE_POM / JNI_NATIVE_POM would be clearer.

  • RELEASE_PROCESS.md change crams a lot of information into a single bullet point that's now very long and hard to parse.

Verify:

  • Vendored JAR should be checksum-verified against Maven Central; license acknowledged

Copy link
Copy Markdown
Collaborator

@brtnfld brtnfld left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Must Fix" needs to be addressed.

@github-project-automation github-project-automation Bot moved this from To be triaged to In progress in HDF5 - TRIAGE & TRACK Apr 8, 2026
@hyoklee hyoklee added the Component - Wrappers C++, Java & Fortran wrappers label Apr 10, 2026
@hyoklee hyoklee added this to the Backlog milestone Apr 10, 2026
Add org.scijava:native-lib-loader 2.5.0 under java/lib for consistent
classpath use (examples cache includes the JAR alongside SLF4J). Define
HDF5_JAVA_NATIVE_LIB_LOADER_JAR for downstream CMake consumers.
Introduce HDF5JavaNativeBundles.cmake to stage shared libraries under
natives/<platform>/ (SciJava native-lib-loader layout), emit manifests
with Implementation-Version, and add pom-native / pom-jni-native
templates. Wire FFM and JNI hdf.hdf5lib builds and install the
native-lib-loader JAR with other Java dependencies.

Register add_subdirectory(java) before printing Maven deploy status so
Java targets and globals exist before the examples subdirectory pulls in
HDF5ExampleCache.cmake.

Add Hdf5NativeLoader (thin wrapper over NativeLoader) and call it from
H5 for FFM and JNI before loading JNI. Declare org.scijava:native-lib-loader
in generated POMs and put the vendored JAR on the Java compile classpath.
Preserves skip/override properties documented in INSTALL.
Include HDF5_JAVA_NATIVE_LIB_LOADER_JAR alongside logging JARs so unit
tests compile and run the same classpath layout as production JARs.
Extend artifact validation for native classifier JARs and adjust
workflows (ctest, maven-staging, maven-deploy) for the new native
bundle layout and dependencies.
Describe hdf5-native / classpath layout, SciJava loader usage,
Hdf5NativeLoader, and release notes touch-up for published artifacts.

Document that bundled native loading may partially succeed (libhdf5
loaded even if JNI bridge load fails) and clarify that transitive
native dependencies like zlib/szip/plugins are not bundled in the
Maven native JARs.
@matteodg matteodg force-pushed the 6355-add-hdf5-native-maven-artifacts branch from 51bdf4a to 9fdea83 Compare April 24, 2026 21:08
Wrap Hdf5NativeLoader.loadBundledNativesIfPresent(false) in try/catch
(Throwable) with printStackTrace(), matching the JNI loadH5Lib() path.
Prevents ExceptionInInitializerError on H5 when native-lib-loader is
missing from the classpath (e.g. NoClassDefFoundError for NativeLoader),
so the class can still initialize and other library loading may succeed.
…les CMake

Generate the five OS-activated hdf5-bundled-native-* profile blocks from
HDF5JavaMavenBundledProfiles.cmake and inject them into both pom.xml.in
templates via @HDF5_MAVEN_BUNDLED_OS_PROFILES@, with an optional second
dependency on hdf5-jni-native for JNI builds.

Refactor HDF5JavaNativeBundles.cmake into hdf5_java_add_native_maven_bundles()
so internal variables (including the former HDF5_NATIVE_LOADER_PLATFORM) stay
function-local instead of leaking into the parent scope after include().
Replace the long grep alternation with an explicit allow-list and
grep -Fq loops over known HDF5 Maven artifact IDs.
Copy POMs from FFM vs JNI CMake output paths instead of matching any
pom.xml under the build tree. ctest uses ACTIVE_PRESET *Maven-FFM*
and fails if both trees contain pom.xml. maven-staging branches on
matrix.implementation and validates exactly one pom.xml in artifacts.
Break the Deployment Process sub-step into separate bullets for
deployed artifacts, POM pairing, and release-note guidance so the
section is easier to scan.
Introduce java/lib/vendored-jars.sha256 and a CI workflow that runs
sha256sum -c against it (no network). Document SLF4J and SciJava
native-lib-loader in java/lib/NOTICES.txt. Force LF on the new text
files via .gitattributes.
@matteodg matteodg force-pushed the 6355-add-hdf5-native-maven-artifacts branch from 9fdea83 to 3169dea Compare April 24, 2026 21:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Component - Wrappers C++, Java & Fortran wrappers

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

Publish hdf5-native Maven artifacts (bundled libhdf5 for classpath loading)

3 participants