Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
159 changes: 159 additions & 0 deletions .github/verify-bytecode-version.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
#!/usr/bin/env bash

# SPDX-FileCopyrightText: 2026 Bernard Ladenthin <bernard.ladenthin@gmail.com>
#
# SPDX-License-Identifier: MIT OR Apache-2.0

# Cross-repo shared script — kept BYTE-IDENTICAL in java-llama.cpp, srcmorph,
# BitcoinAddressFinder and streambuffer (sync any edit to all four, and to the checksum table in
# workspace/crossrepostatus.md). Fails when a built jar contains a class file newer than the Java
# release the artifact claims to support.
#
# Why: `maven.compiler.release` governs only the code WE compile. A dependency compiled for a newer
# Java lands in the jar untouched, and nothing in a normal build objects. The failure surfaces at a
# consumer's JVM as UnsupportedClassVersionError, which is the worst possible place to find it.
# This has happened twice here: checker-qual 4.x (Java 11 bytecode, and its annotations are
# @Retention(RUNTIME), so anything reflecting over an annotated element loads them), and
# logback-classic from 1.4.0 on, whose LogbackServiceProvider SLF4J's ServiceLoader loads at
# startup — a guaranteed crash rather than a latent one.
#
# Scan the BUILT ARTIFACT, not a resolved classpath: an uber jar is what a user actually runs, and
# `dependency:build-classpath` answers a different question (and answers it with an empty file when
# it fails, which reads as a pass).
#
# Usage: verify-bytecode-version.sh --max-major <N> [--allow <pattern>]... <path>...
# --max-major <N> highest class-file major version a consumer JVM may be asked to load.
# 52 = Java 8, 55 = Java 11, 61 = Java 17, 65 = Java 21.
# Pass it from the pipeline so the value lives next to the release it belongs
# to, instead of being duplicated here per repo.
# --allow <pattern> repeatable, optional. Glob matched against "<jar-basename>:<entry-path>", so
# it can waive a whole jar (`--allow 'foo-*.jar:*'`) or a single entry
# (`--allow '*:com/example/Legacy.class'`). Use sparingly and say why in the
# workflow: every entry here is a hole in the guarantee, and a hole nobody
# revisits is how a gate stops gating.
# <path>... jars, and/or directories searched recursively for *.jar.
#
# ALWAYS skipped, not configurable — a classpath JVM never loads these, so a high version in them
# is not a defect and waiving them per-repo would only invite blanket exceptions:
# * module-info.class (any directory) — read only in module mode, and Java 8 has none
# * META-INF/versions/** — multi-release overlays, invisible below their own release
#
# Exit codes: 0 clean · 1 violations found · 2 nothing to scan / bad usage. 2 matters as much as 1:
# a run that scanned no jars must never be reported as a pass.

set -euo pipefail

MAX_MAJOR=""
ALLOW=()
PATHS=()

fail_usage() {
echo "::error::$*" >&2
echo "usage: verify-bytecode-version.sh --max-major <N> [--allow <pattern>]... <path>..." >&2
exit 2
}

while [ $# -gt 0 ]; do
case "$1" in
--max-major) [ $# -ge 2 ] || fail_usage "--max-major needs a value"; MAX_MAJOR="$2"; shift 2 ;;
--allow) [ $# -ge 2 ] || fail_usage "--allow needs a value"; ALLOW+=("$2"); shift 2 ;;
--) shift; while [ $# -gt 0 ]; do PATHS+=("$1"); shift; done ;;
-*) fail_usage "unknown option '$1'" ;;
*) PATHS+=("$1"); shift ;;
esac
done

[ -n "$MAX_MAJOR" ] || fail_usage "--max-major is required"
case "$MAX_MAJOR" in ''|*[!0-9]*) fail_usage "--max-major must be a number, got '$MAX_MAJOR'" ;; esac
[ "${#PATHS[@]}" -gt 0 ] || fail_usage "at least one jar or directory is required"

for p in "${PATHS[@]}"; do
[ -e "$p" ] || fail_usage "path '$p' does not exist"
done

command -v python3 >/dev/null 2>&1 || fail_usage "python3 is required to read class-file headers"

# The scan itself: one pass per jar, reading the 8-byte class-file header of every entry. Kept in
# python because the alternative (unzip -p per entry) spawns a process per class — thousands for a
# fat jar — and because a zip reader must not be reimplemented in shell.
python3 - "$MAX_MAJOR" "${#ALLOW[@]}" "${ALLOW[@]}" "${PATHS[@]}" <<'PYTHON'
import fnmatch, os, sys, zipfile

max_major = int(sys.argv[1])
n_allow = int(sys.argv[2])
allow = sys.argv[3:3 + n_allow]
paths = sys.argv[3 + n_allow:]

jars = []
for p in paths:
if os.path.isdir(p):
for root, _dirs, files in os.walk(p):
jars.extend(os.path.join(root, f) for f in files if f.endswith(".jar"))
elif p.endswith(".jar"):
jars.append(p)
jars = sorted(set(jars))

# An empty scan is a broken measurement, never a pass. A glob that matched nothing, a download step
# that silently produced no artifact, a renamed output directory: all of them yield "0 violations"
# from a scanner that just shrugs, and that is indistinguishable from a clean run.
if not jars:
print(f"::error::no jars found under: {', '.join(paths)} -- refusing to report a pass", file=sys.stderr)
sys.exit(2)

def skipped_always(entry):
# A plain classpath JVM never loads either of these, at any Java level.
return (entry == "module-info.class"
or entry.endswith("/module-info.class")
or entry.startswith("META-INF/versions/"))

violations = []
waived = 0
scanned = 0

for jar in jars:
base = os.path.basename(jar)
try:
zf = zipfile.ZipFile(jar)
except Exception as exc: # noqa: BLE001 - report, do not crash
print(f"::error::cannot read '{jar}': {exc}", file=sys.stderr)
sys.exit(2)
with zf:
for entry in zf.namelist():
if not entry.endswith(".class") or skipped_always(entry):
continue
try:
with zf.open(entry) as handle:
head = handle.read(8)
except Exception: # noqa: BLE001 - unreadable entry
continue
if len(head) < 8 or head[:4] != b"\xca\xfe\xba\xbe":
continue
scanned += 1
major = int.from_bytes(head[6:8], "big")
if major <= max_major:
continue
key = f"{base}:{entry}"
if any(fnmatch.fnmatch(key, pattern) for pattern in allow):
waived += 1
continue
violations.append((base, entry, major))

# One line per offending jar, naming an example entry: a full listing of a fat jar's thousands of
# classes buries the answer, and the jar is the unit somebody acts on.
by_jar = {}
for base, entry, major in violations:
prev = by_jar.get(base)
if prev is None or major > prev[1]:
by_jar[base] = (entry, major)

for base in sorted(by_jar):
entry, major = by_jar[base]
print(f"::error::{base}: class-file major {major} (Java {major - 44}) exceeds the "
f"allowed {max_major} (Java {max_major - 44}) -- e.g. {entry}")

print(f"scanned {scanned} class file(s) in {len(jars)} jar(s); "
f"{len(by_jar)} jar(s) over major {max_major}"
+ (f"; {waived} entr(y/ies) waived by --allow" if waived else ""))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent implementation. The bytecode version verification is properly scoped:

  • Skips module-info.class and META-INF/versions/** (classpath JVM never loads these)
  • Exits with code 2 on empty scan (prevents false positives)
  • Clear error messages showing the highest offending major version
  • Cross-repo synchronized script is a good pattern for consistency

Minor note: The python3 requirement check on line 74 is good defensive practice.

sys.exit(1 if violations else 0)
PYTHON
9 changes: 9 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -465,6 +465,15 @@ jobs:
with:
java-version: '21'
distribution: temurin
# Class-file floor. Production code here targets Java 8, so anything a consumer's JVM
# can load must be major 52 or lower -- a single Java 11 class (logback's
# LogbackServiceProvider was the real case) kills the process with
# UnsupportedClassVersionError before any of our code runs. The script recurses into the
# whole download directory, so every jar the build produced is checked, not just the fat
# jar; module-info.class and META-INF/versions/** are skipped unconditionally because a
# classpath JVM never loads them. Kept byte-identical across all four sibling repos.
- name: Verify Java 8 bytecode (no class newer than major 52)
run: .github/verify-bytecode-version.sh --max-major 52 fatjar
- name: Run fat-jar smoke test
run: |
.github/smoke-fatjar-cli.sh fatjar 'srcmorph-cli-*-jar-with-dependencies.jar' \
Expand Down
32 changes: 32 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,38 @@ The release procedure (prompt template and step-by-step instructions) lives in [

## [Unreleased]

### Changed
- **The CLI fat jar now ships `slf4j-simple` instead of logback, and no `checker-qual` at all.**
Production code here targets Java 8, but every logback release from 1.4.0 on is Java 11 bytecode:
SLF4J's `ServiceLoader` finds `LogbackServiceProvider` at startup, so a Java 8 JVM died with
`UnsupportedClassVersionError` before a single line was logged. The Java 8 logback line (1.3.x) is
end-of-life and its CVEs are fixed only in 1.5.x/1.6.x with no backport, so downgrading was not an
option either. `slf4j-simple` is six classes from the same release train as `slf4j-api`, with no
configuration or socket layer for a CVE to live in.

`checker-qual` moved to `provided` scope. Its annotations are major 55 from 4.0.0 on and
`@Retention(RUNTIME)`, so anything reflecting over an annotated element — Jackson, which binds the
CLI's whole configuration — loads them. `provided` keeps 4.2.2 on the compile classpath, where the
Checker Framework processor needs it, while shipping none of it: `jar-with-dependencies` filters on
scope, which is also why `<optional>true</optional>` alone would not have been enough.

Consequences for users: `examples/logbackConfiguration.xml` is replaced by
`examples/simplelogger.properties`; the fat jar carries its own defaults (INFO, stdout, timestamps)
which any `-Dorg.slf4j.simpleLogger.*` system property or a classpath `simplelogger.properties`
overrides. The published `srcmorph` and `srcmorph-cli` **library** jars are unchanged in this
respect — they impose no binding and carry no logging configuration; a binding is an application
concern and the fat jar is the application. `net.ladenthin:llama`'s own binding is now excluded
transitively, so embedding `srcmorph` no longer forces one on the caller and the Maven plugin no
longer ends up with two providers alongside Maven's `maven-slf4j-provider`.

### Added
- **CI gates every built jar on the Java 8 class-file floor.** `release 8` governs only the code we
compile; a dependency built for a newer Java lands in the jar untouched and surfaces as
`UnsupportedClassVersionError` on a consumer's JVM. The new `.github/verify-bytecode-version.sh`
(kept byte-identical across the four sibling repos) runs in the `smoke-fatjar` job with
`--max-major 52` and fails on any class above it, skipping only `module-info.class` and
`META-INF/versions/**`, which a classpath JVM never loads.

### Added
- **`srcmorph:calibrate` now writes a machine-readable report.** The goal built a `CalibrationReport`
and printed it as `INFO` lines, and that was the only output — so the numbers a calibration run
Expand Down
60 changes: 53 additions & 7 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ llamacpp-ai-index-maven-plugin/ (repo root; reactor parent)
│ └── configuration/ CConfiguration + CCommand (BAF public-field style)
├── srcmorph-maven-plugin/ Maven plugin net.ladenthin:srcmorph-maven-plugin, goalPrefix srcmorph
│ └── src/main/java/net/ladenthin/maven/srcmorph/mojo/ (4 goal mojos + the abstract AbstractAiIndexMojo base; renamed package/properties)
├── examples/ config_*.json/.yaml + run_*.sh/.bat + logbackConfiguration.xml
├── examples/ config_*.json/.yaml + run_*.sh/.bat + simplelogger.properties
├── docs/ RELEASE.md + the ai-index model-benchmark writeups
└── .github/workflows/ CI adapted to the 3-module reactor
```
Expand Down Expand Up @@ -103,7 +103,7 @@ Framework-free: **no dependency on `org.apache.maven..`** anywhere (enforced by
`org.slf4j.Logger` (a private static final field per class), not a Maven `Log` — this is what makes
the module Maven-free; Maven's own `maven-slf4j-provider` (ships since Maven ≥ 3.1) makes these lines
surface as ordinary `[INFO]`/`[WARN]` output inside a plugin execution with zero glue, and the CLI
ships a logback binding for the same log lines outside Maven.
ships an SLF4J binding for the same log lines outside Maven.
- **`document/`** — the `.ai.md` model + codecs (`AiMdDocument`, `AiMdHeader`, `AiMdDocumentCodec`,
`AiMdHeaderCodec`, `AiMdHeaderSupport`, `AiMdChildEntryLineFormatter`, `AiMdLeadExtractor`,
`AiGenerationRequest`/`AiGenerationResult`).
Expand Down Expand Up @@ -166,10 +166,13 @@ CLI driven by a single JSON or YAML configuration file:
- The fat jar (`srcmorph-cli-<version>-jar-with-dependencies.jar`, main class
`net.ladenthin.srcmorph.cli.Main`) is bound **unconditionally** to the `package` phase (a deliberate
divergence from BAF's `-P assembly` opt-in — for this module the fat jar IS the deliverable).
- Ships its own logback binding (`ch.qos.logback:logback-classic`, runtime scope) — unlike the library
- Ships its own SLF4J binding (`org.slf4j:slf4j-simple`, runtime scope) — unlike the library
(consumer picks any SLF4J binding) and the plugin (gets one for free from Maven's own
`maven-slf4j-provider`), a standalone `java -jar` process needs to bring its own or every log line is
silently dropped.
silently dropped. It is **not** logback: see "Java 8 bytecode floor" below. Fat-jar defaults live in
`srcmorph-cli/src/main/assembly-resources/simplelogger.properties`, added by the custom assembly
descriptor `srcmorph-cli/src/assembly/fat-jar.xml` (the predefined `jar-with-dependencies` ref
verbatim plus that one file) so they never reach the published `srcmorph-cli` jar.
- **Architecture rules** (`CliArchitectureTest`): `cliIsLeaf` (nothing else in the reactor may depend on
this module — it is the leaf-most consumer), `noPublicMutableFields` (with the `configuration`
package carve-out), `noSystemExit`, `mavenFree` (must never depend on the Maven Plugin API — that
Expand Down Expand Up @@ -394,9 +397,10 @@ assume it has already been updated.

| Dependency | Version | Used by |
|---|---|---|
| `net.ladenthin:llama` | 5.1.0 | `srcmorph` (`provider` package only) — llama.cpp JNI binding |
| `net.ladenthin:llama` | 5.2.0 | `srcmorph` (`provider` package only) — llama.cpp JNI binding; its own SLF4J binding is excluded transitively (see "Java 8 bytecode floor") |
| `org.slf4j:slf4j-api` | 2.0.18 (converged in the parent) | `srcmorph`, `srcmorph-cli`, the plugin |
| `ch.qos.logback:logback-classic` | 1.6.3 (converged in the parent) | `srcmorph-cli` (runtime binding) |
| `org.slf4j:slf4j-simple` | 2.0.18 (converged in the parent) | `srcmorph-cli` (runtime binding) |
| `ch.qos.logback:logback-classic` | 1.6.3 (converged in the parent) | `srcmorph` (**test scope only** — `ListAppender` capture) |
| `com.fasterxml.jackson.core:jackson-databind` | pinned in parent | `srcmorph-cli` (JSON config) |
| `com.fasterxml.jackson.dataformat:jackson-dataformat-yaml` | pinned in parent | `srcmorph-cli` (YAML config) |
| `org.apache.maven:maven-plugin-api` | 3.9.16 | `srcmorph-maven-plugin` (provided) |
Expand Down Expand Up @@ -451,6 +455,48 @@ See [`../workspace/workflows/pull-request-workflow.md`](../workspace/workflows/p
existing consumer mid-migration; the rename to `srcmorph-maven-plugin` is a deliberately isolated,
later step (see `TODO.md`).

## Java 8 bytecode floor — what may ship

Production code in all three modules targets **Java 8** (`release 8`), so **every class a
consumer's JVM can load must be class-file major 52 or lower**. Two entries exist only for that,
and both are easy to undo by accident:

- **`slf4j-simple`, not logback, is `srcmorph-cli`'s shipped SLF4J binding.** Every logback release
from 1.4.0 on is Java 11 bytecode, so `LogbackServiceProvider` cannot load on Java 8 — SLF4J's
`ServiceLoader` finds it at startup and the JVM throws `UnsupportedClassVersionError` before a
single log line is written. The Java 8 line (1.3.x) is end-of-life and every logback CVE disclosed
since has been fixed only in 1.5.x/1.6.x with no backport, so it is not an option either.
`slf4j-simple` is six classes from the same release train as `slf4j-api`, with no configuration or
socket layer for a CVE to live in. logback stays, but **test scope only**, for
`AiFieldGenerationSupportTest`'s `ListAppender`.
- **`checker-qual` is `provided` scope, not a pinned old version.** It is major 55 from 4.0.0 on and
its annotations are `@Retention(RUNTIME)`, so anything reflecting over an annotated element (Jackson
does — the CLI binds its whole configuration with it) loads them. **Pinning the shipped copy to the
last Java 8 line (3.55.1) does not work**: the Nullness Checker resolves its own qualifiers through
javac's symbol table, i.e. the *compile classpath*, so a 3.x checker-qual under the 4.x processor
fails every build with `Could not load type:
org.checkerframework.framework.qual.DoesNotUnrefineReceiver`. Processor and qualifiers must share a
major version. `provided` satisfies both: 4.2.2 where the checker needs it, and excluded from
consumers' graph **and** from the fat jar (`jar-with-dependencies` takes scope `runtime`).
`<optional>true</optional>` alone would not have been enough — that descriptor filters on scope
only. Safe because no source in this reactor imports `org.checkerframework`.

**The gate: `.github/verify-bytecode-version.sh`.** Kept **byte-identical** across
java-llama.cpp / BitcoinAddressFinder / streambuffer / srcmorph (checksum table in
`workspace/crossrepostatus.md`). It opens every `.class` in every jar it is given and fails on any
whose class-file major version exceeds `--max-major`:

```bash
.github/verify-bytecode-version.sh --max-major 52 [--allow '<jar>:<entry>']... <jar-or-dir>...
```

Paths may be jars or directories (searched recursively for `*.jar`), so one invocation covers a
whole artifact set. `module-info.class` and `META-INF/versions/**` are skipped unconditionally —
a classpath JVM never loads either, which is why a `release 9` `module-info` is fine. `--allow`
is a repeatable glob matched against `<jar-basename>:<entry-path>` for anything else that must be
tolerated. Exit codes: 0 clean, 1 violations, **2 nothing to scan** (an empty input is a failure,
never a pass). Wired into the `smoke-fatjar` job at `--max-major 52`.

## Javadoc Conventions

See [`../workspace/policies/javadoc-conventions.md`](../workspace/policies/javadoc-conventions.md).
Expand Down Expand Up @@ -487,7 +533,7 @@ See [`../workspace/policies/ci-test-diagnostics.md`](../workspace/policies/ci-te
See [`../workspace/policies/pit-mutation-testing.md`](../workspace/policies/pit-mutation-testing.md).
Run PIT with the lifecycle prefix. Reactor-wide (what CI does):
`mvn test-compile org.pitest:pitest-maven:mutationCoverage`; or scoped to one module with
`-f srcmorph/pom.xml`. All three modules gate at `mutationThreshold` 100 — `srcmorph` (775 mutations),
`-f srcmorph/pom.xml`. All three modules gate at `mutationThreshold` 100 — `srcmorph` (807 mutations),
`srcmorph-maven-plugin` (62, the five mojo classes) and `srcmorph-cli` (16). The CLI's
`Main.main(String[])` is the one documented exclusion: it is the process entry point, and the
`smoke-fatjar` release-gating job already runs the real `java -jar` artifact and asserts
Expand Down
Loading
Loading