Skip to content

Fix main CI (Maven plugin drift), stale dep locks, dev-container cleanup - #130

Merged
mjohns-databricks merged 4 commits into
mainfrom
beta/0.5.0
Oct 13, 2026
Merged

mjohns-databricks merged 4 commits into
mainfrom
beta/0.5.0

Conversation

@mjohns-databricks

Copy link
Copy Markdown
Collaborator

Summary

Fixes the failed build main run on 0ff6e736 and cleans up dependency and dev-container issues found while investigating.

Fix main CI: pin maven-clean and maven-site plugins

build main failed the PGP signature gate (pgpverify:check, "Not allowed artifact … keyID"). The pom doesn't pin maven-clean-plugin or maven-site-plugin, so CI used the runner Maven's built-in defaults, which moved from site 3.12.1 → 3.22.0 and clean 3.2.0 → 3.5.0. The new site plugin pulled in ~60 new transitive dependencies (doxia 2.1.0, jetty 9.4.58, flexmark 0.62.2) signed by keys that aren't in .maven-keys.list. Both plugins are now pinned at the previously verified versions, next to the existing lifecycle-plugin pins, so the reviewed key list stays authoritative. Local scripts/security/maven-pgp-verify: BUILD SUCCESS (630 signatures OK).

Regenerate 3 stale dependency locks

The pre-0.5.2 Dependabot bumps (requests 2.34.2, geopandas 1.1.4, botocore <1.44) updated the .in files but deferred regenerating the hashed locks. requirements-light-env6-ci.txt (the light CI job's lock), requirements-light-env5-ci.txt and requirements-dev-container.txt still had the old pins, so CI kept testing them. All three are regenerated (Linux target, seeded with the current pins): requests 2.34.2, geopandas 1.1.4, botocore 1.43.106, and aiobotocore 3.9.2, which botocore 1.43 requires. The other five .in/.txt pairs were already consistent.

Dev container cleanup

  • Drop unused keplergl. Nothing imports it (Genie Map uses the separate @kepler.gl npm packages). Its source-only build needs jupyterlab 3.x, which blocked local image rebuilds. jupyterlab stays pinned because ipywidgets 7 requires it transitively.
  • Make JDWP opt-in. The start scripts no longer set JAVA_TOOL_OPTIONS. It attached a debug agent on port 5005 to every JVM, so any second concurrent JVM failed, and about 20 call sites had to unset it. The per-run opt-in already in pom.xml (-DagentLib='${jdwp.agent}') is now documented. Ports 5005, 8888 and 4040 stay mapped, each with a comment.

Testing

  • maven-pgp-verify locally: BUILD SUCCESS.
  • CI's light job run against the new requirements-light-env6-ci.txt (clean Python 3.12 venv, --require-hashes): 3,924 passed. The 14 vizx failures are tippecanoe not running under x86 emulation on Apple Silicon; they fail the same way on the old pins and pass in CI on real x86.
  • Local geobrix-dev image rebuilt from the new lock and container started without JAVA_TOOL_OPTIONS: earthdata 6/6, STAC 36/36, scalastyle clean.

This pull request and its description were written by Isaac.

mjohns-databricks and others added 4 commits October 12, 2026 16:36
main CI failed the PGP gate after the runner's Maven changed its
super-POM defaults: maven-site-plugin 3.12.1 -> 3.22.0 and
maven-clean-plugin 3.2.0 -> 3.5.0. The new site plugin pulled ~60 new
transitives (doxia 2.1.0, jetty 9.4.58, flexmark 0.62.2) signed by keys
absent from .maven-keys.list. These two were the only lifecycle plugins
left unpinned after the earlier Category A drift fix; pin them at the
previously verified versions so the reviewed key list stays
authoritative. Local maven-pgp-verify: BUILD SUCCESS.

Co-authored-by: Isaac <no-reply@databricks.com>
c55069d bumped requests 2.34.2, geopandas 1.1.4 and botocore <1.44 in
the .in files but deferred the hashed locks; the light-env5/env6 CI
locks and the dev-container lock were never regenerated, so CI kept
installing requests 2.33.0, geopandas 1.1.3, botocore 1.40.70 and the
Dependabot alerts stayed open. Regenerate them (Linux target, seeded
with the current pins so only the required changes move): requests
2.34.2, geopandas 1.1.4, botocore 1.43.106, and aiobotocore 3.9.2,
which botocore 1.43 requires. The other five .in/.txt pairs were
already consistent. The CI light job on the new env6 lock: 3924
passed; the 14 vizx failures are tippecanoe not running under x86
emulation on Apple Silicon and fail identically on the old pins.

Co-authored-by: Isaac <no-reply@databricks.com>
keplergl 0.3.2 was pinned in the dev-container stack but nothing in the
repo imports it (Genie Map uses the separate @kepler.gl npm packages),
and its sdist-only build needs jupyterlab 3.x, which the dev PyPI proxy
no longer serves, so a local geobrix-dev rebuild failed. Remove it and
its sole dependency traittypes; install it per example if one ever
needs it. jupyterlab stays pinned with a note: ipywidgets 7 ->
widgetsnbextension -> notebook requires it. Also correct the recorded
GEOPANDAS/REQUESTS versions. Local rebuild now succeeds; earthdata,
stac and scalastyle pass in the new container.

Co-authored-by: Isaac <no-reply@databricks.com>
The start scripts set JAVA_TOOL_OPTIONS with a JDWP agent on 5005,
which attaches to every JVM in the container; a second concurrent JVM
fails to bind, so about 20 call sites had to unset it. Stop setting it
(kept as a comment showing how to re-add it) and document the per-run
opt-in already in pom.xml (-DagentLib='${jdwp.agent}'). Ports 5005,
8888 and 4040 stay mapped, each with a comment on what it is for.

Co-authored-by: Isaac <no-reply@databricks.com>
@mjohns-databricks
mjohns-databricks requested a review from a team as a code owner October 13, 2026 01:16
@mjohns-databricks
mjohns-databricks merged commit 59cc8d4 into main Oct 13, 2026
8 checks passed
mjohns-databricks added a commit that referenced this pull request Oct 13, 2026
main squash-merged #130 as 59cc8d4, so beta's four commits and main's squash commit diverge in history while the trees are identical. Merge main back so beta contains main without rewriting beta's history.

Co-authored-by: Isaac <no-reply@databricks.com>

This branch was previously deployed

1 inactive deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant