Skip to content

Fix mixed-collection boundary clipping in Scala and Python BNG tessellation - #131

Closed
sllynn wants to merge 3 commits into
databrickslabs:mainfrom
sllynn:fix/collection-aware-jts-overlay
Closed

sllynn wants to merge 3 commits into
databrickslabs:mainfrom
sllynn:fix/collection-aware-jts-overlay

Conversation

@sllynn

@sllynn sllynn commented Oct 19, 2026 •

Copy link
Copy Markdown

Valid polygons can produce mixed GeometryCollections when clipped against a BNG cell boundary. The Scala tier calls JTS difference directly on the collection and raises Operation does not support GeometryCollection arguments. The Python tier accepts the operation, but can retain boundary-only points and then silently discard the cell through an exact geometry-type filter. Polygon-to-MultiPolygon clipping also loses cells in both original implementations.

Scala changes:

  • Add JTS.intersection and JTS.difference helpers that recursively handle mixed collections on either side, compact results through unary union, and preserve the left SRID and factory. Ordinary geometries retain existing JTS overlay operations.
  • Expose opt-in JTS.implicits._ extensions and use collectionAwareIntersection / collectionAwareDifference in BNG tessellation.
  • Filter chips by dimension, allowing Polygon/MultiPolygon transitions, and handle omitted core geometries safely.

Python changes:

  • Add _clean_chip_geometry in _bng.py to subtract cell boundaries from individual collection leaves, discard empty results, and compact surviving members while preserving SRID. Ordinary clips remain unchanged to preserve vertex order for core-cell promotion.
  • Match Scala's dimension-based filtering, retaining polygonal chips while excluding boundary-only points and lines.
  • Cover both synthetic failure patterns, both keep-core modes, MultiPolygon inputs, nested/empty collections, SRID, and the public WKB wrapper.

Validation:

  • Python: 75 Spark-free BNG tests passed across codec, cell math, vector encoder, neighbourhood, geometry, polyfill, and tessellation (Shapely 2.2.0). Includes eight new regression cases. The Spark-dependent UDF test was deselected; parent conftests were disabled because PySpark is unavailable locally.
  • Black and Flake8 passed for both changed Python files; whitespace checks and commit secret scans passed.
  • Scala: compiled the actual JTS and BNG sources and executed all 11 regression test bodies in an isolated local harness with JTS 1.20.0 and Scala 2.13.18. Spark API types, ScalaTest registration, and unrelated dilation were stubbed outside the repository; this is not a full Maven/Spark test run.
  • The same Scala BNG regression with the original BNG source reproduces the exact GeometryCollection exception; patched source passes. Before the Python fix, synthetic inputs lost area (6/10 and 5/7 square metres); regression tests now verify complete cell sets and area preservation.
  • Full Maven/Spark tests and Scalastyle remain for CI: Maven/Docker are unavailable locally; the repository Scala lint script stops at its Docker check.

General overlay helpers retain legitimate lines and points; dimension filtering remains in tessellation. Broader migration of direct JTS callers is outside this change. All regression geometries are synthetic; no customer data is included.

@CLAassistant

CLAassistant commented Oct 19, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@sllynn sllynn self-assigned this Oct 19, 2026
@sllynn sllynn added the bug Something isn't working label Oct 19, 2026
@sllynn sllynn changed the title Fix BNG clipping with collection-aware JTS overlays Fix mixed-collection boundary clipping in Scala and Python BNG tessellation Oct 20, 2026
@sllynn

sllynn commented Oct 20, 2026

Copy link
Copy Markdown
Author

Superseded by #132, containing the same changes on an upstream branch. The fork PR's CI could not obtain the required OIDC permissions and the doc-inventory job expected an upstream branch.

This branch was previously deployed

1 inactive deployment
runtime — 2e99df05 Deployed Oct 20, 2026 by sllynn via build (3.12.3, 8.4.2, 2.1.3, 3.11.4, 4.0.0) #886
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants