Repository navigation
Wireless Coverage workshop fixes, run screenshots, notebook tooling - #129
Merged
Merged
Conversation
v0.5.2 is now cut and published, so the release-notes section no longer carries the pre-release "under active development -- may change before release" disclaimer. The `Current version: 0.5.2` banner is unchanged. Co-authored-by: Isaac <no-reply@databricks.com>
The LiDAR LAS/LAZ writer (new in 0.5.2) was present in the Readers table but missing from Writers; add the lightweight-only row so the README reflects the registered writer. Co-authored-by: Isaac <no-reply@databricks.com>
The exif/file/lidar reader & writer pages appended the redundant `*_gbx` DataSource id to their sidebar_label, diverging from the bare-format-name convention the other 10+ pages use -- and leaving the LiDAR reader labeled `LiDAR (lidar_gbx)` while its writer was just `LiDAR`. Drop the `(exif_gbx)` / `(file_gbx)` / `(lidar_gbx)` suffixes. `GeoJSONL (multi-file)` is kept: it is an informative descriptor, not a redundant DataSource id. Co-authored-by: Isaac <no-reply@databricks.com>
The WC executed-output screenshots sat inside the Notebook Series tab, so readers landing on the default Lakeflow tab never saw them; they now sit above the tabs, labeled as pulled from the notebook series. The Orthomosaic page had no executed output at all, so add four renders (raw inputs, color before/after, PMTiles preview, land cover) taken from the committed notebook outputs. Also fix the double-rendered raw-input gallery in 01a/01b (the returned figure was echoed as execute_result) and Visualise -> Visualize. Co-authored-by: Isaac <no-reply@databricks.com>
Workshop feedback on the Wireless Coverage example: hyphenated catalog or schema names failed on Serverless (unquoted identifiers), and the default DEMO run broke at nb04 because it read the full-SF surface tables instead of the _demo ones. Both flavors now backtick-quote every catalog/schema/table name and nb04 reads surfaces through _tbl. Per-site viewsheds were computed but never kept, so map/BI tools could not score sites against coverage or population layers. Add wc_h3_site_viewshed_res12 (one row per sited tower and visible cell, keyed by the tower's res-9 cellid) in the notebook series and the Lakeflow siting task. H3 outputs now carry an h3 string twin of each BIGINT cell column for map tools, are liquid-clustered on the BIGINT cell column (CLUSTER BY, not Z-ORDER), and are OPTIMIZEd after each write; gold MVs use cluster_by. Verified end to end on Serverless env 6 in a hyphenated schema (DEMO run, all five notebooks green). Co-authored-by: Isaac <no-reply@databricks.com>
The Lakeflow tabs described the job and pipeline only in prose and a diagram, so readers never saw what a deployed run looks like. Add the job-run and pipeline-lineage screenshots with a short narrative (WC before its table catalog; Vapor-Eyes at the top of How it works, with the flow diagram moved to the end of that section). The WC page also opened straight on its executed-output screenshots, so give it a brief introduction and a 0.5.2+ requirement note like the Vapor-Eyes page. Co-authored-by: Isaac <no-reply@databricks.com>
The docs pages gained Lakeflow run screenshots (WC, Vapor-Eyes) and an Orthomosaic executed-output section, but readers browsing the example folders on GitHub never saw them. Copy the same images and narrative into the WC and Vapor-Eyes lakeflow READMEs and the Orthomosaic README, following each file's image-path convention. The WC lakeflow README also still said the siting task writes six tables; it writes seven now that wc_h3_site_viewshed_res12 exists. Co-authored-by: Isaac <no-reply@databricks.com>
Committed example notebooks carry stored outputs, but refreshing them after a Serverless validation run meant hand work. Add --harvest-outputs to gbx:test:notebooks-serverless: after a SUCCESS it exports the run (jobs export-run), decodes the embedded notebook model, and writes outputs into the local .ipynb, matching cells by source and preserving the file's serialization. It re-exports until results stop growing, since an export taken as the run ends was seen to lack a multi-MB plot; a requested harvest that fails now fails the run. --set-var injected its override right after the %run cell, so a series notebook's own parameter cell (nb04: FORCE_RELOAD = False) silently undid it. Inject after the last cell assigning that name. Co-authored-by: Isaac <no-reply@databricks.com>
Re-running nb04 over tables from an older run crashed (it reused a surface bin that still had the old `cell` column). _persist now reuses an existing table only if its columns match what the code writes, and rewrites it otherwise, so upgrades work without FORCE_RELOAD. The Lakeflow validate task failed on the gold materialized views: DESCRIBE DETAIL rejects MVs. Read clusteringColumns from table properties instead (works for Delta tables and MVs); the full-SF job now validates green (13 tables + 9 clustering checks). Simplify the h3-twin/clustering write in _persist and siting._write, and refresh nb04's stored outputs from a Serverless run against the canonical schema. Co-authored-by: Isaac <no-reply@databricks.com>
nb04's closing SHOW TABLES listed the whole shared schema, so the committed output exposed other examples' tables and stale dev leftovers (wc_spike_*, old towers/visible_pairs tables). Filter it to wc_* and refresh the stored outputs from a Serverless run after the leftovers were dropped from geospatial_docs.geobrix. Co-authored-by: Isaac <no-reply@databricks.com>
Isaac loads every changed file in full, so a scope with executed notebooks (multi-MB base64 plots) overflowed the context window and the review failed. By default the command now reviews the same change in a scratch worktree with every changed .ipynb's outputs stripped on both sides (--keep-outputs opts out); the checkout is untouched. It also fails loudly when isaac reports "No changes found" for a non-empty scope, and notes an expired dbcert certificate. Co-authored-by: Isaac <no-reply@databricks.com>
From Isaac Review: a --set-var override landed before a %run config include whenever the notebook assigned the name earlier, so the include re-assigned the default; it now goes after whichever comes last. And a harvested cell with a result type the harvester can't convert (e.g. a display() table) had its stored outputs replaced with nothing; such cells are now left untouched and reported. Co-authored-by: Isaac <no-reply@databricks.com>
The QC judge flagged _export_model as untested; it holds the NOTEBOOK-view lookup that broke once already. Stub the CLI call to cover it, the no-NOTEBOOK-view error, and harvest_run_outputs re-exporting until a too-early export's missing results appear. Co-authored-by: Isaac <no-reply@databricks.com>
mjohns-databricks
temporarily deployed
to
runtime
October 12, 2026 18:35 — with 
GitHub Actions
Inactive
GitHub Actions
Inactive
mjohns-databricks
temporarily deployed
to
runtime
October 12, 2026 18:35 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
mjohns-databricks
had a problem deploying
to
runtime
October 12, 2026 19:04 — with
GitHub Actions
Failure
GitHub Actions
Failure
mjohns-databricks
had a problem deploying
to
runtime
October 12, 2026 19:05 — with
GitHub Actions
Failure
GitHub Actions
Failure
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-ups to v0.5.2, mostly driven by running the Wireless Coverage example for a workshop, plus docs screenshots and notebook tooling.
Wireless Coverage (notebook series + Lakeflow SDP)
USE CATALOG/USE SCHEMA,_tbl,siting._fqn, validate notebook). Hyphenated names previously failed on Serverless withINVALID_IDENTIFIER._demosurface tables through_tbl.wc_h3_site_viewshed_res12. One row per (sited tower, visible res-12 cell), keyed by the tower's res-9site_cellid, in both the notebook series and the Lakeflow siting task. Map/BI tools can then score each site against coverage or population layers without double counting.h3string twin. Tables are liquid-clustered on the BIGINT cell column (CLUSTER BY, not Z-ORDER) and OPTIMIZEd after each write. Gold MVs usecluster_by.FORCE_RELOAD._persistrewrites an existing table whose columns no longer match what the code writes.DESCRIBE DETAILrejects MVs.SHOW TABLESnow lists onlywc_*tables.Docs and READMEs
Tooling (
gbx:*)gbx:test:notebooks-serverless --harvest-outputswrites a Serverless run's cell outputs back into the local.ipynb. It matches cells by source, preserves the file's serialization, and leaves cells with unconvertible result types untouched.--set-varoverrides now land after the notebook's own assignment or a later%run, whichever comes last; before, nb04's parameter cell silently undid them.gbx:review:roundstrips notebook outputs in a scratch worktree so Isaac doesn't overflow its context, and fails loudly on an empty review.Testing
geospatial_docs.wc-verify-hyphen): 01, 02a, 03a, 03b and 04 all green. Clustering keys,h3 = h3_h3tostring(cellid), and site-viewshed counts checked via SQL._writechange.This pull request and its description were written by Isaac.