Skip to content

trailview: bake the coarse stamps and the structure meshes (#3194) - #3209

Merged
timohueser merged 8 commits into
ground-planfrom
claude/gp-coarse-stamp-bake
Nov 8, 2026
Merged

timohueser merged 8 commits into
ground-planfrom
claude/gp-coarse-stamp-bake

Conversation

@timohueser

@timohueser timohueser commented Nov 8, 2026 •

Copy link
Copy Markdown
Owner

Summary

Step 1c of the ground-plan load plan (#3194). The coarse stamps (levels 9 to 12) and the structure meshes (level 15) are a pure function of the release data. This change bakes them, and the viewer reads them before it asks the stamp worker.

  • Bake step. planner/trailview-stamps is a step of the layer planner/trailview-features (a step reads only client files of another layer, so it belongs to the layer that it reads). It runs the viewer's own stamp code in Node, as the owner decided on TrailView ground plan, step 1c: bake the coarse stamps (L9–L12) and the structure meshes #3194: builder/web/scripts/trailview-stamps.mjs. Rust (host/obc-trailview-features/src/stamps.rs) reads the feature archive, sends each feature tile to up to 8 Node processes, and writes trailview-stamps.pmtiles (gzip tiles, tile z/x/y = terrain node z/x/y).
  • What a node holds. At levels 9 to 12, the whole result that the stamp worker posts with the ground plan. At level 15, only its structure meshes, and only for a node that has any; at zoom 13, a u16 mask of the level-15 nodes that the layer holds. The bytes: terrain/node-bytes.ts (a u32 JSON length, the JSON, then each typed array at an offset of 8 bytes). Each water, structure and anchored mesh is packed: its equal vertices once, in the order of their first use (the encoder welds; MeshBuilder and the live meshes do not change), x and y as int16 from the node's middle where they fit, and meshopt's vertex and index-sequence codes. The viewer unpacks a node to the exact bytes that the worker posts. The bake drives stamp-worker.ts's own onmessage.
  • Version. stamp_version = the start of the SHA-256 of the bake module. vite/stamp-bundle.ts makes the module with a Vite SSR build of terrain/stamp-bake.ts. The viewer gets the same version at build time from virtual:stamp-version. The step's option stamp_sources is the digest of the files that the stamp code can read, so obc data shows the step stale ("options") and bakes only it again when the stamp code changes.
  • Viewer. stream/terrain.ts bakedNodes opens the layer once, at the start. Stamps.features reads a baked node first. A missing layer, a layer that fails to open, a layer of another stamp_version, a zoom outside the layer, an absent node and a node whose bytes do not decode all go to the stamp worker, as today. For a level-15 node, the mask of its feature tile says whether to fetch it; the worker job (meshes: false) runs in parallel with the fetch.
  • The step runs up to 8 Node processes, as many as the free memory gives 1.5 GiB each (V8 heap capped at 1 GiB; measured peak 1.29 GB), with 300 s for each tile. The first process that fails stops the others. obc data setup needs Node 22.18 for the planner; the stamp process says so on an older Node.
  • Without ground=plan the viewer reads only the level-15 meshes: the coarse stamps are baked with the ground plan.

The change stays in the stamp worker's fetch path, the worker's message (one flag) and the new bake files, so #3193 merges with it.

Mode: change #3194
Requirements: none

Gates (after the review fixes, 4472bd7 and e2ff9dd)

Measured on timosPC (Ryzen 5 3600, GTX 1660 SUPER, shared, load 2–8). Fixture = the Engelberg fixture of tests/browser/trailview/fixture. base = ground-plan f518fd8. none = this branch without the layer (it also holds the merge of ground-plan e810ec3, so base and none differ by more than this change). layer = this branch with the layer. Clean frame = the first screenshot after which each screenshot differs from the final frame in less than 1 % of the pixels; 1920 × 960, ground=plan, GPU. Repetitions a / b.

Invariants

Gate Result
A baked node equals the worker's output (fixture) The step on the fixture; the same stamp code in Chromium (Playwright) unpacks each baked node and compares it with what it stamps itself: 0 of 192 nodes differ, and the 9 masks name exactly the 81 owner nodes with meshes. The packed codec has a vitest of its own on real nodes (node-bytes.test.ts).
No baked byte depends on the client The bake reads only the feature tile and the node; the ground plan is fixed for levels 9–12.
A missing, stale or bad layer costs no more than today Missing: no request. Stale: one TileJSON request, then none (first round). A node that does not decode, at level 9–12 or 15: the worker makes it (stamps.test.ts). An empty owner node: no request.
The default path is unchanged MeshBuilder, the live meshes and the worker's output do not change; without a layer the viewer asks the worker as before.

Time to a clean frame (s, a / b)

View Mode base none layer
v17 (fixture) cold 4.9 / 6.9 3.7 / 4.0 3.2 / 4.1
v17 (fixture) warm 4.8 / 5.0 3.6 / 3.6 2.9 / 3.0
m1 (fixture) cold 2.9 / 2.9 2.8 / 3.0 2.9 / 3.0
m1 (fixture) warm 2.6 / 2.7 2.8 / 2.8 4.0 (1) / 2.9
Lucerne old town (local release) cold 62.1 (1) / 35.0 53.1 (1) / 35.5 24.3 / 24.4
Lucerne old town (local release) warm 34.0 / 35.0 35.0 / 33.5 24.2 / 24.3

(1) Single slow runs while the PC had other load; the other repetition is the usual value.

Throttled network (Chromium emulation 20 Mbit/s, 40 ms; one cold load each; local release)

View none layer (stamps uncompressed) layer (stamps gzip)
v17 8.6 9.3 8.8
Lucerne old town 34.0 28.0 27.0

The layer still wins in the town (−7 s): its stamp worker work ends at 18 s instead of 55 s. At v17 it is even: the 28 MB of the view's other downloads take about 11 s at 20 Mbit/s, and the stamp worker is no longer the long pole. The stamp nodes of a view are 1.8 MB (v17) and 2.9 MB (Lucerne) gzip. My local proxy served the layer; the release tile service and its block index add round trips that this does not show.

Stamp worker per level

Summed job time (from the message to the post; jobs overlap while a tile decodes), repetition a (Lucerne: b), s:

View Side L9 L10 L11 L12 L15 All levels Last job ends
v17 base – – – 0.03 6.1 17.5 5.4
v17 layer – – – 0 0.18 5.5 5.1
m1 base – – – 0.05 0.78 3.7 4.0
m1 layer – – – 0 0.12 2.3 4.0
Lucerne base 4.4 3.0 1.5 0.67 36.3 54.6 55.5
Lucerne layer 0 0 0 0 0.16 12.7 23.5

Chrome trace of one cold Lucerne load: stamp worker busy base 52.0 s / none 53.2 s / layer 12.1 s; idle at 53.0 / 54.2 / 24.3 s; all threads 151 / 155 / 91 s. With the layer the plan workers end last.

The frame time (GP0's flights) and the 20 owner views were measured in the first round (no change, 0 pixels differ); the review fixes change no GPU data and no draw, so I did not run them again.

Owner decision: the size

As built (8db6f5d) Packed (this push)
Fixture (Engelberg, 25 feature tiles) 1.92 MB 0.34 MB
Central Switzerland, levels 9–12 (all 356 nodes, measured) 8.8 MB 4.5 MB
Central Switzerland, level 15: sample of 120 of about 905 z13 tiles (measured) 23.4 MB 5.2 MB
Central Switzerland, level 15, extrapolated (× 905 / 120) about 176 MB about 39 MB
Central Switzerland, all about 185 MB about 44 MB
Share of the 1.3 GB release +14 % +3.4 %

All sizes are gzip, as the archive stores the nodes. The packed level 15 is 0.043 MB a z13 tile, against 0.057 MB for the feature tile that the client fetches anyway. The review's variant X (about 33 MB) stores welded indices; to give back the worker's exact bytes, this codec also stores which distinct vertex each vertex is, which costs about 1 byte a vertex. The local layer of the measurements (all z8–z11 tiles of the box and 111 z13 tiles about Lucerne and Engelberg) went from 27.5 MB for 49 z13 tiles to 10.6 MB for 111.

Bake time: the CS sample (89 low and 120 z13 tiles) takes 125 s in one process (114 s before the packing); Central Switzerland is about 15 min of CPU. On timosPC now, the free memory gives the step 1 process (1.5 GiB each); with 2 processes the peak is about 2.6 GB. The Lucerne and Engelberg layer took 173 s in 1 process.

Options for the owner: ship both levels packed (about 44 MB a region), or only levels 9–12 (about 4.5 MB; one condition in nodesOf; Lucerne then keeps its 36 s of level-15 jobs).

Checks

Run (after the review fixes):

  • obc test -p obc-trailview-features -p obc-data-steps -p obc-data: 432 passed, 1 skipped.
  • obc exec cargo clippy -p obc-trailview-features -p obc-data-steps -p obc-data --all-targets -- -D warnings: clean. cargo fmt --all: clean.
  • npx vitest run src/lib/trailview: 60 files, 414 tests passed. npm run check: 0 errors, 0 warnings.
  • obc prose --check: pass.
  • The fixture step and the Chromium equality check, the fixture and Lucerne clean-frame matrices, the Lucerne traces, the throttled loads.
  • First round also: the repository-tools Python suite (327 passed), npm run check:wgsl, the docs link check, obc suites check, GP0's flights, the owner views.

Not run: the whole workspace, the browser suites (CI), a step through obc data update (no ad-hoc bake of a release; the step ran through its function on the fixture and on release tiles in temporary folders on OSM1), Safari.

Deviations

  • The issue says "a PMTiles layer"; the stamps are a second step of the layer planner/trailview-features, not a layer of their own: the test each_step_belongs_to_the_layer_of_its_producer forbids a step that reads non-client files of another layer. A new feature bake bakes the stamps too.
  • The step's option is stamp_sources, a digest of the files that the stamp code can read, not the exact stamp_version: the exact version needs Node and the npm packages at plan time, which the Rust tests and a plan on another machine do not have. The digest changes with every change of the version, and with some others (any TypeScript of src/lib/trailview/): then only this step bakes again. The layer is useful only when the bake and the deployed viewer come from the same stamp code; rebakes follow the owner's schedule.
  • Not changed from the review: F7 (the first nodes wait for the layer's TileJSON, which starts with eyeTiles), F11 (a region without a feature tile fails the step; zoom 12 tiles go through the pipe for nothing).
  • The bake makes the whole worker result of a level-15 node and keeps only the meshes (simple; the bake time is small).
  • Measured with two local patches that are not in the branch: a timer in stamp-worker.ts (job time per level, [diag] lines) and OBC_STAMPS in the harness to serve a layer to the fixture page. The measured layers were baked with the patched code, so their version matched the page.

Public docs changed: no (the contracts specs/trailview-tiles.md and specs/obc-data.md changed).
Pictures and runs: /media/timo/OSM1/tmp/ground-plan/stamp-bake/ (runs/, shots/, flights/, STATUS.md).

🤖 Generated with Claude Code

timohueser and others added 4 commits November 7, 2026 23:36
The obc data step planner/trailview-stamps runs the stamp worker's code
in Node on each tile of planner/trailview-features. It writes the bytes
that the worker posts for each node of levels 9 to 12 (with the ground
plan) and the structure meshes of each level-15 node into
trailview-stamps.pmtiles. Its stamp_version is the start of the SHA-256
of the bake module, which Vite makes of the stamp code and the tables
that it reads.

The viewer reads a node from the layer before it asks the stamp worker,
when the layer's stamp_version is the version of its own build. Else
the worker makes the node, as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A step reads only client files of another layer, so the stamp step is a
step of planner/trailview-features, as its own reads demand. A new
version of the feature layer bakes it again.

Workers bundle their imports with plugins of their own: they get the
stamp version plugin too, and the page and the workers share one
bundle. The stamp process writes logs of the stamp code to standard
error, because standard output holds its frames.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test ran past the 5 s timeout in a loaded vitest run. A smaller
owner node with structure meshes, points and anchors gives the same
cover in about 1 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: b155d252-b0b6-4409-847c-6eefdbabc70f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

timohueser and others added 3 commits November 8, 2026 01:48
The stamp layer stores each water, structure and anchored mesh packed:
its equal vertices once, in the order of their first use, x and y as
int16 from the node's middle where they fit, and meshopt's vertex and
index sequence codes. The viewer unpacks a node to the bytes that the
worker posts. The bake welds only in the encoder; MeshBuilder and the
live meshes do not change.

The layer holds only the owner nodes that have a structure mesh, and a
mask of them for each zoom 13 feature tile, so an empty owner node
costs no request. The worker job of an owner node starts with its
baked meshes. A node whose bytes do not decode comes from the worker.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…stops

The option stamp_sources of planner/trailview-stamps is the digest of
the files that the stamp code can read, so a change of the code stales
and bakes only this step. obc data setup needs Node 22.18 for the
planner (the stamp process imports TypeScript), and the process says
so on an older Node.

Each stamp process has a capped heap and 300 s for a tile. The count of
processes follows the free memory. The first process that fails stops
the others, and its error is the step's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A vitest lists the modules of the stamp bundle and fails for each that
the digest rule of stamp_sources does not hold, so a new import outside
it stops a test run instead of leaving the step fresh. The memory
budget comment of the stamp step holds no measurement.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@timohueser
timohueser marked this pull request as ready for review November 8, 2026 09:00
@timohueser
timohueser enabled auto-merge November 8, 2026 09:00
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
timohueser added a commit that referenced this pull request Nov 8, 2026
With the ground plan, Stamps reads a node from the device cache before it
asks the stamp worker, and writes each node that the worker makes. The
key holds the release and the stamp version of #3209: the SHA-256 of the
stamp worker bundled with every module that it imports. The node bytes
are encodeNode of #3209, which gives a node back exactly. The device
cache takes a kind, so the cells and the nodes keep caches of their own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@timohueser
timohueser merged commit 5edc816 into ground-plan Nov 8, 2026
56 of 59 checks passed
@timohueser
timohueser deleted the claude/gp-coarse-stamp-bake branch November 8, 2026 11:53
timohueser added a commit that referenced this pull request Nov 8, 2026
# Conflicts:
#	builder/web/src/lib/trailview/engine.ts
#	builder/web/src/lib/trailview/terrain/stamps.ts
#	builder/web/src/lib/trailview/terrain/terrain.ts
#	builder/web/vite.config.ts
#	builder/web/vite/stamp-bundle.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant