Repository navigation
trailview: bake the coarse stamps and the structure meshes (#3194) - #3209
Merged
Merged
Conversation
The obc data step planner/trailview-stamps runs the stamp worker's code in Node on each tile of planner/trailview-features. It writes the bytes that the worker posts for each node of levels 9 to 12 (with the ground plan) and the structure meshes of each level-15 node into trailview-stamps.pmtiles. Its stamp_version is the start of the SHA-256 of the bake module, which Vite makes of the stamp code and the tables that it reads. The viewer reads a node from the layer before it asks the stamp worker, when the layer's stamp_version is the version of its own build. Else the worker makes the node, as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A step reads only client files of another layer, so the stamp step is a step of planner/trailview-features, as its own reads demand. A new version of the feature layer bakes it again. Workers bundle their imports with plugins of their own: they get the stamp version plugin too, and the page and the workers share one bundle. The stamp process writes logs of the stamp code to standard error, because standard output holds its frames. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test ran past the 5 s timeout in a loaded vitest run. A smaller owner node with structure meshes, points and anchors gives the same cover in about 1 s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
|
The stamp layer stores each water, structure and anchored mesh packed: its equal vertices once, in the order of their first use, x and y as int16 from the node's middle where they fit, and meshopt's vertex and index sequence codes. The viewer unpacks a node to the bytes that the worker posts. The bake welds only in the encoder; MeshBuilder and the live meshes do not change. The layer holds only the owner nodes that have a structure mesh, and a mask of them for each zoom 13 feature tile, so an empty owner node costs no request. The worker job of an owner node starts with its baked meshes. A node whose bytes do not decode comes from the worker. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…stops The option stamp_sources of planner/trailview-stamps is the digest of the files that the stamp code can read, so a change of the code stales and bakes only this step. obc data setup needs Node 22.18 for the planner (the stamp process imports TypeScript), and the process says so on an older Node. Each stamp process has a capped heap and 300 s for a tile. The count of processes follows the free memory. The first process that fails stops the others, and its error is the step's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A vitest lists the modules of the stamp bundle and fails for each that the digest rule of stamp_sources does not hold, so a new import outside it stops a test run instead of leaving the step fresh. The memory budget comment of the stamp step holds no measurement. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
timohueser
marked this pull request as ready for review
November 8, 2026 09:00
timohueser
enabled auto-merge
November 8, 2026 09:00
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
timohueser
added a commit
that referenced
this pull request
Nov 8, 2026
With the ground plan, Stamps reads a node from the device cache before it asks the stamp worker, and writes each node that the worker makes. The key holds the release and the stamp version of #3209: the SHA-256 of the stamp worker bundled with every module that it imports. The node bytes are encodeNode of #3209, which gives a node back exactly. The device cache takes a kind, so the cells and the nodes keep caches of their own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
timohueser
added a commit
that referenced
this pull request
Nov 8, 2026
# Conflicts: # builder/web/src/lib/trailview/engine.ts # builder/web/src/lib/trailview/terrain/stamps.ts # builder/web/src/lib/trailview/terrain/terrain.ts # builder/web/vite.config.ts # builder/web/vite/stamp-bundle.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Step 1c of the ground-plan load plan (#3194). The coarse stamps (levels 9 to 12) and the structure meshes (level 15) are a pure function of the release data. This change bakes them, and the viewer reads them before it asks the stamp worker.
planner/trailview-stampsis a step of the layerplanner/trailview-features(a step reads only client files of another layer, so it belongs to the layer that it reads). It runs the viewer's own stamp code in Node, as the owner decided on TrailView ground plan, step 1c: bake the coarse stamps (L9–L12) and the structure meshes #3194:builder/web/scripts/trailview-stamps.mjs. Rust (host/obc-trailview-features/src/stamps.rs) reads the feature archive, sends each feature tile to up to 8 Node processes, and writestrailview-stamps.pmtiles(gzip tiles, tile z/x/y = terrain node z/x/y).terrain/node-bytes.ts(a u32 JSON length, the JSON, then each typed array at an offset of 8 bytes). Each water, structure and anchored mesh is packed: its equal vertices once, in the order of their first use (the encoder welds;MeshBuilderand the live meshes do not change), x and y as int16 from the node's middle where they fit, and meshopt's vertex and index-sequence codes. The viewer unpacks a node to the exact bytes that the worker posts. The bake drivesstamp-worker.ts's ownonmessage.stamp_version= the start of the SHA-256 of the bake module.vite/stamp-bundle.tsmakes the module with a Vite SSR build ofterrain/stamp-bake.ts. The viewer gets the same version at build time fromvirtual:stamp-version. The step's optionstamp_sourcesis the digest of the files that the stamp code can read, soobc datashows the step stale ("options") and bakes only it again when the stamp code changes.stream/terrain.tsbakedNodesopens the layer once, at the start.Stamps.featuresreads a baked node first. A missing layer, a layer that fails to open, a layer of anotherstamp_version, a zoom outside the layer, an absent node and a node whose bytes do not decode all go to the stamp worker, as today. For a level-15 node, the mask of its feature tile says whether to fetch it; the worker job (meshes: false) runs in parallel with the fetch.obc data setupneeds Node 22.18 for the planner; the stamp process says so on an older Node.ground=planthe viewer reads only the level-15 meshes: the coarse stamps are baked with the ground plan.The change stays in the stamp worker's fetch path, the worker's message (one flag) and the new bake files, so #3193 merges with it.
Mode: change #3194
Requirements: none
Gates (after the review fixes, 4472bd7 and e2ff9dd)
Measured on timosPC (Ryzen 5 3600, GTX 1660 SUPER, shared, load 2–8). Fixture = the Engelberg fixture of
tests/browser/trailview/fixture. base =ground-planf518fd8. none = this branch without the layer (it also holds the merge ofground-plane810ec3, so base and none differ by more than this change). layer = this branch with the layer. Clean frame = the first screenshot after which each screenshot differs from the final frame in less than 1 % of the pixels; 1920 × 960,ground=plan, GPU. Repetitions a / b.Invariants
node-bytes.test.ts).stamps.test.ts). An empty owner node: no request.MeshBuilder, the live meshes and the worker's output do not change; without a layer the viewer asks the worker as before.Time to a clean frame (s, a / b)
(1) Single slow runs while the PC had other load; the other repetition is the usual value.
Throttled network (Chromium emulation 20 Mbit/s, 40 ms; one cold load each; local release)
The layer still wins in the town (−7 s): its stamp worker work ends at 18 s instead of 55 s. At v17 it is even: the 28 MB of the view's other downloads take about 11 s at 20 Mbit/s, and the stamp worker is no longer the long pole. The stamp nodes of a view are 1.8 MB (v17) and 2.9 MB (Lucerne) gzip. My local proxy served the layer; the release tile service and its block index add round trips that this does not show.
Stamp worker per level
Summed job time (from the message to the post; jobs overlap while a tile decodes), repetition a (Lucerne: b), s:
Chrome trace of one cold Lucerne load: stamp worker busy base 52.0 s / none 53.2 s / layer 12.1 s; idle at 53.0 / 54.2 / 24.3 s; all threads 151 / 155 / 91 s. With the layer the plan workers end last.
The frame time (GP0's flights) and the 20 owner views were measured in the first round (no change, 0 pixels differ); the review fixes change no GPU data and no draw, so I did not run them again.
Owner decision: the size
All sizes are gzip, as the archive stores the nodes. The packed level 15 is 0.043 MB a z13 tile, against 0.057 MB for the feature tile that the client fetches anyway. The review's variant X (about 33 MB) stores welded indices; to give back the worker's exact bytes, this codec also stores which distinct vertex each vertex is, which costs about 1 byte a vertex. The local layer of the measurements (all z8–z11 tiles of the box and 111 z13 tiles about Lucerne and Engelberg) went from 27.5 MB for 49 z13 tiles to 10.6 MB for 111.
Bake time: the CS sample (89 low and 120 z13 tiles) takes 125 s in one process (114 s before the packing); Central Switzerland is about 15 min of CPU. On timosPC now, the free memory gives the step 1 process (1.5 GiB each); with 2 processes the peak is about 2.6 GB. The Lucerne and Engelberg layer took 173 s in 1 process.
Options for the owner: ship both levels packed (about 44 MB a region), or only levels 9–12 (about 4.5 MB; one condition in
nodesOf; Lucerne then keeps its 36 s of level-15 jobs).Checks
Run (after the review fixes):
obc test -p obc-trailview-features -p obc-data-steps -p obc-data: 432 passed, 1 skipped.obc exec cargo clippy -p obc-trailview-features -p obc-data-steps -p obc-data --all-targets -- -D warnings: clean.cargo fmt --all: clean.npx vitest run src/lib/trailview: 60 files, 414 tests passed.npm run check: 0 errors, 0 warnings.obc prose --check: pass.npm run check:wgsl, the docs link check,obc suites check, GP0's flights, the owner views.Not run: the whole workspace, the browser suites (CI), a step through
obc data update(no ad-hoc bake of a release; the step ran through its function on the fixture and on release tiles in temporary folders on OSM1), Safari.Deviations
planner/trailview-features, not a layer of their own: the testeach_step_belongs_to_the_layer_of_its_producerforbids a step that reads non-client files of another layer. A new feature bake bakes the stamps too.stamp_sources, a digest of the files that the stamp code can read, not the exactstamp_version: the exact version needs Node and the npm packages at plan time, which the Rust tests and a plan on another machine do not have. The digest changes with every change of the version, and with some others (any TypeScript ofsrc/lib/trailview/): then only this step bakes again. The layer is useful only when the bake and the deployed viewer come from the same stamp code; rebakes follow the owner's schedule.eyeTiles), F11 (a region without a feature tile fails the step; zoom 12 tiles go through the pipe for nothing).stamp-worker.ts(job time per level,[diag]lines) andOBC_STAMPSin the harness to serve a layer to the fixture page. The measured layers were baked with the patched code, so their version matched the page.Public docs changed: no (the contracts
specs/trailview-tiles.mdandspecs/obc-data.mdchanged).Pictures and runs:
/media/timo/OSM1/tmp/ground-plan/stamp-bake/(runs/,shots/,flights/,STATUS.md).🤖 Generated with Claude Code