Pass DIVE camera registrations to 2-cam/3-cam VIAME pipelines - #1780
Open
romleiaj wants to merge 9 commits into
Open
Pass DIVE camera registrations to 2-cam/3-cam VIAME pipelines#1780romleiaj wants to merge 9 commits into
romleiaj wants to merge 9 commits into
Conversation
romleiaj
marked this pull request as draft
July 21, 2026 19:21
Collaborator
Author
|
Paired with VIAME/VIAME#278. |
romleiaj
marked this pull request as ready for review
August 17, 2026 16:55
romleiaj
force-pushed
the
sealtk/registration-json
branch
from
August 17, 2026 17:29
b0a8b15 to
5811f58
Compare
Multicam pipelines previously received no transform information from DIVE; the old h5 workflow relied on paths hardcoded inside the pipes. Desktop and web now hand the dataset's camera registration to the pipeline's warp processes, mirroring the stereo calibration flow: - One standard <camera>_to_<reference>_registration.json per non-reference camera is written into the job work dir; warpN (matching the pipeline camera position) receives its file via -s warpN:transformation_file=..., with the pair and direction pinned through the reader's from_camera/to_camera config since a pair may be stored in either orientation. - Pipeline camera order is the registration reference first (the pipes warp everything onto camera 1's frame), then display order. Which detector a pipe runs on which input is the pipe's documented contract, not something DIVE infers. - Only pairs registering a camera directly onto the reference are supported; pairs between two non-reference cameras are explicitly unsupported for pipelines and never reach the job. Cameras with points-only (unfitted) pairs get no settings. - Web sends the registration as a multicam_registration job param built from dataset meta; the worker writes the files. Requires VIAME's dive transform_2d_io reader and warp_detections process. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The web path built its registration files from the pre-v2 correspondence
shape: a flat list of {id, a, b} per camera pair, emitted as pairs[].points
and stamped version 1. Format v2 replaced that with observations -- one
entry per image pair, each carrying its own points -- so this did not
merely write an outdated file, it raised KeyError: 'a' on the first
correspondence and failed the job in the worker. Every web multicam
pipeline run on a v2-registered dataset would have died there.
Build observations instead, as the inverse of
registration_output._from_registration_pairs: the store's imageA/imageB
become the file's imageLeft/imageRight, frame/enabled/source/stats carry
through, and each point's a/b pair becomes one `leftX leftY rightX rightY`
row. Stamp version 2.
The desktop path was never affected -- it writes through
buildPerCameraRegistrationFiles, which the format-v2 work already moved --
which is exactly why this was easy to miss: desktop produces correct v2
files while the worker crashes.
VIAME's dive transform reader only consumes the matrices, so the
observations travel for provenance and so a file round-trips back into
DIVE without losing which frame contributed what. Tests now assert the
version, which nothing did before.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014S8CVLjQDzaBGbCSKL7FiF
- cameraRegistration.spec.ts: JsonMeta does not exist (use JsonConfig); type the fs-extra mock the way common.spec.ts does so vue-tsc accepts the default export and statSync parameter types. - crud_rpc.py: drop unused pathlib import. - multicam_pipeline.py: black formatting. - test_multicam_pipeline.py: VIAME's dive reader ignores the file version; it is DIVE's format-v2 loader that rejects other versions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MD6kx1uDv4NuiKdFNnYpG2
Which dataset camera feeds which inputN of a multicam pipe is the pipe's contract -- the arctic seal 3-cam pipe runs the thermal detector on input3 and projects optical boxes onto input2 -- but DIVE was inferring it from display order (reference camera first, then the rest), so a dataset imported as EO, IR, UV silently ran the thermal detector on UV. Pipes now state their slots with a `# Camera Order: EO, UV, IR` header, parsed into pipeline metadata on desktop and web. At run time each slot is matched to a dataset camera by name: an exact camera name, or a name segment sharing the slot's role (EO ~ eo/rgb/optical/color/vis, IR ~ ir/thermal/lwir/flir, UV ~ uv/ultraviolet); other tokens match literally. A slot matching no camera or several refuses the run with a message that names the pipe's slots and the dataset's cameras. Camera 1 of the resolved order is the warp target the registration files are built onto. Pipes without the header keep the reference-first fallback. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MD6kx1uDv4NuiKdFNnYpG2
… datasets Which dataset camera feeds which inputN of a 2-cam/3-cam pipe was decided silently -- from the pipe's `# Camera Order:` header matched against camera names, and before that from display order. Make it explicit and visible: - Datasets carry a sensor role per camera (`cameraRoles`: eo / ir / uv), inferred once at import from the camera name and, failing that, from tokens in its image file names (KAMERA style _rgb / _ir / _uv). Only a unanimous answer is recorded. Desktop and web import both set it; it is a mutable config key so it round-trips through load/save on both. - Running a 2-cam/3-cam pipe opens a camera-assignment step: one row per pipeline camera (the header's slots, or bare input1..N) with the dataset camera DIVE proposes -- by role when both sides have one, else by name -- which the user confirms or changes. Unfilled or duplicated slots block the run. Confirming role-labelled slots saves the roles back onto the dataset (opt-out checkbox), so a corrected role wins over a misleading name next time and for every other pipe. - The confirmed order travels as pipelineParams.cameraOrder; desktop and web validate it against the dataset's cameras and use it verbatim. Runs without it (CLI) still resolve the header by role then name; pipes with no header keep the reference-first fallback. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MD6kx1uDv4NuiKdFNnYpG2
…re time A 2-cam/3-cam pipe whose warpN camera had no registration onto camera 1 launched anyway and died inside kwiver with "Path does not exist: registration_camera2_to_camera1.json". Do the check before the job exists: - Pipeline discovery (desktop + web) records registrationWarps: the input positions of `process warpN :: warp_detections | warp_image` in the pipe body, e.g. [2, 3]. - The camera-assignment dialog shows, for each warped row, whether the chosen camera has a fitted Aligned View registration onto camera 1, and blocks Run with "Register X -> Y in Aligned View first" when it does not. - Desktop (viame.ts) and web (crud_rpc) refuse the run with the same message before creating the job, so CLI and API callers get it too. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MD6kx1uDv4NuiKdFNnYpG2
…stration-missing message Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MD6kx1uDv4NuiKdFNnYpG2
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MD6kx1uDv4NuiKdFNnYpG2
The design settled on the camera-assignment step always supplying the input1..N order, which left the earlier automatic header resolution as a dead path with a lot of code behind it. Remove it and flatten what is left: - Drop resolvePipelineCameraOrder / candidatesForSlot / camerasMatchingSlot / pipelineCameraNames / pipelineOrderedCameraNames on the client and resolve_pipeline_camera_order / candidates_for_slot / cameras_matching_slot / pipeline_camera_order on the server. Without a confirmed cameraOrder (API callers) both backends now use the dataset's stored camera order, exactly as main does. One matcher, camerasForSlot, remains for the dialog prefill. - buildRegistrationPipelineArgs takes the pipe's warp positions and throws the register-first error itself, instead of viame.ts reverse-engineering the fitted set from the -s keys it had just built; its three guards for "this camera has a fitted pair onto camera 1" become one. - Web import infers camera roles from names only (no per-folder item scan); the server's infer_camera_role drops its image-name fallback accordingly. Desktop keeps the image-name fallback since it has the list in hand. No change on the dialog path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MD6kx1uDv4NuiKdFNnYpG2
romleiaj
force-pushed
the
sealtk/registration-json
branch
from
August 19, 2026 16:58
5811f58 to
58e1959
Compare
Collaborator
Author
Pass DIVE camera registrations to 2-cam/3-cam VIAME pipesMulticam pipes got no transform info from DIVE; paths were hard-coded in the This PR sends the dataset's registration to the pipe's warp processes and makes Registration hand-off
Camera assignment
Registration check before the run
Cleanup
Manual testsNeeds VIAME from
StackStandalone on Overall: 21 files, +1537/-10. Commits: 8bb338f through 58e1959. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Multicam pipelines previously received no transform information from DIVE; the old h5 workflow relied on paths hardcoded inside the pipes. Desktop and web now hand the dataset's camera registration to the pipeline's warp processes, mirroring the stereo calibration flow:
Requires VIAME's dive transform_2d_io reader and warp_detections process.