Integrate CoolscanPy LS-5000 whole-roll scanning#613
Closed
rohanpandula wants to merge 51 commits into
Closed
Conversation
coolscanpy is a standalone, SANE-free library for whole-roll scanning on a Nikon Coolscan LS-5000 with a roll feeder. Add it as a new dependency-group, following the same pattern as the existing "scanner" group for python-sane: opt-in, not a hard dependency of the rest of the application.
CoolscanRoll wraps coolscanpy's Device/Roll workflow (preview, spacing correction, approval, batch fine-scan, safe stop) behind a small RollHandle, translating coolscanpy's typed PyCoolscanError hierarchy to plain RuntimeErrors the same way SaneBackend already reports failures elsewhere in this package. available() is a cheap importlib.util.find_spec() check, mirroring ScanSidebar's existing _sane_available() gate, so the feature degrades to fully absent when coolscanpy isn't installed. open_roll() is the single place this adapter resolves a device handle, marked as the intended re-point target once the maintainer's own generic SANE-based coolscan route defines a real backend-selection seam on ScannerService. Tests inject a fake coolscanpy module (tests/scanners/conftest.py) so none of this requires real hardware or the real package installed.
RollScanningService orchestrates the coolscanpy_roll adapter: open/ close a roll (one at a time, matching coolscanpy's own reservation lock), preview, approve, spacing correction, batch scan_many, and safe_stop. write_frame() writes each scanned Frame to disk in NegPy's existing scan-output shape: a 16-bit TIFF master, an "_IR" TIFF sidecar when the frame carries an infrared plane (matching writer.write_tiff_16bit's own suffix), and an "_receipt.json" sidecar with the full Receipt. The filename pattern is the same Jinja2 template the plain Scan panel uses, but seq is seeded from the frame's physical slot number rather than probed for a free name, so redoing one bad slot overwrites that slot's files instead of accumulating a second copy. ir_validity is intentionally not persisted; nothing downstream reads it yet.
Explains what the integration adds, install steps, the LS-5000/SA-30 hardware caveat, the on-disk output convention, error handling, and where the single integration point lives for a future re-point onto the maintainer's own generic SANE-based coolscan route. Also documents the follow-up: no contact-sheet GUI panel yet, and where one would plug in.
The camera capture route lives beside scanners/ and scanning/ as its own infrastructure/capture + services/capture pair, gated on an optional dependency, rather than inside the single-frame ScannerBackend protocol. Give roll scanning the same shape: infrastructure/roll/coolscanpy_roll.py and services/roll/service.py, instead of sitting inside the plain-SANE scanners/scanning packages where it doesn't implement that protocol and never will. Also adds a RollScanningError for this service's own lifecycle misuse (mirrors CaptureError), an is_safe_stop() helper next to the existing exception translation so a future worker can tell a deliberate safe_stop() outcome from a real failure, and precise on_progress/return typing sourced from coolscanpy's actual Progress/Thumbnail/Frame types. No behavior change to the preview/approve/scan/write workflow itself.
Mirrors CaptureWorker's shape: a QObject moved to its own QThread, request dataclasses for the operations with a progress/status lifecycle (RollPreviewRequest, RollBatchScanRequest), and one open hardware session held across calls rather than reopened per operation. _ensure_open() lazily opens the roll for whichever device a preview or batch-scan request names, closing a previous reservation on a device switch, mirroring _acquire_camera()'s idempotent-session pattern. Cancellation has no local threading.Event the way CaptureWorker's does: a fine-scan in flight can't be aborted mid-pull, so safe_stop() (thread- safe, called directly like CaptureWorker.cancel()) is the only stop, and it always lets the in-flight frame finish. run_batch_scan tells that clean stop apart from a genuine failure via coolscanpy_roll.is_safe_stop(). Extends the tests/roll/ fake_coolscanpy fixture's FakeRoll to actually call on_progress during preview()/scan_many(), matching coolscanpy's real Progress-object shape, so the worker's progress-forwarding has real coverage instead of being untestable.
Structured like ScanlightSidebar: a pinned primary action, a gate hint that names what's still missing, sections for device and output settings, one status line plus progress bar, and settings persisted the same way (RollScanSettings under a global-setting key, loaded/saved like ScanlightSettings). _apply_gating() mirrors _missing_requirements() / _apply_gating() there, split into a narrower preview gate and a fuller scan gate. The contact sheet, per-slot spacing/approval controls and the safe-stop button have no camera-route analogue -- coolscanpy's preview/approve/ batch-scan workflow doesn't exist on that side. Built the simplest way that fits the surrounding code: a plain QListWidget in icon mode (not the file browser's heavier ThumbnailGridView, which assumes NegPy's own asset model) with a small local numpy-to-QPixmap conversion, since a scanner preview frame isn't a color-managed pipeline buffer. Safe Stop is its own control, not a Scan/Stop button toggle: unlike a camera exposure, a fine-scan in flight can't be aborted mid-pull, so the button explicitly says what it does -- finish the current frame, then stop -- rather than implying an instant cancel.
AppController gets a roll_thread + RollWorker pair, lazily started like the capture thread (_ensure_roll_thread mirrors _ensure_capture_thread), request/result signal pairs for every RollWorker operation, and public methods the sidebar calls (request_roll_devices, start_roll_preview, set_roll_spacing_offset, approve_roll_slot, start_roll_scan, roll_safe_stop). roll_safe_stop() calls the worker directly, same as cancel_capture()/cancel_scan() -- no thread needs to be running for a thread-safe Event.set() to be worth queuing. _on_roll_scan_finished feeds written frames into NegPy's file list via request_asset_discovery(), the same seam the plain Scan and Camera Scanning routes already use for their own output. Unlike _on_capture_finished it does no rgbscan_mode/process_mode bookkeeping: each roll frame is already one complete per-slot RGB TIFF, not an R/G/B triplet needing a merge flag. RightPanel's Scan tab gains a third collapsible section, "Roll Scanning", alongside the existing "Scanner (SANE)" and "Camera Scanning" sections, and on_activated() now reaches all three sidebars on tab switch.
Mirrors test_capture_controller.py: exercises AppController._on_roll_scan_finished directly against a mock controller, so a future rename of roll_finished/request_asset_discovery fails a fast unit test instead of only showing up as scanned frames silently not appearing in the file browser.
Rewrites the module layout section around infrastructure/roll and services/roll instead of the old scanners/scanning locations, documents RollScanningError and is_safe_stop() alongside the existing exception- translation section, points the integration-point cross-reference at the new file paths, and replaces the old "not built yet" note (written before this branch had a UI at all) with a description of what the panel actually does and the three deliberate simplifications in it: no material picker, no separate close-roll button, and Safe Stop as its own control rather than a Scan/Stop toggle. Also comments the coolscan-roll dependency group in pyproject.toml, matching the existing camera group's comment.
Types _thumbnails and the worker-signal handler parameters against coolscanpy's real Thumbnail/DeviceInfo (TYPE_CHECKING-only, matching the lazy-import discipline everywhere else) and RollFrameOutput, instead of bare object/list. Guards contact_sheet.item(i) against the None a QListWidgetItem lookup can return per the PyQt6 stubs, even though the loop's own range() never actually produces an out-of-range index.
The previous wording implied the camera route's threading.Event cancel is "instant"; it isn't, it's checked between channels the same way safe_stop is checked between frames here. The real reason RollWorker carries no local cancel flag of its own is that coolscanpy already exposes Roll.safe_stop() for this, where gphoto2 exposes nothing equivalent and CaptureWorker had to build its own.
RollScanningService will write a frame's dust/scratch-repaired RGBI as its own output tier, but no repair engine ships in this package. This adds the plug point: available()/register_engine()/repair(), with nothing registered by default so a caller degrades instead of writing data mislabeled as repaired.
render_positive() inverts a scanner-linear RGB buffer to a positive by calling ImageProcessor.run_pipeline directly on the in-memory array -- the same DarkroomEngine entry point a freshly imported negative reaches on first open, and the same one NegPy's own export pipeline calls after decoding a file. A stock WorkspaceConfig is used (C41, print intent, auto exposure on): the ordinary default conversion, not a custom one, since this tier is a regenerable view rather than an edit. Each call gets a fresh source_hash rather than one derived from the frame, deliberately: the engine's stage cache reuses a prior render when both the source hash and the settings hash match, and every Tier-3 call here shares the same stock settings, so a stable hash reused across two different frames would silently hand back a previous frame's pixels.
write_unrepaired, write_repaired and write_positive select which of the three output tiers a batch scan writes, independently of one another rather than as a single three-way choice. write_unrepaired defaults on: it is the archival master and the only tier the scanner itself can reproduce. repair_mode carries the Tier-2 variant (exact or hybrid) once a repair engine is registered.
write_frame() now writes up to three output tiers per frame instead of always writing the one it used to: unrepaired (Tier 1, the archival master), repaired (Tier 2, via the repair seam), and positive (Tier 3, Tier 2 inverted). Any combination is valid. Tier 3 always derives from Tier 2's in-memory result, computing repair whenever either flag needs it even if Tier 2 itself isn't written to disk, and never from a Tier 1 or Tier 2 file already on disk. Tier 2 and Tier 3 degrade with a status in the receipt instead of raising when they can't be produced -- no repair engine registered, no infrared plane to guide one, the engine or the render failing -- so a problem in a derived tier never costs the archival capture. Naming follows the existing _IR-sidecar convention: _repaired.tif plus _repaired_IR.tif for Tier 2 (the infrared sidecar is Tier 1's own, retained unchanged since repair consumes it rather than producing a repaired version of it), _positive.tif for Tier 3. The receipt's existing top-level fields (coolscanpy's own per-frame receipt) are unchanged; a new "outputs" key records what was written per tier, and for Tier 2/3 the provenance needed to audit or regenerate them -- engine name, version and mode for Tier 2, and render intent, process mode, auto-exposure setting and the repair that fed it for Tier 3. RollFrameOutput's rgb_path is now optional to match: Tier 1 can be turned off same as the other two.
RollBatchScanRequest gains write_unrepaired/write_repaired/ write_positive/repair_mode, defaulting to RollScanSettings.defaults() (Tier 1 only) so a request built without naming them behaves exactly as before. run_batch_scan forwards them to write_frame().
Three checkboxes (Unrepaired/Repaired/Positive) and a repair-mode dropdown in the Output section, seeded from and persisted through RollScanSettings the same way the folder and filename-pattern fields already are. Scanning is gated on at least one tier being selected, matching the existing device/slots/folder gates. Unchecking Unrepaired shows a warning label: it's the only tier the scanner itself can reproduce, so turning it off is a real tradeoff the UI should say out loud rather than a setting like any other.
rgb_path can now be None (Tier 1 turned off in favor of Tier 2/3 alone), which _on_roll_scan_finished's discovery seam didn't account for. Filters those out instead of handing NegPy's asset discovery a None path; a batch with at least one real Tier-1 output still discovers those normally.
Explains Tier 1 (unrepaired, the archival master), Tier 2 (repaired, cached because it's expensive but deterministic), and Tier 3 (positive, cheap and treated as a regenerable view since NegPy's color rendering is still being tuned), the capture-then-repair-then- invert ordering, what each tier's receipt entry records, how the degrade paths behave, and the storage cost of writing every tier.
Contributor
Author
|
Closing this in favor of #615 — a thin slice of just the glue (optional dependency group + the The broader pieces that were in this PR — the desktop sidebar, background worker, repair tiers, Nikon-exact color builders, and live-evidence contracts — will come back as separate stacked PRs on top of #615, so each can be reviewed on its own. Nothing is lost; the full branch is preserved locally. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This integrates the standalone CoolscanPy LS-5000/SA-21 whole-roll backend into NegPy as an optional dependency and desktop workflow.
The integration provides:
Hardware validation
Validated live on 2026-07-23 with a Nikon Super Coolscan 5000 ED (firmware 1.03) and converted SA-21/SA-30 feeder:
Software validation
mainis merged; the PR is conflict-free and mergeable.tycheck for the touched SANE eject backend passes.970d18e4b092737a633bd20141d3eaa62b25dcc6; its integration PR is merged inrohanpandula/coolscanpyand CI passes on macOS/Ubuntu, Python 3.13/3.14.Review note
This is intentionally opened as a draft because the branch is broad and the maintainer may prefer the desktop integration, evidence contracts, and packaging changes split into smaller review units. The implementation and live validation are complete; feedback on how to slice it for upstream review is welcome.