feat(cellpose): add optional gamma arg to CellposeV4#14
Open
hinderling wants to merge 27 commits into
Open
Conversation
Per-cell DMD-targeted stim that tiles each FOV into 14 px patches, keeps only patches fully inside a cell's mask, and stamps a 10 px-diameter dot on one patch per cell. Selection is seeded by (fov, fov_timestep) so post-processing reruns reproduce the live patches. Patch coordinates are accumulated in-memory and merged into exp_data.parquet on (fov, fov_timestep, label) so per-cell responses can later be correlated with image features extracted from the patch. Includes the matching experiment notebook (24 FOVs, 30 s intervals, 5 min baseline / single stim / 15 min response, optocheck on the last frame) using TrackerTrackpy.
Adds BINNING = "2x2" class attr on Moench and applies it in init_scope right after loading System,Startup. Most MM camera drivers reset the ROI on a binning change so binning has to come first. set_roi() now computes ROI_X / ROI_Y / ROI_WIDTH dynamically against the live camera dimensions — full image width (no x crop) and a symmetric top-bottom crop down to ROI_HEIGHT — and writes the values back to self so experiment notebooks reading mic.ROI_X still work idempotently. Class-attr defaults updated to the Prime BSI 2x2 case (1024x1024 -> 1024x800).
Calibration capture loop: - verbose=True now plots all captured frames in a single grid figure with the detected (max_x, max_y) overlaid as an open red circle, so you can eyeball whether the argmax-based detector landed on each projected disk. Replaces the per-frame plt.show() that printed the raw images one by one. Verbose validation step: - Test points were hardcoded as (100,500), (600,350), (500,800) in camera pixels, which fall outside the 1024x800 ROI under 2x2 binning — skimage.draw.disk silently clips out-of-FOV centers, leaving img_p empty and img_warp blank. Replaced with values scaled from the live getImageHeight/Width (h/4, h/2, 3h/4) so the test points always sit inside the FOV. - Dropped exposure=exposure on the test SLMImage. With OverlapMode=Off (where the setup-notebook focus-aid cells leave Mosaic3) a 25 ms SLM exposure blanks the DMD before snapImage opens the camera, so the validation frames came back empty even though calibration captured fine. Phase-1 events don't set SLMImage.exposure either; matching that behavior keeps the long KeepDMDAlive exposure in effect. - Switched the four-panel overlay to open circles for consistency with the new capture grid, and aligned panel-3 colors with panels 0..2 (red = detected, lime = requested) — they were swapped in the original code.
Cells 6c/6d wrap the existing arm/disarm pattern from 6a/6b but park Wheel-A on is_empty (the Sutter emission wheel's empty slot) so napari Live shows the DMD checkerboard reflected off the sample instead of emission-band fluorescence. Useful for focusing the DMD when the stim wavelength would otherwise excite (and bleach) the reporter. The arm cell defines arm_dmd_focus_reflection() / disarm_dmd_focus_ reflection() so the arm/disarm pair can be re-run with different channels without retyping setup. Default channel is OPTOCHECK_CHANNEL (mCitrine in cell 5's pinning), and the cyan stim LED level is forced on top of the channel preset since that's what fires through the DMD. Flow: start Live in napari-mm -> run 6c -> refocus -> run 6d to restore Wheel-A and OverlapMode.
apply_fov_batching and merge_rtm_sequences used to offset both min_start_time and the ``t`` index for overflow batches, so batch 1's FOVs ran at t=N..2N-1, batch 2 at t=2N..3N-1, etc. The OmeZarrWriter sized its time axis from max(event.t)+1, so the on-disk array was (n_pos, n_batches * N, …) with each FOV occupying only its own batch's slab and the rest left as fill. Loaders that grouped frames by t alone saw concatenated batches instead of aligned per-FOV traces. Drop the t-offset from both functions; keep only the wall-clock min_start_time offset so the controller still fires events in chronological order. Every FOV now uses t=0..N-1 and overflow batches stack along p. Zarr time-axis size collapses from n_batches*N to N. Side-effect for downstream consumers: (t, p) is no longer globally unique — multiple FOVs share the same t. Group/sort by (p, t), not t alone. Verified with the existing motile-free test suites (event ordering, frame dispenser, validate hardware, events_to_dataframe — 120 tests pass) and a smoke test confirming batched FOVs all carry t=0..N-1 while min_start_time still spaces batches correctly.
generate_fov_positions_from_list was reading the attribute as ONLY_USE_PFS but every microscope class declares USE_ONLY_PFS (Moench, Jungfrau set True; Niesen, demo set False). The getattr default of False was always returned, so z values were carried over from the FOV record into every event regardless of the scope's intent. Downstream the engine called setPosition on TIZDrive at every FOV move, which on a Nikon Ti with PFS active forces the focus motor away from the PFS-locked offset — i.e. PFS was effectively bypassed for the whole run. One-character fix; PFS now stays on across the experiment for scopes that declare USE_ONLY_PFS = True.
The Feb 28 controller refactor (f26b54e) replaced per-channel queueing with RTMSequence.to_mda_events but dropped the slm_image=_make_slm(...) that the old _queue_channels / _queue_optocheck attached to imaging and ref events. After the refactor only stim events carry an slm_image, so non-stim events leave whatever pattern was last latched on the DMD. _make_slm itself stayed on Controller as dead code. Under OverlapMode=On (the MDA path) the DMD re-pulses its currently loaded pattern on every camera TTL for the currently set SLM exposure. After a stim event fires, the DMD has the stim pattern + a short stim exposure latched; the next imaging events at the same t for subsequent FOVs in an FOV-batched timepoint pulse that stim pattern instead of an all-on frame. The result: imaging frames at the stim timepoint come back dark for every FOV after the first, segmentation finds nothing, and the random-patch stimulator records masks only for the first FOV. KeepDMDAlive's 60 s refresh saves quiet timepoints but is too slow to catch the tight burst of events at a stim timepoint under FOV batching. Fix: in _run_mda_with_events, wrap every non-stim planned event with SLMImage(data=True, device=dmd.name, exposure=ev.exposure) when the microscope reports dmd_needs_to_be_waken. Stim events untouched. Restores the pre-refactor invariant that every camera capture explicitly drives the DMD all-on for the matching exposure. Verified with the motile-free test suites (event ordering, frame dispenser, validate hardware, events_to_dataframe — 120 tests pass). The behavioural confirmation is that the next experiment run produces imaging signal across all FOV-batched stim frames rather than only the first FOV.
Concurrent writer.write() calls — one from the storage thread (raw
imaging frames) plus up to four from the Analyzer's pipeline-worker
threads (stim, stim_mask, label arrays) — race on zarr's atomic
rename of shard files.
The store shards all channels for a given (t, p) into a single shard
file (shards = (t_shard, 1, total_channels, h, w)), so a write to
one channel triggers a read-modify-write of the whole shard via
zarr's encode_partial path. encode_partial writes a `*.partial`
sidecar then renames it to the final chunk filename. Two threads
hitting different channels of the same shard each create a
`*.partial` and try to rename to the same target. On Windows / SMB
(Z:\) the second rename surfaces as
PermissionError: [WinError 5] Access is denied:
'acquisition.ome.zarr/0/33.0.0.0.<uuid>.partial' ->
'acquisition.ome.zarr/0/33.0.0.0.0'
Earlier 1-stim-frame schedules got lucky on timing; the new 30/10/50
structure (10 contiguous stim frames per FOV) plus the FOV-batching
alignment fix put many more concurrent writes on the same (t, p)
shards and tripped the race reliably.
Add ``self._write_lock = threading.Lock()`` to OmeZarrWriter.__init__
and wrap the body of ``write()`` with it. OmeZarrWriterPlate inherits
the locked entry point automatically. Tracking, segmentation, and
feature extraction continue running in parallel across pipeline
workers — only the on-disk write step serializes, which it had to be
at the shard level anyway. No measurable throughput hit at our
acquisition cadence.
Verified with the motile-free test suites (event ordering, frame
dispenser, validate hardware, events_to_dataframe — 120 tests pass).
Analyzer.executor runs up to four pipeline tasks concurrently. Under FOV batching that means four FOVs at the same t fire pipeline submissions nearly simultaneously, and they all called ``self.model.eval`` on a single shared CellposeModel instance. CellposeModel.eval is not thread-safe: it shares internal buffers and runs through PyTorch's CUDA context, where overlapping calls have been observed to silently return empty / corrupt masks for an entire batch of FOVs while the next batch works. The downstream effect is "all four FOVs in the same batch produced no labels" while trackpy's linker has nothing to advance — which is exactly what we saw on exp_425. Add ``self._eval_lock = threading.Lock()`` to the constructor and wrap just ``self.model.eval(...)`` with it. Tracking, feature extraction, and storage stay parallel across FOVs — only the GPU inference step serializes, which it had to be at the CUDA layer anyway. ``remove_small_objects`` post-processing stays outside the lock (CPU-only, thread-safe). Same regression pattern lurks in faro/segmentation/cellpose.py (the older v3 wrapper); not touched here since current experiments use v4. Stardist / convpaint segmenters carry the same shared-model pattern and would need the same lock if revisited.
trackpy's track_cells ``else`` branch (df_old non-empty, df_new non-empty) called ``fov_state.linker.next_level(...)`` unconditionally, assuming the linker had been initialised on a prior call. That invariant is broken whenever ``df_old`` is recovered from the parquet file fallback in pipeline.py: get_predecessor times out (frame N-1 took too long, e.g. cellpose stalled), the file fallback reads the last successful frame's parquet from disk, but the in-memory FovState.linker for this run is still ``None``. The very next frame that *does* have detections then crashes with AttributeError: 'NoneType' object has no attribute 'next_level' at trackpy.py:58 inside the executor's done-callback. exp_425 hit exactly this on the FOVs whose first segmentations came back empty from the cellpose-concurrency bug — the trackpy crash compounded the empty-label damage downstream. Treat both ``df_old.empty`` and ``fov_state.linker is None`` as first-frame conditions: discard the recovered df_old (its particle IDs can't be reconciled with a fresh linker), spin up a new trackpy.linking.Linker, and init_level on the current frame. The first-frame log line now also fires when we drop a stale predecessor, so the discontinuity is visible in the run output. Particle IDs do *not* carry over across the recovery boundary — that's the price of the fallback. Offline post-processing has to treat each FOV's pre-recovery and post-recovery segments as independent track tables, but a clean restart is strictly better than the crash + cascade we had before. Verified with a smoke test that exercises both paths (stale predecessor: emits the warning, drops df_old, init's a fresh linker; normal sequence: still init then next_level as before).
RandomStimPerCell14pxPatches now picks the per-cell patch selection once per FOV (at the first stim frame) and reuses the same fixed-in-image-space mask across every subsequent stim frame in that FOV. Lets you stimulate the same spots across a contiguous block of stim frames without the position rerolling each frame. Records are still appended on every call so the merge into exp_data.parquet populates patch coords on each stim-frame row, not just the first. The seed is fov-only now (no timestep dependency) so a replay is stable. Notebook config switched from a single-frame stim pulse to the 30 baseline / 10 stim / 50 recovery block this stimulator was written for. base_path uses a raw string so the Z: backslashes don't get parsed as escapes.
Reads a corner-positions JSON saved from napari-MM (2 or 4 corners),
builds an axis-aligned 10x10 grid across the spanned region, and
acquires one frame at each of the 100 FOVs in both imaging channels
through the same CellposeV4 + FE_ErkKtr pipeline as the stim
experiment. After acquisition: 10x10 collage per channel, a filter
cell on area_nuc / mean_intensity_C{0,1}_nuc with cropped previews
of matching cells, and an export of the surviving FOVs as
filtered_positions.json — ready to drop into the stim notebook for
the actual run.
Lives under the random_stim_per_cell folder since that's the
experiment it feeds. Bilinear / rotated-region interpolation can
replace the bounding-box linspace when we move past the simple case.
Two related fixes to make Controller.run_experiment robust against the
napari-micromanager-in-live-mode + MDA race that surfaces as
"Camera image buffer read failed":
1. _run_mda_with_events now stops continuous sequence acquisition
before starting MDA. If live mode is still running when the MDA's
first snapImage fires, napari-mm's _core_link._image_snapped handler
reads the snap buffer before the engine gets to it, and the engine
raises. Stop it unconditionally up front; suppress exceptions in
case the camera adapter is in an odd state.
2. Controller.__init__ accepts an optional `viewer` parameter. When
provided, each acquired frame is mirrored to a layer named
Controller.PREVIEW_LAYER_NAME ("pipeline_preview"), so the user can
see what the pipeline is processing without having to keep
napari-mm's live mode on. The update is marshaled through
superqt.utils.ensure_main_thread because frameReady is emitted from
the MDA worker thread.
The preview layer is intentionally distinct from napari-mm's "preview"
so the two don't clobber each other.
…-per-cell-patches
…ombine fix: use combine(..., axis="t") in optogenetic demo notebook
Squash-imports the faro/async-run-handle work onto this branch so the async experiment flow can be tested on the real microscope. Brings: - faro/core/controller.py -- run_experiment / continue_experiment now return a RunHandle and run the MDA feed loop on a worker thread. Overwrites the earlier preview + stop-sequence changes (superseded). - faro/core/run_status.py -- RunStatus snapshot + RunHandle (wait / cancel / pause / resume + psygnal statusChanged signal). - faro/widgets/ -- ExperimentStatusWidget (event strip, FOV map, stop button). - faro/microscope/base.py -- image-dimension fields on AbstractMicroscope. - experiments/02_demo_sim_optogenetic/ -- async demo notebooks. uv.lock intentionally not touched (async added no new direct deps; pyproject.toml unchanged on the async branch). Imported as a single commit rather than a merge so the async branch (PR pertzlab#10) keeps a clean independent history.
Add FrameDispenser.cancel() and the FrameWaitCancelled exception so a thread blocked in wait_for_frame / get_predecessor is woken immediately instead of sitting out the full timeout. This lets an experiment abort promptly: a feed loop parked in an up-to-80s stim-mask wait is released the instant the run is cancelled.
Cancellation: RunHandle gains an on_cancel hook, invoked synchronously from cancel(), that wakes a feed loop blocked in a stim-mask wait via Analyzer.cancel_pending_waits(). Previously a cancel issued during that wait took up to the stim-mask timeout (~80s) to take effect, leaving the frame handler connected in the meantime. Queue stats: Analyzer.queue_stats() / Controller.queue_stats() expose storage, pipeline and deferred queue depths for the status widget. finish_experiment runs its teardown (run wait + Analyzer drain) on a worker thread and pumps Qt, so napari stays responsive during the drain. Lag is anchored to the first frame's acquisition start rather than the worker's start time, so worker/engine startup (~1s) is no longer charged to every lag reading.
Stop now cancels the run and then runs finish_experiment(), so the next run starts clean instead of leaking the old Analyzer; the state banner shows STOPPING... while the drain runs. Stats are split into three panels (timing / queues / errors). The storage and pipeline queue depths render as grayscale fill bars that turn red past 80% of capacity; deferred shows as a plain count. The FovMap is freely resizable instead of pinned square.
apply_fov_batching gains offset_min_start_time (default True). FOVs in a batch are imaged sequentially, not simultaneously, so the k-th FOV of a batch only starts ~k * time_per_fov after the batch's first FOV. Encoding that offset into each event's min_start_time keeps the scheduled per-FOV frame interval consistent and makes lag measurement meaningful (lag is acquisition-start minus min_start_time). The first FOV of every batch gets a 0 within-batch offset; batches after the first still get their batch wall-clock offset on top.
…start-offset feat: stagger per-FOV min_start_time in apply_fov_batching
…-atexit-unload fix: release Micro-Manager devices on interpreter exit
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
gamma: float = 1.0toCellposeV4.__init__.segment(), applyimage = image**self.gammawhengamma != 1.0(skipped otherwise to avoid an unnecessary float upcast).mainis unchanged unless callers opt in.Motivated by per-cell stim experiments where boosting the dark end of low-contrast nuclear images noticeably improves Cellpose segmentation. Previously this was being done inline (
image**0.3) insidesegment, which forced gamma on every caller. Making it a constructor arg keeps the default behavior untouched and lets each pipeline configure it explicitly.Test plan
CellposeV4()produces identical masks to pre-change main on the same input image.CellposeV4(gamma=0.3)produces masks consistent with the previously-hardcodedimage**0.3path.