docs: reference catch-up for chain params (#120, #122-#125)#126
Closed
FIrgolitsch wants to merge 77 commits into
Closed
docs: reference catch-up for chain params (#120, #122-#125)#126FIrgolitsch wants to merge 77 commits into
FIrgolitsch wants to merge 77 commits into
Conversation
Squashed pr-m-gpu-kvikio: - linumpy.gpu.zarr_io: high-level read_zarr_to_gpu dispatcher (kvikio/GDS native -> zarr.config.enable_gpu fallback) - linumpy.gpu.kvikio_zarr: kvikio-backed zarr->GPU reader using contiguous CuFile.pread scratch buffer - GPU keep-on-device for resize/affine/map_coordinates; resample writer uses gpu_zarr_context - linum_aip / linum_aip_png GPU pipelines stay on device end-to-end - Device-aware find_tissue_interface, find_z_overlap; vectorized focal curvature roll - Wire --use_gpu for focal_curvature + stack into nextflow - Increase GPU resample maxForks; tune GPU memory management in _run_pipelined - deps: bump ome-zarr to >=0.16.0 (NGFF 0.5) - Server: nvidia-fs 2.28.4 patch script for kernel 7.0 - Benchmark script linum_benchmark_kvikio_zarr.py - Bug fix: detect_focal_curvature broadcasts per-tile correction across tile positions
Co-authored-by: Copilot <copilot@github.com>
Squash of resample/GPU/workflow work on dev: - Round-robin GPU assignment for resample_mosaic_grid (linumpy.gpu) - Per-gpu active-slot counters via Helpers.gpuPinBlock for fair maxForks balance - CUDA device pin in prefetch worker thread + workflow CUDA_VISIBLE_DEVICES - Keeps simple per-tile CPU-read -> GPU-rescale -> CPU-write path
Heavy downsampling of small axes (e.g. Z=5 by factor 0.1) used to round to zero, producing zero-sized output and ZeroDivisionError downstream. Clamp output_shape to a minimum of 1 in both rescale() and out_tile_shape.
…ip_tiles for manual overrides
- pyproject.toml: requires-python >=3.14 - Drop `from __future__ import annotations` across the codebase (no-op on 3.14) - linumpy.config.threads: adopt os.process_cpu_count() over os.cpu_count() - Update CI matrices (.github/workflows/python-app.yml, publish.yaml) to py3.14 - Bump .readthedocs.yaml + Dockerfile base image to py3.14; drop unused .python-version copy - Remove .pep8speaks.yml (deprecated service); add docker-publish.yml for GHCR - Misc workflow cleanup: drop redundant python-version pins, fix nextflow-ci checkout - ruff: enable FURB rule set + apply modernization sweep (pathlib, dict.fromkeys, list comprehensions, etc.) - README + docs/NEXTFLOW_WORKFLOWS: update python version refs
…il-alignment, sort-declarations)
…n kernel 7.0
- create_nv.symvers.sh: case-insensitive grep for relative CRCs ('r' vs 'R'),
needed because kernel 7.0 / open-gpu-kmd emits __crc_nvidia_p2p_* as
section-local rodata. Without this, modversion fallback was skipped and
bogus relocation offsets were written, producing an unloadable nvidia-fs.
- create_nv.symvers.sh: 'zstd -df --rm' (force) so reruns don't stall on the
'overwrite (y/n)?' prompt when nvidia.ko already exists.
- nvfs-mmap.c patch: idempotent (skip if already applied).
…r-python PR #2863)
Previously the correlation-fallback branch overwrote the measured NCC with 0.0, making it indistinguishable from a true zero-correlation match and impossible to derive a sensible `z_overlap_min_corr` from stacking_decisions.csv -- every fallback row reported 0.0 regardless of how close the actual measurement was to the threshold. The fallback now keeps the measured correlation and reports `overlap_source = 'correlation_fallback'` (plus a new `correlation_fallback_used` z-match flag) so QC can tell which pairs were trusted vs. demoted to the slicing-interval estimate while still seeing the real NCC distribution. Manually aligned slices are already trusted via the registration branch and do not enter this code path.
…test
The Allen ABC Atlas docs describe `average_template_<res>.nrrd` as ASL
("first axis is anterior-to-posterior, second is superior-to-inferior,
third is left-to-right"), while the rest of linumpy uses the
NIfTI/radiology convention where each letter names the direction the
axis points toward -- under which the same layout is PIR. Both names
describe the identical voxel layout, but the existing docstring only
mentioned PIR, which made it easy to suspect the conversion code when a
downstream RAS preview looked flipped.
* Rewrite the `download_template_ras_aligned` docstring to spell out
both names, cite the empirical landmarks (olfactory tip vs. cerebellum
along SITK X), and note that the legacy informatics-archive nrrd and
the newer ABC-Atlas template share this orientation. The
`PermuteAxes((2, 0, 1)) + Flip(F, T, T)` recipe is unchanged --
verified correct on the cached 100 µm template.
* Add `TestRealAllenTemplateOrientation` (skipped when the nrrd is not
cached) so future changes to either the upstream file or the RAS
recipe trip a regression test:
- raw nrrd: anterior tip < posterior mass on SITK X (PIR/ASL)
- RAS-aligned: +dim1 mass concentrated at anterior end
- RAS-aligned: dim2 (R) mass roughly L/R symmetric within 10 %
…strength in attenuation compensation
…script (sub-voxel roll, single load, GPU verbose), document 10x Mitutoyo objective context
Contributor
Author
|
Doc changes cherry-picked directly onto dev (history rebuild). This branch is no longer needed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Documentation catch-up for the stacked-PR chain
devalready contains the implementation of several new params introduced by the 22-PR chain (#98 → #125), but the reference docs were not updated alongside the code. This PR fills those gaps. It targetsdevdirectly because every change here is a documentation-only addendum to commits that are already ondev.docs/NEXTFLOW_WORKFLOWS.mdscratch_dir(per-taskscratchdirective,true/false/<path>semantics — from perf: pipeline-wide GPU + scratch optimizations #120 perf pipeline).fix_illum_fit_max_samples,fix_illum_max_iterations,fix_illum_darkfield— from fix_illum: pool tiles across Z + darkfield/fit knobs (chain tip = dev) #125) and the axial PSF correction parameters (compensate_psf_enabled,compensate_psf_method,compensate_psf_zr_initial— from PSF compensation + focal curvature follow-ups #124).z_overlap_min_corrrow to mention the newoverlap_source = 'correlation_fallback'value and thecorrelation_fallback_usedflag instacking_decisions.csv(from metrics: split package + stacking decisions hardening #122).docs/SCRIPTS_REFERENCE.mdlinum_detect_focal_curvature.py: surfaced the real flags (--n_levels,--n_processes,--block_size,--use_log,--use_gpu,--verbose) instead of just the bare two-argument signature.linum_compensate_psf_from_model.py: documented--zr_initialwith default and the 10× Mitutoyo guidance (from PSF compensation + focal curvature follow-ups #124).linum_compensate_attenuation.py/ newlinum_compensate_attenuation_inplace.py: documented all in-place attenuation flags introduced by the four-method DRE work (--method,--strength,--k,--zshift,--snr_threshold_db,--min_bias,--mask_smoothing_sigma,--n_levels). Cross-links todocs/ATTENUATION_METHODS.mdfor the math. (from Depth-resolved attenuation: 4 estimators, Li 2020 default #123)No code or config changes — only documentation. Safe to merge independently of the chain.