Preserve library RT coordinates and calibrate search dimensions independently

Cargo.toml

@@ -77,18 +77,12 @@ insta = { version = "1.34.0" }
 bon = "3.8.1"
 tinyvec = { features = ["alloc", "serde"], version = "1.10.0" }
 smallvec = { version = "1.13", features = ["const_generics", "union"] }
-# `mzannotate` carries the mzSpecLib reader. Two fixes are not in a release yet:
-# named attribute sets were never applied, so every decoy in a target/decoy
-# library read back as a target, and retention time was dropped for any spectrum
-# whose writer grouped other attributes with it. `mzcore` and `mzcv` come from
-# the same source so the graph holds one copy of each and the ontology types
-# unify.
-#
-# Pinned by revision, not by branch: a branch can be force-pushed or deleted.
-# All three become version numbers once the fixes ship.
-mzannotate = { git = "https://github.com/jspaezp/rustyms", rev = "142c96ed1d31c56936c003af5f599d0c8f740005" }
-mzcore = { git = "https://github.com/jspaezp/rustyms", rev = "142c96ed1d31c56936c003af5f599d0c8f740005" }
-mzcv = { git = "https://github.com/jspaezp/rustyms", rev = "142c96ed1d31c56936c003af5f599d0c8f740005" }
+# mzSpecLib record API preserves RT terms, grouped units and attribute-set
+# provenance before projection (jspaezp/rustyms PR #6). Keep all ontology crates
+# on the same reviewed revision so their types unify.
+mzannotate = { git = "https://github.com/jspaezp/rustyms", rev = "c033092561166e1b9960fa2a6678299af52a94e1" }
+mzcore = { git = "https://github.com/jspaezp/rustyms", rev = "c033092561166e1b9960fa2a6678299af52a94e1" }
+mzcv = { git = "https://github.com/jspaezp/rustyms", rev = "c033092561166e1b9960fa2a6678299af52a94e1" }
 csv = "1.3"
 
 tempfile = "3.23.0"

docs/development.md

@@ -31,6 +31,41 @@ q-value columns (`result_mode=raw` Parquet metadata). It bypasses calibration,
 rescoring and q-value filtering. Supplied decoys do not by themselves validate a
 decoy strategy for a new analyte class.
 
+Library RT is declared once per library: seconds, normalized index, unspecified
+coordinates, or absent. Measured minutes convert to seconds; normalized indices
+retain their values, including zero and negative values. Declared normalized-scale
+metadata is preserved from mzSpecLib headers and prediction provenance; records
+select the actual axis. Mixed RT availability or
+axes are rejected. An RT-free library searches unrestricted on RT, omits RT
+prediction/residual features, and retains decoy competition, rescoring and q-values
+when decoys are available. RT, m/z and mobility calibrate independently. Without
+an RT fit, score-selected apexes still supply m/z and mobility measurements,
+without ridge filtering. RT-free prescoring retains bounded candidates across
+observed RT bands. Mobility calibration requires a searchable run axis; FAIMS and
+absent mobility do not contribute sentinel measurements. An axis without measurements keeps its configured tolerance
+in primary and secondary extraction. A failed RT fit never reinterprets an index
+as seconds; initial extraction searches unrestricted RT. Secondary queries still
+center on detected apex RT and observed mobility.
+
+Results format 5 includes `library_rt_axis` Parquet metadata. Unavailable library
+RT, calibrated RT and RT residuals are NaN; observed apex RT remains seconds.
+Calibration format v4 records the input library axis and effective tolerance
+enums; an empty RT snapshot means no RT fit, while residuals can still contain
+m/z and mobility calibration. Older calibration files
+must be regenerated. Explicit `rt_seconds` JSON/Python inputs remain seconds. JSON `rt_axis` is
+preserved whether optional precursor/fragment labels are supplied or filled in.
+Rust source geometry exposes `LibraryRT<f32>` through `Target` and `OwnedTarget`.
+`ExtractionQuery` borrows that geometry with a separately resolved acquisition
+window. `RtSelection::Centered(ObservedRTSeconds(...))` or `FullRun` resolves
+against the acquisition extent once. Collectors copy that window and reuse their
+geometry/intensity buffers; they retain no source borrow. Index queries accept
+`PeakTolerance` for m/z, mobility and quadrupole only. Isotope extraction shifts
+collector buffers directly, without an owned target or library-axis clone.
+The viewer searches full-run without an RT fit, including libraries measured in
+seconds. Direct-query CLI restricted RT inputs explicitly mean acquisition
+seconds; normalized library indices require calibration first. Python target
+`rt_seconds` likewise specifies an acquisition coordinate.
+
 `run_report.json` records the resolved `scoring_plan` once for the search library,
 plus `calibration_scoring_plan` when a separate calibration library is supplied.
 These are the same plans serialized in Parquet metadata: sequence-operation
@@ -49,7 +84,7 @@ intensities, so those reader routes remain extraction-only.
 Standalone `calib_dash` reads saved `calibration.json`, not a spectral library.
 
 [Analyte module documentation](../rust/timsquery/src/chemistry/analyte.rs) documents chemistry storage, reader mappings,
-library-wide sequence eligibility, and results format version 4.
+library-wide sequence eligibility, and results format version 5.
 
 Precursor isotope envelopes retain the three-bin C/S approximation. Scoring
 finalization includes known modification C/S deltas or an explicitly based

python/timsquery_pyo3/src/index.rs

@@ -7,7 +7,6 @@ use timsquery::traits::queriable_data::QueriableData;
 use timsquery::{
     ChromatogramCollector,
     MzMobilityStatsCollector,
-    OptionallyRestricted,
     SpectralCollector,
     Tolerance,
 };
@@ -21,28 +20,20 @@ use crate::spectrum::{
 use crate::target::PyTarget;
 use crate::tolerance::PyTolerance;
 
-/// Compute the RT range in milliseconds for a chromatogram query.
-///
-/// If the tolerance is restricted, returns the tolerance-derived range.
-/// If unrestricted, falls back to the full acquisition RT range from the
-/// cycle mapping (giving a chromatogram spanning the entire run).
-pub(crate) fn rt_range_ms_for_chromatogram(
+/// Python target `rt_seconds` is an explicit acquisition coordinate.
+pub(crate) fn resolve_query_rt(
     tol: &Tolerance,
     rt_seconds: f32,
     handle: &IndexedPeaksHandle,
-) -> PyResult<timsquery::TupleRange<u32>> {
-    match tol.rt_range_as_milis(rt_seconds) {
-        OptionallyRestricted::Restricted(range) => Ok(range),
-        OptionallyRestricted::Unrestricted => {
-            let (start, end) = handle.ms1_cycle_mapping().range_milis();
-            timsquery::TupleRange::try_new(start, end).map_err(|e| {
-                PyErr::new::<pyo3::exceptions::PyValueError, _>(format!(
-                    "Empty RT range in index: {:?}",
-                    e
-                ))
-            })
-        }
-    }
+) -> PyResult<timsquery::ResolvedRt> {
+    timsquery::ResolvedRt::from_mapping(
+        timsquery::RtSelection::Centered(timsquery::ObservedRTSeconds(rt_seconds)),
+        &tol.rt,
+        handle.ms1_cycle_mapping(),
+    )
+    .map_err(|e| {
+        PyErr::new::<pyo3::exceptions::PyValueError, _>(format!("Invalid query RT: {e:?}"))
+    })
 }
 
 /// Resolved tolerances: either one shared or one per query.
@@ -133,15 +124,18 @@ impl PyTimsIndex {
         target: &PyTarget,
         tolerance: &PyTolerance,
     ) -> PyResult<PyChromatogramResult> {
-        let target = target.inner.clone();
+        let target = &target.inner;
         let tol = &tolerance.inner;
-        let rt_range_ms = rt_range_ms_for_chromatogram(tol, target.rt_seconds(), &self.handle)?;
+        let resolved_rt = resolve_query_rt(tol, target.rt_seconds(), &self.handle)?;
         let ref_rt = self.handle.ms1_cycle_mapping();
 
-        let mut collector = ChromatogramCollector::<usize, f32>::new(&target, rt_range_ms, ref_rt)
-            .map_err(|e| PyErr::new::<pyo3::exceptions::PyValueError, _>(format!("{:?}", e)))?;
+        let mut collector = ChromatogramCollector::<usize, f32>::new(
+            &timsquery::ExtractionQuery::new(target, resolved_rt),
+            ref_rt,
+        )
+        .map_err(|e| PyErr::new::<pyo3::exceptions::PyValueError, _>(format!("{:?}", e)))?;
 
-        self.handle.add_query(&mut collector, tol);
+        self.handle.add_query(&mut collector, &tol.peak_tolerance());
 
         Ok(PyChromatogramResult::new(collector, target.id()))
     }
@@ -161,18 +155,22 @@ impl PyTimsIndex {
         target: &PyTarget,
         tolerance: &PyTolerance,
     ) -> PyResult<()> {
-        let target = target.inner.clone();
+        let target = &target.inner;
         let tol = &tolerance.inner;
-        let rt_range_ms = rt_range_ms_for_chromatogram(tol, target.rt_seconds(), &self.handle)?;
+        let resolved_rt = resolve_query_rt(tol, target.rt_seconds(), &self.handle)?;
         let ref_rt = self.handle.ms1_cycle_mapping();
 
         result
             .collector
-            .try_reset_with(&target, rt_range_ms, ref_rt)
+            .try_reset_with(
+                &timsquery::ExtractionQuery::new(target, resolved_rt),
+                ref_rt,
+            )
             .map_err(|e| PyErr::new::<pyo3::exceptions::PyValueError, _>(format!("{:?}", e)))?;
 
         result.source_id.set_from(target.id());
-        self.handle.add_query(&mut result.collector, tol);
+        self.handle
+            .add_query(&mut result.collector, &tol.peak_tolerance());
 
         Ok(())
     }
@@ -189,10 +187,13 @@ impl PyTimsIndex {
         target: &PyTarget,
         tolerance: &PyTolerance,
     ) -> PyResult<PySpectralResult> {
-        let target = target.inner.clone();
+        let target = &target.inner;
         let tol = &tolerance.inner;
-        let mut collector = SpectralCollector::<usize, f32>::new(&target);
-        self.handle.add_query(&mut collector, tol);
+        let mut collector = SpectralCollector::<usize, f32>::new(&timsquery::ExtractionQuery::new(
+            target,
+            resolve_query_rt(tol, target.rt_seconds(), &self.handle)?,
+        ));
+        self.handle.add_query(&mut collector, &tol.peak_tolerance());
         Ok(PySpectralResult::new(collector, target.id()))
     }
 
@@ -209,10 +210,15 @@ impl PyTimsIndex {
         target: &PyTarget,
         tolerance: &PyTolerance,
     ) -> PyResult<PyMzMobilityResult> {
-        let target = target.inner.clone();
+        let target = &target.inner;
         let tol = &tolerance.inner;
-        let mut collector = SpectralCollector::<usize, MzMobilityStatsCollector>::new(&target);
-        self.handle.add_query(&mut collector, tol);
+        let mut collector = SpectralCollector::<usize, MzMobilityStatsCollector>::new(
+            &timsquery::ExtractionQuery::new(
+                target,
+                resolve_query_rt(tol, target.rt_seconds(), &self.handle)?,
+            ),
+        );
+        self.handle.add_query(&mut collector, &tol.peak_tolerance());
         Ok(PyMzMobilityResult::new(collector, target.id()))
     }
 
@@ -239,24 +245,34 @@ impl PyTimsIndex {
             .iter()
             .enumerate()
             .map(|(i, target)| {
-                let inner = target.inner.clone();
+                let inner = &target.inner;
                 let tol = tolerances.get(i);
-                let rt_range_ms =
-                    rt_range_ms_for_chromatogram(tol, inner.rt_seconds(), &self.handle)?;
-                ChromatogramCollector::<usize, f32>::new(&inner, rt_range_ms, ref_rt).map_err(|e| {
-                    PyErr::new::<pyo3::exceptions::PyValueError, _>(format!("{:?}", e))
-                })
+                let resolved_rt = resolve_query_rt(tol, inner.rt_seconds(), &self.handle)?;
+                ChromatogramCollector::<usize, f32>::new(
+                    &timsquery::ExtractionQuery::new(inner, resolved_rt),
+                    ref_rt,
+                )
+                .map_err(|e| PyErr::new::<pyo3::exceptions::PyValueError, _>(format!("{:?}", e)))
             })
             .collect::<PyResult<Vec<_>>>()?;
 
         // Release GIL for the parallel query work
         let handle = &*self.handle;
         py.detach(|| match &tolerances {
             ResolvedTolerances::Single(tol) => {
-                handle.par_add_query_multi(&mut collectors[..], rayon::iter::repeat_n(tol, n));
+                handle.par_add_query_multi(
+                    &mut collectors[..],
+                    rayon::iter::repeat_n(&tol.peak_tolerance(), n),
+                );
             }
             ResolvedTolerances::PerQuery(tols) => {
-                handle.par_add_query_multi(&mut collectors[..], &tols[..]);
+                handle.par_add_query_multi(
+                    &mut collectors[..],
+                    &tols
+                        .iter()
+                        .map(Tolerance::peak_tolerance)
+                        .collect::<Vec<_>>()[..],
+                );
             }
         });