Formalize LibCapabilities into declared, reported graceful degradation

[jspaezp]

Contract

Capability decisions are runtime and library-wide. Every scored target and decoy in a run must use the same enabled feature set. Per-row coverage explains a decision; it never enables a score only for some entries. A fallback must be explicitly justified as producing comparable values across the library; producing a number is insufficient.

Separate observed library facts from what an operation can compute. Extraction geometry, residue sequence, modification sites/counts, molecular composition, fragment-ion chemistry, and reference intensities are distinct capabilities. Extensions (especially “JSON”) and label types alone do not establish support. Describe concrete input schemas and record reader transformations: discarded information cannot be recovered by inspecting the final arena.

Data model: names are independent of chemistry

• Preserve entry labels/names and supplied source IDs exactly through round trips. An entry name that looks like a sequence or charge is still just a name; never parse it as authoritative chemistry. Fragment annotations are a separate concept again.
• Keep external identity separate from stored-row handles, expanded variant handles, and competition groups. Stored and generated decoys remain explicit policy/variant decisions.
• Retain structured chemistry produced during loading: residues/stripped sequence, located modifications, and independently supplied composition or ion/adduct information. Avoid throwing parsed chemistry away only to reparse sequence strings during rescoring.
• Original sequence spelling is not a round-trip requirement. Generate sequence notation from structured chemistry for output; no redundant original-sequence string is required.
• Preserve unresolved modification content and known sites explicitly. Unknown composition must not erase usable residues or known modification counts. A mass delta alone does not establish an atomic formula.
• Distinguish no modifications from unknown modifications, and available sequence from missing/unresolved or explicitly not-applicable sequence. Do not infer not-applicable merely from an absent field.

#142 is merged: structured analyte storage, modification registry, borrowed accessors and independent names/source IDs are implemented. Public builders validate stored and recovered facts.

Small molecules are legitimate library entries

A library entry does not imply a peptide. A small molecule may have usable precursor/fragment geometry, reference intensities, and molecular formula but legitimately no sequence.

• Extraction and reference-spectrum comparison should be assessed independently of peptide annotations; supporting this in scoring requires adapting the current bridge.
• Residue/modification features are not applicable to such entries. In a mixed library they remain disabled for the whole run under the uniform-feature contract.
• Composition-based isotope envelopes may be available without sequence, given adequate molecular and ion/adduct information.
• Peptide-specific decoy generation must not be assumed applicable. Library decoy presence is only an FDR prerequisite; an appropriate strategy and checks on surviving candidates remain necessary.

Downstream data flow

After #142, candidates borrow chemistry through their library row; the owned Peptide and repeated sequence parsing are removed. The continuing direction is:

read explicit chemistry → retain structured information + independent names/IDs
                       → inspect library-wide coverage
                       → resolve one execution plan
                       → compute using borrowed row views + observed candidate values

Pass the owning ReferenceLibrary into downstream batch operations, including rescoring; merged #144 stores its resolved plan on the library itself. Observed scores remain candidate data; library metadata is resolved through the existing row handle. Keep variant/decoy information wherever transformed chemistry matters: a stored-row handle alone cannot describe every expanded query. Pair the plan with its library at the context boundary.

Per-element access is fine; per-element policy is not. Resolve operation eligibility in the owning library plan; candidate projection supplies values, not policy. If a required library fact is unavailable after the plan promised it, diagnose an invariant failure rather than silently changing that candidate's features.

Score macro and execution

ScoreBlock currently generates Parquet projections, ordered ML names, fixed-size linear/nonlinear value arrays, and unary transforms; score calculations are handwritten. Extending the derive with requirements alone will not stop computation.

Merged #144 implements this for residue and modification-count operations using #[score(requires(...))], with the plan owned by ReferenceLibrary. Extending the same approach to other operations remains follow-up work:

• Declare requirements on independently computed operations. Split mixed blocks where needed: residue counts, modification counts, and composition-dependent calculations have different requirements.
• Generate requirement metadata and ordered feature descriptors from the same declarations that generate projections.
• Resolve enabled operations and output column ranges once per run. A dispatcher owns calls to handwritten computations, so disabled operations are never computed.
• Use the same plan for feature names, values, and training/prediction schema. Blocks can retain fixed-size arrays while the assembled matrix width is resolved at runtime.
• Disable all projections of an unavailable score, including isna; otherwise a disabled score can still contribute a missingness indicator.
• Declare dependencies for cross-score operations such as Derived; disabling an input disables dependent features.

Keep the all-NaN scan as a guard for enabled features that prove unusable on actual data, rather than the primary discovery mechanism for predictable library limitations. In #146, disabled fragment-isotope score fields are NaN in Parquet and omitted entirely from model projections. Raw-mode output omits competition, discriminant-score and q-value columns; do not turn unavailable scores into valid zeros.

Vertical implementation slices

1. Stored analyte facts — merged Store analyte facts in the library and borrow them during scoring #142. Readers retain structure and independent labels; scoring borrows row facts; output supports canonical nullable sequence and supplied formula. Old candidate sequence copies and legacy parser removed.
2. Partial chemistry / Sequence features: a non-Unimod mod discards the residues too #107 — merged Enable residue and modification counts independently across each library #144. Resolve residue and modification-count operations independently across the entire library. Derive requirement/name/value metadata, dispatch only enabled operations, and share their layout across GBM, LDA/hybrid, streaming MLP and dashboard. Report decisions in CLI, viewer and Parquet metadata. Missing composition alone must not disable known modification counts.
3. Modification-aware isotope composition / Isotope envelopes ignore the atomic composition of modifications #108 — merged Include modification C/S counts in library-wide isotope envelopes #145. Retain the existing three-bin C/S model; include modification C/S deltas or explicitly based supplied formulas. Select composition-derived counts only when every stored target and shipped decoy supports them; otherwise use mass-estimated C/S for the whole library. Generated mass-shift variants reuse their parent envelope (and unshifted parent mass in mass-estimated mode). Report the resolved method and unavailable reasons through the owning library plan. Slice 2 itself left isotope behavior unchanged.
4. Sequence-free scoring — draft Score sequence-free libraries with library-wide chemistry fallbacks #146. mzSpecLib with missing sequence/unknown peaks and programmatic string-labelled libraries with reference intensities reach common scoring. Arena sealing disables automatic mass-shift generation library-wide when any retained label lacks the required chemistry. The scoring plan gates fragment-isotope extraction and every matching ML projection. Supplied decoys retain the supervised route; no decoys selects full-acquisition-RT raw scoring without calibration, competition, rescoring, q-value filtering or FDR output columns. Packed opaque keys currently limit this bridge to 255 peaks per entry, with source labels preserved separately. Current Target/ElutionGroupInput JSON schemas remain geometry-only. Real degraded-HeLa and raw-output acceptance results are recorded in the PR.

Order: 1 → 2, then 3 and 4 can proceed separately. Each PR delivers reader-to-output behavior and removes its replaced path; no horizontal scaffolding-only PRs. #118 remains independent.

Example acceptance case: 99 entries with resolved modifications plus one with a known, located mass-only modification. If all residues and modification counts are recoverable, those features can be enabled for all 100; full-composition features remain a separate decision. A missing modification list disables modification counts for every entry but leaves complete residue features usable. Labels/IDs remain unchanged.

Related work

• Validate scoring libraries once and extract directly from RefQuery #136 merged / ExpectedIntensities: validate once at scoring-library construction #111 closed: validates scoring-library invariants and passes RefQuery directly into mutable extraction buffers; it does not implement this capability plan.
• Sequence features: a non-Unimod mod discards the residues too #107: preserve residues when modification representation is unsupported.
• Isotope envelopes ignore the atomic composition of modifications #108: modification composition in isotope envelopes.
• Represent mobility observables explicitly and degrade searches safely #117: measurement compatibility and mobility degradation.
• Finish arena ownership: pair intensities with geometry and filter decoys at seal #118: shared builder/sidecar ownership; remains an independent structural change.