Add an optional stable_id to every extraction entity

examples/run/records/x7Ex4mpleA01.extraction.json

@@ -7,6 +7,7 @@
     "repaired_by": null,
     "paper_sections": null
   },
+  "stable_id": null,
   "local_id": "x7Ex4mpleA01",
   "description": null,
   "hypothesis": null,

extraction-readme.md

@@ -244,6 +244,14 @@ is the `FactorLevel.level` label pointing at it — `baseline`, `T0`, `week 12`,
 paper writes. What is normalized is placement (`relation_to_intervention`) and sequence
 (`order`), which carry every query an occasion vocabulary would have served.
 
+### An entity's stable_id is minted by the store
+
+Every entity may carry a `stable_id`, a UUID that the store mints the first time the entity
+appears and keeps across re-runs, so a later extraction matches the entity it describes. An
+extractor never sets it and leaves it null. A `local_id` is unique only within one record, so
+it cannot tell the same entity from a new one after a rename or a pipeline re-run; the
+`stable_id` can.
+
 ### Cells carry direction; weights do not
 
 Direction is recorded on the cell, never as a numeric contrast weight. One `Cell` is one level

src/study_schema/linkml/extraction-deviations.yaml

@@ -150,12 +150,138 @@ required_additions:
 
   slots:
 
+    # The store's key for an entity across re-runs; storage's `id` stays the store's own and is never extracted.
     study.Study:
       extraction_metadata:
         range: ExtractionMetadata
         required: true
         inlined: true
         description: Provenance for this extraction record.
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
+    acquisition.Acquisition:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
+    acquisition.Device:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
+    analysis.Analysis:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
+    analysis.Measure:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
+    analysis.Preprocessing:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
+    analysis.ModelEstimation:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
+    analysis.ModelTerm:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
+    analysis.InferenceSettings:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
+    analysis.Table:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
+    analysis.CoordinateSet:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
+    assessment.Assessment:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
+    design.Arm:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
+    design.Timepoint:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
+    group.Group:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
+    region.Region:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
+    task.Task:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
+    task.Condition:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
 
 # Nothing here yet. The projection is the baseline; add entries as data shows a different
 # shape extracts better. Candidates carried over from the hand-written schema that was

src/study_schema/linkml/neuroimaging-study-extraction/acquisition.yaml

@@ -96,6 +96,12 @@ classes:
       lists together. Everything modality-specific lives in the class the Modality value instantiates
       -- MRI, EEG, FNIRS, PET, or OtherModality.
     attributes:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
       local_id:
         identifier: true
         range: string
@@ -281,6 +287,12 @@ classes:
       The scanner, amplifier, or recording system an acquisition was made on, split into the two parts a source reliably reports. Either part can be missing: papers name a model with no manufacturer as often as the reverse.
       One record is one physical machine, referenced by every acquisition made on it: a study that collected functional, structural and diffusion scans in one session has three Acquisitions and one Device. Two records mean the study says it used two machines -- a second site, or a rescan on different hardware -- which is a claim about the data and not a way of recording that two sequences differed.
     attributes:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
       local_id:
         identifier: true
         range: string

src/study_schema/linkml/neuroimaging-study-extraction/analysis.yaml

@@ -32,6 +32,12 @@ classes:
       A distinct reported inferential map or tested effect, linking its source acquisitions, sample, paradigm context, tested effect, preprocessing, model estimation, inference settings, and source result tables.
       Create one Analysis for each distinct tested effect. Separate Analyses are required when normalized direction, participant-group comparison, method, the pattern of `effect.cells`, seed or connectivity model, decoded variable, component identity, or spatial_scope differ, and when an omnibus effect and its directional post-hoc contrast are both reported. ROI definitions, acquisition summaries, descriptive sample tables, and masks without inferential effects are not Analyses. Nor is a map reported with no inferential test at all -- an ICA or PCA component map presented descriptively, a connectivity matrix given without a test -- there being nothing for such a map to be the effect of.
     attributes:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
       local_id:
         identifier: true
         range: string
@@ -270,6 +276,12 @@ classes:
       One record may be referenced by several Analyses, so every field must hold for all of them; an analysis of a different quantity takes its own record. The test is mechanical: two analyses share a record when `family`, `type`, `specific_metric` and `unit` all match. A study measures one or two quantities and tests them many ways, so sharing is the common case -- eight analyses of one connectivity measure are one record, not eight.
       Not a property of the thresholding scheme: one correction scheme routinely covers several measures, and one measure is often tested under several schemes. See storage-schema-design-notes.md for the reasoning behind this shape.
     attributes:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
       local_id:
         identifier: true
         range: string
@@ -377,6 +389,12 @@ classes:
       An ordered preprocessing pipeline applied to acquired data before or as part of statistical
       modeling. This can be expanded with COBIDAS.
     attributes:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
       local_id:
         identifier: true
         range: string
@@ -415,6 +433,12 @@ classes:
       The test for whether two analyses share a record is mechanical: list the columns each was fitted with. A different dependent variable, a different covariate set, or a different subset of participants means a different design matrix and therefore a separate record, even when family, estimator and software are identical. Two BPM regressions of the same measure on the same regressor, one over the patients and one over patients and controls pooled, are two records.
       Sharing one record across matrices corrupts the derived adjustment set. The adjustment set is the model's terms minus the terms named by its cells. If several matrices share one record, they claim adjustments that their original models never made, and the record cannot expose the error.
     attributes:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
       local_id:
         identifier: true
         range: string
@@ -498,6 +522,12 @@ classes:
 
       A term is something the measure was regressed *on*. The quantity being modelled is `Analysis.measure` and never a term of the model that models it. A table laid out with one row-block per measured parameter -- four diffusion parameters, three frequency bands -- is reporting one analysis per parameter over one measure each, not one analysis over a factor whose levels are the parameters.
     attributes:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
       local_id:
         identifier: true
         range: string
@@ -636,6 +666,12 @@ classes:
       One record may be referenced by several Analyses, so every field must hold for all of them; an analysis thresholded or corrected differently takes its own record. The test is mechanical: two analyses share a record when the threshold, its type, the correction method, the scope corrected over and the alpha are all the same. A paper that states one correction scheme in Methods and applies it throughout has one record.
       Inference attaches to the analysis, not to the fit: one model estimated once is often thresholded several ways -- an ROI test Bonferroni-corrected over four regions and a searchlight over the same fit corrected by FDR. See storage-schema-design-notes.md for the reasoning behind this shape.
     attributes:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
       local_id:
         identifier: true
         range: string
@@ -705,6 +741,12 @@ classes:
       cell structure and point-level provenance may remain in the repository's coordinate-table
       representation and be linked with source_path.
     attributes:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
       local_id:
         identifier: true
         range: string
@@ -740,6 +782,12 @@ classes:
       A group of coordinates reported together -- a row group of a table, or a run of sentences in the text -- and what it is for. The coordinates stay in the parse, as for tables; this records their role and links them to what they belong to: the analysis a result reports, the regions an anchor locates.
       A set has one role: its dominant one. The groups come from the table parse, which splits a table by what was tested rather than by what each row is for, so a set can mix roles -- a connectivity results table that lists its own seed as a row. Such a set keeps the role of most of its points, and the odd point takes that role with it. This is a known error, about 1.3% of sets, accepted rather than splitting the parse's groups.
     attributes:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
       local_id:
         identifier: true
         range: string

src/study_schema/linkml/neuroimaging-study-extraction/assessment.yaml

@@ -24,6 +24,12 @@ classes:
       procedure that produces participant-level measurements independently of the neuroimaging
       acquisition or experimental conditions.
     attributes:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
       local_id:
         identifier: true
         range: string

src/study_schema/linkml/neuroimaging-study-extraction/design.yaml

@@ -190,6 +190,12 @@ classes:
     description: One thing participants were assigned to receive, or to receive in place of
       it.
     attributes:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
       local_id:
         identifier: true
         range: string
@@ -219,6 +225,12 @@ classes:
       One occasion on which data were collected, and where it sits relative to the intervention. Declared once on `StudyDesign.timepoints` and referenced by identifier; `FactorLevel.timepoints` is the only slot that references one, so it is the whole of how an occasion reaches a model.
       A study that *compared* occasions did so with a `ModelTerm` whose levels name them, paradigm or no paradigm. An occasion no level references was collected but never modelled: legitimate where the record only describes when follow-up happened, a defect where the analyses report change over time, which `validate_record.py` flags. Names stay in the source's wording; what is normalized is where an occasion sits and in what order.
     attributes:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
       local_id:
         identifier: true
         range: string

src/study_schema/linkml/neuroimaging-study-extraction/group.yaml

@@ -46,6 +46,12 @@ classes:
   Group:
     description: An aggregate participant group and its reported demographics.
     attributes:
+      stable_id:
+        range: string
+        pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
+        description: >-
+          UUID minted by the store the first time this entity appears, never by an extractor,
+          and kept across re-runs so a later extraction matches the same entity.
       local_id:
         identifier: true
         range: string