Ship the LinkML source with the package

README.md

@@ -12,7 +12,7 @@ runs, the extraction pipeline, and the three modules that define what a record m
 submodule, so a schema change and the code that consumes it are versioned together without
 either being a copy of the other.
 
-`neuroimaging-study-storage.yaml` and the modules under `neuroimaging-study-storage/` are
+The LinkML lives under `src/study_schema/linkml/`. `neuroimaging-study-storage.yaml` and the modules under `neuroimaging-study-storage/` are
 the full storage schema: everything we might ever want to represent. It is the source of
 truth and is never narrowed in place.
 
@@ -282,6 +282,8 @@ from study_schema.models.paper_parse import CoordinateParse
 parse = CoordinateParse.model_validate_json(path.read_bytes())
 ```
 
+**The LinkML itself.** The YAML ships inside the package under `study_schema.linkml`, so a consumer that needs the source reads it from the installed copy rather than from a checkout: `study_schema.linkml.EXTRACTION`, `STORAGE` and `PAPER_PARSE` are `pathlib.Path`s to the three top-level schemas, which import their modules by relative path, and `study_schema.linkml.EXTRACTION.parent` is the directory they sit in. The contract files that pondie checks, `extraction-deviations.yaml` and `extraction-to-storage.map.yaml`, are packaged alongside them. `storage-parameter-priorities.yaml` is reviewer triage data and is not packaged. `tools/generate_models.py` reads the same files, so the copy a consumer holds is the one the committed models were generated from.
+
 **Reference properties.** A string property that holds another entity's `local_id` carries `"x-reference": "<TargetClass>"` on the property itself, beside its `type`; for a multivalued slot that property is the array, not its `items`. The keyword is set from the LinkML induced slots, so `is_a` and `slot_usage` apply, and only where the range class has an identifier and the slot is not inlined. JSON Schema validators ignore it, and `study_schema.jsonschema.references(name)` returns the same relationships as `Reference(cls, slot, target, multivalued)` named tuples, sorted by class and slot.
 
 **Unique keys.** A class with a LinkML `unique_keys` carries `"x-unique-keys": [[slot, ...], ...]` on its definition, its own keys first and then inherited ones with each key's slots in declared order, and `study_schema.jsonschema.unique_keys(name)` returns them as a dict from class to tuples of slot names.
@@ -388,7 +390,7 @@ the priority inventory from here.
 |---|---|
 | [schema-tutorial.md](schema-tutorial.md) | The course: twelve chapters teaching the whole schema — what every entity is for, what each field holds and what it must not be confused with, the rules for filling a record, and the reasoning steps to take when a paper is ambiguous. Start here if you are new to the schema; the documents below are the references it teaches from |
 | [extraction-readme.md](extraction-readme.md) | Rules the schema cannot state: the gates that skip a paper, extraction conventions, validator invariants, mapper responsibilities, and known limits |
-| [extraction-deviations.yaml](extraction-deviations.yaml) | Every way the extraction schema is not a projection of storage. Currently: pipeline provenance, and nothing else. Its trailing comment lists the six shapes the hand-written schema used to have, as candidates to re-test |
+| [extraction-deviations.yaml](src/study_schema/linkml/extraction-deviations.yaml) | Every way the extraction schema is not a projection of storage. Currently: pipeline provenance, and nothing else. Its trailing comment lists the six shapes the hand-written schema used to have, as candidates to re-test |
 | [storage-schema-design-notes.md](storage-schema-design-notes.md) | Why the storage schema is shaped the way it is |
 | [analysis-entities.md](analysis-entities.md) | What each entity around an analysis is called, what owns it, and what points at what: the three layers, a crosswalk from the words papers use, and the joins a synthesis reads a record back out through. Start here before representing-models.md |
 | [representing-models.md](representing-models.md) | How to put a reported analysis into the schema: which facts belong to the model and which to the contrast, where each class's job ends, and what to do when a paper does not divide things the way the schema does. Its YAML fragments and the worked records under `pondie/tests/fixtures/examples/` are checked on every test run |

analysis-entities.md

@@ -4,7 +4,7 @@ This document names each object around an `Analysis` and shows its ownership and
 
 This is the map, not the instructions. [representing-models.md](representing-models.md) says how to
 encode a reported result and what to do when a paper resists;
-[neuroimaging-study-storage/](neuroimaging-study-storage/) says what each field is and how to fill
+[neuroimaging-study-storage/](src/study_schema/linkml/neuroimaging-study-storage/) says what each field is and how to fill
 it. Read this first if the class names do not yet mean anything, or if you know the statistics and
 want to know which of these words the schema has borrowed for something narrower.
 
@@ -265,6 +265,6 @@ acquired count when the analysis dropped participants.
 | The four values of `direction`, and the three ways to be non-directional | [§4](representing-models.md#4-the-four-values-of-direction-and-the-three-ways-to-be-non-directional) |
 | Twelve worked records, from a simple contrast to a two-stage model | [§5](representing-models.md#5-worked-models) |
 | A paper that does not divide the world the way the schema does | [§6](representing-models.md#6-when-the-paper-does-not-fit) |
-| What each field is, and how to fill it | [neuroimaging-study-storage/](neuroimaging-study-storage/) |
+| What each field is, and how to fill it | [neuroimaging-study-storage/](src/study_schema/linkml/neuroimaging-study-storage/) |
 | The rest of the record — cohorts, paradigm, acquisitions — taught rather than mapped | [schema-tutorial.md](schema-tutorial.md) |
 | Why the schema is shaped this way | [storage-schema-design-notes.md](storage-schema-design-notes.md) |

extraction-readme.md

@@ -185,7 +185,7 @@ The extraction schema used to carry eight sibling slots here and five on `Acquis
 "all eight empty" standing in for mass-univariate. Naming the type is one field instead of
 eight and cannot be silently ambiguous, but it does ask the extractor to make a choice it
 previously made by omission. If that turns out to extract worse, it is candidate 2 in
-[extraction-deviations.yaml](extraction-deviations.yaml).
+[extraction-deviations.yaml](src/study_schema/linkml/extraction-deviations.yaml).
 
 ### A task's name is split into its paradigm and the ways it varies
 
@@ -265,7 +265,7 @@ recorded verbatim and a 23-entry synonym table in the map did the translating.
 Extraction used to spread direction across five per-axis slots, one each for group, condition,
 timepoint, arm and factor, because comparative wording is per axis and that is what an
 extractor can point at. Storage always had the one slot, and now so does extraction. It is
-candidate 3 in [extraction-deviations.yaml](extraction-deviations.yaml) — the biggest of the
+candidate 3 in [extraction-deviations.yaml](src/study_schema/linkml/extraction-deviations.yaml) — the biggest of the
 six, and the one most worth measuring, since the collapse asks the extractor to resolve an
 entity to the model term whose level names it *and* to pick a direction in the same step.
 
@@ -676,7 +676,7 @@ impossible by construction rather than merely documented, and
 copies of the real schemas, because a completeness check that cannot fail is worth nothing.
 
 The shapes that used to need absorbing are listed as candidates at the end of
-[extraction-deviations.yaml](extraction-deviations.yaml). They were not obviously wrong --
+[extraction-deviations.yaml](src/study_schema/linkml/extraction-deviations.yaml). They were not obviously wrong --
 they were chosen because they read more naturally to an extractor -- and the point of
 starting from the projection is to find out which of them actually earn their cost. Adding
 one back is an entry in that file plus a matching entry in the map.
@@ -729,7 +729,7 @@ extraction schema still asks for them.
 
 `ConnectivityEdge.directionality` is `deterministic` without any outside source: the mapper looks
 it up from the containing analysis's `connectivity_method` through `value_map` in
-[extraction-to-storage.map.yaml](extraction-to-storage.map.yaml). Only DCM and Granger causality
+[extraction-to-storage.map.yaml](src/study_schema/linkml/extraction-to-storage.map.yaml). Only DCM and Granger causality
 support a claim about which region influences which, and no wording of the paper changes that, so
 the field is not one to ask a reader for. It is left unset where the method is written as free text,
 since there is nothing to look up — the one case where an edge's directionality is genuinely unknown

pyproject.toml

@@ -26,6 +26,7 @@ where = ["src"]
 [tool.setuptools.package-data]
 "study_schema" = ["py.typed"]
 "study_schema.jsonschema" = ["*.schema.json"]
+"study_schema.linkml" = ["*.yaml", "*/*.yaml"]
 
 [tool.pytest.ini_options]
 testpaths = ["tests"]

representing-models.md

@@ -3,7 +3,7 @@
 How to put a reported analysis into the storage schema, and how to tell which shape it is. This is
 the practical companion to [storage-schema-design-notes.md](storage-schema-design-notes.md), which
 holds the reasoning behind the shape rather than instructions for using it. The schema's field
-definitions are in [neuroimaging-study-storage/](neuroimaging-study-storage/) and say what each
+definitions are in [neuroimaging-study-storage/](src/study_schema/linkml/neuroimaging-study-storage/) and say what each
 field *is*; this says how the classes fit together and what to do when a paper does not divide the
 world the way the schema does. If the class names are not yet familiar,
 [analysis-entities.md](analysis-entities.md) is the map: what each entity is called, what owns it,

schema-tutorial.md

@@ -5,7 +5,7 @@ connect, what each field holds and what it must not be confused with, and how to
 is ambiguous.
 
 This is the teaching document. The schema files under
-[neuroimaging-study-storage/](neuroimaging-study-storage/) are the source of truth for what a field
+[neuroimaging-study-storage/](src/study_schema/linkml/neuroimaging-study-storage/) are the source of truth for what a field
 holds, and they win any disagreement with what follows.
 
 ---
@@ -1795,11 +1795,11 @@ required column leaves out `id`, which every class that has one requires; "Field
 ## Appendix B. Where everything is
 
 **The schema.** `neuroimaging-study-storage.yaml` plus eleven modules under
-[neuroimaging-study-storage/](neuroimaging-study-storage/): `subsets`, `design`, `study`, `group`,
+[neuroimaging-study-storage/](src/study_schema/linkml/neuroimaging-study-storage/): `subsets`, `design`, `study`, `group`,
 `task`, `assessment`, `region`, `acquisition`, `analysis_enums`, `analysis_details`, `analysis`.
 `analysis_enums` holds 27 of the 38 vocabularies, separately so that the analysis and
 analysis-details modules can both bind to them without either importing the other.
-[neuroimaging-study-extraction/](neuroimaging-study-extraction/) is the generated projection and is
+[neuroimaging-study-extraction/](src/study_schema/linkml/neuroimaging-study-extraction/) is the generated projection and is
 never hand-edited.
 
 **The uniqueness rules**, each saying "recorded twice is one thing": `ModelTerm.name` within its

src/study_schema/__init__.py

@@ -4,6 +4,7 @@
     study_schema.models.extraction    Study (an extraction record)   pondie -> ingester
     study_schema.models.storage       Study (the storage shape)      ingester -> neurostore
     study_schema.jsonschema           the same contracts as JSON Schema, for non-Python readers
+    study_schema.linkml               the LinkML source, as paths into the installed package
     study_schema.keys                 the analysis key rule
     study_schema.statistics           which statistic kinds are signed; a point's side
     study_schema.layouts              where those files sit on disk (needs the `layouts` extra)

src/study_schema/linkml/__init__.py

@@ -0,0 +1,14 @@
+"""The LinkML schemas the package is generated from, as paths into the installed package.
+
+    EXTRACTION     neuroimaging-study-extraction.yaml   pondie -> ingester
+    STORAGE        neuroimaging-study-storage.yaml      ingester -> neurostore
+    PAPER_PARSE    neuroimaging-paper-parse.yaml        ingestion -> pondie, ingester
+"""
+
+from pathlib import Path
+
+_DIRECTORY = Path(__file__).parent
+
+EXTRACTION = _DIRECTORY / "neuroimaging-study-extraction.yaml"
+STORAGE = _DIRECTORY / "neuroimaging-study-storage.yaml"
+PAPER_PARSE = _DIRECTORY / "neuroimaging-paper-parse.yaml"

extraction-deviations.yaml → src/study_schema/linkml/extraction-deviations.yaml

@@ -1,7 +1,7 @@
 # extraction-deviations.yaml
 #
 # Everything in the extraction schema that is not a mechanical projection of the storage
-# schema. gen_extraction_schema.py applies this file on top of the projection; nothing
+# schema. pondie.schema.generate applies this file on top of the projection; nothing
 # else may differ, and check_extraction_mirrors_storage.py enforces that.
 #
 # The file has two parts, and the difference between them matters.
@@ -35,7 +35,7 @@ header:
     Immutable, evidence-first structured output from extracting a neuroimaging
     publication.
 
-    This schema is generated from neuroimaging-study-storage by gen_extraction_schema.py.
+    This schema is generated from neuroimaging-study-storage by pondie.schema.generate.
     It has the same classes with the same slots in the same nesting; what differs is that
     every source-derived value is wrapped. An ExtractedValue records whether a value was
     found (extraction_status), the value itself when it was, how it relates to the source

storage-schema-design-notes.md

@@ -2,7 +2,7 @@
 
 Why the storage schema is shaped the way it is: the reasoning behind a decision, the
 alternatives weighed against it, and the measurement that settled it. The field
-definitions themselves are in [neuroimaging-study-storage/](neuroimaging-study-storage/)
+definitions themselves are in [neuroimaging-study-storage/](src/study_schema/linkml/neuroimaging-study-storage/)
 and say what each field *is*; [representing-models.md](representing-models.md) says how to
 use them; [analysis-entities.md](analysis-entities.md) says what owns what. This holds the
 part none of those should carry.
@@ -407,7 +407,7 @@ it is asking for a chance to disagree with the method, and the corpus took the c
 on 2026-08-14, 16 sat on `dcm` and **five of those were marked `undirected`**, which DCM cannot be.
 
 It therefore became a lookup in `derivations.ConnectivityEdge.directionality.value_map` in
-[extraction-to-storage.map.yaml](extraction-to-storage.map.yaml) — two directed values, six
+[extraction-to-storage.map.yaml](src/study_schema/linkml/extraction-to-storage.map.yaml) — two directed values, six
 undirected, one line each.
 
 ### What the derivation cannot do

tests/test_linkml.py

@@ -0,0 +1,82 @@
+from __future__ import annotations
+
+import shutil
+import subprocess
+import sys
+import zipfile
+from pathlib import Path
+
+import pytest
+import yaml
+
+from study_schema import linkml
+
+ROOT = Path(__file__).resolve().parents[1]
+LINKML = linkml.EXTRACTION.parent.resolve()
+SCHEMAS = ("EXTRACTION", "STORAGE", "PAPER_PARSE")
+
+
+def _imports(path: Path) -> list[str]:
+    document = yaml.safe_load(path.read_text("utf-8")) or {}
+    return [entry for entry in document.get("imports", []) if not entry.startswith("linkml:")]
+
+
+@pytest.mark.parametrize("name", SCHEMAS)
+def test_constant_points_at_a_packaged_schema(name):
+    schema = getattr(linkml, name)
+
+    assert isinstance(schema, Path)
+    assert schema.is_file()
+    assert schema.resolve().is_relative_to(LINKML)
+
+
+@pytest.mark.parametrize("name", SCHEMAS)
+def test_imports_resolve_inside_the_packaged_tree(name):
+    pending = [getattr(linkml, name)]
+    seen = set()
+    while pending:
+        current = pending.pop()
+        if current in seen:
+            continue
+        seen.add(current)
+        for target in _imports(current):
+            imported = (current.parent / f"{target}.yaml").resolve()
+            assert imported.is_file(), f"{current.name} imports {target!r}, which is not packaged"
+            assert imported.is_relative_to(LINKML)
+            pending.append(imported)
+    assert len(seen) > 1
+
+
+def test_wheel_ships_the_linkml_tree(tmp_path):
+    pytest.importorskip("setuptools.build_meta", reason="building a wheel needs setuptools")
+    project = tmp_path / "project"
+    project.mkdir()
+    for name in ("pyproject.toml", "README.md"):
+        shutil.copy(ROOT / name, project / name)
+    shutil.copytree(ROOT / "src", project / "src")
+    wheels = tmp_path / "wheels"
+    subprocess.run(
+        [
+            sys.executable,
+            "-c",
+            "import sys; from setuptools import build_meta; print(build_meta.build_wheel(sys.argv[1]))",
+            str(wheels),
+        ],
+        cwd=project,
+        check=True,
+        capture_output=True,
+    )
+    (wheel,) = wheels.glob("*.whl")
+    with zipfile.ZipFile(wheel) as archive:
+        shipped = {
+            name for name in archive.namelist()
+            if name.startswith("study_schema/linkml/") and name.endswith(".yaml")
+        }
+    expected = {
+        f"study_schema/linkml/{path.relative_to(LINKML).as_posix()}"
+        for path in LINKML.rglob("*.yaml")
+    }
+    assert expected
+    assert shipped == expected
+    assert "study_schema/linkml/extraction-deviations.yaml" in shipped
+    assert "study_schema/linkml/extraction-to-storage.map.yaml" in shipped

tests/test_statistics.py

@@ -1,15 +1,14 @@
-from pathlib import Path
-
 import yaml
 
+from study_schema import linkml
 from study_schema.statistics import (
     DIRECTIONAL_KINDS,
     NON_DIRECTIONAL_KINDS,
     is_directional,
     point_side,
 )
 
-ENUMS = Path(__file__).parent.parent / "neuroimaging-paper-parse" / "parse_enums.yaml"
+ENUMS = linkml.EXTRACTION.parent / "neuroimaging-paper-parse" / "parse_enums.yaml"
 
 
 def test_kind_sets_match_the_schema_annotations():

tests/test_unique_keys.py

@@ -6,7 +6,7 @@
 
 # A golden check of the YAML, not a sync with neurostore: these pins are neurostore's
 # DECLARED_KEYS in neurostore.study_schema_tools on its review branch. Each key matches a
-# unique_keys declaration in neuroimaging-study-storage/analysis.yaml, with its slots in
+# unique_keys declaration in src/study_schema/linkml/neuroimaging-study-storage/analysis.yaml, with its slots in
 # declared order.
 NEUROSTORE_DECLARED_KEYS = {
     "Cell": (("term", "level"),),

tools/generate_models.py

@@ -17,51 +17,53 @@
 from dataclasses import dataclass
 from pathlib import Path
 
+from study_schema import linkml
+
 ROOT = Path(__file__).resolve().parents[1]
 PACKAGE = ROOT / "src" / "study_schema"
 
 
 @dataclass(frozen=True)
 class Contract:
     module: str
-    schema: str
+    schema: Path
     #: JSON Schema file name -> top-level class.
     roots: dict[str, str]
 
 
 CONTRACTS = (
     Contract(
         module="paper_parse",
-        schema="neuroimaging-paper-parse.yaml",
+        schema=linkml.PAPER_PARSE,
         roots={
             "parsed-paper": "ParsedPaper",
             "coordinate-parse": "CoordinateParse",
         },
     ),
     Contract(
         module="extraction",
-        schema="neuroimaging-study-extraction.yaml",
+        schema=linkml.EXTRACTION,
         roots={"extraction-record": "Study"},
     ),
     Contract(
         module="storage",
-        schema="neuroimaging-study-storage.yaml",
+        schema=linkml.STORAGE,
         roots={"storage-study": "Study"},
     ),
 )
 
 
 def _header(contract: Contract) -> str:
     return (
-        f"# Generated by tools/generate_models.py from {contract.schema}.\n"
+        f"# Generated by tools/generate_models.py from {contract.schema.name}.\n"
         "# Do not edit: change the YAML and regenerate.\n"
     )
 
 
 def _view(contract: Contract):
     from linkml_runtime.utils.schemaview import SchemaView
 
-    view = SchemaView(str(ROOT / contract.schema))
+    view = SchemaView(str(contract.schema))
     view.merge_imports()
     return view