Include file metadata and schema version to extraction/validation

.github/CONTRIBUTING.md

@@ -1,6 +1,6 @@
-# Contributing to REPO-NAME
+# Contributing to biosim-extractor
 
-Thank you for your interest in contributing to **REPO-NAME**!
+Thank you for your interest in contributing to **biosim-extractor**!
 
 We’re excited to collaborate with developers, researchers, and community members to make REPO-NAME better for everyone.
 
@@ -62,4 +62,4 @@ Well-documented issues help us address problems faster and keep REPO-NAME stable
 
 ---
 
-Thank you for helping improve **REPO-NAME**, your contributions make open source stronger!
+Thank you for helping improve **biosim-extractor**, your contributions make open source stronger!

.gitignore

@@ -70,6 +70,7 @@ instance/
 
 # Sphinx documentation
 docs/_build/
+.DS_Store
 
 # PyBuilder
 .pybuilder/

README.md

@@ -1,6 +1,14 @@
 biosim-extractor
 ================
-A repository for extracting simulation data from output files produced from molecular dynamics (MD) simulations of biomolecules and validating against biosim-schem.
+
+<p align="center">
+    <img src="docs/source/_static/logos/biosim-extractor-logo-light-text.png#gh-dark-mode-only" alt="biosim-extractor logo" width="300"/>
+    <img src="docs/source/_static/logos/biosim-extractor-logo-dark-text.png#gh-light-mode-only" alt="biosim-extractor logo" width="300"/>
+</p>
+
+biosim-extractor is a repository for extracting simulation data from output files produced from molecular dynamics (MD) simulations of biomolecules and validating against [biosim-schema](https://github.com/CCPBioSim/biosim-schema).
+
+See the [biosim-extractor documentation](https://biosim-extractor.readthedocs.io/en/latest/) for more information.
 
 ## Project Status
 

biosim_extractor/metadata/filemetadata.py

@@ -0,0 +1,68 @@
+"""Utility to extract binary file metadata (size, hash)."""
+
+import hashlib
+from pathlib import Path
+
+
+def file_metadata(path, role="other", hash_algorithm="md5"):
+    """Extract name, size (MB), and hash of a single file.
+
+    Args:
+        path: Input file path (str or :class:`pathlib.Path`).
+        role: Optional category label for the file. Defaults to "other".
+        hash_algorithm: Hash function identifier ('md5', 'sha256'). Default is 'md5'.
+
+    Returns:
+        Dictionary containing 'file_name', 'file_size' (dict), 'file_hash',
+        'file_hash_algorithm', and 'file_role'.
+    """
+    path = Path(path)
+    digest = hashlib.new(hash_algorithm)
+    size_mb = path.stat().st_size / 1_000_000
+
+    with path.open("rb") as handle:
+        for chunk in iter(lambda: handle.read(1024 * 1024), b""):
+            digest.update(chunk)
+
+    return {
+        "file_name": path.name,
+        "file_size": {"value": round(size_mb, 6), "value_unit": "MB"},
+        "file_hash": digest.hexdigest(),
+        "file_hash_algorithm": hash_algorithm,
+        "file_role": role,
+    }
+
+
+def files_metadata(saved_files):
+    """Compile metadata for multiple files grouped by role.
+
+    Args:
+        saved_files: Dictionary mapping roles to lists of file paths
+            (:data:`str` or :class:`pathlib.Path`).
+
+    Returns:
+        Flat list of ``file_metadata`` dictionaries for all provided paths.
+    """
+    return [
+        file_metadata(path, role)
+        for role, paths in saved_files.items()
+        for path in paths
+    ]
+
+
+def group_files(filepaths, saved_files=None, role="other"):
+    """Add file paths to a role-based dictionary.
+
+    Args:
+        filepaths: Iterable of file paths to add.
+        saved_files: Dictionary mapping roles to lists of file paths.
+        role: Role name under which to store the file paths.
+
+    Returns:
+        Updated dictionary of file paths grouped by role.
+    """
+    if saved_files is None:
+        saved_files = {}
+    for filepath in filepaths:
+        saved_files.setdefault(role, []).append(filepath)
+    return saved_files

biosim_extractor/metadata/populatemetadata.py

@@ -13,6 +13,7 @@
 from biosim_extractor.helpers.metadata_utils import round_floats
 from biosim_extractor.mdanalysis.toptraj import TopTrajParser
 from biosim_extractor.metadata.fetchschema import get_schema, update_schema
+from biosim_extractor.metadata.filemetadata import files_metadata, group_files
 from biosim_extractor.metadata.validatemetadata import validate_metadata
 from biosim_extractor.units.unitconversion import UnitConverter
 
@@ -200,19 +201,34 @@ def __init__(
         engine=None,
         top_file=None,
         traj_file=None,
+        store_file_metadata=True,
     ):
-        """
-        Args:
-            schema_path: Path to the extraction schema JSON file.
-            log_file: Path to the MD engine log file.
-            engine: MD engine name (``"amber"``, ``"gromacs"``).
-            top_file: Path to the topology file.
-            traj_file: Path to the trajectory file (or list of paths).
+        """Orchestrates extraction of MD engine metadata from log files (Amber/GROMACS)
+        or topology/trajectory inputs, mapping results to the `biosim-schema` format.
+
+        Features supported by this class:
+            - Reads logs via specialized parsers (`AmberLogParser`, etc.).
+            - Flattens nested structures and applies reverse-forward schema mappings.
+            - Handles unit conversions where required (e.g., kcal/mol → eV).
+            - Stores metadata about input files if `store_file_metadata=True`.
+
+        Use cases:
+            1. Batch processing logs via CLI (`--logfile`, `--engine`).
+            2. Populating a single simulation from top/traj without logs via MDAnalysis integration.
+
+        Args (constructor):
+            schema_path: Path to the engine mapping JSON file.
+            log_file: Optional MD engine log file path.
+            engine: MD engine name, such as "amber" or "gromacs".
+            top_file: Optional topology file path.
+            traj_file: Optional trajectory file path or list of trajectory paths.
+            store_file_metadata: If True, include input file metadata in the output.
         """
         self.schema_path = schema_path
         self.log_file = log_file
         self.top_file = top_file
         self.traj_file = traj_file
+        self.store_file_metadata = store_file_metadata
         self.engine = engine
         if engine:
             self.engine = engine.lower()
@@ -229,7 +245,6 @@ def populate(self):
             entries removed.
         """
         self.load_schema()
-
         if self.engine:
             self.engine_data = self.parse_log()
 
@@ -247,6 +262,18 @@ def populate(self):
         # self.data["SimulationMetadata"]["@type"] = "SimulationMetadata"
         result = self.data["SimulationMetadata"]
 
+        # save file metadata in dict
+        if self.store_file_metadata:
+            saved_files = {}
+            if self.log_file:
+                saved_files = group_files([self.log_file], saved_files, role="log")
+            if self.top_file and self.traj_file:
+                saved_files = group_files([self.top_file], saved_files, role="topology")
+                saved_files = group_files(
+                    self.traj_file, saved_files, role="trajectory"
+                )
+            result["files"] = files_metadata(saved_files)
+
         # Remove any dict that contains a None field
         result = remove_null_parents(result) or {}
         # Round all floats
@@ -490,7 +517,12 @@ def convert_values(self, value, term, is_vector=False):
 
 
 def resolve_schema_inputs(args):
-    """Resolve mapping and biosim schema paths from args or remote schema bundle."""
+    """Resolve mapping and biosim schema paths from args or remote schema bundle.
+
+    If either path argument (`mappingschema`, `biosimschema`) is missing, the function
+    fetches a bundled schema release (optionally updating if requested). This ensures
+    downstream processing has valid JSON/YAML sources without requiring manual caching setup
+    """
     mapping_path = args.mappingschema
     biosim_path = args.biosimschema
 
@@ -559,6 +591,18 @@ def parse_args():
     parser.add_argument("--top", help="Topology file path")
     parser.add_argument("--traj", nargs="+", help="Trajectory file path")
     parser.add_argument("--config", help="Configuration file path")
+    parser.add_argument(
+        "--exclude-file-metadata",
+        action="store_false",
+        help="Include flag to include file metadata in metadata output",
+    )
+    parser.add_argument(
+        "--file-metadata",
+        dest="store_file_metadata",
+        action=argparse.BooleanOptionalAction,
+        default=True,
+        help="Include file metadata in output (use --no-file-metadata to disable).",
+    )
     parser.add_argument("--output", "-o", help="Output file path")
 
     return parser.parse_args()
@@ -576,6 +620,7 @@ def main():
         engine=args.engine,
         top_file=args.top,
         traj_file=args.traj,
+        store_file_metadata=args.store_file_metadata,
     )
 
     result = populator.populate()

biosim_extractor/metadata/validatemetadata.py

@@ -4,11 +4,38 @@
 """
 
 import os
+import re
 import warnings
+from pathlib import Path
 
 from linkml.validator import validate
 
 
+def extract_schema_version(schema_path):
+    """Extract the biosim schema version from a local schema YAML file.
+
+    Args:
+    schema_path (str | Path): Path to the biosim schema YAML file.
+
+    Returns:
+    str | None: Parsed schema version string, or None if the path is missing,
+    points to a URL, the file does not exist, or no version field is found.
+    """
+    if not schema_path or str(schema_path).startswith("http"):
+        return None
+
+    p = Path(schema_path)
+    if not p.exists():
+        return None
+
+    text = p.read_text(encoding="utf-8")
+    m = re.search(
+        r"(?m)^version:\s*['\"]?([^\r\n'\"#]+)['\"]?\s*(?:#.*)?$",
+        text,
+    )
+    return m.group(1).strip() if m else None
+
+
 def validate_metadata(result, biosimschema_path=None, strict=False):
     """Validate a populated metadata dict, optionally against a biosim schema.
 
@@ -48,6 +75,11 @@ def validate_extracted(instance, schema_path):
     Returns:
         list: Validation error messages. An empty list means the instance is valid.
     """
+
+    schema_version = extract_schema_version(schema_path)
+    if schema_version and isinstance(instance, dict):
+        instance.setdefault("biosim_schema_version", schema_version)
+
     errors = []
 
     errors.extend(_validate_all_vector_values(instance))

docs/source/conf.py

@@ -40,3 +40,7 @@
 
 html_theme = 'furo'
 html_static_path = ['_static']
+html_theme_options = {
+    "dark_logo": "logos/biosim-extractor-logo-light-text.png",
+    "light_logo": "logos/biosim-extractor-logo-dark-text.png",
+}

docs/source/index.rst

@@ -1,37 +1,32 @@
-.. CCPBioSim-Python-Template documentation master file, created by
-   sphinx-quickstart on Wed Dec 10 16:51:51 2025.
-   You can adapt this file completely to your liking, but it should at least
-   contain the root `toctree` directive.
-
 BioSim Extractor Documentation
 ==============================
 
-A toolkit for extracting and standardizing metadata from molecular dynamics simulations.
+``biosim-extractor`` is available at http://github.com/CCPBioSim/biosim-extractor
 
-Contents
---------
+.. image:: /_static/logos/biosim-extractor-logo-dark-text.png
+   :width: 250px
+   :align: center
+   :class: only-light
 
-.. toctree::
-   :maxdepth: 2
-   :caption: Contents:
+.. image:: /_static/logos/biosim-extractor-logo-light-text.png
+   :width: 250px
+   :align: center
+   :class: only-dark
 
-   installation
-   usage
-   contributing
-   api
+A toolkit for extracting standardised metadata from molecular dynamics (MD) simulations.
 
 Overview
 --------
 
-BioSim Extractor provides tools to parse, extract, and standardize metadata from MD simulation outputs (AMBER, GROMACS, LAMMPS, etc.), and validate against biosim-schema.
+BioSim Extractor provides tools to parse, extract, and standardize metadata from MD simulation outputs (AMBER, GROMACS, etc.), and validate against `biosim-schema <https://github.com/CCPBioSim/biosim-schema>`_.
 
 Features
 --------
 
 - Parse log files from multiple MD engines
 - Extract topology and trajectory metadata
-- Standardize units and schema
-- Validate metadata against biosim-schema
+- Standardise units and schema
+- Validate metadata against biosim-schema `biosim-schema <https://github.com/CCPBioSim/biosim-schema>`_
 
 Getting Started
 ---------------
@@ -43,9 +38,37 @@ API Reference
 
 The full API documentation is available in :doc:`api`.
 
+Contents
+--------
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents:
+
+   installation
+   usage
+   contributing
+   api
+
+Funding
+=======
+
+Contributors to biosim-extractor were supported by
+
+.. image::  _static/logos/psdi-logo.png
+   :height: 100px
+   :target: https://www.psdi.ac.uk/
+
+.. image:: _static/logos/cosec-logo.png
+   :height: 100px
+   :target: https://www.cosec.ac.uk/
+
+.. image:: _static/logos/CCPBioSim-logo.png
+   :height: 100px
+   :target: https://www.ccpbiosim.ac.uk/
+
 Indices and tables
 ------------------
 
 * :ref:`genindex`
 * :ref:`modindex`
-* :ref:`search`