feat: write the calibration report as JSON and YAML

CHANGELOG.md

@@ -11,6 +11,24 @@ The release procedure (prompt template and step-by-step instructions) lives in [
 
 ## [Unreleased]
 
+### Added
+- **`srcmorph:calibrate` now writes a machine-readable report.** The goal built a `CalibrationReport`
+  and printed it as `INFO` lines, and that was the only output — so the numbers a calibration run
+  produces (prefill / decode throughput, chars per token per model) could not be diffed across runs,
+  committed as a baseline, or fed back into `aiDefinitions` by anything other than a human re-reading
+  the console, which is most of the point of measuring them.
+
+  `execute()` now also writes `srcmorph-calibration.json` and `srcmorph-calibration.yaml` into the
+  configured `outputDirectory`, creating it if needed. Both carry the same keys in the same order and
+  add `loadSeconds`, `midPrefillTokensPerSecond` and `cachedPromptTokens`, which the pasteable
+  `<calibration>` block does not. The three figures the XML *does* carry are formatted identically, so
+  the JSON and the paste block can never disagree about what was measured.
+
+  The renderers are hand-rolled in `CalibrationReport` rather than delegated to Jackson: the core
+  module is framework-free and carries no JSON dependency (only `srcmorph-cli` does), and
+  `renderXml()` already set that precedent. Model keys come from user configuration, so they are
+  escaped for both formats — an unescaped quote would produce a file neither parser accepts.
+
 ### Added
 - **`flashAttn` works.** The knob was documented and settable since it was introduced, but could not
   be forwarded: `--flash-attn` takes a mandatory `on|off|auto` and `net.ladenthin:llama` offered only

TODO.md

@@ -24,15 +24,6 @@ everything below is genuinely still open.
   not dropping the idea. Deliberately out of scope for 1.2.0: it is a build-time question, not a
   correctness one.
 
-- **`srcmorph:calibrate` reports only through the log.** `CalibrateEngine` builds a
-  `CalibrationReport` and `CalibrateMojo` prints it as `INFO` lines. There is no machine-readable
-  output, so the numbers a calibration run produces (prefill / decode throughput, chars per token per
-  model) cannot be diffed across runs, fed back into `aiDefinitions`, or committed as a baseline --
-  which is most of the point of measuring them. Emitting the same report as JSON and YAML next to
-  the log (the CLI already carries both Jackson mappers, and `SrcMorphConfiguration` round-trips
-  through them) would close it. **Was announced during the 1.2.0 audit cycle and never landed**;
-  it is a feature, not a fix, so it is not a 1.2.0 blocker.
-
 - **The sixteen GPU classifier fat jars are verified structurally, never launched.** Since 1.2.0
   `.github/verify-classifier-fatjars.sh` asserts each is the artifact its name claims (one jar per
   classifier, a native for the promised OS/arch, a native set that differs from the default jar's, so

srcmorph-maven-plugin/README.md

@@ -480,6 +480,22 @@ src/site/ai/
                 ├── AnotherClass.java.ai.md
                 └── package.ai.md
 ```
+
+The `calibrate` goal writes two more files into the same `outputDirectory`:
+
+```
+src/site/ai/
+├── srcmorph-calibration.json
+└── srcmorph-calibration.yaml
+```
+
+Both carry one entry per measured model — `modelKey`, `loadSeconds`,
+`prefillTokensPerSecond`, `decodeTokensPerSecond`, `charsPerToken`,
+`midPrefillTokensPerSecond`, `cachedPromptTokens` — so a calibration run can be diffed
+across machines or committed as a baseline, instead of existing only as console output.
+The three figures that also appear in the paste-ready `<calibration>` block are formatted
+identically, so the two can never disagree about what was measured.
+
 ## Design Principles
 - Deterministic metadata (hash-based change detection)
 - Separation of concerns (header = metadata, body = summary)

srcmorph-maven-plugin/src/test/java/net/ladenthin/maven/srcmorph/mojo/MojoConfigurationMappingTest.java

@@ -9,6 +9,7 @@
 import static org.hamcrest.MatcherAssert.assertThat;
 
 import java.io.File;
+import java.nio.file.Path;
 import java.lang.reflect.Field;
 import java.util.Arrays;
 import java.util.Collections;
@@ -19,6 +20,7 @@
 import net.ladenthin.srcmorph.config.SrcMorphConfiguration;
 import net.ladenthin.srcmorph.prompt.AiPromptDefinition;
 import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
 
 /**
  * Covers each concrete goal's <em>own</em> {@code @Parameter} mapping.
@@ -35,6 +37,10 @@
  */
 public class MojoConfigurationMappingTest {
 
+    /** Per-test output directory, so a goal that writes files cannot pollute the checkout. */
+    @TempDir
+    Path outputDirectory;
+
     // <editor-fold defaultstate="collapsed" desc="fixture">
 
     /** The goal parameters are private, so the same reflective assignment the sibling skip test uses. */
@@ -199,6 +205,10 @@ public void calibrateMojo_execute_logsTheInstructionsAndTheRenderedReport() thro
         final CalibrateMojo mojo = new CalibrateMojo();
         fillSharedParameters(mojo);
         wireMockModelAndRule(mojo);
+        // execute() writes the machine-readable report into outputDirectory; fillSharedParameters
+        // points that at the relative path "out-dir", so without this the goal leaves two files
+        // behind in the checkout on every run.
+        mojo.outputDirectory = outputDirectory.toFile();
         final CapturingLog log = new CapturingLog();
         mojo.setLog(log);
 

srcmorph/src/main/java/net/ladenthin/srcmorph/engine/CalibrateEngine.java

@@ -4,6 +4,11 @@
 package net.ladenthin.srcmorph.engine;
 
 import java.io.IOException;
+import java.io.OutputStreamWriter;
+import java.io.Writer;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
 import java.util.ArrayList;
 import java.util.LinkedHashMap;
 import java.util.List;
@@ -103,7 +108,50 @@ public CalibrationReport execute() throws SrcMorphException {
                     providerFactory));
         }
 
-        return new CalibrationReport(measurements);
+        final CalibrationReport report = new CalibrationReport(measurements);
+        writeMachineReadableReport(report);
+        return report;
+    }
+
+    /**
+     * Writes the report next to the log, as JSON and YAML, into the configured output directory.
+     *
+     * <p>Without this the numbers a calibration run produces exist only as log lines: they cannot be
+     * diffed across runs, committed as a baseline, or fed back into {@code aiDefinitions} by anything
+     * other than a human re-reading the console &#x2014; which is most of the point of measuring
+     * them.</p>
+     *
+     * @param report the report to write
+     * @throws SrcMorphException if the directory cannot be created or a file cannot be written
+     */
+    private void writeMachineReadableReport(final CalibrationReport report) throws SrcMorphException {
+        final Path directory = config.getOutputDirectory().toPath();
+        final Path json = directory.resolve(CalibrationReport.JSON_FILE_NAME);
+        final Path yaml = directory.resolve(CalibrationReport.YAML_FILE_NAME);
+        try {
+            Files.createDirectories(directory);
+            writeUtf8(json, report.renderJson());
+            writeUtf8(yaml, report.renderYaml());
+        } catch (final IOException e) {
+            throw new SrcMorphException("Failed to write the calibration report to " + directory + ": " + e, e);
+        }
+        LOGGER.info("");
+        LOGGER.info("Machine-readable calibration report written to:");
+        LOGGER.info("  {}", json);
+        LOGGER.info("  {}", yaml);
+    }
+
+    /**
+     * Writes UTF-8 text to a file, replacing any previous content.
+     *
+     * @param target the file
+     * @param text   the content
+     * @throws IOException if the write fails
+     */
+    private static void writeUtf8(final Path target, final String text) throws IOException {
+        try (Writer writer = new OutputStreamWriter(Files.newOutputStream(target), StandardCharsets.UTF_8)) {
+            writer.write(text);
+        }
     }
 
     /**

srcmorph/src/main/java/net/ladenthin/srcmorph/engine/CalibrationReport.java

@@ -57,6 +57,155 @@ public String renderXml() {
         return out.toString();
     }
 
+    /**
+     * File name of the machine-readable JSON report, written by {@link CalibrateEngine#execute()}
+     * into the configured output directory.
+     */
+    public static final String JSON_FILE_NAME = "srcmorph-calibration.json";
+
+    /** File name of the machine-readable YAML report, written alongside {@link #JSON_FILE_NAME}. */
+    public static final String YAML_FILE_NAME = "srcmorph-calibration.yaml";
+
+    /** Format of the two throughput figures; identical to the one {@link #renderXml()} pastes. */
+    private static final String FORMAT_TOKENS_PER_SECOND = "%.1f";
+
+    /** Format of the chars-per-token figure; identical to the one {@link #renderXml()} pastes. */
+    private static final String FORMAT_CHARS_PER_TOKEN = "%.2f";
+
+    /** Format of the load duration, which has no counterpart in the XML block. */
+    private static final String FORMAT_LOAD_SECONDS = "%.3f";
+
+    /**
+     * Renders the report as JSON.
+     *
+     * <p>Hand-rolled rather than delegated to Jackson on purpose: this module is framework-free and
+     * carries no JSON dependency (only {@code srcmorph-cli} does), and {@link #renderXml()} sets the
+     * same precedent. The document is small and fully determined by six numbers and a key.</p>
+     *
+     * <p>The three figures that also appear in the {@code <calibration>} block are formatted
+     * identically, so the JSON and the paste-ready XML can never disagree about what was measured.</p>
+     *
+     * @return the JSON document; a {@code models} array that is empty when no model was measured
+     */
+    public String renderJson() {
+        if (measurements.isEmpty()) {
+            return "{\n  \"models\": []\n}\n";
+        }
+        final StringBuilder out = new StringBuilder("{\n  \"models\": [\n");
+        for (int i = 0; i < measurements.size(); i++) {
+            final ModelMeasurement entry = measurements.get(i);
+            final AiCalibrationMeasurement m = entry.measurement();
+            out.append("    {\n");
+            out.append("      \"modelKey\": ").append(quote(entry.modelKey())).append(",\n");
+            appendJsonNumber(out, "loadSeconds", FORMAT_LOAD_SECONDS, m.loadSeconds());
+            appendJsonNumber(out, "prefillTokensPerSecond", FORMAT_TOKENS_PER_SECOND, m.prefillTokensPerSecond());
+            appendJsonNumber(out, "decodeTokensPerSecond", FORMAT_TOKENS_PER_SECOND, m.decodeTokensPerSecond());
+            appendJsonNumber(out, "charsPerToken", FORMAT_CHARS_PER_TOKEN, m.charsPerToken());
+            appendJsonNumber(out, "midPrefillTokensPerSecond", FORMAT_TOKENS_PER_SECOND, m.midPrefillTokensPerSecond());
+            out.append("      \"cachedPromptTokens\": ").append(m.cachedPromptTokens()).append('\n');
+            out.append("    }");
+            out.append(i + 1 < measurements.size() ? ",\n" : "\n");
+        }
+        out.append("  ]\n}\n");
+        return out.toString();
+    }
+
+    /**
+     * Renders the report as YAML, carrying exactly the same keys, order and number formatting as
+     * {@link #renderJson()}.
+     *
+     * <p>Every scalar is emitted double-quoted or numeric, which keeps the output inside the subset of
+     * YAML that needs no emitter to get right &#x2014; no block scalars, no anchors, no indentation
+     * subtleties beyond a fixed two-level indent.</p>
+     *
+     * @return the YAML document; {@code models: []} when no model was measured
+     */
+    public String renderYaml() {
+        if (measurements.isEmpty()) {
+            return "models: []\n";
+        }
+        final StringBuilder out = new StringBuilder("models:\n");
+        for (final ModelMeasurement entry : measurements) {
+            final AiCalibrationMeasurement m = entry.measurement();
+            out.append("  - modelKey: ").append(quote(entry.modelKey())).append('\n');
+            appendYamlNumber(out, "loadSeconds", FORMAT_LOAD_SECONDS, m.loadSeconds());
+            appendYamlNumber(out, "prefillTokensPerSecond", FORMAT_TOKENS_PER_SECOND, m.prefillTokensPerSecond());
+            appendYamlNumber(out, "decodeTokensPerSecond", FORMAT_TOKENS_PER_SECOND, m.decodeTokensPerSecond());
+            appendYamlNumber(out, "charsPerToken", FORMAT_CHARS_PER_TOKEN, m.charsPerToken());
+            appendYamlNumber(out, "midPrefillTokensPerSecond", FORMAT_TOKENS_PER_SECOND, m.midPrefillTokensPerSecond());
+            out.append("    cachedPromptTokens: ").append(m.cachedPromptTokens()).append('\n');
+        }
+        return out.toString();
+    }
+
+    /**
+     * Appends one {@code "key": number,} line to the JSON buffer.
+     *
+     * @param out    the buffer
+     * @param key    the JSON key
+     * @param format the {@link String#format} pattern for the value
+     * @param value  the value
+     */
+    private static void appendJsonNumber(
+            final StringBuilder out, final String key, final String format, final double value) {
+        out.append("      \"")
+                .append(key)
+                .append("\": ")
+                .append(String.format(Locale.ROOT, format, value))
+                .append(",\n");
+    }
+
+    /**
+     * Appends one {@code key: number} line to the YAML buffer.
+     *
+     * @param out    the buffer
+     * @param key    the YAML key
+     * @param format the {@link String#format} pattern for the value
+     * @param value  the value
+     */
+    private static void appendYamlNumber(
+            final StringBuilder out, final String key, final String format, final double value) {
+        out.append("    ")
+                .append(key)
+                .append(": ")
+                .append(String.format(Locale.ROOT, format, value))
+                .append('\n');
+    }
+
+    /**
+     * Renders a string as a double-quoted scalar that is valid in both JSON and YAML.
+     *
+     * <p>A model key comes from user configuration, so it can carry a quote, a backslash or a control
+     * character; without escaping, one such key would produce a file neither parser accepts.</p>
+     *
+     * @param value the string
+     * @return the quoted, escaped scalar
+     */
+    private static String quote(final String value) {
+        final StringBuilder out = new StringBuilder(value.length() + 2);
+        out.append('"');
+        for (int i = 0; i < value.length(); i++) {
+            final char c = value.charAt(i);
+            if (c == '"') {
+                out.append("\\\"");
+            } else if (c == '\\') {
+                out.append("\\\\");
+            } else if (c == '\n') {
+                out.append("\\n");
+            } else if (c == '\r') {
+                out.append("\\r");
+            } else if (c == '\t') {
+                out.append("\\t");
+            } else if (c < ' ') {
+                out.append(String.format(Locale.ROOT, "\\u%04x", (int) c));
+            } else {
+                out.append(c);
+            }
+        }
+        out.append('"');
+        return out.toString();
+    }
+
     /**
      * Appends a copy-pasteable {@code <calibration>} block (with a comment naming the model) to the buffer.
      *