diff --git a/CLAUDE.md b/CLAUDE.md index 0d1717c..861d854 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -353,6 +353,14 @@ were the ones intended. A reading of the diff proves nothing; the DOM does. - **A French option name stays accepted for ever**, silently, now that its English equivalent has arrived. Scripts already deployed never break. Only `faits.jsonl` will break, in format 2.0, and that is assumed: it is three days old. +- **A produced file keeps one vocabulary, and `diagnostic.json`'s is French.** Its keys — + `executions`, `rapprochement`, `fichiersSansSource` — predate the switch to English and are + frozen like every other produced key. New keys there follow them (`empreinte`, + `fichiersEcrits`, `seuilAlerte`): a document half in one language and half in the other + reads worse than either, and the choice is between consistency inside one file and + consistency with the code around it. `faits.jsonl` went the other way and paid for it with + a format break at 2.0 — worth it there, because that file is *read by others*; the + diagnostic is read by us. - **A commit explains why**, not what the diff already shows. The subject is a sentence, not a label. - **A test guards a decision.** The tests here do not check lines but choices: that the Maven @@ -412,12 +420,24 @@ were the ones intended. A reading of the diff proves nothing; the DOM does. out MSYS2's POSIX translation as the cause. The JVM escapes it because it opens few files and keeps its handles. + **That sentence is about a *walk*, and it must not be read as "concurrency never helps".** + An archiver, an explorer, a backup agent goes through the files one after another and + nothing overlaps. When we hold the *producer*, on the other hand, two producers do overlap + — which is what makes the parallel rendering below worth its extra pass. + + **The cost model was measured rather than assumed**, and the measurement is what settled + the rest. Injecting a fixed latency on every `open` under the output — an `LD_PRELOAD` that + changes nothing else — gives a total that tracks **one open per file written**: ~185 + delayed opens for the 178 files of a run's two sites. So the time paid is the file count + times the filter's latency, and any change is worth exactly what it does to that product. + **`JACOCO_REPORTS` decides what is written of them**, and nothing else: `full` both sites - (the default, unchanged), `detailed` the complete one alone, `data` neither. The setting - touches neither the measurement nor the display — the coverage rendered comes from - `jacoco.xml`, written in every case, and `Coverage.parse` reads it directly. On the example - application, a run goes from 186 to 101 files in `detailed`, to 9 in `data`, with the same - coverage to the figure. + (the default, unchanged), `detailed` the complete one alone, `data` neither, `minimal` + not even the campaign's merged site — its XML and CSV are still written, so the figure + survives its rendering. The setting touches neither the measurement nor the display — the + coverage rendered comes from `jacoco.xml`, written in every case, and `Coverage.parse` + reads it directly. On the example application, a run goes from 186 to 101 files in + `detailed`, to 9 in `data`, with the same coverage to the figure. Three decisions hold it, and they are the ones to know before touching it: @@ -439,9 +459,61 @@ were the ones intended. A reading of the diff proves nothing; the DOM does. copy goes through the zip filesystem, so the intermediate files never exist. Writing them in order to delete them would have cost exactly what one is trying to avoid. + **Three things are now done without being asked, because they take nothing away.** + + - **The two sites are rendered at once.** The focused report learns which classes ran from + `jacoco.xml`, so the data pass has to come first — and it used to carry the complete site + with it, which left the two renderings strictly one after the other. Split into three + passes, the data pass writes two files and the two renderings overlap. Measured **on the + rendering step**, which is the only thing this changes: the two `jacococli` calls alone + go from 1.69 s to 1.24 s at 5 ms per open (−27 %), 2.62 → 1.73 at 10 ms (−34 %), 4.44 → + 2.68 at 20 ms (−40 %) — and *0.75 against 0.76* with no filter at all, the extra + start-up being exactly paid by the overlap. On the tool itself, read off the files' + timestamps, the phase goes **4.26 s → 2.45 s at 20 ms (−42 %)**. + + **Do not quote those percentages of a whole run.** The same run goes 7.84 s → 7.22 s + (−8 %), because it also pays for the application, the components and the assembly, none + of which this touches. The gain is on the rendering and grows with the number of + classes; everything else is fixed. The parallelism is of degree two and stays there: + each rendering parses the whole class set, so this phase's peak memory doubles. + - **The focused report is not written when every analysed class ran.** It would then list + exactly what the complete one lists. The test is strict and made on the coverage alone, + before staging: a single class never entered, or a hidden package — hidden on our side, + unknown to the CLI, hence present in the complete site — puts the two out of step and the + shortcut off. + - **The count is said.** Past `Footprint.NOTABLE` files the tool prints it at the end of a + campaign and names the directory to exclude; `diagnostic.json` carries it run by run. + Counting opens each *directory* and never a file, so it costs the tree's directories and + not its files — 0.63 s for one walk at 20 ms per open, against some four seconds for a + single pass over the files that walk describes. Cheap, not free: which is why the + diagnostic **sums the runs it has already counted** instead of walking them a second + time, and why the total is taken once, at the very end. Before this, the figure was + reachable only by counting by hand on the machine where counting is itself slow, which + is to say never: the setting existed, was documented, and nothing ever pointed at it. + + **`ARCHIVE` gathers `runs/` into one `runs.zip`**, `keep` beside the tree or `replace` + instead of it. `replace` is the one that restricts — the page's links to the JaCoCo sites + stop resolving and `--report-only` has nothing left to rebuild from — so it is off by + default and says so where it is offered. Nothing is removed before the entries written are + counted against the files walked: this deletes measurements. And writing the archive does + read every file, so the gesture costs, once, exactly what it stops costing at every later + walk. + + **A cache of rendered pages across runs is not possible, and the example application hides + it.** JaCoCo's source page carries the per-line colouring: the same class, with an `exec` + and without, gives 14 lines `fc` against 14 lines `nc`. But every run of `sample-app` + covers the same lines, so its 26 source pages come out identical from one run to the next — + a cache keyed on the code would pass every test here and be wrong exactly in a campaign + whose runs differ, which is the only reason to run a campaign. What *is* run-independent is + `jacoco-resources/`: 20 files per site, 22 % of a site on the example, ~2 % on an analysis + of 447 classes. It shrinks where the problem grows. + `runs/` remains, moreover, the right candidate for an antivirus exclusion: a single directory, containing only generated artefacts, of which nothing is executed and everything - is reproducible. + is reproducible. It is the only measure that **removes** the cost rather than trimming it, + and it is the first thing the help's section on filtered machines says — that section lists + every setting that reduces the count **with what it gives up**, and `FileCostTest` holds it + to that: a value added to `JACOCO_REPORTS` and left out of the section fails the build. - **Windows terminal**: the tool writes in UTF-8. A terminal in cp850 — the default on many machines — renders the accents unreadable. Fix it on the terminal's side (mintty → Options → Text → UTF-8), or launch with `-Dstdout.encoding=cp850`. Do not "fix" this in the code: when diff --git a/bin/acceptance-local.sh b/bin/acceptance-local.sh index 3233fe2..5dddcbe 100755 --- a/bin/acceptance-local.sh +++ b/bin/acceptance-local.sh @@ -128,7 +128,7 @@ echo echo "4 bis. What is written to disk is a setting, and does not touch the measurement" pct(){ grep -o '"pct":[0-9.]*' "$1"/faits.jsonl | head -1; } count(){ find "$1"/runs -type f | wc -l; } -for v in detailed data; do +for v in detailed data minimal; do java -jar "$JAR" \ --java "java -jar $APP --iterations 200000" \ --level coverage --sources "$REPO_DIR/sample-app/src/main/java" \ @@ -145,11 +145,57 @@ done step $? "in \"data\", no HTML site is written for the run" grep -q 'toujours: true' "$REPO_DIR/orchestrator/src/main/resources/lab/xray/dashboard.html" step $? "and the page still names the absent reports, with their command" +# The whole point of the settings: they give up a RENDERING, never a figure. The count of +# files written per run is in the diagnostic, so the trade is checkable after the fact. +grep -q '"fichiersEcrits"' rep-data/diagnostic.json +step $? "the diagnostic says, run by run, what was left on disk" +grep -q '"conseil":"Exclude' rep-data/diagnostic.json +step $? "and names the directory to exclude from an antivirus scan" +echo + +# The only value that also gives up the campaign's merged rendering. It is the one that +# must not take the figure with it: an aggressive setting stays defensible only as long as +# the last possible reading survives it. +echo "4 ter. \"minimal\" gives up the merged SITE, never the merged figure" +java -jar "$JAR" \ + --java "java -jar $APP --iterations 120000" \ + --level coverage --sources "$REPO_DIR/sample-app/src/main/java" \ + --classes "$APP" --jacoco-reports minimal --name "minimal bis" --out rep-minimal \ + > rep-minimal-2.log 2>&1 +step $? "a second run under \"minimal\" ends, so there is something to merge" +[ -f rep-minimal/jacoco-fusion/html/jacoco.xml ] +step $? "the merged XML is written" +[ ! -f rep-minimal/jacoco-fusion/html/index.html ] +step $? "and its site is not" +echo + +# Gathering the run directories into one file, and — only if asked — putting it in their +# place. Nothing is removed before the archive has been counted against what it replaces. +echo "4 quater. --archive gathers the files, and \"replace\" takes their place" +java -jar "$JAR" \ + --java "java -jar $APP --iterations 120000" \ + --level coverage --sources "$REPO_DIR/sample-app/src/main/java" \ + --classes "$APP" --archive --name "archive kept" --out arch-keep > arch-keep.log 2>&1 +step $? "a measurement with --archive ends" +[ -f arch-keep/runs.zip ] && [ -d arch-keep/runs ] +step $? " the archive is written, and the tree stays: \"keep\" takes nothing away" +[ "$(unzip -l arch-keep/runs.zip | tail -1 | awk '{print $2}')" = "$(find arch-keep/runs -type f | wc -l)" ] +step $? " and it holds exactly as many entries as there are files" +java -jar "$JAR" \ + --java "java -jar $APP --iterations 120000" \ + --level coverage --sources "$REPO_DIR/sample-app/src/main/java" \ + --classes "$APP" --archive replace --name "archive replaced" --out arch-rep \ + > arch-rep.log 2>&1 +step $? "a measurement with --archive replace ends" +[ -f arch-rep/runs.zip ] && [ ! -d arch-rep/runs ] +step $? " the tree is gone, the archive is there" +[ -f arch-rep/index.html ] && [ -f arch-rep/diagnostic.json ] && [ -f arch-rep/faits.jsonl ] +step $? " and what is read without the measurements still stands" echo # A report is often reassembled without passing the launch options again. What was given to # the measurement must not get lost on the way — and above all, must not be denied. -echo "4 ter. Reassembling without --sources finds the annotated code again" +echo "4 quinquies. Reassembling without --sources finds the annotated code again" rm -rf reassemble && cp -r out reassemble java -jar "$JAR" --report-only --out reassemble > reassemble.log 2>&1 step $? "reassembling without --sources ends" diff --git a/docs/outil/mode-emploi.md b/docs/outil/mode-emploi.md index 320e55d..6b26869 100644 --- a/docs/outil/mode-emploi.md +++ b/docs/outil/mode-emploi.md @@ -185,7 +185,8 @@ the first go — **[Reducing the footprint on a large codebase](empreinte.md)** | `LEVEL` / `--level` | `full` | How far to observe: `coverage` (JaCoCo alone), `tree` (+ sampling), `full` (+ values). The first knob to turn down when the measurement costs too much | | `COVER_INCLUDES` / `--cover` | everything | Classes JaCoCo instruments, e.g. `com.example.*`. **Without it, every class loaded is instrumented**, dependencies included: it is the main cost centre | | `SAMPLE_INTERVAL_MS` / `--interval` | 1 ms | Stack sampling interval. At 10 ms, ten times fewer samples | -| `JACOCO_REPORTS` / `--jacoco-reports` | `full` | How much of JaCoCo's own rendering to write per run — see [When the number of files costs](#when-the-number-of-files-costs) | +| `JACOCO_REPORTS` / `--jacoco-reports` | `full` | How much of JaCoCo's own rendering to write per run. **Every value but the default gives a rendering up** — see [When the number of files costs](#when-the-number-of-files-costs) | +| `ARCHIVE` / `--archive` | — | Gather `runs/` into `runs.zip` once the report is built: `keep` beside the tree, `replace` **instead of it** — same section | | `FOLLOW_PORT` / `--follow` | — | Port of the follow page. Without it nothing is served — but `progression.jsonl` is written all the same | | `EXPORT` / `--export` | — | Rewrites the measurements for other tools: `perf`, `cpuprofile`, `lcov`, `values`, or `all` — see [the exports](exports.md) | @@ -257,7 +258,8 @@ runtime-xray-out/ ├── config.json ← annotations for THIS run (optional, takes priority) ├── execution.log ← the application's output ├── jacoco/html/ ← the detailed coverage, all the analysed code - ├── jacoco-focused/html/ ← the same, restricted to the classes that ran + ├── jacoco-focused/html/ ← the same, restricted to the classes that ran (absent when + │ every analysed class ran: it would repeat the other) ├── classes-executees.jar ← the bytecode kept for that second report ├── async-profiler/ ← the folded stacks, plus the profile rendered by the tool │ itself (flamegraph.html and its inverse) @@ -277,22 +279,65 @@ size of the analysed code, not with the measurement. On a machine where every fi crosses a stack of filters — antivirus, EDR, DLP — that count is what a campaign pays for: at write time, at every later walk, and when archiving it to pass it on. -`JACOCO_REPORTS` (or `--jacoco-reports`) decides how much of it is written: - -| Value | What is written | What is lost | +**The tool says the figure itself.** Past ten thousand files it prints the count at the end +of a campaign, names the directory to exclude, and points at `--help`; `diagnostic.json` +carries the count run by run, and the count of `runs/`. Before that, knowing what a campaign +had cost meant going and counting on the machine where counting is itself slow — so nobody +did, and a setting that reduces it stayed documented and unused. + +**Two things are done without being asked, because they take nothing away.** The two sites +of a run are **rendered at the same time** rather than one after the other: the cost is one +file open apiece, paid serially, so two writers overlap where one could not. Measured on the +example application, with a fixed latency injected on every file open, **on the rendering +step** — which is the only thing this changes: **−27 % at 5 ms per open, −34 % at 10 ms, +−42 % at 20 ms**, and *nothing at all* — 0.75 s against 0.76 s — with no filter, which is +what makes it free. Read as a share of a whole run the figure is much smaller, since a run +also pays for the application, the components and the assembly. And the focused report is +**not written when every analysed class ran**: it would then list exactly what the complete +one lists. + +### First, and it takes nothing away: exclude the directory + +`runs/` is the right candidate for an antivirus exclusion: a single directory, holding only +generated artefacts, of which nothing is executed and everything is reproducible. It is the +only measure that **removes** the cost. Everything below merely reduces it, and everything +below has a price. + +### Then, if that is refused — and each of these gives something up + +> **These settings restrict the report.** None of them is on by default, none of them +> changes the coverage the page displays — read from `jacoco.xml`, written in every case — +> and an absent report is never silent: the page keeps naming it, greyed out, and clicking +> gives the command that produces it. But each takes away something a reader might have +> opened. That is the trade being offered, and it is offered here rather than left to be +> discovered. + +`JACOCO_REPORTS` (or `--jacoco-reports`) decides how much of JaCoCo's own rendering is +written: + +| Value | What is written | What is given up | |---|---|---| | `full` *(default)* | both sites | nothing | | `detailed` | the complete site alone | a framing: the focused report holds **no datum** the complete one lacks | | `data` | `jacoco.xml` and `jacoco.csv` only | JaCoCo's rendering **per run**; the campaign's (`jacoco-fusion/`) stays | +| `minimal` | the same, and no merged site either | the rendering of the figure one hands on; the merged XML and CSV are still written | -This setting touches **neither the measurement nor what the page shows**: the coverage -rendered line by line comes from `jacoco.xml`, written in every case. And an absent report is -never silent — the page keeps naming it, greyed out, and clicking gives the command that -produces it. +`ARCHIVE` (or `--archive`) gathers `runs/` into one `runs.zip` once the report is built: -`runs/` is, moreover, the right candidate for an antivirus exclusion: a single directory, -holding only generated artefacts, of which nothing is executed and everything is -reproducible. +| Value | What it does | What is given up | +|---|---|---| +| `keep` *(what a bare `--archive` means)* | writes the archive, leaves the tree | nothing — and it reduces nothing either: one object to hand on, to back up, to have scanned once | +| `replace` | writes it, checks it, **then removes `runs/`** | the page's links to the JaCoCo sites stop resolving, and `--report-only` and `--serve` have nothing left to rebuild from. The page, the diagnostic, the facts and the Markdown still read | + +Nothing is removed before the archive has been verified — the entries written are counted +against the files walked, and the tree only goes if they agree. Writing the archive does read +every file: the gesture costs, once and deliberately, exactly what it stops costing at every +later walk. + +Two other settings reduce the count and are documented elsewhere because they do more than +that: `--cover` restricts what JaCoCo instruments — by far the biggest lever, and the only +one that **changes the measurement**, since what it leaves out is no longer covered at all — +and `--level coverage` gives up the call tree and the captured values. ### `rapport.md` — for a forge diff --git a/orchestrator/src/main/java/lab/xray/Config.java b/orchestrator/src/main/java/lab/xray/Config.java index 2c0d5d5..24723f2 100644 --- a/orchestrator/src/main/java/lab/xray/Config.java +++ b/orchestrator/src/main/java/lab/xray/Config.java @@ -46,6 +46,18 @@ public final class Config { public String sourceDirs = ""; public String classFilter = ""; public String outDir = "runtime-xray-out"; + /** + * What to do with {@code runs/} once the report is assembled: {@code ""} nothing, + * {@code "keep"} also write {@code runs.zip}, {@code "replace"} write it and remove the + * tree. + * + *

A campaign is a great many small files, and on a machine where a filter inspects + * every open, they are paid at every later walk — a backup, a search, the archive one + * makes to send it. Gathering them into one file pays that count once and stops paying + * it. {@code replace} is the only value that actually reduces anything, and it is the + * only one that takes something away: see {@link #ARCHIVE}. + */ + public String archive = ""; public String runName = ""; public int attachAfterSeconds = 8; public int maxSeconds = 600; @@ -163,14 +175,25 @@ public static String level(String value) { public static final String FULL = "full"; public static final String DETAILED = "detailed"; public static final String DATA = "data"; + /** + * Neither site per run, nor the campaign's merged rendering. + * + *

Apart from the other three, and it must stay apart. They give up a convenience per + * run while the campaign's figure — the one that is handed on and that makes authority — + * stays rendered; this one gives that up too. It is the last reading JaCoCo produces by + * itself, so nothing here should ever make it a default, and the documentation states + * what it costs before saying how to turn it on. The merged XML and CSV are still + * written: the figure survives, its rendering does not. + */ + public static final String MINIMAL = "minimal"; - /** The three values, in decreasing order of what is written. */ + /** The four values, in decreasing order of what is written. */ public static final java.util.List JACOCO_REPORTS = - java.util.List.of(FULL, DETAILED, DATA); + java.util.List.of(FULL, DETAILED, DATA, MINIMAL); /** Whether an HTML site is written for each run at all. */ public boolean jacocoHtmlWanted() { - return !DATA.equals(jacocoReports); + return !DATA.equals(jacocoReports) && !MINIMAL.equals(jacocoReports); } /** Whether the focused report — and the class staging it needs — is produced. */ @@ -178,6 +201,38 @@ public boolean focusedReportWanted() { return FULL.equals(jacocoReports); } + /** Whether the campaign's merged coverage gets its HTML site. */ + public boolean mergedHtmlWanted() { + return !MINIMAL.equals(jacocoReports); + } + + public static final String KEEP = "keep"; + public static final String REPLACE = "replace"; + + /** + * The accepted values of {@code ARCHIVE}, and the only restricting one is the second. + * + *

{@code keep} adds {@code runs.zip} beside the tree: one object to hand on, to + * back up, to have scanned once instead of file by file. It takes nothing away, and it + * reduces nothing either — the tree is still there. + * + *

{@code replace} removes the tree once the archive is verified, and that one + * restricts: the report's links to the JaCoCo sites stop resolving, and neither + * {@code --report-only} nor {@code --serve} can rebuild anything from this output any + * more. The page, the diagnostic, the facts and the Markdown stay where they are and + * still read. It is off by default and says what it costs where it is documented. + */ + public static final java.util.List ARCHIVE = java.util.List.of(KEEP, REPLACE); + + public boolean archiveWanted() { + return !archive.isBlank(); + } + + /** Whether {@code runs/} is removed once the archive has been checked. */ + public boolean archiveReplaces() { + return REPLACE.equals(archive); + } + public static Config load(Path file) throws IOException { Config c = new Config(); for (String raw : Files.readAllLines(file, StandardCharsets.UTF_8)) { @@ -221,6 +276,7 @@ void set(String key, String value) { case "LEVEL", "NIVEAU" -> level = value; case "COVER_INCLUDES" -> coverIncludes = value; case "JACOCO_REPORTS" -> jacocoReports = value; + case "ARCHIVE" -> archive = value; case "SAMPLE_INTERVAL_MS" -> sampleIntervalMs = parse(value, sampleIntervalMs); case "FOLLOW_PORT", "SUIVI_PORT" -> followPort = parse(value, followPort); case "TRACE_COUNT" -> traceCount = parse(value, traceCount); @@ -471,10 +527,26 @@ public static void writeTemplate(Path file) throws IOException { # full both sites (the default) # detailed the complete site alone — the focused report holds no datum it lacks # data jacoco.xml and .csv only; the campaign's merged report still stands + # minimal and not the merged site either; its XML and CSV are still written # - # An absent report stays named in the page, with the command that produces it. + # Every value but the default GIVES A RENDERING UP. None of them changes the + # coverage the page shows — it is read from jacoco.xml, written in every case — + # and an absent report stays named in the page, with the command that produces + # it. "runtime-xray --help" weighs them up one by one. #JACOCO_REPORTS="detailed" + # Gathering runs/ into a single runs.zip once the report is built, for a machine + # where every file open crosses a security filter and the count is paid again at + # each walk. + # + # keep the archive beside the tree — takes nothing away, reduces nothing + # replace the archive INSTEAD of the tree, once it has been verified + # + # "replace" gives something up: the page's links to the JaCoCo sites stop + # resolving, and --report-only and --serve have nothing left to rebuild from. + # The page, the diagnostic, the facts and the Markdown still read. + #ARCHIVE="keep" + # Classes JaCoCo instruments, in its agent's format (patterns separated by ':'). # Without this setting, EVERY class loaded is instrumented, dependencies # included: it is the main cost centre on an enterprise application. diff --git a/orchestrator/src/main/java/lab/xray/Main.java b/orchestrator/src/main/java/lab/xray/Main.java index ed0ac95..c9a4fc2 100644 --- a/orchestrator/src/main/java/lab/xray/Main.java +++ b/orchestrator/src/main/java/lab/xray/Main.java @@ -4,6 +4,7 @@ import lab.xray.report.Coverage; import lab.xray.report.Dashboard; import lab.xray.report.Exports; +import lab.xray.report.Footprint; import java.io.IOException; import java.io.InputStream; @@ -22,6 +23,7 @@ import java.util.Set; import java.util.zip.ZipEntry; import java.util.zip.ZipFile; +import java.util.zip.ZipOutputStream; import java.util.UUID; import java.util.concurrent.TimeUnit; @@ -111,6 +113,19 @@ private static int run(String[] args) throws Exception { case "--level", "--niveau" -> config.level = args[++i]; case "--cover" -> config.coverIncludes = args[++i]; case "--jacoco-reports" -> config.jacocoReports = args[++i]; + // The value attaches to the option, and "keep" is what a bare --archive + // means: it is the one that takes nothing away, so it is the one somebody + // typing the option without reading further should get. + case "--archive" -> { + config.archive = Config.KEEP; + // Anything that is not another option is taken as the value, even one + // that does not exist: it then gets the message naming the values that + // do, where leaving it in place would have made the parser complain + // about an option, over a word the reader typed as a value. + if (i + 1 < args.length && !args[i + 1].startsWith("-")) { + config.archive = args[++i]; + } + } case "--interval" -> config.sampleIntervalMs = Integer.parseInt(args[++i]); // The port attaches to the option, as for --serve: "--suivi" alone takes // the default port, and "--suivi 9100" the one given to it. @@ -212,6 +227,10 @@ private static int run(String[] args) throws Exception { require(Config.JACOCO_REPORTS.contains(config.jacocoReports), "unknown value for --jacoco-reports: " + config.jacocoReports + " (known: " + String.join(", ", Config.JACOCO_REPORTS) + ")"); + config.archive = config.archive.trim().toLowerCase(Locale.ROOT); + require(config.archive.isBlank() || Config.ARCHIVE.contains(config.archive), + "unknown value for --archive: " + config.archive + + " (known: " + String.join(", ", Config.ARCHIVE) + ")"); require(Config.LEVELS.contains(Config.level(config.level)), "--level expects coverage, tree or full (got: " + config.level + ")"); // The classes serve to MEASURE. Reassembling a view from existing measurements @@ -242,6 +261,13 @@ private static int run(String[] args) throws Exception { config.hidden(), launch(config, tools, sourceRoots(config))); sayWhatWasFound(outDir); sayTheWeight(page); + archiveRuns(config, outDir); + // Counted last, once nothing more is going to be written: it is the figure a reader + // can check with a single "find", and one that cannot be checked is one nobody + // believes twice. + for (String line : Footprint.of(outDir).console()) { + System.out.println(line); + } System.out.println(); System.out.println("Done — open: " + page); @@ -419,7 +445,11 @@ private static void mergeCoverage(Config config, Toolbox tools, Path outDir) { Path html = dir.resolve("html"); Files.createDirectories(html); exec(mergedReportCommand(cli, merged, html, samples.size(), - classes, sourceRoots(config))); + classes, sourceRoots(config), config.mergedHtmlWanted())); + if (!config.mergedHtmlWanted()) { + System.out.println(" JACOCO_REPORTS=" + config.jacocoReports + + " — merged figure written as XML and CSV, site not rendered"); + } } catch (Exception e) { // The merge is a bonus: the view can already accumulate on the page side. Its // failure must not take the report down, it must be said. @@ -447,15 +477,19 @@ private static void mergeCoverage(Config config, Toolbox tools, Path outDir) { * does not help, since the damage is done by the parent as it writes the arguments. */ static List mergedReportCommand(Path cli, Path merged, Path html, int runs, - List classes, List sources) { + List classes, List sources, + boolean site) { List report = new ArrayList<>(List.of( RunSession.javaExecutable(), "-jar", cli.toString(), "report", merged.toString(), - "--html", html.toString(), "--xml", html.resolve("jacoco.xml").toString(), "--csv", html.resolve("jacoco.csv").toString(), "--name", "Cumulative coverage over " + runs + " runs", "--quiet")); + if (site) { + report.add("--html"); + report.add(html.toString()); + } for (Path entry : classes) { report.add("--classfiles"); report.add(entry.toString()); @@ -487,6 +521,30 @@ private static List runDirectories(Path outDir) throws IOException { * Two renderings from the same measurement: the complete report, and a focused * report restricted to the classes that actually ran. On a real project the second is * often the only readable one — the first lists thousands of irrelevant classes. + * + *

Three passes rather than two, so the two sites can be written at once. The + * focused report needs to know which classes ran, and it learns that from + * {@code jacoco.xml} — so the data pass has to come first, and until now it carried the + * complete site with it, leaving the two renderings strictly one after the other. Split + * this way, the data pass writes two files, and the two HTML renderings overlap. + * + *

That was measured before being written, because the gain is entirely a matter of + * where the time goes. On a filtered machine the cost is one open per file, paid + * serially: injecting a fixed latency on every file open under the output gives, on the + * sample application and on this step alone, −27 % at 5 ms per open, −34 % at + * 10 ms and −42 % at 20 ms. With no filter at all the two shapes are indistinguishable + * — the extra JVM start-up is exactly paid by the overlap — so this costs nothing where + * it brings nothing. As a share of a whole run it is far less: the rest of a run pays + * for the application, the components and the assembly, none of which this touches. + * + *

The parallelism is of degree two and stays there: each rendering parses the whole + * class set, so the peak memory of this phase doubles. Two is what the work naturally + * offers; more would only buy memory pressure. + * + *

None of this contradicts the note in {@code CLAUDE.md} that neither memory nor + * cores buy the latency back. That one is about a walk — an archiver, an explorer + * — going through the files one after another, where nothing can be overlapped. Here we + * hold the producer, and two producers do overlap. */ private static void renderCoverage(Config config, Toolbox tools, Path runDir) throws Exception { Path exec = runDir.resolve("jacoco/jacoco.exec"); @@ -500,29 +558,24 @@ private static void renderCoverage(Config config, Toolbox tools, Path runDir) th // report read. A produced file never moves. Path html = runDir.resolve("jacoco/html"); Files.createDirectories(html); - - List cmd = new ArrayList<>(List.of( - RunSession.javaExecutable(), "-jar", cli.toString(), "report", exec.toString(), - "--xml", html.resolve("jacoco.xml").toString(), - "--csv", html.resolve("jacoco.csv").toString(), - "--name", "Runtime X-Ray", "--quiet")); - if (config.jacocoHtmlWanted()) { - cmd.add("--html"); - cmd.add(html.toString()); - } - // One --classfiles entry per directory or jar: the option is repeatable, and that - // is the tool's intended mechanism for analysing several bytecode sources. - for (Path entry : config.classesPaths()) { - cmd.add("--classfiles"); - cmd.add(entry.toString()); + List classes = config.classesPaths(); + List sources = sourceRoots(config); + + // With a single HTML rendering to produce, the data pass carries it: there would be + // nothing to overlap, and splitting would cost a start-up for nothing. + boolean twoSites = config.focusedReportWanted(); + List data = reportCommand(cli, exec, "Runtime X-Ray", classes, sources); + data.add("--xml"); + data.add(html.resolve("jacoco.xml").toString()); + data.add("--csv"); + data.add(html.resolve("jacoco.csv").toString()); + if (config.jacocoHtmlWanted() && !twoSites) { + data.add("--html"); + data.add(html.toString()); } - for (Path src : sourceRoots(config)) { - cmd.add("--sourcefiles"); - cmd.add(src.toString()); - } - exec(cmd); + exec(data); - if (!config.focusedReportWanted()) { + if (!twoSites) { System.out.println(" JACOCO_REPORTS=" + config.jacocoReports + " — coverage read from jacoco.xml, " + (config.jacocoHtmlWanted() ? "focused report not written" @@ -530,28 +583,87 @@ private static void renderCoverage(Config config, Toolbox tools, Path runDir) th return; } + Coverage coverage = Coverage.parse(html.resolve("jacoco.xml"), config.hidden()); + List> renderings = new ArrayList<>(); + List complete = reportCommand(cli, exec, "Runtime X-Ray", classes, sources); + complete.add("--html"); + complete.add(html.toString()); + renderings.add(complete); + // Focused report: only the classes with at least one covered instruction are // presented to the CLI. That is the tool's native mechanism, not a home-made // filter. - Coverage coverage = Coverage.parse(html.resolve("jacoco.xml"), config.hidden()); Path staging = runDir.resolve("classes-executees.jar"); - int kept = stageExecutedClasses(coverage, config.classesPaths(), staging); - if (kept == 0) { - return; + int kept = 0; + boolean duplicate = focusedWouldRepeatTheComplete(config, coverage); + if (!duplicate) { + kept = stageExecutedClasses(coverage, classes, staging); + if (kept > 0) { + Path focused = runDir.resolve("jacoco-focused/html"); + Files.createDirectories(focused); + List focusedCmd = reportCommand( + cli, exec, "Code actually executed", List.of(staging), sources); + focusedCmd.add("--html"); + focusedCmd.add(focused.toString()); + renderings.add(focusedCmd); + } } - Path focused = runDir.resolve("jacoco-focused/html"); - Files.createDirectories(focused); - List focusedCmd = new ArrayList<>(List.of( + execTogether(renderings); + if (duplicate) { + System.out.println(" focused report not written: every analysed class ran, so it" + + " would list exactly what the complete one lists"); + } else if (kept > 0) { + System.out.println(" " + kept + " executed classes kept for the focused report"); + } + } + + /** + * Whether the focused report would name exactly the classes of the complete one. + * + *

When it would, it is a copy: two files per class, written and then walked and then + * archived, for a framing that frames nothing. Not writing it restricts nothing — which + * is the condition for doing it without being asked. The page keeps naming it, greyed + * out, with the command that produces it. + * + *

The test is deliberately strict, and it is made on the coverage alone, before the + * classes are staged. Two things could make the two sites differ, and both disqualify + * the shortcut: a class the run never entered, and a hidden package — hidden here on + * reading, but not by the CLI, which knows nothing of it and puts it in the complete + * site. As for a class whose bytecode could not be staged, it would have made the + * focused report poorer than the complete one, never richer: nothing is lost by + * not producing it. + */ + static boolean focusedWouldRepeatTheComplete(Config config, Coverage coverage) { + if (!config.hiddenPackages.isBlank()) return false; + int analysed = 0; + for (Object value : coverage.packages.values()) { + if (!(value instanceof List classes)) continue; + for (Object o : classes) { + if (!(o instanceof Map cls)) continue; + analysed++; + if (((Number) cls.get("covered")).intValue() == 0) return false; + } + } + return analysed > 0; + } + + /** The invariable part of a {@code jacococli report} call: what it reads, and its name. */ + static List reportCommand(Path cli, Path exec, String name, + List classes, List sources) { + List cmd = new ArrayList<>(List.of( RunSession.javaExecutable(), "-jar", cli.toString(), "report", exec.toString(), - "--classfiles", staging.toString(), - "--html", focused.toString(), - "--name", "Code actually executed", "--quiet")); - for (Path src : sourceRoots(config)) { - focusedCmd.add("--sourcefiles"); - focusedCmd.add(src.toString()); + "--name", name, "--quiet")); + // One --classfiles entry per directory or jar: the option is repeatable, and that + // is the tool's intended mechanism for analysing several bytecode sources. + for (Path entry : classes) { + cmd.add("--classfiles"); + cmd.add(entry.toString()); + } + for (Path src : sources) { + cmd.add("--sourcefiles"); + cmd.add(src.toString()); } - exec(focusedCmd); - System.out.println(" " + kept + " executed classes kept for the focused report"); + return cmd; } /** @@ -979,6 +1091,103 @@ private static void exec(List cmd) throws IOException, InterruptedExcept } } + /** + * Gathers {@code runs/} into a single archive, and — only if asked — removes the tree. + * + *

A campaign is a great many small files, and each of them is paid again at every + * later walk: a backup, a search, the zip somebody makes to send the report on. One + * archive pays that count once. Writing it does read every file, so the gesture + * costs exactly what it is meant to stop costing — once, deliberately, instead of at + * each pass. + * + *

Nothing is removed before the archive has been checked. The count of entries + * written is compared with the count of files walked, and the tree only goes if they + * agree. This deletes measurements: a zip that is short by a file, discovered a week + * later, is a campaign to run again. + * + *

A failure here never takes the report down — it is a convenience applied after the + * fact, and the report was the point. But it is said, and it stops: a warning followed + * by a deletion would be the worst of both. + */ + private static void archiveRuns(Config config, Path outDir) { + if (!config.archiveWanted()) return; + Path runs = outDir.resolve("runs"); + if (!Files.isDirectory(runs)) return; + Path zip = outDir.resolve("runs.zip"); + try { + long files = Footprint.count(runs); + System.out.println("▶ Archiving " + Footprint.grouped(files) + " files into " + + zip.getFileName()); + long entries = zipTree(runs, zip); + require(entries == files, "the archive holds " + entries + " entries for " + + files + " files: nothing removed"); + if (config.archiveReplaces()) { + deleteRecursively(runs); + System.out.println(" runs/ removed — the page and the diagnostic still" + + " read; the links to the JaCoCo sites no longer resolve, and" + + " --report-only has nothing left to rebuild from"); + } + } catch (Exception e) { + System.out.println(" ⚠️ archive not produced: " + e.getMessage()); + } + } + + /** Writes every regular file under {@code dir} into {@code zip}; returns the count. */ + private static long zipTree(Path dir, Path zip) throws IOException { + long entries = 0; + try (ZipOutputStream out = new ZipOutputStream(Files.newOutputStream(zip))) { + try (var walk = Files.walk(dir)) { + for (Path file : walk.filter(Files::isRegularFile).toList()) { + // A zip entry is separated by '/' on every system: a Windows path put in + // as it comes gives one entry whose name contains backslashes, which no + // reader splits back into directories. + String name = dir.relativize(file).toString().replace('\\', '/'); + out.putNextEntry(new ZipEntry(name)); + Files.copy(file, out); + out.closeEntry(); + entries++; + } + } + } + return entries; + } + + /** + * Runs several commands at once, and waits for every one of them. + * + *

Waiting for all before reporting anything is the point: a command left running + * behind a thrown exception writes into a directory the caller believes finished. The + * first failure is the one raised, the others travel with it as suppressed exceptions — + * two renderings that both fail usually fail for the same reason, and hiding the second + * would make that look like a coincidence. + */ + private static void execTogether(List> commands) + throws IOException, InterruptedException { + if (commands.size() == 1) { + exec(commands.get(0)); + return; + } + List started = new ArrayList<>(); + for (List cmd : commands) { + started.add(new ProcessBuilder(cmd).inheritIO().start()); + } + IOException failure = null; + for (int i = 0; i < started.size(); i++) { + Process p = started.get(i); + String what = String.join(" ", commands.get(i)); + IOException problem = null; + if (!p.waitFor(10, TimeUnit.MINUTES)) { + p.destroy(); + problem = new IOException("commande interrompue : " + what); + } else if (p.exitValue() != 0) { + problem = new IOException("failed (exit " + p.exitValue() + ") : " + what); + } + if (problem == null) continue; + if (failure == null) failure = problem; else failure.addSuppressed(problem); + } + if (failure != null) throw failure; + } + private static void deleteRecursively(Path dir) throws IOException { try (var walk = Files.walk(dir)) { walk.sorted(java.util.Comparator.reverseOrder()).forEach(p -> { @@ -1033,9 +1242,16 @@ private static void usage() { Without it every class the JVM loads is instrumented. --jacoco-reports How much of JaCoCo's own rendering to write per run: full (both sites, the default), detailed (the complete - site alone), data (jacoco.xml and .csv only). The page + site alone), data (jacoco.xml and .csv only), minimal + (and not the campaign's merged site either). The page reads the XML, so it shows the same coverage either way; this only decides how many FILES land on disk. + Every value but the default takes a rendering away — + see LIVING WITH A SECURITY FILTER. + --archive [v] Gather runs/ into runs.zip once the report is built: + keep (the default of this option) leaves the tree in + place, replace removes it after checking the archive. + "replace" takes something away — see the same section. --interval Stack sampling interval (default: 1). --attach-after Delay before inspecting values (default: 8). --max-seconds Guard rail on the run duration (default: 600). @@ -1098,9 +1314,51 @@ private static void usage() { --print-options Run nothing: print the JVM options to add to any command line, then assemble with --report-only. + LIVING WITH A SECURITY FILTER (antivirus, EDR, DLP) + A report is a great many small FILES, not many bytes: JaCoCo writes two per + class, and the tool asks it for two sites per run. Where every file open + crosses a filter, the time paid is that count times the filter's latency — + measured, one open per file — and it is paid again at every later walk: + a backup, a search, the archive made to send the report on. The latency is + serial, so neither the memory nor the cores of the machine buy a walk back. + + FIRST, AND IT TAKES NOTHING AWAY + Exclude /runs from the scan. It holds only generated artefacts, + nothing in it is executed, and everything in it is reproducible. This + is the only measure that REMOVES the cost. Everything below reduces it, + and every line below has a price written next to it. + + IF THAT IS REFUSED — each of these gives something up + --cover "com.example.*" By far the fewest files, and the only one here + that touches the MEASUREMENT: what it leaves + out is not covered at all any more. + --jacoco-reports detailed Drops the focused site. Gives up a framing + ("only the code that ran") — no datum: the + complete site holds everything it held. + --jacoco-reports data Drops both sites of each run. Gives up JaCoCo's + own rendering per run; the campaign's merged + one still stands. + --jacoco-reports minimal ... and the merged site as well. Gives up the + rendering of the figure one hands on; the + merged XML and CSV are still written. + --archive replace Gathers runs/ into runs.zip, then removes the + tree. The page, the diagnostic and the facts + still read; the links to the JaCoCo sites stop + resolving, and --report-only and --serve have + nothing left to rebuild from. + --level coverage Measures less, and that is the point: no call + tree and no captured values. + + WHAT NONE OF THEM CHANGES + The coverage this page shows. It is read from jacoco.xml, written in + every case. A report not produced stays named in the page, greyed out, + and the click gives the command that produces it. And the tool says the + figure by itself past files, so nobody has to go and count. + EXIT STATUS 0 success, or --help 2 bad usage: unknown option, unknown fact family - """); + """.replace("", lab.xray.report.Footprint.grouped( + lab.xray.report.Footprint.NOTABLE))); } } diff --git a/orchestrator/src/main/java/lab/xray/report/Diagnostic.java b/orchestrator/src/main/java/lab/xray/report/Diagnostic.java index 6fa75f2..09a9f95 100644 --- a/orchestrator/src/main/java/lab/xray/report/Diagnostic.java +++ b/orchestrator/src/main/java/lab/xray/report/Diagnostic.java @@ -70,7 +70,9 @@ public static Map write(Path commonDir, List runs, d.put("machine", environment()); if (context != null && !context.isEmpty()) d.put("lancement", context); d.put("sortie", commonDir.toAbsolutePath().normalize().toString()); - d.put("executions", runs(runs)); + List runViews = runs(commonDir, runs); + d.put("executions", runViews); + d.put("empreinte", footprint(commonDir, runViews)); d.put("sources", index.diagnostic()); List bytecode = bytecode(context); d.put("bytecode", bytecode); @@ -331,7 +333,7 @@ private static String conclusion(int expected, int missing, Sources.Index index) return missing + " measured class(es) out of " + expected + " have no source."; } - private static List runs(List runs) { + private static List runs(Path commonDir, List runs) { List out = new ArrayList<>(); for (Object r : runs) { if (!(r instanceof Map run)) continue; @@ -346,11 +348,57 @@ private static List runs(List runs) { e.put("classesMesurees", count(run.get("packages"))); e.put("fichiersCouverts", size(run.get("coverage"))); e.put("methodesInspectees", size(run.get("values"))); + e.put("fichiersEcrits", writtenFiles(commonDir, run.get("chemin"))); out.add(e); } return out; } + /** + * What this run left on disk, in number of files. + * + *

Placed with each run rather than as a single total, and that is deliberate: a run's + * directory is finished once its reports are written, so this figure is stable and a + * reader can reproduce it with a single {@code find}. A total over the whole output + * would be taken here, while the page, the facts and the diagram are still to be + * written — it would be wrong by a handful the moment it was read, and a figure one + * cannot reproduce is a figure one stops believing. + */ + private static long writtenFiles(Path commonDir, Object relative) { + if (!(relative instanceof String path) || path.isBlank()) return 0; + try { + return Footprint.count(commonDir.resolve(path)); + } catch (IOException e) { + return 0; + } + } + + /** + * The file count of {@code runs/}, and what to do about it. + * + *

{@code runs/} and not the whole output: it is the directory the advice names, so it + * is the one whose figure must be checkable against it. + * + *

Summed from the runs rather than walked again. Walking costs a directory open + * apiece, and on the machine this figure is written for, a directory open is not free + * either — measured at 0.63 s for one walk under a 20 ms filter. Two walks of the same + * tree to print the same number would have been the small, self-inflicted version of + * exactly what is being reported. + */ + private static Map footprint(Path commonDir, List runViews) { + Map m = new LinkedHashMap<>(); + long total = 0; + for (Object r : runViews) { + if (r instanceof Map run && run.get("fichiersEcrits") instanceof Number n) { + total += n.longValue(); + } + } + m.put("fichiersDansRuns", total); + m.put("seuilAlerte", Footprint.NOTABLE); + m.put("conseil", Footprint.exclusionAdvice(commonDir)); + return m; + } + private static int count(Object packages) { int n = 0; if (packages instanceof Map m) { diff --git a/orchestrator/src/main/java/lab/xray/report/Footprint.java b/orchestrator/src/main/java/lab/xray/report/Footprint.java new file mode 100644 index 0000000..fcee8cd --- /dev/null +++ b/orchestrator/src/main/java/lab/xray/report/Footprint.java @@ -0,0 +1,138 @@ +package lab.xray.report; + +import java.io.IOException; +import java.nio.file.FileVisitResult; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.SimpleFileVisitor; +import java.nio.file.attribute.BasicFileAttributes; +import java.util.ArrayList; +import java.util.List; +import java.util.Locale; + +/** + * What a report weighs in number of files. + * + *

That is the figure which decides how long a campaign takes on a machine where every + * file open crosses a stack of security filters — antivirus, EDR, DLP. The cost model was + * measured, not assumed: injecting a fixed latency on every {@code open} under the output + * directory gives a total that tracks one open per file written. So the time paid is + * the file count multiplied by the filter's latency, and nothing else — not the volume, not + * the number of runs by itself. + * + *

The count therefore had to stop being invisible. Before this, the only way to know + * what a campaign had cost was to go and count by hand, after the fact, on the machine + * where it hurt — which is to say: never. A setting that reduces the count existed and was + * documented, and nothing in the tool's output ever suggested reaching for it. + * + *

Counting is far cheaper than reading, and it is not free. A walk opens each + * directory and reads the entries' attributes; it never opens a file. The cost is + * therefore proportional to the tree's directories and not to its files — measured at + * 0.63 s for one walk of a run's output under a 20 ms filter, against some four seconds for + * a single pass over the files it holds. That ratio is what makes the figure affordable at + * every assembly; it is also why the count is taken once at the end and summed from the + * runs in the diagnostic, rather than walked twice for the same number. + */ +public final class Footprint { + + /** + * Above this many files, the tool says so on its own. + * + *

Ten thousand is a convention, not a measurement: at the latency of a filtered + * machine — of the order of a millisecond per open — it is where a single walk of the + * output starts to be counted in tens of seconds, hence where the settings begin to be + * worth knowing about. Set lower, the message would fire on runs nobody waits for; set + * higher, it would arrive after the campaign that hurt. + */ + public static final long NOTABLE = 10_000; + + /** Where the advice sends the reader, and what it names. */ + public static final String EXCLUDE = "runs"; + + public final Path out; + public final long files; + + private Footprint(Path out, long files) { + this.out = out; + this.files = files; + } + + public static Footprint of(Path out) throws IOException { + return new Footprint(out, count(out)); + } + + /** Regular files under {@code dir}, or 0 when it does not exist. */ + public static long count(Path dir) throws IOException { + if (!Files.isDirectory(dir)) { + return Files.isRegularFile(dir) ? 1 : 0; + } + long[] n = {0}; + Files.walkFileTree(dir, new SimpleFileVisitor() { + @Override + public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) { + if (attrs.isRegularFile()) n[0]++; + return FileVisitResult.CONTINUE; + } + + @Override + public FileVisitResult visitFileFailed(Path file, IOException e) { + // A file that cannot be read still exists, and still gets scanned. + n[0]++; + return FileVisitResult.CONTINUE; + } + }); + return n[0]; + } + + public boolean notable() { + return files >= NOTABLE; + } + + /** + * What the console says once the report is assembled — empty below the threshold. + * + *

It says the figure, then the one measure that removes the cost instead of reducing + * it, then where to read the rest. It does not list the settings: they restrict + * what the report holds, and a line printed at the end of a campaign is the wrong place + * to weigh that up. The help section is written for it. + */ + public List console() { + return message(out, files); + } + + /** + * Built apart from any directory so a test can read it. + * + *

Checking the message otherwise would mean creating ten thousand files — that is, + * paying in the test exactly the cost the message exists to report. + */ + static List message(Path out, long files) { + List lines = new ArrayList<>(); + if (files < NOTABLE) return lines; + lines.add(" 📁 " + grouped(files) + " files written under " + + out.toAbsolutePath().normalize()); + lines.add(" On a machine where a security filter inspects every file open, that"); + lines.add(" count is what a campaign pays — at every walk, and when archiving it."); + lines.add(" " + exclusionAdvice(out)); + lines.add(" \"runtime-xray --help\" explains what can be reduced, and at what price."); + return lines; + } + + /** + * The sentence that names the directory to exclude, and says why it is safe to. + * + *

It lives here, in one place, because it is said in three: the console at the end of + * a campaign, {@code diagnostic.json}, and the help. Three copies would have drifted, + * and the one a reader gets is decided by where they happened to look. + */ + public static String exclusionAdvice(Path out) { + return "Exclude " + out.toAbsolutePath().normalize().resolve(EXCLUDE) + + " from the scan: it holds only generated artefacts, nothing there is" + + " executed, and everything in it is reproducible."; + } + + /** English grouping, and the tool's locale rather than the machine's. */ + public static String grouped(long n) { + return String.format(Locale.ROOT, "%,d", n); + } +} diff --git a/orchestrator/src/main/resources/lab/xray/dashboard.html b/orchestrator/src/main/resources/lab/xray/dashboard.html index c1d9ab1..1adc29c 100644 --- a/orchestrator/src/main/resources/lab/xray/dashboard.html +++ b/orchestrator/src/main/resources/lab/xray/dashboard.html @@ -1,5 +1,5 @@ - + @@ -1747,7 +1747,9 @@

Runtime X-Ray

cmd: "JaCoCo's HTML sites hold two files per class, and the tool produces two of them" + " per run: on a large application, that is the bulk of a campaign's file count." + " JACOCO_REPORTS decides what is written of them — full both, detailed the" - + " complete one alone, data neither.\n\nTo get them back:\n\n" + + " complete one alone, data neither, minimal not even the merged site.\n\n" + + "The focused report is also left out when every analysed class ran: it would then" + + " list exactly what the complete one lists.\n\nTo get them back:\n\n" + " java -jar runtime-xray.jar --config --jacoco-reports full\n\n" + "The coverage displayed in this page, for its part, does not depend on this" + " setting: it is read from jacoco.xml, which is written in every case.", diff --git a/orchestrator/src/test/java/lab/xray/FileCostTest.java b/orchestrator/src/test/java/lab/xray/FileCostTest.java new file mode 100644 index 0000000..16f921c --- /dev/null +++ b/orchestrator/src/test/java/lab/xray/FileCostTest.java @@ -0,0 +1,156 @@ +package lab.xray; + +import lab.xray.report.Coverage; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; + +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.regex.Matcher; +import java.util.regex.Pattern; + +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertTrue; + +/** + * What the tool refuses to write, and what it says about the rest. + * + *

Two decisions meet here, and they pull in opposite directions. One says that a report + * nobody asked to lose must never go missing — so the only thing the tool drops on its own + * is an exact copy. The other says that a machine where a filter inspects every file open + * has to be given a way out, and that a way out which takes something away must say so + * where it is offered, not in a paragraph somebody may never reach. + */ +class FileCostTest { + + /** Two classes, both entered: the focused site would list exactly the complete one's. */ + private static final String ALL_RAN = """ + + + + + + + + + + + + """; + + /** The same, with one class the run never entered. */ + private static final String ONE_NEVER_RAN = ALL_RAN.replace( + "", + ""); + + private static Coverage coverage(Path dir, String xml) throws Exception { + Path file = dir.resolve("jacoco.xml"); + Files.writeString(file, xml, StandardCharsets.UTF_8); + return Coverage.parse(file); + } + + private static String main() throws IOException { + return Files.readString(Path.of("").toAbsolutePath().getParent() + .resolve("orchestrator/src/main/java/lab/xray/Main.java").normalize(), + StandardCharsets.UTF_8); + } + + @Test + @DisplayName("When every analysed class ran, the focused report is a copy") + void everyClassRanMakesItACopy(@TempDir Path dir) throws Exception { + // Two files per class, written then walked then archived, to frame nothing. This is + // the only case where the tool drops a report without being asked, and it is the + // only case where dropping it takes nothing away. + assertTrue(Main.focusedWouldRepeatTheComplete(new Config(), coverage(dir, ALL_RAN))); + } + + @Test + @DisplayName("One class that never ran, and the focused report has something to say") + void oneClassThatNeverRanIsEnough(@TempDir Path dir) throws Exception { + // The threshold is one class, deliberately. A "nearly identical" report is exactly + // the one a reader opens to find what is missing from it. + assertFalse(Main.focusedWouldRepeatTheComplete( + new Config(), coverage(dir, ONE_NEVER_RAN))); + } + + @Test + @DisplayName("A hidden package puts the two reports back out of step") + void hidingAPackageDisablesTheShortcut(@TempDir Path dir) throws Exception { + // Hiding happens on OUR side, on reading. The CLI knows nothing of it and puts the + // hidden classes in the complete site — so the two sites differ, and the coverage + // read here can no longer see it: everything it holds ran. + Config config = new Config(); + config.hiddenPackages = "org.slf4j"; + assertFalse(Main.focusedWouldRepeatTheComplete(config, coverage(dir, ALL_RAN))); + } + + @Test + @DisplayName("A measurement with no class at all is not a copy of anything") + void anEmptyCoverageIsNotADuplicate(@TempDir Path dir) throws Exception { + // "Every class ran" is vacuously true of no class. Answering yes would have made an + // empty measurement — the failure this tool exists to explain — silently produce + // one report fewer. + assertFalse(Main.focusedWouldRepeatTheComplete(new Config(), + coverage(dir, ""))); + } + + @Test + @DisplayName("The help has a section for the filtered machine, and it names the exclusion") + void theHelpNamesTheExclusion() throws IOException { + String help = section(); + assertTrue(help.contains("runs"), + "the directory to exclude is the first thing to say: it is the only measure " + + "that removes the cost instead of trimming it"); + assertTrue(help.contains("reproducible"), + "and why it is safe to exclude, because the reader has to convince somebody"); + } + + @Test + @DisplayName("Every setting that takes something away says so where it is offered") + void everyRestrictingSettingCarriesItsPrice() throws IOException { + String help = section(); + // The rule this guards: nothing restricts by default, and what restricts is written + // down beside the thing it restricts. A value added later and left out of here is a + // capability nobody weighs before turning it on. + for (String value : Config.JACOCO_REPORTS) { + if (Config.FULL.equals(value)) continue; + assertTrue(help.contains("--jacoco-reports " + value), + "\"" + value + "\" reduces what is written and is not in the section " + + "that lists what each value costs"); + } + assertTrue(help.contains("--archive " + Config.REPLACE), + "the value that removes the measurements must be named with its price"); + assertFalse(help.contains("--archive " + Config.KEEP + " "), + "\"keep\" takes nothing away: listing it among the trade-offs would suggest " + + "it does"); + } + + @Test + @DisplayName("No option quoted in that section is an option that does not exist") + void theSectionQuotesOnlyRealOptions() throws IOException { + String main = main(); + int start = main.indexOf("switch (a) {"); + int end = main.indexOf("unknown option", start); + String options = main.substring(start, end); + Matcher m = Pattern.compile("(--[a-z-]{3,})").matcher(section()); + while (m.find()) { + String option = m.group(1); + assertTrue(options.contains('"' + option + '"'), + "the help sends the reader to " + option + ", which Main does not accept " + + "— on the machine where this section gets read, there is nothing to " + + "check it against"); + } + } + + /** The help's section on filtered machines, cut out of the source rather than run. */ + private static String section() throws IOException { + String main = main(); + int start = main.indexOf("LIVING WITH A SECURITY FILTER"); + int end = main.indexOf("EXIT STATUS", start); + assertTrue(start > 0 && end > start, "the section must stay locatable in the help"); + return main.substring(start, end); + } +} diff --git a/orchestrator/src/test/java/lab/xray/MergedCoverageTest.java b/orchestrator/src/test/java/lab/xray/MergedCoverageTest.java index 344e5c2..211340f 100644 --- a/orchestrator/src/test/java/lab/xray/MergedCoverageTest.java +++ b/orchestrator/src/test/java/lab/xray/MergedCoverageTest.java @@ -33,7 +33,12 @@ class MergedCoverageTest { private static final Path SOURCES = Path.of("/app/src"); private static List command() { - return Main.mergedReportCommand(CLI, MERGED, HTML, 3, List.of(CLASSES), List.of(SOURCES)); + return command(true); + } + + private static List command(boolean site) { + return Main.mergedReportCommand(CLI, MERGED, HTML, 3, + List.of(CLASSES), List.of(SOURCES), site); } @Test @@ -90,4 +95,19 @@ void theBytecodeAndTheSourcesArePassed() { assertTrue(command.contains("Cumulative coverage over 3 runs"), "the report says how many runs it unites: " + command); } + + @Test + @DisplayName("Without its site, the merged report still writes the figure") + void withoutItsSiteTheFigureSurvives() { + // JACOCO_REPORTS=minimal gives up the campaign's RENDERING, never its figure: the + // XML and the CSV are what one goes back to, and what another tool reads. Dropping + // them with the site would have made the setting a way of measuring less, which is + // exactly what every value of this setting refuses to be. + List command = command(false); + assertFalse(command.contains("--html"), "no site was asked for: " + command); + assertTrue(command.contains("--xml"), "the merged XML is written all the same"); + assertTrue(command.contains("--csv"), "and so is the CSV"); + assertFalse(command.contains(HTML.toString()), + "the site's directory has no business in the command: " + command); + } } diff --git a/orchestrator/src/test/java/lab/xray/report/CharacterisationTest.java b/orchestrator/src/test/java/lab/xray/report/CharacterisationTest.java index a55eb8c..4819be5 100644 --- a/orchestrator/src/test/java/lab/xray/report/CharacterisationTest.java +++ b/orchestrator/src/test/java/lab/xray/report/CharacterisationTest.java @@ -94,7 +94,7 @@ class CharacterisationTest { /** The options the command line accepts, today. */ private static final List EXPECTED_OPTIONS = List.of( - "--attach-after", "--classes", "--components", "--composants", "--config", + "--archive", "--attach-after", "--classes", "--components", "--composants", "--config", "--context", "--contexte", "--cover", "--export", "--families", "--familles", "--filter", "--follow", "--help", "--hide", "--interval", "--jacoco-reports", "--java", "--level", diff --git a/orchestrator/src/test/java/lab/xray/report/FootprintTest.java b/orchestrator/src/test/java/lab/xray/report/FootprintTest.java new file mode 100644 index 0000000..d54ad16 --- /dev/null +++ b/orchestrator/src/test/java/lab/xray/report/FootprintTest.java @@ -0,0 +1,122 @@ +package lab.xray.report; + +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; + +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertTrue; + +/** + * What a report costs in number of files, and the moment the tool says so. + * + *

The figure existed before, in the sense that anybody could go and count. Nobody did: + * the machines where the count hurts are the ones where running a {@code find} over the + * output is itself slow. A setting that reduces it was documented, and nothing in the run + * ever pointed at it. These tests guard the three decisions that fix that — count without + * paying, say it only when it matters, and always name the one measure that costs nothing. + */ +class FootprintTest { + + private static void files(Path dir, int n) throws IOException { + Files.createDirectories(dir); + for (int i = 0; i < n; i++) { + Files.writeString(dir.resolve("f" + i + ".html"), "x", StandardCharsets.UTF_8); + } + } + + @Test + @DisplayName("Files are counted, and the directories holding them are not") + void countsFilesAndNotDirectories(@TempDir Path dir) throws IOException { + files(dir.resolve("runs/one/jacoco/html"), 3); + files(dir.resolve("runs/two/jacoco/html"), 4); + // Seven files under five directories: a count that included the directories would + // read as twelve, and the figure is meant to be checkable with "find -type f". + assertEquals(7, Footprint.count(dir)); + assertEquals(3, Footprint.count(dir.resolve("runs/one"))); + } + + @Test + @DisplayName("A directory that does not exist counts zero, it is not an error") + void missingDirectoryCountsZero(@TempDir Path dir) throws IOException { + // A run stopped before writing anything, or an output already archived away: the + // figure is 0, and the campaign carries on. Counting is a comfort. + assertEquals(0, Footprint.count(dir.resolve("never-written"))); + } + + @Test + @DisplayName("Below the threshold the tool says nothing") + void staysQuietBelowTheThreshold(@TempDir Path dir) throws IOException { + files(dir.resolve("runs/one"), 5); + Footprint small = Footprint.of(dir); + assertFalse(small.notable()); + assertTrue(small.console().isEmpty(), + "five files is every ordinary run: a warning there would teach nothing and " + + "would be ignored the day it matters"); + } + + @Test + @DisplayName("Past the threshold it gives the figure and names the directory to exclude") + void pastTheThresholdItNamesTheExclusion(@TempDir Path dir) { + // Built rather than written: creating ten thousand real files to check a message + // would make the test pay exactly the cost the message is about. + String said = String.join("\n", Footprint.message(dir, 186_400)); + assertTrue(said.contains("186,400"), "the figure itself, grouped: " + said); + assertTrue(said.contains(Footprint.EXCLUDE), + "and the directory to exclude — a warning with no way out is a complaint: " + + said); + assertTrue(said.contains("--help"), + "the rest is a matter of trade-offs, and belongs where they are written out"); + // Exactly at the threshold it speaks: a boundary read as "strictly above" would + // make the round number in the documentation a lie by one file. + assertFalse(Footprint.message(dir, Footprint.NOTABLE).isEmpty()); + assertTrue(Footprint.message(dir, Footprint.NOTABLE - 1).isEmpty()); + } + + @Test + @DisplayName("The advice names a path, and says why removing it from the scan is safe") + void theAdviceSaysWhyItIsSafe(@TempDir Path dir) throws IOException { + String advice = Footprint.exclusionAdvice(dir); + assertTrue(advice.contains(dir.toAbsolutePath().normalize() + .resolve(Footprint.EXCLUDE).toString()), + "an exclusion advice that does not give the path to exclude is a slogan: " + + advice); + // Whoever reads this has to convince somebody else, and "trust me" does not carry. + // The three properties are the argument: generated, not executed, reproducible. + assertTrue(advice.contains("generated"), advice); + assertTrue(advice.contains("executed"), advice); + assertTrue(advice.contains("reproducible"), advice); + } + + @Test + @DisplayName("The figure is grouped, and in the tool's locale rather than the machine's") + void theFigureIsGroupedInTheToolsLocale() { + // The page and the console read as English wherever they run — a machine set to a + // French locale must not render 186 400 with a space and look like two numbers. + assertEquals("186,400", Footprint.grouped(186_400)); + assertEquals("9", Footprint.grouped(9)); + } + + @Test + @DisplayName("The per-run count travels in the diagnostic, run by run") + void theDiagnosticCarriesThePerRunCount(@TempDir Path dir) throws Exception { + files(dir.resolve("runs/one/jacoco/html"), 6); + Sources.Index index = Sources.load(List.of()); + Diagnostic.write(dir, List.of(java.util.Map.of( + "nom", "one", "uuid", "u-1", "chemin", "runs/one/")), index, null); + String written = Files.readString(dir.resolve("diagnostic.json"), StandardCharsets.UTF_8); + assertTrue(written.contains("\"fichiersEcrits\":6"), + "each run says what it left on disk: " + written); + assertTrue(written.contains("\"fichiersDansRuns\":6"), + "and the directory the advice names carries its own figure: " + written); + assertTrue(written.contains("\"seuilAlerte\":" + Footprint.NOTABLE), + "with the threshold, so a reader knows why nothing was said: " + written); + } +} diff --git a/orchestrator/src/test/java/lab/xray/report/ViewLanguageTest.java b/orchestrator/src/test/java/lab/xray/report/ViewLanguageTest.java index b6aaf6c..674a6e0 100644 --- a/orchestrator/src/test/java/lab/xray/report/ViewLanguageTest.java +++ b/orchestrator/src/test/java/lab/xray/report/ViewLanguageTest.java @@ -197,4 +197,19 @@ void theSelectorIsTwoLettersAndThePageFetchesNothing() { assertFalse(PAGE.contains("🇫🇷") || PAGE.contains("🇬🇧"), "no flag: a flag names a country, not a language"); } + + @Test + @DisplayName("The template declares the language it is written in") + void theTemplateDeclaresItsOwnLanguage() throws Exception { + // The file carries English since the template was inverted; its went on + // saying "fr". The script puts that right on load, so nothing showed — but the page + // as it sits on disk announced French over English text, to a screen reader, to a + // translation prompt, and on the first paint. The whole point of the inversion is + // that the language the tool is read in is the one the file carries: the attribute + // is part of the file. + String template = read(); + assertTrue(template.contains(""), + "the template is written in English and must say so"); + assertFalse(template.contains(""), template.substring(0, 120)); + } } diff --git a/skills/runtime-xray/SKILL.md b/skills/runtime-xray/SKILL.md index a528165..6fb06fd 100644 --- a/skills/runtime-xray/SKILL.md +++ b/skills/runtime-xray/SKILL.md @@ -91,19 +91,34 @@ per run**. A campaign's file count therefore grows with the size of the code, no measurement — and on a machine where every file open crosses an antivirus, an EDR, a DLP, that is often the real brake. +**Start with the exclusion, it takes nothing away.** The `runs/` directory is the right +candidate for an antivirus exclusion: nothing in it is executed, everything in it is +generated and reproducible. It is the only measure that *removes* the cost. Everything below +reduces it, and each line below gives something up — the tool prints the file count itself +past ten thousand, and `runtime-xray --help` weighs the settings one by one. + ```sh --jacoco-reports detailed # the complete site alone, without the focused one --jacoco-reports data # jacoco.xml and .csv only, no site per run +--jacoco-reports minimal # and no merged site either — its XML and CSV are still written +--archive # gather runs/ into runs.zip, and keep the tree +--archive replace # gather it, then put the archive IN PLACE of the tree ``` -This setting changes **neither the measurement nor what the page shows**: the coverage comes +These settings change **neither the measurement nor what the page shows**: the coverage comes from `jacoco.xml`, written in every case. `detailed` loses no data — the focused report holds none the complete one lacks. `data` loses JaCoCo's rendering per run, but keeps the -whole campaign's. An absent report stays named in the page, with the command that produces -it. +whole campaign's; `minimal` loses that one too, keeping its figure. An absent report stays +named in the page, with the command that produces it. + +`--archive replace` is the one to weigh: it removes the measurements once the archive has +been checked against them, so the page's links to the JaCoCo sites stop resolving and +`--report-only` has nothing left to rebuild from. The page, the diagnostic, the facts and +the Markdown still read. -The `runs/` directory is, moreover, the right candidate for an antivirus exclusion: nothing -in it is executed, everything in it is generated and reproducible. +Two things happen on their own and need no setting: the two sites of a run are rendered at +the same time, and the focused report is not written when every analysed class ran — it +would repeat the complete one. ## Observing a process one does not launch oneself