From 2899210e2d9e842e66e9e782b6a2f691a2a3cf94 Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Fri, 9 Oct 2026 03:34:49 +0000 Subject: [PATCH 1/4] docs(perf): measure 0.10.0 against 0.9.2 Co-Authored-By: jason.han --- .../unreleased/perf-0.10.0-record.added.md | 1 + docs/internals/performance.md | 2 +- docs/project/README.md | 7 +- .../performance-release-0.10.0-vs-0.9.2.md | 736 ++++++++++++++++++ mkdocs.yml | 1 + 5 files changed, 743 insertions(+), 4 deletions(-) create mode 100644 changes/unreleased/perf-0.10.0-record.added.md create mode 100644 docs/project/performance-release-0.10.0-vs-0.9.2.md diff --git a/changes/unreleased/perf-0.10.0-record.added.md b/changes/unreleased/perf-0.10.0-record.added.md new file mode 100644 index 0000000000..e8821dceec --- /dev/null +++ b/changes/unreleased/perf-0.10.0-record.added.md @@ -0,0 +1 @@ +- **The release-gate performance record of 0.10.0 against 0.9.2.** `docs/project/performance-release-0.10.0-vs-0.9.2.md` measures `main` at the 0.10.0 release against the `v0.9.2` tag — every benchmark on both revisions under `benchstat`, the rows in doubt re-run interleaved, the stress constellation, and whole-binary validation, start-up and size — and names the change responsible for each confirmed regression. Nothing in it blocks the release; loading is 16–20% slower per element on connector-dense models, executing a step 20–70%, checking a constraint 16–68%, editing a workspace document allocates 2–4 times the bytes, and the import-ambiguity warning fires 134 times on the Annex A vehicle example, each with a follow-up named. diff --git a/docs/internals/performance.md b/docs/internals/performance.md index c63b5cfaf2..863404a81f 100644 --- a/docs/internals/performance.md +++ b/docs/internals/performance.md @@ -5,7 +5,7 @@ say about where the remaining cost is. Figures below were taken on an `Intel Xeon Platinum 8559C`, Go 1.25, `GOMAXPROCS=8`; treat them as ratios rather than absolutes. Each release is measured against the one before it in a record under `docs/project/`; the latest is -[release 0.9.1 against release 0.9.0](../project/performance-release-0.9.1-vs-0.9.0.md). +[release 0.10.0 against release 0.9.2](../project/performance-release-0.10.0-vs-0.9.2.md). How far one realistic model scales — a satellite constellation with every spacecraft modeled to its components, from 2 to 12 800 satellites — and where validation, satisfaction checking and editing each stop being practical is in diff --git a/docs/project/README.md b/docs/project/README.md index 88a0d0990b..a3479e5169 100644 --- a/docs/project/README.md +++ b/docs/project/README.md @@ -90,9 +90,10 @@ within these records and means nothing outside this repository. - **[StateMachines library](statemachines-library.md)** — pseudostates as bundled `SemanticMetadata`, so a state machine's `choice`, `junction` and history pseudostates spell standard, conforming SysML v2 -- **[Performance: 0.9.2 against 0.9.1](performance-release-0.9.2-vs-0.9.1.md)** — the release-gate - measurement of the 0.9.2 patch line against 0.9.1: every benchmark on both revisions, the rows in - doubt re-run interleaved, and whole-binary scaling; the +- **[Performance: 0.10.0 against 0.9.2](performance-release-0.10.0-vs-0.9.2.md)** — the release-gate + measurement of 0.10.0 against 0.9.2: every benchmark on both revisions, the rows in doubt re-run + interleaved, whole-binary scaling, and each confirmed regression with the change responsible; the + [0.9.2 against 0.9.1](performance-release-0.9.2-vs-0.9.1.md), [0.9.1 against 0.9.0](performance-release-0.9.1-vs-0.9.0.md), [0.8.0 against 0.7.0](performance-release-0.8-vs-0.7.0.md), [0.6 against 0.4.2](performance-release-0.6-vs-0.4.2.md) and diff --git a/docs/project/performance-release-0.10.0-vs-0.9.2.md b/docs/project/performance-release-0.10.0-vs-0.9.2.md new file mode 100644 index 0000000000..47e7d0ee5f --- /dev/null +++ b/docs/project/performance-release-0.10.0-vs-0.9.2.md @@ -0,0 +1,736 @@ +# Performance: release 0.10.0 against release 0.9.2 + +The release-gate measurement of the 0.10.0 candidate — `main` at +`ffe51571c` (2026-10-08, the `release/0.10.0` merge plus a CI-only +hotfix), 1 676 commits past the `v0.9.2` tag (`4226ad9a3`, 2026-10-05) — +against 0.9.2 itself. It follows the method of the +[0.9.2 against 0.9.1](performance-release-0.9.2-vs-0.9.1.md) record: every +benchmark on both revisions, `benchstat` over six runs, the rows the machine +left in doubt re-run interleaved, and whole-binary timings on generated and +real models. See [performance](../internals/performance.md) for what the +figures mean and where the remaining cost is. + +0.10.0 is a minor release carrying everything on `develop` since 0.9.2: +exact Rationals and unbounded Integers in the evaluator, the exploration of +every order of a body's unordered statements, executors that interleave move +by move, reflective relationships and indexed connector ends in the semantic +model, incremental re-analysis in the gRPC service, views, code lenses and +inlay hints in the language server, and the fixes of a 272-entry +changelog. The +expectation going in was movement on the execution and analysis paths, and +that is what the measurement finds — it is not a parity record. Every row +that regresses is confirmed by the interleaved re-run, attributed to the +change responsible below, and left for the follow-up work the summary +names; nothing is fixed here. + +## Summary + +**Nothing here blocks the release.** Every row scales as it did — +nothing superlinear arrived, no path stopped fitting in memory, and the +whole binary validates a 12 000-declaration model in 0.29 s against 0.25 s +and the Apollo 11 model in 0.32 s against 0.29 s, holding 19–26 MiB more. +But 0.10.0 is slower than 0.9.2 on most of what it measures, by constant +factors that are the price of its features in some places and avoidable in +others, and the follow-ups below should land in a 0.10.x patch rather than +wait for 0.11. + +Confirmed by the interleaved re-run, with the change responsible: + +- **Loading a model costs 16–20% more time, 14–22% more bytes and 17–19% + more allocations per element** on the connector-dense stress + constellation (2.5–4.6% on the REPL's attribute-dense fixture), and + `sysml -validate` is 10–20% slower with 4–26 MiB more resident on + 3 000–12 000-declaration models. The connector, binding and flow ends + are now owned, journaled, indexed and reflected as metaobjects + (`00b8499a8`, `da8fba4a3`, `32890be28`, `63552cbc4`). Explained; the + structural cost of the release. +- **Executing actions and state machines is 20–70% slower per step with + 1.3–2 times the allocations** (`AssignmentLoopStep` +70%, `ActionLoop` + +36–52%, `StateLoop` +22–27%, `ExecuteActionFreshContext` +30%, + `RunCalc` +20%). Three changes share it: a body's unordered statements + are explored in every order (`71fda4cd5`, `807fcf591`), body feature + bindings are tracked on every write (`a4e1f6ef9`), and control-node + values are held with the flow that brought them (`a0f77861d`). The + profile also shows the binding check building its `"action node …"` + description string on every write that passes — a third of the new + allocations — which is avoidable. **Follow-up.** +- **Checking a constraint is 16–68% slower** (`BatchConstraints` +68%, + `Satisfy` +18–26%, `GRPCVerifyConstraint` +21%, `BatchSatisfy` +9.5%): + ordering performances by outside successions (`72498ad90`) walks the + succession graph on every check (`successionCycle`, 10% of the time) + and the exact-arithmetic dispatch is on the evaluation path. The walk + is memoizable. **Follow-up.** +- **Editing a document in a workspace allocates 2–4 times the bytes it + did** (`WorkspaceUpdate` +99%, `WorkspaceEdit/vehicle` +86%, + `WorkspaceEditSmallDocBesideLarge` +293%, `EditBeside` +143–280%), + growing with the *other* documents in the workspace and +26–43% in time. + The index snapshots each name's registration before a change so that a + name registered again as it was is no change (`4ca321508`, + `fe6e01fb9`); the snapshot copies the re-export claims of every symbol + the name resolves to, and on an 84-byte edit beside a 96 000-element + model that is 15–40% of all bytes allocated. **Follow-up.** +- **The Annex A vehicle example now reports 134 warnings**, one per + member the model's recursive `::**` imports bring in twice + (`eb4b7583c`, the import-ambiguity rule), where 0.9.2 reported none; + both binaries still exit 0 with `no errors`. Every path that returns + the model's diagnostics pays for them: `Analyze/vehicle` +64% time and + +38% allocations (the suggestion tables the warnings build), + `GRPCParseFileCached` 22 → 299 allocations (each diagnostic converted to + protobuf per call), `ConnectParseFileHTTPCached` 3.5× (each serialised), + `GRPCParseFileUncached` +69%. Whether the rule should fire on a + recursive import of a namespace that is also reached through another is + a semantics question for the release notes, not this record; the + performance follows from the answer. **Follow-up.** +- **Interpreted and compiled calcs pay for exact numerics**: + `Fib(25)/interpreted` 8.1 → 41 ms (+409%), `Collatz(27)/interpreted` + +60%, `Collatz(27)/go` +56%, `Hypot(3.0,4.0)/go` 11.5 → 60 ns, and + `SumTo(1000000)/go` 0.78 → 7.2 ms once its budget is raised. Unbounded + Integers with an int64 fast path (`79629923b`), exact Rationals + (`923ec81f0`) and the budget compiled calcs now spend (`d6eafc870`). + The price of the feature; the interpreted `Fib` is the one row where + it is steep enough to deserve a profile-driven pass. +- **Instantiating an object is 18–42% slower with 3–276 more + allocations** (`Instantiate` +18–32%, `InstantiateWarmModel` +42%, + `FleetInstantiate` +21–39%) and up to 16% fewer bytes: the implicit + relationship ends an instance now carries (`fa8a7220c`, `ca0e6cca0`). + Explained. +- `DecodeSnapshot` +19% allocations and `AnalyseResolved` +10%: the + snapshot holds the owned-relationship journal; the analysis runs the + three constraint-tier passes added in the interval. Explained. + +Three rows the candidate cannot run, reported rather than compared: +`CompiledCalc/SumTo(1000000)` exceeds the default step budget that it fit +in 0.9.2 (a million-iteration loop now needs `OPENSYSML_MAX_STEPS`); the +C target refuses the three calcs that do Integer arithmetic, since a C +`int64` cannot hold an unbounded Integer; and the stress constellation's +`AnalyzeSplitPerDocument` fails because its registry turns on the new +decimal-rounding lint, which the generated model's `0.2` literals trip. +The first two belong in the release notes. + +Faster: `HydratePlane` −9% time and −12–31% allocations (indexed +connector ends), `ConvertAPIJSON/32` −15%, `LookupQualified` −15%, +`FleetInstantiate` −7–16% bytes. Unchanged: lexing, parsing, formatting +(on the same input), the migration writer, `ExpandWildcardImports`, +`FeaturesOf`, `ReferencesWarm`, `RenameWarm`, `Diagnostics`, +`REPLEvalExpr`, the empty session's start time. + +The binary is 47.00 MiB (+4.12 MiB) and an empty session maps 68 MiB +(+3 MiB). + +## Method + +Machine: `INTEL(R) XEON(R) PLATINUM 8559C`, 8 CPUs, 31 GiB, Go +`go1.25.11 linux/amd64`, `GOMAXPROCS=8`. 0.9.2 is a worktree at the tag, +built and benchmarked with its own tree; the candidate is `main` at +`ffe51571c`, built and benchmarked with its own. Package benchmarks are +`go test ./ -run '^$' -bench . -benchmem -count 6` on each revision, +compared with `benchstat` (rows are significant at p < 0.05; `~` is no +significant difference). The first pass ran each package's six counts on +0.9.2 and then on the candidate; rows it left in doubt — a significant +difference, or a spread above 20% on either side — were re-run with the +two revisions interleaved, `-count 1` six times each, nothing else on the +machine. Where the first pass left most of a package in doubt +(`tests/perf`, `tests/stressmodel`) the whole package was re-run that way, +and the re-run is the table shown. Whole-binary figures are +`/usr/bin/time -f '%e %M'` over five interleaved runs of each binary, both +built with `make build-sysml` (`-s -w`, `-trimpath`); medians are reported. +The pilot corpora the `vehicle` rows and the gRPC warm-instantiation row +read were fetched into both trees with `scripts/download-pilot-corpora.sh` +after the first pass, so those rows appear in the re-run only. Causes +were read from `git log v0.9.2..ffe51571c -- ` and, where the +history alone did not name one, from `-cpuprofile` and `-memprofile` of +the row on both revisions. + +### What is and is not comparable + +- **Ten packages define benchmarks on both revisions**; nine produce rows. + `internal/syntax/parser` defines no benchmark that runs without + `OPENSYSML_BENCH_MODEL`; neither revision produced a row and the package + is omitted, as before. +- **Candidate-only rows.** `internal/exec/runtime` gains seven rows for the + exact numerics and behavior ordering (`IntegerArithmeticLoop`, + `IntegerArithmeticLoopBeyondInt64`, `IntegerCollectionSum`, + `RationalDecimalLoop`, `RationalHarmonicLoop`, `RealDecimalLoop`, + `BehaviorOrdersLibraryModel`); `tests/perf` gains the eight + `ImportValues` rows of the CSV/TSV/JSON value import. They have no 0.9.2 + counterpart and are listed with their candidate figures, not compared. +- **Rows the candidate cannot run.** In `internal/frontend/repl`, + `CompiledCalc/SumTo(1000000)` fails on the candidate for the interpreted + and Go targets (`evaluation step limit exceeded (10000000 steps)`), and + `Fib`, `SumTo` and `Collatz` for the C target are refused + (`the interpreter's Integers are unbounded and a C program holds int64`); + the five rows have 0.9.2 figures and no candidate figure, and are + discussed with that package. In `tests/stressmodel`, the six + `AnalyzeSplitPerDocument` rows fail on the candidate because the + benchmark asserts a clean analysis and the per-document registry now + reports the decimal literals of the generated model as rounded by their + `Real` features; they have 0.9.2 figures and no candidate figure. +- **`InstantiateWarmModel`** (`internal/frontend/grpc`) now exists on both + revisions and is compared; both first-pass runs skipped it for want of + the vehicle corpus, so its figures are the sequential six-count run made + after the download. +- **`tests/stressmodel`**: `ConvertAPIJSON/satellites=512` completed one + count on 0.9.2 (156.0 s) before the run was killed for memory (78 GiB + allocated per operation on a 31 GiB machine) and five on the candidate + (118.7–121.9 s); it is excluded as no distribution, as in the previous + record. +- **Diagnostics**: on every model measured both binaries return the same + exit status and the same report — `no errors` on the generated models, + the two example models and all 28 Apollo 11 files. (The candidate's + decimal-rounding report is an opt-in lint on the command line; only the + stress benchmark's registry turns it on.) + +## Package benchmarks + +Time per operation, 0.9.2 then the candidate, from the interleaved re-run +wherever a row was re-run (the first pass agrees in sign and roughly in +size on every row named here); bytes and allocations where they moved. +Allocation counts are deterministic to the unit on both revisions and are +the reading that separates a change in the code from a change in the +machine. + +### `internal/exec/runtime` + +| benchmark | 0.9.2 | 0.10.0 | +| --------- | ----- | ------ | +| `MaterializePlainAttributes` | 46.8 µs / 42.69 KiB / 599 allocs | 55.3 µs (+18%, p=0.002) / 43.58 KiB (+2.1%) / 618 (+3.2%) | +| `SetFeatureValueNoDependents` | 423 ns | 442 ns (~) | +| `DerivedReadWriteRead` | 5.65 µs ±16% | 6.53 µs ±2% (+16%, p=0.015); allocations identical | +| `AssignmentLoopStep` | 917 µs / 521.4 KiB / 4 923 allocs | 1 563 µs (+70%, p=0.002) / 837.7 KiB (+61%) / 8 527 (+73%) | +| `IntegerArithmeticLoop` | — | 3.18 ms | +| `IntegerArithmeticLoopBeyondInt64` | — | 3.40 ms | +| `IntegerCollectionSum` | — | 897 µs | +| `RationalDecimalLoop` | — | 3.53 ms | +| `RationalHarmonicLoop` | — | 844 µs | +| `RealDecimalLoop` | — | 4.00 ms ±37% | +| `BehaviorOrdersLibraryModel` | — | 328 µs | + +As found the first two moved rows were +12% and +66%, and +`MaterializePlainAttributes` was `~` at ±18–21%; interleaved all three are +significant. `AssignmentLoopStep` — one assignment step of an action loop, +the figure the 0.9.1 and 0.9.2 records watched — allocates 3 600 more +objects per step. The profile of the equivalent +`tests/perf` row (`ActionLoop/for1000`, 3 181 → 6 220 allocations) names the +new allocators: `stmtEngine.evalIn` (31% of the objects; the frame chain +built per statement to track body feature bindings, `a4e1f6ef9`), +`actionStmtHost.describe` (17%; the `"action node " + name` string the +binding check receives on every write, built whether or not the check +fails), `stmtEngine.runWithOrder` (15%; the exploration of a body's +statement orders, `71fda4cd5`), and `EvalContext.heldValue` is 39% of +the CPU time (control-node values held with their flow, `a0f77861d`). +`MaterializePlainAttributes` and `DerivedReadWriteRead` carry the +implicit-end constancy (`fa8a7220c`) and the exact-number dispatch. + +### `internal/frontend/repl` + +28 rows on 0.9.2, 23 on the candidate (the five it cannot run are above); +of the 23, 8 are `~` as found and 15 moved. Interleaved: + +| benchmark | 0.9.2 | 0.10.0 | +| --------- | ----- | ------ | +| `LoadModel/elements=0` | 405 µs ±39% / 216.2 KiB / 1 875 allocs | 415 µs (~) / 221.6 KiB (+2.5%) / 1 931 (+3.0%) | +| `LoadModel/elements=250` | 28.5 ms ±38% / 13.22 MiB | 29.0 ms (~) / 13.56 MiB (+2.6%) / allocs +4.4% | +| `LoadModel/elements=1000` | 107 ms ±36% / 52.08 MiB | 115 ms (~) / 53.48 MiB (+2.7%) / allocs +4.5% | +| `LoadModel/elements=4000` | 448 ms ±27% / 207.2 MiB / 1.895 M allocs | 494 ms (~, p=0.240) / 213.6 MiB (+3.1%) / 1.982 M (+4.6%) | +| `RunStateMachine/elements=250, 1000` | 13.1 µs / 15.23 KiB / 213 allocs | 13.7 µs, 13.6 µs (~) / 15.52 KiB (+1.9%) / 217 | +| `RunStateMachine/elements=4000` | 13.1 µs | 13.8 µs ±28% (+5.4%, p=0.009) | +| `RunCalc/elements=250, 1000, 4000` | 4.53–4.57 µs / 4.242 KiB / 76 allocs | 5.44–5.49 µs (+20%, p=0.002 at every size) / 4.453 KiB (+5.0%) / 79 | +| `Instantiate/elements=250` | 6.46 µs / 6.415 KiB / 53 allocs | 8.49 µs (+32%, p=0.002) / 7.105 KiB (+11%) / 56 | +| `Instantiate/elements=1000` | 6.02 µs | 7.09 µs (+18%, p=0.002) / +12% B | +| `Instantiate/elements=4000` | 6.02 µs ±36% | 6.85 µs (~, p=0.093) / +12% B | +| `Diagnostics/attributes=50, 200, 800` | 86.0 µs, 300 µs, 1.17 ms | (~) at every size; +0.1% B | +| `CompiledCalc/Fib(25)/interpreted` | 8.13 ms / 44.04 KiB / 368 allocs | 41.4 ms ±71% (+409%, p=0.002) / 269.6 KiB (+512%) / 2 125 (+477%) | +| `CompiledCalc/Fib(25)/go` | 959 µs ±94% / 6 allocs | 1.71 ms ±5% (~, p=0.065; +77% as found) / 12 allocs | +| `CompiledCalc/Fib(25)/c` | 229 µs | not compilable | +| `CompiledCalc/SumTo(1000000)/interpreted` | 1.55 s | step limit exceeded | +| `CompiledCalc/SumTo(1000000)/go` | 783 µs | step limit exceeded | +| `CompiledCalc/SumTo(1000000)/c` | 385 µs | not compilable | +| `CompiledCalc/Collatz(27)/interpreted` | 290 µs / 29.43 KiB / 1 000 allocs | 465 µs (+60%, p=0.002) / 81.91 KiB (+178%) / 2 115 (+112%) | +| `CompiledCalc/Collatz(27)/go` | 5.43 µs ±25% | 8.49 µs ±6% (+56%, p=0.002); 0 allocs on both | +| `CompiledCalc/Collatz(27)/c` | 1.00 µs | not compilable | +| `CompiledCalc/Hypot(3.0,4.0)/interpreted` | 7.30 µs ±38% / 88 allocs | 8.93 µs (~, p=0.065) / 91 allocs (+3.4%) | +| `CompiledCalc/Hypot(3.0,4.0)/go` | 11.5 ns | 60.3 ns ±27% (+423%, p=0.002); 0 allocs on both | +| `CompiledCalc/Hypot(3.0,4.0)/c` | 13.2 ns | 13.9 ns (+5.0%, p=0.026) | + +Three things are going on in this package. + +**The model loader costs 2.5–3.1% more bytes and 3–4.6% more +allocations per element**, with the time within the machine's spread +(+6–9% as found, `~` interleaved at every size). This is the semantic +model's new bookkeeping — owned relationships journaled and reflected as +metaobjects (`da8fba4a3`, `00b8499a8`, `32890be28`), the classification of +extended declarations (`ca0e6cca0`) and the implicit-end constancy derived +once for reflection and writes (`fa8a7220c`) — and the same per-element +growth the stress constellation and `tests/perf` show below. **Explained**; +it is the fourth record running in which the loader grows by a few percent +per release. + +**Instantiating one object and evaluating one calc are 18–32% and 20% +slower**, three allocations more per operation each. The gRPC +`InstantiateWarmModel` row below shows the same +42% with +18% +allocations. An instance now carries the +implicit relationship ends of its type (`fa8a7220c`, `ca0e6cca0`) and +its attribute values go through the exact-number constructors +(`79629923b`, `923ec81f0`); the bytes rise 11–26% on these small objects +and fall on the stress constellation's fleet, where the Integer and +Rational fast paths' allocation-free representation (`d1eb16edb`) wins. + +**The interpreted and compiled calcs pay for unbounded Integers, exact +Rationals and the step budget.** Each of the four fixtures is affected +differently, and the benchmark's own fixture file grew from 597 to 930 +lines (the same four calcs; the new content is other compiled fixtures), +which the `LoadModel`-sized setup absorbs but the `FormatEdits` row in +`internal/frontend/lsp` does not: + +- `Fib(25)/interpreted` is five times slower and allocates six times the + bytes and 5.8 times the objects. The profile shows where the time went: +`compiledCalc.invoke`, `callNode`, `conditionalNode` and `leafPair` +(the interpreted target is the closure-compiled calc evaluator) and +`scalarCheck.held` at 11% flat — the exact-Rational evaluation +(`923ec81f0`) checks the kind of every scalar a node yields and pairs the +leaves of each arithmetic node, 250 000 calls deep. Allocation counts rose +5.8× with it (the `big`-free fast path still allocates a result per +call), and the candidate's ±71% spread is the collector. The price of +exactness, but the steepest row in the record and the one a +profile-driven pass should look at first. +- `Collatz(27)/interpreted` is +60% with 2.1 times the allocations; + `Collatz(27)/go` is +56% with none on either side, and + `Hypot(3.0,4.0)/go` goes from 11.5 ns to 60 ns, also allocation-free. + The Go target's compiled values now +keep an Integer an Integer inside a Real-typed value and hold negative +powers as exact Rationals (`b69ea6840`, `0aa4aca55`), so each arithmetic +node carries its number kind and branches on it; `Hypot`'s four +operations went from 11.5 ns to 60 ns for it, still allocation-free, and +compiled calcs spend the interpreter's step budget (`d6eafc870`). +- `SumTo(1000000)` sums a million integers in a loop. On 0.9.2 the + interpreted run took 1.55 s inside the default budget of 10 000 000 + steps; on the candidate the same loop exceeds it, for the interpreted + and the Go target both, because compiled calcs now spend the + interpreter's step budget (`d6eafc870`) and the interpreter counts the + statements of a loop body it explores in order (`71fda4cd5`, + `43548d741`). The row is not a timing regression but a change in what + fits the default budget: a loop of a million iterations no longer does + without `OPENSYSML_MAX_STEPS`. +- The C target refuses every calc that does Integer arithmetic + (`Fib`, `SumTo`, `Collatz`) because a C `int64` cannot hold the + interpreter's unbounded Integers (`79629923b`); only `Hypot`, on Reals, + still compiles to C. This is a deliberate narrowing of the C target, not + a performance figure, and belongs in the release notes. + +### `internal/frontend/grpc` + +| benchmark | 0.9.2 | 0.10.0 | +| --------- | ----- | ------ | +| `ParseFileColdShared` | 10.7 ms ±34% / 4.689 MiB / 31.12 k allocs | 7.67 ms ±21% (~) / 4.700 MiB (+0.2%) / 31.29 k (+0.5%) | +| `ParseFileColdInline` | 7.69 ms ±29% / 4.916 MiB / 31.47 k | 7.74 ms (~) / 4.932 MiB (~) / 31.69 k (+0.7%) | +| `InstantiateWarmModel` | 134 µs ±8% / 82.87 KiB / 1 555 allocs | 191 µs ±1% (+42%, p=0.002) / 104.13 KiB (+26%) / 1 831 (+18%) | + +The cold-parse rows are the parser and the loader on the vehicle example +and move with the loader's few percent of allocations. The warm +instantiation is the `Instantiate` row of the REPL at the service's +scale: 276 more objects per instantiation. See the `Instantiate` cause under the +REPL: the same implicit relationship ends and exact-number constructors, +on the vehicle example's objects. + +### `internal/frontend/lsp` + +| benchmark | 0.9.2 | 0.10.0 | +| --------- | ----- | ------ | +| `FormatEdits` | 515 µs ±11% / 433.7 KiB / 78 allocs | 871 µs ±18% (+69%, p=0.002) / 704.3 KiB (+62%) / 85 | +| `SpanToRangeLarge` | 2.99 ms | 2.87 ms (~) | +| `ReferencesWarm` | 206 µs | 211 µs (~) | +| `ReferencesCold` | 45.4 ms ±19% | 37.9 ms (~, p=0.065) | +| `RenameWarm` | 20.0 ms ±76% | 18.9 ms ±28% (~; −14.5% as found) | +| `WorkspaceUpdate` | 198 ms ±100% / 49.39 MiB / 605.6 k allocs | 270 ms ±16% (~, p=0.065) / 98.46 MiB (+99%) / 930.6 k (+54%) | + +`FormatEdits` formats `internal/frontend/repl/testdata/compile_calcs.sysml`, +which grew from 597 to 930 lines on the candidate with the new compiled +fixtures; the bytes move by the ratio of the file sizes (+62% against ++56% more lines) and the formatter itself has one commit in the interval +(the indexed connector ends, `63552cbc4`). **Not a regression in the +code.** + +`WorkspaceUpdate` — one edit to one of the workspace's documents, with the +re-analysis the server does for it — allocates twice the bytes and 1.5 +times the objects it did, and the time is within the machine's spread +only because the baseline's six counts ran 100% apart. The profile puts 39% of the bytes +(810 MB over the run) and 30% of the objects in one function new since +0.9.2, `symbols.registrationOf`, reached from `Index.noteBefore`: before +a name's registration changes, the index snapshots what a lookup of it +would read — the symbols under it, their re-export and hidden flags, and +a copy of every document's re-export claims and routes — so that a name +registered again exactly as it was does not count as a change +(`4ca321508`, `fe6e01fb9`). The snapshot is per name touched by the +re-registration, which on a workspace update is every name of the +document and of what imports it, and the claim maps are deep-copied. The +intent is the right one (fewer downstream invalidations); the copy is +avoidable. **Follow-up.** The rest of the growth is the resolver's new +`checkImportedNames` (`eb4b7583c`). + +### `internal/translate/migrate`, `internal/workspace/libs`, `internal/workspace/model` + +| benchmark | 0.9.2 | 0.10.0 | +| --------- | ----- | ------ | +| `migrate WriterSiblingBlocks` | 1.93 ms | 1.87 ms (~); bytes and allocations identical | +| `libs ExpandWildcardImports` | 26.8 ms | 27.8 ms (~) | +| `libs IndexLibrary` | 13.8 ms / 13.56 MiB / 104.6 k allocs | 14.1 ms (~; +5.1% as found) / 14.07 MiB (+3.8%) / 108.2 k (+3.4%) | +| `libs ExpandModelImports` | 6.36 ms | 6.50 ms (~) | +| `libs DecodeSnapshot` | 9.52 ms / 35.32 MiB / 56.63 k allocs | 9.98 ms (+4.9%, p=0.009) / 37.85 MiB (+7.2%) / 67.45 k (+19%) | +| `libs SetDigest` | 629 µs | 624 µs (~) | +| `model AnalyseUnresolved` | 20.2 ms | 19.9 ms (~) | +| `model AnalyseResolved` | 1.26 ms / 543.0 KiB / 5 049 allocs | 1.39 ms (+10%, p=0.002) / 590.4 KiB (+8.7%) / 5 971 (+18%) | + +The migration writer is unchanged. Indexing the standard library +allocates 3.4% more objects (the reflective relationship ends, the +`StateActivity` extension library `0188fb308`), and decoding its snapshot +19% more: the snapshot carries the owned-relationship journal and the +reflected implicit relationships (`da8fba4a3`, `32890be28`) that the +0.9.2 snapshot did not hold. Analysing a resolved document is +10% with +18% more allocations: the constraint-tier passes added in the interval — +the indexed-end multiplicity check (`cf85beb36`), the import-ambiguity +warning (`eb4b7583c`), the reflective derivations for the batch +constraints (`70bfa8ee2`) — each walk the document once more. The +`tests/perf` `Analyze` rows below show the same +4.7% allocations on the +synthetic model and much more on the vehicle example. + +### `tests/perf` + +37 rows on 0.9.2 and 45 on the candidate as found (the eight `ImportValues` +rows are candidate-only); 48 and 56 once the vehicle corpus was present, +and the whole package was re-run interleaved. Of the 48 compared rows 20 +are `~` in time; the rows that moved, and the rows whose allocations moved: + +| benchmark | 0.9.2 | 0.10.0 | +| --------- | ----- | ------ | +| `Lex`, `Parse` (synthetic, vehicle) | | (~) time; bytes and allocations identical | +| `IndexAddExpand/synthetic` | 42.5 ms ±41% / 39.70 MiB / 290.8 k allocs | 44.9 ms (~) / 42.45 MiB (+6.9%) / 315.5 k (+8.5%) | +| `IndexAddOnly/synthetic` | 39.7 ms / 37.59 MiB / 281.2 k | 41.0 ms (~) / 40.34 MiB (+7.3%) / 305.9 k (+8.8%) | +| `IndexAddOnly/vehicle` | 1.07 ms / 1.091 MiB / 9 276 | 1.06 ms (~) / 1.138 MiB (+4.3%) / 9 632 (+3.8%) | +| `Analyze/synthetic` | 1.13 s ±17% / 466.1 MiB / 4.435 M | 1.24 s ±29% (+9.2%, p=0.026) / 487.1 MiB (+4.5%) / 4.644 M (+4.7%) | +| `Analyze/vehicle` | 118 ms ±20% / 73.64 MiB / 627.5 k | 193 ms ±44% (+64%, p=0.002) / 130.0 MiB (+77%) / 865.1 k (+38%) | +| `WorkspaceEdit/synthetic` | 204 ms ±74% / 77.98 MiB / 596.3 k | 256 ms ±42% (~) / 85.73 MiB (+9.9%) / 680.6 k (+14%) | +| `WorkspaceEdit/synthetic/reindex+diagnostics` | 1.52 s ±60% / 493.2 MiB / 4.787 M | 1.62 s (~) / 519.7 MiB (+5.4%) / 5.090 M (+6.3%) | +| `WorkspaceEdit/vehicle` | 52.0 ms / 12.33 MiB / 139.1 k | 66.0 ms (+27%, p=0.002) / 22.95 MiB (+86%) / 214.4 k (+54%) | +| `WorkspaceEdit/vehicle/reindex+diagnostics` | 167 ms / 71.25 MiB / 651.7 k | 223 ms ±27% (+34%, p=0.002) / 129.8 MiB (+82%) / 917.6 k (+41%) | +| `WorkspaceEditSmallDocBesideLarge` | 42.4 ms ±51% / 7.952 MiB / 81.74 k | 60.8 ms ±64% (+43%, p=0.041) / 31.24 MiB (+293%) / 162.6 k (+99%) | +| `FQNOf` | 1.37 ms ±23% / 36 000 symbols | 1.52 ms (~) / 40 000 symbols (+11%); 38.0 ns against 38.1 ns per symbol (~) | +| `LookupQualified` | 1.39 ms ±18% | 1.18 ms (−15%, p=0.041) | +| `FeaturesOf` | 931 ms ±55% | 945 ms (~); bytes and allocations identical | +| `REPLLoadFile` | 1.32 s ±26% / 595.5 MiB / 5.383 M | 1.48 s (~, p=0.065) / 616.5 MiB (+3.5%) / 5.595 M (+4.0%) | +| `REPLSubmitSnippet` | 1.03 s / 513.6 MiB / 4.841 M | 1.11 s (+7.6%, p=0.015) / 545.7 MiB (+6.3%) / 5.048 M (+4.3%) | +| `REPLEvalExpr` | 4.50 ms / 101 allocs | 4.46 ms (~) / 104 | +| `LowerActionGraph` | 1.73 µs / 2.625 KiB / 25 allocs | 2.01 µs (+16%, p=0.002) / 2.906 KiB (+11%) / 27 | +| `LowerStateGraph` | 8.30 µs | 8.08 µs (~); bytes −0.5% | +| `LowerChain/chain10, 100, 1000` | 8.44 µs, 145 µs, 8.94 ms | 9.53 µs (+13%), 150 µs (~), 8.55 ms (~); bytes +13–15% | +| `ExecuteAction` | 75.3 µs / 192 allocs | 78.9 µs (+4.9%, p=0.004) / 253 (+32%) | +| `ExecuteActionFreshContext` | 30.4 µs ±22% / 26.26 KiB / 268 | 39.5 µs ±4% (+30%, p=0.002) / 34.94 KiB (+33%) / 347 (+29%) | +| `ExecuteState` | 704 µs / 483.4 KiB / 10.38 k | 838 µs (+19%, p=0.002) / 608.2 KiB (+26%) / 11.66 k (+12%) | +| `ActionLoop/for10` | 25.0 µs / 20.85 KiB / 204 | 33.9 µs (+36%) / 28.37 KiB (+36%) / 273 (+34%) | +| `ActionLoop/for100` | 101 µs / 47.97 KiB / 477 | 152 µs (+50%) / 70.26 KiB (+46%) / 816 (+71%) | +| `ActionLoop/for1000` | 850 µs / 279.0 KiB / 3 181 | 1 290 µs (+52%) / 449.2 KiB (+61%) / 6 220 (+96%) | +| `ActionChain/chain10` | 88.6 µs / 675 allocs | 101 µs (+14%) / 784 (+16%) | +| `ActionChain/chain100` | 562 µs / 522.8 KiB / 5 757 | 696 µs (+24%) / 705.9 KiB (+35%) / 6 595 (+15%) | +| `ActionChain/chain1000` | 13.0 ms / 5.153 MiB / 56.28 k | 14.5 ms (+11%) / 7.077 MiB (+37%) / 64.77 k (+15%) | +| `StateLoop/count50, 500, 5000` | 653 µs, 6.23 ms, 61.6 ms | 798 µs (+22%), 7.92 ms (+27%), 78.4 ms (+27%); bytes +26%, allocs +12% at every size | +| `BatchConstraints` | 8.97 ms ±31% / 2.223 MiB / 55.19 k | 15.1 ms ±11% (+68%, p=0.002) / 2.387 MiB (+7.4%) / 55.86 k (+1.2%) | +| `SameConstraintManyInstances` | 3.56 µs / 1 016 B | 4.12 µs (+16%, p=0.002) / 1 000 B (−1.6%) | +| `BatchSatisfy` | 1.18 s / 417.0 MiB / 4.211 M | 1.29 s (+9.5%, p=0.009) / 427.2 MiB (+2.4%) / 4.357 M (+3.5%) | +| `Instantiate` | 508 ms ±102% / 173.9 MiB ±57% | 529 ms ±6% (~) / 180.2 MiB (~, p=0.065) | +| `GRPCParseFileCached` | 58.8 µs / 216.7 KiB / 22 allocs | 69.7 µs (+19%, p=0.002) / 242.1 KiB (+12%) / 299 (13.6×) | +| `GRPCParseFileUncached` | 124 ms ±31% / 81.41 MiB / 660.5 k | 210 ms ±5% (+69%, p=0.002) / 142.1 MiB (+74%) / 905.1 k (+37%) | +| `GRPCEvaluate` | 8.97 µs / 8.426 KiB / 110 | 9.75 µs (+8.7%, p=0.009) / 9.406 KiB (+12%) / 116 | +| `GRPCVerifyConstraint` | 795 µs / 510.6 KiB / 9 188 | 958 µs (+21%, p=0.002) / 541.8 KiB (+6.1%) / 9 768 (+6.3%) | +| `ConnectEvaluateHTTP` | 150 µs / 201 allocs | 153 µs (~) / 206 | +| `ConnectParseFileHTTPCached` | 251 µs / 352.7 KiB / 166 allocs | 882 µs (+252%, p=0.002) / 488.3 KiB (+38%) / 1 292 (7.8×) | +| `ImportValues/flat/parts=100, 500, 2000 (attrs=3)` | — | 21.3 ms, 104 ms, 548 ms; 300, 1 500, 6 000 values | +| `ImportValues/flat/parts=2000/attrs=5` | — | 844 ms ±37%; 10 000 values | +| `ImportValues/nested/parts=100, 500, 2000 (attrs=3)` | — | 29.4 ms, 141 ms, 620 ms | +| `ImportValues/nested/parts=2000/attrs=5` | — | 847 ms | +| geomean (compared rows) | | +18% time | + +Four groups, each with a cause of its own: + +1. **Execution.** Every row that runs an action or state machine is + slower by a constant factor per step: an action loop iteration +36–52% + with 1.3–2 times the allocations, a state-machine event +19–27% with + +12% allocations, a succession chain +11–24%, a fresh action context + +30%, and `ExecuteAction` +4.9% with 32% more allocations. The + `internal/exec/runtime` rows above (`AssignmentLoopStep` +70%) and the + REPL's `RunCalc` (+20%) are the same cost. The runtime profile (under +`internal/exec/runtime` above) splits the new work three ways: the +frames built per statement to track body feature bindings +(`a4e1f6ef9`), the exploration of a body's statement orders +(`71fda4cd5`, `807fcf591`, `c8891457a`), and control-node values held +with their flow (`a0f77861d`); the description string the binding check +builds on every passing write is the avoidable part. `LowerActionGraph` ++16% and `LowerChain/chain10` +13% are the lowering carrying the +succession orders the executor explores. +2. **Constraint checking.** `BatchConstraints` is +68% per check with + +1.2% allocations, `SameConstraintManyInstances` +16% with fewer bytes, + `GRPCVerifyConstraint` +21%, `BatchSatisfy` +9.5%. The profile of `BatchConstraints` +puts 10% of the candidate's CPU in `Context.successionCycle`, a function +that did not exist: ordering performances by outside successions +(`72498ad90`) walks the succession graph when a constraint's body is +evaluated, on every check, for a model whose successions do not change +between checks. The remainder is the exact-number dispatch on the +evaluation path (`923ec81f0`, `79629923b`) and the reflective +derivations the constraint passes read (`70bfa8ee2`). The walk is +memoizable per model. **Follow-up.** +3. **The gRPC parse path.** `ParseFile` on a document the service has + already parsed allocated 22 objects on 0.9.2 and allocates 299 on the + candidate (+19% time); through Connect it is 3.5 times slower, and an + uncached parse is +69% time and +74% bytes. The service's `ParseSources` + now names the documents an edit reached and re-analyzes only those + (`fcc123b40`, `2d921fae9`, `af2604ba8`): the cached path that used to + return a stored result now computes the affected set, and the uncached + path carries the lineage bookkeeping. That is not where the allocations are, +though. The profile puts 70% of the cached path's objects in +`DiagnosticToProto` and `sourceSpanToProto`, called from +`modelDiagnostics`: the vehicle example now carries 134 warnings +(below), and each `ParseFile` converts every one of them to protobuf +again — the 22 → 299 allocations are 2 per warning plus the response. +The service reports the parser's and resolver's warnings as the +workspace does (`71d4bacea`), which it did not in 0.9.2; the cost is the +number of warnings times the number of calls, and would be gone with +either a cached conversion or no warnings on this model. +4. **Workspace edits and analysis.** Editing a document beside others + allocates 1.9–3.9 times the bytes it did (`WorkspaceEdit/vehicle` +86%, + `WorkspaceEditSmallDocBesideLarge` +293%; the LSP's `WorkspaceUpdate` + +99% and the stress constellation's `EditBeside` +143–280% are the same + path), and analysing the vehicle example — a model dense in connectors, + bindings, flows and decimal literals — is +64% time and +38% + allocations against +4.7% on the synthetic model. Two causes. The bytes are +`symbols.registrationOf` (see `WorkspaceUpdate` above): the stress +`EditBeside` profile puts 15% of the bytes and 13% of the objects there +on a 32-satellite constellation, and the share grows with the model +beside the edit (`EditBeside` +143% → +280% bytes from 32 to 512 +satellites). The vehicle rows have a second cause: 0.9.2 validated the +Annex A `SimpleVehicleModel` with no diagnostic, and the candidate +reports 134 warnings, `Duplicate of imported member name '…'`, one per +member that `VehicleConfiguration_b::**` and +`VehicleConfiguration_b::PartsTree::**` both bring into the same +namespace (`eb4b7583c`, the import-ambiguity rule). The profile of +`Analyze/vehicle` shows the rule's `checkImportedNames` (21% of bytes) +and the `suggest.EditDistance` tables each diagnostic builds (12% of +objects) as the whole of the growth; the synthetic model, which has no +such imports, grows 4.7%. Whether a recursive import should warn about +a member it also reaches through a nested recursive import is for the +semantics to decide; until it does, every tool that analyses this model +pays 134 diagnostics' worth of work per analysis, and every gRPC client +receives them. + +The import of values from CSV, TSV and JSON files (`ImportValues`) is new +in 0.10.0; 2 000 parts take 0.55–0.85 s flat and 0.62–0.85 s nested, which +is 90–280 µs per imported value. The first record to carry these rows, +they are the baseline for the next. + +### `tests/stressmodel` + +Generated satellite constellations at 32, 128 and 512 satellites (6 227, +24 167 and 95 927 elements), one document each; the first pass left every +time row in doubt (0.9.2's counts ran 10–60% apart), so the whole package +was re-run interleaved and the re-run is shown. 36 rows compared; six +`AnalyzeSplitPerDocument` rows are 0.9.2-only (above) and +`ConvertAPIJSON` has no distribution on either side. + +| benchmark | 0.9.2 | 0.10.0 | +| --------- | ----- | ------ | +| `Load/satellites=32` | 346 ms / 132.7 MiB / 1.994 M allocs | 415 ms (+20%) / 162.1 MiB (+22%) / 2.365 M (+19%) | +| `Load/satellites=128` | 1.38 s / 501.3 MiB / 7.682 M | 1.60 s (+16%) / 579.1 MiB (+16%) / 9.040 M (+18%) | +| `Load/satellites=512` | 6.11 s / 1.934 GiB / 30.43 M | 7.12 s (+16%) / 2.208 GiB (+14%) / 35.74 M (+17%) | +| `Satisfy/satellites=32` (96 assertions) | 4.10 ms / 1.665 MiB / 28.90 k | 4.83 ms (+18%) / 1.831 MiB (+10%) / 34.57 k (+20%) | +| `Satisfy/satellites=128` (384) | 17.3 ms / 6.652 MiB | 20.7 ms (+20%) / 7.316 MiB (+10%) / +20% allocs | +| `Satisfy/satellites=512` (1 536) | 82.8 ms ±7% / 26.62 MiB | 104 ms ±7% (+26%) / 29.27 MiB (+10%) / +20% allocs | +| `FleetInstantiate/satellites=32` (1 179 elements) | 4.06 ms / 7.987 MiB / 12.72 k | 4.94 ms (+21%) / 7.417 MiB (−7.1%) / 13.96 k (+9.7%) | +| `FleetInstantiate/satellites=128` (1 715) | 11.4 ms / 24.10 MiB / 48.37 k | 14.6 ms (+29%) / 24.45 MiB (+1.4%) / 52.97 k (+9.5%) | +| `FleetInstantiate/satellites=512` (3 947) | 59.8 ms / 160.2 MiB ±7% / 190.9 k | 82.9 ms ±9% (+39%) / 134.1 MiB (−16%) / 209.0 k (+9.5%) | +| `FleetSatisfy/satellites=32, 128, 512` | 1.01 ms, 1.60 ms, 5.06 ms | 1.19 ms (+18%), 1.88 ms (+17%), 6.30 ms (+24%); +8.8–9.2% B, +17–18% allocs | +| `EditBeside/satellites=32` | 1.05 ms / 295.5 KiB / 3 442 | 1.32 ms (+26%) / 719.5 KiB (+143%) / 5 533 (+61%) | +| `EditBeside/satellites=128` | 3.21 ms / 719.9 KiB / 8 377 | 4.21 ms (+31%) / 2 435.7 KiB (+238%) / 15 922 (+90%) | +| `EditBeside/satellites=512` | 12.9 ms / 2.408 MiB / 28.11 k | 16.9 ms (+31%) / 9.147 MiB (+280%) / 57.44 k (+104%) | +| `ValidateSplit/satellites=32/files=6/jobs=1` | 508 ms / 186.1 MiB / 2.480 M | 572 ms (+13%) / 217.5 MiB (+17%) / 2.898 M (+17%) | +| `ValidateSplit/satellites=32/files=6/jobs=8` | 236 ms | 253 ms (+7.0%) / +17% B / +17% allocs | +| `ValidateSplit/satellites=128/files=6/jobs=1, 8` | 1.79 s, 737 ms | 2.04 s (+14%), 840 ms ±9% (+14%) / +15% B / +17% allocs | +| `ValidateSplit/satellites=512/files=6/jobs=1` | 7.39 s ±18% / 2.335 GiB / 33.69 M | 8.34 s ±2% (~, p=0.065) / 2.646 GiB (+13%) / 39.59 M (+18%) | +| `ValidateSplit/satellites=512/files=6/jobs=8` | 2.90 s ±18% | 3.48 s ±2% (+20%, p=0.004) / +13% B / +18% allocs | +| `LoadFiles/satellites=32/files=6` | 435 ms ±20% / 147.7 MiB / 1.971 M | 508 ms ±1% (~, p=0.065) / 176.8 MiB (+20%) / 2.365 M (+20%) | +| `LoadFiles/satellites=128, 512/files=6` | 1.62 s, 6.66 s ±10% | 1.85 s (+14%), 7.69 s (+16%) / +16–18% B / +21% allocs | +| `EditImported/satellites=32, 128, 512/files=6` | 473 ms ±43%, 1.60 s ±44%, 6.54 s ±36% | 532 ms ±2%, 1.82 s ±3%, 7.45 s ±4% (all ~) / +19–30% B / +22–25% allocs | +| `OpenSplit/satellites=32/files=6/cold` | 228 ms ±11% / 169.1 MiB / 2.362 M | 239 ms ±8% (+4.8%, p=0.041) / 199.9 MiB (+18%) / 2.777 M (+18%) | +| `OpenSplit/satellites=32/files=6/warm` | 96.6 ms ±13% / 65.62 MiB / 571.1 k | 122 ms ±39% (+27%) / 86.03 MiB (+31%) / 706.8 k (+24%) | +| `OpenSplit/satellites=128/files=6/cold` | 693 ms ±13% / 541.3 MiB | 804 ms ±6% (+16%, p=0.009) / 632.2 MiB (+17%) / +18% allocs | +| `OpenSplit/satellites=128/files=6/warm` | 180 ms ±7% / 166.6 MiB / 1.385 M | 250 ms ±45% (+39%) / 215.2 MiB (+29%) / 1.730 M (+25%) | +| `OpenSplit/satellites=512/files=6/cold` | 2.70 s ±5% / 2.042 GiB | 3.35 s ±17% (+24%, p=0.009) / 2.355 GiB (+15%) / +19% allocs | +| `OpenSplit/satellites=512/files=6/warm` | 586 ms ±14% / 613.4 MiB / 4.639 M | 849 ms ±113% (+45%) / 897.7 MiB (+46%) / 5.821 M (+25%) | +| `HydratePlane/satellites=32/files=6` | 85.5 ms ±9% / 25.80 MiB / 314.8 k | 77.7 ms ±10% (−9.1%, p=0.015) / 25.67 MiB (−0.5%) / 278.5 k (−12%) | +| `HydratePlane/satellites=128/files=6` | 224 ms ±23% / 76.61 MiB / 919.8 k | 202 ms ±19% (~) / 62.71 MiB (−18%) / 680.3 k (−26%) | +| `HydratePlane/satellites=512/files=6` | 932 ms ±3% / 287.2 MiB / 3.336 M | 767 ms ±64% (~, p=0.394; −37% as found) / 214.2 MiB (−25%) / 2.287 M (−31%) | +| `ConvertAPIJSON/satellites=32` (as found) | 7.86 s ±4% / 5.021 GiB / 40.91 M | 6.69 s ±4% (−15%) / 5.156 GiB (+2.7%) / 44.96 M (+9.9%) | +| `ConvertAPIJSON/satellites=128` (as found) | 31.9 s ±4% / 19.69 GiB / 157.8 M | 31.2 s ±2% (~) / 20.22 GiB (+2.7%) / 173.4 M (+9.9%) | +| `ConvertAPIJSON/satellites=512` | 156 s, one count, then killed | 119–122 s, five counts; no distribution on either side | +| `AnalyzeSplitPerDocument/satellites=32, 128, 512/files=6/jobs=1, 8` | 2.75 ms, 11.6 ms, 50.4 ms (jobs=1); 1.23 ms, 2.26 ms, 10.7 ms (jobs=8) | fails: the generated model's decimal literals are reported as rounded | +| geomean (36 compared rows) | 167 ms / 78.87 MiB / 911.8 k | 195 ms (+17%) / 95.95 MiB (+22%) / 1.081 M (+19%) | + +Unless marked, every time row is p=0.002 over six interleaved counts; +every byte and allocation row is p=0.002 with 0% spread. + +The constellation says in one place what the packages say severally: + +- **Loading is +16–20% time, +14–22% bytes and +17–19% allocations per + element**, at every size, in every path that loads (`Load`, + `LoadFiles`, `ValidateSplit`, `OpenSplit/cold`, `EditImported`'s bytes). + This is more than the REPL's `LoadModel` (+2.5–3.1% bytes) because the + constellation is nothing but parts, ports, connections and flows between + them, and it is the connector, binding and flow ends whose + representation changed: their ends are owned by the connector and + journaled as owned relationships (`00b8499a8`, `da8fba4a3`), inherited + ends are replaced positionally, flows are paired by end, each end may + carry an index (`63552cbc4`, `7514a97bf`), and every implicit + relationship the resolver derives is reflected as a metaobject + (`32890be28`). A model of 95 927 elements holds 5.3 million more objects + for it and takes a second longer to load. **Explained**, and the largest + single cost of the release. +- **Satisfying requirements is +18–26% time, +10% bytes and +20% + allocations per assertion**, the same as `tests/perf`'s + `GRPCVerifyConstraint` (+21%) and `BatchConstraints`; the fleet + variant +17–24%. See the constraint-checking cause under `tests/perf`. +- **Instantiating a fleet is +21–39% time with 9.5% more allocations and + up to 16% fewer bytes**: the same `Instantiate` cost as the REPL and + gRPC rows (+18–42%), with the bytes going the other way because the + instantiated objects' attribute storage is smaller (the + allocation-free Integer and Rational fast paths, `d1eb16edb`). +- **Editing a small document beside the constellation is +26–31% time + and 2.4–3.8 times the bytes**, growing with the size of the document + beside it — the edited document is 84 bytes and does not change. The + bytes scale with the *other* document: the workspace's re-analysis after + an edit now re-derives something proportional to the whole workspace. + See the workspace-edit cause under `tests/perf`. +- **Opening a workspace warm** (the documents already analysed once) is + +27–45% with spreads of 39–113% on the candidate, the only rows whose + spread the re-run did not settle; the bytes (+29–46%) are the loader's + growth plus the edit path's. +- **Hydrating a plane is 9–37% faster with 12–31% fewer allocations** — + the one improvement at scale, from the indexed connector ends: the + hydrator finds an end by index rather than by scanning. +- **Converting to API JSON is 15% faster at 32 satellites** with 9.9% more + allocations (`fcc123b40`'s affected-set bookkeeping applies here too), + unchanged at 128; at 512 the first-pass run was killed on 0.9.2 after + its second count (78 GiB allocated per operation) and the candidate's + five counts of 119–122 s are reported for what they are. + +## Whole-binary scaling + +`sysml -validate` over generated compliant models (the satellite-network +generator at ≈3 000, 6 000 and 12 000 declarations: 3 804, 7 260 and +14 172 lines) that report `no errors`, five runs each with the two +binaries interleaved; median wall seconds and median maximum resident set. + +| declarations | 0.9.2 | 0.10.0 | +| ------------ | ----- | ------ | +| ≈3 000 | 0.10 s / 97 MiB | 0.12 s / 101 MiB | +| ≈6 000 | 0.16 s / 109 MiB | 0.17 s / 115 MiB | +| ≈12 000 | 0.25 s / 132 MiB | 0.29 s / 158 MiB | + +Loading and validating a model from the command line is 10–20% slower +and holds 4–26 MiB more at the end, growing with the model: the loader's +per-element bytes and the analysis passes above, seen whole. At 12 000 +declarations that is 40 ms and 26 MiB; the figures scale linearly on both +binaries (0.10→0.25 s and 0.12→0.29 s from 3 000 to 12 000), so nothing +superlinear arrived with the release. + +### Process start and session floor + +| figure | 0.9.2 | 0.10.0 | +| ------ | ----- | ------ | +| binary size | 42.88 MiB (44 957 880 B) | 47.00 MiB (49 275 064 B; +4.12 MiB) | +| empty session (`sysml -e 1`), median of 5 | 0.02 s / 65 MiB | 0.02 s / 68 MiB | +| empty-session load (`LoadModel/elements=0`) | 405 µs / 216.2 KiB | 415 µs (~) / 221.6 KiB (+2.5%) | + +The binary grows 4.12 MiB (+9.6%) with the release's new surfaces — the +views and their D2, Mermaid, DOT and PlantUML renderers, the graphs +export, the api-json loader, the value import, the browser engine's +shared code, the `big.Int` and `big.Rat` arithmetic — and the empty +session maps 3 MiB more for it; its start time is unchanged at the clock's +resolution. + +### Example models and the Apollo 11 model + +Median wall time and maximum resident set of five interleaved runs; every +run returned the same exit status and report on both binaries (the +candidate tree's copies of the two example files were given to both). The +Apollo 11 model is the public +[Apollo 11 SysML v2 model](https://github.com/airbus/apollo-11-sysml-v2) +(commit `6e9c93f`, 28 files, 7 221 lines). + +| model / command | 0.9.2 | 0.10.0 | +| --------------- | ----- | ------ | +| `action-executor-demo.sysml -validate` | 0.02 s / 66 MiB | 0.03 s / 70 MiB | +| `disposal-robot-demo/robot.sysml -validate` | 0.04 s / 72 MiB | 0.04 s / 78 MiB | +| Apollo 11, `-validate` all 28 files | 0.29 s / 150 MiB | 0.32 s / 169 MiB | + +The Apollo 11 model loads and validates in 0.32 s against 0.29 s (+10%, +three clock ticks) and holds 19 MiB more, in line with the generated +models of its size. + +## Re-runs + +The first pass left 71 of the 190 compared rows in doubt: 48 by a +significant difference, 23 by a spread above 20% on one side with no +significant difference (`WorkspaceUpdate` ±100%, `Fib(25)/go` ±94%, +`Instantiate` ±102%, most of the stress constellation's time rows). All +were re-run interleaved — `-count 1` of the row on 0.9.2, then on the +candidate, six times — and the `tests/perf` and `tests/stressmodel` +packages whole, since most of each was in doubt. The pilot corpora were +downloaded between the first pass and the re-run, so the `vehicle` rows +and `InstantiateWarmModel` have one sequential six-count run each +(reported above) and no first pass. + +What the re-run changed: + +- **Nothing changed sign.** Every row significant in the first pass is + significant in the re-run in the same direction; the stress + constellation's time regressions shrank from the first pass's +39–87% + to +16–45% once the two revisions ran under the same conditions, and + that is the figure reported. The first pass had run all of 0.9.2's + counts first and the candidate's after; the re-run shows that about + half of the first-pass difference on the long rows was the machine. +- **Cleared:** `RenameWarm` (−14.5% as found, `~`), `ReferencesCold` + (`~`), `HydratePlane/128, 512` (−10%, −37% as found; `~` in time, the + byte and allocation gains stand), `WorkspaceUpdate` (+36% as found; + `~` in time at ±16%, the +99% bytes stand), `Fib(25)/go` (+77% as + found; `~` at p=0.065, 1.71 ms ±5% against 959 µs ±94%), `Instantiate` + in `tests/perf` (`~`), `EditImported` (`~` at every size), the REPL + `LoadModel` rows (`~` in time at every size), `Hypot/interpreted` + (`~`). +- **Confirmed** (significant both times, within a few points of each + other): `AssignmentLoopStep` (+66% → +70%), `MaterializePlainAttributes` + (`~` → +18%), `DerivedReadWriteRead` (+12% → +16%), `RunCalc` (+20%), + `Instantiate/250, 1000` (+32%, +18%), `Fib(25)/interpreted` (+434% → + +409%), `Collatz(27)/interpreted` (+68% → +60%), `Collatz(27)/go` + (+56%), `Hypot(3.0,4.0)/go` (+398% → +423%), `FormatEdits` (+69%), + `InstantiateWarmModel` (+42%), `DecodeSnapshot` (+4.9%), + `AnalyseResolved` (+10%), and every `tests/perf` and stress row in the + tables above. +- **Still in doubt:** `OpenSplit/*/warm` (±39–113% on the candidate's + counts, every one of them above 0.9.2's); `RealDecimalLoop` ±37% and + `ImportValues/flat/parts=2000/attrs=5` ±37%, candidate-only. + +Raw `benchstat` output for every package, first pass and re-run, is in the +tables above; the per-row p-values are p=0.002 (n=6) unless shown. + +## What this does not measure + +- The new surfaces with no 0.9.2 counterpart — the view renderers, the + graphs export, the api-json loader, the browser engine, the Python and + Node clients' new calls — have benchmarks only where listed above + (`ImportValues`, the exact-numeric loops); the rest are covered by their + test suites, not by this record. +- The compiled-calc C target is measured on `Hypot` only; the three + Integer calcs it refuses have no candidate figure, so the C target's + speed on 0.10.0 is read from one row. +- Everything ran on one 8-CPU virtual machine. Its neighbours were + quieter than during the 0.9.2 record (the baseline's re-run spreads are + mostly under 20%), but the stress constellation's `Satisfy` and + `EditBeside` rows still show 30–80% spreads on single counts, and a + one-CPU host should read the package figures rather than the + whole-binary ones, as the previous records said. diff --git a/mkdocs.yml b/mkdocs.yml index 2405b76842..5ffb7f3db3 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -175,6 +175,7 @@ exclude_docs: | project/performance-release-0.8-vs-0.7.0.md project/performance-release-0.9.1-vs-0.9.0.md project/performance-release-0.9.2-vs-0.9.1.md + project/performance-release-0.10.0-vs-0.9.2.md project/satellite-network-stress-test.md project/large-model-scaling-design.md project/native-compilation.md From 0d6e0a41e75133d2f356c52f0288bc32894e71db Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Fri, 9 Oct 2026 17:58:19 +0000 Subject: [PATCH 2/4] docs(perf): measure the 0.10.0 candidate with its performance fixes against 0.9.2 Co-Authored-By: jason.han --- .../unreleased/perf-0.10.0-record.added.md | 2 +- .../performance-release-0.10.0-vs-0.9.2.md | 1259 +++++++++-------- 2 files changed, 662 insertions(+), 599 deletions(-) diff --git a/changes/unreleased/perf-0.10.0-record.added.md b/changes/unreleased/perf-0.10.0-record.added.md index e8821dceec..714468b539 100644 --- a/changes/unreleased/perf-0.10.0-record.added.md +++ b/changes/unreleased/perf-0.10.0-record.added.md @@ -1 +1 @@ -- **The release-gate performance record of 0.10.0 against 0.9.2.** `docs/project/performance-release-0.10.0-vs-0.9.2.md` measures `main` at the 0.10.0 release against the `v0.9.2` tag — every benchmark on both revisions under `benchstat`, the rows in doubt re-run interleaved, the stress constellation, and whole-binary validation, start-up and size — and names the change responsible for each confirmed regression. Nothing in it blocks the release; loading is 16–20% slower per element on connector-dense models, executing a step 20–70%, checking a constraint 16–68%, editing a workspace document allocates 2–4 times the bytes, and the import-ambiguity warning fires 134 times on the Annex A vehicle example, each with a follow-up named. +- **The release-gate performance record of 0.10.0 against 0.9.2.** `docs/project/performance-release-0.10.0-vs-0.9.2.md` measures the 0.10.0 candidate — `main` at the release merge with `develop`'s post-cut changes and the six performance fixes folded in — against the `v0.9.2` tag: every benchmark on both revisions under `benchstat`, the rows in doubt re-run interleaved, the stress constellation, and whole-binary validation, start-up and size, with a column for the release merge alone where a fix moved a row. Nothing in it blocks the release. The fixes cleared the interpreted `Fib(25)` (now 20% faster than 0.9.2), the batch constraint check and the cached gRPC diagnostics, and halved the workspace-edit allocations; what remains is loading 10–16% slower per element on connector-dense models, executing a step 20–60%, workspace edits beside a large document at 2.9–3.0 times the bytes, the import-ambiguity warning firing 134 times on the Annex A vehicle example, and the exact-numeric calcs, each with its cause named. diff --git a/docs/project/performance-release-0.10.0-vs-0.9.2.md b/docs/project/performance-release-0.10.0-vs-0.9.2.md index 47e7d0ee5f..f7424ac23c 100644 --- a/docs/project/performance-release-0.10.0-vs-0.9.2.md +++ b/docs/project/performance-release-0.10.0-vs-0.9.2.md @@ -1,8 +1,10 @@ # Performance: release 0.10.0 against release 0.9.2 -The release-gate measurement of the 0.10.0 candidate — `main` at -`ffe51571c` (2026-10-08, the `release/0.10.0` merge plus a CI-only -hotfix), 1 676 commits past the `v0.9.2` tag (`4226ad9a3`, 2026-10-05) — +The release-gate measurement of the 0.10.0 candidate — +`hotfix/0.10.0-develop-sync` at `82f0fac87` (2026-10-09), which is `main` +at `ffe51571c` (the `release/0.10.0` merge plus a CI-only hotfix) with +everything `develop` gained after the release branch was cut merged back +into it, 1 754 commits past the `v0.9.2` tag (`4226ad9a3`, 2026-10-05) — against 0.9.2 itself. It follows the method of the [0.9.2 against 0.9.1](performance-release-0.9.2-vs-0.9.1.md) record: every benchmark on both revisions, `benchstat` over six runs, the rows the machine @@ -15,131 +17,124 @@ exact Rationals and unbounded Integers in the evaluator, the exploration of every order of a body's unordered statements, executors that interleave move by move, reflective relationships and indexed connector ends in the semantic model, incremental re-analysis in the gRPC service, views, code lenses and -inlay hints in the language server, and the fixes of a 272-entry -changelog. The -expectation going in was movement on the execution and analysis paths, and -that is what the measurement finds — it is not a parity record. Every row -that regresses is confirmed by the interleaved re-run, attributed to the -change responsible below, and left for the follow-up work the summary -names; nothing is fixed here. +inlay hints in the language server, repository and notebook commands in the +REPL, and the fixes of a 300-entry changelog. The expectation going in was +movement on the execution and analysis paths, and that is what the +measurement finds — it is not a parity record. + +The candidate also carries six performance fixes made against an earlier +measurement of `ffe51571c` alone, and this record shows what they bought: +where a row moved between `ffe51571c` and `82f0fac87` the tables carry a +third column with the `ffe51571c` figure from the same interleaved run. +The six, by the change on the branch: + +- a cached model's diagnostics are converted to protobuf once, not on + every `ParseFile` that returns them (`70242d872`); +- the symbol index no longer deep-copies a document's re-export claims to + decide whether a name's registration changed (`4b4b35121`); +- the binding check on a body write builds its description only when it + refuses the write (`37b4a17c9`); +- the succession-cycle check is memoized while no behavior is held + (`8bd97fcf6`); +- a compiled calc's scalars are packed into three words instead of being + boxed per node (`4a1b937f2`); +- a symbol's qualified name is compared without being built (`367ca7e3a`). + +Every row that regresses is confirmed by the interleaved re-run and +attributed to the change responsible below; the fixes that remain open are +named in the summary, and nothing is fixed here. ## Summary -**Nothing here blocks the release.** Every row scales as it did — -nothing superlinear arrived, no path stopped fitting in memory, and the -whole binary validates a 12 000-declaration model in 0.29 s against 0.25 s -and the Apollo 11 model in 0.32 s against 0.29 s, holding 19–26 MiB more. -But 0.10.0 is slower than 0.9.2 on most of what it measures, by constant -factors that are the price of its features in some places and avoidable in -others, and the follow-ups below should land in a 0.10.x patch rather than -wait for 0.11. - -Confirmed by the interleaved re-run, with the change responsible: - -- **Loading a model costs 16–20% more time, 14–22% more bytes and 17–19% - more allocations per element** on the connector-dense stress - constellation (2.5–4.6% on the REPL's attribute-dense fixture), and - `sysml -validate` is 10–20% slower with 4–26 MiB more resident on - 3 000–12 000-declaration models. The connector, binding and flow ends - are now owned, journaled, indexed and reflected as metaobjects - (`00b8499a8`, `da8fba4a3`, `32890be28`, `63552cbc4`). Explained; the - structural cost of the release. -- **Executing actions and state machines is 20–70% slower per step with - 1.3–2 times the allocations** (`AssignmentLoopStep` +70%, `ActionLoop` - +36–52%, `StateLoop` +22–27%, `ExecuteActionFreshContext` +30%, - `RunCalc` +20%). Three changes share it: a body's unordered statements - are explored in every order (`71fda4cd5`, `807fcf591`), body feature - bindings are tracked on every write (`a4e1f6ef9`), and control-node - values are held with the flow that brought them (`a0f77861d`). The - profile also shows the binding check building its `"action node …"` - description string on every write that passes — a third of the new - allocations — which is avoidable. **Follow-up.** -- **Checking a constraint is 16–68% slower** (`BatchConstraints` +68%, - `Satisfy` +18–26%, `GRPCVerifyConstraint` +21%, `BatchSatisfy` +9.5%): - ordering performances by outside successions (`72498ad90`) walks the - succession graph on every check (`successionCycle`, 10% of the time) - and the exact-arithmetic dispatch is on the evaluation path. The walk - is memoizable. **Follow-up.** -- **Editing a document in a workspace allocates 2–4 times the bytes it - did** (`WorkspaceUpdate` +99%, `WorkspaceEdit/vehicle` +86%, - `WorkspaceEditSmallDocBesideLarge` +293%, `EditBeside` +143–280%), - growing with the *other* documents in the workspace and +26–43% in time. - The index snapshots each name's registration before a change so that a - name registered again as it was is no change (`4ca321508`, - `fe6e01fb9`); the snapshot copies the re-export claims of every symbol - the name resolves to, and on an 84-byte edit beside a 96 000-element - model that is 15–40% of all bytes allocated. **Follow-up.** -- **The Annex A vehicle example now reports 134 warnings**, one per - member the model's recursive `::**` imports bring in twice - (`eb4b7583c`, the import-ambiguity rule), where 0.9.2 reported none; - both binaries still exit 0 with `no errors`. Every path that returns - the model's diagnostics pays for them: `Analyze/vehicle` +64% time and - +38% allocations (the suggestion tables the warnings build), - `GRPCParseFileCached` 22 → 299 allocations (each diagnostic converted to - protobuf per call), `ConnectParseFileHTTPCached` 3.5× (each serialised), - `GRPCParseFileUncached` +69%. Whether the rule should fire on a - recursive import of a namespace that is also reached through another is - a semantics question for the release notes, not this record; the - performance follows from the answer. **Follow-up.** -- **Interpreted and compiled calcs pay for exact numerics**: - `Fib(25)/interpreted` 8.1 → 41 ms (+409%), `Collatz(27)/interpreted` - +60%, `Collatz(27)/go` +56%, `Hypot(3.0,4.0)/go` 11.5 → 60 ns, and - `SumTo(1000000)/go` 0.78 → 7.2 ms once its budget is raised. Unbounded - Integers with an int64 fast path (`79629923b`), exact Rationals - (`923ec81f0`) and the budget compiled calcs now spend (`d6eafc870`). - The price of the feature; the interpreted `Fib` is the one row where - it is steep enough to deserve a profile-driven pass. -- **Instantiating an object is 18–42% slower with 3–276 more - allocations** (`Instantiate` +18–32%, `InstantiateWarmModel` +42%, - `FleetInstantiate` +21–39%) and up to 16% fewer bytes: the implicit - relationship ends an instance now carries (`fa8a7220c`, `ca0e6cca0`). - Explained. -- `DecodeSnapshot` +19% allocations and `AnalyseResolved` +10%: the - snapshot holds the owned-relationship journal; the analysis runs the - three constraint-tier passes added in the interval. Explained. - -Three rows the candidate cannot run, reported rather than compared: -`CompiledCalc/SumTo(1000000)` exceeds the default step budget that it fit -in 0.9.2 (a million-iteration loop now needs `OPENSYSML_MAX_STEPS`); the -C target refuses the three calcs that do Integer arithmetic, since a C -`int64` cannot hold an unbounded Integer; and the stress constellation's -`AnalyzeSplitPerDocument` fails because its registry turns on the new -decimal-rounding lint, which the generated model's `0.2` literals trip. -The first two belong in the release notes. - -Faster: `HydratePlane` −9% time and −12–31% allocations (indexed -connector ends), `ConvertAPIJSON/32` −15%, `LookupQualified` −15%, -`FleetInstantiate` −7–16% bytes. Unchanged: lexing, parsing, formatting -(on the same input), the migration writer, `ExpandWildcardImports`, -`FeaturesOf`, `ReferencesWarm`, `RenameWarm`, `Diagnostics`, -`REPLEvalExpr`, the empty session's start time. - -The binary is 47.00 MiB (+4.12 MiB) and an empty session maps 68 MiB -(+3 MiB). +**Nothing here blocks the release.** 0.10.0 is slower than 0.9.2 on most +of the paths a model goes through, by a constant factor on each; nothing +is superlinear, nothing that fit 0.9.2's budgets stops fitting except the +one calc noted below, and the fleet's instantiation and hydration are +faster. Against the earlier measurement of `ffe51571c`, the six fixes on +the candidate cleared three of the regressions that record named as +avoidable and halved a fourth: + +- `CompiledCalc/Fib(25)/interpreted` went from 5.3× 0.9.2 to **20% + faster than 0.9.2** (the three-word scalar packing); +- `BatchConstraints` from +68% to `~`, `Satisfy` from +22–28% to + +10–14% (the memoized succession-cycle check); +- `GRPCParseFileCached` from 299 allocations to 24 against 0.9.2's 22 + and from +22% to `~` (the cached protobuf diagnostics); +- the workspace-edit rows from 3.8–3.9× 0.9.2's bytes to 2.9–3.0×, and + from +61–104% allocations to +32–48% (the re-export claims compared in + place); +- `AssignmentLoopStep` lost 1 200 of the 3 604 allocations the release + had added, and `ActionLoop/for1000` 2 000 of 3 039 (the lazy + binding-check description); +- the loader lost 2–3 points of allocations (the qualified-name compare). + +What remains, confirmed by the interleaved re-run: + +- **Loading** is +15–18% allocations and +12–20% bytes per element on + connector-dense models, for +10–16% time: `Load` +16% at 32 + satellites, `sysml -validate` +20–22% and +6–23 MiB at 3 000–12 000 + declarations, Apollo 11 +14% and +12 MiB. Each connector, binding and + flow end is now owned, journaled, indexed and reflected. Structural; + linear. +- **Action and state execution** is +20–60% per step with 1.3–1.5× the + allocations: `AssignmentLoopStep` +61%, `ActionLoop` +38–56%, + `StateLoop` +22–40%, `ExecuteState` +21%, `ExecuteActionFreshContext` + +38%, `RunCalc` +10–14%. The exploration of a body's statement orders, + the per-statement binding frames and the held control values are the + semantics of the release; the per-step constant is where a 0.10.x pass + should look next. +- **Workspace edits beside a large document** still allocate 2.9–3.0× + 0.9.2's bytes (`WorkspaceEditSmallDocBesideLarge` 7.95 → 23.7 MiB, + `EditBeside/512` 2.41 → 6.66 MiB) for +20–31% time: the per-name + snapshot of re-export claims across the large neighbor. The one row + in this record with an evident fix still open. +- **The Annex A vehicle example now emits 134 + `Duplicate of imported member name` warnings** (0 in 0.9.2), and every + path that returns them pays: `Analyze/vehicle` +85%, + `GRPCParseFileUncached` +64%, `ConnectParseFileHTTPCached` 271 → 879 µs + (the HTTP transport has no cache of its JSON). Whether the rule should + fire on nested `::**` imports is a semantics question; if not, the + group is one fix away from 0.9.2. +- **Exact numerics**: `Collatz(27)` +59–65% (an allocation per Integer + multiply or divide), `Fib(25)/go` +81% and `Hypot/go` 11 → 56 ns (the + Go target's values carry their number kind), `Instantiate` +11–31% per + object (implicit ends and exact-number constructors). The price of the + feature. +- **Three things the candidate cannot run**, for the release notes: + `SumTo(1000000)` exceeds the default 10 000 000-step budget it fit in + 0.9.2 (compiled calcs now spend the interpreter's budget); the C target + refuses calcs on Integers (unbounded against `int64`); the stress + benchmark's per-document analysis trips the new decimal-rounding lint + on its generated `Real` literals. +- Binary 42.88 → 47.23 MiB; empty session 64 → 69 MiB, start time + unchanged. ## Method Machine: `INTEL(R) XEON(R) PLATINUM 8559C`, 8 CPUs, 31 GiB, Go `go1.25.11 linux/amd64`, `GOMAXPROCS=8`. 0.9.2 is a worktree at the tag, -built and benchmarked with its own tree; the candidate is `main` at -`ffe51571c`, built and benchmarked with its own. Package benchmarks are +built and benchmarked with its own tree; the candidate is +`hotfix/0.10.0-develop-sync` at `82f0fac87`, built and benchmarked with its +own; `main` at `ffe51571c`, where it is shown, is a third worktree treated +the same way. Package benchmarks are `go test ./ -run '^$' -bench . -benchmem -count 6` on each revision, compared with `benchstat` (rows are significant at p < 0.05; `~` is no significant difference). The first pass ran each package's six counts on 0.9.2 and then on the candidate; rows it left in doubt — a significant difference, or a spread above 20% on either side — were re-run with the -two revisions interleaved, `-count 1` six times each, nothing else on the +revisions interleaved, `-count 1` six times each, nothing else on the machine. Where the first pass left most of a package in doubt -(`tests/perf`, `tests/stressmodel`) the whole package was re-run that way, -and the re-run is the table shown. Whole-binary figures are -`/usr/bin/time -f '%e %M'` over five interleaved runs of each binary, both -built with `make build-sysml` (`-s -w`, `-trimpath`); medians are reported. -The pilot corpora the `vehicle` rows and the gRPC warm-instantiation row -read were fetched into both trees with `scripts/download-pilot-corpora.sh` -after the first pass, so those rows appear in the re-run only. Causes -were read from `git log v0.9.2..ffe51571c -- ` and, where the -history alone did not name one, from `-cpuprofile` and `-memprofile` of -the row on both revisions. +(`internal/frontend/repl`, `tests/perf`, `tests/stressmodel`) the whole +package or the whole of its moving rows was re-run that way, and the re-run +is the table shown. Whole-binary figures are `/usr/bin/time -f '%e %M'` +over nine interleaved runs of each binary, all built with +`make build-sysml` (`-s -w`, `-trimpath`); medians are reported. The pilot +corpora the `vehicle` rows and the gRPC warm-instantiation row read were +fetched into every tree with `scripts/download-pilot-corpora.sh`; 0.9.2's +first pass predates the download, so those rows appear in the re-run only. +Causes were read from `git log v0.9.2..82f0fac87 -- ` and, where the +history alone did not name one, from `-cpuprofile` and `-memprofile` of the +row on both revisions. ### What is and is not comparable @@ -165,15 +160,14 @@ the row on both revisions. benchmark asserts a clean analysis and the per-document registry now reports the decimal literals of the generated model as rounded by their `Real` features; they have 0.9.2 figures and no candidate figure. -- **`InstantiateWarmModel`** (`internal/frontend/grpc`) now exists on both - revisions and is compared; both first-pass runs skipped it for want of - the vehicle corpus, so its figures are the sequential six-count run made - after the download. +- **`InstantiateWarmModel`** (`internal/frontend/grpc`) exists on both + revisions and is compared from the interleaved re-run; 0.9.2's first + pass skipped it for want of the vehicle corpus. - **`tests/stressmodel`**: `ConvertAPIJSON/satellites=512` completed one count on 0.9.2 (156.0 s) before the run was killed for memory (78 GiB - allocated per operation on a 31 GiB machine) and five on the candidate - (118.7–121.9 s); it is excluded as no distribution, as in the previous - record. + allocated per operation on a 31 GiB machine) and one on the candidate + (135.6 s) before the same; it is excluded as no distribution, as in the + previous record. - **Diagnostics**: on every model measured both binaries return the same exit status and the same report — `no errors` on the generated models, the two example models and all 28 Apollo 11 files. (The candidate's @@ -185,123 +179,144 @@ the row on both revisions. Time per operation, 0.9.2 then the candidate, from the interleaved re-run wherever a row was re-run (the first pass agrees in sign and roughly in size on every row named here); bytes and allocations where they moved. -Allocation counts are deterministic to the unit on both revisions and are -the reading that separates a change in the code from a change in the -machine. +Where a row moved between `ffe51571c` and the candidate, the `ffe51571c` +figure from the same interleaved run stands between them. Allocation +counts are deterministic to the unit on every revision and are the reading +that separates a change in the code from a change in the machine. ### `internal/exec/runtime` -| benchmark | 0.9.2 | 0.10.0 | -| --------- | ----- | ------ | -| `MaterializePlainAttributes` | 46.8 µs / 42.69 KiB / 599 allocs | 55.3 µs (+18%, p=0.002) / 43.58 KiB (+2.1%) / 618 (+3.2%) | -| `SetFeatureValueNoDependents` | 423 ns | 442 ns (~) | -| `DerivedReadWriteRead` | 5.65 µs ±16% | 6.53 µs ±2% (+16%, p=0.015); allocations identical | -| `AssignmentLoopStep` | 917 µs / 521.4 KiB / 4 923 allocs | 1 563 µs (+70%, p=0.002) / 837.7 KiB (+61%) / 8 527 (+73%) | -| `IntegerArithmeticLoop` | — | 3.18 ms | -| `IntegerArithmeticLoopBeyondInt64` | — | 3.40 ms | -| `IntegerCollectionSum` | — | 897 µs | -| `RationalDecimalLoop` | — | 3.53 ms | -| `RationalHarmonicLoop` | — | 844 µs | -| `RealDecimalLoop` | — | 4.00 ms ±37% | -| `BehaviorOrdersLibraryModel` | — | 328 µs | - -As found the first two moved rows were +12% and +66%, and -`MaterializePlainAttributes` was `~` at ±18–21%; interleaved all three are -significant. `AssignmentLoopStep` — one assignment step of an action loop, -the figure the 0.9.1 and 0.9.2 records watched — allocates 3 600 more -objects per step. The profile of the equivalent -`tests/perf` row (`ActionLoop/for1000`, 3 181 → 6 220 allocations) names the -new allocators: `stmtEngine.evalIn` (31% of the objects; the frame chain -built per statement to track body feature bindings, `a4e1f6ef9`), -`actionStmtHost.describe` (17%; the `"action node " + name` string the -binding check receives on every write, built whether or not the check -fails), `stmtEngine.runWithOrder` (15%; the exploration of a body's -statement orders, `71fda4cd5`), and `EvalContext.heldValue` is 39% of -the CPU time (control-node values held with their flow, `a0f77861d`). -`MaterializePlainAttributes` and `DerivedReadWriteRead` carry the -implicit-end constancy (`fa8a7220c`) and the exact-number dispatch. +| benchmark | 0.9.2 | `ffe51571c` | 0.10.0 (`82f0fac87`) | +| --------- | ----- | ----------- | -------------------- | +| `MaterializePlainAttributes` | 45.2 µs ±4% / 42.68 KiB / 599 allocs | 53.1 µs (+17%) / 43.58 KiB / 618 | 51.5 µs ±2% (+14%, p=0.002) / 43.58 KiB (+2.1%) / 618 (+3.2%) | +| `SetFeatureValueNoDependents` | 423 ns ±28% / 352 B / 9 | 442 ns (~) | 490 ns ±24% (~, p=0.132) / 352 B / 9 | +| `DerivedReadWriteRead` | 5.39 µs ±2% / 2.484 KiB / 65 | 6.09 µs (+13%) | 5.96 µs ±2% (+11%, p=0.002) / 2.484 KiB / 65 | +| `AssignmentLoopStep` | 940 µs ±2% / 521.4 KiB / 4 923 | 1 568 µs (+67%) / 837.7 KiB (+61%) / 8 527 (+73%) | 1 509 µs ±2% (+61%, p=0.002) / 817.3 KiB (+57%) / 7 327 (+49%) | + +Candidate-only rows (no 0.9.2 counterpart): `IntegerArithmeticLoop` +3.57 ms ±59% / 279.7 KiB / 9 070 allocs; `IntegerArithmeticLoopBeyondInt64` +3.51 ms ±7% / 514.1 KiB / 13 070; `IntegerCollectionSum` 879 µs ±5% / +4.961 MiB / 44; `RationalDecimalLoop` 3.60 ms ±34% / 279.7 KiB / 9 069; +`RationalHarmonicLoop` 849 µs ±4% / 178.9 KiB / 4 650; `RealDecimalLoop` +3.75 ms ±9% / 279.7 KiB / 9 069; `BehaviorOrdersLibraryModel` 392 µs ±10% / +163.2 KiB / 1 922. They are the baseline for the next record. + +The interleaved re-run settled all three compared rows in doubt at ±2%. +**Executing one assignment statement of an action loop is +61% with 1.6 +times the bytes and 1.5 times the allocations**; materializing an +instance's plain attributes +14% with 19 more allocations, and a derived +read-write-read +11% with none. The fix on this path — the binding check +describing a body write's host only when it refuses the write +(`37b4a17c9`) — took `AssignmentLoopStep` from 8 527 to 7 327 allocations +per step (1 200 of the 3 604 the release had added, the description +strings the earlier profile showed) and 4% of its time; the remaining ++61% is the three changes the profile splits it into: the frames built +per statement to track body feature bindings (`a4e1f6ef9`), the +exploration of a body's statement orders (`71fda4cd5`, `807fcf591`, +`c8891457a`) and control-node values held with the flow that brought them +(`a0f77861d`). Those are the semantics of the release — a body's unordered +statements are now explored in every order, and that costs frames — and +not a bug; the per-step constant is where a 0.10.x pass should look next. +The `tests/perf` `ActionLoop`, `StateLoop` and `ExecuteActionFreshContext` +rows below and the REPL's `RunCalc` are the same cost from other +entrances. The 19 allocations on `MaterializePlainAttributes` and the 2.1% +bytes are the implicit relationship ends an instance now carries +(`fa8a7220c`, `ca0e6cca0`); see `Instantiate` under the REPL. ### `internal/frontend/repl` -28 rows on 0.9.2, 23 on the candidate (the five it cannot run are above); -of the 23, 8 are `~` as found and 15 moved. Interleaved: - -| benchmark | 0.9.2 | 0.10.0 | -| --------- | ----- | ------ | -| `LoadModel/elements=0` | 405 µs ±39% / 216.2 KiB / 1 875 allocs | 415 µs (~) / 221.6 KiB (+2.5%) / 1 931 (+3.0%) | -| `LoadModel/elements=250` | 28.5 ms ±38% / 13.22 MiB | 29.0 ms (~) / 13.56 MiB (+2.6%) / allocs +4.4% | -| `LoadModel/elements=1000` | 107 ms ±36% / 52.08 MiB | 115 ms (~) / 53.48 MiB (+2.7%) / allocs +4.5% | -| `LoadModel/elements=4000` | 448 ms ±27% / 207.2 MiB / 1.895 M allocs | 494 ms (~, p=0.240) / 213.6 MiB (+3.1%) / 1.982 M (+4.6%) | -| `RunStateMachine/elements=250, 1000` | 13.1 µs / 15.23 KiB / 213 allocs | 13.7 µs, 13.6 µs (~) / 15.52 KiB (+1.9%) / 217 | -| `RunStateMachine/elements=4000` | 13.1 µs | 13.8 µs ±28% (+5.4%, p=0.009) | -| `RunCalc/elements=250, 1000, 4000` | 4.53–4.57 µs / 4.242 KiB / 76 allocs | 5.44–5.49 µs (+20%, p=0.002 at every size) / 4.453 KiB (+5.0%) / 79 | -| `Instantiate/elements=250` | 6.46 µs / 6.415 KiB / 53 allocs | 8.49 µs (+32%, p=0.002) / 7.105 KiB (+11%) / 56 | -| `Instantiate/elements=1000` | 6.02 µs | 7.09 µs (+18%, p=0.002) / +12% B | -| `Instantiate/elements=4000` | 6.02 µs ±36% | 6.85 µs (~, p=0.093) / +12% B | -| `Diagnostics/attributes=50, 200, 800` | 86.0 µs, 300 µs, 1.17 ms | (~) at every size; +0.1% B | -| `CompiledCalc/Fib(25)/interpreted` | 8.13 ms / 44.04 KiB / 368 allocs | 41.4 ms ±71% (+409%, p=0.002) / 269.6 KiB (+512%) / 2 125 (+477%) | -| `CompiledCalc/Fib(25)/go` | 959 µs ±94% / 6 allocs | 1.71 ms ±5% (~, p=0.065; +77% as found) / 12 allocs | -| `CompiledCalc/Fib(25)/c` | 229 µs | not compilable | -| `CompiledCalc/SumTo(1000000)/interpreted` | 1.55 s | step limit exceeded | -| `CompiledCalc/SumTo(1000000)/go` | 783 µs | step limit exceeded | -| `CompiledCalc/SumTo(1000000)/c` | 385 µs | not compilable | -| `CompiledCalc/Collatz(27)/interpreted` | 290 µs / 29.43 KiB / 1 000 allocs | 465 µs (+60%, p=0.002) / 81.91 KiB (+178%) / 2 115 (+112%) | -| `CompiledCalc/Collatz(27)/go` | 5.43 µs ±25% | 8.49 µs ±6% (+56%, p=0.002); 0 allocs on both | -| `CompiledCalc/Collatz(27)/c` | 1.00 µs | not compilable | -| `CompiledCalc/Hypot(3.0,4.0)/interpreted` | 7.30 µs ±38% / 88 allocs | 8.93 µs (~, p=0.065) / 91 allocs (+3.4%) | -| `CompiledCalc/Hypot(3.0,4.0)/go` | 11.5 ns | 60.3 ns ±27% (+423%, p=0.002); 0 allocs on both | -| `CompiledCalc/Hypot(3.0,4.0)/c` | 13.2 ns | 13.9 ns (+5.0%, p=0.026) | +Attribute-dense generated models at 0, 250, 1 000 and 4 000 elements, and +the compiled-calc fixtures. 28 rows on 0.9.2, 23 on the candidate: the five +`CompiledCalc` rows the candidate cannot run are listed with their 0.9.2 +figures. The first pass left the whole package in doubt (the candidate's +`LoadModel` counts ran 11–32% apart); the moving rows were re-run +interleaved and the re-run is shown. + +| benchmark | 0.9.2 | `ffe51571c` | 0.10.0 (`82f0fac87`) | +| --------- | ----- | ----------- | -------------------- | +| `LoadModel/elements=0` | 399 µs ±20% / 216.2 KiB / 1 875 allocs | 399 µs (~) | 402 µs ±3% (~) / 221.6 KiB (+2.5%) / 1 931 (+3.0%) | +| `LoadModel/elements=250` | 28.0 ms ±8% / 13.22 MiB / 126.1 k | 30.1 ms (+7.4%) | 29.3 ms ±7% (+4.7%, p=0.015) / 13.51 MiB (+2.2%) / 130.3 k (+3.4%) | +| `LoadModel/elements=1000` | 104 ms ±17% / 52.06 MiB / 480.0 k | 117 ms (+12%) | 114 ms ±9% (+9.5%, p=0.041) / 53.25 MiB (+2.3%) / 496.8 k (+3.5%) | +| `LoadModel/elements=4000` | 458 ms ±6% / 207.2 MiB / 1.895 M | 507 ms (+11%) | 493 ms ±18% (+7.6%, p=0.009) / 212.7 MiB (+2.7%) / 1.962 M (+3.5%) | +| `RunStateMachine/elements=250` | 12.6 µs ±3% / 15.23 KiB / 213 | 13.9 µs (+10%) | 13.7 µs ±22% (+8.9%, p=0.002) / 15.52 KiB (+1.9%) / 217 (+1.9%) | +| `RunStateMachine/elements=1000` | 12.9 µs ±5% | 13.5 µs (+4.5%) | 13.2 µs ±20% (~, p=0.240) / same | +| `RunStateMachine/elements=4000` | 13.0 µs ±1% | 14.2 µs (+9.6%) | 13.7 µs ±23% (+5.8%, p=0.026) / same | +| `RunCalc/elements=250` | 4.42 µs ±6% / 4.242 KiB / 76 | 5.32 µs (+20%) | 5.02 µs ±27% (+14%, p=0.002) / 4.453 KiB (+5.0%) / 79 (+3.9%) | +| `RunCalc/elements=1000` | 4.50 µs ±5% | 5.30 µs (+18%) | 4.99 µs ±6% (+11%, p=0.002) / same | +| `RunCalc/elements=4000` | 4.49 µs ±5% | 5.31 µs (+18%) | 4.94 µs ±8% (+10%, p=0.002) / same | +| `Instantiate/elements=250` | 6.38 µs ±4% / 6.420 KiB / 53 | 8.49 µs (+33%) | 8.36 µs ±6% (+31%, p=0.002) / 7.114 KiB (+11%) / 56 (+5.7%) | +| `Instantiate/elements=1000` | 6.19 µs ±6% / 6.404 KiB | 6.96 µs (+12%) | 7.06 µs ±5% (+14%, p=0.002) / 7.197 KiB (+12%) / 56 | +| `Instantiate/elements=4000` | 6.08 µs ±4% / 6.375 KiB | 6.83 µs (+12%) | 6.73 µs ±5% (+11%, p=0.002) / 7.187 KiB (+13%) / 56 | +| `Diagnostics/attributes=50` | 86.0 µs ±8% / 72.67 KiB / 545 | — | 89.1 µs ±7% (~) / 73.60 KiB (+1.3%) / 559 (+2.6%) | +| `Diagnostics/attributes=200` | 300 µs ±14% / 248.0 KiB / 1 453 | — | 309 µs ±4% (~) / 248.9 KiB (+0.4%) / 1 467 (+1.0%) | +| `Diagnostics/attributes=800` | 1.21 ms ±8% / 895.2 KiB / 5 607 | 1.14 ms (−5.7%) | 1.21 ms ±20% (~, p=0.937) / 896.2 KiB (+0.1%) / 5 621 (+0.2%) | +| `CompiledCalc/Fib(25)/interpreted` | 7.69 ms ±8% / 42.67 KiB / 353 | 40.9 ms (+433%) / 261.2 KiB (+512%) / 2 059 (+483%) | 6.16 ms ±9% (−20%, p=0.002) / 45.73 KiB (+7.2%, p=0.041) / 381 (+7.8%, p=0.039) | +| `CompiledCalc/Fib(25)/c` | 225 µs ±5% / 1 B / 0 | refused | refused (Integer arithmetic) | +| `CompiledCalc/Fib(25)/go` | 949 µs ±13% / 7 B / 0 | 1 704 µs (+80%) | 1 720 µs ±53% (+81%, p=0.002) / 12 B (+71%) / 0 | +| `CompiledCalc/SumTo(1000000)/interpreted` | 1.549 s ±7% / 58.84 MiB / 3.043 M | step limit | step limit exceeded (10 000 000 steps) | +| `CompiledCalc/SumTo(1000000)/c` | 385 µs ±10% / 2 B / 0 | refused | refused (Integer arithmetic) | +| `CompiledCalc/SumTo(1000000)/go` | 783 µs ±4% / 5 B / 0 | step limit | step limit exceeded (10 000 000 steps) | +| `CompiledCalc/Collatz(27)/interpreted` | 275 µs ±7% / 29.30 KiB / 999 | 443 µs (+61%) | 455 µs ±6% (+65%, p=0.002) / 81.84 KiB (+179%) / 2 115 (+112%) | +| `CompiledCalc/Collatz(27)/c` | 1.01 µs ±10% / 0 B / 0 | refused | refused (Integer arithmetic) | +| `CompiledCalc/Collatz(27)/go` | 5.40 µs ±4% / 0 B / 0 | 8.36 µs (+55%) | 8.58 µs ±10% (+59%, p=0.002) / 0 B / 0 | +| `CompiledCalc/Hypot(3.0,4.0)/interpreted` | 7.10 µs ±31% / 7.846 KiB / 88 | 8.82 µs (~) | 8.21 µs ±34% (+16%, p=0.041) / 8.078 KiB (+3.0%) / 91 (+3.4%) | +| `CompiledCalc/Hypot(3.0,4.0)/c` | 13.3 ns ±14% / 0 B / 0 | 13.4 ns (~) | 13.8 ns ±4% (~, p=0.240) / 0 B / 0 | +| `CompiledCalc/Hypot(3.0,4.0)/go` | 11.0 ns ±81% / 0 B / 0 | 54.5 ns (+395%) | 56.4 ns ±2% (+412%, p=0.002) / 0 B / 0 | Three things are going on in this package. -**The model loader costs 2.5–3.1% more bytes and 3–4.6% more -allocations per element**, with the time within the machine's spread -(+6–9% as found, `~` interleaved at every size). This is the semantic -model's new bookkeeping — owned relationships journaled and reflected as -metaobjects (`da8fba4a3`, `00b8499a8`, `32890be28`), the classification of -extended declarations (`ca0e6cca0`) and the implicit-end constancy derived -once for reflection and writes (`fa8a7220c`) — and the same per-element -growth the stress constellation and `tests/perf` show below. **Explained**; -it is the fourth record running in which the loader grows by a few percent -per release. - -**Instantiating one object and evaluating one calc are 18–32% and 20% -slower**, three allocations more per operation each. The gRPC -`InstantiateWarmModel` row below shows the same +42% with +18% -allocations. An instance now carries the -implicit relationship ends of its type (`fa8a7220c`, `ca0e6cca0`) and -its attribute values go through the exact-number constructors -(`79629923b`, `923ec81f0`); the bytes rise 11–26% on these small objects -and fall on the stress constellation's fleet, where the Integer and -Rational fast paths' allocation-free representation (`d1eb16edb`) wins. - -**The interpreted and compiled calcs pay for unbounded Integers, exact -Rationals and the step budget.** Each of the four fixtures is affected -differently, and the benchmark's own fixture file grew from 597 to 930 -lines (the same four calcs; the new content is other compiled fixtures), -which the `LoadModel`-sized setup absorbs but the `FormatEdits` row in +**The model loader costs 2.2–2.7% more bytes and 3.0–3.5% more +allocations per element**, and 5–10% more time (p ≤ 0.041 interleaved, +with the 4 000-element row's candidate counts 18% apart). This is the +semantic model's new bookkeeping — owned relationships journaled and +reflected as metaobjects (`da8fba4a3`, `00b8499a8`, `32890be28`), the +classification of extended declarations (`ca0e6cca0`) and the implicit-end +constancy derived once for reflection and writes (`fa8a7220c`) — and the +same per-element growth the stress constellation and `tests/perf` show +below. The qualified-name fix (`367ca7e3a`) took 5 000 allocations per +1 000 elements off `ffe51571c`'s loader (501.8 k → 496.8 k) and 1–2% of +its time. **Explained**; it is the fourth record running in which the +loader grows by a few percent per release. + +**Instantiating one object is 11–31% slower and evaluating one calc +10–14%**, three allocations more per operation each. The gRPC +`InstantiateWarmModel` row below shows the same +35% with +18% +allocations. An instance now carries the implicit relationship ends of +its type (`fa8a7220c`, `ca0e6cca0`) and its attribute values go through +the exact-number constructors (`79629923b`, `923ec81f0`); the bytes rise +11–13% on these small objects and fall on the stress constellation's +fleet, where the Integer and Rational fast paths' allocation-free +representation (`d1eb16edb`) wins. `RunCalc` was +18–20% on `ffe51571c`; +the memoized succession-cycle check (`8bd97fcf6`) took a third of that +off. **Explained.** + +**The compiled calcs pay for unbounded Integers, exact Rationals and the +step budget.** Each of the four fixtures is affected differently, and the +benchmark's own fixture file grew from 597 to 930 lines (the same four +calcs; the new content is other compiled fixtures), which the +`LoadModel`-sized setup absorbs but the `FormatEdits` row in `internal/frontend/lsp` does not: -- `Fib(25)/interpreted` is five times slower and allocates six times the - bytes and 5.8 times the objects. The profile shows where the time went: -`compiledCalc.invoke`, `callNode`, `conditionalNode` and `leafPair` -(the interpreted target is the closure-compiled calc evaluator) and -`scalarCheck.held` at 11% flat — the exact-Rational evaluation -(`923ec81f0`) checks the kind of every scalar a node yields and pairs the -leaves of each arithmetic node, 250 000 calls deep. Allocation counts rose -5.8× with it (the `big`-free fast path still allocates a result per -call), and the candidate's ±71% spread is the collector. The price of -exactness, but the steepest row in the record and the one a -profile-driven pass should look at first. -- `Collatz(27)/interpreted` is +60% with 2.1 times the allocations; - `Collatz(27)/go` is +56% with none on either side, and - `Hypot(3.0,4.0)/go` goes from 11.5 ns to 60 ns, also allocation-free. - The Go target's compiled values now -keep an Integer an Integer inside a Real-typed value and hold negative -powers as exact Rationals (`b69ea6840`, `0aa4aca55`), so each arithmetic -node carries its number kind and branches on it; `Hypot`'s four -operations went from 11.5 ns to 60 ns for it, still allocation-free, and -compiled calcs spend the interpreter's step budget (`d6eafc870`). +- `Fib(25)/interpreted` **is 20% faster than 0.9.2** with 7% more + allocations. On `ffe51571c` it was 5.3 times slower and allocated six + times the objects: the exact-Rational evaluation (`923ec81f0`) boxed + every scalar a node yielded and paired the leaves of each arithmetic + node, 250 000 calls deep. Packing a compiled calc's scalars into three + words (`4a1b937f2`) removed the boxing — 2 059 → 381 allocations per + call — and the row is now the fastest of the three records it appears + in. The steepest row of the earlier measurement is cleared. +- `Collatz(27)/interpreted` is +65% with 2.1 times the allocations, and + the fix did not reach it: `Collatz` multiplies and divides Integers, + and the unbounded-Integer path (`79629923b`) still allocates a result + per operation where `Fib`'s additions stay in the packed word (999 → + 2 115 allocations is one per arithmetic node). `Collatz(27)/go` is +59% + with none on either side, and `Hypot(3.0,4.0)/go` goes from 11 ns to + 56 ns, also allocation-free: the Go target's compiled values now keep + an Integer an Integer inside a Real-typed value and hold negative powers + as exact Rationals (`b69ea6840`, `0aa4aca55`), so each arithmetic node + carries its number kind and branches on it, and compiled calcs spend + the interpreter's step budget (`d6eafc870`). The price of exactness; the + Integer multiply path is where a profile-driven pass should look next. - `SumTo(1000000)` sums a million integers in a loop. On 0.9.2 the interpreted run took 1.55 s inside the default budget of 10 000 000 steps; on the candidate the same loop exceeds it, for the interpreted @@ -311,409 +326,454 @@ compiled calcs spend the interpreter's step budget (`d6eafc870`). `43548d741`). The row is not a timing regression but a change in what fits the default budget: a loop of a million iterations no longer does without `OPENSYSML_MAX_STEPS`. -- The C target refuses every calc that does Integer arithmetic - (`Fib`, `SumTo`, `Collatz`) because a C `int64` cannot hold the - interpreter's unbounded Integers (`79629923b`); only `Hypot`, on Reals, - still compiles to C. This is a deliberate narrowing of the C target, not - a performance figure, and belongs in the release notes. +- The C target refuses every calc that does Integer arithmetic (`Fib`, + `SumTo`, `Collatz`) because a C `int64` cannot hold the interpreter's + unbounded Integers (`79629923b`); only `Hypot`, on Reals, still + compiles to C. This is a deliberate narrowing of the C target, not a + performance figure, and belongs in the release notes. ### `internal/frontend/grpc` -| benchmark | 0.9.2 | 0.10.0 | -| --------- | ----- | ------ | -| `ParseFileColdShared` | 10.7 ms ±34% / 4.689 MiB / 31.12 k allocs | 7.67 ms ±21% (~) / 4.700 MiB (+0.2%) / 31.29 k (+0.5%) | -| `ParseFileColdInline` | 7.69 ms ±29% / 4.916 MiB / 31.47 k | 7.74 ms (~) / 4.932 MiB (~) / 31.69 k (+0.7%) | -| `InstantiateWarmModel` | 134 µs ±8% / 82.87 KiB / 1 555 allocs | 191 µs ±1% (+42%, p=0.002) / 104.13 KiB (+26%) / 1 831 (+18%) | +| benchmark | 0.9.2 | `ffe51571c` | 0.10.0 (`82f0fac87`) | +| --------- | ----- | ----------- | -------------------- | +| `ParseFileColdShared` | 10.7 ms ±34% / 4.689 MiB / 31.12 k allocs | 7.67 ms (~) | 7.61 ms ±6% (~, p=0.394) / 4.700 MiB (+0.2%) / 31.29 k (+0.5%) | +| `ParseFileColdInline` | 7.69 ms ±29% / 4.916 MiB / 31.47 k | 7.74 ms (~) | 7.66 ms ±8% (~, p=0.818) / 4.945 MiB (~) / 31.71 k (+0.8%) | +| `InstantiateWarmModel` | 150 µs ±12% / 83.06 KiB / 1 555 allocs | 211 µs (+41%) / 104.02 KiB / 1 838 | 198 µs ±10% (+32%, p=0.002) / 104.15 KiB (+25%) / 1 836 (+18%) | The cold-parse rows are the parser and the loader on the vehicle example -and move with the loader's few percent of allocations. The warm -instantiation is the `Instantiate` row of the REPL at the service's -scale: 276 more objects per instantiation. See the `Instantiate` cause under the -REPL: the same implicit relationship ends and exact-number constructors, -on the vehicle example's objects. +and move with the loader's few percent of allocations; the time is `~` on +every revision. The warm instantiation is the `Instantiate` row of the +REPL at the service's scale: 281 more objects per instantiation, the same +implicit relationship ends and exact-number constructors on the vehicle +example's objects. The cached-diagnostics fix (`70242d872`) does not +touch these rows; its effect is on `tests/perf`'s `GRPCParseFileCached` +and `ConnectParseFileHTTPCached` below. ### `internal/frontend/lsp` -| benchmark | 0.9.2 | 0.10.0 | -| --------- | ----- | ------ | -| `FormatEdits` | 515 µs ±11% / 433.7 KiB / 78 allocs | 871 µs ±18% (+69%, p=0.002) / 704.3 KiB (+62%) / 85 | -| `SpanToRangeLarge` | 2.99 ms | 2.87 ms (~) | -| `ReferencesWarm` | 206 µs | 211 µs (~) | -| `ReferencesCold` | 45.4 ms ±19% | 37.9 ms (~, p=0.065) | -| `RenameWarm` | 20.0 ms ±76% | 18.9 ms ±28% (~; −14.5% as found) | -| `WorkspaceUpdate` | 198 ms ±100% / 49.39 MiB / 605.6 k allocs | 270 ms ±16% (~, p=0.065) / 98.46 MiB (+99%) / 930.6 k (+54%) | +| benchmark | 0.9.2 | `ffe51571c` | 0.10.0 (`82f0fac87`) | +| --------- | ----- | ----------- | -------------------- | +| `FormatEdits` | 512 µs ±4% / 433.7 KiB / 78 allocs | 891 µs (+74%) / 704.3 KiB / 85 | 878 µs ±5% (+71%, p=0.002) / 704.3 KiB (+62%) / 85 (+9.0%) | +| `SpanToRangeLarge` | 2.99 ms ±17% / 0 B | — | 2.92 ms ±40% (~) / 0 B | +| `ReferencesWarm` | 206 µs ±7% / 339.4 KiB / 1 017 | — | 204 µs ±2% (~) / 339.4 KiB / 1 017 | +| `ReferencesCold` | 45.4 ms ±19% / 9.713 MiB / 44.51 k | — | 39.5 ms ±17% (~) / 9.720 MiB (~) / 44.51 k (~) | +| `RenameWarm` | 16.5 ms ±11% / 1.252 MiB / 12.75 k | 16.9 ms (~) | 16.5 ms ±6% (~, p=0.818) / 1.244 MiB (~) / 12.62 k (~) | +| `WorkspaceUpdate` | 194 ms ±7% / 49.38 MiB / 605.6 k | 262 ms (+35%) / 98.50 MiB (+99%) / 930.6 k (+54%) | 224 ms ±4% (+16%, p=0.002) / 57.18 MiB (+16%) / 710.0 k (+17%) | `FormatEdits` formats `internal/frontend/repl/testdata/compile_calcs.sysml`, which grew from 597 to 930 lines on the candidate with the new compiled -fixtures; the bytes move by the ratio of the file sizes (+62% against -+56% more lines) and the formatter itself has one commit in the interval -(the indexed connector ends, `63552cbc4`). **Not a regression in the -code.** +fixtures; the bytes move by the ratio of the file sizes (+62% against +56% +more lines) and the formatter itself has one commit in the interval (the +indexed connector ends, `63552cbc4`). **Not a regression in the code.** +References and rename are unchanged, to the allocation. `WorkspaceUpdate` — one edit to one of the workspace's documents, with the -re-analysis the server does for it — allocates twice the bytes and 1.5 -times the objects it did, and the time is within the machine's spread -only because the baseline's six counts ran 100% apart. The profile puts 39% of the bytes -(810 MB over the run) and 30% of the objects in one function new since -0.9.2, `symbols.registrationOf`, reached from `Index.noteBefore`: before -a name's registration changes, the index snapshots what a lookup of it -would read — the symbols under it, their re-export and hidden flags, and -a copy of every document's re-export claims and routes — so that a name -registered again exactly as it was does not count as a change -(`4ca321508`, `fe6e01fb9`). The snapshot is per name touched by the -re-registration, which on a workspace update is every name of the -document and of what imports it, and the claim maps are deep-copied. The -intent is the right one (fewer downstream invalidations); the copy is -avoidable. **Follow-up.** The rest of the growth is the resolver's new -`checkImportedNames` (`eb4b7583c`). +re-analysis the server does for it — is where the re-export fix landed. +On `ffe51571c` it allocated twice the bytes of 0.9.2 and 1.5 times the +objects, 39% of them in `symbols.registrationOf` deep-copying every +document's re-export claims before each name's re-registration +(`4ca321508`, `fe6e01fb9`); with the claims compared in place instead of +copied (`4b4b35121`) the row is **+16% time, +16% bytes and +17% +allocations** against 0.9.2 — 41 MiB per update less than `ffe51571c`, +and the first-pass spread of ±46% on 0.9.2 settled to ±7% interleaved. The +remainder is the resolver's new `checkImportedNames` (`eb4b7583c`) and +the loader's per-element growth, which the `tests/perf` and stress edit +rows below share. **Explained.** ### `internal/translate/migrate`, `internal/workspace/libs`, `internal/workspace/model` -| benchmark | 0.9.2 | 0.10.0 | -| --------- | ----- | ------ | -| `migrate WriterSiblingBlocks` | 1.93 ms | 1.87 ms (~); bytes and allocations identical | -| `libs ExpandWildcardImports` | 26.8 ms | 27.8 ms (~) | -| `libs IndexLibrary` | 13.8 ms / 13.56 MiB / 104.6 k allocs | 14.1 ms (~; +5.1% as found) / 14.07 MiB (+3.8%) / 108.2 k (+3.4%) | -| `libs ExpandModelImports` | 6.36 ms | 6.50 ms (~) | -| `libs DecodeSnapshot` | 9.52 ms / 35.32 MiB / 56.63 k allocs | 9.98 ms (+4.9%, p=0.009) / 37.85 MiB (+7.2%) / 67.45 k (+19%) | -| `libs SetDigest` | 629 µs | 624 µs (~) | -| `model AnalyseUnresolved` | 20.2 ms | 19.9 ms (~) | -| `model AnalyseResolved` | 1.26 ms / 543.0 KiB / 5 049 allocs | 1.39 ms (+10%, p=0.002) / 590.4 KiB (+8.7%) / 5 971 (+18%) | - -The migration writer is unchanged. Indexing the standard library -allocates 3.4% more objects (the reflective relationship ends, the -`StateActivity` extension library `0188fb308`), and decoding its snapshot +| benchmark | 0.9.2 | 0.10.0 (`82f0fac87`) | +| --------- | ----- | -------------------- | +| `migrate` `WriterSiblingBlocks` | 1.93 ms ±7% / 1.906 MiB / 20.02 k allocs | 1.90 ms ±9% (~) / 1.906 MiB (~) / 20.02 k (~) | +| `libs` `ExpandWildcardImports` | 26.8 ms ±10% / 14.46 MiB / 128.3 k | 31.0 ms ±20% (+15%, p=0.009, first pass) / 15.36 MiB (+6.2%) / 145.7 k (+14%) | +| `libs` `IndexLibrary` | 16.3 ms ±15% / 13.56 MiB / 104.6 k | 16.2 ms ±14% (~, p=0.818) / 14.07 MiB (+3.8%) / 108.2 k (+3.4%) | +| `libs` `ExpandModelImports` | 6.36 ms ±4% / 4.315 MiB / 36.16 k | 6.93 ms ±30% (+9.0%, p=0.002, first pass) / 4.376 MiB (+1.4%) / 38.15 k (+5.5%) | +| `libs` `DecodeSnapshot` | 9.89 ms ±17% / 35.32 MiB / 56.63 k | 10.8 ms ±21% (~, p=0.132) / 37.85 MiB (+7.2%) / 67.45 k (+19%) | +| `libs` `SetDigest` | 629 µs ±12% / 1.710 MiB / 966 | 641 µs ±7% (~) / 1.733 MiB (+1.4%) / 998 (+3.3%) | +| `model` `AnalyseUnresolved` | 20.2 ms ±8% / 13.69 MiB / 149.3 k | 20.6 ms ±6% (~) / 13.80 MiB (+0.8%) / 150.7 k (+1.0%) | +| `model` `AnalyseResolved` | 1.51 ms ±22% / 543.0 KiB / 5 049 | 1.59 ms ±12% (~, p=0.699) / 590.4 KiB (+8.7%) / 5 971 (+18%) | + +None of these rows moved between `ffe51571c` and the candidate, to the +allocation. The migration writer is unchanged. Indexing the standard +library allocates 3.4% more objects (the reflective relationship ends, the +`StateActivity` extension library `0188fb308`) and decoding its snapshot 19% more: the snapshot carries the owned-relationship journal and the -reflected implicit relationships (`da8fba4a3`, `32890be28`) that the -0.9.2 snapshot did not hold. Analysing a resolved document is +10% with -18% more allocations: the constraint-tier passes added in the interval — -the indexed-end multiplicity check (`cf85beb36`), the import-ambiguity -warning (`eb4b7583c`), the reflective derivations for the batch -constraints (`70bfa8ee2`) — each walk the document once more. The -`tests/perf` `Analyze` rows below show the same +4.7% allocations on the +reflected implicit relationships (`da8fba4a3`, `32890be28`) that the 0.9.2 +snapshot did not hold; the time of both is `~` interleaved. Expanding the +library's wildcard imports allocates 14% more objects and was +15% in the +first pass with the candidate's counts 20% apart; the import-ambiguity +rule (`eb4b7583c`) now walks what each wildcard brings in, and the row was +not re-run. Analysing a resolved document allocates 18% more with the time +`~` interleaved (first pass +12%): the constraint-tier passes added in the +interval — the indexed-end multiplicity check (`cf85beb36`), the +import-ambiguity warning (`eb4b7583c`), the reflective derivations for the +batch constraints (`70bfa8ee2`) — each walk the document once more. The +`tests/perf` `Analyze` rows below show the same +3.3% allocations on the synthetic model and much more on the vehicle example. ### `tests/perf` -37 rows on 0.9.2 and 45 on the candidate as found (the eight `ImportValues` -rows are candidate-only); 48 and 56 once the vehicle corpus was present, -and the whole package was re-run interleaved. Of the 48 compared rows 20 -are `~` in time; the rows that moved, and the rows whose allocations moved: - -| benchmark | 0.9.2 | 0.10.0 | -| --------- | ----- | ------ | -| `Lex`, `Parse` (synthetic, vehicle) | | (~) time; bytes and allocations identical | -| `IndexAddExpand/synthetic` | 42.5 ms ±41% / 39.70 MiB / 290.8 k allocs | 44.9 ms (~) / 42.45 MiB (+6.9%) / 315.5 k (+8.5%) | -| `IndexAddOnly/synthetic` | 39.7 ms / 37.59 MiB / 281.2 k | 41.0 ms (~) / 40.34 MiB (+7.3%) / 305.9 k (+8.8%) | -| `IndexAddOnly/vehicle` | 1.07 ms / 1.091 MiB / 9 276 | 1.06 ms (~) / 1.138 MiB (+4.3%) / 9 632 (+3.8%) | -| `Analyze/synthetic` | 1.13 s ±17% / 466.1 MiB / 4.435 M | 1.24 s ±29% (+9.2%, p=0.026) / 487.1 MiB (+4.5%) / 4.644 M (+4.7%) | -| `Analyze/vehicle` | 118 ms ±20% / 73.64 MiB / 627.5 k | 193 ms ±44% (+64%, p=0.002) / 130.0 MiB (+77%) / 865.1 k (+38%) | -| `WorkspaceEdit/synthetic` | 204 ms ±74% / 77.98 MiB / 596.3 k | 256 ms ±42% (~) / 85.73 MiB (+9.9%) / 680.6 k (+14%) | -| `WorkspaceEdit/synthetic/reindex+diagnostics` | 1.52 s ±60% / 493.2 MiB / 4.787 M | 1.62 s (~) / 519.7 MiB (+5.4%) / 5.090 M (+6.3%) | -| `WorkspaceEdit/vehicle` | 52.0 ms / 12.33 MiB / 139.1 k | 66.0 ms (+27%, p=0.002) / 22.95 MiB (+86%) / 214.4 k (+54%) | -| `WorkspaceEdit/vehicle/reindex+diagnostics` | 167 ms / 71.25 MiB / 651.7 k | 223 ms ±27% (+34%, p=0.002) / 129.8 MiB (+82%) / 917.6 k (+41%) | -| `WorkspaceEditSmallDocBesideLarge` | 42.4 ms ±51% / 7.952 MiB / 81.74 k | 60.8 ms ±64% (+43%, p=0.041) / 31.24 MiB (+293%) / 162.6 k (+99%) | -| `FQNOf` | 1.37 ms ±23% / 36 000 symbols | 1.52 ms (~) / 40 000 symbols (+11%); 38.0 ns against 38.1 ns per symbol (~) | -| `LookupQualified` | 1.39 ms ±18% | 1.18 ms (−15%, p=0.041) | -| `FeaturesOf` | 931 ms ±55% | 945 ms (~); bytes and allocations identical | -| `REPLLoadFile` | 1.32 s ±26% / 595.5 MiB / 5.383 M | 1.48 s (~, p=0.065) / 616.5 MiB (+3.5%) / 5.595 M (+4.0%) | -| `REPLSubmitSnippet` | 1.03 s / 513.6 MiB / 4.841 M | 1.11 s (+7.6%, p=0.015) / 545.7 MiB (+6.3%) / 5.048 M (+4.3%) | -| `REPLEvalExpr` | 4.50 ms / 101 allocs | 4.46 ms (~) / 104 | -| `LowerActionGraph` | 1.73 µs / 2.625 KiB / 25 allocs | 2.01 µs (+16%, p=0.002) / 2.906 KiB (+11%) / 27 | -| `LowerStateGraph` | 8.30 µs | 8.08 µs (~); bytes −0.5% | -| `LowerChain/chain10, 100, 1000` | 8.44 µs, 145 µs, 8.94 ms | 9.53 µs (+13%), 150 µs (~), 8.55 ms (~); bytes +13–15% | -| `ExecuteAction` | 75.3 µs / 192 allocs | 78.9 µs (+4.9%, p=0.004) / 253 (+32%) | -| `ExecuteActionFreshContext` | 30.4 µs ±22% / 26.26 KiB / 268 | 39.5 µs ±4% (+30%, p=0.002) / 34.94 KiB (+33%) / 347 (+29%) | -| `ExecuteState` | 704 µs / 483.4 KiB / 10.38 k | 838 µs (+19%, p=0.002) / 608.2 KiB (+26%) / 11.66 k (+12%) | -| `ActionLoop/for10` | 25.0 µs / 20.85 KiB / 204 | 33.9 µs (+36%) / 28.37 KiB (+36%) / 273 (+34%) | -| `ActionLoop/for100` | 101 µs / 47.97 KiB / 477 | 152 µs (+50%) / 70.26 KiB (+46%) / 816 (+71%) | -| `ActionLoop/for1000` | 850 µs / 279.0 KiB / 3 181 | 1 290 µs (+52%) / 449.2 KiB (+61%) / 6 220 (+96%) | -| `ActionChain/chain10` | 88.6 µs / 675 allocs | 101 µs (+14%) / 784 (+16%) | -| `ActionChain/chain100` | 562 µs / 522.8 KiB / 5 757 | 696 µs (+24%) / 705.9 KiB (+35%) / 6 595 (+15%) | -| `ActionChain/chain1000` | 13.0 ms / 5.153 MiB / 56.28 k | 14.5 ms (+11%) / 7.077 MiB (+37%) / 64.77 k (+15%) | -| `StateLoop/count50, 500, 5000` | 653 µs, 6.23 ms, 61.6 ms | 798 µs (+22%), 7.92 ms (+27%), 78.4 ms (+27%); bytes +26%, allocs +12% at every size | -| `BatchConstraints` | 8.97 ms ±31% / 2.223 MiB / 55.19 k | 15.1 ms ±11% (+68%, p=0.002) / 2.387 MiB (+7.4%) / 55.86 k (+1.2%) | -| `SameConstraintManyInstances` | 3.56 µs / 1 016 B | 4.12 µs (+16%, p=0.002) / 1 000 B (−1.6%) | -| `BatchSatisfy` | 1.18 s / 417.0 MiB / 4.211 M | 1.29 s (+9.5%, p=0.009) / 427.2 MiB (+2.4%) / 4.357 M (+3.5%) | -| `Instantiate` | 508 ms ±102% / 173.9 MiB ±57% | 529 ms ±6% (~) / 180.2 MiB (~, p=0.065) | -| `GRPCParseFileCached` | 58.8 µs / 216.7 KiB / 22 allocs | 69.7 µs (+19%, p=0.002) / 242.1 KiB (+12%) / 299 (13.6×) | -| `GRPCParseFileUncached` | 124 ms ±31% / 81.41 MiB / 660.5 k | 210 ms ±5% (+69%, p=0.002) / 142.1 MiB (+74%) / 905.1 k (+37%) | -| `GRPCEvaluate` | 8.97 µs / 8.426 KiB / 110 | 9.75 µs (+8.7%, p=0.009) / 9.406 KiB (+12%) / 116 | -| `GRPCVerifyConstraint` | 795 µs / 510.6 KiB / 9 188 | 958 µs (+21%, p=0.002) / 541.8 KiB (+6.1%) / 9 768 (+6.3%) | -| `ConnectEvaluateHTTP` | 150 µs / 201 allocs | 153 µs (~) / 206 | -| `ConnectParseFileHTTPCached` | 251 µs / 352.7 KiB / 166 allocs | 882 µs (+252%, p=0.002) / 488.3 KiB (+38%) / 1 292 (7.8×) | -| `ImportValues/flat/parts=100, 500, 2000 (attrs=3)` | — | 21.3 ms, 104 ms, 548 ms; 300, 1 500, 6 000 values | -| `ImportValues/flat/parts=2000/attrs=5` | — | 844 ms ±37%; 10 000 values | -| `ImportValues/nested/parts=100, 500, 2000 (attrs=3)` | — | 29.4 ms, 141 ms, 620 ms | -| `ImportValues/nested/parts=2000/attrs=5` | — | 847 ms | -| geomean (compared rows) | | +18% time | - -Four groups, each with a cause of its own: - -1. **Execution.** Every row that runs an action or state machine is - slower by a constant factor per step: an action loop iteration +36–52% - with 1.3–2 times the allocations, a state-machine event +19–27% with - +12% allocations, a succession chain +11–24%, a fresh action context - +30%, and `ExecuteAction` +4.9% with 32% more allocations. The - `internal/exec/runtime` rows above (`AssignmentLoopStep` +70%) and the - REPL's `RunCalc` (+20%) are the same cost. The runtime profile (under -`internal/exec/runtime` above) splits the new work three ways: the -frames built per statement to track body feature bindings -(`a4e1f6ef9`), the exploration of a body's statement orders -(`71fda4cd5`, `807fcf591`, `c8891457a`), and control-node values held -with their flow (`a0f77861d`); the description string the binding check -builds on every passing write is the avoidable part. `LowerActionGraph` -+16% and `LowerChain/chain10` +13% are the lowering carrying the -succession orders the executor explores. -2. **Constraint checking.** `BatchConstraints` is +68% per check with - +1.2% allocations, `SameConstraintManyInstances` +16% with fewer bytes, - `GRPCVerifyConstraint` +21%, `BatchSatisfy` +9.5%. The profile of `BatchConstraints` -puts 10% of the candidate's CPU in `Context.successionCycle`, a function -that did not exist: ordering performances by outside successions -(`72498ad90`) walks the succession graph when a constraint's body is -evaluated, on every check, for a model whose successions do not change -between checks. The remainder is the exact-number dispatch on the -evaluation path (`923ec81f0`, `79629923b`) and the reflective -derivations the constraint passes read (`70bfa8ee2`). The walk is -memoizable per model. **Follow-up.** -3. **The gRPC parse path.** `ParseFile` on a document the service has - already parsed allocated 22 objects on 0.9.2 and allocates 299 on the - candidate (+19% time); through Connect it is 3.5 times slower, and an - uncached parse is +69% time and +74% bytes. The service's `ParseSources` - now names the documents an edit reached and re-analyzes only those - (`fcc123b40`, `2d921fae9`, `af2604ba8`): the cached path that used to - return a stored result now computes the affected set, and the uncached - path carries the lineage bookkeeping. That is not where the allocations are, -though. The profile puts 70% of the cached path's objects in -`DiagnosticToProto` and `sourceSpanToProto`, called from -`modelDiagnostics`: the vehicle example now carries 134 warnings -(below), and each `ParseFile` converts every one of them to protobuf -again — the 22 → 299 allocations are 2 per warning plus the response. -The service reports the parser's and resolver's warnings as the -workspace does (`71d4bacea`), which it did not in 0.9.2; the cost is the -number of warnings times the number of calls, and would be gone with -either a cached conversion or no warnings on this model. -4. **Workspace edits and analysis.** Editing a document beside others - allocates 1.9–3.9 times the bytes it did (`WorkspaceEdit/vehicle` +86%, - `WorkspaceEditSmallDocBesideLarge` +293%; the LSP's `WorkspaceUpdate` - +99% and the stress constellation's `EditBeside` +143–280% are the same - path), and analysing the vehicle example — a model dense in connectors, - bindings, flows and decimal literals — is +64% time and +38% - allocations against +4.7% on the synthetic model. Two causes. The bytes are -`symbols.registrationOf` (see `WorkspaceUpdate` above): the stress -`EditBeside` profile puts 15% of the bytes and 13% of the objects there -on a 32-satellite constellation, and the share grows with the model -beside the edit (`EditBeside` +143% → +280% bytes from 32 to 512 -satellites). The vehicle rows have a second cause: 0.9.2 validated the -Annex A `SimpleVehicleModel` with no diagnostic, and the candidate -reports 134 warnings, `Duplicate of imported member name '…'`, one per -member that `VehicleConfiguration_b::**` and -`VehicleConfiguration_b::PartsTree::**` both bring into the same -namespace (`eb4b7583c`, the import-ambiguity rule). The profile of -`Analyze/vehicle` shows the rule's `checkImportedNames` (21% of bytes) -and the `suggest.EditDistance` tables each diagnostic builds (12% of -objects) as the whole of the growth; the synthetic model, which has no -such imports, grows 4.7%. Whether a recursive import should warn about -a member it also reaches through a nested recursive import is for the -semantics to decide; until it does, every tool that analyses this model -pays 134 diagnostics' worth of work per analysis, and every gRPC client -receives them. - -The import of values from CSV, TSV and JSON files (`ImportValues`) is new -in 0.10.0; 2 000 parts take 0.55–0.85 s flat and 0.62–0.85 s nested, which -is 90–280 µs per imported value. The first record to carry these rows, -they are the baseline for the next. - -### `tests/stressmodel` - -Generated satellite constellations at 32, 128 and 512 satellites (6 227, -24 167 and 95 927 elements), one document each; the first pass left every -time row in doubt (0.9.2's counts ran 10–60% apart), so the whole package -was re-run interleaved and the re-run is shown. 36 rows compared; six -`AnalyzeSplitPerDocument` rows are 0.9.2-only (above) and -`ConvertAPIJSON` has no distribution on either side. - -| benchmark | 0.9.2 | 0.10.0 | -| --------- | ----- | ------ | -| `Load/satellites=32` | 346 ms / 132.7 MiB / 1.994 M allocs | 415 ms (+20%) / 162.1 MiB (+22%) / 2.365 M (+19%) | -| `Load/satellites=128` | 1.38 s / 501.3 MiB / 7.682 M | 1.60 s (+16%) / 579.1 MiB (+16%) / 9.040 M (+18%) | -| `Load/satellites=512` | 6.11 s / 1.934 GiB / 30.43 M | 7.12 s (+16%) / 2.208 GiB (+14%) / 35.74 M (+17%) | -| `Satisfy/satellites=32` (96 assertions) | 4.10 ms / 1.665 MiB / 28.90 k | 4.83 ms (+18%) / 1.831 MiB (+10%) / 34.57 k (+20%) | -| `Satisfy/satellites=128` (384) | 17.3 ms / 6.652 MiB | 20.7 ms (+20%) / 7.316 MiB (+10%) / +20% allocs | -| `Satisfy/satellites=512` (1 536) | 82.8 ms ±7% / 26.62 MiB | 104 ms ±7% (+26%) / 29.27 MiB (+10%) / +20% allocs | -| `FleetInstantiate/satellites=32` (1 179 elements) | 4.06 ms / 7.987 MiB / 12.72 k | 4.94 ms (+21%) / 7.417 MiB (−7.1%) / 13.96 k (+9.7%) | -| `FleetInstantiate/satellites=128` (1 715) | 11.4 ms / 24.10 MiB / 48.37 k | 14.6 ms (+29%) / 24.45 MiB (+1.4%) / 52.97 k (+9.5%) | -| `FleetInstantiate/satellites=512` (3 947) | 59.8 ms / 160.2 MiB ±7% / 190.9 k | 82.9 ms ±9% (+39%) / 134.1 MiB (−16%) / 209.0 k (+9.5%) | -| `FleetSatisfy/satellites=32, 128, 512` | 1.01 ms, 1.60 ms, 5.06 ms | 1.19 ms (+18%), 1.88 ms (+17%), 6.30 ms (+24%); +8.8–9.2% B, +17–18% allocs | -| `EditBeside/satellites=32` | 1.05 ms / 295.5 KiB / 3 442 | 1.32 ms (+26%) / 719.5 KiB (+143%) / 5 533 (+61%) | -| `EditBeside/satellites=128` | 3.21 ms / 719.9 KiB / 8 377 | 4.21 ms (+31%) / 2 435.7 KiB (+238%) / 15 922 (+90%) | -| `EditBeside/satellites=512` | 12.9 ms / 2.408 MiB / 28.11 k | 16.9 ms (+31%) / 9.147 MiB (+280%) / 57.44 k (+104%) | -| `ValidateSplit/satellites=32/files=6/jobs=1` | 508 ms / 186.1 MiB / 2.480 M | 572 ms (+13%) / 217.5 MiB (+17%) / 2.898 M (+17%) | -| `ValidateSplit/satellites=32/files=6/jobs=8` | 236 ms | 253 ms (+7.0%) / +17% B / +17% allocs | -| `ValidateSplit/satellites=128/files=6/jobs=1, 8` | 1.79 s, 737 ms | 2.04 s (+14%), 840 ms ±9% (+14%) / +15% B / +17% allocs | -| `ValidateSplit/satellites=512/files=6/jobs=1` | 7.39 s ±18% / 2.335 GiB / 33.69 M | 8.34 s ±2% (~, p=0.065) / 2.646 GiB (+13%) / 39.59 M (+18%) | -| `ValidateSplit/satellites=512/files=6/jobs=8` | 2.90 s ±18% | 3.48 s ±2% (+20%, p=0.004) / +13% B / +18% allocs | -| `LoadFiles/satellites=32/files=6` | 435 ms ±20% / 147.7 MiB / 1.971 M | 508 ms ±1% (~, p=0.065) / 176.8 MiB (+20%) / 2.365 M (+20%) | -| `LoadFiles/satellites=128, 512/files=6` | 1.62 s, 6.66 s ±10% | 1.85 s (+14%), 7.69 s (+16%) / +16–18% B / +21% allocs | -| `EditImported/satellites=32, 128, 512/files=6` | 473 ms ±43%, 1.60 s ±44%, 6.54 s ±36% | 532 ms ±2%, 1.82 s ±3%, 7.45 s ±4% (all ~) / +19–30% B / +22–25% allocs | -| `OpenSplit/satellites=32/files=6/cold` | 228 ms ±11% / 169.1 MiB / 2.362 M | 239 ms ±8% (+4.8%, p=0.041) / 199.9 MiB (+18%) / 2.777 M (+18%) | -| `OpenSplit/satellites=32/files=6/warm` | 96.6 ms ±13% / 65.62 MiB / 571.1 k | 122 ms ±39% (+27%) / 86.03 MiB (+31%) / 706.8 k (+24%) | -| `OpenSplit/satellites=128/files=6/cold` | 693 ms ±13% / 541.3 MiB | 804 ms ±6% (+16%, p=0.009) / 632.2 MiB (+17%) / +18% allocs | -| `OpenSplit/satellites=128/files=6/warm` | 180 ms ±7% / 166.6 MiB / 1.385 M | 250 ms ±45% (+39%) / 215.2 MiB (+29%) / 1.730 M (+25%) | -| `OpenSplit/satellites=512/files=6/cold` | 2.70 s ±5% / 2.042 GiB | 3.35 s ±17% (+24%, p=0.009) / 2.355 GiB (+15%) / +19% allocs | -| `OpenSplit/satellites=512/files=6/warm` | 586 ms ±14% / 613.4 MiB / 4.639 M | 849 ms ±113% (+45%) / 897.7 MiB (+46%) / 5.821 M (+25%) | -| `HydratePlane/satellites=32/files=6` | 85.5 ms ±9% / 25.80 MiB / 314.8 k | 77.7 ms ±10% (−9.1%, p=0.015) / 25.67 MiB (−0.5%) / 278.5 k (−12%) | -| `HydratePlane/satellites=128/files=6` | 224 ms ±23% / 76.61 MiB / 919.8 k | 202 ms ±19% (~) / 62.71 MiB (−18%) / 680.3 k (−26%) | -| `HydratePlane/satellites=512/files=6` | 932 ms ±3% / 287.2 MiB / 3.336 M | 767 ms ±64% (~, p=0.394; −37% as found) / 214.2 MiB (−25%) / 2.287 M (−31%) | -| `ConvertAPIJSON/satellites=32` (as found) | 7.86 s ±4% / 5.021 GiB / 40.91 M | 6.69 s ±4% (−15%) / 5.156 GiB (+2.7%) / 44.96 M (+9.9%) | -| `ConvertAPIJSON/satellites=128` (as found) | 31.9 s ±4% / 19.69 GiB / 157.8 M | 31.2 s ±2% (~) / 20.22 GiB (+2.7%) / 173.4 M (+9.9%) | -| `ConvertAPIJSON/satellites=512` | 156 s, one count, then killed | 119–122 s, five counts; no distribution on either side | -| `AnalyzeSplitPerDocument/satellites=32, 128, 512/files=6/jobs=1, 8` | 2.75 ms, 11.6 ms, 50.4 ms (jobs=1); 1.23 ms, 2.26 ms, 10.7 ms (jobs=8) | fails: the generated model's decimal literals are reported as rounded | -| geomean (36 compared rows) | 167 ms / 78.87 MiB / 911.8 k | 195 ms (+17%) / 95.95 MiB (+22%) / 1.081 M (+19%) | - -Unless marked, every time row is p=0.002 over six interleaved counts; -every byte and allocation row is p=0.002 with 0% spread. - -The constellation says in one place what the packages say severally: - -- **Loading is +16–20% time, +14–22% bytes and +17–19% allocations per - element**, at every size, in every path that loads (`Load`, - `LoadFiles`, `ValidateSplit`, `OpenSplit/cold`, `EditImported`'s bytes). - This is more than the REPL's `LoadModel` (+2.5–3.1% bytes) because the - constellation is nothing but parts, ports, connections and flows between - them, and it is the connector, binding and flow ends whose - representation changed: their ends are owned by the connector and - journaled as owned relationships (`00b8499a8`, `da8fba4a3`), inherited - ends are replaced positionally, flows are paired by end, each end may - carry an index (`63552cbc4`, `7514a97bf`), and every implicit - relationship the resolver derives is reflected as a metaobject - (`32890be28`). A model of 95 927 elements holds 5.3 million more objects - for it and takes a second longer to load. **Explained**, and the largest - single cost of the release. -- **Satisfying requirements is +18–26% time, +10% bytes and +20% - allocations per assertion**, the same as `tests/perf`'s - `GRPCVerifyConstraint` (+21%) and `BatchConstraints`; the fleet - variant +17–24%. See the constraint-checking cause under `tests/perf`. -- **Instantiating a fleet is +21–39% time with 9.5% more allocations and - up to 16% fewer bytes**: the same `Instantiate` cost as the REPL and - gRPC rows (+18–42%), with the bytes going the other way because the - instantiated objects' attribute storage is smaller (the - allocation-free Integer and Rational fast paths, `d1eb16edb`). -- **Editing a small document beside the constellation is +26–31% time - and 2.4–3.8 times the bytes**, growing with the size of the document - beside it — the edited document is 84 bytes and does not change. The - bytes scale with the *other* document: the workspace's re-analysis after - an edit now re-derives something proportional to the whole workspace. - See the workspace-edit cause under `tests/perf`. -- **Opening a workspace warm** (the documents already analysed once) is - +27–45% with spreads of 39–113% on the candidate, the only rows whose - spread the re-run did not settle; the bytes (+29–46%) are the loader's - growth plus the edit path's. -- **Hydrating a plane is 9–37% faster with 12–31% fewer allocations** — - the one improvement at scale, from the indexed connector ends: the - hydrator finds an end by index rather than by scanning. -- **Converting to API JSON is 15% faster at 32 satellites** with 9.9% more - allocations (`fcc123b40`'s affected-set bookkeeping applies here too), - unchanged at 128; at 512 the first-pass run was killed on 0.9.2 after - its second count (78 GiB allocated per operation) and the candidate's - five counts of 119–122 s are reported for what they are. +The synthetic 4 000-part model and the Annex A vehicle example through +every entrance: lexer, parser, index, analysis, workspace edits, REPL, +lowering, execution, constraints, the gRPC service in-process and over +HTTP. 49 rows on 0.9.2 and 57 on the candidate (the eight `ImportValues` +rows are new). The first pass left most of the package in doubt, so the +whole package was re-run interleaved across the three revisions; that +re-run is the table. The vehicle rows need the pilot corpus, which 0.9.2's +first pass predates, so those have the re-run only. + +| benchmark | 0.9.2 | `ffe51571c` | 0.10.0 (`82f0fac87`) | +| --------- | ----- | ----------- | -------------------- | +| `Lex/synthetic` | 8.16 ms ±14% / 483.0 k allocs | 7.84 ms (~) | 7.97 ms ±6% (~) / 483.0 k | +| `Lex/vehicle` | 277 µs ±17% / 10.85 k | 268 µs (~) | 272 µs ±8% (~) / 10.85 k | +| `Parse/synthetic` | 99.0 ms ±16% / 293.2 k | 89.0 ms (~) | 88.5 ms ±25% (~) / 293.2 k | +| `Parse/vehicle` | 2.51 ms ±12% / 7 559 | 2.59 ms (~) | 2.49 ms ±9% (~) / 7 559 | +| `IndexAddExpand/synthetic` | 51.4 ms ±19% / 294.6 k | 51.7 ms (~) | 51.6 ms ±8% (~) / 317.6 k (+7.8%) | +| `IndexAddExpand/vehicle` | 27.3 ms ±33% / 131.2 k | 27.7 ms (~) | 28.1 ms ±18% (~) / 139.7 k (+6.5%) | +| `IndexAddOnly/synthetic` | 45.8 ms ±20% / 283.0 k | 47.5 ms (~) | 48.0 ms ±17% (~) / 306.6 k (+8.3%) | +| `IndexAddOnly/vehicle` | 1.14 ms ±10% / 9 277 | 1.13 ms (~) | 1.18 ms ±11% (~) / 9 633 (+3.8%) | +| `Analyze/synthetic` | 1.26 s ±20% / 466.3 MiB / 4.435 M | 1.33 s (~) / 487.2 MiB / 4.644 M | 1.35 s ±21% (+7.5%, p=0.041) / 484.5 MiB (+3.9%) / 4.582 M (+3.3%) | +| `Analyze/vehicle` | 117 ms ±17% / 73.65 MiB / 627.6 k | 221 ms (+89%) / 130.0 MiB / 865.3 k | 216 ms ±22% (+85%, p=0.002) / 129.9 MiB (+76%) / 869.3 k (+39%) | +| `WorkspaceEdit/synthetic` | 210 ms ±10% / 77.97 MiB / 596.3 k | 278 ms (+32%) / 85.84 MiB / 680.6 k | 285 ms ±68% (+35%, p=0.002) / 85.74 MiB (+10%) / 680.5 k (+14%) | +| `WorkspaceEdit/synthetic/reindex+diagnostics` | 1.62 s ±5% / 493.5 MiB / 4.787 M | 1.82 s (+13%) / 519.6 MiB / 5.089 M | 1.80 s ±16% (+11%, p=0.002) / 517.4 MiB (+4.8%) / 5.027 M (+5.0%) | +| `WorkspaceEdit/vehicle` | 56.5 ms ±12% / 12.33 MiB / 139.1 k | 78.6 ms (+39%) / 22.94 MiB (+86%) / 214.4 k (+54%) | 67.9 ms ±11% (+20%, p=0.002) / 14.60 MiB (+18%) / 168.1 k (+21%) | +| `WorkspaceEdit/vehicle/reindex+diagnostics` | 176 ms ±7% / 71.24 MiB / 651.7 k | 262 ms (+49%) / 129.7 MiB (+82%) / 917.6 k (+41%) | 250 ms ±10% (+42%, p=0.002) / 121.0 MiB (+70%) / 867.1 k (+33%) | +| `WorkspaceEditSmallDocBesideLarge` | 45.4 ms ±34% / 7.952 MiB / 81.74 k | 69.5 ms (+53%) / 31.24 MiB (+293%) / 162.6 k (+99%) | 59.4 ms ±35% (+31%, p=0.026) / 23.67 MiB (+198%) / 114.6 k (+40%) | +| `FQNOf` | 1.59 ms ±32% / 36.00 k | 1.68 ms (~) | 1.62 ms ±12% (~) / 40.00 k (+11%) | +| `LookupQualified` | 1.22 ms ±34% / 11.00 k | 1.26 ms (~) | 1.26 ms ±14% (~) / 11.00 k | +| `FeaturesOf` | 1.01 s ±11% / 4.154 M | 1.03 s (~) | 1.06 s ±13% (~) / 4.156 M (~) | +| `REPLLoadFile` | 1.53 s ±7% / 595.5 MiB / 5.383 M | 1.63 s (~) / 616.7 MiB / 5.595 M | 1.64 s ±10% (~, p=0.065) / 613.0 MiB (+2.9%) / 5.533 M (+2.8%) | +| `REPLSubmitSnippet` | 1.10 s ±9% / 4.805 M | 1.22 s (+11%) / 5.048 M | 1.25 s ±18% (+14%, p=0.026) / 4.986 M (+3.8%) | +| `REPLEvalExpr` | 4.76 ms ±15% / 101 | 4.66 ms (~) | 4.86 ms ±8% (~) / 104 (+3.0%) | +| `LowerActionGraph` | 1.85 µs ±11% / 25 | 2.16 µs (+17%) | 2.15 µs ±7% (+16%, p=0.004) / 27 (+8.0%) | +| `LowerStateGraph` | 9.02 µs ±13% / 103 | 8.48 µs (~) | 9.08 µs ±17% (~) / 103 | +| `ExecuteAction` | 79.9 µs ±10% / 192 | 86.7 µs (~) / 254 | 80.4 µs ±5% (~, p=0.937) / 234 (+22%) | +| `ExecuteActionFreshContext` | 31.0 µs ±16% / 268 | 41.8 µs (+35%) / 347 | 42.8 µs ±9% (+38%, p=0.002) / 327 (+22%) | +| `ExecuteState` | 729 µs ±12% / 483.8 KiB / 10.38 k | 903 µs (+24%) / 608.9 KiB / 11.66 k | 880 µs ±6% (+21%, p=0.002) / 607.5 KiB (+26%) / 11.61 k (+12%) | +| `BatchConstraints` | 9.83 ms ±26% / 2.241 MiB / 55.26 k | 16.5 ms (+68%) / 2.424 MiB (+8.1%) / 55.99 k (+1.3%) | 10.5 ms ±4% (~, p=0.093) / 2.240 MiB (~) / 55.35 k (+0.2%) | +| `SameConstraintManyInstances` | 3.85 µs ±11% / 48 | 4.40 µs (+14%) | 4.38 µs ±6% (+14%, p=0.015) / 48 | +| `BatchSatisfy` | 1.27 s ±13% / 4.211 M | 1.33 s (~) | 1.33 s ±6% (~, p=0.310) / 4.354 M (+3.4%) | +| `Instantiate` | 525 ms ±12% / 174.0 MiB / 1.871 M | 574 ms (~) | 563 ms ±16% (~, p=0.240) / 180.2 MiB (+3.5%) / 1.955 M (+4.5%) | +| `GRPCParseFileCached` | 63.4 µs ±23% / 216.7 KiB / 22 | 77.5 µs (+22%) / 242.0 KiB (+12%) / 299 (+1 259%) | 66.0 µs ±5% (~, p=0.485) / 218.0 KiB (+0.6%) / 24 (+9.1%) | +| `GRPCParseFileUncached` | 131 ms ±12% / 661.3 k | 225 ms (+71%) / 905.1 k | 215 ms ±20% (+64%, p=0.002) / 909.0 k (+37%) | +| `GRPCEvaluate` | 9.04 µs ±7% / 110 | 10.6 µs (+18%) | 10.1 µs ±13% (+11%, p=0.004) / 116 (+5.5%) | +| `GRPCVerifyConstraint` | 831 µs ±16% / 9 188 | 1 047 µs (+26%) | 1 009 µs ±43% (+21%, p=0.009) / 9 768 (+6.3%) | +| `ConnectEvaluateHTTP` | 153 µs ±6% / 201 | 151 µs (~) | 153 µs ±4% (~) / 206 (+2.5%) | +| `ConnectParseFileHTTPCached` | 271 µs ±7% / 351.6 KiB / 166 | 926 µs (+241%) / 491.7 KiB / 1 292 | 879 µs ±24% (+224%, p=0.002) / 465.2 KiB (+32%) / 1 017 (+513%) | +| `ActionLoop/for10` | 25.8 µs ±10% / 204 | 36.5 µs (+41%) / 273 | 35.7 µs ±19% (+38%, p=0.002) / 253 (+24%) | +| `ActionLoop/for100` | 104 µs ±31% / 477 | 166 µs (+59%) / 816 | 163 µs ±17% (+56%, p=0.002) / 616 (+29%) | +| `ActionLoop/for1000` | 954 µs ±14% / 279.0 KiB / 3 181 | 1 369 µs (+44%) / 449.3 KiB (+61%) / 6 220 (+96%) | 1 332 µs ±19% (+40%, p=0.002) / 410.2 KiB (+47%) / 4 220 (+33%) | +| `ActionChain/chain10` | 94.7 µs ±13% / 675 | 111 µs (+18%) | 105 µs ±22% (~, p=0.065) / 764 (+13%) | +| `ActionChain/chain100` | 575 µs ±20% / 5 757 | 735 µs (+28%) | 712 µs ±7% (+24%, p=0.009) / 6 394 (+11%) | +| `ActionChain/chain1000` | 14.3 ms ±22% / 56.28 k | 16.3 ms (~) | 15.7 ms ±9% (~, p=0.180) / 62.83 k (+12%) | +| `StateLoop/count50` | 706 µs ±7% / 10.37 k | 870 µs (+23%) | 861 µs ±22% (+22%, p=0.002) / 11.60 k (+12%) | +| `StateLoop/count500` | 6.45 ms ±10% / 100.9 k | 8.95 ms (+39%) | 9.03 ms ±21% (+40%, p=0.002) / 112.9 k (+12%) | +| `StateLoop/count5000` | 64.9 ms ±14% / 45.27 MiB / 1.006 M | 85.4 ms (+32%) | 85.5 ms ±12% (+32%, p=0.002) / 57.16 MiB (+26%) / 1.126 M (+12%) | +| `LowerChain/chain10` | 8.72 µs ±15% / 73 | 10.5 µs (+21%) | 10.2 µs ±17% (+17%, p=0.015) / 78 (+6.8%) | +| `LowerChain/chain100` | 149 µs ±14% / 460 | 159 µs (+7.0%) | 162 µs ±19% (+8.5%, p=0.026) / 471 (+2.4%) | +| `LowerChain/chain1000` | 9.13 ms ±19% / 4 115 | 8.74 ms (−4.3%) | 9.99 ms ±13% (~, p=0.699) / 4 136 (+0.5%) | + +Candidate-only rows (first pass, no 0.9.2 counterpart), the CSV/TSV/JSON +value import of `-import-values`: `ImportValues/flat/parts=100/attrs=3` +22.1 ms ±5% / 131.0 k allocs; `/flat/parts=500/attrs=3` 107 ms ±13% / +615.0 k; `/flat/parts=2000/attrs=3` 513 ms ±42% / 2.433 M; +`/flat/parts=2000/attrs=5` 743 ms ±25% / 3.672 M; +`/nested/parts=100/attrs=3` 30.1 ms ±5% / 175.4 k; +`/nested/parts=500/attrs=3` 145 ms ±5% / 829.4 k; +`/nested/parts=2000/attrs=3` 664 ms ±57% / 3.284 M; +`/nested/parts=2000/attrs=5` 1.05 s ±18% / 4.569 M. Linear in parts × +attributes, 300 allocations per 1 000 values; the nested layout costs a +third more than the flat one. + +Reading the table by cause: + +- **Lexing, parsing and indexing are unchanged** in time, with 4–8% more + allocations in the index (the reflective relationships each declaration + now registers, `00b8499a8`). +- **Analysis of the synthetic model** is +7.5% with +3.3% allocations, the + constraint-tier passes `AnalyseResolved` names. **Analysis of the vehicle + example is +85% with 1.8 times the bytes**, and the fixes did not touch + it, because the cost is not in the analysis but in its result: the + candidate reports 134 `Duplicate of imported member name` warnings on + the Annex A vehicle example where 0.9.2 reported none (the resolver's + `checkImportedNames`, `eb4b7583c`, which now sees the names a nested + `::**` import brings alongside the package's own). Every path that + returns the example's diagnostics pays to build and carry them: + `WorkspaceEdit/vehicle/reindex+diagnostics` +42%, + `GRPCParseFileUncached` +64% with 248 000 more allocations, + `ConnectParseFileHTTPCached` 271 → 879 µs with 851 more allocations + (the HTTP transport serializes the 134 warnings to JSON on every + request; the gRPC transport's cached protobuf conversion, `70242d872`, + is why `GRPCParseFileCached` went from 299 allocations back to 24 — two + more than 0.9.2, the cached slice's header and its once — and from + +22% to `~`). Whether the rule should fire on the vehicle example's + imports is a semantic question for the resolver's owners; **if it + should not, every row in this group is one fix away from 0.9.2**, and + if it should, the HTTP path wants the same cache the gRPC path got. + Either way, a model with no such warnings does not pay it: the + synthetic model's rows above are the same code without them. +- **Workspace edits.** The re-export fix (`4b4b35121`) is what moved these + rows against `ffe51571c`: `WorkspaceEdit/vehicle` from +86% bytes to + +18% and `WorkspaceEditSmallDocBesideLarge` — a one-line edit to a small + document in the same workspace as the 4 000-part model — from 3.9 times + 0.9.2's bytes to 3.0 times, and from +99% allocations to +40%. What + remains is the re-registration itself: `symbols.registrationOf` still + snapshots each name's re-export claims by reference (`4ca321508`), + which is a slice per name of the large neighbor per edit, 15.7 MiB per + edit of the small document. **Confirmed, and the remaining 3.0× is + the one row in this record a 0.10.x pass should start with.** The + synthetic edit's +35% time stands on a ±68% candidate spread; its + bytes (+10%) and allocations (+14%) are the stable reading. +- **Execution is where the release's semantics cost**: `ActionLoop` + +38–56%, `StateLoop` +22–40%, `ExecuteState` +21%, + `ExecuteActionFreshContext` +38%, with 12–33% more allocations. The + lazy binding-check description (`37b4a17c9`) took `ActionLoop/for1000` + from 6 220 to 4 220 allocations per run (`ffe51571c` had doubled + 0.9.2's) and 3% of its time; the time that remains is the statement-order + exploration and the per-statement binding frames described under + `internal/exec/runtime`, which a loop body of one assignment pays on + every iteration. **Confirmed; structural.** `ExecuteAction` (a single + action, no loop) is `~` in time with 42 more allocations. +- **Constraints.** `BatchConstraints` was +68% on `ffe51571c` and is `~` + on the candidate, allocations +0.2%: the memoized succession-cycle check + (`8bd97fcf6`) **cleared it**. `SameConstraintManyInstances` +14% and + `BatchSatisfy` `~` in time with +3.4% allocations are the exact-number + evaluation of the constraint bodies. `GRPCVerifyConstraint` +21% on a + ±43% candidate spread, +6.3% allocations; `GRPCEvaluate` +11%, six + allocations more per call. +- **Instantiate**, the 4 000-part model's instantiation, is `~` in time + interleaved (the first pass's +39% was two outliers at ±79% and ±109%) + with +4.5% allocations and +3.5% bytes; the per-object cost is the REPL's + `Instantiate` row. +- **Lowering** is +16% on a single action graph and +17% on a ten-step + chain (two and five more allocations: the guard and effect spans the + lowered graph now carries, `a0f77861d`), fading to `~` at 1 000 steps. +- **REPL**: loading the 4 000-part file is `~` in time interleaved (the + first pass's +34% was the candidate's ±37% spread) with +2.8% + allocations; submitting a snippet to the loaded model +14%. + +## Stress constellation (`tests/stressmodel`) + +The generated satellite-network constellation at 32, 128 and 512 +satellites (6 227, 24 167 and 95 927 elements), the sizes of the previous +records. Time per operation, 0.9.2, `ffe51571c` and the candidate, from the +interleaved re-run of every row but `ConvertAPIJSON` and +`AnalyzeSplitPerDocument` (first pass, see below); bytes and allocations +per operation where they moved. 51 rows on 0.9.2, 45 on the candidate. + +| benchmark | 0.9.2 | `ffe51571c` | 0.10.0 (`82f0fac87`) | +| --------- | ----- | ----------- | -------------------- | +| `Load/32` (6 227 el.) | 379 ms ±7% / 132.7 MiB / 1.994 M allocs | 456 ms (+20%) / 162.2 MiB / 2.365 M | 440 ms ±4% (+16%, p=0.002) / 159.7 MiB (+20%) / 2.316 M (+16%) | +| `Load/128` (24 167 el.) | 1.55 s ±20% / 501.3 MiB / 7.682 M | 1.85 s (+20%) | 1.71 s ±10% (+10%, p=0.041) / 568.2 MiB (+13%) / 8.836 M (+15%) | +| `Load/512` (95 927 el.) | 6.67 s ±23% / 1.935 GiB / 30.43 M | 7.81 s (+17%) | 7.50 s ±6% (~, p=0.132) / 2.166 GiB (+12%) / 34.92 M (+15%) | +| `Satisfy/32` (96 assertions) | 4.42 ms ±11% / 1.665 MiB / 28.90 k | 5.10 ms (+16%) / 1.831 MiB / 34.57 k | 4.88 ms ±3% (+11%, p=0.015) / 1.768 MiB (+6.2%) / 33.42 k (+16%) | +| `Satisfy/128` (384) | 18.9 ms ±14% / 114.9 k | 23.5 ms (+25%) | 21.1 ms ±15% (+12%, p=0.026) / 7.064 MiB (+6.2%) / 132.9 k (+16%) | +| `Satisfy/512` (1 536) | 103 ms ±11% / 458.8 k | 132 ms (+28%) | 109 ms ±15% (~, p=0.132) / 28.26 MiB (+6.2%) / 530.9 k (+16%) | +| `FleetInstantiate/32` (1 179 el.) | 4.33 ms ±5% / 7.665 MiB / 12.72 k | 5.13 ms (+19%) | 4.31 ms ±6% (~, p=1.000) / 7.592 MiB (~) / 13.93 k (+9.5%) | +| `FleetInstantiate/128` (1 715) | 12.1 ms ±12% / 24.10 MiB / 48.37 k | 16.6 ms (+37%) | 12.9 ms ±28% (~, p=0.093) / 24.44 MiB (~) / 52.94 k (+9.5%) | +| `FleetInstantiate/512` (3 947) | 63.2 ms ±7% / 151.4 MiB / 190.9 k | 84.6 ms (~, ±29%) | 62.6 ms ±9% (~, p=0.937) / 144.1 MiB (~) / 209.0 k (+9.5%) | +| `FleetSatisfy/32` (24 assertions) | 1.07 ms ±8% / 456.7 KiB / 7 811 | 1.32 ms (+23%) | 1.22 ms ±15% (+14%, p=0.002) / 480.1 KiB (+5.1%) / 8 852 (+13%) | +| `FleetSatisfy/128` (36) | 1.70 ms ±5% / 12.10 k | 2.07 ms (+22%) | 1.90 ms ±9% (+12%, p=0.002) / 743.5 KiB (+4.9%) / 13.65 k (+13%) | +| `FleetSatisfy/512` (108) | 5.35 ms ±17% / 37.06 k | 6.63 ms (+24%) | 5.90 ms ±36% (+10%, p=0.041) / 2.221 MiB (+4.6%) / 41.55 k (+12%) | +| `EditBeside/32` | 1.10 ms ±5% / 295.6 KiB / 3 442 | 1.48 ms (+34%) / 719.5 KiB (+143%) / 5 533 (+61%) | 1.32 ms ±4% (+20%, p=0.002) / 558.6 KiB (+89%) / 4 537 (+32%) | +| `EditBeside/128` | 3.41 ms ±3% / 719.9 KiB / 8 377 | 4.60 ms (+35%) / 2.379 MiB (+238%) / 15.92 k (+90%) | 4.06 ms ±12% (+19%, p=0.002) / 1.756 MiB (+150%) / 11.97 k (+43%) | +| `EditBeside/512` | 13.4 ms ±7% / 2.409 MiB / 28.11 k | 19.1 ms (+43%) / 9.146 MiB (+280%) / 57.44 k (+104%) | 16.1 ms ±4% (+20%, p=0.002) / 6.662 MiB (+177%) / 41.68 k (+48%) | +| `ValidateSplit/32/jobs=1` | 541 ms ±9% / 186.1 MiB / 2.480 M | 612 ms (+13%) | 614 ms ±24% (+14%, p=0.002) / 217.1 MiB (+17%) / 2.881 M (+16%) | +| `ValidateSplit/32/jobs=8` | 254 ms ±5% / 2.499 M | 275 ms (+8.2%) | 267 ms ±5% (+4.9%, p=0.041) / 219.0 MiB (+16%) / 2.896 M (+16%) | +| `ValidateSplit/128/jobs=1` | 1.96 s ±17% / 8.723 M | 2.17 s (+11%) | 2.15 s ±6% (~, p=0.093) / 700.6 MiB (+14%) / 10.07 M (+15%) | +| `ValidateSplit/128/jobs=8` | 802 ms ±19% / 8.740 M | 901 ms (~) | 853 ms ±12% (~, p=0.589) / 703.0 MiB (+14%) / 10.08 M (+15%) | +| `ValidateSplit/512/jobs=1` | 7.86 s ±17% / 33.69 M | 8.94 s (+14%) | 8.70 s ±8% (+11%, p=0.026) / 2.623 GiB (+12%) / 38.80 M (+15%) | +| `ValidateSplit/512/jobs=8` | 3.22 s ±20% / 33.69 M | 3.69 s (~) | 3.64 s ±4% (~, p=0.065) / 2.626 GiB (+12%) / 38.82 M (+15%) | +| `LoadFiles/32` | 473 ms ±15% / 147.7 MiB / 1.971 M | 555 ms (+17%) | 547 ms ±43% (+16%, p=0.009) / 175.7 MiB (+19%) / 2.325 M (+18%) | +| `LoadFiles/128` | 1.88 s ±12% / 6.998 M | 2.03 s (~) | 2.02 s ±6% (~, p=0.240) / 565.6 MiB (+16%) / 8.244 M (+18%) | +| `LoadFiles/512` | 7.62 s ±7% / 27.10 M | 8.51 s (+12%) | 8.39 s ±22% (+10%, p=0.002) / 2.123 GiB (+14%) / 31.91 M (+18%) | +| `EditImported/32` | 547 ms ±25% / 116.3 MiB / 1.979 M | 574 ms (~) / 150.5 MiB (+29%) | 572 ms ±112% (~, p=0.485) / 135.4 MiB (+16%) / 2.360 M (+19%) | +| `EditImported/128` | 1.82 s ±10% / 6.796 M | 2.01 s (+11%) / 468.1 MiB (+23%) | 1.96 s ±10% (+7.5%, p=0.026) / 447.7 MiB (+17%) / 8.090 M (+19%) | +| `EditImported/512` | 7.17 s ±12% / 26.06 M | 8.30 s (+16%) / 1.759 GiB (+19%) | 8.12 s ±6% (+13%, p=0.004) / 1.720 GiB (+16%) / 31.00 M (+19%) | +| `OpenSplit/32/cold` | 251 ms ±18% / 169.0 MiB / 2.360 M | 263 ms (~) | 256 ms ±12% (~, p=0.818) / 199.0 MiB (+18%) / 2.757 M (+17%) | +| `OpenSplit/32/warm` | 106 ms ±16% / 65.62 MiB / 571.1 k | 135 ms (+27%) / 86.05 MiB (+31%) | 132 ms ±5% (+24%, p=0.002) / 79.75 MiB (+22%) / 694.8 k (+22%) | +| `OpenSplit/128/cold` | 770 ms ±13% / 8.208 M | 834 ms (~) | 857 ms ±8% (+11%, p=0.041) / 627.4 MiB (+16%) / 9.554 M (+16%) | +| `OpenSplit/128/warm` | 203 ms ±13% / 166.7 MiB / 1.385 M | 284 ms (+40%) / 215.3 MiB (+29%) | 268 ms ±12% (+32%, p=0.002) / 208.9 MiB (+25%) / 1.717 M (+24%) | +| `OpenSplit/512/cold` | 2.93 s ±8% / 31.59 M | 3.61 s (+23%) | 3.52 s ±18% (+20%, p=0.026) / 2.332 GiB (+14%) / 36.72 M (+16%) | +| `OpenSplit/512/warm` | 661 ms ±6% / 613.2 MiB / 4.639 M | 941 ms (+42%) / 897.5 MiB (+46%) | 939 ms ±20% (+42%, p=0.002) / 890.2 MiB (+45%) / 5.804 M (+25%) | +| `HydratePlane/32` | 95.2 ms ±8% / 25.83 MiB / 314.8 k | 83.6 ms (−12%) | 78.6 ms ±32% (~, p=0.065) / 20.92 MiB (−19%) / 255.0 k (−19%) | +| `HydratePlane/128` | 263 ms ±8% / 76.62 MiB / 919.8 k | 208 ms (−21%) | 211 ms ±52% (~, p=0.065) / 57.47 MiB (−25%) / 652.9 k (−29%) | +| `HydratePlane/512` | 1.03 s ±3% / 287.1 MiB / 3.336 M | 828 ms (−20%) | 846 ms ±4% (−18%, p=0.002) / 207.0 MiB (−28%) / 2.243 M (−33%) | +| `ConvertAPIJSON/32` (first pass) | 9.74 s ±14% / 4.645 GiB / 25.53 M | — | 7.21 s ±8% (−26%, p=0.002) / 4.677 GiB (+0.7%) / 25.13 M (−1.5%) | +| `ConvertAPIJSON/128` (first pass) | 35.7 s ±38% / 18.22 GiB / 100.8 M | — | 29.7 s ±3% (~, p=0.065) / 18.34 GiB (+0.7%) / 99.31 M (−1.5%) | +| `ConvertAPIJSON/512` | one count, 156 s / 72.84 GiB | — | one count, 136 s / 73.31 GiB — no distribution | +| `AnalyzeSplitPerDocument/32/jobs=1` | 425 ms ±7% / 133.1 MiB / 1.636 M | fails | fails (see below) | +| `AnalyzeSplitPerDocument/32/jobs=8` | 108 ms ±5% | fails | fails | +| `AnalyzeSplitPerDocument/128/jobs=1` | 1.69 s ±5% / 489.0 MiB / 6.207 M | fails | fails | +| `AnalyzeSplitPerDocument/128/jobs=8` | 415 ms ±8% | fails | fails | +| `AnalyzeSplitPerDocument/512/jobs=1` | 7.18 s ±25% / 2.025 GiB / 24.49 M | fails | fails | +| `AnalyzeSplitPerDocument/512/jobs=8` | 1.77 s ±11% | fails | fails | + +Allocation counts — deterministic to the unit — split the package into +four stories: + +- **The loader costs 15–18% more allocations and 12–20% more bytes per + element** at every scale and through every entrance (`Load`, + `LoadFiles`, `ValidateSplit`, `OpenSplit/cold`, `EditImported`), for + 10–16% more time where the spread lets the difference show and `~` + where it does not (the 512-satellite `Load` and the parallel + `ValidateSplit`). Each connector, binding and flow end the constellation + is made of is now owned, journaled, indexed and reflected + (`00b8499a8`, `da8fba4a3`, `32890be28`, `63552cbc4`), and the extended + declarations classified (`ca0e6cca0`). The qualified-name fix + (`367ca7e3a`) took 2–3 points off `ffe51571c`'s allocations and 3–7 off + its time. **Confirmed; structural, linear, and the cost of 0.10.0's + relationship model.** The whole-binary table below shows the same from + outside the process. +- **The satisfaction checks are +16% allocations and +10–14% time on the + fleet** where `ffe51571c` was +20% and +22–28%: the memoized + succession-cycle check (`8bd97fcf6`) took the walk out of every check + and left the exact-number evaluation of the assertions (`923ec81f0`, + `79629923b`), which is 4 500 allocations per 96 assertions. **Confirmed; + explained.** +- **Editing beside the constellation allocates 2.9 times the bytes of + 0.9.2** (was 3.8 times) and +32–48% objects (was +61–104%), for +20% + time (was +34–43%); the warm `OpenSplit` and `WorkspaceUpdate` in the + LSP table are the same. The re-export fix (`4b4b35121`) removed the + deep copy of each name's claims; the per-name snapshot of claims by + reference across the 512-satellite neighbor (`4ca321508`) is what + remains, 4.3 MiB per edit at 512. **Confirmed; the open fix.** + `OpenSplit/512/warm` (+42%, +45% bytes) adds the warm cache's + re-validation of the owned-relationship journal it now carries. +- **Instantiation of the fleet is `~` in time with 9.5% more allocations + and the same or fewer bytes** — the three-word scalar packing + (`4a1b937f2`) and the allocation-free Integer and Rational fast paths + (`d1eb16edb`) took `ffe51571c`'s +19–37% back to 0.9.2, and + `HydratePlane` is **18–33% fewer allocations and 18–28% fewer bytes** + than 0.9.2, 18% faster at 512: the hydration of a plane's instances no + longer rebuilds the implicit ends it can derive (`fa8a7220c`). The + API-JSON conversion is 26% faster at 32 satellites with the same bytes, + from the same source; its 512-satellite row still does not fit this + machine on either revision. + +`AnalyzeSplitPerDocument` fails on the candidate at every size. The +benchmark analyses each of the constellation's six files in its own +per-document registry and asserts a clean result; the candidate's +per-document analysis now reports each decimal literal the generator +writes into a `Real`-typed attribute (`0.2 is rounded to the nearest Real, +0.20000000000000001, since a feature typed by Real holds a binary64 +value; type the feature by Rational to keep the literal exact`, +`014fceb7b`), ten per file, and the assertion trips. The six rows are +baseline-only figures in this record; the benchmark's generator or its +assertion needs the release's view of decimal literals (a `Rational` +attribute type, or the lint filtered) before the row can be compared +again. The whole-workspace analysis the other rows and `sysml -validate` +do is not affected — the report is a lint the per-document registry turns +on — which is why `ValidateSplit` and the whole-binary runs are clean on +both. ## Whole-binary scaling `sysml -validate` over generated compliant models (the satellite-network generator at ≈3 000, 6 000 and 12 000 declarations: 3 804, 7 260 and -14 172 lines) that report `no errors`, five runs each with the two -binaries interleaved; median wall seconds and median maximum resident set. - -| declarations | 0.9.2 | 0.10.0 | -| ------------ | ----- | ------ | -| ≈3 000 | 0.10 s / 97 MiB | 0.12 s / 101 MiB | -| ≈6 000 | 0.16 s / 109 MiB | 0.17 s / 115 MiB | -| ≈12 000 | 0.25 s / 132 MiB | 0.29 s / 158 MiB | - -Loading and validating a model from the command line is 10–20% slower -and holds 4–26 MiB more at the end, growing with the model: the loader's -per-element bytes and the analysis passes above, seen whole. At 12 000 -declarations that is 40 ms and 26 MiB; the figures scale linearly on both -binaries (0.10→0.25 s and 0.12→0.29 s from 3 000 to 12 000), so nothing -superlinear arrived with the release. +14 172 lines) that report `no errors`, nine runs each with the three +binaries interleaved; median wall seconds and median maximum resident set, +every run exiting 0. + +| declarations | 0.9.2 | `ffe51571c` | 0.10.0 (`82f0fac87`) | +| ------------ | ----- | ----------- | -------------------- | +| ≈3 000 | 0.09 s / 95 MiB | 0.11 s / 100 MiB | 0.11 s / 101 MiB | +| ≈6 000 | 0.14 s / 107 MiB | 0.16 s / 116 MiB | 0.17 s / 115 MiB | +| ≈12 000 | 0.23 s / 134 MiB | 0.27 s / 157 MiB | 0.28 s / 157 MiB | + +Linear in the declarations on every binary, as before: doubling the model +doubles the time (0.11 → 0.17 → 0.28 s) and adds 14–42 MiB. The candidate +is 20–22% slower than 0.9.2 and holds 6–23 MiB more, growing with the +model — the loader's per-element cost of the stress constellation +(+14–20% bytes, +15–18% allocations) seen from outside the process. The +fixes did not move these figures against `ffe51571c`, and were not +expected to: none of the six is on the load path. The spreads were tight +on 0.9.2 and `ffe51571c` (±1 sample in the second decimal) and the +candidate's first round of each model was its slowest (0.51, 0.75 and +1.48 s, a cold page cache for the freshly built binary); the medians are +unaffected and the remaining eight rounds agree with them. ### Process start and session floor -| figure | 0.9.2 | 0.10.0 | -| ------ | ----- | ------ | -| binary size | 42.88 MiB (44 957 880 B) | 47.00 MiB (49 275 064 B; +4.12 MiB) | -| empty session (`sysml -e 1`), median of 5 | 0.02 s / 65 MiB | 0.02 s / 68 MiB | -| empty-session load (`LoadModel/elements=0`) | 405 µs / 216.2 KiB | 415 µs (~) / 221.6 KiB (+2.5%) | - -The binary grows 4.12 MiB (+9.6%) with the release's new surfaces — the -views and their D2, Mermaid, DOT and PlantUML renderers, the graphs -export, the api-json loader, the value import, the browser engine's -shared code, the `big.Int` and `big.Rat` arithmetic — and the empty -session maps 3 MiB more for it; its start time is unchanged at the clock's -resolution. +| measure | 0.9.2 | `ffe51571c` | 0.10.0 (`82f0fac87`) | +| ------- | ----- | ----------- | -------------------- | +| `sysml -e 1` wall time | 0.02 s | 0.02 s | 0.02 s | +| `sysml -e 1` max RSS (session floor) | 64 MiB | 68 MiB | 69 MiB | +| `bin/sysml` size (`-s -w`) | 42.88 MiB | 46.99 MiB | 47.23 MiB | + +The floor is the standard library's bundled index decoded at start, and +rose 5 MiB with the snapshot's new contents (the owned-relationship +journal and the reflected implicit relationships, `DecodeSnapshot` ++7.2% bytes). The binary is 4.35 MiB larger than 0.9.2's: the new +frontends (views, code lenses, inlay hints, the REPL's repository and +notebook commands, `-list`), the exact-number evaluator, the value import +and the SysML v2 API client; 0.24 MiB of it arrived after the release +branch was cut. Start time is unchanged. ### Example models and the Apollo 11 model -Median wall time and maximum resident set of five interleaved runs; every -run returned the same exit status and report on both binaries (the -candidate tree's copies of the two example files were given to both). The -Apollo 11 model is the public -[Apollo 11 SysML v2 model](https://github.com/airbus/apollo-11-sysml-v2) -(commit `6e9c93f`, 28 files, 7 221 lines). - -| model / command | 0.9.2 | 0.10.0 | -| --------------- | ----- | ------ | -| `action-executor-demo.sysml -validate` | 0.02 s / 66 MiB | 0.03 s / 70 MiB | -| `disposal-robot-demo/robot.sysml -validate` | 0.04 s / 72 MiB | 0.04 s / 78 MiB | -| Apollo 11, `-validate` all 28 files | 0.29 s / 150 MiB | 0.32 s / 169 MiB | +| model | 0.9.2 | `ffe51571c` | 0.10.0 (`82f0fac87`) | +| ----- | ----- | ----------- | -------------------- | +| `examples/action-executor-demo.sysml` | 0.02 s / 66 MiB | 0.02 s / 69 MiB | 0.02 s / 71 MiB | +| `examples/disposal-robot-demo/robot.sysml` | 0.04 s / 72 MiB | 0.04 s / 78 MiB | 0.04 s / 77 MiB | +| Apollo 11 (28 files, `scripts/download-apollo11.sh`) | 0.28 s / 149 MiB | 0.33 s / 169 MiB | 0.32 s / 161 MiB | -The Apollo 11 model loads and validates in 0.32 s against 0.29 s (+10%, -three clock ticks) and holds 19 MiB more, in line with the generated -models of its size. +The two small examples sit on the session floor and move with it. The +Apollo 11 model — the one real model in the set, connector- and +flow-dense like the stress constellation — validates in 0.32 s against +0.28 s (+14%) holding 12 MiB more; 8 MiB less than `ffe51571c` held, the +smaller claim snapshots of the re-export fix on a multi-document model. +Every file validates with the same report on all three binaries. ## Re-runs -The first pass left 71 of the 190 compared rows in doubt: 48 by a -significant difference, 23 by a spread above 20% on one side with no -significant difference (`WorkspaceUpdate` ±100%, `Fib(25)/go` ±94%, -`Instantiate` ±102%, most of the stress constellation's time rows). All -were re-run interleaved — `-count 1` of the row on 0.9.2, then on the -candidate, six times — and the `tests/perf` and `tests/stressmodel` -packages whole, since most of each was in doubt. The pilot corpora were -downloaded between the first pass and the re-run, so the `vehicle` rows -and `InstantiateWarmModel` have one sequential six-count run each -(reported above) and no first pass. +The first pass left 83 of the 196 compared rows in doubt: 61 by a +significant difference, 22 by a spread above 20% on one side with no +significant difference (`Fib(25)/interpreted` ±65%, `ReferencesCold` +±19/17%, `Instantiate` ±79/109%, most of the stress constellation's time +rows). All were re-run interleaved — `-count 1` of the row on 0.9.2, then +on `ffe51571c`, then on the candidate, six times — and the +`internal/frontend/repl`, `tests/perf` and `tests/stressmodel` packages +whole or in their moving rows, since most of each was in doubt. The pilot +corpora were downloaded between 0.9.2's first pass and the re-run, so the +`vehicle` rows and `InstantiateWarmModel` have the re-run only. What the re-run changed: - **Nothing changed sign.** Every row significant in the first pass is - significant in the re-run in the same direction; the stress - constellation's time regressions shrank from the first pass's +39–87% - to +16–45% once the two revisions ran under the same conditions, and - that is the figure reported. The first pass had run all of 0.9.2's - counts first and the candidate's after; the re-run shows that about - half of the first-pass difference on the long rows was the machine. -- **Cleared:** `RenameWarm` (−14.5% as found, `~`), `ReferencesCold` - (`~`), `HydratePlane/128, 512` (−10%, −37% as found; `~` in time, the - byte and allocation gains stand), `WorkspaceUpdate` (+36% as found; - `~` in time at ±16%, the +99% bytes stand), `Fib(25)/go` (+77% as - found; `~` at p=0.065, 1.71 ms ±5% against 959 µs ±94%), `Instantiate` - in `tests/perf` (`~`), `EditImported` (`~` at every size), the REPL - `LoadModel` rows (`~` in time at every size), `Hypot/interpreted` - (`~`). + significant in the re-run in the same direction or `~`; the stress + constellation's time regressions shrank from the first pass's +14–27% + to +10–20% once the three revisions ran under the same conditions, and + that is the figure reported. +- **Cleared:** `REPLLoadFile` (+34% as found; `~` at p=0.065), + `Instantiate` in `tests/perf` (+39% as found; `~`), `BatchSatisfy` + (+17% as found; `~`), `IndexLibrary` (+19% as found; `~`), + `AnalyseResolved` (+12% as found; `~`, the +18% allocations stand), + `DecodeSnapshot` (`~`, the +19% allocations stand), `IndexAddExpand` + and `IndexAddOnly` (`~`), `FleetInstantiate` at every size (`~`), + `Load/512` and `Satisfy/512` (`~` at ±6% and ±15%), `LoadFiles/128`, + `OpenSplit/32/cold`, `EditImported/32`, `ValidateSplit/128` (`~`), + `ExecuteAction` (`~`), `ActionChain/chain10, chain1000` (`~`), + `LowerStateGraph` (`~`), `RunStateMachine/1000` (`~`), + `Diagnostics/attributes=800` (`~`), `GRPCParseFileCached` (`~`). - **Confirmed** (significant both times, within a few points of each - other): `AssignmentLoopStep` (+66% → +70%), `MaterializePlainAttributes` - (`~` → +18%), `DerivedReadWriteRead` (+12% → +16%), `RunCalc` (+20%), - `Instantiate/250, 1000` (+32%, +18%), `Fib(25)/interpreted` (+434% → - +409%), `Collatz(27)/interpreted` (+68% → +60%), `Collatz(27)/go` - (+56%), `Hypot(3.0,4.0)/go` (+398% → +423%), `FormatEdits` (+69%), - `InstantiateWarmModel` (+42%), `DecodeSnapshot` (+4.9%), - `AnalyseResolved` (+10%), and every `tests/perf` and stress row in the - tables above. -- **Still in doubt:** `OpenSplit/*/warm` (±39–113% on the candidate's - counts, every one of them above 0.9.2's); `RealDecimalLoop` ±37% and - `ImportValues/flat/parts=2000/attrs=5` ±37%, candidate-only. + other): `AssignmentLoopStep` (+90% → +61%), `MaterializePlainAttributes` + (`~` → +14%), `DerivedReadWriteRead` (+39% → +11%), `FormatEdits` + (+71%), `WorkspaceUpdate` (+42% → +16%), the REPL `LoadModel` rows + (+17–41% → +5–10%), `RunCalc` (+13–19% → +10–14%), `Instantiate/250, + 1000, 4000` (+13–27% → +11–31%), `Collatz(27)` (+71%/+53% → + +65%/+59%), `Fib(25)/go` (+94% → +81%), `Hypot(3.0,4.0)/go` (+394% → + +412%), `InstantiateWarmModel` (+32%), and the `tests/perf` and stress + rows the tables above mark significant. +- **Moved the other way:** `Fib(25)/interpreted` was `~` in the first pass + (6.61 ms ±65% against 7.54 ms) and is −20% at ±9% interleaved. +- **Still in doubt:** `WorkspaceEdit/synthetic` (+35% at ±68% on the + candidate's counts; the +10% bytes and +14% allocations are the + reading), `GRPCVerifyConstraint` (+21% at ±43%), + `EditImported/32` (±112%), `HydratePlane/32, 128` (−17%, −20% at + ±32% and ±52%, `~` at p=0.065; the allocation gains stand), + `LoadFiles/32` (+16% at ±43%); `IntegerArithmeticLoop` ±59% and + `RationalDecimalLoop` ±34%, candidate-only. Raw `benchstat` output for every package, first pass and re-run, is in the tables above; the per-row p-values are p=0.002 (n=6) unless shown. @@ -721,16 +781,19 @@ tables above; the per-row p-values are p=0.002 (n=6) unless shown. ## What this does not measure - The new surfaces with no 0.9.2 counterpart — the view renderers, the - graphs export, the api-json loader, the browser engine, the Python and - Node clients' new calls — have benchmarks only where listed above - (`ImportValues`, the exact-numeric loops); the rest are covered by their - test suites, not by this record. + graphs export, the api-json loader, the browser engine, the REPL's + repository and notebook commands, the Python and Node clients' new + calls — have benchmarks only where listed above (`ImportValues`, the + exact-numeric loops); the rest are covered by their test suites, not by + this record. - The compiled-calc C target is measured on `Hypot` only; the three Integer calcs it refuses have no candidate figure, so the C target's speed on 0.10.0 is read from one row. -- Everything ran on one 8-CPU virtual machine. Its neighbours were - quieter than during the 0.9.2 record (the baseline's re-run spreads are - mostly under 20%), but the stress constellation's `Satisfy` and - `EditBeside` rows still show 30–80% spreads on single counts, and a - one-CPU host should read the package figures rather than the +- The stress constellation's per-document analysis + (`AnalyzeSplitPerDocument`) has no candidate figure at any size. +- Everything ran on one 8-CPU virtual machine. The baseline's re-run + spreads are mostly under 20%, but the stress constellation's + `EditImported/32` and `HydratePlane` rows and the candidate's + `WorkspaceEdit/synthetic` still show 30–110% spreads on single counts, + and a one-CPU host should read the package figures rather than the whole-binary ones, as the previous records said. From 57ae83c3f6badf0c67261ee41822b693729741eb Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Fri, 9 Oct 2026 18:04:20 +0000 Subject: [PATCH 3/4] docs(perf): re-run the library import rows interleaved and state the failing-row handling Co-Authored-By: jason.han --- .../unreleased/perf-0.10.0-record.added.md | 2 +- .../performance-release-0.10.0-vs-0.9.2.md | 29 ++++++++++++------- 2 files changed, 19 insertions(+), 12 deletions(-) diff --git a/changes/unreleased/perf-0.10.0-record.added.md b/changes/unreleased/perf-0.10.0-record.added.md index 714468b539..dbbbb97ef5 100644 --- a/changes/unreleased/perf-0.10.0-record.added.md +++ b/changes/unreleased/perf-0.10.0-record.added.md @@ -1 +1 @@ -- **The release-gate performance record of 0.10.0 against 0.9.2.** `docs/project/performance-release-0.10.0-vs-0.9.2.md` measures the 0.10.0 candidate — `main` at the release merge with `develop`'s post-cut changes and the six performance fixes folded in — against the `v0.9.2` tag: every benchmark on both revisions under `benchstat`, the rows in doubt re-run interleaved, the stress constellation, and whole-binary validation, start-up and size, with a column for the release merge alone where a fix moved a row. Nothing in it blocks the release. The fixes cleared the interpreted `Fib(25)` (now 20% faster than 0.9.2), the batch constraint check and the cached gRPC diagnostics, and halved the workspace-edit allocations; what remains is loading 10–16% slower per element on connector-dense models, executing a step 20–60%, workspace edits beside a large document at 2.9–3.0 times the bytes, the import-ambiguity warning firing 134 times on the Annex A vehicle example, and the exact-numeric calcs, each with its cause named. +- **The release-gate performance record of 0.10.0 against 0.9.2.** `docs/project/performance-release-0.10.0-vs-0.9.2.md` measures the 0.10.0 candidate — `main` at the release merge with `develop`'s post-cut changes and the six performance fixes folded in — against the `v0.9.2` tag: every benchmark on both revisions under `benchstat`, the rows in doubt re-run interleaved, the stress constellation, and whole-binary validation, start-up and size, with a column for the release merge alone where a fix moved a row. Nothing in it blocks the release. The fixes cleared the interpreted `Fib(25)` (now 20% faster than 0.9.2), the batch constraint check and the cached gRPC diagnostics, and cut the workspace-edit bytes and allocations by a quarter; what remains is loading 10–16% slower per element on connector-dense models, executing a step 20–60%, workspace edits beside a large document at 2.9–3.0 times the bytes, the import-ambiguity warning firing 134 times on the Annex A vehicle example, and the exact-numeric calcs, each with its cause named. diff --git a/docs/project/performance-release-0.10.0-vs-0.9.2.md b/docs/project/performance-release-0.10.0-vs-0.9.2.md index f7424ac23c..63fd72f900 100644 --- a/docs/project/performance-release-0.10.0-vs-0.9.2.md +++ b/docs/project/performance-release-0.10.0-vs-0.9.2.md @@ -52,7 +52,7 @@ is superlinear, nothing that fit 0.9.2's budgets stops fitting except the one calc noted below, and the fleet's instantiation and hydration are faster. Against the earlier measurement of `ffe51571c`, the six fixes on the candidate cleared three of the regressions that record named as -avoidable and halved a fourth: +avoidable and cut a fourth by a quarter: - `CompiledCalc/Fib(25)/interpreted` went from 5.3× 0.9.2 to **20% faster than 0.9.2** (the three-word scalar packing); @@ -126,7 +126,11 @@ revisions interleaved, `-count 1` six times each, nothing else on the machine. Where the first pass left most of a package in doubt (`internal/frontend/repl`, `tests/perf`, `tests/stressmodel`) the whole package or the whole of its moving rows was re-run that way, and the re-run -is the table shown. Whole-binary figures are `/usr/bin/time -f '%e %M'` +is the table shown. A benchmark that fails on one revision (`b.Fatal`) +prints `--- FAIL` and no result line while the package's other benchmarks +run on; `benchstat` then has no row for it, and the failure is reported in +that package's section as a candidate-only or baseline-only row. Nothing +was excluded from `-bench .`. Whole-binary figures are `/usr/bin/time -f '%e %M'` over nine interleaved runs of each binary, all built with `make build-sysml` (`-s -w`, `-trimpath`); medians are reported. The pilot corpora the `vehicle` rows and the gRPC warm-instantiation row read were @@ -385,9 +389,9 @@ rows below share. **Explained.** | benchmark | 0.9.2 | 0.10.0 (`82f0fac87`) | | --------- | ----- | -------------------- | | `migrate` `WriterSiblingBlocks` | 1.93 ms ±7% / 1.906 MiB / 20.02 k allocs | 1.90 ms ±9% (~) / 1.906 MiB (~) / 20.02 k (~) | -| `libs` `ExpandWildcardImports` | 26.8 ms ±10% / 14.46 MiB / 128.3 k | 31.0 ms ±20% (+15%, p=0.009, first pass) / 15.36 MiB (+6.2%) / 145.7 k (+14%) | +| `libs` `ExpandWildcardImports` | 27.4 ms ±1% / 14.46 MiB / 128.3 k | 28.6 ms ±3% (+4.2%) / 15.36 MiB (+6.2%) / 145.7 k (+14%) | | `libs` `IndexLibrary` | 16.3 ms ±15% / 13.56 MiB / 104.6 k | 16.2 ms ±14% (~, p=0.818) / 14.07 MiB (+3.8%) / 108.2 k (+3.4%) | -| `libs` `ExpandModelImports` | 6.36 ms ±4% / 4.315 MiB / 36.16 k | 6.93 ms ±30% (+9.0%, p=0.002, first pass) / 4.376 MiB (+1.4%) / 38.15 k (+5.5%) | +| `libs` `ExpandModelImports` | 6.50 ms ±11% / 4.315 MiB / 36.16 k | 6.67 ms ±4% (~, p=0.485) / 4.376 MiB (+1.4%) / 38.15 k (+5.5%) | | `libs` `DecodeSnapshot` | 9.89 ms ±17% / 35.32 MiB / 56.63 k | 10.8 ms ±21% (~, p=0.132) / 37.85 MiB (+7.2%) / 67.45 k (+19%) | | `libs` `SetDigest` | 629 µs ±12% / 1.710 MiB / 966 | 641 µs ±7% (~) / 1.733 MiB (+1.4%) / 998 (+3.3%) | | `model` `AnalyseUnresolved` | 20.2 ms ±8% / 13.69 MiB / 149.3 k | 20.6 ms ±6% (~) / 13.80 MiB (+0.8%) / 150.7 k (+1.0%) | @@ -400,10 +404,11 @@ library allocates 3.4% more objects (the reflective relationship ends, the 19% more: the snapshot carries the owned-relationship journal and the reflected implicit relationships (`da8fba4a3`, `32890be28`) that the 0.9.2 snapshot did not hold; the time of both is `~` interleaved. Expanding the -library's wildcard imports allocates 14% more objects and was +15% in the -first pass with the candidate's counts 20% apart; the import-ambiguity -rule (`eb4b7583c`) now walks what each wildcard brings in, and the row was -not re-run. Analysing a resolved document allocates 18% more with the time +library's wildcard imports allocates 14% more objects and is +4.2% in time +interleaved (+15% in the first pass, with the candidate's counts 20% +apart); the import-ambiguity rule (`eb4b7583c`) now walks what each +wildcard brings in. Expanding a model's imports is `~` in time interleaved +(+9.0% in the first pass). Analysing a resolved document allocates 18% more with the time `~` interleaved (first pass +12%): the constraint-tier passes added in the interval — the indexed-end multiplicity check (`cf85beb36`), the import-ambiguity warning (`eb4b7583c`), the reflective derivations for the @@ -482,8 +487,8 @@ value import of `-import-values`: `ImportValues/flat/parts=100/attrs=3` `/nested/parts=500/attrs=3` 145 ms ±5% / 829.4 k; `/nested/parts=2000/attrs=3` 664 ms ±57% / 3.284 M; `/nested/parts=2000/attrs=5` 1.05 s ±18% / 4.569 M. Linear in parts × -attributes, 300 allocations per 1 000 values; the nested layout costs a -third more than the flat one. +attributes at roughly 400 allocations per value (437 at 300 values, 367 +at 10 000); the nested layout costs a third more than the flat one. Reading the table by cause: @@ -748,6 +753,7 @@ What the re-run changed: - **Cleared:** `REPLLoadFile` (+34% as found; `~` at p=0.065), `Instantiate` in `tests/perf` (+39% as found; `~`), `BatchSatisfy` (+17% as found; `~`), `IndexLibrary` (+19% as found; `~`), + `ExpandModelImports` (+9.0% as found; `~`), `AnalyseResolved` (+12% as found; `~`, the +18% allocations stand), `DecodeSnapshot` (`~`, the +19% allocations stand), `IndexAddExpand` and `IndexAddOnly` (`~`), `FleetInstantiate` at every size (`~`), @@ -763,7 +769,8 @@ What the re-run changed: (+17–41% → +5–10%), `RunCalc` (+13–19% → +10–14%), `Instantiate/250, 1000, 4000` (+13–27% → +11–31%), `Collatz(27)` (+71%/+53% → +65%/+59%), `Fib(25)/go` (+94% → +81%), `Hypot(3.0,4.0)/go` (+394% → - +412%), `InstantiateWarmModel` (+32%), and the `tests/perf` and stress + +412%), `InstantiateWarmModel` (+32%), `ExpandWildcardImports` (+15% → +4.2%), + and the `tests/perf` and stress rows the tables above mark significant. - **Moved the other way:** `Fib(25)/interpreted` was `~` in the first pass (6.61 ms ±65% against 7.54 ms) and is −20% at ±9% interleaved. From 6d5170052bb3521e9ae8008fb0602a89988fc10d Mon Sep 17 00:00:00 2001 From: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Date: Mon, 12 Oct 2026 07:50:58 +0000 Subject: [PATCH 4/4] docs(perf): measure the v0.10.0 tag against v0.9.2 Co-Authored-By: jason.han --- .../unreleased/perf-0.10.0-record.added.md | 2 +- .../performance-release-0.10.0-vs-0.9.2.md | 1253 ++++++++--------- 2 files changed, 598 insertions(+), 657 deletions(-) diff --git a/changes/unreleased/perf-0.10.0-record.added.md b/changes/unreleased/perf-0.10.0-record.added.md index dbbbb97ef5..5c1aa74c21 100644 --- a/changes/unreleased/perf-0.10.0-record.added.md +++ b/changes/unreleased/perf-0.10.0-record.added.md @@ -1 +1 @@ -- **The release-gate performance record of 0.10.0 against 0.9.2.** `docs/project/performance-release-0.10.0-vs-0.9.2.md` measures the 0.10.0 candidate — `main` at the release merge with `develop`'s post-cut changes and the six performance fixes folded in — against the `v0.9.2` tag: every benchmark on both revisions under `benchstat`, the rows in doubt re-run interleaved, the stress constellation, and whole-binary validation, start-up and size, with a column for the release merge alone where a fix moved a row. Nothing in it blocks the release. The fixes cleared the interpreted `Fib(25)` (now 20% faster than 0.9.2), the batch constraint check and the cached gRPC diagnostics, and cut the workspace-edit bytes and allocations by a quarter; what remains is loading 10–16% slower per element on connector-dense models, executing a step 20–60%, workspace edits beside a large document at 2.9–3.0 times the bytes, the import-ambiguity warning firing 134 times on the Annex A vehicle example, and the exact-numeric calcs, each with its cause named. +- **The release-gate performance record of 0.10.0 against 0.9.2.** `docs/project/performance-release-0.10.0-vs-0.9.2.md` measures the `v0.10.0` tag against the `v0.9.2` tag: every benchmark on both revisions under `benchstat`, the rows in doubt re-run interleaved, the stress constellation, and whole-binary validation, start-up and size, with a column for the release merge before the state-machine and inherited-step work where that work moved a row. Nothing in it blocks the release. Loading is 15–18% more allocations per element on connector-dense models (`sysml -validate` +14–15%), executing a step +40–66%, lowering an action chain twice the bytes for the inherited steps it now carries (linear in the chain), checks with arguments 1 016 → 1 440 B each, workspace edits beside a large document 3.0 times the bytes, the import-ambiguity warning firing 134 times on the Annex A vehicle example, and the exact-numeric calcs, each with its cause named; the fleet's hydration is 19–33% fewer allocations and the interpreted `Fib(25)` 8% faster. diff --git a/docs/project/performance-release-0.10.0-vs-0.9.2.md b/docs/project/performance-release-0.10.0-vs-0.9.2.md index 63fd72f900..8d16c8b432 100644 --- a/docs/project/performance-release-0.10.0-vs-0.9.2.md +++ b/docs/project/performance-release-0.10.0-vs-0.9.2.md @@ -1,32 +1,35 @@ # Performance: release 0.10.0 against release 0.9.2 -The release-gate measurement of the 0.10.0 candidate — -`hotfix/0.10.0-develop-sync` at `82f0fac87` (2026-10-09), which is `main` -at `ffe51571c` (the `release/0.10.0` merge plus a CI-only hotfix) with -everything `develop` gained after the release branch was cut merged back -into it, 1 754 commits past the `v0.9.2` tag (`4226ad9a3`, 2026-10-05) — -against 0.9.2 itself. It follows the method of the -[0.9.2 against 0.9.1](performance-release-0.9.2-vs-0.9.1.md) record: every -benchmark on both revisions, `benchstat` over six runs, the rows the machine -left in doubt re-run interleaved, and whole-binary timings on generated and -real models. See [performance](../internals/performance.md) for what the -figures mean and where the remaining cost is. +The release-gate measurement of 0.10.0 — the `v0.10.0` tag at +`08ef39784` (2026-10-12), 2 001 commits past the `v0.9.2` tag +(`4226ad9a3`, 2026-10-05) — against 0.9.2 itself. It follows the method +of the [0.9.2 against 0.9.1](performance-release-0.9.2-vs-0.9.1.md) +record: every benchmark on both revisions, `benchstat` over six runs, the +rows the machine left in doubt re-run interleaved, and whole-binary +timings on generated and real models. See +[performance](../internals/performance.md) for what the figures mean and +where the remaining cost is. 0.10.0 is a minor release carrying everything on `develop` since 0.9.2: -exact Rationals and unbounded Integers in the evaluator, the exploration of -every order of a body's unordered statements, executors that interleave move -by move, reflective relationships and indexed connector ends in the semantic -model, incremental re-analysis in the gRPC service, views, code lenses and -inlay hints in the language server, repository and notebook commands in the -REPL, and the fixes of a 300-entry changelog. The expectation going in was -movement on the execution and analysis paths, and that is what the +exact Rationals and unbounded Integers in the evaluator, the exploration +of every order of a body's unordered statements, executors that +interleave move by move, state machines whose change triggers survive a +rollback, actions that perform the steps they inherit, reflective +relationships and indexed connector ends in the semantic model, +incremental re-analysis in the gRPC service, views, code lenses and inlay +hints in the language server, repository and notebook commands in the +REPL, and the fixes of a 300-entry changelog. The expectation going in +was movement on the execution and analysis paths, and that is what the measurement finds — it is not a parity record. -The candidate also carries six performance fixes made against an earlier -measurement of `ffe51571c` alone, and this record shows what they bought: -where a row moved between `ffe51571c` and `82f0fac87` the tables carry a -third column with the `ffe51571c` figure from the same interleaved run. -The six, by the change on the branch: +The release was measured twice on its way to the tag, and the tables +keep one earlier point where it helps: `82f0fac87`, the release merge +with `develop`'s six performance fixes and before the state-machine and +inherited-step work (`internal/exec/runtime`, `internal/ir/lower`) +landed. Where a row moved between `82f0fac87` and the tag the tables +carry its `82f0fac87` figure from the same interleaved run, so the cost +of that last work is read directly. The six fixes, by the change in the +history: - a cached model's diagnostics are converted to protobuf once, not on every `ParseFile` that returns them (`70242d872`); @@ -40,105 +43,110 @@ The six, by the change on the branch: boxed per node (`4a1b937f2`); - a symbol's qualified name is compared without being built (`367ca7e3a`). +A seventh landed with the inherited-step work itself: an action step's +symbol is found through its scope's declaration index rather than by +scanning the scope's members (`baea6252b`), which is what keeps the +lowering of a chain linear in its length below. + Every row that regresses is confirmed by the interleaved re-run and -attributed to the change responsible below; the fixes that remain open are -named in the summary, and nothing is fixed here. +attributed to the change responsible below; the fixes that remain open +are named in the summary, and nothing is fixed here. ## Summary **Nothing here blocks the release.** 0.10.0 is slower than 0.9.2 on most of the paths a model goes through, by a constant factor on each; nothing is superlinear, nothing that fit 0.9.2's budgets stops fitting except the -one calc noted below, and the fleet's instantiation and hydration are -faster. Against the earlier measurement of `ffe51571c`, the six fixes on -the candidate cleared three of the regressions that record named as -avoidable and cut a fourth by a quarter: - -- `CompiledCalc/Fib(25)/interpreted` went from 5.3× 0.9.2 to **20% - faster than 0.9.2** (the three-word scalar packing); -- `BatchConstraints` from +68% to `~`, `Satisfy` from +22–28% to - +10–14% (the memoized succession-cycle check); -- `GRPCParseFileCached` from 299 allocations to 24 against 0.9.2's 22 - and from +22% to `~` (the cached protobuf diagnostics); -- the workspace-edit rows from 3.8–3.9× 0.9.2's bytes to 2.9–3.0×, and - from +61–104% allocations to +32–48% (the re-export claims compared in - place); -- `AssignmentLoopStep` lost 1 200 of the 3 604 allocations the release - had added, and `ActionLoop/for1000` 2 000 of 3 039 (the lazy - binding-check description); -- the loader lost 2–3 points of allocations (the qualified-name compare). - -What remains, confirmed by the interleaved re-run: +one calc noted below, and the fleet's hydration is faster. Confirmed by +the interleaved re-run, in the order a model meets them: - **Loading** is +15–18% allocations and +12–20% bytes per element on - connector-dense models, for +10–16% time: `Load` +16% at 32 - satellites, `sysml -validate` +20–22% and +6–23 MiB at 3 000–12 000 - declarations, Apollo 11 +14% and +12 MiB. Each connector, binding and - flow end is now owned, journaled, indexed and reflected. Structural; - linear. -- **Action and state execution** is +20–60% per step with 1.3–1.5× the - allocations: `AssignmentLoopStep` +61%, `ActionLoop` +38–56%, - `StateLoop` +22–40%, `ExecuteState` +21%, `ExecuteActionFreshContext` - +38%, `RunCalc` +10–14%. The exploration of a body's statement orders, + connector-dense models: `sysml -validate` +14–15% and +5–23 MiB at + 3 000–12 000 declarations, Apollo 11 +13% and +13 MiB; the package + rows' spreads on this machine leave most of their time columns at `~` + and the allocations are the reading. Each connector, binding and flow + end is now owned, journaled, indexed and reflected. Structural; linear. +- **Action and state execution** is +40–66% per step with 1.2–1.5× the + allocations: `AssignmentLoopStep` +66%, `ActionLoop` +42–56%, + `StateLoop` +33–51%, `ExecuteState` +40%, `ExecuteActionFreshContext` + +46%, `RunCalc` +7–10%. The exploration of a body's statement orders, the per-statement binding frames and the held control values are the - semantics of the release; the per-step constant is where a 0.10.x pass - should look next. -- **Workspace edits beside a large document** still allocate 2.9–3.0× - 0.9.2's bytes (`WorkspaceEditSmallDocBesideLarge` 7.95 → 23.7 MiB, - `EditBeside/512` 2.41 → 6.66 MiB) for +20–31% time: the per-name - snapshot of re-export claims across the large neighbor. The one row - in this record with an evident fix still open. + semantics of the release; the state rows carry a further + per-executor cost at the tag — `ExecuteState` 610 → 783 KiB, + `StateLoop/count50` 605 → 802 KiB against `82f0fac87` — for the + change-trigger bookkeeping that now survives a rollback. The per-step + constant is where a 0.10.x pass should look next. +- **Lowering an action chain** costs 2.0× the bytes and 1.76× the + allocations of 0.9.2 at every length (`LowerChain`, `82f0fac87`: +15% + / `~`), `LowerActionGraph` +46% / +52%: the inherited steps, + successions, pins, gates and assertions a specialization's graph now + carries. Linear (56 allocations per step at 10 and at 1 000 steps); + `ActionChain/chain1000` +52% bytes, `~` time. The same cost reaches + `FleetInstantiate` (+13% allocations, +30% bytes, `82f0fac87`: +9.5% + / `~`) through each satellite's state machine and action. +- **Checks with arguments**: `BatchConstraints` +22% allocations and + +39% bytes, `SameConstraintManyInstances` 1 016 → 1 440 B per check + (`82f0fac87`: `~`) — a constraint's `in` parameters are bound at check + time and such a check no longer shares its verdict. +- **Workspace edits beside a large document** allocate 3.0× 0.9.2's + bytes (`WorkspaceEditSmallDocBesideLarge` 7.95 → 23.7 MiB, + `EditBeside/512` 2.41 → 6.66 MiB) for +26–56% time: the per-name + snapshot of re-export claims across the large neighbor. The one row in + this record with an evident fix still open. - **The Annex A vehicle example now emits 134 `Duplicate of imported member name` warnings** (0 in 0.9.2), and every - path that returns them pays: `Analyze/vehicle` +85%, - `GRPCParseFileUncached` +64%, `ConnectParseFileHTTPCached` 271 → 879 µs - (the HTTP transport has no cache of its JSON). Whether the rule should - fire on nested `::**` imports is a semantics question; if not, the - group is one fix away from 0.9.2. -- **Exact numerics**: `Collatz(27)` +59–65% (an allocation per Integer - multiply or divide), `Fib(25)/go` +81% and `Hypot/go` 11 → 56 ns (the - Go target's values carry their number kind), `Instantiate` +11–31% per + path that returns them pays: `Analyze/vehicle` +79%, + `GRPCParseFileUncached` +72% bytes, `ConnectParseFileHTTPCached` + 343 µs → 1.03 ms (the HTTP transport has no cache of its JSON). Whether + the rule should fire on nested `::**` imports is a semantics question; + if not, the group is one fix away from 0.9.2. +- **Exact numerics**: `Collatz(27)` +56–61% (an allocation per Integer + multiply or divide), `Fib(25)/go` +125% and `Hypot/go` 15 → 64 ns (the + Go target's values carry their number kind), `Instantiate` +7–29% per object (implicit ends and exact-number constructors). The price of the - feature. -- **Three things the candidate cannot run**, for the release notes: + feature; `Fib(25)/interpreted` is 8% faster than 0.9.2. +- **Three things 0.10.0 cannot run**, for the release notes: `SumTo(1000000)` exceeds the default 10 000 000-step budget it fit in 0.9.2 (compiled calcs now spend the interpreter's budget); the C target refuses calcs on Integers (unbounded against `int64`); the stress benchmark's per-document analysis trips the new decimal-rounding lint - on its generated `Real` literals. -- Binary 42.88 → 47.23 MiB; empty session 64 → 69 MiB, start time - unchanged. + on its generated `Real` literals. And one thing neither can: + the stress constellation's `PlaneResidency` row fails its own setup on + both revisions. +- Faster: `HydratePlane` −19–33% allocations, −8–15% time; + `ExecuteAction` −16% bytes. Binary 42.88 → 47.55 MiB; empty session + 65 → 69 MiB, start time unchanged. ## Method -Machine: `INTEL(R) XEON(R) PLATINUM 8559C`, 8 CPUs, 31 GiB, Go -`go1.25.11 linux/amd64`, `GOMAXPROCS=8`. 0.9.2 is a worktree at the tag, -built and benchmarked with its own tree; the candidate is -`hotfix/0.10.0-develop-sync` at `82f0fac87`, built and benchmarked with its -own; `main` at `ffe51571c`, where it is shown, is a third worktree treated -the same way. Package benchmarks are +Machine: `Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz`, 8 CPUs, +31 GiB, Go `go1.25.11 linux/amd64`, `GOMAXPROCS=8`. 0.9.2 is a worktree +at the `v0.9.2` tag, built and benchmarked with its own tree; 0.10.0 is a +worktree at the `v0.10.0` tag, built and benchmarked with its own; +`82f0fac87`, where it is shown, is a third worktree treated the same way. +Package benchmarks are `go test ./ -run '^$' -bench . -benchmem -count 6` on each revision, compared with `benchstat` (rows are significant at p < 0.05; `~` is no significant difference). The first pass ran each package's six counts on -0.9.2 and then on the candidate; rows it left in doubt — a significant +0.10.0 and then on 0.9.2; rows it left in doubt — a significant difference, or a spread above 20% on either side — were re-run with the revisions interleaved, `-count 1` six times each, nothing else on the machine. Where the first pass left most of a package in doubt (`internal/frontend/repl`, `tests/perf`, `tests/stressmodel`) the whole -package or the whole of its moving rows was re-run that way, and the re-run -is the table shown. A benchmark that fails on one revision (`b.Fatal`) -prints `--- FAIL` and no result line while the package's other benchmarks -run on; `benchstat` then has no row for it, and the failure is reported in -that package's section as a candidate-only or baseline-only row. Nothing -was excluded from `-bench .`. Whole-binary figures are `/usr/bin/time -f '%e %M'` -over nine interleaved runs of each binary, all built with -`make build-sysml` (`-s -w`, `-trimpath`); medians are reported. The pilot -corpora the `vehicle` rows and the gRPC warm-instantiation row read were -fetched into every tree with `scripts/download-pilot-corpora.sh`; 0.9.2's -first pass predates the download, so those rows appear in the re-run only. -Causes were read from `git log v0.9.2..82f0fac87 -- ` and, where the -history alone did not name one, from `-cpuprofile` and `-memprofile` of the -row on both revisions. +package was re-run that way, and the re-run is the table shown. A +benchmark that fails on one revision (`b.Fatal`) prints `--- FAIL` and no +result line while the package's other benchmarks run on; `benchstat` +then has no row for it, and the failure is reported in that package's +section as a candidate-only, baseline-only or failed row. Nothing was +excluded from `-bench .`. Whole-binary figures are +`/usr/bin/time -f '%e %M'` over nine interleaved runs of each binary, all +built with `make build-sysml` (`-s -w`, `-trimpath`); medians are +reported. The pilot corpora the `vehicle` rows and the gRPC +warm-instantiation row read were fetched into every tree with +`scripts/download-pilot-corpora.sh`. Causes were read from +`git log v0.9.2..v0.10.0 -- ` and, where the history alone did not +name one, from `-cpuprofile` and `-memprofile` of the row on both +revisions. ### What is and is not comparable @@ -152,525 +160,462 @@ row on both revisions. `RationalDecimalLoop`, `RationalHarmonicLoop`, `RealDecimalLoop`, `BehaviorOrdersLibraryModel`); `tests/perf` gains the eight `ImportValues` rows of the CSV/TSV/JSON value import. They have no 0.9.2 - counterpart and are listed with their candidate figures, not compared. -- **Rows the candidate cannot run.** In `internal/frontend/repl`, - `CompiledCalc/SumTo(1000000)` fails on the candidate for the interpreted - and Go targets (`evaluation step limit exceeded (10000000 steps)`), and + counterpart and are listed with their 0.10.0 figures, not compared. +- **Rows 0.10.0 cannot run.** In `internal/frontend/repl`, + `CompiledCalc/SumTo(1000000)` fails on 0.10.0 for the interpreted and + Go targets (`evaluation step limit exceeded (10000000 steps)`), and `Fib`, `SumTo` and `Collatz` for the C target are refused (`the interpreter's Integers are unbounded and a C program holds int64`); - the five rows have 0.9.2 figures and no candidate figure, and are + the five rows have 0.9.2 figures and no 0.10.0 figure, and are discussed with that package. In `tests/stressmodel`, the six - `AnalyzeSplitPerDocument` rows fail on the candidate because the - benchmark asserts a clean analysis and the per-document registry now - reports the decimal literals of the generated model as rounded by their - `Real` features; they have 0.9.2 figures and no candidate figure. -- **`InstantiateWarmModel`** (`internal/frontend/grpc`) exists on both - revisions and is compared from the interleaved re-run; 0.9.2's first - pass skipped it for want of the vehicle corpus. -- **`tests/stressmodel`**: `ConvertAPIJSON/satellites=512` completed one - count on 0.9.2 (156.0 s) before the run was killed for memory (78 GiB - allocated per operation on a 31 GiB machine) and one on the candidate - (135.6 s) before the same; it is excluded as no distribution, as in the - previous record. + `AnalyzeSplitPerDocument` rows fail on 0.10.0 because the benchmark + asserts a clean analysis and the per-document registry now reports the + decimal literals of the generated model as rounded by their `Real` + features; they have 0.9.2 figures and no 0.10.0 figure. +- **Rows neither revision can run.** `tests/stressmodel`'s four + `PlaneResidency` rows fail on both revisions in the benchmark's own + setup (`ErrRecordStale`); no figure on either side. +- **`ConvertAPIJSON/satellites=512`** allocates 73 GiB per operation; + 0.9.2's first pass was killed for memory after two counts of it (as in + the previous record), 0.10.0's completed, and the interleaved re-run + produced the row on both revisions, from which it is compared. - **Diagnostics**: on every model measured both binaries return the same exit status and the same report — `no errors` on the generated models, - the two example models and all 28 Apollo 11 files. (The candidate's + the two example models and all 28 Apollo 11 files. (0.10.0's decimal-rounding report is an opt-in lint on the command line; only the stress benchmark's registry turns it on.) +- **The machine.** This record's machine is slower than the previous + records' (see [Re-runs](#re-runs)); every time here is from both + revisions run on it, and no time is compared with an earlier record. ## Package benchmarks -Time per operation, 0.9.2 then the candidate, from the interleaved re-run +Time per operation, 0.9.2 then 0.10.0, from the interleaved re-run wherever a row was re-run (the first pass agrees in sign and roughly in size on every row named here); bytes and allocations where they moved. -Where a row moved between `ffe51571c` and the candidate, the `ffe51571c` -figure from the same interleaved run stands between them. Allocation -counts are deterministic to the unit on every revision and are the reading -that separates a change in the code from a change in the machine. +Where a row moved between `82f0fac87` — the release merge before the +state-machine and inherited-step work landed, measured in the same +interleaved runs — and the tag, the `82f0fac87` figure stands between +them. Allocation counts are deterministic to the unit on every revision +and are the reading that separates a change in the code from a change in +the machine. ### `internal/exec/runtime` -| benchmark | 0.9.2 | `ffe51571c` | 0.10.0 (`82f0fac87`) | -| --------- | ----- | ----------- | -------------------- | -| `MaterializePlainAttributes` | 45.2 µs ±4% / 42.68 KiB / 599 allocs | 53.1 µs (+17%) / 43.58 KiB / 618 | 51.5 µs ±2% (+14%, p=0.002) / 43.58 KiB (+2.1%) / 618 (+3.2%) | -| `SetFeatureValueNoDependents` | 423 ns ±28% / 352 B / 9 | 442 ns (~) | 490 ns ±24% (~, p=0.132) / 352 B / 9 | -| `DerivedReadWriteRead` | 5.39 µs ±2% / 2.484 KiB / 65 | 6.09 µs (+13%) | 5.96 µs ±2% (+11%, p=0.002) / 2.484 KiB / 65 | -| `AssignmentLoopStep` | 940 µs ±2% / 521.4 KiB / 4 923 | 1 568 µs (+67%) / 837.7 KiB (+61%) / 8 527 (+73%) | 1 509 µs ±2% (+61%, p=0.002) / 817.3 KiB (+57%) / 7 327 (+49%) | - -Candidate-only rows (no 0.9.2 counterpart): `IntegerArithmeticLoop` -3.57 ms ±59% / 279.7 KiB / 9 070 allocs; `IntegerArithmeticLoopBeyondInt64` -3.51 ms ±7% / 514.1 KiB / 13 070; `IntegerCollectionSum` 879 µs ±5% / -4.961 MiB / 44; `RationalDecimalLoop` 3.60 ms ±34% / 279.7 KiB / 9 069; -`RationalHarmonicLoop` 849 µs ±4% / 178.9 KiB / 4 650; `RealDecimalLoop` -3.75 ms ±9% / 279.7 KiB / 9 069; `BehaviorOrdersLibraryModel` 392 µs ±10% / -163.2 KiB / 1 922. They are the baseline for the next record. - -The interleaved re-run settled all three compared rows in doubt at ±2%. -**Executing one assignment statement of an action loop is +61% with 1.6 -times the bytes and 1.5 times the allocations**; materializing an -instance's plain attributes +14% with 19 more allocations, and a derived -read-write-read +11% with none. The fix on this path — the binding check -describing a body write's host only when it refuses the write -(`37b4a17c9`) — took `AssignmentLoopStep` from 8 527 to 7 327 allocations -per step (1 200 of the 3 604 the release had added, the description -strings the earlier profile showed) and 4% of its time; the remaining -+61% is the three changes the profile splits it into: the frames built -per statement to track body feature bindings (`a4e1f6ef9`), the -exploration of a body's statement orders (`71fda4cd5`, `807fcf591`, -`c8891457a`) and control-node values held with the flow that brought them -(`a0f77861d`). Those are the semantics of the release — a body's unordered -statements are now explored in every order, and that costs frames — and -not a bug; the per-step constant is where a 0.10.x pass should look next. -The `tests/perf` `ActionLoop`, `StateLoop` and `ExecuteActionFreshContext` -rows below and the REPL's `RunCalc` are the same cost from other -entrances. The 19 allocations on `MaterializePlainAttributes` and the 2.1% -bytes are the implicit relationship ends an instance now carries -(`fa8a7220c`, `ca0e6cca0`); see `Instantiate` under the REPL. +| benchmark | 0.9.2 | 0.10.0 | Δ time | Δ B/op | Δ allocs | +| --------- | ----- | ------ | ------ | ------ | -------- | +| `MaterializePlainAttributes` | 65.2 µs | 76.6 µs | +17.6% | +2.1% | 599 → 618 | +| `SetFeatureValueNoDependents` | 722 ns | 816 ns | +13% | ~ | 9 → 9 | +| `DerivedReadWriteRead` | 8.75 µs | 10.5 µs | +20% | ~ | 65 → 65 | +| `AssignmentLoopStep` | 1.38 ms | 2.30 ms | +66% | 522 → 818 KiB (+57%) | 4 924 → 7 329 (+49%) | + +`AssignmentLoopStep` runs an action whose loop body assigns an attribute +on every iteration, and is the per-step cost of the release's execution +semantics in isolation: each iteration now explores the orders of its +body's statements, holds a binding frame per statement and keeps the +control values it held across the step (`71fda4cd5`, `807fcf591`, +`a4e1f6ef9`, `a0f77861d`). The lazy binding-check description +(`37b4a17c9`) already took 1 200 allocations off the step; the +2 405 that remain are the frames. `82f0fac87` and the tag agree on this +row to the allocation (7 330 against 7 329) — the state-machine work did +not touch it. `MaterializePlainAttributes` and `DerivedReadWriteRead` are ++18–20% time for 0–19 more allocations: the exact-number constructors +(`923ec81f0`) in the value path. `SetFeatureValueNoDependents` is +13% +with identical allocations, below the spread of its single counts. + +Seven rows exist only on 0.10.0 and are listed, not compared (first +pass, this machine): `IntegerArithmeticLoop` 5.30 ms / 280 KiB / 9 071 +allocs; `IntegerArithmeticLoopBeyondInt64` 5.72 ms / 514 KiB / 13 070; +`IntegerCollectionSum` 1.77 ms / 4.96 MiB / 45; `RationalDecimalLoop` +7.07 ms / 280 KiB / 9 071; `RationalHarmonicLoop` 1.96 ms / 179 KiB / +4 651; `RealDecimalLoop` 5.58 ms / 280 KiB / 9 070; +`BehaviorOrdersLibraryModel` 629 µs / 163 KiB / 1 922. The Integer loop +costs the same as the Real loop to the allocation while its values fit a +word, and 44% more allocations once they do not. ### `internal/frontend/repl` -Attribute-dense generated models at 0, 250, 1 000 and 4 000 elements, and -the compiled-calc fixtures. 28 rows on 0.9.2, 23 on the candidate: the five -`CompiledCalc` rows the candidate cannot run are listed with their 0.9.2 -figures. The first pass left the whole package in doubt (the candidate's -`LoadModel` counts ran 11–32% apart); the moving rows were re-run -interleaved and the re-run is shown. - -| benchmark | 0.9.2 | `ffe51571c` | 0.10.0 (`82f0fac87`) | -| --------- | ----- | ----------- | -------------------- | -| `LoadModel/elements=0` | 399 µs ±20% / 216.2 KiB / 1 875 allocs | 399 µs (~) | 402 µs ±3% (~) / 221.6 KiB (+2.5%) / 1 931 (+3.0%) | -| `LoadModel/elements=250` | 28.0 ms ±8% / 13.22 MiB / 126.1 k | 30.1 ms (+7.4%) | 29.3 ms ±7% (+4.7%, p=0.015) / 13.51 MiB (+2.2%) / 130.3 k (+3.4%) | -| `LoadModel/elements=1000` | 104 ms ±17% / 52.06 MiB / 480.0 k | 117 ms (+12%) | 114 ms ±9% (+9.5%, p=0.041) / 53.25 MiB (+2.3%) / 496.8 k (+3.5%) | -| `LoadModel/elements=4000` | 458 ms ±6% / 207.2 MiB / 1.895 M | 507 ms (+11%) | 493 ms ±18% (+7.6%, p=0.009) / 212.7 MiB (+2.7%) / 1.962 M (+3.5%) | -| `RunStateMachine/elements=250` | 12.6 µs ±3% / 15.23 KiB / 213 | 13.9 µs (+10%) | 13.7 µs ±22% (+8.9%, p=0.002) / 15.52 KiB (+1.9%) / 217 (+1.9%) | -| `RunStateMachine/elements=1000` | 12.9 µs ±5% | 13.5 µs (+4.5%) | 13.2 µs ±20% (~, p=0.240) / same | -| `RunStateMachine/elements=4000` | 13.0 µs ±1% | 14.2 µs (+9.6%) | 13.7 µs ±23% (+5.8%, p=0.026) / same | -| `RunCalc/elements=250` | 4.42 µs ±6% / 4.242 KiB / 76 | 5.32 µs (+20%) | 5.02 µs ±27% (+14%, p=0.002) / 4.453 KiB (+5.0%) / 79 (+3.9%) | -| `RunCalc/elements=1000` | 4.50 µs ±5% | 5.30 µs (+18%) | 4.99 µs ±6% (+11%, p=0.002) / same | -| `RunCalc/elements=4000` | 4.49 µs ±5% | 5.31 µs (+18%) | 4.94 µs ±8% (+10%, p=0.002) / same | -| `Instantiate/elements=250` | 6.38 µs ±4% / 6.420 KiB / 53 | 8.49 µs (+33%) | 8.36 µs ±6% (+31%, p=0.002) / 7.114 KiB (+11%) / 56 (+5.7%) | -| `Instantiate/elements=1000` | 6.19 µs ±6% / 6.404 KiB | 6.96 µs (+12%) | 7.06 µs ±5% (+14%, p=0.002) / 7.197 KiB (+12%) / 56 | -| `Instantiate/elements=4000` | 6.08 µs ±4% / 6.375 KiB | 6.83 µs (+12%) | 6.73 µs ±5% (+11%, p=0.002) / 7.187 KiB (+13%) / 56 | -| `Diagnostics/attributes=50` | 86.0 µs ±8% / 72.67 KiB / 545 | — | 89.1 µs ±7% (~) / 73.60 KiB (+1.3%) / 559 (+2.6%) | -| `Diagnostics/attributes=200` | 300 µs ±14% / 248.0 KiB / 1 453 | — | 309 µs ±4% (~) / 248.9 KiB (+0.4%) / 1 467 (+1.0%) | -| `Diagnostics/attributes=800` | 1.21 ms ±8% / 895.2 KiB / 5 607 | 1.14 ms (−5.7%) | 1.21 ms ±20% (~, p=0.937) / 896.2 KiB (+0.1%) / 5 621 (+0.2%) | -| `CompiledCalc/Fib(25)/interpreted` | 7.69 ms ±8% / 42.67 KiB / 353 | 40.9 ms (+433%) / 261.2 KiB (+512%) / 2 059 (+483%) | 6.16 ms ±9% (−20%, p=0.002) / 45.73 KiB (+7.2%, p=0.041) / 381 (+7.8%, p=0.039) | -| `CompiledCalc/Fib(25)/c` | 225 µs ±5% / 1 B / 0 | refused | refused (Integer arithmetic) | -| `CompiledCalc/Fib(25)/go` | 949 µs ±13% / 7 B / 0 | 1 704 µs (+80%) | 1 720 µs ±53% (+81%, p=0.002) / 12 B (+71%) / 0 | -| `CompiledCalc/SumTo(1000000)/interpreted` | 1.549 s ±7% / 58.84 MiB / 3.043 M | step limit | step limit exceeded (10 000 000 steps) | -| `CompiledCalc/SumTo(1000000)/c` | 385 µs ±10% / 2 B / 0 | refused | refused (Integer arithmetic) | -| `CompiledCalc/SumTo(1000000)/go` | 783 µs ±4% / 5 B / 0 | step limit | step limit exceeded (10 000 000 steps) | -| `CompiledCalc/Collatz(27)/interpreted` | 275 µs ±7% / 29.30 KiB / 999 | 443 µs (+61%) | 455 µs ±6% (+65%, p=0.002) / 81.84 KiB (+179%) / 2 115 (+112%) | -| `CompiledCalc/Collatz(27)/c` | 1.01 µs ±10% / 0 B / 0 | refused | refused (Integer arithmetic) | -| `CompiledCalc/Collatz(27)/go` | 5.40 µs ±4% / 0 B / 0 | 8.36 µs (+55%) | 8.58 µs ±10% (+59%, p=0.002) / 0 B / 0 | -| `CompiledCalc/Hypot(3.0,4.0)/interpreted` | 7.10 µs ±31% / 7.846 KiB / 88 | 8.82 µs (~) | 8.21 µs ±34% (+16%, p=0.041) / 8.078 KiB (+3.0%) / 91 (+3.4%) | -| `CompiledCalc/Hypot(3.0,4.0)/c` | 13.3 ns ±14% / 0 B / 0 | 13.4 ns (~) | 13.8 ns ±4% (~, p=0.240) / 0 B / 0 | -| `CompiledCalc/Hypot(3.0,4.0)/go` | 11.0 ns ±81% / 0 B / 0 | 54.5 ns (+395%) | 56.4 ns ±2% (+412%, p=0.002) / 0 B / 0 | - -Three things are going on in this package. - -**The model loader costs 2.2–2.7% more bytes and 3.0–3.5% more -allocations per element**, and 5–10% more time (p ≤ 0.041 interleaved, -with the 4 000-element row's candidate counts 18% apart). This is the -semantic model's new bookkeeping — owned relationships journaled and -reflected as metaobjects (`da8fba4a3`, `00b8499a8`, `32890be28`), the -classification of extended declarations (`ca0e6cca0`) and the implicit-end -constancy derived once for reflection and writes (`fa8a7220c`) — and the -same per-element growth the stress constellation and `tests/perf` show -below. The qualified-name fix (`367ca7e3a`) took 5 000 allocations per -1 000 elements off `ffe51571c`'s loader (501.8 k → 496.8 k) and 1–2% of -its time. **Explained**; it is the fourth record running in which the -loader grows by a few percent per release. - -**Instantiating one object is 11–31% slower and evaluating one calc -10–14%**, three allocations more per operation each. The gRPC -`InstantiateWarmModel` row below shows the same +35% with +18% -allocations. An instance now carries the implicit relationship ends of -its type (`fa8a7220c`, `ca0e6cca0`) and its attribute values go through -the exact-number constructors (`79629923b`, `923ec81f0`); the bytes rise -11–13% on these small objects and fall on the stress constellation's -fleet, where the Integer and Rational fast paths' allocation-free -representation (`d1eb16edb`) wins. `RunCalc` was +18–20% on `ffe51571c`; -the memoized succession-cycle check (`8bd97fcf6`) took a third of that -off. **Explained.** - -**The compiled calcs pay for unbounded Integers, exact Rationals and the -step budget.** Each of the four fixtures is affected differently, and the -benchmark's own fixture file grew from 597 to 930 lines (the same four -calcs; the new content is other compiled fixtures), which the -`LoadModel`-sized setup absorbs but the `FormatEdits` row in -`internal/frontend/lsp` does not: - -- `Fib(25)/interpreted` **is 20% faster than 0.9.2** with 7% more - allocations. On `ffe51571c` it was 5.3 times slower and allocated six - times the objects: the exact-Rational evaluation (`923ec81f0`) boxed - every scalar a node yielded and paired the leaves of each arithmetic - node, 250 000 calls deep. Packing a compiled calc's scalars into three - words (`4a1b937f2`) removed the boxing — 2 059 → 381 allocations per - call — and the row is now the fastest of the three records it appears - in. The steepest row of the earlier measurement is cleared. -- `Collatz(27)/interpreted` is +65% with 2.1 times the allocations, and - the fix did not reach it: `Collatz` multiplies and divides Integers, - and the unbounded-Integer path (`79629923b`) still allocates a result - per operation where `Fib`'s additions stay in the packed word (999 → - 2 115 allocations is one per arithmetic node). `Collatz(27)/go` is +59% - with none on either side, and `Hypot(3.0,4.0)/go` goes from 11 ns to - 56 ns, also allocation-free: the Go target's compiled values now keep - an Integer an Integer inside a Real-typed value and hold negative powers - as exact Rationals (`b69ea6840`, `0aa4aca55`), so each arithmetic node - carries its number kind and branches on it, and compiled calcs spend - the interpreter's step budget (`d6eafc870`). The price of exactness; the - Integer multiply path is where a profile-driven pass should look next. -- `SumTo(1000000)` sums a million integers in a loop. On 0.9.2 the - interpreted run took 1.55 s inside the default budget of 10 000 000 - steps; on the candidate the same loop exceeds it, for the interpreted - and the Go target both, because compiled calcs now spend the - interpreter's step budget (`d6eafc870`) and the interpreter counts the - statements of a loop body it explores in order (`71fda4cd5`, - `43548d741`). The row is not a timing regression but a change in what - fits the default budget: a loop of a million iterations no longer does - without `OPENSYSML_MAX_STEPS`. -- The C target refuses every calc that does Integer arithmetic (`Fib`, - `SumTo`, `Collatz`) because a C `int64` cannot hold the interpreter's - unbounded Integers (`79629923b`); only `Hypot`, on Reals, still - compiles to C. This is a deliberate narrowing of the C target, not a - performance figure, and belongs in the release notes. +| benchmark | 0.9.2 | 0.10.0 | Δ time | Δ B/op | Δ allocs | +| --------- | ----- | ------ | ------ | ------ | -------- | +| `LoadModel/elements=0` | 586 µs | 607 µs | ~ | +2.5% | +2.9% | +| `LoadModel/elements=250` | 40.6 ms | 42.8 ms | ~ | +2.3% | +3.6% | +| `LoadModel/elements=1000` | 161 ms | 172 ms | +7.0% | +2.3% | +3.8% | +| `LoadModel/elements=4000` | 710 ms | 774 ms | +9.0% | +2.8% | +3.8% | +| `RunStateMachine/elements=250…4000` | 20.9 µs | 22.5 µs | +6.7–8.2% | 15.23 → 15.93 KiB | 213 → 223 | +| `RunCalc/elements=250…4000` | 6.6–6.7 µs | 7.2–7.3 µs | +7.1–9.9% | 4.24 → 4.45 KiB | 76 → 79 | +| `Instantiate/elements=250` | 8.73 µs | 11.3 µs | +29% | +12.9% | 53 → 56 | +| `Instantiate/elements=1000` | 8.27 µs | 9.31 µs | +13% | +12.4% | 53 → 56 | +| `Instantiate/elements=4000` | 8.39 µs | 8.97 µs | +6.9% | +12.4% | 53 → 56 | +| `Diagnostics/attributes=50…800` | 155 µs – 1.91 ms | 145 µs – 1.99 ms | ~ | +0.1–1.3% | +0.3–2.6% | +| `CompiledCalc/Fib(25)/interpreted` | 11.1 ms | 10.2 ms | **−8.4%** | +18% | 489 → 574 | +| `CompiledCalc/Fib(25)/go` | 1.02 ms | 2.30 ms | +125% | 7 → 16 B | 0 | +| `CompiledCalc/Collatz(27)/interpreted` | 399 µs | 644 µs | +61% | 29.9 → 82.8 KiB (+177%) | 1 004 → 2 122 (+111%) | +| `CompiledCalc/Collatz(27)/go` | 7.31 µs | 11.4 µs | +56% | 0 | 0 | +| `CompiledCalc/Hypot(3.0,4.0)/interpreted` | 10.7 µs | 12.1 µs | +13% | +2.9% | 88 → 91 | +| `CompiledCalc/Hypot(3.0,4.0)/c` | 21.3 ns | 22.3 ns | +4.8% | 0 | 0 | +| `CompiledCalc/Hypot(3.0,4.0)/go` | 15.3 ns | 63.6 ns | +317% | 0 | 0 | + +Five rows have a 0.9.2 figure and no 0.10.0 figure, as in the previous +measurement of the candidate: `CompiledCalc/SumTo(1000000)` for the +interpreted and Go targets fails with +`evaluation step limit exceeded (10000000 steps)` (0.9.2: 3.23 s and +866 µs) because a compiled calc now spends the interpreter's step budget, +and `Fib(25)`, `SumTo(1000000)` and `Collatz(27)` for the C target are +refused (`the interpreter's Integers are unbounded and a C program holds +int64`; 0.9.2: 286 µs, 540 µs, 1.23 µs). Both belong in the release +notes; neither is a performance regression. + +The interpreted `Fib(25)` is the one row the six fixes turned around: the +three-word scalar packing (`4a1b937f2`) took it from 5.3× 0.9.2 to 8% +faster on this machine (20% on the earlier one; the spread is ±4% on +both sides). `Collatz(27)/interpreted` doubles its allocations: an +Integer multiply or divide allocates its exact result (`923ec81f0`, +`79629923b`), where 0.9.2 did `float64` arithmetic in place. +`Fib(25)/go` and `Hypot/go` are the Go target's values carrying their +number kind (`d6eafc870`): a tagged word per operation instead of a bare +`float64`, 15 → 64 ns on a two-operation calc. `Instantiate` is +12.4% +bytes per object for the implicit relationship ends (`fa8a7220c`, +`ca0e6cca0`), and `RunStateMachine` gained 6 allocations per run at the +tag — the change-trigger bookkeeping journaled per transition +(`bc1baced5`; `82f0fac87` had 217). `LoadModel` is the loader's +3.8% +allocations per element on a connector-light generated model (the +constellation below shows the connector-dense figure). ### `internal/frontend/grpc` -| benchmark | 0.9.2 | `ffe51571c` | 0.10.0 (`82f0fac87`) | -| --------- | ----- | ----------- | -------------------- | -| `ParseFileColdShared` | 10.7 ms ±34% / 4.689 MiB / 31.12 k allocs | 7.67 ms (~) | 7.61 ms ±6% (~, p=0.394) / 4.700 MiB (+0.2%) / 31.29 k (+0.5%) | -| `ParseFileColdInline` | 7.69 ms ±29% / 4.916 MiB / 31.47 k | 7.74 ms (~) | 7.66 ms ±8% (~, p=0.818) / 4.945 MiB (~) / 31.71 k (+0.8%) | -| `InstantiateWarmModel` | 150 µs ±12% / 83.06 KiB / 1 555 allocs | 211 µs (+41%) / 104.02 KiB / 1 838 | 198 µs ±10% (+32%, p=0.002) / 104.15 KiB (+25%) / 1 836 (+18%) | - -The cold-parse rows are the parser and the loader on the vehicle example -and move with the loader's few percent of allocations; the time is `~` on -every revision. The warm instantiation is the `Instantiate` row of the -REPL at the service's scale: 281 more objects per instantiation, the same -implicit relationship ends and exact-number constructors on the vehicle -example's objects. The cached-diagnostics fix (`70242d872`) does not -touch these rows; its effect is on `tests/perf`'s `GRPCParseFileCached` -and `ConnectParseFileHTTPCached` below. +| benchmark | 0.9.2 | 0.10.0 | Δ time | Δ B/op | Δ allocs | +| --------- | ----- | ------ | ------ | ------ | -------- | +| `InstantiateWarmModel` | 200 µs | 291 µs | +46% | 83.4 → 107 KiB (+28%) | 1 555 → 1 862 (+20%) | +| `ParseFileColdShared` | 10.7 ms | 11.0 ms | ~ | +0.2% | +0.5% | +| `ParseFileColdInline` | 11.0 ms | 11.3 ms | ~ | +0.7% | +0.9% | + +`InstantiateWarmModel` instantiates the vehicle example's parts over a +warm session: +20% allocations, the implicit relationship ends and the +exact-number attribute values (`fa8a7220c`, `923ec81f0`), unchanged from +`82f0fac87` (1 860). The cold parse rows are the parser and loader on a +small file and are within 1%. ### `internal/frontend/lsp` -| benchmark | 0.9.2 | `ffe51571c` | 0.10.0 (`82f0fac87`) | -| --------- | ----- | ----------- | -------------------- | -| `FormatEdits` | 512 µs ±4% / 433.7 KiB / 78 allocs | 891 µs (+74%) / 704.3 KiB / 85 | 878 µs ±5% (+71%, p=0.002) / 704.3 KiB (+62%) / 85 (+9.0%) | -| `SpanToRangeLarge` | 2.99 ms ±17% / 0 B | — | 2.92 ms ±40% (~) / 0 B | -| `ReferencesWarm` | 206 µs ±7% / 339.4 KiB / 1 017 | — | 204 µs ±2% (~) / 339.4 KiB / 1 017 | -| `ReferencesCold` | 45.4 ms ±19% / 9.713 MiB / 44.51 k | — | 39.5 ms ±17% (~) / 9.720 MiB (~) / 44.51 k (~) | -| `RenameWarm` | 16.5 ms ±11% / 1.252 MiB / 12.75 k | 16.9 ms (~) | 16.5 ms ±6% (~, p=0.818) / 1.244 MiB (~) / 12.62 k (~) | -| `WorkspaceUpdate` | 194 ms ±7% / 49.38 MiB / 605.6 k | 262 ms (+35%) / 98.50 MiB (+99%) / 930.6 k (+54%) | 224 ms ±4% (+16%, p=0.002) / 57.18 MiB (+16%) / 710.0 k (+17%) | +| benchmark | 0.9.2 | 0.10.0 | Δ time | Δ B/op | Δ allocs | +| --------- | ----- | ------ | ------ | ------ | -------- | +| `FormatEdits` | 639 µs | 1.13 ms | +78% | 434 → 704 KiB (+62%) | 78 → 85 | +| `SpanToRangeLarge` | 3.54 ms | 3.44 ms | −3.0% | 0 | 0 | +| `ReferencesWarm` | 303 µs | 284 µs | ~ | ~ | ~ | +| `ReferencesCold` | 58.1 ms | 58.1 ms | ~ | +0.1% | ~ | +| `RenameWarm` | 25.9 ms | 25.6 ms | ~ | ~ | ~ | +| `WorkspaceUpdate` | 324 ms | 402 ms | +24% | 49.7 → 57.0 MiB (+15%) | 607k → 712k (+17%) | `FormatEdits` formats `internal/frontend/repl/testdata/compile_calcs.sysml`, -which grew from 597 to 930 lines on the candidate with the new compiled -fixtures; the bytes move by the ratio of the file sizes (+62% against +56% -more lines) and the formatter itself has one commit in the interval (the -indexed connector ends, `63552cbc4`). **Not a regression in the code.** -References and rename are unchanged, to the allocation. - -`WorkspaceUpdate` — one edit to one of the workspace's documents, with the -re-analysis the server does for it — is where the re-export fix landed. -On `ffe51571c` it allocated twice the bytes of 0.9.2 and 1.5 times the -objects, 39% of them in `symbols.registrationOf` deep-copying every -document's re-export claims before each name's re-registration -(`4ca321508`, `fe6e01fb9`); with the claims compared in place instead of -copied (`4b4b35121`) the row is **+16% time, +16% bytes and +17% -allocations** against 0.9.2 — 41 MiB per update less than `ffe51571c`, -and the first-pass spread of ±46% on 0.9.2 settled to ±7% interleaved. The -remainder is the resolver's new `checkImportedNames` (`eb4b7583c`) and -the loader's per-element growth, which the `tests/perf` and stress edit -rows below share. **Explained.** +which grew from 597 to 930 lines on 0.10.0 with the new compiled +fixtures; the bytes move by the ratio of the file sizes (+62% against ++56% more lines) and the formatter itself has one commit in the interval +(the indexed connector ends, `63552cbc4`). Not a regression in the code. +`WorkspaceUpdate` edits a document in +a workspace of 128 and re-analyses: +17% allocations is the symbol +index's owned connector ends and the diagnostics of the re-analysis +(`00b8499a8`, `da8fba4a3`); the re-export claim fix (`4b4b35121`) took it +from +53% to +17%. Both unchanged from `82f0fac87` to the allocation. +References and rename are unchanged. ### `internal/translate/migrate`, `internal/workspace/libs`, `internal/workspace/model` -| benchmark | 0.9.2 | 0.10.0 (`82f0fac87`) | -| --------- | ----- | -------------------- | -| `migrate` `WriterSiblingBlocks` | 1.93 ms ±7% / 1.906 MiB / 20.02 k allocs | 1.90 ms ±9% (~) / 1.906 MiB (~) / 20.02 k (~) | -| `libs` `ExpandWildcardImports` | 27.4 ms ±1% / 14.46 MiB / 128.3 k | 28.6 ms ±3% (+4.2%) / 15.36 MiB (+6.2%) / 145.7 k (+14%) | -| `libs` `IndexLibrary` | 16.3 ms ±15% / 13.56 MiB / 104.6 k | 16.2 ms ±14% (~, p=0.818) / 14.07 MiB (+3.8%) / 108.2 k (+3.4%) | -| `libs` `ExpandModelImports` | 6.50 ms ±11% / 4.315 MiB / 36.16 k | 6.67 ms ±4% (~, p=0.485) / 4.376 MiB (+1.4%) / 38.15 k (+5.5%) | -| `libs` `DecodeSnapshot` | 9.89 ms ±17% / 35.32 MiB / 56.63 k | 10.8 ms ±21% (~, p=0.132) / 37.85 MiB (+7.2%) / 67.45 k (+19%) | -| `libs` `SetDigest` | 629 µs ±12% / 1.710 MiB / 966 | 641 µs ±7% (~) / 1.733 MiB (+1.4%) / 998 (+3.3%) | -| `model` `AnalyseUnresolved` | 20.2 ms ±8% / 13.69 MiB / 149.3 k | 20.6 ms ±6% (~) / 13.80 MiB (+0.8%) / 150.7 k (+1.0%) | -| `model` `AnalyseResolved` | 1.51 ms ±22% / 543.0 KiB / 5 049 | 1.59 ms ±12% (~, p=0.699) / 590.4 KiB (+8.7%) / 5 971 (+18%) | - -None of these rows moved between `ffe51571c` and the candidate, to the -allocation. The migration writer is unchanged. Indexing the standard -library allocates 3.4% more objects (the reflective relationship ends, the -`StateActivity` extension library `0188fb308`) and decoding its snapshot -19% more: the snapshot carries the owned-relationship journal and the -reflected implicit relationships (`da8fba4a3`, `32890be28`) that the 0.9.2 -snapshot did not hold; the time of both is `~` interleaved. Expanding the -library's wildcard imports allocates 14% more objects and is +4.2% in time -interleaved (+15% in the first pass, with the candidate's counts 20% -apart); the import-ambiguity rule (`eb4b7583c`) now walks what each -wildcard brings in. Expanding a model's imports is `~` in time interleaved -(+9.0% in the first pass). Analysing a resolved document allocates 18% more with the time -`~` interleaved (first pass +12%): the constraint-tier passes added in the -interval — the indexed-end multiplicity check (`cf85beb36`), the -import-ambiguity warning (`eb4b7583c`), the reflective derivations for the -batch constraints (`70bfa8ee2`) — each walk the document once more. The -`tests/perf` `Analyze` rows below show the same +3.3% allocations on the -synthetic model and much more on the vehicle example. +| benchmark | 0.9.2 | 0.10.0 | Δ time | Δ B/op | Δ allocs | +| --------- | ----- | ------ | ------ | ------ | -------- | +| migrate `WriterSiblingBlocks` | 2.23 ms | 3.03 ms | +36%¹ | ~ | 20 022 → 20 022 | +| libs `ExpandWildcardImports` | 42.3 ms | 73.6 ms | +74%¹ | +6.2% | +13.5% | +| libs `IndexLibrary` | 21.1 ms | 22.5 ms | +6.6% | +3.8% | +3.4% | +| libs `ExpandModelImports` | 10.7 ms | 15.8 ms | +48%¹ | +1.4% | +5.5% | +| libs `DecodeSnapshot` | 14.0 ms | 14.7 ms | ~ | 35.3 → 37.9 MiB (+7.2%) | 56.6k → 67.5k (+19%) | +| libs `SetDigest` | 772 µs | 996 µs | +29%¹ | +1.6% | +3.9% | +| model `AnalyseUnresolved` | 29.7 ms | 44.8 ms | +51%¹ | +0.8% | +0.9% | +| model `AnalyseResolved` | 1.72 ms | 1.88 ms | +9.8% | +8.7% | 5 049 → 5 970 (+18%) | + +¹ First-pass figure (this machine, `-count 6` on each revision in turn, +not interleaved); the rows' allocations are unchanged or within 14%, and +their 20–40% single-count spreads on this machine leave the time column +unconfirmed. The interleaved rows (`IndexLibrary`, `DecodeSnapshot`, +`AnalyseResolved`) are the ones to read. `DecodeSnapshot` is the bundled +standard library's index decoded at start, 2.5 MiB and 10 800 +allocations more per decode for the owned-relationship journal and the +reflected implicit relationships in the snapshot (`da8fba4a3`, +`fa8a7220c`); it is the session floor's +4 MiB below. `AnalyseResolved` +is +18% allocations for the connector ends the symbol table now owns and +indexes. `WriterSiblingBlocks` allocates identically on both sides. ### `tests/perf` -The synthetic 4 000-part model and the Annex A vehicle example through -every entrance: lexer, parser, index, analysis, workspace edits, REPL, -lowering, execution, constraints, the gRPC service in-process and over -HTTP. 49 rows on 0.9.2 and 57 on the candidate (the eight `ImportValues` -rows are new). The first pass left most of the package in doubt, so the -whole package was re-run interleaved across the three revisions; that -re-run is the table. The vehicle rows need the pilot corpus, which 0.9.2's -first pass predates, so those have the re-run only. - -| benchmark | 0.9.2 | `ffe51571c` | 0.10.0 (`82f0fac87`) | -| --------- | ----- | ----------- | -------------------- | -| `Lex/synthetic` | 8.16 ms ±14% / 483.0 k allocs | 7.84 ms (~) | 7.97 ms ±6% (~) / 483.0 k | -| `Lex/vehicle` | 277 µs ±17% / 10.85 k | 268 µs (~) | 272 µs ±8% (~) / 10.85 k | -| `Parse/synthetic` | 99.0 ms ±16% / 293.2 k | 89.0 ms (~) | 88.5 ms ±25% (~) / 293.2 k | -| `Parse/vehicle` | 2.51 ms ±12% / 7 559 | 2.59 ms (~) | 2.49 ms ±9% (~) / 7 559 | -| `IndexAddExpand/synthetic` | 51.4 ms ±19% / 294.6 k | 51.7 ms (~) | 51.6 ms ±8% (~) / 317.6 k (+7.8%) | -| `IndexAddExpand/vehicle` | 27.3 ms ±33% / 131.2 k | 27.7 ms (~) | 28.1 ms ±18% (~) / 139.7 k (+6.5%) | -| `IndexAddOnly/synthetic` | 45.8 ms ±20% / 283.0 k | 47.5 ms (~) | 48.0 ms ±17% (~) / 306.6 k (+8.3%) | -| `IndexAddOnly/vehicle` | 1.14 ms ±10% / 9 277 | 1.13 ms (~) | 1.18 ms ±11% (~) / 9 633 (+3.8%) | -| `Analyze/synthetic` | 1.26 s ±20% / 466.3 MiB / 4.435 M | 1.33 s (~) / 487.2 MiB / 4.644 M | 1.35 s ±21% (+7.5%, p=0.041) / 484.5 MiB (+3.9%) / 4.582 M (+3.3%) | -| `Analyze/vehicle` | 117 ms ±17% / 73.65 MiB / 627.6 k | 221 ms (+89%) / 130.0 MiB / 865.3 k | 216 ms ±22% (+85%, p=0.002) / 129.9 MiB (+76%) / 869.3 k (+39%) | -| `WorkspaceEdit/synthetic` | 210 ms ±10% / 77.97 MiB / 596.3 k | 278 ms (+32%) / 85.84 MiB / 680.6 k | 285 ms ±68% (+35%, p=0.002) / 85.74 MiB (+10%) / 680.5 k (+14%) | -| `WorkspaceEdit/synthetic/reindex+diagnostics` | 1.62 s ±5% / 493.5 MiB / 4.787 M | 1.82 s (+13%) / 519.6 MiB / 5.089 M | 1.80 s ±16% (+11%, p=0.002) / 517.4 MiB (+4.8%) / 5.027 M (+5.0%) | -| `WorkspaceEdit/vehicle` | 56.5 ms ±12% / 12.33 MiB / 139.1 k | 78.6 ms (+39%) / 22.94 MiB (+86%) / 214.4 k (+54%) | 67.9 ms ±11% (+20%, p=0.002) / 14.60 MiB (+18%) / 168.1 k (+21%) | -| `WorkspaceEdit/vehicle/reindex+diagnostics` | 176 ms ±7% / 71.24 MiB / 651.7 k | 262 ms (+49%) / 129.7 MiB (+82%) / 917.6 k (+41%) | 250 ms ±10% (+42%, p=0.002) / 121.0 MiB (+70%) / 867.1 k (+33%) | -| `WorkspaceEditSmallDocBesideLarge` | 45.4 ms ±34% / 7.952 MiB / 81.74 k | 69.5 ms (+53%) / 31.24 MiB (+293%) / 162.6 k (+99%) | 59.4 ms ±35% (+31%, p=0.026) / 23.67 MiB (+198%) / 114.6 k (+40%) | -| `FQNOf` | 1.59 ms ±32% / 36.00 k | 1.68 ms (~) | 1.62 ms ±12% (~) / 40.00 k (+11%) | -| `LookupQualified` | 1.22 ms ±34% / 11.00 k | 1.26 ms (~) | 1.26 ms ±14% (~) / 11.00 k | -| `FeaturesOf` | 1.01 s ±11% / 4.154 M | 1.03 s (~) | 1.06 s ±13% (~) / 4.156 M (~) | -| `REPLLoadFile` | 1.53 s ±7% / 595.5 MiB / 5.383 M | 1.63 s (~) / 616.7 MiB / 5.595 M | 1.64 s ±10% (~, p=0.065) / 613.0 MiB (+2.9%) / 5.533 M (+2.8%) | -| `REPLSubmitSnippet` | 1.10 s ±9% / 4.805 M | 1.22 s (+11%) / 5.048 M | 1.25 s ±18% (+14%, p=0.026) / 4.986 M (+3.8%) | -| `REPLEvalExpr` | 4.76 ms ±15% / 101 | 4.66 ms (~) | 4.86 ms ±8% (~) / 104 (+3.0%) | -| `LowerActionGraph` | 1.85 µs ±11% / 25 | 2.16 µs (+17%) | 2.15 µs ±7% (+16%, p=0.004) / 27 (+8.0%) | -| `LowerStateGraph` | 9.02 µs ±13% / 103 | 8.48 µs (~) | 9.08 µs ±17% (~) / 103 | -| `ExecuteAction` | 79.9 µs ±10% / 192 | 86.7 µs (~) / 254 | 80.4 µs ±5% (~, p=0.937) / 234 (+22%) | -| `ExecuteActionFreshContext` | 31.0 µs ±16% / 268 | 41.8 µs (+35%) / 347 | 42.8 µs ±9% (+38%, p=0.002) / 327 (+22%) | -| `ExecuteState` | 729 µs ±12% / 483.8 KiB / 10.38 k | 903 µs (+24%) / 608.9 KiB / 11.66 k | 880 µs ±6% (+21%, p=0.002) / 607.5 KiB (+26%) / 11.61 k (+12%) | -| `BatchConstraints` | 9.83 ms ±26% / 2.241 MiB / 55.26 k | 16.5 ms (+68%) / 2.424 MiB (+8.1%) / 55.99 k (+1.3%) | 10.5 ms ±4% (~, p=0.093) / 2.240 MiB (~) / 55.35 k (+0.2%) | -| `SameConstraintManyInstances` | 3.85 µs ±11% / 48 | 4.40 µs (+14%) | 4.38 µs ±6% (+14%, p=0.015) / 48 | -| `BatchSatisfy` | 1.27 s ±13% / 4.211 M | 1.33 s (~) | 1.33 s ±6% (~, p=0.310) / 4.354 M (+3.4%) | -| `Instantiate` | 525 ms ±12% / 174.0 MiB / 1.871 M | 574 ms (~) | 563 ms ±16% (~, p=0.240) / 180.2 MiB (+3.5%) / 1.955 M (+4.5%) | -| `GRPCParseFileCached` | 63.4 µs ±23% / 216.7 KiB / 22 | 77.5 µs (+22%) / 242.0 KiB (+12%) / 299 (+1 259%) | 66.0 µs ±5% (~, p=0.485) / 218.0 KiB (+0.6%) / 24 (+9.1%) | -| `GRPCParseFileUncached` | 131 ms ±12% / 661.3 k | 225 ms (+71%) / 905.1 k | 215 ms ±20% (+64%, p=0.002) / 909.0 k (+37%) | -| `GRPCEvaluate` | 9.04 µs ±7% / 110 | 10.6 µs (+18%) | 10.1 µs ±13% (+11%, p=0.004) / 116 (+5.5%) | -| `GRPCVerifyConstraint` | 831 µs ±16% / 9 188 | 1 047 µs (+26%) | 1 009 µs ±43% (+21%, p=0.009) / 9 768 (+6.3%) | -| `ConnectEvaluateHTTP` | 153 µs ±6% / 201 | 151 µs (~) | 153 µs ±4% (~) / 206 (+2.5%) | -| `ConnectParseFileHTTPCached` | 271 µs ±7% / 351.6 KiB / 166 | 926 µs (+241%) / 491.7 KiB / 1 292 | 879 µs ±24% (+224%, p=0.002) / 465.2 KiB (+32%) / 1 017 (+513%) | -| `ActionLoop/for10` | 25.8 µs ±10% / 204 | 36.5 µs (+41%) / 273 | 35.7 µs ±19% (+38%, p=0.002) / 253 (+24%) | -| `ActionLoop/for100` | 104 µs ±31% / 477 | 166 µs (+59%) / 816 | 163 µs ±17% (+56%, p=0.002) / 616 (+29%) | -| `ActionLoop/for1000` | 954 µs ±14% / 279.0 KiB / 3 181 | 1 369 µs (+44%) / 449.3 KiB (+61%) / 6 220 (+96%) | 1 332 µs ±19% (+40%, p=0.002) / 410.2 KiB (+47%) / 4 220 (+33%) | -| `ActionChain/chain10` | 94.7 µs ±13% / 675 | 111 µs (+18%) | 105 µs ±22% (~, p=0.065) / 764 (+13%) | -| `ActionChain/chain100` | 575 µs ±20% / 5 757 | 735 µs (+28%) | 712 µs ±7% (+24%, p=0.009) / 6 394 (+11%) | -| `ActionChain/chain1000` | 14.3 ms ±22% / 56.28 k | 16.3 ms (~) | 15.7 ms ±9% (~, p=0.180) / 62.83 k (+12%) | -| `StateLoop/count50` | 706 µs ±7% / 10.37 k | 870 µs (+23%) | 861 µs ±22% (+22%, p=0.002) / 11.60 k (+12%) | -| `StateLoop/count500` | 6.45 ms ±10% / 100.9 k | 8.95 ms (+39%) | 9.03 ms ±21% (+40%, p=0.002) / 112.9 k (+12%) | -| `StateLoop/count5000` | 64.9 ms ±14% / 45.27 MiB / 1.006 M | 85.4 ms (+32%) | 85.5 ms ±12% (+32%, p=0.002) / 57.16 MiB (+26%) / 1.126 M (+12%) | -| `LowerChain/chain10` | 8.72 µs ±15% / 73 | 10.5 µs (+21%) | 10.2 µs ±17% (+17%, p=0.015) / 78 (+6.8%) | -| `LowerChain/chain100` | 149 µs ±14% / 460 | 159 µs (+7.0%) | 162 µs ±19% (+8.5%, p=0.026) / 471 (+2.4%) | -| `LowerChain/chain1000` | 9.13 ms ±19% / 4 115 | 8.74 ms (−4.3%) | 9.99 ms ±13% (~, p=0.699) / 4 136 (+0.5%) | - -Candidate-only rows (first pass, no 0.9.2 counterpart), the CSV/TSV/JSON -value import of `-import-values`: `ImportValues/flat/parts=100/attrs=3` -22.1 ms ±5% / 131.0 k allocs; `/flat/parts=500/attrs=3` 107 ms ±13% / -615.0 k; `/flat/parts=2000/attrs=3` 513 ms ±42% / 2.433 M; -`/flat/parts=2000/attrs=5` 743 ms ±25% / 3.672 M; -`/nested/parts=100/attrs=3` 30.1 ms ±5% / 175.4 k; -`/nested/parts=500/attrs=3` 145 ms ±5% / 829.4 k; -`/nested/parts=2000/attrs=3` 664 ms ±57% / 3.284 M; -`/nested/parts=2000/attrs=5` 1.05 s ±18% / 4.569 M. Linear in parts × -attributes at roughly 400 allocations per value (437 at 300 values, 367 -at 10 000); the nested layout costs a third more than the flat one. +The harness's end-to-end rows: the vehicle example (Annex A) and a +synthetic 4 000-part model through lexing, parsing, indexing, analysis, +workspace edits, execution, constraints, the gRPC and HTTP transports. +Time per operation, 0.9.2, `82f0fac87` and 0.10.0, from the interleaved +re-run of the whole package; bytes and allocations against 0.9.2. + +| benchmark | 0.9.2 | `82f0fac87` | 0.10.0 | Δ B/op | Δ allocs | +| --------- | ----- | ----------- | ------ | ------ | -------- | +| `Lex/synthetic`, `Lex/vehicle` | 9.85 ms, 370 µs | ~ | ~ | 0 | 0 | +| `Parse/synthetic`, `Parse/vehicle` | 118 ms, 3.20 ms | ~ | ~ | +0.0% | +0.0% | +| `IndexAddExpand/synthetic` | 83.4 ms | ~ | ~ | +6.4% | +7.6% | +| `IndexAddExpand/vehicle` | 42.7 ms | ~ | ~ | +1.9% | +6.5% | +| `IndexAddOnly/synthetic` | 70.9 ms | ~ | ~ | ~ | ~ | +| `IndexAddOnly/vehicle` | 1.54 ms | ~ | ~ | +4.3% | +3.8% | +| `Analyze/synthetic` | 1.89 s | ~ | ~ | +4.5% | +4.0% | +| `Analyze/vehicle` | 185 ms | 338 ms (+83%) | 330 ms (+79%) | 73.7 → 130 MiB (+76%) | 628k → 871k (+39%) | +| `WorkspaceEdit/synthetic` | 292 ms | 379 ms (+30%) | 461 ms (+58%)¹ | +10% | +14% | +| `WorkspaceEdit/synthetic/reindex+diagnostics` | 2.37 s | ~ | ~ | +5.3% | +5.7% | +| `WorkspaceEdit/vehicle` | 83.2 ms | 107 ms (+29%) | 106 ms (+28%) | +18% | +21% | +| `WorkspaceEdit/vehicle/reindex+diagnostics` | 253 ms | 352 ms (+39%) | 422 ms (+66%)¹ | 71.3 → 121 MiB (+70%) | +33% | +| `WorkspaceEditSmallDocBesideLarge` | 67.8 ms | 95.5 ms (+41%) | 106 ms (+56%) | 7.95 → 23.7 MiB (+198%) | 81.7k → 115k (+40%) | +| `FQNOf` | 2.00 ms | +18% | 2.45 ms (+23%) | +9.5% | 36 000 → 40 000 | +| `LookupQualified` | 1.97 ms | ~ | ~ | 0 | 0 | +| `FeaturesOf` | 1.41 s | +4.5% | ~ | ~ | ~ | +| `REPLLoadFile` | 2.16 s | +9.3% | 2.45 s (+14%) | +3.4% | +3.4% | +| `REPLSubmitSnippet` | 1.65 s | +13% | 1.93 s (+17%) | +6.6% | +4.4% | +| `REPLEvalExpr` | 5.37 ms | ~ | ~ | +0.0% | 101 → 104 | +| `LowerActionGraph` | 2.67 µs | ~ (2.91 KiB, 27) | 4.52 µs (+69%) | 2.63 → 3.84 KiB (+46%) | 25 → 38 (+52%) | +| `LowerStateGraph` | 13.0 µs | ~ | ~ | +1.7% | 146 → 149 | +| `ExecuteAction` | 98.6 µs | ~ | ~ | −16% | 193 → 244 (+26%) | +| `ExecuteActionFreshContext` | 48.2 µs | 67.0 µs (+39%) | 70.1 µs (+46%) | 26.4 → 35.9 KiB (+36%) | 268 → 338 (+26%) | +| `ExecuteState` | 1.13 ms | ~ (610 KiB, 11 620) | 1.58 ms (+40%) | 486 → 783 KiB (+61%) | 10 390 → 12 140 (+17%) | +| `BatchConstraints` | 18.5 ms | ~ (2.63 MiB, 56 720) | 23.2 ms (+26%) | 2.53 → 3.52 MiB (+39%) | 56.3k → 68.9k (+22%) | +| `SameConstraintManyInstances` | 5.39 µs | ~ (1 000 B, 29) | ~ | 1 016 → 1 440 B (+42%) | 29 → 35 (+21%) | +| `BatchSatisfy` | 1.76 s | ~ | ~ | +3.7% | +3.7% | +| `Instantiate` | 1.20 s | ~ | +6.4% | +4.6% | +4.0% | +| `GRPCParseFileCached` | 94.4 µs | ~ | ~ | +0.6% | 22 → 24 | +| `GRPCParseFileUncached` | 197 ms | ~ | ~¹ | 83.5 → 144 MiB (+72%) | 664k → 913k (+38%) | +| `GRPCEvaluate` | 13.0 µs | ~ | ~ | +12% | 110 → 116 | +| `GRPCVerifyConstraint` | 1.28 ms | ~ | ~ | +6.4% | +6.5% | +| `ConnectEvaluateHTTP` | 158 µs | ~ | ~ | +3.0% | 201 → 206 | +| `ConnectParseFileHTTPCached` | 343 µs | 1.03 ms (+201%) | 1.03 ms (+201%) | 352 → 462 KiB (+31%) | 166 → 1 017 (+513%) | +| `ActionLoop/for10` | 41.5 µs | +43% | 64.6 µs (+56%) | +39% | 204 → 263 (+29%) | +| `ActionLoop/for100` | 169 µs | +43% | 239 µs (+42%) | +40% | 477 → 626 (+31%) | +| `ActionLoop/for1000` | 1.42 ms | ~ | ~¹ | 279 → 411 KiB (+47%) | 3 181 → 4 231 (+33%) | +| `ActionChain/chain10` | 125 µs | ~ | ~ | +13% | 675 → 805 (+19%) | +| `ActionChain/chain100` | 857 µs | +26% | 1.17 ms (+37%) | 513 → 756 KiB (+48%) | 5 758 → 6 639 (+15%) | +| `ActionChain/chain1000` | 20.5 ms | ~ (7.08 MiB) | ~ (25.6 ms)¹ | 5.16 → 7.85 MiB (+52%) | 56.3k → 65.3k (+16%) | +| `StateLoop/count50` | 1.10 ms | ~ (605 KiB) | 1.66 ms (+51%) | 481 → 802 KiB (+67%) | +17% | +| `StateLoop/count500` | 10.6 ms | +28% | 14.0 ms (+33%) | 4.52 → 6.11 MiB (+35%) | +17% | +| `StateLoop/count5000` | 104 ms | ~ | ~¹ | 45.3 → 59.7 MiB (+32%) | 1.006M → 1.176M (+17%) | +| `LowerChain/chain10` | 12.0 µs | ~ (8.41 KiB, 78) | 23.5 µs (+96%) | 7.42 → 14.8 KiB (+99%) | 73 → 129 (+77%) | +| `LowerChain/chain100` | 209 µs | ~ (75.0 KiB, 471) | 324 µs (+55%) | 65.1 → 138 KiB (+112%) | 460 → 813 (+77%) | +| `LowerChain/chain1000` | 13.6 ms | ~ (1.06 MiB, 4 140) | ~ (14.7 ms)¹ | 922 → 1 857 KiB (+101%) | 4 118 → 7 227 (+76%) | + +¹ Spread above 35% on one side after six interleaved rounds; the time +column of these rows is not confirmed and the bytes and allocations are +the reading. The first pass (not interleaved, this machine) agrees in sign +on each. + +Eight rows exist only on 0.10.0 (the CSV/TSV/JSON value import; first +pass, this machine): `ImportValues/flat` 54.2 ms, 279 ms, 999 ms and +1.19 s for 100, 500 and 2 000 parts at three attributes and 2 000 at five +(131k, 615k, 2.43M and 3.67M allocations); `ImportValues/nested` 56.0 ms, +312 ms, 1.21 s and 1.88 s (175k, 829k, 3.28M and 4.57M). Linear in parts +times attributes, ≈400 allocations per imported value. Reading the table by cause: - **Lexing, parsing and indexing are unchanged** in time, with 4–8% more allocations in the index (the reflective relationships each declaration now registers, `00b8499a8`). -- **Analysis of the synthetic model** is +7.5% with +3.3% allocations, the - constraint-tier passes `AnalyseResolved` names. **Analysis of the vehicle - example is +85% with 1.8 times the bytes**, and the fixes did not touch - it, because the cost is not in the analysis but in its result: the - candidate reports 134 `Duplicate of imported member name` warnings on - the Annex A vehicle example where 0.9.2 reported none (the resolver's - `checkImportedNames`, `eb4b7583c`, which now sees the names a nested - `::**` import brings alongside the package's own). Every path that - returns the example's diagnostics pays to build and carry them: - `WorkspaceEdit/vehicle/reindex+diagnostics` +42%, - `GRPCParseFileUncached` +64% with 248 000 more allocations, - `ConnectParseFileHTTPCached` 271 → 879 µs with 851 more allocations - (the HTTP transport serializes the 134 warnings to JSON on every - request; the gRPC transport's cached protobuf conversion, `70242d872`, - is why `GRPCParseFileCached` went from 299 allocations back to 24 — two - more than 0.9.2, the cached slice's header and its once — and from - +22% to `~`). Whether the rule should fire on the vehicle example's - imports is a semantic question for the resolver's owners; **if it - should not, every row in this group is one fix away from 0.9.2**, and - if it should, the HTTP path wants the same cache the gRPC path got. - Either way, a model with no such warnings does not pay it: the - synthetic model's rows above are the same code without them. -- **Workspace edits.** The re-export fix (`4b4b35121`) is what moved these - rows against `ffe51571c`: `WorkspaceEdit/vehicle` from +86% bytes to - +18% and `WorkspaceEditSmallDocBesideLarge` — a one-line edit to a small - document in the same workspace as the 4 000-part model — from 3.9 times - 0.9.2's bytes to 3.0 times, and from +99% allocations to +40%. What +- **Analysis of the synthetic model** is `~` in time with +4.0% + allocations (+3.3% on `82f0fac87`; the tag adds seven-tenths of a + point with its semantic fixes to satisfy and verify relationships). **Analysis of the vehicle example is +79% with + 1.8 times the bytes** because the cost is not in the analysis but in + its result: 0.10.0 reports 134 `Duplicate of imported member name` + warnings on the Annex A vehicle example where 0.9.2 reported none (the + resolver's `checkImportedNames`, `eb4b7583c`, which now sees the names + a nested `::**` import brings alongside the package's own). Every path + that returns the example's diagnostics pays to build and carry them: + `WorkspaceEdit/vehicle/reindex+diagnostics` +70% bytes, + `GRPCParseFileUncached` +72% bytes with 249 000 more allocations, + `ConnectParseFileHTTPCached` 343 µs → 1.03 ms with 851 more + allocations (the HTTP transport serializes the 134 warnings to JSON on + every request; the gRPC transport's cached protobuf conversion, + `70242d872`, is why `GRPCParseFileCached` is 24 allocations against + 0.9.2's 22 and `~` in time). Whether the rule should fire on the + vehicle example's imports is a semantic question for the resolver's + owners; **if it should not, every row in this group is one fix away + from 0.9.2**, and if it should, the HTTP path wants the same cache the + gRPC path got. A model with no such warnings does not pay it: the + synthetic model's rows are the same code without them. +- **Workspace edits.** `WorkspaceEditSmallDocBesideLarge` — a one-line + edit to a small document in the same workspace as the 4 000-part model + — allocates 3.0 times 0.9.2's bytes and +40% objects for +56% time; + `WorkspaceEdit/vehicle` +18% bytes, +21% allocations, +28% time. The + re-export fix (`4b4b35121`) took the first from 3.9× to 3.0×; what remains is the re-registration itself: `symbols.registrationOf` still snapshots each name's re-export claims by reference (`4ca321508`), which is a slice per name of the large neighbor per edit, 15.7 MiB per - edit of the small document. **Confirmed, and the remaining 3.0× is - the one row in this record a 0.10.x pass should start with.** The - synthetic edit's +35% time stands on a ±68% candidate spread; its - bytes (+10%) and allocations (+14%) are the stable reading. + edit of the small document. **Confirmed, and the one row in this + record a 0.10.x pass should start with.** The synthetic edit's +58% + time stands on ±34–48% spreads; its bytes (+10%) and allocations + (+14%) are the stable reading. Nothing in this group moved between + `82f0fac87` and the tag. - **Execution is where the release's semantics cost**: `ActionLoop` - +38–56%, `StateLoop` +22–40%, `ExecuteState` +21%, - `ExecuteActionFreshContext` +38%, with 12–33% more allocations. The - lazy binding-check description (`37b4a17c9`) took `ActionLoop/for1000` - from 6 220 to 4 220 allocations per run (`ffe51571c` had doubled - 0.9.2's) and 3% of its time; the time that remains is the statement-order - exploration and the per-statement binding frames described under - `internal/exec/runtime`, which a loop body of one assignment pays on - every iteration. **Confirmed; structural.** `ExecuteAction` (a single - action, no loop) is `~` in time with 42 more allocations. -- **Constraints.** `BatchConstraints` was +68% on `ffe51571c` and is `~` - on the candidate, allocations +0.2%: the memoized succession-cycle check - (`8bd97fcf6`) **cleared it**. `SameConstraintManyInstances` +14% and - `BatchSatisfy` `~` in time with +3.4% allocations are the exact-number - evaluation of the constraint bodies. `GRPCVerifyConstraint` +21% on a - ±43% candidate spread, +6.3% allocations; `GRPCEvaluate` +11%, six - allocations more per call. -- **Instantiate**, the 4 000-part model's instantiation, is `~` in time - interleaved (the first pass's +39% was two outliers at ±79% and ±109%) - with +4.5% allocations and +3.5% bytes; the per-object cost is the REPL's + +42–56%, `StateLoop` +33–51%, `ExecuteState` +40%, + `ExecuteActionFreshContext` +46%, with 17–33% more allocations. The + loop rows are the statement-order exploration and the per-statement + binding frames described under `internal/exec/runtime`, which a loop + body of one assignment pays on every iteration — the lazy + binding-check description (`37b4a17c9`) already took `ActionLoop/for1000` + from 6 220 to 4 231 allocations — and are unchanged from `82f0fac87`. + **Confirmed; structural.** The state rows moved at the tag: `ExecuteState` + from 610 to 783 KiB and `StateLoop/count50` from 605 to 802 KiB, with + 5% more allocations, while `count5000` moved 4% — a per-executor cost, + not per step: the change-trigger bookkeeping journaled per transition + and the change-condition read sets kept across a snapshot restore + (`bc1baced5`, `3dfbdb7c6`, `91f9d384a`), which a machine entered once + pays once. **Confirmed; the price of restoring trigger state correctly + on rollback.** `ExecuteAction` (a single action, no loop) is `~` in time + with 51 more allocations and fewer bytes. +- **Lowering an action chain costs twice the bytes and 1.76× the + allocations at every length** (`LowerChain`, `LowerActionGraph` +46% / + +52%), from `82f0fac87`'s +15% / `~`: the inherited action steps, + successions, pins, gates and assertions a specialization's graph now + carries from its general action (`896d35582`, `fa8f50125`, `9d25656eb`, + `a5dbc9d3b`, `c8da68c1c`, `308d6a625`). The cost is linear — 10 → + 1 000 steps is 129 → 7 227 allocations, 56 per step at every length — + because the step's symbol is found through the scope's declaration + index (`baea6252b`); looked up by scanning the scope's members, as the + work first did, the same row was 53.9 MiB at 1 000 steps against 7.85 + here. `ActionChain` (lowering plus execution) is +52% bytes and +16% + allocations, `~` in time at 1 000 steps. **Confirmed; structural, and + the per-step constant of lowering is the second place a 0.10.x pass + should look.** +- **Constraints.** `BatchConstraints` is +22% allocations and +39% bytes + at the tag where `82f0fac87` was `~` (the memoized succession-cycle + check, `8bd97fcf6`, had cleared the release merge's +68%), and + `SameConstraintManyInstances` 1 016 → 1 440 B per check, 29 → 35 + allocations: a constraint's or requirement's `in` parameters are now + bound at check time (`645df3d19`) and a check that takes arguments no + longer shares its verdict (`d070d92a6`) — six allocations per check for + the argument frame. **Confirmed; the price of checks with arguments.** + `BatchSatisfy` and `GRPCVerifyConstraint` are `~` with +3.7% and + +6.5% allocations (the exact-number evaluation of the bodies); + `GRPCEvaluate` six allocations more per call. +- **Instantiate**, the 4 000-part model's instantiation, is +6.4% in + time with +4.0% allocations; the per-object cost is the REPL's `Instantiate` row. -- **Lowering** is +16% on a single action graph and +17% on a ten-step - chain (two and five more allocations: the guard and effect spans the - lowered graph now carries, `a0f77861d`), fading to `~` at 1 000 steps. -- **REPL**: loading the 4 000-part file is `~` in time interleaved (the - first pass's +34% was the candidate's ±37% spread) with +2.8% - allocations; submitting a snippet to the loaded model +14%. +- **REPL**: loading the 4 000-part file is +14% in time with +3.4% + allocations; submitting a snippet to the loaded model +17% with +4.4%. + `FQNOf` is +11% allocations, 4 000 more per 36 000 names. ## Stress constellation (`tests/stressmodel`) -The generated satellite-network constellation at 32, 128 and 512 -satellites (6 227, 24 167 and 95 927 elements), the sizes of the previous -records. Time per operation, 0.9.2, `ffe51571c` and the candidate, from the -interleaved re-run of every row but `ConvertAPIJSON` and -`AnalyzeSplitPerDocument` (first pass, see below); bytes and allocations -per operation where they moved. 51 rows on 0.9.2, 45 on the candidate. - -| benchmark | 0.9.2 | `ffe51571c` | 0.10.0 (`82f0fac87`) | -| --------- | ----- | ----------- | -------------------- | -| `Load/32` (6 227 el.) | 379 ms ±7% / 132.7 MiB / 1.994 M allocs | 456 ms (+20%) / 162.2 MiB / 2.365 M | 440 ms ±4% (+16%, p=0.002) / 159.7 MiB (+20%) / 2.316 M (+16%) | -| `Load/128` (24 167 el.) | 1.55 s ±20% / 501.3 MiB / 7.682 M | 1.85 s (+20%) | 1.71 s ±10% (+10%, p=0.041) / 568.2 MiB (+13%) / 8.836 M (+15%) | -| `Load/512` (95 927 el.) | 6.67 s ±23% / 1.935 GiB / 30.43 M | 7.81 s (+17%) | 7.50 s ±6% (~, p=0.132) / 2.166 GiB (+12%) / 34.92 M (+15%) | -| `Satisfy/32` (96 assertions) | 4.42 ms ±11% / 1.665 MiB / 28.90 k | 5.10 ms (+16%) / 1.831 MiB / 34.57 k | 4.88 ms ±3% (+11%, p=0.015) / 1.768 MiB (+6.2%) / 33.42 k (+16%) | -| `Satisfy/128` (384) | 18.9 ms ±14% / 114.9 k | 23.5 ms (+25%) | 21.1 ms ±15% (+12%, p=0.026) / 7.064 MiB (+6.2%) / 132.9 k (+16%) | -| `Satisfy/512` (1 536) | 103 ms ±11% / 458.8 k | 132 ms (+28%) | 109 ms ±15% (~, p=0.132) / 28.26 MiB (+6.2%) / 530.9 k (+16%) | -| `FleetInstantiate/32` (1 179 el.) | 4.33 ms ±5% / 7.665 MiB / 12.72 k | 5.13 ms (+19%) | 4.31 ms ±6% (~, p=1.000) / 7.592 MiB (~) / 13.93 k (+9.5%) | -| `FleetInstantiate/128` (1 715) | 12.1 ms ±12% / 24.10 MiB / 48.37 k | 16.6 ms (+37%) | 12.9 ms ±28% (~, p=0.093) / 24.44 MiB (~) / 52.94 k (+9.5%) | -| `FleetInstantiate/512` (3 947) | 63.2 ms ±7% / 151.4 MiB / 190.9 k | 84.6 ms (~, ±29%) | 62.6 ms ±9% (~, p=0.937) / 144.1 MiB (~) / 209.0 k (+9.5%) | -| `FleetSatisfy/32` (24 assertions) | 1.07 ms ±8% / 456.7 KiB / 7 811 | 1.32 ms (+23%) | 1.22 ms ±15% (+14%, p=0.002) / 480.1 KiB (+5.1%) / 8 852 (+13%) | -| `FleetSatisfy/128` (36) | 1.70 ms ±5% / 12.10 k | 2.07 ms (+22%) | 1.90 ms ±9% (+12%, p=0.002) / 743.5 KiB (+4.9%) / 13.65 k (+13%) | -| `FleetSatisfy/512` (108) | 5.35 ms ±17% / 37.06 k | 6.63 ms (+24%) | 5.90 ms ±36% (+10%, p=0.041) / 2.221 MiB (+4.6%) / 41.55 k (+12%) | -| `EditBeside/32` | 1.10 ms ±5% / 295.6 KiB / 3 442 | 1.48 ms (+34%) / 719.5 KiB (+143%) / 5 533 (+61%) | 1.32 ms ±4% (+20%, p=0.002) / 558.6 KiB (+89%) / 4 537 (+32%) | -| `EditBeside/128` | 3.41 ms ±3% / 719.9 KiB / 8 377 | 4.60 ms (+35%) / 2.379 MiB (+238%) / 15.92 k (+90%) | 4.06 ms ±12% (+19%, p=0.002) / 1.756 MiB (+150%) / 11.97 k (+43%) | -| `EditBeside/512` | 13.4 ms ±7% / 2.409 MiB / 28.11 k | 19.1 ms (+43%) / 9.146 MiB (+280%) / 57.44 k (+104%) | 16.1 ms ±4% (+20%, p=0.002) / 6.662 MiB (+177%) / 41.68 k (+48%) | -| `ValidateSplit/32/jobs=1` | 541 ms ±9% / 186.1 MiB / 2.480 M | 612 ms (+13%) | 614 ms ±24% (+14%, p=0.002) / 217.1 MiB (+17%) / 2.881 M (+16%) | -| `ValidateSplit/32/jobs=8` | 254 ms ±5% / 2.499 M | 275 ms (+8.2%) | 267 ms ±5% (+4.9%, p=0.041) / 219.0 MiB (+16%) / 2.896 M (+16%) | -| `ValidateSplit/128/jobs=1` | 1.96 s ±17% / 8.723 M | 2.17 s (+11%) | 2.15 s ±6% (~, p=0.093) / 700.6 MiB (+14%) / 10.07 M (+15%) | -| `ValidateSplit/128/jobs=8` | 802 ms ±19% / 8.740 M | 901 ms (~) | 853 ms ±12% (~, p=0.589) / 703.0 MiB (+14%) / 10.08 M (+15%) | -| `ValidateSplit/512/jobs=1` | 7.86 s ±17% / 33.69 M | 8.94 s (+14%) | 8.70 s ±8% (+11%, p=0.026) / 2.623 GiB (+12%) / 38.80 M (+15%) | -| `ValidateSplit/512/jobs=8` | 3.22 s ±20% / 33.69 M | 3.69 s (~) | 3.64 s ±4% (~, p=0.065) / 2.626 GiB (+12%) / 38.82 M (+15%) | -| `LoadFiles/32` | 473 ms ±15% / 147.7 MiB / 1.971 M | 555 ms (+17%) | 547 ms ±43% (+16%, p=0.009) / 175.7 MiB (+19%) / 2.325 M (+18%) | -| `LoadFiles/128` | 1.88 s ±12% / 6.998 M | 2.03 s (~) | 2.02 s ±6% (~, p=0.240) / 565.6 MiB (+16%) / 8.244 M (+18%) | -| `LoadFiles/512` | 7.62 s ±7% / 27.10 M | 8.51 s (+12%) | 8.39 s ±22% (+10%, p=0.002) / 2.123 GiB (+14%) / 31.91 M (+18%) | -| `EditImported/32` | 547 ms ±25% / 116.3 MiB / 1.979 M | 574 ms (~) / 150.5 MiB (+29%) | 572 ms ±112% (~, p=0.485) / 135.4 MiB (+16%) / 2.360 M (+19%) | -| `EditImported/128` | 1.82 s ±10% / 6.796 M | 2.01 s (+11%) / 468.1 MiB (+23%) | 1.96 s ±10% (+7.5%, p=0.026) / 447.7 MiB (+17%) / 8.090 M (+19%) | -| `EditImported/512` | 7.17 s ±12% / 26.06 M | 8.30 s (+16%) / 1.759 GiB (+19%) | 8.12 s ±6% (+13%, p=0.004) / 1.720 GiB (+16%) / 31.00 M (+19%) | -| `OpenSplit/32/cold` | 251 ms ±18% / 169.0 MiB / 2.360 M | 263 ms (~) | 256 ms ±12% (~, p=0.818) / 199.0 MiB (+18%) / 2.757 M (+17%) | -| `OpenSplit/32/warm` | 106 ms ±16% / 65.62 MiB / 571.1 k | 135 ms (+27%) / 86.05 MiB (+31%) | 132 ms ±5% (+24%, p=0.002) / 79.75 MiB (+22%) / 694.8 k (+22%) | -| `OpenSplit/128/cold` | 770 ms ±13% / 8.208 M | 834 ms (~) | 857 ms ±8% (+11%, p=0.041) / 627.4 MiB (+16%) / 9.554 M (+16%) | -| `OpenSplit/128/warm` | 203 ms ±13% / 166.7 MiB / 1.385 M | 284 ms (+40%) / 215.3 MiB (+29%) | 268 ms ±12% (+32%, p=0.002) / 208.9 MiB (+25%) / 1.717 M (+24%) | -| `OpenSplit/512/cold` | 2.93 s ±8% / 31.59 M | 3.61 s (+23%) | 3.52 s ±18% (+20%, p=0.026) / 2.332 GiB (+14%) / 36.72 M (+16%) | -| `OpenSplit/512/warm` | 661 ms ±6% / 613.2 MiB / 4.639 M | 941 ms (+42%) / 897.5 MiB (+46%) | 939 ms ±20% (+42%, p=0.002) / 890.2 MiB (+45%) / 5.804 M (+25%) | -| `HydratePlane/32` | 95.2 ms ±8% / 25.83 MiB / 314.8 k | 83.6 ms (−12%) | 78.6 ms ±32% (~, p=0.065) / 20.92 MiB (−19%) / 255.0 k (−19%) | -| `HydratePlane/128` | 263 ms ±8% / 76.62 MiB / 919.8 k | 208 ms (−21%) | 211 ms ±52% (~, p=0.065) / 57.47 MiB (−25%) / 652.9 k (−29%) | -| `HydratePlane/512` | 1.03 s ±3% / 287.1 MiB / 3.336 M | 828 ms (−20%) | 846 ms ±4% (−18%, p=0.002) / 207.0 MiB (−28%) / 2.243 M (−33%) | -| `ConvertAPIJSON/32` (first pass) | 9.74 s ±14% / 4.645 GiB / 25.53 M | — | 7.21 s ±8% (−26%, p=0.002) / 4.677 GiB (+0.7%) / 25.13 M (−1.5%) | -| `ConvertAPIJSON/128` (first pass) | 35.7 s ±38% / 18.22 GiB / 100.8 M | — | 29.7 s ±3% (~, p=0.065) / 18.34 GiB (+0.7%) / 99.31 M (−1.5%) | -| `ConvertAPIJSON/512` | one count, 156 s / 72.84 GiB | — | one count, 136 s / 73.31 GiB — no distribution | -| `AnalyzeSplitPerDocument/32/jobs=1` | 425 ms ±7% / 133.1 MiB / 1.636 M | fails | fails (see below) | -| `AnalyzeSplitPerDocument/32/jobs=8` | 108 ms ±5% | fails | fails | -| `AnalyzeSplitPerDocument/128/jobs=1` | 1.69 s ±5% / 489.0 MiB / 6.207 M | fails | fails | -| `AnalyzeSplitPerDocument/128/jobs=8` | 415 ms ±8% | fails | fails | -| `AnalyzeSplitPerDocument/512/jobs=1` | 7.18 s ±25% / 2.025 GiB / 24.49 M | fails | fails | -| `AnalyzeSplitPerDocument/512/jobs=8` | 1.77 s ±11% | fails | fails | - -Allocation counts — deterministic to the unit — split the package into -four stories: +The satellite-network generator at 32, 128 and 512 satellites (6 227, +24 167 and 95 927 elements) — connector-, binding- and flow-dense, with a +state machine and a `DownlinkPass` action per satellite — loaded, split +across six files, edited beside, validated, instantiated, hydrated and +converted. Time per operation, 0.9.2 and 0.10.0, from the interleaved +re-run of the whole package (six rounds; `82f0fac87` stands between them +where a row moved at the tag); bytes and allocations from the same run. + +| benchmark | 0.9.2 | 0.10.0 | Δ time | Δ B/op | Δ allocs | +| --------- | ----- | ------ | ------ | ------ | -------- | +| `Load/32` | 679 ms | 775 ms | ~¹ | 133 → 160 MiB (+20%) | +16% | +| `Load/128` | 2.85 s | 2.55 s | ~¹ | 501 → 569 MiB (+13%) | +15% | +| `Load/512` | 9.47 s | 10.1 s | ~ | 1.94 → 2.17 GiB (+12%) | +15% | +| `Satisfy/32…512` | 6.55 ms – 130 ms | 7.52 ms – 152 ms | ~, ~, +17% | +6.2% | +15.7% | +| `FleetInstantiate/32` | 5.67 ms | 5.73 ms | ~ (+21% in the first re-run) | 5.27 → 6.88 MiB (+31%) | 12 720 → 14 350 (+13%) | +| `FleetInstantiate/128` | 21.3 ms | 23.4 ms | +9.8% | 24.1 → 31.4 MiB (+30%) | 48 370 → 54 600 (+13%) | +| `FleetInstantiate/512` | 84.1 ms | 83.5 ms | ~ | +15%¹ | 190.9k → 215.6k (+13%) | +| `FleetSatisfy/32…512` | 1.81 – 7.96 ms | 1.86 – 9.46 ms | ~, ~, +19%¹ | +4.7–5.1% | +12–13% | +| `EditBeside/32` | 1.84 ms | 1.76 ms | ~ | 296 → 559 KiB (+89%) | 3 442 → 4 536 (+32%) | +| `EditBeside/128` | 4.26 ms | 5.31 ms | +25%¹ | 720 KiB → 1.76 MiB (+150%) | 8 377 → 11 973 (+43%) | +| `EditBeside/512` | 18.6 ms | 23.6 ms | +26%¹ | 2.41 → 6.66 MiB (+177%) | 28 110 → 41 680 (+48%) | +| `ValidateSplit/32…512/jobs=1` | 0.81 s – 11.0 s | 0.89 s – 12.5 s | ~ | +12–17% | +15–16% | +| `ValidateSplit/32…512/jobs=8` | 0.38 s – 4.47 s | 0.37 s – 4.28 s | ~ | +12–16% | +15–16% | +| `LoadFiles/32…512` | 0.80 s – 10.5 s | 0.81 s – 11.7 s | ~, +12%, ~ | +14–19% | +18% | +| `EditImported/32…512` | 1.01 s – 10.5 s | 0.87 s – 11.4 s | ~, ~, +9.0% | +16–17% | +19% | +| `OpenSplit/32…512/cold` | 0.39 s – 4.39 s | 0.39 s – 4.38 s | ~ | +14–18% | +16–17% | +| `OpenSplit/32/warm` | 162 ms | 228 ms | +41%¹ | 65.6 → 79.7 MiB (+21%) | +22% | +| `OpenSplit/128/warm` | 357 ms | 440 ms | +23% | 167 → 209 MiB (+25%) | +24% | +| `OpenSplit/512/warm` | 1.04 s | 1.40 s | +34% | 613 → 891 MiB (+45%) | +25% | +| `HydratePlane/32` | 149 ms | 134 ms | ~ | 25.8 → 20.9 MiB (**−19%**) | **−19%** | +| `HydratePlane/128` | 356 ms | 328 ms | **−7.9%** | 76.6 → 57.5 MiB (**−25%**) | **−29%** | +| `HydratePlane/512` | 1.36 s | 1.16 s | **−15%** | 287 → 206 MiB (**−28%**) | **−33%** | +| `ConvertAPIJSON/32` | 9.00 s | 9.02 s | ~ | +0.7% | −1.5% | +| `ConvertAPIJSON/128` | 38.8 s | 38.6 s | ~ | +0.7% | −1.5% | +| `ConvertAPIJSON/512` | 169 s² | 171 s | ~ | 72.8 → 73.3 GiB | 402M → 396M | + +¹ Spread above 20% on one side after six rounds; time unconfirmed, bytes +and allocations are the reading. ² One count on 0.9.2 (see below). + +Twelve rows have a 0.9.2 figure and no 0.10.0 figure (`AnalyzeSplitPerDocument`, +six rows) or no figure on either revision (`PlaneResidency`, four rows at +32, 128, 512 and 1 600 satellites), discussed below. - **The loader costs 15–18% more allocations and 12–20% more bytes per element** at every scale and through every entrance (`Load`, `LoadFiles`, `ValidateSplit`, `OpenSplit/cold`, `EditImported`), for - 10–16% more time where the spread lets the difference show and `~` - where it does not (the 512-satellite `Load` and the parallel - `ValidateSplit`). Each connector, binding and flow end the constellation - is made of is now owned, journaled, indexed and reflected - (`00b8499a8`, `da8fba4a3`, `32890be28`, `63552cbc4`), and the extended - declarations classified (`ca0e6cca0`). The qualified-name fix - (`367ca7e3a`) took 2–3 points off `ffe51571c`'s allocations and 3–7 off - its time. **Confirmed; structural, linear, and the cost of 0.10.0's - relationship model.** The whole-binary table below shows the same from - outside the process. -- **The satisfaction checks are +16% allocations and +10–14% time on the - fleet** where `ffe51571c` was +20% and +22–28%: the memoized - succession-cycle check (`8bd97fcf6`) took the walk out of every check - and left the exact-number evaluation of the assertions (`923ec81f0`, - `79629923b`), which is 4 500 allocations per 96 assertions. **Confirmed; - explained.** -- **Editing beside the constellation allocates 2.9 times the bytes of - 0.9.2** (was 3.8 times) and +32–48% objects (was +61–104%), for +20% - time (was +34–43%); the warm `OpenSplit` and `WorkspaceUpdate` in the - LSP table are the same. The re-export fix (`4b4b35121`) removed the - deep copy of each name's claims; the per-name snapshot of claims by - reference across the 512-satellite neighbor (`4ca321508`) is what - remains, 4.3 MiB per edit at 512. **Confirmed; the open fix.** - `OpenSplit/512/warm` (+42%, +45% bytes) adds the warm cache's - re-validation of the owned-relationship journal it now carries. -- **Instantiation of the fleet is `~` in time with 9.5% more allocations - and the same or fewer bytes** — the three-word scalar packing - (`4a1b937f2`) and the allocation-free Integer and Rational fast paths - (`d1eb16edb`) took `ffe51571c`'s +19–37% back to 0.9.2, and - `HydratePlane` is **18–33% fewer allocations and 18–28% fewer bytes** - than 0.9.2, 18% faster at 512: the hydration of a plane's instances no - longer rebuilds the implicit ends it can derive (`fa8a7220c`). The - API-JSON conversion is 26% faster at 32 satellites with the same bytes, - from the same source; its 512-satellite row still does not fit this - machine on either revision. - -`AnalyzeSplitPerDocument` fails on the candidate at every size. The -benchmark analyses each of the constellation's six files in its own -per-document registry and asserts a clean result; the candidate's -per-document analysis now reports each decimal literal the generator -writes into a `Real`-typed attribute (`0.2 is rounded to the nearest Real, + 9–12% more time where the spread lets the difference show and `~` + where it does not (on this machine, most of the rows: the loader's + single counts spread 15–55% on 0.9.2). Each connector, binding and flow + end the constellation is made of is now owned, journaled, indexed and + reflected (`00b8499a8`, `da8fba4a3`, `32890be28`, `63552cbc4`), and the + extended declarations classified (`ca0e6cca0`). **Confirmed by the + allocations at every size; structural, linear, and the cost of + 0.10.0's relationship model.** The whole-binary table below shows the + same from outside the process, where the spread is small enough to + show the time. Unchanged from `82f0fac87` to the allocation. +- **The satisfaction checks are +16% allocations and +6% bytes on the + fleet**, `~` to +17% time: the exact-number evaluation of the + assertions (`923ec81f0`, `79629923b`), 4 500 allocations per 96 + assertions, with the succession-cycle walk memoized (`8bd97fcf6`). + **Confirmed; explained.** +- **Editing beside the constellation allocates 1.9–2.8 times the bytes + of 0.9.2** and +32–48% objects, for `~` to +26% time; the warm + `OpenSplit` (+21–45% bytes) and `WorkspaceUpdate` in the LSP table are + the same cost. The per-name snapshot of claims by reference across the + 512-satellite neighbor (`4ca321508`) is what remains after the + re-export fix (`4b4b35121`), 4.3 MiB per edit at 512. **Confirmed; the + open fix.** `OpenSplit/512/warm` adds the warm cache's re-validation of + the owned-relationship journal it now carries. +- **Instantiation of the fleet is +13% allocations and +30% bytes** at + the tag, where `82f0fac87` was +9.5% and `~`: each satellite's state + machine and action are lowered with their inherited steps and + change-trigger bookkeeping (the `LowerChain` and `ExecuteState` costs + above, per satellite); +9.8% time at 128, `~` at 32 and 512 (one + earlier interleaved run had +21% at 32). **Confirmed; the same cause as + the lowering rows.** `HydratePlane` is **19–33% fewer allocations and + 19–28% fewer bytes** than 0.9.2, 8–15% faster: the hydration of a + plane's instances no longer rebuilds the implicit ends it can derive + (`fa8a7220c`). The API-JSON conversion is unchanged in time with 1.5% + fewer allocations. + +`AnalyzeSplitPerDocument` fails on 0.10.0 at every size, as it did on +`82f0fac87`. The benchmark analyses each of the constellation's six files +in its own per-document registry and asserts a clean result; 0.10.0's +per-document analysis reports each decimal literal the generator writes +into a `Real`-typed attribute (`0.2 is rounded to the nearest Real, 0.20000000000000001, since a feature typed by Real holds a binary64 value; type the feature by Rational to keep the literal exact`, `014fceb7b`), ten per file, and the assertion trips. The six rows are -baseline-only figures in this record; the benchmark's generator or its -assertion needs the release's view of decimal literals (a `Rational` -attribute type, or the lint filtered) before the row can be compared -again. The whole-workspace analysis the other rows and `sysml -validate` -do is not affected — the report is a lint the per-document registry turns -on — which is why `ValidateSplit` and the whole-binary runs are clean on -both. +baseline-only figures in this record (0.9.2: 656 ms / 159 ms, 2.27 s / +597 ms and 11.3 s / 2.50 s at one and eight jobs); the benchmark's +generator or its assertion needs the release's view of decimal literals +before the row can be compared again. The whole-workspace analysis the +other rows and `sysml -validate` do is not affected — the report is a +lint the per-document registry turns on — which is why `ValidateSplit` +and the whole-binary runs are clean on both. + +`PlaneResidency` — the residency of a plane's interface record among the +constellation's documents — fails on **both** revisions at every size with +`workspace: the interface record was written among other documents: +plane000.sysml` (`ErrRecordStale`): the benchmark writes the record after +the plane's documents and the workspace refuses to open it as stale. It +never ran in 0.9.2's full pass, where the package was killed first (next +paragraph), and a focused run of the row on 0.9.2 fails the same way; it +is a defect of the benchmark's setup, not a difference between the +revisions. No figure on either side. + +`ConvertAPIJSON/satellites=512` allocates 73 GiB per operation. On 0.9.2 +the package's first pass completed two counts (169 s) before the process +was killed for memory on this 31 GiB machine (`signal: killed`, as in the +previous record); 0.10.0's first pass completed all six (169–235 s) and +the interleaved re-run produced the row on both revisions (`~`), so it is +compared here, from the re-run. The row is a memory test of the machine +as much as of the converter and is read with that in mind. ## Whole-binary scaling @@ -680,110 +625,101 @@ generator at ≈3 000, 6 000 and 12 000 declarations: 3 804, 7 260 and binaries interleaved; median wall seconds and median maximum resident set, every run exiting 0. -| declarations | 0.9.2 | `ffe51571c` | 0.10.0 (`82f0fac87`) | -| ------------ | ----- | ----------- | -------------------- | -| ≈3 000 | 0.09 s / 95 MiB | 0.11 s / 100 MiB | 0.11 s / 101 MiB | -| ≈6 000 | 0.14 s / 107 MiB | 0.16 s / 116 MiB | 0.17 s / 115 MiB | -| ≈12 000 | 0.23 s / 134 MiB | 0.27 s / 157 MiB | 0.28 s / 157 MiB | +| declarations | 0.9.2 | `82f0fac87` | 0.10.0 | +| ------------ | ----- | ----------- | ------ | +| ≈3 000 | 0.13 s / 95 MiB | 0.16 s / 100 MiB | 0.15 s / 100 MiB | +| ≈6 000 | 0.20 s / 108 MiB | 0.24 s / 114 MiB | 0.23 s / 115 MiB | +| ≈12 000 | 0.36 s / 133 MiB | 0.42 s / 157 MiB | 0.41 s / 156 MiB | Linear in the declarations on every binary, as before: doubling the model -doubles the time (0.11 → 0.17 → 0.28 s) and adds 14–42 MiB. The candidate -is 20–22% slower than 0.9.2 and holds 6–23 MiB more, growing with the -model — the loader's per-element cost of the stress constellation -(+14–20% bytes, +15–18% allocations) seen from outside the process. The -fixes did not move these figures against `ffe51571c`, and were not -expected to: none of the six is on the load path. The spreads were tight -on 0.9.2 and `ffe51571c` (±1 sample in the second decimal) and the -candidate's first round of each model was its slowest (0.51, 0.75 and -1.48 s, a cold page cache for the freshly built binary); the medians are -unaffected and the remaining eight rounds agree with them. +doubles the time (0.15 → 0.23 → 0.41 s) and adds 15–41 MiB. 0.10.0 is +14–15% slower than 0.9.2 and holds 5–23 MiB more, growing with the model +— the loader's per-element cost of the stress constellation (+12–20% +bytes, +15–18% allocations) seen from outside the process. The tag and +`82f0fac87` agree within a sample: none of the state-machine work is on +the load path. The spreads were within ±0.01 s on 0.9.2 and `82f0fac87`; +0.10.0's first round of each model was its slowest (0.77, 1.54 and 2.46 +s, a cold page cache for the freshly built binary) and the remaining +eight rounds agree with the medians. ### Process start and session floor -| measure | 0.9.2 | `ffe51571c` | 0.10.0 (`82f0fac87`) | -| ------- | ----- | ----------- | -------------------- | -| `sysml -e 1` wall time | 0.02 s | 0.02 s | 0.02 s | -| `sysml -e 1` max RSS (session floor) | 64 MiB | 68 MiB | 69 MiB | -| `bin/sysml` size (`-s -w`) | 42.88 MiB | 46.99 MiB | 47.23 MiB | +| measure | 0.9.2 | `82f0fac87` | 0.10.0 | +| ------- | ----- | ----------- | ------ | +| `sysml -e 1` wall time | 0.03 s | 0.02 s | 0.03 s | +| `sysml -e 1` max RSS (session floor) | 65 MiB | 69 MiB | 69 MiB | +| `bin/sysml` size (`-s -w`) | 42.88 MiB | 47.23 MiB | 47.55 MiB | The floor is the standard library's bundled index decoded at start, and -rose 5 MiB with the snapshot's new contents (the owned-relationship +rose 4 MiB with the snapshot's new contents (the owned-relationship journal and the reflected implicit relationships, `DecodeSnapshot` -+7.2% bytes). The binary is 4.35 MiB larger than 0.9.2's: the new ++7.2% bytes). The binary is 4.67 MiB larger than 0.9.2's: the new frontends (views, code lenses, inlay hints, the REPL's repository and -notebook commands, `-list`), the exact-number evaluator, the value import -and the SysML v2 API client; 0.24 MiB of it arrived after the release -branch was cut. Start time is unchanged. +notebook commands, `-list`), the exact-number evaluator, the value import, +the SysML v2 API client and, in the last 0.32 MiB, the inherited-step +lowering and the audit and migration fixes of the release's final +hotfixes. Start time is unchanged within the timer's resolution. ### Example models and the Apollo 11 model -| model | 0.9.2 | `ffe51571c` | 0.10.0 (`82f0fac87`) | -| ----- | ----- | ----------- | -------------------- | -| `examples/action-executor-demo.sysml` | 0.02 s / 66 MiB | 0.02 s / 69 MiB | 0.02 s / 71 MiB | -| `examples/disposal-robot-demo/robot.sysml` | 0.04 s / 72 MiB | 0.04 s / 78 MiB | 0.04 s / 77 MiB | -| Apollo 11 (28 files, `scripts/download-apollo11.sh`) | 0.28 s / 149 MiB | 0.33 s / 169 MiB | 0.32 s / 161 MiB | +| model | 0.9.2 | `82f0fac87` | 0.10.0 | +| ----- | ----- | ----------- | ------ | +| `examples/action-executor-demo.sysml` | 0.03 s / 66 MiB | 0.03 s / 70 MiB | 0.03 s / 71 MiB | +| `examples/disposal-robot-demo/robot.sysml` | 0.06 s / 72 MiB | 0.06 s / 77 MiB | 0.06 s / 77 MiB | +| Apollo 11 (28 files, `scripts/download-apollo11.sh`) | 0.55 s / 146 MiB | 0.61 s / 159 MiB | 0.62 s / 160 MiB | The two small examples sit on the session floor and move with it. The Apollo 11 model — the one real model in the set, connector- and -flow-dense like the stress constellation — validates in 0.32 s against -0.28 s (+14%) holding 12 MiB more; 8 MiB less than `ffe51571c` held, the -smaller claim snapshots of the re-export fix on a multi-document model. -Every file validates with the same report on all three binaries. +flow-dense like the stress constellation — validates in 0.62 s against +0.55 s (+13%) holding 13 MiB more. Every file validates with the same +report on all three binaries. ## Re-runs -The first pass left 83 of the 196 compared rows in doubt: 61 by a -significant difference, 22 by a spread above 20% on one side with no -significant difference (`Fib(25)/interpreted` ±65%, `ReferencesCold` -±19/17%, `Instantiate` ±79/109%, most of the stress constellation's time -rows). All were re-run interleaved — `-count 1` of the row on 0.9.2, then -on `ffe51571c`, then on the candidate, six times — and the -`internal/frontend/repl`, `tests/perf` and `tests/stressmodel` packages -whole or in their moving rows, since most of each was in doubt. The pilot -corpora were downloaded between 0.9.2's first pass and the re-run, so the -`vehicle` rows and `InstantiateWarmModel` have the re-run only. - -What the re-run changed: - -- **Nothing changed sign.** Every row significant in the first pass is - significant in the re-run in the same direction or `~`; the stress - constellation's time regressions shrank from the first pass's +14–27% - to +10–20% once the three revisions ran under the same conditions, and - that is the figure reported. -- **Cleared:** `REPLLoadFile` (+34% as found; `~` at p=0.065), - `Instantiate` in `tests/perf` (+39% as found; `~`), `BatchSatisfy` - (+17% as found; `~`), `IndexLibrary` (+19% as found; `~`), - `ExpandModelImports` (+9.0% as found; `~`), - `AnalyseResolved` (+12% as found; `~`, the +18% allocations stand), - `DecodeSnapshot` (`~`, the +19% allocations stand), `IndexAddExpand` - and `IndexAddOnly` (`~`), `FleetInstantiate` at every size (`~`), - `Load/512` and `Satisfy/512` (`~` at ±6% and ±15%), `LoadFiles/128`, - `OpenSplit/32/cold`, `EditImported/32`, `ValidateSplit/128` (`~`), - `ExecuteAction` (`~`), `ActionChain/chain10, chain1000` (`~`), - `LowerStateGraph` (`~`), `RunStateMachine/1000` (`~`), - `Diagnostics/attributes=800` (`~`), `GRPCParseFileCached` (`~`). -- **Confirmed** (significant both times, within a few points of each - other): `AssignmentLoopStep` (+90% → +61%), `MaterializePlainAttributes` - (`~` → +14%), `DerivedReadWriteRead` (+39% → +11%), `FormatEdits` - (+71%), `WorkspaceUpdate` (+42% → +16%), the REPL `LoadModel` rows - (+17–41% → +5–10%), `RunCalc` (+13–19% → +10–14%), `Instantiate/250, - 1000, 4000` (+13–27% → +11–31%), `Collatz(27)` (+71%/+53% → - +65%/+59%), `Fib(25)/go` (+94% → +81%), `Hypot(3.0,4.0)/go` (+394% → - +412%), `InstantiateWarmModel` (+32%), `ExpandWildcardImports` (+15% → +4.2%), - and the `tests/perf` and stress - rows the tables above mark significant. -- **Moved the other way:** `Fib(25)/interpreted` was `~` in the first pass - (6.61 ms ±65% against 7.54 ms) and is −20% at ±9% interleaved. -- **Still in doubt:** `WorkspaceEdit/synthetic` (+35% at ±68% on the - candidate's counts; the +10% bytes and +14% allocations are the - reading), `GRPCVerifyConstraint` (+21% at ±43%), - `EditImported/32` (±112%), `HydratePlane/32, 128` (−17%, −20% at - ±32% and ±52%, `~` at p=0.065; the allocation gains stand), - `LoadFiles/32` (+16% at ±43%); `IntegerArithmeticLoop` ±59% and - `RationalDecimalLoop` ±34%, candidate-only. - -Raw `benchstat` output for every package, first pass and re-run, is in the -tables above; the per-row p-values are p=0.002 (n=6) unless shown. +The first pass on this machine left 114 of the 130 compared time rows +in doubt: 74 by a significant difference, 40 by a spread above 20% on one +side with no significant difference (the stress constellation's time rows +almost entirely, `Lex/synthetic` ±107%, `LookupQualified` ±111%, +`Fib(25)/interpreted` ±111% on 0.10.0). All were re-run interleaved — +`-count 1` of the row on 0.9.2, then on `82f0fac87`, then on 0.10.0, six +times, nothing else on the machine — and where most of a package was in +doubt (`internal/frontend/repl`, `tests/perf`, `tests/stressmodel`) the +whole package was re-run that way and the re-run is the table shown. The +re-run: + +- **confirmed** every regression named in the summary, with the bytes + and allocations identical to the first pass to the unit on every row, + and the time deltas smaller than the first pass's on most + (`WorkspaceUpdate` +127% → +24%, `FormatEdits` +117% → +78%, + `AnalyseResolved` +53% → +9.8%, `InstantiateWarmModel` +74% → +46%, + `LoadModel/4000` +35% → +9.0%, `IndexLibrary` +35% → +6.6%, + `BatchConstraints` +123% → +26%, `ExecuteState` +73% → +40%, + `LowerActionGraph` +118% → +69%); +- **dismissed** the first pass's time differences on `Instantiate` + (first pass +12%, re-run +6.4%), `LowerStateGraph` (+29% → `~`), + `REPLEvalExpr` (+35% → `~`), `GRPCVerifyConstraint` (+44% → `~`), + `ActionChain/chain1000` and `LowerChain/chain1000` (`~` both ways), + `IndexAddExpand` and `IndexAddOnly` (+31–39% → `~`), + `RunStateMachine` (+13–30% → +7–8%), `Collatz(27)/interpreted` + (+94% → +61%), `Hypot/interpreted` (+30% → +13%), `ReferencesWarm` + (+15% → `~`) and, in the stress constellation, `Satisfy/512` + (+17% in both, the one loader-side time row that held), + `EditBeside/128` and `/512` (+25–26% on ±25–41% spreads, unconfirmed); +- **left unconfirmed** in time, with spreads above 35% on this machine + after six rounds, `WorkspaceEdit/synthetic` and + `/vehicle/reindex+diagnostics`, `GRPCParseFileUncached`, + `ActionLoop/for1000`, `StateLoop/count5000`, the 1 000-step chain rows + and the loader's stress rows — on each of them the bytes and + allocations are the figure to read, and they are deterministic. + +The machine this record was measured on is not the machine of the +previous records: it is 1.3–1.8× slower on the same code (`FormatEdits` +on 0.9.2: 512 µs there, 639 µs here; `LoadModel/4000` 458 ms against +710 ms), with the same allocations to the unit. Every time in this record +is from runs of both revisions on this machine, interleaved where it +mattered; no time is compared across machines. Readers comparing against +the 0.9.2 record's absolute figures should scale accordingly or, better, +read the allocation columns. ## What this does not measure @@ -794,13 +730,18 @@ tables above; the per-row p-values are p=0.002 (n=6) unless shown. exact-numeric loops); the rest are covered by their test suites, not by this record. - The compiled-calc C target is measured on `Hypot` only; the three - Integer calcs it refuses have no candidate figure, so the C target's - speed on 0.10.0 is read from one row. + Integer calcs it refuses have no 0.10.0 figure, so the C target's speed + on 0.10.0 is read from one row. - The stress constellation's per-document analysis - (`AnalyzeSplitPerDocument`) has no candidate figure at any size. -- Everything ran on one 8-CPU virtual machine. The baseline's re-run - spreads are mostly under 20%, but the stress constellation's - `EditImported/32` and `HydratePlane` rows and the candidate's - `WorkspaceEdit/synthetic` still show 30–110% spreads on single counts, - and a one-CPU host should read the package figures rather than the - whole-binary ones, as the previous records said. + (`AnalyzeSplitPerDocument`) has no 0.10.0 figure at any size, and its + `PlaneResidency` row has no figure on either revision. +- Inherited action steps are measured only where the benchmarks' + fixtures specialize an action (`LowerChain`, `FleetInstantiate`); a + model whose actions redefine deep hierarchies of inherited steps has no + row here. +- Everything ran on one 8-CPU virtual machine, slower than the previous + records', with single-count spreads of 20–100% on many rows that the + earlier machine held under 20%; the allocation columns are the + machine-independent reading, and a one-CPU host should read the + package figures rather than the whole-binary ones, as the previous + records said.