perf(validate): read the cycle already run instead of running another one - #98
Conversation
… one The shortfall accounting launched a second full scan just to read three counters. On the container's real WordPress a full scan costs about eight minutes, and three checks added in one day each doing that took `make validate-engines` from roughly a quarter of an hour to nearly an hour. That matters more than it looks. A validation nobody has an hour for is one that stops being run, and this script's whole value is that it gets run before trusting a scan. Slowness is how it fails in practice — not by being wrong. The counters were already on screen. Since #92 the text report prints each engine's gaps and notes on their own lines, and $scan_output was captured forty lines earlier in the same section. Same cycle, same numbers, no second walk. unknown_rule is deliberately not counted as an excuse for a shortfall: the finding it describes IS in the report, so it explains nothing about one that is missing. Verified against a realistic sample rather than assumed — outside_requested_scope=853 plus vanished_before_hashing=2 sums to 855, and the unknown_rule=260 on the same line stays out of it. Six full scans, now five. The remaining pair in the evasion section is inherent: comparing the terminal against the JSON needs one invocation of each.
|
Measured after merging, and the commit message understates the problemThe change does what it claims. The shortfall check still works and now costs nothing: One second, where it previously launched a full scan. But the whole run took 100m42s, and Where the time actually goes, from the timestamps of one run:
So a full scan in that container now costs 16–24 minutes. Earlier today one reported What is solid: four full scans remain, each of them tens of minutes, and removing one of the six was worth doing but does not fix the shape of the problem. The next lever, not taken hereThe quarantine section and the resource-limit measurement each run their own full scan. The memory-peak measurement could plausibly wrap the quarantine one instead of adding a third, though they run under different |
1 participant
The shortfall accounting launched a second full scan just to read three counters. On the container's real WordPress a full scan costs about eight minutes, and three checks added in one day — #87's, #91's and #93's — each did that.
make validate-engineswent from roughly a quarter of an hour to nearly an hour.That matters more than it looks. A validation nobody has an hour for is one that stops being run, and this script's entire value is that it gets run before trusting a scan. Slowness is how it fails in practice, not wrongness.
The counters were already on screen
Since #92 the text report prints each engine's gaps and notes on their own lines:
and
$scan_outputwas captured forty lines earlier in the same section. Same cycle, same numbers, no second walk.unknown_ruleis deliberately not counted as an excuse for a shortfall: the finding it describes is in the report, so it explains nothing about one that is missing. Verified against a realistic sample rather than assumed —outside_requested_scope=853plusvanished_before_hashing=2sums to 855, and theunknown_rule=260on the same line stays out of it.Six full scans, now five
The remaining pair in the evasion section is inherent: comparing the terminal against the JSON needs one invocation of each, and they have to be adjacent so they describe the same state.
The other three are each doing a distinct job — the main cycle, the post-quarantine cycle with
observation_modeoff, and the memory-peak measurement — so they stay.Self-inflicted, and worth saying so
I added all three of the expensive checks in this session, and each looked cheap in isolation. The cost only became visible when a run sat at fifty-eight minutes and I went to find out whether it had hung. It had not — it was doing exactly what I told it to, three times over.