Skip to content

Measure what the commands cost, and save a baseline - #126

Merged
vmillet-dev merged 5 commits into
mainfrom
test/what-the-commands-cost
Sep 15, 2026
Merged

vmillet-dev merged 5 commits into
mainfrom
test/what-the-commands-cost

Conversation

@vmillet-dev

@vmillet-dev vmillet-dev commented Sep 15, 2026

Copy link
Copy Markdown
Owner

Closes #78.

A criterion harness under the command boundary, a baseline to compare against, and the three
things that came out of taking it seriously.

What is measured

store::* + view::* + the serde round-trip — a command is four lines, so that is nearly all
of its cost. Six groups against a file-backed database of 8000 notes of ~13 kB (~104 MB,
seeded once per group); an in-memory one has no pager, no page cache and no I/O, and would
measure something the application never does.

Not in CI, and not in cargo test either — cargo gives a [[bench]] target test = false,
so the suite never builds it. A timing assertion on a shared runner flaps.

Command Cost 800 → 8000
query_notes, search matching nothing 406 ms ×15.1
query_notes, unfiltered 403 ms ×14.7
query_notes, search folding accents 392 ms ×15.2
export_notes / import_notes 377 ms / 98 ms ×14.1 / ×11.8
list_tags 37.9 ms ×64
delete_notes then restore_notes, 100 notes 10.4 ms ×1.2
rename_tag across the corpus 8.2 ms ×2.2
move_notes / tag_notes, 100 notes 5.1 ms / 4.2 ms ×1.9 / ×1.8
list_trash 4.5 ms ×10.5
update_note 1.5 ms ×1.0
list_global_placeholders 1.5 µs ×1.0

query_notes has gone past its own debounce. It runs on every keystroke behind 150 ms; at
800 notes its 27 ms sat comfortably inside, at 8000 the query fired for one keystroke is still
running when the third one after it arrives. That is what #21 is about, and it has stopped being
theoretical.

The search still is not what costs. Folding accents is the cheapest of the three variants
— fewer notes survive to be serialised. Fetching 8000 × 13 kB out of SQLite and turning the view
into JSON is the whole cost.

list_tags degrades faster than the corpus. ×64 for ×10 the data, the only entry that does,
and invisible at 800 notes where it cost 591 µs. It joins note_tags to notes to read one
nullable column, so each of the 16 000 tag rows dereferences a ~13 kB note row. ⚠️ Likely cause,
not confirmed — no query plan was taken.

⚠️ The baseline --save-baseline main writes lands in the gitignored target/: local to one
machine, gone with cargo clean. The table in docs/architecture.md is the durable record.

The corpus leaked, every run

impl Drop for Corpus erased the temp directory — but Drop::drop runs before the struct's
fields, so SQLite still held the file open, and Windows refuses to delete over an open handle.
let _ = swallowed the error. Measured: 18 directories, 709 MB in %TEMP%, six added per
run, and at 8000 notes each one is ~104 MB.

The erasure moved onto a TempDir field declared last — fields drop in declaration order, so
the connection closes first. Verified on a 50-note corpus: six built and torn down, zero left.

Two pieces of housekeeping

Cargo.toml went from 128 lines to 94 with no comments left. What was load-bearing there
(autobenches = false, bench = false, and now Corpus not implementing Drop) is in
docs/architecture.md.

cargo-deny is down to check advisories, and deny.toml from 65 lines to 14. Across the
last 25 security.yml runs, licenses, bans and sources never made a single decision; the
one time the job went red it was this branch's rustls advisory. Advisories stay because
repos/…/dependabot/alerts answers "Dependabot alerts are disabled for this repository" — it
is the only net there is. GPL-3.0 compatibility becomes a release-time review rather than a
per-pull-request gate.

Also on this branch

rustls past RUSTSEC-2026-0285 (0.23.43 → 0.23.45), which is what turned the Security job
red in the first place.

⚠️ Both packages on one command, and that is necessary: cargo update -p rustls alone stops
at 0.23.43. 0.23.45 requires webpki ^0.103.14 and the lockfile held 0.103.13; a single-package
update is conservative and will not move a sibling to satisfy the package it was asked about, so
it falls back to the newest version compatible with what is already locked.

cargo update -p rustls -p rustls-webpki

Lockfile only. No --precise pin and no MSRV change — every crate involved declares
rust-version = "1.71", so rust-version = "1.88" was never the gate, and raising it as far as
1.93 changes nothing. Cargo's "latest Rust 1.88 compatible version" line describes its resolution
mode, not a refusal.

Checks

cargo fmt --check, cargo clippy --all-targets --all-features -- -D warnings, cargo test
(321 tests) — all green locally.

🤖 Generated with Claude Code

Thirty-eight commands and not one number attached to any of them. criterion
benchmarks below the command boundary, against a file-backed database of 800
notes of ~13 kB — the shape already quoted in notes/view.rs.

The first baseline is in docs/architecture.md. Two things it says: query_notes
costs 27 ms on 800 notes and runs on every keystroke, and the search is not
what costs — filtering is cheaper than not filtering, so the accent fold is
not the dominant term.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
vmillet-dev and others added 4 commits September 15, 2026 18:05
TLS 1.3 handshake messages were accepted across encryption level boundaries.
The handshake transcript stays authenticated, so a network-position attacker
cannot alter or complete one — the effect is that a peer could send in
plaintext what should have been encrypted without the connection being
rejected.

Reached through tauri-plugin-updater, which is the only thing here that
speaks TLS.

Both packages named on one command: 0.23.45 wants webpki ^0.103.14 and the
lockfile held 0.103.13, so updating rustls alone falls back to 0.23.43 —
which is still below the fix — rather than move a sibling.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
query_notes costs 403 ms there, past its own 150 ms debounce, and list_tags
is the only command that degrades faster than the corpus.

Corpus dropped its temp directory while SQLite still held the file open,
which Windows refuses to delete over: six groups leaked a corpus each.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
licenses, bans and sources never made a decision in 25 runs. Advisories
stay: Dependabot alerts are off, so this is the only net there is.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@vmillet-dev
vmillet-dev merged commit 9082bb3 into main Sep 15, 2026
11 checks passed
@vmillet-dev
vmillet-dev deleted the test/what-the-commands-cost branch September 15, 2026 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Nothing says what any of the 38 commands costs

1 participant