fix(daemon): settings startup survives busy and interrupted writes

[DeusData]

Summary

Four nondeterministic Windows reds and one permanent failure mode share one area: how startup reads _config.db while another process writes it, or after a writer died.

Where	Symptom
test-windows-guards section_start_status_port / daemon_lifecycle (#2643 rounds, #2648, the v0.12.0 matrix)	error: the daemon did not accept the UI configuration
test-windows-guards section_churn_stability	codebase-memory-mcp: could not load saved runtime settings
v0.12.0 release smoke, windows-2025, Phase 15	the --ui=true --port=N server exited about 5 s after start, read as "no embedded UI assets"
test-windows-guards section_crash_recovery, and any real crash, taskkill /F or power loss during a settings write	every later command fails with "could not load saved runtime settings", permanently

Root cause:

• daemon start and the stdio server gave the set_context and set_ui_config replies the 1 s probe budget (MAIN_CONNECT_TIMEOUT_MS). The daemon answers set_ui_config only after journaled commits to _config.db. With --port a miss is fatal.
• Every UI daemon start committed even when nothing changed: the upsert rewrote equal values and fired the revision trigger.
• cbm_runtime_settings_init, at the top of main(), gave a committing writer 50 ms and then exited the process. cbm_config_open and the UI load had no busy wait at all, so a concurrent commit failed the daemon's own UI save at once and made the load silently keep the defaults.
• A writer killed mid-transaction leaves a hot rollback journal. Settings startup and the UI load open _config.db read-only, and SQLite cannot roll a hot journal back on a read-only connection (SQLITE_READONLY_ROLLBACK, not BUSY). Nothing in the product opened it read-write first, so the install stayed broken. New since v0.11.0.

Change

• cbm_runtime_settings_wait_for_writers(): one busy handler for every _config.db connection. It waits out a committing writer, logs config_db.busy_wait once per busy wait, and gives up only after 3 s (a wedged holder). It is used by settings startup, settings_open, the UI load, cbm_config_open and the recovery below.
• settings_read_snapshot(): the read-only snapshot used by settings startup and the UI load. On SQLITE_READONLY_ROLLBACK it recovers ONCE through a read-write open that reads the schema, so SQLite rolls the interrupted write back (config_db.hot_journal_recovered), then retries read-only. A journal is hot only while no live writer holds its lock, so this never undoes a write in progress. If the recovery itself fails (read-only volume, ACL), config_db.hot_journal_unrecovered names the -journal path, and startup stays fatal: reading the settings without the rollback would be inconsistent.
• write_value: ON CONFLICT ... DO UPDATE ... WHERE value IS NOT excluded.value. An unchanged value is not rewritten, so no commit and no revision bump.
• daemon start (MAIN_DAEMON_CTL_START_TIMEOUT_MS) and the stdio server (its bootstrap startup budget) wait for the two handshake replies. The connection is up, so the reply is an event that comes unless the daemon dies.

Tests (red on 8df0639, green on this branch)

• runtime_settings suite:
  • Three single-threaded lock tests. A second connection holds the lock, and a log sink commits it when the waiter logs config_db.busy_wait; no test depends on a duration.
    • startup_waits_out_a_committing_writer: base ASSERT(initialized) fails.
    • ui_save_and_load_wait_out_a_committing_writer: base ASSERT(saved) fails.
    • unchanged_ui_save_does_not_rewrite: base revision 6, expected 2.
  • Two hot-journal tests on a portable fixture. It snapshots the database and its journal while a spilled write transaction is open, rolls that transaction back, then restores the snapshot. It checks that the journal is hot (SQLite magic and a nonzero record count) before the code under test runs.
    • startup_recovers_a_hot_journal: base ASSERT(cbm_runtime_settings_init(...)) fails. Fixed: the pre-crash value is read and the journal is gone.
    • ui_load_recovers_a_hot_journal: base falls back to the legacy defaults (ASSERT(loaded.ui_enabled)).
• tests/windows/test_daemon_stability.py:
  • New section start_port_waits_for_settings_writer. It holds BEGIN IMMEDIATE against a real daemon start --port and releases on events: client exit, or 1.5 s after the daemon logged the wait. It releases by closing the holder connection, so the OS drops the lock with the handle. Base: exited_under_lock=True rc=1, "the daemon did not accept the UI configuration". Fix: PASS. Its cleanup always runs daemon stop for its cache, so a failure cannot leave a daemon that blocks the next run's residue cleanup.
  • section_crash_recovery now prints the last daemon status output instead of only "kept reporting it", which hid the settings error.

Proof on real binaries (stub-UI builds of 8df0639 and this branch)

Scenario	8df0639	this branch
config list while a second process holds BEGIN EXCLUSIVE	rc 1 after 0.09 s, "could not load saved runtime settings"	rc 0, waited, then read
daemon start --port=N while a second process holds BEGIN IMMEDIATE	rc 1 under the lock, "did not accept the UI configuration"	rc 0, daemon: started (permanent ...)
daemon up, --ui=true --port=N under BEGIN IMMEDIATE (smoke Phase 15)	exited after 1.12 s, "daemon UI configuration update failed"	stayed up and served the UI
a writer killed mid-transaction (SIGKILL on macOS, TerminateProcess on Windows), then config list twice	rc 1 both times, "could not load saved runtime settings" (macOS and Windows)	rc 0, recovered once (config_db.hot_journal_recovered), then rc 0 with nothing left to recover (macOS and Windows)

Verification

• macOS ASan, final tree: cli runtime_settings ui httpd daemon daemon_application daemon_bootstrap daemon_runtime daemon_frontend userconfig: 706 passed, 1 skipped (runtime_settings 14/14).
• Windows VM (arm64), CI-shaped binary (--with-ui TEST_SEAMS=1, as _test.yml builds it) of this tree. All 8 Windows guards are GREEN in 4 consecutive full runs: daemon_lifecycle including both UI-refusal checks, and daemon_stability including the new section, kill -9 recovery, churn and cold storm. runtime_settings is 14/14. The real-binary lock scenarios and the hot-journal scenario PASS.
• Revert checks:
  • The production files reverted, tests kept: each new lock test fails on its own assertion (see above).
  • Only the hot-journal hunk reverted: both hot-journal tests fail on their own assertion; the other 12 pass.
  • Only src/main.c reverted: config list still passes, but daemon start --port and --ui=true --port fail under the held lock while the daemon logs that it is waiting. The new guard section fails with daemon_waited=True. Every part of the change is needed.
• cppcheck 2.20 (the CI version) and clang-format (Homebrew LLVM) are clean on the changed production files.
• Skipped: the Linux container legs. The change is platform-neutral SQLite and IPC-timeout code, and PR CI runs both Linux legs.

Release relevance

The maintainer decided this ships in v0.12.0. The binaries change, so the release re-dispatch runs the full test matrix (skip_tests=false).

Found, not fixed here

• test-infrastructure/vm/win.sh guards builds without TEST_SEAMS=1, unlike CI, so test_daemon_lifecycle.py's refusal check is red on the VM for any tree.
• Every settings open re-secures the cache directory and, on Windows, rewrites owner and DACL on every file in it (win_repair_runtime_children), including files other processes are writing.
• Other set_context callers (one-shot CLI and hook paths) keep the 1 s bound; no failure was observed there.
• Test harness, Windows: a COMMIT of a change-free BEGIN IMMEDIATE transaction on the test's own Python connection (SQLite 3.53.3) returned plain SQLITE_BUSY (code 5, no journal) while another process's connection retried the lock. That happened in 3 of 5 full guard runs and 0 of 9 standalone runs. The section therefore releases by closing the connection. The SQLite-internal path that returns BUSY there is not pinned. An empty _config.db at BEGIN, where page 1 would be written and COMMIT would need EXCLUSIVE, is excluded: the section's warm-up daemon start / daemon stop creates the database first. This is not a product gap: after this PR every product connection to _config.db (settings_open, startup, the UI load, cbm_config_open and the hot-journal recovery) carries the busy handler, and SQLite invokes it for the lock waits at commit. A product commit therefore waits there instead of failing.
• On CI, the handshake budget change in main.c is exercised only by the Windows guard section. The POSIX legs never run it (the guard runs on POSIX only locally).