Reduce unnecessary boot USB writes during idle operation and UI navigation

[borg-codex-bot]

Problem and user impact

A user reported regular writes to the Unraid boot USB stick under /boot/config/borg-backup/config/ while using the UI without changing settings.

Investigation confirmed unnecessary background saves of the notification queue, redundant permission updates on an inventory lock file, and repeated write-access probes in the configured data directory during ordinary UI usage.

Reduce these writes without changing notification delivery, locking, authentication, or storage safety behavior.

Measurements

The maintainer captured file events on the current test-channel installation on 2026-09-08:

• Idle: 07:26:01 to 07:31:01, 300 seconds, plugin UI closed.
• UI: 07:33:10 to 07:43:10, 600 seconds. Normal page navigation until 07:42, followed by rapid navigation across all menu pages.
• Both captures monitored 180 directories and completed without reported monitoring gaps or queue overflows.
• Evidence files: bbui-io-idle.jsonl and bbui-io-ui.jsonl. This issue records the relevant aggregate findings; the raw captures have not been uploaded.

Observed operation	Idle (5 min)	UI before 07:42	UI from 07:42
Replace notification-queue.json	10	18	2
Save users.json	0	1	0
Save sessions.json	0	1	0
Create, write, and delete .borg-ui-write-test	0	12	11
Update metadata of .inventory.lock	0	1	5

The two UI phases have different durations (approximately 8 min 50 sec and 1 min 10 sec); their raw counts are not equal-duration rates.

Boot USB findings

• notification-queue.json is saved about every 30 seconds independently of UI navigation. Each cycle creates and writes a temporary file, updates its metadata, and renames it over the destination. A local reproduction confirmed that even an empty queue is rewritten with identical content.
• At the configured 30-second interval this extrapolates to approximately 2,880 unnecessary saves per day of continuous operation. These are real file writes on the USB-backed filesystem. This is not a full-day measurement or a measurement of physical device write commands, bytes, or USB lifespan.
• users.json and sessions.json were each saved once at 07:33:32, consistent with sign-in. Repeated session saves during normal or rapid navigation were not reproduced.
• .inventory.lock received six metadata updates. The lock helper unconditionally reapplies permissions when acquiring the lock. No file-content WRITE event was observed for this lock.
• notification-delivery.lock produced WRITE_CLOSE events (20 idle, 67 during UI usage), but no content WRITE events. Closing a writable handle must not be counted as proof that file content was written.
• Other observed configuration and widget accesses were reads. Read-event counts are not physical USB read/write counts.

Data-directory finding

/mnt/user/borg_backup_ui/status/.borg-ui-write-test was created, written, and deleted 23 times during UI usage, including 11 times during rapid navigation. This is a filesystem write-access probe, not a restore test. On this installation it affects the configured data storage rather than the boot USB stick.

Code findings

Inspected source: f6bfd99dbdb9b53b33de60213acba1dc01b31739. The relevant queue, setup-validation, and inventory-lock functions are identical to origin/main at the time of investigation; these behaviors were not introduced by the job-ID work.

• runtime/lib/notification_events.py: drain_notification_queue() always calls _write_json() after selecting pending items, including when the queue is empty or unchanged.
• borg_backup_ui.py: _start_notification_delivery_loop() defaults to a 30-second interval.
• api/inventory_store.py: inventory_lock() calls os.fchmod(fd, 0o600) on every outer acquisition.
• api/config_api.py: get_setup_status() calls validate_runtime_config(), which calls ensure_data_dirs(). That helper creates and deletes .borg-ui-write-test, causing writes from a routine status request.

Requested changes

• Persist the notification queue only when its stored contents actually change. Preserve pending deliveries, retry timing, concurrency protection, and durable updates when work is enqueued or processed.
• Avoid redundant permission updates on the inventory lock when its required permissions are already satisfied. Preserve locking and enforce required private permissions when necessary.
• Remove write-access probes from routine page/setup-status queries. Preserve appropriate validation during setup and actual write operations, including existing mount and error handling.
• Add a user-facing release note describing the reduction in unnecessary background and UI-triggered writes.

Scope boundaries

• No relocation of configuration, secrets, sessions, or notification stores.
• No authentication/session redesign or change to login/logout behavior.
• No broad caching or read-performance refactor.
• No removal of locking or storage safety checks.
• No changes to job-ID migration or the SSH compatibility task. At the maintainer's explicit request on 2026-09-08, implement this on codex/issue-486-job-ids in PR Use permanent job IDs, explicit settings and compatible SSH with fewer idle writes (#486, #495, #496, #497) #494, using a dedicated Reduce unnecessary boot USB writes during idle operation and UI navigation #497 commit and release-note fragment. This replaces the initial plan for an independent Main-based branch. Handle optional SSH warnings compatibly and lower minimum Unraid to 6.12.5 #496 and Persist Borg security state to prevent unencrypted backups failing after reboot #498 remain outside this change.

Acceptance criteria and validation

• Repeated checks of an empty or unchanged notification queue do not create, rewrite, or replace its JSON file; future retries that are not yet due also leave it unchanged.
• Enqueueing, delivery, retry, and failure handling remain functional and persist real changes correctly.
• Repeated lock acquisition no longer reapplies already-satisfied permissions; required permissions and cross-process/thread exclusion remain enforced.
• Routine status/setup GET requests do not create the write-test file. Required checks still work during setup and actual write operations, including unavailable or unwritable storage.
• Focused tests cover these cases and existing notification, authentication, and storage checks continue to pass.
• Repeat the 5-minute idle and 10-minute UI captures, including rapid navigation. Compare actual file-write/replacement and metadata events, rather than interpreting READ or WRITE_CLOSE alone as content writes.
• Document expected sign-in saves separately; do not claim a fix for repeated session writes that were not reproduced in the UI capture.
• Publish and verify a test-channel candidate for the maintainer before any stable release, following the repository workflow. Record the plugin and Unraid versions used for retesting.

Follow-up approved on 2026-09-09

Maintainer captures from candidate 2026.09.08.2345 verified the queue, inventory-lock and setup-status changes. The five-minute idle capture had no content writes or metadata changes, and the ten-minute UI capture had no queue saves or inventory chmod events. Both monitored 180 directories without reported gaps.

Two remaining findings are included in the explicitly approved follow-up:

• Ten data-directory write probes remained during UI navigation. The restore archive/file lists, repository statistics, target-directory browsing and restore-state GET handlers still called the writing storage guard. Switch these queries to the existing read-only guard; actual backup, restore, check and download actions retain the write probe.
• Ten distinct Python .pyc files were written once each alongside plugin sources on the boot USB stick. The maintainer selected a RAM cache: set the service's Python cache prefix to /run/borg-backup-ui/pycache, inherited by its child processes. Reuse it across service restarts and recreate it after reboot. This is Python program bytecode, not Borg data or configuration. It does not change job-ID migration or authentication. If the optional cache cannot be prepared, continue without bytecode writes rather than falling back to USB.

The RAM cache is allocated on demand, with no fixed 32 MiB cap. The preliminary estimate is 10-20 MiB, with 32 MiB as a planning allowance only; measure actual allocation on Unraid after navigation and job execution.

The UI capture also contained one users.json save and two sessions.json saves at the beginning, with no subsequent session saves. The extra earlier session save cannot be attributed from file events alone; no authentication changes are part of this follow-up.

Validation must cover all five restore GET handlers, missing/unmounted/unwritable storage, retained write failures for actual actions, real Python child-process cache inheritance, cache reuse, and startup without an available optional cache. Publish an updated candidate and repeat the Unraid file-event capture before considering this issue complete.

Follow-up measurements and diagnostic attachment (2026-09-09)

Candidate 2026.09.09.0916, source 9be8b322b269cc0f73b95279e9012c04525b0887, data root /mnt/user/borg_backup_ui. The maintainer previously reported Unraid 7.4.0 beta 2; the monitor itself does not record versions. Times are CEST (UTC+02:00).

• Idle: 09:23:04 to 09:28:04, 300 seconds.
• UI: 09:30:37 to 09:40:37, 600 seconds.
• Both monitored 180 directories, completed with incomplete: false, and contained no reported warnings/gaps. Individual event records agree with the summaries and final counts.

Observed operation	Idle (5 min)	UI (10 min)
Queue replacements	0	0
Inventory-lock permission/metadata changes	0	0
Restore write-test file create/write/delete cycles	0	0
Python bytecode writes on boot USB	0	0
User/session saves	0	0
Repository-info scheduler state saves	1	0

The UI capture contains only 67,684 READ events and 76 writable-handle closes on existing lock files, without any content WRITE events, file replacements, creations/deletions or metadata changes. The idle capture's one save at 09:25:23 was repository-info-refresh-state.json, consistent with the scheduler persisting planning state after its five-minute startup delay. Exact startup timing and changes beyond timestamps cannot be proven from file events alone. This is separate from the removed recurring queue writes.

The maintainer measured 13M (approximately 13 MiB) for /run/borg-backup-ui/pycache with du -sh. The RAM path is outside the standard monitoring roots. The targeted #497 write sources did not recur in the observed windows. The device captures do not demonstrate cache reuse after service restart, reconstruction after Unraid reboot or real notification delivery; those remain separate manual checks. No merge or stable release approval is implied.

Attachments and reproduction

• Detailed measurement report, evidence hashes, limitations and commands
• Diagnostic script attachment: bbui-io-watch.py
• Download the unchanged Python script

The script is attached as a versioned repository file. It uses Linux inotify and the Python standard library, records paths/events without reading monitored file contents, and is not part of the installed plugin. Its SHA-256 is a01135a0d51873770f3046854f46d073ba8dd4e28b5c01d7ec0a48ac986ee34c. Raw production captures are not uploaded; their hashes and exact capture windows are in the report because the filenames were reused.

Download on Unraid:

curl -fsSL 'https://raw.githubusercontent.com/borgforge/borg-backup-ui/98210d69f2294dcafe75d001a94f0c2d76836911/docs/maintainer/attachments/bbui-io-watch.py' -o /tmp/bbui-io-watch.py

After startup finishes, close the plugin UI and capture five idle minutes:

python3 /tmp/bbui-io-watch.py \
  --phase idle --seconds 300 \
  --data-root /mnt/user/borg_backup_ui \
  | tee /tmp/bbui-io-idle-497.jsonl

Then capture ten UI minutes, visiting all pages including Browse & Restore without saving settings, with rapid navigation in the final minute:

python3 /tmp/bbui-io-watch.py \
  --phase ui --seconds 600 \
  --data-root /mnt/user/borg_backup_ui \
  | tee /tmp/bbui-io-ui-497.jsonl

Use the actual configured data root. The two boot configuration roots are watched automatically. Copy results out of /tmp before rebooting. The report also includes the optional RAM-cache capture command: --root replaces default roots, so the boot roots must be repeated when adding the RAM directory. Do not add summary counts to the individual events again; WRITE_CLOSE alone does not establish a content write, and these counts do not measure physical USB writes or lifespan.

This documentation commit does not change the deployable source digest. The verified test package remains 2026.09.09.0916; no new test build is required for the attachment/report.