Phase 5/9: Cut over backup runtime, status, logs, locks, recovery, and notifications to job_id

[borg-codex-bot]

Parent: #447

Depends on: Phase 4/9

Delivery branch: codex/issue-447-immutable-job-ids

Delivery model: This phase is committed to the shared #447 integration branch and the single draft pull request against main. It is not merged or released independently.

Objective

Convert backup execution and all live or newly persisted operational state to immutable job_id, while retaining readable descriptors as snapshots. A running or completed backup must remain associated with the same job across later edits.

Runtime and execution

• Key JobManager state by job_id.
• Pass job_id explicitly through runner subprocess boundaries.
• Resolve repository, sources, retention and the complete current archive prefix from canonical job metadata.
• Use archive_prefixes[0] exactly for new archive names without hidden additions.
• Scope prune to the ordered explicit prefix list in the current repository.
• Address run, live log and cancellation operations by ID.
• Keep a separate immutable run_id for individual executions.

Locks, cancellation, and recovery

• Store lock ownership by job_id and show resolved friendly labels in conflicts.
• Use ID-based control files and runtime state.
• Preserve cancellation ownership while names or prefixes change.
• Convert runtime recovery state that must survive restart.
• Ensure Docker and VM recovery still completes safely after job metadata edits.
• Do not infer identity from readable filenames.

Status and logs

New status payloads must contain:

• full job_id;
• run-time job name;
• archive prefix;
• repository key and location snapshots;
• archive name and normal run statistics.

New filenames may include a readable bounded slug plus a short UUID suffix, but readers must trust payload identity only. Logs remain administrator-readable while lookup uses the full ID from structured metadata.

Legacy status filenames may remain unchanged when their payload is migrated or classified. Do not rename historical files solely for appearance.

Notifications and reminders

• Correlate notification events by job_id.
• Preserve readable run snapshots in messages.
• Convert reminder and deduplication keys without resending old events.
• Ensure overdue and restore-test-specific consumers can adopt the same ID contract in later phases.
• Never expose secrets in identity diagnostics or logs.

Archive and cache continuity

• Preserve the existing Borg repository and archive data unchanged.
• Define cache path migration or reuse without forcing unnecessary re-chunking.
• Preserve archive-prefix history for restore and retention.
• Do not treat UUIDs as archive names or Borg credentials.

Tests

• manual and scheduled backup execution by ID;
• exact archive prefix and retention filters;
• status payload and lookup after job edits;
• readable log filenames with payload-based identity;
• cancellation, concurrent-run prevention and repository locks;
• interrupted Docker/VM recovery;
• notification correlation and reminder deduplication;
• legacy status mapping and unassigned classification;
• Borg cache continuity where current cache paths depend on Type ID/location;
• no new operational state keyed only by mutable descriptors.

Test-channel policy

Do not publish a test-channel version yet. Although backup execution may work in isolation, dashboard, reports, restore workflows and startup migration are still incomplete. Validate with automated tests and isolated local fixtures only.

Acceptance criteria

• Every new run, lock, control record, status and notification carries job_id.
• Editing a job cannot detach an active or completed run.
• Existing archives and useful Borg cache state are preserved.
• New archive naming uses the explicit current prefix exactly.
• Legacy records are mapped or classified without silent deletion.

[borg-codex-bot]

Phase 5 implementation checkpoint is pushed to the shared #447 branch and draft PR #480.

Commit: ae5f47f
Author and committer: borg-codex-bot

• Immutable private start contexts and independent full job/run IDs now cover manual/scheduled execution, exact archive naming, logs, cancellation, locks, status, recovery and backup notifications.
• Existing normal SSE and complete RAM activity logging remain supported, including reconnect after WebUI restart and final persistence.
• Existing Borg caches are referenced explicitly across migration/rename. Retention uses one shared policy across the current/historical prefix union for automatic and manual maintenance, completing the pending Phase 4/9: Cut over job APIs, repository assignments, and schedules to job_id #474 integration.
• 633 distinct focused tests passed, including real Borg backups in both log modes, real union retention/Borg check, cancellation and Docker/VM recovery. Python/JavaScript checks and authenticated synthetic HTTP smoke passed.

The final malformed historical-status regression was fixed and all 71 record tests passed afterward. Full details, test limits and the external Borg concurrency limitation are documented in docs/maintainer/canonical-job-runtime.md and the rewritten PR description.

This phase remains integration-only. No final preflight, version bump or test-channel package is published before #479. Reporting/overdue presentation (#476), restore (#477), transfer/cleanup (#478) and the guarded migration/startup integration (#479) remain open. This issue is not closed independently of the umbrella integration.