Phase 2/9: Build the inactive migration planner, snapshot, journal, and verifier

[borg-codex-bot]

Parent: #447

Depends on: Phase 1/9

Delivery branch: codex/issue-447-immutable-job-ids

Delivery model: This phase is committed to the shared #447 integration branch and the single draft pull request against main. It is not merged or released independently.

Objective

Implement the non-destructive foundation for migration immutable_job_id_v1: detection, complete planning, stable UUID mapping, snapshots, audit records and post-migration verification. The migration must remain inactive at application startup until Phase 9.

Scope

Detection and inventory

• Inventory every persistent object listed in Introduce immutable job IDs across all job-dependent data and workflows #447 and the Phase 1 dependency checklist.
• Detect missing IDs, mutable active references, malformed records, ambiguous aliases and orphaned historical data.
• Return the migration runner contract with a boolean required field.
• Avoid changing files, refreshing caches, applying schedules or starting background writers during detection.

Stable migration plan

• Build the complete legacy-key-to-UUID mapping before any write.
• Generate UUIDv4 values once and persist the plan before application.
• Reuse the journaled mapping after interruption or retry.
• Match canonical legacy metadata by exact keys and explicit aliases only.
• Never match by display name alone.
• Normalize existing archive-prefix lists deterministically without losing valid previous prefixes.
• Classify ambiguous historical objects as unassigned; block on ambiguous or inconsistent active configuration.

Snapshot

Create a recoverable snapshot of every object the final migration can change. The snapshot manifest must include:

• migration ID and timestamp;
• source paths and object types;
• checksums and sizes;
• the planned UUID mapping and actions;
• enough metadata to document a manual recovery;
• no logged secret values.

Snapshot creation must finish and verify successfully before the apply phase can begin.

Audit and retry journal

Use the existing migration contract and record:

• pending, applied, skipped, failed, blocked or not_applicable;
• detection, planning, snapshot, apply and verification phases;
• affected objects and concrete actions;
• masked exception type and cause;
• reuse of an existing migration plan after interruption.

Verification library

Implement reusable checks for:

• unique valid IDs for active jobs;
• complete archive-prefix lists;
• resolvable active schedules and repository assignments;
• resolvable status and restore-test mappings;
• absence of forbidden mutable active references;
• classification rather than deletion of unresolved history;
• blocking of writable services when verification fails.

Activation boundary

Do not register an apply path that can partially convert a real installation. Planning and verification utilities may be exercised against isolated fixtures, but startup activation is reserved for Phase 9 after all readers and writers have been converted.

Tests

Cover all Phase 1 fixtures plus:

• idempotent repeated detection;
• stable UUID reuse after simulated interruption;
• snapshot completeness and checksum verification;
• snapshot failure before writes;
• malformed detector and apply return values;
• first-failure blocking of later migrations;
• masked audit errors without secrets;
• deterministic treatment of ambiguous and orphaned records;
• no filesystem changes during detection or dry-run planning.

Test-channel policy

Do not publish a test-channel version. This phase intentionally has no active end-to-end migration and is not safe or useful for installation testing.

Acceptance criteria

• A complete deterministic plan can be produced for every successful fixture.
• Invalid active states block before any write.
• Snapshots and audit logs satisfy the repository migration contract.
• Interrupted plans reuse the same UUIDs.
• Verification functions cover every target store.
• No real startup migration is activated.

[borg-codex-bot]

Phase 2 implementation delivered to the shared integration PR

Implemented on codex/issue-447-immutable-job-ids in draft PR #480. This is an
inactive foundation, not an installable migration or independent release.
Commit: e8fb69e (Build inactive identity migration foundation (#472)).

Delivered

• Direct read-only inventory and sealed complete UUID plan; no lazy runtime
  discovery, schedule cleanup or writes to installation data.
• Exact ownership checks and deterministic projection for the known dependent
  stores. Preserve historical evidence and former UUIDs; unresolved history
  stays explicitly unassigned. Conflicting active state blocks.
• Private persisted allocation, exact-byte snapshots with UTC timestamp,
  checksum manifest, ID map and action metadata; sequenced, hash-linked audit
  journal with fixed reason codes instead of raw secret-bearing errors.
• Resume validates the original complete footprint and exact before/after
  fingerprints and reuses the original plan and UUIDs. Mixed legacy/UUID state
  without its original plan, missing replacements and unexplained edits block.
• Independent actual-target verification, including changes between reads.
  Snapshot, bound confirmation, independent-backup acknowledgement, quiescence
  and actual captured cron revalidation are default-deny prerequisites.
• Developer documentation: docs/maintainer/identity-migration-foundation.md.

Important safety prerequisite for later activation

The private plan/snapshot/journal directory needs an explicitly selected,
persistent mounted filesystem supporting private 0700/0600 permissions and
hard links. The FAT /boot USB is unsuitable for this implementation; do not
silently fall back to weaker permissions. Reading configuration originals on
/boot is still supported. #479 must choose and validate the private location,
authenticate approval, capture real cron state and hold all writer exclusions.

Plans and snapshots can contain sensitive configuration or notification
payloads. They must not be attached to public issues, ordinary support bundles
or exposed through an unprotected download. Permissions are not encryption.

Verification

• 427 focused tests passed, including 244 new phase-2 tests and the phase-1
  fixture/contract suite plus existing migration, startup, source-path,
  schedule, status and runner regression tests.
• Python compilation and diff whitespace checks passed.
• The four phase-1 scenarios that need writer/source-change/snapshot/resume
  evidence use actual controlled filesystem/process-state tests, not fixture
  booleans presented as proof of authorization.

Deliberate remaining boundaries

There is no production apply, registry/startup activation, HTTP endpoint,
automatic rollback or live Unraid installation test in this phase. A successful
library verifier still cannot enable writable services. Complete job-option
validation and application readers/writers remain in #473-#478. #479 owns the
actual transaction, authenticated maintenance gate, full crash/restart tests
and first test-channel candidate. Incomplete/torn private publication state or
journals block for diagnosis instead of discarding questionable evidence.

Per the approved shared-branch delivery plan, final source preflight and all
package/release work are deferred to #479. PR #480 remains in draft and this
issue stays open for review/integration. Main remains available for hotfixes.