engineering: support inline dm-verity in COSI-derived Host Configurations

[Paco Huelsz (frhuelsz)]

🔍 Description

engineering. Adds support for inline dm-verity to Host Configuration derivation from COSI, so trident grpc-client stream-disk can install Azure Container Linux (ACL) images.

Inline verity stores the hash tree inside the same partition as the data, at a byte offset, rather than in a dedicated hash partition.

🤔 Rationale

Streaming a real ACL image failed during derivation:

Derived Host Configuration is invalid: Referrer 'verity-1' of kind
'verity-device' references target 'partition-2' more than once

Derivation resolved the verity hash partition purely by image path. With inline verity the verity entry points at the same image as the filesystem, so it emitted a VerityDevice whose data and hash devices were the same partition, which the storage graph rejected. Trident also had nowhere to record where the hash tree began, and never passed an offset to veritysetup open.

This was not a regression — inline verity was never supported. git log --all --grep=inline is empty and nothing referenced hashOffset. #673 made ACL work for A/B update, but that path never derives a Host Configuration; it reads the root hash from usrhash= on the UKI cmdline and mounts /usr directly, so it never had to model the layout.

Changes

• Read hashOffset from COSI verity metadata. Image Customizer already emits this field for inline layouts — it is declared in its cosiapi package and was added upstream by azure-linux-image-tools#702 ("Add support for inline verity"), gated so it appears only when inline, and covered by an IC test. This change consumes what the producer already declares, and brings Trident's COSI reference up to date with it: hashOffset is now documented in Composable-OS-Image.md, the v1.2 JSON Schema, and the changelog. The addition is descriptive and backward compatible — the field is optional and absent for the ordinary separate-hash-partition layout, so every existing COSI stays valid.
• Carry the offset on the OS image, not the Host Configuration. Like the root hash, the offset is a property of the image, so it is read from image metadata at servicing time. It does not appear in the Host Configuration or Host Status.
• Allow a verity device to name the same partition as both data and hash device. hashDeviceId stays required, so the Host Configuration schema is completely unchanged — host-config-schema.json and the generated VerityDevice.md are byte-identical to main. The graph models the inline case as the single device it is rather than as a second reference to the same node, which leaves the duplicate-target and referrer-sharing invariants untouched. Verity referrer cardinality becomes 1..=2, and the hash partition-type cross-check is skipped when there is no distinct hash partition.
• Pass --hash-offset to veritysetup open, so an inline-verity device is actually opened and verified during servicing rather than skipped.

Two documentation sets are treated differently on purpose. The COSI reference describes the format Trident consumes, so it documents hashOffset as a factual property of images IC already produces. The Host Configuration reference describes what users may configure, and inline dm-verity is not an officially supported configuration, so it is deliberately not documented there — host-config-schema.json and the generated VerityDevice.md are byte-identical to main. The only generated-doc change is the verity-device row of the referrer cardinality table in Storage-Rules.md (2..2 → 1..2), which states a reference count without mentioning inline verity.

Derivation fails with a clear error naming hashOffset if an image uses inline verity but does not say where the hash tree starts, instead of producing a device that cannot be opened.

📝 Checks

• Check dev-docs/manual-validation.md

Manually validated end to end on a real ACL image (acl-base-3.0.20260706), installed via stream-disk from an Azure Linux installer ISO built with RPMs from this branch, in QEMU/KVM:

• derivation succeeds; storage graph builds; full HostConfiguration::validate() passes
• all five partitions streamed; Servicing completed successfully; reboot
• verity activated during servicing and reported verified
• installed disk boots with no install media attached: UEFI → GRUB → ACL kernel 6.6.143.1, with systemd activating dm-verity on /usr from the UKI cmdline (sysusr-usr.mount, /dev/mapper/usr, zero mount failures), 38 targets reached

First boot then stops at emergency.target because Ignition cannot reach Azure IMDS (169.254.169.254) — expected off-Azure for an image carrying flatcar.oem.id=azure, and downstream of everything this PR touches (verity was already mounted and verified before Ignition ran).

Before writing any code, the underlying veritysetup invocation was confirmed against the real disk image:

veritysetup open /dev/loopXp2 <name> /dev/loopXp2 270ed371... --hash-offset=1065345024
→ status: verified

Automated: 22 test suites pass workspace-wide, cargo fmt --check clean, cargo clippy introduces no new warnings. Adds test_derive_host_configuration_inner_inline_verity (pinning ACL's real usr partition type GUID, which exercises partition-type rules a generic Linux GUID would mask) and ..._inline_verity_missing_offset.

📌 Follow-ups

TODO:

• The v1.2 schema requires osPackages to be an array, but real ACL COSI files set it to null and Trident already tolerates that (acl COSI files do not contain ospackages #707). Validating an ACL image's metadata against the checked-in schema fails on osPackages alone — a pre-existing divergence, unrelated to this PR and left alone here.

🗒️ Notes

Opened as a draft: functionally validated end to end, but wanted review on the graph modelling choice before marking ready.

The main design question is how inline verity is represented. Two alternatives were built and discarded:

1. Drop the verity device entirely, mounting /usr read-only straight off the partition and leaving verity to systemd at boot. Smallest change and it validates, but Trident never verifies the image it just wrote. It also has a trap: the non-verity path drops the ro mount option, and resize is skipped only when is_read_only() || !fs_type.is_ext(). ACL survives on the btrfs clause alone; an ext4 inline-verity image would be resized and have its appended hash tree silently destroyed.
2. hash_device_id: Option<BlockDeviceId>. Cleaner conceptually, but it is a Host Status wire-format change that rippled through ~30 files.

The approach here keeps the Host Configuration and Host Status untouched while still verifying at install time.

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).
There may be pipelines that require an authorized user to comment /azp run to run.