feat: Detect a stale cold-pool aggregate during checkpoint validation

[Dfinity-Bjoern]

CanisterStates keeps a precomputed ColdStats aggregate over the cold pool so that total_consumed_cycles() and total_canister_memory_usage() are O(|hot|) rather than O(|all canisters|).

Nothing verified that the aggregate still matches the pool it summarizes, outside of debug_asserts — even though it is already consensus-critical: cold_stats.memory_usage feeds total_canister_memory_usage(), which message routing writes to subnet_metrics.canister_state_bytes (message_routing.rs:1503), which is certified in the state tree. So a silently drifted aggregate diverges subnets today, with nothing to say so.

This adds CanisterStates::validate_cold_stats(), which recomputes the aggregate from the cold pool and reports a mismatch, and calls it from checkpoint validation.

Scope and limits

Stated up front, because it would be easy to over-read what this buys:

• It is advisory, not enforcing. validate_eq_checkpoint discards the error via unwrap_or_else, logs CRITICAL_ERROR_REPLICATED_STATE_ALTERED_AFTER_CHECKPOINT, increments replicated_state_altered_after_checkpoint, and finalizes the checkpoint regardless. Detection also lags a full checkpoint interval, so a drifted value has already been served before the alert fires. This is detection and attribution, not prevention.

  Making it actually enforce would change shared state-manager behaviour for every subnet and every field that path validates. That is a deliberate non-goal here; if wanted it should be its own change.

• The cost is negligible. O(|cold|) arithmetic over already-resident canisters, on a path that already loads every canister from disk and deep-compares it against the in-memory state.

• The ordering is deliberate. It runs after the per-canister comparison, and the two errors are combined rather than ?-propagated. In the exact scenario where this check fires, the per-canister diagnostics are what tell the operator which canister drifted — an advisory check must not cost them that information.

Provenance

Split out of the subnet_metrics management endpoint work (#11032), where it was written after review challenged the determinism argument for reading the hot/cold partition. That endpoint does not depend on this check, and the two were separated so this hardening of pre-existing code can be reviewed on its own merits rather than alongside a new public API.

Verification

cargo check --all-targets clean, cargo fmt -- --check exit 0, targeted clippy with --deny warnings clean apart from three pre-existing unused_imports. bazel test on the two affected targets passes. Contains no reference to subnet_metrics and stands alone on master.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR adds an explicit runtime validation that the CanisterStates precomputed cold_stats aggregate still matches the actual contents of the cold pool, and wires that check into checkpoint validation so drift is detected (and attributed) in release builds.

Changes:

• Introduces CanisterStates::validate_cold_stats() to recompute cold-pool aggregates and report mismatches.
• Extends checkpoint canister-state validation to run validate_cold_stats() and combine its error with per-canister validation errors (without short-circuiting).
• Adds unit tests covering both the consistent case and a deliberately-staled aggregate case.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
rs/state_manager/src/checkpoint.rs	Runs cold-pool aggregate validation during checkpoint equality validation and combines outcomes with per-canister validation results.
rs/replicated_state/src/canister_states.rs	Adds validate_cold_stats() to detect stale cold_stats vs a recomputation over the cold pool, with rationale documented for checkpoint validation usage.
rs/replicated_state/src/canister_states/tests.rs	Adds tests to ensure validate_cold_stats() accepts consistent aggregates and rejects intentionally-staled aggregates.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.