fix: declare container identity for systemd nodes - #59
Merged
tianyuzhou95 merged 2 commits intoSep 10, 2026
Merged
Conversation
Set container=oci in the final all-in-one image and explicitly pass it through Helm and standalone launchers so existing images also identify their PID 1 systemd as running inside a container. Without this identity, privileged shutdown can remount shared host filesystems read-only and leave replacement nodes unable to write logs or state. Check image metadata and PID 1 container detection in standalone CI, and document upgrade verification and the recovery order for mounts already affected by this issue. Signed-off-by: Tianyu Zhou <albert.zty@antgroup.com>
Keep the deployment guide focused on the container identity requirement, replacement requirement, and existing read-only filesystem limitation. Remove the detailed incident recovery and acceptance procedure, and update the agent guidance link description to match the shorter section. Signed-off-by: Tianyu Zhou <albert.zty@antgroup.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Deleting a privileged AKernel node Pod can leave its host-backed data filesystem read-only when PID 1 systemd fails to detect that it is running in a container. Systemd's shutdown remount changes the filesystem superblock, so a replacement Pod inherits the read-only state and cannot start sandboxd successfully.
Declare lowercase
container=ociin the final all-in-one image and explicitly pass it in the Helm node environment and standalone launcher. The launcher settings also protect deployments using older images. Both Terraform providers consume the same Helm node template. Add checks for the built image environment and PID 1 container detection to standalone CI, and briefly document the deployment implications.This follows the systemd container interface and its v255 shutdown guard. Existing shutdown signals and service cleanup remain intact; the fix does not automatically remount filesystems or delete sandbox state.
Validation completed locally:
make deploy-script-check, entrypoint Bash syntax, CI YAML and embedded shell syntax, andgit diff --checkpassed.templatefilewith synthetic inputs and optional features off/on. Each rendered node environment contains exactly onecontainer=ociand retains existing variables.Full image build and live PID 1 checks were not run locally; they are included in the existing standalone E2E workflow. Target containerd/ACK validation of normal deletion, rolling replacement, and recreation with dual XFS filesystems and checkpoint bind mounts remains a deployment acceptance check. The deployment guide notes that replacement is required and already read-only filesystems are not automatically repaired.