You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Kata] Add an experimental Kata Containers backend - #1577
Experimental Linux kata backend: runs a request in a Kata microVM via containerd-shim-kata-v2 over ttrpc. Design and limits are in docs/backends/kata/kata-backend.md. Draft until #1610 and its spec are agreed.
Fedora x86-64, Kata 4.3.1 runtime-rs: scripts/ci/run_backend_validation_tests.sh kata passed 19, failed 0 (QEMU, Cloud Hypervisor, Dragonball, Firecracker). fmt and clippy clean. cargo test -p mxc-sdk --lib: only the 3 failures already on main.
Not verified: CI pool (/dev/kvm?), arm64, other Kata versions.
…ontract
Adds the experimental kata backend value and its kata section
(rootfs, hypervisor) to the 1.1.0-alpha contract, the runtime model,
and backend registration. The Kata backend itself follows in later
commits, so selecting it is still refused.
Published contracts are unchanged.
Adds a small ttrpc client and the protobuf messages that the Kata shim's
containerd.task.v2.Task service needs. The tests compare the encodings with
ones produced from containerd's own .proto files.
It adds no dependencies. The ttrpc and containerd-shim-protos crates would add
about 45 packages to Cargo.lock, including second versions of nix, thiserror,
and syn. Nothing uses the client yet.
The kata backend starts containerd-shim-kata-v2 itself and drives it over
ttrpc, with no containerd or Kubernetes. A request provisions a sandbox, runs
the command as an exec process in the started VM, and releases the sandbox on
every path. The hypervisor (qemu, clh, firecracker, dragonball) is chosen by
passing a Kata configuration file in the shim's runtime options.
The root filesystem is a directory (read-only overlay lower layer, writable
layer on a capped tmpfs) or an ext4 image (private copy on a loop device;
required for Firecracker). Filesystem policy becomes bind mounts, denied paths
become masks. The VM gets an empty network namespace, so only a deny-all
network policy is accepted; everything the backend cannot enforce is rejected
before a VM exists.
A request killed without cleaning up is reclaimed by the next one.
The streaming path now checks the experimental opt-in for this backend.
Lifecycle operations follow in a later commit.
Provision, start, exec, stop, and deprovision work for kata through the raw
exact 1.1.0-alpha JSON entry points. The sandbox id is kata:<token>; state is
kept on disk, so a later call can come from another process. Provision fixes
the filesystem and network policy. Stop keeps the writable layer, so a restart
sees earlier files. Typed SDK requests still do not reach kata.
Adds the kata provision root to the development contract and its generated
schema and TypeScript types. The .NET parity check treats kata like Windows
Sandbox: a lifecycle only the raw JSON lane reaches.
Adds the kata suite: lxc-exec runs on every hypervisor (guest kernel, exit
code, streams, deny-all network, filesystem policy, timeout, rejections), then
the prebuilt streaming_kata test covers streaming, kill, drop, a killed
request, and the container lifecycle. Every run checks that no sandbox
directory, mount, process, or loop device is left behind.
The build job ships streaming_kata with lxc-exec and runs the Kata unit tests
and the request-refusal tests, which need no root, KVM, or Kata. The matrix
runs kata on ubuntu-24.04 x64: the host prep installs a pinned Kata release
and Alpine root filesystem, and a missing KVM, Kata, or root filesystem fails
the job instead of skipping.
Documents the backend in docs/backends/kata and the schema, policy, and CI
docs.
Adds an experimental Linux Kata Containers backend to MXC’s execution engine, with create-and-run execution, streaming, and container lifecycle operations through the development JSON contract.
Changes:
Adds Kata runtime integration, filesystem isolation, deny-all networking, and resource cleanup.
Extends development contracts, generated artifacts, routing, and compatibility tests.
Adds Linux validation jobs, host preparation, and backend documentation.
File
Description
tests/scripts/run_kata_all_tests.sh
Adds multi-hypervisor functional validation and leak checks.
Addressed the review findings in 17cc774: relayed output no longer ends on EAGAIN (new test, red before), a failed wait thread now cleans up, SIGKILL needs a real Wait result, the loop device detaches if autoclear fails, and a relayed timeout is an error. Not changed: the netns test passes unprivileged here (umount2 returns ENOENT first), and CloseIO on native stdio transfer is a known limitation for a follow-up.
Forward Kata cleanup warnings from run-to-completion waits
src/mxc-sdk/src/core/mxc_engine/run.rs:315
[P2] This run-to-completion route drops Kata cleanup warnings: KataProcess::finish records teardown failures in warnings(), but Runner::wait_for_child never reads them. As a result, lxc-exec can report success after leaving mounts or sandbox state behind without the warning promised by the backend guide. Forward the child's warnings through the shared runner's response or logger before dropping it, while preserving streaming warning propagation.
Pushed ec65cea: stdin goes through a cancellable pump, fixing the early CloseIO on native stdio and the stuck stdin write; descendant kill and guest-side denied paths are follow-ups.
[P2] Cleanup failures are only saved in SandboxProcess::warnings(), but the lxc-exec path uses Runner::wait_for_child, which never reads those warnings or forwards them to its Logger/ScriptResponse. Consequently an unmount or deprovision failure is silently discarded when the handle is dropped, and the CLI can exit successfully with resources left behind. Forward post-wait warnings through the shared Runner to the logger used by lxc-exec, including on timeout/error outcomes.
Timeout kill fails to preserve timed-out result in wait and poll
[P2] With no process.timeout configured, mxc_sandbox_kill_for_timeout uses the inherited kill implementation without recording that termination was requested for a timeout. Subsequent wait/poll therefore reports an ordinary exit rather than setting out_timed_out. Override kill_for_timeout to record a separate pending-timeout state, confirm termination and perform teardown, then preserve the timeout result in both wait and polling paths.
[P2] This non-Linux arm skips the experimental authorization check performed by the Linux arm and by resolve_runner/run_state_aware. Consequently, mxc_spawn_json with Kata and experimental=0 returns unsupported_containment on Windows/macOS instead of the documented backend_unavailable experimental-opt-in error. Apply the same gate before returning the host-platform error so streaming preserves the other entry points' failure semantics.
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
Cleanup failures remain silent on the lxc-exec path because child warnings are not forwarded to Logger, allowing commands to report success despite leftover resources.
Before implementing a new feature or API, please first open a feature request and discuss the proposed design with the maintainers. This allows us to understand the motivating scenario, review the API surface, consider the impact across our supported platforms and SDKs, and agree on an approach before implementation begins.
Our contribution process is documented in the following guides:
This PR does not currently have a corresponding issue, feature specification, or agreed-upon design. If you would like us to consider this feature, please open a feature request describing the scenario and proposed API. We can continue the design discussion there, and ultimately may ask for a feature spec PR first before any implementation.
Thanks again for your interest in contributing to MXC.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Needs-AttentionRequires attention or a decision from the MXC maintainers.
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📖 Description
Experimental Linux
katabackend: runs a request in a Kata microVM viacontainerd-shim-kata-v2over ttrpc. Design and limits are indocs/backends/kata/kata-backend.md. Draft until #1610 and its spec are agreed.🔗 References
🔍 Validation
Fedora x86-64, Kata 4.3.1
runtime-rs:scripts/ci/run_backend_validation_tests.sh katapassed 19, failed 0 (QEMU, Cloud Hypervisor, Dragonball, Firecracker). fmt and clippy clean.cargo test -p mxc-sdk --lib: only the 3 failures already onmain.Not verified: CI pool (
/dev/kvm?), arm64, other Kata versions.✅ Checklist
Cargo.lock, thedependency-feed-checkcheck passes (see pull request builds)