Add openvmm as vmm backend option for wslc vms#40629
Draft
damanm24 wants to merge 10 commits into
Draft
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces an optional OpenVMM-backed virtual machine implementation for WSLC sessions, including the plumbing needed to orchestrate VM lifecycle and networking via ttrpc/protobuf, plus Windows-side socket I/O adjustments to support AF_UNIX-based vsock bridging.
Changes:
- Add an OpenVMM-based
IWSLCVirtualMachineimplementation and a minimal ttrpc client/envelope codec for vmservice RPCs. - Add a new Consomme networking mode and a new mini_init message to configure guest networking for that backend.
- Extend socket handling to support non-overlapped (blocking) I/O paths required by Windows AF_UNIX sockets, and wire service/session code to use a backend-agnostic “connect/accept vsock port” VM API.
Reviewed changes
Copilot reviewed 30 out of 30 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
| src/windows/wslcsession/WSLCVirtualMachine.h | Expose underlying VM interface pointer; adjust crash dump collection signature. |
| src/windows/wslcsession/WSLCVirtualMachine.cpp | Route vsock/crashdump connections through VM backend; add Consomme guest network setup; AF_UNIX crashdump relay path. |
| src/windows/wslcsession/WSLCSession.cpp | Adjust Docker client wiring and add mount-format-retry behavior for pre-attached disks. |
| src/windows/wslcsession/DockerHTTPClient.h | Detect socket family for Boost.Asio stream protocol assignment (HV socket vs AF_UNIX). |
| src/windows/wslcsession/DockerHTTPClient.cpp | Use VM backend to connect to per-request vsock ports. |
| src/windows/wslc/CMakeLists.txt | Add OpenVMM protobuf generation hook for wslc library when enabled. |
| src/windows/service/inc/wslc.idl | Add Consomme networking mode; extend VM interface with vsock/crashdump connection methods. |
| src/windows/service/exe/WSLCSessionManager.cpp | Choose VM backend (OpenVMM vs HCS) based on user setting; adjust settings for OpenVMM constraints. |
| src/windows/service/exe/TtrpcEnvelopeCodec.h | Define ttrpc header/envelope codec interface. |
| src/windows/service/exe/TtrpcEnvelopeCodec.cpp | Implement ttrpc request encoding and response decoding. |
| src/windows/service/exe/TtrpcClient.h | Define minimal ttrpc client + VM configuration helpers. |
| src/windows/service/exe/TtrpcClient.cpp | Implement AF_UNIX ttrpc transport, request/response handling, and vmservice RPC wrappers. |
| src/windows/service/exe/OpenVmmVirtualMachine.h | Define OpenVMM-backed IWSLCVirtualMachine implementation. |
| src/windows/service/exe/OpenVmmVirtualMachine.cpp | Implement OpenVMM process launch, vsock bridge listeners, ttrpc orchestration, and disk management. |
| src/windows/service/exe/HcsVirtualMachine.h | Extend HCS backend declaration to implement new vsock/crashdump methods. |
| src/windows/service/exe/HcsVirtualMachine.cpp | Implement new vsock/crashdump methods; create crashdump listen socket. |
| src/windows/service/exe/CMakeLists.txt | Conditionally include OpenVMM sources, proto generation, and compile defs. |
| src/windows/common/WSLCUserSettings.h | Add user setting key for enabling OpenVMM backend. |
| src/windows/common/WSLCUserSettings.cpp | Update default settings template and validator for OpenVMM option. |
| src/windows/common/ConsommeNetworking.h | Add Consomme networking engine and associated constants. |
| src/windows/common/ConsommeNetworking.cpp | Implement Consomme networking engine stubs and initial config fill. |
| src/windows/common/CMakeLists.txt | Add ConsommeNetworking sources; add ATL include-dir discovery logic. |
| src/shared/inc/SocketChannel.h | Add AF_UNIX detection and blocking send/receive paths for non-overlapped sockets on Windows. |
| src/shared/inc/lxinitshared.h | Add WSLC_CONFIGURE_NETWORKING message type and payload. |
| src/linux/init/WSLCInit.cpp | Handle WSLC_CONFIGURE_NETWORKING by configuring guest network + resolv.conf; register handler. |
| packages.config | Bump Microsoft.WSL.DeviceHost nuget version to include OpenVMM + proto. |
| msipackage/package.wix.in | Add openvmm.exe to MSI payload. |
| msipackage/CMakeLists.txt | Add OpenVMM-related DeviceHost binaries to packaging dependencies when enabled. |
| CMakeLists.txt | Add INCLUDE_OPENVMM option; conditionally fetch/build protobuf; add proto generation helper function. |
| cgmanifest.json | Add protobuf component metadata for CG compliance. |
Comment on lines
+601
to
603
| wil::unique_socket socket; | ||
| THROW_IF_FAILED(m_vm->ConnectToVsockPort(port, reinterpret_cast<HANDLE*>(&socket))); | ||
|
|
Comment on lines
617
to
619
| ConnectedSocket socket; | ||
| socket.Socket = wsl::windows::common::hvsocket::Connect(m_vmId, response.Result, m_vmTerminatingEvent.get(), m_initChannelTimeout); | ||
| THROW_IF_FAILED(m_vm->ConnectToVsockPort(response.Result, reinterpret_cast<HANDLE*>(&socket.Socket))); | ||
|
|
Comment on lines
+1259
to
1267
| wil::unique_socket socket; | ||
| HRESULT hr = m_vm->AcceptCrashDumpConnection(reinterpret_cast<HANDLE*>(&socket)); | ||
| if (hr == E_ABORT) | ||
| { | ||
| // VM is exiting. | ||
| break; | ||
| } | ||
| THROW_IF_FAILED(hr); | ||
|
|
Comment on lines
+1326
to
+1338
| constexpr size_t bufferSize = 65536; | ||
| std::vector<char> buf(bufferSize); | ||
| for (;;) | ||
| { | ||
| int bytesRead = ::recv(channel.Socket(), buf.data(), static_cast<int>(buf.size()), 0); | ||
| if (bytesRead <= 0) | ||
| { | ||
| break; | ||
| } | ||
|
|
||
| DWORD bytesWritten{}; | ||
| THROW_IF_WIN32_BOOL_FALSE(WriteFile(file.get(), buf.data(), static_cast<DWORD>(bytesRead), &bytesWritten, nullptr)); | ||
| } |
Comment on lines
+610
to
613
| // Connect the new socket via the VM interface. | ||
| wil::unique_socket connSocket; | ||
| THROW_IF_FAILED(m_vm->ConnectToVsockPort(response.Port, reinterpret_cast<HANDLE*>(&connSocket))); | ||
| wsl::shared::SocketChannel newChannel{ |
Comment on lines
481
to
+497
| HRESULT RemoveShare([in] REFGUID ShareId); | ||
|
|
||
| // Returns an event that is signaled when the VM exits (graceful or forced). | ||
| HRESULT GetTerminationEvent([out, system_handle(sh_event)] HANDLE* Event); | ||
|
|
||
| // Connects to a vsock port in the VM. Returns a socket handle. | ||
| // For HCS VMs, this uses hvsocket (supports overlapped I/O). | ||
| // For OpenVMM VMs, this uses the hybrid_vsock Unix domain socket bridge. | ||
| // AF_UNIX sockets do not support overlapped I/O | ||
| HRESULT ConnectToVsockPort([in] ULONG Port, [out, system_handle(sh_socket)] HANDLE* Socket); | ||
|
|
||
| // Accepts a crash dump connection from the VM. Blocks until a crash dump | ||
| // connection arrives or the VM exits. Returns E_ABORT if the VM exits | ||
| // before a connection is received. | ||
| // For HCS VMs, this uses an HV socket listener on the crash dump port. | ||
| // For OpenVMM VMs, this uses the hybrid_vsock Unix domain socket bridge. | ||
| HRESULT AcceptCrashDumpConnection([out, system_handle(sh_socket)] HANDLE* Socket); |
Comment on lines
+797
to
+804
| HRESULT HcsVirtualMachine::ConnectToVsockPort(_In_ ULONG Port, _Out_ HANDLE* Socket) | ||
| try | ||
| { | ||
| auto socket = wsl::windows::common::hvsocket::Connect(m_vmId, Port); | ||
| *Socket = reinterpret_cast<HANDLE>(socket.release()); | ||
| return S_OK; | ||
| } | ||
| CATCH_RETURN() |
Comment on lines
+287
to
+289
| // OpenVMM provides networking via its built-in consomme backend. | ||
| // Use ConsommeNetworking mode so the session process skips GNS but | ||
| // still configures the networking engine and port relay. |
Comment on lines
+726
to
+737
| // Try to reap the child. If WSLC_WATCH_PROCESSES already reaped it, we | ||
| // get ECHILD which is fine — the pipe close confirms the child exited. | ||
| int status = -1; | ||
| if (TEMP_FAILURE_RETRY(waitpid(childPid, &status, 0)) < 0) | ||
| { | ||
| if (errno == ECHILD) | ||
| { | ||
| // Child was already reaped by the WatchProcesses handler. | ||
| // The pipe confirmed it exited, so treat as success. | ||
| status = 0; | ||
| } | ||
| else |
Comment on lines
+131
to
+133
| // Set socket timeouts to prevent blocking forever if OpenVMM hangs. | ||
| setsockopt(m_socket, SOL_SOCKET, SO_RCVTIMEO, reinterpret_cast<const char*>(&c_socketTimeoutMs), sizeof(c_socketTimeoutMs)); | ||
| setsockopt(m_socket, SOL_SOCKET, SO_SNDTIMEO, reinterpret_cast<const char*>(&c_socketTimeoutMs), sizeof(c_socketTimeoutMs)); |
benhillis
reviewed
May 26, 2026
| endforeach() | ||
|
|
||
| if (INCLUDE_OPENVMM) | ||
| set(WSL_DEVICE_HOST_BINARIES wsldevicehost.dll;openvmm.exe) |
Member
There was a problem hiding this comment.
wsldevicehost.dll should be included all the time.
Member
There was a problem hiding this comment.
maybe that's an existing issue?
benhillis
reviewed
May 27, 2026
| Transaction.SendResultMessage(result < 0 ? errno : 0); | ||
| } | ||
|
|
||
| void HandleMessageImpl( |
Member
There was a problem hiding this comment.
I think we probably want to use the existing GNS implementation to do network configuration. it should support all of this stuff.
benhillis
reviewed
May 27, 2026
| set_target_properties(common PROPERTIES FOLDER windows) | ||
| target_include_directories(common PUBLIC ${CMAKE_CURRENT_BINARY_DIR}/../service/mc/${TARGET_PLATFORM}/${CMAKE_BUILD_TYPE}) | ||
|
|
||
| # ATL headers (atlsafe.h) are needed by precomp.h but not automatically included |
benhillis
reviewed
May 27, 2026
|
|
||
| // Disable features not yet supported by the OpenVMM backend. | ||
| WI_ClearFlag(m_featureFlags, WslcFeatureFlagsGPU); | ||
| WI_ClearFlag(m_featureFlags, WslcFeatureFlagsVirtioFs); |
Member
There was a problem hiding this comment.
what does openvmm use for host shares if not virtiofs?
Comment on lines
+45
to
+50
| # # Uncomment to use a locally built openvmm.exe instead of the NuGet package version. | ||
| # # Requires INCLUDE_OPENVMM=ON (pass -DINCLUDE_OPENVMM=ON when generating). | ||
| # # Point to the cargo output directory containing openvmm.exe: | ||
| # # Debug: set(OPENVMM_BUILD_DIR "d:/src/openvmm/target/debug") | ||
| # # Release: set(OPENVMM_BUILD_DIR "d:/src/openvmm/target/release") | ||
| set(OPENVMM_/BUILD_DIR "d:/src/openvmm/target/release") |
Comment on lines
+1399
to
+1404
| m_fileExisted = std::filesystem::exists(m_settingsPath); | ||
| if (m_fileExisted) | ||
| { | ||
| std::ifstream existing(m_settingsPath); | ||
| m_originalContent = std::string{std::istreambuf_iterator<char>(existing), {}}; | ||
| } |
Comment on lines
+1416
to
+1419
| { | ||
| other.m_fileExisted = false; | ||
| other.m_originalContent.reset(); | ||
| } |
Comment on lines
483
to
+504
| // Returns an event that is signaled when the VM exits (graceful or forced). | ||
| HRESULT GetTerminationEvent([out, system_handle(sh_event)] HANDLE* Event); | ||
|
|
||
| // Connects to a vsock port in the VM. Returns a socket handle. | ||
| // For HCS VMs, this uses hvsocket. | ||
| // For OpenVMM VMs, this uses the hybrid_vsock Unix domain socket bridge; | ||
| HRESULT ConnectToVsockPort([in] ULONG Port, [out, system_handle(sh_socket)] HANDLE* Socket); | ||
|
|
||
| // Accepts a crash dump connection from the VM. Blocks until a crash dump | ||
| // connection arrives or the VM exits. Returns E_ABORT if the VM exits | ||
| // before a connection is received. | ||
| // For HCS VMs, this uses an HV socket listener on the crash dump port. | ||
| // For OpenVMM VMs, this uses the hybrid_vsock Unix domain socket bridge. | ||
| HRESULT AcceptCrashDumpConnection([out, system_handle(sh_socket)] HANDLE* Socket); | ||
|
|
||
| // Maps a host port to a guest port via the VMM's built-in port forwarding. | ||
| // For OpenVMM consomme VMs, this creates a host-side listener in the VMM. | ||
| // For HCS VMs, returns E_NOTIMPL (port relay is used instead). | ||
| HRESULT MapPort([in] int Family, [in] unsigned short HostPort, [in] unsigned short GuestPort); | ||
|
|
||
| // Unmaps a previously mapped host port. | ||
| HRESULT UnmapPort([in] int Family, [in] unsigned short HostPort, [in] unsigned short GuestPort); |
Comment on lines
+499
to
+501
| wil::unique_event acceptEvent(wil::EventOptions::ManualReset); | ||
| WSAEventSelect(m_initListenSocket, acceptEvent.get(), FD_ACCEPT); | ||
|
|
Comment on lines
+287
to
+290
| // OpenVMM provides networking via its built-in consomme backend. | ||
| // Use ConsommeNetworking mode so the session process skips GNS but | ||
| // still configures the networking engine and port relay. | ||
| openVmmSettings.NetworkingMode = WSLCNetworkingModeConsomme; |
Comment on lines
+331
to
+348
| // Set OPENVMM_LOG so the openvmm tracing subscriber emits detailed logs. | ||
| // Without this, only INFO-level messages appear (the default), which omits | ||
| // most operational output from VM creation and runtime. | ||
| // The variable is set in the current process environment and inherited by the | ||
| // child; restore it after Start() to avoid polluting the service environment. | ||
| wil::unique_hlocal_string previousLog; | ||
| DWORD prevLen = GetEnvironmentVariableW(L"OPENVMM_LOG", nullptr, 0); | ||
| if (prevLen > 0) | ||
| { | ||
| previousLog.reset(static_cast<PWSTR>(LocalAlloc(LMEM_FIXED, prevLen * sizeof(WCHAR)))); | ||
| THROW_IF_NULL_ALLOC(previousLog.get()); | ||
| GetEnvironmentVariableW(L"OPENVMM_LOG", previousLog.get(), prevLen); | ||
| } | ||
|
|
||
| SetEnvironmentVariableW(L"OPENVMM_LOG", L"info,openvmm=debug"); | ||
| auto restoreEnv = wil::scope_exit([&] { | ||
| SetEnvironmentVariableW(L"OPENVMM_LOG", previousLog.get()); | ||
| }); |
Comment on lines
+502
to
+505
| HANDLE waitHandles[] = { acceptEvent.get(), m_vmExitEvent.get() }; | ||
| auto waitResult = WaitForMultipleObjects(ARRAYSIZE(waitHandles), waitHandles, FALSE, m_bootTimeoutMs); | ||
| THROW_HR_IF(E_ABORT, waitResult != WAIT_OBJECT_0); | ||
|
|
Comment on lines
+709
to
+712
| HANDLE waitHandles[] = { acceptEvent.get(), m_vmExitEvent.get() }; | ||
| auto waitResult = WaitForMultipleObjects(ARRAYSIZE(waitHandles), waitHandles, FALSE, INFINITE); | ||
| THROW_HR_IF(E_ABORT, waitResult != WAIT_OBJECT_0); | ||
|
|
Comment on lines
243
to
+245
| <File Id="wsldevicehost.dll" Source="${WSL_DEVICE_HOST_SOURCE_DIR}/bin/${TARGET_PLATFORM}/wsldevicehost.dll" /> | ||
| <File Id="wslvmservice.dll" Source="${WSL_VM_SERVICE_SOURCE_DIR}/bin/${TARGET_PLATFORM}/wslvmservice.dll" /> | ||
| <File Id="openvmm.exe" Source="${WSL_DEVICE_HOST_SOURCE_DIR}/bin/${TARGET_PLATFORM}/openvmm.exe" /> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds OpenVMM as a possible VMM backing WSLC VMs.
This feature is gated on both a compile time gate and a WSLC feature flag. Non-exhaustive list of changes made to implement this:
WSLC_CONFIGURE_NETWORKINGmessage that allows us to configure networking within the VM using OpenVMM's consomme networking stack.OpenVmmVirtualMachineclass that implements theIWSLCVirtualMachineallowing users to switch between traditional HCS VMs and OpenVMM VMs.SocketChannelclass to support non-overlapped I/O which is not supported when using AF_UNIX sockets. This is required to bridge the connection between themini_initprocess from the guest to thewslcservice.