Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/integration-wsl.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,10 +49,10 @@ jobs:
wslc version

- name: Restore
run: dotnet restore src/WSLTestContainers.slnx
run: dotnet restore src/Containers.slnx

- name: Build
run: dotnet build src/WSLTestContainers.slnx --configuration Release --no-restore
run: dotnet build src/Containers.slnx --configuration Release --no-restore

- name: WSLC backend contract
# Real containers on a shared WSLC session; ~27 tests and ~4 minutes.
Expand Down Expand Up @@ -86,4 +86,4 @@ jobs:

if ($failed.Count -gt 0) {
throw "WSLC integration suites failed: $($failed -join ', ')"
}
}
7 changes: 2 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,17 +22,14 @@ whenever a target framework, a backend package or the `buildTransitive` assets c

| Path | Purpose |
| --- | --- |
| `src/WSLTestContainers.slnx` | Canonical solution for restore, build, test and pack |
| `src/Containers.slnx` | Canonical solution for restore, build, test and pack |
| `src/src/Containers` | Umbrella package (`Purview.Containers`, `net10.0`): references `Core` and both backends; no code of its own |
| `src/src/Core` | Backend-neutral abstractions (`Purview.Containers.Core`, `net10.0`, namespace `Purview.Containers`): `IContainer`/`ContainerConfiguration`, `ContainerBuilder`, `ContainerBase`, `IContainerBackend`/`ContainerBackends`, wait strategies, images, networking, mounts, diagnostics |
| `src/src/Wsl` | WSL Containers backend (`Purview.Containers.Wsl`, multi-target `net10.0` facade + `net10.0-windows10.0.19041.0` implementation): `WslContainerBackend`, the shared session runtime, `WslContainer`, `WslContainerSession`, and the portable facade (`WslPayload`, `*.Facade.cs`) that loads the implementation at run time |
| `src/src/Docker` | Docker backend (`Purview.Containers.Docker`, `net10.0`): `DockerContainerBackend` + `DockerContainer` over Testcontainers |
| `src/src/<Module>` | Service modules (`Purview.Containers.<Module>`, `net10.0`); each is a thin layer over the abstractions and carries a bespoke `Sdk/README.md` |
| `src/src/<Project>/Sdk` | Package-only assets. `Sdk/README.md` is packed as the package README (suppressing the repo-root README); `Sdk/buildTransitive/**` ships MSBuild assets to consumers (the abstractions' backend registration, each backend package's own registration entry, and the WSL backend's consumer defaults and guards) |
| `src/tests` | TUnit unit (`*.UnitTests`) and container integration (`*.IntegrationTests`) projects: the WSLC suites need a WSLC host, `Docker.IntegrationTests` and `Modules.DockerIntegrationTests` need a Docker daemon |
| `spikes/WslcSpikes` | Phase 0 investigation harness (`s1`..`s14` probes); not part of the test run |
| `spikes/DynamicLoadSpike` | Phase 0 feasibility probe for the portable facade: loads the WSLC projection from a local payload and drives a session from a plain `net10.0` process; not part of the test run |
| `spikes/PortableConsumerSpike` | Acceptance probe for the portable facade: a `net10.0` project that binds the facade and runs a real WSLC container through a `wslc/` payload; not part of the test run |
| `docs/wiki` | User-facing documentation wiki, aggregated by the purview-dev website. `Backends.md` is the consumer guide to choosing and configuring WSLC or Docker |
| `samples/getting-started` | Runnable consumer-shaped samples (WSLC and Docker) built with the solution; `just sample-wsl` / `just sample-docker` |
| `mkdocs.yml` | Wiki site configuration (`docs_dir: docs/wiki`) |
Expand Down Expand Up @@ -188,7 +185,7 @@ Commit messages follow Conventional Commits enforced by the `commit-msg` lefthoo
[Testing](docs/wiki/Testing.md)); `.github/actionlint.yaml` declares that label. It is never attached to
`pull_request` or `push`.
- The workflows pin `dotnet-version` to `global.json`'s `sdk.version`; keep them in sync, and keep
`purview-build.json` pointing at `src/WSLTestContainers.slnx`. The shared workflow runs on
`purview-build.json` pointing at `src/Containers.slnx`. The shared workflow runs on
`ubuntu-latest`, which is why `EnableWindowsTargeting=true` must stay in `src/Directory.Build.props`.

## Completion checklist
Expand Down
2 changes: 1 addition & 1 deletion Justfile
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ export DOTNET_SKIP_FIRST_TIME_EXPERIENCE := "1"
export DO_NOT_TRACK := "1"

root_folder := "./src/"
solution_file := root_folder + "WSLTestContainers.slnx"
solution_file := root_folder + "Containers.slnx"
test_solution := solution_file
build_configuration := "Debug"

Expand Down
80 changes: 28 additions & 52 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,56 +118,40 @@ await rabbitMq.StartAsync();
string amqp = rabbitMq.GetConnectionString();
```

> These are the target APIs. The generic container is implemented; the module builders arrive in
> Phases 3–6. See the docs for the designed surface.

## Key design decisions (verified by spikes)
## Key design decisions

| Decision | Evidence |
| --- | --- |
| One shared process-wide session, shared storage path | image store is keyed by storage path; session start ~20 ms (S2); concurrent sessions can't share the VHD (S18) → auto-fallback to isolated store |
| Unique session names `wslc-{pid}-{rand}` | session names are machine-reserved (S1, S13) |
| Default `NetworkingMode = Bridged` | port mappings require Bridged; default is `none` (S4) |
| Random host ports via native `windowsPort=0` | race-free random assignment (S4) |
| Serialize session/container lifecycle ops | concurrent `Start` can race (`0x8000FFFF`) (S8) |
| No Ryuk-style reaper needed | `Session.Dispose()` frees the session; orphans block only their own name (S10, S14) |
| No container-name DNS | containers reach each other by IP only (S9) |
| One shared process-wide session, shared storage path | the image store is keyed by storage path; session start is ~20 ms; concurrent sessions cannot share the VHD, so the runtime falls back to an isolated store |
| Unique session names `wslc-{pid}-{rand}` | session names are machine-reserved |
| Default `NetworkingMode = Bridged` | port mappings require Bridged; the default is `none` |
| Random host ports via native `windowsPort=0` | race-free random assignment |
| Serialize session/container lifecycle ops | a concurrent `Start` can race (`0x8000FFFF`) |
| No Ryuk-style reaper needed | `Session.Dispose()` frees the session; orphans block only their own name |
| No container-name DNS | containers reach each other by IP only |
| `Image`/`WithTag` fail fast | invalid images/tags rejected at configuration time, never at pull time |

## Repository layout

```
src/
Wsl/ core runtime + WSL Containers backend (package Purview.Containers.Wsl)
PostgreSql/ PostgreSQL module (builder, container, Npgsql connection string)
Redis/ Redis module (builder, container, StackExchange.Redis connection string)
MsSql/ SQL Server module (builder, container, SqlClient connection string)
RabbitMq/ RabbitMQ module (builder, container, AMQP + management endpoints)
Azurite/ Azurite module (builder, container, blob/queue/table endpoints)
Nats/ NATS module (builder, container, client + monitoring endpoints)
MySql/ MySQL module (builder, container, MySqlConnector connection string)
tests/
SharedTestingFramework/ shared WSLC skip/helper for integration tests
Wsl.UnitTests/
Wsl.IntegrationTests/
PostgreSql.UnitTests/
PostgreSql.IntegrationTests/
Redis.UnitTests/
Redis.IntegrationTests/
MsSql.UnitTests/
MsSql.IntegrationTests/
RabbitMq.UnitTests/
RabbitMq.IntegrationTests/
Azurite.UnitTests/
Azurite.IntegrationTests/
spikes/
WslcSpikes/ Phase 0 investigation harness (run: see below)
Containers.slnx canonical solution (restore, build, test, pack)
src/
Core/ backend-neutral abstractions (Purview.Containers.Core)
Wsl/ WSL Containers backend (Purview.Containers.Wsl)
Docker/ Docker backend (Purview.Containers.Docker)
Containers/ umbrella package (Purview.Containers): Core + both backends
PostgreSql/ Redis/ MsSql/ MySql/
RabbitMq/ Azurite/ Nats/ service modules
tests/ TUnit unit + integration projects (WSLC and Docker suites)
samples/
getting-started/ runnable samples (WslSample, DockerSample, AutoSample)
docs/
wiki/ project wiki (mkdocs.yml -> docs_dir: docs/wiki)
index.md Home.md _Sidebar.md Getting-Started.md Testing.md
Architecture.md Lifecycle.md Networking.md Wait-Strategies.md Modules.md
Packaging.md Release-Flow.md Contributing.md Contributing-Modules.md
Wslc-Api-Investigation.md Wslc-Capability-Matrix.md
wiki/ project wiki (mkdocs.yml -> docs_dir: docs/wiki)
index.md Home.md _Sidebar.md Getting-Started.md Using-in-Your-Tests.md
Backends.md Consumer-Requirements.md Architecture.md Lifecycle.md
Networking.md Wait-Strategies.md Modules.md Testing.md Packaging.md
Release-Flow.md Contributing.md Contributing-Modules.md
```

## Documentation
Expand All @@ -188,7 +172,7 @@ The project documentation lives in [`docs/wiki`](docs/wiki/Home.md) and is publi
Every package also ships its own `README.md` (from `src/src/<Project>/Sdk/README.md`), so
`dotnet add package Purview.Containers.<Module>` brings documentation specific to that package.

The PostgreSQL, Redis, SQL Server and RabbitMQ modules work today:
All seven service modules work today:

```csharp
await using var postgres = new PostgreSqlBuilder()
Expand Down Expand Up @@ -275,13 +259,13 @@ just sample-wsl # needs WSL Containers
just sample-docker # needs a Docker daemon
```

> The `Wsl.IntegrationTests` module runs 27 real containers in one session and takes ~4
> minutes because WSLC serialises container operations; expect slow-test warnings while it runs.
> The `Wsl.IntegrationTests` module starts many real containers in one session and can take several
> minutes, because WSLC serialises container operations; expect slow-test warnings while it runs.

### Verifying the consumer contract

```powershell
just verify-consumers # pack, then build 16 throwaway consumer projects against ./artifacts
just verify-consumers # pack, then build 21 throwaway consumer projects against ./artifacts
just verify-consumers -Keep # same, keeping the generated projects for inspection
```

Expand All @@ -290,14 +274,6 @@ the happy path, the shipped `buildTransitive` defaults, the `PCC0001`/`PCC0002`
non-.NET-11 escape hatch that deliberately does not work. It needs network access and is therefore a
local step rather than part of the `[Category=Unit]` pipeline filter.

## Running the Phase 0 spikes

```powershell
dotnet build spikes/WslcSpikes/WslcSpikes.csproj
dotnet run --project spikes/WslcSpikes -- sfull
# or s1..s14 for individual behaviour probes
```

## License

MIT. This project is not affiliated with, or endorsed by, the Testcontainers project or Microsoft.
28 changes: 14 additions & 14 deletions docs/wiki/Architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,18 +63,18 @@ cache. Package consumers get registration from the generated module initializer
`Register(...)`. Consumer-facing guidance (including the CI example) is in
[Backends: WSLC or Docker](Backends.md).

## Session lifetime model (chosen after spikes)
## Session lifetime model

**One shared, process-wide session.** Findings that drove this:

- Session start is cheap (~20-30 ms) because the WSL VM is shared under the session manager. (EXP)
- The image store is keyed by the **storage path** (EXP, S2): a stable storage path gives a warm image cache across sessions and process restarts.
- Session start is cheap (~20-30 ms) because the WSL VM is shared under the session manager.
- The image store is keyed by the **storage path**: a stable storage path gives a warm image cache across sessions and process restarts.
- Per-container sessions would force a fresh storage path per container → a cold image store (re-pull) per container, which is the dominant cost (alpine ~2-3 s, python ~3-4 s). Not acceptable for test throughput.
- `Session`/`Container` lifecycle calls are **not reliably thread-safe**: a concurrent `Container.Start()` occasionally fails with `0x8000FFFF` (E_UNEXPECTED) (EXP, S8). Container isolation is achieved by unique container names + unique host ports, not separate sessions.
- `Session`/`Container` lifecycle calls are **not reliably thread-safe**: a concurrent `Container.Start()` occasionally fails with `0x8000FFFF` (E_UNEXPECTED). Container isolation is achieved by unique container names + unique host ports, not separate sessions.

Design rules:

1. **One lazily-started session per process** with a deterministic name `wslc-{processId}-{8 hex}` (unique per machine; session names are reserved until the session is disposed — EXP S1/S14) and a **stable storage path** (default `%LOCALAPPDATA%\Purview.WslContainers\sessions\{name}\`), configurable.
1. **One lazily-started session per process** with a deterministic name `wslc-{processId}-{8 hex}` (unique per machine; session names are reserved until the session is disposed) and a **stable storage path** (default `%LOCALAPPDATA%\Purview.WslContainers\sessions\{name}\`), configurable.
2. The session VM is capped at **4096 MB by default** (`WslContainerRuntimeOptions.Default`). This is required for SQL Server (which refuses to start below 2000 MB — `sqlservr: This program requires a machine with at least 2000 megabytes of memory`) and harmless for lighter containers. Override via `WslContainerRuntimeOptions.MemorySizeInMB`.
2. `IContainerBackend` is the seam for backends: a backend package (starting with `Purview.Containers.Wsl`) supplies `IContainer` instances, and `ContainerBackends` resolves which one runs. `IContainerRuntime` remains the WSLC-internal seam so advanced users/tests can substitute a per-container-session runtime for isolation experiments.
3. Container `DisposeAsync` never terminates the shared session; it deletes the container only.
Expand All @@ -88,25 +88,25 @@ Design rules:
process runs. Set `StorageMode.PerSession` (or an explicit `StoragePath` / `PURVIEW_CONTAINERS_STORAGE_PATH`)
for isolation. Session names stay unique per process; only the path is shared.
- **Concurrent sharing is not possible**: a running session exclusively locks its `storage.vhdx`
(a second session on the same path fails with `0x80070020` — EXP S18). The lock is taken lazily on the
(a second session on the same path fails with `0x80070020`). The lock is taken lazily on the
first store access, so contention can surface on `GetImages()` rather than at session start; the runtime
detects it there too and falls back to an isolated per-process store.
Sequential reuse works (EXP S2/S13): after a session ends, a new session on the same path sees its images.
- Cleanup: normal shutdown terminates+disposes the session (frees the name, EXP S14). Orphaned sessions from crashed processes block only their own name (EXP S13); they do not block other sessions or storage-path reuse.
Sequential reuse works: after a session ends, a new session on the same path sees its images.
- Cleanup: normal shutdown terminates+disposes the session (frees the name). Orphaned sessions from crashed processes block only their own name; they do not block other sessions or storage-path reuse.

## Image management

- `PullPolicy { Missing, Always, Never }`, default `Missing`.
- `EnsureImageAsync`: consult `GetImages()` (session store) → pull when missing.
- Concurrent pulls of the same image are **deduplicated** by a keyed async lock (WSLC does not dedupe; EXP S11).
- Concurrent pulls of the same image are **deduplicated** by a keyed async lock (WSLC does not dedupe).
- Registry auth via `Session.Authenticate(uri, user, pass)` → `AuthenticateResult.IdentityToken` → `PullImageOptions.RegistryAuth`. Credentials live in a `Secret` type; never logged or serialized.

## Ports

- `PortBinding(containerPort, hostPort?, protocol, hostAddress?)`.
- **Random host port = native `windowsPort=0`** (race-free, EXP S4). The assigned port is read from `Inspect().Ports`.
- **Random host port = native `windowsPort=0`** (race-free). The assigned port is read from `Inspect().Ports`.
- Explicit host ports are bound by WSLC at `Start`; a conflict throws `0x80072740` (surface as a clear `WslContainerPortInUseException`).
- Default host bind is IPv4 loopback only; IPv6 is not mapped (EXP S12).
- Default host bind is IPv4 loopback only; IPv6 is not mapped.
- UDP → `ContainerNotSupportedException`.

## Concurrency
Expand All @@ -129,10 +129,10 @@ fallback. Isolated stores are transient and are removed when their session termi

**No Ryuk-style sidecar process is needed.**

- `Session.Dispose()` removes the session from the manager and frees its name (EXP S14).
- `Session.Dispose()` removes the session from the manager and frees its name.
- The library registers a process-exit handler so the session is always disposed on normal termination.
- Crash residue: an orphaned session from a dead process remains registered with a dead `Creator PID` (EXP S10). It reserves only its own name. It can be swept with `wslc --session <name> system session terminate`; the library documents this as an optional maintenance step and may offer a dev-time sweep utility (CLI-based, clearly isolated from the core runtime).
- `Container.DisposeAsync` is idempotent: stop (SIGTERM→SIGKILL) then `Delete(Force)`; swallow `RPC_E_DISCONNECTED`/`ContainerNotFound` on double-delete (EXP S7).
- Crash residue: an orphaned session from a dead process remains registered with a dead `Creator PID`. It reserves only its own name. It can be swept with `wslc --session <name> system session terminate`; the library documents this as an optional maintenance step and may offer a dev-time sweep utility (CLI-based, clearly isolated from the core runtime).
- `Container.DisposeAsync` is idempotent: stop (SIGTERM→SIGKILL) then `Delete(Force)`; swallow `RPC_E_DISCONNECTED`/`ContainerNotFound` on double-delete.

## Observability

Expand Down
2 changes: 1 addition & 1 deletion docs/wiki/Getting-Started.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ just sample-docker # needs a Docker daemon
## 5. Build and pack locally

```powershell
just build # dotnet build of src/WSLTestContainers.slnx (Debug)
just build # dotnet build of src/Containers.slnx (Debug)
just pack # build + dotnet pack into ./artifacts
just pipeline-pack-validate # shared pipeline: restore, build, lint, test, pack, validate
```
Expand Down
3 changes: 1 addition & 2 deletions docs/wiki/Home.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,15 +107,14 @@ example and the troubleshooting reference.

| Path | Purpose |
| --- | --- |
| `src/WSLTestContainers.slnx` | Canonical solution for restore, build, test and pack. |
| `src/Containers.slnx` | Canonical solution for restore, build, test and pack. |
| `src/src/Containers` | Umbrella package (`Purview.Containers`): references Core and both backends; no code of its own. |
| `src/src/Core` | Backend-neutral abstractions (`Purview.Containers.Core`, namespace `Purview.Containers`): the container contract, builders and backend selection. |
| `src/src/Wsl` | WSL Containers backend (`Purview.Containers.Wsl`): containers, images, runtime, networking, mounts, diagnostics. |
| `src/src/Docker` | Docker backend (`Purview.Containers.Docker`): the same containers on a Docker daemon, via Testcontainers. |
| `src/src/<Module>` | Service modules; each carries a bespoke `Sdk/README.md` that ships as the package README. |
| `src/tests` | TUnit unit and integration test projects (WSLC and Docker suites). |
| `samples/getting-started` | Runnable WSLC and Docker samples (`just sample-wsl`, `just sample-docker`). |
| `spikes/WslcSpikes` | Phase 0 investigation harness (`s1`..`s14` behaviour probes). |
| `docs/wiki` | This wiki, aggregated by the purview-dev website. |
| `purview-build.json` | Shared `Purview.Build` pipeline configuration, including the exhaustive pack manifest. |
| `.agents` | Package-delivered agent skills, agents and prompts. |
Expand Down
Loading
Loading