Skip to content

feat(ec2): per-subnet docker networks for L3 isolation (#1745 phase 2)#1755

Merged
vieiralucas merged 4 commits into
mainfrom
worktree-ec2-netiso-batch2-subnet-networks
Jun 17, 2026
Merged

feat(ec2): per-subnet docker networks for L3 isolation (#1745 phase 2)#1755
vieiralucas merged 4 commits into
mainfrom
worktree-ec2-netiso-batch2-subnet-networks

Conversation

@vieiralucas

@vieiralucas vieiralucas commented Jun 17, 2026

Copy link
Copy Markdown
Member

Summary

Phase 2 of EC2 real network isolation (#1745). Stacked on #1754 (phase 1) — base will retarget to main once that merges.

Backing containers all shared the default Docker bridge, so instances in different VPCs could reach each other and there was no L3 segmentation. This attaches each instance's container to a per-subnet daemon network:

  • RunInstances computes an InstanceNetwork { subnet_id, internal } from the resolved subnet (internal = the subnet has no 0.0.0.0/0 -> igw route) and passes it to the runtime.
  • The Docker backend ensures fakecloud-subnet-<id> exists (idempotent; --internal for private subnets), labels it fakecloud-subnet=<id> + the shared fakecloud-instance ownership label so the startup reaper prunes it, and attaches with --network.
  • Same-subnet instances share a bridge and can talk; different VPCs/subnets get different bridges and cannot route to each other.
  • Best-effort: if network creation fails the instance still boots on the default bridge (no regression vs metadata-only).
  • k8s pods keep their flat network (isolation there is a NetworkPolicy concern, phase 4). Subnet placement is recorded so persisted instances recover onto the same network, and phase-5 introspection can report it.

Test plan

  • crates/fakecloud-e2e/tests/ec2_network_isolation.rs (Docker-gated, hard-fails in CI): same-subnet reachability, cross-VPC isolation (ping passes/fails accordingly), private subnet -> --internal network, public/default subnet -> not internal.
  • cargo test -p fakecloud-ec2, cargo clippy -p fakecloud-ec2 --all-targets -- -D warnings, cargo fmt --all --check clean.

Summary by cubic

Add per-subnet Docker networks for EC2 instances to enforce L3 isolation across VPCs/subnets. Phase 2 of #1745; persists subnet placement so restarts keep the same network; k8s remains flat.

  • New Features

    • RunInstances derives InstanceNetwork { subnet_id, internal } (private if no 0.0.0.0/0 -> igw) and passes it to the runtime.
    • Docker ensures fakecloud-subnet-<id> exists (idempotent), labels it fakecloud-subnet=<id> and fakecloud-instance=<daemon>, attaches with --network; uses --internal for private subnets. Falls back to the default bridge if creation fails.
    • Runtime stores subnet placement and returns it in RunningInstance.network; restarts re-attach to the same network. k8s returns network: None.
    • New Docker-gated E2E test verifies same-subnet reachability, cross-VPC isolation, and --internal on private vs public/default subnets.
  • Bug Fixes

    • Updated the k8s integration test to the new 4-arg run_instance signature.
    • Dropped an unused binding in the E2E test to satisfy clippy -D warnings.

Written for commit 800114f. Summary will update on new commits.

Review in cubic

Base automatically changed from worktree-ec2-netiso-batch1-default-vpc to main June 17, 2026 14:54
Backing containers all shared the default bridge, so instances in different
VPCs could reach each other and there was no L3 segmentation. Attach each
instance's container to a per-subnet daemon network instead:

- RunInstances computes an `InstanceNetwork { subnet_id, internal }` from the
  resolved subnet (internal = the subnet has no `0.0.0.0/0 -> igw` route) and
  passes it to the runtime.
- The Docker backend ensures `fakecloud-subnet-<id>` exists (idempotent;
  `--internal` for private subnets), labels it `fakecloud-subnet=<id>` plus the
  shared `fakecloud-instance` ownership label so the startup reaper prunes it,
  and attaches the container with `--network`.
- Same-subnet instances share a bridge and can talk; different VPCs/subnets get
  different bridges and cannot route to each other. Network creation is
  best-effort: on failure the instance still boots on the default bridge (no
  regression vs metadata-only).
- k8s pods keep their flat network (isolation there is a NetworkPolicy concern,
  phase 4). Subnet placement is captured in the runtime record so persisted
  instances recover onto the same network after a restart, and so phase-5
  introspection can report the backing network.

Tests: e2e (Docker-gated, hard-fails in CI) proving same-subnet reachability,
cross-VPC isolation (ping passes/fails accordingly), and that private subnets
back onto `--internal` networks while public/default subnets do not.
The per-subnet network arg added for phase-2 changed run_instance's
signature; the feature-gated k8s integration test (only compiled in the kind
CI job) still called the 3-arg form and would fail to compile there.
@vieiralucas vieiralucas force-pushed the worktree-ec2-netiso-batch2-subnet-networks branch from ee40f92 to 5b8c1af Compare June 17, 2026 14:56
@codecov

codecov Bot commented Jun 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 83 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
crates/fakecloud-ec2/src/runtime/mod.rs 0.00% 49 Missing ⚠️
crates/fakecloud-ec2/src/service/instance.rs 0.00% 16 Missing ⚠️
crates/fakecloud-ec2/src/service/mod.rs 0.00% 15 Missing ⚠️
crates/fakecloud-ec2/src/runtime/k8s.rs 0.00% 3 Missing ⚠️

📢 Thoughts on this report? Let us know!

@vieiralucas vieiralucas merged commit 1e9f05d into main Jun 17, 2026
54 checks passed
@vieiralucas vieiralucas deleted the worktree-ec2-netiso-batch2-subnet-networks branch June 17, 2026 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant