[docker-in-docker] - Move the iptables switching logic in the docker-init script and isolated tests for specific cases

[Kaniska244]

Summary

Fixes #1659 and fixes #1662.

This PR updates the docker-in-docker feature to support selecting the iptables backend at container runtime instead of only during image build. This is needed for hosts where the correct backend depends on the host kernel state at runtime, such as systems where the ip_tables kernel module is not loaded and iptables-legacy exists but does not work.

It also updates Docker Compose handling so dockerDashComposeVersion supports latest and uses it as the default.

Changes

• Adds a new docker-in-docker option:
  • iptablesSwitchAtRuntime:
    • Default: false
    • When enabled, the feature writes the iptables alternatives selection logic into docker-init.sh so it runs before dockerd starts.
• Keeps the existing install-time behavior when iptablesSwitchAtRuntime is not enabled.
• Updates the runtime selection logic to:
  • Prefer iptables-legacy only when the legacy backend is actually available through the host kernel.
  • Fall back to iptables-nft when legacy support is unavailable.
• Updates Docker Compose version support:
  • Adds latest as a valid dockerDashComposeVersion value.
  • Changes the default dockerDashComposeVersion from v2 to latest.
  • Updates install.sh, devcontainer-feature.json, and the README option table accordingly.
• Updates the docker-in-docker README with the new/updated options.
• Bumps the docker-in-docker feature version to 3.1.0.

Test Coverage

Adds isolated test scenarios for:

• Build-time iptables switching behavior.
• Runtime iptables switching behavior.
• Debian-based containers with ip_tables loaded.
• Debian-based containers without ip_tables loaded.
• Ubuntu-based containers with ip_tables loaded.
• Ubuntu-based containers without ip_tables loaded.
• Stress test coverage for iptablesSwitchAtRuntime=true.
• Docker Compose latest behavior with Moby enabled.
• Docker Compose latest behavior with Moby disabled.

Also updates CI to isolate the iptables-specific scenarios so they do not interfere with the standard docker-in-docker scenario test matrix.

Validation

The new tests verify that:

• docker-init.sh contains the runtime switching block only when iptablesSwitchAtRuntime=true.
• The selected iptables alternative is either iptables-legacy or iptables-nft as appropriate.
• iptables commands work.
• dockerd starts successfully.
• docker ps works inside the dev container.
• Docker daemon logs show successful initialization.
• docker compose version returns a valid version when dockerDashComposeVersion=latest.
• docker-compose --version returns a valid version when dockerDashComposeVersion=latest.
• /usr/local/bin/docker-compose is installed for the latest Docker Compose scenario.

[SrzStephen]

LGTM @ 7f2a5ed77143a90f735a7b6c799ac0d9cb44b55f but you pushed while I was typing ;)

I don't see anything wrong with 7360873 if the tests pass

There are cases where iptables can be loaded but iptables nat (same case for iptables-legacy) isn't... I can't think of a decent reason why this would come into play.

If you can think of one then it might be worth checking for iptables nat - I cant remember what case I ran into since it was ~ a month ago, entirely possible that I was doing something silly.

Example:
Both available

docker run --rm -it --cap-add=NET_ADMIN debian:bookworm-slim bash -c "apt-get update -qq && apt-get install -y iptables >/dev/null && echo '== iptables version ==' && iptables --version && echo '== NAT test ==' && iptables -t nat -L"

iptable-nat not available.

docker run --rm -it debian:bookworm-slim bash -c "apt-get update -qq && apt-get install -y iptables >/dev/null && echo '== iptables version ==' && iptables --version && echo '== NAT test (expected to fail) ==' && iptables -t nat -L || echo 'NAT not available or cannot access kernel tables'"

[Kaniska244]

LGTM @ 7f2a5ed77143a90f735a7b6c799ac0d9cb44b55f but you pushed while I was typing ;)

I don't see anything wrong with 7360873 if the tests pass

There are cases where iptables can be loaded but iptables nat (same case for iptables-legacy) isn't... I can't think of a decent reason why this would come into play.... If you can think of one then it might be worth checking for iptables nat - I cant remember what case I ran into since it was ~ a month ago, entirely possible that I was doing something silly.

Example: Both available

docker run --rm -it --cap-add=NET_ADMIN debian:bookworm-slim bash -c "apt-get update -qq && apt-get install -y iptables >/dev/null && echo '== iptables version ==' && iptables --version && echo '== NAT test ==' && iptables -t nat -L"

iptable-nat not available.

docker run --rm -it debian:bookworm-slim bash -c "apt-get update -qq && apt-get install -y iptables >/dev/null && echo '== iptables version ==' && iptables --version && echo '== NAT test (expected to fail) ==' && iptables -t nat -L || echo 'NAT not available or cannot access kernel tables'"

Hi @SrzStephen

Thank you for the review. Let me have a look for that scenario.

[SrzStephen]

Probably the only thing I'd try to make a case for, but I'm not overly familiar with this repos general policy here so if it tends to more conservative then that's ok:

If you're moving to a v4 release it might be worth making this a default: true since you're already cutting a major rev.

Reason: The only time it matters what iptables is present is at runtime so it's probably the correct approach, someone can always opt into the old behaviour if they wish (by setting it to false) but you'll probably see some "why doesn't docker run" issues similar to #1659.

[Kaniska244]

Makes sense. I would much rather put it back to version 3 as the flag could help doing this seamlessly.

[thediveo]

I'm wondering since 3.0.0 and 3.0.1 break on Ubuntu 24.04 devcontainer base images on a nUbuntu 26.04 host anyway and my testing indicates that the 3.1.0/4.0.0 feature works also on an Ubuntu 22.04 why not considering this a fix with a new compatible feature with the default of iptablesSwitchAtRuntime:true? This makes the 3.x feature working; are there any side effects of the iptablesSwitchAtRuntime?