Skip to content

feat(eval): support configurable container runtime - #3894

Merged
Sayt-0 merged 1 commit into
mainfrom
feat/3887-container-runtime
Aug 3, 2026
Merged

feat(eval): support configurable container runtime#3894
Sayt-0 merged 1 commit into
mainfrom
feat/3887-container-runtime

Conversation

@Sayt-0

@Sayt-0 Sayt-0 commented Aug 3, 2026

Copy link
Copy Markdown
Member

Summary

  • add an eval-only --container-runtime flag, defaulting to docker
  • use the selected Docker-compatible CLI for both image builds and container runs
  • record the selected runtime in logs and evaluation JSON output
  • update evaluation documentation with a Podman example

Issue expectations

Expectation Implementation
Run evaluations without requiring the Docker CLI --container-runtime accepts an alternative Docker-compatible executable or path
Support Podman as the motivating runtime documented with docker agent eval agent.yaml --container-runtime podman
Preserve existing behavior Docker remains the default and the fallback for programmatic callers
Keep evaluation isolation evaluations continue to build and run in containers with the existing flags and cancellation behavior

Validation

  • go test ./...
  • go test ./cmd/root ./pkg/evaluation
  • ./scripts/build.sh
  • golangci-lint run
  • go run ./lint .
  • go mod tidy --diff

A real Podman daemon was not available locally. Build and run selection are covered with fake-executable tests against the Docker-compatible CLI contract.

Closes #3887

@Sayt-0
Sayt-0 requested a review from a team as a code owner August 3, 2026 12:35
@Sayt-0
Sayt-0 force-pushed the feat/3887-container-runtime branch from 3ef9bad to 94c7b59 Compare August 3, 2026 12:40
@aheritier aheritier added area/cli CLI commands, flags, output formatting area/core Core agent runtime, session management area/docs Documentation changes kind/feat PR adds a new feature (maps to feat:). Use on PRs only. labels Aug 3, 2026
@Sayt-0
Sayt-0 merged commit b20c73a into main Aug 3, 2026
19 of 20 checks passed
@Sayt-0
Sayt-0 deleted the feat/3887-container-runtime branch August 3, 2026 13:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/cli CLI commands, flags, output formatting area/core Core agent runtime, session management area/docs Documentation changes kind/feat PR adds a new feature (maps to feat:). Use on PRs only.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow eval command to be run with any container runtime

3 participants