agentsh + Runloop

Runtime security governance for AI agents using agentsh v0.20.3 with Runloop Devboxes.

Why agentsh + Runloop?

Runloop provides isolation. agentsh provides governance.

Runloop Devboxes give AI agents a secure, isolated compute environment. But isolation alone doesn't prevent an agent from:

• Exfiltrating data to unauthorized endpoints
• Accessing cloud metadata (AWS/GCP/Azure credentials at 169.254.169.254)
• Leaking secrets in outputs (API keys, tokens, PII)
• Running dangerous commands (sudo, ssh, kill, nc)
• Reaching internal networks (10.x, 172.16.x, 192.168.x)
• Deleting workspace files permanently

agentsh adds the governance layer that controls what agents can do inside the sandbox, providing defense-in-depth:

+---------------------------------------------------------+
|  Runloop Devbox (Isolation)                             |
|  +---------------------------------------------------+  |
|  |  agentsh (Governance)                             |  |
|  |  +---------------------------------------------+  |  |
|  |  |  AI Agent                                   |  |  |
|  |  |  - Commands are policy-checked              |  |  |
|  |  |  - Network requests are filtered            |  |  |
|  |  |  - File I/O is policy-enforced              |  |  |
|  |  |  - Secrets are redacted from output         |  |  |
|  |  |  - All actions are audited                  |  |  |
|  |  +---------------------------------------------+  |  |
|  +---------------------------------------------------+  |
+---------------------------------------------------------+

What agentsh Adds

Runloop Provides	agentsh Adds
Compute isolation	Seccomp BPF syscall interception (execve, file, signal)
Process sandboxing	File I/O policy (seccomp file_monitor + Linux permissions)
API access to sandbox	Domain allowlist/blocklist
Persistent environment	Cloud metadata blocking
	Command blocking (shell shim + seccomp execve + BASH_ENV)
	Signal blocking (kill/tgkill/tkill via seccomp BPF)
	Secret detection and redaction (DLP)
	Shell RC file persistence prevention
	Hostname/identity spoofing protection
	Soft-delete file quarantine
	LLM request auditing + rate limiting
	Threat intelligence feeds (URLhaus)
	Package install security scanning (OSV)
	Transparent command unwrapping
	Dangerous binary removal (chmod 000)
	Complete audit logging

Quick Start

Prerequisites

• Python 3 with pip install runloop-api-client
• Runloop account and API key

Deploy and Test

git clone https://github.com/canyonroad/agentsh-runloop.git
cd agentsh-runloop

pip install runloop-api-client

# Set your API key
export RUNLOOP_API_KEY="your-api-key"

# Run the security demo (73 tests)
python example.py

How It Works

agentsh replaces /bin/bash with a shell shim that routes every command through the policy engine:

execute_sync runs: /bin/bash -lc "sudo whoami"
                     |
                     v
            +-------------------+
            |  Shell Shim       |  /bin/bash -> agentsh-shell-shim
            |  (intercepts)     |
            +--------+----------+
                     |
                     v
            +-------------------+
            |  agentsh server   |  Policy evaluation
            |  (auto-started)   |  + network proxy
            +--------+----------+
                     |
              +------+------+
              v             v
        +----------+  +----------+
        |  ALLOW   |  |  BLOCK   |
        | exit: 0  |  | exit: 126|
        +----------+  +----------+

Every command that Runloop's execute_sync() executes is automatically intercepted -- no explicit agentsh exec calls needed. The AGENTSH_SHIM_FORCE=1 environment variable ensures the shim routes through agentsh even without a TTY (Runloop runs commands via HTTP API).

This configuration targets agentsh v0.20.3. The v0.20 line ships an "honest detect" probe (#392) that reports security capabilities based on whether the seccomp listener can really be installed, plus a sandbox.seccomp.shellc.opaque knob (#386) controlling how the shim treats unparseable bash -c script bodies. Both matter on Runloop -- see Enforcement model on Runloop below.

Capabilities on Runloop Devboxes

agentsh detect reports Security Mode: minimal, Protection Score 65/100 on Runloop with this configuration:

Capability	Status	Notes
seccomp_user_notify	Working	Installs cleanly only with the unix_sockets wrapper disabled -- see below
seccomp-execve (command control)	Working	execve interception for server-routed commands (sudo/su/etc. blocked by policy)
seccomp-notify (file control)	Working	openat/stat enforcement for server-spawned processes
seccomp BPF	Working	Signal blocking (kill/tgkill/tkill), dangerous syscall blocking
capabilities_drop	Working	41/41 caps dropped, privilege reduction
Network policy (proxy)	Working	Exfil/SSRF/metadata/k8s blocked by rule, not by timeout
cgroups_v2	Not available	Runloop's own cgroup is missing the pids controller
FUSE	Detected, not mounting	/dev/fuse exists (mode 0600), deferred chmod via sudo
Landlock	Not available	CONFIG_SECURITY_LANDLOCK not enabled in kernel (ABI v0 only)
eBPF	Not available	Kernel built without CONFIG_DEBUG_INFO_BTF (no /sys/kernel/btf/vmlinux)
ptrace	Not available	No CAP_SYS_PTRACE; raw PTRACE_TRACEME returns EPERM. agentsh ptrace backend stays inactive
pid_namespace	Not available	Host PID namespace

Kernel: 6.18.5 | Latest v0.20.3 harness run: 70 pass / 0 fail / 3 platform-limited | Active backend: seccomp_user_notify (execve + openat interception)

Enforcement model on Runloop

On v0.16-v0.18 this integration used the unix_sockets wrapper (agentsh-unixwrap) to enable seccomp_user_notify. On the v0.20 line that wrapper now conflicts with agentsh's own enforcement: both try to install a seccomp_user_notify NEW_LISTENER, and the kernel allows only one per process, so the second install fails with device or resource busy (errno 16). With the wrapper enabled, agentsh falls back to "minimal" mode (15/100) and network policy is not actually enforced.

The fix is to disable the unix_sockets wrapper (sandbox.unix_sockets.enabled: false) and let agentsh own the listener directly. That restores real seccomp-execve + seccomp-notify interception (65/100) and, crucially, real network policy enforcement -- curl https://evil.com, cloud-metadata, internal-network, and Kubernetes requests are denied by rule (rule=block-evil-domains, etc.), not merely by connection timeout.

Two settings make this work:

• sandbox.unix_sockets.enabled: false -- avoid the NEW_LISTENER EBUSY conflict.
• sandbox.seccomp.shellc.opaque: allow -- Runloop's execute_sync launches the shim directly as /bin/bash -lc "<cmd>", which the shim sees as one opaque script. The default enforce fails closed and denies every opaque command (anything with a pipe, redirect, &&, $(), or nested bash -c) because the shim's own process isn't under the listener. allow lets those run; parseable single commands are still routed through full policy.

Defense Layers (deepest first)

1. seccomp_user_notify (execve) -- sudo/su/nsenter/etc. policed at the syscall level for server-routed commands
2. seccomp_user_notify (openat) -- file policy (workspace allow, system deny) for server-spawned processes
3. Seccomp BPF -- kill/tgkill/tkill and dangerous syscalls (mount, ptrace, sethostname...) blocked at kernel level
4. chmod 000 -- privilege escalation binaries (sudo, su, kill, nsenter, etc.) made non-executable in launch_commands
5. Network proxy -- domain/CIDR policy enforced by rule (exfil/SSRF/metadata/internal-net/k8s)
6. Shell shim -- /bin/bash replaced with policy-enforcing shim
7. BASH_ENV -- bash builtins (kill, enable, ulimit) disabled
8. capability drop -- 41/41 caps dropped from the bounding set
9. Linux permissions -- non-root user can't write system paths

Known limitations on Runloop

Runloop launches the agentsh shell shim directly as /bin/bash -lc "...", so agentsh cannot attach seccomp interception to the shim's own process -- only to processes the agentsh server spawns. This is intrinsic to the hosted-sandbox model (agentsh classifies Runloop as "minimal" mode) and leaves three protections unenforced. The harness reports these as [LIMITATION], separate from pass/fail:

Protection	Why it's unavailable	Mitigated by
sh-builtin signals (os.system('kill ...'))	The kill syscall from the shim's children isn't seccomp-policed	/usr/bin/kill/pkill/killall are chmod 000; kill of PID 1 fails (non-root EPERM)
Safe wrapper commands (env whoami, nice ls)	The shim fail-closes wrappers it can't police post-unwrap (rule=shellc-wrapper-bypass) -- conservative, not insecure	Dangerous wrappers (env sudo, nice sudo) remain blocked

Note: shell-RC persistence (.bashrc/.profile redirect appends) would also fall in this category, but we block it explicitly via the immutable bit (chattr +i in launch_commands) -- the agent can't clear it because CAP_LINUX_IMMUTABLE is dropped from the bounding set.

These would be lifted entirely if Runloop ran the shim under an agentsh-managed wrap (so its process tree inherits the listener), or granted the container enough privilege for a second NEW_LISTENER.

For Runloop Engineers: Optional Improvements

With agentsh v0.20.3 and unix_sockets disabled, 70 of 73 harness checks pass and 3 are platform-limited (see above). These platform changes would add further defense-in-depth:

Run the shim under an agentsh wrap -- High Impact

Current state: Runloop's execute_sync() spawns /bin/bash -lc ... directly, so the shim's own process is outside agentsh's seccomp listener. The sh-builtin signal path and wrapper unwrap path therefore can't be policed via seccomp.

What it unlocks: Full file and signal enforcement on every command path, lifting all three known limitations (and removing the need for the chattr +i shell-RC workaround).

How to improve: Launch the agent shell through agentsh wrap (or set the listener up before the shim execs) so the shim's process tree inherits the seccomp filter.

FUSE Mount -- Medium Impact

Current state: /dev/fuse exists but with mode 0600 (root-only). agentsh uses deferred mode with sudo chmod 666 /dev/fuse, but the FUSE mount doesn't trigger.

What it unlocks: VFS-level file interception, soft-delete quarantine (recoverable file deletion), symlink escape prevention.

How to improve: Create /dev/fuse with mode 0666 during container initialization.

Landlock -- Medium Impact

Current state: Kernel 6.18.5 exposes Landlock ABI v0 only; CONFIG_SECURITY_LANDLOCK path restrictions are not usable.

What it unlocks: Kernel-level filesystem and network restrictions as defense-in-depth.

How to enable: Rebuild kernel with CONFIG_SECURITY_LANDLOCK=y and add landlock to LSM boot parameters.

eBPF -- Low Impact

Current state: Kernel built without CONFIG_DEBUG_INFO_BTF, so no /sys/kernel/btf/vmlinux; cilium/ebpf CO-RE programs can't relocate.

What it unlocks: Kernel-level network monitoring.

How to enable: Rebuild the kernel with CONFIG_DEBUG_INFO_BTF=y (and ideally CONFIG_DEBUG_INFO_BTF_MODULES=y).

cgroups_v2 -- Low Impact

Current state: Disabled in config.yaml (sandbox.cgroups.enabled: false). Runloop's own cgroup is missing the pids controller and non-root processes can't create a sub-cgroup, so the v0.18.0+ probe fails closed.

What it unlocks: Per-command CPU/memory/PID limits enforced by the kernel (Runloop already enforces resource limits at the Devbox level).

How to enable: Delegate a cgroup subtree with the pids controller available so non-root processes can manage cgroup.subtree_control (standard systemd pattern).

Configuration

Security policy is defined in two files:

• config.yaml -- Server configuration: seccomp (unix_sockets wrapper disabled, shellc.opaque: allow, file_monitor, syscall blocking), network interception, DLP patterns, LLM proxy with rate limiting, FUSE settings, env_inject (BASH_ENV for builtin blocking), threat intelligence feeds, package install scanning, capability drop
• default.yaml -- Policy rules: command rules, network rules, file rules, transparent command unwrapping, hostname/shell RC protection

See the agentsh documentation for the full policy reference.

Project Structure

agentsh-runloop/
├── Dockerfile          # Blueprint image with agentsh v0.20.3 on Ubuntu 24.04
├── config.yaml         # Server config (seccomp, file_monitor, DLP, network, threat feeds, rate limits)
├── default.yaml        # Security policy (commands, network, files, deny-before-allow ordering)
└── example.py          # Python SDK integration tests (73 tests, 11 categories)

Testing

The example.py script creates a Runloop Blueprint and Devbox, then runs 73 security tests across 11 categories. Tests are reported as pass, fail, or [LIMITATION] (a protection Runloop's minimal mode cannot provide -- see Known limitations). It exits non-zero only if a real check fails.

• Diagnostics -- agentsh version, server health, shell shim, FUSE mount, BASH_ENV, HTTPS_PROXY, config files, agentsh detect
• AI agent protection -- rm -rf, data exfiltration, reverse shell, ssh blocked
• Cloud infrastructure -- AWS/GCP metadata, private networks, Kubernetes API blocked by network policy
• Multi-tenant isolation -- sudo, su, nsenter, docker, pkill, kill blocked (seccomp-execve + chmod)
• File access control -- workspace/tmp writes allowed; /etc, /usr/bin, /var writes blocked
• Filesystem protection -- cp/touch/tee/mkdir to protected paths blocked; Python read/write to /etc, /usr/bin, /root blocked; symlink escape blocked
• Multi-context blocking -- env/xargs/find -exec/nested script/Python subprocess sudo blocked
• Allowed operations -- echo, ls, git, bash, npm registry access verified
• Soft delete -- file create, rm, verify gone, trash list
• New security features (v0.11+) -- transparent command unwrapping (nice/nohup sudo blocked)
• v0.16.9 hardening -- hostname/domainname blocked, shell-RC writes blocked (.bashrc/.profile via chattr +i immutable bit; agent can't clear it because CAP_LINUX_IMMUTABLE is dropped), kill syscall blocked via seccomp BPF

export RUNLOOP_API_KEY="your-api-key"
python example.py

v0.20.3 Runloop Status

The latest full run against agentsh v0.20.3 (with the unix_sockets wrapper disabled and the chattr +i shell-RC guard) reports 70 passing, 0 failing, 3 platform-limited across the 73 checks. Network policy is enforced by rule (exfiltration, cloud-metadata, internal-network, and Kubernetes requests are denied with rule=..., not by connection timeout). Shell-RC persistence is blocked by setting the immutable bit on .bashrc/.profile/etc. in launch_commands -- the agent can't clear it because CAP_LINUX_IMMUTABLE is dropped. The 3 platform-limited checks all stem from one cause -- Runloop launches the shim directly, so agentsh can't attach seccomp to the shim's own process -- and are detailed in Known limitations on Runloop. The Docker build verifies the image installs agentsh 0.20.3+be96a4d2.

Runloop Devbox Environment

Property	Value
Base OS	Ubuntu 24.04
Kernel	6.18
Package Manager	apt (deb)
User	user (uid 4444)
Workspace	/home/user
Git	/usr/bin/git
Python	python3 available

Related Projects

• agentsh -- Runtime security for AI agents (docs)
• Runloop -- AI development platform

License

MIT