Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 52 additions & 35 deletions docs/container.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
# The lifecycle of a task container

Every task in panopticon runs in its own **container** — a throwaway Docker container with a
`claude` agent inside it, working on a private clone of the repo. This doc explains that
container's life from the operator's chair: how one comes up, the statuses you watch on the
Every container-backed task in panopticon runs in its own **container** — a throwaway Docker
container running its selected agent harness (Claude Code, Codex, Pi, or Outfitter), working
on a private clone of the repo. Shell workflows such as `setup-repo` run on the host instead.
This doc explains the container's life from the operator's chair: how one comes up, the statuses you watch on the
dashboard, what each means, and how a container recovers or is torn down. It's about *observable
behaviour* from the operator's side.

Expand All @@ -11,9 +12,9 @@ For the token a container authenticates with, see [auth](auth.md); for the image

## One task, one container, one session

A task's work happens in exactly one place: its container. Nothing else in panopticon runs an
A container-backed task's agent work happens in its container. Nothing else in panopticon runs an
LLM — the task service (control plane) and the session service (runner) are deterministic host
processes that never call `claude`. That's the **determinism invariant**: the container is the
processes that never call an LLM. That's the **determinism invariant**: the container is the
*only* LLM-bearing component, so everything the agent does is scoped to it.

Concretely, each running task has:
Expand All @@ -30,31 +31,43 @@ service** only ever *records* and *displays* what the runner reports; it spawns

## The status you see on the dashboard

The dashboard shows one **container status** per task — a single word the task service computes
by folding three signals together: the spawn *phase* the runner is reporting, whether the
container has an open **registration** (its live connection), and whether the **runner** itself
is still connected. First match wins, so a status higher in this table always overrides a lower
one.
The dashboard shows one **container status** per task. The task service combines the task's
terminal state, launch pause, dependencies, and runner claim with the reported lifecycle
*phase*, the container's open **registration** (its live connection), and runner liveness.
The table explains the statuses; the selection rules below determine which one appears.

| Status | What it means | What you do |
|---|---|---|
| `queued` | Non-terminal but unclaimed — no runner has picked it up yet. | Wait; if it sticks, check a runner is actually running (`make start`). |
| `–` | Terminal task (COMPLETE / DROPPED) — no container concept. | Nothing. |
| `paused` | Launch is paused and no container registration is open; no failed phase takes precedence. | Resume launch when ready. |
| `gated` | The task is unclaimed and dependencies block launch. | Wait for the dependencies to clear, or inspect the blocking tasks. |
| `queued` | The task is unclaimed and eligible for a runner to pick up. | Wait; if it sticks, check a runner is actually running (`make start`). |
| `live` | The claimed task has an open container registration and no reported failed phase. The registration alone does not prove the agent is healthy. | Attach with `t` to watch or steer. |
| `disconnected` | The task is claimed, has no open registration, and its runner is no longer connected. | Bring that runner's host back, or the task stays stranded until its claim is released. |
| `claiming` | A runner just claimed it; the spawn is about to start. | Nothing — transient. |
| `preparing` | Readying the per-task clone / workspace. | Nothing — transient. |
| `building` | Composing and `docker build`-ing the image. **The slow step on a first run** for a repo (later runs hit Docker's cache). | Wait; first build of a repo image can take minutes. |
| `building` | Composing and building the image. Uncached steps can take time. | Wait; a first image build can take minutes. |
| `starting` | `docker run` and the tmux session are coming up. | Nothing — transient. |
| `awaiting` | Container and tmux are up; waiting for the agent to open its `/live` connection. | Nothing — transient; if it lingers, see *When it goes wrong*. |
| `live` | A container registration is open — the agent is running and reachable. | Attach with `t` to watch or steer. |
| `down` | Claimed, the runner is alive, but the container is gone and unregistered. It came up and vanished, or never reported. | Respawn from the dashboard with `R` (the runner also self-heals — see below). |
| `failed` | A spawn step raised an error (hover / inspect for the detail). | Read the detail; fix the cause (bad image layer, missing secret) and respawn. |
| `disconnected` | Claimed by a runner that's **no longer connected** to the task service. | Bring that runner's host back, or the task stays stranded until its claim is released. |
| `–` | Terminal task (COMPLETE / DROPPED) — no container concept. | Nothing. |

The dashboard only *displays* this status; it does no liveness guessing of its own. The five
middle statuses (`claiming`→`awaiting`) are the **spawn phases** the runner pushes as it works;
`queued`, `live`, `down`, and `disconnected` are *derived* by the task service from
registration + runner liveness, so the runner never invents them. (See
`compose_container_status` in `core/models.py` for the exact precedence.)
| `awaiting` | Container and tmux are up; waiting for the container's `/live` registration. | Nothing — transient; if it lingers, see *When it goes wrong*. |
| `healing` | The runner is recovering the task's missing session. | Wait for recovery; inspect repeated failures. |
| `failed` | A launcher or runner failure has been reported. It can remain visible even while the container's liveness process is registered. | Read the detail; fix the cause (bad image layer, missing secret) and respawn. |
| `down` | The task is claimed, its runner is live, and neither a registration nor a lifecycle phase remains. | Respawn from the dashboard with `R` (the runner also self-heals — see below). |

The selection rules in `compose_container_status` (`core/models.py`) run in this order:

1. A terminal task shows `–`.
2. A launch-paused task without a registration shows `paused`, or `failed` if that phase
was reported.
3. An unclaimed task shows `gated` when dependencies block it, otherwise `queued`.
4. A claimed task with an open registration shows `live`, except that a reported `failed`
phase wins. Other stale phases and runner disconnection do not override that registration.
5. Without a registration, a claimed task whose runner is disconnected shows `disconnected`,
even if a lifecycle phase remains.
6. Otherwise the reported phase shows through; with no phase, the task shows `down`.

The dashboard only displays this result. The runner reports `claiming`, `preparing`,
`building`, `starting`, and `awaiting` during spawn, plus `healing` during recovery and
`failed` on failure. Those phases are inputs to the rules, not a separate priority order.

## Coming up: the spawn sequence

Expand All @@ -71,9 +84,9 @@ reporting each as a status above.
and its `origin` is pointed at the real forge rather than the local cache.

3. **Build** (`building`). The runner composes and `docker build`s the task's image. It's tagged
per `(workflow, repo)`, so once built it's cached — only the **first** task for a repo pays the
full build cost, which is why `building` can be slow that first time. See [layers](layers.md)
for how the image is composed.
per `(harness, workflow, repo)`, with an optional runtime suffix. Docker reuses unchanged
layers, which is why the first build can be slow and later builds are usually faster.
See [layers](layers.md) for how the image is composed.

4. **Start** (`starting`). The runner does `docker run --detach` (injecting the repo's secrets
via `--env-file`, mounting `/workspace`, and a small config volume that persists the agent's
Expand Down Expand Up @@ -104,17 +117,21 @@ For what a slug, branch, clone, and `provisioned` mean as task concepts, see [ta
connection to the task service. While it's open the task reads `live`; if it drops (crash,
kill, network blip) the registration clears and the task stops being `live`. The container
reconnects with backoff across transient blips.
- **The agent.** In the tmux pane, the launcher wires everything the agent needs and then execs
`claude`: it renders the workflow's **skills** and **operations** as slash-commands, points the
agent's **MCP** client at the task service (so it can read/write artifacts and drive its own
state), and puts the workflow's state-machine overview in the system prompt.
- **The agent.** In the tmux pane, the launcher bootstraps and starts the task's selected
harness. It renders workflow **skills** and **operations** in that harness's format and
supplies the workflow's state-machine overview. Claude Code and Codex connect their **MCP**
clients to the task service; Pi and Outfitter use rendered **REST** instructions for task
operations and artifacts because they have no MCP client.
- **The task memo is pre-filled.** On a task's first spawn the runner pastes the task's
description into the agent's input box (unsent), so you see the ask waiting when you attach.
- **Turn and blocked.** As the agent and user hand the work back and forth, the task's **turn**
flips (`agent` ↔ `user`) via in-container hooks; a separate **blocked** marker is a deliberate
"waiting on something" flag the agent sets. Both are shown on the dashboard.
- **Auth.** The agent authenticates from a `CLAUDE_CODE_OAUTH_TOKEN` injected from the repo's
env-file — see [auth](auth.md).
- **Auth.** Credentials depend on the selected harness: Claude Code uses an OAuth token or
API key from the repo's env-file; Codex uses its API credentials or a mounted `auth.json`;
Pi and Outfitter use Pi-compatible provider credentials or a mounted credential directory.
These model-provider credentials are separate from the task-service capability token used
by MCP and REST calls. See [auth](auth.md) for setup and credential layouts.

## When it goes wrong

Expand Down Expand Up @@ -166,6 +183,6 @@ wedges the runner. After cleanup the task shows `–`: no container, nothing to

## See also

- [auth](auth.md) — giving a repo its `claude` token.
- [layers](layers.md) — how a task's image is composed (base → workflow → repo).
- [auth](auth.md) — setting up model-provider and task-service credentials.
- [layers](layers.md) — how a task's image is composed (base → harness → workflow → repo).
- [tasks](tasks.md) — slug, branch, clone, and provisioning as task concepts.
46 changes: 23 additions & 23 deletions docs/design/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,11 +115,12 @@ The execution-backend ABC realized as a separate **host process** (ADR 0008). Pe
- owns **one tmux session per task** — *not inside the agent container*; its panes exec into
the task's container(s), so a task with separate planner + implementer containers (M2)
shares one session;
- **builds/selects the composed image** for a task (base → workflow → repo, ADR 0005);
- **builds/selects the composed image** for a task (base → harness → workflow → repo, ADR 0005);
- **injects per-repo secrets** at launch (env vars + creds mount, ADR 0007);
- **provisions the worktree** once the slug is set — observing the task over its pull loop, it
builds the slug-named worktree on its host and repoints the container into it, then runs the
workflow's provisioning (ADR 0010);
- **prepares a writable per-task clone before spawn**, mounted at `/workspace`, with `origin`
pointed at the forge; once the slug is set, its pull loop branches that existing clone as
`panopticon/<slug>` and records the branch and clone path with the task service (ADR 0011).
The container keeps the same mount and working directory;
- **registers with the task service** and reports session status.

Concrete adapters behind the same interface, over time: local Docker+tmux (now), remote
Expand Down Expand Up @@ -237,9 +238,9 @@ tool/REST. A container **hook** detects an unset slug and instructs the agent to
decouples identity from naming and lets the agent (which has the most context) choose the
slug. *(Refines PARITY §14's "YYYY-MM-DD-<slug>" id.)*

The task's **worktree/branch are named from the slug**, so the slug must be set before the
core can create the (always-required) worktree — and therefore before any workflow
provisioning that needs the branch (e.g. the parity workflow's PR). See the §9 lifecycle.
The task's **branch is named from the slug**. Its writable clone already exists at spawn,
keyed by task id; setting the slug lets the runner branch that clone in place. Work needing
the named branch, such as opening a PR, follows that provisioning step. See the §9 lifecycle.

### 8.4 Secrets (ADR 0007)

Expand All @@ -252,31 +253,30 @@ mounted creds. Values never enter the DB, artifacts, or image layers.
1. **Idea → task.** A user captures an idea / promotes it via the dashboard (REST). The task
service creates a Task row (internal id, chosen workflow, `repo_id`), in the workflow's
initial state, with the turn assigned appropriately. No slug yet.
2. **Assign & spawn.** The task service assigns the task to a session service (runner). The
runner builds/selects the composed image (ADR 0005), injects the repo's secrets (ADR 0007),
creates the task container (sibling, DooD) and its tmux session, and starts the agent. The
container begins on a **read-only checkout** of the repo (there's no slug, hence no worktree,
yet) for the agent to plan against; step 5 upgrades it to the writable worktree.
2. **Claim & spawn.** A session service (runner) claims the task and creates its writable
per-task clone from the repository cache, pointing `origin` at the forge. It builds/selects
the composed image (base → harness → workflow → repo, ADR 0005), injects the repo's secrets
(ADR 0007), and starts the container on the host Docker daemon plus its host tmux session.
The clone is mounted read-write at `/workspace` from the start, before a slug exists.
3. **Connect & register.** The container entrypoint **connects to the task service and stays
connected** — registering that this container is working on this task (liveness). It loads
the workflow's in-container skills (from the active workflow) on top of the core operations.
4. **Slug.** A hook notices the slug is unset and instructs the agent to set one via a task
tool; the task service records it.
5. **Worktree & provisioning.** The **session service**, observing the task over its pull loop
(ADR 0010), sees the slug land and creates the task's **worktree/branch** on the host where
the container runs — always required, named from the slug, so it cannot precede step 4 — then
repoints the container's working path to it (the agent `cd`s in). The active workflow's
**provisioning** then runs (e.g. the parity workflow opens its PR); because it needs that
branch, PR creation is transitively gated on the slug too.
6. **Work.** The agent plans/implements; artifacts (plan/notes) flow over MCP; the agent runs
workflow skills (e.g. `babysit-ci`) that may use `gh`/git and call back over REST/MCP to
5. **Branch & provisioning.** The **session service**, observing the task over its pull loop
(ADR 0011), sees the slug land and creates `panopticon/<slug>` from the existing clone's
current HEAD. It records the branch and clone path with the task service. The container's
`/workspace` mount and working directory stay in place; `origin` was set during spawn.
The agent can then perform workflow work that needs the named branch, such as opening a PR.
6. **Work.** The agent plans/implements; artifacts (plan/notes) flow over MCP or REST,
depending on the harness; the agent runs workflow skills (e.g. `babysit-ci`) that may use `gh`/git and call back over REST/MCP to
request transitions. The task service deterministically enforces the workflow's state
machine and responsibility gating, flips the turn per the fg/bg classification, and appends history.
7. **Observe & steer.** The dashboard reflects state/turn/history live (REST); the user
presses `t` to drop into a task's tmux and back.
8. **Terminal states & cleanup.** On COMPLETE/DROPPED (or the workflow's terminals), cleanup
runs the core's agnostic teardown (tmux/worktree/branch) plus the workflow's specific
steps; the session service stops the container/session.
8. **Terminal states & cleanup.** On COMPLETE/DROPPED (or the workflow's terminals), the
session service stops the container/session, releases its claim, and removes the per-task
clone. Durable task artifacts remain in the task service's artifact store.

## 10. Deployment topology (ADR 0008)

Expand Down
Loading
Loading