Skip to content

Managed Agents ceiling program: findings, experiments, and provisioner extensions - #19

Open
devin-ai-integration[bot] wants to merge 44 commits into
mainfrom
swarm/integration
Open

Managed Agents ceiling program: findings, experiments, and provisioner extensions#19
devin-ai-integration[bot] wants to merge 44 commits into
mainfrom
swarm/integration

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Summary

Managed Agents ceiling program: all ten workstreams' findings, experiments, provisioner extensions — and the final report.

  • context/findings/REPORT.md (replaces the earlier ONE-PAGER.md): the comprehensive capability-ceiling report — what Managed Agents are, architecture (loop vs self-hosted sandbox, three durability planes), a per-primitive can-build table with the measured numbers, the class-D register (self-healing, event ingress, fork/checkpoint, knowledge retrieval, model routing, fleet ops), the lifecycle-moat analysis, custom-build audit (clean — no class-D replacements built), the untested register, and the conclusion.
  • context/findings/{A..K}*.md + experiments/{A..K}/: per-workstream evidence (session/event/request IDs, harnesses, raw JSON).
  • context/findings/SYNTHESIS.md: parity table, provenance ledger, class-D register.
  • packages/provision/: class-B provisioner extensions only — drift.ts (client-side desired-vs-live comparison splitting real drift from server-added fields), subagent/Skill ID plumbing, discovery prompt paragraphs.

Core finding: the loop is self-diagnosing, not self-healing — dead-worker detection is native (~360 s lease expiry) and narrow transient errors retry, but nothing re-dispatches abandoned work (work_id == session_id), billing-exhausted sessions stay inert, and there is no event ingress, fork/checkpoint, knowledge injection, or model routing. Execution is near Devin parity; the lifecycle/control plane is the ceiling.

Verification: pnpm verify (25 tests), ruff format/check, mypy, pytest (32 tests) all green.

Link to Devin session: https://app.devin.ai/sessions/f50c9cdc5d874afdae743036b0e39862
Open in Devin Desktop: https://app.devin.ai/desktop/session/f50c9cdc5d874afdae743036b0e39862?variant=devin
Requested by: @hrabbani


Devin Review

Status Commit
⚪ Not started

Run Devin Review

Devin Review (Staging)

devin-ai-integration Bot and others added 30 commits August 28, 2026 02:06
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
…iscovery

Adds experiments/K drivers and evidence for mid-run steering, ask-and-block, deployment polling, the GitHub MCP review/CI loop, Skills as playbooks, memory write-back, and session forking, plus the findings file. Lands the skill-discovery paragraph and env-driven skills list in the agent definition.

Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Workstream D: agent-as-code and configuration lifecycle
…omation

H: Deployments and automation — native ceiling, evidence, and irreducible gaps
Workstream E: native Memory Store findings and probes
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
…t drivers

Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
…ility

Workstream C: runtime reliability, recovery, and the tool surface
Document Managed Agents control-plane semantics
Workstream F: built-in subagent findings + production roster
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
…oster

Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
…tion

Workstream K findings: Devin-parity interaction model (steering, ask-and-block, wake-on-event, PR/CI loop, Skills, self-improvement, forking)
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
…ector

Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
devin-ai-integration Bot and others added 8 commits August 28, 2026 05:02
…economics

workstream I: observability and economics
…ntlet

J: integrated gauntlet — native composition to a CI-green PR, and where it stops
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
B: long-horizon agent quality — findings and experiment harness
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
… re-dispatch no)

Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
J follow-up: the managed loop is self-diagnosing, not self-healing
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

devin-ai-integration Bot and others added 6 commits August 28, 2026 06:09
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
…ade remote development for agents?

Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
…tten agents

Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant