Managed Agents ceiling program: findings, experiments, and provisioner extensions - #19
Open
devin-ai-integration[bot] wants to merge 44 commits into
Open
Managed Agents ceiling program: findings, experiments, and provisioner extensions#19devin-ai-integration[bot] wants to merge 44 commits into
devin-ai-integration[bot] wants to merge 44 commits into
Conversation
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
…iscovery Adds experiments/K drivers and evidence for mid-run steering, ask-and-block, deployment polling, the GitHub MCP review/CI loop, Skills as playbooks, memory write-back, and session forking, plus the findings file. Lands the skill-discovery paragraph and env-driven skills list in the agent definition. Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Workstream D: agent-as-code and configuration lifecycle
…omation H: Deployments and automation — native ceiling, evidence, and irreducible gaps
Workstream E: native Memory Store findings and probes
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
…t drivers Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
…ility Workstream C: runtime reliability, recovery, and the tool surface
Document Managed Agents control-plane semantics
Workstream F: built-in subagent findings + production roster
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
…oster Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
…tion Workstream K findings: Devin-parity interaction model (steering, ask-and-block, wake-on-event, PR/CI loop, Skills, self-improvement, forking)
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
…ector Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
…economics workstream I: observability and economics
…ntlet J: integrated gauntlet — native composition to a CI-green PR, and where it stops
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
B: long-horizon agent quality — findings and experiment harness
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
… re-dispatch no) Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
J follow-up: the managed loop is self-diagnosing, not self-healing
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Contributor
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
…ade remote development for agents? Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
…tten agents Co-Authored-By: Humza Rabbani <humza.rabbani@cognition.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Managed Agents ceiling program: all ten workstreams' findings, experiments, provisioner extensions — and the final report.
context/findings/REPORT.md(replaces the earlierONE-PAGER.md): the comprehensive capability-ceiling report — what Managed Agents are, architecture (loop vs self-hosted sandbox, three durability planes), a per-primitive can-build table with the measured numbers, the class-D register (self-healing, event ingress, fork/checkpoint, knowledge retrieval, model routing, fleet ops), the lifecycle-moat analysis, custom-build audit (clean — no class-D replacements built), the untested register, and the conclusion.context/findings/{A..K}*.md+experiments/{A..K}/: per-workstream evidence (session/event/request IDs, harnesses, raw JSON).context/findings/SYNTHESIS.md: parity table, provenance ledger, class-D register.packages/provision/: class-B provisioner extensions only —drift.ts(client-side desired-vs-live comparison splitting real drift from server-added fields), subagent/Skill ID plumbing, discovery prompt paragraphs.Core finding: the loop is self-diagnosing, not self-healing — dead-worker detection is native (~360 s lease expiry) and narrow transient errors retry, but nothing re-dispatches abandoned work (
work_id == session_id), billing-exhausted sessions stay inert, and there is no event ingress, fork/checkpoint, knowledge injection, or model routing. Execution is near Devin parity; the lifecycle/control plane is the ceiling.Verification:
pnpm verify(25 tests), ruff format/check, mypy, pytest (32 tests) all green.Link to Devin session: https://app.devin.ai/sessions/f50c9cdc5d874afdae743036b0e39862
Open in Devin Desktop: https://app.devin.ai/desktop/session/f50c9cdc5d874afdae743036b0e39862?variant=devin
Requested by: @hrabbani
Devin Review