From 01d62efcd6a5111c6034274f08ab9a5fdac8a40b Mon Sep 17 00:00:00 2001 From: Lakshman Patel Date: Thu, 16 Jul 2026 08:59:24 +0530 Subject: [PATCH 1/2] docs: add comprehensive Grok-to-hawk port plan documents These long-horizon planning documents provide: - FULL-GROK-ECO-TO-HAWK-ECO-PORT-PLAN.md: Complete crate-by-crate port roadmap with detailed implementation phases and feature mapping - GROK-CLASS-CAPABILITY-LONG-HORIZON-PLAN.md: 12-18 month timeline focused on agent control plane quality improvements Both documents outline the roadmap for bringing Hawk to Grok-class agent control plane quality while preserving the multi-repo Go platform architecture. --- .../FULL-GROK-ECO-TO-HAWK-ECO-PORT-PLAN.md | 697 ++++++++++++ ...GROK-CLASS-CAPABILITY-LONG-HORIZON-PLAN.md | 994 ++++++++++++++++++ 2 files changed, 1691 insertions(+) create mode 100644 docs/plans/FULL-GROK-ECO-TO-HAWK-ECO-PORT-PLAN.md create mode 100644 docs/plans/GROK-CLASS-CAPABILITY-LONG-HORIZON-PLAN.md diff --git a/docs/plans/FULL-GROK-ECO-TO-HAWK-ECO-PORT-PLAN.md b/docs/plans/FULL-GROK-ECO-TO-HAWK-ECO-PORT-PLAN.md new file mode 100644 index 00000000..1afdfe96 --- /dev/null +++ b/docs/plans/FULL-GROK-ECO-TO-HAWK-ECO-PORT-PLAN.md @@ -0,0 +1,697 @@ +# Full Port Plan: grok-eco → hawk-eco (Go reimplementation) + +**Status:** Master long-term plan +**Date:** 2026-07-16 +**Meaning of “port”:** Reimplement **every Grok Build capability** in idiomatic **Go** across hawk-eco repos. +**Not meaning:** Copy Rust crates, depend on Grok binaries, or collapse hawk into a Rust monorepo. + +**Source tree:** `grok-eco/grok-build` (~1.35M LOC Rust, 1 product monorepo) +**Target tree:** `hawk-eco/*` (multi-repo Go platform + cloud/TS/Python) + +**Related shorter plan:** `GROK-CLASS-CAPABILITY-LONG-HORIZON-PLAN.md` (control-plane focus). +**This document:** **complete** crate/tool/doc/slash inventory, including small items, with owner repo + status + phase. + +--- + +## 0. Port principles + +1. **Behavior parity over code structure** — same user-visible contracts, not same file tree. +2. **Go multi-repo stays** — map Grok crates into hawk / engines / contracts / cloud / skills. +3. **Prefer wire-first** — Hawk already has partial types; complete wiring before greenfield. +4. **Privacy-first defaults** — Grok Mixpanel/Sentry defaults become **opt-in** OTEL/privacy-safe telemetry. +5. **Multi-provider stays** — Grok sampler/auth maps to **eyrie**, not a single-vendor clone. +6. **Memory stays yaad** — Grok markdown memory maps to **yaad APIs + UX**, not a second store. +7. **Tokens stay tok** — Grok `xai-token-estimation` is obsolete relative to tok. +8. **Track everything** — even 70-line crates appear in the matrix (Done / Partial / Port / Skip / N/A). + +### Status legend + +| Status | Meaning | +|--------|---------| +| **Done** | Hawk-eco already has equivalent or better | +| **Partial** | Exists but incomplete vs Grok | +| **Port** | Must reimplement in Go | +| **Skip** | Intentionally not porting (vendor/stack conflict) | +| **N/A** | Grok internal leaf; absorbed into larger port | + +### Effort bands (one senior Go eng) + +| Band | Meaning | +|------|---------| +| XS | < 1 week | +| S | 1–2 weeks | +| M | 3–6 weeks | +| L | 2–3 months | +| XL | 3–6 months | +| XXL | 6–12+ months (TUI/parity programs) | + +### Suggested calendar (full port) + +| Horizon | Scope | +|---------|--------| +| **Y0 (0–6 mo)** | Control plane + trust + spawn + hooks (critical path) | +| **Y0–1 (6–12 mo)** | Plugins/marketplace, tasks/monitor, ACP, user-guide | +| **Y1–2** | Workspace depth, foreign sessions, hunks, mermaid, PTY, enterprise policy | +| **Y2–3** | Computer-hub class remote tools (if product needs), full slash parity, IDE polish | +| **Y3–5** | Hardening, multi-editor, fleet/agents at scale, continuous Grok-diff audits | + +Full “every small thing green” is a **multi-year product program**, not a 10-year rewrite of Rust. +Critical path remains **months**; **complete parity** including TUI polish and enterprise is **~2–4 years** of continuous work depending on headcount. + +--- + +## 1. Repo mapping (where Grok monorepo goes) + +```text +grok-eco/grok-build (one Rust workspace) + │ + ├─► hawk product CLI/TUI/engine/tools/hooks/plugins/ACP + ├─► hawk-core-contracts pure DTOs (tool/spawn/hooks/policy) + ├─► eyrie LLM routing/stream/retry/catalog/auth credentials + ├─► yaad memory graph + dream UX + ├─► tok compression / secrets / token cost + ├─► trace session capture/import/replay + ├─► sight / inspect review & live audit (no Grok 1:1) + ├─► hawk-mcpkit MCP server helpers + ├─► hawk-community-skills skills + marketplace content + ├─► hawk-cloud managed policy / tenancy / usage + ├─► hawk-sdk-go/python ACP/daemon clients + └─► graycode-core web dashboard only (optional) +``` + +**There is only one Grok product repo (`grok-build`).** +“Port all grok-eco repos” = port **all crates/capabilities inside grok-build** into the **appropriate hawk-eco repos**. + +--- + +## 2. Master crate → hawk-eco matrix + +### 2.1 Product / TUI / shell (codegen) + +| Grok crate | ~LOC | Capability | Target repo | Status | Effort | Notes | +|------------|-----:|------------|-------------|--------|--------|-------| +| `xai-grok-pager` | 415k | Full TUI | hawk `cmd/` | Partial | XXL | Bubble Tea already; parity of panes/modals/slash is multi-year | +| `xai-grok-pager-render` | 35k | Render pipeline | hawk | Partial | L | Streaming render, blocks, media | +| `xai-grok-pager-minimal` | 5.6k | `--minimal` native scrollback | hawk | Partial | M | `hawk --repl` / print modes exist | +| `xai-grok-pager-bin` | 2.9k | Binary composition root | hawk `cmd/hawk` | Done | — | | +| `xai-grok-pager-pty-harness` | 10k | PTY test harness | hawk tests | Partial | M | Test infra | +| `xai-grok-shell` | 336k | Agent runtime host | hawk `internal/engine` | Partial | XXL | Largest runtime port | +| `xai-grok-shell-base` | 2.7k | Shared shell foundation | hawk | Partial | S | Absorb | +| `xai-grok-shell-session-support` | 1.5k | Session support extract | hawk `session` | Partial | S | Absorb | +| `xai-grok-tools` | 112k | Tool implementations | hawk `internal/tool` | Partial | XL | See tool matrix §3 | +| `xai-grok-tools-api` | 0.7k | Tool API / slash wording | hawk + contracts | Partial | S | | +| `xai-grok-workspace` | 77k | FS/VCS/permissions/hub | hawk + sandbox + session | Partial | XL | | +| `xai-grok-workspace-types` | 9.2k | Workspace wire types | contracts | Partial | M | | +| `xai-grok-workspace-client` | 0.8k | Workspace RPC client | hawk (if hub) | Port | M | Only if computer-hub ported | +| `xai-grok-agent` | 21k | Agent defs + system prompt | hawk agent/personas | Partial | L | | +| `xai-grok-subagent-resolution` | 2.6k | Capability/isolation resolve | hawk + contracts | Port | M | Critical | +| `xai-chat-state` | 13k | Chat state actor | hawk session/engine | Partial | L | | +| `xai-grok-markdown` | 22k | Streaming MD TUI | hawk markdown | Partial | L | | +| `xai-grok-markdown-core` | 1.1k | Headless MD analysis | hawk | Partial | S | | +| `xai-ratatui-textarea` | 12k | Input widget | hawk (Bubble Tea) | N/A | — | Different UI stack | +| `xai-ratatui-inline` | 3.7k | Inline render | hawk | N/A | — | UI stack | +| `xai-fast-worktree` | 19k | CoW worktree speed | hawk worktree | Port | L | Perf enhancement | +| `xai-file-utils` | 15k | Event tracking / upload | hawk observability | Partial | M | Privacy opt-in | +| `xai-grok-telemetry` | 14k | Events + OTEL + Mixpanel + Sentry | hawk observability | Partial | L | **Skip Mixpanel default**; keep OTEL | +| `xai-hunk-tracker` | 13k | Agent vs external hunks | hawk | Port | L | | +| `xai-grok-sampling-types` | 13k | Chat API types | **eyrie** | Done* | — | Different shape; eyrie owns | +| `xai-grok-sampler` | 11k | HTTP stream + retry | **eyrie** | Done* | — | Do not replace eyrie | +| `xai-grok-update` | 11k | Auto-update | hawk | Partial | M | | +| `xai-grok-mcp` | 10k | MCP client (oauth, wire) | hawk `mcp` | Partial | L | | +| `xai-codebase-graph` | 9.7k | Tree-sitter graph | hawk codegraph/repomap | Partial | L | | +| `xai-grok-memory` | 9.7k | Cross-session memory | **yaad** | Done* | M | Port UX only | +| `xai-grok-hooks` | 8.3k | File/HTTP hooks | hawk hooks | Port | L | | +| `xai-fsnotify` | 6.7k | FS events | hawk (fsnotify) | Partial | S | | +| `xai-grok-config` | 6k | Config layers + managed | hawk config + cloud | Port | L | | +| `xai-grok-config-types` | 2.7k | Config DTOs | contracts/config | Partial | S | | +| `xai-grok-plugin-marketplace` | 5.3k | Marketplace | hawk + community-skills | Port | L | | +| `xai-grok-shared` | 5.2k | Shared utils | hawk | N/A | — | Absorb | +| `xai-grok-test-support` | 4.6k | Test harness | hawk testutil | Partial | M | | +| `xai-grok-sandbox` | 3.9k | OS sandbox profiles | hawk sandbox | Port | L | | +| `xai-grok-voice` | 2.7k | Streaming STT | hawk | Partial | M | whisper path exists | +| `xai-acp-lib` | 2.3k | ACP protocol | hawk acp | Port | L | | +| `xai-grok-mermaid` | 2.2k | Mermaid→PNG | hawk | Port | M | | +| `xai-crash-handler` | 1.9k | Crash + startup detect | hawk | Port | S | | +| `ptyctl` | 2.3k | Headless PTY control | hawk | Port | M | | +| `ptyctl-cli` | 0.8k | PTY CLI | hawk tests/tools | Optional | S | | +| `xai-tty-utils` | 1.2k | TTY-safe spawn | hawk | Port | S | | +| `xai-hooks-plugins-types` | 1.2k | Hooks/plugins ACP DTOs | contracts | Port | S | | +| `xai-sqlite-journal` | 0.8k | SQLite journal mode | hawk/yaad/tok | Partial | XS | | +| `xai-system-power` | 0.7k | Sleep/wake notify | hawk sleep_prevent | Partial | S | | +| `xai-grok-http` | 0.6k | Shared HTTP client | hawk/netutil | Partial | XS | | +| `xai-agent-lifecycle` | 0.6k | Lifecycle hooks data | hawk hooks/engine | Partial | S | | +| `xai-gix-status` | 0.6k | Fast git status | hawk git tools | Partial | S | | +| `xai-grok-paths` | 0.6k | AbsPath types | hawk | N/A | XS | Go path.Clean enough | +| `xai-grok-secrets` | 0.6k | Secrets helpers | tok + eyrie | Partial | S | | +| `xai-grok-announcements` | 0.4k | Release announcements | hawk tips/notify | Port | S | | +| `xai-grok-auth` | 0.4k | Auth seam | eyrie + hawk auth | Partial | M | Browser OAuth optional | +| `xai-token-estimation` | 0.3k | Bytes/4 heuristic | **tok** | Skip | — | tok superior | +| `xai-tracing-macros` | 0.2k | Log macros | hawk observability | N/A | XS | | +| `xai-grok-env` | 0.2k | Backend env presets | eyrie/hawk | Partial | XS | | +| `xai-prompt-queue` | 0.2k | Prompt queue types | hawk | Port | S | | +| `xai-mixpanel` | 0.1k | Mixpanel client | — | **Skip** | — | Privacy; OTEL opt-in | +| `xai-grok-version` | 0.1k | Version | hawk VERSION | Done | — | | +| `xai-grok-models` | 0.1k | Default model IDs | eyrie catalog | Done* | — | | + +### 2.2 Common crates (tool protocol / hub / compaction) + +| Grok crate | ~LOC | Capability | Target | Status | Effort | Notes | +|------------|-----:|------------|--------|--------|--------|-------| +| `xai-computer-hub-sdk` | 14k | Remote tool server SDK | new or hawk | Port (Y2+) | XL | Only if multi-host tools wanted | +| `xai-computer-hub-core` | 4.2k | Transport/registry | same | Port (Y2+) | L | | +| `xai-computer-hub-mcp-adapter` | 1k | MCP into hub | hawk-mcpkit | Port (Y2+) | M | | +| `xai-tool-protocol` | 6.6k | Wire protocol | contracts + hawk | Port (Y2) | L | | +| `xai-tool-runtime` | 5.4k | Tool trait runtime | hawk tool | Partial | L | | +| `xai-tool-types` | 3.6k | Spawn/task types | **contracts** | Port | M | **P0** | +| `xai-grok-compaction` | 6.8k | Compaction engine | hawk engine + tok | Partial | L | | +| `xai-circuit-breaker` | 2.2k | HTTP breaker | eyrie/resilience | Partial | S | | +| `xai-tracing` | 0.8k | Tracing | hawk OTEL | Partial | S | | +| `xai-test-utils` | 0.4k | Hermetic git tests | hawk testutil | Partial | S | | +| `xai-interjection-core` | 0.3k | Interjection messaging | hawk | Port | S | Mid-turn user inject | +| `xai-proto-build` | — | Build tooling | — | Skip | — | Rust build | + +\*Done* = domain covered by hawk engine with different API. + +--- + +## 3. Tool-by-tool port matrix (Grok tools) + +### 3.1 Grok Build native tools + +| Grok tool | Hawk equivalent | Status | Port work | +|-----------|-----------------|--------|-----------| +| `bash` / `run_terminal_command` | `Bash` | Partial | background flag, timeout, safe-bash, kill | +| `read_file` | `Read` | Partial | media/PDF page ranges parity | +| `search_replace` | `Edit` | Partial | hashline / strict range policy optional | +| `list_dir` | `LS` | Done | | +| `grep` | `Grep` | Done | | +| `web_search` | `WebSearch` | Done | | +| `web_fetch` | `WebFetch` / AgenticFetch | Partial | SSRF hardening audit | +| `todo` | `TodoWrite` | Done | | +| `task` (spawn) | `Agent` | **Port** | full schema §A | +| `task_output` / wait | bg managers | **Port** | unify + tools | +| `kill_task` | partial | **Port** | | +| `monitor` | missing | **Port** | line stream + rate limit | +| `scheduler` | `Cron*` | Partial | `/loop` UX | +| `ask_user_question` | `AskUserQuestion` | Partial | multi-option UI | +| `enter_plan_mode` | `/spec` + plan | Partial | unify | +| `exit_plan_mode` | ApproveImplementation | Partial | | +| `update_goal` | todos/mission | Partial | goal tool | +| `lsp` | `LSP` / MCPLSP | Partial | | +| `image_gen` | missing | Port / Optional | provider-backed | +| `image_edit` | missing | Port / Optional | | +| `video_gen` | missing | Port / Optional | xAI-specific → multi-provider later | +| skills invoke | `Skill` | Partial | multi-harness discovery | +| MCP meta tools | MCP tools | Partial | search_tool / use_tool patterns | + +### 3.2 Codex / OpenCode compatibility tool ports (in Grok tree) + +Grok vendors codex/opencode tool implementations for compatibility profiles. + +| Compat surface | Hawk | Action | +|----------------|------|--------| +| Codex apply_patch / read / list / grep | hawk tools | Optional compat profile | +| OpenCode read/write/edit/bash/glob/grep/todo/skill | hawk | Optional `compat.opencode` | +| Claude import | missing | **Port** into trace/hawk | +| Cursor skills scan | missing | **Port** | + +--- + +## 4. Slash commands & UX (pager) + +Grok slash modules (port checklist → hawk `/` commands or CLI): + +| Grok slash | Status in hawk | Action | +|------------|----------------|--------| +| help | Partial | Port completeness | +| model / effort | Partial | Port effort levels | +| theme / vim_mode | Partial | Port gaps | +| compact | Partial | | +| config_agents / personas | Partial | Port agents modal parity | +| plugins / mcps / marketplace | Partial | Port | +| plan / view_plan | Partial | Align with /spec | +| loop | Partial (cron) | Port `/loop` | +| remember / memory | Partial (yaad) | Port UX | +| voice | Partial | Streaming STT | +| dashboard | Partial (HUD) | Port | +| usage | Partial | Port | +| import_claude | Missing | Port | +| fork / rewind / resume / rename | Partial | Port gaps | +| queue | Missing | Port prompt queue | +| btw (interjection) | Missing | Port | +| always_approve / auto | Partial | Map autonomy | +| login/logout | N/A keychain | Optional OAuth | +| share | Partial | | +| export / transcript | Partial | | +| announcements / release_notes | Missing | Port | +| feedback | Partial | | +| privacy | Partial | Port | +| terminal_setup | Missing | Port | +| timestamps | Partial | | +| history / find / copy | Partial | | +| imagine / imagine_video | Missing | Optional media tools | +| session_info / recap | Partial | | +| home / cd / new / exit | Partial | | +| multiline / mouse / screen_mode | Partial | TUI polish | +| scroll_debug / gboom | Dev only | Optional | + +--- + +## 5. User-guide docs (port as hawk user-guide) + +| Grok doc | Port to | +|----------|---------| +| 01–05 essentials | `hawk/docs/user-guide/` | +| 06 theming | same | +| 07 MCP | same + mcp-servers.md merge | +| 08 skills | same | +| 09 plugins | same | +| 10 hooks | same | +| 11 custom models | eyrie + hawk | +| 12 project rules AGENTS.md | exists Partial | +| 13 memory | yaad UX | +| 14 headless | CLI flags doc | +| 15 agent/ACP | acp doc | +| 16 subagents | spawn doc | +| 17 sessions | session doc | +| 18 sandbox | sandbox.toml doc | +| 19 plan mode | /spec doc | +| 20 background tasks | taskruntime doc | +| 21 terminal support | same | +| 22 permissions | autonomy+pipeline doc | +| 23 dashboard | same | +| 24 monitoring usage | cost/usage + cloud | + +**Effort:** M continuous across phases (write as features land). + +--- + +## 6. Small systems often missed (explicit) + +These are “small” crates/modules but required for **full** port claims: + +| Item | Grok home | Hawk action | Effort | +|------|-----------|-------------|--------| +| Folder trust | `workspace/folder_trust.rs` | **Port** `internal/trust` | M | +| envrc / direnv load | `workspace/envrc.rs` | **Port** | S | +| Foreign sessions (Claude/Codex) | `foreign_sessions/` | **Port** via trace | L | +| Managed config sync | `shell/managed_config` | **Port** via hawk-cloud | L | +| Claude import | `shell/claude_import*` | **Port** | M | +| Prompt queue | `xai-prompt-queue` | **Port** | S | +| Interjection (“btw”) | `xai-interjection-core` | **Port** | S | +| Announcements | `xai-grok-announcements` | **Port** | S | +| Crash handler | `xai-crash-handler` | **Port** | S | +| Auto-update | `xai-grok-update` | Complete Partial | M | +| Mermaid PNG | `xai-grok-mermaid` | **Port** (Go/SVG or subprocess) | M | +| Voice streaming | `xai-grok-voice` | Upgrade Partial | M | +| PTY control | `ptyctl` | **Port** for harness/tools | M | +| TTY-safe child spawn | `xai-tty-utils` | **Port** | S | +| Sleep/wake on auth | `xai-system-power` | Complete Partial | S | +| Hashline edit protocol | tools hashline | Optional Port | M | +| Leader/cluster multi-agent UI | pager leader_cluster | Optional (mission covers) | L | +| Relay / remote agent WS | shell relay/remote | Port for ACP serve | L | +| Upload / share pipeline | shell upload | Partial + privacy | M | +| Heap profile | shell heap_profile | Optional dev | S | +| Diag server | workspace diag_server | Optional | S | +| Preview supervisor | workspace | Optional | M | +| Hub auth/channel/server | workspace hub* | Y2 computer-hub | XL | +| Campaigns / version_overrides config | config | Port if enterprise | M | +| MDM macOS managed prefs | config macos_managed | Optional enterprise | M | +| Signed requirements fail-closed | config signed_policy | Port with cloud | L | +| Skills ignore/disabled/paths | skills config | Port | S | +| Compat cursor/claude toggles | config compat | Port | S | +| Safe-bash allowlist | permissions docs | Port | M | +| dontAsk / acceptEdits modes | permissions | Port | M | +| Dashboard view | pager views/dashboard | Port polish | M | +| Notifications system | pager notifications | Partial | S | +| Tips | pager tips | Partial | XS | +| Client identity | pager | Partial | XS | +| Inline media ffmpeg | pager | Optional | M | +| Hyperlink routing | pager | Port | S | +| Config TOML live edit | pager | Port | S | +| Goal classifier | shell session | Port | S | +| Trace classifier | shell | map to hawk trace | S | +| Repo changes tracking | shell session | Partial | M | +| Active sessions multi | shell | Partial | M | +| MCP doctor | shell mcp_doctor | Port | S | +| Bundle / extensions suggest | shell extensions | Partial | M | +| Instrumentation / inspect modules | shell | map observability | S | + +--- + +## 7. Year-by-year program (full port) + +### Year 0 — Foundation (months 0–12) + +**Theme:** Model can do what Grok’s agent control plane does; secure by default. + +| Quarter | Deliverables | Repos | +|---------|--------------|-------| +| **Q1** | Contracts spawn DTOs; wire explore/plan/general; stop hardcoded explore; explore bash hard gate; unify taskruntime; structured SpawnResult | contracts, hawk | +| **Q2** | sandbox.toml; folder trust; safe-bash; permission pipeline hooks-first; PreToolUse deny | hawk | +| **Q3** | File+HTTP hooks; vendor aliases; multi-harness skills; multi-component plugins; marketplace MVP | hawk, community-skills | +| **Q4** | Monitor/Wait/Kill; /loop; structured AskUser; plan/spec alignment; user-guide 01–12; crash handler; announcements; prompt queue; interjection | hawk | + +**Exit Year 0:** Contributor-ready “Grok-class agent controls” without claiming full TUI parity. + +### Year 1 — Integration & product depth (months 12–24) + +| Quarter | Deliverables | +|---------|--------------| +| **Q5** | ACP session/load/resume; richer updates; OpenAPI; sdk-go/python fields | +| **Q6** | Managed policy (signed) hawk-cloud → hawk; IT tier; config layer order | +| **Q7** | Foreign session import (Claude/Codex); envrc; hunk tracker; fast worktree; mermaid render | +| **Q8** | Voice streaming upgrade; update checker parity; MCP doctor; slash parity batch 1; user-guide 13–24 | + +### Year 2 — Workspace / hub / polish (months 24–36) + +| Focus | Items | +|-------|--------| +| Computer hub (optional product decision) | tool-protocol, hub server/client, remote tools, MCP adapter | +| TUI parity program | dashboard, modals, mouse, minimal mode, notifications polish | +| Compat | Claude/Cursor/OpenCode/Codex profiles | +| Media tools | image_gen/edit/video if multi-provider available | +| Relay/WS agent serve | remote IDE/web | +| Performance | CoW worktrees, git status, compaction quality | + +### Year 3–5 — Continuous parity + +- Diff audit bot: scan Grok public releases → open hawk issues +- Enterprise: MDM, SCIM (cloud), fleet policies +- Full editor suite (VS Code/Zed via ACP) +- Hardening, fuzz, SLSA releases +- Deprecate temporary adapters + +--- + +## 8. Phase implementation packs (engineering) + +Each pack is a shippable program of work with Definition of Done. + +### PACK-00: Inventory & gates (1–2 weeks) + +- [ ] Freeze this matrix in repo +- [ ] ADR: “Full behavioral port, Go multi-repo” +- [ ] Feature-flag registry +- [ ] CI job: `docs/plans` status table lint (optional later) + +### PACK-01: Contracts (2–3 weeks) — **start here** + +**Repo:** `hawk-core-contracts` + +- [ ] `agent` package: CapabilityMode, IsolationMode, SubagentType, SpawnRequest, SpawnResult +- [ ] Hook event name constants +- [ ] Tests for aliases and validation +- [ ] Version release + +**DoD:** hawk can import types; no engine deps. + +### PACK-02: Spawn control plane (6–8 weeks) + +**Repo:** `hawk` + +- [ ] Change `AgentSpawnFn` to SpawnRequest/Result +- [ ] Agent tool full schema (type, capability, isolation, resume, cwd, model, thoroughness, description, background) +- [ ] Wire plan/general (not only explore) +- [ ] Worktree isolation lifecycle +- [ ] Transcript resume_from +- [ ] MultiAgent typed tasks +- [ ] Explore bash AST allowlist +- [ ] Unify BackgroundAgentManager / BackgroundRunner / BackgroundAgentPool → `taskruntime` +- [ ] E2E + race tests + +**DoD:** Model can spawn plan agent that cannot Write; resume works. + +### PACK-03: Trust & sandbox (5–7 weeks) + +- [ ] `internal/trust` folder trust store + CLI +- [ ] Gate project hooks/MCP/LSP/plugins +- [ ] sandbox.toml loader (additive project merge) +- [ ] Profiles: off, workspace, read-only, strict, devbox, custom +- [ ] Deny globs fail-closed +- [ ] OS matrix tests + +### PACK-04: Hooks complete (4 weeks) + +- [ ] Event set expansion +- [ ] Claude/Cursor aliases +- [ ] File discovery paths +- [ ] HTTP runner +- [ ] Inject into PermissionEngine before autonomy +- [ ] Plugin env HAWK_PLUGIN_ROOT/DATA + +### PACK-05: Extensions (8–10 weeks) + +- [ ] Multi-component plugin layout +- [ ] Discovery scopes +- [ ] Marketplace multi-source install +- [ ] community-skills index schema for plugins +- [ ] Multi-harness skill scan +- [ ] Audit hardening + +### PACK-06: Task tools & loop (4 weeks) + +- [ ] GetTaskOutput, WaitTasks, KillTask tools +- [ ] Monitor tool (line processor, rate limit, auto-kill) +- [ ] `/loop` over cron with caps/expiry +- [ ] Shell background demote hotkey (optional TUI) + +### PACK-07: Plan & AskUser (3 weeks) + +- [ ] Structured AskUserQuestion UI +- [ ] enter/exit plan alignment with /spec +- [ ] update_goal tool if needed + +### PACK-08: Small UX systems batch (4 weeks) + +- [ ] Crash handler +- [ ] Announcements +- [ ] Prompt queue +- [ ] Interjection (btw) +- [ ] MCP doctor +- [ ] Terminal setup slash +- [ ] Privacy / usage docs + +### PACK-09: ACP + SDKs (7 weeks) + +- [ ] session/load, richer updates +- [ ] OpenAPI fields +- [ ] sdk-go + sdk-python +- [ ] Optional WS serve/relay + +### PACK-10: Enterprise policy (6 weeks) + +- [ ] Signed managed policy schema (cloud) +- [ ] hawk apply layers +- [ ] fail-open vs fail-closed +- [ ] IT non-excludable tier end-to-end + +### PACK-11: Session ecosystem (6 weeks) + +- [ ] Foreign sessions import (trace) +- [ ] Claude import CLI +- [ ] envrc load +- [ ] Hunk attribution tracker +- [ ] Fast worktree (CoW where available) + +### PACK-12: Media & mermaid (4–6 weeks) + +- [ ] Mermaid render path +- [ ] Image gen/edit if providers allow +- [ ] Video optional +- [ ] Inline media handling + +### PACK-13: Voice & power (3 weeks) + +- [ ] Streaming STT pipeline +- [ ] Sleep/wake aware auth refresh + +### PACK-14: TUI parity program (ongoing XL) + +Track Grok pager features against hawk Bubble Tea: + +- [ ] Dashboard view parity +- [ ] Agents/personas modal +- [ ] Plugins/marketplace modal +- [ ] Minimal mode +- [ ] Mouse/scroll polish +- [ ] Notifications +- [ ] Theme completeness +- [ ] Slash completeness audit (table §4) + +### PACK-15: Computer hub (optional Y2 XL) + +Only if product requires remote tool hosts: + +- [ ] tool-protocol Go +- [ ] hub server/client +- [ ] MCP adapter +- [ ] workspace RPC + +### PACK-16: Docs full port (parallel) + +- [ ] `docs/user-guide/01` … `24` +- [ ] Architecture notes stay separate + +### PACK-17: Continuous parity (forever) + +- [ ] Quarterly grok-build public tree diff +- [ ] Issue auto-filing for new Grok features +- [ ] Security review cadence + +--- + +## 9. Per hawk-eco repo full ownership checklist + +### `hawk` (majority) + +Port/finish: spawn, tools, sandbox, trust, hooks, plugins, marketplace client, TUI, ACP, headless, slash, taskruntime, hunks, mermaid, voice, crash, announcements, queue, interjection, plan, ask user, update, PTY harness, envrc, permissions pipeline, user-guide. + +### `hawk-core-contracts` + +Port: spawn/capability/isolation types, hook events, optional sandbox policy DTO, tool protocol types (later). + +### `eyrie` + +Absorb: sampling/stream/retry/circuit-breaker lessons; catalog already ahead; managed deployment hooks; **do not** become Grok-only auth. + +### `yaad` + +Port memory **UX**: dream, toggle priority, flush; keep graph store. + +### `tok` + +Keep; optionally align display token estimates; secrets scanning already stronger. + +### `trace` + +Foreign session import; share/export; session indexing parity with Grok session features that are capture-oriented. + +### `sight` / `inspect` + +No direct Grok crates; keep peer engines; ensure hawk composition matches Grok “review/inspect” product moments if any. + +### `hawk-mcpkit` + +Liveness, shared OAuth helpers, hub MCP adapter if hub lands. + +### `hawk-community-skills` + +Marketplace content + multi-component packages + validation. + +### `hawk-cloud` + +Managed policy, usage ledger (exists), entitlements, SSO/SCIM (exists primitives). + +### `hawk-sdk-go` / `hawk-sdk-python` + +Spawn options, plugins meta, permission/sandbox, stream formats. + +### `graycode-core` + +Dashboard/usage/marketplace web UI only. + +--- + +## 10. Explicit Skip list (still “accounted for”) + +| Grok thing | Why skip / transform | +|------------|----------------------| +| Entire Rust monorepo layout | Wrong language/org model | +| Mixpanel default | Privacy | +| Sentry hard-dependency | Opt-in error reporting | +| SpaceXAI-only browser auth as sole path | Multi-provider + keychain | +| `xai-token-estimation` | tok wins | +| Replace yaad with markdown memory files | Graph better | +| Replace eyrie with xai-grok-sampler | Multi-provider better | +| ratatui widgets | Bubble Tea stack | +| `xai-proto-build` | Rust build | +| Closed contribution policy | Hawk is open | + +Skipping is **documented completion**, not unfinished work. + +--- + +## 11. Tracking spreadsheet schema (use in GitHub Projects) + +Columns: + +1. ID (e.g. PACK-02-07) +2. Grok source (crate/path) +3. Capability name +4. Target repo +5. Target package +6. Status (Done/Partial/Port/Skip) +7. Pack +8. Effort +9. Priority (P0–P3) +10. Owner +11. PR links +12. Tests (unit/integ/OS) +13. User-guide section +14. Feature flag +15. Notes + +Import every row from §2, §3, §4, §6. + +--- + +## 12. Definition of “full port complete” + +Declare complete when: + +1. Every row in §2 is Done, Partial-accepted (with written gap < 10% behavior), or Skip. +2. Every tool in §3 is Done or Explicitly Optional. +3. Every slash in §4 is Done or Explicitly Dev-only. +4. User-guide 01–24 exists and matches behavior. +5. ACP multi-turn + permissions works in at least one external editor. +6. Folder trust + sandbox deny globs pass security tests on macOS+Linux. +7. Marketplace can install a multi-component plugin. +8. Subagent explore/plan/general + resume + worktree isolation work. +9. No second memory/token/provider stack duplicated against yaad/tok/eyrie. +10. Continuous parity process (PACK-17) running for 2 releases. + +--- + +## 13. Resource model for “all including small things” + +| Team size | Time to Year-0 exit | Time to full port claim | +|-----------|---------------------|-------------------------| +| 1 senior | ~12 mo | ~4–5 years | +| 2 seniors | ~7–9 mo | ~2.5–3.5 years | +| 3–4 (core+cloud+skills) | ~6 mo | ~2–3 years | + +**Bottleneck:** TUI parity (pager 415k LOC class) and optional computer-hub — not the spawn contracts. + +--- + +## 14. Immediate next 90 days (actionable) + +| Week | Work | +|------|------| +| 1–2 | PACK-01 contracts + ADR | +| 3–8 | PACK-02 spawn control plane | +| 9–12 | Start PACK-03 trust/sandbox (parallel safe-bash) | + +Do **not** start marketplace or computer-hub before PACK-02/03. + +--- + +## 15. Relationship to shorter plan + +| Document | Scope | +|----------|--------| +| `GROK-CLASS-CAPABILITY-LONG-HORIZON-PLAN.md` | Agent control + trust + hooks + marketplace focus (~12–18 mo) | +| **This document** | **Every crate, tool, slash, small system** → multi-year full port | + +Execute shorter plan as **Year 0** of this master plan. + +--- + +## 16. One-line summary + +**Port all of grok-eco into hawk-eco = reimplement Grok Build’s full capability surface in Go across hawk-eco repos, map each crate to an owner engine, skip vendor/privacy conflicts, wire Hawk’s existing partial systems first, and run a multi-year program ending in behavioral parity—not a Rust code transplant.** + +--- + +*Maintainers: update Status column as PRs merge; re-run crate inventory when grok-build updates.* diff --git a/docs/plans/GROK-CLASS-CAPABILITY-LONG-HORIZON-PLAN.md b/docs/plans/GROK-CLASS-CAPABILITY-LONG-HORIZON-PLAN.md new file mode 100644 index 00000000..af50db42 --- /dev/null +++ b/docs/plans/GROK-CLASS-CAPABILITY-LONG-HORIZON-PLAN.md @@ -0,0 +1,994 @@ +# Hawk-Eco Long-Horizon Plan: Grok-Class Agent Control Plane (Go) + +**Status:** Planning +**Date:** 2026-07-16 +**Horizon:** ~12–18 months (phased; can compress with parallel teams) +**Language:** Go only (no Rust ports; reimplement contracts and behavior) +**Primary product:** `hawk` +**Supporting repos:** `hawk-core-contracts`, `eyrie`, `yaad`, `tok`, `trace`, `sight`, `inspect`, `hawk-mcpkit`, `hawk-community-skills`, `hawk-cloud`, `hawk-sdk-go`, `hawk-sdk-python`, `graycode-core` (UI only) + +--- + +## 0. Executive summary + +### Goal + +Bring Hawk to **Grok-class agent control-plane quality** (typed subagents, sandbox profiles, folder trust, hooks, plugins/marketplace, ACP depth, monitor/scheduler, enterprise managed policy) **without** abandoning Hawk’s multi-repo Go platform advantages (eyrie multi-provider, yaad graph memory, tok compression, mission mode, contracts, cloud ledger). + +### Non-goals + +- Rewrite Hawk in Rust or monorepo-collapse engines into hawk. +- Vendor lock-in to a single LLM host auth model. +- Replace eyrie/yaad/tok/trace with Grok-shaped internal crates. +- Default opt-in product telemetry that violates privacy-first posture. + +### Strategy + +1. **Close wiring gaps first** — Hawk already has many *types* and *partial systems* that are not exposed on the Agent tool or unified. +2. **Stabilize contracts** in `hawk-core-contracts` before multi-repo consumers. +3. **Ship vertical slices** (end-to-end user value) every quarter. +4. **Keep engines peer-independent**; hawk remains the only product orchestrator. + +--- + +## 1. Deep audit: current state (verified 2026-07-16) + +### 1.1 What already exists (do not rebuild) + +| Area | Evidence | Maturity | +|------|----------|----------| +| Subagent **modes as types** | `internal/engine/agent/agent_types.go`: `explore`, `general`, `plan`; thoroughness quick/medium/very-thorough; `ExploreTools` / `PlanTools` | **Types strong** | +| Mode tool allowlists + budgets | `internal/engine/agent/subagent_budget.go`: `ModeToolAllowlist`, `FilterToolsForMode`, turn budgets | **Library strong** | +| Subagent spawn implementation | `internal/engine/agent_session_tool.go`: `spawnSubAgent`, model cascade, tool filter, max depth, inherits spec stage | **Partial** | +| Agent tool surface | `internal/tool/agent.go`: only `prompt`, `run_in_background`, `agent_id`, `retry_of` | **Weak vs types** | +| Default wire path | `WireAgentTool()` **always** calls `spawnSubAgent(..., SubAgentExplore, 0)` | **Critical gap** | +| Background systems (3+) | `tool.BackgroundAgentManager`, `engine.BackgroundRunner`, `agent.BackgroundAgentPool` | **Duplicated** | +| Permission + spec gate | `engine/safety/permission_engine.go`: DryRun → SpecStage → autonomy → classifier → auto-mode → memory → prompt | **Strong core** | +| Sandbox backends | seatbelt, landlock, seccomp, docker, gvisor; modes strict/workspace/off | **Strong backends, weak profiles** | +| Personas | `multiagent/agents`: ReadOnly, tools, hooks, YAML | **Strong** | +| Mission worktrees | `multiagent/worker.go` creates git worktrees | **Strong for missions only** | +| Worktree tools | `EnterWorktree` / `ExitWorktree` | **User-driven, not spawn isolation** | +| Plugins V1/V2 | subprocess/daemon/WASM, hooks on manifest | **Partial packaging** | +| Skills install + community registry | git install + huge `hawk-community-skills` | **Content strong** | +| ACP first cut | `internal/acp` + `cmd/acp.go`: init/new/prompt/cancel/permission | **Thin** | +| Session checkpoints/fork | `internal/session/checkpoint.go`, fork, export | **Strong** | +| Cron | `internal/tool/cron.go` max 256 jobs | **Present, UX thin** | +| Cloud enterprise policy DTO | `hawk-cloud` enterprise: model allow/deny, capability allow/deny | **Primitive exists** | +| Contracts | severity, tools, events, policy, review, verify, sessions phases | **Missing agent spawn types** | +| Compaction | multiple `engine/compact*.go` | **Present** | +| Memory | yaad graph engine | **Architecturally ahead of Grok markdown memory** | +| Token/cost | tok library | **Architecturally ahead of Grok heuristic** | + +### 1.2 Critical gap pattern + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Types & budgets exist (explore/plan/general) │ +│ spawnSubAgent(mode) exists │ +│ WireAgentTool hardcodes explore │ +│ Agent tool schema cannot pass mode / isolation / resume │ +│ → Model cannot use plan mode or general-purpose by design │ +└─────────────────────────────────────────────────────────────┘ +``` + +This is the single highest-ROI fix: **wire what already exists**, then extend. + +### 1.3 Permission pipeline (actual vs target) + +**Actual order** (`PermissionEngine.CheckTool`): + +1. DryRun deny-all +2. Spec stage gate (independent of autonomy) +3. Autonomy preset / “safe tool” short-circuit +4. Bypass killswitch +5. Bash classifier “safe” +6. AutoMode +7. Permission memory +8. User prompt + +**Missing relative to Grok-class:** + +- PreToolUse **hooks before** all of the above (hooks exist as registry but not as first deny gate in this path) +- Explicit ordered **deny rules** package as distinct stage +- Segment-aware **safe-bash allowlist** (classifier is different, weaker documentation) +- Named modes `dontAsk` / `acceptEdits` as first-class (partially in jsonc `defaultMode`) +- Folder trust gate for project automation + +### 1.4 Duplication / debt that the plan must fix early + +| Debt | Risk if ignored | +|------|-----------------| +| 3 background task managers | Inconsistent ids, race bugs, broken Wait/Kill | +| Agent tool signature `func(ctx, prompt string)` | Blocks mode/isolation/model forever | +| explore Bash allowlisted without hard read-only bash AST gate | Explore mode can still mutate via shell | +| Plugin = tools binary, not multi-component package | Marketplace cannot ship team bundles | +| No folder trust | Project hooks/MCP = RCE vector | + +--- + +## 2. Target architecture (Go) + +```text + ┌──────────────────────────────┐ + │ hawk TUI / CLI / daemon/ACP │ + └──────────────┬───────────────┘ + │ + ┌─────────────────────────┼─────────────────────────┐ + ▼ ▼ ▼ +┌─────────────────┐ ┌──────────────────┐ ┌──────────────────┐ +│ SpawnController │ │ Trust & Policy │ │ Extension Host │ +│ typed subagents │ │ folder trust │ │ plugins/skills │ +│ isolation │ │ sandbox.toml │ │ marketplace │ +│ resume │ │ hooks pipeline │ │ MCP/LSP load │ +└────────┬────────┘ └────────┬─────────┘ └────────┬─────────┘ + │ │ │ + ▼ ▼ ▼ +┌─────────────────┐ ┌──────────────────┐ ┌──────────────────┐ +│ engine.Session │ │ PermissionEngine │ │ plugin.Manager │ +│ + TaskRuntime │ │ + HookRunner │ │ skill.Loader │ +└────────┬────────┘ └──────────────────┘ └──────────────────┘ + │ + ┌────┴────┬──────────┬──────────┬──────────┐ + ▼ ▼ ▼ ▼ ▼ + eyrie yaad tok trace sight/inspect +``` + +**New core package (proposed):** `hawk/internal/spawn` (or expand `engine/agent`) owning: + +- `SpawnRequest` / `SpawnResult` (Go structs aligned with contracts) +- capability filter, isolation worktree lifecycle +- transcript persistence for resume +- unified task registry (shell bg + subagents + monitors) + +**Contracts package (proposed):** `hawk-core-contracts/agent` (or `spawn`): + +- enums + DTOs only; no engine imports + +--- + +## 3. Concept inventory → implementation status + +Legend: **Done** | **Partial** | **Missing** | **N/A (keep engine)** + +### Wave A — Agent control (P0) + +| # | Concept | Status | Target | +|---|---------|--------|--------| +| A1 | Typed spawn schema on Agent tool | Missing on tool; Partial in engine | Full | +| A2 | `subagent_type` explore/plan/general-purpose | Types Partial; wire Missing | Full | +| A3 | Thoroughness for explore | Types Done; wire Missing | Full | +| A4 | Capability modes read-only/read-write/execute/all | Partial (modes only) | Full enum + filter | +| A5 | Isolation none/worktree | Mission Partial; Agent Missing | Full | +| A6 | True `resume_from` transcript | Missing (agent_id status only) | Full | +| A7 | Structured SpawnResult | Partial envelope | Full | +| A8 | MultiAgent typed task objects | Missing | Full | +| A9 | Unify background task runtime | Partial (3 systems) | Full | +| A10 | Explore Bash hard read-only | Partial (comment only) | Full (AST allowlist) | + +### Wave B — Trust & sandbox (P1) + +| # | Concept | Status | Target | +|---|---------|--------|--------| +| B1 | Built-in profiles incl. read-only, devbox | Partial (3 modes) | Full | +| B2 | `sandbox.toml` extends/deny globs | Missing | Full | +| B3 | Project additive-only profile merge | Missing | Full | +| B4 | Folder trust store | Missing | Full | +| B5 | Gate hooks/MCP/LSP/plugins on trust | Missing | Full | +| B6 | Safe-bash allowlist (segment-aware) | Partial classifier | Full | +| B7 | Permission pipeline doc + hooks first | Partial | Full | +| B8 | Named modes acceptEdits/dontAsk | Partial jsonc | Full product | + +### Wave C — Hooks (P2) + +| # | Concept | Status | Target | +|---|---------|--------|--------| +| C1 | Full event set (subagent, stop, failure) | Partial | Full | +| C2 | Vendor event aliases | Missing | Full | +| C3 | File-discovered hooks | Partial (in-process) | Full | +| C4 | HTTP hooks | Missing | Full | +| C5 | Plugin env HAWK_PLUGIN_ROOT/DATA | Missing | Full | +| C6 | PreToolUse can deny in CheckTool path | Missing | Full | + +### Wave D — Plugins & marketplace (P3) + +| # | Concept | Status | Target | +|---|---------|--------|--------| +| D1 | Multi-component plugin layout | Missing | Full | +| D2 | Discovery scopes + priority | Partial | Full | +| D3 | Marketplace multi-source | Design only | Full | +| D4 | Multi-harness skill scan | Missing | Full | +| D5 | Audit on install (keep) | Partial | Harden | + +### Wave E — Tasks / monitor / scheduler (P4) + +| # | Concept | Status | Target | +|---|---------|--------|--------| +| E1 | Shell background task_id | Partial | Full | +| E2 | GetTaskOutput / WaitTasks / KillTask | Missing as tools | Full | +| E3 | Monitor line-stream tool | Missing | Full | +| E4 | /loop UX over cron | Partial cron | Full | + +### Wave F — Plan UX & AskUser (P5) + +| # | Concept | Status | Target | +|---|---------|--------|--------| +| F1 | Align /spec with plan subagent | Partial | Full | +| F2 | Structured AskUserQuestion | Partial free-text | Full | + +### Wave G — ACP & SDKs (P6) + +| # | Concept | Status | Target | +|---|---------|--------|--------| +| G1 | session/load + resume | Missing | Full | +| G2 | Richer session/update | Partial | Full | +| G3 | Optional agent serve/WS | Missing | Optional | +| G4 | SDK spawn/plugin fields | Missing | Full | +| G5 | OpenAPI sync | Partial | Full | + +### Wave H — Sessions / import / attribution (P7) + +| # | Concept | Status | Target | +|---|---------|--------|--------| +| H1 | Foreign session import | Missing | Full (trace) | +| H2 | Hunk agent vs external | Missing | Full | +| H3 | Checkpoint already | Done | Keep | + +### Wave I — Memory UX (P8) + +| # | Concept | Status | Target | +|---|---------|--------|--------| +| I1 | Toggle priority order | Partial | Full | +| I2 | Dream/consolidate product UX | Partial yaad | Full | +| I3 | Graph memory core | Done (yaad) | Keep | + +### Wave J — Enterprise (P9) + +| # | Concept | Status | Target | +|---|---------|--------|--------| +| J1 | Config layer merge order | Partial | Full | +| J2 | Managed signed policy pull | Partial cloud DTO | Full | +| J3 | Fail-open default / fail-closed org | Missing | Full | + +### Wave K — Docs & polish (P10) + +| # | Concept | Status | Target | +|---|---------|--------|--------| +| K1 | User-guide tiered docs | Missing | Full | +| K2 | Mermaid render optional | Partial text | Optional | +| K3 | Keep engines independent | Done | Keep | + +--- + +## 4. Multi-quarter roadmap (long horizon) + +Assumes ~1–2 senior Go engineers on hawk core + fractional engine/cloud; scale parallelizes quarters. + +```text +Q1 Foundation: contracts + spawn wiring + background unify +Q2 Trust: sandbox.toml + folder trust + safe-bash + hooks gate +Q3 Extensions: plugins multi-component + marketplace MVP + skill multi-harness +Q4 Runtime: monitor/wait/loop + AskUser structured + plan alignment +Q5 Integration: ACP phase-2 + OpenAPI + SDKs +Q6 Enterprise: managed policy + cloud apply + IT tier +Q7 Ecosystem: foreign import (trace) + hunk attribution + memory UX +Q8 Hardening: perf, fuzz, security audit, docs completion, GA polish +``` + +Calendar can slip; **order of waves should not reverse** (contracts before marketplace consumers, trust before project plugins). + +--- + +## 5. Detailed phase plans + +### Phase 0 — Prep (1–2 weeks) + +**Objectives** + +- Freeze concept list (this document). +- Inventory all `AgentSpawnFn` call sites and background managers. +- Add ADR: “Typed spawn is the only subagent entrypoint.” + +**Deliverables** + +- [ ] ADR in `docs/architecture/adr/` +- [ ] Call-site map in this plan appendix +- [ ] Test plan template for spawn matrix +- [ ] Dependency freeze: no new third background system + +**Exit criteria** + +- Team agrees Wave A is first PR stack; no marketplace before folder trust. + +--- + +### Phase 1 — Contracts foundation (2–3 weeks) + +**Repo:** `hawk-core-contracts` +**Module:** new package `agent` (or `spawn`) — stdlib only + +#### 1.1 Types to add + +```go +// CapabilityMode +const ( + CapReadOnly = "read-only" + CapReadWrite = "read-write" + CapExecute = "execute" + CapAll = "all" +) + +// IsolationMode +const ( + IsoNone = "none" + IsoWorktree = "worktree" +) + +// SubagentType +const ( + TypeGeneralPurpose = "general-purpose" // maps to general + TypeExplore = "explore" + TypePlan = "plan" +) + +type SpawnRequest struct { + Prompt string + Description string + SubagentType string + CapabilityMode string // optional; derived from type if empty + Isolation string + ResumeFrom string + CWD string + Model string + Background bool + Thoroughness string // explore only + ParentSession string +} + +type SpawnResult struct { + SubagentID string + SubagentType string + Status string // running|completed|failed + Output string + Summary string + ToolCalls int + Turns int + DurationMs int64 + WorktreePath string + Persona string + Error string +} +``` + +#### 1.2 Work items + +| ID | Task | Effort | Tests | +|----|------|--------|-------| +| C-1 | Package + Parse helpers (aliases: general↔general-purpose, ReadOnly) | S | table tests | +| C-2 | Validate mutual exclusion cwd vs worktree | S | unit | +| C-3 | Version bump + CHANGELOG | S | CI module | +| C-4 | Proto optional later — **not required Q1** | — | — | + +**Exit criteria** + +- `go test ./...` green; hawk can depend on new pseudo-version without engine imports. + +--- + +### Phase 2 — Spawn controller & Agent tool (4–6 weeks) ★ highest ROI + +**Repo:** `hawk` +**Packages:** `internal/engine/agent`, `internal/engine`, `internal/tool` + +#### 2.1 Change `AgentSpawnFn` signature + +**Today:** + +```go +AgentSpawnFn func(ctx context.Context, prompt string) (string, error) +``` + +**Target:** + +```go +AgentSpawnFn func(ctx context.Context, req agentcontracts.SpawnRequest) (agentcontracts.SpawnResult, error) +``` + +Migration: temporary adapter for one release if needed; prefer single breaking change in 0.x. + +#### 2.2 Wire `WireAgentTool` correctly + +| Step | Change | +|------|--------| +| 1 | Parse Agent tool JSON → `SpawnRequest` | +| 2 | Map `subagent_type` → `SubAgentMode` (+ thoroughness) | +| 3 | Default capability from mode; allow override | +| 4 | Apply `FilterToolsForMode` **and** capability filter | +| 5 | Isolation worktree: create under `.hawk/worktrees/` using shared helper from mission worker | +| 6 | Persist child session transcript under `~/.hawk/subagents//` | +| 7 | `resume_from`: load transcript, append prompt, re-spawn with same type | +| 8 | Return structured `SpawnResult` JSON to model | + +#### 2.3 Explore Bash hard gate + +- Reuse `internal/tool` bash AST if present (`bash_ast.go`). +- For `IsReadOnlyMode`: only allowlist commands (Grok-class list). +- Deny `rm`, redirects, `git commit`, network tools, etc. + +#### 2.4 MultiAgent + +```json +{ + "tasks": [ + { + "prompt": "...", + "description": "scan auth", + "subagent_type": "explore", + "capability_mode": "read-only", + "thoroughness": "very-thorough" + } + ], + "run_in_background": true +} +``` + +#### 2.5 Unify background runtime + +**Target single package:** `internal/taskruntime` + +| Capability | API | +|------------|-----| +| Register | shell / subagent / monitor | +| Get | status + output | +| Wait | any/all + timeout | +| Kill | cancel context + process group | +| List | active | + +Deprecate: + +- Prefer migrating call sites off `BackgroundRunner` and `BackgroundAgentPool` into one manager; keep thin wrappers one release. + +#### 2.6 PR stack (Graphite-friendly) + +1. Contracts bump in hawk +2. `SpawnRequest` internal + adapter +3. Agent tool schema + parse +4. Wire modes plan/general/explore +5. Worktree isolation +6. Transcript resume +7. Background unify +8. Explore bash hard gate +9. MultiAgent objects +10. Docs + e2e tests + +**Exit criteria** + +- Model can spawn plan agent that cannot Write. +- Explore cannot `rm` via Bash. +- Resume continues conversation with prior tool results present. +- Single task id namespace for bg subagents. +- Race tests pass under `-race`. + +**Effort:** ~6–8 eng-weeks. + +--- + +### Phase 3 — Sandbox profiles + folder trust (4–5 weeks) + +**Repo:** `hawk` +**Packages:** `internal/sandbox`, `internal/trust` (new), `internal/mcp`, `internal/hooks`, `internal/plugin` + +#### 3.1 sandbox.toml + +```toml +# ~/.hawk/sandbox.toml +[profiles.ci] +extends = "strict" +restrict_network = true +deny = ["**/.env", "**/*.pem", "**/*credentials*"] +``` + +**Loader rules (security-critical):** + +1. Load user global. +2. Load project `.hawk/sandbox.toml` **additive only** (new names only). +3. Warn on conflicting redefinition; ignore project redefinition. +4. Deny globs applied on top of backend (seatbelt/landlock/bwrap-equivalent). +5. Fail closed when deny requested but OS cannot enforce. + +#### 3.2 Profile matrix + +| Name | FS read | FS write | Network | Maps from today | +|------|---------|----------|---------|-----------------| +| off | all | all | all | ModeOff / TierOff | +| workspace | all | cwd+state+tmp | allow | ModeWorkspace | +| read-only | all | state+tmp only | deny where possible | **new name** | +| strict | cwd+sys | cwd+state+tmp | deny | ModeStrict | +| devbox | all | broad except protected | allow | **new** | +| custom | extends | extends | flag | **new** | + +#### 3.3 Folder trust + +**Store:** `~/.hawk/trusted_folders.toml` + +```toml +[[folders]] +path = "/Users/me/proj" +trusted_at = "2026-07-16T00:00:00Z" +``` + +**Commands:** `/trust`, `/trust status`, `/trust revoke`, CLI `--trust` + +**Gates when untrusted:** + +- Project hooks +- Project MCP +- Project LSP +- Project plugins +- Optional: project skills that execute scripts + +**Exit criteria** + +- Malicious project cannot redefine global `strict` profile. +- Untrusted project hooks never run (tested). +- Custom deny `**/.env` blocks Read and Bash cat (integration tests per OS). + +**Effort:** ~5–7 eng-weeks (OS matrix heavy). + +--- + +### Phase 4 — Hooks pipeline completion (3–4 weeks) + +**Repo:** `hawk` +**Package:** `internal/hooks` + +#### 4.1 Event model expansion + +Add: `stop`, `stop_failure`, `post_tool_failure`, `permission_denied`, `user_prompt_submit`, `notification`, `subagent_start`, `subagent_stop` +Keep existing; alias map for Claude/Cursor names. + +#### 4.2 Runners + +| Type | Behavior | +|------|----------| +| command | shell with timeout, env HAWK_* | +| http | POST JSON, timeout, optional deny on non-2xx for pre_tool | + +#### 4.3 Integration into PermissionEngine + +```text +CheckTool: + 1. DryRun + 2. PreTool hooks (deny wins) + 3. Spec stage + 4. Explicit deny rules + 5. ... rest +``` + +#### 4.4 File discovery + +- `~/.hawk/hooks/*.json` +- `.hawk/hooks/*.json` (trust required) +- Compat: `.claude/settings.json` hooks, `.cursor/hooks.json` behind flags + +**Exit criteria** + +- Pre-tool hook can block `Bash(rm -rf)` even in Autonomous. +- Project hook skipped until trust. +- HTTP hook timeouts do not hang session (fail open or closed by config). + +**Effort:** ~4 eng-weeks. + +--- + +### Phase 5 — Plugins multi-component + marketplace MVP (6–8 weeks) + +**Repos:** `hawk`, `hawk-community-skills` + +#### 5.1 Plugin layout (convention) + +```text +my-plugin/ + plugin.json # optional + skills/**/SKILL.md + commands/*.md + agents/*.yaml|*.md + hooks/hooks.json + .mcp.json + .lsp.json +``` + +Loader merges components; tools optional. + +#### 5.2 Discovery priority + +1. Session meta / SDK +2. `--plugin-dir` +3. Project `.hawk/plugins` (trust) +4. User `~/.hawk/plugins` +5. Config extra paths + +#### 5.3 Marketplace + +| Component | Owner | +|-----------|--------| +| Source list config | hawk | +| Index schema | community-skills + hawk | +| Install resolve (git) | hawk | +| Audit | hawk plugin malware_check (extend) | +| CLI `hawk plugins` / TUI tab | hawk | +| Optional web gallery | graycode-core later | + +#### 5.4 Multi-harness skills + +Scan `.claude`, `.cursor`, `.agents` with toggles in settings; dedupe by name. + +**Exit criteria** + +- Install plugin with skill+hook+MCP in one command. +- Marketplace install from official + one third-party source. +- Audit fails install of skill with shell exfil pattern. + +**Effort:** ~8–10 eng-weeks (product + content). + +--- + +### Phase 6 — Task runtime: monitor, wait, loop (3–4 weeks) + +Depends on Phase 2 unified `taskruntime`. + +| Tool | Spec | +|------|------| +| `GetTaskOutput` | id, optional timeout_ms | +| `WaitTasks` | ids[], mode any\|all, timeout | +| `KillTask` | id | +| `Monitor` | command, persistent?, timeout; line events → stream | + +**Loop UX** + +- `/loop 5m ` → cron wrapper with caps (max jobs, 7-day expiry) +- Fire immediate + interval + +**Exit criteria** + +- Dev server monitor produces line events without flooding (rate limit). +- Wait any returns when first subagent completes. + +**Effort:** ~4 eng-weeks. + +--- + +### Phase 7 — Plan alignment + structured AskUser (2–3 weeks) + +| Work | Detail | +|------|--------| +| Plan subagent | Default tool for “design only”; writes plan file under `.hawk/plans/` or specs | +| Spec workflow | Document mapping: plan agent → `/spec` stages; avoid double systems | +| AskUserQuestion | questions[], options, multi_select, other, cancel message | +| TUI | Reuse autonomy/spec pickers | + +**Exit criteria** + +- Structured multi-question UI in chat. +- Plan agent never writes production code tools. + +**Effort:** ~3 eng-weeks. + +--- + +### Phase 8 — ACP phase-2 + SDKs + OpenAPI (5–7 weeks) + +**Repos:** `hawk`, `hawk-sdk-go`, `hawk-sdk-python` + +| Milestone | Scope | +|-----------|--------| +| ACP-1 | session/load, list sessions | +| ACP-2 | richer updates (tool start/end, thoughts if available) | +| ACP-3 | permission round-trip hardened | +| API | OpenAPI fields: spawn options, sandbox, autonomy, plugins meta | +| SDK | AgentConfig + ChatRequest fields; plugin dirs | +| Optional | `hawk agent serve --bind` WS | + +**Exit criteria** + +- Zed/VS Code can drive hawk ACP for multi-turn with permissions. +- SDK e2e against daemon contract snapshot tests. + +**Effort:** ~7 eng-weeks (+ extension work separate). + +--- + +### Phase 9 — Enterprise managed policy (4–6 weeks) + +**Repos:** `hawk-cloud`, `hawk`, optional `eyrie` + +| Piece | Detail | +|-------|--------| +| Policy document | models allow/deny, capabilities, sandbox deny, max budget, tool denylist | +| Signing | Ed25519 envelope; hawk verifies before apply | +| Apply path | `~/.hawk/managed_policy.json` layers under user config | +| Default | fail-open for individual; org can require fail-closed | +| IT tier | non-excludable rules (already sketched in product — finish) | +| Cloud UI | graycode-core admin later | + +Build on existing enterprise policyInput (model/capability lists). + +**Exit criteria** + +- Org policy denies model X on device within TTL. +- Tampered signature rejected. +- Offline individual still works without cloud. + +**Effort:** ~6 eng-weeks. + +--- + +### Phase 10 — Ecosystem polish (trace import, hunks, yaad UX) (4–6 weeks) + +| Item | Repo | Detail | +|------|------|--------| +| Foreign import | trace + hawk CLI | Claude/Codex session metadata → index | +| Hunk attribution | hawk | agent vs external edits via fsnotify | +| Memory UX | yaad + hawk | toggle priority, `/dream` consolidate | +| Mermaid optional | hawk | sandbox render path | + +**Exit criteria** + +- Import at least Claude sessions list. +- “Agent changed files this turn” accurate in TUI. + +**Effort:** ~5 eng-weeks. + +--- + +### Phase 11 — Hardening & GA (ongoing last quarter) + +- Fuzz bash allowlist, sandbox deny globs, spawn JSON +- Race + stress multiagent +- Security review: trust, hooks, marketplace install +- User-guide `docs/user-guide/01–22` +- Performance: spawn latency, worktree creation +- Compatibility matrix: macOS seatbelt, Linux landlock, Windows best-effort +- Release: feature flags default-on after bake + +--- + +## 6. Cross-repo ownership matrix + +| Concept family | Primary | Secondary | Contracts? | +|----------------|---------|-----------|------------| +| Spawn / capability / isolation | hawk | — | yes | +| Sandbox profiles | hawk | — | optional DTO | +| Folder trust | hawk | — | no | +| Hooks | hawk | — | event names yes | +| Plugins / marketplace | hawk | community-skills | manifest schema | +| Multi-harness skills | hawk | community-skills | no | +| Monitor/tasks | hawk | — | optional | +| ACP | hawk | sdk-go/python | OpenAPI | +| Managed policy | hawk-cloud | hawk, graycode-core | yes DTO | +| Foreign import | trace | hawk | optional | +| Memory dream UX | yaad | hawk | no | +| Token | tok | — | no change | +| Providers | eyrie | hawk | no change | +| Review engines | sight/inspect | hawk | findings already | + +--- + +## 7. Testing strategy (all phases) + +### 7.1 Unit + +- Parse/validate SpawnRequest (aliases, mutual exclusion) +- FilterToolsForMode × capability matrix +- sandbox.toml merge security +- hook alias map +- safe-bash segments + +### 7.2 Integration + +- Spawn explore/plan/general end-to-end with fake LLM +- Worktree isolation: parent tree unchanged +- Resume transcript +- Untrusted project hooks never execute +- Deny glob blocks file read + +### 7.3 Race / stress + +- MultiAgent max concurrency +- Background collect +- Monitor rate limit + +### 7.4 OS matrix + +| Feature | macOS | Linux | Windows | +|---------|-------|-------|---------| +| Seatbelt profiles | required | n/a | n/a | +| Landlock/seccomp | n/a | required | n/a | +| Worktree isolation | yes | yes | best-effort | +| Folder trust | yes | yes | yes | + +### 7.5 Compatibility / contract CI + +- `hawk-core-contracts` version pin +- OpenAPI snapshot for SDKs +- Engine import boundary scripts (existing ecosystem boundary checks) + +--- + +## 8. Feature flags & rollout + +| Flag | Default early | GA | +|------|---------------|-----| +| `spawn.v2` | on for contributors | on | +| `sandbox.profiles` | off → on | on | +| `trust.folder` | on (secure default) | on | +| `hooks.file_discovery` | off → on | on | +| `plugins.marketplace` | off | on | +| `monitor.tool` | off | on | +| `acp.v2` | off | on | +| `policy.managed` | off | org-only | + +Prefer **secure defaults** (folder trust on) even if noisy. + +--- + +## 9. Risk register + +| Risk | Impact | Mitigation | +|------|--------|------------| +| Signature change of AgentSpawnFn breaks plugins | High | Adapter release; version min_hawk | +| Worktree disk bloat | Med | GC old worktrees; symlink shared dirs (already pattern) | +| Explore still escapes via Bash | High | AST allowlist + sandbox | +| Marketplace supply chain | High | audit, pin commit SHAs, signatures later | +| Hook RCE via project | Critical | folder trust + fail closed | +| Three bg systems mid-migration | Med | single package + deprecation window | +| Over-scoping vs release readiness | High | Phases 1–2 only until contributor GA | +| Cloud policy vs local-first conflict | Med | fail-open default; explicit org enroll | + +--- + +## 10. Success metrics + +| Metric | Baseline (now) | Target (post Phase 2) | Target (GA) | +|--------|----------------|------------------------|-------------| +| Agent tool can select plan/explore/general | No (hardcoded explore) | Yes | Yes | +| Explore shell mutation blocked | Partial | 100% allowlist tests | Fuzz green | +| Untrusted project hooks run | Possible | 0 | 0 | +| Time to implement team plugin (skill+mcp+hook) | High friction | <30 min | <10 min marketplace | +| ACP multi-turn with permissions | First-cut | Full load/resume | Editor-certified | +| Subagent resume fidelity | Status only | Full transcript | Full + worktree | +| Doc user-guide chapters | ~0 numbered | 8 | 20+ | + +--- + +## 11. Suggested PR / milestone naming + +```text +feat(contracts): agent spawn DTOs +feat(spawn): wire explore/plan/general through Agent tool +feat(spawn): capability modes + tool filter +feat(spawn): worktree isolation +feat(spawn): resume_from transcripts +refactor(taskruntime): unify background managers +feat(sandbox): sandbox.toml profiles +feat(trust): folder trust store +feat(hooks): pre_tool deny + file/http runners +feat(plugin): multi-component layout +feat(marketplace): multi-source install MVP +feat(tools): monitor + wait + kill +feat(acp): session load/resume +feat(cloud): managed policy apply +docs(user-guide): 01-getting-started … +``` + +--- + +## 12. Resource plan (long horizon) + +| Role | Q1–Q2 | Q3–Q4 | Q5–Q8 | +|------|-------|-------|-------| +| Hawk core (Go) | 1.5 FTE | 1.5 FTE | 1 FTE | +| Security-minded sandbox | 0.5 FTE | 0.5 FTE | 0.25 FTE | +| Community-skills / marketplace content | 0.25 | 0.75 | 0.5 | +| Cloud (TS) | 0 | 0.25 | 0.75 | +| SDK / ACP | 0.25 | 0.25 | 0.75 | +| Docs / DX | 0.25 | 0.5 | 0.5 | + +**Minimum viable long-horizon track** (one engineer): Phases 1→2→3→4 only (~4–5 months) yields most product quality jump. + +--- + +## 13. Appendix A — Call sites to migrate (AgentSpawnFn) + +Verified entry points (non-exhaustive; re-grep at start of Phase 2): + +| File | Usage | +|------|--------| +| `internal/engine/session.go` | field declaration | +| `internal/engine/agent_session_tool.go` | WireAgentTool / spawnSubAgent | +| `internal/engine/stream_tool_exec.go` | ToolContext injection | +| `internal/tool/agent.go` | Agent / MultiAgent | +| `internal/tool/agentic_fetch.go` | research spawn | +| `internal/tool/tool.go` | ToolContext | + +Background: + +| File | System | +|------|--------| +| `internal/tool/background.go` | BackgroundAgentManager | +| `internal/engine/background_runner.go` | BackgroundRunner | +| `internal/engine/agent/background_agent.go` | BackgroundAgentPool | + +--- + +## 14. Appendix B — Mapping Grok names → Hawk names + +| Grok | Hawk | +|------|------| +| `task` tool | `Agent` / `Task` alias | +| `general-purpose` | `general` / `general-purpose` | +| `capability_mode` | same | +| `isolation: worktree` | same | +| `resume_from` | same (replace weak `agent_id` semantics) | +| `sandbox.toml` | `~/.hawk/sandbox.toml` | +| folder trust | `~/.hawk/trusted_folders.toml` | +| `/loop` | `/loop` over CronScheduler | +| plugin marketplace | `hawk plugins` + community registry | +| managed_config | managed policy via hawk-cloud | +| ACP | `hawk acp` | + +--- + +## 15. Appendix C — First 30-day execution checklist + +Week 1 + +- [ ] Land contracts package + tests +- [ ] Document ADR +- [ ] Inventory + delete path for bg managers (design only) + +Week 2 + +- [ ] SpawnRequest internal +- [ ] Agent tool schema expanded (flags behind feature if needed) +- [ ] Wire plan + general (stop hardcoding explore) + +Week 3 + +- [ ] Explore bash allowlist enforcement +- [ ] SpawnResult structured output +- [ ] Integration tests fake LLM + +Week 4 + +- [ ] Worktree isolation MVP +- [ ] Transcript save (resume stub OK) +- [ ] CHANGELOG + user-facing `/help` snippet + +--- + +## 16. Decision log (planning) + +| Decision | Choice | Rationale | +|----------|--------|-----------| +| Language | Go only | Hawk ecosystem is Go | +| First work | Wire existing modes | Types already exist; highest ROI | +| Contracts location | hawk-core-contracts | Engines/SDKs may later honor modes | +| Trust default | On | Security | +| Marketplace before trust? | No | Supply chain | +| Replace yaad? | No | Graph memory superior | +| Replace eyrie? | No | Multi-provider is differentiator | +| Three bg systems | Unify early | Blocks monitor/wait | + +--- + +## 17. Document maintenance + +- Update this plan at each phase exit. +- Mark concept table statuses. +- Link PRs under each phase. +- Supersedes marketing claims in `docs/IMPLEMENTATION-ROADMAP.md` for agent-control scope (that doc remains for marketplace/IDE product ideas). + +--- + +**End of plan.** +Next action if approved: execute **Phase 1 (contracts)** then **Phase 2 PR stack** without waiting for marketplace or cloud. From 27ba2324f6beae541001cd56ff91ac7ac72873fc Mon Sep 17 00:00:00 2001 From: Lakshman Patel Date: Thu, 16 Jul 2026 09:08:27 +0530 Subject: [PATCH 2/2] fix: add proper error handling for io.ReadAll in telegram gateway The error return from io.ReadAll was previously ignored, which could lead to silent failures when reading partial responses from Telegram or the hawk daemon. Now logs warnings when partial reads occur. --- internal/daemon/telegram.go | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/internal/daemon/telegram.go b/internal/daemon/telegram.go index d9ff1361..c69480b4 100644 --- a/internal/daemon/telegram.go +++ b/internal/daemon/telegram.go @@ -227,11 +227,15 @@ func (tg *TelegramGateway) forwardToHawk(ctx context.Context, prompt string) (st defer func() { _ = resp.Body.Close() }() // Limit response body to 1 MiB to prevent memory exhaustion. - body, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20)) + body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20)) + if err != nil { + slog.Warn("partial read in forwardToHawk response", "error", err) + } var chatResp struct { Response string `json:"response"` } if err := json.Unmarshal(body, &chatResp); err != nil { + // JSON unmarshal may fail with partial body; return raw body as fallback return string(body), nil } return chatResp.Response, nil @@ -255,7 +259,10 @@ func (tg *TelegramGateway) sendMessage(ctx context.Context, chatID int64, text s return err } defer func() { _ = resp.Body.Close() }() - body, _ := io.ReadAll(io.LimitReader(resp.Body, 4096)) + body, err := io.ReadAll(io.LimitReader(resp.Body, 4096)) + if err != nil { + slog.Warn("partial read in sendMessage response", "error", err) + } var result struct { OK bool `json:"ok"` Description string `json:"description"`