Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,32 @@ All notable changes to DevMap are documented in this file.

## [Unreleased]

### Added

- `devmap mcp`: run DevMap as an MCP server over stdio so AI agents can query the project map
with tool calls (`--root`, `--auto-refresh`, `--allow-ai`)
- Tools: `devmap_overview`, `devmap_map`, `devmap_flow`, `devmap_routes`, `devmap_impact`
(direct and transitive dependents, affected features, critical dependents), `devmap_find`,
`devmap_analyze` and, only with `--allow-ai`, `devmap_explain`
- Resources `devmap://snapshot`, `devmap://devmap-md`, `devmap://feature/{id}` and prompts
`devmap_onboard`, `devmap_plan_change`
- Read-only and static by default: no AI provider is contacted unless `--allow-ai` is set;
every result reports whether the snapshot is stale; results are size-capped and say when
they are truncated
- `devmap_analyze` refreshes the snapshot (single analysis shared by concurrent calls,
progress notifications, `running` status past the timeout, `aiDataDropped` when a static
rebuild loses AI-derived data); `--auto-refresh` does it before answering and falls back to
the older snapshot when it fails
- `.devmap/analyze.lock` prevents two MCP servers from writing the snapshot at once
- `mcpName` in `package.json` for the MCP Registry
- `docs/mcp.md` and a landing-site docs page for the MCP server

### Changed

- `runMap`, `runFlow`, `runAnalyze` and `runExplain` are exported; `map` and `flow` accept `write`
and `loaded`, `flow` accepts `narrate`, and `analyze` accepts `offline`. CLI behavior is unchanged
- The CI and publish workflows run `test/mcp-smoke.mjs` against the build before publishing

### Planned

- Go, Java, C# and PHP (PSR-4) import resolution, after JS/TS is mature
Expand Down
20 changes: 17 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,11 @@ devmap/
│ │ │ └── registry/ ← Feature signal vocabulary (18 files)
│ │ ├── ai/ ← Provider abstraction (Groq, OpenRouter, Custom)
│ │ ├── cache/ ← Snapshot, file hash, agent navigation files
│ │ ├── commands/ ← Command handlers (init, analyze, map, flow, explain, onboarding, config, doctor)
│ │ ├── commands/ ← Command handlers (init, analyze, map, flow, explain, onboarding, config, doctor, mcp)
│ │ ├── mcp/ ← MCP server: tools, resources, prompts, snapshot provider, refresher + lock
│ │ ├── onboarding/ ← Onboarding model and builder
│ │ └── utils/ ← Config, errors, output, prompts, markdown renderer
│ ├── test/ ← 42 test files (`*.test.ts`, auto-discovered by `run-tests.ts`)
│ ├── test/ ← 48 test files (`*.test.ts`, auto-discovered by `run-tests.ts`) + `mcp-smoke.mjs`, `package-e2e.mjs`
│ └── package.json
├── apps/web/ ← Astro marketing site (unrelated to CLI)
├── docs/ ← Public documentation
Expand Down Expand Up @@ -72,6 +73,10 @@ Everything else (files, graph, routes, features, flows, critical files) is 100%
| Domain inference | `src/analyzers/inference/domainInference.ts` | AI call — structured metadata only |
| Snapshot enrichment | `src/ai/snapshotEnrichment.ts` | AI call — file purposes + feature terms |
| Agent navigation | `src/cache/agentNavigation.ts` | `writeAgentNavigationFiles()` — index.json + features/*.json |
| MCP server | `src/mcp/server.ts` | `createDevmapServer()` — tools (`devmap_overview/map/flow/routes/impact/find/analyze`, `devmap_explain` only with `--allow-ai`), resources, prompts |
| MCP queries | `src/mcp/queries.ts` | Pure read-only queries over a `ProjectMap` (overview, routes, impact, find) |
| MCP snapshot provider | `src/mcp/snapshotProvider.ts` | mtime-cached snapshot, TTL-cached staleness, opt-in auto-refresh |
| MCP refresher | `src/mcp/refresh.ts` | Single-flight analysis + `.devmap/analyze.lock` |

## Data Model

Expand Down Expand Up @@ -124,12 +129,13 @@ Order for a non-relative specifier: `#subpath` (package.json `imports`) → tsco

## Testing

42 test files in `packages/cli/test/`. Key covered areas:
48 test files in `packages/cli/test/`. Key covered areas:
- Analyzers, AI clients, agent navigation, entity extractors
- Dependency graph: aliases, workspace packages, `#imports`, Nuxt auto-imports/aliases, entry points (`graph-resolution`, `dependency-graph-alias`, `js-ts-ecosystem`, `js-ts-frameworks`)
- Feature similarity/merge, frontend features, framework routes (`framework-routes`; Remix/React Router/TanStack/Expo/Hono and bundler aliases in `js-ts-frameworks`; client-side routers, Hono factories, Next normalization and critical-file ranking in `js-ts-gaps`; SolidStart, Qwik City, Angular, Elysia, Koa, AdonisJS and constant-resolved paths in `js-ts-more-frameworks`), detector fixes
- Heuristic language readers (`heuristic-languages`), scanner robustness, project scope/metadata
- Commands (init, analyze, map, flow, explain, onboarding, config, doctor)
- MCP server (`core-api`, `mcp-server`, `mcp-freshness`, `mcp-explain`): tools, resources, prompts, stale/missing snapshots, auto-refresh and lock, offline-never-touches-AI, stdio process with a clean protocol stream. `test/mcp-smoke.mjs` runs the compiled `dist/` as a server (CI and publish workflow)
- Utils (markdown terminal, JSON output, prompts, errors)

**No dedicated unit tests**: capability detector (exercised through feature/onboarding tests), cache layer internals. Domain inference is covered by `analyze-ai`; the pipeline orchestrator, service detector, filter engine, dependency graph builder and entry point detector are covered by `analyzers`, `graph-resolution` and `js-ts-ecosystem`.
Expand Down Expand Up @@ -159,6 +165,8 @@ pnpm dev
pnpm devmap <command>
```

MCP server (after `pnpm build:cli`): `node packages/cli/test/mcp-smoke.mjs`.

## Important Constraints

- No source files should be modified during reverse-engineering tasks
Expand All @@ -167,6 +175,8 @@ pnpm devmap <command>
- Snapshot is regenerated on every `devmap analyze` run
- `AGENTS.md` is never overwritten by init (append-only)
- `.devmap/` is always gitignored
- `devmap mcp` owns stdout: it is the JSON-RPC stream. Never `console.log` from code the MCP server can reach; use `output.*` inside `withJsonOutput()` (silent) or stderr
- The MCP server is read-only and static by default. Only `devmap_analyze` / `--auto-refresh` write, and only inside `.devmap/`. AI is reachable only with `--allow-ai`; `devmap_explain` is registered only then

## Known Limitations

Expand All @@ -179,12 +189,16 @@ pnpm devmap <command>
- Elysia, Koa and Adonis detectors follow literal call chains and mounts; routers built inside factory functions, `.use()` of computed values and runtime-assembled paths are not followed. Angular reads literal `Routes` arrays and `loadChildren` imports
- Feature similarity threshold (0.35) is tuned but may cause false merges
- Domain inference is single-shot — no conversation context
- `.devmap/analyze.lock` is taken by MCP-driven analysis only; `devmap analyze` does not take or check it
- The MCP staleness check re-scans project files (cached for 10 s), so it is not free on very large repositories

## Engineering Notes

- All commands take a Dependencies object for testability
- `runSafely` wrapper handles errors and sets process.exitCode
- `AsyncLocalStorage` used for output mode (human/json)
- `runMap`, `runFlow`, `runAnalyze` and `runExplain` are exported and shared by the CLI and the MCP server; options `write:false`, `narrate:false`, `offline:true` and `loaded` make them safe for a long-lived process
- MCP tests copy a fixture to a temp dir and analyze it there; analyzing in place and saving the snapshot elsewhere changes the fingerprint and every result reports `stale: true`
- p-limit(50) concurrency for file scanning
- MD5 content hashing for fingerprinting
- `.js` → `.ts` extension swap in import resolution (shared in `graph/localPaths.ts`)
Expand Down
38 changes: 38 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,43 @@ Reusable across sessions, tools, and agents.

---

## MCP Server

Run DevMap as an [MCP](https://modelcontextprotocol.io) server and agents can
query the project map with tool calls instead of reading a dumped context file.

```bash
# Claude Code
claude mcp add devmap -- npx -y @flaid/devmap mcp
```

```json
{
"mcpServers": {
"devmap": { "command": "npx", "args": ["-y", "@flaid/devmap", "mcp"] }
}
}
```

The server is read-only and static by default (no AI provider is contacted unless
you start it with `--allow-ai`).

| Tool | Answers |
| ---- | ------- |
| `devmap_overview` | What is this project? Entry points, critical files, features, flows |
| `devmap_map` | How is this feature or file wired? |
| `devmap_flow` | How does this request or feature work end to end? |
| `devmap_routes` | Which pages and API routes exist? |
| `devmap_impact` | What breaks if I change this file? |
| `devmap_find` | Where is the code for this concept? |
| `devmap_analyze` | Refresh the snapshot when it is stale |
| `devmap_explain` | Explain this file, feature or function (needs `--allow-ai`) |

Every answer says whether the snapshot is stale. Setup for Claude Code, Cursor,
opencode and Codex, plus the privacy model, is in [docs/mcp.md](./docs/mcp.md).

---

## Vision

DevMap is not an AI coding assistant.
Expand Down Expand Up @@ -313,6 +350,7 @@ Node.js 22.12+
## Documentation

- [docs/commands.md](./docs/commands.md)
- [docs/mcp.md](./docs/mcp.md)
- [docs/architecture.md](./docs/architecture.md)
- [docs/generated-files.md](./docs/generated-files.md)
- [docs/design.md](./docs/design.md)
Expand Down
38 changes: 38 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -534,3 +534,41 @@ aggregation, and fine-grained incremental derived-feature computation.
required tests, work packages, acceptance criteria, verification commands, and
repository delivery rules. Implementation must follow it incrementally; do not
attempt this architecture as a single rewrite.

## 15. Front-ends: CLI and MCP

The analyzer has two front-ends over the same snapshot:

```txt
createProjectMap() ──▶ .devmap/snapshot.json ──┬──▶ CLI commands (init, analyze, map, flow, explain, ...)
└──▶ devmap mcp (tools, resources, prompts)
```

`devmap mcp` adds no analysis of its own. The command modules export their core
(`runAnalyze`, `runMap`, `runFlow`, `runExplain`), and the MCP server calls them
with options that make them safe for a long-lived process: `write: false`,
`narrate: false`, `offline: true` and a preloaded snapshot. Reads are served by
pure query functions over the `ProjectMap` (`mcp/queries.ts`).

Consequences for the architecture:

- **The AI boundary does not move.** The three call sites in section 9 stay the
only ones. The MCP server is static by default and reaches them only when the
user starts it with `--allow-ai`, because the caller is an agent and not the
person who owns the API key.
- **stdout becomes a protocol.** Commands must never assume `console.log` is
free. All run functions already route output through `output.*`, which
`withJsonOutput()` silences; the MCP process also redirects `console.log` to stderr.
- **Freshness is part of every answer.** A long-lived server cannot rely on the
user having just run `analyze`, so each result carries `snapshot.stale`, and the
snapshot can be refreshed on request (`devmap_analyze`) or automatically
(`--auto-refresh`).
- **Concurrency now exists.** Two agents, or an agent and a terminal, can ask for
analysis at once. In-process calls are single-flight; MCP-driven analysis also
takes `.devmap/analyze.lock`. The CLI `analyze` does not take the lock yet.

Deferred: a remote (HTTP) transport, serving several projects from one server, and
watching files for changes instead of checking staleness per request.

Implementation notes are in [guide/commands/09-mcp.md](./guide/commands/09-mcp.md);
usage is in [mcp.md](./mcp.md).
51 changes: 50 additions & 1 deletion docs/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,16 @@

## Overview

DevMap ships the following core project commands and one configuration command:
DevMap ships the following core project commands, a server command, and one
configuration command:

* `devmap init`
* `devmap analyze`
* `devmap onboarding`
* `devmap map`
* `devmap flow`
* `devmap explain`
* `devmap mcp`
* `devmap doctor`
* `devmap config model`

Expand Down Expand Up @@ -441,6 +443,51 @@ The `<target>` argument is required.

---

## `devmap mcp`

Run DevMap as an MCP (Model Context Protocol) server over stdio, so AI agents
can query the project map with tool calls. Full reference: [mcp.md](./mcp.md).

### Purpose

> Let an agent ask "what does this feature touch?" or "what breaks if I change
> this file?" without reading the whole repository first.

### Usage

```bash
devmap mcp # serve the project in the current folder
devmap mcp --root <path> # serve another project
devmap mcp --auto-refresh # rebuild a stale or missing snapshot before answering
devmap mcp --allow-ai # let analysis use the AI provider; adds devmap_explain
```

Clients start it for you; see [mcp.md](./mcp.md#client-setup) for Claude Code,
Cursor, opencode and Codex.

### Responsibilities

* Serve tools over stdio: `devmap_overview`, `devmap_map`, `devmap_flow`,
`devmap_routes`, `devmap_impact`, `devmap_find`, `devmap_analyze`, and
`devmap_explain` (only with `--allow-ai`)
* Serve resources (`devmap://snapshot`, `devmap://devmap-md`,
`devmap://feature/{id}`) and prompts (`devmap_onboard`, `devmap_plan_change`)
* Answer from `.devmap/snapshot.json`; every result reports `snapshot.stale`
* Never print to stdout except the protocol stream; logs go to stderr

### Rules

* Read-only, except `devmap_analyze` (and `--auto-refresh`), which write only
inside `.devmap/`
* Static by default: no AI provider is contacted, and the AI config is not read,
unless `--allow-ai` is set
* `--auto-refresh` is off by default: a question never writes to disk
* Analysis takes `.devmap/analyze.lock` so two servers do not write the snapshot
at once
* `devmap mcp` has no `--json` flag; its output is the protocol

---

## `devmap doctor`

Run diagnostics for DevMap setup.
Expand Down Expand Up @@ -630,6 +677,8 @@ Contract:
* progress sections and Markdown rendering are omitted
* AI responses are buffered instead of streamed
* runtime failures return a JSON object with `status`, `error`, and optional `hint`
* `devmap mcp` is the exception: it has no `--json` flag because stdout is the
MCP protocol stream
* `init --json` never prompts and therefore requires `GROQ_API_KEY`,
`OPENROUTER_API_KEY`, or an existing API key
* package-manager wrapper warnings may appear on stderr and are not part of the
Expand Down
43 changes: 42 additions & 1 deletion docs/for-me-personal/DEBUG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Catatan Debugging DevMap

Terakhir diperbarui: 2026-08-05
Terakhir diperbarui: 2026-10-30

Dokumen ini menyimpan masalah teknis yang pernah ditemukan selama development
DevMap. Tujuannya supaya penyebab, solusi, dan cara verifikasinya tidak perlu
Expand Down Expand Up @@ -1596,3 +1596,44 @@ Test ketiga (bahasa `id` via prompt palsu) sudah hermetic dan tidak diubah.
lingkungan runner.
- Durasi test yang melonjak (detik → menit) adalah sinyal test menunggu
input interaktif, bukan test yang lambat.
---

## 30. Semua Hasil MCP Melaporkan `stale: true` pada Snapshot yang Baru Dibuat

**Tanggal:** 2026-10-29

### Gejala

Tes `devmap_overview` untuk server MCP gagal: `snapshot.stale` bernilai `true`
padahal snapshot baru saja dibuat dari fixture yang sama dan tidak ada file yang
diubah.

### Akar Masalah

Helper tes menganalisis fixture di tempatnya
(`createProjectMap(test/fixtures/nextjs-project)`) lalu menyimpan snapshot ke
direktori temp lain (`saveSnapshot(tempRoot, snapshot)`). Pemeriksaan stale
memindai file di `projectRoot` yang diberikan ke server (direktori temp, yang
hanya berisi `.devmap/`), sehingga fingerprint-nya tidak pernah sama dengan
fingerprint di snapshot. Kodenya benar; fixture-nya yang tidak mewakili project
yang nyata.

### Solusi

- Salin fixture ke direktori temp dengan `cp(..., { recursive: true })`, lalu
jalankan `createProjectMap` dan `saveSnapshot` pada salinan itu.
- Tes baru yang butuh perubahan file (`addFile`) memakai project mini yang dibuat
langsung di direktori temp.

### Verifikasi

- `node --import tsx --test test/mcp-server.test.ts` → 13 pass / 0 fail.
- Tes `results flag a stale snapshot after the project changes` membuktikan arah
sebaliknya: `stale` berubah menjadi `true` hanya setelah file ditambahkan.

### Pelajaran

- Fingerprint snapshot terikat pada isi `projectRoot`. Tes yang membandingkan
snapshot dengan project harus menganalisis dan menyimpan di direktori yang sama.
- Jika semua hasil tiba-tiba `stale: true`, periksa dulu apakah snapshot dan project
yang dibandingkan memang folder yang sama sebelum mencurigai logika stale.
60 changes: 59 additions & 1 deletion docs/for-me-personal/PROGRESS.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,64 @@
# Progress DevMap

Terakhir diperbarui: 2026-10-03
Terakhir diperbarui: 2026-10-30

## Update 2026-10-30

### MCP Server (`devmap mcp`) — PR #130 s.d. #133, docs di branch `docs/mcp-server`

Tujuan: DevMap terasa "for AI agent" — agent bertanya ke peta proyek lewat tool
call, bukan membaca file context yang di-dump. Satu package, satu analyzer,
MCP hanya lapisan tipis di atas snapshot.

1. **Fase 0 (#130)** — `runMap`/`runFlow`/`runExplain` di-export; opsi `write`,
`narrate`; helper `loadSnapshot()` (`{ snapshot, stale }`).
2. **Fase 1 (#131)** — command `devmap mcp` (stdio, read-only): tool
`devmap_overview/map/flow/routes/impact/find`, resource
`devmap://snapshot|devmap-md|feature/{id}`. Dependency baru
`@modelcontextprotocol/sdk` dan `zod`.
3. **Fase 2 (#132)** — `devmap_analyze` (single-flight, lock
`.devmap/analyze.lock`, progress notification, status `running` setelah
timeout), `--auto-refresh` (default off), `devmap_impact` transitive
(`depth` 1–5), prompt `devmap_onboard` / `devmap_plan_change`,
`runAnalyze` di-export dengan opsi `offline`.
4. **Fase 3 (#133)** — `devmap_explain` (hanya terdaftar dengan `--allow-ai`),
`test/mcp-smoke.mjs` dijalankan di `ci.yml` dan `publish.yml` sebelum
`npm publish`, `mcpName: io.github.itsflaid/devmap` di `package.json`.
5. **Fase 4 (sedang jalan)** — `docs/mcp.md`, bagian MCP di README, `commands.md`,
`architecture.md` (bagian 15), `generated-files.md`, `docs/guide/commands/09-mcp.md`,
`CLAUDE.md`, `CHANGELOG.md` (Unreleased); lalu landing page.

**Keputusan penting**
- Nama command `devmap mcp`. `devmap_analyze` baru di Fase 2 (Fase 1 murni read-only).
- Auto-refresh default **off**.
- AI di MCP default **mati**, walaupun API key sudah di-set; hanya hidup dengan
`--allow-ai`. Alasan: yang memanggil tool adalah agent, bukan pemilik API key
(kuota dan metadata kode ke provider). `devmap_analyze` statis melaporkan
`aiDataDropped` kalau snapshot lama punya data AI.
- `server.json` dan bump versi **tidak** dimasukkan ke PR Fase 3: versi di
`server.json` harus sama dengan versi yang benar-benar terbit. Bump hanya
setelah konfirmasi eksplisit (aturan `AGENTS.md`).

**Verifikasi (Fase 3)**: `pnpm test:cli` → 482 pass / 0 fail, `pnpm build:cli`,
`node packages/cli/test/mcp-smoke.mjs`, `pnpm test:package-e2e`.

**Belum / sisa**
- Landing page (`apps/web`) + halaman Docs MCP.
- PR rilis: bump versi (tunggu konfirmasi level bump), `server.json`,
`packages/cli/README.md` (ikut tarball npm), tag + publish manual via OIDC,
lalu `mcp-publisher publish`.
- `devmap analyze` di terminal belum memakai `.devmap/analyze.lock`.
- Tes Windows/macOS untuk tes yang memakai proses sungguhan (stdio, lock,
smoke) baru akan tervalidasi oleh matriks CI; run CI terakhir yang terlihat
dari GitHub API bertanggal 2026-06 dan gagal (`Ubuntu / Node 20`), jadi
status CI perlu dicek langsung di tab Actions.

**Catatan proses**: commit PR #130–#133 memakai format `feat(cli): ...` dan trailer
co-author, padahal `AGENTS.md` meminta `<type>: <description>` tanpa scope dan
tanpa trailer bila `.agents/config.local.md` tidak ada. Sudah ter-merge, tidak
diubah; PR berikutnya mengikuti aturan.

---

## Update 2026-10-03

Expand Down
Loading
Loading