Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "code-context",
"description": "Local hybrid keyword + semantic search and SQL over your codebase, for coding agents.",
"version": "0.4.0",
"version": "0.5.0",
"author": { "name": "Infino AI" },
"homepage": "https://github.com/infino-ai/code-context"
}
6 changes: 3 additions & 3 deletions .devin/wiki.json
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
{
"repo_notes": [
{
"content": "code-context is local code search for AI coding agents: a CLI (bins `code-context` and `cx`) and an MCP server over a ranked index that lives in plain files inside the repo, under `.infino/`. It fuses exact keyword matching (BM25) with semantic similarity (vectors) into one ranked pass, and exposes read-only SQL over the index so search composes with aggregation (`GROUP BY`). Drop it into Claude Code with `claude mcp add code-context -- npx -y @infino-ai/code-context mcp`, or install the CLI with `npm install -g @infino-ai/code-context` then `cx index` in a repo. Document it as a tool developers and coding agents use to answer questions about a codebase without crawling files into the context window: lead with the MCP setup, the three MCP tools (search, sql, reindex), and how the index stays fresh."
"content": "code-context is local code search for AI coding agents: a CLI (bins `code-context` and `cx`) and an MCP server over a ranked index that lives in plain files inside the repo, under `.infino/`. It fuses exact keyword matching (BM25) with semantic similarity (vectors) into one ranked pass, and exposes read-only SQL over the index so search composes with aggregation (`GROUP BY`). Drop it into Claude Code with `claude mcp add code-context -- npx -y @infino-ai/code-context mcp`, or install the CLI with `npm install -g @infino-ai/code-context` then `cx index` in a repo. Document it as a tool developers and coding agents use to answer questions about a codebase without crawling files into the context window: lead with the MCP setup, the three MCP tools (find, search, sql), and how the index stays fresh on its own (first query builds it, every query re-syncs it; there is no reindex tool)."
},
{
"content": "Common misconceptions to avoid when documenting code-context, with the corrected statements: (1) It is NOT a cloud service and needs NO account, API key, or database server. It runs locally; embedding uses a small local model downloaded once, and code never leaves the machine. (2) It is NOT semantic/vector search only. Search is hybrid: BM25 and vector rankings fuse in one pass, and until vectors finish backfilling it degrades to keyword-only rather than failing. (3) The index is NOT a proprietary database tier. It is plain files in `.infino/` that you can copy, cache in CI, or gitignore. (4) It is NOT a structural code-intelligence tool. It does ranked content retrieval and content-relevance aggregation, not call-graph tracing, dead-code detection, or type resolution; graph/LSP tools are complementary and stack alongside it. (5) It is for questions that span the repo (understanding a subsystem, ranking or aggregating across files); for jumping to one known symbol a plain grep is already cheap."
"content": "Common misconceptions to avoid when documenting code-context, with the corrected statements: (1) It is NOT a cloud service and needs NO account, API key, or database server. It runs locally; embedding uses a small local model downloaded once, and code never leaves the machine. (2) It is NOT semantic/vector search only. Search is hybrid: BM25 and vector rankings fuse in one pass, and until vectors finish backfilling it degrades to keyword-only rather than failing. (3) The index is NOT a proprietary database tier. It is plain files in `.infino/` that you can copy, cache in CI, or gitignore. (4) It is NOT a structural code-intelligence tool. It does ranked content retrieval and content-relevance aggregation, not call-graph tracing, dead-code detection, or type resolution; graph/LSP tools are complementary and stack alongside it. (5) It is NOT only for questions that span the repo. Those (understanding a subsystem, ranking or aggregating across files) are where it saves the most, but the grep case - every occurrence of a known symbol or literal - is served by the find tool from the same index, complete and unranked, with no file scanned."
},
{
"content": "How it works: files are chunked at definition boundaries with tree-sitter (WASM, no native compile) for common languages, with a fixed-window fallback for the rest; every chunk carries path, start_line, end_line, lang, and content. Chunks are indexed into one table named `chunks` in `.infino/`, with a BM25 full-text index and an IVF vector index, queried in-process through the infino engine's Node binding (no server). Readiness is staged: the keyword index commits in seconds so search is live immediately, while vectors backfill in the background and hybrid ranking unlocks automatically when they land. Sync is incremental: a per-file state map (size/mtime prefilter, then content hash) re-chunks and re-embeds only changed files, and the MCP server auto-syncs in the background as queries arrive. The MCP surface is exactly three tools by design: search (hybrid ranked retrieval), sql (read-only SELECT/WITH, with the ranked search functions bm25_search/hybrid_search usable as table-valued relations so search composes with GROUP BY aggregation), and reindex (incremental sync). code-context is built on the infino engine (https://github.com/infino-ai/infino), whose same index format also serves logs, docs, and agent memory."
"content": "How it works: files are chunked at definition boundaries with tree-sitter (WASM, no native compile) for common languages, with a fixed-window fallback for the rest; every chunk carries path, start_line, end_line, lang, and content. Chunks are indexed into one table named `chunks` in `.infino/`, with a BM25 full-text index and an IVF vector index, queried in-process through the infino engine's Node binding (no server). Readiness is staged: the keyword index commits in seconds so search is live immediately, while vectors backfill in the background and hybrid ranking unlocks automatically when they land. Sync is incremental: a per-file state map (size/mtime prefilter, then content hash) re-chunks and re-embeds only changed files, and the MCP server auto-syncs in the background as queries arrive. The MCP surface is exactly three tools by design: find (every line containing an exact string, cited path:line; the index's token match picks candidate chunks and each line is checked for the literal, so it is complete and unranked like grep -n), search (hybrid ranked retrieval), and sql (read-only SELECT/WITH, with the ranked search functions bm25_search/hybrid_search usable as table-valued relations so search composes with GROUP BY aggregation). Freshness is not a tool: the first query builds the index and every query re-syncs it; cx index --full rebuilds from a shell. code-context is built on the infino engine (https://github.com/infino-ai/infino), whose same index format also serves logs, docs, and agent memory."
}
]
}
2 changes: 1 addition & 1 deletion .mcp.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"mcpServers": {
"code-context": {
"command": "npx",
"args": ["-y", "@infino-ai/code-context@0.4.0", "mcp"],
"args": ["-y", "@infino-ai/code-context@0.5.0", "mcp"],
"alwaysLoad": true
}
}
Expand Down
24 changes: 17 additions & 7 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,17 +24,18 @@ the honest limits in [docs/tradeoffs.md](docs/tradeoffs.md).
## Repo map

- `src/cli.ts`: the `cx` / `code-context` command entry (commander).
- `src/mcp/server.ts`: the MCP server, three tools (`search`, `sql`,
`reindex`). Each takes an optional `path` (repo root) so one server serves
multiple repos in a session, defaulting to the startup root.
- `src/mcp/server.ts`: the MCP server, three tools (`find`, `search`, `sql`).
Each takes an optional `path` (repo root) so one server serves multiple
repos in a session, defaulting to the startup root. Freshness is not a
tool: the first query builds the index and every query re-syncs it.
- `src/mcp/repos.ts`: the per-repo registry - resolves and validates a
requested root, one engine connection per repo, LRU-capped.
- `src/mcp/ensure.ts`: auto-index on first query - a `search`/`sql` on a
never-indexed repo builds the index inline, then answers on the same call
(`CX_AUTO_INDEX=0` restores the strict "index it first" error).
- `src/core/`: the engine-facing core. `chunker` (tree-sitter chunking),
`indexer` (build + staged readiness + incremental sync), `searcher`
(hybrid search + SQL), `embedder` (local model), `filestate` (incremental
(find, hybrid search, SQL), `embedder` (local model), `filestate` (incremental
sync state), `walker`, `manifest`, `config`, `context`, `output`.
- `src/commands/`: CLI command implementations (`index-cmd`, `query-cmds`).
- `test/`: vitest suites. `bench/`: the benchmark harness. `docs/`: docs.
Expand All @@ -53,11 +54,20 @@ before opening a PR.
## Conventions

- TypeScript, ES modules. Every source file carries an SPDX header.
- The MCP surface is deliberately three tools: one way to find (`search`),
one way to count (`sql`), one way to stay fresh (`reindex`). Adding
near-duplicate retrieval tools worsens an agent's tool selection; resist it.
- The MCP surface is deliberately three tools, one per question: where does
this exact text occur (`find`, unranked and complete - the grep
replacement), what is most relevant (`search`, ranked top-k), how much of
what is where (`sql`). Adding near-duplicate retrieval tools worsens an
agent's tool selection; resist it. A new tool must answer a question none
of these three does. A `reindex` tool was the fourth until it was measured
(docs/benchmark.md, "The tool surface"): no Sonnet run called it, Haiku
called it where it hurt, and auto-sync already does the job.
- Search results carry chunk content plus `path:line` ranges so answers cite
code; keep that contract when touching `searcher` or the tool descriptions.
- Tool descriptions and server instructions are prompt text on every turn
and were measured to steer tool selection sentence by sentence. Change
them with the bench (`bench/`, the four question sets and
`compare-builds.mjs`), not by taste.

## Boundaries

Expand Down
67 changes: 48 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,10 +75,10 @@ servers - where clients defer tool definitions behind a tool-search step - the
agent doesn't miss the index and fall back to plain file search. (Use *either*
the plugin or this command, not both.)

Then just ask a question about the code. The first `search` or `sql` on an
unindexed repo builds the index inline and answers on the same call: keyword
Then just ask a question about the code. The first `find`, `search`, or `sql`
on an unindexed repo builds the index inline and answers on the same call: keyword
search is live in seconds, and vectors backfill in the background. (Prefer to
kick it off yourself? The `reindex` tool does the same build on demand.)
kick it off yourself? `cx index` does the same build from a shell.)

CI-tested on Linux x64 (glibc) and macOS arm64; linux-arm64, musl, and
Windows-via-WSL are expected to work through the engine's prebuilt bindings
Expand Down Expand Up @@ -108,6 +108,20 @@ explore less efficiently, so the savings tend to be **larger** there. On
pinpoint symbol lookup, where a single grep is already cheap, an index
matches file tools rather than beating them.

Adding `find` was measured the same way, against the three-tool build on the
same repo, questions, model, and a blind judge: answer quality level (judge
29 / 22 / 13 main / find / tie over 64 pairs, no out-of-bounds citation in
128 answers), exact-lookup questions **-35% tokens, -17% dollars, -38% tool
calls**, the shipped question set flat (-3% tokens, +1% dollars), and about a
thousand tokens per turn of added prompt for the fourth tool.

The tool surface itself was then measured lever by lever - names, result
shapes, and every sentence of description - on two models with a blind
judge: the shipped text is the one that kept selection where it was, cut the
per-turn prompt cost of the tool definitions by more than half, and judged
51 to 33 over the previous surface. That run is also why there are three
tools and not four.

Full methodology and per-question tables are in
[docs/benchmark.md](docs/benchmark.md), with the harness in
[`bench/`](bench/) so you can run the same lanes on your own repo.
Expand All @@ -118,14 +132,18 @@ One index and a deliberately small tool surface for agents:

| Tool | What it does | When agents use it |
|---|---|---|
| `find` | Every line containing an exact string, cited `path:line` like `grep -n`, plus matching lines per file like `grep -c`. Complete and unranked: the index's token match picks the candidate chunks, then each line is checked for the literal, so no file is scanned and every hit is a real occurrence. | Where an agent would grep: every use or definition of an identifier, an error message, a config key. |
| `search` | One ranked pass fusing exact keyword matching (BM25) with semantic similarity (reciprocal-rank fusion). Hits carry the chunk content, so answers come straight from results. | A strong default for finding and understanding code: how a subsystem works, code by meaning or exact term, context before a change, similar implementations - exact identifiers and paraphrases in the same call. |
| `sql` | Read-only SQL over the index, with the ranked search functions (`bm25_search`/`hybrid_search`) usable as table-valued relations. | Counts, rankings, aggregates over the whole repo in one query. |
| `reindex` | Incremental sync (the server also auto-syncs in the background). | After significant edits. |

Three tools is a deliberate design: one way to find, one way to count, one
way to stay fresh. Every additional near-duplicate retrieval tool worsens an
agent's tool selection, and hybrid search's keyword half already ranks
exact identifier terms highly, so a separate lexical tool has no job left.
Three tools, each a different question: where does this exact text occur,
what is most relevant to this, how much of what is where. Freshness is not
a tool: the first query on an unindexed repo builds the index, every query
re-syncs it against the working tree, and `cx index --full` rebuilds from a
shell. There are no near-duplicate retrieval tools, because those worsen an
agent's tool selection: `find` is unranked and complete where `search` is
ranked and top-k, and hybrid search's keyword half already ranks exact
identifier terms highly, so no separate lexical *ranking* tool exists.

### The SQL move

Expand Down Expand Up @@ -164,7 +182,7 @@ export and pass around.
## Setup for agents

code-context is an MCP server over stdio, so any MCP client works. Register
it once and the tools (`search`, `sql`, `reindex`) become available to the
it once and the tools (`find`, `search`, `sql`) become available to the
agent.

<details>
Expand Down Expand Up @@ -252,9 +270,9 @@ when the client's working directory is not the repo.

</details>

Tools: `search`, `sql`, `reindex` (incremental sync: an unchanged repo is
a fast no-op, and the server also auto-syncs in the background as queries
arrive, so results track your edits without anyone asking).
Tools: `find`, `search`, `sql`. The server auto-syncs in the background as
queries arrive (an unchanged repo is a fast no-op), so results track your
edits without anyone asking; `cx index --full` from a shell forces a rebuild.

**Multiple repos in one session.** Each tool takes an optional `path` (an
absolute repo root). Omit it and the server uses its startup root; set it to
Expand All @@ -268,15 +286,16 @@ no restart, no per-repo config.
|---|---|---|
| `CX_INDEX_DIR` | `<repo>/.infino` | where the index lives |
| `CX_SEARCH_K` | 10 | default number of hits `search` returns (also settable per call and via the CLI `-k` flag) |
| `CX_MAX_FILES` / `CX_MAX_FILE_BYTES` | 20000 / 1MB | indexing caps (files over the file cap are left out; `search`/`sql` then flag the index as partial so an absence isn't read as proof) |
| `CX_FIND_LIMIT` | 500 | default number of matching lines `find` returns, which is also the hard cap, so it only bites on a flood (also settable per call and via the CLI `--limit` flag); `total` and `byFile` are complete either way |
| `CX_MAX_FILES` / `CX_MAX_FILE_BYTES` | 20000 / 1MB | indexing caps (files over the file cap are left out; `find`/`search`/`sql` then flag the index as partial so an absence isn't read as proof) |
| `CX_ROOT` | current directory | default repo root for the MCP server / CLI when not run from the repo (each tool call can override it with a `path` argument) |
| `CX_AUTO_INDEX` | on | `0` makes a query on an unindexed repo error instead of building the index inline on the first `search`/`sql` |
| `CX_AUTO_INDEX` | on | `0` makes a query on an unindexed repo error instead of building the index inline on the first `find`/`search`/`sql` |
| `CX_AUTO_SYNC` | on | `0` disables the MCP server's background staleness sync |
| `CX_SYNC_INTERVAL_SECS` | 30 | auto-sync debounce between staleness checks |
| `CX_NO_EMBED` | off | keyword-only mode for the MCP server (skip the vector stage) |
| `CX_NO_RECEIPT` | off | `1` turns off usage accounting - the per-call receipt on results and the `cx usage` ledger |

Every `search` / `sql` result carries a **usage receipt** - a terse, local line
Every `find` / `search` / `sql` result carries a **usage receipt** - a terse, local line
showing the tokens it returned, the files it spanned, and a running session
total (e.g. `returned ~1.2k tokens | 4 chunks / 3 files | session ~8.4k over 7
queries`). Every figure is a `~` estimate, computed in-process - nothing about
Expand All @@ -294,17 +313,19 @@ npm install -g @infino-ai/code-context

```
cx index [path] sync the index (incremental; --full rebuilds, --watch follows edits)
cx find <text> every line containing the exact text, path:line (-i, -c per-file counts, --limit)
cx search <query> exact terms + meaning, one ranked pass (-k hits)
cx sql <statement> read-only SQL; --embed q="text" fills {{q}}
cx status what the index holds, how fresh, vector readiness
cx usage ledger of queries run and what each returned (-n, --all, --clear, --json)
cx mcp serve the MCP tools over stdio
```

`cx usage` reads the local ledger at `.infino/usage.jsonl` - every `search` /
`sql` (from the CLI or the MCP server) appends one line recording the query and
a compact summary of what came back (paths and line ranges for search, row
count for sql), plus the token figures from the receipt. It's a deterministic,
`cx usage` reads the local ledger at `.infino/usage.jsonl` - every `find` /
`search` / `sql` (from the CLI or the MCP server) appends one line recording the
query and a compact summary of what came back (`path:line` for find, paths and
line ranges for search, row count for sql), plus the token figures from the
receipt. It's a deterministic,
model-independent view of what went through the index - no running server or
agent needed to read it back. `CX_NO_RECEIPT=1` turns off both the inline
receipt and this ledger.
Expand Down Expand Up @@ -335,6 +356,14 @@ to your Claude Code settings (`~/.claude/settings.json` or a project
and prints nothing. If you run code-context via `npx`, use
`npx -y @infino-ai/code-context usage --hook` as the command.

The same tally breaks the calls down by tool (`by tool: find 4 · search 2 ·
sql 1`) and records which tool the agent reached for first in each prompt
(`first tool of a prompt: find 4 · Grep 2`), which is what tells you whether
the tool surface steers as intended. With the matcher above only
code-context's own tools are forwarded, so the first-tool line names them
alone; set the `PostToolUse` matcher to `.*` to see Grep, Read, and the rest
in that line too, at the cost of one hook process per tool call.

## What it is, and what it isn't

code-context's lane is ranked **content** retrieval and content-relevance
Expand Down
Loading