Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
- `packages/ragmir-core` provides `@jcode.labs/ragmir`: the `rgr` CLI, TypeScript library, MCP server, and portable skills.
- `packages/ragmir-chat` is the optional local GGUF synthesis add-on used by `rgr chat`.
- `packages/ragmir-tts` is the optional local/offline audio add-on used by `rgr audio`.
- Core must install and start without Chat or TTS. Keep both as optional peer integrations and load them only when their command is used.
- `packages/ragmir-landing` is a self-contained, telemetry-free Astro site. Keep it static, open-source focused, and free of vendor deployment configuration.
- Ragmir Core stays retrieval-first: `local-hash` supports offline retrieval, `transformers` is the explicit semantic option, and local chat remains a separate add-on.
- Long-running Node.js processes use one `RagmirClient` per project root and close it during shutdown. Keep the top-level API for one-shot scripts.
Expand Down
1 change: 1 addition & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ Use `pnpm --filter @jcode.labs/ragmir <script>` for Core-only work. The pinned N
- `packages/ragmir-core`: published CLI, library, MCP server, and skills.
- `packages/ragmir-chat`: optional local chat add-on.
- `packages/ragmir-tts`: optional audio add-on.
- Core must install and start without Chat or TTS. Keep both as optional peer integrations and load them only when their command is used.
- `packages/ragmir-landing`: self-contained static Astro documentation and product site.

Generated `dist/`, `.astro/`, `release-artifacts/`, and `.ragmir/` directories are ignored. Do not
Expand Down
11 changes: 7 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,11 @@

**Local RAG for your coding agents.**

Ragmir indexes the project files you choose, keeps the index on your machine, and gives coding
agents and local scripts bounded evidence with verifiable citations. Connect through project-scoped
agent skills, a local MCP server, the CLI, or the TypeScript API. The default path needs no account,
hosted document store, or model download.
Ragmir indexes the project files you choose on your machine and retrieves bounded, cited evidence
offline by default. The corpus and generated index remain local, so confidential source files are
not uploaded to a hosted RAG service. Connect through project-scoped agent skills, a local MCP
server, the CLI, or the TypeScript API. The default `local-hash` path needs no account, API key, or
model download.

Bring the coding agent or automation you already use. Ragmir Core retrieves evidence without
calling a model. If no retrieved passage may leave the machine, use a local consumer or Ragmir Chat
Expand Down Expand Up @@ -235,6 +236,8 @@ Redaction reduces accidental exposure but is not a compliance certification. Rev
| [`@jcode.labs/ragmir-chat`](./packages/ragmir-chat/README.md) | Optional cited generation with a local GGUF model |
| [`@jcode.labs/ragmir-tts`](./packages/ragmir-tts/README.md) | Optional local audio or explicit online voice rendering |

Installing Core does not install Chat or TTS. Add only the optional package needed by the workflow.

## Runnable examples

| Example | What it proves |
Expand Down
8 changes: 6 additions & 2 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,9 @@ or private document disclosure.

## Data Boundary

Ragmir is designed to index local project documents. Raw project documents, `.ragmir/`,
environment files, credentials, and customer records must remain outside commits.
Ragmir indexes selected project documents on the user's machine. The default `local-hash` path keeps
ingestion and retrieval offline, and Core does not upload the corpus to a hosted RAG service. A cloud
consumer can still receive returned passages when the user explicitly chooses that handoff.

Raw project documents, `.ragmir/`, environment files, credentials, and customer records must remain
outside commits.
2 changes: 1 addition & 1 deletion context7.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "https://context7.com/schema/context7.json",
"projectTitle": "Ragmir",
"description": "Local RAG for coding agents and scripts, with verifiable citations. The TypeScript CLI, library, read-focused MCP server, and portable skills keep the index under ignored local state. Optional local Chat and TTS remain separate packages.",
"description": "Local RAG for coding agents and scripts, with offline cited retrieval. The TypeScript CLI, library, read-focused MCP server, and portable skills keep the corpus and index under ignored local state. Optional local Chat and TTS remain separate packages.",
"excludeFolders": [
"node_modules",
"packages/ragmir-core/dist",
Expand Down
5 changes: 3 additions & 2 deletions docs/agent-integration.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# Agent integration

Ragmir gives the AI or automation you choose cited local passages through CLI or one stdio MCP
server. Core is model-agnostic and never calls a model itself.
Ragmir indexes the selected project files locally and gives the AI or automation you choose cited
passages through CLI or one stdio MCP server. The default `local-hash` path keeps ingestion and
retrieval offline. Core is model-agnostic, never uploads the corpus, and never calls a model itself.

Choose the handoff that matches the corpus:

Expand Down
18 changes: 15 additions & 3 deletions docs/api-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,9 @@ Ragmir publishes three ESM packages for Node.js 20 or later:

Use the CLI or MCP server when an agent or automation only needs to retrieve evidence. Use these
APIs when a Node.js process owns the workflow. All paths resolve from `cwd` or the current working
directory, and generated state stays under the project's ignored `.ragmir/` directory.
directory, and generated state stays under the project's ignored `.ragmir/` directory. With the
default `local-hash` provider, Core indexes and retrieves private project files locally and
offline. Only passages a caller explicitly hands to an external consumer cross that boundary.

## Core: cited retrieval

Expand Down Expand Up @@ -279,6 +281,15 @@ import { doctor, renderSpeech } from "@jcode.labs/ragmir-tts"
const runtime = await doctor()
console.log(runtime.transformersAvailable)

await renderSpeech({
cwd: process.cwd(),
text: "Non-sensitive model preload text.",
outputPath: "/tmp/ragmir-tts-preload.wav",
engine: "transformers",
language: "en",
allowRemoteModels: true,
})

const result = await renderSpeech({
cwd: process.cwd(),
textFile: ".ragmir/reports/release-brief.md",
Expand All @@ -292,8 +303,9 @@ console.log(result.outputPath, result.samplingRate)
```

TTS renders text supplied by the caller. It does not retrieve evidence or write a summary. The
default Transformers.js path produces local WAV output after an explicit model preload. The Edge
path is explicit and sends narration text to the external service.
first call explicitly preloads the local model from non-sensitive text. Later calls can keep
`allowRemoteModels: false` for confidential content. The Edge path is explicit and sends narration
text to the external service.

### TTS runtime exports

Expand Down
6 changes: 5 additions & 1 deletion docs/cli-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,8 @@ rgr models pull --enable
rgr ocr doctor
rgr ocr setup --language eng+fra
rgr chat setup --profile fast
printf '%s\n' "Non-sensitive model preload text." > /tmp/ragmir-tts-preload.txt
rgr audio /tmp/ragmir-tts-preload.txt --allow-remote-models --out .ragmir/audio/preload.wav
rgr audio ./brief.md --offline --out .ragmir/audio/brief.wav
```

Expand All @@ -77,7 +79,9 @@ rgr audio ./brief.md --offline --out .ragmir/audio/brief.wav
| `audio <file>` | Render text with the optional TTS add-on. |

OCR runs only for PDF pages without embedded text. The strict privacy profile disables external
extractors. Normal chat and offline audio rendering do not download models.
extractors. The first audio command above explicitly downloads the model from non-sensitive text;
the second uses the prepared cache and does not download anything. See the
[offline TTS guide](./offline-tts-preload.md) for model paths and verification.

## Agents, maintenance, and JSON

Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"name": "jcode-ragmir",
"version": "0.4.12",
"private": true,
"description": "Monorepo for Ragmir, local RAG with verifiable citations for coding agents and scripts, plus optional local Chat and TTS.",
"description": "Monorepo for Ragmir, local RAG with offline cited retrieval for coding agents and scripts, plus optional local Chat and TTS.",
"type": "module",
"license": "MIT",
"author": "Jean-Baptiste Thery (https://github.com/jb-thery)",
Expand Down
8 changes: 4 additions & 4 deletions packages/ragmir-chat/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@

**Optional local answers for your coding-agent RAG workflow.**

Ragmir Core retrieves cited project evidence for your coding agent or local script.
`@jcode.labs/ragmir-chat` adds an optional answer step on the same machine: it runs a verified local
model, limits generation to the retrieved passages, and validates the source markers in the visible
answer.
Ragmir Core indexes selected project files on your machine and retrieves cited evidence offline by
default, without uploading the source corpus to a hosted RAG service. `@jcode.labs/ragmir-chat` adds
an optional answer step on the same machine: it runs a verified local model, limits generation to
the retrieved passages, and validates the source markers in the visible answer.

Core does not require Chat or any generative model. Use this package only when the answer itself,
not just retrieval, must stay on the workstation.
Expand Down
10 changes: 5 additions & 5 deletions packages/ragmir-core/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@

**Local RAG for your coding agents.**

`@jcode.labs/ragmir` indexes the project files you select, keeps the index on your machine, and
returns bounded evidence with verifiable citations. Your coding agent or local script gets the
context it needs without a hosted document store. The default `local-hash` path works offline,
without an account or model download.
`@jcode.labs/ragmir` indexes the project files you select on your machine and retrieves bounded,
cited evidence offline by default. The corpus and generated index remain local, so confidential
source files are not uploaded to a hosted RAG service. Your coding agent or local script gets the
context it needs without an account, API key, or model download.

Core is model-agnostic. Connect the coding agent or local script you already use through generated
project skills, local MCP, or the JSON CLI. Use the typed Node.js API when your application owns the
Expand All @@ -34,7 +34,7 @@ control flow.
Core retrieves cited evidence. It does not require a generative model or generate answers. Use
[Ragmir Chat](https://www.npmjs.com/package/@jcode.labs/ragmir-chat) only when local GGUF synthesis
is useful, or [Ragmir TTS](https://www.npmjs.com/package/@jcode.labs/ragmir-tts) when you need audio
output.
output. Core declares both as optional peer integrations and does not install either add-on.

## Connect a coding agent in minutes

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ Ragmir.

Use this example when you want to prove more than "a relevant file appeared." Its six golden queries
check that Ragmir retrieves the expected path and the exact file, line, chunk, or PDF-page citation.
Every document is fictional and safe to commit.
Every document is fictional and safe to commit. The benchmark keeps its corpus and generated index
on the machine and runs the default `local-hash` retrieval offline.

## What it measures

Expand Down
3 changes: 2 additions & 1 deletion packages/ragmir-core/examples/library-api-demo/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ workflows.

Use this example when you are changing Ragmir Core or evaluating it as a library rather than as a
CLI. It imports the same package name an external consumer uses, runs the full retrieval loop, and
prints cited results from a fictional local corpus.
prints cited results from a fictional local corpus. The default `local-hash` path keeps the corpus
and generated index on the machine and performs retrieval offline.

## What it proves

Expand Down
7 changes: 4 additions & 3 deletions packages/ragmir-core/examples/sovereign-rag-demo/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,10 @@
A complete, fictional workspace for learning how agents and scripts retrieve cited local project
context with Ragmir.

Use this example to ingest several file types, search with citations, inspect privacy posture, find an
unsupported file, evaluate golden queries, and prepare agent access. It runs with offline
`local-hash` retrieval and contains no customer or production data.
Use this example to ingest several file types, search with citations, inspect privacy posture, find
an unsupported file, evaluate golden queries, and prepare agent access. It keeps the fictional
corpus and generated index on the machine, runs with offline `local-hash` retrieval, and contains no
customer or production data.

## Scenario

Expand Down
18 changes: 15 additions & 3 deletions packages/ragmir-core/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@jcode.labs/ragmir",
"version": "0.4.12",
"description": "Local RAG for coding agents, with cited project evidence through CLI, MCP, and TypeScript.",
"description": "Local RAG for coding agents, with offline cited retrieval through CLI, MCP, and TypeScript.",
"type": "module",
"license": "MIT",
"author": {
Expand Down Expand Up @@ -69,8 +69,6 @@
},
"dependencies": {
"@huggingface/transformers": "^4.2.0",
"@jcode.labs/ragmir-chat": "workspace:*",
"@jcode.labs/ragmir-tts": "workspace:*",
"@lancedb/lancedb": "^0.30.0",
"@modelcontextprotocol/sdk": "^1.29.0",
"commander": "^14.0.2",
Expand All @@ -84,7 +82,21 @@
"yaml": "^2.9.0",
"zod": "^4.1.13"
},
"peerDependencies": {
"@jcode.labs/ragmir-chat": "workspace:*",
"@jcode.labs/ragmir-tts": "workspace:*"
},
"peerDependenciesMeta": {
"@jcode.labs/ragmir-chat": {
"optional": true
},
"@jcode.labs/ragmir-tts": {
"optional": true
}
},
"devDependencies": {
"@jcode.labs/ragmir-chat": "workspace:*",
"@jcode.labs/ragmir-tts": "workspace:*",
"@types/html-to-text": "^9.0.4",
"@types/node": "^24.10.1",
"publint": "^0.3.21",
Expand Down
14 changes: 10 additions & 4 deletions packages/ragmir-core/src/cli-options.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
import { isTtsLanguage, TTS_LANGUAGES, type TtsLanguage } from "@jcode.labs/ragmir-tts"
import type { AgentInstallMode, AgentInstallScope } from "./skill.js"

/**
Expand All @@ -12,6 +11,9 @@ import type { AgentInstallMode, AgentInstallScope } from "./skill.js"

export type AudioEngine = "auto" | "edge" | "transformers"

export const SUPPORTED_AUDIO_LANGUAGES = ["en", "es", "fr", "ja", "th", "zh"] as const
export type AudioLanguage = (typeof SUPPORTED_AUDIO_LANGUAGES)[number]

export interface AudioOptions {
out?: string
engine?: string
Expand Down Expand Up @@ -78,14 +80,18 @@ export function audioAllowRemoteModels(options: AudioOptions): boolean | undefin
* Resolve the spoken language from `--lang`. Throws on an unsupported value so
* the operator is told which languages are available.
*/
export function audioLanguage(options: AudioOptions): TtsLanguage | undefined {
export function audioLanguage(options: AudioOptions): AudioLanguage | undefined {
if (options.lang === undefined) {
return undefined
}
if (isTtsLanguage(options.lang)) {
if (isSupportedAudioLanguage(options.lang)) {
return options.lang
}
throw new Error(`Expected --lang to be one of: ${TTS_LANGUAGES.join(", ")}.`)
throw new Error(`Expected --lang to be one of: ${SUPPORTED_AUDIO_LANGUAGES.join(", ")}.`)
}

function isSupportedAudioLanguage(value: string): value is AudioLanguage {
return SUPPORTED_AUDIO_LANGUAGES.some((language) => language === value)
}

/**
Expand Down
6 changes: 6 additions & 0 deletions packages/ragmir-core/src/cli.test.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import { TTS_LANGUAGES } from "@jcode.labs/ragmir-tts"
import { describe, expect, it } from "vitest"
import {
audioAllowRemoteModels,
Expand All @@ -8,6 +9,7 @@ import {
parseNumber,
parsePositiveInt,
parseRecallThreshold,
SUPPORTED_AUDIO_LANGUAGES,
} from "./cli-options.js"

describe("parsePositiveInt", () => {
Expand Down Expand Up @@ -72,6 +74,10 @@ describe("audioAllowRemoteModels", () => {
})

describe("audioLanguage", () => {
it("matches the optional TTS package language contract", () => {
expect(SUPPORTED_AUDIO_LANGUAGES).toEqual(TTS_LANGUAGES)
})

it("returns undefined when no language is provided", () => {
expect(audioLanguage({})).toBeUndefined()
})
Expand Down
37 changes: 33 additions & 4 deletions packages/ragmir-core/src/cli.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
#!/usr/bin/env node
import path from "node:path"
import type { ChatSource } from "@jcode.labs/ragmir-chat"
import type { TtsLanguage } from "@jcode.labs/ragmir-tts"
import { Command } from "commander"
import pc from "picocolors"
import { accessLogUsageReport } from "./access-log.js"
import {
type AudioLanguage,
audioAllowRemoteModels,
audioEngine,
audioLanguage,
Expand Down Expand Up @@ -1568,7 +1568,7 @@ interface TtsRenderOptions {
textFile: string
outputPath?: string
engine: "auto" | "edge" | "transformers"
language?: TtsLanguage
language?: AudioLanguage
model?: string
modelPath?: string
allowRemoteModels?: boolean
Expand All @@ -1579,7 +1579,17 @@ interface TtsRenderOptions {
}

async function loadChat(): Promise<ChatModule> {
const module: unknown = await import(CHAT_PACKAGE_NAME)
let module: unknown
try {
module = await import(CHAT_PACKAGE_NAME)
} catch (error) {
if (isMissingOptionalPackage(error, CHAT_PACKAGE_NAME)) {
throw new Error(
`Ragmir Chat is optional and is not installed. Add ${CHAT_PACKAGE_NAME} to this project before using \`rgr chat\`.`,
)
}
throw error
}
if (!isChatModule(module)) {
throw new Error(`${CHAT_PACKAGE_NAME} did not expose the expected chat API.`)
}
Expand Down Expand Up @@ -1630,13 +1640,32 @@ function toChatSource(source: Awaited<ReturnType<typeof search>>[number]): ChatS
}

async function loadTts(): Promise<TtsModule> {
const module: unknown = await import(TTS_PACKAGE_NAME)
let module: unknown
try {
module = await import(TTS_PACKAGE_NAME)
} catch (error) {
if (isMissingOptionalPackage(error, TTS_PACKAGE_NAME)) {
throw new Error(
`Ragmir TTS is optional and is not installed. Add ${TTS_PACKAGE_NAME} to this project before using \`rgr audio\`.`,
)
}
throw error
}
if (!isTtsModule(module)) {
throw new Error(`${TTS_PACKAGE_NAME} did not expose the expected TTS API.`)
}
return module
}

function isMissingOptionalPackage(error: unknown, packageName: string): boolean {
return (
error instanceof Error &&
"code" in error &&
error.code === "ERR_MODULE_NOT_FOUND" &&
error.message.includes(packageName)
)
}

function isTtsModule(value: unknown): value is TtsModule {
return (
typeof value === "object" &&
Expand Down
Loading