Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 2 additions & 23 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,29 +39,8 @@ jobs:
- name: Install dependencies
run: pnpm install --frozen-lockfile

- name: Security dependency audit
run: pnpm audit:security

- name: Lint and format check
run: pnpm lint

- name: Typecheck
run: pnpm check

- name: Test
run: pnpm test

- name: Build
run: pnpm build

- name: Smoke test production CLI and MCP
run: pnpm smoke

- name: Verify npm package metadata
run: pnpm package:check

- name: Generate release verification artifacts
run: pnpm release:artifacts
- name: Validate repository
run: pnpm validate

commitlint:
name: Commitlint
Expand Down
12 changes: 10 additions & 2 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,24 @@ permissions:

jobs:
analyze:
name: Analyze TypeScript
name: Analyze ${{ matrix.label }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- language: javascript-typescript
label: TypeScript
- language: python
label: Python
steps:
- name: Checkout
uses: actions/checkout@v6

- name: Initialize CodeQL
uses: github/codeql-action/init@v4
with:
languages: javascript-typescript
languages: ${{ matrix.language }}

- name: Autobuild
uses: github/codeql-action/autobuild@v4
Expand Down
8 changes: 0 additions & 8 deletions .github/workflows/npm-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,6 @@ jobs:
environment:
name: npm-publish
url: https://www.npmjs.com/package/@jcode.labs/ragmir
outputs:
version: ${{ steps.version.outputs.version }}
steps:
- name: Verify main branch
run: test "${GITHUB_REF}" = "refs/heads/main"
Expand Down Expand Up @@ -62,9 +60,3 @@ jobs:
with:
name: ragmir-release-${{ github.sha }}
path: release-artifacts/

- name: Resolve released version
id: version
run: |
VERSION=$(git describe --tags --abbrev=0 2>/dev/null | sed 's/^v//' || echo "")
echo "version=$VERSION" >> "$GITHUB_OUTPUT"
8 changes: 4 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,26 +57,26 @@ The repository is indexed as `jcode-ragmir`.
- Use `gitnexus_rename` for symbol renames. If the index is stale, run `npx gitnexus analyze` before relying on it.

<!-- gitnexus:start -->
# GitNexus Code Intelligence
# GitNexus: Code Intelligence

This project is indexed by GitNexus as **jcode-ragmir** (2005 symbols, 4448 relationships, 165 execution flows). Use the GitNexus MCP tools to understand code, assess impact, and navigate safely.

> Index stale? Run `node .gitnexus/run.cjs analyze` from the project root — it auto-selects an available runner. No `.gitnexus/run.cjs` yet? `npx gitnexus analyze` (npm 11 crash → `npm i -g gitnexus`; #1939).
> Index stale? Run `node .gitnexus/run.cjs analyze` from the project root. It auto-selects an available runner. No `.gitnexus/run.cjs` yet? `npx gitnexus analyze` (npm 11 crash → `npm i -g gitnexus`; #1939).

## Always Do

- **MUST run impact analysis before editing any symbol.** Before modifying a function, class, or method, run `impact({target: "symbolName", direction: "upstream"})` and report the blast radius (direct callers, affected processes, risk level) to the user.
- **MUST run `detect_changes()` before committing** to verify your changes only affect expected symbols and execution flows. For regression review, compare against the default branch: `detect_changes({scope: "compare", base_ref: "main"})`.
- **MUST warn the user** if impact analysis returns HIGH or CRITICAL risk before proceeding with edits.
- When exploring unfamiliar code, use `query({search_query: "concept"})` to find execution flows instead of grepping. It returns process-grouped results ranked by relevance.
- When you need full context on a specific symbol callers, callees, which execution flows it participates in — use `context({name: "symbolName"})`.
- When you need full context on a specific symbol (callers, callees, and participating execution flows), use `context({name: "symbolName"})`.
- For security review, `explain({target: "fileOrSymbol"})` lists taint findings (source→sink flows; needs `analyze --pdg`).

## Never Do

- NEVER edit a function, class, or method without first running `impact` on it.
- NEVER ignore HIGH or CRITICAL risk warnings from impact analysis.
- NEVER rename symbols with find-and-replace — use `rename` which understands the call graph.
- NEVER rename symbols with find-and-replace. Use `rename`, which understands the call graph.
- NEVER commit changes without running `detect_changes()` to check affected scope.

## Resources
Expand Down
11 changes: 6 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ pnpm example
```

Use `pnpm --filter @jcode.labs/ragmir <script>` for Core-only work. The pinned Node version lives in
`mise.toml`; activate mise in your shell or use any compatible Node 20+ runtime locally.
`mise.toml`; activate mise in your shell or run local workspace commands with that pinned version.
Published packages retain their documented Node.js 20 compatibility.

## Workspace

Expand Down Expand Up @@ -38,26 +39,26 @@ close it during shutdown. Ragmir does not provide an HTTP server or fixed port;
own transport security, authentication, authorization, and rate limits.

<!-- gitnexus:start -->
# GitNexus Code Intelligence
# GitNexus: Code Intelligence

This project is indexed by GitNexus as **jcode-ragmir** (2005 symbols, 4448 relationships, 165 execution flows). Use the GitNexus MCP tools to understand code, assess impact, and navigate safely.

> Index stale? Run `node .gitnexus/run.cjs analyze` from the project root — it auto-selects an available runner. No `.gitnexus/run.cjs` yet? `npx gitnexus analyze` (npm 11 crash → `npm i -g gitnexus`; #1939).
> Index stale? Run `node .gitnexus/run.cjs analyze` from the project root. It auto-selects an available runner. No `.gitnexus/run.cjs` yet? `npx gitnexus analyze` (npm 11 crash → `npm i -g gitnexus`; #1939).

## Always Do

- **MUST run impact analysis before editing any symbol.** Before modifying a function, class, or method, run `impact({target: "symbolName", direction: "upstream"})` and report the blast radius (direct callers, affected processes, risk level) to the user.
- **MUST run `detect_changes()` before committing** to verify your changes only affect expected symbols and execution flows. For regression review, compare against the default branch: `detect_changes({scope: "compare", base_ref: "main"})`.
- **MUST warn the user** if impact analysis returns HIGH or CRITICAL risk before proceeding with edits.
- When exploring unfamiliar code, use `query({search_query: "concept"})` to find execution flows instead of grepping. It returns process-grouped results ranked by relevance.
- When you need full context on a specific symbol callers, callees, which execution flows it participates in — use `context({name: "symbolName"})`.
- When you need full context on a specific symbol (callers, callees, and participating execution flows), use `context({name: "symbolName"})`.
- For security review, `explain({target: "fileOrSymbol"})` lists taint findings (source→sink flows; needs `analyze --pdg`).

## Never Do

- NEVER edit a function, class, or method without first running `impact` on it.
- NEVER ignore HIGH or CRITICAL risk warnings from impact analysis.
- NEVER rename symbols with find-and-replace — use `rename` which understands the call graph.
- NEVER rename symbols with find-and-replace. Use `rename`, which understands the call graph.
- NEVER commit changes without running `detect_changes()` to check affected scope.

## Resources
Expand Down
28 changes: 18 additions & 10 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Use the [issue templates](https://github.com/jcode-works/jcode-ragmir/issues/new
Confirm the feature stays compatible with Ragmir's local-first, zero-telemetry posture.

Before opening a new issue, search [existing issues](https://github.com/jcode-works/jcode-ragmir/issues)
to avoid duplicates. Security vulnerabilities must not be reported through public issues — follow
to avoid duplicates. Security vulnerabilities must not be reported through public issues. Follow
[`SECURITY.md`](./SECURITY.md).

## Development
Expand All @@ -26,19 +26,26 @@ version CI uses. Install mise, then run the single onboarding command:

```bash
pnpm bootstrap
pnpm validate
```

`pnpm bootstrap` runs `mise install && pnpm install`. Without mise, any Node.js 20+ and pnpm install
works too — just run `pnpm install` directly.
`pnpm bootstrap` runs `mise install && pnpm install`. Without mise, install the Node.js 22 release
pinned in `mise.toml` and pnpm, then run `pnpm install` directly. Published packages retain their
documented Node.js 20 runtime support; the repository toolchain itself requires Node.js 22.

Activate mise in your shell (`mise activate`, per the
[mise docs](https://mise.jdx.dev/getting-started.html)) so that entering this repository puts the
pinned Node on your `PATH` automatically. Then `pnpm dev:landing` and `pnpm example` run on the
same toolchain as CI without per-script wiring.

`pnpm validate` runs Biome, a dependency security audit, TypeScript, Vitest, the production CLI/MCP
smoke test, and npm package metadata checks.
Before opening a pull request, run:

```bash
pnpm validate
```

`pnpm validate` runs Biome, a dependency security audit, TypeScript checks, coverage-gated Vitest,
production CLI and MCP smoke tests, the static landing build, public API checks, and npm package and
release-artifact checks.

To smoke-test the library API against your local build while developing Ragmir Core, run
`pnpm example` (see
Expand All @@ -54,17 +61,18 @@ pnpm audit:security

- Branch from `develop` for normal feature work, using `feature/<short-name>`.
- Open feature pull requests against `develop`.
- Use `release/<version-or-topic>` branches from `develop` when preparing a production release, then
open the release pull request against `main`.
- Maintainers prepare `release/<version-or-topic>` from `main`, apply the validated `develop` tree,
and open the release pull request against `main`.
- Use `hotfix/<short-name>` branches from `main` for urgent production fixes, then back-merge the fix
into `develop`.
- Keep changes focused and include tests or smoke coverage for behavior changes.
- Do not commit private documents, generated vector stores, generated `.ragmir/` state, environment
files, tokens, credentials, or interview notes.
- Use conventional commit messages such as `feat: add source parser` or
`fix: handle empty index`.
- Non-release branches run CI only. npm publishing is restricted to the protected semantic-release
workflow from `main`; versions are derived from Conventional Commits, not manual package bumps.
- Non-release branches run CI only. npm publishing of Core, Chat, and TTS is restricted to the
protected semantic-release workflow from `main`; versions come from Conventional Commits, not
manual package bumps.

## Security

Expand Down
25 changes: 23 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@

**Confidential local RAG for your coding agents.**

*Stop sending confidential documents directly to the cloud.*

Ragmir indexes the project files you choose on your machine and retrieves bounded, cited evidence
offline by default. The corpus and generated index remain local, so confidential source files are
not uploaded to a hosted RAG service. Connect through project-scoped agent skills, a local MCP
Expand Down Expand Up @@ -106,7 +108,11 @@ connector.
The MCP surface is intentionally bounded and read-focused. Agents can request compact evidence
first, then expand one returned citation without opening a second index or reading arbitrary files.
MCP clients can read `ragmir://context` for a compact base, readiness, freshness, and capability
overview before choosing a tool.
overview before choosing a tool. Every tool advertises non-destructive behavior, and every tool and
resource JSON response is subject to a byte budget. Search, ask, research, and evaluation
conservatively advertise open-world behavior because explicitly enabled semantic models may
download public weights. Tools that may initialize local state or append metadata-only access logs
do not claim read-only or idempotent behavior.

Core is model-agnostic: any compatible CLI, TypeScript, or MCP consumer can use the returned
citations. A hosted AI receives the passages you return to it under that provider's data policy. A
Expand All @@ -125,6 +131,19 @@ root base for shared knowledge and isolated bases in individual apps. `rgr bases
base, generated MCP helpers pin their project root, and nested bases receive distinct server names
so agents do not silently query the wrong index.

### Share one source of truth across a team

Ragmir is local and confidential by design. It has no built-in cloud synchronization. Give every
developer the same source folder through a tool such as the Google Drive desktop app or a team
script, keep the Ragmir version, configuration, embedding provider, and model aligned, then ingest
locally. The result is an equivalent local index on each workstation, not a shared database.

Always sync before `rgr ingest`, then run `rgr audit`. A missing, partially synced, or extra file in
the selected raw or source folder makes that developer's index diverge. Ragmir hashes file content
on every inventory pass, so a changed file is still detected when a sync tool preserves its size
and modification time. Teams can automate setup with `initProject`, `addSourceEntries`, and
`createRagmirClient`; Ragmir remains the local retrieval layer, not the synchronization layer.

### Audit a knowledge base

```bash
Expand Down Expand Up @@ -169,7 +188,9 @@ pnpm exec rgr status --json
Ragmir records per-file progress atomically under ignored `.ragmir/storage/` state. Files from a
committed batch are not parsed or embedded again after a restart. A full `--rebuild` writes to an
isolated generation and activates it only after row and manifest validation, so an interrupted
rebuild leaves the previous searchable index active.
rebuild leaves the previous searchable index active. After activation, older generated tables stay
available so searches that already opened them can finish safely. `rgr destroy-index` removes all
generated index storage.

### Search scanned PDFs

Expand Down
20 changes: 13 additions & 7 deletions RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,23 @@ Do not publish from a local machine.

1. Start feature work from `develop` on a `feature/*`, `fix/*`, or `chore/*` branch.
2. Open a pull request into `develop` and wait for required checks.
3. Promote `develop` to `main` through a release pull request after integration is green.
3. From `main`, prepare a `release/*` branch containing the validated `develop` tree, then open a
release pull request into `main`.
4. Let the `Release npm` workflow run on `main`, or dispatch it manually from `main`.
5. Approve the protected `npm-publish` environment when GitHub asks for review.
6. Verify npm, GitHub release notes, release artifacts, and the landing deploy.
6. Verify all three npm packages, GitHub release notes, and release artifacts.
7. Build and deploy the static landing through the external deployment process, then verify the
published package version and canonical site URL.

The workflow runs `pnpm validate`, then semantic-release derives the next version from
Conventional Commits. It prepares `packages/ragmir-tts` and `packages/ragmir-core`, publishes
`@jcode.labs/ragmir-tts` first, then publishes `@jcode.labs/ragmir` with npm provenance.

After a successful npm release, the workflow dispatches `deploy-landing.yml` on `main` with the
released version so the production landing can show the current package version.
Conventional Commits. It prepares and publishes, in order, `@jcode.labs/ragmir-tts`,
`@jcode.labs/ragmir-chat`, and `@jcode.labs/ragmir`, all with npm provenance. Installing Core still
keeps Chat and TTS optional.

The repository intentionally contains no cloud-vendor landing configuration and the npm workflow
does not deploy the site. The external deployment must build with the released version in
`PUBLIC_RAGMIR_VERSION`. Production must use `PUBLIC_RAGMIR_LANDING_URL=https://ragmir.com`; staging
must use its own public URL so canonical and robots metadata cannot fall back to production.

## Versioning Notes

Expand Down
16 changes: 13 additions & 3 deletions SECURITY-HARDENING.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Ragmir Security Hardening

*Stop sending confidential documents directly to the cloud.*

Ragmir reduces document exposure by keeping its index and normal retrieval local. It is not a
security certification and does not replace encrypted storage, operating-system isolation, or review
of an agent that receives retrieved passages.
Expand All @@ -11,9 +13,14 @@ of an agent that receives retrieved passages.
- Built-in redaction runs before indexing.
- Secret-like filenames are skipped and unsupported files are reported.
- Access logs contain metadata and salted query identifiers, not raw prompts or retrieved text.
- MCP tools are read-focused and retrieval is bounded.
- Remote model downloads require an explicit setup action; normal confidential indexing keeps remote
model loading disabled.
- MCP tools advertise non-destructive behavior, and every tool or resource JSON response is
byte-bounded. Search, ask, research, and evaluation conservatively advertise open-world behavior
because explicitly enabled semantic models may download public weights. Tools that can append
metadata-only logs or initialize local state use conservative read-only and idempotency hints.
- Strict MCP diagnostics mask configured model, storage, source, and access-log paths. Evaluation
returns a project-relative golden path and replaces operational errors with a generic message.
- Remote model loading is disabled on the normal confidential path. Downloads are possible only
after the operator explicitly enables remote Transformers models or runs the model-pull setup.

## Check a workspace

Expand All @@ -32,6 +39,9 @@ not index. The strict audit is the local readiness check before using a sensitiv
- Store the repository and `.ragmir/` on an encrypted disk when at-rest protection matters.
- Use one checkout per trust boundary.
- Keep raw documents, environment files, credentials, and generated state out of commits.
- Treat a team file-sync service as a separate trust boundary. Ragmir does not upload the corpus,
but a Google Drive account or another sync tool follows its own access and retention policy.
- Synchronize source files, not an actively written `.ragmir/storage/` directory.
- Preload optional embedding, chat, or TTS models with non-sensitive text before using them offline.
- Treat cited passages as evidence to review, not as automatically correct conclusions.

Expand Down
1 change: 1 addition & 0 deletions biome.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
"*.json",
"scripts/**/*.mjs",
"packages/*/scripts/**/*.mjs",
"packages/*/vitest.config.ts",
"packages/*/src/**/*.{ts,tsx}"
]
},
Expand Down
Loading