Pocket DB — the local database for Electron, desktop and CLI apps. One file, zero native dependencies, a familiar MongoDB-style API.

---

Add persistent local storage to a Node.js or Electron app — no server, no daemon, and no native bindings. No node-gyp, no rebuilding against every Electron version, none of the better-sqlite3 recompile dance. Run npm install and you have a working document store backed by a single file.

import { pocketDb } from "@axfab/pocket-db";

const db = pocketDb("./data.pdb");
const users = db.collection("users");

users.insertOne({ name: "Ada", role: "admin" });
const admins = users.find({ role: "admin" }).toArray();

db.close();

That's the whole setup — open a file, work with collections of JSON documents, close.

Why pocket-db

• 🪶 Zero native dependencies — pure TypeScript, no node-gyp, no per-Electron rebuilds
• 📄 Single file — back up your whole database by copying one .pdb file
• 🍃 MongoDB-style API — find / insert / update with the operators you already know
• ⚡ Append-only writes — every mutation is a fast sequential append, with crash-safe batches
• 🛡️ Safe reads — every query returns an independent copy, so mutating a result never corrupts your stored data

Inspired by SQLite (one embedded file) and MongoDB (document model), but intentionally small. The core rule — never reserialise the whole database on a write — makes every insert, update and delete a fast append, and reads seek straight to a document's offset.

Good fit: Electron & desktop apps, CLI tools, local servers, plugins, structured caches, offline-first prototypes.
Not a fit: multiple processes writing the same file at once, complex aggregation pipelines, or anything that really wants a server database.

---

Install

npm install @axfab/pocket-db

No native binaries. No optional dependencies. Pure TypeScript compiled to ESM. Node.js ≥ 18 is required (the package only relies on structuredClone and Object.hasOwn, both available since Node 17/16.9).

Bun / Deno: not officially tested, but should work — the package touches only node:fs, node:path, node:os, node:crypto, and Buffer, all well-covered by both runtimes' Node compatibility layers, and with { type: 'json' } isn't used here so there's no import-attribute version floor to worry about.

---

Quick start

import { pocketDb } from "@axfab/pocket-db";

const db = pocketDb("./data.pdb");
const users = db.collection("users");

// Insert
const { insertedId } = users.insertOne({ name: "Ada", role: "admin", age: 37 });

// Find
const ada = users.findOne({ name: "Ada" });
console.log(ada); // { _id: "...", name: "Ada", role: "admin", age: 37 }

// Query with operators
const admins = users.find({ role: "admin", age: { $gte: 18 } }).toArray();

// Update
users.updateOne(insertedId, { $set: { age: 38 }, $inc: { loginCount: 1 } });

// Delete
users.deleteOne(insertedId);

db.close();

---

Core concepts

Single file, append-only

All writes are appended to the end of the file. Reads go directly to the byte offset of the document — no full-file scan. The in-memory state is rebuilt by replaying the log when open() is called. Deleted and updated documents leave dead records behind; db.compact() reclaims that space in a single forward pass.

Collections

A database holds any number of named collections. Collections are created implicitly on first access and persisted to the log. Each collection has its own primary index (keyed by _id) and optional secondary indexes.

Document IDs

Every document gets a _id: a 24-character lowercase hex string (12-byte ObjectId layout — 4-byte timestamp, 5-byte random, 3-byte counter). You can supply your own _id on insert as long as it matches that format.

Safe reads

Every document returned by findOne, find, or a cursor is a fresh, independent object that you fully own. Mutating a query result never affects what is stored — exactly what you'd expect from a real database. This holds whether the read came from disk or from the hot-document cache (cached reads are deep-cloned on the way out).

This is a deliberate guarantee: some in-memory stores return references to their internal objects by default, which is faster but means modifying a query result silently corrupts the database. Pocket DB always isolates your results.

---

API

Opening a database

import { pocketDb } from "@axfab/pocket-db";
const db = pocketDb("./data.pdb");

You can also merge both arguments by setting the path property in the options object, or pass all options together:

const db = pocketDb({ path: "./data.pdb" });

pocketDb accepts an optional OpenOptions object as its second argument (or first, when using the object form):

const db = pocketDb("./data.pdb", {
  durability: "strict",      // default: "relaxed"
  serialization: "bson",     // default: "json" — only applies when creating a new file
  maxReadBytes: 4 * 1024 * 1024, // default: 8MiB
});

durability

Controls whether fsync is called after every write.

• "relaxed" (default) — skips fsync. Writes reach the OS page cache but may be lost on a power failure or OS crash before the cache is flushed. Faster; suitable when losing the last few writes on a hard crash is acceptable.
• "strict" — calls fsync after every appendOperation, guaranteeing data is on durable storage before the call returns. Safest; incurs one extra syscall per write.

serialization

Selects the document encoding format when creating a new database file. Opening an existing file always uses the format recorded in the file header — this option is ignored.

• "json" (default) — documents stored as UTF-8 JSON. Human-readable, universally compatible.
• "bson" — documents stored as BSON (Binary JSON). Supports double, string, document, array, boolean, null, int32, int64. More compact than JSON for numeric-heavy documents.
• "amf3" — documents stored as AMF3 (Action Message Format 3). A compact binary format supporting undefined, null, boolean, integer, double, string, array, and object.

maxReadBytes

Upper bound, in bytes, of a single read from the database file (default 8MiB). Replay at open(), query scans and index builds read the file in windows of at most this size: peak read memory stays around this value plus the largest document, whatever the size of the database. Query windows start at 64KiB and double up to this bound, so a query that stops early (findOne()) reads little. Lower it on memory-constrained hosts, raise it on slow-seek storage. See ADR 0021.

verify

When true, open() decodes every live document of the collections that have no index and fails on the first one that cannot be decoded (default false). open() always checks record CRCs and decodes the live documents of indexed collections (to build their indexes), but not those of unindexed collections. See db.verify() below for an on-demand check that returns a report.

Database

db.collection(name: string): Collection
db.getCollections(): string[]              // names of all registered collections
db.existsCollection(name: string): boolean
db.compact(): void                         // reclaim space from dead records
db.verify(options?): VerifyResult          // decode every live document, report invalid ones
db.close(): void

Collection

collection.insertOne(doc): InsertOneResult
collection.insertMany(docs): InsertManyResult

collection.findOne(query?): Record | null
collection.find(query?): Cursor
collection.countDocuments(query?): number
collection.distinct(field, query?, options?: { limit?: number }): unknown[]

collection.updateOne(id | query, update): UpdateResult
collection.updateMany(query, update): UpdateResult

collection.replaceOne(id, doc): ReplaceOneResult
collection.replaceOne(doc & { _id }): ReplaceOneResult

collection.deleteOne(id | query): DeleteOneResult
collection.deleteMany(query?): DeleteManyResult

collection.createIndex(field, { type: "string" | "number", unique?: boolean, partialFilterExpression?: Query }): CreateIndexResult
collection.dropIndex(field): DropIndexResult
collection.getIndexes(): { name: string; type: string; unique: boolean; partialFilterExpression?: Query }[]
collection.existsIndex(name: string): boolean
collection.drop(): DropResult

collection.enableCache(maxBytes: number): void   // hot-document cache (off by default)
collection.disableCache(): void
collection.cacheStats(): DocumentCacheStats | null

Cursor

cursor.next(): Record | null
cursor.toArray(): Record[]
cursor.count(): number

cursor.sort(spec: Record<string, 1 | -1>): Cursor   // up to 4 fields
cursor.limit(n: number): Cursor
cursor.skip(n: number): Cursor

---

Query operators

Queries are plain objects. A bare value is shorthand for $eq.

Operator	Description
$eq	Strict equality (no type coercion); on an array field, any element (or the whole array) equal
$ne	Not equal (on an array field: no element equal)
$gt / $gte	Greater than / greater than or equal
$lt / $lte	Less than / less than or equal
$in	Field value is in the given array
$nin	Field value is not in the given array
$exists	Field is present (true) or absent (false)
$regex	String matches a regular expression (flags via $options; g/y rejected)
$not	Negates an operator expression
$and	Logical AND of sub-queries
$or	Logical OR of sub-queries
$nor	Logical NOR of sub-queries

// Compound query
users.find({
  $and: [
    { role: { $in: ["admin", "editor"] } },
    { age: { $gte: 18, $lt: 65 } }
  ]
});

// Negation
users.find({ status: { $not: { $eq: "banned" } } });

// Array fields: matches when any element matches (see docs/query.md#array-fields)
posts.find({ tags: "typescript" });
posts.find({ scores: { $gt: 90 } });

// OR
users.find({ $or: [{ role: "admin" }, { role: "editor" }] });

// Regex (string pattern + $options, RegExp value, or bare RegExp shorthand)
users.find({ name: { $regex: "^ada", $options: "i" } });
users.find({ name: { $regex: /^ada/i } });
users.find({ name: /^Ada/ });

---

Update operators

Updates are expressed as operator objects applied to the current document.

Operator	Description
$set	Set one or more fields
$unset	Remove one or more fields
$inc	Increment a numeric field
$mul	Multiply a numeric field by a factor
$min / $max	Set field only if new value is lower / higher
$rename	Rename a field (missing source is a no-op)
$currentDate	Set field to the current date (true / { $type: "date" } → ISO string, { $type: "timestamp" } → epoch ms)
$push	Append a value to an array field
$addToSet	Append a value only if no equal element exists
$pop	Remove the last (1) or first (-1) array element
$pull	Remove array elements equal to a value or matching a condition
$pullAll	Remove array elements equal to any listed value

users.updateOne(id, {
  $set: { role: "editor" },
  $inc: { loginCount: 1 },
  $mul: { score: 1.1 },
  $currentDate: { lastLogin: true },
  $addToSet: { tags: "active" },
  $pull: { scores: { $lt: 10 } }
});

_id is immutable and cannot be modified by any update operator.

---

Indexes

Secondary indexes speed up equality and range queries. They are rebuilt from the log at every open.

// Create
users.createIndex("role", { type: "string" });
users.createIndex("age",  { type: "number" });

// Unique: rejects any write that would duplicate an existing value
users.createIndex("email", { type: "string", unique: true });

// Multikey: array fields are indexed element by element (automatic)
posts.createIndex("tags", { type: "string" });
posts.find({ tags: "typescript" });          // posts whose tags array contains "typescript"

// Partial: only index documents matching a filter
users.createIndex("email", {
  type: "string",
  unique: true,
  partialFilterExpression: { deletedAt: { $exists: false } }  // unique among non-deleted users
});

// Drop
users.dropIndex("role");

StringIndex supports $eq and $in lookups. NumberIndex additionally supports $gt, $gte, $lt, $lte range scans. The query planner automatically picks the most selective available index for each query.

Multikey. An array field is indexed once per distinct element of the index's type, so find({ tags: "x" }) and { scores: { $gt: 90 } } use the index. This relies on MongoDB-style array matching in queries (any element matching is enough, for indexed or unindexed fields — see docs/query.md).

Partial. partialFilterExpression (any query, stored as JSON — no RegExp instances) restricts the index to matching documents. The planner only uses a partial index for a query that itself contains each top-level condition of the filter (same field, operator and operand); other queries use another index or a full scan, with identical results. With unique, only matching documents are constrained. See docs/indexes.md.

unique: true (default false) turns an index into a uniqueness constraint, checked on every insertOne/insertMany/replaceOne/updateOne/updateMany before the write is committed. Only values matching the index's own type participate — a missing field or a value of a different type never conflicts. Creating a unique index over a collection that already has conflicting values throws and leaves the collection unchanged. See docs/indexes.md for details.

---

Hot-document cache

By default every read decodes its document from the file. For workloads that read the same documents repeatedly, you can opt into an in-memory cache that keeps parsed hot documents around, so repeated reads skip both the file read and the decode.

users.enableCache(16 * 1024 * 1024); // 16 MB budget; least-recently-used docs are evicted
users.cacheStats();                  // { hits, misses, evictions, bytes, documentCount, ... }
users.disableCache();                // free everything, back to zero overhead

The cache is off by default — when disabled it costs nothing (no object is even allocated). It is keyed by _id and versioned by file offset, so it stays correct across updates, deletes, compaction, and open cursors (snapshot reads are preserved).

The gain: on a 2,000-document JSON dataset, single-document reads get ~2.9× faster and even full scans ~1.8× faster, because skipping the read + JSON parse outweighs the cost of cloning the cached object.

The drawbacks: it trades memory for speed, so size the budget for your hot working set. It is rebuilt empty on every open() (in-memory only, never persisted). And large scans populate the cache as they read — if the budget is smaller than a scan's footprint, a one-off scan can evict genuinely hot documents.

See docs/cache.md for the full design, internals, and benchmark methodology.

---

Sorting and pagination

const page = users
  .find({ role: "admin" })
  .sort({ age: -1, name: 1 })   // up to 4 sort fields
  .skip(20)
  .limit(10)
  .toArray();

Sort accepts 1 (ascending) and -1 (descending). Missing values sort first in ascending order and last in descending order. Sorting is eager — every matching document is read before the first result — except when the first sort field is _id. Narrow the candidate set with an indexed query before sorting other fields over large collections.

To walk a large collection page by page, prefer keyset pagination on _id over skip(): each page costs O(log n + page size), and pages stay exact when documents are written between them (no document skipped or returned twice).

let last: string | undefined;
for (;;) {
  const page = users
    .find(last ? { _id: { $gt: last } } : {})
    .sort({ _id: 1 })
    .limit(250)
    .toArray();
  if (page.length === 0) break;
  // … process the page
  last = page[page.length - 1]._id as string;
}

$gt/$gte/$lt/$lte on _id and sort({ _id: 1 | -1 }) are served by the primary index (find().sort({ _id: -1 }).limit(10) returns the 10 most recent documents without reading the others). See ADR 0022.

---

Compaction

Dead records accumulate as documents are updated or deleted. compact() rewrites the file in a single forward pass, keeping only live data:

db.compact();

After compaction, all in-memory indexes are refreshed automatically.

---

Batch atomicity

insertMany, updateMany, and deleteMany are crash-safe: if the process is killed mid-batch, the partial batch is silently discarded on the next open. Either all operations are visible or none are.

---

Crash recovery

Every write is a single sequential append — Pocket DB never reserialises the whole database on a write (see Single file, append-only above). That means a crash can only ever affect the one record that was being written when the process died; everything written before it is untouched on disk.

If open() finds an incomplete or corrupt record at the very end of the file — evidence of a crash mid-write on a previous run — it recovers automatically: the file is truncated back to its last valid record and the database opens normally, discarding only that one unfinished write. db.recovered reports whether this happened, and open() also logs a warning naming the file and the number of bytes discarded. See docs/storage.md and ADR 0019 for the exact scope (this covers the trailing record only — corruption elsewhere in the file is a different, not-yet-implemented problem, and still fails open() loudly rather than guessing).

const db = pocketDb("./data.pdb");
if (db.recovered) {
  console.log("Recovered from an incomplete write left by a previous crash.");
}

---

File locking

pocketDb() creates a .lock file next to the database file. A second pocketDb() on the same path — from another process, or from the same process while the first handle is still open — will throw. Stale locks are detected and cleared automatically.

The lock file contains pid:hostname (for example 1:3f9c2a1b7d4e). Locks written by earlier versions contain only the PID and are still understood. A lock is reclaimed when:

• its PID is dead (same hostname, or legacy PID-only lock), or
• it carries this process's own PID and hostname but this process does not currently hold the path. This is the typical Docker case: the container restarts, the service gets the same PID again (often PID 1) and finds its own previous lock.

A lock is never reclaimed when it comes from another hostname (its PID belongs to another machine or container and proves nothing), or when its PID is alive. A blocked open throws an error naming the lock path, PID and hostname.

Known limitations (a lock that survives must be deleted by hand; there is no TTL, heartbeat or force option):

• worker_threads: each worker keeps its own in-memory registry of held locks. Workers opening the same file share the PID and hostname but see an empty registry, so one worker may wrongly reclaim another worker's lock.
• Two containers with the same fixed hostname, the same PID and a shared volume would reclaim each other's lock. This is an atypical setup, and a simple file lock does not support sharing a volume between instances anyway.
• A container recreated with a new hostname on a persistent volume finds the old lock still held. This is the safe behaviour: delete the .lock file manually.

Pocket DB is designed for single-process use. Multiple concurrent writers on the same file are not supported.

---

TypeScript

Pocket DB is written in TypeScript and ships its own type declarations. All public types are exported from the package root:

import { pocketDb } from "@axfab/pocket-db";
import type {
  Database, Collection, Cursor,
  InsertOneResult, InsertManyResult,
  UpdateResult, ReplaceOneResult,
  DeleteOneResult, DeleteManyResult,
  CreateIndexResult, DropIndexResult, DropResult,
  CreateIndexOptions, IndexInfo, OpenOptions, SortDirection,
  Query, DocumentCacheStats
} from "@axfab/pocket-db";

---

Documentation

The docs/ folder contains in-depth documentation available as a wiki:

• File format — binary layout, record structure, U29 encoding
• Storage semantics — replay rules, write path, crash recovery
• Query & update model — operators, compilation, cursor semantics
• Indexes — primary index, StringIndex, NumberIndex, multikey (array) indexing, partial indexes, unique constraints, query planner
• Hot-document cache — LRU cache internals, offset versioning, eviction, benchmarks
• Compaction — algorithm, invariants, secondary index refresh

---

Tooling

• pocket-desk — a desktop GUI for browsing and inspecting .pdb files without writing any code: collections, documents, indexes, and raw storage stats.
• pocket-shell — an interactive, mongosh-style REPL for .pdb files: zero install, zero server (npx @axfab/pocket-shell ./data.pdb). Full CRUD, index management, compaction, import/export, and a non-interactive --eval mode for scripting. A CLI alternative to pocket-desk's GUI.

---

Performance

Pocket DB is built for fast, durable writes. Benchmarked against other embedded stores on 1,000 documents across the operations in our benchmark suite — ops/sec, higher is better.

The headline: for durable writes, pocket-db is 40–700× faster than every other file-backed store here, and lands within ~12% of in-memory SQLite — which isn't even durable.

📊 Full results table → benchmarks/RESULTS.md — all adapters and operations side by side. Regenerate anytime with npm run bench, which prints a width-aware ranked view to the console and refreshes RESULTS.md and results.json.

Reading the results

Writes — where pocket-db shines. Every mutation is a single sequential append: no B-tree rebalancing, no page allocation, no full-file reserialisation. That's why insertOne, updateOne and deleteOne leave every other persistent store far behind and sit right next to in-memory SQLite. For a desktop or Electron app writing to disk on every user action, this is exactly the path that matters — and it stays durable.

Single-document reads are fast too. findById runs at ~140k ops/sec — more than enough for typical app workloads, even though it reads from disk rather than RAM.

Full scans and sorts — the current tradeoff, by design. json-file, lowdb and LokiJS win on findAll and sortByScore because they hold the entire dataset in memory and serve reads from there. That speed has a hard ceiling: your database can never grow larger than available RAM. Pocket DB keeps only its indexes in memory and reads each document from its file offset on demand — so it can back a database far larger than RAM would ever allow. The price today is slower full scans.

A hot-document cache now closes most of that read gap without touching the append-only write model. Opt in with collection.enableCache(maxBytes) and hot documents stay parsed in memory: single-document reads get ~2.9× faster and full scans ~1.8× faster on the 2,000-document JSON benchmark. It is off by default and trades memory for speed — see docs/cache.md.

Reading the read numbers fairly. pocket-db returns an independent copy of every document, so mutating a result never touches stored data. Some in-memory stores in this table return references to their internal objects by default — fast, but modifying a query result silently corrupts the database. Part of their read-throughput lead is simply the defensive copy pocket-db makes and they skip; it buys a correctness guarantee we keep on purpose.

In short: if your workload is write-heavy, needs durability, or outgrows memory, pocket-db is the right tool. If you need pure in-memory read throughput on a dataset that comfortably fits in RAM, an in-memory store still wins today.

Run the benchmarks yourself:

npm install
npm run bench

---

License

MIT © Fabien Bavent