Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ One domain, many stores: the same activity feed modeled on relational, document,

## What this demonstrates

Polyglot persistence is the idea that one product rarely has one ideal database. A follow graph wants adjacency lookups, a home timeline wants a clustered time key, and a unique handle wants a relational constraint. This repo keeps one domain and one `ActivityStore` port, then implements that port on each paradigm so query shape, consistency, and scaling stay comparable. The first backend is in-memory fan-in-on-read: per-author timelines merged into a home feed. Later backends (Postgres, Redis, MongoDB, Cassandra, CockroachDB, Neo4j) keep the same interface and the same contract tests.
Polyglot persistence is the idea that one product rarely has one ideal database. A follow graph wants adjacency lookups, a home timeline wants a clustered time key, and a unique handle wants a relational constraint. This repo keeps one domain and one `ActivityStore` port, then implements that port on each paradigm so query shape, consistency, and scaling stay comparable. The in-memory backend is fan-in on read: per-author timelines merged into a home feed. The Postgres backend is the same port over a 3NF schema, composite B-tree indexes, and keyset SQL. Later backends (Redis, MongoDB, Cassandra, CockroachDB, Neo4j) keep the same interface and the same contract tests.

## Concepts demonstrated

Expand All @@ -16,11 +16,18 @@ Polyglot persistence is the idea that one product rarely has one ideal database.
- Compound cursor pagination over a total order `(createdAt DESC, id DESC)`
- Unique constraints on user id and handle
- Directed follow graph with idempotent edges and live (not snapshot) feeds
- Third normal form: users, a composite follow edge, posts
- Composite B-tree indexes `(author_id, created_at DESC, id DESC)` for a newest-first timeline
- Keyset pagination with a row comparison `(created_at, id) < cursor` (first page and keyed page are separate statements)
- Fan-in via `CROSS JOIN LATERAL`: each followee's newest-N posts walk `posts_author_timeline_idx`, then the outer query merges
- Table-level `CHECK` (no self-follow) and `UNIQUE` (handle) as the source of integrity
- Query planner: after `ANALYZE`, `EXPLAIN` on the author timeline is `Index Scan using posts_author_timeline_idx` with no Sort. The lateral feed uses that same scan per followee.

## What's implemented

- Project scaffold with TypeScript strict mode, Vitest, and CI
- Activity-feed domain, `ActivityStore` port, in-memory backend, and a shared contract suite
- Relational backend (Postgres): schema, indexes, the SQL queries for the domain

## Usage

Expand All @@ -42,7 +49,7 @@ const page = await store.feed('ada', { limit: 20 })
const next = await store.feed('ada', { limit: 20, before: post })
```

A new backend implements `ActivityStore` and calls `defineStoreContract(name, factory)` from `test/contract.ts`. If the contract is green, the store matches the domain.
`PostgresStore` takes any `{ query(sql, params) }` client. Tests drive it with PGlite (`@electric-sql/pglite` is a devDependency, not a runtime dep). A server `pg` Pool works the same way. A new backend implements `ActivityStore` and calls `defineStoreContract(name, factory)` from `test/contract.ts`.

## Running the tests

Expand Down
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "polyglot-persistence",
"version": "0.1.0",
"version": "0.2.0",
"description": "One domain modeled across relational, document, key-value, wide-column, distributed SQL, and graph stores. A reference for when to reach for which database.",
"type": "module",
"license": "MIT",
Expand All @@ -11,6 +11,7 @@
"build": "tsc"
},
"devDependencies": {
"@electric-sql/pglite": "^0.5.5",
"typescript": "^5.6.0",
"vitest": "^2.1.0"
}
Expand Down
8 changes: 8 additions & 0 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion src/index.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
export const VERSION = '0.1.0'
export const VERSION = '0.2.0'
export { BODY_MAX, StoreError } from './domain'
export type {
FeedCursor,
Expand All @@ -11,3 +11,5 @@ export type {
} from './domain'
export type { ActivityStore, CreateUserInput, PublishInput } from './store'
export { MemoryStore } from './memory-store'
export { PostgresStore, SCHEMA_STATEMENTS, SQL } from './postgres/store'
export type { SqlQuery } from './postgres/store'
79 changes: 79 additions & 0 deletions src/postgres/schema.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
export const SCHEMA_STATEMENTS = [
`CREATE TABLE IF NOT EXISTS users (
id TEXT PRIMARY KEY,
handle TEXT NOT NULL,
created_at BIGINT NOT NULL
)`,
`CREATE UNIQUE INDEX IF NOT EXISTS users_handle_uidx ON users (handle)`,
`CREATE TABLE IF NOT EXISTS follows (
follower_id TEXT NOT NULL REFERENCES users (id) ON DELETE CASCADE,
followee_id TEXT NOT NULL REFERENCES users (id) ON DELETE CASCADE,
PRIMARY KEY (follower_id, followee_id),
CONSTRAINT follows_no_self CHECK (follower_id <> followee_id)
)`,
`CREATE INDEX IF NOT EXISTS follows_inbound_idx ON follows (followee_id)`,
`CREATE TABLE IF NOT EXISTS posts (
id TEXT PRIMARY KEY,
author_id TEXT NOT NULL REFERENCES users (id) ON DELETE CASCADE,
body TEXT NOT NULL,
created_at BIGINT NOT NULL,
CONSTRAINT posts_body_len CHECK (char_length(body) BETWEEN 1 AND 280)
)`,
`CREATE INDEX IF NOT EXISTS posts_author_timeline_idx
ON posts (author_id, created_at DESC, id DESC)`,
] as const

export const SQL = {
insertUser: `INSERT INTO users (id, handle, created_at)
VALUES ($1, $2, $3)
RETURNING id, handle, created_at`,
selectUser: `SELECT id, handle, created_at FROM users WHERE id = $1`,
selectUserByHandle: `SELECT id, handle, created_at FROM users WHERE handle = $1`,
insertFollow: `INSERT INTO follows (follower_id, followee_id)
VALUES ($1, $2)
ON CONFLICT (follower_id, followee_id) DO NOTHING`,
deleteFollow: `DELETE FROM follows WHERE follower_id = $1 AND followee_id = $2`,
hasFollow: `SELECT 1 FROM follows WHERE follower_id = $1 AND followee_id = $2`,
following: `SELECT followee_id FROM follows WHERE follower_id = $1 ORDER BY followee_id`,
followers: `SELECT follower_id FROM follows WHERE followee_id = $1 ORDER BY follower_id`,
insertPost: `INSERT INTO posts (id, author_id, body, created_at)
VALUES ($1, $2, $3, $4)
RETURNING id, author_id, body, created_at`,
selectPost: `SELECT id, author_id, body, created_at FROM posts WHERE id = $1`,
authorTimeline: `SELECT id, author_id, body, created_at
FROM posts
WHERE author_id = $1
ORDER BY created_at DESC, id DESC
LIMIT $2`,
authorTimelineBefore: `SELECT id, author_id, body, created_at
FROM posts
WHERE author_id = $1
AND (created_at, id) < ($2::bigint, $3::text)
ORDER BY created_at DESC, id DESC
LIMIT $4`,
feed: `SELECT p.id, p.author_id, p.body, p.created_at
FROM follows f
CROSS JOIN LATERAL (
SELECT p.id, p.author_id, p.body, p.created_at
FROM posts p
WHERE p.author_id = f.followee_id
ORDER BY p.created_at DESC, p.id DESC
LIMIT $2
) p
WHERE f.follower_id = $1
ORDER BY p.created_at DESC, p.id DESC
LIMIT $2`,
feedBefore: `SELECT p.id, p.author_id, p.body, p.created_at
FROM follows f
CROSS JOIN LATERAL (
SELECT p.id, p.author_id, p.body, p.created_at
FROM posts p
WHERE p.author_id = f.followee_id
AND (p.created_at, p.id) < ($2::bigint, $3::text)
ORDER BY p.created_at DESC, p.id DESC
LIMIT $4
) p
WHERE f.follower_id = $1
ORDER BY p.created_at DESC, p.id DESC
LIMIT $4`,
} as const
Loading
Loading