System Design Knowledge Base

A comprehensive collection of system design case studies, architectural patterns, production-grade implementations, and interview preparation resources.

---

📖 Glossary & Quick Navigation

mindmap
  root((System Design))
    Case Studies
      Financial Systems
      E-Commerce
      Food Delivery
      IoT & Streaming
      Collaboration
      Media & Analytics
    Patterns
      Communication
      Resilience
      Data
      Messaging
      Deployment
    Implementations
      URL Shortener
      Rate Limiter
      Leaderboard
      OAuth 2.0
      JWT Auth
      Products API
      Log Ingestion
      Stock Signals
    Resources
      Interview Guide
      Compliance Guide
      Job Search HQ
      Kubernetes Guide
      Amazon SDM Prep
      Workflow Tools
    AI & LLM
      DSAR Agent

Loading

---

🏗️ System Design Case Studies

Financial Systems

Topic	Description	Key Concepts	Links
Financial Clearing House	Interbank settlement system with multilateral netting achieving 93%+ efficiency	Graph algorithms, Two-phase settlement, Saga pattern	📘 Overview • 🏛️ Architecture • 🔄 Flow
Seller Payment System	E-commerce seller payouts with configurable schedules and fee optimization	Exactly-once semantics, State machines, Idempotency	📘 Overview • 🏛️ Architecture • 💾 Data Models
Fintech Data Platform	End-to-end data architecture with CDC, event streaming, and federated queries	CDC (Debezium), Kafka, HTAP, Data Lake	📘 Overview • 📊 E2E Diagram
Digital Remittance Platform	Cross-border money transfer platform (Wise/Remitly-scale) with 1M+ transfers/day across 50+ corridors	FX corridors, Compliance (AML/KYC), Payment rails, Saga pattern	📘 Overview • 📋 Blueprint

E-Commerce & Marketplace

Topic	Description	Key Concepts	Links
Merchandise Browsing	Large-scale product discovery for 1M+ DAU with real-time trending	Popularity scoring, Flink streaming, Personalization	📘 Overview • 📊 Diagrams
Uber Cart System	Multi-merchant cart with family accounts and offline-first behavior	CRDT sync, Multi-tenant, Event sourcing	📘 Overview • 🏛️ Architecture • 📱 API
Issue Tracking System	Multi-tenant Jira-like platform for 50M DAU with 10B issues	Row-level security, Elasticsearch, Tenant isolation	📘 Overview • 🏛️ Architecture • 🔍 Search

Food Delivery & Logistics

Topic	Description	Key Concepts	Links
Uber Eats Feed	Restaurant feed with H3 spatial indexing handling 10K+ views/sec	H3 hexagonal grid, Geo-sharding, ML ranking	📘 Overview • 🗺️ Spatial Indexing • 📊 Ranking
Instant Grocery Delivery	10-minute grocery delivery across 40 dark stores at 100K orders/day	Geo-routing, Inventory reservation (Redis), Rider dispatch, Kafka backbone	📋 API Spec • 📐 ADRs • 🏛️ Plan
Cheapest Flights & Dynamic Pricing	Google Flights-like aggregation for 100M+ daily searches across 500+ suppliers	Dynamic pricing (ML), Price prediction, Supplier fan-out, Cache hierarchy	📘 Overview • 🏛️ Architecture • 🔌 API
Notification Service	Unified SMS/Email/Push platform at 500M+ notifications/day with priority tiers and quota enforcement	Priority queuing, Deduplication, Template engine, Retry/backoff	📘 Overview • 🏛️ Architecture • 📋 API

Real-Time & Collaboration

Topic	Description	Key Concepts	Links
Collaborative Editor	Google Docs-like editor with CRDTs for 100 concurrent editors	CRDT, WebSocket, Offline-first, Causal consistency	📘 Overview • 🧬 CRDT Design • 🔄 Sync Protocol
Real-Time Leaderboard	Gaming leaderboard for 100M users with global/regional rankings	Redis Sorted Sets, Kafka, WebSocket push	📘 Overview • 🏛️ Architecture • 📊 Redis Deep Dive

Media & Analytics

Topic	Description	Key Concepts	Links
YouTube Video Views	10B view events/day with near-real-time counters (<10s freshness) and OLAP creator analytics	Lambda/Kappa architecture, HyperLogLog dedup, ClickHouse OLAP, Flink streaming	📘 Overview • 📊 Data Modeling • ⚡ Fault Tolerance

IoT & Streaming Systems

Topic	Description	Key Concepts	Links
Log Ingestion (10 PB/Day)	Petabyte-scale distributed log ingestion for debugging and compliance with ~115 GB/s average throughput	Fluent Bit/Vector, Kafka, Object storage tiering, Near-real-time queryability	📘 Overview • 📄 Docs

---

📐 Distributed System Patterns

📚 Complete Pattern Reference — Decision flowcharts, trade-off analysis, and implementation examples

API & Communication Patterns

Pattern	When to Use	Trade-offs	Link
REST API	Public APIs, CRUD operations	Simple ↔ Over/under-fetching	📄 Docs
GraphQL	Mobile apps, complex queries	Flexible ↔ Caching complexity	📄 Docs
gRPC	Internal microservices, high performance	Fast ↔ Browser support	📄 Docs
WebSockets	Real-time bidirectional	Low latency ↔ Connection overhead	📄 Docs

Gateway & Routing Patterns

Pattern	When to Use	Trade-offs	Link
API Gateway	Centralized entry point	Single entry ↔ SPOF risk	📄 Docs
Backend for Frontend	Multi-platform clients	Optimized UX ↔ Code duplication	📄 Docs
Aggregator	Composite responses	Reduced round trips ↔ Complexity	📄 Docs

Resilience Patterns

Pattern	When to Use	Trade-offs	Link
Circuit Breaker	Prevent cascading failures	Fail-fast ↔ Implementation complexity	📄 Docs
Retry with Backoff	Handle transient failures	Reliability ↔ Thundering herd	📄 Docs
Bulkhead	Isolate resource pools	Fault isolation ↔ Resource underutilization	📄 Docs
Rate Limiting	Traffic spike protection	System protection ↔ User experience	📄 Docs
Timeout	Prevent hung connections	Responsiveness ↔ False positives	📄 Docs

Data Patterns

Pattern	When to Use	Trade-offs	Link
CQRS	Separate read/write scaling	Performance ↔ Complexity	📄 Docs
Event Sourcing	Audit trails, temporal queries	Complete history ↔ Storage costs	📄 Docs
Saga Pattern	Distributed transactions	Eventual consistency ↔ Coordination	📄 Docs
Outbox Pattern	Reliable event publishing	Guaranteed delivery ↔ At-least-once	📄 Docs
Two-Phase Commit	Strong consistency needed	ACID guarantees ↔ Availability	📄 Docs

Messaging Patterns

Pattern	When to Use	Trade-offs	Link
Pub/Sub	Fan-out notifications	Decoupling ↔ Message ordering	📄 Docs
Message Queue	Work distribution	Reliability ↔ Latency	📄 Docs
Event-Driven Architecture	Reactive systems	Flexibility ↔ Debugging complexity	📄 Docs

Service Discovery & Mesh

Pattern	When to Use	Trade-offs	Link
Service Registry	Dynamic service discovery	Flexibility ↔ Additional infrastructure	📄 Docs
Sidecar	Cross-cutting concerns	Separation of concerns ↔ Resource overhead	📄 Docs
Service Mesh	Complex microservices	Full observability ↔ Operational complexity	📄 Docs

Deployment & Infrastructure

Pattern	When to Use	Trade-offs	Link
Blue-Green Deployment	Zero-downtime releases	Instant rollback ↔ 2x infrastructure	📄 Docs
Canary Deployment	Gradual rollouts	Lower risk ↔ Complexity	📄 Docs
Rolling Deployment	Resource-efficient updates	Simple ↔ Slower rollback	📄 Docs
Feature Flags	Runtime feature control	Flexibility ↔ Tech debt	📄 Docs
Strangler Fig	Legacy migration	Incremental ↔ Longer timeline	📄 Docs
Database Per Service	Microservices data isolation	Autonomy ↔ Distributed complexity	📄 Docs

---

🔧 Production Implementations

URL Shortener

Full-stack URL shortener scaling from local to 500M URLs/month globally

Tier	Scale	Architecture
1	Local	SQLite + Single binary
2	Startup (100K/mo)	PostgreSQL + Redis
3	Growth (10M/mo)	Multi-instance + Replicas
4	Scale (100M/mo)	Multi-region + DynamoDB
5	Global (500M/mo)	Edge computing + Sharded

Stack: Rust (Axum) • DynamoDB Global Tables • CloudFront • Terraform • Kubernetes

Resource	Link
Rust Version	📘 README
Java Version	📘 README
Architecture	🏛️ Docs
Security & Compliance	🔒 GDPR/SOC2

Distributed Rate Limiter

High-performance rate limiter for API Gateway (100K-1M RPS)

Features: Sliding window counter • Composite keys (user + endpoint) • Fail-open/closed modes • Circuit breaker integration

Stack: Java 21 (Spring Boot) • Redis Cluster • Resilience4j • Prometheus

Resource	Link
Overview	📘 README
Architecture	🏛️ Docs
Algorithms	🧮 Docs

Real-Time Leaderboard

Gaming leaderboard for 100M users with 50M DAU, supporting global/regional/friend rankings

Features: O(log N) Redis Sorted Sets • WebSocket push notifications • Multiple time windows • Circuit breakers

Stack: Java 21 (Spring Boot) • Redis Cluster • Kafka • PostgreSQL • WebSocket

Resource	Link
Overview	📘 README
Architecture	🏛️ Docs
Redis Deep Dive	📊 Docs
Demo Scripts	🎮 Scripts

OAuth 2.0 Demo

Complete OAuth 2.0 implementation with Authorization Server and Resource Server

Features: Authorization Code + PKCE • Client Credentials • Refresh Tokens • JWT with custom claims • PostgreSQL persistence

Stack: Java (Spring Boot 3.2) • Spring Authorization Server • Spring Security • PostgreSQL

Resource	Link
Overview	📘 README
Postman Collection	📬 Collection

JWT Authentication

Simple JWT auth system with access/refresh tokens

Features: Token rotation • Secure refresh flow • Spring Security integration

Stack: Java (Spring Boot 3.2) • Spring Security • MySQL • jjwt

Resource	Link
Overview	📘 README
Architecture	🏛️ Docs
Postman Collection	📬 Collection

Products API (Cassandra/ScyllaDB)

RESTful CRUD API demonstrating Cassandra/ScyllaDB patterns

Features: CQL operations • Docker Compose setup • Seed data

Stack: Java (Spring Boot) • Apache Cassandra 4.1 • Docker

Resource	Link
Overview	📘 README

Codec Library

MySQL column type codecs with zero-copy decoding (Go)

Features: Text/Binary protocol • Temporal types with timezone • DECIMAL precision • database/sql helpers

Resource	Link
Overview	📘 README
Source	📦 Go Package

Log Ingestion System

Petabyte-scale distributed log ingestion platform handling 10 PB/day (~115 GB/s average, ~345 GB/s peak)

Features: Multi-tier storage (hot/warm/cold) • Near-real-time queryability (<5 min) • Compliance audit trails • Auto-scaling collection agents

Stack: Fluent Bit / Vector • Apache Kafka • Object Storage (S3/GCS) • ClickHouse / OpenSearch

Resource	Link
Overview	📘 README
Docs	📄 Docs

Stock Signal Platform

Automated Indian equity screener with composite BUY/EXIT signal scoring and real-time alerts

Features: Fundamental screener (500 → ~50–100 stocks) • Technical + Momentum + Valuation scoring • Telegram/Email alerts • Web dashboard

Stack: Python • yfinance / NSE data • Telegram Bot API

Resource	Link
Overview	📘 README
Telegram Setup	📱 Guide
Backend	📦 backend/
Frontend	🖥️ frontend/

---

🎓 Interview & Career Resources

System Design Interview Guide

📚 Complete Guide — 12-part comprehensive preparation resource

#	Topic	Description
01	Interview Framework	Step-by-step approach for any interview
02	Requirements & Estimation	Capacity planning basics
02a	Back-of-Envelope (Detailed)	Mental math tricks & calculations
03	Core Building Blocks	DBs, caching, load balancers
04	Scalability Patterns	Sharding, replication, scaling
05	Distributed Concepts	CAP, consistency, consensus
06	Data Storage	SQL vs NoSQL, partitioning
07	Caching Strategies	Patterns, invalidation, CDNs
08	Messaging & Async	Queues, event-driven
09	API Design	REST, GraphQL, gRPC
10	Observability	Monitoring, fault tolerance
11	Common Problems	URL shortener, chat, feed
12	Quick Reference	One-page cheatsheet

Job Search HQ

Complete job search framework with trackers, templates, and guides

Category	Topics	Link
Resume	Quantified achievements, LinkedIn optimization	📄 Resume Guide
Behavioral	STAR stories, EM philosophy	📄 STAR Bank
System Design	Quick reference sheets	📄 Cheat Sheets
LLD	Patterns, problem bank	📄 LLD Guide
Coding	Pattern-based approach	📄 Coding Patterns
Negotiation	Compensation strategy	📄 Negotiation
Tracker	Company research template	📄 Tracker

📚 Full Job Search Guide

Kubernetes for Engineers

📚 Complete K8s Guide — Tailored for MAANG-level system design and EM interviews

#	Topic	Link
01	Architecture Overview (control plane, worker nodes, EKS)	📄 Docs
02	Structure & Components (Pods, Deployments, StatefulSets)	📄 Docs
03	Networking (Services, Ingress, CNI)	📄 Docs
04	Storage & Volumes	📄 Docs
05	Secrets Management	📄 Docs

Amazon SDM Interview Prep

Leadership principle stories and behavioral frameworks for Amazon SDM (L5/L6) interviews

Resource	Link
Amazon 16 LPs — Story Bank	📄 Guide

Interview Experiences

Company	Type	Link
Agoda	System Design + LLD	📁 Folder
Kuvera	System Design	📁 Folder
Fintech Org	Architecture Review	📁 Folder

---

🤖 AI & LLM Systems

DSAR Agent (LLM-Powered Privacy Compliance)

Automated Data Subject Access Request processing with multi-agent LLM orchestration for GDPR/CCPA compliance

Architecture: Classifier → Extractor → Validator → Responder (multi-agent pipeline) with RAG over policy docs and human-in-the-loop escalation

Stack: Python • LLM orchestration • RAG pipeline • Multi-agent workflow

Resource	Link
PRD	📋 prd.md
Source	📦 src/
Config	⚙️ config/

---

📋 Compliance & Regulations

Global Financial Compliance Guide

📚 Complete Guide — 42 compliance frameworks across 6 regions

Comprehensive reference covering:

Region	Key Regulations
Global	Basel III/IV, FATF, PCI DSS, ISO 27001
North America	SOX, GLBA, BSA/AML, Dodd-Frank, CCPA, NYDFS
Europe	GDPR, PSD2/PSD3, MiFID II, DORA, AMLD 5/6
Asia Pacific	RBI Master Directions, DPDP Act, PIPL, MAS TRM, CPS 234
Middle East	CBUAE, SAMA Cybersecurity, PDPL
Latin America	LGPD, BCB Resolution 4893
Africa	POPIA, SARB, NDPR

---

🛠️ Tools & Workflow

Apache Airflow

Workflow orchestration with practical DAG examples

Use Cases: ETL pipelines • ML orchestration • Data quality • Report generation

Resource	Link
Getting Started	📘 README
Core Concepts	📄 Concepts
Best Practices	📄 Guide
Example DAGs	📦 dags/

---

🗺️ Learning Paths

Path 1: System Design Fundamentals

flowchart LR
    A[Interview Guide] --> B[Patterns Reference]
    B --> C[URL Shortener]
    C --> D[Rate Limiter]
    D --> E[Leaderboard]

Loading

1. Foundation: System Design Interview Guide
2. Patterns: Distributed Patterns
3. Build: URL Shortener
4. Scale: Rate Limiter
5. Real-time: Leaderboard

Path 2: Authentication & Security

flowchart LR
    A[JWT Auth] --> B[OAuth 2.0]
    B --> C[Compliance Guide]
    C --> D[Seller Payments]

Loading

1. Basics: JWT Authentication
2. OAuth: OAuth 2.0 Demo
3. Compliance: Financial Compliance Guide
4. Apply: Seller Payment System

Path 3: Real-Time Systems

flowchart LR
    A[Event-Driven Patterns] --> B[Collaborative Editor]
    B --> C[Uber Eats Feed]
    C --> D[Crash Detection]

Loading

1. Foundation: Messaging Patterns
2. Collaboration: Collaborative Editor
3. Geo-Spatial: Uber Eats Feed
4. Streaming Ingestion: Log Ingestion

Path 4: Multi-Tenant Systems

flowchart LR
    A[Data Patterns] --> B[Issue Tracker]
    B --> C[Seller Payments]
    C --> D[Clearing House]

Loading

1. Patterns: Data Patterns
2. Multi-Tenant: Issue Tracking System
3. Payments: Seller Payment System
4. Finance: Clearing House

Path 5: Interview Preparation

flowchart LR
    A[Interview Guide] --> B[Quick Reference]
    B --> C[Case Studies]
    C --> D[Mock Practice]

Loading

1. Guide: Interview Framework
2. Reference: Cheatsheet
3. Study: Any case study from above
4. Review: Interview Experiences

---

📊 Pattern Cross-Reference

Pattern	Used In
Event Sourcing / CDC	Fintech Data Platform, E-Commerce, Collaborative Editor
CQRS	E-Commerce, Uber Eats Feed, Issue Tracking
Saga Pattern	Financial Clearing House, Seller Payments, Digital Remittance
Exactly-Once Semantics	Clearing House, Seller Payments, Leaderboard
CRDT	Collaborative Editor, Uber Cart
Redis Sorted Sets	Leaderboard, Rate Limiter
Batch + Real-time (Lambda/Kappa)	E-Commerce, YouTube Views, Log Ingestion
H3 Spatial Indexing	Uber Eats Feed, Instant Grocery
Circuit Breaker	Rate Limiter, Seller Payments, Leaderboard, Notification Service
Multi-Tenancy (RLS)	Issue Tracking System
Priority Queuing	Notification Service
HyperLogLog / Count-Min Sketch	YouTube Views
Supplier Fan-out / Aggregation	Cheapest Flights
Multi-Agent LLM Orchestration	DSAR Agent
WebSocket	Collaborative Editor, Leaderboard
OAuth 2.0 / JWT	OAuth 2.0 Demo, JWT Auth
Sharding / Partitioning	All case studies

---

🚀 Quick Start

Run OAuth 2.0 Demo

cd oauth2-demo && ./run.sh
# Auth Server: http://localhost:9001
# Resource Server: http://localhost:8080

Run Leaderboard Demo

cd leaderboard
docker-compose up -d redis zookeeper kafka postgres
./mvnw spring-boot:run
./scripts/demo-data.sh

Run Financial Clearing House Demo

# Python
cd financial-clearing-house/src/python && python3 demo.py

# Java
cd financial-clearing-house
mkdir -p target && find src/java -name "*.java" | xargs javac -d target
java -cp target com.clearinghouse.SettlementApp

Run URL Shortener

cd url-shortener && docker-compose up -d
# API available at http://localhost:8080

Run Rate Limiter

cd rate-limiter && docker-compose up -d redis && ./mvnw spring-boot:run

Run JWT Auth

cd jwt-auth && docker-compose up -d --build
# API available at http://localhost:8080

Run Products API (Cassandra)

cd products-api && docker-compose up -d --build
# API available at http://localhost:8080

Run Airflow Examples

cd apache-airflow && docker-compose up -d
# UI at http://localhost:8080 (admin/admin)

---

📝 License

MIT — See LICENSE for details.

---

Built for engineers who want depth over breadth in system design.