Skip to content

ParvJain227/FastDB

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

11 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

fastDB

fastDB is an in-memory Bank database program written in low-latency C++ with multi-threading with a Python backend.It uses a custom multi threaded architecture to process transactions asynchronously and maintain real-time indexes along with user balances

Overview

fastDB uses custom data structures(skip lists,segment trees,Thread safe queue,heaps) to minimize latency for continuous querying and database updation and insertion.

  • Engine: C++ for low-latency data insertion and querying
  • Bindings: pybind11 Python module to bind c++ code and use it as a library inside python
  • API: FastAPI server for handling web traffic and API
  • Client: HTML/CSS/JS dashboard

Performance

fastDB uses advanced data structures(skip lists and segment trees and heap) to achieve $O(1)$ and $O(\log N)$ time complexities for core operations.

Operation / Endpoint Time Complexity Underlying Data Structure
Register User (/register) $O(\log N)$ Skip List + Segment Tree
Transfer Money (/transfer) $O(1)$ API
$O(\log N)$ Engine
Thread-Safe Queue (API)
+ Skip List + Segment Trees
Check Balance (/balance/{id}) $O(1)$ Direct Array Indexing (std::vector)
Leaderboard (/leaderboard) $O(K)$ Min/Max Heap ($K$ is top_n)
Regional Wealth (/balanceinrange) $O(\log N)$ Min/Max or Sum Segment Trees

Architecture

fastDB uses a single C++ worker thread that runs continuously,isolated from the main web server thread.The Python API pushes transaction requests into a queue,and mutex locks ensure the web server and database engine don't collide.

  • Direct Memory Arrays(std::vector):User balances map directly to array indices.The engine accesses balances at balances[userid] for $O(1)$ read/write operations.
  • Thread-Safe Queues (std::queue + std::mutex):Transfer requests are pushed to a C++ queue in $O(1)$ time.A background thread processes the queue asynchronously.
  • Skip List (std::set):Balances are stored in nodes in skip list(multiple linked list )data structure.On a transaction,the old balance is removed and the new one inserted in $O(\log N)$ time to keep global rankings sorted for leaderboard queries.
  • Segment Trees:The engine uses three concurrent Segment Trees for wealth queries in a range (Sum, Min, Max).These update on every transaction $O(\log N)$ time complexity.

Tech Stack

  • Core Engine: C++11
  • Python Bindings: pybind11
  • Backend API: Python 3.8+,FastAPI,Uvicorn
  • Frontend UI: HTML5,CSS3,Vanilla JavaScript

Prerequisites

  • A C++ compiler
  • Python 3.8+
pip install fastapi uvicorn pybind11

Commands to run

  • step-1 c++ -O3 -Wall -shared -std=c++11 -fPIC$(python3 -m pybind11 --includes) main.cpp -o fastdb$(python3-config--extension-suffix)
  • step-2 uvicorn main:app --reload
  • step-3open in a new terminal and run = python3 -m http.server 8080
image

About

fastDB is an ultra-fast,in-memory financial bank system built from scratch in low latency multi-threaded C++ with a python backend .It replaces the use of SQL entirely,using custom memory arrays, thread-safe queues,skip list,segment trees/heap/ to process live transactions and rankings instantly and updates the frontend using FastAPI.

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages