Levels

A multi-level time-series database for financial market data -- orderbook events, price streaming, and OHLCV bars. Built on .NET 10.0 with cross-language support for C++, Python, and TypeScript.

Features

• Custom binary format -- 56-byte core records with schema-extensible fields, CRC32 integrity, little-endian layout
• Three operational modes -- embed in C# apps, run as a standalone TCP server, or deploy as a REST API
• Automatic data lifecycle -- compaction, period promotion with data quality checks, configurable retention and archival
• Cross-language support -- generate readers/writers for C#, Python, TypeScript, and C++ from a single .fbs schema
• Real-time and batch -- OHLCV bar emission, top-of-book snapshots, orderbook L1/L2/L3 projection
• Export -- CSV, Parquet, and Avro adapters
• Observability -- OpenTelemetry metrics and tracing, Prometheus scraping endpoint

Prerequisites

• .NET 10.0 SDK (preview)

Quick Start

# Build
dotnet build

# Run tests
dotnet test

# Run the sample (writes synthetic orderbook data)
dotnet run --project samples/CryptoExchangeSample

See the Quickstart Guide for a full walkthrough.

Operational Modes

Mode	Use case	Entry point
Embedded	In-process C# ingestion	AddLevels<TSchema>() + AddLevelsDataSink<T>()
TCP Server	Standalone daemon, multi-language clients	src/Levels.Server with YAML config
REST API	HTTP queries and export	src/Levels.Web with ASP.NET Core

CLI

The levels CLI tool provides file inspection, compaction, code generation, resampling, and data export.

dotnet run --project src/Levels.Cli -- --help

See the CLI Reference for all commands.

Documentation

• Quickstart -- get up and running
• Architecture -- data pipeline, binary format, system internals
• Embedded Mode -- in-process C# usage
• Server Mode -- standalone TCP server
• Web API -- REST endpoints
• Schemas -- FlatBuffers schema system and code generation
• Configuration -- complete options reference
• CLI -- command-line tool reference
• Cross-Language -- C++, Python, TypeScript interop

Project Structure

Levels/
  src/
    Levels.Core/           Core binary format, interfaces, WAL, diagnostics
    Levels.SourceGen/      Roslyn source generator for .fbs schemas (netstandard2.0)
    Levels.Sinks/          Partitioned ingestion, circuit breaker, backpressure
    Levels.DataFlow/       Channel-based async pub/sub bus
    Levels.Query/          In-memory file index, orderbook projection
    Levels.Compaction/     Event-sourcing replay, synthetic snapshots, archival
    Levels.Period/         Data quality checks, promotion/demotion
    Levels.Resampled/      OHLCV bars, top-of-book snapshots
    Levels.Export/         CSV, Parquet, Avro adapters
    Levels.Hosting/        DI registration (AddLevels), OpenTelemetry setup
    Levels.Hints/          LiteDB-based query hints
    Levels.Protocol/       Wire protocol (frame reader/writer)
    Levels.Server/         Standalone TCP server
    Levels.Client/         TCP client library
    Levels.Web/            ASP.NET Core REST API
    Levels.Cli/            Command-line tool
  samples/
    CryptoExchangeSample/  Simulated exchange (synthetic data)
    BinanceLiveSample/     Live Binance WebSocket feed
  tests/
    Levels.Tests/          xUnit tests + cross-language round-trips
    Levels.Benchmarks/     BenchmarkDotNet performance suite
  cross-language/
    schema.fbs             Canonical schema definition
    cpp/                   C++ generated code and tests
    python/                Python generated code and tests
    typescript/            TypeScript generated code and tests

License

Apache-2.0