Crypto Trading System - Complete Documentation

Table of Contents

1. Project Overview
2. Quick Start Guide
3. System Architecture
4. Core Components
5. CI/CD
6. Development Tools & Automation
7. Performance & Testing
8. Contact & Contribution

---

---

Project Overview

Mission Statement

The Crypto Trading System is a high-performance, multi-threaded cryptocurrency trading simulation platform built with modern C++ and Python integration. It demonstrates advanced software engineering practices including real-time data processing, automated code generation, comprehensive parameter validation, and sophisticated development tooling.

Key Features

• Multi-threaded Architecture: 3 dedicated threads for market data, trading logic, and execution
• Real-Time Data Integration: Live Bitcoin market data from Binance API with fallback simulation
• Thread-safe Communication: Hybrid architecture with socket-based Python-C++ communication and internal C++ thread synchronization using SafeQueue with mutex and condition variables
• Automated Parameter Validation: Template-based compile-time type checking and logging
• Dynamic Strategy Management: Configurable trading strategies with automated code generation
• Advanced Development Tools: Python-based automation for logging, validation, and deployment
• One-command Execution: Complete automation from development to production
• Error Handling: Comprehensive validation with ErrorLogger and graceful error recovery
• Modern C++ Compatibility: Uses C++17 features for maximum type safety and performance
• Real-time Performance Auditing: Dedicated Python monitor tracking CPU, Memory, and Handles with automated trend chart generation.

System Specifications

• Runtime: Configurable trading duration (default: 30 seconds via config.yaml)
• Data Processing: Real-time Bitcoin price updates from Binance with CSV logging
• Trading Strategies: Multiple algorithms (SMA, Bollinger Bands, Momentum RSI)
• Portfolio Management: Real-time P&L tracking and portfolio analytics
• Error Handling: Multi-layered validation with parameter checking and exception logging
• Logging: Custom logging with detailed execution tracing

---

Quick Start Guide

Prerequisites

Cross-Platform Compatibility:

• Windows: MSVC or MinGW with C++17 support, Winsock2 library
• Linux: GCC with C++17 support, POSIX socket support

Required Software:

# Core Requirements
- C++ compiler with C++17 support
- Python 3.x
- python3-pip
- Make
- Docker (optional)

# Python Dependencies
pip3 install jinja2 pyyaml requests

Configuration

Edit config/strategy_config.yaml to customize: （Requires Rebuild)

# Strategy Selection (uncomment desired strategy)
selected_class: SimpleMovingAverageStrategy # Active algorithm
# selected_class: BollingerBandsStrategy
# selected_class: MomentumRSIStrategy

obj_name: simpleMovingAverage               # Strategy instance name
# obj_name: bollingerBands
# obj_name: momentumRSI

Edit config/config.cfg to customize: (No Rebuild Required)

# Trading Configuration
RUN_DURATION=60
DEFAULT_CASH=10000.0
MAX_HISTORY=70
MIN_HISTORY=10
# 0=Main, 1=MarketData, 2=Strategy, 3=Execution, 4=DEBUG...
LOG_LEVEL=0

Edit functionEnhanced.yaml to adds parameter validation and error handling to existing C++ code: （Requires Rebuild)

Format

filename.cpp:
  - ReturnType ClassName::FunctionName(param1, param2, ...)
  - ReturnType ClassName::AnotherFunction(param1, param2, ...)

another_file.cpp:
  - ReturnType Function(parameters)

Example Configuration

TradeExecutor.cpp:
  - bool TradeExecutor::ExecuteBuyOrder(price, amount)
  - bool TradeExecutor::HandleActionSignal(action, price, amount)

TradingStrategy.cpp:
  - ActionType TradingStrategy::CalculateSimpleMovingAverageStrategy(priceHistory)

One-Command Deployment

The system features a "Smart Automation" layer via RunTradeSystem.py. This script acts as the central orchestrator, managing the lifecycle of both C++ and Python components.

Automated Workflow

By running a single command, the system automatically evaluates which parts of the environment need to be refreshed:

# Windows or Linux
python RunTradeSystem.py

1. The "Smart" Build Logic

The script performs a dependency check and enhancement sequence:

• Static Config Check (.yaml): If you changed the strategy in strategy_config.yaml or added functions to functionEnhanced.yaml, the script triggers:

  • Code Generation: Re-runs Jinja2 templates to update StrategyWrapper.cpp.

  • Logic Injection: Re-runs Add_check_all.py and CppLogInjector.py to bake safety and logging into the source.

  • Compilation: Calls make all to produce a new trading_system binary.

• Dynamic Config Check (.cfg): If you only changed values in config.cfg (like RUN_DURATION or LOG_LEVEL), the script recognizes that the binary is already up-to-date and proceeds directly to execution, saving time.

2. Execution Orchestration

Once the binary is ready, the script manages the multi-process startup sequence:

1. Launch Server: Starts the C++ trading_system.

2. Attach Monitor: Launches the Performance Monitor to track the PID of the C++ process.

3. Launch Client: Starts MarketFetch.py to begin streaming Binance data via Sockets.

4. Graceful Shutdown: After the RUN_DURATION (defined in config.cfg) expires, the script sends a signal to all processes to close cleanly and save logs.

3. Post-Run Analysis

Immediately after the system stops, the deployment script:

• Aggregates all logs into the build_result/ folder.

• Triggers plot_performance.py to generate visual reports (.png) of CPU and Memory usage.

Manual Deployment

# 1. Cleanup previous builds
rm -f src/*.cpp.bak src/TradeStrategy/*.cpp.bak

# 2. Configuration and code generation
python3 utilLocal/GenerateStrategy/generate_code.py

# 3. Optional enhancements
python3 utilLocal/CppLogInjector.py
python3 tools/Add_check_all.py src

# 4. Build system
make clean && make all

# 5. Execution (two terminals required)
# Terminal 1: Trading system (start first as server)
./output/trading_system

# Terminal 2: Market data feed (start after trading system is running)
python3 src/MarketFetch.py

# Terminal 3: Performance Monitor (start after trading system is running)
pythons tools/performance_monitor/run_monitor.py

---

System Architecture

Overview

The Crypto Trading System implements a hybrid multi-process architecture combining C++ multi-threading with Python real-time data acquisition. The system uses socket-based inter-process communication, thread-safe mechanisms, and a sophisticated strategy pattern.

Architecture Diagram

┌─────────────────────────────────────────────────────────────────────────────────────┐
│                           Main Thread (C++ - Thread 1)                              │
│  ┌─────────────┐  ┌──────────────┐  ┌─────────────┐  ┌─────────────────────────┐   │
│  │   System    │  │ Create Shared│  │  Wait for   │  │   Display Final         │   │
│  │ Initialize  │  │ Components & │  │ WAIT_SECONDS│  │   Portfolio Results     │   │
│  │   & Launch  │  │Launch Threads│  │             │  │                         │   │
│  └─────────────┘  └──────────────┘  └─────────────┘  └─────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────────────────┘
           │                    │                    │
           │                    │                    │
    ┌─────────────┐      ┌─────────────┐      ┌─────────────┐
    │  Python     │      │C++ Thread 2 │      │C++ Thread 3 │
    │MarketFetch  │◄────►│ Strategy    │─────►│ Trade       │
    │   Process   │Socket│ Engine +    │Queue │ Executor    │
    │             │      │SocketServer │      │             │
    └─────────────┘      └─────────────┘      └─────────────┘
           │                     │                     │
    ┌─────────────┐      ┌─────────────┐      ┌─────────────┐
    │   Binance   │      │ Strategy    │      │ Portfolio   │
    │ BTCUSDT API │      │ Wrapper     │      │ Management  │
    │ + CSV Log   │      │ Pattern     │      │ + P&L Calc  │
    └─────────────┘      └─────────────┘      └─────────────┘

Performance Monitoring Layer

The system now includes a non-intrusive monitoring suite located in tools/performance_monitor/:

1. run_monitor.py: Attaches to the trading_system PID to collect high-frequency metrics.
2. plot_performance.py: Post-processes CSV data into professional trend charts using Pandas and Matplotlib.
3. build_result/: A centralized directory for all execution artifacts, including raw_performance.csv and performance PNG reports.

Component Responsibilities

Main Thread (C++ - main.cpp)

• System Orchestration: Initialize logging, create shared SafeQueue instances
• Component Creation: Create StrategyEngine and TradeExecutor with shared ActionSignal queue
• Thread Management: Launch strategy and execution threads using std::thread
• Wait Management: Sleep for WAIT_SECONDS (configurable from config.yaml)
• Final Reporting: Display portfolio status and P&L from TradeExecutor

Key Constants (Auto-updated from config/config.cfg): (Read from file)

RUN_DURATION=60
DEFAULT_CASH=10000.0
MAX_HISTORY=70
MIN_HISTORY=10
# 0=Main, 1=MarketData, 2=Strategy, 3=Execution, 4=DEBUG...
LOG_LEVEL=0

Python Process: MarketFetch.py

• Live Data Fetching: Primary Binance BTCUSDT API with 1-second intervals
• Data Management: 2000-entry rotating deque buffer with CSV persistence
• Socket Client: TCP connection to localhost:9999 with JSON messaging
• Error Handling: 5-second timeouts, automatic reconnection, comprehensive logging

Data Sources:

# Primary: Binance API
url = "https://api.binance.com/api/v3/ticker/price"
params = {"symbol": "BTCUSDT"}

Thread 2: Strategy Engine (C++)

• Socket Server: Cross-platform TCP server (Windows Winsock2/Linux POSIX)
• Message Processing: JSON parsing with buffer management for newline-delimited messages
• Price History: Sliding window using std::deque<double> with configurable MAX_HISTORY and MIN_HISTORY
• Strategy Integration: StrategyWrapper pattern with auto-generated strategy selection
• Signal Generation: ActionSignal creation for trade recommendations sent to SafeQueue

Thread 3: Trade Executor (C++)

• Signal Processing: Consumes ActionSignal from SafeQueue
• Order Execution: Buy/sell validation with balance/holdings checks
• Portfolio Tracking: Real-time cash and BTC balance management
• Transaction Analytics: Complete trade statistics and P&L calculations

Portfolio State Management:

class TradeExecutor {
    double initialFiatBalance_;     // Starting capital ($10,000)
    double currentFiatBalance_;     // Available cash
    double cryptoAssetAmount_;      // BTC holdings
    uint32_t totalTrades_;          // Total transactions
    uint32_t totalBuyAction_;       // Buy order count
    uint32_t totalSellAction_;      // Sell order count
};

---

Core Components

Thread-Safe Communication Layers

1. Inter-Process Communication (Python ↔ C++)

• Protocol: TCP socket communication on localhost:9999
• Data Format: JSON messages with newline delimiters

// Connection: Python Client ←TCP→ C++ Server (localhost:9999)
{"symbol": "BTC", "price": 29847.52, "timestamp": 1692284400.123}\n

• Direction: Python MarketFetch.py (client) → C++ StrategyEngine (server)
• Purpose: Real-time market data transmission

2. Intra-Process Communication (C++ Threads)

• Mechanism: SafeQueue template with mutex and condition variables

• Direction: StrategyEngine (Thread 2) → TradeExecutor (Thread 3)

• Data Type: ActionSignal objects containing trade recommendations

• Thread Safety: Blocking dequeue operations with proper synchronization

  template<typename T>
  class SafeQueue {
    std::queue<T> queue_;
    std::mutex mutex_;
    std::condition_variable condition_;
  public:
    void enqueue(const T& item);    // Thread-safe insertion
    T dequeue();                    // Thread-safe removal with blocking
    bool empty() const;             // Thread-safe size check
  };
  // Inter-thread communication via SafeQueue
  SafeQueue<ActionSignal> actionSignalQueue;
  // Thread 2: Strategy Engine enqueues signals
  ActionSignal signal{ActionType::BUY, price, amount, timestamp};
  actionSignalQueue.enqueue(signal);
  // Thread 3: Trade Executor dequeues and processes
  ActionSignal signal = actionSignalQueue.dequeue(); // Blocks until available

Strategy Pattern Implementation

Dynamic Strategy Selection via Code Generation

The system uses Jinja2 templates to generate StrategyWrapper.cpp and StrategyWrapper.h based on config.yaml:

// strategy_wrapper_impl.cpp.jinja2
IStrategy* StrategyWrapper::strategy_ = nullptr;

void StrategyWrapper::initialize() 
{
    if (!strategy_) {
        static {{ selected_class }} {{ obj_name }};
        strategy_ = &{{ obj_name }};
    }
}

// Auto-generated StrategyWrapper.cpp
IStrategy* StrategyWrapper::strategy_ = nullptr;

void StrategyWrapper::initialize() {
    if (!strategy_) {
        static SimpleMovingAverageStrategy simpleMovingAverage;
        strategy_ = &simpleMovingAverage;
    }
}

Available Strategies

• SimpleMovingAverageStrategy: Classic SMA crossover signals
• BollingerBandsStrategy: Mean reversion with volatility bands
• MomentumRSIStrategy: Momentum-based relative strength analysis

Cross-Platform

Cross-Platform Build System Integration:

# Cross-platform makefile supporting both Linux and Windows
# Works with GCC on Linux and MinGW/MSVC on Windows

# Platform detection
ifeq ($(OS),Windows_NT)
    PLATFORM = WIN32
    CXX = g++
    LDFLAGS += -lws2_32  # Windows socket library
else
    PLATFORM = POSIX
    CXX = g++
    LDFLAGS += -lpthread  # Linux threading library
endif

Cross-Platform Socket Implementation:

#ifdef _WIN32
    #include 
    #define CLOSESOCKET closesocket
#else
    #include 
    #define SOCKET int
    #define CLOSESOCKET close
#endif 

---

CI/CD Pipeline

The project uses a GitHub Actions CI/CD pipeline to automate the build, test, and deployment processes. The pipeline is defined in main.yml and consists of two main stages: docker_build and build_and_test. The pipeline runs automatically on every push and pull request.

1. Docker Build Stage

This stage creates a custom Docker image to serve as the build and test environment, using Dockerfile_v1.2 to ensure a consistent and isolated environment.

• Dockerfile: Based on ubuntu:22.04, installs all necessary dependencies including git, build-essential, python3, pip3, and Python libraries pyyaml, jinja2, requests, and psutil (required by the performance monitor).
• Image Creation: The build_image job logs into the GitHub Container Registry, builds the Docker image, and tags it with both the commit SHA and the branch name.
• Push to Registry: The image is pushed to the GitHub Container Registry for use in the next stage.

2. Build and Test Stage

This stage uses the custom Docker image to run the full trading system lifecycle — build, execute, monitor, and validate.

• Execution: Runs python RunTradeSystem.py, which handles code generation, C++ compilation, and orchestrates the trading system, market data feed, and performance monitor as coordinated processes.

• Performance Monitoring: While the trading system runs, run_monitor.py attaches to the process and samples memory, thread count, handle count, and context switches at regular intervals, writing results to raw_performance.csv and trend_performance.csv.

• Regression Gate: After the run completes, check_regression.py analyses the trend data and automatically fails the pipeline if any metric:

  • exceeds its absolute threshold (memory > 10MB, threads > 12, handles > 150), or
  • shows a statistically significant upward slope (linear regression slope > 0.05 per trend-point)

  Thresholds are derived from the system's actual baseline with a 2× buffer. A merge is blocked until the regression is resolved.

• Artifacts: The following are saved as pipeline artifacts and retained for one week:

  • build_result/result.txt — trading logs and final P&L
  • build_result/market_data.csv — live Binance price data captured during the run
  • build_result/raw_performance.csv — second-by-second resource samples
  • build_result/trend_performance.csv — aggregated trend data used by the regression gate
  • build_result/report_raw_detail.png — line charts showing per-second resource fluctuations
  • build_result/report_trend_summary.png — trend chart highlighting overall system stability

  Artifacts are uploaded regardless of pipeline result (if: always()), so performance data is always available for inspection even when the regression gate triggers a failure.

🛠 Development Tools & Automation

To achieve high engineering maturity and system reliability, I implemented a suite of Python-based automation tools. This "Infrastructure as Code" approach eliminates manual repetition and enforces rigorous safety standards.

1. Automated Defensive Guardrails (Add_check_all.py)

Instead of manually wrapping hundreds of functions, this orchestration tool automatically injects a non-invasive defensive layer based on functionEnhanced.yaml.

• Static Analysis & Injection: Parses C++ source files to inject check_all validation headers and wraps core logic in try-catch blocks.

• Compile-Time Type Dispatching: The core engine (ParameterCheck.h) utilizes C++17 if constexpr for zero-overhead validation.

• Fail-Fast Mechanism: Uses static_assert to catch unsupported types during compilation, preventing unprotected data from entering the trading loop.

// Logic automatically injected by Add_check_all.py
bool TradeExecutor::ExecuteBuyOrder(double price, double amount) {
    if (!check_all("ExecuteBuyOrder", price, amount)) return false; // Zero-cost validation
    try {
        /* Original trading logic */
    } catch (const std::exception& e) {
        ErrorLogger::LogError("TradeExecutor", "ExecuteBuyOrder", "std::exception", e.what());
        return false;
    }
}

2. Automated Execution Tracing (CppLogInjector.py)

Provides "transparent" observability across the multi-threaded system without manual code changes.

• Instrumentation: Automatically injects entry (start) and exit (stop) logging statements at the scope boundaries of C++ functions.

• Traceability: Enables microsecond-level tracking of the execution flow across market data, strategy, and execution threads.

  // Before transformation
  bool TradeExecutor::ExecuteBuyOrder(double price, double amount) 
  {
      // Original logic
  }
  
  // After transformation
  bool TradeExecutor::ExecuteBuyOrder(double price, double amount) 
  {
      LOG(CustomerLogLevel::INFO) << " This is start of " << __FILE__ << "::" << __FUNCTION__;
      // Original logic
      LOG(CustomerLogLevel::INFO) << " This is stop of " << __FILE__ << "::" << __FUNCTION__;
  }

3. Smart Build & Orchestration (RunTradeSystem.py)

A central orchestrator that manages the entire system lifecycle:

• Dependency-Aware Rebuilds: Detects changes in .yaml or .cfg files to trigger code generation (Jinja2) or logic injection only when necessary.

• Multi-Process Coordination: Synchronizes the startup of the C++ Server, Python Market Client, and the Real-time Performance Monitor.

Performance & Testing

System Specifications

Real-Time Performance Metrics

• Data Latency: Sub-second market data delivery (Binance API → Strategy processing)
• CPU Optimization: O(1) sliding window operations using std::deque
• Network Resilience: 5-second timeouts, automatic fallback to simulation
• Memory Management: Rotating 2000-entry deque buffer, automatic cleanup

Current System Specifications

• Runtime Duration: 30 seconds (configurable via tradeTime)
• Price History Window: 70 data points maximum (maxHistory)
• Strategy Activation: 10 data points minimum (minHistory)
• Default Portfolio: $10,000 starting capital (DEFAULT_CASH)
• Trade Size: 0.01 BTC per transaction (hardcoded)
• Data Frequency: 1-second intervals from Binance API

Automated Performance Auditing

The system features an integrated monitoring suite that captures real-time metrics during execution.

• High-Frequency Monitoring: Tracks CPU, Memory, and System Handles every second.
• Trend Analysis: Aggregates raw data into smooth trend lines to detect long-term resource leaks.
• Visual Reporting: Automatically generates professional charts after each run.

🔴 Live Monitoring: To view real-time system performance, visit the Performance Monitoring Center

---

Contact & Contribution

I welcome feedback, questions, and contributions. You can connect with the project maintainer through the following channels:

• GitHub Branch: https://github.com/Celia2024-bit/TradeSystem.git
• Email: qiang.qiaoyan@gmail.com

This documentation accurately reflects the implemented Crypto Trading System, including the actual error handling mechanisms, configuration management, and development automation tools as shown in the provided code files.