Warlock is a modular Reinforcement Learning framework for developing and evaluating cryptocurrency trading agents. It provides an end-to-end research pipeline — historical market data ingestion, feature engineering, a realistic portfolio simulator, and a custom Gymnasium environment — purpose-built for training and stress-testing sequence-aware RL policies such as Recurrent PPO with an LSTM backbone. Every stage is config-driven, meaning the whole experiment surface can be reshaped from a single YAML file without touching core code.

Table of Contents

Backtest Snapshot • Key Features • Architecture • Repository Structure • Pipeline • Core Modules • Reward Design • Configuration • Getting Started • Tech Stack • Roadmap • Contributing • License

---

Latest Backtest Snapshot

Metric	Weakest Checkpoint		Best Checkpoint
Sharpe Ratio	-8.18	→	0.4033
Total Return	-29.81%	→	+3.08%
Profit Factor	0.517	→	1.2639
Expectancy	-4.93	→	+1.97

Full Metric Breakdown

Category	Metric	Value
Returns	Total Return	+3.08%
	Annualized Return / CAGR	+4.42%
	Annualized Volatility	12.74%
	Final Capital	$10,307.61
Risk-Adjusted	Sharpe Ratio	0.4033
	Sortino Ratio	0.0061
	Calmar Ratio	0.5504
	Peak Capital	$10,625.75
Drawdown	Max Drawdown	8.03%
	Average Drawdown	2.54%
	Longest Drawdown (steps)	3,205
	Minimum Capital	$9,316.84
Trade Quality	Total Trades	1,264
	Closing Trades	533
	Win Rate	46.72%
	Profit Factor	1.2639
	Expectancy	+1.97

---

Key Features

Data & Features Modular data pipeline — Binance via ccxt, gap-filling, anomaly detection Feature engineering across 5 indicator families Auto-generated diagnostic plots (correlation, Sharpe, distributions)	Environment & Execution Custom Gymnasium env for sequence-aware policies Realistic spot simulator — fees, slippage, min notionals ATR-based SL/TP, dynamic sizing, drawdown protection
Agent & Training Recurrent PPO (LSTM policy) via sb3-contrib Optuna-based hyperparameter optimization Multi-seed runs for robustness validation	Reward & Analytics Rolling Sharpe-ratio objective with return blending Drawdown and overtrading penalties VectorBT-powered metrics, leaderboards, reporting

---

Architecture Overview

flowchart LR
    A[(" Data Manager")] --> B[[" Feature Engineering"]]
    B --> C{{" Gymnasium Env"}}
    C --> D[[" Portfolio Simulator"]]
    D --> E((" Recurrent PPO Agent"))
    E --> F[[" Analytics & Backtesting"]]
    F -. tune reward/features .-> B
    F -. tune hyperparameters .-> E

    classDef stage fill:#2E1065,stroke:#A78BFA,stroke-width:1.5px,color:#F5F3FF;
    classDef agent fill:#5B21B6,stroke:#C4B5FD,stroke-width:2px,color:#F5F3FF;
    class A,B,C,D,F stage;
    class E agent;

Loading

---

Repository Structure

warlock/
├── main.py                     # Runs the data + feature pipeline end-to-end
├── config.yaml                 # Single source of truth for the entire system
├── requirements.txt
│
├── src/
│   ├── data_manager/           → downloading, cleaning, anomaly detection
│   ├── features/               → indicator pipeline + feature plots
│   ├── env/                    → Gymnasium env, reward engineering
│   ├── portfolio/              → order execution, sizing, trade/equity history
│   ├── agent/                  → PPO trainer, HPO, multi-seed, evaluation
│   ├── analytics/              → checkpoint evaluation, leaderboards, reports
│   ├── benchmark/              → buy & hold / random-agent baselines
│   ├── utils/                  → config loader, seeding, path helpers
│   └── tests/                  → env, portfolio & reward verification suite
│
├── experiments/                → per-run configs, checkpoints, logs
├── graphs/features/            → auto-generated feature diagnostic plots
├── notebooks/                  → exploratory analysis
│
├── docs/
│   └── OPTUNA_HPO.md           → Hyperparameter optimization methodology, search space, pruning strategy, and experiment 
│
└── scripts/
    ├── launch_optuna.ps1       → PowerShell launcher for Optuna HPO experiments 
    └── launch_optuna.sh        → Bash launcher for Optuna HPO experiments 

---

The Pipeline

Stage	Module	What Happens
1 · Ingest	data_manager	Historical OHLCV pulled from Binance, cleaned, gap-filled, flagged for wick anomalies
2 · Engineer	features	Price, candle, momentum, volatility & volume features computed + plotted
3 · Simulate	env + portfolio	Custom Gym env wraps a realistic execution simulator (fees, slippage, ATR SL/TP)
4 · Train	agent	Recurrent PPO (LSTM) trained against the risk-aware reward signal
5 · Evaluate	analytics	Checkpoints scored, ranked on a leaderboard, and reported via VectorBT metrics
6 · Iterate	agent.hpo	Optuna sweeps hyperparameters and reward weights against evaluation results

---

Core Modules

1 · Data Management — src/data_manager/

• Downloader & Cleaner — automates historical OHLCV downloads from exchanges (e.g. Binance), with duplicate removal and missing-candle handling.
• Anomaly Detection — flags structural anomalies such as extreme wick deviations using rolling windows and configurable wick multipliers.

2 · Feature Engineering Suite — src/features/

• Builds distinct features across Price Action, Candlestick, Momentum, Volatility, and Volume categories.
• Automated feature profiling generates diagnostic plots in graphs/features/ — correlation profiles, rolling Sharpe ratios, trend strength, and distribution histograms.

3 · Custom Gymnasium Environment — src/env/

• gym_bitcoin.py implements a custom Gymnasium interface that streams price tensors and historical lookback windows into standard RL networks — including recurrent (LSTM) policies.

4 · Advanced Portfolio Simulator — src/portfolio/

• Emulates realistic spot trading: configurable maker/taker fees, slippage models, minimum trade notional limits, and rebalancing.
• Includes ATR-based Stop Loss / Take Profit, dynamic position sizing, and portfolio-level drawdown protection.
• Maintains a full trade and equity history per episode for post-hoc analysis.

5 · Reward Engineering — src/env/rewards.py

• Risk-aware reward combining immediate portfolio returns with a rolling, aggregated Sharpe-ratio objective.
• Returns are aggregated over a short window before entering the Sharpe buffer, so the ratio reflects sustained performance rather than single-tick noise.
• Additional drawdown and overtrading penalties discourage churn and excessive risk-taking in favor of stable, risk-adjusted behavior.

6 · Agent & Experimentation — src/agent/

• trainer.py — orchestrates Recurrent PPO training (sb3-contrib), environment vectorization, and callbacks.
• hpo.py — Optuna-based hyperparameter optimization.
• multi_seed.py — multi-seed runs for robustness checks.
• evaluate.py / quick_eval.py — checkpoint evaluation utilities.
• experiment.py — experiment tracking and run-directory management (see experiments/).

7 · Analytics & Reporting — src/analytics/

• Checkpoint evaluation, leaderboard generation, and cross-run comparison.
• vbt_metrics.py — VectorBT-powered performance metrics (Sharpe, returns, profit factor, expectancy).
• Report and plot generation for backtest results.

---

Reward Design

The reward function blends four signals into a single risk-adjusted scalar:

$$R_t = w_r \cdot r_t ;+; w_s \cdot \text{Sharpe}_t ;-; \lambda_{dd} \cdot \text{Drawdown}_t ;-; \lambda_{ot} \cdot \text{Overtrade}_t$$

Term	Purpose
step_return_weight · r_t	Rewards immediate, realized portfolio return
sharpe_weight · Sharpe_t	Rewards consistency of returns over a rolling, aggregated window
drawdown_penalty_scale	Penalizes portfolio-level drawdown beyond safe thresholds
overtrade_penalty_scale	Penalizes excessive turnover / churn

All four weights, plus the Sharpe window length and aggregation step size, are exposed directly in config.yaml under reward: — enabling systematic HPO sweeps over reward shape itself, not just network hyperparameters.

---

Configuration

Every module is fully decentralized and governed by a single config.yaml. This lets you instantly:

• Swap exchange, symbol, and timeframe settings
• Toggle active technical indicators
• Tune trading fees, slippage, and leverage (spot & short)
• Configure lookback/observation windows
• Adjust risk parameters (ATR-based SL/TP, max drawdown)
• Reshape the reward function without touching core code

Example config snippet

env:
  action_scale: 0.5
  window_len: 48
  max_trade_step: 0.2
  transaction_cost_rate: 0.0005
  max_drawdown: 0.3
  initial_capital: 10000.0

risk:
  stop_loss_atr_multiple: 1.5
  take_profit_atr_multiple: 3.0
  target_atr_pct: 1.0

---

Getting Started

Prerequisites

• Python 3.8+
• TA-Lib C-library dependencies (required for technical indicators)

1 · Installation

# Clone the repository
git clone https://github.com/darkisthenight07/warlock
cd warlock

# Create and activate a virtual environment
python -m venv venv
venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

2 · Run the Data & Feature Pipeline

Downloads data, builds clean feature matrices, and generates diagnostic charts in graphs/:

python main.py

3 · Train the Agent

python -m src.agent.train

4 · Component Verification Tests

# Verify reward scaling, buffer mechanics, and penalties
python -m src.tests.test_rewards

# Verify trade execution, fee charges, slippage, and liquidations
python -m src.tests.test_portfolio

# Verify Gymnasium state handling, lookback observations, step updates, and resets
python -m src.tests.test_env

---

Tech Stack

---

Roadmap

• Expand to multi-asset portfolios (ETH/USDT and beyond)
• Live / paper-trading execution bridge
• Extended HPO sweeps across reward-shaping variants
• Model export & inference API for trained checkpoints
• Walk-forward validation harness for out-of-sample robustness

---

License

Distributed under the MIT License.

---

"An edge isn't found — it's engineered, back-tested, and earned one Sharpe ratio at a time."