Skip to content

fix: update workflows and docs for Render deployment #26

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
SamoraDC merged 1 commit into from
Jan 22, 2026
Merged

fix: update workflows and docs for Render deployment #26

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 42 additions & 9 deletions .github/workflows/daily-report.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,20 +25,41 @@ jobs:
run: |
pip install -r reports/requirements.txt

- name: Download database from Fly.io
- name: Fetch data from Render API
env:
FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}
RENDER_SERVICE_URL: ${{ secrets.RENDER_SERVICE_URL }}
run: |
# Install flyctl
curl -L https://fly.io/install.sh | sh
export FLYCTL_INSTALL="/home/runner/.fly"
export PATH="$FLYCTL_INSTALL/bin:$PATH"
# Use secret URL or fallback to default service name
SERVICE_URL="${RENDER_SERVICE_URL:-https://orp-flow-trading.onrender.com}"

# Download database file
echo "📊 Fetching trading data from: $SERVICE_URL"
mkdir -p data
flyctl ssh console -C "cat /data/trades.db" > data/trades.db 2>/dev/null || echo "No database found, using empty"

# Fetch metrics
echo "Fetching /metrics..."
curl -s "$SERVICE_URL/metrics" --connect-timeout 30 --max-time 60 > data/metrics.json || echo '{}' > data/metrics.json

# Fetch trades history
echo "Fetching /trades..."
curl -s "$SERVICE_URL/trades" --connect-timeout 30 --max-time 60 > data/trades.json || echo '[]' > data/trades.json

# Fetch account status
echo "Fetching /account..."
curl -s "$SERVICE_URL/account" --connect-timeout 30 --max-time 60 > data/account.json || echo '{}' > data/account.json

# Show fetched data summary
echo ""
echo "📈 Fetched data summary:"
echo "Metrics:"
cat data/metrics.json | jq . 2>/dev/null || cat data/metrics.json
echo ""
echo "Trades count: $(cat data/trades.json | jq 'length' 2>/dev/null || echo 'N/A')"
echo "Account:"
cat data/account.json | jq . 2>/dev/null || cat data/account.json

- name: Generate report
env:
DATA_SOURCE: api
run: |
python reports/generate.py

Expand All @@ -59,7 +80,19 @@ jobs:
run: |
if [ -n "$TELEGRAM_BOT_TOKEN" ] && [ -n "$TELEGRAM_CHAT_ID" ]; then
# Extract key metrics for notification
MESSAGE="📊 QuantumFlow Daily Report Generated\n\nCheck the updated README for performance metrics."
TOTAL_TRADES=$(cat data/metrics.json | jq -r '.total_trades // 0' 2>/dev/null || echo "0")
WIN_RATE=$(cat data/metrics.json | jq -r '.win_rate // 0' 2>/dev/null || echo "0")
TOTAL_PNL=$(cat data/metrics.json | jq -r '.total_pnl // 0' 2>/dev/null || echo "0")

MESSAGE="📊 ORPFlow Daily Report Generated

📈 Performance Summary:
• Total Trades: $TOTAL_TRADES
• Win Rate: ${WIN_RATE}%
• Total P&L: \$${TOTAL_PNL}

Check the updated README for full metrics and charts."

curl -s -X POST "https://api.telegram.org/bot${TELEGRAM_BOT_TOKEN}/sendMessage" \
-d "chat_id=${TELEGRAM_CHAT_ID}" \
-d "text=${MESSAGE}" \
Expand Down
54 changes: 38 additions & 16 deletions .github/workflows/health-check.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,40 +14,58 @@ jobs:
steps:
- name: Check API Health
id: health
env:
RENDER_SERVICE_URL: ${{ secrets.RENDER_SERVICE_URL }}
run: |
response=$(curl -s -o response.json -w "%{http_code}" https://quantumflow-hft.fly.dev/health || echo "000")
# Use secret URL or fallback to default service name
SERVICE_URL="${RENDER_SERVICE_URL:-https://orp-flow-trading.onrender.com}"

echo "Checking health at: $SERVICE_URL/health"
response=$(curl -s -o response.json -w "%{http_code}" "$SERVICE_URL/health" --connect-timeout 30 --max-time 60 || echo "000")

if [ "$response" = "200" ]; then
echo "status=healthy" >> $GITHUB_OUTPUT
echo "API is healthy"
cat response.json
echo "✅ API is healthy"
cat response.json | jq . 2>/dev/null || cat response.json
else
echo "status=unhealthy" >> $GITHUB_OUTPUT
echo "API returned status: $response"
echo "❌ API returned status: $response"
cat response.json 2>/dev/null || echo "No response body"
fi

- name: Check if Shabbat pause
id: shabbat
if: steps.health.outputs.status == 'healthy'
run: |
status=$(cat response.json | jq -r '.details.shabbat_pause // false')
status=$(cat response.json | jq -r '.details.shabbat_pause // false' 2>/dev/null || echo "false")
echo "shabbat_pause=$status" >> $GITHUB_OUTPUT
echo "Shabbat pause: $status"
echo "🕯️ Shabbat pause: $status"

- name: Get system metrics
if: steps.health.outputs.status == 'healthy'
env:
RENDER_SERVICE_URL: ${{ secrets.RENDER_SERVICE_URL }}
run: |
SERVICE_URL="${RENDER_SERVICE_URL:-https://orp-flow-trading.onrender.com}"

echo "📊 Fetching metrics..."
curl -s "$SERVICE_URL/metrics" --connect-timeout 10 | jq . 2>/dev/null || echo "Metrics not available"

- name: Restart if unhealthy
if: steps.health.outputs.status == 'unhealthy'
env:
FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}
RENDER_DEPLOY_HOOK_URL: ${{ secrets.RENDER_DEPLOY_HOOK_URL }}
run: |
echo "System unhealthy, attempting restart..."

# Install flyctl
curl -L https://fly.io/install.sh | sh
export FLYCTL_INSTALL="/home/runner/.fly"
export PATH="$FLYCTL_INSTALL/bin:$PATH"
echo "⚠️ System unhealthy, attempting restart..."

# Restart the app
flyctl apps restart quantumflow-hft
if [ -n "$RENDER_DEPLOY_HOOK_URL" ]; then
# Trigger redeploy via Render deploy hook
curl -X POST "$RENDER_DEPLOY_HOOK_URL"
echo "🔄 Redeploy triggered via Render deploy hook"
else
echo "⚠️ RENDER_DEPLOY_HOOK_URL not configured. Cannot trigger restart."
echo "Set up a deploy hook in Render Dashboard → Settings → Deploy Hook"
fi

- name: Send alert on failure
if: failure()
Expand All @@ -56,7 +74,11 @@ jobs:
TELEGRAM_CHAT_ID: ${{ secrets.TELEGRAM_CHAT_ID }}
run: |
if [ -n "$TELEGRAM_BOT_TOKEN" ] && [ -n "$TELEGRAM_CHAT_ID" ]; then
MESSAGE="⚠️ QuantumFlow Health Check Failed\n\nThe system may be down. Automatic restart attempted."
MESSAGE="⚠️ ORPFlow Health Check Failed

The system may be down. Automatic restart attempted.

Check: https://dashboard.render.com"
curl -s -X POST "https://api.telegram.org/bot${TELEGRAM_BOT_TOKEN}/sendMessage" \
-d "chat_id=${TELEGRAM_CHAT_ID}" \
-d "text=${MESSAGE}" \
Expand Down
147 changes: 88 additions & 59 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,18 +1,19 @@
# ORPflow - HFT Paper Trading

> **O**Caml + **R**ust + **P**ython Flow
> **O**Caml + **R**ust + **P**erformance Flow

[![CI](https://github.com/SamoraDC/ORPflow/actions/workflows/ci.yml/badge.svg)](https://github.com/SamoraDC/ORPflow/actions/workflows/ci.yml)
[![Health Check](https://github.com/SamoraDC/ORPFlow/actions/workflows/health-check.yml/badge.svg)](https://github.com/SamoraDC/ORPFlow/actions/workflows/health-check.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Render](https://img.shields.io/badge/Deploy-Render-46E3B7)](https://render.com)

A high-frequency paper trading system demonstrating multi-language systems engineering with **OCaml**, **Rust**, and **Python**. Designed as a portfolio project showcasing low-latency market data processing, type-safe risk management, and quantitative trading strategies.
A high-frequency paper trading system demonstrating multi-language systems engineering with **OCaml** and **Rust**. Built following **Jane Street architecture principles**: no interpreted languages in the hot path. Uses ONNX Runtime for ML inference directly in Rust.

## Features

- **Real-time Market Data** (Rust): WebSocket connection to Binance with order book reconstruction
- **Type-Safe Risk Engine** (OCaml): Position limits, drawdown circuit breakers, P&L calculation
- **Trading Strategy** (Python): Order flow imbalance strategy with volatility adjustment
- **Unified Trading Engine** (Rust): WebSocket market data, order book, strategy, paper broker, REST API - all in one binary
- **ML Inference in Hot Path** (Rust + ONNX): LightGBM, XGBoost, CNN, LSTM models loaded via ONNX Runtime
- **Type-Safe Risk Gateway** (OCaml): Position limits, drawdown circuit breakers, P&L validation
- **Paper Trading**: Realistic execution simulation with slippage, market impact, and fees
- **Shabbat Pause**: Automatic trading pause from Friday to Saturday sunset
- **Live Dashboard**: Auto-updating README with performance charts via GitHub Actions
Expand All @@ -26,41 +27,47 @@ A high-frequency paper trading system demonstrating multi-language systems engin
## Architecture

```
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Binance │────▶│ Rust │────▶│ OCaml │
│ WebSocket │ │ Market Data │ │ Risk Engine │
└─────────────┘ └─────────────┘ └─────────────┘
│ │
└───────┬───────────┘
▼
┌─────────────────┐
│ Python │
│ Strategy Engine │
│ + REST API │
└────────┬────────┘
│
┌────────▼────────┐
│ SQLite │
│ Database │
└─────────────────┘
┌─────────────┐ ┌────────────────────────────────────────┐
│ Binance │────▶│ Rust (Unified) │
│ WebSocket │ │ ┌──────────┐ ┌──────────┐ ┌──────┐ │
└─────────────┘ │ │ WebSocket│ │ Strategy │ │ REST │ │
│ │ + Order │──▶│ + Paper │──▶│ API │ │
│ │ Book │ │ Broker │ │:8000 │ │
│ └──────────┘ └──────────┘ └──────┘ │
│ │ │ │
│ ▼ ▼ │
│ ┌──────────────────────────────────┐ │
│ │ ONNX Runtime (ML) │ │
│ │ LightGBM XGBoost CNN LSTM │ │
│ └──────────────────────────────────┘ │
└──────────────────┬─────────────────────┘
│ IPC
▼
┌─────────────────┐
│ OCaml │
│ Risk Gateway │
│ (Validation) │
└─────────────────┘
```

### Why This Tech Stack?
### Why This Tech Stack? (Jane Street Style)

| Component | Language | Reason |
|-----------|----------|--------|
| Market Data | **Rust** | Zero-cost abstractions, memory safety, excellent async runtime |
| Risk Engine | **OCaml** | Algebraic data types prevent invalid states, exhaustive pattern matching |
| Strategy | **Python** | Rapid prototyping, rich data science ecosystem, easy modification |
| Unified Engine | **Rust** | Zero-cost abstractions, async I/O, ONNX bindings, no GC pauses |
| ML Inference | **ONNX** | Models trained in Python, deployed in Rust at microsecond latency |
| Risk Gateway | **OCaml** | Algebraic types prevent invalid states, exhaustive pattern matching |

**No Python in Hot Path**: All ML models are exported to ONNX format during training (offline), then loaded by Rust for real-time inference. This eliminates Python interpreter overhead in the trading loop.

## Quick Start

### Prerequisites

- Rust 1.75+
- Rust 1.85+ (for edition2024 support)
- OCaml 5.1+ with opam
- Python 3.11+
- Docker (optional)
- ONNX Runtime 1.19+
- Docker (recommended for deployment)

### Setup

Expand All @@ -86,41 +93,42 @@ docker-compose up --build

```
ORPflow/
├── market-data/ # Rust - WebSocket & order book
├── market-data/ # Rust - Unified trading engine
│ └── src/
│ ├── websocket/ # Connection management
│ ├── parser/ # Message deserialization
│ ├── orderbook/ # Order book data structure
│ └── publisher/ # IPC to other components
│ ├── main.rs # Entry point + REST API (port 8000)
│ ├── websocket/ # Binance WebSocket connection
│ ├── orderbook/ # Order book reconstruction
│ ├── strategy/ # Trading strategy logic
│ ├── broker/ # Paper trading execution
│ └── ml/ # ONNX model loading & inference
│
├── core/ # OCaml - Risk engine
├── core/ # OCaml - Risk gateway
│ └── lib/
│ ├── types/ # Domain types (Order, Trade, Position)
│ ├── orderbook/ # Type-safe order book
│ ├── risk/ # Risk validation & limits
│ └── pnl/ # P&L calculation
│
├── strategy/ # Python - Trading strategy
│ └── src/
│ ├── signals/ # Trading strategies
│ ├── features/ # Microstructure features
│ ├── broker/ # Paper broker
│ ├── api/ # REST API (FastAPI)
│ └── storage/ # SQLite persistence
├── models/ # ML model training (offline, Python)
│ ├── ml/ # LightGBM, XGBoost training
│ ├── dl/ # CNN, LSTM training
│ ├── export/ # ONNX export utilities
│ └── training/ # Training scripts
│
├── reports/ # Report generator
│ ├── generate.py # Chart generation
│ └── templates/ # README templates
├── trained/ # Trained model artifacts
│ └── onnx/ # ONNX models for Rust runtime
│
├── docs/ # Documentation
│ ├── architecture.md # System design
│ ├── strategies.md # Trading logic
│ └── deployment.md # Deploy guide
├── reports/ # Report generator (GitHub Actions)
│ ├── generate.py # Chart generation from API data
│ └── assets/ # Generated charts
│
├── deploy/ # Deployment configuration
│ ├── supervisord.conf # Process management
│ └── entrypoint.sh # Container startup
│
└── .github/workflows/ # CI/CD
├── ci.yml # Tests on every push
├── deploy-render.yml # Deploy to Render
├── daily-report.yml # Update charts daily
├── daily-report.yml # Fetch metrics & update README
└── health-check.yml # Monitor every 15min
```

Expand Down Expand Up @@ -226,10 +234,23 @@ Track and compare strategy variations:

### Environment Variables (set in Render Dashboard)

| Variable | Required | Description |
|----------|----------|-------------|
| `TELEGRAM_BOT_TOKEN` | No | For trade notifications |
| `TELEGRAM_CHAT_ID` | No | Your Telegram chat ID |
| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| `SYMBOLS` | No | `BTCUSDT,ETHUSDT` | Trading symbols |
| `INITIAL_BALANCE` | No | `10000` | Starting paper balance |
| `IMBALANCE_THRESHOLD` | No | `0.3` | Order book imbalance threshold |
| `MIN_CONFIDENCE` | No | `0.6` | Minimum ML confidence for trades |
| `RISK_MAX_POSITION` | No | `1.0` | Maximum position size per symbol |
| `RISK_MAX_DRAWDOWN` | No | `0.05` | Maximum drawdown before circuit breaker |
| `TELEGRAM_BOT_TOKEN` | No | - | For trade notifications |
| `TELEGRAM_CHAT_ID` | No | - | Your Telegram chat ID |

### GitHub Secrets (for workflows)

| Secret | Description |
|--------|-------------|
| `RENDER_SERVICE_URL` | Your Render service URL (e.g., `https://orp-flow-trading.onrender.com`) |
| `RENDER_DEPLOY_HOOK_URL` | Deploy hook URL from Render Dashboard → Settings → Deploy Hook |

### Telegram Setup (Optional)

Expand Down Expand Up @@ -257,16 +278,24 @@ Track and compare strategy variations:
# All tests
make test

# Individual components
# Rust tests (trading engine)
cd market-data && cargo test

# OCaml tests (risk gateway)
cd core && dune test
cd strategy && pytest

# Python tests (model training only - offline)
cd models && pytest
```

### Linting
### Building

```bash
make lint
# Build Rust binary with ML support
cd market-data && cargo build --release --features ml

# Build OCaml risk gateway
cd core && dune build --release
```

### Benchmarks
Expand Down
7 changes: 5 additions & 2 deletions render.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,18 @@
# https://render.com/docs/blueprint-spec

services:
# Main Trading Service (Background Worker - never sleeps)
- type: worker
# Main Trading Service (Web Service for API access)
- type: web
name: orp-flow-trading
runtime: docker
dockerfilePath: ./Dockerfile
dockerContext: .
region: oregon # Closest to major exchanges
plan: starter # $7/month - always on, no spin-down

# Health check configuration
healthCheckPath: /health

# Environment Variables
envVars:
# Logging
Expand Down
Loading
Loading