Skip to content

Repository files navigation

Distributed Demand & Capacity Balancing (DDFN) prototype v2

This system is a truly decentralized, drop-in replacement for traditional centralized Air Traffic Flow Management (ATFM) systems (like the Eurocontrol NM). It utilizes a peer-to-peer pub/sub SWIM network to coordinate traffic flows, assign CASA delays, and renegotiate trajectories without any single node having global authority over the airspace.

Architecture Overview

  • Central SWIM Broker (broker.py): Acts purely as a passive message router for the distributed network and an Audit Logger. It holds no business logic or ATFM state.
  • FMP Nodes (fmp_service.py): Regional Flow Management Positions. Each node runs its own isolated UI, manages ONLY its local FIR sectors, and predicts demand strictly for its local airspace.
  • Airport Nodes (airport_service.py): Local Airport Flight Data Processing Systems (FDPS). These manage ground delay programs and slot auctions at the edge.
  • AOC Nodes (aoc_service.py): Airline Operations Centers. These track fleet schedules, negotiate slot swaps, and validate routes.
  • Simulation Manager (sim_manager.py): A centralized testing interface to dynamically inject new flights and view the global simulation state.

Screenshots

Below are screenshots of the DDFN v2 web dashboards captured from a running local cluster with real AIXM airspace data (BD_2025-09-30_400005921419525.xml) and live injected traffic.

Airspace Visualization

The FMP dashboards feature interactive Leaflet maps with real AIXM sector boundaries, airports, airways, and live flight tracking.

LCCC FIR (Cyprus) – Sector Map Overview

Interactive map showing LCCC FIR sector boundaries and active traffic injected via the Sim Manager.

LCCC Overview

LCCC FIR – Airspace Configuration

The Airspace Config tab with sector selection, airport/airway layer toggles, and the configuration map.

LCCC Config

LCCC FIR – Airspace & Slots

Real-time airspace inventory table showing sector demand, capacity, and slot assignments.

LCCC Airspace

LGGG FIR (Greece) – Sector Map with Active Flights

LGGG FIR showing Greek airspace sectors with injected flights traversing the region.

LGGG Overview

LGGG FIR – Airspace Configuration

Airspace configuration panel for the LGGG node with sector visualization.

LGGG Config

LGGG FIR – Airspace & Slots

Airspace inventory and slot management view for LGGG FIR.

LGGG Airspace

LLLL FIR (Israel) – Sector Map

LLLL FIR sector boundaries and active traffic visualization.

LLLL Overview

LLLL FIR – Airspace Configuration

Configuration panel with map visualization for Israeli airspace.

LLLL Config

LLLL FIR – Airspace & Slots

Airspace inventory table showing sector status for LLLL FIR.

LLLL Airspace

HECC FIR (Egypt) – Sector Map

HECC FIR showing Egyptian airspace sectors.

HECC Overview

HECC FIR – Airspace Configuration

Configuration panel for HECC node with sector map.

HECC Config

HECC FIR – Airspace & Slots

Airspace inventory and slot view for HECC FIR.

HECC Airspace

Broker Audit Dashboard

The central SWIM broker exposes an audit dashboard at http://127.0.0.1:8001, showing all active nodes in the distributed network and a live stream of CAPACITY_STATE_BROADCAST and AIRSPACE_INVENTORY_UPDATE messages.

Broker Dashboard

Simulation Manager

The Sim Manager provides a centralized testing interface at http://127.0.0.1:8050. Here it tracks injected flights (CYP001, CYP002, GRC001, ISR001, EGY001, etc.) originating from real airports such as LCLK, LCPH, LCRA, LGAV, LLBG, HECA.

Sim Manager

FMP Node Dashboards (Legacy Table View)

Each FMP node serves its own isolated dashboard on a dedicated port, configured from real AIXM FIR data.

Node URL FIR Sectors Screenshot
FMP 8010 http://127.0.0.1:8010 LCCC (Cyprus) 18 sectors including LCCC, LCCCNU, LCCCS2, LCCCE2, LCCCWL FMP 8010
FMP 8011 http://127.0.0.1:8011 LGGG (Greece) 7 sectors including LGGG, LGGGKAV FMP 8011
FMP 8012 http://127.0.0.1:8012 LLLL (Israel) 4 sectors including LLLL, LLLLN, LLLLS FMP 8012
FMP 8013 http://127.0.0.1:8013 HECC (Egypt) 3 sectors including HECC FMP 8013

AIXM Parser Integration

The DDFN v2 system integrates with the AIXM Parser library for parsing and visualizing aeronautical information exchange model (AIXM) 4.5 data. This enables:

  • Airspace Visualization: Parse and display airspace boundaries, sectors, and classifications
  • Navaid Support: Extract navigation aids (VOR, NDB, DME, TACAN) for route planning
  • Route Analysis: Parse airways and route segments for traffic flow analysis
  • Airport Data: Import airport locations, runways, and taxiway information

Supported AIXM Features

The parser currently supports:

  • Airspace: Airspace volumes, borders, and authority assignments
  • Aerodrome: Airports, runways, taxiways, aprons
  • Navaid: VOR, NDB, DME, TACAN, ILS, Marker beacons
  • Route: Airways, route segments, waypoints
  • Procedure: SIDs, STARs, instrument approaches

For detailed parser documentation, see the AIXM Parser Architecture and TODO files.

Prerequisites

  • Python 3.10+
  • Required pip packages: fastapi, uvicorn, websockets, python-multipart, aiosqlite

How to Run the System

Method 1: Full Simulation Cluster (Recommended)

To launch the complete European simulation including the Broker, 3 FMP hubs (France, Germany, UK), and 3 Airport FDPS nodes, run:

python run_all.py

All microservices will spin up locally. Check the logs/ directory for individual node outputs. The SWIM audit trail is saved securely to logs/swim_audit.db.

Method 2: Manual Distributed Startup

If you want to simulate running nodes across different terminals or different virtual machines:

  1. Start the Network Broker:

    python broker.py

    Runs on http://127.0.0.1:8080 by default.

  2. Start an FMP Node:

    python fmp_service.py LFFF

    Available nodes: LFFF, EDDD, EGTT. Each node embeds its own dashboard on a dedicated port (e.g., LFFF=8010, EDDD=8011).

  3. Start an Airport Node:

    python airport_service.py LFPG

    Available nodes: LFPG, EDDF, EGLL.

  4. Start an AOC Node:

    python aoc_service.py AFR

    Available airlines: AFR, DLH, BAW.

  5. Start the Simulation Manager:

    python sim_manager.py

    Runs on http://127.0.0.1:8050.

Method 3: One-Click Demo Script (Recommended for Windows)

Launch the full simulation (Broker, 4 FMPs, 3 Airports, 3 AOCs, and Sim Manager) with a single command:

./start_demo.ps1

This script launches all components as background jobs in the current terminal and provides a list of dashboard URLs. Use Ctrl+C to stop the entire cluster.

FMP Dashboard Configuration

Once an FMP node is running, access its isolated UI via browser:

  • LCCC (Cyprus): http://127.0.0.1:8010 (CONFIGURED — 18 sectors, airports LCLK, LCPH, LCRA)
  • LGGG (Greece): http://127.0.0.1:8011 (CONFIGURED — 7 sectors, airports LGIR, LGAV, LGTS, LGRP, LGKO)
  • LLLL (Israel): http://127.0.0.1:8012 (CONFIGURED — 4 sectors, airport LLBG)
  • HECC (Egypt): http://127.0.0.1:8013 (CONFIGURED — 3 sectors)

All four nodes are pre-configured using reference-based JSON files that re-resolve sector geometries from the high-fidelity AIXM source on startup. This ensures that the node's configuration is lightweight and always synchronized with the master airspace definition. For a technical breakdown of this architecture, see Airspace Configuration Design Specification.

Global Network Overview

The network is intentionally designed so no node has a global macroscopic view (to ensure true distribution). However, for analytical purposes, the central SWIM broker natively logs all pub/sub network activity into an immutable SQLite database located at logs/swim_audit.db. You can query this database to see exactly which nodes are publishing, subscribing, or negotiating slot constraints across the entire European mesh.

Important Documentation Reference:

  • Developer Onboarding Guide: Start here if you are new to the project. It explains the decentralized "swarm" paradigm and how to navigate the codebase safely.
  • For a complete reference of the exact message payloads and topics navigating the simulation mesh, see the Protocol Message Dictionary.
  • To view the internal state-machine transitions (such as the Airport Auto-Auction or FMP HITL loops) visualized with Mermaid diagrams, see Node State-Machines.
  • For the strategic vision and upcoming developmental phases needed to rival Eurocontrol NM, see the Future Development Roadmap.

Authentication

The DDFN system now requires API authentication for sensitive endpoints.

Environment Variables

  • DDFN_API_KEY: Set this to configure a custom API key. If not set, a secure key is auto-generated on startup (a generic warning is logged; the key itself is never printed).

Using Authentication

Include the X-API-Key header in requests to protected endpoints:

# Health check (no auth required)
curl http://127.0.0.1:8001/health

# Protected endpoint - requires API key
curl -H "X-API-Key: your-api-key" http://127.0.0.1:8001/api/nodes

Protected Endpoints

The following endpoints require authentication:

  • Broker: /api/nodes, /api/audit
  • FMP Service: All POST endpoints (/api/sectors/, /api/config/, /api/aixm/upload, /api/rad/toggle)
  • Airport Service: /api/aar/override
  • Sim Manager: /api/flights/inject

Public Endpoints

The following endpoints remain public:

  • All /health endpoints (for monitoring)
  • FMP Service: /metadata (for service discovery)

Security & Data Integrity

The DDFN v2 system enforces the following security and reliability guarantees:

  • Input Sanitization: All uploaded files (e.g., AIXM data) are sanitized. Filenames are stripped of path components and stored with UUID-based names to prevent path traversal attacks.
  • Secret Management: API keys and other secrets are never logged to standard output.
  • Audit Logging: All significant state changes are written to an append-only SQLite audit log (logs/swim_audit.db).
  • Async Persistence: The persistence layer uses aiosqlite with WAL mode enabled, ensuring non-blocking I/O across the event loop.
  • Message Durability: The network client buffers outbound messages during disconnections and flushes them automatically on reconnect.
  • CTOT Integrity: The CASA algorithm assigns CTOTs based on the computed assigned_slot time, ensuring correct behavior for in-flight aircraft.

Environment Configuration

The system can be configured via environment variables. Create a .env file or set variables directly:

\`�ash

Broker configuration

export DDFN_BROKER_HOST=127.0.0.1 export DDFN_BROKER_PORT=8001

FMP configuration

export DDFN_FMP_MEASURE_WINDOW=120 export DDFN_FMP_DEFAULT_CAPACITY=40

Airport configuration

export DDFN_AIRPORT_DEFAULT_AAR=30 export DDFN_AIRPORT_DEFAULT_ADR=28

Paths

export DDFN_CONFIG_DIR=data/node_configs export DDFN_LOGS_DIR=logs export DDFN_DATA_DIR=data \\

Docker Deployment

Building the Image

\`�ash docker build -t ddfn-v2 . \\

Running with Docker Compose

\`�ash

Start all services

docker-compose up -d

View logs

docker-compose logs -f

Stop all services

docker-compose down \\

Individual Service Deployment

\`�ash

Run broker only

docker run -p 8001:8001 ddfn-v2 python -m ddfn_v2.core.broker

Run FMP node

docker run -p 8010:8010 -e DDFN_BROKER_HOST=host.docker.internal ddfn-v2 python -m ddfn_v2.services.fmp_service --port 8010


## Features

- **Decentralized Architecture**: True peer-to-peer pub/sub network with no single point of failure
- **Health Check Endpoints**: All services expose `/health` for monitoring
- **API Authentication**: Secure endpoints with API key authentication
- **Prometheus Metrics**: Complete observability with metrics export at `/metrics`
- **Dark Mode Support**: All dashboards support light and dark themes
- **Responsive Design**: Mobile-first design optimized for all devices
- **Kubernetes Ready**: Complete K8s manifests for production deployment
- **Chaos Engineering**: Resilience testing framework included

## Monitoring with Prometheus

All services expose Prometheus metrics at `/metrics`:

```bash
# Scrape metrics from broker
curl http://127.0.0.1:8001/metrics

# Scrape metrics from FMP node
curl http://127.0.0.1:8010/metrics

Available metrics include:

  • Message throughput (published/received)
  • Node health and status
  • Sector capacity and demand
  • WebSocket connection counts
  • API request latency

UI Customization

Dark Mode

All dashboards support dark mode:

  • Click the theme toggle button (🌙/☀️) in the top navigation
  • Theme preference is saved to localStorage
  • System preference is respected on first visit

Mobile Support

Dashboards are fully responsive:

  • Optimized for mobile, tablet, and desktop
  • Touch-friendly controls
  • Adaptive navigation

Kubernetes Deployment

Prerequisites

  • Kubernetes 1.24+
  • kubectl configured
  • Ingress controller (nginx recommended)

Deploy to Kubernetes

# Apply all manifests
kubectl apply -k k8s/

# Verify deployment
kubectl get pods -n ddfn-v2
kubectl get svc -n ddfn-v2

Access Services

After deployment, services are available via Ingress:

  • Broker: http://ddfn.local/broker
  • FMP LCCC: http://ddfn.local/fmp-lccc
  • FMP LFFF: http://ddfn.local/fmp-lfff
  • Sim Manager: http://ddfn.local/sim

Testing

Running Tests

# Run all tests
python -m pytest

# Run with coverage
python -m pytest --cov=src/ddfn_v2 --cov-report=html

# Run specific test categories
python -m pytest tests/unit/ -v
python -m pytest tests/integration/ -v
python -m pytest tests/ui/ -v           # Basic UI tests
python tests/ui/test_airspace_config_stories.py # Airspace config user stories
python -m pytest tests/chaos/ -v

Chaos Engineering Tests

# Run chaos experiments
python -m pytest tests/chaos/ -v

# Run chaos runner
python tests/chaos/chaos_runner.py --duration 30

Coverage Reports

After running tests with coverage, view the HTML report:

open coverage_html/index.html

About

This system is a truly decentralized, drop-in replacement for traditional centralized Air Traffic Flow Management (ATFM) systems (like the Eurocontrol NM). It utilizes a peer-to-peer pub/sub SWIM network to coordinate traffic flows, assign CASA delays, and renegotiate trajectories without any single node having global authority over the airspace.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages