diff --git a/providers/edge3/docs/img/distributed_architecture.svg b/providers/edge3/docs/img/distributed_architecture.svg index 1bbb56c6fce41..d75e9b385589c 100644 --- a/providers/edge3/docs/img/distributed_architecture.svg +++ b/providers/edge3/docs/img/distributed_architecture.svg @@ -1,4 +1 @@ - - - -
Cororate Networks
Remote Corporate Location
Airflow Main Data Center or Cloud Deployment Ecosystem
Kubernetes Cluster (optional)
Airflow Scheduler
& DAG Parser
Postgres DB
HTTPS
Firewall/
AppGW/
Ingress
Edge Worker
Proxy
Client
Client
Client
Airflow API-Server
Any Branch Legacy
Backend System
Distributed Airflow Architecture
Internet Based Client
Airflow Celery Worker (Pool)
Edge Worker
+
Corporate Networks
Corporate Networks
Remote Corporate Location
Remote Corporate Location
Airflow Main Data Center or Cloud Deployment Ecosystem
Airflow Main Data Center or Cloud Deployment Ecosystem
Kubernetes Cluster (optional)
Kubernetes Cluster (optional)
Airflow Scheduler
& DAG Parser
Airflow Sch...
Postgres DB
Postgres DB
HTTPS
HTTPS
Firewall/
AppGW/
Ingress
Firewall/...
Edge Worker
Edge Worker
Proxy
Proxy
Client
Client
Client
Client
Client
Client
Airflow API-Server
Airflow A...
Any Branch Legacy
Backend System
Any Branch Lega...
Distributed Airflow Architecture
Distributed Airflow Architecture
Internet Based Client
Internet Based Cl...
Airflow Celery Worker (Pool)
Airflow Cel...
Edge Worker
Edge Worker
Text is not SVG - cannot display
\ No newline at end of file diff --git a/providers/edge3/docs/img/edge_package.svg b/providers/edge3/docs/img/edge_package.svg index 24408170c9cf5..38327bd72739d 100644 --- a/providers/edge3/docs/img/edge_package.svg +++ b/providers/edge3/docs/img/edge_package.svg @@ -1,4 +1 @@ - - - -
Cororate Network(s) / Remote Location(s)
Airflow Main Data Center or Cloud Deployment Ecosystem
Client(s)
Airflow Edge Worker Components / Provider Package
Airflow Celery Worker
(Pool / Hybrid Execution)
DB
Table: Edge Job
Task Instance Key
State
Queue
Command
(...)
Table: Worker Status
Hostname
Status
(...)
Airflow
Scheduler
Provider Package: Edge3
EdgeExecutor
Add / Report / Cleanup
Airflow
API-Server
Provider Package:
Edge3 Executor Plugin
View(s): Edge Worker - State, Job, Maintenance
Query
Query / Update
HTTPS
REST API(s) Endpoints:
- Fetch Job
- Report Result
- Heartbeat
- Log Upload (...)
Update
Update
Edge
Worker
Provider Package: Edge3
CLI: `airflow edge worker ...`
- Poll & report job status
- Heartbeat, Maintenance
- Send log (chunks)
Task Execution
via Task SDK
Table: Edge Logs
Task Instance Key
Timestamp
Log Text Chunk
Insert
Logs
DAG Sync
Internet Based Client(s)
HTTPS
Query
HTTPS
Proxy
Task SDK API Endpoint
Firewall/
AppGW/
Ingress
+
Corporate Network(s) / Remote Location(s)
Corporate Network(s) / Remote Location(s)
Airflow Main Data Center or Cloud Deployment Ecosystem
Airflow Main Data Center or Cloud Deployment Ecosystem

Airflow Edge Worker Components / Provider Package
Airflow Edge Worker Components / Provider Package
Airflow Celery Worker
(Pool / Hybrid Execution)
Airflo...
DB
DB
Table: Edge Job
Table: Edge Job
Task Instance Key
Task Instance Key
State
State
Queue
Queue
Command
Command
(...)
(...)
Table: Worker Status
Table: Worker Status
Hostname
Hostname
Status
Status
(...)
(...)
Airflow
Scheduler
Airflo...
Provider Package: Edge3
Provider Package: Edge3
EdgeExecutor
EdgeExecutor
Add / Report / Cleanup
Add / Report / Cleanup
Airflow
API-Server
Airf...
Provider Package:
Edge3 Executor Plugin
Provider Package:...
View(s): Edge Worker - State, Job, Maintenance
View(s): Edge Worker -...
Query
Query
Query / Update
Query / Update
HTTPS
HTTPS
REST API(s) Endpoints:
- Fetch Job
- Report Result
- Heartbeat
- Log Upload (...)
REST API(s) Endpoints:...
Update
Update
Update
Update
Edge
Worker
Edge...
Provider Package: Edge3
CLI: `airflow edge worker ...`
Provider Package: Edge3...
- Poll & report job status
- Heartbeat, Maintenance
- Send log (chunks)
- Poll & report job sta...
Task Execution
via Task SDK
Task Execution...
Table: Edge Logs
Table: Edge Logs
Task Instance Key
Task Instance Key
Timestamp
Timestamp
Log Text Chunk
Log Text Chunk
Insert
Insert
Logs
Logs
DAG Sync
DAG Sync
Internet Based Client(s)
Internet...
HTTPS
HTTPS
Query
Query
HTTPS
HTTPS
Proxy
Proxy
Task SDK API Endpoint
Task SDK API Endpoint
Firewall/
AppGW/
Ingress
Firewal...
Client(s)
Client(s)
Text is not SVG - cannot display
\ No newline at end of file