Kraken Real time streaming architecture

Create a virtual environment

python -m venv krakenvenv

Install required packages

pip install -r requirements.txt

Make the shell script executable

chmod +x flowinit.sh

Initialize the database and create admin user

docker compose up airflow-init

start the container services

docker compose up

Open the Airflow UI through http://localhost:8080 Use the credentials:- Username: airflow Password: airflow

Execute postgres source connector configuration

chmod +x debezium_pg_connector.sh

bash debezium_pg_connector.sh

Run