fix(airflow): give the DAGs the object store connection

[alex-mabrouk]

Description

The airflow package now gets its object store configuration from the project's storage Connection, just like the other data-service packages.

It takes a storage connectionRef and an s3SecretRef, then passes the endpoint, region and S3 credentials to Airflow task pods through extraEnv:

• AIRFLOW_ETL_S3_ENDPOINT
• AWS_REGION
• S3 credentials

storage is also added to the package dependencies so the release waits for the object store to be ready.

This follows the same approach on JupyterHub, where the hardcoded endpoint was replaced with {{ storage.endpoints.apiUrl }} from the project's storage Connection.

Package tag bumped to 3.2.1-p07.

Related Issue

Fixes #70

Type of Change

• Bug fix
• New feature
• Documentation update
• Refactor / chore
• Breaking change

How to Test

1. Build the package:

   kubocd package packages/services/airflow/airflow.yaml --ociRepoPrefix <prefix>

2. Try rendering a release without an object store. It should fail validation:

   kubocd render <release>.yaml packages/services/airflow/airflow.yaml

   Expected:

   parameters schema validation error: (root): s3SecretRef is required
   

3. Deploy 3.2.1-p07 with storage and s3SecretRef configured, then check that the Airflow scheduler has the expected S3 environment variables
4. Run an example DAG from okdp-examples and check that it writes to the object store configured by the project's storage Connection, rather than to the hardcoded endpoint.

Checklist

• I have tested my changes
• Documentation updated if needed
• If breaking change: migration path described above
• I hereby declare this contribution to be licensed under the Apache License Version 2.0.
• I hereby agree to grant TOSIT a copyright license to use my contributions.