An API for Element 84 FilmDrop.
FilmDrop is Element 84's geospatial data lake solution, handling everything from ingest to delivery, including processing, archive, indexing, and search and discovery. The FilmDrop API exposes the processing side of a deployment: the workflows that transform data, the runs and metrics they produce, the collections they populate, and the storage that backs them.
A FilmDrop deployment is built using Cirrus, AWS S3, and a STAC API such as stac-server. This API is a read-oriented FastAPI service layered over those components: it reads configuration describing a deployment and surfaces its workflows, executions, collections, and buckets through a single HTTP interface.
- Browse workflow definitions (the underlying AWS Step Functions state machines) and configured workflow instances.
- Query workflow runs by workflow, state, and time range, including their execution events.
- Inspect the sources (S3 / SNS triggers) that drive workflows and the output collections they populate.
- Retrieve aggregated and per-workflow metrics and execution logs.
- List collections and storage buckets for the deployment.
- Interactive OpenAPI documentation, served at
/docs.
All list endpoints are paginated (pagination_token and limit query
parameters). See the interactive docs at /docs for full request/response
schemas.
| Resource | Path | Description |
|---|---|---|
| Root | / |
Landing page, /health, and /version. |
| Collections | /collections |
List and fetch S3-backed, STAC-described data collections. |
| Storage | /storage, /storage/buckets |
List and fetch storage buckets and their metrics. |
| Workflow definitions | /workflows/definitions |
Step Functions definitions for workflows. |
| Workflow instances | /workflows/instances |
Configured workflows (source, parameters, output collections). |
| Workflow runs | /workflows/runs |
Individual workflow executions; filterable by workflow, state, and time. |
| Workflow sources | /workflows/sources |
S3 / SNS triggers that initiate workflows. |
| Workflow metrics | /workflows/metrics |
Aggregated and per-workflow execution metrics. |
| Workflow logs | /workflows/logs |
Lambda and AWS Batch task logs for execution runs. |
The application is configured entirely through environment variables, prefixed
with FILMDROP_API_. Dotenv files are intentionally not parsed by the
application; source them in your shell (or pass --env-file to uvicorn)
instead.
| Variable | Required | Default | Description |
|---|---|---|---|
FILMDROP_API_CONFIG_FILE |
Yes | — | Location of the deployment config file. Accepts a local path or a URL such as s3://bucket/path/config.json. |
FILMDROP_API_CORS_ALLOWED_ORIGINS |
No | '' |
Comma-separated list of allowed CORS origins. |
FILMDROP_API_API_TITLE |
No | Processing API |
Title shown in the OpenAPI docs. |
FILMDROP_API_API_DESCRIPTION |
No | API for data processing |
Description shown in the OpenAPI docs. |
FILMDROP_API_CONFIG_FILE points at a JSON document describing the deployment
the API serves. At a high level it contains:
meta— config version and (optionally) the AWS region.deployments— a mapping of deployment IDs to deployment configurations, each describing its collections, sources, tasks, workflow definitions, and workflows.
Loading a config from an s3:// URL requires AWS credentials available to
the application (via the standard boto3 credential chain).
Note
Only a single deployment is currently supported; configuring more than one will result in a startup error.
Build and start the service (listens on port 8000):
docker compose up --build
The Compose service does not set any environment variables, so supply at least
FILMDROP_API_CONFIG_FILE (and any AWS credentials needed to reach S3 and the
deployment) through your environment or by editing compose.yml.
Install dependencies, then run the app with uvicorn, pointing at a dotenv
file for configuration:
uv sync --dev
source .venv/bin/activate
uvicorn filmdrop.api.main:app --reload --env-file .env.test
With the uvicorn defaults, the app is accessible at
http://localhost:8000/v0/processing, and the interactive docs at
http://localhost:8000/v0/processing/docs.
See CONTRIBUTING.md for full development setup, including testing (some tests require a cirrus-geo deployment running in localstack), pre-commit hooks, and dependency management.
See CONTRIBUTING.md.
Licensed under the Apache License, Version 2.0.