diff --git a/README.md b/README.md index fd4441c..4b82198 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,11 @@ Monorepo for the Cogniro project. +## Documentation + +- [Technical documentation](docs/TECHNICAL-DOCUMENTATION.md) - architecture, technology stack, security and authentication, and the persistence and concurrency model. +- [Deployment guide](docs/DEPLOYMENT.md) - step-by-step setup and run instructions, with and without Docker, including all environment variables. + ## Structure ```text @@ -12,7 +17,7 @@ cogniro/ ## Prerequisites -- [Node.js](https://nodejs.org/) (v20+) +- [Node.js](https://nodejs.org/) 22 - [pnpm](https://pnpm.io/) 10+ - [Python](https://www.python.org/) (>=3.14) - [uv](https://docs.astral.sh/uv/) (for backend) diff --git a/backend/.env.example b/backend/.env.example index 11c186d..27f24b9 100644 --- a/backend/.env.example +++ b/backend/.env.example @@ -1,5 +1,11 @@ # Copy to .env and fill in real values. Never commit .env. +# Deployment stage. Accepts ENVIRONMENT or ENV; defaults to "development". +# Any value other than development/dev/local/test is treated as production: the app then +# refuses to start if ADMIN_PASSWORD_HASH or JWT_SECRET is missing (instead of only warning). +# Set this to production for real deployments so misconfigured auth fails fast. +# ENVIRONMENT=production + # --- Admin auth (POST /admin/auth/login) --- # Generate a bcrypt hash locally, then paste only the hash here: # uv run python scripts/hash_admin_password.py diff --git a/docs/DEPLOYMENT.md b/docs/DEPLOYMENT.md new file mode 100644 index 0000000..0268278 --- /dev/null +++ b/docs/DEPLOYMENT.md @@ -0,0 +1,552 @@ +# Cogniro - Deployment Guide + +This guide explains how to deploy and run Cogniro from scratch. It is written to be followed +step by step and does not assume prior knowledge of the project. If you follow the steps in +order you will not need to read the source code. + +For background on how the system works (architecture, security, the persistence model), see +[TECHNICAL-DOCUMENTATION.md](TECHNICAL-DOCUMENTATION.md). You do not need it to deploy, but it +is useful context. + +There are two ways to run the system, and this guide covers both fully: + +- **Option A: with Docker** (recommended, fewer moving parts). Jump to section 5. +- **Option B: without Docker** (bare metal / VM, install runtimes directly). Jump to + section 6. + +Read sections 1 to 5 first regardless of which option you choose, because the prerequisites, +the secrets, and the environment variables are the same for both. + +--- + +## 1. What you are deploying + +Cogniro has two parts: + +1. **The backend**: a Python (FastAPI) HTTP API. It listens on port **8000** by default. It + stores all data as files in one directory on disk. There is no database to install. +2. **The frontend**: a React website built with Next.js. After it is built it is just static files + (HTML, CSS, JavaScript, images). It listens on port **3000** in the simple setup, or it is + served by your web server. It needs no runtime once built. + +Participants and the administrator use a web browser. The browser loads the frontend, and the +frontend calls the backend over HTTP. + +``` + Browser -> Frontend (static files) -> Backend API (port 8000) -> Data directory on disk +``` + +You will typically put both behind one HTTPS reverse proxy (for example Nginx) so that +everything is served from your domain over HTTPS. Section 8 shows how. + +--- + +## 2. Prerequisites + +### 2.1 For Option A (Docker) + +- A Linux host (a VM or server) with: + - Docker Engine (recent version). + - Docker Compose v2 (the `docker compose` command). +- Outbound internet access during the build (to download base images and packages). + +That is all. The runtimes (Python, Node) are inside the images. + +### 2.2 For Option B (without Docker) + +On the host that runs the **backend**: + +- Python **3.14** +- `uv`, the Python package manager from Astral. Install it from https://docs.astral.sh/uv/ or with + `curl -LsSf https://astral.sh/uv/install.sh | sh`. + +On the machine that **builds** the frontend (can be the same host, or a CI runner, or your +laptop): + +- Node.js **22**. +- `pnpm` **10.33.0**. The easiest way is `corepack enable && corepack prepare pnpm@10.33.0 --activate`. + +On the host that **serves** the frontend in production: + +- A web server that can serve static files (for example Nginx, Apache, Caddy, or a CDN). No + Node.js is required to serve the built site. If you only want a quick start without a web + server, you can serve the static files with the `serve` package (via `pnpm dlx`) instead + (section 6.3). + +--- + +## 3. Create the required secrets + +The backend needs two secrets before it will serve admin login in production: + +1. `ADMIN_PASSWORD_HASH` - a bcrypt hash of the admin password (never the plaintext). +2. `JWT_SECRET` - a long random string used to sign login tokens. + +### 3.1 Generate the admin password hash + +The repository ships a helper script. Run it from the `backend` directory. It will prompt you +for the password and print the hash. + +With Docker available (no local Python needed): + +```bash +docker run --rm -it -v "$PWD/backend:/app" -w /app python:3.14-slim \ + sh -c "pip install --quiet bcrypt && python scripts/hash_admin_password.py" +``` + +Or, on a host that has Python and uv (Option B): + +```bash +cd backend +uv sync +uv run python scripts/hash_admin_password.py +``` + +Copy the printed hash. It looks like `$2b$12$....`. You will paste it into +`ADMIN_PASSWORD_HASH`. Always keep it **single-quoted** in env files and compose files so the +shell does not try to expand the `$` characters. + +### 3.2 Generate the JWT secret + +Any long random value works. For example: + +```bash +openssl rand -base64 48 +``` + +Copy the output into `JWT_SECRET`. + +Keep both secrets out of version control. Do not commit a real `.env` file (the repository's +`.gitignore` already excludes `.env`). + +--- + +## 4. Environment variables + +This section lists what you must set. The backend reads its configuration from environment +variables. A full annotated template is in `backend/.env.example`. The frontend reads a few +`NEXT_PUBLIC_*` variables that are baked into the static build at **build time**. + +### 4.1 Minimum backend variables for production + +| Variable | Set it to | +| --- | --- | +| `ADMIN_PASSWORD_HASH` | The bcrypt hash from step 3.1 (single-quoted). | +| `JWT_SECRET` | The random value from step 3.2. | +| `ENVIRONMENT` | `production` (makes the app refuse to start if a secret is missing). | +| `CORS_ORIGINS` | The exact public origin of your frontend, for example `https://quiz.example.com`. Comma-separate multiple. | +| `FRONTEND_ORIGIN` | The same public frontend origin (used to build participant join links). No trailing slash. | +| `ADMIN_REFRESH_COOKIE_SECURE` | `true` (any HTTPS deployment). | +| `COGNIRO_DATA_DIR` | The directory where data is stored, for example `/var/lib/cogniro`. This is what you back up. | + +If the frontend is served from a **different site** than the API (for example +`quiz.example.com` for the UI and `api.example.com` for the API), also set +`ADMIN_REFRESH_COOKIE_SAMESITE=none`. If they share the same site (recommended, via one +reverse proxy), leave it at the `lax` default. + +### 4.2 Optional backend variables + +These have sensible defaults, override only if needed. See +[TECHNICAL-DOCUMENTATION.md](TECHNICAL-DOCUMENTATION.md) section 8 for the full table. + +- `JWT_EXPIRE_MINUTES`: admin access-token lifetime in minutes (default 15, maximum 60). +- `ADMIN_REFRESH_EXPIRE_DAYS`: refresh-cookie lifetime in days (default 7, maximum 30). +- `ADMIN_REFRESH_COOKIE_SAMESITE`: `lax` (default), `strict`, or `none`. Use `none` for a + separate-site frontend and API (it forces the cookie to be Secure). +- `RESULT_RETENTION_DAYS`: result files older than this many days are deleted (default 30). +- `PURGE_INTERVAL_SECONDS`: how often the cleanup job runs, in seconds (default 3600, minimum 60). +- `MEDIA_PUBLIC_PREFIX`: public URL path prefix for quiz media (default `/media/quiz-assets`). + If you change it, set the frontend `NEXT_PUBLIC_MEDIA_PUBLIC_PREFIX` to the same value. +- `MAX_QUIZ_IMPORT_ZIP_BYTES` (default 100 MiB), `MAX_QUIZ_IMPORT_KQF_BYTES` (default 2 MiB), + `MAX_QUIZ_IMPORT_MEMBER_BYTES` (default 100 MiB), + `MAX_QUIZ_IMPORT_UNCOMPRESSED_TOTAL_BYTES` (default 300 MiB): upper size limits for the quiz + ZIP import. +- `PLAY_RATE_LIMIT_ENABLED` (default true), `PLAY_RATE_LIMIT_WINDOW_SEC` (default 60), + `PLAY_RATE_LIMIT_MAX_REQUESTS` (default 120): per-IP rate limit on the play endpoints (the + default allows 120 requests per 60 seconds per IP). +- `PLAY_RATE_LIMIT_TRUST_X_FORWARDED_FOR` (default false): set to `true` only when the backend + is behind a trusted reverse proxy, so per-IP limits use the real client IP from the + `X-Forwarded-For` header instead of the proxy's IP. + +### 4.3 Frontend build-time variables + +| Variable | Set it to | +| --- | --- | +| `NEXT_PUBLIC_BACKEND_URL` | The public URL the browser uses to reach the backend, for example `https://quiz.example.com` (if proxied under one domain) or `https://api.example.com`. | +| `NEXT_PUBLIC_MEDIA_PUBLIC_PREFIX` | Only if you changed the backend `MEDIA_PUBLIC_PREFIX`; the two must match. Set it in `frontend/.env` for the non-Docker build (section 6.2), or pass it as a `--build-arg` for the Docker build (section 5.2). | + +Important: `NEXT_PUBLIC_BACKEND_URL` is compiled into the static files. If you change it, you +must rebuild the frontend. + +--- + +## 5. Option A: Run with Docker + +There are two scenarios here: a quick local development run, and a production run. + +### 5.1 Quick local run with docker compose (development only) + +The repository includes `docker-compose.yml`. It is configured for **development** (hot +reload, source code mounted into the containers, no production secrets). It is the fastest way +to see the app running locally, but it is **not** a production setup. + +From the repository root: + +```bash +docker compose up --build +``` + +This starts: + +- the backend at http://localhost:8000 (with auto-reload), and +- the frontend at http://localhost:3000 (with hot reload). + +Open http://localhost:3000 in a browser. The admin panel is at http://localhost:3000/admin. + +Note: in this development compose file no `ADMIN_PASSWORD_HASH` or `JWT_SECRET` is set and +`ENVIRONMENT` is the default `development`, so the app starts but admin login will not work +until you provide secrets. For real use, follow the production steps below. + +### 5.2 Production with Docker: build the images + +Both `backend/Dockerfile` and `frontend/Dockerfile` are multi-stage. The production stage in +each is named `runner`. Build those. + +Backend image: + +```bash +docker build -f backend/Dockerfile --target runner -t cogniro-backend:latest ./backend +``` + +The backend build takes no environment variables and no build arguments. All backend +configuration (secrets, CORS, data directory, and so on) is read at **runtime**, so it is +supplied when you start the container, not when you build the image. See section 5.3. + +Frontend image: the frontend is the opposite case. `NEXT_PUBLIC_BACKEND_URL` is baked into +the static files at **build time**, so it must be passed as a build argument here; it cannot +be changed later without rebuilding. Build from the repository root (the build context is the +whole repo). + +```bash +docker build -f frontend/Dockerfile --target runner \ + --build-arg NEXT_PUBLIC_BACKEND_URL=https://quiz.example.com \ + -t cogniro-frontend:latest . +``` + +Replace `https://quiz.example.com` with the URL the browser will use to reach the backend. + +If you changed the backend `MEDIA_PUBLIC_PREFIX` away from the default `/media/quiz-assets`, +pass it as a second build argument so the frontend is built to match (the two must be +identical): + +```bash +docker build -f frontend/Dockerfile --target runner \ + --build-arg NEXT_PUBLIC_BACKEND_URL=https://quiz.example.com \ + --build-arg NEXT_PUBLIC_MEDIA_PUBLIC_PREFIX=/your/custom/prefix \ + -t cogniro-frontend:latest . +``` + +The backend `runner` image runs as a non-root user, creates `/var/lib/cogniro`, sets +`COGNIRO_DATA_DIR=/var/lib/cogniro`, and starts Uvicorn on port 8000 with a single worker (do +not add `--workers`). The frontend `runner` image serves the static `out/` directory with the +`serve` tool on port 3000. + +### 5.3 Production with Docker: run the containers + +Create a file `backend.env` (do not commit it) with your real values. The name is arbitrary +because Docker just reads it and injects the values into the container. This is not the same +as the `backend/.env` used in the non-Docker setup (section 6.1): with Docker you do not need +a `.env` inside the image at all. + +``` +ADMIN_PASSWORD_HASH='$2b$12$your-real-hash-here' +JWT_SECRET=your-real-random-secret +ENVIRONMENT=production +CORS_ORIGINS=https://quiz.example.com +FRONTEND_ORIGIN=https://quiz.example.com +ADMIN_REFRESH_COOKIE_SECURE=true +COGNIRO_DATA_DIR=/var/lib/cogniro +``` + +Create a persistent directory on the host for the data and run the backend with the env file +and a mounted volume so data survives container restarts and rebuilds: + +```bash +mkdir -p /srv/cogniro-data + +docker run -d --name cogniro-backend \ + --env-file backend.env \ + -v /srv/cogniro-data:/var/lib/cogniro \ + -p 8000:8000 \ + --restart unless-stopped \ + cogniro-backend:latest +``` + +Run the frontend: + +```bash +docker run -d --name cogniro-frontend \ + -p 3000:3000 \ + --restart unless-stopped \ + cogniro-frontend:latest +``` + +At this point the backend answers on port 8000 and the frontend on port 3000. In production, +you put both behind a reverse proxy with TLS (section 7). The volume mount +`-v /srv/cogniro-data:/var/lib/cogniro` is the single most important line: it is where all +quizzes and results are stored. Back it up (section 8). + +### 5.4 A note on workers + +Do not change the backend command to use multiple Uvicorn workers. The application keeps live +sessions and locks in process memory and is designed to run as a single process. Running +multiple workers will cause race conditions and split sessions. See +[TECHNICAL-DOCUMENTATION.md](TECHNICAL-DOCUMENTATION.md) section 6.5. To handle more load, use +a bigger host, not more workers. + +--- + +## 6. Option B: Run without Docker + +Here you install the runtimes directly on the host. You run the backend as a long-lived +service, build the frontend once into static files, and serve those files with a web server. + +### 6.1 Backend: install and run + +On the backend host: + +```bash +# 1. Go to the backend directory +cd cogniro/backend + +# 2. Install dependencies into a virtual environment (uv reads pyproject.toml + uv.lock) +uv sync --frozen --no-dev +``` + +**Step 3: create the `backend/.env` file.** This is how you configure the backend in the +non-Docker setup. The backend loads it automatically on startup (`main.py` calls +`load_dotenv()`), so as long as the file sits in the `backend/` directory and you start the +server from there, every value in it becomes an environment variable. You do not need to +export anything by hand. Create `backend/.env` (do not commit it) with at least: + +``` +ADMIN_PASSWORD_HASH='$2b$12$your-real-hash' +JWT_SECRET=your-real-random-secret +ENVIRONMENT=production +CORS_ORIGINS=https://quiz.example.com +FRONTEND_ORIGIN=https://quiz.example.com +ADMIN_REFRESH_COOKIE_SECURE=true +COGNIRO_DATA_DIR=/var/lib/cogniro +``` + +See section 4 for the full list of variables. `backend/.env.example` is an annotated template +you can copy. + +**Step 4: create the data directory.** It must exist and be writable by the user that runs +the backend. The path is up to you: `COGNIRO_DATA_DIR` can point anywhere that user can write +(for example `/var/lib/cogniro`, `/srv/cogniro`, `/home/cogniro/data`). Keep the value in +`backend/.env` and the path you create here the same. `/var/lib/cogniro` is only an example. + +```bash +sudo mkdir -p /var/lib/cogniro +sudo chown "$USER" /var/lib/cogniro +``` + +**Step 5: run the API.** Run it from the `backend/` directory so `backend/.env` is picked up +(single worker, listening on all interfaces, port 8000): + +```bash +uv run uvicorn main:app --host 0.0.0.0 --port 8000 +``` + +That command runs in the foreground. For production, run that same command as a long-lived +managed process with whatever process manager you already use, so it restarts on failure and +on boot. Keep it running from the `backend/` directory (so `backend/.env` is still loaded) and +never add `--workers N`: the backend must run as a single process (section 5.4 and the +technical doc). + +### 6.2 Frontend: build the static site + +On the build machine (Node 22 and pnpm 10.33.0 installed): + +```bash +# 1. From the repository root, install dependencies (the workspace includes the frontend) +pnpm install --frozen-lockfile +``` + +**Step 2: create the `frontend/.env` file.** Just like the backend, the frontend is +configured with a `.env` file. Next.js automatically loads it at build time, so you do not +pass variables on the command line. Create `frontend/.env` (do not commit it) with the +backend URL the browser will use: + +``` +NEXT_PUBLIC_BACKEND_URL=https://quiz.example.com +``` + +`frontend/.env.example` is an annotated template you can copy. This value is compiled into +the static files, so set it correctly for the target environment before building; changing it +later means rebuilding. + +**Step 3: build.** + +```bash +cd frontend +pnpm build +``` + +The build produces the static site in `frontend/out/`. That directory contains everything +needed to serve the site: `index.html`, the `play/`, `admin/`, and `admin/presenter/` +folders, the hashed assets under `_next/`, and optimized images under +`nextImageExportOptimizer/`. + +Copy the contents of `frontend/out/` to your web server's document root, for example: + +```bash +sudo mkdir -p /var/www/cogniro +sudo cp -r frontend/out/* /var/www/cogniro/ +``` + +If you ever need to serve the site under a sub-path (for example `https://example.com/quiz/`) +rather than at the domain root, you must add a `basePath` to `frontend/next.config.ts` and +rebuild. Serving at the domain root needs no change. + +### 6.3 Frontend: simplest possible serving (optional, no web server) + +If you just want to serve the built files without configuring Nginx, you can use the `serve` +package on the host. Run it with `pnpm dlx` (no global install needed): + +```bash +pnpm dlx serve@14 -s /var/www/cogniro -l 3000 +``` + +This listens on port 3000. For production, prefer a real web server with TLS (section 7). + +--- + +## 7. Reverse proxy and HTTPS (recommended for both options) + +In production you should serve everything over HTTPS. The configuration of the reverse proxy +and the TLS certificate is the responsibility of whoever runs the infrastructure, so this +guide does not ship a proxy config. What follows is what the proxy needs to do for this +application; translate it into your proxy of choice. + +The cleanest setup serves the static frontend and the backend API from one domain over HTTPS: + +- Terminate TLS at the proxy. +- Serve the contents of `frontend/out/` as static files, with a single-page-style fallback so + unknown paths resolve to the matching `index.html` (the build uses `trailingSlash`, so each + route is emitted as `route/index.html`). +- Forward the backend's API paths to the backend process (port 8000). The backend's route + roots are `/admin`, `/play`, `/media`, and `/health`. If you change `MEDIA_PUBLIC_PREFIX` + away from the default `/media/quiz-assets`, forward that prefix as well. +- Forward the standard proxy headers, including `X-Forwarded-For` (needed for the rate-limit + client IP) and `X-Forwarded-Proto`. +- Optionally serve the hashed build assets under `/_next/` with a long, immutable cache. + +With this same-domain layout the frontend and API are on the **same site**, so the admin +cookie works with the default `SameSite=lax`. Set the frontend `NEXT_PUBLIC_BACKEND_URL` and +the backend `CORS_ORIGINS` and `FRONTEND_ORIGIN` to that one public origin (for example +`https://quiz.example.com`), set `ADMIN_REFRESH_COOKIE_SECURE=true`, and because the API is +behind a proxy set `PLAY_RATE_LIMIT_TRUST_X_FORWARDED_FOR=true` so rate limiting uses the real +client IP. + +If instead you serve the API on a separate domain (for example `api.example.com`), set +`CORS_ORIGINS` and `FRONTEND_ORIGIN` to the frontend's domain, build the frontend with +`NEXT_PUBLIC_BACKEND_URL=https://api.example.com`, and set +`ADMIN_REFRESH_COOKIE_SAMESITE=none` (which forces the cookie to be Secure). + +Use any standard tool for the TLS certificate (for example Certbot / Let's Encrypt). + +--- + +## 8. Data, persistence, and backups + +All durable data lives under the backend's data directory (`COGNIRO_DATA_DIR`, for example +`/var/lib/cogniro` in Docker or `/srv/cogniro-data` on the host). Inside it: + +- `storage/quizzes/` - quiz definitions and their media. +- `storage/results/` - one JSON file per finished session, organized by date. +- `storage/uploads/quiz-assets/` - temporary editor staging (auto-cleaned after 24 hours). +- `storage/admin/password_hash` - the admin password hash, if it was changed at runtime. + +There is no database to dump. To back up the whole system, back up this one directory. A +simple approach is a scheduled copy or snapshot: + +```bash +tar czf cogniro-backup-$(date +%F).tar.gz -C /srv/cogniro-data . +``` + +Notes: + +- Old results are automatically deleted after `RESULT_RETENTION_DAYS` (default 30). If you + need to keep results longer, raise that value or back up before the purge runs. +- Restoring is just unpacking the directory back into place while the backend is stopped. + +--- + +## 9. Verify the deployment + +After starting both parts, confirm they work: + +1. **Backend health**: `curl https://quiz.example.com/health` should return + `{"status":"ok"}`. (Or `curl http://127.0.0.1:8000/health` directly on the host.) +2. **Frontend loads**: open `https://quiz.example.com/` in a browser. It should redirect to + the participant screen at `/play`. +3. **Admin login works**: open `https://quiz.example.com/admin`, log in with the password you + hashed in step 3.1. If login fails with a server error, the secrets are not set correctly + (check `ADMIN_PASSWORD_HASH` and `JWT_SECRET`); if it returns "invalid password", the hash + does not match the password you typed. +4. **End to end**: create a quiz, activate it, open the join link or scan the QR on the + presenter screen on a second device, join as a participant, submit, and confirm the score + appears on the leaderboard. + +--- + +## 10. Operations and maintenance + +- **Restarts end live sessions.** A backend restart clears all running sessions and loses any + in-progress (not yet stopped) scores. Restart and deploy when no live session is running. +- **Logs.** The backend logs to standard output. With Docker, view them with + `docker logs -f cogniro-backend`; without Docker, read them wherever your process manager + collects stdout. The background purge loop and any auth misconfiguration are logged here. +- **Changing the admin password.** The admin can change it from the admin panel; the new hash + is written to `storage/admin/password_hash` and survives restarts. All existing login + sessions are invalidated immediately. +- **Updating to a new version.** Pull the new code, then: for Docker rebuild the images and + recreate the containers (keep the same data volume); for bare metal run `uv sync --frozen + --no-dev` and restart the backend service, and rebuild the frontend (`pnpm build`) and + redeploy `frontend/out/`. The data directory is untouched by updates. + +--- + +## 11. Troubleshooting + +| Symptom | Likely cause | Fix | +| --- | --- | --- | +| Backend refuses to start with an error about missing env vars | `ENVIRONMENT` is `production` and `ADMIN_PASSWORD_HASH` or `JWT_SECRET` is not set | Set both secrets (section 3). | +| Admin login returns a 503 | `JWT_SECRET` not configured | Set `JWT_SECRET` and restart. | +| Admin login returns "invalid password" | The hash does not match the password typed | Regenerate the hash for the correct password (step 3.1). | +| Browser shows CORS errors in the console | `CORS_ORIGINS` does not include the frontend origin | Set `CORS_ORIGINS` to the exact public frontend origin and restart. | +| Admin stays logged out after refresh / cookie not stored | Cookie blocked over HTTP, or cross-site without correct flags | Use HTTPS, set `ADMIN_REFRESH_COOKIE_SECURE=true`; for separate-site setups set `ADMIN_REFRESH_COOKIE_SAMESITE=none`. | +| Frontend calls the wrong backend URL | `NEXT_PUBLIC_BACKEND_URL` was wrong at build time | Rebuild the frontend with the correct value (it is baked in at build). | +| Quiz images do not load during a game | `MEDIA_PUBLIC_PREFIX` mismatch, or media accessed outside an active session | Ensure frontend and backend prefixes match; media is only served while a session is active. | +| All participants share one IP and hit the rate limit | Behind a proxy without forwarded-IP trust | Set `PLAY_RATE_LIMIT_TRUST_X_FORWARDED_FOR=true` (only behind a trusted proxy), or raise `PLAY_RATE_LIMIT_MAX_REQUESTS`. | +| Data disappeared after a container rebuild | No volume mounted for the data directory | Always run the backend with the data volume mounted (section 5.3). | + +--- + +## 12. Production checklist + +Before going live, confirm: + +- [ ] `ADMIN_PASSWORD_HASH` and `JWT_SECRET` are set to real values. +- [ ] `ENVIRONMENT=production` is set. +- [ ] `CORS_ORIGINS` and `FRONTEND_ORIGIN` are set to your public frontend origin. +- [ ] `ADMIN_REFRESH_COOKIE_SECURE=true` (and `SameSite=none` if frontend and API are on + different sites). +- [ ] The backend runs as a **single** Uvicorn process (no `--workers`). +- [ ] The data directory is on persistent storage and is backed up. +- [ ] The frontend was built with the correct `NEXT_PUBLIC_BACKEND_URL`. +- [ ] HTTPS is terminated by the reverse proxy, and `PLAY_RATE_LIMIT_TRUST_X_FORWARDED_FOR=true` + if behind a proxy. +- [ ] `GET /health` returns `{"status":"ok"}` and an end-to-end test game works. diff --git a/docs/TECHNICAL-DOCUMENTATION.md b/docs/TECHNICAL-DOCUMENTATION.md new file mode 100644 index 0000000..9aa3626 --- /dev/null +++ b/docs/TECHNICAL-DOCUMENTATION.md @@ -0,0 +1,756 @@ +# Cogniro - Technical Documentation + +This document is the technical reference for the Cogniro platform. It describes the +architecture, the technology stack, the security and authentication model, and the +persistence and concurrency model in detail. + +For the step-by-step setup and run instructions, see the companion document +[DEPLOYMENT.md](DEPLOYMENT.md). + +Other existing reference documents you may want alongside this one: + +- [api-and-frontend-contract.md](api-and-frontend-contract.md) - the HTTP contract between frontend and backend. +- [storage-and-lifecycle.md](storage-and-lifecycle.md) - on-disk layout, session lifecycle, retention purge. +- [quiz-format-docs.md](quiz-format-docs.md) - the KQF quiz file format. +- [network-configuration.md](network-configuration.md) - CORS and network setup notes. + +--- + +## 1. What the system is + +Cogniro is a live quiz / knowledge-competition platform. It +has two kinds of users: + +- A single **administrator**, who creates and edits quizzes, starts and stops live + sessions, and reviews past results. +- Many anonymous **participants**, who join a running session either by entering a short PIN + code or by scanning a QR code (the QR encodes the join link, so scanning it opens the play + screen with the PIN already filled in), then pick + a nickname, answer the questions in their browser, and submit a final score that appears + on a live leaderboard. + +The product is intentionally small in scope. Several different quizzes can be live at the +same time, each with its own session and PIN; the only restriction is that a single quiz +cannot be started more than once concurrently (one active session per quiz). Each session is +hosted for a single audience (a classroom, a meeting, a conference talk) and participants +count in the tens to low hundreds, not millions. This scale is the reason the architecture +can stay simple, which is explained in detail in section 6. + +--- + +## 2. High-level architecture + +The system is two independent deployable units plus the participants' browsers. + +``` + +-------------------------------+ + | Participant / | + | Admin browser | + +---------------+----------------+ + | HTTPS + +---------------+----------------+ + | Static frontend (React + | + | Next.js static export, | + | plain files) on Nginx/CDN | + +---------------+----------------+ + | HTTPS (JSON REST) + +---------------+----------------+ + | Backend API (FastAPI on | + | Uvicorn, single process) | + +---------------+----------------+ + | local filesystem + +---------------+----------------+ + | Data directory on disk: | + | quizzes, results, media | + | (JSON / KQF / image files) | + +-------------------------------+ +``` + +Key points: + +- The **frontend is fully static**. After it is built, it is just HTML, CSS, JavaScript and + image files. It needs no Node.js runtime in production and can be served by any web + server or CDN. +- The **backend is a single Python process**. It holds all live-session state in memory and + persists durable data (quizzes, results, uploaded media) as files on a local disk. +- There is **no database** and **no separate cache / message broker**. The only stateful + resource the backend depends on is a writable directory on disk. +- Communication is **plain HTTP REST with JSON**. There are no WebSockets and no polling. + Requests are made in response to user actions and at specific moments in the quiz flow. + The countdown timers in the participant UI are computed locally in the browser and do not + call the backend. + +--- + +## 3. Technology stack + +### 3.1 Backend + +| Concern | Choice | Version (minimum) | +| --- | --- | --- | +| Language | Python | 3.14 (pinned in `backend/.python-version` and `pyproject.toml`) | +| Web framework | FastAPI | >= 0.135.3 | +| ASGI server | Uvicorn (with `standard` extras: uvloop, httptools) | >= 0.38.0 | +| Data validation | Pydantic | >= 2.12.5 | +| Password hashing | bcrypt | >= 5.0.0 | +| JSON Web Tokens | PyJWT | >= 2.12.1 | +| Image processing | Pillow | >= 12.2.0 | +| Quiz file parsing | python-frontmatter | >= 1.1.0 | +| Multipart uploads | python-multipart | >= 0.0.27 | +| Profanity filtering | glin-profanity | >= 3.4.0 | +| Env file loading | python-dotenv | >= 1.0.0 | +| Package / venv manager | uv (Astral) | latest | +| Lint / format | Ruff | >= 0.15.9 (dev only) | +| Tests | pytest, httpx | dev only | + +The exact pinned versions live in `backend/pyproject.toml` and `backend/uv.lock`. + +### 3.2 Frontend + +| Concern | Choice | Version | +| --- | --- | --- | +| UI library | React | 19.2.3 | +| Framework | Next.js (React framework, App Router) | 16.1.6 | +| Language | TypeScript | 5.9.3 | +| Styling | Tailwind CSS | 4.2.1 | +| Forms / validation | react-hook-form + Zod | 7.65.0 / 4.1.12 | +| Icons | lucide-react | 0.577.0 | +| Static image optimization | next-image-export-optimizer | 1.20.1 | +| QR codes | qrcode | 1.5.4 | +| Date handling | date-fns, react-day-picker | 4.4.0 / 10.0.1 | +| Package manager | pnpm | 10.33.0 (pinned in root `package.json`) | +| Node.js runtime (build only) | Node | 22 (the version used by Docker and CI) | + +Node is only needed to **build** the frontend; it is not needed to **serve** it. + +### 3.3 Repository layout + +This is a pnpm + uv monorepo. + +``` +cogniro/ + backend/ FastAPI application (Python, managed with uv) + main.py App composition, CORS, lifespan, background purge loop + core/settings.py Environment-driven settings and limits + security/ Admin auth: bcrypt verify, JWT issue/decode, revocation + routes/ HTTP routers (admin_auth, admin_quiz, admin_results, play, media) + services/ Business logic (storage, sessions, results, kqf, media, etc.) + schemas/ Pydantic request/response models + scripts/ Helper scripts (e.g. hash_admin_password.py) + tests/ pytest suite + Dockerfile Multi-stage image (dev / prod-deps / runner) + .env.example Template for backend environment variables + .python-version Python 3.14 + frontend/ Next.js app (TypeScript, managed with pnpm) + app/ App Router pages (/, /play, /admin, /admin/presenter) + app/components/ UI components + lib/ API clients, backend URL + media URL resolution, auth client + next.config.ts Static export configuration + Dockerfile Multi-stage image (dev / build / runner) + .env.example Template for frontend environment variables + docs/ This documentation + docker-compose.yml Local development orchestration (dev targets) + package.json Root workspace scripts (validate, test, fix) and git hooks + pnpm-workspace.yaml Workspace definition +``` + +--- + +## 4. Backend in detail + +### 4.1 Application composition + +The application is assembled in `backend/main.py`: + +1. `load_dotenv()` loads a local `.env` file if one is present. This works in any + environment; values already set in the real environment take precedence over the file, so + you can use a `.env` file or inject variables directly (or both). +2. A FastAPI app is created with a **lifespan** context manager. On startup the lifespan: + - initializes the storage directories, + - loads admin auth configuration from environment variables (and fails fast in + production if required secrets are missing, see section 5), + - loads a persisted admin password override if one exists on disk, + - starts a single background `asyncio` task that runs two cleanup jobs once at startup + and then on a fixed interval: one deletes result files past their retention age, the + other deletes abandoned editor image uploads. Both are described in detail in + section 9. +3. CORS middleware is added (see section 5.6). +4. Five routers are mounted, plus a `GET /health` endpoint that returns `{"status": "ok"}`. + +The app is served by Uvicorn. In production it must run as a **single worker** process. The +reason is explained in section 6.5. + +### 4.2 Routers and endpoints + +The full request/response contract is documented in +[api-and-frontend-contract.md](api-and-frontend-contract.md). The summary below lists every +route grouped by router. + +**Admin authentication** (`routes/admin_auth.py`, mounted under `/admin`): + +| Method | Path | Purpose | +| --- | --- | --- | +| POST | `/admin/auth/login` | Verify the admin password, return an access token and set the refresh cookie. Returns 401 `invalid_password` on a bad password. | +| POST | `/admin/auth/refresh` | Issue a new access token from the refresh cookie, and rotate the refresh token (the old one is revoked, a new one is set). | +| POST | `/admin/auth/logout` | Revoke the current access and refresh tokens and clear the refresh cookie. | +| POST | `/admin/auth/change-password` | Verify the current password, set a new one, invalidate all previously issued tokens, and reissue a session for the current device. | + +**Admin quiz management** (`routes/admin_quiz.py`, mounted under `/admin`, all require a valid admin access token): + +| Method | Path | Purpose | +| --- | --- | --- | +| POST | `/admin/quiz` | Create a quiz. | +| GET | `/admin/quiz/all` | List all quizzes. | +| GET | `/admin/quiz/{id}` | Get one quiz for editing. | +| PUT | `/admin/quiz/{id}` | Update a quiz. | +| DELETE | `/admin/quiz/{id}` | Delete a quiz. | +| POST | `/admin/quiz/{id}/activate` | Start a live session, return PIN and join URL. | +| POST | `/admin/quiz/{id}/stop` | Stop the session and write the result file. | +| GET | `/admin/quiz/{id}/session` | Snapshot of the running session (participants, scores). | +| POST | `/admin/quiz/{id}/session/block` | Block a participant from the leaderboard. | +| PATCH | `/admin/quiz/{id}/availability` | Schedule or manually open/close a quiz. | +| POST | `/admin/assets` | Upload an image (max 5 MiB); re-encoded to WebP image + thumbnail, returned as a content-addressed `asset_{uuid}`. | +| GET | `/admin/assets/{asset_id}/{file}` | Serve a staged editor asset for preview before the quiz is saved. | +| GET | `/admin/quiz/{id}/media/{file}` | Serve quiz-owned media for the editor. Does not require an active session (unlike the public `/media` route). | +| GET | `/admin/quiz/{id}/export` | Download the quiz directory as a ZIP archive. | +| POST | `/admin/quiz/import` | Import a quiz from a ZIP archive. | + +All routes in this router require a valid admin access token (the router declares +`dependencies=[Depends(require_admin)]`). + +**Admin results** (`routes/admin_results.py`, require admin token): + +| Method | Path | Purpose | +| --- | --- | --- | +| GET | `/admin/results` | List dates that have results. | +| GET | `/admin/results/{date}` | List result files for a date. | +| GET | `/admin/results/{date}/{file}` | Read one result file. | +| DELETE | `/admin/results/{date}/{file}` | Delete one result file. | +| DELETE | `/admin/results/{date}` | Delete all results for a date. | + +**Participant play** (`routes/play.py`, public, rate limited): + +| Method | Path | Purpose | +| --- | --- | --- | +| GET | `/play/{pin}/check` | Check whether a PIN is valid / joinable. | +| POST | `/play/{pin}/join` | Join with a nickname. | +| POST | `/play/{pin}/submit` | Submit the final score (clamped server-side, see 6.4). | +| GET | `/play/{pin}/leaderboard` | Get the ranked leaderboard. | + +Availability gating: `check` and `join` resolve the PIN to a live session and then check the +quiz's availability window (`services/admin_quiz.check_availability`). A quiz can be +force-opened or force-closed (`manual_status`) or scheduled with a start/end window. These two +endpoints translate the outcome into distinct HTTP codes so the participant UI can show the +right message: 404 `pin_not_active`, 423 `not_yet` (with the `opens_at` time), 410 `expired`, +and 403 `manually_closed`. A taken nickname returns 409; a profane nickname or a submit from an +unknown/blocked nickname returns 400. + +`submit` and `leaderboard` do **not** re-check the availability window; they only require the +PIN to still map to an active session. So once a session is running, scores can be submitted +and the leaderboard stays readable even if the quiz has been manually closed or has passed its +`schedule_end`. The window only gates entry (`check` / `join`); the live session itself ends +only when the admin stops it. + +**Media** (`routes/media.py`, public but gated): + +| Method | Path | Purpose | +| --- | --- | --- | +| GET | `/media/{quiz_id}/{file}` | Serve quiz media, but only while that quiz has an active session. | +| GET | `{MEDIA_PUBLIC_PREFIX}/{asset_path}` | Serve a staged editor asset (default prefix `/media/quiz-assets`). Public, no session required; only `.webp` files are served and the asset id is an unguessable UUID. | + +--- + +## 5. Security and authentication + +### 5.1 Authentication model: a single administrator + +There is exactly one privileged user, the administrator. There is no user database, no +sign-up, and no per-user accounts. The admin proves identity with a single password. +Participants are never authenticated; they are anonymous and identified only by the +nickname they choose inside a session. + +The implementation is in `backend/security/admin_auth.py`. + +### 5.2 Password storage + +The admin password is never stored in plaintext and the application never sees it written +anywhere in clear form. + +- The password is stored as a **bcrypt hash** (cost factor 12). bcrypt is a deliberately + slow, salted hashing algorithm designed to resist brute-force and rainbow-table attacks. +- The hash comes from one of two places, checked in this order: + 1. A hash persisted on disk at `storage/admin/password_hash` (written when the admin + changes the password at runtime). This file is created with `0600` permissions + (owner read/write only) where the operating system supports it. + 2. The `ADMIN_PASSWORD_HASH` environment variable, used as the initial value. +- Login verifies the submitted password against the active hash with `bcrypt.checkpw`. A + bad password returns HTTP 401, and a malformed stored hash is treated as a failed match + rather than an error. The verification runs in a worker thread (`asyncio.to_thread`) + because bcrypt is intentionally slow and would otherwise block the event loop. +- bcrypt only uses the first 72 bytes of a password. The login schema accepts 1 to 512 + characters; the change-password flow additionally rejects a new password longer than 72 + bytes with a clear `password_too_long` error instead of letting bcrypt truncate silently. + +To generate the initial hash, run `uv run python scripts/hash_admin_password.py` in the +backend directory. See [DEPLOYMENT.md](DEPLOYMENT.md) for the exact steps. + +### 5.3 Tokens: short-lived access plus refresh + +After a successful login the backend issues two JSON Web Tokens (HS256, signed with +`JWT_SECRET`): + +- **Access token**: short-lived (default 15 minutes, configurable 1 to 60 minutes via + `JWT_EXPIRE_MINUTES`). It is sent by the frontend in the `Authorization: Bearer ...` + header on every protected admin request. It is held only in memory in the browser tab + and is intentionally not persisted (see section 7.3). +- **Refresh token**: longer-lived (default 7 days, configurable 1 to 30 days via + `ADMIN_REFRESH_EXPIRE_DAYS`). It is delivered as an **HttpOnly cookie** scoped to the + path `/admin/auth`, so it cannot be read by JavaScript and is only sent to the auth + endpoints. When the access token expires, the frontend silently calls + `/admin/auth/refresh` to obtain a new one. + +Refresh tokens are **rotated**: each call to `/admin/auth/refresh` revokes the refresh +token it was given (by its `jti`, see 5.4) and issues a brand new refresh token in the +response cookie, alongside the new access token. A stolen refresh token therefore has a +limited useful life, and a replayed (already-used) refresh token is rejected. + +Each token carries: a subject (`admin`), a type (`access` or `refresh`), a unique id +(`jti`), the issue time, the expiry, and a **password fingerprint** (`pwd`). + +### 5.4 Token revocation and password-change invalidation + +Two mechanisms invalidate tokens before their natural expiry: + +- **Explicit revocation (logout)**: the token `jti` is added to an in-memory revocation + set with a time-to-live equal to the token's own expiry. Any later request presenting a + revoked `jti` is rejected. The set is pruned of expired entries on each access. +- **Password change invalidates everything**: every token embeds a short SHA-256 + fingerprint of the password hash that was active when it was issued. When the admin + changes the password, the active hash changes, so its fingerprint changes, and every + previously issued token (access and refresh) instantly fails the fingerprint check and is + rejected. This is how a password change forces a global logout without any shared session + store. The change-password endpoint requires the correct current password and a matching + confirmation, and to avoid logging out the device that just made the change, it issues a + fresh access and refresh token in the same response. + +Important operational consequence: the revocation set lives in process memory only. It is +cleared on restart and is not shared between processes. This is one of the reasons the +backend must run as a single worker (section 6.5). After a restart, an already-issued token +that was explicitly logged out could in principle be replayed until its short natural +expiry; keeping the access token lifetime short (the 15-minute default) bounds this window. + +### 5.5 Cookie flags + +The refresh cookie is configured for safe cross-site behaviour: + +- `HttpOnly` is always set. +- `Secure` is controlled by `ADMIN_REFRESH_COOKIE_SECURE` (set it to `true` in any + HTTPS deployment so the cookie is never sent over plain HTTP). +- `SameSite` is controlled by `ADMIN_REFRESH_COOKIE_SAMESITE` (`lax`, `strict`, or `none`). + If you host the admin UI on a different site than the API (for example + `app.example.com` calling `api.example.com`), set this to `none`. The backend then forces + `Secure=true` automatically, because browsers require it for `SameSite=None`. + +### 5.6 CORS + +Cross-Origin Resource Sharing is configured in `main.py`. Allowed origins come from the +`CORS_ORIGINS` environment variable (comma-separated). When it is unset the defaults are +`http://localhost:3000` and `http://127.0.0.1:3000` for local development. Credentials are +allowed (required so the refresh cookie works), and all methods and headers are permitted. +In production you must set `CORS_ORIGINS` to the exact public origin(s) of your frontend. + +### 5.7 Fail-fast on missing secrets + +On startup, if `ADMIN_PASSWORD_HASH` or `JWT_SECRET` is missing, the behaviour depends on +the environment: + +- In a development-like environment (`ENVIRONMENT` or `ENV` set to `development`, `dev`, + `local`, or `test`, which is also the default when neither is set) the app logs a warning + and keeps running, but admin login will fail until the values are set. +- In any other environment the app **refuses to start** with a clear error. This prevents a + misconfigured production deploy from silently serving broken authentication. + +For production you should set `ENVIRONMENT=production` (or any value not in the dev list) so +this guard is active. See [DEPLOYMENT.md](DEPLOYMENT.md). + +### 5.8 Input validation and abuse protection + +- **Pydantic models** validate every request body. Nicknames are stripped and bounded (1 + to 128 characters), submitted scores must be non-negative integers (`Field(ge=0)`), the + login password is 1 to 512 characters, and a new password is 8 to 512 characters (with the + extra 72-byte bcrypt cap from section 5.2). +- **Profanity filtering**: participant nicknames are checked with `glin-profanity` on join + and rejected with HTTP 400 if they violate the filter. +- **Image upload pipeline** (`services/media_assets.py`): the client-declared content type + must be `image/jpeg`, `image/png`, or `image/webp`, and the body is read in 64 KiB chunks + aborting at 5 MiB. The real format check happens after Pillow opens the file (only JPEG, + PNG, WebP are accepted), the pixel count is capped at 20 million to stop decompression + bombs, EXIF orientation is normalized, and the image is re-encoded to WebP (a main variant + capped at 720 px wide plus a 256 px thumbnail). Output is content-addressed as + `asset_{uuid}` and served with an immutable long-cache header. The asset server only + serves `.webp` files and rejects any symlink in the path. +- **ZIP import** (`services/admin_quiz.py`): the archive must contain a `quiz.kqf`, which is + parsed and validated before anything is written. Extraction enforces a maximum archive + size, a maximum `quiz.kqf` size, a maximum per-member uncompressed size, and a maximum + total uncompressed size (defending against zip bombs); every member path is checked for + traversal and absolute paths and rejected if unsafe. A single oversized media member is + dropped and reported in a `skipped` list (the editor shows a broken-media placeholder), + while any harder failure rolls back the whole quiz directory. All limits are configurable + (section 8). +- **Path traversal protection**: every endpoint that maps a request value to a file path + rejects `..`, path separators, and other unsafe input, so a request can never read or + write files outside the intended directory. Result-file reads validate the date format and + reject any filename containing `/` or `..`. +- **Per-IP rate limiting** on the public play endpoints (section 6.6). +- **Media gating**: `GET /media/{quiz_id}/{file}` only serves files while that quiz has an + active live session. When no session is active it returns 403, so quiz content is not + publicly browsable outside of a running game. + +--- + +## 6. Data persistence and the storage model + +This section explains the persistence model in detail: what is stored where, and how the +code guarantees correctness under concurrency (no lost updates, no dirty reads, no torn +writes, no time-of-check-to-time-of-use races). + +### 6.1 The storage model + +Durable state is kept as plain files on disk, and hot live-session state is kept in process +memory. The shape of the workload makes this model straightforward to reason about: + +- **Durable data has effectively one writer.** Quiz definitions are written only by the + single administrator, editing one quiz at a time. Participants never write durable files + during a game; their scores live in memory and are written to disk exactly once, when the + session is stopped. +- **The data set is small.** A handful of quiz definitions and one small JSON file per + finished session. There are no joins, reporting queries, or analytical workloads. +- **Write volume is low.** A live session produces one in-memory update per participant + action and exactly one durable file at the end. + +The correctness properties for this model (atomic durable writes, serialized writers, and +race-free live-session state) are provided by explicit, auditable code, described in the rest +of this section. + +### 6.2 What is stored, and where + +The on-disk layout is documented in full in +[storage-and-lifecycle.md](storage-and-lifecycle.md). In summary, under the data directory: + +- `storage/quizzes/{quiz_id}/` + - `quiz.kqf` - the canonical quiz definition (Markdown plus front matter). This is the + single source of truth for a quiz. + - `meta.json` - a derived cache (title, slug, timestamps, question count) used for fast + listing. If it is missing or corrupt it is rebuilt from `quiz.kqf`. + - `media/` - images belonging to the quiz. +- `storage/results/{YYYY-MM-DD}/{slug}_{timestamp}_{seq}.json` - one immutable JSON snapshot + per finished session. +- `storage/uploads/quiz-assets/` - temporary staging for images uploaded in the editor + before the quiz is saved. +- `storage/admin/password_hash` - the persisted admin password hash, if changed at runtime. + +The location of the data directory is controlled by `COGNIRO_DATA_DIR` (or the more specific +`COGNIRO_STORAGE_DIR`). In the production Docker image it defaults to `/var/lib/cogniro`. See +section 8 and the deployment guide. + +### 6.3 In-memory live-session state + +While a quiz is running, the session (its PIN, its participants, their scores, the shuffled +question order) lives only in memory, in `backend/services/sessions.py`. This is the hot, +frequently-mutated state. It is deliberately not written to disk on every change. When the +admin stops the session, the final scores are written once, atomically, to a single result +file. Because this state is in-memory, **a server restart ends all active sessions** (across +every running quiz) and the in-progress (not yet stopped) scores are lost. This is an +accepted trade-off for the expected scale. + +### 6.4 How every ACID-type concern is handled + +The properties usually associated with a database transaction (atomicity, isolation, +durability, no lost updates, no torn reads) are all provided here by a small set of explicit +techniques. + +**Atomic, durable writes (no torn or partial files).** + +Every durable file is written with a write-to-temp-then-rename pattern in +`services/storage.py` (`write_text_atomic`): + +1. The full content is written to a temporary file in the same directory. +2. The file buffer is flushed and `os.fsync` is called so the bytes reach the disk. +3. `os.replace` atomically renames the temp file over the destination. On a POSIX + filesystem `rename` is atomic, so a reader either sees the complete old file or the + complete new file, never a half-written mix. +4. The containing directory is itself fsynced so the rename survives a crash. + +This guarantees that a quiz file, a `meta.json` (written via `write_meta_json_atomic`), a +result file, or the persisted password hash is never observed in a partially-written state, +even if the process is killed mid-write. + +**Serialized writes (no lost updates, single writer).** + +Quiz writes are serialized by a process-wide reentrant lock, `QUIZ_WRITE_LOCK` +(`threading.RLock`) in `services/storage.py`. Every code path that mutates a quiz directory +holds this lock for the whole read-modify-write: creating a quiz, updating one (which reads +the existing KQF, merges the payload, writes `quiz.kqf` and `meta.json`, and removes media +files that are no longer referenced), patching availability, recording an activation +timestamp, clearing the manual status on stop, importing, and deleting. Because the lock +spans the read and the write, two overlapping saves cannot interleave and clobber each other, +and combined with the atomic rename above the on-disk quiz is always internally consistent. + +**Time-of-check-to-time-of-use (TOCTOU) safety in live sessions.** + +All live-session reads and mutations go through a single lock, `_LOCK` (`threading.RLock`) in +`services/sessions.py`. The critical point is that each operation performs its check and its +action **inside the same lock acquisition**, so there is no window between checking a +condition and acting on it. Examples: + +- `start_session` checks "is this quiz already running?" and, if not, generates a unique PIN + and registers the session, all while holding the lock. Two near-simultaneous activate + requests cannot both create a session for the same quiz, and cannot be assigned the same + PIN. If one request loses the race, `start_session` raises and the activate handler simply + looks up and returns the session the winner created (`activate_quiz` in + `services/admin_quiz.py`), so the loser still gets a correct, consistent response instead + of an error. Activation is therefore idempotent. +- `delete_quiz` refuses with HTTP 409 if a session for that quiz is running, so a quiz cannot + be deleted out from under a live game. +- `register_participant` checks "is this nickname already taken in this session?" and adds + the participant in the same locked block. Two players cannot both claim the same nickname + because the check and the insert are atomic with respect to each other. +- `record_submission` looks up the participant and records the score under the lock, so a + block action and a submit cannot race into an inconsistent state. + +Because the check and the use are never separated, the classic TOCTOU race simply cannot +happen. + +**Idempotent submission (no double-counting).** + +`record_submission` records a score only if the participant has not already submitted. A +duplicate submit (for example caused by a client retry, or React's development-mode double +effect) returns the existing participant unchanged rather than overwriting or erroring. The +operation is therefore idempotent. + +**A justified lock-free fast path.** + +One hot read path, `get_or_create_session_shuffle`, has a deliberate lock-free fast path for +performance. It is safe because of two facts: reading an attribute and building a list are +atomic under the Python Global Interpreter Lock, and the shuffled-order field is only ever +**replaced** as a whole, never mutated in place. So a reader always sees either the old +complete list or the new complete list. If the fast-path check is ambiguous (first join, or +the quiz was edited mid-session) it falls back to the locked slow path. This is the one place +where concurrency reasoning is subtle, and it is documented inline in the code. + +**Server-authoritative scoring (never trust the client).** + +On submit, the client-supplied score is clamped server-side to the quiz's maximum possible +points (`routes/play.py`: `score = min(body.score, max_score)`). A tampered client cannot +report an impossible score. + +**Single-IP rate-limit state.** + +The play-endpoint rate limiter keeps per-IP counters in memory, guarded by a dedicated +`threading.Lock`, with bounded memory (it tracks at most 10000 distinct IPs and evicts the +least-recently-used under flooding). See section 6.6. + +### 6.5 The one hard constraint: run a single worker + +All of the locks above are **in-process** locks. They serialize work within one Python +process. They do **not** coordinate across multiple OS processes. Therefore the backend must +be run with a **single Uvicorn worker**. + +If you ran Uvicorn with `--workers N` (N greater than 1) you would have N independent +processes, each with its own locks, its own in-memory sessions, and its own revocation set. +That would reintroduce exactly the race conditions the design avoids (two workers could both +"win" a PIN, sessions would be split across workers, logout would only affect one worker). + +For the application to function correctly it must therefore run with a **single Uvicorn +worker**. Never run multiple workers. + +### 6.6 Rate limiting + +All four public play endpoints (`GET /play/{pin}/check`, `POST /play/{pin}/join`, +`POST /play/{pin}/submit`, and `GET /play/{pin}/leaderboard`) share a per-IP sliding-window +rate limiter (`services/play_rate_limit.py`). Defaults: enabled, a 60-second window, and 120 +requests per window per IP. When the limit is exceeded the endpoint +returns HTTP 429 with a `Retry-After` header. If the backend runs behind a trusted reverse +proxy, set `PLAY_RATE_LIMIT_TRUST_X_FORWARDED_FOR=true` so the real client IP from +`X-Forwarded-For` is used instead of the proxy's IP. All values are configurable (section 8). + +--- + +## 7. Frontend in detail + +### 7.1 Static export + +The frontend is a **React** application built with Next.js (the React framework), using the +App Router and configured with `output: 'export'` +(`frontend/next.config.ts`). Building it produces a directory of static files +(`frontend/out/`) with no server-side rendering and no Node.js runtime requirement. With +`trailingSlash: true`, each route is emitted as `route/index.html`, which works on any plain +file server. A custom image loader plus `next-image-export-optimizer` processes images in +`public/images/` at build time into `out/nextImageExportOptimizer/` (responsive WebP variants +at quality 75 and tiny blurred placeholders). The dev-only `allowedDevOrigins` list is +derived from `NEXT_PUBLIC_ALLOWED_DEV_ORIGIN` or `NEXT_PUBLIC_BACKEND_URL` so hot reload works +when the dev server is reached on a non-localhost hostname; it has no effect on the +production static build. + +### 7.2 Routes + +| Route | Purpose | Auth | +| --- | --- | --- | +| `/` | Redirects to `/play`. | none | +| `/play` | Participant flow: enter the PIN (or arrive via a scanned QR with `?code=PIN` prefilled), choose nickname, answer, submit, leaderboard. | none | +| `/admin` | Admin login and dashboard (quiz CRUD, sessions, results, settings). | admin token | +| `/admin/presenter` | Projector screen showing the join QR code and PIN for participants. The QR is also shown in the admin running-session view, which can additionally generate a printable QR board. | none | + +### 7.3 How the frontend talks to the backend + +- The backend base URL comes from the build-time variable `NEXT_PUBLIC_BACKEND_URL` + (`lib/backend-url.ts`), defaulting to `http://localhost:8000`. Because it is a + `NEXT_PUBLIC_*` variable, its value is baked into the static build, so it must be set + **before** building for a given environment. The helper that builds request URLs strips a + trailing `/api` segment if present, so either form of base URL works. +- Quiz media URLs are resolved against `NEXT_PUBLIC_MEDIA_PUBLIC_PREFIX` (`lib/media-url.ts`), + which must match the backend's `MEDIA_PUBLIC_PREFIX`. +- The admin access token is stored **in memory only** (a module variable in + `lib/admin-auth/client.ts`); it is never written to localStorage, sessionStorage, or a + JavaScript-readable cookie. On a page reload it is gone, and the app silently re-acquires + one via the refresh cookie. This reduces the blast radius of any cross-site-scripting issue. +- Admin requests attach `Authorization: Bearer {token}`. The auth client + (`lib/admin-auth/client.ts`) additionally uses `credentials: 'include'` so the refresh + cookie is sent to the auth endpoints, and the admin-quiz client (`lib/admin-quiz/client.ts`) + transparently refreshes the access token once when a request fails with `token_expired` and + then retries. The other admin clients (sessions, results, import/export) send the Bearer + token but do not include the cookie or implement the refresh-and-retry path, so a request + there that hits an expired token surfaces the error to the UI rather than refreshing in + place. +- There is **no realtime transport and no polling**: no WebSockets, no server-sent events, + and no background interval that refetches state. The frontend calls the backend in response + to user actions and at specific points in the quiz flow (for example checking a PIN, + joining, and submitting). The admin session view and leaderboard are loaded on demand + rather than continuously refreshed. Timers shown to participants (the overall quiz timer + and per-question timers) are computed locally from timestamps in the browser. The one + scheduled re-check is narrow: when a participant opens a quiz that is scheduled but not yet + open, the join screen re-checks availability once around the scheduled start time so it can + unlock without a manual reload. + +--- + +## 8. Configuration reference + +All backend configuration is read from environment variables. The full, copy-ready +descriptions (with example values) are in `backend/.env.example`, and the deployment steps +are in [DEPLOYMENT.md](DEPLOYMENT.md). This is the consolidated reference. + +### 8.1 Backend environment variables + +| Variable | Required | Default | Purpose | +| --- | --- | --- | --- | +| `ADMIN_PASSWORD_HASH` | Yes (prod) | none | bcrypt hash of the admin password. Single-quote it so the shell does not expand `$`. | +| `JWT_SECRET` | Yes (prod) | none | Secret used to sign admin JWTs. Use at least 32 random bytes. | +| `ENVIRONMENT` / `ENV` | Recommended | `development` | Set to `production` so the app fails fast when secrets are missing. | +| `JWT_EXPIRE_MINUTES` | No | 15 | Access-token lifetime in minutes (1 to 60). | +| `ADMIN_REFRESH_EXPIRE_DAYS` | No | 7 | Refresh-token lifetime in days (1 to 30). | +| `ADMIN_REFRESH_COOKIE_SECURE` | No | false | Set `true` for HTTPS so the refresh cookie is HTTPS-only. | +| `ADMIN_REFRESH_COOKIE_SAMESITE` | No | lax | `lax`, `strict`, or `none`. `none` is for separate-site frontend/API and forces Secure. | +| `CORS_ORIGINS` | Yes (prod) | localhost:3000, 127.0.0.1:3000 | Comma-separated allowed browser origins. Set to your public frontend origin. | +| `FRONTEND_ORIGIN` | Recommended | http://localhost:3000 | Frontend origin used to build `/play/?code=PIN` join links. No trailing slash. | +| `COGNIRO_DATA_DIR` | Recommended | backend package dir (`/var/lib/cogniro` in Docker) | Root directory that contains `storage/`. The volume to back up. | +| `COGNIRO_STORAGE_DIR` | No | none | Points directly at the `storage/` directory. Overrides `COGNIRO_DATA_DIR` for quiz/result paths. | +| `MEDIA_PUBLIC_PREFIX` | No | /media/quiz-assets | Public path prefix for quiz media. Must match the frontend value. | +| `MEDIA_ABSOLUTE_ORIGIN` | No | none | Override the absolute origin for media URLs when the API is reached via localhost but clients load media from a public host. | +| `RESULT_RETENTION_DAYS` | No | 30 | Delete result folders older than this many days. | +| `PURGE_INTERVAL_SECONDS` | No | 3600 | How often the background purge loop runs (minimum 60). | +| `MAX_QUIZ_IMPORT_ZIP_BYTES` | No | 100 MiB | Max ZIP upload size for quiz import. | +| `MAX_QUIZ_IMPORT_KQF_BYTES` | No | 2 MiB | Max uncompressed size of `quiz.kqf` inside an import. | +| `MAX_QUIZ_IMPORT_MEMBER_BYTES` | No | 100 MiB | Max uncompressed size of any single media member. | +| `MAX_QUIZ_IMPORT_UNCOMPRESSED_TOTAL_BYTES` | No | 300 MiB | Max total uncompressed bytes per import. | +| `PLAY_RATE_LIMIT_ENABLED` | No | true | Enable per-IP rate limiting on play endpoints. | +| `PLAY_RATE_LIMIT_WINDOW_SEC` | No | 60 | Rate-limit window in seconds. | +| `PLAY_RATE_LIMIT_MAX_REQUESTS` | No | 120 | Max requests per IP per window. | +| `PLAY_RATE_LIMIT_TRUST_X_FORWARDED_FOR` | No | false | Trust the first `X-Forwarded-For` hop. Enable only behind a trusted proxy. | + +### 8.2 Frontend environment variables + +These are baked into the static build at build time. Set them before running the build. + +| Variable | Required | Default | Purpose | +| --- | --- | --- | --- | +| `NEXT_PUBLIC_BACKEND_URL` | Yes | http://localhost:8000 | Base URL of the backend API. | +| `NEXT_PUBLIC_MEDIA_PUBLIC_PREFIX` | No | /media/quiz-assets | Public path prefix for quiz media. Must match backend `MEDIA_PUBLIC_PREFIX`. | +| `NEXT_PUBLIC_ALLOWED_DEV_ORIGIN` | No | none | Dev-only. Allows hot-reload when the frontend dev server is reached on a non-localhost hostname (for example a tunnel). | + +--- + +## 9. Background tasks + +A single `asyncio` task is started in the app lifespan (`main.py`, the `_purge_loop` +coroutine). It runs both jobs below once immediately at startup, then sleeps for +`PURGE_INTERVAL_SECONDS` (default 3600, that is one hour, minimum 60) and repeats, for as +long as the process is alive. The task is cancelled cleanly on shutdown. There is no external +scheduler, cron, or task queue, so these jobs only run while the backend process is running. + +**Job 1: result retention purge** (`services/results.py`, `purge_results_older_than`). +Results are stored one folder per day under `storage/results/{YYYY-MM-DD}/`. This job +computes a cutoff date of `today - RESULT_RETENTION_DAYS` (default 30 days) and permanently +deletes every dated result folder whose date is older than the cutoff (the whole folder is +removed with its result files inside). Folders whose name is not a valid date are skipped. The +effect is that finished-session results are retained for roughly the retention window and then +removed automatically, so the disk does not grow without bound. If you need to keep results +longer, raise `RESULT_RETENTION_DAYS` or copy them out before they age out. + +**Job 2: stale editor-upload purge** (`services/media_assets.py`, `purge_stale_editor_staging`). +When the admin uploads an image in the quiz editor, it is staged under +`storage/uploads/quiz-assets/asset_{uuid}/` before the quiz is saved. Once the quiz is saved, +its media is copied into that quiz's own `media/` folder, so the staged copy is no longer +needed. This job deletes any `asset_*` staging directory whose last-modified time is older +than 24 hours (`ORPHAN_ASSET_RETENTION_SECONDS`). The 24-hour delay means an upload that is +still being worked on is never removed mid-edit; only genuinely abandoned staging (an upload +that was never attached to a saved quiz) is cleaned up. Saved quiz media is never touched by +this job. + +Both jobs run the blocking filesystem work in a worker thread (`asyncio.to_thread`) so they +never block the request-handling event loop, and any error in either job is logged and +swallowed so a cleanup failure can never crash the application or stop the loop. + +--- + +## 10. Testing and CI + +- The backend has a pytest suite under `backend/tests/` (auth, sessions, play, rate limiting, + storage, results, import/export, KQF parsing, media gating, and more). +- The frontend has unit tests run with `tsx --test`. +- Continuous integration (`.github/workflows/ci.yml`) runs on every pull request and on push + to `main`. It has three jobs: + - **Frontend**: install, validate (ESLint + Prettier + TypeScript), test, build. + - **Backend**: install with uv, Ruff lint, Ruff format check, pytest. + - **Docker**: build the production `runner` image for both backend and frontend. +- Git hooks (Husky) enforce lint-staged on commit, conventional commit messages, and the full + `validate` suite on push. + +--- + +## 11. Known limitations + +These are intrinsic to the design and are listed so there are no surprises in operation: + +- **Single process only.** The application must run with exactly one Uvicorn worker + (section 6.5). Never run multiple workers. +- **Live sessions are not durable.** A backend restart ends all running sessions and loses + in-progress (not yet stopped) scores. Plan restarts for when no session is live. +- **Token revocation is in-memory.** A logout does not survive a restart and is not shared + across processes. Short access-token lifetimes bound the risk. +- **The data directory is the system of record.** Everything durable (quizzes, results, + uploaded media, the persisted password hash) lives under one directory. Back it up + (section in the deployment guide). Losing it loses all quizzes and historical results. +- **Frontend backend URL is baked in at build time.** Changing `NEXT_PUBLIC_BACKEND_URL` + requires a rebuild of the frontend. + +--- + +## 12. Where to go next + +- To set the system up and run it, read [DEPLOYMENT.md](DEPLOYMENT.md). +- For the exact request/response shapes, read + [api-and-frontend-contract.md](api-and-frontend-contract.md). +- For the data lifecycle and retention details, read + [storage-and-lifecycle.md](storage-and-lifecycle.md). +- For the quiz file format, read [quiz-format-docs.md](quiz-format-docs.md). diff --git a/docs/storage-and-lifecycle.md b/docs/storage-and-lifecycle.md index fedb736..067b18c 100644 --- a/docs/storage-and-lifecycle.md +++ b/docs/storage-and-lifecycle.md @@ -23,8 +23,6 @@ Under that directory: KQF on disk is the **source of truth**; `meta.json` is a cache for listing and UI. -**Upgrade note:** editor staging used to live at `{data_dir}/uploads/quiz-assets/`. It now lives at `{data_dir}/storage/uploads/quiz-assets/`. Move any in-flight staged `asset_*` folders into the new path (or re-upload) before relying on unsaved editor previews. - ## In-memory sessions Live runs are tracked only in memory: PIN → session and quiz id → session indices in [backend/services/sessions.py](../backend/services/sessions.py). Restarting the process clears all active sessions. diff --git a/frontend/Dockerfile b/frontend/Dockerfile index 689d7aa..0827a3d 100644 --- a/frontend/Dockerfile +++ b/frontend/Dockerfile @@ -20,7 +20,9 @@ CMD ["pnpm", "exec", "next", "dev", "--webpack"] FROM deps AS build ARG NEXT_PUBLIC_BACKEND_URL +ARG NEXT_PUBLIC_MEDIA_PUBLIC_PREFIX ENV NEXT_PUBLIC_BACKEND_URL=$NEXT_PUBLIC_BACKEND_URL +ENV NEXT_PUBLIC_MEDIA_PUBLIC_PREFIX=$NEXT_PUBLIC_MEDIA_PUBLIC_PREFIX COPY frontend ./frontend WORKDIR /app/frontend RUN pnpm build diff --git a/frontend/README.md b/frontend/README.md index b6fb48b..d80cfc4 100644 --- a/frontend/README.md +++ b/frontend/README.md @@ -8,7 +8,7 @@ This is a [Next.js](https://nextjs.org) project bootstrapped with [`create-next- ### Prerequisites -- **Node.js 24** — With nvm: `nvm use` (run `nvm install` if needed) +- **Node.js 22** — With nvm: `nvm install 22 && nvm use 22` - **pnpm 10+** — `corepack enable` then `corepack prepare pnpm@latest --activate` - **GitHub CLI** — optional, for skills; `brew install gh` then `gh auth login` - **CodeRabbit CLI** — optional, for skills; `brew install coderabbit` then `coderabbit auth login`