[HC-ONLINE]

Pull Request Template

Description

Bound the in-memory download registry so token storage can no longer grow without limit, stop loading export files into memory before their size is validated, and never issue download tokens while downloads are disabled.

• Atomic budget: reservations are made before reading the export file (download.max_tokens, default 100, and download.max_total_bytes, default 1 GiB) and committed only after the read succeeds; failures release the reservation. There is no eviction of live tokens — when a budget is exhausted the export still completes on disk, the response returns 200 with download_token: null, and the rejection is logged.
• Size validation before read: stat().st_size is checked before read_bytes() (no more loading an oversized file to reject it afterwards), plus a defensive re-check after the read (TOCTOU).
• Disabled gate: generate_download_token and the scrape batch helper short-circuit with no token while download.enabled=false (log INFO / DEBUG respectively).
• Docs: registry budget documented in docs/CONFIGURATION.md (JSON, reference table, new section) and .env.example (CIBERWEBSCAN_DOWNLOAD_MAX_TOKENS, CIBERWEBSCAN_DOWNLOAD_MAX_TOTAL_BYTES); download_token: null responses documented in docs/API.md (plus two obsolete env var names corrected there).

Verification: 11 new regression tests (registry budget ×9, disabled helper, disabled batch); full suite 1896 passed / 3 skipped; end-to-end against the real API 21/21 checks across 3 scenarios (disabled, byte budget, token cap); Ruff, Pyright and all pre-commit hooks pass.

Related Issue

N/A (no public issue; tracked internally)

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update

Checklist

• I have followed the style guidelines of this project (Ruff & Pyright)
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I have updated the documentation accordingly

Screenshots (if applicable)

N/A (backend/API changes)