Add calculate variable usage analytics#1499
Merged
Merged
Conversation
hua7450
approved these changes
May 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1498
Fixes #1500
Summary
Adds privacy-safe variable usage analytics for authenticated
/calculaterequests and introduces Alembic-managed schema changes for the analytics database. The existingvisitsrequest-count analytics remains intact.Calculate variable analytics
calculate_requestslinked tovisits.id.calculate_request_variablesrows for variable name, source, entity type, period granularity, counts, and availability status.person,tax_unit, orhousehold, and collapses unknown caller-supplied groups tounknown....and storesvariable_name_truncated=truewhen capping occurs.Required analytics behavior
ANALYTICS__ENABLED=falseskips analytics entirely.ANALYTICS__ENABLED=truemakes analytics a required runtime dependency.Alembic migration infrastructure
visitstable.uv run alembic revision --autogenerate --rev-id 20260508_0002 -m "calculate variable usage"against a temp DB upgraded to the baseline revision.20260512_0003withuv run alembic revision --autogenerate --rev-id 20260512_0003 -m "cap analytics variable names", then adjusts the generated boolean column to backfill existing rows safely.AGENTS.mdis the shared entry point,CLAUDE.mdand.github/copilot-instructions.mdare thin adapters, and database migration guidance lives atdocs/engineering/skills/database-migrations.md.uv run alembic revision --autogenerate -m "Description", with only the documented existing-production-schema baseline exception.db.create_all()for analytics schema changes.ANALYTICS__ENABLED=true.Request validation
Test structure
tests/fixtures/.tests/conftest.py, avoiding nested pytest plugin loaders.Manual rollout required
Existing analytics databases that already have
visitsbut noalembic_versiontable must be stamped exactly once before the workflow migration step can run successfully. Run this manually before the first deploy that includes Alembic; otherwiseuv run alembic upgrade headwill try to create the existingvisitstable and fail:This should be done first for staging, then for production after staging verification. Staging and production must configure
USER_ANALYTICS_DB_CONNECTION_NAME,USER_ANALYTICS_DB_USERNAME, andUSER_ANALYTICS_DB_PASSWORD; missing secrets fail the deploy before the App Engine version is created.Deferred
Daily rollups are intentionally left out of this PR and should be handled separately with offline aggregation or atomic database upsert semantics.
Tests
uv run ruff format --check .uv run ruff check alembic policyengine_household_api/data/analytics_setup.py policyengine_household_api/decorators/analytics.py policyengine_household_api/utils/config_loader.py tests/unit/data/test_alembic_migrations.pyANALYTICS_DATABASE_URL=sqlite:////private/tmp/household_api_alembic_smoke.db uv run alembic upgrade headuv run pytest -q tests/unit/data/test_alembic_migrations.py tests/unit/data/test_analytics_setup.pyuv run pytest -q tests/unit/decorators/test_analytics.py tests/unit/endpoints/test_household.pyuv run pytest -q --confcutdir=tests/unit/utils tests/unit/utils/test_variable_usage_analytics.pyuv run pytest -q tests/unit/data/test_alembic_migrations.py tests/unit/utils/test_config_loader.pyuv run pytest -q --timeout=150 .github/scripts tests/to_refactor tests/unituv run pytest -q tests/unit/data/test_analytics_setup.py tests/unit/data/test_alembic_migrations.py tests/unit/decorators/test_analytics.pyuv run ruff format --check policyengine_household_api/data/analytics_setup.py tests/unit/data/test_analytics_setup.pyuv run ruff check policyengine_household_api/data/analytics_setup.pyuv run pytest -q tests/unit/data/test_analytics_setup.py tests/unit/data/test_alembic_migrations.py tests/unit/decorators/test_analytics.py tests/unit/endpoints/test_household.py tests/unit/utils/test_variable_usage_analytics.pyuv run pytest -q tests/unit/data/test_analytics_setup.py tests/unit/decorators/test_analytics.py tests/unit/endpoints/test_household.py tests/unit/data/test_alembic_migrations.py tests/unit/utils/test_variable_usage_analytics.pyuv run ruff format --check policyengine_household_api/data/analytics_setup.py policyengine_household_api/decorators/analytics.py tests/unit/data/test_analytics_setup.py tests/unit/decorators/test_analytics.pyuv run ruff check policyengine_household_api/data/analytics_setup.py policyengine_household_api/decorators/analytics.pyANALYTICS_DATABASE_URL=sqlite:////tmp/household_api_alembic_smoke_autogen.db uv run alembic upgrade headuv run pytest -q tests/unit/data/test_alembic_migrations.py tests/unit/data/test_analytics_setup.py tests/unit/decorators/test_analytics.py tests/unit/endpoints/test_household.py tests/unit/utils/test_variable_usage_analytics.pyuv run ruff check alembic/versions/20260508_0002_calculate_variable_usage.py policyengine_household_api/data/analytics_setup.py policyengine_household_api/decorators/analytics.pyuv run ruff format --check alembic/versions/20260508_0002_calculate_variable_usage.py policyengine_household_api/data/analytics_setup.py policyengine_household_api/decorators/analytics.pyuv run pytest -q tests/unit/data/test_alembic_migrations.pyuv run pytest -q tests/unit/data/test_alembic_migrations.py tests/unit/data/test_analytics_setup.py tests/unit/decorators/test_analytics.py tests/unit/endpoints/test_household.py tests/unit/utils/test_variable_usage_analytics.pyuv run ruff check policyengine_household_api/utils/variable_usage_analytics.py tests/fixtures/data/analytics_setup.py tests/fixtures/data/analytics_setup_patches.py tests/fixtures/decorators/analytics_patches.py tests/fixtures/endpoints/household.py tests/conftest.pyuv run ruff check tests/conftest.py tests/unit/data/test_analytics_setup.py tests/unit/decorators/test_analytics.py tests/unit/endpoints/test_household.pyuv run pytest -q tests/unit/data/test_analytics_setup.py tests/unit/decorators/test_analytics.py tests/unit/endpoints/test_household.py tests/unit/utils/test_variable_usage_analytics.py tests/unit/data/test_alembic_migrations.pyuv run ruff check policyengine_household_api/endpoints/household.py policyengine_household_api/utils/variable_validation.py policyengine_household_api/utils/variable_usage_analytics.py policyengine_household_api/decorators/analytics.py policyengine_household_api/data/models.py policyengine_household_api/data/analytics_setup.py alembic/versions/20260512_0003_cap_analytics_variable_names.py tests/fixtures/data/analytics_setup_patches.py tests/unit/data/test_alembic_migrations.py tests/unit/endpoints/test_household.py tests/unit/utils/test_variable_usage_analytics.pyANALYTICS_DATABASE_URL=sqlite:////private/tmp/household_api_alembic_verify_20260512_0003.db uv run alembic upgrade headuv run pytest -q tests/unit/endpoints/test_household.py tests/unit/utils/test_variable_validation.py tests/unit/utils/test_variable_usage_analytics.py tests/unit/data/test_alembic_migrations.py tests/unit/data/test_analytics_setup.py tests/unit/decorators/test_analytics.pygit diff --check