Feat: Compatibility Verification Suite (k6 test harness)#41
Merged
Conversation
Add AI-DLC documentation for the new Compatibility Verification Suite unit (UOW-05): state tracking, audit log entries, requirements analysis, workflow planning, and NFR requirements plan for k6-based Firefly-III compatibility testing. Co-Authored-By: GitHub Copilot <noreply@github.com>
…patibility Verification Suite
…r UOW-05 Compatibility Verification Suite
Add a comprehensive k6-based automated test suite that validates Utopia's API responses against Firefly-III compatibility requirements. - Seed data generator (scripts/seed/): TypeScript/Bun script that truncates tables and inserts deterministic test fixtures - k6 test fixtures (k6/fixtures/): JSON schemas for strict response validation of auth, accounts, and transactions endpoints - k6 test harness (k6/harness.ts): Shared utilities for auth, response validation, pagination checks, and Firefly-III envelope verification - k6 test scripts: auth.ts (6 scenarios), accounts.ts (7 scenarios), transactions.ts (8 scenarios) - k6 runner script (k6/run-all.sh): Orchestrates full test suite execution - Docker Compose: Added k6 service with testing profile - GitHub Actions: CI workflow for automated compatibility testing on PR - Updated .env.example with k6 environment variables - Updated aidlc-state.md and audit.md for UOW-05 Code Generation Co-Authored-By: GitHub Copilot <<EMAIL>>
Fix multiple issues causing the compatibility-check workflow to fail: 1. Health check: Use /metrics endpoint (unauthenticated) on port 3000 instead of /api/v1/about (requires auth, redirects through Caddy) 2. Seed data: Expose postgres port 5432 to host so bun seed script can connect from the GitHub Actions runner 3. k6 profile: Add --profile testing flag to docker compose run commands so the k6 service is activated 4. .env for CI: Create .env from .env.example with Docker Compose network settings (postgres host, sslmode=disable, strict_ssl=false) 5. Dockerfile: Add curl for container healthcheck support 6. docker-compose: Add healthcheck to utopia-api, expose ports, use bind mount for k6 results, depend on healthy state
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a comprehensive k6-based automated test suite that validates Utopia's API responses against Firefly-III compatibility requirements.
What's Included
Seed Data Generator (
scripts/seed/)k6 Test Suite (
k6/)harness.ts): Shared utilities for auth, response validation, pagination checks, and Firefly-III envelope verificationauth.ts): 6 scenarios — token issuance, authenticated request, unauthenticated rejection, token revocation, revoked token rejection, invalid bootstrap keyaccounts.ts): 7 scenarios — list, get, create, update, delete, pagination, type filtertransactions.ts): 8 scenarios — list, get, create, update, delete, account-scoped listing, pagination, type filterfixtures/): JSON schemas for strict response validationInfrastructure
k6service withtestingprofile, volume mounts for k6/ and scripts/seed/.github/workflows/compatibility-check.yml): CI workflow that starts the stack, loads seed data, runs the full k6 suite, and uploads results as artifacts on every PR.env.example: AddedAPP_BASE_URLandK6_OUTPUT_DIRvariablesDocumentation
aidlc-docs/construction/compatibility-verification-suite/code/code-summary.md)aidlc-state.mdandaudit.mdfor UOW-05 Code GenerationNFR Compliance
http_req_duration: ["p(95)<500"]TRUNCATE ... CASCADEbefore seedTest Plan
bun run scripts/seed/index.ts./k6/run-all.sh