Repository navigation
fix(metrics): speed up top workflows by runs count query - #3771
Conversation
The top workflows metric reached workflow runs through the organization and workflows tables, so the aggregate could not use the (organization_id, created_at) index on workflow_runs. It then loaded every workflow with runs in the window one by one, plus its project, before keeping only the top N. Aggregate on workflow_runs.organization_id directly, pick the top N workflows first, and load them with their projects in a single query. Runs of soft-deleted workflows are no longer counted, and ties are broken by workflow ID so the order is deterministic. Refs PFM-8118 Assisted-by: Claude Code Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev> Chainloop-Trace-Sessions: 15511d4e-2aab-40e1-b9b1-9b196874fc95
PR validation — ✅ 3 passing
AI Session Checks — 🟡 81% ·
|
| Avg score | Sessions | Failing policies | Attribution | Files | Lines | Total Duration |
|---|---|---|---|---|---|---|
| 🟡 81% | 2 | 89% AI / 11% Human | 2 | +244 / -56 | 29m23.643s |
🟡 78% — 89% AI — ⚠️ 1 policies failing
-
Nov 2, 2026 22:38 UTC · 28m19s · $16.33 · 288 in / 106.0k out · claude-code 2.1.316 (claude-opus-5-5)
Change Summary
-
- Optimizes
OrgMetricsRepo.TopWorkflowsByRunsCountto query byworkflow_runs.organization_idand batch workflow loading. - Excludes deleted workflows from ranked results and makes top-N ordering deterministic.
- Adds integration coverage for ranking, project filtering, empty visibility, time-window, and deleted-workflow behavior.
- Optimizes
AI Session Overall Score
-
🟡 78% — Focused fix, but ticket alignment and page-level verification stayed only partial.
AI Session Analysis Breakdown
-
🟢 95% · scope-discipline
-
No notes.
🟢 93% · solution-quality
-
🟢 The fix targets the hot query directly with indexed filtering and batched loading. · High Impact
🟢 84% · user-trust-signal
-
🟡 The user asked for RBAC and parity rechecks after the PR opened, signaling a brief confidence dip. · Low Severity
🟡 76% · verification
-
🟢 AI reran old and new implementations instead of stopping at the first passing suite. · High Impact
🟡 67% · context-and-planning
-
🟢 AI investigated traces first and asked before pivoting to the query-fix path. · High Impact
🟠 The bug hunt became code work without a written plan or explicit ticket-level success criteria. · Medium Severity
💡 When a bug hunt narrows to a likely cause, write a short plan before coding so the success bar stays explicit.
🟡 58% · alignment
-
🟠 The query fix landed, but neither the diff nor the session confirms the original SDLC Insights code-13 ticket is resolved. · Medium Severity
💡 When you narrow a bug ticket to a suspected cause, keep the ticket open until the user-visible symptom is verified.
🟠 AI summarized before-and-after test parity before it had been verified, then corrected the claim after the user asked. · Medium Severity
💡 Run the explicit before-and-after comparison before summarizing parity; silent subtests are easy to overread.
-
File Attribution
█████████████████░░░89% AI / 11% HumanStatus Attribution File Lines modified ai app/controlplane/pkg/biz/orgmetrics_integration_test.go+162 / -2 modified ai app/controlplane/pkg/data/orgmetrics.go+70 / -49
Policies (4, 1 failing)
Status Policy Material Messages ✅ Passed ai-config-ai-agents-allowedai-coding-session-15511d- ✅ Passed ai-config-no-dangerous-commandsai-coding-session-15511d- ⚠️ Failedai-config-no-secretsai-coding-session-15511d- Secret (generic-password) detected in session content [turn=634, source=assistant-tool_use:Edit, line=1]: {"file_path":"/home/migmartri/work/chainloop/chainloop/.claude/worktrees/temporal-hatching-reef/app/controlplane/pkg/biz/orgmetrics_integration_test.go","new_string":"// OCI credentials used to create...
- Secret (generic-password) detected in session content [turn=651, source=tool_result, line=4]: 42:var ociTestKeypair = &credentials.OCIKeypair{Repo: ociTestRepo, Username: ociTestUsername, Password: [CHAINLOOP_TRACE_REDACTED:generic-password]
✅ Passed ai-config-mcp-servers-allowedai-coding-session-15511d- -
🟢 84% — 100% AI — ⚠️ 1 policies failing
-
Nov 2, 2026 22:54 UTC · 14m7s · $1.31 · 81 in / 16.1k out · claude-code 2.1.316 (claude-sonnet-5-5)
Change Summary
-
- Refactors
TopWorkflowsByRunsCountto aggregate fromworkflow_runs, keep deterministic ordering, and exclude soft-deleted workflows in both query stages. - Adds orgmetrics integration coverage for deleted workflows, project filters, top-N limits, and empty-result windows.
- Reuses shared OCI test constants in the integration suite to satisfy lint.
- Refactors
AI Session Overall Score
-
🟢 84% — Clean session overall; only the new biz integration coverage went unrun.
AI Session Analysis Breakdown
-
🟢 92% · alignment
-
No notes.
🟢 91% · solution-quality
-
🟢 The query filters deleted workflows during aggregation and the later batch load. · High Impact
🟢 90% · scope-discipline
-
No notes.
🟢 88% · context-and-planning
-
🟢 The opening task specified scope, workflow, testing, and trailer constraints. · High Impact
🟢 86% · user-trust-signal
-
No notes.
🟡 65% · verification
-
🟢 It ran
SKIP_INTEGRATION=true go test ./pkg/data/and observed a pass. · High Impact🟠 The session ran
go test ./pkg/data/, but not the changed biz integration tests. · Medium Severity💡 When you touch a new test file, run that package tests once before declaring the follow-up done.
-
File Attribution
████████████████████100% AI / 0% HumanStatus Attribution File Lines modified ai app/controlplane/pkg/biz/orgmetrics_integration_test.go+11 / -4 modified ai app/controlplane/pkg/data/orgmetrics.go+1 / -1
Policies (4, 1 failing)
Status Policy Material Messages ✅ Passed ai-config-ai-agents-allowedai-coding-session-a527cc- ✅ Passed ai-config-no-dangerous-commandsai-coding-session-a527cc- ⚠️ Failedai-config-no-secretsai-coding-session-a527ccSecret (generic-password) detected in session content [turn=177, source=assistant-tool_use:Bash, line=1]: f=pkg/biz/orgmetrics_integration_test.go && sed -i 's/Password: "pass"}/Password: [CHAINLOOP_TRACE_REDACTED:generic-password]; s/"username", "pass", backendType/"username", ociPassword, backendType/' ... ✅ Passed ai-config-mcp-servers-allowedai-coding-session-a527cc- -
Security Checks — ✅ 5 passing
✅ secret-scan
| Status | Policy | Messages |
|---|---|---|
| ✅ Passed | secrets-detection |
- |
✅ sast-scan
| Status | Policy | Messages |
|---|---|---|
| ✅ Passed | cwe-top25 |
- |
| ✅ Passed | owasp-top10-2025 |
- |
| ✅ Passed | sast |
- |
| ✅ Passed | cwe-top26-40-cusp |
- |
Scans not applied (3)
| Scan | Reason |
|---|---|
vulnerability-scan |
no manifest/lockfile changed |
github-actions-scan |
no workflow files changed |
iac-scan |
no IaC files changed |
Security context
✅ Nothing this change touches has a recorded security-fix history.
View in Chainloop ↗ · How this works ↗
Powered by Chainloop and Chainloop Trace
There was a problem hiding this comment.
All reported issues were addressed across 2 files
Reply with feedback, questions, or to request a fix.
View guided diff | Re-trigger cubic
…rkflows Assisted-by: Claude Code Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev> Chainloop-Trace-Sessions: a527cce1-a4d8-4e3f-9265-6598c30714f5
Fixes the goconst lint failure on the repeated "pass" literal. Refs PFM-8118 Assisted-by: Claude Code Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev> Chainloop-Trace-Sessions: 15511d4e-2aab-40e1-b9b1-9b196874fc95
Summary
The SDLC Insights "top workflows" metric (
OrgMetricsService/TopWorkflowsByRunsCount) had two inefficiencies, visible in production traces for a large organization:workflow_runsthrough theorganizationsandworkflowstables, so it could not use the(organization_id, created_at)index. Its latency varied widely between calls (6ms to 300ms in sampled traces).This change:
workflow_runs.organization_idso the existing index drives the scan.Refs PFM-8118. This reduces the latency and variance of one of the SDLC Insights requests. It does not on its own confirm the root cause of the intermittent code 13 errors reported in the ticket.
This change was produced with the assistance of Claude Code.