Plugin for per-sub-project DORA metrics in a monorepo - #9037
Open
eyal4D wants to merge 12 commits into
Open
Conversation
…repo Repos containing multiple logically separate projects (each deployed by its own CI job, PRs tagged by label) previously collapsed into a single set of DORA numbers, since DevLake's scope model is one-scope-per-repo. Adds a new metric plugin, monorepo, that runs after dora and attributes deployments (by CI job name) and merged pull requests (by label) to a configured sub-project, writing per-sub-project deployment and change lead time metrics to two new tables. Nothing existing is modified: dora and the core scope model are untouched, and PR coding/pickup/review time are reused from dora's own project_pr_metrics rather than recomputed. Includes unit tests, an e2e test with fixtures, and Grafana dashboards (mysql + postgresql) to view the output.
Adds a Project Settings panel for the monorepo plugin, letting users define sub-projects (name, PR labels, deploy job pattern) directly in config-ui rather than only via the raw API, following the existing pattern used for the linker plugin's fields.
RunAfter() is advisory-only: core's blueprint plan builder (GeneratePlanJsonV200) merges every enabled metric plugin's plan with ParallelizePipelinePlans, which zips stages together by index and never consults RunAfter. With dora and monorepo both enabled on a project, monorepo's single stage ran concurrently with dora's first stage instead of after dora's third stage, where project_pr_metrics and cicd_deployment_commits actually get written — silently producing nil coding/pickup/review times with no error. Pads monorepo's plan with empty stages so its real work lands strictly after dora's plan completes. This is a workaround for the unenforced RunAfter contract, not a true fix; it's coupled to dora's current 3-stage plan (documented in the code). Adds a regression test locking in the padded shape, and corrects the RunAfter doc comment which previously claimed ordering was guaranteed. Verified: fresh blueprint plan generation on the rebuilt binary shows org -> dora(x3) -> [empty x3] -> monorepo; migration and e2e tests pass on both MySQL and a real Postgres instance.
…' into feature/subprojects-for-monorepo
…able Adds the core schema for unified monorepo support (design §2): a nullable sub_project column on pull_requests, pull_request_commits and project_pr_metrics, and a new cicd_deployment_subprojects mapping table for the many-to-many relationship between a deployment and the sub-project(s) it deployed. Three migrations register the new columns/table and backfill sub_project from the monorepo plugin's existing compat tables for projects already using it; the backfill is written with portable correlated-subquery SQL (no MySQL-only JOIN or Postgres-only FROM forms) and skips cleanly when the monorepo plugin's tables don't exist on this install. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
ConvertApiMrCommitsMeta.Dependencies was missing ExtractApiMrCommitsMeta, so commit conversion could race ahead of extraction and silently convert zero or partial commits from _tool_gitlab_mr_commits. This was one of two previously opened-and-closed fix MRs folded into the monorepo design doc (Fix 1). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ibuted Rewrites the monorepo plugin's subtask split per the design doc's §3.3 redesign: - attributeDeployments now also writes the core cicd_deployment_subprojects mapping table (dual-writing monorepo_subproject_deployments for one release, for backward compatibility), and writes 'unattributed' rows for monorepo deployments that match no configured sub-project. - attributePullRequests is attribution-only: it tags pull_requests.sub_project (and, from there, pull_request_commits.sub_project) for every pull request regardless of merge status, removing the pr.merged_date IS NOT NULL filter (Fix 2) and the old merge-date-nearest-deployment heuristic (firstDeploymentAfter/loadSubProjectDeployments/loadDoraPrMetrics) along with the coding/pickup/review/deploy/cycle-time computation it used to do inline. - A new third subtask, updateProjectPrMetricsSubProject, tags project_pr_metrics.sub_project from pull_requests.sub_project, cross-checks it against the deployment DORA attributed the PR to via project_pr_metrics.deployment_commit_id (logging - not failing on - disagreements as a config hygiene signal), and backfills monorepo_subproject_pr_metrics using DORA's already-computed numbers instead of recomputing them. This is a correction: existing monorepo users will see deploy_time/cycle_time change on upgrade. - MonorepoOptions gains includeUnattributed (design decision 3, default true): when false, unmatched PRs/deployments are left unclassified instead of getting the 'unattributed' sentinel, matching the pre-existing behaviour. - NewSubProjectMatcher now rejects 'unattributed'/'All' as sub-project names and rejects empty prLabels entries. - impl.go registers all three subtasks and threads includeUnattributed through to task options. Test coverage: unit tests for the new validation and the attribution decision logic (resolveSubProject); an e2e dataflow test covering all three subtasks including the label/deploy-pattern mismatch case (both the "noise" case from a multi-sub-project deployment and a genuine misconfiguration) and the includeUnattributed=false path; and two new migration tests (TestMigrationAddsSubProjectColumnsAndTable, TestBackfillSubProjectFromMonorepo) that run the real registered core + monorepo migrations against an isolated database and verify the backfill is idempotent. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Mirrors the backend validation in NewSubProjectMatcher (backend/plugins/monorepo/tasks/task_data.go): a sub-project cannot be named 'unattributed' or 'All', since those collide with the sentinel values the monorepo plugin and dashboards use for unmatched/ungrouped rows. Shows an inline warning and disables Save while any configured sub-project uses a reserved name, instead of only finding out after the backend rejects the blueprint plan. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Rewrites all 6 panels of monorepo-subprojects.json (MySQL and PostgreSQL) to read from the new core tables (cicd_deployment_commits joined through cicd_deployment_subprojects, and project_pr_metrics) instead of the deprecated monorepo_subproject_deployments/monorepo_subproject_pr_metrics compat tables, per design §5.2's recommendation to migrate this dashboard rather than keep it on the compat tables. Deployment/PR-count panels use COALESCE(sub_project, 'All') so single-repo (non-monorepo) projects still render a sensible group. The "Unattributed merged PRs" panel simplifies to a direct pull_requests.sub_project = 'unattributed' filter now that attribution lives on the PR row itself. The "Attributed Deployments" detail table drops its CI-job-name column, since that information isn't reconstructable from the mapping table alone without re-deriving the regex match in SQL; its description is updated to say so. Note: this is the one dashboard fully migrated in this change. The broader dashboard file list in design §5.2 (Gitlab.json, dora-details-*, engineering-throughput-and-cycle-time*, engineering-overview.json, dora-by-team.json) was surveyed but deliberately NOT modified - see the implementation report for why. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…eries Applies the mechanical COALESCE(sub_project, 'All') transform (design §5.1/§5.3) to every panel across Gitlab.json, engineering-overview.json and engineering-throughput-and-cycle-time.json (MySQL + PostgreSQL) that is safely convertible: a time series with exactly one time column and one numeric value column, which Grafana's long-format convention already renders as a single line and will now render as one line per sub-project once a text grouping column is added. sub_project is inserted as the second SELECT column (matching the design doc's own example) and added to GROUP BY; original value column names/aliases are preserved. Panels deliberately left untouched, and why: - Grafana "stat" panels (single KPI tiles): their reduceOptions is configured with values:false, meaning they collapse all returned rows into one number via "last non-null". Adding a GROUP BY would silently make the tile show an arbitrary single sub-project's value instead of the whole-project total - fixing that needs a reduceOptions change too, which needs Grafana to verify renders correctly. - Wide-format time series with multiple value columns (e.g. "PRs Opened/Merged" - two metric columns in one row) and barchart/table panels already grouping by something else (e.g. "Top 20 Contributors by Merged PRs" groups by author) - adding sub_project would multiply or reshape the series in ways that need a Grafana-side call, not a mechanical SQL edit. - Every dora-details-*.json and dora-by-team.json panel, and most of engineering-throughput-and-cycle-time-team-view.json: multi-CTE queries with window functions (percent_rank, row_number, calendar-generation CTEs for medians). Retrofitting a grouping dimension into these correctly means restructuring most of the CTEs, which is real engineering work that needs a live Grafana instance to verify - not safe to do blind. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…-for-monorepo # Conflicts: # backend/core/models/migrationscripts/register.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Repos containing multiple logically separate projects (each deployed by its own CI job, PRs tagged by label) previously collapsed into a single set of DORA numbers, since DevLake's scope model is one-scope-per-repo.
Adds a new metric plugin, monorepo, that runs after dora and attributes deployments (by CI job name) and merged pull requests (by label) to a configured sub-project, writing per-sub-project deployment and change lead time metrics to two new tables. Nothing existing is modified: dora and the core scope model are untouched, and PR coding/pickup/review time are reused from dora's own project_pr_metrics rather than recomputed.
Includes unit tests, an e2e test with fixtures, and Grafana dashboards (mysql + postgresql) to view the output.