Skip to content

Eliminate N+1 queries on Project & Person admin changelists (#1346, Phase 2)#1349

Merged
jonfroehlich merged 1 commit into
masterfrom
1346-admin-perf-phase2
Jun 19, 2026
Merged

Eliminate N+1 queries on Project & Person admin changelists (#1346, Phase 2)#1349
jonfroehlich merged 1 commit into
masterfrom
1346-admin-perf-phase2

Conversation

@jonfroehlich

Copy link
Copy Markdown
Member

Phase 2 of #1346 (admin changelist audit). The two heaviest admin list pages were doing per-row DB work that scaled with the number of rows:

  • Projects — ~10 DB-touching callable columns (get_banner_count, the member counts, get_most_recent_artifact_* ×2, the contributor union…) × the default 100 rows/page = 1,000+ queries for one list view.
  • People — ~14 columns, nearly all funneling through per-row position lookups (.latest()/.earliest()), plus the PositionRoleListFilter looping every Person on every load.

Both are now roughly constant-query regardless of row count, and the count columns became sortable as a bonus.

Project (project_admin.py)

  • get_queryset() annotates every count as an independent scalar subquery (related_count_subquery), so the joins don't multiply each other (stacking Count() over several M2M relations both blows up rows and risks wrong counts). Covers pubs, talks, videos, banners, distinct people, current/past members, and the role∪publication-author Contributors union (kept per your call, computed as one correlated subquery).
  • Most-recent-artifact date and type derived from three max-date subqueries combined with Greatest().
  • Columns repointed at the annotations; the model methods stay for the public site/detail views. list_per_page = 50.

Person (person.py + person_admin.py + position_role_list_filter.py)

  • get_latest_position / get_earliest_position now pick from position_set.all() in Python instead of .latest()/.earliest(), so a single prefetch resolves them — and with them every "current title/role/date/duration", is_active, and the Role filter that all funnel through them. Behavior preserved. Same prefetch-friendly rewrite for get_total_time_in_role and is_alumni_member (they used .filter() on positions).
  • PersonAdmin.get_queryset() prefetches position_set + projectrole_set__project and annotates the 3 count columns; the Role filter prefetches position_set in its per-person loop. list_per_page = 50.

Shared helper related_count_subquery() added to admin/utils.py.

Tests (test_admin_perf.py)

  • Correctness: each annotation / rewritten model method asserted equal to the original per-row result (e.g. pub_count == get_publication_count(), contributor_count == get_contributor_count(), most-recent date/type, the position rewrites).
  • N+1 regression: renders the real admin ChangeList (filters + every column cell) and asserts the steady-state query count stays flat as rows grow 4 → 12. (Verified the fix directly: a fully-warmed 12-row People changelist is 10 queries, flat — down from ~1 + 8/row.)

Full suite green locally (362 tests, in-container with --settings=makeabilitylab.settings_test).

Merge note

Touches project_admin.py, which Phase 1 (#1347) also edits — verified they auto-merge with no conflicts in either order (different regions of the file).

Screenshots

Admin changelist sort arrows on the new count columns — to be added.

🤖 Generated with Claude Code

…1346)

Phase 2 of the admin changelist audit. The Project list page could fire
1,000+ queries (≈10 DB-touching callable columns × default 100 rows) and
the People list page was similar, dominated by per-row position lookups.
Both are now ~constant-query regardless of row count.

Project (project_admin.py):
- get_queryset() annotates every count column as an independent scalar
  subquery (related_count_subquery), avoiding the row-multiplication of
  stacked Count() joins: pubs, talks, videos, banners, distinct people,
  current/past members, and the role∪author "contributors" union.
- Most-recent-artifact date/type derived from three max-date subqueries
  (Greatest()), sortable. Columns repointed at the annotations; model
  methods kept for the public site. list_per_page=50.

Person (person.py + person_admin.py + position_role_list_filter.py):
- get_latest_position/get_earliest_position now pick from
  position_set.all() in Python instead of .latest()/.earliest(), so a
  prefetch resolves them (and every "current ..." column + the Role
  filter that funnel through them) with zero per-row queries. Behavior
  preserved. Same prefetch-friendly rewrite for get_total_time_in_role
  and is_alumni_member (they used .filter() on positions).
- PersonAdmin.get_queryset() prefetches position_set and
  projectrole_set__project and annotates the 3 count columns; the Role
  filter prefetches position_set in its per-person loop. list_per_page=50.

Shared helper related_count_subquery() added to admin/utils.py.

Tests (test_admin_perf.py): assert each annotation/ rewritten method
equals the original per-row result, plus a steady-state guard that the
changelist query count stays flat as rows grow 4 -> 12 (the N+1 regression
test). Full suite green (362).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jonfroehlich jonfroehlich merged commit b9f5df8 into master Jun 19, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant