Eliminate N+1 queries on Project & Person admin changelists (#1346, Phase 2)#1349
Merged
Conversation
…1346) Phase 2 of the admin changelist audit. The Project list page could fire 1,000+ queries (≈10 DB-touching callable columns × default 100 rows) and the People list page was similar, dominated by per-row position lookups. Both are now ~constant-query regardless of row count. Project (project_admin.py): - get_queryset() annotates every count column as an independent scalar subquery (related_count_subquery), avoiding the row-multiplication of stacked Count() joins: pubs, talks, videos, banners, distinct people, current/past members, and the role∪author "contributors" union. - Most-recent-artifact date/type derived from three max-date subqueries (Greatest()), sortable. Columns repointed at the annotations; model methods kept for the public site. list_per_page=50. Person (person.py + person_admin.py + position_role_list_filter.py): - get_latest_position/get_earliest_position now pick from position_set.all() in Python instead of .latest()/.earliest(), so a prefetch resolves them (and every "current ..." column + the Role filter that funnel through them) with zero per-row queries. Behavior preserved. Same prefetch-friendly rewrite for get_total_time_in_role and is_alumni_member (they used .filter() on positions). - PersonAdmin.get_queryset() prefetches position_set and projectrole_set__project and annotates the 3 count columns; the Role filter prefetches position_set in its per-person loop. list_per_page=50. Shared helper related_count_subquery() added to admin/utils.py. Tests (test_admin_perf.py): assert each annotation/ rewritten method equals the original per-row result, plus a steady-state guard that the changelist query count stays flat as rows grow 4 -> 12 (the N+1 regression test). Full suite green (362). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 2 of #1346 (admin changelist audit). The two heaviest admin list pages were doing per-row DB work that scaled with the number of rows:
get_banner_count, the member counts,get_most_recent_artifact_*×2, the contributor union…) × the default 100 rows/page = 1,000+ queries for one list view..latest()/.earliest()), plus thePositionRoleListFilterlooping every Person on every load.Both are now roughly constant-query regardless of row count, and the count columns became sortable as a bonus.
Project (
project_admin.py)get_queryset()annotates every count as an independent scalar subquery (related_count_subquery), so the joins don't multiply each other (stackingCount()over several M2M relations both blows up rows and risks wrong counts). Covers pubs, talks, videos, banners, distinct people, current/past members, and the role∪publication-author Contributors union (kept per your call, computed as one correlated subquery).max-datesubqueries combined withGreatest().list_per_page = 50.Person (
person.py+person_admin.py+position_role_list_filter.py)get_latest_position/get_earliest_positionnow pick fromposition_set.all()in Python instead of.latest()/.earliest(), so a single prefetch resolves them — and with them every "current title/role/date/duration",is_active, and the Role filter that all funnel through them. Behavior preserved. Same prefetch-friendly rewrite forget_total_time_in_roleandis_alumni_member(they used.filter()on positions).PersonAdmin.get_queryset()prefetchesposition_set+projectrole_set__projectand annotates the 3 count columns; the Role filter prefetchesposition_setin its per-person loop.list_per_page = 50.Shared helper
related_count_subquery()added toadmin/utils.py.Tests (
test_admin_perf.py)pub_count == get_publication_count(),contributor_count == get_contributor_count(), most-recent date/type, the position rewrites).Full suite green locally (362 tests, in-container with
--settings=makeabilitylab.settings_test).Merge note
Touches
project_admin.py, which Phase 1 (#1347) also edits — verified they auto-merge with no conflicts in either order (different regions of the file).Screenshots
Admin changelist sort arrows on the new count columns — to be added.
🤖 Generated with Claude Code