Async org-indexing worker + per-member monthly resolution#102
Draft
peetzweg wants to merge 1 commit into
Draft
Conversation
The request path builds orgs at lifetime-totals resolution only. This adds a background worker (scripts/refresh-orgs.ts) that upgrades orgs to monthly resolution — the data behind a new company commit chart — paced well under GitHub's rate limit with a reserved floor for live traffic. - org_member_monthly table (one row per org/member/month) + migration - fetchOrgMemberWindows returns per-window totals; fetchOrgMemberContributions now sums them for the request path - getOrgSummary assembles a cumulative chart series from monthly_commits rows - OrgView renders the company chart once the worker has written monthly rows - oversized orgs are recorded (builtAt null) so the worker can pick them up
|
The preview deployment for peetzweg/commit-history:coolify-bwc8csg4w8gkkgcg8s404cks is ready. 🟢 Open Preview | Open Build Logs | Open Application Logs Last updated at: 2026-07-08 19:32:47 CET |
This was referenced Jul 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #84. Adds a background worker that indexes orgs ("companies") at monthly resolution — the data behind a company commit chart — instead of the request path's lifetime-totals-only build.
Why: the request path deliberately stores lifetime totals only (~2 requests/member); monthly resolution costs ~12× that, too heavy for a live build. And mega-orgs (e.g.
google, 635 members) can't be built on-demand at all.How: a new
scripts/refresh-orgs.tsworker enumerates each org's members and fetches monthly org-scoped windows, paced under GitHub's 5k/hr limit with a 500-request floor reserved for live traffic (same politeness model asbackfill-contributions.ts). Neworg_member_monthlytable holds per-member months; rolled up into the org'smonthly_commitsrows for the chart. The request path now records oversized orgs (builtAtnull) so the worker can pick them up.Draft — landing the lightweight "record looked-up orgs" slice separately first; this merges once we've run the worker end-to-end.