Load Citation Impact tab from a slim index, fetch timelines on demand - #9
Open
LukasWallrich wants to merge 2 commits into
Open
Load Citation Impact tab from a slim index, fetch timelines on demand#9LukasWallrich wants to merge 2 commits into
LukasWallrich wants to merge 2 commits into
Conversation
Opening the tab downloaded data/originals.json (6.18 MB), of which the table needs only the 1.13 MB index; the remaining 5 MB is per-original citation timelines and replication lists used solely by the row popup. scripts/split_originals.py writes data/originals_index.json (table rows plus a `file` pointer) and one data/originals/<key>.json per original. refresh_data.py calls it at the end of each run; running the module directly rebuilds both from an existing originals.json. Both outputs are rerunnable: unchanged files are left untouched and files for originals that dropped out are removed. The tab now fetches the index on open and a study's detail file when its row is clicked, caching it for the session. Filenames come from the index, so the browser never has to reproduce the Python sanitiser. originals.json is still written; run_fect.R and clean-json.yml read it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DpdpuV7MGiLcHoh9eTJB9N
|
Important Review skippedToo many files! This PR contains 2150 files, which is 2050 over the limit of 100. To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch. Upgrade to a paid plan to raise the limit. Usage-priced reviews support at most 300 files. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (2150)
You can disable this status message by setting the |
The index file is now small enough that a deep link to the tab can finish fetching before the Plotly CDN script has loaded, which raised 'Plotly is not defined' and showed the placeholder. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DpdpuV7MGiLcHoh9eTJB9N
Contributor
Author
|
Browser-checked locally on this branch: KPIs, table and the per-study modal (detail file 7 kB) render correctly. Deep-linking to the tab hit a Plotly load-order race that the smaller index now exposes; fixed in the second commit by waiting for Plotly before rendering. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Opening the Citation Impact tab downloaded
data/originals.jsonin full (6.18 MB). The paginated "Browse all originals" table uses only theindexpart of that file; the other ~5 MB is per-original citation timelines and replication lists, needed only when a row is clicked.scripts/split_originals.py(new) writesdata/originals_index.json(the table rows, each with afilepointer) and onedata/originals/<key>.jsonper original.scripts/refresh_data.pycallswrite_split()at the end ofwrite_outputs(), so a normal weekly run produces the split files. Runningpython3 scripts/split_originals.pyon its own rebuilds them from an existingoriginals.json— no API calls.assets/citation-impact.jsfetchesoriginals_index.jsonon tab open, and a study's detail file when its row is clicked (cached for the session).findStudyDoinow resolves?doi=deep links against the index.data/originals.jsonis still written:scripts/run_fect.Rand.github/workflows/clean-json.ymlread it. It is no longer served to the browser, so it could be retired in a follow-up ifrun_fect.Ris rewired.clean-json.ymlalso cleansoriginals_index.json; README data tree updated.Filenames are derived from the DOI in Python (lowercased, anything outside
[a-z0-9._-]replaced with_, hash suffix on collision or over-length) and carried in the index, so the JS never has to reproduce the sanitiser.Payload sizes
data/originals.json(before)data/originals_index.json(on open)Detail files: 2,144 of them, median 2.2 kB, max 7.8 kB. On-open transfer drops from ~816 kB to ~221 kB gzipped (−73%); a row click adds under 1 kB.
Verification
originals.json; every index row'sfileexists, filenames are unique and match^[a-z0-9][a-z0-9._-]*\.json$, and each detail file is byte-identical to the correspondingstudiesentry in the monolith. Index rows equal the oldindexrows plusfile.git statusclean; originals that drop out have their file deleted.node --check assets/citation-impact.jspasses.python3 -m http.serverand confirmeddata/originals_index.json, a detail file,data/meta.jsonand the JS all return 200 with the expected sizes.renderKPIs,renderAggregateandrenderCocitBreakdownread onlymeta.json,aggregate.json,fect_results.jsonandcocit_breakdown.json— unaffected.Not verified: the visual result has not been checked in a browser. The modal's loading state, the deep-link path (
?tab=citations&doi=…) and the fast-double-click guard were reviewed by reading the code only.🤖 Generated with Claude Code
https://claude.ai/code/session_01DpdpuV7MGiLcHoh9eTJB9N