Keep the folder size index current from the write, not from the sweep - #443
Merged
Merged
Conversation
added 2 commits
September 26, 2026 20:17
`exifr` was last published in 2022. It is the one thing in the image that opens a file somebody else wrote using code nobody maintains any more, and it is loaded lazily in the metadata route precisely because nobody was sure of it. `exif-reader` replaces it: maintained alongside sharp, and it does nothing but walk a TIFF-shaped block with a bounds check on every read. The block itself needs no file access — sharp already opens the image to report its dimensions and hands the raw EXIF block back with them — so a photograph's details now come from one pass instead of two, and `exifr` leaves the dependencies. `utils/exifDetails.js` holds what the block is read for: when the picture was taken, the camera and lens, the exposure, the orientation, the place. Written as its own file because the metadata route is about what kind of thing a file is, and this is about what a photograph says of itself. ## Checks `exif-details.test.js`, 20 tests over the reading itself — a block that is not EXIF, a truncated one, dates in the three shapes cameras write them, a rational that divides by zero, coordinates in both hemispheres. Making the reader answer nothing turns six of them red. `metadata.test.js`, 13 tests through the route. Whole backend suite: 2 509 passed, 2 failed — the two that fail on `main` on its own. `capabilities.js`, `documentText.js` and `pdfTextExtract.js` were in this batch at first and are not any more: probing which optional tools the machine has, and reading a document's text, are their own ground, and bringing them here turned three of `main`'s own tests red.
The index has three ways of learning a folder's size: a first pass, a watcher, and an
adaptive sweep that walks folders again looking for changes. Nothing told it about a write
the application itself had just made — and the application knows the exact number of bytes
at that moment, which is the one case where no traversal is needed at all.
So after an upload, a save in the editor, a new folder or a document created from a
template, the size shown for the folder was the one from before, until a sweep happened to
come round to it. On a large volume that is minutes.
`services/folderSizeHooks.js` takes the delta each operation already has and propagates it
to the folder and its ancestors. Four call sites here: the upload, the editor save, a new
folder, and a document created with contents. Every hook is best-effort — gated on the
feature being on, and swallowing its own errors — because a folder size is not worth
failing a write for.
The editor save reads what the file weighed before it wrote, so the folder gains the
difference rather than the whole of the new file. That is the whole reason the hook takes
two numbers.
`folderSizeIndex` also records which mode the index was built in, and `folderSizeManager`
measures again when it starts in a different one: sizes measured as "apparent" and sizes
measured on disk are not the same numbers, and keeping the other mode's answers was
showing a total nobody could reconcile with anything.
## Checks
81 tests across `folder-size-hooks`, `folder-size-flush`, `folder-size-disabled`,
`folder-size-mode-change`, `folderSizeIndexer` and the `folderSize` route. Making
`onFileWritten` say nothing — the behaviour being replaced — turns 34 of them red.
Whole backend suite: 2 545 passed, 2 failed, the two that fail on `main` on its own.
`frontend/.eslintrc.cjs` gains `env: { browser, es2022 }`. `ecmaVersion: 'latest'` sets
what syntax is allowed and not what exists at run time, so `globalThis` — standard since
ES2020 — read as an undefined name in nine places. That is nine errors fewer in
`npm run lint`, and it would have applied to any file that reached for it.
The hooks belong in three more places — the two office editors and the archive extraction —
and those call sites travel with their own batches.
cerede2000
pushed a commit
to cerede2000/NextExplorer
that referenced
this pull request
Sep 26, 2026
The batch is reversed, so its findings move from PORT to DONE in the same commit that sends them. What is left to reverse is whatever `scripts/parity.mjs` still reports.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
P3-11 of phase 3 (see #373). Stacked on #433–#442.
The index is never told about a write the application just made
The folder size index learns a size three ways: a first pass, a watcher, and an adaptive
sweep that walks folders again looking for changes. Nothing tells it about a write the
application itself performed — and that is the one case where it needs no traversal at all,
because the exact number of bytes is already in hand.
So after an upload, a save in the editor, a new folder or a document created from a
template, the size shown for the folder is the one from before, until a sweep happens to
come round to it. On a large volume that is minutes.
The write hands over its own delta
services/folderSizeHooks.jstakes the number each operation already has and propagates itto the folder and its ancestors. Four call sites here — the upload, the editor save, a new
folder, a document created with contents — and every hook is best-effort: gated on the
feature being on, and swallowing its own errors, because a folder size is not worth failing
a write for.
The editor save reads what the file weighed before it wrote, so the folder gains the
difference rather than the whole of the new file. That is why the hook takes two numbers.
folderSizeIndexalso records which mode the index was built in, andfolderSizeManagermeasures again when it starts in a different one: sizes measured as apparent and sizes
measured on disk are not the same numbers, and keeping the other mode's answers showed a
total nobody could reconcile with anything.
Also here
frontend/.eslintrc.cjsgainsenv: { browser, es2022 }.ecmaVersion: 'latest'sets whatsyntax is allowed, not what exists at run time, so
globalThis— standard since ES2020, andwhat a store reaches for to touch a timer the browser owns — read as an undefined name in
nine places. Nine errors fewer in
npm run lint, and it would have applied to any file thatreached for it.
Checks
81 tests across
folder-size-hooks,folder-size-flush,folder-size-disabled,folder-size-mode-change,folderSizeIndexerand thefolderSizeroute.Making
onFileWrittensay nothing — the behaviour being replaced — turns 34 of themred.
mainalone.npm run format:checkreports 21 files againstmain's 22: one of them is a file thisbatch rewrites, which comes back clean.
The hooks belong in three more places — the two office editors and the archive extraction —
and those call sites travel with their own batches.