[03/40] Say what the cache directory holds, and what the process is costing - #444
Merged
Merged
Conversation
cerede2000
pushed a commit
to cerede2000/NextExplorer
that referenced
this pull request
Sep 26, 2026
The batch is reversed, so its findings move from PORT to DONE in the same commit that sends them. What is left to reverse is whatever `scripts/parity.mjs` still reports.
cerede2000
pushed a commit
to cerede2000/NextExplorer
that referenced
this pull request
Sep 26, 2026
The batch is reversed, so its findings move from PORT to DONE in the same commit that sends them. What is left to reverse is whatever `scripts/parity.mjs` still reports.
cerede2000
force-pushed
the
p3-12
branch
4 times, most recently
from
September 27, 2026 12:29
b18026e to
c1cca5f
Compare
added 3 commits
September 27, 2026 19:24
Two routes built a command line with values from the request in it and gave the line to a shell. `routes/usage.js` pasted a folder's path into `du -sb "…"` and `df -Pk "…"`; `routes/permissions.js` pasted an owner, a group, a mode and a path into `chown`, `chgrp`, `chmod -R` and two `id` lookups. A name is not a shell string. Anything that closes the quoting leaves the rest for `/bin/sh` to run, as the user the server runs as. The usage one needs no privilege at all: any account that can create a folder can name one, and where AUTH_ENABLED is false that is anybody who can reach the server. `execFile` takes the arguments as a list, so there is no line for a shell to read and no shell. The commands, their output and the answers are unchanged — this is deliberately the smallest change that closes it, and not the rewrite that stands in nxzai#450 and nxzai#453. An argument list is not a free pass on its own: `chown` reads a leading dash as an option, so `--reference=/etc/shadow` would have copied another file's ownership onto the target. An account or group name has to look like one. `tests/routes/no-shell.test.js` — three cases. Two of them name a folder, and an owner, with a payload that creates a file, and check the file is not there; the third asks for `--reference=/etc/shadow` and expects a refusal. All three fail against the routes as they are, the first two by running the command. The payload only ever touches the working directory, and it is removed whatever an assertion does, so a run that does execute leaves nothing behind.
Two things about OIDC that this fixes.
**Which of two failures it was.** A sign-in that cannot start has one of two causes:
nothing was configured, or what was configured could not be made to work. The first
is answered by filling in OIDC_ISSUER and the rest; the second is answered by looking
at the provider. Both answered 404 "OIDC is not configured", so an administrator whose
provider was unreachable was sent to change a configuration that was already right.
`configureOidc` now records what it concluded and why, and the routes read it: 404
AUTH_OIDC_NOT_CONFIGURED for the first, 503 AUTH_OIDC_PROVIDER_UNAVAILABLE for the
second, and neither carries the network's own words — no ENOTFOUND, no internal host
name, in the body or in the address bar.
A browser asking for one of those addresses is looking at a page, not reading JSON, so
it is sent back to the sign-in screen with the code beside the sentence. The screen can
then say what the code means in the reader's own language.
**Where the callback comes back to.** The return address was built from a fixed
`baseURL`, so a deployment reached through a reverse proxy on a different name sent
people back to the wrong origin. It is now resolved from the request — but only from a
forwarded host behind a trusted proxy, and only when the result exactly matches an
origin the operator configured. A redirect target that a request header could choose
is an open redirect with extra steps.
The path is still held to a relative, same-site one, as before.
Also here, because it is the same files:
- `claimsFromIdToken` in the middleware and `uniqueOrigins` in the routes were local
copies of things `utils/idToken.js` and the new `utils/oidcRedirect.js` do; both go.
- The guest-session cookie was cleared on `/api` only, at five sign-in and sign-out
paths. A guest session left on `/` outlived the sign-in that should have ended it.
- `ServiceUnavailableError` (503) and the two AUTH_OIDC_* codes, which nothing had yet.
The account lockout that `routes/auth.js` also differs in is deliberately not here: it
is a different subject and goes with releasing a locked account.
## Checks
Seven test files, 107 tests, including three this fork had and `main` did not:
`oidc-middleware`, `oidcOrigin` and `auth-oidc-routes`. Neutralising
`getOidcAvailability` so it reports one verdict for both causes turns three of them
red — the three that tell the two apart.
Whole backend suite: 2 391 passed, 2 failed, the two that fail on `main` on its own
(`auth.test.js` on the current password, `browse-hidden-files.test.js`). `npm run lint`
reports 146 against `main`'s 143, the three being the parse error on `backend/tests/**`
that 141 of `main`'s own test files already draw. Formatting clean. Frontend builds,
backend loads, documentation site builds.
One test assertion was rewritten rather than ported as it stood: it matched the wording
`express-openid-connect` uses for a callback with no sign-in in progress, and that
wording differs between 2.19 and 2.20. It now asserts the refusal.
One box, one name for it
------------------------
The sign-in box takes an email address or a username, so it is neither: it is
whatever was typed, and this calls it `identifier` from the screen to the route.
The screen and the client were renamed and the store and the route were not, so
the store passed `email` to a client expecting `identifier`. `JSON.stringify`
drops a key whose value is undefined, and the request went out carrying a
password and nobody to sign in. The answer was "invalid credentials", which is
what a wrong password looks like — so nothing about it read as a defect.
`attemptLocalLogin` already took `identifier`; the route is what had not caught
up. `email` and `username` still work, for a script or an older client that
sends them.
The identifier was the half that failed loudly. The screen also reads
`totpPending`, `oidcStatus`, `cancelTotp`, `ensureStatus` and `forgetSession`
off the store, and none of them were there: `totpPending` read undefined, so the
box for the code from the authenticator never appeared, and a correct password
on an account with a second factor landed on a screen that looked like it had
done nothing. Undefined is not an error in a template — it is a `v-if` that is
false. The store is here in full, with the code step read back from the server
on every start so a reload in the middle of one lands back on the code.
tests/routes/sign-in-identifier.test.js signs in with each of the three names,
refuses a wrong password, and reads the three frontend files to check they all
use the one name — the chain is four files long and three of them have no runner
here, which is how it broke silently in the middle.
And what a refusal says
-----------------------
A code that names the kind of refusal — FORBIDDEN, NOT_FOUND, CONFLICT,
RATE_LIMIT_EXCEEDED — is translated for the reader, and the server's own
sentence, which says *which* refusal, went underneath rather than being lost.
A lock that arrives with a duration gets a sentence of its own rather than a
placeholder in the plain one, so it can never read "{minutes}". And the
handler no longer asks vue-i18n for a key before checking it has it, which was
a console warning for every refusal the catalogue has no entry for, twice.
Two reports the server makes about itself at start, neither of which existed. **What releases up to 1.1.7 left in the cache.** The database and app-config.json lived in CACHE_DIR until 1.1.8, which moved them to CONFIG_DIR and left links behind; 2.0.3 removed that move from the entrypoint. So an installation that started on 1.1.7 or earlier and skipped the releases in between comes up on a new, empty app.db in CONFIG_DIR with its accounts, shares and settings sitting unread in the cache. Nothing said so: the server started, the sign-in page offered to create the first administrator, and the answer looked like a fresh installation rather than a lost one. It is a notice and nothing more — nothing is moved, nothing is deleted. It names what it found and where, and says what to do with it. **What the process is costing.** `services/performanceDiagnostics.js` samples CPU, resident memory as the cgroup sees it rather than as the host does, event-loop delay at p99, and the queues that can grow: thumbnails, folder sizes, transfers. Off unless PERFORMANCE_DIAGNOSTICS_ENABLED is set, and then it reports only the intervals that pass a threshold — a diagnostic that logs every interval by default is a diagnostic that fills a disk. PERFORMANCE_DIAGNOSTICS_LOG_EVERY_INTERVAL asks for all of them. An interval below its floor is held to the default: a sampler on a 1 ms interval costs more than whatever it was meant to diagnose, and 1 ms is what an emptied field sends. Each queue reports itself through an optional call. One that has no report is a queue this installation has nothing to say about, not a reason for the whole record to fail — a diagnostic that throws says nothing at the moment it is most wanted. ## Checks `legacy-cache-check.test.js`, 3 tests: a cache holding the old names is named, one holding the links 1.1.8 left is named differently, and an ordinary cache says nothing. Making the inspection always see an empty cache turns all three red. `performance-diagnostics.test.js`, 7 tests: silent unless asked for, says what it will watch and by which thresholds, holds an interval below its floor to the default, reports nothing of an ordinary interval, reports one that passes a threshold and says which kind it was, reports every interval when told to, and samples the machine rather than guessing. Making it start whether or not it was asked for turns the first red. Whole backend suite: 2 556 passed, 2 failed — the two that fail on `main` on its own.
cerede2000
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
P3-12 and P3-13 of phase 3 (see #373), together because they are one subject: what the
server says about itself at start. Stacked on #433–#443.
An installation that lost its database, and nothing said so
The database and
app-config.jsonlived inCACHE_DIRuntil 1.1.8, which moved them toCONFIG_DIRand left links behind; 2.0.3 removed that move from the entrypoint. So aninstallation that started on 1.1.7 or earlier and skipped the releases in between comes up
on a new, empty
app.dbinCONFIG_DIR, with its accounts, shares and settings sittingunread in the cache directory.
Nothing said so. The server started, the sign-in page offered to create the first
administrator, and it looked like a fresh installation rather than a lost one.
services/legacyCacheCheck.jsis a notice and nothing more — nothing is moved, nothing isdeleted. It names what it found and where, and says what to do with it.
What the process is costing
services/performanceDiagnostics.js: CPU, resident memory as the cgroup sees it ratherthan as the host does, event-loop delay at p99, and the queues that can grow — thumbnails,
folder sizes, transfers.
It exists for the case nobody can reproduce: an installation that goes slow after hours, on
storage nobody here has, under a load nobody here makes.
PERFORMANCE_DIAGNOSTICS_ENABLEDis set.every interval by default is a diagnostic that fills a disk;
PERFORMANCE_DIAGNOSTICS_LOG_EVERY_INTERVALasks for all of them.more than whatever it was meant to diagnose, and 1 ms is what an emptied field sends.
this installation has nothing to say about, not a reason for the record to fail — a
diagnostic that throws says nothing at the moment it is most wanted.
Checks
legacy-cache-check.test.js, 3 tests: a cache holding the old names is named, oneholding the links 1.1.8 left is named differently, an ordinary cache says nothing. Making
the inspection always see an empty cache turns all three red.
performance-diagnostics.test.js, 7 tests: silent unless asked for; says what it willwatch and by which thresholds; holds an interval below its floor to the default; reports
nothing of an ordinary interval; reports one that passes a threshold and says which kind
it was; reports every interval when told to; samples the machine rather than guessing.
Making it start whether or not it was asked for turns the first red.
mainalone.