[07/40] Ask where to put something, and offer the folders somebody actually uses - #448
Merged
Merged
Conversation
cerede2000
pushed a commit
to cerede2000/NextExplorer
that referenced
this pull request
Sep 26, 2026
The batch is reversed, so its findings move from PORT to DONE in the same commit that sends them. The dialog, the composable, the service, the table and the strings travelled together, because a dialog nothing can open and a service nothing can call are both green and useless. What is left to reverse is whatever `scripts/parity.mjs` still reports.
cerede2000
force-pushed
the
p3-17
branch
4 times, most recently
from
September 27, 2026 12:29
ffc74d5 to
8d0aa76
Compare
added 7 commits
September 27, 2026 19:24
Two routes built a command line with values from the request in it and gave the line to a shell. `routes/usage.js` pasted a folder's path into `du -sb "…"` and `df -Pk "…"`; `routes/permissions.js` pasted an owner, a group, a mode and a path into `chown`, `chgrp`, `chmod -R` and two `id` lookups. A name is not a shell string. Anything that closes the quoting leaves the rest for `/bin/sh` to run, as the user the server runs as. The usage one needs no privilege at all: any account that can create a folder can name one, and where AUTH_ENABLED is false that is anybody who can reach the server. `execFile` takes the arguments as a list, so there is no line for a shell to read and no shell. The commands, their output and the answers are unchanged — this is deliberately the smallest change that closes it, and not the rewrite that stands in nxzai#450 and nxzai#453. An argument list is not a free pass on its own: `chown` reads a leading dash as an option, so `--reference=/etc/shadow` would have copied another file's ownership onto the target. An account or group name has to look like one. `tests/routes/no-shell.test.js` — three cases. Two of them name a folder, and an owner, with a payload that creates a file, and check the file is not there; the third asks for `--reference=/etc/shadow` and expects a refusal. All three fail against the routes as they are, the first two by running the command. The payload only ever touches the working directory, and it is removed whatever an assertion does, so a run that does execute leaves nothing behind.
Two things about OIDC that this fixes.
**Which of two failures it was.** A sign-in that cannot start has one of two causes:
nothing was configured, or what was configured could not be made to work. The first
is answered by filling in OIDC_ISSUER and the rest; the second is answered by looking
at the provider. Both answered 404 "OIDC is not configured", so an administrator whose
provider was unreachable was sent to change a configuration that was already right.
`configureOidc` now records what it concluded and why, and the routes read it: 404
AUTH_OIDC_NOT_CONFIGURED for the first, 503 AUTH_OIDC_PROVIDER_UNAVAILABLE for the
second, and neither carries the network's own words — no ENOTFOUND, no internal host
name, in the body or in the address bar.
A browser asking for one of those addresses is looking at a page, not reading JSON, so
it is sent back to the sign-in screen with the code beside the sentence. The screen can
then say what the code means in the reader's own language.
**Where the callback comes back to.** The return address was built from a fixed
`baseURL`, so a deployment reached through a reverse proxy on a different name sent
people back to the wrong origin. It is now resolved from the request — but only from a
forwarded host behind a trusted proxy, and only when the result exactly matches an
origin the operator configured. A redirect target that a request header could choose
is an open redirect with extra steps.
The path is still held to a relative, same-site one, as before.
Also here, because it is the same files:
- `claimsFromIdToken` in the middleware and `uniqueOrigins` in the routes were local
copies of things `utils/idToken.js` and the new `utils/oidcRedirect.js` do; both go.
- The guest-session cookie was cleared on `/api` only, at five sign-in and sign-out
paths. A guest session left on `/` outlived the sign-in that should have ended it.
- `ServiceUnavailableError` (503) and the two AUTH_OIDC_* codes, which nothing had yet.
The account lockout that `routes/auth.js` also differs in is deliberately not here: it
is a different subject and goes with releasing a locked account.
## Checks
Seven test files, 107 tests, including three this fork had and `main` did not:
`oidc-middleware`, `oidcOrigin` and `auth-oidc-routes`. Neutralising
`getOidcAvailability` so it reports one verdict for both causes turns three of them
red — the three that tell the two apart.
Whole backend suite: 2 391 passed, 2 failed, the two that fail on `main` on its own
(`auth.test.js` on the current password, `browse-hidden-files.test.js`). `npm run lint`
reports 146 against `main`'s 143, the three being the parse error on `backend/tests/**`
that 141 of `main`'s own test files already draw. Formatting clean. Frontend builds,
backend loads, documentation site builds.
One test assertion was rewritten rather than ported as it stood: it matched the wording
`express-openid-connect` uses for a callback with no sign-in in progress, and that
wording differs between 2.19 and 2.20. It now asserts the refusal.
One box, one name for it
------------------------
The sign-in box takes an email address or a username, so it is neither: it is
whatever was typed, and this calls it `identifier` from the screen to the route.
The screen and the client were renamed and the store and the route were not, so
the store passed `email` to a client expecting `identifier`. `JSON.stringify`
drops a key whose value is undefined, and the request went out carrying a
password and nobody to sign in. The answer was "invalid credentials", which is
what a wrong password looks like — so nothing about it read as a defect.
`attemptLocalLogin` already took `identifier`; the route is what had not caught
up. `email` and `username` still work, for a script or an older client that
sends them.
The identifier was the half that failed loudly. The screen also reads
`totpPending`, `oidcStatus`, `cancelTotp`, `ensureStatus` and `forgetSession`
off the store, and none of them were there: `totpPending` read undefined, so the
box for the code from the authenticator never appeared, and a correct password
on an account with a second factor landed on a screen that looked like it had
done nothing. Undefined is not an error in a template — it is a `v-if` that is
false. The store is here in full, with the code step read back from the server
on every start so a reload in the middle of one lands back on the code.
tests/routes/sign-in-identifier.test.js signs in with each of the three names,
refuses a wrong password, and reads the three frontend files to check they all
use the one name — the chain is four files long and three of them have no runner
here, which is how it broke silently in the middle.
And what a refusal says
-----------------------
A code that names the kind of refusal — FORBIDDEN, NOT_FOUND, CONFLICT,
RATE_LIMIT_EXCEEDED — is translated for the reader, and the server's own
sentence, which says *which* refusal, went underneath rather than being lost.
A lock that arrives with a duration gets a sentence of its own rather than a
placeholder in the plain one, so it can never read "{minutes}". And the
handler no longer asks vue-i18n for a key before checking it has it, which was
a console warning for every refusal the catalogue has no entry for, twice.
Two reports the server makes about itself at start, neither of which existed. **What releases up to 1.1.7 left in the cache.** The database and app-config.json lived in CACHE_DIR until 1.1.8, which moved them to CONFIG_DIR and left links behind; 2.0.3 removed that move from the entrypoint. So an installation that started on 1.1.7 or earlier and skipped the releases in between comes up on a new, empty app.db in CONFIG_DIR with its accounts, shares and settings sitting unread in the cache. Nothing said so: the server started, the sign-in page offered to create the first administrator, and the answer looked like a fresh installation rather than a lost one. It is a notice and nothing more — nothing is moved, nothing is deleted. It names what it found and where, and says what to do with it. **What the process is costing.** `services/performanceDiagnostics.js` samples CPU, resident memory as the cgroup sees it rather than as the host does, event-loop delay at p99, and the queues that can grow: thumbnails, folder sizes, transfers. Off unless PERFORMANCE_DIAGNOSTICS_ENABLED is set, and then it reports only the intervals that pass a threshold — a diagnostic that logs every interval by default is a diagnostic that fills a disk. PERFORMANCE_DIAGNOSTICS_LOG_EVERY_INTERVAL asks for all of them. An interval below its floor is held to the default: a sampler on a 1 ms interval costs more than whatever it was meant to diagnose, and 1 ms is what an emptied field sends. Each queue reports itself through an optional call. One that has no report is a queue this installation has nothing to say about, not a reason for the whole record to fail — a diagnostic that throws says nothing at the moment it is most wanted. ## Checks `legacy-cache-check.test.js`, 3 tests: a cache holding the old names is named, one holding the links 1.1.8 left is named differently, and an ordinary cache says nothing. Making the inspection always see an empty cache turns all three red. `performance-diagnostics.test.js`, 7 tests: silent unless asked for, says what it will watch and by which thresholds, holds an interval below its floor to the default, reports nothing of an ordinary interval, reports one that passes a threshold and says which kind it was, reports every interval when told to, and samples the machine rather than guessing. Making it start whether or not it was asked for turns the first red. Whole backend suite: 2 556 passed, 2 failed — the two that fail on `main` on its own.
`/api/features` already offers `preview.maxRenderBytes` to the screen, and the configuration had no such value, so it answered `null` and the preview rendered whatever it was given. A document large enough to render slowly renders slowly for everybody on the machine, and nobody could raise or lower the point at which it stops trying. PREVIEW_MAX_RENDER_SIZE sets it; 16 MB when it is not set, and a value below zero or unparseable is that default rather than a ceiling of nothing. And the listing: `GET /api/browse` answers with what is true at that moment — which documents somebody has open in an editor, what a folder weighs, whether a write would be refused. A GET with no cache header is cacheable by default, so a proxy or a browser was free to keep it and serve a folder as it was: a deleted file still listed, a document shown as open by somebody who closed it an hour ago. `private, no-store` says what it is. ## Checks `browse-caching.test.js` asserts both halves of the header, and taking the header away turns it red. The ceiling is read through `/api/features`, which has offered the field since the batch that brought the features route and was answering `null` for it. Whole backend suite: 2 556 passed, 2 failed — the two that fail on `main` on its own.
`getDb` checks whether a connection is already open and opens one when it is not. Every
caller that arrives before the first has finished sees "not open" and opens another —
and everything that starts with the server asks at once: the session store, the settings,
the trash sweep, the search index, the favourites. So a start opened app.db four times,
ran `migrate` over the same file four times in parallel, and whichever finished last
became the one everybody used.
One opening is shared now. `openDb` does the work, `getDb` hands every caller the same
promise while it is in flight and the same connection afterwards, and `closeDb` lets a
test take it away again — which is what a suite opening a database per case needs.
And `prepared(db, sql)`: `db.prepare` compiles the SQL every time it is called, and the
hot paths called it per row — a listing asking whether each of a thousand entries is a
favourite compiled the same statement a thousand times. Kept in a WeakMap keyed on the
connection, so the cache goes when the connection does and a statement is never handed to
a connection that did not compile it.
## Checks
`db-single-open.test.js`, 5 tests. Five callers in one turn get one connection — counted
on the connection objects rather than on anything the code says about itself. A later
caller gets the one already open. After `closeDb` the next caller gets a new, working one.
The same SQL twice compiles once, and the same SQL against a new connection compiles again.
Putting the per-caller opening back turns the first red.
Whole backend suite: 2 561 passed, 2 failed — the two that fail on `main` on its own.
Two files were in this batch and are not: `betterSqliteSessionStore.js` and
`bootstrap.js`. `main`'s versions are the newer ones — it opens the session database on
first use rather than when the module is required, which is what keeps
`require('./backend/src/app.js')` working where the cache directory does not exist yet.
Bringing the fork's versions over them would have broken that check.
Each account's per-folder choices were two JSON values under `user_settings`:
one map of sorts, one of views, read and rewritten whole on every change. Three
things followed, and all three are gone here.
Two tabs on different folders overwrote each other. Each save sent the whole
map, so whichever tab saved last won and the other folder's choice was lost.
A preference is now written one folder at a time, through
`PATCH /api/settings` with `{ folderSort: { path, sort } }`, and the row's
UPSERT keeps the half it was not given: setting a folder's view no longer
erases its sort.
The map had a ceiling of a hundred folders, because it shipped entire on every
load and was rewritten entire on every change; the hundred-and-first folder
silently forgot the oldest. Rows have no ceiling.
And nothing could ever clean it up. A folder deleted or renamed left its
preferences behind on every account that had ever opened it, on a path that no
longer existed. As rows they can be removed with the folder they describe.
Schema 20 creates `folder_preferences` and carries the old values over: one row
per account and folder, merged under the later of the two times, and the values
it read are removed so nothing can read them back. A value it cannot parse
costs that folder its remembered sort rather than failing the startup.
Also here, because they are the same screen and the same service:
- A default view for folders that have none of their own. `null` means the
built-in one; a mode there is no such thing as is refused rather than read as
null, which used to put every folder back to the built-in view.
- Markdown opens in the editor, for whoever mostly writes it, instead of going
through the preview and clicking Edit every time. Only markdown: it is the one
kind of file that has both, so it is the only one where opening it is a choice.
- An access rule that cannot be stored is refused with its reason instead of
being dropped from the answer. Saving `../Secret` used to answer 200 with a
list the page then adopted, and an administrator was left believing a folder
was hidden that never was. Worse, a permission that was not one of the three
became `rw`, so a mistyped `readonly` opened a folder for writing.
- Every section of a save is checked before any of it is written. A payload
carrying a valid section and a refused one used to store the first and answer
400 — a request reported as refused that had changed something.
- One list of what a preference is. It was three: the keys the route allowed,
the chain of if/else that sanitised them, and the defaults in the client store.
A key in one and missing from another was accepted, silently dropped, and
answered with its previous value, which the client applied — so the switch
flicked itself back off. `markdownOpensInEditor` did exactly that.
`PATCH /api/settings` now answers with the settings read back from storage
rather than an echo of what was sent, so what the caller applies to its own
state is what a later request would read. `tests/routes/settings-preferences.js`
was asserting the echo, and now asserts the value of each preference it saved.
`USER_SETTING_KEYS` is `WRITABLE_USER_SETTINGS`, which says what it holds.
tests/services/folder-preferences-as-rows.test.js covers the eight claims
above. Each was put back to the old behaviour to check the right one goes red:
overwriting instead of keeping the other half, deleting the account's rows on
every write, reading without the owner, a LIMIT of 100, accepting any word as a
view mode, and dropping the carry-over.
Whole suite: 2685 passed, 2 failed — the two that already fail on main
(auth.test.js on the wrong current password, browse-hidden-files.test.js on a
FOREIGN KEY). Built, started, and driven in a browser: a folder keeps the view
it was left in while its neighbour keeps the default, and markdown goes to the
editor only when the preference says so.
Moving or copying meant typing a path, or navigating to the destination first and coming back. This is the dialog the rest of the application will use for that — mounted once with the layout, opened from anywhere, and answering with a folder or with nothing. It opens on a list rather than at the root, because the folder somebody wants is nearly always one they have used before. Nobody curates that list: it is written by the transfers themselves, so it stays true to how the person really files things, and every route into a folder counts — the picker, a drag onto a favorite, a paste. Kept per account: a shared favourite is a deliberate bookmark, this is a trace of one person's habits, and showing someone else's would be both wrong and a small leak of where they work. The list is filtered against what the account can reach right now. A folder can be deleted, or have its access revoked, long after it was last used, and offering it as a destination would only produce a failure at the end of the flow. Anything gone is forgotten on the way out, so the list heals itself. Schema 21 adds `recent_destinations`, one row per account and path, trimmed to ten on write so it cannot grow for somebody who never opens the picker. A destination that fails to be remembered never fails the transfer that reached it: remembering is a convenience, and a convenience must not break the thing it is convenient for. `ModalDialog` gains a footer. Without it the dialog rendered — title, recents, the folder tree, all correct — and had no confirm button anywhere in the page: what it asked for sat inside the scrolling part, and there was no way to answer except Escape. Driving the dialog in a browser is the only thing that would have caught it, and it is why the footer is here rather than in the batch that rewrites this component. The archive preview is the first screen to use it: "Extract to…" beside "Extract here", so an archive can be unpacked somewhere other than the folder it sits in. Closing the dialog without choosing means do nothing — not the root, and not the default. tests/routes/recent-destinations.test.js covers the six claims: that a transfer is remembered without being asked to, that the most recent comes first, that using a destination again moves it up rather than listing it twice, that one account's habits stay out of another's list, that a destination since deleted is dropped, and that a write it cannot perform is swallowed rather than failing the transfer. Each was put back to the old behaviour to check the right one goes red: no recording at all, no filtering, the error rethrown, and the read without its owner. The route is described in the OpenAPI document and walked by the tour, because upstream holds every mounted route to both. Whole suite: 2691 passed, 2 failed — the two that already fail on main. Built, started, and driven in a browser end to end: two moves, the dialog opened from an archive, the folder chosen from the recents, and the entries landing there rather than beside the archive.
cerede2000
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #433–#447. The last commit is the one to read.
What changes
Moving or copying something meant typing a path, or navigating to the destination first and coming back. This adds the dialog the rest of the application will use for that: mounted once with the layout, opened from anywhere, answering with a folder or with nothing.
It opens on a list rather than at the root, because the folder somebody wants is nearly always one they have used before. Nobody curates that list — it is written by the transfers themselves, so it stays true to how the person really files things, and every route into a folder counts: the picker, a drag onto a favorite, a paste.
Kept per account. A shared favourite is a deliberate bookmark; this is a trace of one person's habits, and showing someone else's would be both wrong and a small leak of where they work.
The list is filtered against what the account can reach right now. A folder can be deleted, or have its access revoked, long after it was last used, and offering it as a destination would only produce a failure at the end of the flow. Anything gone is forgotten on the way out, so the list heals itself.
Schema 21 adds
recent_destinations— one row per account and path, trimmed to ten on write so it cannot grow for somebody who never opens the picker. A destination that fails to be remembered never fails the transfer that reached it.One defect this found
ModalDialoghad no footer slot. The dialog rendered — title, recents, folder tree, all correct — and had no confirm button anywhere in the page: what it asked for sat inside the scrolling part, and the only way out was Escape. Nothing in the build, the types or the suite says so; it took driving the dialog in a browser. The slot is here rather than in the batch that rewrites that component, because the dialog cannot be used without it.The first screen to use it
The archive preview: Extract to… beside Extract here, so an archive can be unpacked somewhere other than the folder it sits in. Closing the dialog without choosing means do nothing — not the root, and not the default.
Proof
tests/routes/recent-destinations.test.js— six tests: a transfer is remembered without being asked to; the most recent comes first; using a destination again moves it up rather than listing it twice; one account's habits stay out of another's list; a destination since deleted is dropped; a write that cannot be performed is swallowed rather than failing the transfer. Each was put back to the old behaviour to check the right one goes red — no recording, no filtering, the error rethrown, the read without its owner.The route is described in the OpenAPI document and walked by the tour, because this repository holds every mounted route to both.
Whole suite: 2691 passed, 2 failed — the two that already fail on
main.npm run buildclean; eslint and prettier at theirmainbaselines.Driven end to end in a browser: two moves, the dialog opened from an archive, the folder chosen from the recents list, and the entries landing there rather than beside the archive — the request carrying
"destination":"Archive", which before the footer slot it never could.