Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
d4f7f1e
cli: clickmonk version, and one version for every package
Sep 26, 2026
284db1d
stack: a disk clickhouse can write its own backups to
Sep 26, 2026
174425e
cli: version answers before either environment is parsed
Sep 26, 2026
9bda127
stack: anchor the backups disk assertion past comments
Sep 26, 2026
0810f08
ops: backup.sh, with the worker stopped only while clickhouse is copied
Sep 26, 2026
1a58560
test: pin the backup's missing-disk and same-second refusals
Sep 26, 2026
a868979
ops: backup.sh reads the env file before asking compose anything
Sep 26, 2026
a375d07
ops: backup.sh takes a lock, sweeps old archives and masks signals in…
Sep 26, 2026
bbf709f
test: pin the backup's lock, signal mask, pipefail, closed stdin and …
Sep 26, 2026
ae64d79
test: clear a leftover backup lock before and after the suite
Sep 26, 2026
3bf99b4
test: pin that the backup lock is held while the worker is started again
Sep 26, 2026
b8a2cc5
ops: backup.sh waits for a backup or restore clickhouse is still running
Sep 26, 2026
598fe3f
test: pin the sweep under the lock, the release after a failed restar…
Sep 26, 2026
25af8ea
test: let the lock tests clean up after a failed assertion
Sep 26, 2026
890accb
ops: restore.sh, which checks everything before it stops anything
Sep 26, 2026
386da59
ops: backup.sh refuses while a restore is unfinished, and restore.sh …
Sep 26, 2026
a90acf7
test: pin an interrupted restore, the full disk and the clickmonk-fre…
Sep 26, 2026
575845c
ops: restore.sh refuses when any one of its size readings is missing
Sep 26, 2026
6548eba
test: pin the refusal when clickhouse cannot say how much its databas…
Sep 26, 2026
955f990
ops: backup.sh refuses when either of its size readings is missing
Sep 26, 2026
f339372
test: pin backup.sh's refusal when clickhouse cannot say how much its…
Sep 26, 2026
8a150b9
ops: restore.sh writes its marker beside the old one and renames it
Sep 26, 2026
e1bb2ca
test: pin the earlier marker, the marker after a failed verification …
Sep 26, 2026
e613162
stack: a backup restored into an empty stack counts every click once
Sep 26, 2026
eb0b8f0
stack: the round trip requires the restored ledger, not the empty sta…
Sep 26, 2026
ebb4012
stack: pin the spool copy before the worker restarts, and a restore t…
Sep 26, 2026
46db9a0
docs: backup, restore and upgrading
Sep 26, 2026
6c2d10e
install: point at the web interface and backup.sh, and name the compo…
Sep 26, 2026
87a55a6
docs: how fast a redirect is, what an install exposes, and the rest m…
Sep 26, 2026
ff17177
docs: the pre-0.1.0 upgrade by checkout age, and each claim made exact
Sep 26, 2026
bafdded
ops: back up and restore an install from before the rollups, and retr…
Sep 26, 2026
20f321e
test: pin a backup from before the rollups and a restart that fails once
Sep 26, 2026
0812f18
docs: re-wrap the lines the last pass left long
Sep 26, 2026
f8e99f4
test: pin the backup without the raw clicks table and a start that st…
Sep 26, 2026
8624b58
ops: finish a stop before starting, wait for a running step, and key …
Sep 26, 2026
a9db070
test: pin stops still draining, waited steps, two installs and an int…
Sep 26, 2026
3f83808
docs: the lock and marker per install, a step waited for, and advice …
Sep 26, 2026
3b72561
test: pin the project name from the env file and the wait for pg_restore
Sep 26, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -51,3 +51,9 @@ test/stack/tmp/
# dist/ (already ignored above); this is a separate directory so the two
# builds never delete each other's output, and nothing reads it.
packages/ui/dist-ts/

# The lock backup.sh and restore.sh hold while they run.
.backup-restore-*.lock/
# The mark restore.sh leaves while the stores may be half restored, one per
# install, and the file it is written to first.
.restore-incomplete-*
27 changes: 21 additions & 6 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,9 @@ built on the admin API and nothing it can reach that a script cannot), retention
periods on the settings row, enforced hourly by the worker, dropping raw clicks a partition
at a time and blanking the address on a click in place), the CLI (`migrate`,
`domain add|list|verify`, `link add`, `settings show|set`, `ipdata status|update`,
`admin create|passwd`, `admin totp disable`, `apikey create|list|revoke`), `install.sh`, a
Compose stack, and the restart durability and stack test suites.
`admin create|passwd`, `admin totp disable`, `apikey create|list|revoke`, `version`),
`install.sh`, `backup.sh` and `restore.sh`, a Compose stack, and the restart durability and
stack test suites.

**A retention period is a floor, not a deadline, and it is never defaulted by a reader
that deletes.** `clicks` is partitioned by month and dropped whole, so 90 days keeps 90 to
Expand Down Expand Up @@ -106,7 +107,9 @@ What does not exist yet, and must not be implied by any documentation:
recognised by ASN only, since no cloud provider's range file states a licence. The
region and city columns exist on a click and are always empty, and a breakdown by
either would be a new materialized view and a backfill of it.
- **Backup and restore.**
- **Backups that encrypt, rotate or leave the server**, and **a restore of one part of a
backup**. `backup.sh` writes one directory; `restore.sh` replaces everything and is an
outage while it runs.
- Segments ClickHouse rejects are set aside as `.bad` files, and nothing reports them.
- One redirect process per spool directory.
- A full spool stops recording without stopping redirects; the drop count is on
Expand Down Expand Up @@ -165,7 +168,8 @@ packages/ui/ the web interface: React 19, Tailwind 4, a Vite build the ad
Radix overlay primitive, no runtime import from a service package,
one <h1> per screen. Every time shown is the browser's own zone, except a
click's raw fields, which are labelled UTC.
packages/cli/ `clickmonk migrate | domain | link add | settings | ipdata | admin | apikey`.
packages/cli/ `clickmonk migrate | domain | link add | settings | ipdata | admin | apikey |
version`.
Two commands here are deliberately not API routes: `admin create`,
because nothing can authenticate before the account exists, and
`admin totp disable`, because it is the way back in when the second
Expand All @@ -176,6 +180,13 @@ caddy/ the Caddyfile, and the tls.d/ and proxy.d/ drop-in directori
admin service and every other name to the redirect, with a CEL
expression rather than a host matcher so an unset value starts.
install.sh writes .env once, with fresh secrets, and starts the stack.
backup.sh copies Postgres, ClickHouse, the spool, caddy-data and .env into one
directory, stopping only the worker, and only while the spool and
ClickHouse are copied.
restore.sh checks a backup, then replaces all four stores with it; the only
script here that deletes data. Both source backup-lib.sh, which
holds the lock they share: one run at a time per install.
clickhouse/ backup-disk.xml, the disk ClickHouse's own BACKUP writes to.
test/stack/ brings the whole stack up against a local certificate authority
and DNS server; proves TLS issuance, domain verification, real
client addresses over IPv4 and IPv6, and the admin host — its
Expand Down Expand Up @@ -267,7 +278,9 @@ up, sign in to it over HTTPS on the admin host, and drive a password-protected l
the form to the redirect. One of them clicks a link through Caddy and then reads that
click back through the admin host as a summary, a chart, a breakdown, a log page and a
CSV, and lowers a retention period and watches the worker enforce it — the one path where
the admin service's own ClickHouse configuration has to be right. They bind **80 and 443**; the
the admin service's own ClickHouse configuration has to be right. One of them takes backups
of a running stack, refuses damaged and newer ones without touching anything, then destroys
every volume and restores into an empty stack. They bind **80 and 443**; the
durability suite binds 8080 and 8123 and the test databases 8123 and 5433. Run one at a
time.

Expand Down Expand Up @@ -329,7 +342,9 @@ That rule is easy to satisfy badly, so:

Anything that runs on an operator's host rather than in a container — an installer,
backup or restore — targets **bash 3.2**, the version macOS still ships: no associative
arrays, no `mapfile`. `install.sh` is the first. `shellcheck` is pinned as a Docker image
arrays, no `mapfile`. `install.sh` is the first. `backup.sh`, `backup-lib.sh` and
`restore.sh` follow the same rules, and the backup suite parses all three under a real
bash 3.2. `shellcheck` is pinned as a Docker image
and run by `pnpm lint:sh` over every `*.sh` in the repository, so a new script is linted
without anything being added to the script, and by CI with the same command, so a failure
there is reproducible here.
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Thanks for your interest. ClickMonk is early: the redirect and the click pipelin

## Development setup

You need Node 22, Docker with Compose v2, and `corepack` (it ships with Node). Then:
You need Node 22, Docker with Compose 2.23 or later, and `corepack` (it ships with Node). Then:

corepack enable
pnpm install
Expand Down
Loading
Loading