A self-chaining GitHub Actions pipeline that mirrors a Dropbox account into Proton Drive. After each run Proton Drive holds what Dropbox held at listing time, under one folder of your choosing. Changed files become new Proton revisions, files that left Dropbox move to Proton's trash, and nothing is recorded as mirrored until Proton's own upload summary has accounted for it. The only durable state is one SQLite database, age-encrypted in a Cloudflare R2 bucket; no run is ever told where to start.
Dropbox stays the primary. Nothing written in Proton Drive flows back.
Dropbox (primary, read-only credentials)
-> GitHub Actions run (this repo, inside one pinned toolbox image)
inventory -> delta -> plan -> per batch: fetch -> verify -> upload -> confirm -> checkpoint
-> trash -> reconcile (weekly) -> report -> ping -> chain if batches remain
-> Proton Drive <destination folder> (Proton's version history is the cloud-side history)
-> R2 .state/ (encrypted state database, dated history, CLI session)
The repository is public and holds no account: every credential and every account identifier (the Dropbox account id, the Proton destination folder and its UID) lives in one 1Password vault and reaches a run by name.
The infrastructure modules under src/migrator/ (SQLite evidence schema, one-pass hasher,
atomic writes, redacting logger, path guards, and the two providers for the Dropbox API
and the official proton-drive CLI) come from
donphi/dropbox_proton at commit cfd0e57,
MIT, whose copyright notice is retained in LICENSE. The mirror phases and the
Taskfile are this repo's own; the toolbox that runs them, its image and the two workflows
this repository calls are katoptra/lib's, at v2.
One run is task pipeline, executed inside the toolbox image. Each step is one
python -m migrator <command>; the Taskfile owns sequencing, the Python owns every
decision, and the toolbox from katoptra/lib owns the
run around them: how it starts, is contained, resolves its secrets, is checked and is
reported. Steps before inventory run without a state database and are called
commands; the rest are phases and record their evidence in the state.
| Step | What it does |
|---|---|
clock |
The toolbox's stamp to .run/start.txt, then the migrator's own (clock-phase): the run start epoch, UTC hour and weekday to .run/clock.json; clears staging/, the report and the chain marker. |
session |
Fetches .state/session.tar.age from R2, decrypts it with the age identity, unpacks it to .run/session/, which is PROTON_DRIVE_CACHE_DIR for every later Proton call. |
state |
Fetches .state/state.sqlite.xz.age and starts the run row. A missing state is accepted as an empty mirror only when .state/history/ is empty too and the bucket answers a probe; a missing state beside history is refused, because a lost state must never look like an empty mirror. |
inventory |
Recursive Dropbox files/list_folder walk, each page committed with its cursor. Entries with no content_hash (Paper docs, cloud files) are recorded as non-downloadable and excluded. Rebuilds every entry's display path from its ancestor folders' own names, because Dropbox cases the parent segments of path_display inconsistently between entries of one folder; that rebuilt path is what staging and Proton see, and the lowercased path stays the key. Keeps only the newest listing and drops each entry's raw API JSON once the walk is complete, since nothing reads either and every checkpoint ships the state to R2. |
delta |
Compares the inventory against mirror_objects on (path_lower, size, content_hash); this run's delta replaces every earlier run's. Refuses a listing under half the mirrored file count so a truncated listing can never become a trash list. |
plan |
Refuses a tree over ceiling_gb or a batch the runner's disk cannot hold in staging. Leaves out any file over max_file_gb, or over what the disk can stage beside its headroom, and counts it as oversized in the report; a file that would fail its batch must not stop the chain. Packs the rest into batches of at most batch_gb and batch_files, a file over batch_gb being a batch by itself. Earlier PLANNED batches are dropped; this run's plan is the only plan. |
batches |
Touches the Proton session once with a listing of the destination's parent, then runs each batch through the five steps below. Before each batch it stops when the elapsed time plus the longest batch so far would pass the budget; stopping with batches left is a success that marks the run for chaining. |
fetch |
Empties staging and downloads the batch from Dropbox over the API by listed path, download_workers files in flight, each file landing under its display path so Proton receives Dropbox's casing. A path that vanished since listing is counted and skipped. |
verify |
Recomputes every staged file's Dropbox content hash and records SHA-1 and SHA-256. A mismatch is a file edited between listing and fetch: removed, counted, never recorded. A batch where every file mismatches fails. |
upload |
One proton-drive filesystem upload of the staging tree with -f create-new-revision -d merge --skip-thumbnails; Proton skips files whose content it already holds. |
confirm |
Reads the upload summary: transferred, skipped and failed items must account for every verified file plus every folder, and every failure must name a file in the batch. Those files alone are recorded as failed, with the CLI's error text, and the rest confirm; a failure that names nothing in the batch, or counts that do not add up, leave the whole batch unrecorded. Proton's server verifies every ciphertext block hash at upload; the weekly reconcile walk is the independent observation of what Proton holds. |
checkpoint |
Merges the confirmed rows into mirror_objects and pushes the state to R2 under .state/history/<epoch>-<batch> and then, server-side, to the canonical key. A batch with failed files still checkpoints what confirmed; those files are absent from mirror_objects, so the next run's delta lists them again. Only a batch that recorded nothing fails the run. Always the last step of a batch, so a killed run repeats at most one batch. |
trash |
Only when every planned batch landed: groups deleted rows by parent folder, one listing and one filesystem trash per folder. A folder that cannot be listed is recorded and retried the next run. |
reconcile |
On the first run of the configured weekday, or with RECONCILE=true: a full Proton walk, proton.walk_workers CLI listings in flight, every folder below the root addressed by UID so the CLI resolves it in one lookup, children queued to the worker that listed their parent so its decrypted keys are already cached. Each worker runs from its own copy of the CLI session, because a rejected token refresh signs a copy out; the copy a refresh rewrote is adopted and the others re-seeded. The walk compares Proton's own listed size and SHA-1 against mirror_objects: rows Proton lacks, mis-sizes or mismatches are dropped so they re-upload; Proton nodes under the destination that neither Dropbox nor the state knows are trashed, and so are folders Dropbox no longer has once nothing on record is left under them (topmost only; a folder's trash takes its subtree). A walk that does not fit one run's budget resumes where it stopped on the next reconcile run, and a partial walk drops and trashes nothing. |
report |
Builds the step summary from the state alone, finishes the run row, writes the chain marker, pushes the state, and returns the run status so a failed run stops before the success ping. |
ping |
The toolbox's: pings HEALTHCHECK_URL; a failed run pings /fail instead, from the toolbox's failure path. |
Between the migrator's report and the toolbox's ping runs the toolbox's own report, which puts its
rows on the job page (when the run started and how long it took, the image, whether the
next run is queued) and the migrator's report under them.
Every step is plan-by-default. batches, trash, reconcile, report and empty-trash
change anything only with --apply, which the Taskfile passes in task pipeline and never in
task plan-pipeline. No mutation is trusted on its exit status: the state push is trusted
only once the object lands, and nothing is recorded as uploaded on a command's exit code
alone. The evidence has three layers: the upload summary per batch (transferred, skipped
and failed items matched against what was verified, each failure named); Proton's own
server-side block hashes, checked at upload time and out of this repo's hands; and the
weekly reconcile walk, which compares Proton's own listing, its size and SHA-1 for every
node under the destination, against mirror_objects, independent of anything a batch
claimed.
- A container engine, running: Apple
containeron macOS, or Docker. The Taskfile picks Applecontainerwhen its daemon is up, else Docker; override withENGINE=docker. Every command in this repo, tests included, runs inside the toolbox image,ghcr.io/katoptra/toolbox:proton-v2from katoptra/lib (Python,proton-drive,age, go-task, every one checksum-pinned by that repository's lock). Nothing else is installed on the host. - go-task:
brew install go-task. - The 1Password CLI
op, signed in and unlocked, for anything that needs the vault on the laptop. Nothing from the vault touches disk:task op -- <cmd>wraps a command inop run --env-file=op.env, which resolves theop://references in op.env at run time and masks their values in output. - Accounts: Dropbox, Proton Drive, a Cloudflare R2 bucket, a healthchecks.io check, and a 1Password vault dedicated to this repo.
Every value that names an account is stored in exactly one place, the 1Password vault,
and referenced in exactly one: op.env, twelve op://<vault>/<item>/<field>
lines. The laptop and CI both run op run --env-file=op.env, so there is no second list
to keep in step. The twelve references resolve five vault items:
| Item | Fields | Reaches a run as |
|---|---|---|
dropbox |
app_key, app_secret, refresh_token, account_id |
MIRROR_DROPBOX_APP_KEY, MIRROR_DROPBOX_APP_SECRET, MIRROR_DROPBOX_REFRESH_TOKEN, MIRROR_DROPBOX_ACCOUNT_ID |
proton |
destination, destination_uid |
MIRROR_PROTON_DESTINATION, MIRROR_PROTON_DESTINATION_UID |
r2 |
access_key_id, secret_access_key, endpoint, bucket |
AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_ENDPOINT_URL_S3, MIRROR_R2_BUCKET |
age |
identity |
MIRROR_AGE_IDENTITY |
healthcheck |
url |
HEALTHCHECK_URL |
Create a vault for this repo and a service account scoped to that vault alone. Store the
service-account token in your personal vault (never in the vault it reads) and as the one
GitHub repository secret, OP_SERVICE_ACCOUNT_TOKEN. Put the vault into the references
in op.env, by name if the name has no slash (op://katoptra-dropbox/...), otherwise by
UUID from op vault get <name> --format json: a secret reference has exactly three
segments, so a slash in a vault name cannot be written. Neither the name nor the UUID is a
secret: without the service-account token it opens nothing.
The pipeline reads Dropbox over the API alone, both for listing and for downloading.
-
At https://www.dropbox.com/developers/apps create a Scoped access, Full Dropbox app. Under Permissions enable only
files.metadata.readandfiles.content.read. The mirror can never write to Dropbox. Note the App key and App secret. -
Get a refresh token. Open, with your key filled in,
https://www.dropbox.com/oauth2/authorize?client_id=APP_KEY&response_type=code&token_access_type=offline.token_access_type=offlineis what makes the token permanent. Approve, copy the code, and exchange it:curl https://api.dropboxapi.com/oauth2/token \ -d code=THE_CODE -d grant_type=authorization_code -u APP_KEY:APP_SECRET
The response holds both an
access_token(four hours) and arefresh_token(never expires, never rotates). Storeapp_key,app_secretandrefresh_tokenas fields of the vault itemdropbox. -
Record the account id. Mint an access token from the refresh token (this is what every run does) and ask who you are:
curl https://api.dropboxapi.com/oauth2/token \ -d grant_type=refresh_token -d refresh_token=REFRESH_TOKEN -u APP_KEY:APP_SECRET curl -X POST https://api.dropboxapi.com/2/users/get_current_account \ -H "Authorization: Bearer ACCESS_TOKEN"Store the
account_id(the wholedbid:...string) as fieldaccount_idof the same item. Every run verifies the account it is reading against this value and refuses a mismatch, so a swapped credential can never turn another account's listing into a trash list.
Proton's CLI can only be seeded by a browser sign-in, and fresh sign-ins from datacenter addresses are blocked, so the session is created once on the laptop and carried to CI as an encrypted file. Every run restores it before the first CLI call and writes it back after every CLI call, because its refresh token rotates.
-
Turn telemetry off in Proton account settings.
-
Install the macOS
proton-driveCLI from https://proton.me/download/drive/cli at the version katoptra/lib'stoolchain.lock.tomlpins for the image. The session file format is tied to the version; the Linux binary in the toolbox must be able to read what the laptop wrote. -
Sign in, with the session written as plain files under
.run/pd(the directory must be inside this repo, since only the repo is mounted into the toolbox;.run/is ignored by git and deleted bytask clean):PROTON_DRIVE_CACHE_DIR=.run/pd PROTON_DRIVE_CREDENTIALS_STORE=unsafe_file proton-drive auth login
Afterwards
.run/pdholdsauth-session.jsonandclientUid.json. -
Create the destination folder. In the CLI,
/my-filesis the existing "My files" root of your drive, not a folder you create. Create one folder under it:PROTON_DRIVE_CACHE_DIR=.run/pd PROTON_DRIVE_CREDENTIALS_STORE=unsafe_file \ proton-drive filesystem create-folder /my-files Dropbox PROTON_DRIVE_CACHE_DIR=.run/pd PROTON_DRIVE_CREDENTIALS_STORE=unsafe_file \ proton-drive filesystem list -j /my-files
Store the folder's CLI path (
/my-files/Dropbox) as fielddestinationof vault itemproton. In the listing find the entry whosename.valueisDropboxand store itsuidas fielddestination_uid. Every run lists the parent, finds the folder by name, and compares its UID with this value before touching it. A listing that shows noDropboxentry, or one whose UID differs, fails the run withconfigured Proton destination did not resolve to exactly one folderordid not exactly match the listing; the usual cause is the folder created one level too deep or the wrong entry's UID copied. -
Seal the session into R2 (after step 5 below has created the bucket):
task session-seal -- .run/pd
Checked against CLI 0.8.0: the upload flags in
src/migrator/providers/proton_cli.py (-f create-new-revision -d merge --json --skip-thumbnails) match filesystem upload --help;
-t is the short form of --skip-thumbnails and --json is a general option. Re-check
this whenever the pinned CLI version changes.
Run age-keygen once and store the AGE-SECRET-KEY-... line as field identity of vault
item age. Both the state and the session in R2 are encrypted to this identity, since the
state holds every mirrored path name.
Create a bucket and an API token scoped to it, and a lifecycle rule that expires objects
under .state/history/ after 30 days. Store access_key_id, secret_access_key,
endpoint (https://<account-id>.r2.cloudflarestorage.com) and bucket as fields of
vault item r2; op.env resolves them as AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY,
AWS_ENDPOINT_URL_S3 and MIRROR_R2_BUCKET, the names boto3's S3 client reads directly.
AWS_REGION=auto is a literal ENV line in the image, not a vault value: R2 has one
region, and op run masking the word auto would corrupt ordinary output.
Create a check whose schedule matches the nightly dispatch, with a grace period long enough
for a queued run plus a full one. Store the ping URL as field url of vault item
healthcheck. Chained runs ping more often than the schedule, which the check ignores.
task image # pulls the toolbox image once; every later task reuses it
task test # the pytest suite, offline
task check # every pipeline command rendered inside the image, diffed against render.txt
task plan # the real thing, read-only: lists Dropbox, fetches the state, prints the plantask plan needs the vault and the session, and mutates nothing in Proton. Its output is
the same report a run produces, with the mirror status and the batches it would move.
Dispatch the sync workflow once from the Actions tab or with
gh workflow run sync.yml. The first run finds no state and no history, treats the whole
tree as the delta, and chains itself run after run until the tree is mirrored: each run
stops starting batches at its budget, checkpoints what landed, and queues the next run,
which picks up from the state in R2. Each run's step summary shows percent mirrored and
projected runs remaining.
The default budget is 335 minutes under a 355-minute job timeout, just under GitHub's six-hour limit for one job: every run pays one image pull, one Dropbox listing and one unused tail of up to a batch, so fewer, longer runs waste less. The 20-minute gap leaves the last batch's upload and the report room to finish. Actions minutes on a public repository are free, so the seed can run on Actions alone. For a 200,000-file tree expect about 36 hours of Proton upload at the measured 0.65 s per file plus 6 to 10 hours of Dropbox downloads, around 45 hours in all, or roughly sixteen chained runs at the default budget.
task sync from a laptop behaves exactly like one chained run: it stops on
RUN_BUDGET_MIN, reports percent mirrored, and the next task sync picks up from R2. Use
it only while no Actions run is in progress.
Add schedules/dropbox.ts to jshvn/dispatch with 10 2 * * *. A scheduled run
queues behind a chained one in the workflow's concurrency group. After three green nights,
edit one file and delete one file in Dropbox and confirm both appear in Proton the next
morning: the edit as a new revision, the delete in Proton's trash.
task alone prints the menu, grouped by effect. Everything runs inside the toolbox.
task plan # fetch the state, list Dropbox, print what a sync would move
task status # counts and the last run's figures from the state in R2task plan starts a run row in its local copy of the state and never pushes it. task status fetches the state, prints, and starts nothing; both leave .run/state.sqlite
behind for inspection. Neither makes a Proton call, so both are safe while CI may be
running.
task test # pytest inside the toolbox
task lint # ruff check and format check
task fmt # ruff format
task check # render every pipeline command inside the image, diff against render.txt
task render-update # accept the current render as render.txttask sync # one budgeted run, the same thing CI runs
task sync -- RUN_BUDGET_MIN=30 # with a shorter budget
task sync -- RECONCILE=true # force the weekly Proton walk (the literal word true)
task empty-trash # permanently delete Proton trash; asks first; never scheduled
task state-rollback # list the dated history objects in R2
task state-rollback -- <key> # copy one of them over the canonical state
task session-seal -- .run/pd # encrypt a laptop Proton CLI session into R2Never run task sync or task empty-trash from the laptop while a CI run may be in
progress. Two processes holding the Proton session race its rotating refresh token, the
loser's copy dies, and the next run needs a fresh login.
task image # pull the toolbox image (no-op while it exists)
task image-build # build it from a local katoptra/lib checkout instead (LIB_DIR=../lib)
task image-clean # remove it so the next task pulls again
task clean # delete .run, the caches and every other file git ignores
task run -- <cmd> # any command in the toolbox with the repo at /work
task op -- <cmd> # the same with secrets from 1Password via op.envtask clean is git clean -fdX: it removes only files git already ignores, so the
decrypted state, a laptop Proton session under .run/pd, staging, the caches, the
toolbox include under .task/ and stray lockfiles go, and nothing tracked or unignored
is touched.
sync.yml is dispatch-only and calls
katoptra/lib's reusable sync.yml at v2, which
installs go-task and the 1Password CLI at the versions in lib's lock, pulls the image, and
runs task sync -- <vars>. The one input, vars, carries KEY=value pairs for the
pipeline: gh workflow run sync.yml -f vars='RECONCILE=true RUN_BUDGET_MIN=30'.
concurrency: {group: sync, cancel-in-progress: false} is what queues a chained or
scheduled run behind a running one; overlap would kill the Proton session.
timeout-minutes: 355 against a default budget of 335 leaves the last batch's upload and
the report room to finish. OP_SERVICE_ACCOUNT_TOKEN is the repository's one secret,
inherited by the called workflow; task sync runs the pipeline through task op, the
same op run --env-file=op.env wrapper the laptop uses, and a pipeline that fails still
reports and pings /fail inside that same container, and a run the runner cut off (the
timeout, a cancellation) gets the same from the workflow. When report left a .run/chain
marker, the called workflow queues the next run with gh workflow run, which is the only
reason the caller grants actions: write. Nothing else in this repo starts a run; the
nightly dispatch comes from jshvn/dispatch.
check.yml runs on pull requests: lib's reusable check.yml,
which renders the pipeline inside the image and diffs it against render.txt,
and a second job for task test and task lint. Neither has access to the vault, so a
pull request from a fork can run it safely. GitHub registers it when the first pull
request is opened.
On a public repository the run logs and step summaries are public too. What they carry:
phase lines, counts, retry warnings with the provider's error class, and the report
tables. What they never carry: a path name, a credential, or an account identifier; the
op run masks every value it resolved, MIRROR_VERBOSE is off by default so an
error prints as its class, and the report is built from counts alone. Only collaborators
can dispatch the workflow.
config/mirror.toml is the one behavior input; the schema is strict and rejects unknown keys. It names no account.
| Key | Meaning |
|---|---|
mirror.id |
Name recorded in the state. |
dropbox.root |
Subtree to mirror; empty means the whole Dropbox. |
dropbox.page_limit, minimum_call_interval_seconds |
Listing page size and the serialised call spacing. |
dropbox.download_workers |
Files fetched in flight during fetch, under the shared Dropbox rate limit. |
budget.batch_gb, batch_files |
A batch's byte and file caps; a file over batch_gb is a batch by itself. |
budget.max_file_gb |
Largest file a plan will stage; bigger files are left out and counted as oversized. A GitHub runner guarantees 14 GB of disk. |
budget.run_budget_minutes |
Wall-clock budget from the run's start; batches stop starting when it runs out. |
budget.ceiling_gb |
Refuse a Dropbox tree larger than this. |
budget.disk_headroom_gb |
Free disk the runner must keep beyond a batch's staging. |
budget.listing_floor_ratio |
Refuse a listing smaller than this share of the mirrored file count. |
proton.walk_workers |
Folder listings in flight during the reconcile walk, each from its own copy of the CLI session (1 to 32). |
reconcile.weekday |
UTC weekday (0 is Monday) whose first run does the Proton walk. |
The account is the environment, all of it read in src/migrator/env.py and
src/migrator/config.py: the twelve names in op.env, of which
MIRROR_DROPBOX_ACCOUNT_ID (the dbid: the run must be reading; anything else is
refused), MIRROR_PROTON_DESTINATION (the CLI path of the mirror root) and
MIRROR_PROTON_DESTINATION_UID (its UID, verified on every run before any write) override
the TOML keys dropbox.expected_account_id, proton.destination and
proton.expected_destination_uid, which exist for a private fork that prefers a file.
Run overrides go after the double dash, task sync -- RUN_BUDGET_MIN=30 RECONCILE=true,
or in the workflow's vars input; the Taskfile maps them to the environment the migrator
reads. MIRROR_VERBOSE=1 in the environment prints an error's full text instead of its
class; MIRROR_WORK_DIR (default .run) and MIRROR_CONFIG (default
config/mirror.toml) are set by the Taskfile. The non-vault AWS_REGION literal is an
ENV line in the image so every process in the toolbox sees it.
The step summary is built from the state database alone, so task status on a laptop
shows the same figures as the Actions page. It carries counts only, never a path name:
- Mirror status: inventory files and bytes, mirrored files and bytes, percent mirrored, non-downloadable entries, oversized files and bytes, batches and bytes remaining, projected runs remaining, chain.
- This run: budget used, batches planned and completed, files fetched, vanished and hash-mismatched, files and bytes uploaded, skipped as content-identical by Proton, confirmed, checkpointed, trashed.
- Throughput: GB/h down from Dropbox and up to Proton; batch durations (min, median, max).
- Throttling per provider: rate-limit responses, seconds waited, longest wait.
- Errors and issues by class, and Verification: files confirmed this run and cumulatively, the last reconcile walk's state (complete or partial), matched, dropped, strays trashed, folders trashed and mismatches.
- Phases: the status of every phase of the run.
Error text lives in the encrypted state, in the events table. After task status:
task run -- python -c "import sqlite3; c=sqlite3.connect('.run/state.sqlite'); print(*c.execute(\"SELECT timestamp, phase, message, safe_raw_error FROM events WHERE level='ERROR' ORDER BY id DESC LIMIT 20\"), sep='\n')".state/state.sqlite.xz.age the state: evidence tables + mirror_objects, runs, batches, deletions
.state/history/<epoch>-<label>.sqlite.xz.age one copy per checkpoint; label is the batch number, trash, reconcile or report
.state/session.tar.age the Proton CLI session (auth-session.json, clientUid.json); no history
R2 has no object versioning, so every checkpoint writes the dated copy first and then a
server-side copy to the canonical key; a bad state rolls back with task state-rollback.
The session gets no history because a stale copy holds a rotated-out refresh token and
cannot be restored. The work directory .run/ on the runner holds the decrypted state,
the session and staging for the life of one job; it is never uploaded as an artifact and
is ignored by git.
login firstin the state events, or a run that fails at the first Proton call. The session is gone. Repeat setup step 3's sign-in andtask session-seal -- .run/pd.configured Proton destination did not resolve to exactly one folder. The folder named in the vault'sproton/destinationis not a direct child of its parent in Proton. List the parent withfilesystem list -jand either move the folder or fix the field.did not exactly match the listing. The folder exists but its UID differs from the vault'sproton/destination_uid. Copy the UID from the listing if the folder was recreated on purpose.MIRROR_DROPBOX_ACCOUNT_ID ... must be a full dbid: identifierat startup. The vault field is empty or the reference inop.envorsync.ymlnames the wrong field.state object is missing but history exists. Roll back withtask state-rollback. Never delete the history to make a run start fresh.- The state looks wrong after a run.
task state-rollbacklists the dated copies;task state-rollback -- <key>copies one over the canonical state. The next run repeats from there, and re-uploads skip content Proton already holds. - The report shows files confirm failed. Proton's CLI refused those uploads, usually a passing server error; the error text is in the state under the batch item. The batch checkpointed without them and the next run re-uploads them. A file that fails every night is worth reading the error for.
- A run stops on budget every night without finishing. Lower
batch_filesorbatch_gb; the throughput and batch-duration rows say which. A run that checkpointed nothing does not chain and fails instead, so a batch that fails identically cannot loop. - Proton 429s or a collapsing upload rate. The throttling table is the gauge; lower
batch_gb. Dropbox stays at 10 requests a second. - A weekly reconcile does not finish in one run. Normal on a large tree: the walk
resumes where it stopped on the next run that reconciles, whether that is the following
scheduled weekday or a run forced with
RECONCILE=true, and finishes over as many runs as it needs. - A flag such as
RUN_BUDGET_MINseems ignored. The report's "budget minutes" row shows what the run saw.RECONCILEtakes the literal wordtrue. Both go after the double dash,task sync -- RUN_BUDGET_MIN=30; before it they set a host-side task var that never reaches the container. - The report shows oversized files. They are over
max_file_gbor over what the runner's disk could stage, and percent mirrored stays short by their bytes. Upload them by hand to their Dropbox path under the destination: the reconcile walk leaves alone any Proton node whose path the Dropbox listing knows, so a manual copy is never trashed. Or shrink or move the file in Dropbox. - A case-only rename in Dropbox does not reach Proton. Files are keyed by their lowercased path, so changing only the case of a name changes nothing in the delta and Proton keeps the old spelling. Rename to something else and back if the case matters.
- Move the mirror folder in Proton. Rename or move it anywhere under My files, then
change the vault's
proton/destination. The UID survives both, and every run verifies it. - Switch the Dropbox account. New
refresh_tokenandaccount_idfields from setup step 2 signed in as the new account; no commit is needed. The next run trashes what the old account had and mirrors the new tree; to start clean instead, empty the Proton folder and delete both the state object and everything under.state/history/before the switch. - Move the runner. The same image, Taskfile and
op.envrun anywhere with a container engine, go-task andop:task syncis the whole job and the next cron tick is the chain.
The mirrored tree includes personal documents. During a batch its files sit decrypted on the runner's ephemeral disk and in memory, which is inherent: Dropbox serves plaintext and Proton encrypts client-side inside the CLI. What bounds it: the runner is a single-tenant VM destroyed after the job; logs and the step summary carry counts, never names; no workflow artifact is ever uploaded; the state, which holds every path name, is age-encrypted at rest; the Dropbox credentials cannot write, the R2 token reaches one bucket, and the service account reads one vault.
What the public repository holds: code, the Taskfile, the behavior config, and
op:// references made of a vault UUID and field names. What it does not hold: any
credential, the Dropbox account id, the Proton folder path or UID, the bucket name or
endpoint, the healthcheck URL, or any mirrored path name.
Taskfile.yml the mirror's own verbs around katoptra/lib's toolbox, included at v2
.taskrc.yml trusts raw.githubusercontent.com for the include; refetched hourly at most
render.txt the committed dry run of the pipeline; task check diffs against it
op.env op:// references, committed; the one place vault names are listed for the laptop
config/mirror.toml the one behavior input; names no account
src/migrator/ the package: commands, phases/, providers/, state, store, crypt, session
tests/ pytest suite, no network; tests/fixtures/live/ is ignored by git
.github/workflows/ sync.yml and check.yml, callers of katoptra/lib's workflows at v2
.run/ work directory at run time; ignored by git, removed by task clean
.task/ the include cache; rides into the image, so a run needs no network