Repository navigation
fix(restore): restore documents before uploading media, fail fast on a read-only token - #3
Merged
amenophis1er merged 1 commit intoSep 10, 2026
Conversation
…a read-only token A token with content:read/media:read but no content:write made 'trokky restore' upload every media file first — minting fresh media IDs — then fail every document write with HTTP 403, then print "Restore completed" and exit 0. The target instance was left with a full media store under new IDs while its documents still referenced the old ones, so every image 404s. Restore is now three ordered phases: documents are written first with their references untouched, media is uploaded second, and a third pass rewrites media references only into documents that carry them. Media upload is the destructive half, so nothing is uploaded until the token has proven it can write documents; the first 401/403 on a document write aborts with "token lacks content:write (required to restore documents)" and a non-zero exit, before any upload happens. The PUT->POST singleton fallback and the --overwrite POST->PUT fallback no longer mask a permission denial. Summary counters now report work that actually landed: documents written, media uploaded, references contained in documents that were successfully rewritten, and collections that actually received documents. "Restore completed" prints only on a fully clean run; any failure exits non-zero. If media was uploaded and some documents could not be pointed at it, a loud orphaned-media warning names the counts. Also normalize --url and TROKKY_URL to the API root, so '--url http://host:3210' no longer fails with "HTTP 404: Cannot GET /collections". Closes #2
amenophis1er
deleted the
foreman/prevent-silent-restore-failures-and-186b
branch
September 10, 2026 17:03
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2.
Root cause
restoreuploaded media before it wrote any documents. Media upload is the destructive half — it assigns new media IDs — so when the document writes then failed with 403, the target was left with a fully populated media store that nothing referenced, while the pre-existing documents still pointed at the old IDs. Every image and file 404s, site-wide.Two things turned a permission error into corruption:
PUT→POSTand--overwritePOST→PUTfallbacks swallowed the first error, so a 403 looked like an ordinary retry rather than a hard stop.The summary then reported
Media restored: 141,References updated: 93,Restore completedand exit 0 — three separate untruths, the last of which meant no script or CI step could detect the failure.The ordering decision
Restore now runs in three ordered phases: documents → media → reference rewrite.
Documents are written first with the backup's original media references intact. Only once at least one document has landed does media upload run; phase 3 then rewrites the references to the newly assigned IDs and PUTs back only the documents whose bodies actually changed.
This was chosen over "preflight the permissions, then keep the old order". A preflight is a check that can be wrong — permissions can be scoped per collection, and a token that passes a probe can still be refused on a specific write. Making the first document write be the gate means the destructive step is unreachable unless a real write has already succeeded. Preflighting is a second line of defence, not the fix.
Phase 2 is additionally skipped when zero documents were restored, so uploads cannot orphan themselves.
Changes
token lacks content:write (required to restore documents)and a non-zero exit, before any upload. The two fallback paths no longer mask a denial.Restore completedprints only on a fully clean run.References updatedcounts references in documents that were successfully rewritten. AddedDocuments failed.Collections restoredcounts collections that actually received a document. Non-permission write failures are counted instead of silently skipped.--urlpapercut.NormalizeBaseURLnow appends/apionly when the URL's path lacks anapisegment, so/api,/api/v1and/backend/apiare left alone and the function is idempotent. Applied to both the--urlflag andTROKKY_URL.cmd.OutOrStdout()/cmd.ErrOrStderr();SilenceUsage: trueso a runtime failure isn't buried under a usage dump.Deliberately not done
No automatic rollback of uploaded media. Deleting media after a partial failure is itself destructive and cannot distinguish files this run uploaded from pre-existing ones that happen to share a checksum. Given the new ordering makes the orphaning case much harder to reach, a loud warning is the safer trade. Revisit if it shows up in practice.
isPermissionDeniedmatches on error-message substrings (" (HTTP 403)","HTTP 403") becauseinternal/clientdoes not expose typed status codes. It is pragmatic and documented, but it is string-matching on error text and will drift if that formatting changes. A typed error ininternal/clientis the durable fix and belongs in its own PR.Tests
Six new tests in
cmd/restore_test.go, driven againsthttptestservers that record every request:TestRestoreAbortsBeforeUploadingMediaWhenDocumentWritesAreForbidden— asserts zero requests reach/media/uploadTestRestoreFailedDocumentWritesExitNonZeroTestRestoreCountersReportOnlyRealWorkTestRestoreHappyPathReportsTrueCountsTestRestoreDryRunMakesNoWrites— dry-run behaviour is unchangedTestIsPermissionDeniedPlus
TestNormalizeBaseURLand twoResolveCredentialstests ininternal/config. Three existing config tests were updated because they asserted the old raw passthrough, which is the bug being fixed;cmd/documents_test.gowas updated for the same reason. Nothing was deleted or weakened.go build ./...,go vet ./...andgo test ./...all pass.