feat(walrus): Harbor + Seal encrypted private storage (P0 #5 / #6 / #7)#29
Merged
Conversation
Port the proven Walrus Harbor + Seal flow (reference: benhaq/walrus-harbor-mcp) into packages/walrus as plain async/await TypeScript (no Effect). - HarborClient: Bearer REST surface (spaces/buckets/files) — reserve→sign→ finalize bucket creation, multipart upload, status poll, download. - SealCrypto: Seal encrypt/decrypt + sponsored-tx signing (@mysten/seal@1.1.3, pinned to match the existing @mysten/sui@2.17.0). - HarborStorage: createPrivateBucket / putEncrypted / getDecrypted, with a mirror_missing_grant post-finalize retry (the reference's retry was dead code — the server error code was being overwritten; preserved here). - harborConfigFromEnv() for Worker/Node env wiring. - scripts/harbor-smoke.ts: manual live testnet round-trip (CLI only, not CI). Verified end-to-end on Sui testnet: encrypt→upload→download→decrypt is byte-exact; bundles for a Cloudflare Worker at ~65KB gzip with no Node built-ins, no WASM, no eval. Refs #5, #6
Route private-namespace artifacts through Seal encryption + Harbor storage in the hosted Worker, mirroring how MEMWAL secrets are handled: - storeNamespaceImport / readArtifact / updateNamespaceArtifact encrypt-on-write and decrypt-on-read via HarborStorage when a namespace is Harbor-backed; public namespaces keep the plaintext R2 path unchanged. - Version-scope the artifact write/classify path to current_version_id so a private edit never misclassifies a stale R2 row and leaks plaintext. - Persist-before-delete on rotation: commit the new harbor_file_id before dropping old ciphertext (no dangling pointer / silent loss). - putEncrypted: bounded settlement-retry for transient upload_funding_timeout; storeNamespaceImport: orphan-cleanup rolls back a freshly-created bucket on upload failure so a failed run leaks no bucket (live-validated). - migration 0007: add storage_provider + seal_identity_salt columns and the per-namespace Harbor bucket / Seal policy / per-artifact file pointer columns. - wrangler.example.jsonc: document HARBOR_* vars and required secrets. - scripts: live + FakeD1 round-trip smoke harness (reads creds from env only).
Deploying contextmem with
|
| Latest commit: |
0f6fe00
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://c0e9508f.contextmem.pages.dev |
| Branch Preview URL: | https://feat-harbor-seal-storage.contextmem.pages.dev |
harrymove-ctrl
changed the title
packages/walrus imports @mysten/sui across proof/bcs/resources/resolve
and harbor/{constants,seal}, but only @mysten/seal was declared in
package.json — @mysten/sui resolved via workspace hoist only. Pin it
explicitly (^2.9.1, matching apps/api) so the package is self-contained
and survives a non-hoisted install. Surfaced by the open-issue audit (#6).
harrymove-ctrl
pushed a commit
that referenced
this pull request
Jun 20, 2026
… bundle worker.ts imported `harbor` via the @contextmem/walrus barrel, whose siblings (bcs/resolve/quilt) import the @contextmem/core barrel, which re-exports html.ts → jsdom. jsdom uses __dirname at import time, which the Workers runtime rejects (validation error 10021), so the post-#29 deploy failed. The harbor subtree imports no core, so add a ./harbor subpath export and import from it — matching the existing leaf-import convention for @contextmem/core/{chunks,facts,utils}. Drops the Worker bundle from ~11.8MB to ~3.8MB; jsdom no longer in the executable bundle.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Private-namespace artifacts are now Seal-encrypted and stored in per-namespace Harbor (Walrus) buckets, while public namespaces keep the plaintext R2 path. Covers the P0 Harbor/Seal storage track: the encrypt/decrypt spike (#5), the HarborClient (#6), and the Worker-side namespace→bucket→Seal mapping + service-key handling (#7).
What's in here
packages/walrus— Harbor + Seal client (#5, #6)HarborStorage: reserve→sign→finalize bucket lifecycle, multipart upload, status poll, download.putEncrypted/getDecrypted: client-side SEAL encrypt-on-write, decrypt-on-read (@mysten/seal+@mysten/sui).upload_funding_timeoutfunding-settlement failure.apps/api/src/worker.ts— Worker wiring (#7)storeNamespaceImport/readArtifact/updateNamespaceArtifactroute throughHarborStoragewhen a namespace is Harbor-backed; public namespaces unchanged.current_version_idso a private edit never misclassifies a stale R2 row and leaks plaintext.harbor_file_idbefore dropping old ciphertext (no dangling pointer / silent loss).Schema / config
0007:storage_provider+seal_identity_saltcolumns, per-namespace Harbor bucket / Seal policy columns, per-artifact file-pointer columns (all nullable — existing/public artifacts keep working).wrangler.example.jsonc: documentsHARBOR_*vars and the required secrets (HARBOR_API_KEY,HARBOR_SERVICE_PRIVATE_KEY); real values live only as wrangler secrets / gitignored.dev.vars.Testing
apps/api/scripts/harbor-wiring-smoke.ts: full encrypt→upload→edit→decrypt round-trip against a FakeD1 + live Harbor (creds read from env only). FakeD1 path green.upload_funding_timeoutoutage (testnet-wide, not a code issue); a self-cleaning retry is armed to confirm once the network recovers.bun run typecheckclean forpackages/walrusandapps/api.Notes
HARBOR_API_KEY/HARBOR_SERVICE_PRIVATE_KEYare testnet credentials held only in wrangler secrets / gitignored.dev.vars; recommend rotating them after the hackathon.