Repository navigation
OUT-4176: support Dropbox uploads over 150 MiB via upload sessions - #153
Merged
Merged
Conversation
…ffers Reads a byte stream into fixed-size buffers (last one may be smaller) so a large file can be fed to a Dropbox upload session one chunk per append. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add _uploadFileSession (start, append_v2, finish) for files too large for /files/upload, exposed as the retry-wrapped uploadFileSession. getBody yields a fresh stream so a retry restarts the whole session, and the streamed byte count is checked against the expected size before finishing so a short download can never commit a partial file. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
uploadFileInDropbox now branches on the Assembly file size: files over the 150 MiB single-request limit (or with an unknown size) go through the chunked upload session, everything else keeps the single-shot /files/upload. Add a shared download step that fails on a non-ok response so an error page is never uploaded as file content, plus a log line showing which path each file took. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Object logs were capped at depth 2, so nested payloads printed as [Object] (e.g. a batched item's opts/file). Raise the default depth to 4 and add a LOG_DEPTH override, keeping the array and string caps so a line stays bounded. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Drive an Assembly -> Dropbox sync for a file with no reported size, which routes through the upload session, and assert the session start/finish ran and the mapping row was created. The single-shot upload is left unmocked so a routing regression trips msw. Adds a mockDropboxUploadSession helper. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Collaborator
Author
|
@greptileai rereview |
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Files larger than 150 MiB failed to upload to Dropbox with
payload_too_largebecauseuploadFileInDropboxused the single-shot/files/uploadendpoint (150 MiB cap). Large files now stream through a Dropbox upload session (OUT-4176).What changed
chunkStreamhelper — reads a byte stream into fixed-size buffers (last one smaller); peak memory ≈ one chunk.DropboxClientupload session —_uploadFileSessiondrivesstart→append_v2(per chunk) →finish, exposed as retry-wrappeduploadFileSession.getBodyyields a fresh stream so a retry restarts the whole session; the streamed byte count is checked against the expected size beforefinish, so a short/expired download can never commit a partial file.uploadFileInDropboxbranches onfile.size: over 150 MiB (or unknown size) → session, else the unchanged single-shot path. A shared download step fails on a non-ok response so an error page is never uploaded as content. A log line records which path each file took.DROPBOX_SINGLE_UPLOAD_MAX_BYTES(150 MiB),DROPBOX_UPLOAD_CHUNK_BYTES(16 MiB, a multiple of 4 MiB per Dropbox guidance).LOG_DEPTHoverride) so nested payloads stop printing as[Object]. Cross-cutting fix, bundled here.Design notes
session_id; the orphaned partial session expires on Dropbox's side (no duplicate,mode: 'add').>boundary: a file of exactly 150 MiB stays on the single-shot path.Testing
pnpm typecheck,pnpm lint,pnpm test— 311 unit tests pass.pnpm test:integration— 189 integration tests pass (22 files), including a new large-file flow (assembly-to-dropbox.large-file.integration.test.ts) that drives the session path end-to-end and asserts start/finish ran and the mapping row was created. The single-shot upload is left unmocked, so a routing regression trips msw.Before merge
file.sizeis actually populated from Assembly on real files (log it in staging). If it's ever missing, files still upload correctly but always take the slower session path and skip the byte-count guard.🤖 Generated with Claude Code