Repository navigation
perf(storage): bound concurrent sync and reuse retained ZIP archives - #136
Merged
Merged
Conversation
Cached Blob Range responses keep stable strong ETags and byte lengths while Last-Modified varies. Use the SDK cache validator for cached reads, propagate errors through the owned stream controller, and await each producer before advancing checkpoints. Preserve cache bypass through storage error causes.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Parsed snapshot synchronization uploads entries serially and writes a full checkpoint for every file. Add configurable bounded Range concurrency, count/time checkpoint batching, and reuse a streamed retained result.zip through an immutable, versioned archive marker. Default concurrency/checkpoint interval remain one; byte-only adapters, historical checkpoints, result.zip paths and commit version 1 remain compatible.
Only the contiguous successful prefix is durable. Count/time thresholds drain active writes before saving progress; cancellation and errors wait for extraction and storage cleanup. Four files with a ten-file checkpoint interval bound abrupt-process replay to thirteen files. Cached transport validates strong ETag, length, Content-Range and received bytes without relying on Blob CDN Last-Modified. A pinned canonical origin proof permits cache-to-sequential fallback while checking saved prefix hashes.
Validation: 435 SDK and 18 MCP tests pass, including 55 focused cache/concurrency cases; formatting, lint, types, build and coverage gates pass. Independent review is clear. Generated 281/562 MiB archives and two documents at file concurrency four peak at 264.44 MiB in independent Node processes. A real SIGKILL replays exactly thirteen files and completes.
Published beta2 hosted SICP comparison (same Node24/iad1/2048MiB functions, isolated prefixes, three cold runs each): optimized median 58.052s versus 2.5.0 510.391s and 2.2.0 88.770s. Each optimized run performs one full upstream ZIP transfer, 3,178 requests and 78 progress flushes; all 963 entries complete and page-resource samples are readable. Vercel process RSS ranges 419.83–516.86MiB across three consecutive runs, distinct from the independent SDK memory gate. The latest beta also contains the reviewed cache-to-sequential compatibility correction.
Stable publication remains held in the Changesets release PR until final-beta staging, large hosted fixtures and Hipo acceptance finish. No new Parse jobs, billing changes, production Source mutations, plugin or Gateway deployment are part of this change.