Skip to content

[Push] Resume local-file pushes from receiver-confirmed cursors#367

Merged
adamziel merged 81 commits into
trunkfrom
adamziel/push-files-sender
Jul 19, 2026
Merged

[Push] Resume local-file pushes from receiver-confirmed cursors#367
adamziel merged 81 commits into
trunkfrom
adamziel/push-files-sender

Conversation

@adamziel

@adamziel adamziel commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator

Adds PushFilesSender, a resumable processor that turns a completed local-files plan into a committed push.

Background

#366 made local-files planning resumable, but stopped at the two path lists. Sending those paths can span multiple PHP processes and cannot assume enough time or memory to finish in one run. A request may also end after the target accepted complete parts but before the sender received its response.

This change

start() owns the completed fresh local index and acquires the sender lock. Each next_step() call processes at most one planning path, one file chunk, one deleted path, or one lifecycle transition. Multipart requests may span calls. When the caller stops, cancel() abandons the open request at the preceding durable boundary and close() releases resources and the lock without finishing it. After reopening, the target's cursors determine which bytes were accepted.

The completed deletion plan is sent without re-reading the fresh local index or checking the live local tree. A path that reappears after planning belongs to the next push. File uploads still check type, size, and ctime around each read so a resumed upload cannot combine bytes from different file versions.

After the target commits the work, the fresh local index becomes the local index at the previous push and the planning files are removed.

Commit 068e025c is the last version before the processor refactor. This stack currently ends at 3e2c4b1e.

Usage

Start after producing a complete fresh local index. A later process resumes the same sender from sender.json:

$sender = $first_run
    ? PushFilesSender::start($options)
    : PushFilesSender::resume($options);

try {
    while ($has_time_remaining() && $has_memory_available()) {
        if (!$sender->next_step()) {
            break;
        }
    }
    $status = $sender->get_status();
} finally {
    if ($sender->get_status() === 'continue') {
        $sender->cancel();
    }
    $sender->close();
}

complete ends the push. restart means the caller must produce another fresh local index and call start() again. failed reports its classification and explanation through get_reason() and get_detail(); the same durable sender state remains available to a later resume() call.

Testing

There is no command integration for this processor yet. Review the interruption and continuation tests and rely on CI.

@github-actions

github-actions Bot commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Pull pipeline performance — large-directory

Site: large-directory · 2,000+ plus targeted file-transfer scenarios files · 10,000 posts · 25,000 postmeta · PHP 8.5.8

Stage PR trunk Δ Status Details
playground-sqlite-db-pull 6.35 s 6.05 s ⚪ +296 ms (+4.9%) condition=db-pull in PHP.wasm
runtime=php.wasm 8.3
wp_mysql_parser=enabled
mode=lexer
native_lexer=verified
native_token_stream=WP_MySQL_Native_Token_Stream
native_token_count=18
native_parser=selected
trunk: condition=db-pull in PHP.wasm
runtime=php.wasm 8.3
wp_mysql_parser=enabled
mode=lexer
native_lexer=verified
native_token_stream=WP_MySQL_Native_Token_Stream
native_token_count=18
native_parser=selected
playground-sqlite-db-apply 2.48 s 2.46 s ⚪ +25 ms (+1.0%) condition=db-apply to SQLite in PHP.wasm
runtime=php.wasm 8.3
wp_mysql_parser=enabled
mode=parser
native_lexer=verified
native_token_stream=WP_MySQL_Native_Token_Stream
native_token_count=18
native_parser=verified
native_ast=WP_MySQL_Native_Parser_Node
sqlite_driver_parser=verified
trunk: condition=db-apply to SQLite in PHP.wasm
runtime=php.wasm 8.3
wp_mysql_parser=enabled
mode=parser
native_lexer=verified
native_token_stream=WP_MySQL_Native_Token_Stream
native_token_count=18
native_parser=verified
native_ast=WP_MySQL_Native_Parser_Node
sqlite_driver_parser=verified
Total 8.83 s 8.51 s ⚪ +321 ms (+3.8%)

Numbers carry runner noise; treat single-run deltas as directional, not authoritative.

📈 Trunk performance history — commit-by-commit timeline.

@adamziel
adamziel force-pushed the adamziel/push-files-sender branch 4 times, most recently from 2799a4b to 7e1c9a5 Compare July 18, 2026 10:57
@adamziel
adamziel force-pushed the adamziel/push-files-sender branch from 7e1c9a5 to 93e7125 Compare July 18, 2026 11:56
@adamziel
adamziel force-pushed the adamziel/push-files-sender branch 2 times, most recently from 1f8c861 to 3ee6a4e Compare July 18, 2026 13:30
@adamziel
adamziel force-pushed the adamziel/push-files-sender branch from 3ee6a4e to a595d9c Compare July 18, 2026 13:37
@adamziel
adamziel force-pushed the adamziel/push-files-sender branch from 218819c to e014b28 Compare July 18, 2026 22:57
@adamziel
adamziel force-pushed the adamziel/push-files-sender branch from e014b28 to 038cfab Compare July 18, 2026 22:58
@adamziel
adamziel merged commit ccc6c0d into trunk Jul 19, 2026
28 of 30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant