First stab at implementing asynchronously pushed-back task results#2
Open
First stab at implementing asynchronously pushed-back task results#2
Conversation
The general idea is to not keep workers waiting for tasks that may take a while to complete. Things like backups and upgrades may take the client a minute to process - the worker does not need to sit idle during that time; it might as well process other tasks simultaniously. Flow: 1. Worker pushes the task to the client, with a uri and one-time signing key 2. Client acknowledges receipt and support for the async return push with a 202 3. Worker marks task as awaiting async response and continues to whatever other tasks are queued 4. Client processes slow task, and when done pushes the response to the sitedash endpoint from (1). Endpoint stores received data, marks related job exec as having received data to process. 5. Same worker (when free) is assigned again to job exec and finishes processing the data
Mark-H
force-pushed
the
feature-async-return-push
branch
from
87ef56e to
72a2bf4
Compare
… the added complexity The return push uri is now a 96-character random string that's only valid once (when SiteDash expects a response) so that's going to be incredibly difficult to cause arbitrary data to get sent back by an attacker. Lack of a signature does theoretically allow MITM, but nearly impossible to execute server-to-server and production runs on HTTPS with HSTS.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.