Skip to content

add idempotency key to leader selected mode collection job requests - #4893

Closed
tgeoghegan wants to merge 4 commits into
mainfrom
timg/leader-selected-collection-job-idempotency-key
Closed

tgeoghegan wants to merge 4 commits into
mainfrom
timg/leader-selected-collection-job-idempotency-key

Conversation

@tgeoghegan

Copy link
Copy Markdown
Contributor

Implements the proposed DAP change in ietf-wg-ppm/draft-ietf-ppm-dap#797. One unexpected quirk here: because the collection job req is bound into the AAD on the aggregate shares, the idempotency key has to be transmitted to the helper. That's not a problem, as such, it's just odd, since there's absolutely no decision that the helper could ever make with that value. But then as I observed in the DAP PR, the problem we're solving is irreducibly ugly, and that will be reflected in its solution.

@tgeoghegan

Copy link
Copy Markdown
Contributor Author

Draft so I can see what breaks in CI.

@tgeoghegan

tgeoghegan commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor Author

I was pleasantly surprised to see that this doesn't actually require any DB schema changes: we already store the encoded collection job query in collection_jobs, and since idempotency keys are opaque byte strings anyway, that suffices for recognizing replayed requests.

That said, we should add an index for looking up collection jobs by (task_id, query).

@tgeoghegan tgeoghegan added allow-changed-migrations Override the ci-migrations check to allow migrations that have changed. draft-ietf-ppm-dap-17+ Stuff for implementing draft-ietf-ppm-dap-17 and beyond labels Oct 9, 2026
Moving toward server-selected resource identifiers means that HTTP
clients need to parse resource identifiers from URIs provided by servers
in `Location` HTTP headers.

This adds an extension trait `HeaderMapExt` with method
`ids_from_collection_job_location`. It finds a `Location` header and
parses its value, extracting the task and collection job IDs.

In the future, we'll need to do this for aggregate shares and
aggregation jobs, too, at which point we can make this method generic
over types that can be parsed from Base64URL. But for now, we specialize
to collection jobs.
@tgeoghegan
tgeoghegan force-pushed the timg/leader-selected-collection-job-idempotency-key branch from 4281657 to b2a2523 Compare October 9, 2026 20:34
@tgeoghegan

Copy link
Copy Markdown
Contributor Author

Closing, these changes are being handled elsewhere.

@tgeoghegan tgeoghegan closed this Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

allow-changed-migrations Override the ci-migrations check to allow migrations that have changed. draft-ietf-ppm-dap-17+ Stuff for implementing draft-ietf-ppm-dap-17 and beyond

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant