Skip to content

Sleep 30s between batches in ingestor upload session#9

Open
raptormitch wants to merge 1 commit into
masterfrom
feature/mitchellhause/sleep-between-api-calls
Open

Sleep 30s between batches in ingestor upload session#9
raptormitch wants to merge 1 commit into
masterfrom
feature/mitchellhause/sleep-between-api-calls

Conversation

@raptormitch
Copy link
Copy Markdown
Collaborator

Summary

  • Adds a 30-second sleep between batch requests in create_ingestor_upload_session to avoid rate-limiting
  • Sleep is skipped after the final batch

Test plan

  • Run with a URL count exceeding 1000 and confirm 30s pause between batches

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@raptormitch raptormitch requested review from cr-rm and hdehaven May 27, 2026 19:51
Copy link
Copy Markdown

@hdehaven hdehaven left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants