Skip to content

Resumable historical backfill orchestrator #116

Description

@Miracle656

Background

Bringing up a new deployment requires backfilling history. A resumable orchestrator makes this reliable over millions of ledgers.

What to build

A backfill job that processes ledger ranges in chunks, persists a durable cursor, parallelizes within bounds, and resumes exactly where it left off after a crash — without conflicting with live ingest.

Key files

  • src/ingest/backfill.ts (new)
  • src/cli.ts (backfill command)

Acceptance criteria

  • Cursor persisted; restart resumes mid-range
  • Backfill coexists with live ingest without gaps/dupes
  • Tests for chunking and resume

Drips Wave · Complexity: Advanced · 200 points


Required: Before submitting, join the contributor Telegram so your work can be tracked and counted toward the Stellar Wave: https://t.me/+fxHXq8f1SwlkZDBk

Metadata

Metadata

Assignees

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions