Skip to content

Backfill orchestrator should support batched execution #570

@kbighorse

Description

@kbighorse

Context

The batch_size parameter was removed from the backfill orchestrator and backfill_radionuclides() in #558 because it was accepted but never used — all rows were loaded and committed in a single transaction.

Work Required

Implement real batched execution for backfill jobs:

  • Process rows in configurable batch sizes using LIMIT/OFFSET or cursor-based pagination
  • Commit after each batch so partial progress survives failures
  • Re-add --batch-size CLI flag once batching is implemented
  • Log per-batch progress

Refs

Removed in #558

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions