Skip to content

Batch DB commits during score/tailor/cover instead of one commit at the end - #100

Open
msohailse wants to merge 1 commit into
Pickle-Pixel:mainfrom
msohailse:batch-commit-feature
Open

Batch DB commits during score/tailor/cover instead of one commit at the end#100
msohailse wants to merge 1 commit into
Pickle-Pixel:mainfrom
msohailse:batch-commit-feature

Conversation

@msohailse

Copy link
Copy Markdown

Summary

score, tailor, and cover-letter stages each collected every result in memory and only wrote to the DB in a single commit after the whole job list finished. If the run got interrupted partway through -- rate limit, network drop, killed process, crash -- nothing already completed that run had been persisted, so it was all lost.

  • All three stages now flush to the DB every 5 jobs (and on the final job), instead of one commit at the very end.
  • Pure DB-durability fix, no provider/LLM-specific changes -- applies cleanly on top of current main.

Test plan

  • ruff check / ruff format --check on all three files -- identical finding count to unmodified main (18 pre-existing, 0 new)
  • Verified behavior: killed a run mid-batch and confirmed only the in-progress batch (up to 5 jobs) was lost, not the whole run's prior progress

🤖 Generated with Claude Code

score/tailor/cover-letter stages each held every result in memory and
wrote them to the DB in a single commit after the entire job list
finished. If the run got interrupted, crashed, or hit an external error
partway through (rate limit, network drop, killed process), everything
already completed that run was lost -- nothing had been persisted yet.

All three stages now flush to the DB every 5 jobs (and on the final job),
so an interruption only loses the batch in progress, not the whole run.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant