Skip to content

feat(directory): run sync as a retryable background job#66

Merged
WhiteMuush merged 1 commit into
mainfrom
feat/54-sync-jobs-retry
Jun 19, 2026
Merged

feat(directory): run sync as a retryable background job#66
WhiteMuush merged 1 commit into
mainfrom
feat/54-sync-jobs-retry

Conversation

@WhiteMuush

Copy link
Copy Markdown
Owner

What

Moves directory sync out of the request cycle into a retryable job queue. Closes #54.

Why

syncDirectoryConnection was awaited inside POST /api/directory/[id]/sync: large directories risk an HTTP timeout, and a failure left no retry or recovery.

How

  • SyncJob table (status, attempts, maxAttempts, lastError, runAfter, timestamps) + migration.
  • Enqueue: the route scopes the connection to the caller's company (404 otherwise), enqueueSyncJob dedups against a pending/running job, and the handler returns 202 with the job id instead of blocking.
  • Processing: processSyncJobs claims due jobs with FOR UPDATE SKIP LOCKED (safe across instances), runs the sync, and on failure retries with exponential backoff (30s → 60s → 120s) up to maxAttempts, then marks FAILED.
  • The route kicks the queue fire-and-forget; a scheduler (Add a periodic scheduler (auto scan + sync) #53) can also drain it. Concurrency is now enforced by the job claim, which supersedes the in-request advisory lock from Replace in-memory lock with pg_advisory_lock #55.

Tests

backoffMs, enqueue dedup vs create, and processSyncJobs success / re-queue-with-backoff / exhausted-FAILED paths; updated route tests (auth, 404 scoping, 202 + job id).

Notes

  • Migration authored by hand (no local DB in this environment); prisma validate passes and the client generates cleanly. Run prisma migrate deploy on deploy.
  • The fire-and-forget kick relies on a long-running node server; serverless deployments should rely on the scheduler to drain the queue.

Closes #54

🤖 Generated with Claude Code

syncDirectoryConnection was awaited inside the POST handler: timeout
risk on large directories, no retry or recovery (#54).

- New SyncJob table (status, attempts, lastError, runAfter) + migration.
- enqueueSyncJob dedups against a pending/running job; the route now
  scopes the connection to the caller's company, enqueues, and returns
  202 with the job id instead of blocking.
- processSyncJobs claims due jobs with FOR UPDATE SKIP LOCKED (safe
  across instances), runs the sync, and on failure retries with
  exponential backoff (30s/60s/120s) up to maxAttempts before FAILED.
- The route kicks the queue fire-and-forget; a scheduler can also drain
  it. Concurrency is now handled by the job claim, superseding the
  in-request advisory lock from #55.
- Unit tests for backoff, enqueue dedup, and the success/retry/failure
  paths, plus the updated route behavior.

Closes #54
@WhiteMuush WhiteMuush merged commit 5be9003 into main Jun 19, 2026
11 checks passed
@WhiteMuush WhiteMuush deleted the feat/54-sync-jobs-retry branch June 19, 2026 08:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Move sync out of the request cycle: jobs + retry

1 participant