dofs: hasObjects exceeds the Durable Object SQLite binding limit

[CrazyBoyM]

Describe the bug

hasObjects() binds up to 256 hashes into one IN (?, ...) query. Durable Object SQLite accepts at most 100 bound parameters per query, so a valid sync batch with 101 or more unique chunks fails before it can apply or checkpoint the batch.

The current comment says that 256 is below SQLite's limit, but the relevant limit is the Durable Object platform limit rather than the limit in Node SQLite:

https://developers.cloudflare.com/durable-objects/platform/limits/#sql-storage-limits

The failing path is packages/dofs/src/sync/fetch.ts. pullOnce() collects all unique chunk hashes referenced by a batch of up to 256 entries and passes that list to both the remote and local hasObjects() calls. The error happens before applyChanges() and the fetch cursor update, so a retry reaches the same batch and fails again.

Steps to reproduce

Add 101 complete objects to a database backed by real Durable Object SqlStorage, then call hasObjects(db, hashes) with all 101 hashes. The existing workerd test setup makes this a focused regression test:

it("probes more objects than Durable Object SQLite accepts in one query", async () => {
  await withDB(async (db) => {
    const hashes: Uint8Array[] = [];
    for (let i = 0; i < 101; i++) {
      const bytes = new TextEncoder().encode(`object-${i}`);
      const hash = chunksOf(bytes)[0].hash;
      stageBlob(db, hash, bytes, i);
      hashes.push(hash);
    }

    expect(hasObjects(db, hashes)).toEqual(hashes);
  });
});

Run:

npm run test:workers --workspace @cloudflare/dofs -- src/sync/fetch.test.ts

On main at 76d9e75, the new test fails with:

Error: too many SQL variables at offset 417: SQLITE_ERROR
at Database.all (src/storage.ts:81)
at hasObjects (src/sync/fetch.ts:53)

This can be reached by 101 unique one-chunk files in one pull batch, or by one file that contains more than 100 unique 512 KiB chunks.

Expected behavior

hasObjects() should accept any hash list produced by the sync driver and split its SQL work into queries that stay within the Durable Object SQLite binding limit.

Proposed fix

Limit each hasObjects() probe to 100 hashes. The function already loops over windows, so this only changes the window size. It does not change the wire format, schema, ordering, duplicate handling, or pull batch size.

A tested patch and changeset are ready here:

main...CrazyBoyM:computer:agent/dofs-bound-object-probes

Commit: CrazyBoyM@41ec4a2

Following CONTRIBUTING.md, I am opening the fix proposal before a pull request. If this direction fits, please add allow-pr or ask me to open the pull request.

Verification

The regression test was run before and after the change against real Durable Object SQLite. Before the change, 1 of 11 focused tests failed with the error above. After the change:

test:workers fetch.test.ts: 1 file, 11 tests passed
node fetch.test.ts:         1 file, 11 tests passed
@cloudflare/dofs:          37 files, 437 tests passed
typecheck:                 passed
build:                     passed
npm run check:             passed

There is no compatibility or migration concern. A 256-hash probe becomes three indexed queries instead of one invalid query on Durable Object SQLite.

Environment

• cloudflare/computer at 76d9e75c5688713b656bce85540d9e0071cece8b
• Node 22.22.2
• Vitest 4.1.10
• @cloudflare/vitest-pool-workers 0.16.20

[daviskathy6643]

📌🙌 覆盖率门槛设多少比较合理

[wilsondonna51]

🙌🧪 标注规范有没有简版说明

[medinakenneth6634]

文档写得很友好，感谢开源 👍🔥