diff --git a/.changeset/update-agent-skills.md b/.changeset/update-agent-skills.md new file mode 100644 index 0000000000..245ca1f288 --- /dev/null +++ b/.changeset/update-agent-skills.md @@ -0,0 +1,11 @@ +--- +'@tanstack/angular-db': patch +'@tanstack/db': patch +'@tanstack/offline-transactions': patch +'@tanstack/react-db': patch +'@tanstack/solid-db': patch +'@tanstack/svelte-db': patch +'@tanstack/vue-db': patch +--- + +Update agent skills to match current APIs and behavior. diff --git a/packages/angular-db/skills/angular-db/SKILL.md b/packages/angular-db/skills/angular-db/SKILL.md index e6e17f85b3..11f6ac3e4f 100644 --- a/packages/angular-db/skills/angular-db/SKILL.md +++ b/packages/angular-db/skills/angular-db/SKILL.md @@ -10,7 +10,7 @@ description: > type: framework library: db framework: angular -library_version: '0.6.0' +library_version: '0.6.17' requires: - db-core sources: @@ -101,6 +101,10 @@ const query = injectLiveQuery({ }) ``` +A bare `{ query }` config defaults to `startSync: true` and `gcTime: 0`, like +the query-function overload. Explicit values in the config override those +defaults. + ## Angular-Specific Patterns ### Reactive params with signals diff --git a/packages/db/skills/db-core/SKILL.md b/packages/db/skills/db-core/SKILL.md index 19c2ac27d4..496c7405bc 100644 --- a/packages/db/skills/db-core/SKILL.md +++ b/packages/db/skills/db-core/SKILL.md @@ -3,14 +3,14 @@ name: db-core description: > TanStack DB core concepts: createCollection with queryCollectionOptions, electricCollectionOptions, powerSyncCollectionOptions, rxdbCollectionOptions, - trailbaseCollectionOptions, localOnlyCollectionOptions. Live queries via + trailBaseCollectionOptions, localOnlyCollectionOptions. Live queries via query builder (from, where, join, select, groupBy, orderBy, limit). Optimistic mutations with draft proxy (collection.insert, collection.update, collection.delete). createOptimisticAction, createTransaction, createPacedMutations. Entry point for all TanStack DB skills. type: core library: db -library_version: '0.6.0' +library_version: '0.6.17' --- # TanStack DB — Core Concepts @@ -60,4 +60,4 @@ For framework-specific hooks: ## Version -Targets @tanstack/db v0.6.0. +Targets @tanstack/db v0.6.17. diff --git a/packages/db/skills/db-core/collection-setup/SKILL.md b/packages/db/skills/db-core/collection-setup/SKILL.md index bdbd6e20f2..9673266c20 100644 --- a/packages/db/skills/db-core/collection-setup/SKILL.md +++ b/packages/db/skills/db-core/collection-setup/SKILL.md @@ -4,16 +4,17 @@ description: > Creating typed collections with createCollection. Adapter selection: queryCollectionOptions (REST/TanStack Query), electricCollectionOptions (ElectricSQL real-time sync), powerSyncCollectionOptions (PowerSync SQLite), - rxdbCollectionOptions (RxDB), trailbaseCollectionOptions (TrailBase), + rxdbCollectionOptions (RxDB), trailBaseCollectionOptions (TrailBase), localOnlyCollectionOptions, localStorageCollectionOptions. CollectionConfig options: getKey, schema, sync, gcTime, autoIndex (default off), defaultIndexType, syncMode (eager/on-demand, plus progressive for Electric). StandardSchema validation with Zod/Valibot/ArkType. Collection lifecycle (idle/loading/ready/error). Adapter-specific sync patterns including Electric txid tracking, Query direct - writes, and PowerSync query-driven sync with onLoad/onLoadSubset hooks. + writes, Query initial data and scoped factories, and PowerSync query-driven + sync with onLoad/onLoadSubset hooks. type: sub-skill library: db -library_version: '0.6.0' +library_version: '0.6.17' sources: - 'TanStack/db:docs/overview.md' - 'TanStack/db:docs/guides/schemas.md' @@ -80,7 +81,7 @@ const todoCollection = createCollection( | ElectricSQL (real-time Postgres) | `electricCollectionOptions` | `@tanstack/electric-db-collection` | | PowerSync (SQLite offline) | `powerSyncCollectionOptions` | `@tanstack/powersync-db-collection` | | RxDB (reactive database) | `rxdbCollectionOptions` | `@tanstack/rxdb-db-collection` | -| TrailBase (event streaming) | `trailbaseCollectionOptions` | `@tanstack/trailbase-db-collection` | +| TrailBase (event streaming) | `trailBaseCollectionOptions` | `@tanstack/trailbase-db-collection` | | No backend (UI state) | `localOnlyCollectionOptions` | `@tanstack/db` | | Browser localStorage | `localStorageCollectionOptions` | `@tanstack/db` | @@ -218,7 +219,10 @@ queryCollectionOptions({ }) ``` -`queryFn` result is treated as complete server state. Returning `[]` means "server has no items", deleting all existing collection data. +In eager mode, `queryFn` is complete collection state. Returning `[]` means +"the server has no items" and removes all rows. In on-demand mode, a result is +complete only for that exact subset/Query key; an empty result releases that +subset's ownership, while overlapping subsets can keep shared rows. Source: docs/collections/query-collection.md @@ -309,7 +313,9 @@ queryCollectionOptions({ }) ``` -`queryFn` result replaces all collection data. For incremental fetches, merge with existing data. +An eager `queryFn` result replaces all collection data. For incremental eager +fetches, merge with existing data. In on-demand mode, return the complete state +for the requested subset instead. Source: docs/collections/query-collection.md @@ -390,11 +396,20 @@ When a schema transforms types, `TInput` must accept both the pre-transform and Source: docs/guides/schemas.md -### HIGH React Native missing crypto.randomUUID polyfill +### HIGH Runtime has no secure random number generator -TanStack DB uses `crypto.randomUUID()` internally. React Native doesn't provide this. Install `react-native-random-uuid` and import it at your app entry point. +TanStack DB's `safeRandomUUID()` uses `crypto.randomUUID()` when available and +falls back to `crypto.getRandomValues()`, including on non-secure HTTP origins. +Add a Web Crypto polyfill only in runtimes, including some React Native +versions, that provide neither API. -Source: docs/overview.md +```ts +import { safeRandomUUID } from '@tanstack/db' + +collection.insert({ id: safeRandomUUID(), text: 'New item' }) +``` + +Source: packages/db/src/utils/uuid.ts, packages/db/tests/uuid.test.ts ### MEDIUM Providing both explicit type parameter and schema diff --git a/packages/db/skills/db-core/collection-setup/references/electric-adapter.md b/packages/db/skills/db-core/collection-setup/references/electric-adapter.md index 38f0e07b20..5886791c35 100644 --- a/packages/db/skills/db-core/collection-setup/references/electric-adapter.md +++ b/packages/db/skills/db-core/collection-setup/references/electric-adapter.md @@ -30,10 +30,23 @@ const collection = createCollection( | `id` | (none) | Unique collection identifier | | `schema` | (none) | StandardSchema validator | | `shapeOptions.params` | (none) | Additional shape params (e.g. `{ table: 'todos' }`) | +| `syncMode` | `eager` | `eager`, `on-demand`, or `progressive` | | `onInsert` | (none) | Persistence handler; should return `{ txid }` | | `onUpdate` | (none) | Persistence handler; should return `{ txid }` | | `onDelete` | (none) | Persistence handler; should return `{ txid }` | +## Sync Modes + +- `eager` loads the full shape before use. +- `on-demand` loads only the subsets requested by live queries. +- `progressive` loads the requested subset first, then completes the full sync + in the background. + +With SQLite persistence, progressive resume keeps hydrated rows visible while +the full Electric stream resumes. On-demand subset snapshots use a bounded +refresh wait, so a stalled native long-poll refresh does not block loading +forever. + ## Three Sync Strategies ### 1. Txid Return (Recommended) diff --git a/packages/db/skills/db-core/collection-setup/references/local-adapters.md b/packages/db/skills/db-core/collection-setup/references/local-adapters.md index cc3ffea28e..f145aa8aad 100644 --- a/packages/db/skills/db-core/collection-setup/references/local-adapters.md +++ b/packages/db/skills/db-core/collection-setup/references/local-adapters.md @@ -24,19 +24,18 @@ import { const collection = createCollection( localOnlyCollectionOptions({ - id: 'ui-state', getKey: (item) => item.id, }), ) ``` -- `id` -- unique collection identifier - `getKey` -- extracts unique key from each item ### Optional Config | Option | Default | Description | | ------------- | ------- | -------------------------------------- | +| `id` | UUID | Unique collection identifier | | `schema` | (none) | StandardSchema validator | | `initialData` | (none) | Array of items to populate on creation | | `onInsert` | (none) | Handler before confirming inserts | @@ -101,27 +100,26 @@ import { const collection = createCollection( localStorageCollectionOptions({ - id: 'user-preferences', storageKey: 'app-user-prefs', getKey: (item) => item.id, }), ) ``` -- `id` -- unique collection identifier - `storageKey` -- localStorage key for all collection data - `getKey` -- extracts unique key from each item ### Optional Config -| Option | Default | Description | -| ----------------- | -------------- | -------------------------------------------------------------------- | -| `schema` | (none) | StandardSchema validator | -| `storage` | `localStorage` | Custom storage (`sessionStorage` or any localStorage-compatible API) | -| `storageEventApi` | `window` | Event API for cross-tab sync | -| `onInsert` | (none) | Handler on insert | -| `onUpdate` | (none) | Handler on update | -| `onDelete` | (none) | Handler on delete | +| Option | Default | Description | +| ----------------- | ---------------- | -------------------------------------------------------------------- | +| `id` | From storage key | `local-collection:${storageKey}` | +| `schema` | (none) | StandardSchema validator | +| `storage` | `localStorage` | Custom storage (`sessionStorage` or any localStorage-compatible API) | +| `storageEventApi` | `window` | Event API for cross-tab sync | +| `onInsert` | (none) | Handler on insert | +| `onUpdate` | (none) | Handler on update | +| `onDelete` | (none) | Handler on delete | ### Using sessionStorage diff --git a/packages/db/skills/db-core/collection-setup/references/powersync-adapter.md b/packages/db/skills/db-core/collection-setup/references/powersync-adapter.md index 8a61414c88..72e6b6e82a 100644 --- a/packages/db/skills/db-core/collection-setup/references/powersync-adapter.md +++ b/packages/db/skills/db-core/collection-setup/references/powersync-adapter.md @@ -9,7 +9,7 @@ pnpm add @tanstack/powersync-db-collection @powersync/web @journeyapps/wa-sqlite ## Required Config ```typescript -import { createCollection } from '@tanstack/react-db' +import { createCollection, safeRandomUUID } from '@tanstack/react-db' import { powerSyncCollectionOptions } from '@tanstack/powersync-db-collection' import { Schema, Table, column, PowerSyncDatabase } from '@powersync/web' @@ -164,7 +164,7 @@ const APP_SCHEMA = new Schema({ }) await collection.insert( - { id: crypto.randomUUID(), name: 'Report' }, + { id: safeRandomUUID(), name: 'Report' }, { metadata: { source: 'web-app', userId: 'user-123' } }, ).isPersisted.promise ``` @@ -187,7 +187,7 @@ const tx = createTransaction({ }) tx.mutate(() => { documentsCollection.insert({ - id: crypto.randomUUID(), + id: safeRandomUUID(), name: 'Doc 1', created_at: new Date().toISOString(), }) diff --git a/packages/db/skills/db-core/collection-setup/references/query-adapter.md b/packages/db/skills/db-core/collection-setup/references/query-adapter.md index b5d31c3563..3f0b07f08e 100644 --- a/packages/db/skills/db-core/collection-setup/references/query-adapter.md +++ b/packages/db/skills/db-core/collection-setup/references/query-adapter.md @@ -32,19 +32,28 @@ const collection = createCollection( ## Optional Config (with defaults) -| Option | Default | Description | -| ----------------- | ------------ | ----------------------------------------------- | -| `id` | (none) | Unique collection identifier | -| `schema` | (none) | StandardSchema validator | -| `select` | (none) | Extracts array items when wrapped with metadata | -| `enabled` | `true` | Whether query runs automatically | -| `refetchInterval` | `0` | Polling interval in ms; 0 = disabled | -| `retry` | (TQ default) | Retry config for failed queries | -| `retryDelay` | (TQ default) | Delay between retries | -| `staleTime` | (TQ default) | How long data is considered fresh | -| `meta` | (none) | Metadata passed to queryFn context | -| `startSync` | `true` | Start syncing immediately | -| `syncMode` | (none) | Set `"on-demand"` for predicate push-down | +| Option | Default | Description | +| ---------------------- | ------------ | ----------------------------------------------- | +| `id` | (none) | Unique collection identifier | +| `schema` | (none) | StandardSchema validator | +| `select` | (none) | Extracts rows from the original response shape | +| `enabled` | `true` | Whether query runs automatically | +| `refetchInterval` | (TQ default) | Polling interval | +| `retry` / `retryDelay` | (TQ default) | Retry policy | +| `staleTime` / `gcTime` | (TQ default) | Freshness and unused-cache retention | +| `refetchOnWindowFocus` | (TQ default) | Refetch when the window regains focus | +| `refetchOnReconnect` | (TQ default) | Refetch after reconnecting | +| `refetchOnMount` | (TQ default) | Refetch when the observer mounts | +| `networkMode` | (TQ default) | TanStack Query network mode | +| `initialData` | (none) | Initial response for eager collections | +| `initialDataUpdatedAt` | (none) | Timestamp used to judge initial-data freshness | +| `meta` | (none) | Metadata merged into the query function context | +| `startSync` | `true` | Start syncing immediately | +| `syncMode` | `eager` | Set `"on-demand"` for predicate push-down | + +Query Client defaults apply when these pass-through fields are omitted. +`placeholderData` is intentionally unsupported: it is observer-local UI state, +not cache data, so it must not become collection-wide rows. ### Persistence Handlers @@ -81,6 +90,124 @@ collection.utils.writeBatch(() => { }) ``` +## Response Shape, Initial Data, and Query Options + +`select` is a Query Collection row-extraction hook, not TanStack Query's +observer-level projection. The Query cache keeps the original response while +the collection materializes the returned row array: + +```typescript +const collection = createCollection( + queryCollectionOptions({ + queryKey: ['todos'], + queryFn: fetchTodosResponse, + initialData: { + items: [{ id: '1', title: 'Initial todo' }], + total: 1, + }, + initialDataUpdatedAt: Date.now(), + staleTime: 60_000, + select: (response) => response.items, + queryClient, + getKey: (todo) => todo.id, + }), +) +``` + +The same `select` applies to fetched and initial responses. Cached or hydrated +data for the same exact Query key takes precedence over a later `initialData` +value. `initialData` is supported only in eager mode; seed the exact derived +Query cache entries for on-demand subsets. + +Passing either `initialData` or `initialDataUpdatedAt` in on-demand mode throws +`InitialDataInOnDemandModeError`. When stale initial data triggers a fetch, the +rows remain visible while it runs. A failed fetch retains them; a successful +fetch reconciles them normally. + +Direct writes can preserve simple wrappers such as `{ data: [...] }`, +`{ items: [...] }`, or `{ results: [...] }`. For a derived projection such as +`response.edges.map((edge) => edge.node)`, refetch or invalidate when the +wrapped cache must reflect the write exactly. + +You may spread compatible `queryOptions(...)` output into +`queryCollectionOptions`, but provide `queryFn` explicitly. Do not pass a +TanStack Query observer-level `select`; Query Collection gives that name the +row-extraction contract above. + +## Runtime QueryClient and Business Scopes + +When a `QueryClient` is request-, router-, tenant-, or test-scoped, put shared +options in a factory and create one stable collection per client and business +scope: + +```typescript +function createProjectTodosCollection( + queryClient: QueryClient, + projectId: string, +) { + return createCollection( + queryCollectionOptions({ + queryKey: ['projects', projectId, 'todos'], + queryFn: () => fetchProjectTodos(projectId), + queryClient, + getKey: (todo) => todo.id, + }), + ) +} + +type ProjectTodosCollection = ReturnType + +const projectCollections = new WeakMap< + QueryClient, + Map +>() + +export function getProjectTodosCollection( + queryClient: QueryClient, + projectId: string, +): ProjectTodosCollection { + let collectionsByProject = projectCollections.get(queryClient) + if (!collectionsByProject) { + collectionsByProject = new Map() + projectCollections.set(queryClient, collectionsByProject) + } + + let collection = collectionsByProject.get(projectId) + if (!collection) { + collection = createProjectTodosCollection(queryClient, projectId) + collectionsByProject.set(projectId, collection) + } + + return collection +} + +export async function removeProjectTodosCollection( + queryClient: QueryClient, + projectId: string, +): Promise { + const collectionsByProject = projectCollections.get(queryClient) + if (!collectionsByProject) return + + const collection = collectionsByProject.get(projectId) + if (!collection) return + + collectionsByProject.delete(projectId) + if (collectionsByProject.size === 0) { + projectCollections.delete(queryClient) + } + await collection.cleanup() +} +``` + +Memoize by `QueryClient` and every scope value. Do not create the collection +during every render or in each consumer. Clean up and remove unused entries +from long-lived scope maps when your application owns their lifecycle. + +A business scope names a distinct server resource. A relational subset +(`where`, `orderBy`, `limit`, or `offset`) stays within that collection and, in +on-demand mode, reaches `queryFn` as `ctx.meta.loadSubsetOptions`. Do not create +a collection for each relational subset. + ## Request Cancellation and Cleanup TanStack Query passes an `AbortSignal` through the query function context. @@ -108,6 +235,14 @@ Query cache entries are shared within a `QueryClient`. Explicit cleanup can cancel or remove entries used by another collection or Query consumer with the same exact key. +## Query Invalidation + +Exact-key and prefix invalidation refetch active eager and on-demand queries, +then rematerialize their results. Overlapping subsets keep rows that another +active subset still owns. A failed refetch retains current rows and records the +error in `collection.utils.lastError`. Cleaned-up or otherwise inactive queries +do not rematerialize. + ## Predicate Push-Down (syncMode: "on-demand") Query predicates (where, orderBy, limit, offset) passed to `queryFn` via `ctx.meta.loadSubsetOptions`. @@ -240,8 +375,10 @@ When using a function-based `queryKey`, all derived keys must share the base key ## Key Behaviors -- `queryFn` result is treated as **complete state** -- missing items are deleted -- Empty array from `queryFn` deletes all items +- In eager mode, each `queryFn` result is complete collection state +- In on-demand mode, it is complete state for that exact subset/Query key +- An empty subset removes that subset's ownership; overlapping subsets can keep + the same rows materialized - Direct writes update TQ cache but are overridden by subsequent `queryFn` results - Persistence handlers automatically refetch unless they return `{ refetch: false }` - On-demand `collection.preload()` is a no-op; preload the live query instead diff --git a/packages/db/skills/db-core/collection-setup/references/rxdb-adapter.md b/packages/db/skills/db-core/collection-setup/references/rxdb-adapter.md index fcdcf84b02..18147bb70e 100644 --- a/packages/db/skills/db-core/collection-setup/references/rxdb-adapter.md +++ b/packages/db/skills/db-core/collection-setup/references/rxdb-adapter.md @@ -23,15 +23,15 @@ const todosCollection = createCollection( ## Optional Config (with defaults) -| Option | Default | Description | -| --------------- | ----------------------- | -------------------------------------------------------------------------------------------------- | -| `id` | (none) | Unique collection identifier | -| `schema` | (none) | StandardSchema validator (RxDB has its own validation; this adds TanStack DB-side validation) | -| `startSync` | `true` | Start ingesting RxDB data immediately | -| `syncBatchSize` | `1000` | Max documents per batch during initial sync from RxDB; only affects initial load, not live updates | -| `onInsert` | (default: `bulkUpsert`) | Override default insert persistence | -| `onUpdate` | (default: `patch`) | Override default update persistence | -| `onDelete` | (default: `bulkRemove`) | Override default delete persistence | +| Option | Default | Description | +| --------------- | ------- | -------------------------------------------------------------------------------------------------- | +| `id` | (none) | Unique collection identifier | +| `schema` | (none) | StandardSchema validator (RxDB has its own validation; this adds TanStack DB-side validation) | +| `startSync` | `false` | Start ingesting RxDB data immediately | +| `syncBatchSize` | `1000` | Max documents per batch during initial sync from RxDB; only affects initial load, not live updates | + +The adapter owns `onInsert`, `onUpdate`, and `onDelete` so writes persist to +RxDB. Those handlers cannot be overridden in `RxDBCollectionConfig`. ## Key Behavior: String Keys @@ -96,7 +96,7 @@ RxDB schema indexes do not affect TanStack DB query performance (queries run in- ```typescript import { createRxDatabase } from 'rxdb/plugins/core' import { getRxStorageLocalstorage } from 'rxdb/plugins/storage-localstorage' -import { createCollection } from '@tanstack/react-db' +import { createCollection, safeRandomUUID } from '@tanstack/react-db' import { rxdbCollectionOptions } from '@tanstack/rxdb-db-collection' import { z } from 'zod' @@ -141,7 +141,7 @@ const todosCollection = createCollection( // Usage todosCollection.insert({ - id: crypto.randomUUID(), + id: safeRandomUUID(), text: 'Buy milk', completed: false, }) diff --git a/packages/db/skills/db-core/collection-setup/references/trailbase-adapter.md b/packages/db/skills/db-core/collection-setup/references/trailbase-adapter.md index a01b29ee6b..0ded8f17f7 100644 --- a/packages/db/skills/db-core/collection-setup/references/trailbase-adapter.md +++ b/packages/db/skills/db-core/collection-setup/references/trailbase-adapter.md @@ -17,27 +17,27 @@ const trailBaseClient = initClient('https://your-trailbase-instance.com') const todosCollection = createCollection( trailBaseCollectionOptions({ - id: 'todos', recordApi: trailBaseClient.records('todos'), getKey: (item) => item.id, + parse: {}, + serialize: {}, }), ) ``` -- `id` -- unique collection identifier - `recordApi` -- TrailBase Record API instance from `trailBaseClient.records(tableName)` - `getKey` -- extracts unique key from each item +- `parse` -- field conversions from TrailBase records to collection rows +- `serialize` -- field conversions from collection rows to TrailBase records + +Use empty objects for `parse` and `serialize` when both shapes are identical. ## Optional Config -| Option | Default | Description | -| ----------- | ------- | --------------------------------------------------------------------------------- | -| `schema` | (none) | StandardSchema validator | -| `parse` | (none) | Object mapping field names to functions that transform data coming FROM TrailBase | -| `serialize` | (none) | Object mapping field names to functions that transform data going TO TrailBase | -| `onInsert` | (none) | Handler called on insert | -| `onUpdate` | (none) | Handler called on update | -| `onDelete` | (none) | Handler called on delete | +| Option | Default | Description | +| ---------- | ------- | ---------------------------- | +| `id` | (none) | Unique collection identifier | +| `syncMode` | `eager` | `eager` or `on-demand` | ## Conversions (parse/serialize) @@ -58,8 +58,8 @@ type Todo = { completed: boolean } -const collection = createCollection( - trailBaseCollectionOptions({ +const collection = createCollection( + trailBaseCollectionOptions({ id: 'todos', recordApi: trailBaseClient.records('todos'), getKey: (item) => item.id, @@ -79,36 +79,28 @@ Automatic when `enable_subscriptions` is enabled on the TrailBase server. No add ## Persistence Handlers -```typescript -onInsert: async ({ transaction }) => { - const newItem = transaction.mutations[0].modified -}, -onUpdate: async ({ transaction }) => { - const { original, modified } = transaction.mutations[0] -}, -onDelete: async ({ transaction }) => { - const deletedItem = transaction.mutations[0].original -}, -``` +TrailBase owns `onInsert`, `onUpdate`, and `onDelete`. The adapter writes +through the Record API and waits until subscription events confirm the affected +IDs before removing the optimistic overlay. Custom mutation handlers and +`schema` are not part of `TrailBaseCollectionConfig`. -TrailBase handles persistence through the Record API automatically. Custom handlers are for additional logic only. +Call `collection.utils.cancel()` to cancel the active TrailBase event reader. ## Complete Example ```typescript -import { createCollection } from '@tanstack/react-db' +import { createCollection, safeRandomUUID } from '@tanstack/react-db' import { trailBaseCollectionOptions } from '@tanstack/trailbase-db-collection' import { initClient } from 'trailbase' -import { z } from 'zod' const trailBaseClient = initClient('https://your-trailbase-instance.com') -const todoSchema = z.object({ - id: z.string(), - text: z.string(), - completed: z.boolean(), - created_at: z.date(), -}) +type Todo = { + id: string + text: string + completed: boolean + created_at: Date +} type SelectTodo = { id: string @@ -117,29 +109,23 @@ type SelectTodo = { created_at: number } -type Todo = z.infer - -const todosCollection = createCollection( - trailBaseCollectionOptions({ +const todosCollection = createCollection( + trailBaseCollectionOptions({ id: 'todos', recordApi: trailBaseClient.records('todos'), getKey: (item) => item.id, - schema: todoSchema, parse: { created_at: (ts) => new Date(ts * 1000), }, serialize: { created_at: (date) => Math.floor(date.valueOf() / 1000), }, - onInsert: async ({ transaction }) => { - console.log('Created:', transaction.mutations[0].modified) - }, }), ) // Usage todosCollection.insert({ - id: crypto.randomUUID(), + id: safeRandomUUID(), text: 'Review PR', completed: false, created_at: new Date(), diff --git a/packages/db/skills/db-core/custom-adapter/SKILL.md b/packages/db/skills/db-core/custom-adapter/SKILL.md index 8304c75e23..1d6bb38d38 100644 --- a/packages/db/skills/db-core/custom-adapter/SKILL.md +++ b/packages/db/skills/db-core/custom-adapter/SKILL.md @@ -3,16 +3,18 @@ name: db-core/custom-adapter description: > Building custom collection adapters for new backends. SyncConfig interface: sync function receiving begin, write, commit, markReady, truncate, metadata - primitives. ChangeMessage format (insert, update, delete). loadSubset for - on-demand sync. LoadSubsetOptions (where, orderBy, limit, cursor). Expression - parsing: parseWhereExpression, parseOrderByExpression, + primitives and returning cleanup, loadSubset, and optional unloadSubset + handlers. + ChangeMessage format (insert, update, delete). On-demand LoadSubsetOptions + (where, orderBy, limit, offset, cursor). Expression parsing: + parseWhereExpression, parseOrderByExpression, extractSimpleComparisons, parseLoadSubsetOptions. Collection options creator pattern. rowUpdateMode (partial vs full). Subscription lifecycle and cleanup functions. Persisted sync metadata API (metadata.row and metadata.collection) for storing per-row and per-collection adapter state. type: sub-skill library: db -library_version: '0.6.0' +library_version: '0.6.17' sources: - 'TanStack/db:docs/guides/collection-options-creator.md' - 'TanStack/db:packages/db/src/collection/sync.ts' @@ -26,23 +28,29 @@ This skill builds on db-core and db-core/collection-setup. Read those first. ```ts import { createCollection } from '@tanstack/db' -import type { SyncConfig, CollectionConfig } from '@tanstack/db' +import type { CollectionConfig } from '@tanstack/db' interface MyItem { id: string name: string } -function myBackendCollectionOptions(config: { +interface BackendEvent { + type: 'insert' | 'update' | 'delete' + id: string + data: T +} + +function myBackendCollectionOptions(config: { endpoint: string getKey: (item: T) => string -}): CollectionConfig { +}): CollectionConfig { return { getKey: config.getKey, sync: { - sync: ({ begin, write, commit, markReady, metadata, collection }) => { + sync: ({ begin, write, commit, markReady }) => { let isInitialSyncComplete = false - const bufferedEvents: Array = [] + const bufferedEvents: Array> = [] // 1. Subscribe to real-time events FIRST const unsubscribe = myWebSocket.subscribe(config.endpoint, (event) => { @@ -84,22 +92,28 @@ function myBackendCollectionOptions(config: { rowUpdateMode: 'partial', }, onInsert: async ({ transaction }) => { - await fetch(config.endpoint, { + const response = await fetch(config.endpoint, { method: 'POST', body: JSON.stringify(transaction.mutations[0].modified), }) + await waitForServerObservation(response) }, onUpdate: async ({ transaction }) => { const mut = transaction.mutations[0] - await fetch(`${config.endpoint}/${mut.key}`, { + const response = await fetch(`${config.endpoint}/${mut.key}`, { method: 'PATCH', body: JSON.stringify(mut.changes), }) + await waitForServerObservation(response) }, onDelete: async ({ transaction }) => { - await fetch(`${config.endpoint}/${transaction.mutations[0].key}`, { - method: 'DELETE', - }) + const response = await fetch( + `${config.endpoint}/${transaction.mutations[0].key}`, + { + method: 'DELETE', + }, + ) + await waitForServerObservation(response) }, } } @@ -127,28 +141,54 @@ write({ type: 'delete', key: itemId, value: item }) ### On-demand sync with loadSubset ```ts -import { parseLoadSubsetOptions } from "@tanstack/db" +import { parseLoadSubsetOptions } from '@tanstack/db' +syncMode: 'on-demand', sync: { - sync: ({ begin, write, commit, markReady }) => { - // Initial sync... + sync: ({ begin, write, commit, markReady, collection }) => { + const stopSync = subscribeToBackendChanges() markReady() - return () => {} - }, - loadSubset: async (options) => { - const { filters, sorts, limit, offset } = parseLoadSubsetOptions(options) - // filters: [{ field: ['category'], operator: 'eq', value: 'electronics' }] - // sorts: [{ field: ['price'], direction: 'asc', nulls: 'last' }] - const params = new URLSearchParams() - for (const f of filters) { - params.set(f.field.join("."), `${f.operator}:${f.value}`) + + return { + cleanup: stopSync, + loadSubset: async (options) => { + const { filters, sorts, limit } = parseLoadSubsetOptions(options) + const items = await api.items.list({ + filters, + sorts, + limit, + offset: options.offset, + // Translate cursor.whereFrom/whereCurrent expressions for your API. + cursor: translateCursorExpressions(options.cursor), + }) + + begin() + for (const item of items) { + const key = collection.config.getKey(item) + write( + collection.has(key) + ? { type: 'update', key, value: item } + : { type: 'insert', value: item }, + ) + } + commit() + }, } - const res = await fetch(`/api/items?${params}`) - return res.json() }, + rowUpdateMode: 'full', } ``` +`sync()` returns the handlers in a `SyncConfigRes` object. `loadSubset()` must +write fetched rows through `begin()` → `write()` → `commit()` and resolve +`void` (or return `true` for an immediate synchronous result); it does not +return the fetched rows. `parseLoadSubsetOptions()` returns only `filters`, +`sorts`, and `limit`. Read `offset` and `cursor` from the original options. +`cursor` contains query expressions (`whereFrom` and `whereCurrent`), not an +opaque backend cursor; translate or combine those expressions for your API. +Return `unloadSubset` only when `loadSubset` creates an ongoing resource, such +as a per-subset server subscription, that must be released. + ### Managing optimistic state duration Mutation handlers must not resolve until server changes have synced back to the collection. Five strategies: @@ -163,10 +203,16 @@ Mutation handlers must not resolve until server changes have synced back to the The `metadata` API on the sync config allows adapters to store per-row and per-collection metadata that persists across sync transactions. This is useful for tracking resume tokens, cursors, LSNs, or other adapter-specific state. -The `metadata` object is available as a property on the sync config argument alongside `begin`, `write`, `commit`, etc. It is always provided, but without persistence the metadata is in-memory only and does not survive reloads. With persistence, metadata is durable across sessions. +The `metadata` object is available on the sync config argument alongside +`begin`, `write`, and `commit`. Core supplies it at runtime, but its public type +is optional, so strict TypeScript code must guard it or assert its presence. +Without persistence the metadata is in-memory only and does not survive +reloads. With persistence, it is durable across sessions. ```ts sync: ({ begin, write, commit, markReady, metadata }) => { + if (!metadata) throw new Error('Sync metadata API is unavailable') + // Row metadata: store per-row state (e.g. server version, ETag) metadata.row.get(key) // => unknown | undefined metadata.row.set(key, { version: 3, etag: 'abc' }) @@ -181,7 +227,11 @@ sync: ({ begin, write, commit, markReady, metadata }) => { } ``` -Row metadata writes are tied to the current transaction. When a row is deleted via `write({ type: 'delete', ... })`, its row metadata is automatically deleted. When a row is inserted, its metadata is set from `message.metadata` if provided, or deleted otherwise. +Row metadata writes are tied to the current transaction. Deleting a row also +deletes its metadata. An insert sets metadata from `message.metadata`. A +metadata-less insert deletes stale metadata unless `metadata.row.set()` already +queued an explicit value for that key in the same transaction; that queued +value wins. Collection metadata writes staged before `truncate()` are preserved and commit atomically with the truncate transaction. @@ -189,6 +239,8 @@ Collection metadata writes staged before `truncate()` are preserved and commit a ```ts sync: ({ begin, write, commit, markReady, metadata }) => { + if (!metadata) throw new Error('Sync metadata API is unavailable') + const lastCursor = metadata.collection.get('cursor') as string | undefined const stream = subscribeFromCursor(lastCursor) @@ -225,6 +277,21 @@ const orderBy = parseOrderByExpression(options.orderBy) ## Common Mistakes +### CRITICAL Defining loadSubset beside sync() + +Wrong: + +```ts +sync: { + sync: ({ markReady }) => markReady(), + loadSubset: async () => fetch('/items').then((response) => response.json()), +} +``` + +Correct: return `{ loadSubset, cleanup }` from `sync()` and apply loaded rows +with the sync transaction primitives, as shown above. Add `unloadSubset` when +each loaded subset owns a resource that must be released. + ### CRITICAL Not calling markReady() in sync implementation Wrong: @@ -327,6 +394,14 @@ Sync data must be written within a transaction (`begin` → `write` → `commit` Source: packages/db/src/collection/sync.ts:110 +### HIGH Inserting a different value for an existing synced key + +An `insert` for an existing synced key is normalized to an update only when +the value is unchanged. A different value throws `DuplicateKeySyncError`, +including for plain custom configs with no `utils`. + +Emit an `update`, or delete/truncate the old row before inserting the new one. + ## Tension: Simplicity vs. Correctness in Sync Getting-started simplicity (localOnly, eager mode) conflicts with production correctness (on-demand sync, race condition prevention, proper markReady handling). Agents optimizing for quick setup tend to skip buffering, markReady, and cleanup functions. diff --git a/packages/db/skills/db-core/live-queries/SKILL.md b/packages/db/skills/db-core/live-queries/SKILL.md index fe55967ca9..00da8635ff 100644 --- a/packages/db/skills/db-core/live-queries/SKILL.md +++ b/packages/db/skills/db-core/live-queries/SKILL.md @@ -5,16 +5,18 @@ description: > fullJoin, select, fn.select, groupBy, having, orderBy, limit, offset, distinct, findOne. Operators: eq, gt, gte, lt, lte, like, ilike, inArray, isNull, isUndefined, and, or, not. Aggregates: count, sum, avg, min, max. String - functions: upper, lower, length, concat. Utility: coalesce, caseWhen. Math: add. + functions: upper, lower, length, concat. Utility: coalesce, caseWhen. Math: + add, subtract, multiply, divide. $selected namespace. createLiveQueryCollection. Derived collections. Predicate push-down. Incremental view maintenance via differential dataflow (d2ts). Virtual properties ($synced, $origin, $key, $collectionId). Includes subqueries - for hierarchical data. toArray and concat(toArray(...)) scalar includes. + for hierarchical data. Collection, toArray, materialize, and + concat(toArray(...)) include modes. queryOnce for one-shot queries. createEffect for reactive side effects (onEnter, onUpdate, onExit, onBatch). type: sub-skill library: db -library_version: '0.6.0' +library_version: '0.6.17' sources: - 'TanStack/db:docs/guides/live-queries.md' - 'TanStack/db:packages/db/src/query/builder/index.ts' @@ -106,6 +108,11 @@ Boolean column references work directly: .where(({ user }) => not(user.suspended)) // negated boolean ref ``` +Comparisons follow PostgreSQL semantics. Comparisons involving `null` or +`undefined` are unknown and do not match; use `isNull()` or `isUndefined()`. +`NaN` (and an invalid `Date`) equals itself and sorts after every other +non-null value. + ### 2. Joining two collections Join conditions **must** use `eq()` (equality only -- IVM constraint). Default join type is `left`. Convenience methods: `leftJoin`, `rightJoin`, `innerJoin`, `fullJoin`. @@ -195,11 +202,16 @@ const activeUserPosts = createLiveQueryCollection((q) => Create derived collections once at module scope and reuse them. Do not recreate on every render or navigation. +Live query collections default to `gcTime: 5_000`. An explicit `gcTime: 0` is +preserved and disables garbage collection for that derived collection. + ## Virtual Properties Live query results include computed, read-only virtual properties on every row: -- `$synced`: `true` when the row is confirmed by sync; `false` when it is still optimistic. +- `$synced`: `true` when no pending local optimistic write affects the row; + `false` while one does. This is local mutation status, not proof that a + backend uploaded, confirmed, or read back the row. - `$origin`: `"local"` if the last confirmed change came from this client, otherwise `"remote"`. - `$key`: the row key for the result. - `$collectionId`: the source collection ID. @@ -208,7 +220,9 @@ These props are added automatically and can be used in `where`, `select`, and `o ## Includes (Subqueries in Select) -Embed a correlated subquery inside `select()` to produce hierarchical (nested) data. The subquery must contain a `where` with an `eq()` that correlates a parent field with a child field. Three materialization modes are available. +Embed a correlated subquery inside `select()` to produce hierarchical (nested) +data. The subquery must contain a `where` with an `eq()` that correlates a +parent field with a child field. ### Collection includes (default) @@ -239,7 +253,7 @@ for (const project of projectsWithIssues) { ### Array includes with toArray() -Wrap the subquery in `toArray()` to get a plain array of scalar values instead of a Collection: +Wrap the subquery in `toArray()` to get a plain array instead of a Collection: ```ts import { eq, toArray, createLiveQueryCollection } from '@tanstack/db' @@ -259,6 +273,32 @@ const messagesWithParts = createLiveQueryCollection((q) => // row.contentParts is string[] ``` +### Plain values with materialize() + +Use `materialize()` when the parent row should hold a plain snapshot rather +than a child collection: + +```ts +import { eq, materialize, createLiveQueryCollection } from '@tanstack/db' + +const issuesWithProject = createLiveQueryCollection((q) => + q.from({ issue: issuesCollection }).select(({ issue }) => ({ + ...issue, + project: materialize( + q + .from({ project: projectsCollection }) + .where(({ project }) => eq(project.id, issue.projectId)) + .findOne(), + ), + })), +) +// row.project is Project | undefined +``` + +For a multi-row subquery, `materialize()` returns `Array` like `toArray()`. +For a subquery ending in `findOne()`, it returns `T | undefined`. In both cases, +the parent row is re-emitted when the child result changes. + ### Concatenated scalar with concat(toArray()) Wrap `toArray()` in `concat()` to join the scalar results into a single string: @@ -287,6 +327,9 @@ const messagesWithContent = createLiveQueryCollection((q) => - The subquery **must** have a `where` clause with an `eq()` correlating a parent alias with a child alias. The library extracts this automatically as the join condition. - `toArray()` works with both scalar selects (e.g., `select(({ c }) => c.text)` → `string[]`) and object selects (e.g., `select(({ c }) => ({ id: c.id, title: c.title }))` → `Array<{id, title}>`). +- `materialize()` returns an array, or one value for a `findOne()` subquery. + Like `toArray()`, it must be a top-level value in `select()` and cannot be + nested inside `coalesce()`, `eq()`, or another expression. - `concat(toArray())` requires a **scalar** `select` to concatenate into a string. - Collection includes (bare subquery) require an **object** `select`. - Includes subqueries are compiled into the same incremental pipeline as the parent query -- they are not separate live queries. @@ -385,7 +428,10 @@ const { data } = useLiveQuery((q) => ### HIGH: Not using the full operator set -The library provides string functions (`upper`, `lower`, `length`, `concat`), math (`add`), utility functions (`coalesce`, `caseWhen`), and aggregates (`count`, `sum`, `avg`, `min`, `max`). All are incrementally maintained. Prefer them over JS equivalents. +The library provides string functions (`upper`, `lower`, `length`, `concat`), +math (`add`, `subtract`, `multiply`, `divide`), utility functions (`coalesce`, +`caseWhen`), and aggregates (`count`, `sum`, `avg`, `min`, `max`). All are +incrementally maintained. Prefer them over JS equivalents. ```ts // WRONG @@ -402,6 +448,11 @@ The library provides string functions (`upper`, `lower`, `length`, `concat`), ma })) ``` +Math expressions also work in `orderBy()`. When a computed expression is used +with `limit()`, lazy-loading optimization is skipped and all matching rows load +before sorting. Literal values such as `Date.now()` are captured when the query +is created; recreate the query when the value must advance. + ### HIGH: Missing conditional expression helpers Use `coalesce()` for null/undefined fallbacks and `caseWhen()` for conditional @@ -506,6 +557,12 @@ q.from(usersCollection) q.from({ users: usersCollection }) ``` +### MEDIUM: Using unsafe select alias paths + +Select alias path segments named `__proto__`, `prototype`, or `constructor` +throw `UnsafeAliasPathError`. Use ordinary data-field names; do not suppress +this prototype-pollution guard. + ## Tension: Query expressiveness vs. IVM constraints The query builder looks like SQL but has constraints that SQL does not: diff --git a/packages/db/skills/db-core/live-queries/references/operators.md b/packages/db/skills/db-core/live-queries/references/operators.md index be494de0d0..8d4082ea5c 100644 --- a/packages/db/skills/db-core/live-queries/references/operators.md +++ b/packages/db/skills/db-core/live-queries/references/operators.md @@ -32,6 +32,9 @@ import { concat, // Math add, + subtract, + multiply, + divide, // Utility coalesce, } from '@tanstack/db' @@ -112,6 +115,17 @@ Check if value is `undefined` (absent). Especially useful after left joins where isUndefined(profile) // no matching profile in left join ``` +### Comparison semantics + +Comparisons involving `null` or `undefined` evaluate as unknown and do not +match. Use `isNull()` or `isUndefined()` instead of `eq(value, null)` or +`eq(value, undefined)`. + +`NaN` follows PostgreSQL rather than JavaScript semantics: it equals itself and +is greater than every other non-null value. Invalid `Date` values behave the +same way. This applies to equality, `inArray()`, range comparisons, and +ordering. + --- ## Logical Operators @@ -208,15 +222,32 @@ concat(user.firstName, ' ', user.lastName) ## Math Functions -### add(left, right) -> BasicExpression\ +### add, subtract, multiply (left, right) -> BasicExpression\ -Add two numeric values. +Apply the named operation to two numeric values. ```ts add(order.price, order.tax) -add(user.salary, coalesce(user.bonus, 0)) +subtract(user.salary, user.deductions) +multiply(item.price, item.quantity) +``` + +Nullish operands are treated as `0`. + +### divide(left, right) -> BasicExpression\ + +Divide two numeric values. Nullish operands are treated as `0`; a zero or +nullish divisor returns `null`. + +```ts +divide(order.total, order.itemCount) ``` +These functions may be used in `orderBy()`. With a computed `orderBy()` and +`limit()`, all matching rows load before sorting because lazy-loading +optimization cannot apply. Literal values such as `Date.now()` are captured +when the query is built. + --- ## Utility Functions diff --git a/packages/db/skills/db-core/mutations-optimistic/SKILL.md b/packages/db/skills/db-core/mutations-optimistic/SKILL.md index 64be12d5a9..c249ef8602 100644 --- a/packages/db/skills/db-core/mutations-optimistic/SKILL.md +++ b/packages/db/skills/db-core/mutations-optimistic/SKILL.md @@ -9,7 +9,7 @@ description: > onInsert/onUpdate/onDelete handlers. PendingMutation type. Transaction.isPersisted. type: sub-skill library: db -library_version: '0.6.0' +library_version: '0.6.17' sources: - 'TanStack/db:docs/guides/mutations.md' - 'TanStack/db:packages/db/src/transactions.ts' @@ -24,7 +24,7 @@ sources: > handlers) before you can mutate. TanStack DB mutations follow a unidirectional loop: -**optimistic mutation -> handler persists to backend -> sync back -> confirmed state**. +**optimistic mutation -> handler persists -> handler waits for sync/ack -> confirmed state**. Optimistic state is applied in the current tick and dropped when the handler resolves. --- @@ -34,17 +34,19 @@ Optimistic state is applied in the current tick and dropped when the handler res ### insert ```ts +import { safeRandomUUID } from '@tanstack/db' + // Single item todoCollection.insert({ - id: crypto.randomUUID(), + id: safeRandomUUID(), text: 'Buy groceries', completed: false, }) // Multiple items todoCollection.insert([ - { id: crypto.randomUUID(), text: 'Buy groceries', completed: false }, - { id: crypto.randomUUID(), text: 'Walk dog', completed: false }, + { id: safeRandomUUID(), text: 'Buy groceries', completed: false }, + { id: safeRandomUUID(), text: 'Walk dog', completed: false }, ]) // With metadata / non-optimistic @@ -87,7 +89,9 @@ todoCollection.delete(todo.id, { metadata: { reason: 'completed' } }) ``` All three return a `Transaction` object. Use `tx.isPersisted.promise` to await -persistence or catch rollback errors. +settlement or catch rollback errors. For a non-empty transaction, this normally +means its `mutationFn` returned; it proves upload, confirmation, or read-back +only when that function waits for the backend observation before returning. --- @@ -127,7 +131,7 @@ Multi-collection example: const createProject = createOptimisticAction<{ name: string; ownerId: string }>( { onMutate: ({ name, ownerId }) => { - projectCollection.insert({ id: crypto.randomUUID(), name, ownerId }) + projectCollection.insert({ id: safeRandomUUID(), name, ownerId }) userCollection.update(ownerId, (d) => { d.projectCount += 1 }) @@ -207,7 +211,9 @@ await tx.commit() Inside `tx.mutate(() => { ... })`, the transaction is pushed onto an ambient stack. Any `collection.insert/update/delete` call joins the ambient transaction -automatically via `getActiveTransaction()`. +automatically via `getActiveTransaction()`. That scope is synchronous: +collection operations after an `await` do not join it. Put async work in +`mutationFn`, or call `mutate()` again before committing. For mutations captured by a manual transaction, collection-level `onInsert`/`onUpdate`/`onDelete` handlers are not invoked automatically. The @@ -307,7 +313,7 @@ createOptimisticAction({ // CORRECT createOptimisticAction({ onMutate: (text) => { - collection.insert({ id: crypto.randomUUID(), text }) + collection.insert({ id: safeRandomUUID(), text }) }, ... }) @@ -337,7 +343,7 @@ re-insert. ### HIGH: Inserting item with duplicate key If an item with the same key already exists (synced or optimistic), throws -`DuplicateKeyError`. Always generate a unique key (e.g. `crypto.randomUUID()`) +`DuplicateKeyError`. Always generate a unique key (e.g. `safeRandomUUID()`) or check before inserting. ### HIGH: Manually refetching inside a Query Collection handler diff --git a/packages/db/skills/db-core/mutations-optimistic/references/transaction-api.md b/packages/db/skills/db-core/mutations-optimistic/references/transaction-api.md index 5c8e918489..e4cff2abcf 100644 --- a/packages/db/skills/db-core/mutations-optimistic/references/transaction-api.md +++ b/packages/db/skills/db-core/mutations-optimistic/references/transaction-api.md @@ -6,7 +6,7 @@ import { createTransaction } from "@tanstack/db" const tx = createTransaction({ - id?: string, // defaults to crypto.randomUUID() + id?: string, // defaults to safeRandomUUID() autoCommit?: boolean, // default true -- commit after mutate() mutationFn: MutationFn, // (params: { transaction }) => Promise metadata?: Record, // custom data attached to the transaction @@ -26,7 +26,8 @@ interface Transaction { metadata: Record error?: { message: string; error: Error } - // Deferred promise -- resolves when mutationFn completes, rejects on failure + // Deferred promise -- resolves when the transaction settles, rejects on + // mutation failure or rollback isPersisted: { promise: Promise> resolve: (value: Transaction) => void @@ -51,6 +52,7 @@ interface Transaction { - `rollback()` allowed in `pending` or `persisting` (throws `TransactionAlreadyCompletedRollbackError` if completed) - Failed `mutationFn` automatically triggers `rollback()` - Rollback cascades to other pending transactions sharing the same item keys +- An empty or fully cancelled transaction completes without calling `mutationFn` ## PendingMutation Type @@ -102,6 +104,11 @@ stack. Any `collection.insert/update/delete` call automatically joins the topmost ambient transaction. This is how `createOptimisticAction` and `createPacedMutations` wire collection operations into their transactions. +The ambient scope lasts only for the synchronous `mutate()` callback. A +collection operation after an `await` does not join that transaction. Put async +work in `mutationFn`, or call `mutate()` again while the transaction is still +pending. + ## createOptimisticAction ```ts @@ -116,7 +123,7 @@ const action = createOptimisticAction({ // Optional: same as createTransaction config id?: string, - autoCommit?: boolean, // always true (commit happens after mutate) + autoCommit?: boolean, // default true; false requires manual commit() metadata?: Record, }) @@ -203,5 +210,10 @@ try { The promise is a `Deferred` -- it is created at transaction construction time and settled when `commit()` completes or `rollback()` is called. For -`autoCommit: true` transactions, the promise settles shortly after `mutate()` -returns (the commit runs asynchronously). +`autoCommit: true` transactions, commit starts after `mutate()` returns; the +promise can remain pending as long as `mutationFn` does. + +For a non-empty commit, `mutationFn` is the normal success boundary. +`isPersisted.promise` does not by itself prove that a backend uploaded, +confirmed, or read back the write. It proves those stronger guarantees only +when `mutationFn` waits for them before returning. diff --git a/packages/db/skills/db-core/persistence/SKILL.md b/packages/db/skills/db-core/persistence/SKILL.md index 70d2b9eb69..8a5f657af0 100644 --- a/packages/db/skills/db-core/persistence/SKILL.md +++ b/packages/db/skills/db-core/persistence/SKILL.md @@ -8,10 +8,11 @@ description: > Cloudflare Durable Objects. Multi-tab/multi-process coordination via BrowserCollectionCoordinator / ElectronCollectionCoordinator / SingleProcessCoordinator. schemaVersion for migration resets. Local-only mode - for offline-first without a server. + for offline-first without a server. Applied transaction log pruning and safe + full-reload recovery. type: sub-skill library: db -library_version: '0.6.0' +library_version: '0.6.17' sources: - 'TanStack/db:packages/db-sqlite-persistence-core/src/persisted.ts' - 'TanStack/db:packages/browser-db-sqlite-persistence/src/index.ts' @@ -51,7 +52,6 @@ For purely local data with no sync backend: ```ts import { createCollection } from '@tanstack/react-db' import { - BrowserCollectionCoordinator, createBrowserWASQLitePersistence, openBrowserWASQLiteOPFSDatabase, persistedCollectionOptions, @@ -61,13 +61,8 @@ const database = await openBrowserWASQLiteOPFSDatabase({ databaseName: 'my-app.sqlite', }) -const coordinator = new BrowserCollectionCoordinator({ - dbName: 'my-app', -}) - const persistence = createBrowserWASQLitePersistence({ database, - coordinator, }) const draftsCollection = createCollection( @@ -115,11 +110,15 @@ This works with any adapter: `electricCollectionOptions`, `queryCollectionOption Coordinators handle leader election and cross-instance communication so only one tab/process owns the database writer. -| Platform | Coordinator | Mechanism | -| ------------------------------------- | ------------------------------- | ---------------------------------------------- | -| Browser | `BrowserCollectionCoordinator` | BroadcastChannel + Web Locks | -| Electron | `ElectronCollectionCoordinator` | IPC (main holds DB, renderer accesses via RPC) | -| Single-process (RN, Expo, Node, etc.) | `SingleProcessCoordinator` | No-op (always leader) | +| Platform | Coordinator | Mechanism | +| ------------------------------------- | ------------------------------- | ---------------------------- | +| Browser | `BrowserCollectionCoordinator` | BroadcastChannel + Web Locks | +| Electron | `ElectronCollectionCoordinator` | BroadcastChannel + Web Locks | +| Single-process (RN, Expo, Node, etc.) | `SingleProcessCoordinator` | No-op (always leader) | + +Browser persistence uses single-process semantics by default. That is correct +when the app runs in one tab at a time or each tab has its own database. Pass a +`BrowserCollectionCoordinator` only when multiple tabs share one OPFS database. Browser example: @@ -142,7 +141,12 @@ Electron requires setup in both processes: ```ts // Main process import { exposeElectronSQLitePersistence } from '@tanstack/electron-db-sqlite-persistence' -exposeElectronSQLitePersistence({ persistence, ipcMain }) +import { app, ipcMain } from 'electron' + +const disposeIpc = exposeElectronSQLitePersistence({ persistence, ipcMain }) +app.on('before-quit', () => { + disposeIpc() +}) // Renderer process import { @@ -157,6 +161,10 @@ const persistence = createElectronSQLitePersistence({ }) ``` +Electron persistence calls cross the renderer/main boundary through IPC. The +`ElectronCollectionCoordinator` separately coordinates renderer instances with +`BroadcastChannel` and Web Locks. + ## Schema Versioning `schemaVersion` tracks the shape of persisted data. When the stored version doesn't match the code, the collection resets (drops and reloads from server for synced collections, or throws for local-only). @@ -170,6 +178,36 @@ persistedCollectionOptions({ There is no custom migration function -- a version mismatch triggers a full reset. For synced collections this is safe because the server re-supplies the data. +## Applied Transaction Log Pruning + +The SQLite `applied_tx` log is a replay cache, not permanent history. Browser, +Capacitor, Cloudflare Durable Objects, Expo, Node, React Native, and Tauri +wrappers prune it inside write transactions by default, per collection: + +- `appliedTxPruneMaxRows: 1_000` +- `appliedTxPruneMaxAgeSeconds: 86_400` (24 hours) + +Set either option to `0` to disable that limit, or raise it to retain a longer +replay window: + +```ts +const persistence = createNodeSQLitePersistence({ + database, + appliedTxPruneMaxRows: 5_000, + appliedTxPruneMaxAgeSeconds: 0, +}) +``` + +If a follower asks to recover from a point older than the retained log, it +falls back to a full reload. Pruning does not itself shrink the SQLite file; +use SQLite vacuum settings or separate maintenance when disk reclamation +matters. The defaults are exported as +`DEFAULT_APPLIED_TX_PRUNE_MAX_ROWS` and +`DEFAULT_APPLIED_TX_PRUNE_MAX_AGE_SECONDS`. + +Raw `createSQLiteCorePersistenceAdapter` calls do not inject these defaults. +Electron uses whichever persistence adapter the main process supplies. + ## Key Options | Option | Type | Description | @@ -204,13 +242,13 @@ persistedCollectionOptions({ Without an explicit `id`, the code generates a random UUID each session, so persisted data is silently abandoned on every reload. Local-only persisted collections must always provide an `id`. Synced collections derive it from the adapter config. -### HIGH Forgetting the coordinator in multi-tab apps +### HIGH Sharing one browser database across tabs without a coordinator Wrong: ```ts const persistence = createBrowserWASQLitePersistence({ database }) -// No coordinator — concurrent tabs corrupt the database +// Unsafe if multiple tabs share this database ``` Correct: @@ -220,7 +258,9 @@ const coordinator = new BrowserCollectionCoordinator({ dbName: 'my-app' }) const persistence = createBrowserWASQLitePersistence({ database, coordinator }) ``` -Without a coordinator, multiple browser tabs write to SQLite concurrently, causing data corruption. Always use `BrowserCollectionCoordinator` in browser environments. +Without a coordinator, multiple browser tabs that share one OPFS database can +write concurrently. Use `BrowserCollectionCoordinator` for that case. Do not +add it to a single-tab app merely because the runtime is a browser. ### HIGH Not bumping schemaVersion after changing data shape diff --git a/packages/db/skills/meta-framework/SKILL.md b/packages/db/skills/meta-framework/SKILL.md index e93dc019d6..d8aef668f6 100644 --- a/packages/db/skills/meta-framework/SKILL.md +++ b/packages/db/skills/meta-framework/SKILL.md @@ -9,7 +9,7 @@ description: > preloading with Promise.all. Framework-specific loader APIs. type: composition library: db -library_version: '0.6.0' +library_version: '0.6.17' requires: - db-core - db-core/collection-setup @@ -271,9 +271,10 @@ For on-demand collections, source `collection.preload()` warns and does nothing because no subset has been requested. Create the required live query and await `liveQuery.preload()`. -### Collection module pattern +### Stable collection ownership -Define collections in a shared module, import in both loaders and components: +For one global `QueryClient` and one global server resource, define the +collection in a shared module and import it in loaders and components: ```ts // lib/collections.ts @@ -301,6 +302,15 @@ export const Route = createFileRoute('/todos')({ }) ``` +When the `QueryClient`, tenant, project, account, or route parameter defines +the resource, create one stable collection per `QueryClient` and business +scope. Memoize it and put it in router/request context rather than using a +process-global collection. Remove unused entries and call +`collection.cleanup()` in long-lived scope maps. + +See the +[Query adapter runtime and business-scope pattern](../db-core/collection-setup/references/query-adapter.md#runtime-queryclient-and-business-scopes). + ## Server-Side Integration This skill covers the **client-side** read path only (preloading, live queries). For server-side concerns: @@ -365,7 +375,7 @@ export const Route = createFileRoute('/todos')({ Without preloading, the collection starts syncing only when the component mounts, causing a loading flash. Preloading in the route loader starts sync during navigation, making data available immediately when the component renders. -### MEDIUM Creating separate collection instances +### MEDIUM Creating separate collection instances in one scope Wrong: @@ -385,11 +395,14 @@ export const Route = createFileRoute('/todos')({ Correct: ```ts -// lib/collections.ts — single shared instance +// lib/collections.ts — shared for a global QueryClient and global resource export const todoCollection = createCollection(queryCollectionOptions({ ... })) ``` -Collections are singletons. Creating multiple instances for the same data causes duplicate syncs, wasted bandwidth, and inconsistent state between components. +Collections are stable within a `QueryClient` and business scope; they are not +universal singletons. Creating several instances in one scope causes duplicate +syncs and split state. A request-, router-, tenant-, or route-scoped client +needs a scoped factory instead of the global module pattern. See also: react-db/SKILL.md, vue-db/SKILL.md, svelte-db/SKILL.md, solid-db/SKILL.md, angular-db/SKILL.md — for framework-specific hook usage. diff --git a/packages/offline-transactions/skills/offline/SKILL.md b/packages/offline-transactions/skills/offline/SKILL.md index 1295640dc6..6bf6dfc326 100644 --- a/packages/offline-transactions/skills/offline/SKILL.md +++ b/packages/offline-transactions/skills/offline/SKILL.md @@ -10,7 +10,7 @@ description: > React Native support via separate entry point. type: composition library: db -library_version: '0.6.0' +library_version: '0.6.17' requires: - db-core - db-core/mutations-optimistic @@ -31,6 +31,7 @@ import { startOfflineExecutor, IndexedDBAdapter, } from '@tanstack/offline-transactions' +import { safeRandomUUID } from '@tanstack/db' import { todoCollection } from './collections' const executor = startOfflineExecutor({ @@ -68,7 +69,7 @@ const tx = executor.createOfflineTransaction({ // Mutations run inside tx.mutate() — uses ambient transaction context tx.mutate(() => { - todoCollection.insert({ id: crypto.randomUUID(), text: 'New todo' }) + todoCollection.insert({ id: safeRandomUUID(), text: 'New todo' }) }) tx.commit() ``` @@ -82,7 +83,7 @@ const addTodo = executor.createOfflineAction({ mutationFnName: 'createTodo', onMutate: (variables) => { todoCollection.insert({ - id: crypto.randomUUID(), + id: safeRandomUUID(), text: variables.text, }) }, diff --git a/packages/react-db/skills/react-db/SKILL.md b/packages/react-db/skills/react-db/SKILL.md index fb864ac8f7..dd907b8c03 100644 --- a/packages/react-db/skills/react-db/SKILL.md +++ b/packages/react-db/skills/react-db/SKILL.md @@ -13,7 +13,7 @@ description: > type: framework library: db framework: react -library_version: '0.6.0' +library_version: '0.6.17' requires: - db-core sources: @@ -229,7 +229,7 @@ const { data: projects } = useLiveQuery((q) => ), })), ) -// project.issues is string[] — no subcomponent needed +// project.issues is Array<{ id: string; title: string }> — no subcomponent needed ``` See db-core/live-queries/SKILL.md for full includes rules (correlation conditions, nested includes, aggregates). @@ -238,7 +238,8 @@ See db-core/live-queries/SKILL.md for full includes rules (correlation condition Live query results include computed, read-only virtual properties on every row: -- `$synced`: `true` when the row is confirmed by sync; `false` when it is still optimistic. +- `$synced`: `true` when no pending local optimistic write affects the row; + `false` while one does. It does not prove backend confirmation. - `$origin`: `"local"` if the last confirmed change came from this client, otherwise `"remote"`. - `$key`: the row key for the result. - `$collectionId`: the source collection ID. @@ -253,7 +254,7 @@ const { data } = useLiveQuery( .where(({ todo }) => eq(todo.$synced, false)), [], ) -// Shows only optimistic (unconfirmed) todos +// Shows rows with pending local optimistic writes ``` ## React-Specific Patterns @@ -354,7 +355,12 @@ Source: docs/guides/live-queries.md ### HIGH "Not a Collection" error from duplicate @tanstack/db -If `useLiveQuery` throws `InvalidSourceError: The value provided for alias "todo" is not a Collection`, it usually means two copies of `@tanstack/db` are installed. The collection was created by one copy, but `useLiveQuery` checks `instanceof` against the other. +If a query-builder alias throws +`InvalidSourceError: The value provided for alias "todo" is not a Collection`, +it can mean two copies of `@tanstack/db` are installed. Direct +`useLiveQuery(preCreatedCollection)` detection is structural and works across +package copies or realms, but `q.from({ todo: collection })` still validates +the source with the core collection class. In dev mode, TanStack DB also throws `DuplicateDbInstanceError` if two instances are detected. @@ -372,7 +378,7 @@ If multiple versions appear, fix with one of: { "pnpm": { "overrides": { - "@tanstack/db": "^0.6.0" + "@tanstack/db": "^0.6.17" } } } diff --git a/packages/solid-db/skills/solid-db/SKILL.md b/packages/solid-db/skills/solid-db/SKILL.md index 291d3855c7..0039c6cc89 100644 --- a/packages/solid-db/skills/solid-db/SKILL.md +++ b/packages/solid-db/skills/solid-db/SKILL.md @@ -5,12 +5,13 @@ description: > doubles as data access (call as function) with state/status properties. Fine-grained reactivity: signal reads MUST happen inside the query function for tracking. Config passed as Accessor (() => config). Built-in Suspense - support via createResource. ReactiveMap for state. Import from + support via createResource and errors through Solid ErrorBoundary. + ReactiveMap for state. Import from @tanstack/solid-db (re-exports all of @tanstack/db). type: framework library: db framework: solid -library_version: '0.6.0' +library_version: '0.6.17' requires: - db-core sources: @@ -26,7 +27,7 @@ This skill builds on db-core. Read it first for collection setup, query builder, ```tsx import { useLiveQuery, eq, not } from '@tanstack/solid-db' -import { For, Show, Suspense } from 'solid-js' +import { ErrorBoundary, For, Show, Suspense } from 'solid-js' function TodoList() { const todosQuery = useLiveQuery((q) => @@ -131,12 +132,16 @@ return {(user) =>
{user().name}
}
### Suspense integration ```tsx -Loading...}> - {(todo) =>
  • {todo.text}
  • }
    -
    +
    {error.message}
    }> + Loading...}> + {(todo) =>
  • {todo.text}
  • }
    +
    +
    ``` -`useLiveQuery` integrates with Solid's `createResource` — wrap in `` for loading states. +`useLiveQuery` integrates with Solid's `createResource`. Use `` for +loading and `` for errors. Reading an errored query throws +through the resource, so do not rely on reading `isError` after failure. ## Includes (Hierarchical Data) diff --git a/packages/svelte-db/skills/svelte-db/SKILL.md b/packages/svelte-db/skills/svelte-db/SKILL.md index 5976c43e7a..295103a302 100644 --- a/packages/svelte-db/skills/svelte-db/SKILL.md +++ b/packages/svelte-db/skills/svelte-db/SKILL.md @@ -10,7 +10,7 @@ description: > type: framework library: db framework: svelte -library_version: '0.6.0' +library_version: '0.6.17' requires: - db-core sources: diff --git a/packages/vue-db/skills/vue-db/SKILL.md b/packages/vue-db/skills/vue-db/SKILL.md index 650af931c1..3779be9a5d 100644 --- a/packages/vue-db/skills/vue-db/SKILL.md +++ b/packages/vue-db/skills/vue-db/SKILL.md @@ -10,7 +10,7 @@ description: > type: framework library: db framework: vue -library_version: '0.6.0' +library_version: '0.6.17' requires: - db-core sources: