fix: harden sync error handling, unwrap the gateway envelope, and report RLS denials

.github/workflows/changelog.yml

@@ -3,8 +3,12 @@ name: Release and Update Website Changelog
 on:
   push:
     tags:
-      # main.yml creates release tags without a "v" prefix (e.g. 1.1.3)
-      - "*.*.*"
+      # NOTE: this never matches. main.yml creates release tags without a "v"
+      # prefix (e.g. 1.1.3), so the changelog is published by running this
+      # workflow manually. Widening the filter is not enough on its own: the
+      # called workflow derives the version with ${GITHUB_REF#refs/tags/v} and
+      # rejects an unprefixed tag. Fixing it needs a change in changelog-action.
+      - "v*.*.*"
   workflow_dispatch:
     inputs:
       test_version:

API.md

@@ -53,18 +53,25 @@ This document provides a reference for the SQL functions provided by the `sqlite
 
 **Description:** Stores a global CloudSync setting in the current database. Settings persist across database reopens and are loaded automatically by the extension.
 
-The following payload setting is supported:
+The following settings are supported:
 
 | Key | Description | Default | Minimum | Maximum |
 |---|---|---:|---:|---:|
 | `payload_max_chunk_size` | Maximum transport payload size generated by [`cloudsync_payload_chunks()`](#cloudsync_payload_chunkssince_db_version-filter_site_id-until_db_version-exclude_filter_site_id). Values outside the range are clamped. | `5242880` (5 MB) | `262144` (256 KB) | `33554432` (32 MB) |
+| `network_connect_timeout` | Seconds allowed to connect, name resolution included. Never longer than the request's own deadline. | `30` | `1` | `86400` |
+| `network_request_timeout` | Seconds allowed for a whole API request (check, upload URL, apply, status). | `300` | `1` | `86400` |
+| `network_artifact_timeout` | Absolute limit, in seconds, for a payload upload or download. | `3600` | `1` | `86400` |
+| `network_artifact_low_speed_limit` | A payload transfer slower than this many bytes per second for `network_artifact_low_speed_time` is aborted. | `1024` | `1` | `1073741824` |
+| `network_artifact_low_speed_time` | Seconds a payload transfer may stay below `network_artifact_low_speed_limit`. | `60` | `1` | `86400` |
+
+The `network_*` settings apply from the next network request. A missing, zero or negative value uses the default, and larger values are clamped to the maximum. Settings are stored in the database, so a value tuned for a slow network stays in place until it is changed or removed with `cloudsync_set(key, NULL)`.
 
 `payload_max_chunk_size` affects only chunk generation. [`cloudsync_payload_apply()`](#cloudsync_payload_applypayload) continues to accept legacy payloads, monolithic payloads, and v3 chunk-fragment payloads even when they are larger than the local setting. This preserves compatibility between peers using different settings.
 
 **Parameters:**
 
 - `key` (TEXT): The setting key.
-- `value` (TEXT): The setting value. For `payload_max_chunk_size`, pass the value in bytes.
+- `value` (TEXT): The setting value, in the unit given in the table above. `NULL` removes the setting.
 
 **Returns:** SQLite returns no value. PostgreSQL returns `true` on success.
 
@@ -76,6 +83,10 @@ SELECT cloudsync_set('payload_max_chunk_size', '1048576');
 
 -- Restore the default 5 MB transport chunks
 SELECT cloudsync_set('payload_max_chunk_size', '5242880');
+
+-- Allow API requests up to 2 minutes instead of 5, then restore the default
+SELECT cloudsync_set('network_request_timeout', '120');
+SELECT cloudsync_set('network_request_timeout', NULL);
 ```
 
 ---
@@ -651,7 +662,7 @@ On PostgreSQL, apply chunks as individual statements from the transport/client l
 - Monolithic payloads generated by [`cloudsync_payload_encode()`](#cloudsync_payload_encodetbl-pk-col_name-col_value-col_version-db_version-site_id-cl-seq).
 - Chunk-fragment payloads generated by [`cloudsync_payload_chunks()`](#cloudsync_payload_chunkssince_db_version-filter_site_id-until_db_version-exclude_filter_site_id).
 
-When a v3 fragment payload is received, CloudSync stores the fragment in an internal table and returns after applying zero or more completed values. Once the final fragment for a value is received, the completed value is validated and applied. Duplicate fragment delivery is idempotent.
+When a v3 fragment payload is received, CloudSync stores the fragment in an internal table and returns after applying zero or more completed values. Once the final fragment for a value is received, the completed value is validated and applied. Fragments can arrive in any order, and duplicate fragment delivery is idempotent. Applying a fragment never moves the receive checkpoint. On PostgreSQL, pieces of one value applied by concurrent transactions wait for each other under `READ COMMITTED`, and fail with a retryable serialization error under `SERIALIZABLE` when they conflict; a fragment is refused under `REPEATABLE READ`, where a transaction could miss a piece committed while it waited.
 
 **Parameters:**
 
@@ -665,10 +676,20 @@ When a v3 fragment payload is received, CloudSync stores the fragment in an inte
 SELECT cloudsync_payload_apply(:payload);
 ```
 
+#### Failed writes
+
+The apply stops at the first change whose write fails and returns that error: a constraint, a trigger that raises, a type error, a row-level security policy, a missing privilege, a lock or serialization failure. On PostgreSQL the error keeps its original SQLSTATE. No later change in the payload is applied, the change that failed leaves neither data nor sync metadata behind, and the receive checkpoint does not move.
+
+On SQLite the changes applied before the failure are kept (inside the caller's transaction, if there is one). On PostgreSQL the failing statement is rolled back as a whole.
+
+Fix the cause and deliver the payload again: the changes already applied merge again as no-ops. A row-level security policy that depends on rows later in the same payload is not retried within the payload; the apply fails until the rows it depends on have been applied.
+
 ---
 
 ## Network Functions
 
+Every network request has a deadline, so a stalled server cannot hold the connection indefinitely. The deadlines can be tuned with the `network_*` settings of [`cloudsync_set()`](#cloudsync_setkey-value). A network call in progress can also be cancelled with `sqlite3_interrupt()` on its connection: it stops within about a second and fails with `SQLITE_INTERRUPT` (SQLite 3.41 or later), so a deliberate stop can be told apart from a failure worth retrying.
+
 ### `cloudsync_network_init(managedDatabaseId)`
 
 **Description:** Initializes the `sqlite-sync` network component. This function configures the endpoints for the CloudSync service and initializes the cURL library.
@@ -802,7 +823,7 @@ By default this function **drains all currently-available chunks** in one call.
 SELECT cloudsync_network_receive_changes(5) ->> '$.receive.complete';
 ```
 
-The drain position (the per-stream page cursor) is held **in memory** on the network context, so a capped drain resumes where it left off on the next call — the caller does not manage any cursor; it just loops while `receive.complete` is `false`. If the connection is closed or the process restarts mid-drain, the cursor is lost and the next call safely restarts the drain from the beginning of the stream: already-applied chunks are re-downloaded and re-applied idempotently, so **no rows are skipped** — only redundant download is incurred. This is safe because the durable receive checkpoint (`check_dbversion`/`check_seq`) only advances after a stream has been **fully** applied, never in the middle of a source `db_version`.
+The drain position (the per-stream page cursor) is held **in memory** on the network context, so a capped drain resumes where it left off on the next call — the caller does not manage any cursor; it just loops while `receive.complete` is `false`. If the connection is closed or the process restarts mid-drain, or a call fails, the next call safely restarts the drain from the beginning of the stream: already-applied chunks are re-downloaded and re-applied idempotently, so **no rows are skipped** — only redundant download is incurred. This is safe because the durable receive checkpoint (`check_dbversion`/`check_seq`) stays fixed for the whole stream and only advances once the stream has been **fully** applied. A stream whose final chunk arrives while a fragmented value it delivered is still incomplete fails with an error instead of advancing.
 
 If the network is misconfigured or the remote server is unreachable, the function raises a SQL error. If the received payload cannot be applied locally (for example because of an unknown schema hash), the error is returned as a `receive.error` field in the JSON response. If the server reports an unresolved failed check job (e.g. an `encode_changes` failure), that failure is forwarded as a `receive.lastFailure` object.
 
@@ -816,12 +837,12 @@ If the network is misconfigured or the remote server is unreachable, the functio
 {"receive": {"rows": N, "tables": ["table1", "table2"], "chunks": C, "bytes": B, "complete": true, "error": "...", "lastFailure": {...}}}
 ```
 
-- `receive.rows`: The total number of rows received and applied to the local database, summed across all chunks drained this call. `0` when the receive phase failed, when nothing was available, or when only intermediate fragments were staged without completing a value.
-- `receive.tables`: An array of table names that received changes (the union across all drained chunks). Empty (`[]`) if no changes were applied or the receive phase failed.
-- `receive.chunks`: The number of payload chunks applied by this call. `0` when nothing was ready, `1` for a single monolithic/inline page, and `N` for a drained `N`-chunk stream (bounded by `max_chunks` if given).
+- `receive.rows`: The total number of rows received and applied to the local database, summed across all chunks drained this call, including the rows applied before an error. `0` when nothing was available, or when only intermediate fragments were staged without completing a value.
+- `receive.tables`: An array of table names that received changes (the union across all drained chunks, including changes applied before an error). Empty (`[]`) if no changes were applied.
+- `receive.chunks`: The number of payload chunks fully applied by this call. `0` when nothing was ready, `1` for a single monolithic/inline page, and `N` for a drained `N`-chunk stream (bounded by `max_chunks` if given).
 - `receive.bytes`: The total serialized payload bytes received this call (uncompressed cloudsync payload size, summed across chunks; transport-independent, not the compressed wire size). Useful for byte-budgeted draining together with `max_chunks`.
 - `receive.complete` (boolean): `true` when the receive stream is fully drained (nothing pending), `false` when more chunks remain — because `max_chunks` capped the drain, or it stopped early. When `false`, call this function again to continue.
-- `receive.error` (optional, string): Present when client-side `cloudsync_payload_apply` failed. Contains a human-readable error message describing why the received payload could not be applied.
+- `receive.error` (optional, string): Present when client-side `cloudsync_payload_apply` failed. Contains the error of the first change that could not be applied (see [Failed writes](#failed-writes)); `receive.complete` is then `false`, and the next call receives the changes again from the same checkpoint.
 - `receive.lastFailure` (optional, object): Present only when the server reports a failed check job. Forwarded verbatim from the server's `failures.check` and typically includes `jobId`, `dbVersion`, `seq`, `code`, `stage`, `message`, `retryable`, and `failedAt`. Distinct from `receive.error`: `receive.error` describes a client-side apply failure (string), while `receive.lastFailure` describes a server-side check-job failure (object). Both can coexist in the same response. This function is **check-scoped**: server-reported apply-job failures (`failures.apply`) are not surfaced here — see [`cloudsync_network_send_changes()`](#cloudsync_network_send_changes) and [`cloudsync_network_sync()`](#cloudsync_network_syncwait_ms-max_retries).
 
 **Example:**
@@ -834,7 +855,7 @@ SELECT cloudsync_network_receive_changes();
 -- '{"receive":{"rows":40,"tables":["docs"],"chunks":5,"bytes":1310720,"complete":false}}'
 
 -- With a client-side apply error:
--- '{"receive":{"rows":0,"tables":[],"chunks":0,"bytes":0,"complete":true,"error":"Cannot apply the received payload because the schema hash is unknown 7218827471400075525."}}'
+-- '{"receive":{"rows":0,"tables":[],"chunks":0,"bytes":0,"complete":false,"error":"Cannot apply the received payload because the schema hash is unknown 7218827471400075525."}}'
 
 -- With a server-reported check-job failure:
 -- '{"receive":{"rows":0,"tables":[],"chunks":0,"bytes":0,"complete":true,"lastFailure":{"jobId":456,"dbVersion":15,"seq":1,"code":"tenant_unreachable","stage":"encode_changes","message":"tenant check failed","retryable":true,"failedAt":"2026-04-24T10:22:00Z"}}}'
@@ -876,8 +897,8 @@ When the server delivers changes as a stream of chunks, this function drains the
 - `send.serverVersion`: The latest version confirmed by the server.
 - `send.chunks` / `send.bytes`: Number of payload chunks sent and total serialized payload bytes sent during the send phase. Same semantics as in [`cloudsync_network_send_changes()`](#cloudsync_network_send_changes).
 - `send.lastFailure` (optional): Same semantics as in [`cloudsync_network_send_changes()`](#cloudsync_network_send_changes) — forwarded verbatim from the server's `failures.apply` whenever a failed apply job is reported, regardless of `status`.
-- `receive.rows`: The **total** number of rows received and applied during the receive phase, summed across **all** chunks drained in this call. `0` when the receive phase failed.
-- `receive.tables`: An array of table names that received changes (the union across all drained chunks). Empty (`[]`) if no changes were applied or the receive phase failed.
+- `receive.rows`: The **total** number of rows received and applied during the receive phase, summed across **all** chunks drained in this call, including the rows applied before an error.
+- `receive.tables`: An array of table names that received changes (the union across all drained chunks, including changes applied before an error). Empty (`[]`) if no changes were applied.
 - `receive.chunks`: The number of payload chunks applied in this call. `0` when nothing was ready, `1` for a single monolithic/inline page, and `N` for a fully drained `N`-chunk stream. `cloudsync_network_sync()` always drains the whole stream (it does not cap chunks).
 - `receive.bytes`: The total serialized payload bytes received this call (uncompressed cloudsync payload size, summed across chunks; not the compressed wire size). Same semantics as in [`cloudsync_network_receive_changes()`](#cloudsync_network_receive_changesmax_chunks).
 - `receive.complete` (boolean): `true` when the server stream was fully drained, `false` when the download stopped before the final chunk (an error occurred, or an internal safety bound was reached). When `false`, call `cloudsync_network_sync()` again to resume; re-delivered rows are idempotent.