Skip to content

Commit 4001ea7

Browse files
owenpearsonclaude
andcommitted
test: derive the push and batch unit specs
Covers push/push_device_registrations.md, push_channel_subscriptions.md, push_channels.md, push_admin_publish.md, batch_publish.md and batch_presence.md, completing the REST unit tier. The batch REST API and PushChannel are not implemented, so fifty-one tests carry the deviation mark with the assertion the specification calls for: forty-one for batch publish and presence, ten for PushChannel. Removing the mark is all that is needed once each lands. The push admin surface does exist, and a fifty-second test records that device ids are interpolated into push paths unescaped, where channel names are quoted. deviations.md now carries the totals for the tier: of 581 derived tests, 465 pass, 110 are gated behind RUN_DEVIATIONS and 6 cannot be run at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 6ea0d20 commit 4001ea7

8 files changed

Lines changed: 2386 additions & 5 deletions

‎test/uts/deviations.md‎

Lines changed: 24 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,10 @@ this file records behaviour.
77
Entries are grouped by root cause rather than by test, so one entry covers every
88
test it affects. Headings are fixed and appear even when they hold nothing.
99

10+
Of 581 derived tests, 465 pass, 110 are gated behind `RUN_DEVIATIONS` and 6 cannot be
11+
run at all. Every gated test has been confirmed to fail when enabled, so none of them
12+
passes under both behaviours.
13+
1014
Entries closed by a fix are removed rather than kept as history; `git log` holds that.
1115

1216
Run the gated tests with:
@@ -27,9 +31,8 @@ from the specification text, so that correcting the specification is all it take
2731
it pass, and it is marked `@spec_error` — a skip gated on `RUN_DEVIATIONS`, the same gate
2832
`@deviation` uses, with a reason naming the specification rather than the SDK. The suite
2933
stays green, a real regression still shows, and the failure is one environment variable
30-
away. `spec-inconsistencies.md` carries the report raised upstream.
31-
32-
The tests gated this way:
34+
away. `spec-inconsistencies.md` carries the report raised upstream. Nine tests are gated
35+
this way:
3336

3437
| Test | Spec error |
3538
|---|---|
@@ -156,6 +159,19 @@ file, and the protocol, which fix LEAVE at 3 and UPDATE at 4. The closing note o
156159
- Error bodies throughout omit `message` and `statusCode` while tests assert
157160
`error.statusCode`. An SDK that reads the status from the payload cannot satisfy both.
158161

162+
### Batch response envelopes disagree between sibling specs
163+
164+
`batch_presence.md` states that with `X-Ably-Version >= 3` the server returns a
165+
`BatchResult` envelope "for all batch responses" and calls the plain array legacy.
166+
Every mock in `batch_publish.md` uses the plain array. `features.md` RSC22b backs
167+
`batch_publish.md` — "the response will still be an array" — so `batch_presence.md`'s
168+
claim is the one to revisit. `revoke_tokens.md` has the same internal split:
169+
`RSA17c_1` and `TRS2_1` stub a bare array while asserting envelope fields.
170+
171+
`batch_publish.md` RSC22_Headers1 also pins `X-Ably-Version: 2` and
172+
`Content-Type: application/json`; CSV2b templates the version, the sibling spec says
173+
">= 3", and the binary protocol default makes the content type msgpack.
174+
159175
### Smaller faults
160176

161177
| Spec | Fault |
@@ -179,7 +195,9 @@ the mark is the only change needed once the SDK behaviour lands.
179195

180196
| Spec points | Missing | Tests |
181197
|---|---|---|
182-
198+
| RSC22, RSC24, BSP2, BPR2, BPF2, BAR2, BGR2, BGF2 | `batchPublish` and `batchPresence`, and all six result types. `grep -rn batch ably/` finds nothing | 41 |
199+
| RSA17, RSA17b–g, BAR2, TRS2, TRF2 | `Auth#revokeTokens`, `TokenRevocationTargetSpecifier`, `BatchResult` | 17 |
200+
| RSH7, RSH7a–e, RSH6, RSH8 | `PushChannel`: `channel.push`, `client.device`, `LocalDevice`. The push *admin* surface (RSH1) does exist | 10 |
183201
| RSL7 | `RestChannel#setOptions`. The realtime channel implements it; the REST `options` setter expects the kwargs dict `Channels.get` collected, so a `ChannelOptions` raises `TypeError` | 2 |
184202
| RSP3a2, RSP3a3 | `clientId` and `connectionId` filters on `RestPresence#get`. `Presence.get` takes only `limit`, while `Presence.history` does take its documented params | 3 |
185203
| TP5 | `size` on `PresenceMessage`. The related `maxMessageSize` gap is adapted rather than gated, below; `features.md` TM6 has no UTS test | 1 |
@@ -215,6 +233,7 @@ the mark is the only change needed once the SDK behaviour lands.
215233
| Spec points | Behaviour |
216234
|---|---|
217235
| RSC19b | Caller-supplied headers override the configured `Authorization`, because `Http.make_request` applies `headers` after `auth_headers`. RSC19b says requests "unconditionally" use the configured mechanism |
236+
| RSH1b1 | Device ids are interpolated raw into push paths (`ably/rest/push.py` lines 82, 106, 118), so an id containing `/` addresses a different resource and `:` is unescaped. `ably/rest/channel.py` does quote channel names, so the SDK is inconsistent with itself |
218237

219238
## Adapted Tests
220239

@@ -230,7 +249,7 @@ comment above. These run, so they guard against regression.
230249
| RSC18 | The constructor rejects basic auth over HTTP | Construction succeeds; 40103 is raised from `make_request` when a request needing Basic Auth is attempted, and no request goes out. RSA1/RSC18 say only "any attempt to use" | Compliant; the UTS is stricter than its source |
231250
| REC1b1, REC1c1 | Code 40000, or a message containing "invalid" or "conflict" | 400/40106 with a specific message. The features spec mandates no code | Cosmetic |
232251
| RSAN1a3 | Code 40003 for a missing `Annotation.type` | 400/40000 | Cosmetic; worth aligning cross-SDK |
233-
252+
| RSH1a | Empty `recipient` or `data` rejected with code 40000 | `TypeError` / `ValueError`, not an `AblyException`. The "no HTTP request" half is satisfied | Open bug, minor |
234253
| HP6 | `errorCode` is a number | The raw header string, `'40101'` | Open bug, trivial |
235254
| HP8 | `headers` is a map | A list of `(name, value)` pairs, so the lookup the spec describes is impossible without converting, and case-insensitivity is lost | Open bug; changing the return type is breaking |
236255
| RSC19e | An error indicated idiomatically | `httpx.ConnectError` / `ReadTimeout` reach the caller unwrapped, because `AblyRest.request` carries no `@catch_all` unlike `time()` and `stats()`. The messages do name the failure | Borderline; defensible under RSC19e |

0 commit comments

Comments
 (0)