fix: actionable error and docs for decrypt_events missing-key failures

README.md

@@ -43,7 +43,8 @@ Python/Rust, PascalCase in .NET/Go):
 3. **Initial load** — `decryptEvents(events)`: batch-decrypts a backlog,
    extracting conversation keys from `KeyChange` events and matching signing
    keys by user automatically. Never throws; per-event errors are collected in
-   the result.
+   the result. The events endpoint returns the `KeyChange` events in
+   `meta.conversation_key_events`, separate from `data` — pass both together.
 4. **Each new event** — `decryptEvent(event)`: decrypts one event
    (webhook / poll). Throws on failure.
 5. **Reply** — `encryptMessage({conversationId, text})` or

crates/core/src/core.rs

@@ -1660,8 +1660,9 @@ impl ChatCore {
                                     .join(", ");
                                 SdkError::Crypto(CryptoError::DecryptionFailed(format!(
                                     "Message encrypted with key version '{}' but no matching key \
-                                     found. Available versions: [{}]. Call \
-                                     extract_conversation_keys() first.",
+                                     found. Available versions: [{}]. Include the conversation's \
+                                     KeyChange events in the batch passed to decrypt_events(), \
+                                     or pass the matching key map to decrypt_event().",
                                     version, available
                                 )))
                             })?

docs/API.md

@@ -174,6 +174,12 @@ omitted/empty key arguments and fall back to two opt-in session stores:
   `conversation_key` / `conversation_key_version` pair from it. Disabling
   clears the cache; the cached keys zeroize on drop.
 
+  Only `decrypt_events` populates the cache, and only from KeyChange events
+  in its own batch whose signature verified. `extract_conversation_keys`
+  never feeds it: that method adopts every decryptable key without signature
+  checks, and the cache holds verified keys only. To use its result, pass
+  the returned key map to `decrypt_event` explicitly.
+
 An explicit non-empty argument always wins over the stores. The two decrypt
 contracts are unchanged: `decrypt_events` never throws (per-event errors are
 collected in the result) and `decrypt_event` throws on failure.
@@ -403,7 +409,7 @@ Decrypts multiple events in one call. Handles everything internally:
 
 | Param | JS | Python | Rust | Go | JVM | .NET | Description |
 |---|---|---|---|---|---|---|---|
-| events | `string[]` | `list[str]` | `&[&str]` | `[]string` | `List<String>` | `IEnumerable<string>` | All base64-encoded raw events. **Must include KeyChange events** — without them, messages depending on those keys will land in `errors`. |
+| events | `string[]` | `list[str]` | `&[&str]` | `[]string` | `List<String>` | `IEnumerable<string>` | All base64-encoded raw events. **Must include KeyChange events** — without them, messages depending on those keys will land in `errors`. The events endpoint returns KeyChange events in **`meta.conversation_key_events`**, separate from the `data` array; concatenate both into this argument. |
 | signingKeys | `SigningKeyEntry[]` | `list[dict]` | `&[SigningKeyEntry]` | `[]SigningKeyEntry` or `nil` | `List<SigningKeyEntry>` or `null` | `IEnumerable<SigningKeyEntry>?` | Signing keys for **all participants**. The SDK extracts each event's `senderId` internally and filters to the matching keys. Omitting the parameter (or passing `[]` / `nil` / `null`) falls back to the keys stored via `setSigningKeys`; if none are stored either, under the default reject-unverified policy every **signed** event fails decryption and lands in `errors`. Only after `setRejectUnverified(false)` are such events returned with `verified: false`. |
 
 **Returns: `DecryptEventsResult`** — never throws/raises. Errors are collected.
@@ -434,17 +440,24 @@ interface DecryptedMessage {
 }
 ```
 
-**Some per-event errors are permanent.** Signatures are immutable and verified
-by rebuilding the signed payload from the event, so an event whose signature was
-produced from the wrong input (or never signed at all) fails verification on
-every future load — it cannot be healed by retrying, refreshing keys, or any
-API call. Treat these as tombstones, not transient failures:
+**Rule out a missing-key-events batch first.** The most common cause of
+`Message encrypted with key version '…' but no matching key found` is caller
+input: the KeyChange events were left out of the batch (they arrive in
+`meta.conversation_key_events`, not `data` — see the events argument above).
+That case is fixed by re-batching with the key events included.
+
+**The remaining per-event errors are permanent.** Signatures are immutable and
+verified by rebuilding the signed payload from the event, so an event whose
+signature was produced from the wrong input (or never signed at all) fails
+verification on every future load — it cannot be healed by retrying,
+refreshing keys, or any API call. Treat these as tombstones, not transient
+failures:
 
 | Error | Meaning |
 |---|---|
 | `…signature missing or no matching signing key` on a KeyChange | The key change was never signed (or signed with an unpublished key). Its conversation key is never extracted. |
 | `ECDSA mismatch: key_version=…` | The signer fed different bytes into the signature than the event carries (e.g. a non-canonical conversation id). |
-| `Message encrypted with key version '…' but no matching key found` | The message's key came from an unverifiable KeyChange above — collateral of the first row. |
+| `Message encrypted with key version '…' but no matching key found` (key events included in the batch) | The message's key came from an unverifiable KeyChange above — collateral of the first row. |
 
 New messages are unaffected: rotating the key starts a clean, verifiable
 history from that point forward. (Verifiability only — rotation does not

examples/dotnet/ChatBot.Tests/E2ELiveTests.cs

@@ -52,6 +52,20 @@ private static List<JsonElement> Data(JsonElement page) =>
     private static List<string> EventsB64(IEnumerable<JsonElement> raw) =>
         raw.Select(e => Str(e, "encoded_event")).Where(s => !string.IsNullOrEmpty(s)).Select(s => s!).ToList();
 
+    // KeyChange events arrive in meta.conversation_key_events, separate from
+    // data; they carry the conversation keys and must go into the same
+    // DecryptBatch call as the data events.
+    private static List<string> KeyEvents(JsonElement page) =>
+        page.TryGetProperty("meta", out var meta)
+        && meta.ValueKind == JsonValueKind.Object
+        && meta.TryGetProperty("conversation_key_events", out var arr)
+        && arr.ValueKind == JsonValueKind.Array
+            ? arr.EnumerateArray()
+                .Where(e => e.ValueKind == JsonValueKind.String)
+                .Select(e => e.GetString()!)
+                .ToList()
+            : new List<string>();
+
     private static SigningKeyEntry SigningFrom(JsonElement pk, string userId) => new()
     {
         UserId = userId,
@@ -137,6 +151,7 @@ public async Task E2ELive()
         // -- 1. Inbound history: batch decrypt (+ pagination when available) ----
         var page = await api.GetEventsAsync(conv, 10);
         var raw = Data(page);
+        var keyEventsB64 = KeyEvents(page);
         var nextToken = page.TryGetProperty("meta", out var meta) ? Str(meta, "next_token") : null;
         if (!string.IsNullOrEmpty(nextToken))
         {
@@ -146,6 +161,7 @@ public async Task E2ELive()
             Assert.True(raw2.Count > 0 && !raw2.Any(e => ids1.Contains(Str(e, "id") ?? "")),
                 "pagination made no progress");
             raw.AddRange(raw2);
+            keyEventsB64.AddRange(KeyEvents(page2));
             Console.WriteLine($"pagination: fetched second page with {raw2.Count} events");
         }
 
@@ -170,7 +186,7 @@ public async Task E2ELive()
             }
         }
 
-        var batch = core.DecryptBatch(EventsB64(raw), signing);
+        var batch = core.DecryptBatch(keyEventsB64.Concat(EventsB64(raw)).ToList(), signing);
         var decrypted = batch.Messages.Count(m => !string.IsNullOrEmpty(EventHelpers.MessageText(m.Event)));
         var convKeys = new Dictionary<string, byte[]>(batch.ConversationKeys.Keys);
         Console.WriteLine($"live inbound messages decrypted: {decrypted}; conversation keys: {convKeys.Count}");
@@ -213,7 +229,7 @@ public async Task E2ELive()
         for (var i = 0; i < 5; i++)
         {
             page = await api.GetEventsAsync(conv, 10);
-            batch = core.DecryptBatch(EventsB64(Data(page)), signing);
+            batch = core.DecryptBatch(KeyEvents(page).Concat(EventsB64(Data(page))).ToList(), signing);
             convKeys = new Dictionary<string, byte[]>(batch.ConversationKeys.Keys);
             if (convKeys.ContainsKey(kv)) break;
             await Task.Delay(1500);

examples/dotnet/ChatBot/Bot.cs

@@ -47,6 +47,23 @@ private ConversationState State(string conversationId)
             ? nt.GetString()
             : null;
 
+    /// <summary>
+    /// KeyChange events from a GET events page. They arrive in
+    /// meta.conversation_key_events, separate from data, and carry the
+    /// conversation keys — they must go into the same DecryptEvents batch as
+    /// the data events.
+    /// </summary>
+    private static List<string> KeyEvents(JsonElement page) =>
+        page.TryGetProperty("meta", out var meta)
+        && meta.ValueKind == JsonValueKind.Object
+        && meta.TryGetProperty("conversation_key_events", out var arr)
+        && arr.ValueKind == JsonValueKind.Array
+            ? arr.EnumerateArray()
+                .Where(e => e.ValueKind == JsonValueKind.String)
+                .Select(e => e.GetString()!)
+                .ToList()
+            : new List<string>();
+
     private async Task<List<SigningKeyEntry>> SigningKeysForAsync(IEnumerable<JsonElement> events)
     {
         var senders = events
@@ -88,11 +105,11 @@ public async Task LoadBacklogAsync(string conversationId)
         var raw = page.TryGetProperty("data", out var d) && d.ValueKind == JsonValueKind.Array
             ? d.EnumerateArray().ToList()
             : new List<JsonElement>();
-        var eventsB64 = raw
+        var eventsB64 = KeyEvents(page);
+        eventsB64.AddRange(raw
             .Select(e => e.TryGetProperty("encoded_event", out var ev) ? ev.GetString() : null)
             .Where(s => !string.IsNullOrEmpty(s))
-            .Select(s => s!)
-            .ToList();
+            .Select(s => s!));
 
         var result = _core.DecryptBatch(eventsB64, await SigningKeysForAsync(raw));
         foreach (var (version, key) in result.ConversationKeys.Keys)
@@ -112,6 +129,18 @@ public async Task PollOnceAsync(string conversationId)
             : new List<JsonElement>();
         var signingKeys = await SigningKeysForAsync(raw);
 
+        // Key changes for this page arrive in meta, not data; adopt their
+        // keys before decrypting the messages that need them.
+        var pageKeyEvents = KeyEvents(page);
+        if (pageKeyEvents.Count > 0)
+        {
+            var rotated = _core.DecryptBatch(pageKeyEvents, signingKeys);
+            foreach (var (version, key) in rotated.ConversationKeys.Keys)
+                st.ConversationKeys[version] = key;
+            if (rotated.ConversationKeys.LatestVersion is { } lv)
+                st.LatestKeyVersion = lv;
+        }
+
         foreach (var item in raw)
         {
             if (!item.TryGetProperty("encoded_event", out var ev) || ev.GetString() is not { } eventB64)

examples/go/bot.go

@@ -99,12 +99,14 @@ func (b *Bot) refreshSigningKeys(events []EventItem) {
 // filling the SDK's conversation-key cache from the KeyChange events.
 func (b *Bot) LoadBacklog(conversationID string) error {
 	st := b.stateFor(conversationID)
-	events, next, err := b.api.GetEvents(conversationID, 100, "")
+	events, keyEvents, next, err := b.api.GetEvents(conversationID, 100, "")
 	if err != nil {
 		return err
 	}
 	b.refreshSigningKeys(events)
-	var eventsB64 []string
+	// The key events must be in the same batch as the messages: they are the
+	// only source of the conversation keys the messages decrypt under.
+	eventsB64 := append([]string{}, keyEvents...)
 	for _, e := range events {
 		if e.EncodedEvent != "" {
 			eventsB64 = append(eventsB64, e.EncodedEvent)
@@ -122,11 +124,20 @@ func (b *Bot) LoadBacklog(conversationID string) error {
 // PollOnce fetches new events and replies using the single-event decrypt path.
 func (b *Bot) PollOnce(conversationID string) error {
 	st := b.stateFor(conversationID)
-	events, next, err := b.api.GetEvents(conversationID, 50, st.paginationToken)
+	events, keyEvents, next, err := b.api.GetEvents(conversationID, 50, st.paginationToken)
 	if err != nil {
 		return err
 	}
 	b.refreshSigningKeys(events)
+	// Key changes for this page arrive in meta, not data; only the batch
+	// path feeds the key cache, so route them through it before decrypting.
+	// This runs after the signing-key refresh: a key change from a sender not
+	// yet in the store would fail verification and never be cached.
+	if len(keyEvents) > 0 {
+		if _, err := b.core.DecryptBatch(keyEvents, nil); err != nil {
+			log.Printf("key_events_decrypt_failed conv=%s err=%v", conversationID, err)
+		}
+	}
 	for _, item := range events {
 		if item.EncodedEvent == "" {
 			continue

examples/go/e2e_test.go

@@ -61,15 +61,16 @@ func TestE2ELive(t *testing.T) {
 	}
 
 	// -- 1. Inbound history: batch decrypt (+ pagination when available) ----
-	raw, next, err := api.GetEvents(conv, 10, "")
+	raw, keyEventsPage1, next, err := api.GetEvents(conv, 10, "")
 	if err != nil {
 		t.Fatalf("GetEvents: %v", err)
 	}
 	if next != "" {
-		raw2, _, err := api.GetEvents(conv, 10, next)
+		raw2, keyEventsPage2, _, err := api.GetEvents(conv, 10, next)
 		if err != nil {
 			t.Fatalf("GetEvents page 2: %v", err)
 		}
+		keyEventsPage1 = append(keyEventsPage1, keyEventsPage2...)
 		ids1 := map[string]bool{}
 		for _, e := range raw {
 			ids1[e.ID] = true
@@ -121,7 +122,9 @@ func TestE2ELive(t *testing.T) {
 		t.Fatalf("SetSigningKeys: %v", err)
 	}
 
-	var eventsB64 []string
+	// The KeyChange events from meta.conversation_key_events carry the
+	// conversation keys; they must be in the same batch as the messages.
+	eventsB64 := append([]string{}, keyEventsPage1...)
 	for _, e := range raw {
 		if e.EncodedEvent != "" {
 			eventsB64 = append(eventsB64, e.EncodedEvent)
@@ -203,11 +206,12 @@ func TestE2ELive(t *testing.T) {
 	kv := prep.ConversationKeyVersion
 	var convKeys map[string][]byte
 	for attempt := 0; attempt < 5; attempt++ {
-		raw, _, err = api.GetEvents(conv, 10, "")
+		var pageKeyEvents []string
+		raw, pageKeyEvents, _, err = api.GetEvents(conv, 10, "")
 		if err != nil {
 			t.Fatalf("GetEvents: %v", err)
 		}
-		eventsB64 = eventsB64[:0]
+		eventsB64 = append(eventsB64[:0], pageKeyEvents...)
 		for _, e := range raw {
 			if e.EncodedEvent != "" {
 				eventsB64 = append(eventsB64, e.EncodedEvent)
@@ -488,7 +492,7 @@ func awaitDecrypted(t *testing.T, api *XChatClient, core *ChatCore, conversation
 	t.Helper()
 	var lastErr error
 	for try := 0; try < 10; try++ {
-		events, _, err := api.GetEvents(conversationID, 25, "")
+		events, _, _, err := api.GetEvents(conversationID, 25, "")
 		if err != nil {
 			t.Fatalf("GetEvents: %v", err)
 		}

examples/go/xapi.go

@@ -117,7 +117,12 @@ func (c *XChatClient) GetPublicKeys(userID string) ([]map[string]any, error) {
 }
 
 // GetEvents fetches the raw (encrypted) events for a conversation.
-func (c *XChatClient) GetEvents(conversationID string, maxResults int, paginationToken string) ([]EventItem, string, error) {
+//
+// KeyChange events arrive in meta.conversation_key_events, separate from
+// data; they are returned as keyEvents and must go into the same
+// DecryptEvents batch as the data events — without them no conversation key
+// is extracted and every message lands in the result's errors.
+func (c *XChatClient) GetEvents(conversationID string, maxResults int, paginationToken string) (items []EventItem, keyEvents []string, next string, err error) {
 	q := url.Values{}
 	q.Set("max_results", fmt.Sprintf("%d", maxResults))
 	if paginationToken != "" {
@@ -126,10 +131,17 @@ func (c *XChatClient) GetEvents(conversationID string, maxResults int, paginatio
 	path := fmt.Sprintf("/2/chat/conversations/%s/events?%s", url.PathEscape(apiConvID(conversationID)), q.Encode())
 	var out eventsResponse
 	if err := c.do(http.MethodGet, path, nil, &out); err != nil {
-		return nil, "", err
+		return nil, nil, "", err
+	}
+	if rawKeyEvents, ok := out.Meta["conversation_key_events"].([]any); ok {
+		for _, v := range rawKeyEvents {
+			if s, ok := v.(string); ok {
+				keyEvents = append(keyEvents, s)
+			}
+		}
 	}
-	next, _ := out.Meta["next_token"].(string)
-	return out.Data, next, nil
+	next, _ = out.Meta["next_token"].(string)
+	return out.Data, keyEvents, next, nil
 }
 
 // -- Conversation / key management -------------------------------------------

examples/js/e2e-live.mjs

@@ -104,9 +104,16 @@ const myId = await api.getMyUserId();
 // signingKeyVersion arguments are gone.
 core.setIdentity(myId);
 
+// KeyChange events arrive in meta.conversation_key_events, separate from
+// data; they carry the conversation keys and must go into the same
+// decryptEvents batch as the data events.
+const keyEventsOf = (p) =>
+  p.meta?.conversationKeyEvents ?? p.meta?.conversation_key_events ?? [];
+
 // -- 1. Inbound history: batch decrypt (+ pagination when available) --------
 const page = await api.getEvents(conv, { maxResults: 10 });
 let raw = page.data ?? [];
+const keyEventsB64 = [...keyEventsOf(page)];
 const nextToken = page.meta?.nextToken ?? page.meta?.next_token;
 if (nextToken) {
   const page2 = await api.getEvents(conv, { maxResults: 10, paginationToken: nextToken });
@@ -117,6 +124,7 @@ if (nextToken) {
     "pagination made no progress",
   );
   raw = raw.concat(raw2);
+  keyEventsB64.push(...keyEventsOf(page2));
   console.log(`pagination: fetched second page with ${raw2.length} events`);
 }
 
@@ -137,7 +145,10 @@ for (const id of ids) {
   }
 }
 
-const eventsB64 = raw.map((e) => e.encodedEvent ?? e.encoded_event).filter(Boolean);
+const eventsB64 = [
+  ...keyEventsB64,
+  ...raw.map((e) => e.encodedEvent ?? e.encoded_event).filter(Boolean),
+];
 let batch = core.decryptBatch(eventsB64, signing);
 const decrypted = batch.messages.filter((m) => messageText(m.event)).length;
 let convKeys = { ...batch.conversationKeys.keys };
@@ -190,7 +201,10 @@ const kv = prep.conversationKeyVersion;
 for (let i = 0; i < 5; i++) {
   const page3 = await api.getEvents(conv, { maxResults: 10 });
   batch = core.decryptBatch(
-    (page3.data ?? []).map((e) => e.encodedEvent ?? e.encoded_event).filter(Boolean),
+    [
+      ...keyEventsOf(page3),
+      ...(page3.data ?? []).map((e) => e.encodedEvent ?? e.encoded_event).filter(Boolean),
+    ],
     signing,
   );
   convKeys = { ...batch.conversationKeys.keys };