feat(realtime): add unified set() method for atomic prompt + image updates

[AdirAmsalem]

Summary

• Add realtimeClient.set({ prompt, enhance, image }) — a single method to atomically update prompt and/or reference image in one wire message
• Keep setPrompt() and setImage() for backward compatibility
• Add 20 new unit tests covering validation, wire format, and timeout behavior
• Update all realtime examples to demonstrate set() as the primary API

Motivation

Lucy 2 supports both prompt and reference image. The current API requires separate setPrompt() and setImage() calls, which can't be sent atomically. setImage() also duplicates setPrompt() functionality with its optional prompt parameter, making the API confusing.

The new set() method solves this:

// Atomic prompt + image update
await realtimeClient.set({
  prompt: "A superhero",
  enhance: true,
  image: referenceFile, // Blob | File | base64 | URL | null
});

// Omit = no change, null = clear
await realtimeClient.set({ prompt: "New prompt" });  // image unchanged
await realtimeClient.set({ image: null });            // clears image

Wire Protocol

→ { "type": "set", "prompt": "...", "enhance_prompt": true, "image_data": "base64..." }
← { "type": "set_ack", "success": true, "error": null }

Commits

1. Wire protocol types — SetMessage, SetAckMessage added to type unions
2. Implementation — sendSet() on connection/manager, set() with Zod validation in methods, wired into client
3. Tests — 20 new tests: type structure, sendSet timeout, input validation, wire message correctness
4. Examples — All realtime examples updated to use set() as primary API

Test Results

• 77 tests passing ✅
• Build clean ✅

---

Note

Medium Risk
Introduces a new realtime wire message (set_input) and ack/timeout path through the WebRTC connection, so runtime behavior now depends on server support and correct message/ack handling; existing APIs remain for compatibility.

Overview
Adds a new realtimeClient.set() API to atomically update realtime prompt and/or reference image in a single request, with Zod validation and a 30s ack-based timeout.

Extends the realtime wire protocol with set_input/set_input_ack, plumbs it through WebRTCConnection/WebRTCManager, factors image-to-base64 handling for reuse, and exports the new SetInput type.

Updates realtime examples to prefer set() and adds unit tests covering set() validation/wire format plus sendSet() timeout behavior.

^{Written by Cursor Bugbot for commit 3546912. This will update automatically on new commits. Configure here.}

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/DecartAI/sdk/@decartai/sdk@76

commit: 3546912

[cursor]

Missing ack correlation causes wrong promise resolution on concurrent calls

Medium Severity

sendSet resolves on the first setAck message received, with no correlation to the request that produced it. Unlike the existing setPrompt flow, which matches promptAckMessage.prompt === parsedInput.data.prompt, SetAckMessage has no identifying field, so when multiple set() calls are in-flight (as the examples demonstrate with fire-and-forget in input handlers), the first server ack resolves or rejects all pending promises indiscriminately. A failed request can appear to succeed if an earlier request's success ack arrives first.

Additional Locations (1)

• packages/sdk/src/realtime/types.ts#L70-L75

 

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

[cursor]

Passing image: null silently fails to clear image

High Severity

The set() method accepts image: null (the Zod schema validates it, and the refine check treats it as "provided"), but the condition image !== undefined && image !== null skips adding any image_data field to the wire message. This makes set({ image: null }) send { type: "set" } — identical to omitting image entirely. The server cannot distinguish "clear image" from "no change." The SetMessage type also lacks null in image_data's type union. Compare with the existing setImage path which correctly sends image_data: null on the wire to clear.

Additional Locations (1)

• packages/sdk/src/realtime/types.ts#L63-L69