diff --git a/calls/android/actions.mdx b/calls/android/actions.mdx index 663870464..fcd61a8f6 100644 --- a/calls/android/actions.mdx +++ b/calls/android/actions.mdx @@ -185,6 +185,48 @@ callSession.stopRecording(); +### Start Transcription + +*Available since v5.0.4* + +Begins server-side transcription of the call. This also enables the live closed captions overlay. + + + +```kotlin +callSession.startTranscription() +``` + + +```java +callSession.startTranscription(); +``` + + + + +Transcription requires the feature to be enabled for your CometChat app. + + +### Stop Transcription + +*Available since v5.0.4* + +Stops the current transcription and clears any captions currently on screen. The transcript is saved and can be retrieved with [`TranscriptRequest`](/calls/android/transcription#retrieving-transcripts). + + + +```kotlin +callSession.stopTranscription() +``` + + +```java +callSession.stopTranscription(); +``` + + + ### Mute Participant Mutes a specific participant's audio. This is a moderator action. diff --git a/calls/android/overview.mdx b/calls/android/overview.mdx index e038f09da..cad0cc66e 100644 --- a/calls/android/overview.mdx +++ b/calls/android/overview.mdx @@ -56,6 +56,10 @@ sequenceDiagram Record call sessions for later playback + + Transcribe calls, show live closed captions, and retrieve transcripts + + Retrieve call history and details diff --git a/calls/android/session-settings.mdx b/calls/android/session-settings.mdx index 8235eccca..077716c7e 100644 --- a/calls/android/session-settings.mdx +++ b/calls/android/session-settings.mdx @@ -326,6 +326,56 @@ Automatically starts recording the session as soon as it begins. When enabled, r |-----------|------|---------| | `enabled` | boolean | false | +### Auto Start Transcription + +*Available since v5.0.4* + +**Method:** `enableAutoStartTranscription(boolean)` + +Automatically starts transcribing the session as soon as it begins. See [Transcription](/calls/android/transcription) for details. + + + +```kotlin +.enableAutoStartTranscription(true) +``` + + +```java +.enableAutoStartTranscription(true) +``` + + + +| Parameter | Type | Default | +|-----------|------|---------| +| `enabled` | boolean | false | + +### Caption Language + +*Available since v5.0.4* + +**Method:** `setCaptionLanguage(String)` + +Sets the language used for transcription and closed captions. See [Transcription](/calls/android/transcription) for the full list of supported codes. + + + +```kotlin +.setCaptionLanguage("en-US") +``` + + +```java +.setCaptionLanguage("en-US") +``` + + + +| Parameter | Type | Default | +|-----------|------|---------| +| `captionLanguage` | String | "en-US" | + ### Hide Control Panel **Method:** `hideControlPanel(boolean)` @@ -510,6 +560,56 @@ Hides the recording start/stop button from the control panel. Set to `false` to |-----------|------|---------| | `hide` | boolean | true | +### Hide Transcription Button + +*Available since v5.0.4* + +**Method:** `hideTranscriptionButton(boolean)` + +Hides the transcription start/stop item from the control panel's **More** menu. Set to `false` to show it, allowing users to manually control session transcription. + + + +```kotlin +.hideTranscriptionButton(false) +``` + + +```java +.hideTranscriptionButton(false) +``` + + + +| Parameter | Type | Default | +|-----------|------|---------| +| `hide` | boolean | true | + +### Hide Closed Caption Button + +*Available since v5.0.4* + +**Method:** `hideClosedCaptionButton(boolean)` + +Hides the closed-caption (Show/Hide Captions) item from the control panel's **More** menu. Set to `false` to show it, allowing users to toggle the live captions overlay. Even when set to `false`, the item only appears while transcription is running. + + + +```kotlin +.hideClosedCaptionButton(false) +``` + + +```java +.hideClosedCaptionButton(false) +``` + + + +| Parameter | Type | Default | +|-----------|------|---------| +| `hide` | boolean | true | + ### Hide Screen Sharing Button **Method:** `hideScreenSharingButton(boolean)` diff --git a/calls/android/setup.mdx b/calls/android/setup.mdx index 90ca85ac9..ce6e04d95 100644 --- a/calls/android/setup.mdx +++ b/calls/android/setup.mdx @@ -59,14 +59,14 @@ Add the Calls SDK dependency to your **app level** `build.gradle` file: ```groovy dependencies { - implementation "com.cometchat:calls-sdk-android:5.0.0" + implementation "com.cometchat:calls-sdk-android:5.0.4" } ``` ```kotlin dependencies { - implementation("com.cometchat:calls-sdk-android:5.0.0") + implementation("com.cometchat:calls-sdk-android:5.0.4") } ``` diff --git a/calls/android/transcription.mdx b/calls/android/transcription.mdx new file mode 100644 index 000000000..a32634986 --- /dev/null +++ b/calls/android/transcription.mdx @@ -0,0 +1,508 @@ +--- +title: "Transcription & Closed Captions" +sidebarTitle: "Transcription" +sdk_version: "5.x" +description: "Use CometChat Calls SDK v5 transcription on Android to transcribe calls, show live closed captions, and retrieve transcripts after the call." +--- + + +**Available since v5.0.4** — transcription and closed captions require CometChat Calls SDK v5.0.4 or later for Android. See [Setup](/calls/android/setup) to install or upgrade. + + +Transcribe call sessions in real time and display live closed captions on screen. Transcripts are stored server-side and can be retrieved after the call using `TranscriptRequest`. + + +Transcription must be enabled for your CometChat app. Contact support if you need to enable this feature. + + +## How It Works + +Transcription and closed captions are two related but separate things: + +| Concept | What it does | +|---------|--------------| +| **Transcription** | Server-side speech-to-text for the session. Starting it brings a transcriber into the call, which produces the transcript that is stored for later retrieval. | +| **Closed captions** | The on-screen overlay that renders the live transcript as it arrives. Captions are produced from the running transcription, so they only appear while transcription is active. | + +Starting transcription is a prerequisite for captions — toggling captions on without an active transcription shows nothing. + +## Starting Transcription + +### Auto-Start Transcription + +Configure transcription to start automatically when the session begins: + + + +```kotlin +val sessionSettings = CometChatCalls.SessionSettingsBuilder() + .enableAutoStartTranscription(true) + // ... other settings + .build() +``` + + +```java +SessionSettings sessionSettings = new CometChatCalls.SessionSettingsBuilder() + .enableAutoStartTranscription(true) + // ... other settings + .build(); +``` + + + +**Default:** `false` + +### Manual Transcription Control + +Transcription can be started and stopped during an active call through the `CallSession` singleton. + +#### Start Transcription + + + +```kotlin +val callSession = CallSession.getInstance() +callSession.startTranscription() +``` + + +```java +CallSession callSession = CallSession.getInstance(); +callSession.startTranscription(); +``` + + + +#### Stop Transcription + +Stops the current transcription. Any captions currently on screen are cleared: + + + +```kotlin +val callSession = CallSession.getInstance() +callSession.stopTranscription() +``` + + +```java +CallSession callSession = CallSession.getInstance(); +callSession.stopTranscription(); +``` + + + + +Always check `isSessionActive()` before calling these actions to ensure there's an active call. + + +## Built-in UI Controls + +On Android, both transcription controls live in the control panel's **More** menu and are hidden by default. + +### Transcription Button + +To show the **Start Transcription** / **Stop Transcription** item: + + + +```kotlin +val sessionSettings = CometChatCalls.SessionSettingsBuilder() + .hideTranscriptionButton(false) + .build() +``` + + +```java +SessionSettings sessionSettings = new CometChatCalls.SessionSettingsBuilder() + .hideTranscriptionButton(false) + .build(); +``` + + + +**Default:** `true` + +The menu item toggles between **Start Transcription** and **Stop Transcription** based on the current state. + +### Closed Caption Button + +To show the **Show Captions** / **Hide Captions** item, which toggles the on-screen captions overlay: + + + +```kotlin +val sessionSettings = CometChatCalls.SessionSettingsBuilder() + .hideClosedCaptionButton(false) + .build() +``` + + +```java +SessionSettings sessionSettings = new CometChatCalls.SessionSettingsBuilder() + .hideClosedCaptionButton(false) + .build(); +``` + + + +**Default:** `true` + + +Even with `hideClosedCaptionButton(false)`, the captions item only appears once transcription is running for the session, because captions are generated from the live transcript. + + +## Caption Language + +**Method:** `setCaptionLanguage(String)` + +Sets the language used for transcription and captions. This is fixed for the session — there is no in-call language picker on Android. + + + +```kotlin +val sessionSettings = CometChatCalls.SessionSettingsBuilder() + .setCaptionLanguage("en-US") + .build() +``` + + +```java +SessionSettings sessionSettings = new CometChatCalls.SessionSettingsBuilder() + .setCaptionLanguage("en-US") + .build(); +``` + + + +**Default:** `en-US` + + +| Code | Language | +|------|----------| +| `en-US` | English (United States) | +| `de-DE` | German (Germany) | +| `en-GB` | English (United Kingdom) | +| `es-ES` | Spanish (Spain) | +| `fr-FR` | French (France) | +| `hi-IN` | Hindi (India) | +| `hu-HU` | Hungarian (Hungary) | +| `it-IT` | Italian (Italy) | +| `ja-JP` | Japanese (Japan) | +| `ko-KR` | Korean (South Korea) | +| `lt-LT` | Lithuanian (Lithuania) | +| `ms-MY` | Malay (Malaysia) | +| `nl-NL` | Dutch (Netherlands) | +| `pt-PT` | Portuguese (Portugal) | +| `ru-RU` | Russian (Russia) | +| `sv-SE` | Swedish (Sweden) | +| `tr-TR` | Turkish (Turkey) | +| `zh` | Chinese Mandarin (Simplified, China) | +| `zh-TW` | Chinese Mandarin (Traditional, Taiwan) | + + +## Retrieving Transcripts + +After a call, use `TranscriptRequest` to list the transcript artifacts for a session. Each record is a **pointer to a downloadable transcript file**, not the transcript text itself. + + +The SDK must be initialized with `CometChatCalls.init()` and a user must be logged in. The auth token is read from the logged-in user at fetch time, so it automatically tracks re-logins. + + +### Building a Request + + + +```kotlin +val transcriptRequest = TranscriptRequest.TranscriptRequestBuilder() + .setSessionId("v1.us.2547167fe69871fd.alice") // required + .setLimit(10) // optional + .build() + +transcriptRequest.fetchNext(object : CometChatCalls.CallbackListener>() { + override fun onSuccess(transcripts: List) { + for (transcript in transcripts) { + Log.d(TAG, "Transcript ID: ${transcript.tid}") + Log.d(TAG, "Transcript URL: ${transcript.transcriptUrl}") + } + } + + override fun onError(e: CometChatException) { + Log.e(TAG, "Error: ${e.code} ${e.message}") + } +}) +``` + + +```java +TranscriptRequest transcriptRequest = new TranscriptRequest.TranscriptRequestBuilder() + .setSessionId("v1.us.2547167fe69871fd.alice") // required + .setLimit(10) // optional + .build(); + +transcriptRequest.fetchNext(new CometChatCalls.CallbackListener>() { + @Override + public void onSuccess(List transcripts) { + for (Transcript transcript : transcripts) { + Log.d(TAG, "Transcript ID: " + transcript.getTid()); + Log.d(TAG, "Transcript URL: " + transcript.getTranscriptUrl()); + } + } + + @Override + public void onError(CometChatException e) { + Log.e(TAG, "Error: " + e.getCode() + " " + e.getMessage()); + } +}); +``` + + + +| Method | Required | Description | +|--------|----------|-------------| +| `setSessionId(String)` | Yes | The session ID whose transcripts to fetch. A null or empty value is reported as `ERROR_INVALID_SESSIONID` on the first fetch. | +| `setLimit(int)` | No | Page size. Defaults to `30` and is capped at `1000`. A value of `0` or less is reported as `ERROR_NON_POSITIVE_LIMIT` on the first fetch. | +| `build()` | Yes | Returns a `TranscriptRequest`. Never throws — validation happens at fetch time. | + +### Paginating + +A `TranscriptRequest` is a stateful cursor. Create one per session ID and drive it with `fetchNext()` and `fetchPrevious()`: + + + +```kotlin +// Fetch next page +transcriptRequest.fetchNext(object : CometChatCalls.CallbackListener>() { + override fun onSuccess(transcripts: List) { + if (transcripts.isEmpty()) { + // No more pages + } + } + + override fun onError(e: CometChatException) { + Log.e(TAG, "Error: ${e.message}") + } +}) + +// Fetch previous page +transcriptRequest.fetchPrevious(object : CometChatCalls.CallbackListener>() { + override fun onSuccess(transcripts: List) { + // Handle previous page + } + + override fun onError(e: CometChatException) { + Log.e(TAG, "Error: ${e.message}") + } +}) +``` + + +```java +// Fetch next page +transcriptRequest.fetchNext(new CometChatCalls.CallbackListener>() { + @Override + public void onSuccess(List transcripts) { + if (transcripts.isEmpty()) { + // No more pages + } + } + + @Override + public void onError(CometChatException e) { + Log.e(TAG, "Error: " + e.getMessage()); + } +}); + +// Fetch previous page +transcriptRequest.fetchPrevious(new CometChatCalls.CallbackListener>() { + @Override + public void onSuccess(List transcripts) { + // Handle previous page + } + + @Override + public void onError(CometChatException e) { + Log.e(TAG, "Error: " + e.getMessage()); + } +}); +``` + + + +- `fetchNext()` delivers the next page, or an empty list when there are no more pages. A session with no transcripts delivers an empty list. +- `fetchPrevious()` delivers the previous page, or an empty list when already on the first page. It never requests a page below `1`. +- Only one fetch may be in flight at a time. Calling `fetchNext()` or `fetchPrevious()` while another request is pending reports `ERROR_REQUEST_IN_PROGRESS`; the original call is unaffected and still completes. +- Callbacks are always delivered on the main thread. + +### Transcript Object + +| Property | Type | Description | +|----------|------|-------------| +| `tid` | String | Transcript ID | +| `mid` | String | Meeting ID | +| `roomName` | String | Room name of the meeting | +| `startTime` | long | Meeting start time, in epoch **seconds** | +| `endTime` | long | Meeting end time, in epoch **seconds** | +| `url` | String | Meeting URL | +| `transcriptDate` | String | Transcript date | +| `transcriptUrl` | String | URL of the downloadable transcript JSON | +| `metaData` | JSONObject | The full raw record as returned by the server, so any fields not modelled above are still available | + + +Every property is optional. The server omits keys whose value is empty, so String getters may return `null` and numeric getters `0`. Sparse records are normal and should not be treated as an error. + + +### Reading the Transcript Content + +`transcriptUrl` points at the transcript file. Download it with your HTTP client of choice to read the actual utterances: + + + +```kotlin +transcriptRequest.fetchNext(object : CometChatCalls.CallbackListener>() { + override fun onSuccess(transcripts: List) { + val transcriptUrl = transcripts.firstOrNull()?.transcriptUrl + if (transcriptUrl != null) { + // Download the JSON at transcriptUrl using your HTTP client + } + } + + override fun onError(e: CometChatException) { + Log.e(TAG, "Error: ${e.message}") + } +}) +``` + + +```java +transcriptRequest.fetchNext(new CometChatCalls.CallbackListener>() { + @Override + public void onSuccess(List transcripts) { + if (!transcripts.isEmpty() && transcripts.get(0).getTranscriptUrl() != null) { + String transcriptUrl = transcripts.get(0).getTranscriptUrl(); + // Download the JSON at transcriptUrl using your HTTP client + } + } + + @Override + public void onError(CometChatException e) { + Log.e(TAG, "Error: " + e.getMessage()); + } +}); +``` + + + +### Error Handling + +All errors — including pre-flight validation — are delivered to `onError()` with a `CometChatException` carrying a `code`. `build()` never throws. + +| Condition | Code | +|-----------|------| +| `init()` was not called | `ERROR_COMETCHAT_CALLS_SDK_INIT` | +| No logged-in user (no auth token) | `ERROR_AUTH_TOKEN` | +| `sessionId` missing or empty | `ERROR_INVALID_SESSIONID` | +| `limit` is `0` or negative | `ERROR_NON_POSITIVE_LIMIT` | +| A fetch is already in flight | `ERROR_REQUEST_IN_PROGRESS` | +| Missing or malformed response | `ERROR_JSON_EXCEPTION` | +| Network failure or server-side API error | The network or server's own code (e.g. `AUTH_ERR_EMPTY_APPID`) | + +## Complete Example + + + +```kotlin +// 1. Join a session with transcription enabled +val sessionSettings = CometChatCalls.SessionSettingsBuilder() + .setType(SessionType.VIDEO) + .enableAutoStartTranscription(true) + .hideTranscriptionButton(false) + .hideClosedCaptionButton(false) + .setCaptionLanguage("en-US") + .build() + +CometChatCalls.joinSession(callToken, sessionSettings, callViewContainer, + object : CometChatCalls.CallbackListener() { + override fun onSuccess(callSession: CallSession) { + // 2. Control transcription during the call + callSession.startTranscription() + callSession.stopTranscription() + } + + override fun onError(e: CometChatException) { + Log.e(TAG, "Error: ${e.message}") + } +}) + +// 3. Retrieve transcripts after the call +val transcriptRequest = TranscriptRequest.TranscriptRequestBuilder() + .setSessionId(sessionId) + .setLimit(10) + .build() + +transcriptRequest.fetchNext(object : CometChatCalls.CallbackListener>() { + override fun onSuccess(transcripts: List) { + transcripts.forEach { Log.d(TAG, "Transcript URL: ${it.transcriptUrl}") } + } + + override fun onError(e: CometChatException) { + Log.e(TAG, "Error: ${e.message}") + } +}) +``` + + +```java +// 1. Join a session with transcription enabled +SessionSettings sessionSettings = new CometChatCalls.SessionSettingsBuilder() + .setType(SessionType.VIDEO) + .enableAutoStartTranscription(true) + .hideTranscriptionButton(false) + .hideClosedCaptionButton(false) + .setCaptionLanguage("en-US") + .build(); + +CometChatCalls.joinSession(callToken, sessionSettings, callViewContainer, + new CometChatCalls.CallbackListener() { + @Override + public void onSuccess(CallSession callSession) { + // 2. Control transcription during the call + callSession.startTranscription(); + callSession.stopTranscription(); + } + + @Override + public void onError(CometChatException e) { + Log.e(TAG, "Error: " + e.getMessage()); + } +}); + +// 3. Retrieve transcripts after the call +TranscriptRequest transcriptRequest = new TranscriptRequest.TranscriptRequestBuilder() + .setSessionId(sessionId) + .setLimit(10) + .build(); + +transcriptRequest.fetchNext(new CometChatCalls.CallbackListener>() { + @Override + public void onSuccess(List transcripts) { + for (Transcript transcript : transcripts) { + Log.d(TAG, "Transcript URL: " + transcript.getTranscriptUrl()); + } + } + + @Override + public void onError(CometChatException e) { + Log.e(TAG, "Error: " + e.getMessage()); + } +}); +``` + + + +## Related Documentation + +- [Session Settings](/calls/android/session-settings) +- [Actions](/calls/android/actions) +- [Call Logs](/calls/android/call-logs) diff --git a/calls/flutter/actions.mdx b/calls/flutter/actions.mdx index 71411aefa..6a4776445 100644 --- a/calls/flutter/actions.mdx +++ b/calls/flutter/actions.mdx @@ -99,6 +99,30 @@ Stops the current recording. The recording is saved and accessible via the dashb await CallSession.getInstance()?.stopRecording(); ``` +### Start Transcription + +*Available since v5.0.7* + +Begins server-side transcription of the call. This also enables the live closed captions overlay. + +```dart +await CallSession.getInstance()?.startTranscription(); +``` + + +Transcription requires the feature to be enabled for your CometChat app. + + +### Stop Transcription + +*Available since v5.0.7* + +Stops the current transcription and clears any captions currently on screen. The transcript is saved and can be retrieved with [`TranscriptRequestBuilder`](/calls/flutter/transcription#retrieving-transcripts). + +```dart +await CallSession.getInstance()?.stopTranscription(); +``` + ### Mute Participant Mutes a specific participant's audio. This is a moderator action. @@ -327,6 +351,7 @@ Read-only properties on `CallSession` that return the current state of the local | `isVideoPaused` | `bool` | Whether local video is currently paused | | `isHandRaised` | `bool` | Whether the local user's hand is raised | | `isRecording` | `bool` | Whether the call is currently being recorded | +| `isTranscribing` | `bool` | Whether the call is currently being transcribed | ```dart CallSession? session = CallSession.getInstance(); @@ -335,6 +360,7 @@ bool? muted = session?.isAudioMuted; bool? videoPaused = session?.isVideoPaused; bool? handRaised = session?.isHandRaised; bool? recording = session?.isRecording; +bool? transcribing = session?.isTranscribing; ``` diff --git a/calls/flutter/call-logs.mdx b/calls/flutter/call-logs.mdx index e5a202dcd..145c37098 100644 --- a/calls/flutter/call-logs.mdx +++ b/calls/flutter/call-logs.mdx @@ -40,6 +40,7 @@ Configure the request using the builder methods: | `setSessionType(String)` | String | Filter by call type: `video` or `audio` | | `setCallStatus(String)` | String | Filter by call status | | `setHasRecording(bool)` | bool | Filter calls that have recordings | +| `setHasTranscriptions(bool)` | bool | Filter calls that have transcripts, and attach them to each log | | `setCallCategory(String)` | String | Filter by category: `call` or `meet` | | `setCallDirection(String)` | String | Filter by direction: `incoming` or `outgoing` | | `setUid(String)` | String | Filter calls with a specific user | @@ -119,6 +120,7 @@ Each `CallLog` object contains detailed information about a call: | `totalParticipants` | int | Number of participants | | `hasRecording` | bool | Whether the call was recorded | | `recordings` | List\ | List of recording objects | +| `transcriptions` | List\? | List of transcript objects; null unless the request opted in | | `participantInfoList` | List\ | List of participant details | ## Access Recordings @@ -144,6 +146,35 @@ callLogRequest.fetchNext( ); ``` +## Access Transcripts + +*Available since v5.0.7* + +Use `setHasTranscriptions(true)` to fetch only calls that were transcribed. Opting in also makes the server attach each call's transcripts to the log: + +```dart +CallLogRequest transcribedCallsRequest = CallLogRequest.CallLogRequestBuilder() + .setLimit(30) + .setHasTranscriptions(true) + .build(); + +transcribedCallsRequest.fetchNext( + onSuccess: (List callLogs) { + for (CallLog callLog in callLogs) { + for (final transcript in callLog.getTranscriptions()) { + debugPrint("Transcript ID: ${transcript.tid}"); + debugPrint("Transcript URL: ${transcript.transcriptUrl}"); + } + } + }, + onError: (CometChatCallsException e) { + debugPrint("Error: ${e.message}"); + }, +); +``` + +`getTranscriptions()` returns an empty list when the server omitted transcripts, so it never needs a null check. Leaving the filter off sends no filter at all, so the list comes back unfiltered. To page through a single session's transcripts directly, use [`TranscriptRequestBuilder`](/calls/flutter/transcription#retrieving-transcripts). + | Status | Description | |--------|-------------| diff --git a/calls/flutter/migration-guide-v5.mdx b/calls/flutter/migration-guide-v5.mdx index 7168487fb..4ae34c2f4 100644 --- a/calls/flutter/migration-guide-v5.mdx +++ b/calls/flutter/migration-guide-v5.mdx @@ -13,7 +13,7 @@ Calls SDK v5 is a **drop-in replacement** for v4. All v4 APIs are preserved as d ```yaml dependencies: - cometchat_calls_sdk: ^5.0.3 + cometchat_calls_sdk: ^5.0.7 ``` diff --git a/calls/flutter/session-settings.mdx b/calls/flutter/session-settings.mdx index 04f7361b8..1636ba3f6 100644 --- a/calls/flutter/session-settings.mdx +++ b/calls/flutter/session-settings.mdx @@ -214,6 +214,38 @@ Automatically starts recording the session as soon as it begins. When enabled, r |-----------|------|---------| | `enabled` | bool | false | +### Auto Start Transcription + +*Available since v5.0.7* + +**Method:** `enableAutoStartTranscription(bool)` + +Automatically starts transcribing the session as soon as it begins, without requiring anyone to press the transcription button. See [Transcription](/calls/flutter/transcription) for details. + +```dart +..enableAutoStartTranscription(true) +``` + +| Parameter | Type | Default | +|-----------|------|---------| +| `enabled` | bool | false | + +### Caption Language + +*Available since v5.0.7* + +**Method:** `setCaptionLanguage(String)` + +Sets the language used for transcription and closed captions. See [Transcription](/calls/flutter/transcription#caption-language) for the full list of supported codes. + +```dart +..setCaptionLanguage("en-US") +``` + +| Parameter | Type | Default | +|-----------|------|---------| +| `captionLanguage` | String | en-US | + ### Hide Control Panel **Method:** `hideControlPanel(bool)` @@ -326,6 +358,38 @@ Hides the recording start/stop button from the control panel. Set to `false` to |-----------|------|---------| | `hide` | bool | true | +### Hide Transcription Button + +*Available since v5.0.7* + +**Method:** `hideTranscriptionButton(bool)` + +Hides the transcription start/stop item from the control panel's **More** menu. Set to `false` to show it, allowing users to manually control session transcription. + +```dart +..hideTranscriptionButton(false) +``` + +| Parameter | Type | Default | +|-----------|------|---------| +| `hide` | bool | true | + +### Hide Closed Caption Button + +*Available since v5.0.7* + +**Method:** `hideClosedCaptionButton(bool)` + +Hides the closed-caption (CC) button from the control panel. Even when set to `false`, the button only appears while transcription is running, since captions are generated from the live transcript. + +```dart +..hideClosedCaptionButton(false) +``` + +| Parameter | Type | Default | +|-----------|------|---------| +| `hide` | bool | true | + ### Hide Screen Sharing Button **Method:** `hideScreenSharingButton(bool)` diff --git a/calls/flutter/setup.mdx b/calls/flutter/setup.mdx index 63e0e3d38..5774a33d0 100644 --- a/calls/flutter/setup.mdx +++ b/calls/flutter/setup.mdx @@ -28,7 +28,7 @@ Add the `cometchat_calls_sdk` dependency to your `pubspec.yaml` file: ```yaml dependencies: - cometchat_calls_sdk: ^5.0.3 + cometchat_calls_sdk: ^5.0.7 ``` ### Step 2: Install Dependencies diff --git a/calls/flutter/transcription.mdx b/calls/flutter/transcription.mdx new file mode 100644 index 000000000..cfd02a4ac --- /dev/null +++ b/calls/flutter/transcription.mdx @@ -0,0 +1,342 @@ +--- +title: "Transcription & Closed Captions" +sidebarTitle: "Transcription" +sdk_version: "5.x" +description: "Use CometChat Calls SDK v5 transcription on Flutter to transcribe calls, show live closed captions, and retrieve transcripts after the call." +--- + + +**Available since v5.0.7** — transcription and closed captions require CometChat Calls SDK v5.0.7 or later for Flutter. See [Setup](/calls/flutter/setup) to install or upgrade. + + +Transcribe call sessions in real time and display live closed captions on screen. Transcripts are stored server-side and can be retrieved after the call using `TranscriptRequestBuilder`. + + +Transcription must be enabled for your CometChat app. Contact support if you need to enable this feature. + + +## How It Works + +Transcription and closed captions are two related but separate things: + +| Concept | What it does | +|---------|--------------| +| **Transcription** | Server-side speech-to-text for the session. Starting it brings a transcriber into the call, which produces the transcript that is stored for later retrieval. | +| **Closed captions** | The on-screen overlay that renders the live transcript as it arrives. Captions are produced from the running transcription, so they only appear while transcription is active. | + +Starting transcription is a prerequisite for captions — toggling captions on without an active transcription shows nothing. + +## Starting Transcription + +### Auto-Start Transcription + +Configure transcription to start automatically when the session begins: + +```dart +final sessionSettings = (SessionSettingsBuilder() + ..enableAutoStartTranscription(true)) + .build(); +``` + +**Default:** `false` + +### Manual Transcription Control + +#### Start Transcription + +Begin transcribing during an active call: + +```dart +await CallSession.getInstance()?.startTranscription(); +``` + +#### Stop Transcription + +Stop the current transcription. Any captions currently on screen are cleared: + +```dart +await CallSession.getInstance()?.stopTranscription(); +``` + +#### Check Transcription State + +`CallSession` exposes the local transcription state so you can drive a custom control: + +```dart +final isTranscribing = CallSession.getInstance()?.isTranscribing ?? false; +``` + +Both actions throw a `CometChatCallsException` if the underlying call fails — `ERROR_START_TRANSCRIPTION` and `ERROR_STOP_TRANSCRIPTION` respectively: + +```dart +try { + await CallSession.getInstance()?.startTranscription(); +} on CometChatCallsException catch (e) { + debugPrint("${e.code}: ${e.message}"); +} +``` + +## Built-in UI Controls + +### Transcription Button + +The transcription start/stop item in the control panel's **More** menu is hidden by default. To show it: + +```dart +final sessionSettings = (SessionSettingsBuilder() + ..hideTranscriptionButton(false)) + .build(); +``` + +**Default:** `true` + +The menu item toggles between **Start Transcription** and **Stop Transcription** based on the current state. + +### Closed Caption Button + +The closed-caption (CC) button in the control panel is hidden by default. To show it: + +```dart +final sessionSettings = (SessionSettingsBuilder() + ..hideClosedCaptionButton(false)) + .build(); +``` + +**Default:** `true` + + +Even with `hideClosedCaptionButton(false)`, the CC button only appears once transcription is running for the session, because captions are generated from the live transcript. + + +### Closed Caption Settings + +When the CC button is visible, the settings dialog gains a **Closed Caption** tab where the user can pick the caption language and enable or disable the on-screen captions. The gear icon on the captions overlay opens the dialog directly on that tab. + +## Caption Language + +**Method:** `setCaptionLanguage(String)` + +Sets the language used for transcription and captions. + +```dart +final sessionSettings = (SessionSettingsBuilder() + ..setCaptionLanguage("en-US")) + .build(); +``` + +**Default:** `en-US` + + +| Code | Language | +|------|----------| +| `en-US` | English (United States) | +| `de-DE` | German (Germany) | +| `en-GB` | English (United Kingdom) | +| `es-ES` | Spanish (Spain) | +| `fr-FR` | French (France) | +| `hi-IN` | Hindi (India) | +| `hu-HU` | Hungarian (Hungary) | +| `it-IT` | Italian (Italy) | +| `ja-JP` | Japanese (Japan) | +| `ko-KR` | Korean (South Korea) | +| `lt-LT` | Lithuanian (Lithuania) | +| `ms-MY` | Malay (Malaysia) | +| `nl-NL` | Dutch (Netherlands) | +| `pt-PT` | Portuguese (Portugal) | +| `ru-RU` | Russian (Russia) | +| `sv-SE` | Swedish (Sweden) | +| `tr-TR` | Turkish (Turkey) | +| `zh` | Chinese Mandarin (Simplified, China) | +| `zh-TW` | Chinese Mandarin (Traditional, Taiwan) | + + +## Retrieving Transcripts + +After a call, use `TranscriptRequestBuilder` to list the transcript artifacts for a session. Each record is a **pointer to a downloadable transcript file**, not the transcript text itself. + + +The SDK must be initialized with `CometChatCalls.init()` and a user must be logged in. The auth token is read from the logged-in user at fetch time, so it automatically tracks re-logins — there is no auth token setter on the builder. + + +### Building a Request + +```dart +final request = (TranscriptRequestBuilder() + ..setSessionId("v1.us.2547167fe69871fd.alice") // required + ..setLimit(10)) // optional + .build(); + +request.fetchNext( + onSuccess: (List transcripts) { + for (final transcript in transcripts) { + debugPrint("Transcript URL: ${transcript.transcriptUrl}"); + } + }, + onError: (CometChatCallsException e) { + debugPrint("Error: ${e.code} ${e.message}"); + }, +); +``` + +| Method | Required | Description | +|--------|----------|-------------| +| `setSessionId(String)` | Yes | The session ID whose transcripts to fetch. A missing or blank value is reported to `onError` as `ERR_SESSION_ID_EMPTY`. | +| `setLimit(int)` | No | Page size. Defaults to `30` and is clamped to a maximum of `1000`. A non-positive value is reported as `ERROR_NON_POSITIVE_LIMIT`. | +| `build()` | Yes | Returns a `TranscriptRequest`. It never throws — all validation happens at fetch time and is delivered to `onError`. | + + +Both builder methods have equivalent public fields, so `TranscriptRequestBuilder()..sessionId = "..."` works too. + + +### Paginating + +A `TranscriptRequest` is a stateful cursor. Create one per session ID and drive it with `fetchNext()` and `fetchPrevious()`. Both also return a `Future` that resolves with the same page handed to `onSuccess`, so you can `await` them instead of nesting callbacks: + +```dart +final request = (TranscriptRequestBuilder() + ..setSessionId(sessionId) + ..setLimit(10)) + .build(); + +var page = await request.fetchNext( + onSuccess: (transcripts) {}, + onError: (e) => debugPrint("Error: ${e.message}"), +); + +while (page.isNotEmpty) { + for (final transcript in page) { + debugPrint(transcript.transcriptUrl ?? ""); + } + page = await request.fetchNext( + onSuccess: (transcripts) {}, + onError: (e) => debugPrint("Error: ${e.message}"), + ); +} +``` + +- `fetchNext()` delivers the next page, or an empty list once the last page has been reached. A session with no transcripts delivers an empty list on the first call. +- `fetchPrevious()` delivers the previous page, or an empty list when already on the first page. It never requests a page below `1`. +- Only one fetch may be in flight at a time. Calling `fetchNext()` or `fetchPrevious()` while another request is pending reports `ERROR_REQUEST_IN_PROGRESS` to `onError`; the original call is unaffected and still completes. +- The returned `Future` never completes with an error — failures always arrive through `onError` and the future resolves with an empty list, so a caller that does not `await` can never trip an unhandled async exception. + +### Transcript Properties + +| Property | Type | Description | +|----------|------|-------------| +| `tid` | String? | Transcript ID | +| `mid` | String? | Meeting ID | +| `roomName` | String? | Room name of the meeting | +| `startTime` | int? | Meeting start time, in epoch **seconds** | +| `endTime` | int? | Meeting end time, in epoch **seconds** | +| `url` | String? | Meeting URL | +| `transcriptDate` | String? | Transcript date | +| `transcriptUrl` | String? | URL of the downloadable transcript JSON | +| `metaData` | Map\? | The raw server record, so new or unknown keys are never lost | + + +Every property is optional. The server omits keys whose value is empty, so sparse records are normal and should not be treated as an error. A malformed entry within a page is skipped rather than failing the whole page. + + +### Reading the Transcript Content + +`transcriptUrl` points at the transcript file. Fetch it yourself — with `package:http` or any client of your choice — to read the actual utterances: + +```dart +request.fetchNext( + onSuccess: (List transcripts) async { + for (final transcript in transcripts) { + final url = transcript.transcriptUrl; + if (url == null) continue; + + final response = await http.get(Uri.parse(url)); + debugPrint(response.body); + } + }, + onError: (CometChatCallsException e) { + debugPrint("Error: ${e.message}"); + }, +); +``` + +### Error Handling + +Every failure — pre-flight validation and server errors alike — is delivered to `onError` as a `CometChatCallsException` carrying a `code`, `message` and `details`: + +| Condition | Code | +|-----------|------| +| `init()` was not called | `ERR_SDK_NOT_INITIALIZED` | +| No logged-in user / auth token | `USER_AUTH_TOKEN_NULL` | +| `sessionId` missing or blank | `ERR_SESSION_ID_EMPTY` | +| `limit` is zero or negative | `ERROR_NON_POSITIVE_LIMIT` | +| A fetch is already in flight | `ERROR_REQUEST_IN_PROGRESS` | +| Missing or malformed response | `ERROR_JSON_EXCEPTION` | +| Server-side API error | The server's own code | + +## Transcripts in Call Logs + +Call logs can be filtered to transcribed calls, which also attaches each call's transcripts to the log: + +```dart +CallLogRequest callLogRequest = CallLogRequest.CallLogRequestBuilder() + .setLimit(30) + .setHasTranscriptions(true) + .build(); + +callLogRequest.fetchNext( + onSuccess: (List callLogs) { + for (CallLog callLog in callLogs) { + for (final transcript in callLog.getTranscriptions()) { + debugPrint("${transcript.tid}: ${transcript.transcriptUrl}"); + } + } + }, + onError: (CometChatCallsException e) { + debugPrint("Error: ${e.message}"); + }, +); +``` + +`getTranscriptions()` returns an empty list when the server omitted transcripts, so it never needs a null check. Leaving the filter off sends no filter at all, so the list comes back unfiltered exactly as if it had never been set. + +## Complete Example + +```dart +// 1. Join a session with transcription enabled +final sessionSettings = (SessionSettingsBuilder() + ..setType(SessionType.video) + ..enableAutoStartTranscription(true) + ..hideTranscriptionButton(false) + ..hideClosedCaptionButton(false) + ..setCaptionLanguage("en-US")) + .build(); + +// Pass sessionSettings to the CometChatCallsView / joinSession() call. + +// 2. Control transcription during the call +await CallSession.getInstance()?.startTranscription(); +await CallSession.getInstance()?.stopTranscription(); + +// 3. Retrieve transcripts after the call +final request = (TranscriptRequestBuilder() + ..setSessionId(sessionId) + ..setLimit(10)) + .build(); + +request.fetchNext( + onSuccess: (List transcripts) { + for (final transcript in transcripts) { + debugPrint(transcript.transcriptUrl ?? ""); + } + }, + onError: (CometChatCallsException e) { + debugPrint("Error: ${e.message}"); + }, +); +``` + +## Related Documentation + +- [SessionSettingsBuilder](/calls/flutter/session-settings) +- [Actions](/calls/flutter/actions) +- [Call Logs](/calls/flutter/call-logs) +- [Recording](/calls/flutter/recording) diff --git a/calls/ios/actions.mdx b/calls/ios/actions.mdx index 5d9a770f8..924e83690 100644 --- a/calls/ios/actions.mdx +++ b/calls/ios/actions.mdx @@ -185,6 +185,48 @@ CallSession.shared.stopRecording() +### Start Transcription + +*Available since v5.0.4* + +Begins server-side transcription of the call. This also enables the live closed captions overlay. + + + +```swift +CallSession.shared.startTranscription() +``` + + +```objectivec +[[CallSession shared] startTranscription]; +``` + + + + +Transcription requires the feature to be enabled for your CometChat app. + + +### Stop Transcription + +*Available since v5.0.4* + +Stops the current transcription and clears any captions currently on screen. The transcript is saved and can be retrieved with [`TranscriptsRequest`](/calls/ios/transcription#retrieving-transcripts). + + + +```swift +CallSession.shared.stopTranscription() +``` + + +```objectivec +[[CallSession shared] stopTranscription]; +``` + + + ### Mute Participant Mutes a specific participant's audio. This is a moderator action. diff --git a/calls/ios/call-logs.mdx b/calls/ios/call-logs.mdx index b8b66f643..3b4d64e94 100644 --- a/calls/ios/call-logs.mdx +++ b/calls/ios/call-logs.mdx @@ -14,15 +14,15 @@ Use `CallLogsRequest` to fetch call logs with pagination support. The builder pa ```swift -let callLogRequest = CallLogsRequest.CallLogsRequestBuilder() - .setLimit(30) +let callLogRequest = CallLogsRequest.CallLogsBuilder() + .set(limit: 30) .build() callLogRequest.fetchNext(onSuccess: { callLogs in for callLog in callLogs { - print("Session: \(callLog.sessionID ?? "")") - print("Duration: \(callLog.totalDuration ?? "")") - print("Status: \(callLog.status ?? "")") + print("Session: \(callLog.sessionID)") + print("Duration: \(callLog.totalDuration)") + print("Status: \(callLog.status.value)") } }, onError: { error in print("Error: \(error?.errorDescription ?? "")") @@ -31,15 +31,13 @@ callLogRequest.fetchNext(onSuccess: { callLogs in ```objectivec -CallLogsRequest *callLogRequest = [[[CallLogsRequest CallLogsRequestBuilder] - setLimit:30] - build]; +CallLogsBuilder *builder = [[CallLogsBuilder alloc] init]; +CallLogsRequest *callLogRequest = [[builder setWithLimit:30] build]; [callLogRequest fetchNextOnSuccess:^(NSArray * callLogs) { for (CallLog *callLog in callLogs) { NSLog(@"Session: %@", callLog.sessionID); NSLog(@"Duration: %@", callLog.totalDuration); - NSLog(@"Status: %@", callLog.status); } } onError:^(CometChatCallException * error) { NSLog(@"Error: %@", error.errorDescription); @@ -48,20 +46,25 @@ CallLogsRequest *callLogRequest = [[[CallLogsRequest CallLogsRequestBuilder] -## CallLogsRequestBuilder +## CallLogsBuilder Configure the request using the builder methods: | Method | Type | Description | |--------|------|-------------| -| `setLimit(Int)` | Int | Number of call logs to fetch per request (default: 30, max: 100) | -| `setSessionType(String)` | String | Filter by call type: `video` or `audio` | -| `setCallStatus(String)` | String | Filter by call status | -| `setHasRecording(Bool)` | Bool | Filter calls that have recordings | -| `setCallCategory(String)` | String | Filter by category: `call` or `meet` | -| `setCallDirection(String)` | String | Filter by direction: `incoming` or `outgoing` | -| `setUid(String)` | String | Filter calls with a specific user | -| `setGuid(String)` | String | Filter calls with a specific group | +| `set(limit: Int)` | Int | Number of call logs to fetch per request (default: 30, max: 100) | +| `set(callType: SessionType)` | SessionType | Filter by call type: `.video` or `.voice` | +| `set(callStatus: CallStatus)` | CallStatus | Filter by call status | +| `set(hasRecording: Bool)` | Bool | Filter calls that have recordings | +| `set(hasTranscriptions: Bool)` | Bool | Filter calls that have transcripts, and attach them to each log | +| `set(callCategory: CallCategory)` | CallCategory | Filter by category: `.call`, `.meet`, `.presenter` or `.broadcast` | +| `set(callDirection: CallDirection)` | CallDirection | Filter by direction: `.incoming` or `.outgoing` | +| `set(uid: String)` | String | Filter calls with a specific user | +| `set(guid: String)` | String | Filter calls with a specific group | + + +In Objective-C these are `setWithLimit:`, `setWithCallType:`, `setWithHasTranscriptions:` and so on, and the builder is instantiated directly as `[[CallLogsBuilder alloc] init]`. + ### Filter Examples @@ -69,50 +72,60 @@ Configure the request using the builder methods: ```swift // Fetch only video calls -let videoCallsRequest = CallLogsRequest.CallLogsRequestBuilder() - .setSessionType("video") - .setLimit(20) +let videoCallsRequest = CallLogsRequest.CallLogsBuilder() + .set(callType: .video) + .set(limit: 20) .build() // Fetch calls with recordings -let recordedCallsRequest = CallLogsRequest.CallLogsRequestBuilder() - .setHasRecording(true) +let recordedCallsRequest = CallLogsRequest.CallLogsBuilder() + .set(hasRecording: true) + .build() + +// Fetch calls with transcripts +let transcribedCallsRequest = CallLogsRequest.CallLogsBuilder() + .set(hasTranscriptions: true) .build() // Fetch missed incoming calls -let missedCallsRequest = CallLogsRequest.CallLogsRequestBuilder() - .setCallStatus("missed") - .setCallDirection("incoming") +let missedCallsRequest = CallLogsRequest.CallLogsBuilder() + .set(callStatus: .missed) + .set(callDirection: .incoming) .build() // Fetch calls with a specific user -let userCallsRequest = CallLogsRequest.CallLogsRequestBuilder() - .setUid("user_id") +let userCallsRequest = CallLogsRequest.CallLogsBuilder() + .set(uid: "user_id") .build() ``` ```objectivec // Fetch only video calls -CallLogsRequest *videoCallsRequest = [[[[CallLogsRequest CallLogsRequestBuilder] - setSessionType:@"video"] - setLimit:20] +CallLogsRequest *videoCallsRequest = [[[[[CallLogsBuilder alloc] init] + setWithCallType:SessionTypeVideo] + setWithLimit:20] build]; // Fetch calls with recordings -CallLogsRequest *recordedCallsRequest = [[[CallLogsRequest CallLogsRequestBuilder] - setHasRecording:YES] +CallLogsRequest *recordedCallsRequest = [[[[CallLogsBuilder alloc] init] + setWithHasRecording:YES] + build]; + +// Fetch calls with transcripts +CallLogsRequest *transcribedCallsRequest = [[[[CallLogsBuilder alloc] init] + setWithHasTranscriptions:YES] build]; // Fetch missed incoming calls -CallLogsRequest *missedCallsRequest = [[[[CallLogsRequest CallLogsRequestBuilder] - setCallStatus:@"missed"] - setCallDirection:@"incoming"] +CallLogsRequest *missedCallsRequest = [[[[[CallLogsBuilder alloc] init] + setWithCallStatus:CallStatusMissed] + setWithCallDirection:CallDirectionIncoming] build]; // Fetch calls with a specific user -CallLogsRequest *userCallsRequest = [[[CallLogsRequest CallLogsRequestBuilder] - setUid:@"user_id"] +CallLogsRequest *userCallsRequest = [[[[CallLogsBuilder alloc] init] + setWithUid:@"user_id"] build]; ``` @@ -120,7 +133,7 @@ CallLogsRequest *userCallsRequest = [[[CallLogsRequest CallLogsRequestBuilder] ## Pagination -Use `fetchNext()` and `fetchPrevious()` for pagination: +Use `fetchNext` and `fetchPrevious` for pagination. `fetchPrevious` takes an optional `authToken`; pass `nil` to use the logged-in user's stored token: @@ -133,7 +146,7 @@ callLogRequest.fetchNext(onSuccess: { callLogs in }) // Fetch previous page -callLogRequest.fetchPrevious(onSuccess: { callLogs in +callLogRequest.fetchPrevious(authToken: nil, onSuccess: { callLogs in // Handle previous page }, onError: { error in print("Error: \(error?.errorDescription ?? "")") @@ -150,11 +163,12 @@ callLogRequest.fetchPrevious(onSuccess: { callLogs in }]; // Fetch previous page -[callLogRequest fetchPreviousOnSuccess:^(NSArray * callLogs) { - // Handle previous page -} onError:^(CometChatCallException * error) { - NSLog(@"Error: %@", error.errorDescription); -}]; +[callLogRequest fetchPreviousWithAuthToken:nil + onSuccess:^(NSArray * callLogs) { + // Handle previous page + } onError:^(CometChatCallException * error) { + NSLog(@"Error: %@", error.errorDescription); + }]; ``` @@ -166,21 +180,25 @@ Each `CallLog` object contains detailed information about a call: | Property | Type | Description | |----------|------|-------------| | `sessionID` | String | Unique identifier for the call session | +| `mid` | String | Meeting identifier | | `initiator` | CallEntity | User who initiated the call | | `receiver` | CallEntity | User or group that received the call | -| `receiverType` | String | `user` or `group` | -| `type` | String | Call type: `video` or `audio` | -| `status` | String | Final status of the call | -| `callCategory` | String | Category: `call` or `meet` | +| `receiverType` | CallEntityType | `.callUser` or `.callGroup` | +| `type` | SessionType | Call type: `.video` or `.voice` | +| `status` | CallStatus | Final status of the call | +| `mode` | CallCategory | Category: `.call`, `.meet`, `.presenter` or `.broadcast` | | `initiatedAt` | Int | Timestamp when call was initiated | -| `endedAt` | Int | Timestamp when call ended | +| `startedAt` | Int? | Timestamp when call started | +| `endedAt` | Int? | Timestamp when call ended | | `totalDuration` | String | Human-readable duration (e.g., "5:30") | | `totalDurationInMinutes` | Double | Duration in minutes | | `totalAudioMinutes` | Double | Audio duration in minutes | | `totalVideoMinutes` | Double | Video duration in minutes | | `totalParticipants` | Int | Number of participants | +| `participants` | [Participant] | List of participants who joined | | `hasRecording` | Bool | Whether the call was recorded | | `recordings` | [Recording] | List of recording objects | +| `transcriptions` | [Transcript] | List of transcript objects. Populated only when the request opted in with `set(hasTranscriptions:)` | ## Access Recordings @@ -192,10 +210,10 @@ If a call has recordings, access them through the `recordings` property: callLogRequest.fetchNext(onSuccess: { callLogs in for callLog in callLogs { if callLog.hasRecording { - for recording in callLog.recordings ?? [] { + for recording in callLog.recordings { print("Recording ID: \(recording.rid ?? "")") print("Recording URL: \(recording.recordingURL ?? "")") - print("Duration: \(recording.duration) seconds") + print("Duration: \(recording.duration ?? 0) seconds") } } } @@ -223,6 +241,55 @@ callLogRequest.fetchNext(onSuccess: { callLogs in +## Access Transcripts + +*Available since v5.0.4* + +Opting in with `set(hasTranscriptions: true)` restricts the list to transcribed calls **and** makes the server attach each call's transcripts to the log: + + + +```swift +let callLogRequest = CallLogsRequest.CallLogsBuilder() + .set(limit: 30) + .set(hasTranscriptions: true) + .build() + +callLogRequest.fetchNext(onSuccess: { callLogs in + for callLog in callLogs { + for transcript in callLog.transcriptions { + print("Transcript ID: \(transcript.tid)") + print("Transcript URL: \(transcript.transcriptUrl)") + } + } +}, onError: { error in + print("Error: \(error?.errorDescription ?? "")") +}) +``` + + +```objectivec +CallLogsRequest *callLogRequest = [[[[[CallLogsBuilder alloc] init] + setWithLimit:30] + setWithHasTranscriptions:YES] + build]; + +[callLogRequest fetchNextOnSuccess:^(NSArray * callLogs) { + for (CallLog *callLog in callLogs) { + for (Transcript *transcript in callLog.transcriptions) { + NSLog(@"Transcript ID: %@", transcript.tid); + NSLog(@"Transcript URL: %@", transcript.transcriptUrl); + } + } +} onError:^(CometChatCallException * error) { + NSLog(@"Error: %@", error.errorDescription); +}]; +``` + + + +`transcriptions` is an empty array when the server omitted transcripts, so it never needs a nil check. To page through a single session's transcripts directly, use [`TranscriptsRequest`](/calls/ios/transcription#retrieving-transcripts). + | Status | Description | |--------|-------------| @@ -249,3 +316,8 @@ callLogRequest.fetchNext(onSuccess: { callLogs in | `incoming` | Call received by the user | | `outgoing` | Call initiated by the user | + +## Related Documentation + +- [Transcription](/calls/ios/transcription) +- [Recording](/calls/ios/recording) diff --git a/calls/ios/session-settings.mdx b/calls/ios/session-settings.mdx index 6a8e32243..09ab72cd0 100644 --- a/calls/ios/session-settings.mdx +++ b/calls/ios/session-settings.mdx @@ -303,6 +303,56 @@ Automatically starts recording the session as soon as it begins. When enabled, r |-----------|------|---------| | `enabled` | Bool | false | +### Auto Start Transcription + +*Available since v5.0.4* + +**Method:** `enableAutoStartTranscription(_ enabled: Bool)` + +Automatically starts transcribing the session as soon as it begins. See [Transcription](/calls/ios/transcription) for details. + + + +```swift +.enableAutoStartTranscription(true) +``` + + +```objectivec +[builder enableAutoStartTranscription:YES] +``` + + + +| Parameter | Type | Default | +|-----------|------|---------| +| `enabled` | Bool | false | + +### Caption Language + +*Available since v5.0.4* + +**Method:** `setCaptionLanguage(_ captionLanguage: String)` + +Sets the language used for transcription and closed captions. See [Transcription](/calls/ios/transcription#caption-language) for the full list of supported codes. + + + +```swift +.setCaptionLanguage("en-US") +``` + + +```objectivec +[builder setCaptionLanguage:@"en-US"] +``` + + + +| Parameter | Type | Default | +|-----------|------|---------| +| `captionLanguage` | String | en-US | + ### Hide Control Panel **Method:** `hideControlPanel(_ hidden: Bool)` @@ -487,6 +537,56 @@ Hides the recording start/stop button from the control panel. Set to `false` to |-----------|------|---------| | `enabled` | Bool | true | +### Hide Transcription Button + +*Available since v5.0.4* + +**Method:** `hideTranscriptionButton(_ hidden: Bool)` + +Hides the transcription start/stop item from the control panel's **More** menu. Set to `false` to show it, allowing users to manually control session transcription. + + + +```swift +.hideTranscriptionButton(false) +``` + + +```objectivec +[builder hideTranscriptionButton:NO] +``` + + + +| Parameter | Type | Default | +|-----------|------|---------| +| `hidden` | Bool | true | + +### Hide Closed Caption Button + +*Available since v5.0.4* + +**Method:** `hideClosedCaptionButton(_ hidden: Bool)` + +Hides the closed-caption item from the control panel's **More** menu. Set to `false` to show it, allowing users to toggle the live captions overlay. Captions only appear while transcription is running. + + + +```swift +.hideClosedCaptionButton(false) +``` + + +```objectivec +[builder hideClosedCaptionButton:NO] +``` + + + +| Parameter | Type | Default | +|-----------|------|---------| +| `hidden` | Bool | true | + ### Hide Audio Mode Button **Method:** `hideAudioModeButton(_ enabled: Bool)` diff --git a/calls/ios/setup.mdx b/calls/ios/setup.mdx index 98c54f2ea..5d7b08d8f 100644 --- a/calls/ios/setup.mdx +++ b/calls/ios/setup.mdx @@ -31,7 +31,7 @@ platform :ios, '16.0' use_frameworks! target 'YourApp' do - pod 'CometChatCallsSDK', '~> 5.0.0' + pod 'CometChatCallsSDK', '~> 5.0.4' end ``` diff --git a/calls/ios/transcription.mdx b/calls/ios/transcription.mdx new file mode 100644 index 000000000..426f67a4b --- /dev/null +++ b/calls/ios/transcription.mdx @@ -0,0 +1,492 @@ +--- +title: "Transcription & Closed Captions" +sidebarTitle: "Transcription" +sdk_version: "5.x" +description: "Use CometChat Calls SDK v5 transcription on iOS to transcribe calls, show live closed captions, and retrieve transcripts after the call." +--- + + +**Available since v5.0.4** — transcription and closed captions require CometChat Calls SDK v5.0.4 or later for iOS. See [Setup](/calls/ios/setup) to install or upgrade. + + +Transcribe call sessions in real time and display live closed captions on screen. Transcripts are stored server-side and can be retrieved after the call using `TranscriptsRequest`. + + +Transcription must be enabled for your CometChat app. Contact support if you need to enable this feature. + + +## How It Works + +Transcription and closed captions are two related but separate things: + +| Concept | What it does | +|---------|--------------| +| **Transcription** | Server-side speech-to-text for the session. Starting it brings a transcriber into the call, which produces the transcript that is stored for later retrieval. | +| **Closed captions** | The on-screen overlay that renders the live transcript as it arrives. Captions are produced from the running transcription, so they only appear while transcription is active. | + +Starting transcription is a prerequisite for captions — turning captions on without an active transcription shows nothing. + +## Starting Transcription + +### Auto-Start Transcription + +Configure transcription to start automatically when the session begins: + + + +```swift +let sessionSettings = CometChatCalls.sessionSettingsBuilder + .enableAutoStartTranscription(true) + .build() +``` + + +```objectivec +SessionSettings *sessionSettings = [[[CometChatCalls sessionSettingsBuilder] + enableAutoStartTranscription:YES] + build]; +``` + + + +**Default:** `false` + +### Manual Transcription Control + +#### Start Transcription + +Begin transcribing during an active call: + + + +```swift +CallSession.shared.startTranscription() +``` + + +```objectivec +[[CallSession shared] startTranscription]; +``` + + + +#### Stop Transcription + +Stop the current transcription. Any captions currently on screen are cleared: + + + +```swift +CallSession.shared.stopTranscription() +``` + + +```objectivec +[[CallSession shared] stopTranscription]; +``` + + + +## Built-in UI Controls + +Both transcription controls live in the control panel's **More** menu, and both are hidden by default. + +### Transcription Menu Item + +To show the transcription item: + + + +```swift +let sessionSettings = CometChatCalls.sessionSettingsBuilder + .hideTranscriptionButton(false) + .build() +``` + + +```objectivec +SessionSettings *sessionSettings = [[[CometChatCalls sessionSettingsBuilder] + hideTranscriptionButton:NO] + build]; +``` + + + +**Default:** `true` + +The item toggles between **Start Transcription** and **Stop Transcription** based on the current state. + +### Closed Caption Menu Item + +To show the closed-caption item: + + + +```swift +let sessionSettings = CometChatCalls.sessionSettingsBuilder + .hideClosedCaptionButton(false) + .build() +``` + + +```objectivec +SessionSettings *sessionSettings = [[[CometChatCalls sessionSettingsBuilder] + hideClosedCaptionButton:NO] + build]; +``` + + + +**Default:** `true` + +The item toggles between **Show Captions** and **Hide Captions**. + + +Captions are off by default, so the overlay appears only after the user turns them on **and** transcription is running for the session. Turning captions on before transcription starts shows nothing until the transcript begins arriving. + + +## Caption Language + +**Method:** `setCaptionLanguage(_ captionLanguage: String)` + +Sets the language used for transcription and captions. + + + +```swift +let sessionSettings = CometChatCalls.sessionSettingsBuilder + .setCaptionLanguage("en-US") + .build() +``` + + +```objectivec +SessionSettings *sessionSettings = [[[CometChatCalls sessionSettingsBuilder] + setCaptionLanguage:@"en-US"] + build]; +``` + + + +**Default:** `en-US` + + +| Code | Language | +|------|----------| +| `en-US` | English (United States) | +| `de-DE` | German (Germany) | +| `en-GB` | English (United Kingdom) | +| `es-ES` | Spanish (Spain) | +| `fr-FR` | French (France) | +| `hi-IN` | Hindi (India) | +| `hu-HU` | Hungarian (Hungary) | +| `it-IT` | Italian (Italy) | +| `ja-JP` | Japanese (Japan) | +| `ko-KR` | Korean (South Korea) | +| `lt-LT` | Lithuanian (Lithuania) | +| `ms-MY` | Malay (Malaysia) | +| `nl-NL` | Dutch (Netherlands) | +| `pt-PT` | Portuguese (Portugal) | +| `ru-RU` | Russian (Russia) | +| `sv-SE` | Swedish (Sweden) | +| `tr-TR` | Turkish (Turkey) | +| `zh` | Chinese Mandarin (Simplified, China) | +| `zh-TW` | Chinese Mandarin (Traditional, Taiwan) | + + +## Retrieving Transcripts + +After a call, use `TranscriptsRequest` to list the transcript artifacts for a session. Each record is a **pointer to a downloadable transcript file**, not the transcript text itself. + + +The SDK must be initialized with `CometChatCalls.init()` and a user must be logged in. The auth token is read from the logged-in user at fetch time, so it automatically tracks re-logins — there is no `set(authToken:)` on this builder. + + +### Building a Request + + + +```swift +let request = TranscriptsRequest.TranscriptsBuilder() + .set(sessionId: "v1.us.2547167fe69871fd.alice") // required + .set(limit: 10) // optional + .build() + +request.fetchNext(onSuccess: { transcripts in + for transcript in transcripts { + print(transcript.tid, transcript.transcriptUrl) + } +}, onError: { error in + print("Error: \(error?.errorDescription ?? "")") +}) +``` + + +```objectivec +TranscriptsBuilder *builder = [[TranscriptsBuilder alloc] init]; +TranscriptsRequest *request = [[[builder + setWithSessionId:@"v1.us.2547167fe69871fd.alice"] + setWithLimit:10] + build]; + +[request fetchNextOnSuccess:^(NSArray * transcripts) { + for (Transcript *transcript in transcripts) { + NSLog(@"%@ %@", transcript.tid, transcript.transcriptUrl); + } +} onError:^(CometChatCallException * error) { + NSLog(@"Error: %@", error.errorDescription); +}]; +``` + + + +| Method | Required | Description | +|--------|----------|-------------| +| `set(sessionId: String)` | Yes | The session ID whose transcripts to fetch. A missing or empty value fails the fetch with `ERROR_SESSION_ID_NILL`. | +| `set(limit: Int)` | No | Page size. Defaults to `30` and is clamped to the range `1`–`1000`. | +| `build()` | Yes | Returns a `TranscriptsRequest`. | + + +Callbacks are delivered on the main queue, so it is safe to update UI directly from them. + + +### Paginating + +A `TranscriptsRequest` is a stateful cursor. Create one per session ID and drive it with `fetchNext` and `fetchPrevious`: + + + +```swift +let request = TranscriptsRequest.TranscriptsBuilder() + .set(sessionId: sessionId) + .set(limit: 10) + .build() + +// Next page +request.fetchNext(onSuccess: { transcripts in + // [Transcript] — empty once the last page has been consumed +}, onError: { error in + print("Error: \(error?.errorDescription ?? "")") +}) + +// Previous page +request.fetchPrevious(onSuccess: { transcripts in + // [Transcript] — empty on a fresh request, or when already at the first page +}, onError: { error in + print("Error: \(error?.errorDescription ?? "")") +}) +``` + + +```objectivec +// Next page +[request fetchNextOnSuccess:^(NSArray * transcripts) { + // Empty once the last page has been consumed +} onError:^(CometChatCallException * error) { + NSLog(@"Error: %@", error.errorDescription); +}]; + +// Previous page +[request fetchPreviousOnSuccess:^(NSArray * transcripts) { + // Empty on a fresh request, or when already at the first page +} onError:^(CometChatCallException * error) { + NSLog(@"Error: %@", error.errorDescription); +}]; +``` + + + +- `fetchNext` delivers the next page, or an empty array when there are no more pages. A session with no transcripts delivers an empty array. +- `fetchPrevious` delivers the previous page, or an empty array when already on the first page. It never requests a page below `1`. +- The cursor is committed only from a successful response, so a failed fetch can simply be retried. +- Only one fetch may be in flight per request instance. Calling `fetchNext` or `fetchPrevious` while another is pending fails with `ERROR_REQUEST_IN_PROGRESS`; the original call is unaffected and still completes. + +### Transcript Properties + +| Property | Type | Description | +|----------|------|-------------| +| `tid` | String | Transcript ID | +| `mid` | String | Meeting ID | +| `roomName` | String | Room name of the meeting | +| `startTime` | Int | Meeting start time, in epoch **seconds** | +| `endTime` | Int | Meeting end time, in epoch **seconds** | +| `url` | String | Meeting URL | +| `transcriptDate` | String | Transcript date | +| `transcriptUrl` | String | URL of the downloadable transcript JSON | +| `metaData` | [String: Any] | The raw server entry, so fields not modelled above are still reachable | + + +The server omits keys whose value is empty, so sparse records are normal and should not be treated as an error. Missing string fields arrive as `""` and missing numbers as `0`. A single entry that fails to parse is skipped rather than failing the whole page. + + +### Reading the Transcript Content + +`transcriptUrl` points at the transcript file. Fetch it yourself to read the actual utterances: + + + +```swift +request.fetchNext(onSuccess: { transcripts in + guard let transcript = transcripts.first, + let url = URL(string: transcript.transcriptUrl), + !transcript.transcriptUrl.isEmpty else { return } + + URLSession.shared.dataTask(with: url) { data, _, _ in + guard let data = data else { return } + let content = try? JSONSerialization.jsonObject(with: data) + print(content ?? "") + }.resume() +}, onError: { error in + print("Error: \(error?.errorDescription ?? "")") +}) +``` + + +```objectivec +[request fetchNextOnSuccess:^(NSArray * transcripts) { + Transcript *transcript = transcripts.firstObject; + if (transcript.transcriptUrl.length == 0) { return; } + + NSURL *url = [NSURL URLWithString:transcript.transcriptUrl]; + [[[NSURLSession sharedSession] dataTaskWithURL:url + completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) { + if (data == nil) { return; } + id content = [NSJSONSerialization JSONObjectWithData:data options:0 error:nil]; + NSLog(@"%@", content); + }] resume]; +} onError:^(CometChatCallException * error) { + NSLog(@"Error: %@", error.errorDescription); +}]; +``` + + + +### Error Handling + +Failures are delivered to the `onError` closure as a `CometChatCallException` carrying an `errorCode`: + +| Condition | `errorCode` | +|-----------|-------------| +| `CometChatCalls.init()` was not called | `INIT_NOT_CALLED` | +| No logged-in user / empty auth token | `ERROR_NILL_AUTH_TOKEN` | +| `sessionId` missing or empty | `ERROR_SESSION_ID_NILL` | +| A fetch is already in flight | `ERROR_REQUEST_IN_PROGRESS` | +| No network | `ERROR_INTERNET_UNAVAILABLE` | +| Missing or malformed response | `ERROR_JSON_EXCEPTION` | +| Server-side API error | The server's own code | + +## Transcripts in Call Logs + +Call logs can be filtered to transcribed calls, which also attaches each call's transcripts to the log: + + + +```swift +let callLogRequest = CallLogsRequest.CallLogsBuilder() + .set(limit: 30) + .set(hasTranscriptions: true) + .build() + +callLogRequest.fetchNext(onSuccess: { callLogs in + for callLog in callLogs { + for transcript in callLog.transcriptions { + print(transcript.tid, transcript.transcriptUrl) + } + } +}, onError: { error in + print("Error: \(error?.errorDescription ?? "")") +}) +``` + + +```objectivec +CallLogsBuilder *builder = [[CallLogsBuilder alloc] init]; +CallLogsRequest *callLogRequest = [[[builder + setWithLimit:30] + setWithHasTranscriptions:YES] + build]; + +[callLogRequest fetchNextOnSuccess:^(NSArray * callLogs) { + for (CallLog *callLog in callLogs) { + for (Transcript *transcript in callLog.transcriptions) { + NSLog(@"%@ %@", transcript.tid, transcript.transcriptUrl); + } + } +} onError:^(CometChatCallException * error) { + NSLog(@"Error: %@", error.errorDescription); +}]; +``` + + + +`transcriptions` is an empty array when the server omitted transcripts, so it never needs a nil check. Passing `false` leaves the list unfiltered, exactly as if the filter had never been set — and the server then omits the transcripts. + +## Complete Example + + + +```swift +// 1. Join a session with transcription enabled +let sessionSettings = CometChatCalls.sessionSettingsBuilder + .enableAutoStartTranscription(true) + .hideTranscriptionButton(false) + .hideClosedCaptionButton(false) + .setCaptionLanguage("en-US") + .build() + +// 2. Control transcription during the call +CallSession.shared.startTranscription() +CallSession.shared.stopTranscription() + +// 3. Retrieve transcripts after the call +let request = TranscriptsRequest.TranscriptsBuilder() + .set(sessionId: sessionId) + .set(limit: 10) + .build() + +request.fetchNext(onSuccess: { transcripts in + for transcript in transcripts { + print(transcript.transcriptUrl) + } +}, onError: { error in + print("Error: \(error?.errorDescription ?? "")") +}) +``` + + +```objectivec +// 1. Join a session with transcription enabled +SessionSettings *sessionSettings = [[[[[[CometChatCalls sessionSettingsBuilder] + enableAutoStartTranscription:YES] + hideTranscriptionButton:NO] + hideClosedCaptionButton:NO] + setCaptionLanguage:@"en-US"] + build]; + +// 2. Control transcription during the call +[[CallSession shared] startTranscription]; +[[CallSession shared] stopTranscription]; + +// 3. Retrieve transcripts after the call +TranscriptsBuilder *builder = [[TranscriptsBuilder alloc] init]; +TranscriptsRequest *request = [[[builder + setWithSessionId:sessionId] + setWithLimit:10] + build]; + +[request fetchNextOnSuccess:^(NSArray * transcripts) { + for (Transcript *transcript in transcripts) { + NSLog(@"%@", transcript.transcriptUrl); + } +} onError:^(CometChatCallException * error) { + NSLog(@"Error: %@", error.errorDescription); +}]; +``` + + + +## Related Documentation + +- [SessionSettingsBuilder](/calls/ios/session-settings) +- [Call Actions](/calls/ios/actions) +- [Call Logs](/calls/ios/call-logs) diff --git a/calls/javascript/actions.mdx b/calls/javascript/actions.mdx index de9868562..38ac1267e 100644 --- a/calls/javascript/actions.mdx +++ b/calls/javascript/actions.mdx @@ -91,6 +91,30 @@ Stops the current recording. The recording is saved and accessible via the dashb CometChatCalls.stopRecording(); ``` +## Transcription + +*Available since v5.0.5* + +### Start Transcription + +Begins server-side transcription of the call. This also enables the live closed captions overlay. + +```javascript +CometChatCalls.startTranscription(); +``` + + +Transcription requires the feature to be enabled for your CometChat app. + + +### Stop Transcription + +Stops the current transcription and clears any captions currently on screen. The transcript is saved and can be retrieved with [`TranscriptRequestBuilder`](/calls/javascript/transcription#retrieving-transcripts). + +```javascript +CometChatCalls.stopTranscription(); +``` + ## Participant Management ### Mute Participant diff --git a/calls/javascript/call-logs.mdx b/calls/javascript/call-logs.mdx index b88098dc3..48720ee7b 100644 --- a/calls/javascript/call-logs.mdx +++ b/calls/javascript/call-logs.mdx @@ -46,8 +46,32 @@ See the [Get Call API](/calls/api/get-call) documentation for full details. | `participants` | Array | List of participants who joined | | `recordingUrl` | String | URL to the call recording (if recorded) | +## Transcripts + +*Available since v5.0.5* + +Use `CallLogRequestBuilder` to fetch only calls that were transcribed. Opting in also makes the server attach each call's transcripts to the log: + +```javascript +const callLogRequest = new CometChatCalls.CallLogRequestBuilder() + .setLimit(30) + .setHasTranscriptions(true) + .build(); + +const callLogs = await callLogRequest.fetchNext(); + +callLogs.forEach((callLog) => { + callLog.getTranscriptions().forEach((transcription) => { + console.log(transcription.getTid(), transcription.getTranscriptURL()); + }); +}); +``` + +`getTranscriptions()` returns an empty array when the server omitted transcripts, so it never needs a null check. To page through a session's transcripts directly, use [`TranscriptRequestBuilder`](/calls/javascript/transcription#retrieving-transcripts). + ## Related Documentation +- [Transcription](/calls/javascript/transcription) - [List Calls API](/calls/api/list-calls) - [Get Call API](/calls/api/get-call) diff --git a/calls/javascript/session-settings.mdx b/calls/javascript/session-settings.mdx index 61cb0e078..daca757d2 100644 --- a/calls/javascript/session-settings.mdx +++ b/calls/javascript/session-settings.mdx @@ -97,6 +97,34 @@ autoStartRecording: true **Default:** `false` +### Auto Start Transcription + +*Available since v5.0.5* + +**Property:** `autoStartTranscription` + +Automatically starts transcribing the session as soon as it begins. See [Transcription](/calls/javascript/transcription) for details. + +```javascript +autoStartTranscription: true +``` + +**Default:** `false` + +### Caption Language + +*Available since v5.0.5* + +**Property:** `captionLanguage` + +Sets the language used for transcription and closed captions. See [Transcription](/calls/javascript/transcription) for the full list of supported codes. + +```javascript +captionLanguage: "en-US" +``` + +**Default:** `en-US` + ### Idle Timeout Period Before Prompt **Property:** `idleTimeoutPeriodBeforePrompt` @@ -183,6 +211,34 @@ hideRecordingButton: true **Default:** `true` +### Hide Transcription Button + +*Available since v5.0.5* + +**Property:** `hideTranscriptionButton` + +Hides the transcription start/stop item from the control panel's **More** menu. + +```javascript +hideTranscriptionButton: false +``` + +**Default:** `true` + +### Hide Closed Caption Button + +*Available since v5.0.5* + +**Property:** `hideClosedCaptionButton` + +Hides the closed-caption (CC) button from the control panel. Even when set to `false`, the button only appears while transcription is running. + +```javascript +hideClosedCaptionButton: false +``` + +**Default:** `true` + ### Hide Screen Sharing Button **Property:** `hideScreenSharingButton` @@ -241,6 +297,8 @@ const callSettings = { startAudioMuted: false, startVideoPaused: false, autoStartRecording: false, + autoStartTranscription: false, + captionLanguage: "en-US", // Timeout settings idleTimeoutPeriodBeforePrompt: 60000, @@ -252,6 +310,8 @@ const callSettings = { hideToggleAudioButton: false, hideToggleVideoButton: false, hideRecordingButton: true, + hideTranscriptionButton: true, + hideClosedCaptionButton: true, hideScreenSharingButton: false, hideChangeLayoutButton: false, hideVirtualBackgroundButton: false, @@ -267,6 +327,8 @@ const callSettings = { | `startAudioMuted` | Boolean | `false` | Start with microphone muted | | `startVideoPaused` | Boolean | `false` | Start with camera off | | `autoStartRecording` | Boolean | `false` | Auto-start recording | +| `autoStartTranscription` | Boolean | `false` | Auto-start transcription | +| `captionLanguage` | String | `en-US` | Transcription / caption language code | | `idleTimeoutPeriodBeforePrompt` | Number | `60000` | Idle timeout before prompt (ms) | | `idleTimeoutPeriodAfterPrompt` | Number | `180000` | Idle timeout after prompt (ms) | | `hideControlPanel` | Boolean | `false` | Hide control panel | @@ -274,6 +336,8 @@ const callSettings = { | `hideToggleAudioButton` | Boolean | `false` | Hide audio toggle | | `hideToggleVideoButton` | Boolean | `false` | Hide video toggle | | `hideRecordingButton` | Boolean | `true` | Hide recording button | +| `hideTranscriptionButton` | Boolean | `true` | Hide transcription button | +| `hideClosedCaptionButton` | Boolean | `true` | Hide closed caption button | | `hideScreenSharingButton` | Boolean | `false` | Hide screen share button | | `hideChangeLayoutButton` | Boolean | `false` | Hide layout change button | | `hideVirtualBackgroundButton` | Boolean | `false` | Hide virtual background button | diff --git a/calls/javascript/setup.mdx b/calls/javascript/setup.mdx index 42b91f1e6..144cb5617 100644 --- a/calls/javascript/setup.mdx +++ b/calls/javascript/setup.mdx @@ -27,12 +27,12 @@ Install the CometChat Calls SDK using npm or yarn: ```bash -npm install @cometchat/calls-sdk-javascript@5.0.0 +npm install @cometchat/calls-sdk-javascript@5.0.5 ``` ```bash -yarn add @cometchat/calls-sdk-javascript@5.0.0 +yarn add @cometchat/calls-sdk-javascript@5.0.5 ``` diff --git a/calls/javascript/transcription.mdx b/calls/javascript/transcription.mdx new file mode 100644 index 000000000..24ee75701 --- /dev/null +++ b/calls/javascript/transcription.mdx @@ -0,0 +1,306 @@ +--- +title: "Transcription & Closed Captions" +sidebarTitle: "Transcription" +sdk_version: "5.x" +description: "Use CometChat Calls SDK v5 transcription on JavaScript to transcribe calls, show live closed captions, and retrieve transcripts after the call." +--- + + +**Available since v5.0.5** — transcription and closed captions require CometChat Calls SDK v5.0.5 or later for JavaScript. See [Setup](/calls/javascript/setup) to install or upgrade. + + +Transcribe call sessions in real time and display live closed captions on screen. Transcripts are stored server-side and can be retrieved after the call using `TranscriptRequestBuilder`. + + +Transcription must be enabled for your CometChat app. Contact support if you need to enable this feature. + + +## How It Works + +Transcription and closed captions are two related but separate things: + +| Concept | What it does | +|---------|--------------| +| **Transcription** | Server-side speech-to-text for the session. Starting it brings a transcriber into the call, which produces the transcript that is stored for later retrieval. | +| **Closed captions** | The on-screen overlay that renders the live transcript as it arrives. Captions are produced from the running transcription, so they only appear while transcription is active. | + +Starting transcription is a prerequisite for captions — toggling captions on without an active transcription shows nothing. + +## Starting Transcription + +### Auto-Start Transcription + +Configure transcription to start automatically when the session begins: + +```javascript +const callSettings = { + autoStartTranscription: true, + // ... other settings +}; + +await CometChatCalls.joinSession(callToken, callSettings, container); +``` + +**Default:** `false` + +### Manual Transcription Control + +#### Start Transcription + +Begin transcribing during an active call: + +```javascript +CometChatCalls.startTranscription(); +``` + +#### Stop Transcription + +Stop the current transcription. Any captions currently on screen are cleared: + +```javascript +CometChatCalls.stopTranscription(); +``` + +## Built-in UI Controls + +### Transcription Button + +By default, the transcription button in the control panel's **More** menu is hidden. To show it: + +```javascript +const callSettings = { + hideTranscriptionButton: false, + // ... other settings +}; +``` + +**Default:** `true` + +The menu item toggles between **Start Transcription** and **Stop Transcription** based on the current state. + +### Closed Caption Button + +By default, the closed-caption (CC) button in the control panel is hidden. To show it: + +```javascript +const callSettings = { + hideClosedCaptionButton: false, + // ... other settings +}; +``` + +**Default:** `true` + + +Even with `hideClosedCaptionButton: false`, the CC button only appears once transcription is running for the session, because captions are generated from the live transcript. + + +### Closed Caption Settings + +When the CC button is visible, the settings dialog gains a **Closed Caption** tab where the user can pick the caption language and enable or disable the on-screen captions. The gear icon on the captions overlay opens the dialog directly on that tab. + +## Caption Language + +**Property:** `captionLanguage` + +Sets the language used for transcription and captions. + +```javascript +const callSettings = { + captionLanguage: "en-US", + // ... other settings +}; +``` + +**Default:** `en-US` + + +| Code | Language | +|------|----------| +| `en-US` | English (United States) | +| `de-DE` | German (Germany) | +| `en-GB` | English (United Kingdom) | +| `es-ES` | Spanish (Spain) | +| `fr-FR` | French (France) | +| `hi-IN` | Hindi (India) | +| `hu-HU` | Hungarian (Hungary) | +| `it-IT` | Italian (Italy) | +| `ja-JP` | Japanese (Japan) | +| `ko-KR` | Korean (South Korea) | +| `lt-LT` | Lithuanian (Lithuania) | +| `ms-MY` | Malay (Malaysia) | +| `nl-NL` | Dutch (Netherlands) | +| `pt-PT` | Portuguese (Portugal) | +| `ru-RU` | Russian (Russia) | +| `sv-SE` | Swedish (Sweden) | +| `tr-TR` | Turkish (Turkey) | +| `zh` | Chinese Mandarin (Simplified, China) | +| `zh-TW` | Chinese Mandarin (Traditional, Taiwan) | + + +## Retrieving Transcripts + +After a call, use `TranscriptRequestBuilder` to list the transcript artifacts for a session. Each record is a **pointer to a downloadable transcript file**, not the transcript text itself. + + +The SDK must be initialized with `CometChatCalls.init()` and a user must be logged in. The auth token is read from the logged-in user at fetch time, so it automatically tracks re-logins. + + +### Building a Request + +```javascript +const request = new CometChatCalls.TranscriptRequestBuilder() + .setSessionId("v1.us.2547167fe69871fd.alice") // required + .setLimit(10) // optional + .build(); + +const transcripts = await request.fetchNext(); +``` + +| Method | Required | Description | +|--------|----------|-------------| +| `setSessionId(sessionId: string)` | Yes | The session ID whose transcripts to fetch. An empty value throws `SESSION_ID_REQUIRED` at `build()`. | +| `setLimit(limit: number)` | No | Page size. Defaults to `30` and is clamped to the range `1`–`1000`. | +| `build()` | Yes | Returns a `TranscriptRequest`. Throws synchronously for pre-flight errors. | + +### Paginating + +A `TranscriptRequest` is a stateful cursor. Create one per session ID and drive it with `fetchNext()` and `fetchPrevious()`: + +```javascript +const request = new CometChatCalls.TranscriptRequestBuilder() + .setSessionId(sessionId) + .setLimit(10) + .build(); + +let page = await request.fetchNext(); // page 1 + +while (page.length > 0) { + page.forEach((transcript) => console.log(transcript.transcriptUrl)); + page = await request.fetchNext(); // page 2, 3, ... then [] at the end +} +``` + +- `fetchNext()` resolves the next page, or `[]` when there are no more pages. A session with no transcripts resolves `[]`. +- `fetchPrevious()` resolves the previous page, or `[]` when already on the first page. It never requests a page below `1`. +- Only one fetch may be in flight at a time. Calling `fetchNext()` or `fetchPrevious()` while another request is pending rejects with `REQUEST_IN_PROGRESS`; the original call is unaffected and still completes. + +### Transcript Properties + +| Property | Type | Description | +|----------|------|-------------| +| `tid` | String | Transcript ID | +| `mid` | String | Meeting ID | +| `roomName` | String | Room name of the meeting | +| `startTime` | Number | Meeting start time, in epoch **seconds** | +| `endTime` | Number | Meeting end time, in epoch **seconds** | +| `url` | String | Meeting URL | +| `transcriptDate` | String | Transcript date | +| `transcriptUrl` | String | URL of the downloadable transcript JSON | +| `metaData` | Object | Arbitrary metadata | + + +Every property is optional. The server omits keys whose value is empty, so sparse records are normal and should not be treated as an error. + + +### Reading the Transcript Content + +`transcriptUrl` points at the transcript file. Fetch it yourself to read the actual utterances: + +```javascript +const [transcript] = await request.fetchNext(); + +if (transcript?.transcriptUrl) { + const response = await fetch(transcript.transcriptUrl); + const content = await response.json(); + console.log(content); +} +``` + +### Error Handling + +`build()` throws synchronously; `fetchNext()` and `fetchPrevious()` reject. Both surface a `CometChatCallsException` carrying a `code`: + +| Condition | Code | Raised by | +|-----------|------|-----------| +| `init()` was not called | `NOT_INITIALIZED` | `build()` | +| `sessionId` missing or empty | `SESSION_ID_REQUIRED` | `build()` | +| No logged-in user | `NOT_LOGGED_IN` | `fetchNext()` / `fetchPrevious()` | +| A fetch is already in flight | `REQUEST_IN_PROGRESS` | `fetchNext()` / `fetchPrevious()` | +| Network failure | `NETWORK_ERROR` | `fetchNext()` / `fetchPrevious()` | +| Missing or malformed response | `BAD_RESPONSE` | `fetchNext()` / `fetchPrevious()` | +| Server-side API error | The server's own code (e.g. `AUTH_ERR_EMPTY_APPID`) | `fetchNext()` / `fetchPrevious()` | + +```javascript +try { + const transcripts = await request.fetchNext(); +} catch (error) { + console.error(error.code, error.message); +} +``` + +## Transcripts in Call Logs + +Call logs can be filtered to transcribed calls, which also attaches each call's transcripts to the log: + +```javascript +const callLogRequest = new CometChatCalls.CallLogRequestBuilder() + .setLimit(30) + .setHasTranscriptions(true) + .build(); + +const callLogs = await callLogRequest.fetchNext(); + +callLogs.forEach((callLog) => { + callLog.getTranscriptions().forEach((transcription) => { + console.log(transcription.getTid(), transcription.getTranscriptURL()); + }); +}); +``` + +`getTranscriptions()` returns an empty array when the server omitted transcripts, so it never needs a null check. + + +| Method | Returns | Description | +|--------|---------|-------------| +| `getTid()` | String | The transcript ID | +| `getMid()` | String | The meeting ID | +| `getRoomName()` | String | The room name of the meeting | +| `getStartTime()` | Number | Meeting start time, in epoch seconds | +| `getEndTime()` | Number | Meeting end time, in epoch seconds | +| `getTranscriptDate()` | String | The transcript date | +| `getTranscriptURL()` | String | URL of the downloadable transcript JSON | + + +## Complete Example + +```javascript +// 1. Join a session with transcription enabled +const callSettings = { + sessionType: "VIDEO", + autoStartTranscription: true, + hideTranscriptionButton: false, + hideClosedCaptionButton: false, + captionLanguage: "en-US", +}; + +await CometChatCalls.joinSession(callToken, callSettings, container); + +// 2. Control transcription during the call +CometChatCalls.startTranscription(); +CometChatCalls.stopTranscription(); + +// 3. Retrieve transcripts after the call +const request = new CometChatCalls.TranscriptRequestBuilder() + .setSessionId(sessionId) + .setLimit(10) + .build(); + +const transcripts = await request.fetchNext(); +``` + +## Related Documentation + +- [Session Settings](/calls/javascript/session-settings) +- [Actions](/calls/javascript/actions) +- [Call Logs](/calls/javascript/call-logs) diff --git a/calls/react-native/actions.mdx b/calls/react-native/actions.mdx index cc26f636d..8852ae5fd 100644 --- a/calls/react-native/actions.mdx +++ b/calls/react-native/actions.mdx @@ -69,6 +69,30 @@ The `leaveSession()` method ends the call for the local user. Other participants `leaveSession()` takes no session ID. `CometChatCalls` is a singleton that tracks one active session at a time, so this leaves whichever session is currently in progress. The same applies to every action on this page. See [Single Active Session](/calls/react-native/overview#single-active-session). +## Transcription + +*Available since v5.0.5* + +### Start Transcription + +Begin server-side transcription of the call. This also enables the live closed captions overlay: + +```tsx +CometChatCalls.startTranscription(); +``` + + +Transcription requires the feature to be enabled for your CometChat app. + + +### Stop Transcription + +Stop the current transcription and clear any captions currently on screen. The transcript is saved and can be retrieved with [`TranscriptRequestBuilder`](/calls/react-native/transcription#retrieving-transcripts): + +```tsx +CometChatCalls.stopTranscription(); +``` + ## Raise Hand ### Raise Hand @@ -319,3 +343,4 @@ These methods are deprecated but still available for backward compatibility: - [Events](/calls/react-native/events) - Listen for call events - [Participant Management](/calls/react-native/participant-management) - Manage participants - [Call Layouts](/calls/react-native/call-layouts) - Layout options +- [Transcription](/calls/react-native/transcription) - Transcription and closed captions diff --git a/calls/react-native/call-logs.mdx b/calls/react-native/call-logs.mdx index 406565f2f..81ab5b337 100644 --- a/calls/react-native/call-logs.mdx +++ b/calls/react-native/call-logs.mdx @@ -81,6 +81,7 @@ Each call log contains: | `duration` | number | Call duration in seconds | | `participants` | array | List of participants | | `recordings` | array | List of recordings (if any) | +| `transcriptions` | array | List of transcripts. Only attached when the request opts in with `setHasTranscriptions(true)` | ## Filter by User @@ -141,6 +142,31 @@ const page1 = await manager.fetchNextPage(); const page2 = await manager.fetchNextPage(); ``` +## Transcripts + +*Available since v5.0.5* + +Use the Calls SDK's `CallLogRequestBuilder` to fetch only calls that were transcribed. Opting in also makes the server attach each call's transcripts to the log: + +```tsx +import { CometChatCalls } from '@cometchat/calls-sdk-react-native'; + +const callLogRequest = new CometChatCalls.CallLogRequestBuilder() + .setLimit(30) + .setHasTranscriptions(true) + .build(); + +const callLogs = await callLogRequest.fetchNext(); + +callLogs.forEach((callLog) => { + callLog.getTranscriptions().forEach((transcription) => { + console.log(transcription.getTid(), transcription.getTranscriptURL()); + }); +}); +``` + +`getTranscriptions()` returns an empty array when the server omitted transcripts, so it never needs a null check. To page through a session's transcripts directly, use [`TranscriptRequestBuilder`](/calls/react-native/transcription#retrieving-transcripts). + ## Complete Example ```tsx @@ -301,4 +327,5 @@ export default CallLogsScreen; ## Related Documentation - [Recording](/calls/react-native/recording) - Access call recordings +- [Transcription](/calls/react-native/transcription) - Retrieve call transcripts - [Ringing](/calls/react-native/ringing) - Implement call notifications diff --git a/calls/react-native/overview.mdx b/calls/react-native/overview.mdx index c9d906817..12370565e 100644 --- a/calls/react-native/overview.mdx +++ b/calls/react-native/overview.mdx @@ -57,6 +57,10 @@ sequenceDiagram Record call sessions for later playback + + Transcribe calls, show live closed captions, and retrieve transcripts + + Retrieve call history and details diff --git a/calls/react-native/session-settings.mdx b/calls/react-native/session-settings.mdx index 2f28d0825..3990b5250 100644 --- a/calls/react-native/session-settings.mdx +++ b/calls/react-native/session-settings.mdx @@ -27,6 +27,10 @@ const sessionSettings = { audioMode: 'SPEAKER', hideRecordingButton: true, autoStartRecording: false, + hideTranscriptionButton: true, + hideClosedCaptionButton: true, + autoStartTranscription: false, + captionLanguage: 'en-US', idleTimeoutPeriodBeforePrompt: 180000, enableSpotlightSwap: true, enableSpotlightDrag: true, @@ -84,6 +88,8 @@ const sessionSettings = { | `hideSwitchCameraButton` | boolean | `false` | Hide the switch camera button | | `hideAudioModeButton` | boolean | `false` | Hide the audio output (route) button | | `hideRecordingButton` | boolean | `true` | Hide the recording button | +| `hideTranscriptionButton` | boolean | `true` | Hide the transcription item in the **More** menu | +| `hideClosedCaptionButton` | boolean | `true` | Hide the closed-captions item in the **More** menu | | `hideChangeLayoutButton` | boolean | `false` | Hide the layout-switcher button | | `hideParticipantListButton` | boolean | `false` | Hide the button that opens the participant list | | `hideChatButton` | boolean | `true` | Hide the in-call chat button | @@ -141,6 +147,28 @@ const sessionSettings = { }; ``` +### Transcription & Closed Captions + +*Available since v5.0.5* + +| Property | Type | Default | Description | +|----------|------|---------|-------------| +| `hideTranscriptionButton` | boolean | `true` | Hide the **Start / Stop Transcription** item in the **More** menu | +| `hideClosedCaptionButton` | boolean | `true` | Hide the **Show / Hide Captions** item in the **More** menu. Even when `false`, the item only appears while transcription is running | +| `autoStartTranscription` | boolean | `false` | Auto-start transcription when the call begins | +| `captionLanguage` | string | `'en-US'` | Language code used for transcription and captions | + +```tsx +const sessionSettings = { + hideTranscriptionButton: false, + hideClosedCaptionButton: false, + autoStartTranscription: true, + captionLanguage: 'en-US', +}; +``` + +See [Transcription](/calls/react-native/transcription) for the supported language codes and transcript retrieval. + ### Idle Timeout | Property | Type | Default | Description | @@ -297,6 +325,10 @@ function createSessionSettings(isAudioOnly: boolean = false) { startVideoPaused: false, audioMode: 'SPEAKER', hideRecordingButton: false, + hideTranscriptionButton: false, + hideClosedCaptionButton: false, + autoStartTranscription: false, + captionLanguage: 'en-US', idleTimeoutPeriodBeforePrompt: 180000, }; } diff --git a/calls/react-native/setup.mdx b/calls/react-native/setup.mdx index 9ddb1db0d..723680be9 100644 --- a/calls/react-native/setup.mdx +++ b/calls/react-native/setup.mdx @@ -25,13 +25,13 @@ This guide walks you through installing the CometChat Calls SDK and configuring ### Using npm ```bash -npm install @cometchat/calls-sdk-react-native +npm install @cometchat/calls-sdk-react-native@5.0.5 ``` ### Using Yarn ```bash -yarn add @cometchat/calls-sdk-react-native +yarn add @cometchat/calls-sdk-react-native@5.0.5 ``` ## Install Required Dependencies diff --git a/calls/react-native/transcription.mdx b/calls/react-native/transcription.mdx new file mode 100644 index 000000000..7deb7ee51 --- /dev/null +++ b/calls/react-native/transcription.mdx @@ -0,0 +1,372 @@ +--- +title: "Transcription & Closed Captions" +sidebarTitle: "Transcription" +sdk_version: "5.x" +description: "Use CometChat Calls SDK v5 transcription on React Native to transcribe calls, show live closed captions, and retrieve transcripts after the call." +--- + + +**Available since v5.0.5** — transcription and closed captions require CometChat Calls SDK v5.0.5 or later for React Native. See [Setup](/calls/react-native/setup) to install or upgrade. + + +Transcribe call sessions in real time and display live closed captions on screen. Transcripts are stored server-side and can be retrieved after the call using `TranscriptRequestBuilder`. + + +Transcription must be enabled for your CometChat app. Contact support if you need to enable this feature. + + +## How It Works + +Transcription and closed captions are two related but separate things: + +| Concept | What it does | +|---------|--------------| +| **Transcription** | Server-side speech-to-text for the session. Starting it brings a transcriber into the call, which produces the transcript that is stored for later retrieval. | +| **Closed captions** | The on-screen overlay that renders the live transcript as it arrives. Captions are produced from the running transcription, so they only appear while transcription is active. | + +Starting transcription is a prerequisite for captions — toggling captions on without an active transcription shows nothing. + +## Starting Transcription + +### Auto-Start Transcription + +Configure transcription to start automatically when the session begins: + +```tsx +import { CometChatCalls } from '@cometchat/calls-sdk-react-native'; + +const sessionSettings = { + autoStartTranscription: true, + // ... other settings +}; + + +``` + +**Default:** `false` + +### Manual Transcription Control + +#### Start Transcription + +Begin transcribing during an active call: + +```tsx +CometChatCalls.startTranscription(); +``` + +#### Stop Transcription + +Stop the current transcription. Any captions currently on screen are cleared: + +```tsx +CometChatCalls.stopTranscription(); +``` + +## Built-in UI Controls + +Both transcription controls live in the control panel's **More** menu and are hidden by default. + +### Transcription Menu Item + +Show the **Start Transcription** / **Stop Transcription** item: + +```tsx +const sessionSettings = { + hideTranscriptionButton: false, + // ... other settings +}; +``` + +**Default:** `true` + +The item toggles between **Start Transcription** and **Stop Transcription** based on the current state. + + +Even with `hideTranscriptionButton: false`, the item is hidden while another participant is running transcription for the session. Only the participant who started transcription can stop it from the menu. + + +### Closed Captions Menu Item + +Show the **Show Captions** / **Hide Captions** item, which toggles the on-screen captions overlay: + +```tsx +const sessionSettings = { + hideClosedCaptionButton: false, + // ... other settings +}; +``` + +**Default:** `true` + + +Even with `hideClosedCaptionButton: false`, the item only appears once transcription is running for the session, because captions are generated from the live transcript. + + +### Captions Overlay + +When captions are shown, the SDK renders a captions panel between the call stage and the control panel. Each caption shows the speaker's avatar, name, and text. The panel keeps the most recent captions, auto-scrolls to the newest one, and shows a jump-to-latest button when the user has scrolled up to read earlier captions. Until someone speaks, it displays *Waiting for people to speak…*. + +## Caption Language + +**Property:** `captionLanguage` + +Sets the language used for transcription and captions. + +```tsx +const sessionSettings = { + captionLanguage: 'en-US', + // ... other settings +}; +``` + +**Default:** `en-US` + + +On React Native there is no in-call language picker. Set `captionLanguage` in the session settings before joining. + + + +| Code | Language | +|------|----------| +| `en-US` | English (United States) | +| `de-DE` | German (Germany) | +| `en-GB` | English (United Kingdom) | +| `es-ES` | Spanish (Spain) | +| `fr-FR` | French (France) | +| `hi-IN` | Hindi (India) | +| `hu-HU` | Hungarian (Hungary) | +| `it-IT` | Italian (Italy) | +| `ja-JP` | Japanese (Japan) | +| `ko-KR` | Korean (South Korea) | +| `lt-LT` | Lithuanian (Lithuania) | +| `ms-MY` | Malay (Malaysia) | +| `nl-NL` | Dutch (Netherlands) | +| `pt-PT` | Portuguese (Portugal) | +| `ru-RU` | Russian (Russia) | +| `sv-SE` | Swedish (Sweden) | +| `tr-TR` | Turkish (Turkey) | +| `zh` | Chinese Mandarin (Simplified, China) | +| `zh-TW` | Chinese Mandarin (Traditional, Taiwan) | + + +## Retrieving Transcripts + +After a call, use `TranscriptRequestBuilder` to list the transcript artifacts for a session. Each record is a **pointer to a downloadable transcript file**, not the transcript text itself. + + +The SDK must be initialized with `CometChatCalls.init()` and a user must be logged in. The auth token is read from the logged-in user at fetch time, so it automatically tracks re-logins. + + +### Building a Request + +```tsx +import { CometChatCalls } from '@cometchat/calls-sdk-react-native'; + +const request = new CometChatCalls.TranscriptRequestBuilder() + .setSessionId('v1.us.2547167fe69871fd.alice') // required + .setLimit(10) // optional + .build(); + +const transcripts = await request.fetchNext(); +``` + +| Method | Required | Description | +|--------|----------|-------------| +| `setSessionId(sessionId: string)` | Yes | The session ID whose transcripts to fetch. An empty value throws `SESSION_ID_REQUIRED` at `build()`. | +| `setLimit(limit: number)` | No | Page size. Defaults to `30` and is clamped to the range `1`–`1000`. | +| `build()` | Yes | Returns a `TranscriptRequest`. Throws synchronously for pre-flight errors. | + +### Paginating + +A `TranscriptRequest` is a stateful cursor. Create one per session ID and drive it with `fetchNext()` and `fetchPrevious()`: + +```tsx +const request = new CometChatCalls.TranscriptRequestBuilder() + .setSessionId(sessionId) + .setLimit(10) + .build(); + +let page = await request.fetchNext(); // page 1 + +while (page.length > 0) { + page.forEach((transcript) => console.log(transcript.transcriptUrl)); + page = await request.fetchNext(); // page 2, 3, ... then [] at the end +} +``` + +- `fetchNext()` resolves the next page, or `[]` when there are no more pages. A session with no transcripts resolves `[]`. +- `fetchPrevious()` resolves the previous page, or `[]` when already on the first page. It never requests a page below `1`. +- Only one fetch may be in flight at a time. Calling `fetchNext()` or `fetchPrevious()` while another request is pending rejects with `REQUEST_IN_PROGRESS`; the original call is unaffected and still completes. + +### Transcript Properties + +The `Transcript` type is exported from the package for TypeScript consumers: + +```tsx +import type { Transcript } from '@cometchat/calls-sdk-react-native'; +``` + +| Property | Type | Description | +|----------|------|-------------| +| `tid` | string | Transcript ID | +| `mid` | string | Meeting ID | +| `roomName` | string | Room name of the meeting | +| `startTime` | number | Meeting start time, in epoch **seconds** | +| `endTime` | number | Meeting end time, in epoch **seconds** | +| `url` | string | Meeting URL | +| `transcriptDate` | string | Transcript date | +| `transcriptUrl` | string | URL of the downloadable transcript JSON | +| `metaData` | object | Arbitrary metadata | + + +Every property is optional. The server omits keys whose value is empty, so sparse records are normal and should not be treated as an error. + + +### Reading the Transcript Content + +`transcriptUrl` points at the transcript file. Fetch it yourself to read the actual utterances: + +```tsx +const [transcript] = await request.fetchNext(); + +if (transcript?.transcriptUrl) { + const response = await fetch(transcript.transcriptUrl); + const content = await response.json(); + console.log(content); +} +``` + +### Error Handling + +`build()` throws synchronously; `fetchNext()` and `fetchPrevious()` reject. Both surface a `CometChatCallsException` carrying a `code`: + +| Condition | Code | Raised by | +|-----------|------|-----------| +| `init()` was not called | `NOT_INITIALIZED` | `build()` | +| `sessionId` missing or empty | `SESSION_ID_REQUIRED` | `build()` | +| No logged-in user | `NOT_LOGGED_IN` | `fetchNext()` / `fetchPrevious()` | +| A fetch is already in flight | `REQUEST_IN_PROGRESS` | `fetchNext()` / `fetchPrevious()` | +| Network failure | `NETWORK_ERROR` | `fetchNext()` / `fetchPrevious()` | +| Missing or malformed response | `BAD_RESPONSE` | `fetchNext()` / `fetchPrevious()` | +| Server-side API error | The server's own code (e.g. `AUTH_ERR_EMPTY_APPID`) | `fetchNext()` / `fetchPrevious()` | + +```tsx +try { + const transcripts = await request.fetchNext(); +} catch (error) { + console.error(error.code, error.message); +} +``` + +## Transcripts in Call Logs + +Call logs can be filtered to transcribed calls, which also attaches each call's transcripts to the log: + +```tsx +const callLogRequest = new CometChatCalls.CallLogRequestBuilder() + .setLimit(30) + .setHasTranscriptions(true) + .build(); + +const callLogs = await callLogRequest.fetchNext(); + +callLogs.forEach((callLog) => { + callLog.getTranscriptions().forEach((transcription) => { + console.log(transcription.getTid(), transcription.getTranscriptURL()); + }); +}); +``` + +`getTranscriptions()` returns an empty array when the server omitted transcripts, so it never needs a null check. + + +| Method | Returns | Description | +|--------|---------|-------------| +| `getTid()` | string | The transcript ID | +| `getMid()` | string | The meeting ID | +| `getRoomName()` | string | The room name of the meeting | +| `getStartTime()` | number | Meeting start time, in epoch seconds | +| `getEndTime()` | number | Meeting end time, in epoch seconds | +| `getTranscriptDate()` | string | The transcript date | +| `getTranscriptURL()` | string | URL of the downloadable transcript JSON | + + +## Complete Example + +```tsx +import React, { useEffect, useState } from 'react'; +import { View, TouchableOpacity, Text, StyleSheet } from 'react-native'; +import { CometChatCalls } from '@cometchat/calls-sdk-react-native'; +import type { Transcript } from '@cometchat/calls-sdk-react-native'; + +// 1. Join a session with transcription enabled +const sessionSettings = { + sessionType: 'VIDEO', + autoStartTranscription: true, + hideTranscriptionButton: false, + hideClosedCaptionButton: false, + captionLanguage: 'en-US', +}; + +function CallScreen({ callToken }: { callToken: string }) { + const [isTranscribing, setIsTranscribing] = useState(true); + + // 2. Control transcription during the call + const toggleTranscription = () => { + if (isTranscribing) { + CometChatCalls.stopTranscription(); + } else { + CometChatCalls.startTranscription(); + } + setIsTranscribing(!isTranscribing); + }; + + return ( + + + + + {isTranscribing ? 'Stop Transcription' : 'Start Transcription'} + + + + ); +} + +// 3. Retrieve transcripts after the call +async function fetchTranscripts(sessionId: string): Promise { + const request = new CometChatCalls.TranscriptRequestBuilder() + .setSessionId(sessionId) + .setLimit(10) + .build(); + + return request.fetchNext(); +} + +const styles = StyleSheet.create({ + container: { flex: 1 }, + button: { + backgroundColor: '#333', + paddingHorizontal: 16, + paddingVertical: 12, + borderRadius: 8, + alignSelf: 'center', + }, + buttonText: { color: '#fff', fontSize: 14, fontWeight: '600' }, +}); + +export default CallScreen; +``` + +## Related Documentation + +- [Session Settings](/calls/react-native/session-settings) - Configure transcription options +- [Actions](/calls/react-native/actions) - Start and stop transcription programmatically +- [Call Logs](/calls/react-native/call-logs) - Transcripts attached to call history diff --git a/docs.json b/docs.json index 86152f233..67354129b 100644 --- a/docs.json +++ b/docs.json @@ -5396,6 +5396,7 @@ "calls/javascript/ringing", "calls/javascript/call-layouts", "calls/javascript/recording", + "calls/javascript/transcription", "calls/javascript/call-logs", "calls/javascript/participant-management", "calls/javascript/screen-sharing", @@ -5511,6 +5512,7 @@ "calls/react-native/call-layouts", "calls/react-native/call-logs", "calls/react-native/recording", + "calls/react-native/transcription", "calls/react-native/participant-management", "calls/react-native/screen-sharing", "calls/react-native/audio-modes", @@ -5627,6 +5629,7 @@ "calls/ios/call-layouts", "calls/ios/audio-modes", "calls/ios/recording", + "calls/ios/transcription", "calls/ios/call-logs", "calls/ios/participant-management", "calls/ios/screen-sharing", @@ -5742,6 +5745,7 @@ "calls/android/call-layouts", "calls/android/audio-modes", "calls/android/recording", + "calls/android/transcription", "calls/android/call-logs", "calls/android/participant-management", "calls/android/screen-sharing", @@ -5856,6 +5860,7 @@ "calls/flutter/call-layouts", "calls/flutter/audio-modes", "calls/flutter/recording", + "calls/flutter/transcription", "calls/flutter/call-logs", "calls/flutter/participant-management", "calls/flutter/screen-sharing",