Describe the bug
Native crash in production (Crashlytics) during audio-only calls: a heap corruption, meaning a use-after-free or double free, inside libjingle_peerconnection_so.so. Scudo aborts in free() on a WebRTC internal thread. At the same moment, the AudioRecord thread is inside WebRtcAudioRecord.nativeDataIsRecorded, and another thread is inside PeerConnection.nativeSetRemoteDescription.
It reproduces on both LiveKit 2.27.0 and 2.28.2, so upgrading did not fix it. Scale: 745 events from 725 users in 90 days.
Tombstone (aborting thread), from a 2.28.2 event
Scudo ERROR: corrupted chunk header at address 0x2000072bbbe5c90: chunk header is zero and might indicate memory corruption or a double free
#00 pc 0xb90d0 abort (libc.so)
#01 pc 0x6c8cc scudo::die()
#02 pc 0x6d31c scudo::reportRawError(char const*)
#03 pc 0x6d290 scudo::ScopedErrorReport::~ScopedErrorReport()
#04 pc 0x6d478 scudo::reportHeaderCorruption(void*, void*)
#05 pc 0x6edd8 scudo::Allocator<scudo::AndroidNormalConfig, &scudo_malloc_postinit>::deallocate(...)
#06 pc 0x918954 libjingle_peerconnection_so.so
#07 pc 0x915bb4 libjingle_peerconnection_so.so
#08 pc 0x91541c libjingle_peerconnection_so.so
#09 pc 0x914b54 libjingle_peerconnection_so.so
#10 pc 0x90b08c libjingle_peerconnection_so.so
#11 pc 0x8eacb4 libjingle_peerconnection_so.so
#12 pc 0x3b3228 libjingle_peerconnection_so.so (rtc::Thread run loop)
#13 pc 0x3b226c libjingle_peerconnection_so.so
#14 pc 0x3b3698 libjingle_peerconnection_so.so
#15 pc 0xc9df0 __pthread_start
#16 pc 0xbc4bc __start_thread
BuildId of libjingle_peerconnection_so.so: 8fb495ec1c3cc305 (webrtc-sdk android-prefixed 144.7559.14)
Crashlytics "crashed" thread (the audio record thread)
SIGSEGV 0xb4000072bbc15000
#00 pc 0x42a348 libjingle_peerconnection_so.so
#01 pc 0x42a344 libjingle_peerconnection_so.so
#02 pc 0x4285e4 libjingle_peerconnection_so.so
#03 pc 0x42854c libjingle_peerconnection_so.so
#04 pc 0x427e24 libjingle_peerconnection_so.so
#05 pc 0xa0c0f0 libjingle_peerconnection_so.so
#06 pc 0xa025c0 libjingle_peerconnection_so.so
#07 pc 0x5edeb8 libjingle_peerconnection_so.so
#08 pc 0x86bc48 libjingle_peerconnection_so.so (near Java_livekit_org_webrtc_audio_WebRtcAudioRecord_nativeDataIsRecorded)
#09 pc 0x72ee77a0 (Java frame: WebRtcAudioRecord$AudioRecordThread)
Other relevant thread at crash time
#03 pc 0x3b682c libjingle_peerconnection_so.so (waiting on rtc::Event)
#04 pc 0x3b3a60 libjingle_peerconnection_so.so
#05 pc 0x8dd7b8 libjingle_peerconnection_so.so
#06 pc 0x892344 libjingle_peerconnection_so.so
#07 pc 0x87cb4c libjingle_peerconnection_so.so (Java_livekit_org_webrtc_PeerConnection_nativeSetRemoteDescription)
#08 pc 0x72eea36c
Note: the .so is stripped. Crashlytics labels frames with the nearest exported JNI symbol, such as BuiltinAudioEncoderFactoryFactory_nativeCreateBuiltinAudioEncoderFactory or TurnCustomizer_nativeFreeTurnCustomizer. Those labels are misleading. The raw PCs and the BuildId above should symbolize against your unstripped build of 144.7559.14.
Suspected cause
A race between the capture path and renegotiation. The AudioRecord thread delivers 10 ms frames through nativeDataIsRecorded into the audio transport, then the send stream, APM/encoder, and the Krisp filter. At the same time, setRemoteDescription runs on the signaling thread. That call reconfigures or recreates the audio send stream or encoder on the worker thread and frees objects that the capture thread is still using. The worker thread then frees an already-freed or overwritten chunk, and Scudo aborts.
To Reproduce
We can't reproduce it on demand. It happens in production during active foreground calls, most likely while a renegotiation (a remote SDP update) arrives mid-call. Our app also handles transient audio focus loss during calls (it mutes and unmutes the mic track, and the AudioRecord restarts), which may make the window larger.
Expected behavior
No native heap corruption during renegotiation while the mic is capturing.
Screenshots
N/A. Full tombstone and thread dump are above.
Device Info:
- Device: mostly budget/mid-range OEM devices: Vivo 46%, Xiaomi 20%, Oppo 18%, Realme 5%, other 10%
- OS: Android 16 is 56% of events; also seen on Android 12
- LiveKit SDK version: reproduces on both of these:
- 2.27.0 (webrtc-sdk android-prefixed 144.7559.09): 739 events
- 2.28.2 (webrtc-sdk android-prefixed 144.7559.14): 6 events so far (newer release). The tombstone above is from this version, BuildId
8fb495ec1c3cc305
- krisp-noise-filter 0.0.15 on both. ABI arm64-v8a. Audio only (no video tracks).
- 0% of events happened while the app was in the background, so these calls were in the foreground.
Additional context
- Can you symbolize the PCs above against the 144.7559.14 build (BuildId
8fb495ec1c3cc305)?
- Is this a known issue in these WebRTC revisions, or in the Krisp audio processing integration?
- Is there a recommended workaround in the meantime, for example serializing mic mute/unmute with renegotiation, or disabling Krisp?
Crashlytics issue: 03c9dc9f88fc6ba514302d1571ebed6b
Describe the bug
Native crash in production (Crashlytics) during audio-only calls: a heap corruption, meaning a use-after-free or double free, inside
libjingle_peerconnection_so.so. Scudo aborts infree()on a WebRTC internal thread. At the same moment, theAudioRecordthread is insideWebRtcAudioRecord.nativeDataIsRecorded, and another thread is insidePeerConnection.nativeSetRemoteDescription.It reproduces on both LiveKit 2.27.0 and 2.28.2, so upgrading did not fix it. Scale: 745 events from 725 users in 90 days.
Tombstone (aborting thread), from a 2.28.2 event
BuildId of
libjingle_peerconnection_so.so:8fb495ec1c3cc305(webrtc-sdk android-prefixed 144.7559.14)Crashlytics "crashed" thread (the audio record thread)
Other relevant thread at crash time
Note: the
.sois stripped. Crashlytics labels frames with the nearest exported JNI symbol, such asBuiltinAudioEncoderFactoryFactory_nativeCreateBuiltinAudioEncoderFactoryorTurnCustomizer_nativeFreeTurnCustomizer. Those labels are misleading. The raw PCs and the BuildId above should symbolize against your unstripped build of 144.7559.14.Suspected cause
A race between the capture path and renegotiation. The
AudioRecordthread delivers 10 ms frames throughnativeDataIsRecordedinto the audio transport, then the send stream, APM/encoder, and the Krisp filter. At the same time,setRemoteDescriptionruns on the signaling thread. That call reconfigures or recreates the audio send stream or encoder on the worker thread and frees objects that the capture thread is still using. The worker thread then frees an already-freed or overwritten chunk, and Scudo aborts.To Reproduce
We can't reproduce it on demand. It happens in production during active foreground calls, most likely while a renegotiation (a remote SDP update) arrives mid-call. Our app also handles transient audio focus loss during calls (it mutes and unmutes the mic track, and the
AudioRecordrestarts), which may make the window larger.Expected behavior
No native heap corruption during renegotiation while the mic is capturing.
Screenshots
N/A. Full tombstone and thread dump are above.
Device Info:
8fb495ec1c3cc305Additional context
8fb495ec1c3cc305)?Crashlytics issue:
03c9dc9f88fc6ba514302d1571ebed6b