fix(service): stop blocking getString on Dispatchers.Default-reachable notification paths - #6668
Conversation
…e notification paths Under CMP 1.12.x, compose-resources AsyncCache loads on its own dispatcher-less scope (= Dispatchers.Default) instead of inline on the caller. The blocking getString shim (runBlocking) then turns every Default-pool caller into a worker parked waiting for a load that itself needs a Default worker. Measured on a CMP 1.12.0-rc01 tree (Robolectric, 10 cores): - 2/4/8 concurrent blocking callers on Dispatchers.Default complete; 16 wedge the pool PERMANENTLY. After the wedge every getString in the process hangs forever, main thread included (jstack: DefaultDispatcher workers parked on kotlinx.coroutines.BlockingCoroutine). The failure mode is a hang, not slowness. - First-ever cold getString = ~198ms on the calling thread; later cold keys 0.8-14.6ms; warm ~0.7ms. ServiceScope is CoroutineScope(dispatchers.default + SupervisorJob()) and runs the whole incoming-packet/notification pipeline, so a message burst is exactly the wedge's trigger shape. main is currently on 1.11.1 (revert #6664), where loads resolve inline and the shim is benign — this change is a hard prerequisite for re-landing the CMP 1.12 bump, and is equally correct on 1.11.1. Migrated every Default-reachable site to getStringSuspend: - MeshNotificationManagerImpl: showGroupSummary, createConversationNotification, reply/mark-as-read actions, refreshConversationAfterReply; cancelMessageNotification is now suspend on the MeshNotificationManager interface (all production callers were already in coroutines). - AndroidNotificationManager.ensureChannelsInitialized: suspend behind a Mutex — the mutex is load-bearing; once the guard can suspend, the old boolean check races and a notification could post before its channel exists. - ConversationShortcutPublisher.ensureConversationShortcut: suspend. Deliberately NOT migrated: initChannels() stays eager and blocking on Main. MeshServiceOrchestrator.start() posts the foreground-service notification synchronously right after it, so lazy-gating channel creation into the suspend notify paths would post the FGS notification before its channel exists (watchdog-kill risk). Main is not a Default worker, so the one-time ~198ms block cannot wedge. Also left (latency only, never Default-reachable): fdroid MapView, Glance widget, desktopApp notifications, buildDmShortcut (by-lazy on IO). Reproduction caution: a saturation rig poisons the JVM's Default pool permanently — a poisoned pool surfaces as an unrelated hang elsewhere. One test method per JVM, never in shared CI; the rigs used here were deliberately not committed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
📝 WalkthroughWalkthroughThe notification manager contract now supports suspendable cancellation. Android notification channels, shortcuts, summaries, conversations, and actions use suspendable string-resource loading. Channel initialization is serialized with a mutex, and tests use suspend-aware verification. ChangesNotification asynchronous contract
Estimated code review effort: 3 (Moderate) | ~20 minutes Mergeability Score: 🟠 High · up to The change can still mark notification channels ready after setup fails and can repost an outdated group summary after the last conversation notification is removed. These concrete correctness issues may cause missing or incorrect notifications, so the PR is not ready to merge until they are addressed. Sequence Diagram(s)sequenceDiagram
participant AndroidNotificationManager
participant getStringSuspend
participant NotificationChannel
AndroidNotificationManager->>getStringSuspend: Resolve channel name
getStringSuspend-->>AndroidNotificationManager: Return localized name
AndroidNotificationManager->>NotificationChannel: Create channel
Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 6✅ Passed checks (6 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
core/service/src/androidMain/kotlin/org/meshtastic/core/service/AndroidNotificationManager.kt (1)
68-94: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winSet
channelsInitializedonly after channel creation succeeds.Line 70 marks initialization complete before
getStringSuspendandcreateNotificationChannelsfinish. If either operation throws, laterdispatchcalls skip initialization and can post notifications without their required channel. Keep the flag false until all channel work completes so a later dispatch retries.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@core/service/src/androidMain/kotlin/org/meshtastic/core/service/AndroidNotificationManager.kt` around lines 68 - 94, Update ensureChannelsInitialized so channelsInitialized is set to true only after createChannel and notificationManager.createNotificationChannels complete successfully, including legacy-channel cleanup. Leave it false when any channel setup operation throws, allowing a later dispatch to retry initialization.core/service/src/androidMain/kotlin/org/meshtastic/core/service/MeshNotificationManagerImpl.kt (1)
554-655: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winSerialize group-summary rebuilds.
showGroupSummaryreadsactiveNotificationsbefore suspendable string resolution. If cancellation clears the final child and summary while this call is suspended, this call can resume and repost a summary from the stale snapshot. Guard summary rebuilds with oneMutex, including the empty-summary cancellation path.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@core/service/src/androidMain/kotlin/org/meshtastic/core/service/MeshNotificationManagerImpl.kt` around lines 554 - 655, Serialize all showGroupSummary executions with a dedicated Mutex, acquiring it before reading notificationManager.activeNotifications and holding it through string resolution, summary posting, and the empty-summary cancellation return path. Update showGroupSummary and its callers as needed so concurrent cancellation-triggered rebuilds cannot repost a summary from stale state.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In
`@core/service/src/androidMain/kotlin/org/meshtastic/core/service/AndroidNotificationManager.kt`:
- Around line 68-94: Update ensureChannelsInitialized so channelsInitialized is
set to true only after createChannel and
notificationManager.createNotificationChannels complete successfully, including
legacy-channel cleanup. Leave it false when any channel setup operation throws,
allowing a later dispatch to retry initialization.
In
`@core/service/src/androidMain/kotlin/org/meshtastic/core/service/MeshNotificationManagerImpl.kt`:
- Around line 554-655: Serialize all showGroupSummary executions with a
dedicated Mutex, acquiring it before reading
notificationManager.activeNotifications and holding it through string
resolution, summary posting, and the empty-summary cancellation return path.
Update showGroupSummary and its callers as needed so concurrent
cancellation-triggered rebuilds cannot repost a summary from stale state.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 10f27b80-75b8-4613-9555-6935b9d42ec6
📒 Files selected for processing (8)
androidApp/src/test/kotlin/org/meshtastic/app/service/Fakes.ktcore/repository/src/commonMain/kotlin/org/meshtastic/core/repository/MeshNotificationManager.ktcore/service/src/androidHostTest/kotlin/org/meshtastic/core/service/ReplyReceiverTest.ktcore/service/src/androidMain/kotlin/org/meshtastic/core/service/AndroidNotificationManager.ktcore/service/src/androidMain/kotlin/org/meshtastic/core/service/ConversationShortcutPublisher.ktcore/service/src/androidMain/kotlin/org/meshtastic/core/service/MeshNotificationManagerImpl.ktcore/testing/src/commonMain/kotlin/org/meshtastic/core/testing/FakeMeshNotificationManager.ktdesktopApp/src/main/kotlin/org/meshtastic/desktop/notification/DesktopMeshNotificationManager.kt
Reverts d8361cc, restoring #6662's bump now that every regression it caused has a fix on main: - #6666 settles asynchronous resource loads in the 8 tests that failed deterministically (:core:service, :feature:connections) - #6669 keeps ViewModel coroutines inside the test that started them, covering ConnectionsViewModelTest and RadioConfigViewModelTest - #6668 removes the runBlocking getString shim from every Dispatchers.Default-reachable notification path, which is what made the bump a production hazard rather than only a test one Also drops the Renovate rule that blocked 1.12.0-rc01, since its lift condition is now met. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Why
Under CMP 1.12.x,
compose-resourcesloads strings on its own dispatcher-less scope (=Dispatchers.Default) instead of inline on the caller — the sameAsyncCachechange that broke tests and forced the #6664 revert. That turns our blockinggetStringshim (runBlocking { composeGetString(...) }) into a pool-starvation hazard: each Default-pool caller parks a worker waiting for a load that itself needs a Default worker.Measured on a CMP 1.12.0-rc01 tree (Robolectric, 10 cores, real Android resource reader):
getStringcalls onDispatchers.DefaultgetStringin the process hangs forever, main thread included (jstack:DefaultDispatcherworkers parked onBlockingCoroutine)The failure mode is a hang, not slowness, and it is unrecoverable.
ServiceScopeisCoroutineScope(dispatchers.default + SupervisorJob())and runs the whole incoming-packet/notification pipeline, so a message burst is exactly the trigger shape. Latency numbers for reference: first-ever coldgetString≈ 198 ms on the calling thread; later cold keys 0.8–14.6 ms; warm 0.7 ms.mainis currently on CMP 1.11.1 (revert #6664), where loads resolve inline and the shim is benign — this PR is a hard prerequisite for re-landing the CMP 1.12 bump (#6666 is the test-side groundwork; this is the production-side counterpart). The change is equally correct on 1.11.1, so it can land now.🛠️ Changes
MeshNotificationManagerImpl:showGroupSummary,createConversationNotification, reply/mark-as-read actions, andrefreshConversationAfterReplynow resolve strings viagetStringSuspend.cancelMessageNotificationbecomessuspendon theMeshNotificationManagerinterface (all production callers were already inside coroutines; desktop impl and test fakes updated).AndroidNotificationManager.ensureChannelsInitialized: nowsuspendbehind aMutex. The mutex is load-bearing — once the guard can suspend, the old boolean check races and a notification could post before its channel exists.ConversationShortcutPublisher.ensureConversationShortcut: nowsuspend.initChannels()stays eager and blocking on Main —MeshServiceOrchestrator.start()posts the foreground-service notification synchronously right after it, so lazy-gating channel creation would post the FGS notification before its channel exists (watchdog-kill risk). Main is not a Default worker, so the one-time ~198 ms block cannot wedge. Also left as follow-ups (latency-only, never Default-reachable): fdroidMapView, Glance widget, desktopApp notifications,buildDmShortcut(by lazyon IO).🧪 Testing Performed
spotlessCheck+detekt+assembleDebug+kmpSmokeCompile+test+allTests: the only test failures are byte-identical to the base commit without this change (stash-compared with--rerun-tasks) — i.e. the pre-existing CMP 1.12 test breakage that revert(deps): back out CMP 1.12.0-rc01 until its test regressions are fixed #6664 reverted away, not regressions. On post-revertmainthose failures don't exist.ReplyReceiverTest,ConversationShortcutPublisherTest,MeshNotificationManagerImplConversationTestconversation flows.🤖 Generated with Claude Code
Summary by CodeRabbit