Manual Tools under Network Diagnostics sometime not work

[JohnnyWuLinksys]

Description
Notice a statement when tab already exist for a while(ex: over 10 minutes)
Use Manual Tools such as ping tool it shows timeout error message
(Traceroute and NS Lookup also with problem)
If refresh webpage by F5 or refresh icon on browser to reload LinksysNow UI and try again, it recover workable

20260717_095804-COM3_before_and_after_refresh_ping_record.log
log-2026-07-17_10_25_21_before_refresh_ping_timeout.txt
log-2026-07-17_10_26_09_after_refresh_ping_work.txt
log-2026-07-17_10_53_29_from_ping_work(last_success_10_47)_to_timeout(10_52).txt

Steps

1. Open LinksysNow web over 10 minutes
2. Menu > Network Diagnostics > Manual Tools
3. Try to ping 8.8.8.8(or else)

Current result
Show timeout error message

(not always happen, but easier happen if the tab exist longer)

Expected result
Show correct ping result

Information
Pinnacle 2.0 FW: 1.2.2.26062820
Linksys Now Version: 2.5.0.700525

Severity
S3

[AustinChangLinksys]

🔍 Automated investigation (preliminary triage)

Line: 2.x / USP (TR-369)

Root cause:

• Trigger (firmware/bridge side): after the tab has held the SSE notification stream (/api/v1/notifications) open for a while, the stream keeps emitting heartbeat frames but stops delivering OperationComplete notification frames. The IPPing() Operate is actually accepted and executed on the device (the router's own console shows the ping running), but its async result event never comes back over the aged stream — so the UI's 30s awaiter times out.
• UI defect: PrivacyGUI treats the SSE stream as healthy as long as heartbeats arrive, so there is no staleness detection / auto-reconnect when OperationComplete delivery silently dies. The only recovery is a manual F5 (which opens a fresh SSE stream). The user is left with a bare "timeout" and no self-healing.

Evidence (from the three attached UI logs + the COM3 device console):

1. The Operate HTTP request itself succeeds fast — the failure is not the command being rejected (before_refresh, L6763/L6771):

2026-07-17T10:24:06.208 [D][USPClient]:#78 OPERATE →
2026-07-17T10:24:06.379 [D][USPClient]:#78 OPERATE ← (170ms)
2026-07-17T10:24:06.379 [D][USP][SSE][Operate]: Executing Device.IP.Diagnostics.IPPing() in session (commandKey=ae641adf-...)

2. …but the OperationComplete result never arrives, so the awaiter times out (before_refresh, L6961; repeats again at 10:25:16):

2026-07-17T10:24:36.399 [W][USP][Diagnostics]: Ping timeout: TimeoutException: OperationComplete not received within 30s

3. During that window the SSE stream is alive but only carrying heartbeats — chunk counter advances #66 → #67 → #68 at ~30s spacing with no event: notification frame in between (before_refresh):

2026-07-17T10:24:13.480 [D][USP][SSE]: debug: Chunk #66 (49 chars): event: heartbeat
2026-07-17T10:24:43.467 [D][USP][SSE]: debug: Chunk #67 (49 chars): event: heartbeat
2026-07-17T10:25:13.492 [D][USP][SSE]: debug: Chunk #68 (49 chars): event: heartbeat

4. The device genuinely performed the ping the whole time (COM3 serial console) — so the loss is in the notification-delivery path, not in ping execution:

[07-17 10:24:00] ping 8.8.8.8
[07-17 10:24:00] 64 bytes from 8.8.8.8: seq=0 ttl=115 time=3.713 ms
[07-17 10:24:01] 64 bytes from 8.8.8.8: seq=1 ttl=115 time=3.195 ms

5. Immediately after F5, a fresh SSE stream (chunk counter resets to #1) delivers the OperationComplete right away and ping works (after_refresh, L1039/L1117/L1280):

2026-07-17T10:25:33.025 [D][USP][SSE]: debug: Chunk #1 (49 chars): event: heartbeat
2026-07-17T10:25:56.904 [D][USP][SSE]: debug: Chunk #4 (2284 chars): event: notification
... OperationComplete ... "Status":"Complete","SuccessCount":"3","FailureCount":"0" ...
2026-07-17T10:25:56.904 [D][USP][Diagnostics]: Ping complete — avg=4ms, 3/3 success

6. The staleness is time/age-dependent, not permanent — the 3rd log (work→timeout) shows a ping succeeding at 10:47 (OperationComplete delivered in chunk Clean up auth on pnp #183) then the same stream going quiet again (only heartbeats [NOW-318] Apps & Gaming: It should not prompt the user that the rule could not be created when the overlapping rule was removed #201–Update Locs #207, no notification frame) and timing out at 10:53:24:

2026-07-17T10:47:54.426 [D][USP][SSE]: debug: Chunk #183 (2284 chars): event: notification   ← OperationComplete delivered, ping OK
2026-07-17T10:50:03.095 [D][USP][SSE]: debug: Chunk #201 (49 chars): event: heartbeat
2026-07-17T10:53:02.946 [D][USP][SSE]: debug: Chunk #207 (49 chars): event: heartbeat
2026-07-17T10:53:24.370 [W][USP][Diagnostics]: Ping timeout: TimeoutException: OperationComplete not received within 30s

7. Code note (dev-2.5.0): sse_operation_awaiter.dart already documents a related firmware quirk where "the next OperationComplete is never delivered" after subscribe/unsubscribe churn, and the ping timeout is a fixed Duration(seconds: 30) in manual_tools_notifier.dart:118 with no stream-staleness reconnect on the SSE side — matching the observed symptom.

Confidence: Medium — the UI-side symptom is fully confirmed (Operate ack succeeds, OperationComplete never delivered on an aged stream that is still heartbeating, F5 = fresh stream fixes it). Why the firmware/bridge stops emitting notification frames while heartbeats continue on a long-lived stream needs firmware/bridge-side confirmation. Marked need-fw-confirm.

---

Auto-generated triage by Austin's tooling, pending Austin's review. Not a fix commitment.