Migrate WebSocket stack to data-access and react to site CRUD events

[fmontes]

Description

The legacy WebSocket layer in @dotcms/dotcms-js (Protocol, WebSocketProtocol, LongPollingProtocol, DotWebSocketConfig, DotcmsEventsService) is an outdated abstraction that predates our current Angular/NgRx patterns. It's also missing reactivity to site CRUD events — when a site is published, archived, stopped, or deleted, the UI doesn't reflect the change without a manual refresh, and the UN_PUBLISH_SITE system event isn't even defined on the backend so stop-site events are silently swallowed.

This task replaces the legacy stack with a modern, typed WebSocket service and wires site CRUD events into the global store so the UI reacts automatically.

Acceptance Criteria

• New DotEventsSocket in libs/data-access: native WebSocket, exponential backoff with jitter, typed on<T>(eventType) API, reactive status$() stream
• New withWebSocket() NgRx signal store feature in libs/global-store: manages connection lifecycle, exposes wsStatus signal and typed event observables
• Legacy Protocol, WebSocketProtocol, LongPollingProtocol, DotWebSocketConfig, and getWebSocketConfig() are removed
• All DotcmsEventsService.subscribeTo() / subscribeToEvents() call sites migrated to DotEventsSocket.on<T>() (DotMessageDisplayService, DotToolbarNotificationsComponent, DotLargeMessageDisplayComponent, IframeComponent, IframePortletLegacyComponent)
• DotSiteComponent refreshes the site list (debounced) on any site WebSocket event and auto-switches to the default site when the selected site is archived/stopped/deleted
• Java SystemEventType.UN_PUBLISH_SITE added so stop-site events are dispatched
• Toolbar reacts to SWITCH_SITE events via a dedicated DotSiteNavigationEffect
• WebSocket reconnects automatically after server restart, with a visible reconnecting indicator
• yarn nx test data-access and yarn nx test global-store pass

Additional Context

Tracking PR: #34990

[github-actions]

PRs linked to this issue

• feat(websockets): migrate WebSocket stack to data-access and react to site CRUD events by @fmontes

[erickgonzalez]

✅ QA Test Execution Summary — PR #35378

Environment: https://dotcms-websockets-test.dotcms.dev/ scaled to 2 replicas (cluster) · Chrome · admin@dotcms.com · demo.dotcms.com
Outcome: All 13 manual test cases passed. ✅

TC	Scenario	Result
TC-001	Remote SWITCH_SITE updates the toolbar site selector live in another session	✅ Pass
TC-002	Archiving a non-selected site refreshes the site dropdown without manual reload	✅ Pass
TC-003	Archiving the currently selected site auto-switches the toolbar to the default site	✅ Pass
TC-004	SWITCH_SITE while editing a page redirects to Site Browser	✅ Pass
TC-005	UPDATE_PORTLET_LAYOUTS reloads the navigation menu live	✅ Pass
TC-006	WebSocket auto-reconnects after server restart	✅ Pass (reconnecting indicator not rendered — minor defect, see notes)
TC-007	Stopping (unpublishing) the selected site auto-switches the UI to the default site — the headline fix	✅ Pass
TC-008	Creating/saving a site refreshes the selector live (SAVE_SITE/PUBLISH_SITE)	✅ Pass
TC-009	Deleting a site emits a USER NOTIFICATION (not DELETE_SITE); refresh driven by prior archive	✅ Pass (by design — DELETE_SITE not emitted)
TC-010	Site events are permission-scoped — a user without READ is not refreshed by that site's events	✅ Pass
TC-011	Legacy event-bus consumers still receive server-pushed messages after the migration	✅ Pass
TC-012	Cross-node propagation — event on node A reaches a client on node B (2-replica cluster)	✅ Pass
TC-013	Legacy iframe portlet still reacts to its subscribed events post-migration	✅ Pass

Notes

• UN_PUBLISH_SITE enum is the real fix — site events dispatch dynamically via ContentletSystemEventUtil → SystemEventType.valueOf("<ACTION>_SITE"); the missing constant made valueOf throw and the stop-site event was silently swallowed. Adding it makes the event fire (confirmed via a live WS frame).
• Reconnecting indicator (AC): auto-reconnect works, but no visible wsStatus indicator is bound in any template — minor defect / follow-up.
• Delete path: host delete emits a NOTIFICATION toast, not DELETE_SITE; the frontend DELETE_SITE subscription is effectively dead in the host path (archive already handles refresh).

📋 Full test plan — 35350-35378-test-plan.md

Test Plan — Migrate WebSocket stack to data-access and react to site CRUD events

Mode: qa-postfix (PR #35378 — open/CHANGES_REQUESTED; exercised on the branch instance https://dotcms-websockets-test.dotcms.dev/ scaled to 2 replicas / cluster, per user instruction)

Summary

These manual checks verify the WebSocket-migration change for issue #35350 on the branch build at https://dotcms-websockets-test.dotcms.dev/ — confirming the UI now reacts to site events without a manual refresh, and surfacing regressions/gaps the migration could have introduced. The change replaces the legacy Protocol/WebSocketProtocol/LongPollingProtocol stack with a native DotEventsSocket (data-access) plus a withWebSocket() NgRx-signal store feature, rewires every DotcmsEventsService consumer through the store, and adds the Java UN_PUBLISH_SITE enum constant.

Strategy: 13 Manual hand-checks against the branch instance scaled to 2 replicas (cluster), covering (a) end-to-end reactivity for the site events that fire (switch / save / publish / unpublish / update / archive / unarchive) plus portlet-layout reload, (b) connection resilience and the reconnecting-indicator gap, (c) permission-scoping and legacy-consumer regression — including the delete path, which emits a NOTIFICATION rather than DELETE_SITE — and (d) cross-node propagation, now verifiable because the instance runs 2 nodes (TC-012). Per Q2 default the absent reconnecting indicator is recorded as a defect rather than a hard blocker. No external docs (Q4).

Server-side dispatch reality (verified on branch — corrected after observing a live UN_PUBLISH_SITE frame):
Site events are dispatched dynamically, not by literal constant reference. ContentletSystemEventUtil.getSystemEventType() builds the event name as <ACTION>_<TYPE> (for a Host, TYPE = SITE) and calls SystemEventType.valueOf(name); ESContentletAPIImpl invokes pushSaveEvent / pushPublishEvent / pushUnpublishEvent / pushArchiveEvent / pushUnArchiveEvent / pushDeleteEvent for every contentlet operation, and a Host is a contentlet.

• Confirmed firing via live WS frames: SAVE_SITE (new host), PUBLISH_SITE, UN_PUBLISH_SITE (stop), UPDATE_SITE, ARCHIVE_SITE, UN_ARCHIVE_SITE — these dispatch end-to-end.
• Before this PR, valueOf("UN_PUBLISH_SITE") threw IllegalArgumentException, which getSystemEventType swallows (returns null) → the stop-site event was silently dropped. Adding the UN_PUBLISH_SITE enum constant is therefore the actual dispatch fix — the one-line Java change delivers the AC.
• DELETE_SITE is NOT emitted on host delete. A live delete instead produces a USER-scoped NOTIFICATION ("Site deleted: …"). Because dotCMS requires a site to be archived before deletion, the preceding ARCHIVE_SITE already refreshed the selector / switched away — so the frontend's DELETE_SITE subscription is effectively unused in the host path (see TC-009).
• Other emitters: SWITCH_SITE (SwitchSiteListener), UPDATE_SITE_PERMISSIONS (PermissionBitAPIImpl), UPDATE_PORTLET_LAYOUTS (role/layout APIs). Note ARCHIVE_SITE/UN_ARCHIVE_SITE are also pushed explicitly by HostAPIImpl/WebAssetFactory, so those may arrive twice (debounce coalesces — see TC-002).

Existing Coverage

Audited on the PR branch (fix-websockets-v3).

New automated tests the PR contributes (already in CI on merge — NOT re-listed as manual TCs):

• libs/data-access/.../dot-events-socket.service.spec.ts — connect, on<T>() typed dispatch, status$() transitions, reconnect after unexpected close, no-reconnect after destroy(), no-reconnect on close code 1001. Covers transport-reconnect axis at unit level.
• libs/global-store/.../with-websocket/with-websocket.feature.spec.ts — wsStatus signal moves connecting→connected→reconnecting→closed. Covers the status signal (but not any visible indicator).
• apps/dotcms-ui/.../dot-app-lifecycle/dot-app-lifecycle.effect.spec.ts — SWITCH_SITE → goToSiteBrowser() only on edit page. Covers the edit-page redirect logic (TC-004's logic layer).
• libs/ui/.../dot-site/dot-site.component.spec.ts (modified) — includes should switch to default site when UN_PUBLISH_SITE fires for the selected site. This asserts the frontend reaction to a mocked UN_PUBLISH_SITE event; the backend genuinely emits that event once the enum exists (confirmed via a live WS frame), so TC-007 verifies the same path end-to-end. Gap: there is no integration test guarding that valueOf("UN_PUBLISH_SITE") keeps resolving — if the enum constant is ever removed, the unit test still passes while production silently swallows the event again (see follow-ups).
• apps/dotcms-ui/.../container-list/container-list.component.spec.ts (modified) — reloads containers via switchSiteEvent$() on SWITCH_SITE (TC-001 consumer at unit level).
• libs/global-store/.../store.spec.ts, with-site.feature, dotcms-events.service.spec.ts (modified) — store wiring + feedMessage() bridge.

Tests deleted by the PR (legacy stack removed — confirm they're gone, not silently disabled): long-polling-protocol.spec.ts, websockets-protocol.spec.ts, dot-event-socket.spec.ts.

Bridge that keeps legacy consumers alive: DotcmsEventsService is now a no-op Subject bus fed by withWebSocket().feedLegacyEventBus() via feedMessage(). Consumers (DotMessageDisplayService, DotToolbarNotificationsComponent, DotLargeMessageDisplayComponent, IframeComponent, IframePortletLegacyComponent) still call subscribeTo() — TC-011 regression-checks this bridge end-to-end.

No E2E scaffold found for the toolbar site selector, site-event reactivity, or reconnect — Playwright coverage of these surfaces does not exist (recorded as a follow-up gap, not a reason to skip).

Out of Scope

• Languages — WebSocket site events carry no languageId; rendering/translation paths untouched.
• Workflow — no Workflow* code in the diff.
• Push Publish — no publishable-entity bundle logic touched.
• Persistence (PostgreSQL) — the only Java change is adding an enum constant; no DDL, no startup/upgrade task, no DB read/write.
• Cache invalidation (server CMS/Identifier/Permission caches) — the change is a client-side event-bus migration; it mutates no server-side cached state. (Client reconnect/transport resilience is in scope — TC-006.)
• Keyboard nav / focus / ARIA / responsive / theme / i18n / copy of the site dropdown — the PR changed only data bindings and the event source of DotSiteComponent/toolbar; the PrimeNG Select interactive markup and its a11y are unchanged. The one new user-visible surface the AC demands — the reconnecting indicator — has no template binding and is covered as a defect under TC-006.
• REST response contract / @Schema — no com.dotcms.rest.* resource changed.
• WebSocket endpoint authentication — de-scoped per user request (TC removed); the WS handshake auth behavior is unchanged by this PR.

Test Cases

- Test ID: TC-001
- Test Name: remote SWITCH_SITE updates the toolbar site selector live in another session
- Risk: High
- Scenario: Happy Path
- Steps To Reproduce:
  1. In Chrome, log in to https://dotcms-websockets-test.dotcms.dev/dotAdmin as admin@dotcms.com.
  2. Open a second session in a separate Chrome profile (or incognito) and log in as the same admin; place the two windows side by side. Both show the toolbar site selector pinned to "demo.dotcms.com".
  3. In window B, open the site selector and switch the current site to "System Host" (or any other available site).
  4. Watch window A without touching it.
- Expected Result: Within ~1s window A's toolbar site selector updates to the site chosen in window B (siteDetails synced via withWebSocket → with-site switchSiteEvent$), with no manual page refresh.
- Type of Test: Manual

- Test ID: TC-002
- Test Name: archiving a non-selected site refreshes the site dropdown list without manual reload
- Risk: High
- Scenario: Happy Path
- Steps To Reproduce:
  1. Log in to the branch instance as admin@dotcms.com; keep current site = demo.dotcms.com.
  2. Under Site Browser → Sites, create or identify a secondary site (e.g. "qa-archive-target") that is NOT the selected site; confirm it appears when you open the toolbar site selector dropdown.
  3. Leave the toolbar visible. In the Sites portlet, archive "qa-archive-target".
  4. Open the toolbar site selector dropdown again (do not reload the page).
- Expected Result: The dropdown list reflects the archived state within ~300ms of the ARCHIVE_SITE event (debounced resetFilter re-queries); "qa-archive-target" no longer appears as an active selectable site. No manual browser refresh was needed.
- Type of Test: Manual

- Test ID: TC-003
- Test Name: archiving the currently selected site auto-switches the toolbar to the default site
- Risk: Critical
- Scenario: Edge
- Steps To Reproduce:
  1. Log in as admin@dotcms.com on the branch instance.
  2. In the toolbar site selector, switch the current/pinned site to a non-default secondary site (e.g. "qa-selected-site"); confirm the selector shows it.
  3. In a second admin session (separate profile), open Site Browser → Sites and archive "qa-selected-site".
  4. Observe the first session's toolbar without refreshing.
- Expected Result: The first session detects ARCHIVE_SITE for the pinned site (SITE_UNAVAILABLE_EVENTS), calls switchSite(null), and the toolbar selector switches to the default site (demo.dotcms.com). The user is not left pinned to an inaccessible site. No console error.
- Type of Test: Manual

- Test ID: TC-004
- Test Name: SWITCH_SITE while editing a page redirects to Site Browser
- Risk: High
- Scenario: Edge
- Steps To Reproduce:
  1. Log in as admin@dotcms.com; navigate into the Page Editor / edit-content for any page on demo.dotcms.com (URL under /dotAdmin/#/edit-page or an edit contentlet route).
  2. In a second admin session, switch the current site to a different site (triggers SWITCH_SITE).
  3. Observe the first session (the one on the edit screen) without interacting.
- Expected Result: DotAppLifecycleEffect fires on the SWITCH_SITE event; because DotRouterService.isEditPage() is true, the app navigates away to the Site Browser, preventing the user from editing content under a now-stale site context. On a non-edit screen no redirect occurs (verify by repeating on the dashboard — stays put).
- Type of Test: Manual

- Test ID: TC-005
- Test Name: UPDATE_PORTLET_LAYOUTS reloads the navigation menu live
- Risk: High
- Scenario: Happy Path
- Steps To Reproduce:
  1. Log in as admin@dotcms.com on the branch instance; note the left-hand navigation menu contents.
  2. In a second admin session, go to Roles, and add or remove a portlet/layout from a role assigned to the admin user (this dispatches UPDATE_PORTLET_LAYOUTS).
  3. Watch the first session's left navigation without refreshing.
- Expected Result: DotNavigationService.portletLayoutUpdated$ reloads the menu (dotMenuService.reloadMenu), re-registers dynamic routes, and the navigation reflects the layout change without a manual page reload. Active menu/breadcrumb state is preserved.
- Type of Test: Manual

- Test ID: TC-006
- Test Name: WebSocket auto-reconnects after server restart (and reconnecting indicator gap)
- Risk: High
- Scenario: Edge
- Steps To Reproduce:
  1. Log in as admin@dotcms.com; open Chrome DevTools → Network → WS and confirm a connection to /api/ws/v1/system/events is OPEN.
  2. Cause a server-side disconnect (restart the branch instance, or in DevTools kill the WS frame / toggle offline then online for ~5s to force onclose with a non-1001 code).
  3. Observe the WS panel and the UI during the outage and after the server is back.
  4. Once reconnected, in a second session switch the current site (TC-001 trigger) and confirm window A still reacts.
- Expected Result: The socket reschedules with exponential backoff + jitter and re-establishes a connection automatically when the server returns (status connecting→reconnecting→connected); after reconnect, site events are received again (step 4 still updates the selector). DEFECT TO RECORD (per Q2): no visible "reconnecting" indicator is rendered anywhere — wsStatus signal exists but has no template binding. Auto-reconnect passing + indicator absent = pass-with-defect.
- Type of Test: Manual

- Test ID: TC-007
- Test Name: Verify #35350 — stopping (unpublishing) the selected site auto-switches the UI to the default site
- Risk: High
- Scenario: Edge
- Steps To Reproduce:
  1. Log in as admin@dotcms.com; in the toolbar pin the current site to a published secondary site (e.g. "qa-stop-target").
  2. Open DevTools → Network → WS in the first session.
  3. In a second admin session, open Site Browser → Sites and Stop (unpublish) "qa-stop-target".
  4. Observe the WS frames and the first session's toolbar without refreshing.
- Expected Result: An UN_PUBLISH_SITE frame is received (the enum now resolves in ContentletSystemEventUtil.valueOf, so the host-unpublish event fires). Because UN_PUBLISH_SITE is in SITE_UNAVAILABLE_EVENTS and the stopped site is the pinned one, dot-site calls switchSite(null) and the toolbar switches to the default site (demo.dotcms.com). This is the headline AC and must pass end-to-end (the bug — stop-site silently swallowed — no longer reproduces).
- Type of Test: Manual

- Test ID: TC-008
- Test Name: Verify #35350 — creating/saving a site refreshes the selector live
- Risk: Medium
- Scenario: Happy Path
- Steps To Reproduce:
  1. Log in as admin@dotcms.com; open the toolbar site selector and note the listed sites.
  2. Open DevTools → Network → WS in this session.
  3. In a second admin session, create and save a new site (e.g. "qa-new-site"), then publish it.
  4. In the first session, open the toolbar site selector dropdown again (no page reload) and inspect the WS frames.
- Expected Result: SAVE_SITE (and PUBLISH_SITE on publish) frames are received and dot-site's debounced resetFilter re-queries the list; "qa-new-site" appears as a selectable site without a manual reload. (SAVE_SITE/PUBLISH_SITE are in SITE_REFRESH_EVENTS and the enum constants exist, so valueOf resolves and the events fire.)
- Type of Test: Manual

- Test ID: TC-009
- Test Name: deleting a site emits a NOTIFICATION (not DELETE_SITE); refresh is driven by the prior archive
- Risk: Medium
- Scenario: Edge
- Steps To Reproduce:
  1. Log in as admin@dotcms.com; create and archive a disposable secondary site "qa-delete-target". Confirm via DevTools → Network → WS that archiving emitted an ARCHIVE_SITE frame and the selector already dropped the site.
  2. Keep the toolbar site selector visible and the WS panel open.
  3. In the Sites portlet, delete the archived "qa-delete-target".
  4. Inspect the WS frames and the toolbar notification area.
- Expected Result: Delete emits a USER-scoped NOTIFICATION frame (title "Site deleted") — NO DELETE_SITE frame is sent for the host. The site does not re-appear and no error occurs; selector accuracy was already handled by the ARCHIVE_SITE event in step 1. CONFIRM the "Site deleted" notification renders in the toolbar bell (this exercises the feedMessage bridge — see TC-011). Record that the frontend's DELETE_SITE subscription is dead code in the host path (follow-up).
- Type of Test: Manual

- Test ID: TC-010
- Test Name: site events are permission-scoped — a user without READ on a site is not refreshed by its events
- Risk: Medium
- Scenario: Negative
- Steps To Reproduce:
  1. As admin, create a back-end user "limited@dotcms.com" with access to the toolbar but WITHOUT READ permission on a secondary site "qa-restricted-site".
  2. Log in as limited@dotcms.com in one session; open the toolbar site selector (qa-restricted-site must not appear).
  3. In an admin session, archive "qa-restricted-site" (ARCHIVE_SITE uses Visibility.PERMISSION).
  4. Inspect the limited user's DevTools → Network → WS frames and the selector.
- Expected Result: The limited user's socket receives no ARCHIVE_SITE frame for the restricted site (Payload Visibility.PERMISSION filters by READ permission); their selector does not refresh or error. Events are scoped to sites the user can see.
- Type of Test: Manual

- Test ID: TC-011
- Test Name: legacy event-bus consumers still receive server-pushed messages after migration
- Risk: Medium
- Scenario: Happy Path
- Steps To Reproduce:
  1. Log in as admin@dotcms.com on the branch instance.
  2. Trigger a server-pushed USER NOTIFICATION that flows through the legacy DotcmsEventsService bus — the simplest reliable trigger is deleting an archived site (emits the "Site deleted" NOTIFICATION frame, as seen in DevTools). A push-publish / bundle job that emits LARGE_MESSAGE also works.
  3. Observe the toolbar notifications dropdown (bell) and the large/normal message display area.
- Expected Result: The notification/message renders in the UI, proving withWebSocket().feedLegacyEventBus() → DotcmsEventsService.feedMessage() bridges raw WS messages to the deprecated subscribeTo() consumers (DotMessageDisplayService, DotToolbarNotificationsComponent, DotLargeMessageDisplayComponent). No regression from the no-op-ing of start()/destroy().
- Type of Test: Manual

- Test ID: TC-012
- Test Name: cross-node propagation of site events to a client on a different cluster node
- Risk: High
- Scenario: Edge
- Steps To Reproduce:
  1. Instance scaled to 2 replicas (cluster). Connect client A (admin) and client B (admin) so they land on different nodes (separate browser profiles; confirm via the WS connection / sticky-session behind the load balancer).
  2. In client B, switch the current site (SWITCH_SITE) and separately archive/stop a site (ARCHIVE_SITE / UN_PUBLISH_SITE).
  3. Observe client A without refreshing.
- Expected Result: Client A receives the event generated by the action on the other node within the propagation window and its toolbar/selector reacts (system events are cluster-broadcast, not node-local) — a stale entry that evicts on one node does not stay stale on the peer. Verified on the 2-replica cluster.
- Type of Test: Manual

- Test ID: TC-013
- Test Name: legacy iframe portlet still reacts to its subscribed events post-migration
- Risk: Low
- Scenario: Happy Path
- Steps To Reproduce:
  1. Log in as admin@dotcms.com; open a legacy iframe-based portlet (e.g. a JSP/legacy admin screen rendered through IframePortletLegacyComponent).
  2. Trigger the event that portlet subscribes to (e.g. an action that the legacy screen listens for via DotcmsEventsService).
  3. Observe the iframe content updates/reloads as before the migration.
- Expected Result: The iframe portlet behaves identically to pre-migration — the feedMessage bridge delivers the event and the legacy component reacts. No console errors about missing DotEventsSocket / DotEventsSocketURL providers (those providers were removed in dotcdn/app modules and test beds).
- Type of Test: Manual

Recommended Automation (follow-up)

These are gaps for the PR author / follow-up issues — not manual TCs:

• No integration guard that UN_PUBLISH_SITE (and the other _SITE events) keep resolving. Dispatch is dynamic via ContentletSystemEventUtil → SystemEventType.valueOf("<ACTION>_SITE"), and a missing constant is swallowed silently (the original bug). The frontend unit test passes regardless. Suggested: an Integration test in dotcms-integration that stops/publishes/saves/archives/deletes a Host and asserts the matching SystemEventType is pushed (e.g. capture via a SystemEventsAPI subscriber). Why: this is the actual regression vector — if the enum constant is ever removed again, only an IT catches it; the headline AC would silently break.
• No visible reconnecting indicator. wsStatus signal has no template consumer. Suggested: bind it in the toolbar and add a Jest+Spectator test asserting the indicator shows when wsStatus() !== 'connected'. Why: AC explicitly requires a visible indicator.
• No E2E for the toolbar site selector reactivity / reconnect. Suggested: Playwright spec driving two contexts (switch site in B → selector updates in A) and a forced-disconnect reconnect path. Why: this is the user-facing core of the feature and has zero browser-level coverage.
• Cross-node propagation has no automated integration coverage (manually verified on the 2-replica cluster — TC-012 passed). Suggested: Integration test (or a documented cluster smoke) exercising mutate-on-node-A → event-received-on-node-B. Why: the manual cluster check doesn't run in CI, so a node-local-only regression could slip through.
• Frontend subscribes to DELETE_SITE, which the host-delete path never emits (delete sends a USER NOTIFICATION instead; refresh is covered by the prior ARCHIVE_SITE). Suggested: either drop DELETE_SITE from SITE_REFRESH_EVENTS/SITE_UNAVAILABLE_EVENTS, or have the backend emit DELETE_SITE on host deletion — and add an IT pinning whichever contract is chosen. Why: dead subscription is misleading and a future host-delete refactor could rely on an event that never arrives.