diff --git a/README.md b/README.md
index 72015833a..fb0dcf7f3 100644
--- a/README.md
+++ b/README.md
@@ -342,6 +342,33 @@ Operational visibility is a first-class concern:
- ticket, certificate, and reload lifecycle management
- autoscaling and cluster integration hooks
+For KingRT production call investigations, use:
+
+```bash
+demo/video-chat/scripts/prod-debug.sh
+```
+
+The prod-debug process is read-only. It inspects public runtime health, domains,
+asset/version endpoints, API/WS/SFU reachability, marketplace and call-app
+reachability, Whiteboard Call App CSP/`Allow-CSP-From` frame headers, container
+status, and recent redacted remote logs. Its Call App proof checks both
+`/public/index.html` and `/call-app/whiteboard/public/index.html` on the
+configured Whiteboard host for `https://app.kingrt.com` compatibility, absence
+of `X-Frame-Options`, and absence of nested `*.app.kingrt.com` service origins.
+Remote log sections are labeled for media reconnect, screen-share reconnect
+exhaustion, stale local media capture discard, audio/video track loss, SFU
+reconnect, and Call App frame/CSP errors. It uses existing
+`demo/video-chat/.env.local` values only for production domains and the SSH
+target. `prod-debug.sh` does not deploy, restart, write DB data, change DNS, or
+use admin actions. Set `VIDEOCHAT_PROD_DEBUG_SKIP_REMOTE=1` to run only public
+HTTP/WebSocket/header probes, or `VIDEOCHAT_PROD_DEBUG_DRY_RUN=1` to prove the
+local read-only flow without network or SSH.
+
+Read-only media reconnect, screen-share reconnect exhaustion, stale local media
+capture discard, audio/video track loss, SFU reconnect, and Call App frame/CSP
+checks in prod-debug.sh are non-mutating: prod-debug.sh does not deploy,
+restart, write DB data, change DNS, or use admin actions.
+
## Public Programming Model
The core programming model is:
diff --git a/SPRINT.md b/SPRINT.md
index e3a7c276e..47a8529fd 100644
--- a/SPRINT.md
+++ b/SPRINT.md
@@ -27,11 +27,13 @@ Status:
current fallback can degrade into a matte that swallows the participant.
Sprint goal:
-- Keep background filters stable across browser ML/GPU regressions with a
- deterministic fallback ladder: MediaPipe GPU when healthy, MediaPipe CPU when
- genuinely isolated, SINet/WASM when MediaPipe is unsafe, then a degraded mode
- that keeps the participant visible instead of blending them into the
- background.
+- Keep Pierre Joye's worker/WebGL background pipeline as the production
+ replacement path: if background replacement is possible and the user wants it,
+ apply it.
+- If background replacement is not possible, do not silently degrade the matte
+ or apply synthetic replacement over the person. Keep media alive and ask the
+ user in a modal to choose a standard avatar, upload an avatar, or send
+ unfiltered camera video with background replacement disabled.
- Preserve visual quality: no softmax/sigmoid participant blending, no ghost
translucency, no full-person disappearance. Edge treatment must use contour
alpha smoothing only.
@@ -40,32 +42,58 @@ Sprint goal:
reload loops, audio loss, or video publication failure.
Current baseline:
-- Production no longer depends on the MediaPipe worker fallback path that hit
- Chrome GPU initialization failures.
-- SINet/WASM exists as the insulated segmentation backend.
-- WebGL and canvas compositors exist; WebGL may fall back to canvas.
-- The current matte behavior can still over-apply the background and visually
- swallow the participant.
+- Production uses Pierre's worker-scoped MediaPipe Tasks pipeline with the
+ WebGL compositor contribution preserved.
+- The background-unavailable path is a user choice, not a hidden fallback:
+ standard avatar, uploaded avatar, or unfiltered video. Avatar mode is a
+ static control-state signal plus live audio, not a fake streamed video track.
+- The compositor must render the source frame while the worker has no renderable
+ matte, so a failed or warming segmenter cannot swallow the participant.
- Whiteboard Call App is deployed and installed; the Call Apps attach tab is now
visible for resolved calls and requests a room snapshot after attach.
+- Media reconnect cleanup now has a focused contract proving stale local capture
+ cleanup preserves active camera/audio/screenshare streams and emits
+ reconnect-specific diagnostics instead of looking like an intentional media
+ shutdown. A second browser-oriented smoke runs the real retired-stream cleanup
+ helper in a fake MediaStream sandbox and proves active camera, microphone, and
+ screen-share tracks stay live while retired tracks stop once. The main smoke
+ script exposes a deterministic Node-only release gate for these reconnect and
+ screenshare contracts without requiring real devices.
+- Realtime websocket reconnect now treats transient auth/backend and call-room
+ backfill failures as retryable instead of silently falling back to lobby or
+ closing as a revoked session. Requested call reconnects receive explicit
+ retryable diagnostics and authoritative room/lobby snapshots once backfill is
+ available.
+- Public call-access join now captures the verified logged-in user/session after
+ link verification and sends that context plus the current bearer token into
+ call-access session issuance, so a later login switch cannot bind the link to
+ a different account.
Contract anchors:
- `demo/video-chat/frontend-vue/src/domain/realtime/background/stream.ts`
-- `demo/video-chat/frontend-vue/src/domain/realtime/background/backendSinetWasm.js`
-- `demo/video-chat/frontend-vue/src/domain/realtime/background/maskPostprocess.js`
-- `demo/video-chat/frontend-vue/src/domain/realtime/background/pipeline/compositorCanvasStage.js`
-- `demo/video-chat/frontend-vue/src/domain/realtime/background/pipeline/compositorWebglStage.js`
+- `demo/video-chat/frontend-vue/src/domain/realtime/background/backendWorkerSegmenter.js`
+- `demo/video-chat/frontend-vue/src/domain/realtime/background/workers/imageSegmenterWorker.js`
+- `demo/video-chat/frontend-vue/src/domain/realtime/background/BackgroundReplacementUnavailableModal.vue`
+- `demo/video-chat/frontend-vue/src/domain/realtime/background/pipeline/compositorStage.js`
- `demo/video-chat/frontend-vue/tests/standalone/king-background-segmentation-harness.ts`
- `demo/video-chat/frontend-vue/tests/contract/background-filter-mask-contract.mjs`
- `demo/video-chat/frontend-vue/tests/contract/background-king-wasm-contract.mjs`
-- `demo/video-chat/frontend-vue/tests/contract/background-sinet-defaults-contract.mjs`
- `demo/video-chat/frontend-vue/tests/contract/background-segmentation-harness-contract.mjs`
- `demo/video-chat/frontend-vue/tests/contract/mediapipe-cdn-contract.mjs`
+- `demo/video-chat/deploy.sh`
+- `demo/video-chat/docker-compose.v1.yml`
+- `demo/video-chat/scripts/lib/deploy-hetzner.sh`
+- `demo/video-chat/edge/edge.php`
+- `demo/video-chat/frontend-vue/src/support/backendOrigin.ts`
+- `demo/video-chat/frontend-vue/src/domain/realtime/callApps/callAppWorkspaceState.js`
+- `demo/video-chat/backend-king-php/domain/call_apps/call_app_semantic_dns.php`
+- `demo/video-chat/backend-king-php/domain/marketplace/call_app_marketplace.php`
Execution boundary:
- Preserve Pierre Joye's WebGL/background-removal contribution history and do
not rewrite or squash that work.
-- Do not restore a brittle MediaPipe-only path as the only production backend.
+- Do not replace Pierre's worker pipeline with an unrelated production SINet or
+ "degraded matte" implementation.
- Do not add softmax, sigmoid, or whole-mask alpha curves that make a person
semi-transparent.
- Do not grow `CallWorkspaceView.vue`; new behavior belongs in focused
@@ -75,18 +103,29 @@ Execution boundary:
Acceptance criteria:
- Chrome/Chromium MediaPipe GPU-service init failures do not break calls.
-- If GPU init fails, CPU/SINet/degraded fallback is selected exactly once per
- cooldown window without reload loops.
+- If background replacement init fails, the source remains visible and the user
+ gets a modal choice: standard avatar, uploaded avatar, or unfiltered video.
- The participant center remains opaque in the matte harness; only contour
pixels receive alpha smoothing.
- No `Math.exp`, softmax, sigmoid, or equivalent probabilistic blending exists
in the production fallback matte path.
-- Background filter failure cannot mute audio, remove the local video track, or
- stop media publication.
+- Background filter failure cannot mute audio or stop media publication.
+ Avatar choice must stop avatar/video frame publication and signal one static
+ image state to peers until another media control-state replaces it.
- Diagnostics name the selected backend, failed backend, browser family, GPU
- availability, model source, fallback reason, and cooldown state.
+ availability, model source, fallback reason, and that user choice is required.
- Production deploy smoke proves call app static assets and background model
assets are served from the expected origins.
+- Production service domains are rooted at `kingrt.com` only:
+ `api.kingrt.com`, `ws.kingrt.com`, `sfu.kingrt.com`, `cdn.kingrt.com`,
+ `turn.kingrt.com`, and `registry.kingrt.com`. No service origin may derive
+ from `app.kingrt.com`.
+- Whiteboard is hosted as `whiteboard.kingrt.com`, appears in the Marketplace,
+ can be added to a production `kingrt.com` organization, and then appears in
+ that organization's Call Apps tab.
+- `registry.kingrt.com` is the canonical Semantic-DNS and mothernode join
+ registry, including dev-key approval for KingRT network membership and
+ self-hosted call-app manifests that point at a private mothernode.
Tickets:
- [ ] BGF-01 Browser regression matrix and reproducible failure capture
@@ -96,27 +135,40 @@ Tickets:
and whether CPU delegation still touches GPU internals.
- Add a contract fixture for the known Chrome GPU-service init failure shape.
-- [ ] BGF-02 Backend selection ladder with quarantine
- - Introduce a backend selector contract for MediaPipe GPU, MediaPipe CPU,
- SINet/WASM, and degraded mode.
- - Quarantine a failing backend for a bounded cooldown instead of retrying per
- frame.
+- [x] BGF-02 Backend selection ladder with quarantine
+ - Keep production on Pierre's worker segmenter pipeline, with MediaPipe scoped
+ to the worker boundary.
+ - When worker init fails, render the source frame and open the user-choice
+ modal instead of silently selecting another matte backend.
- Ensure backend switching is idempotent and cannot trigger reload loops.
+ - Proof: `background-regression-matrix-contract.mjs` and
+ `background-king-wasm-contract.mjs` pin the worker boundary, idempotent init,
+ source-visible unavailable state, and explicit modal alternatives.
-- [ ] BGF-03 Matte correctness: hard foreground plus contour smoothing
+- [x] BGF-03 Matte correctness: hard foreground plus contour smoothing
- Remove any remaining softmax/sigmoid-style probability blending from the
fallback path.
- Treat foreground/background classification as hard membership, then apply
alpha only on the contour band.
- Add harness checks that the torso/face center stays opaque and background
pixels do not leak into the participant.
+ - Proof: `background-filter-mask-contract.mjs` pins Pierre's worker pipeline,
+ no softmax/sigmoid fallback blending, source-visible warmup/failure
+ rendering, and explicit modal alternatives.
-- [ ] BGF-04 Degraded mode that preserves the person
- - Define degraded mode as "no synthetic replacement over the participant" when
- segmentation confidence or backend health is unsafe.
- - Prefer original camera video with clear diagnostics over a broken matte that
- hides the participant.
+- [x] BGF-04 Background-unavailable user choice
+ - Ask the user to choose standard avatar, uploaded avatar, or unfiltered video
+ when background replacement is unavailable.
+ - Avatar choice signals one static image state to peers and keeps only live
+ audio in the published stream; it must not stream avatar frames or deltas.
+ - Never apply synthetic replacement over the participant without a renderable
+ matte.
- Keep user media tracks alive while the filter is disabled or warming up.
+ - Proof: `BackgroundReplacementUnavailableModal.vue`,
+ `avatarFallbackSignal.ts`, `staticAvatarRender.ts`,
+ `mediaOrchestration.ts`, and `background-filter-mask-contract.mjs` wire the
+ standard-avatar, uploaded-avatar, and unfiltered-video choices while
+ preserving audio tracks and rendering avatars as static tile media.
- [ ] BGF-05 Compositor and warmup safety
- Make WebGL/canvas compositor warmup deterministic across backend changes.
@@ -126,8 +178,8 @@ Tickets:
segmentation-unavailable states.
- [ ] BGF-06 Runtime diagnostics and field observability
- - Emit throttled diagnostics for backend init, fallback transition, quarantine,
- matte rejection, and degraded mode.
+ - Emit throttled diagnostics for backend init, unavailable transition, modal
+ choice, and matte rejection.
- Include enough local context to debug browser regressions without leaking
media frames, SDP, ICE, or tokens.
- Surface concise state in existing diagnostics channels, not new reload UI.
@@ -138,6 +190,54 @@ Tickets:
screenshare, reconnect, and background filter transitions.
- Record proof commands and results in this sprint before closing.
+- [x] BGF-08 KingRT Domain Contract Cutover
+ - Split deploy configuration into `kingrt.com` as the base domain and
+ `app.kingrt.com` as the frontend application domain.
+ - Serve production services only on `api.kingrt.com`, `ws.kingrt.com`,
+ `sfu.kingrt.com`, `cdn.kingrt.com`, `turn.kingrt.com`, and
+ `registry.kingrt.com`.
+ - Hard-remove old nested service domains such as `cdn.app.kingrt.com` and
+ `api.app.kingrt.com`; do not keep aliases for the cutover.
+ - Add a domain-contract test that fails if any generated service domain ends
+ in `.app.kingrt.com`.
+ - Proof: `codex/domain-registry-cutover` deployed to production, deploy smoke
+ passed, live frontend bundle scan found no `*.app.kingrt.com` service
+ origins, and authoritative Hetzner DNS no longer serves old nested A
+ records.
+
+- [x] BGF-09 Call App Hosting and Semantic Registry
+ - Host Whiteboard at `whiteboard.kingrt.com` and resolve future Call Apps as
+ `{app_key}.kingrt.com`.
+ - Reserve service names such as `app`, `api`, `ws`, `sfu`, `cdn`, `turn`,
+ and `registry` so Call Apps cannot claim platform domains.
+ - Use `registry.kingrt.com` as the dev-key-approved registry for Call App
+ registration, Semantic DNS, and mothernode join announcements.
+ - Allow self-hosted Call App manifests to declare a private mothernode that
+ is not part of the KingRT network.
+ - Proof: production deploy serves Whiteboard from `whiteboard.kingrt.com`,
+ uses `registry.kingrt.com` for mothernode/registry configuration, and
+ semantic Call App DNS reserves platform service labels.
+
+- [x] BGF-10 Whiteboard Marketplace Production Proof
+ - Ensure Whiteboard is visible in the production Marketplace.
+ - Ensure the add-to-organization/install action is present and persists
+ backend entitlements/installations.
+ - Ensure Whiteboard appears in the Call Apps tab for calls owned by that
+ organization after installation.
+ - Proof: production Marketplace read-only probe reported one healthy catalog
+ entry, one enabled installation, one active entitlement, and the seeded
+ call listed Whiteboard as available.
+
+- [x] BGF-11 sicherstellen, dass whiteboard auch bei kingrt.com einer orga zugefügt werden kann
+ - Run the production `kingrt.com` Marketplace journey end to end.
+ - Prove that a real organization can add Whiteboard from Marketplace and use
+ it inside a call without manual database edits.
+ - Record the production proof command/output before closing the sprint.
+ - Proof: production admin Marketplace order/install endpoints were exercised
+ idempotently against `api.kingrt.com`; Whiteboard then appeared in the
+ organization's Call Apps availability response and launched from
+ `whiteboard.kingrt.com`.
+
## Sprint: Whiteboard Call App Hardening And Production Integration
Branch:
@@ -165,7 +265,10 @@ Status:
- [x] GSP-03 Join/snapshot/churn topology hints remains closed by
`gossip-room-state-topology-contract.mjs`.
- [x] GSP-04 Dedicated bounded neighbor lifecycle remains closed by
- `gossip-dedicated-neighbor-lifecycle-contract.mjs`.
+ `gossip-dedicated-neighbor-lifecycle-contract.mjs`; production stack-overflow
+ proof `gossip-neighbor-renegotiate-stack-contract.mjs` now pins queued
+ renegotiation as deduped, timer-based, bounded, and cleared on peer close
+ instead of recursively calling `negotiatePeer`.
- [x] GSP-07 Gossip-native recovery remains closed by
`gossip-native-recovery-contract.mjs`; backend recovery routing stays ops-only
with no media fanout.
@@ -336,6 +439,9 @@ Tickets:
- Owner reconnects, bootstraps from replay, and renders the existing
document state.
- `npm run test:e2e:call-app-whiteboard` passes.
+ - Merged install-sidebar proof keeps the Marketplace install/sidebar access
+ journey in the same command; latest integrated run passed both Whiteboard
+ E2E specs.
- [x] WCA-05 Backend SQLite-runtime proof
- Run PDO-backed backend Call App contracts in a SQLite-enabled PHP runtime.
@@ -369,6 +475,18 @@ Tickets:
`presence.types` and are rejected by CRDT append persistence.
- Whiteboard emits throttled `call_app.presence.publish` messages for
cursor and selection state.
+ - Additional merged proof gates cursor presence by Call App grants, keeps
+ named remote cursors visible only for authorized participants, and clears
+ cursor/selection state after revocation:
+ `call-app-whiteboard-cursors-access-contract`.
+ - Named cursor proof now renders the authorized sender display name into
+ the Whiteboard DOM overlay (`.remote-cursor-label`) as well as the canvas;
+ E2E asserts the participant sees `Owner` and that the label is removed
+ after grant revocation.
+ - Regression proof renders multiple named remote cursors (`Owner`,
+ `Reviewer`, `Facilitator`), removes only the leaving remote cursor on
+ `call_app.presence.leave`, clears all remote cursor labels on revoke, and
+ asserts the participant iframe launch count and URL do not change.
- Move undo/redo for shapes, text, and sticky notes uses CRDT update ops.
- Browser E2E covers duplicate/out-of-order replay injection, throttled
non-persistent presence, snapshot compaction, revoke, and reconnect.
@@ -398,6 +516,19 @@ Tickets:
desktop and mobile, keeping iframe sizing stable.
- Launch grant capabilities now drive visible no-access/read-only notices
in the Call App workspace host.
+ - Additional merged proof makes the Call Apps sidebar responsive at narrow
+ widths, shows Default participant access as Blocked/Allowed choices, and
+ exposes labeled Allow/Revoke controls for participant grants:
+ `call-app-sidebar-access-ux-contract`.
+ - `call-app-whiteboard-install-browser-proof-contract` adds a browser proof
+ path for Marketplace order/install, installed Whiteboard availability in
+ the Call Apps sidebar, default participant access selection, backend
+ participant grant mutation, and narrow-sidebar responsiveness without
+ manual database edits.
+ - The same browser proof now runs the real Whiteboard iframe beside the
+ host/sidebar controls, renders the `Owner` remote cursor label, toggles a
+ sidebar participant grant, and asserts the cursor label, iframe URL, and
+ Whiteboard launch count remain unchanged.
- Exact commands:
- `npm run test:contract:call-apps`
- `npm run build`
@@ -418,6 +549,24 @@ Tickets:
snapshot compaction.
- Frontend Call App bridges emit `king:call-app-diagnostic` browser events
and redact sensitive fields before dispatching diagnostics.
+ - `prod-debug.sh` provides a read-only production diagnostics process for
+ runtime/version, app/CDN/API/WS/SFU reachability, marketplace/call-app
+ checks, container status, and redacted recent logs without deploys,
+ restarts, DB writes, DNS changes, or admin actions.
+ - Merged production-debug proof labels media reconnect, screen-share
+ reconnect exhaustion, stale local media capture discard, audio/video track
+ loss, SFU reconnect, and Call App frame/CSP log slices; dry-run mode now
+ avoids network/SSH and preserves explicit domain overrides over local
+ `.env.local` values.
+ - `call-app-csp-postmessage-contract` proves configured Whiteboard/call-app
+ iframe hosts are normalized safely, launch/CRDT bridge messages are
+ structured-clone safe, and postMessage failure is terminal for the current
+ launch generation instead of causing retry/reload loops.
+ - `call-app-frame-csp-headers-contract` and deploy smoke now require
+ Whiteboard frame responses to deliver compatible `Content-Security-Policy`
+ and `Allow-CSP-From` headers for `app.kingrt.com`, without
+ `X-Frame-Options`, nested app domains, or wildcard script/connect/frame
+ policies.
- `WHITEBOARD_CHECK.md` is an unfilled manual acceptance form for owner,
moderator, participant, guest, revoked participant, reconnect, and export.
- Exact commands:
@@ -467,3 +616,1922 @@ Tickets:
- All commands passed. Host-PHP PDO-SQLite contracts still skip inside
`test:contract:call-apps`; the pinned `php:8.5-cli-trixie` SQLite
proof passes in Docker.
+
+
+
+
+
+# Sprint Task: e2e-enhancement/iam-develop-1.0.8-beta
+
+## Goal
+
+Implement complete end-to-end test coverage for the IAM, call-access, invitation, lobby, guest-account, owner-rights, call-lifecycle, and role-based join flows of the videocall solution.
+
+This sprint is focused on turning the complete permission and identity model into executable E2E tests that run reliably in CI. The tests must validate all relevant user states, organization roles, personalized invitation links, anonymous join links, temporary accounts, account reconciliation flows, lobby behavior, rejoin behavior, ownership transfer rules, invitation invalidation, guest cleanup, call rescheduling, call deletion, call ending, owner absence timeout, and duplicate-link abuse detection.
+
+The expected outcome is not a manual checklist only, but an automated E2E test suite that can be executed in CI and prevents regressions in the IAM and videocall access model.
+
+## Background
+
+The product supports:
+
+- organizations
+- registered users
+- organization-level roles
+- system admins
+- call owners
+- guest lists
+- calendar-based invitations
+- personalized call links
+- anonymous join links
+- temporary guest accounts
+- lobby-based admission
+- temporary moderators
+- ownership transfer
+- explicit and implicit call ending
+- guest-account cleanup when call state changes
+
+A registered user can belong to an organization and can have either an `Admin` or `User` role inside that organization.
+
+A user may be:
+
+- registered and logged in
+- registered and logged out
+- not registered
+- represented by a temporary personalized guest account
+- represented by a temporary anonymous guest account
+
+There are two main link types:
+
+1. Personalized call links
+ These are created through invitation / calendar flows and are associated with a specific invitee or temporary account.
+
+2. Anonymous join links
+ These allow people to attempt joining a call without a personalized identity binding.
+
+The system must correctly distinguish between:
+
+- registered users
+- logged-in users
+- logged-out users
+- organization admins
+- normal organization users
+- system admins
+- call owners
+- temporary guest accounts
+- anonymous temporary accounts
+- users on the guest list
+- users admitted through the lobby
+- users who were kicked
+- users rejoining after admission
+- users opening links that were issued for someone else
+- users whose organization membership changed after invitation
+- users whose invitation link was invalidated
+- users trying to join deleted, ended, or rescheduled calls
+
+---
+
+# Required Work for Codex
+
+Codex must create, extend, or refactor the E2E test suite so that the described IAM and videocall access model is fully covered.
+
+The preferred test framework is Playwright unless the repository already uses another E2E framework.
+
+If Playwright is not yet present, add it in a way that is compatible with the existing frontend, backend, auth, media, and CI setup.
+
+The E2E tests must run automatically in CI.
+
+The implementation must include:
+
+- deterministic test data setup
+- isolated organizations
+- isolated users
+- isolated calls
+- isolated invitations
+- isolated personalized guest links
+- isolated anonymous join links
+- repeatable cleanup
+- CI-safe execution
+- headless browser execution
+- test fixtures for authenticated users
+- test fixtures for unauthenticated users
+- test fixtures for organization admins
+- test fixtures for normal organization users
+- test fixtures for system admins
+- test fixtures for call owners
+- test fixtures for registered invited guests
+- test fixtures for temporary personalized guests
+- test fixtures for temporary anonymous guests
+- test coverage for personalized and anonymous join flows
+- test coverage for temporary guest account creation
+- test coverage for account reconciliation
+- test coverage for lobby admission
+- test coverage for lobby rejection
+- test coverage for kick behavior
+- test coverage for rejoin behavior
+- test coverage for owner transfer
+- test coverage for permission retention / permission loss
+- test coverage for duplicate personalized-link usage
+- test coverage for review-flag creation
+- test coverage for organization membership changes after invitation
+- test coverage for invite invalidation
+- test coverage for guest-account cleanup
+- test coverage for call rescheduling
+- test coverage for call deletion
+- test coverage for explicit call ending
+- test coverage for implicit call ending after owner absence
+- test coverage for final 5-minute countdown display
+
+---
+
+# Playwright / E2E Framework Requirements
+
+Use Playwright unless a different E2E framework already exists and is the established project standard.
+
+If Playwright is already present:
+
+- extend the existing Playwright setup
+- reuse existing fixtures
+- reuse existing auth helpers
+- reuse existing seed helpers
+- reuse existing CI jobs where possible
+- do not create a parallel test architecture
+
+If Playwright is not present:
+
+- add Playwright with browser installation suitable for CI
+- add a dedicated E2E test command
+- add test fixtures
+- add authenticated browser contexts
+- add unauthenticated browser contexts
+- add trace capture on failure
+- add screenshot capture on failure
+- add video capture on failure if feasible
+- add retry policy only where acceptable for CI stability
+- add project-level documentation for running tests locally and in CI
+
+Required capabilities:
+
+- create organizations
+- create registered users
+- assign organization roles
+- log in users
+- create calls
+- create personalized invitations
+- create anonymous join links
+- invalidate links
+- modify guest lists
+- remove users from organizations
+- promote / demote users
+- delete calls
+- reschedule calls
+- end calls
+- simulate owner disconnect
+- simulate owner reconnect
+- assert visible UI states
+- assert backend state when needed
+- assert audit logs when available
+- assert review flags when expected
+
+---
+
+# CI Requirements
+
+The E2E suite must be executable in CI using a single documented command.
+
+The CI job must:
+
+- start all required services
+- migrate the test database
+- seed deterministic test data
+- start the application
+- start auth dependencies if required
+- start media / signaling infrastructure if required
+- start `king` participant containers where needed
+- execute E2E tests headlessly
+- collect Playwright traces on failure
+- collect screenshots on failure
+- collect videos on failure if feasible
+- collect application logs on failure
+- collect backend logs on failure
+- collect media / signaling logs on failure
+- collect `king` container logs on failure
+- fail the pipeline if any required E2E test fails
+- avoid depending on external services unless explicitly mocked or sandboxed
+- avoid leaking real user data
+- avoid real production credentials
+- be deterministic and repeatable
+- run without GPU requirements
+- run without manual interaction
+- not wait 15 real minutes for owner-timeout tests
+
+---
+
+# Virtual Call Participants in CI: `king` Containers
+
+Some tests require multiple concurrent call participants.
+
+For CI, use lightweight virtual participant containers named `king` containers.
+
+A `king` container represents a simulated participant that can join a call and stream deterministic dummy media into the call. These containers do not need to render real UI unless the architecture requires it. Their purpose is to simulate real call participants in a reproducible CI-safe way.
+
+If the existing system already has a test participant simulator, extend it instead of creating a parallel implementation.
+
+If no such simulator exists, add the minimal required `king` container implementation for CI-based E2E coverage.
+
+## `king` Container Capabilities
+
+Each `king` participant container must be able to:
+
+- join a call using a provided link or token
+- optionally act as the call owner
+- optionally act as a logged-in registered user
+- optionally act as an organization admin
+- optionally act as a normal organization user
+- optionally act as a temporary personalized guest
+- optionally act as an anonymous temporary guest
+- provide deterministic fake audio input
+- provide deterministic fake video input
+- stay connected for the duration of the test
+- disconnect gracefully
+- simulate abrupt disconnect
+- simulate browser crash / process kill
+- simulate network loss
+- reconnect with the same identity
+- reconnect with a different identity when explicitly needed
+- simulate multiple participants using the same link
+- simulate multiple participants using different links
+- expose current call state
+- expose participant identity state
+- expose whether the owner-absence countdown is visible
+- expose countdown value or allow Playwright to assert it through UI
+- expose logs for join, disconnect, reconnect, kick, end, and media state
+- terminate cleanly after test completion
+- run in CI without GPU requirements
+
+For owner-timeout tests:
+
+- at least one `king` container must be able to act as the owner participant
+- the owner `king` container must be able to disconnect without explicitly ending the call
+- at least one additional `king` container must remain in the call
+- remaining participants must be able to observe the final 5-minute countdown
+- remaining participants must be able to observe the automatic call-ended state
+
+---
+
+# Product Rule: Membership Revocation After Invitation
+
+A valid explicit call invitation must remain usable even if the invited user is removed from the organization after the invitation was issued.
+
+Organization membership is evaluated for organization-level privileges, but not as the sole condition for invitation validity.
+
+This prevents the following failure case:
+
+- a host invites participants to a call while they are still organization members
+- those participants are removed from the organization before the call starts
+- the participants can no longer join the very call in which the removal / offboarding is supposed to happen
+
+## Rule
+
+If a registered user was explicitly invited to a call before losing organization membership, the invitation remains valid as a call-scoped guest permission.
+
+The user no longer has organization-level rights, but they may still join the specific invited call as an invited participant.
+
+## Resulting Behavior
+
+- removed organization member loses all organization-based permissions immediately
+- removed organization admin loses organization-admin permissions immediately
+- removed user cannot browse, create, manage, or join other organization calls through org membership
+- removed user cannot access organization resources outside the specific invitation
+- removed user may still join the specific call if they have a still-valid explicit invitation
+- the invitation is downgraded to call-scoped guest access
+- the user joins as an invited guest, not as an organization member
+- the user does not regain organization membership
+- the user does not regain organization role permissions
+- the user does not receive admin, org-admin, or owner rights through the old membership
+- the user can be admitted directly if the invitation grants direct access
+- the user can be routed through lobby if the invitation requires host approval
+- the host / admin can manually invalidate the invite if access should be revoked
+- if the invite link is invalidated, the removed user can no longer join
+- if the call is deleted, the removed user can no longer join
+- if the call is ended, the removed user can no longer join
+- if the user was kicked, kick rules override invitation access
+
+## Active Session Rule
+
+If a participant is already inside the call and is then removed from the organization:
+
+- they remain in the current call if their access came from an explicit invitation or call-scoped guest permission
+- they immediately lose organization-level privileges
+- they do not lose ordinary participant presence automatically
+- they cannot perform organization-admin actions anymore
+- they cannot use organization membership to join other calls
+- they may remain until the call ends, they leave, or they are kicked
+- the host / owner / moderator can remove them manually if needed
+
+## Admin / Owner Exception
+
+If the removed participant was an organization admin:
+
+- org-admin privileges are revoked immediately
+- call participation may continue only if they also have call-scoped permission
+- if they were call owner, owner handling must follow the owner-transfer / call-ending rule
+- if no valid call-scoped permission remains, they should be moved to lobby or removed according to the call policy
+
+---
+
+# Product Rule: Personalized Link Opened by Logged-In Different User
+
+When a logged-in user opens a personalized call link that was issued for a temporary account or another invitee:
+
+- the currently logged-in account must remain active
+- the session must not be replaced by the temporary guest account
+- the temporary link account must be compared with the logged-in account
+- if there is no strong mismatch, the logged-in account is used for the call
+- if there is a strong mismatch, the system must show a warning modal
+- strong mismatch means first name and/or last name are materially different
+- the warning modal must not show the other person’s data
+- the user must be asked to provide the host name
+- if the host name is wrong, access is denied or routed to manual review / lobby
+- if the host name is correct, the user may be asked whether to update account data
+- account data differences must not be displayed
+- the user must re-enter the differing values manually
+- if the user wants to update account data, a confirmation email must be sent to the email address of the currently logged-in account
+- account data must not be updated until email confirmation is completed
+- the email must not be sent to the temporary guest account
+- the logged-in account remains the active account throughout the flow
+
+If another logged-in account later uses the same personalized link:
+
+- the account must be flagged for review
+- this must also happen if another account uses the link while the first account is already in the call
+- duplicate personalized-link use must be audit-logged
+
+---
+
+# Product Rule: Anonymous Join Links
+
+When a logged-in user opens an anonymous join link:
+
+- no temporary account should be used as the active identity
+- any temporary anonymous identity created for this flow should be removed or discarded
+- the user joins as the logged-in account
+- the logged-in account’s rights are used
+
+When a non-logged-in user opens an anonymous join link:
+
+- a temporary anonymous guest account may be created
+- the temporary anonymous guest lands in the lobby unless the product explicitly grants direct access
+- the host, temporary moderator, org admin, or system admin may admit the guest
+- once admitted and not kicked, the temporary guest may leave and rejoin without needing to be admitted again
+- if kicked, kick rules override previous admission
+
+---
+
+# Product Rule: Join Permissions
+
+System admins:
+
+- can join every call through normal active-call paths
+- do not need to be on the guest list
+- do not need an invitation
+- cannot bypass deleted or ended call state unless an explicit recovery/debug path exists outside normal join flow
+
+Organization admins:
+
+- can join every active call belonging to their own organization
+- do not need to be on the guest list for calls in their organization
+- cannot join calls of other organizations through org-admin rights
+- cannot bypass deleted or ended call state through normal join flow
+
+Normal users:
+
+- can join calls when they are on the guest list
+- can join calls they own
+- can join calls through a valid explicit invitation
+- cannot directly join unrelated calls
+- can create their own calls
+- become owner of calls they create
+
+Call owners:
+
+- have admin rights in their own call
+- may admit lobby participants
+- may remove / kick participants
+- may transfer ownership
+- may end the call
+
+Owner transfer:
+
+- if a normal org user transfers owner rights to another user, the old owner loses call-admin rights
+- if an organization admin transfers owner rights to another user, the old owner keeps admin rights
+- the new owner receives owner rights
+- there must be exactly one current owner unless the product explicitly supports multiple owners
+
+---
+
+# Product Rule: Call Rescheduling
+
+When a call is rescheduled:
+
+- stale access paths must not remain valid unintentionally
+- old personalized links must be invalidated or migrated according to the product rule
+- old temporary guest accounts must be deleted, invalidated, or migrated according to the product rule
+- old lobby entries must be cleared or migrated according to the product rule
+- admitted temporary participants from the old schedule must be cleared or migrated according to the product rule
+- old links must not join users into the wrong call instance
+- all changes must be audit-logged
+
+---
+
+# Product Rule: Call Deletion
+
+When a call is deleted:
+
+- the deleted call cannot be joined through normal product paths
+- owner cannot join the deleted call
+- org admin cannot join the deleted call
+- system admin cannot join the deleted call through normal join flow
+- personalized invite links become invalid
+- anonymous join links become invalid
+- temporary guest accounts are deleted or invalidated
+- lobby entries are cleared
+- admitted temporary participant state is cleared
+- active participants are disconnected or moved into a safe deleted-call state
+- audit log is preserved
+- registered user accounts are not deleted
+- unrelated calls and guests are not affected
+
+---
+
+# Product Rule: Call Ending
+
+A call can be ended explicitly or implicitly.
+
+## Explicit End
+
+The owner intentionally ends the call or leaves in a way that the product treats as an explicit call end.
+
+Expected behavior:
+
+- call moves to ended state
+- active participants are notified
+- new joins are blocked
+- rejoins are blocked
+- personalized invite links are invalidated
+- anonymous join links are invalidated
+- temporary guest accounts are deleted or invalidated
+- lobby entries are cleared
+- audit log is preserved
+
+## Implicit End by Owner Absence
+
+If the owner is absent from the call for 15 minutes, the call ends automatically.
+
+Examples of owner absence:
+
+- internet outage
+- owner loses connection
+- owner closes browser tab
+- owner browser crashes
+- owner process is killed
+- owner network is disconnected
+
+The final 5 minutes of the implicit-end timer must be visible to the remaining participants.
+
+Expected behavior:
+
+- owner absence starts a 15-minute server-side timer
+- the timer is based on server time, not client time
+- the last 5 minutes of that timer are shown to remaining participants
+- countdown starts when 5 minutes remain
+- countdown updates correctly
+- countdown is synchronized across participants
+- countdown disappears if owner rejoins
+- owner reconnect before timeout cancels pending implicit end
+- call automatically ends when owner has been absent for 15 minutes
+- once ended, the call is no longer joinable through normal links
+- temporary call-scoped access is cleaned up after end
+
+## CI Timer Requirement
+
+Owner-absence timeout must be E2E-testable without making CI wait for real 15-minute durations.
+
+Codex must implement or use a test-safe time-control mechanism.
+
+Preferred approaches:
+
+- configurable timeout values in test environment
+- fake timers at the service layer
+- server-side test clock injection
+- admin/test-only endpoint to advance call lifecycle time
+- deterministic event simulation for owner disconnect and reconnect
+
+The CI tests must not sleep for 15 real minutes.
+
+Required CI timer coverage:
+
+- simulate owner absence
+- advance time to 9 minutes 59 seconds equivalent
+- verify no countdown is visible yet unless product rule says otherwise
+- advance time to 10 minutes equivalent
+- verify final 5-minute countdown is visible
+- advance time during countdown
+- verify countdown updates
+- reconnect owner during countdown
+- verify countdown disappears and call remains active
+- disconnect owner again
+- advance time to 15 minutes equivalent
+- verify call ends
+- verify all participants receive ended state
+- verify new joins are blocked
+- verify guest accounts / links are invalidated or cleaned up
+
+---
+
+# Main Acceptance Criteria
+
+The sprint is complete when:
+
+- the automated E2E suite covers the full IAM and videocall access matrix
+- tests run successfully in CI
+- CI fails on permission regressions
+- CI fails on identity regressions
+- CI fails on guest-link regressions
+- CI fails on lobby regressions
+- CI fails on owner-transfer regressions
+- CI fails on invite-invalidation regressions
+- CI fails on duplicate-link regressions
+- CI fails on call-lifecycle regressions
+- CI fails on owner-timeout regressions
+- virtual participants can be simulated in CI using `king` containers
+- test data is isolated and repeatable
+- failed test runs provide enough artifacts to debug the issue
+- all new tests are documented
+- the manual checklist is mapped to automated test case IDs
+
+---
+
+# E2E Case Checklist
+
+## 1. Organization, User, Roles, Login States
+
+- [ ] Organization can be created
+- [ ] User can be registered in an organization
+- [ ] User can have organization role `User`
+- [ ] User can have organization role `Admin`
+- [ ] User with role `User` can log in
+- [ ] User with role `Admin` can log in
+- [ ] User can be logged out
+- [ ] Logged-in user remains logged in when opening a call link
+- [ ] Logged-out user has no active account session
+- [ ] User without organization cannot receive organization-based rights
+- [x] User from organization A does not receive rights from organization B
+- [x] Organization admin from organization A cannot join organization B calls through org-admin rights
+- [x] Organization role is evaluated server-side
+- [x] Stale organization role in client cache is ignored
+- [x] Stale organization role in session token is revalidated where required
+
+Proof: `call-access-stale-organization-role-contract` issues a session while a
+user has tenant and organization admin roles, downgrades both roles without
+rotating the token, and proves auth/session payloads, local session cache
+fallback, call access, call-admin decisions, forged role input, and stale
+client role cache data all resolve from current backend membership state.
+
+## 2. Call Creation and Owner Rights
+
+- [ ] Registered user with role `User` can create own call
+- [ ] Registered user with role `Admin` can create own call
+- [ ] Call creator becomes call owner
+- [ ] Call creator receives admin rights in own call
+- [ ] Owner can add users to guest list
+- [ ] Owner can manage guest list
+- [ ] Owner can admit lobby participants
+- [ ] Owner can remove / kick participants
+- [ ] Owner rights can be transferred to another user
+- [ ] If organization role `User` transfers owner rights, old owner loses call-admin rights
+- [ ] If organization role `Admin` transfers owner rights, old owner keeps admin rights
+- [ ] New owner receives owner rights
+- [ ] New owner receives admin rights in call
+- [ ] After owner transfer, there is exactly one current owner
+- [ ] Former owner without admin role can no longer perform owner actions
+- [ ] Organization admin can keep call-admin rights after owner transfer
+- [ ] Owner rights cannot be transferred to a non-existent user
+- [ ] Owner rights cannot be transferred across forbidden organization boundaries
+- [ ] Owner transfer is audit-logged
+
+## 3. Join Permissions
+
+- [ ] System admin can join every active call
+- [ ] System admin can join without guest-list entry
+- [ ] System admin can join without invitation
+- [ ] System admin cannot join deleted call through normal join flow
+- [ ] System admin cannot join ended call through normal join flow
+- [ ] Organization admin can join every active call of own organization
+- [ ] Organization admin can join own organization call without guest-list entry
+- [ ] Organization admin cannot join another organization’s call through org-admin rights
+- [ ] User can join call when on guest list
+- [ ] User cannot directly join call when not on guest list
+- [ ] User can join own call as owner
+- [x] User cannot directly join unrelated foreign call
+- [ ] Deleted / disabled user cannot join
+- [ ] Removed guest-list entry revokes direct join access
+- [ ] Newly added guest-list entry grants direct join access
+- [ ] Permissions are checked server-side
+- [ ] Manipulated client role does not grant access
+- [x] Manipulated call ID does not grant access to another call
+
+## 4. Calendar Invitation Flow
+
+- [ ] Host can invite person through calendar flow
+- [ ] Invitee can select appointment in calendar form
+- [ ] Invitee can be registered and logged in
+- [ ] Invitee can be registered and logged out
+- [ ] Invitee can be unregistered
+- [ ] Temporary account is created for non-logged-in invitee
+- [ ] Temporary account contains form data
+- [ ] Personalized call link is associated with temporary account
+- [ ] Personalized call link is unique
+- [ ] Personalized call link is not guessable
+- [ ] Personalized call link is server-side bound to temporary account
+- [ ] Calendar appointment is correctly associated with call / host
+- [ ] Multiple invitees receive different personalized links
+- [ ] Appointment change does not modify unrelated invitations
+- [ ] Invitation cancellation invalidates personalized link
+- [ ] Expired personalized link cannot be used if expiry exists
+- [ ] Reopening same personalized link by same valid context behaves consistently
+
+## 5. Personalized Link: User Not Logged In
+
+- [ ] Not logged-in user opens personalized link
+- [ ] Temporary account from link data is created / used
+- [ ] Temporary account does not automatically log in existing registered account
+- [ ] User enters intended flow with temporary account
+- [ ] Temporary account may be on guest list
+- [ ] Temporary account on guest list can join directly
+- [ ] Temporary account not on guest list lands in lobby
+- [ ] Temporary account cannot see other users’ data
+- [ ] Temporary account receives no registered account rights
+- [ ] Temporary account cannot administer call unless explicitly permitted
+- [ ] Temporary account cannot assume another identity by changing link parameters
+- [ ] Temporary account remains consistent for same link / call
+- [ ] Temporary account can be recognized after leaving
+- [ ] Temporary account cannot receive organization-wide rights
+- [ ] Invalid personalized link is rejected
+- [ ] Manipulated personalized link is rejected
+- [ ] Error state for invalid personalized link leaks no data
+
+## 6. Personalized Link: Logged-In User, No / Light Mismatch
+
+- [ ] Logged-in user opens personalized link
+- [x] Logged-in account remains active
+- [x] Temporary link account does not replace active session
+- [ ] Link account is compared with logged-in account
+- [ ] No mismatch does not show warning modal
+- [ ] Light mismatch does not show strong foreign-link warning
+- [x] Logged-in account is used for call
+- [x] Temporary account is not set as active session
+- [x] Permission check uses logged-in account
+- [ ] User can join if logged-in account is authorized
+- [ ] User lands in lobby if logged-in account is not directly authorized
+- [ ] Temporary link data does not overwrite account data automatically
+- [ ] Light mismatches are optionally logged
+- [ ] No link data is unnecessarily exposed in frontend
+- [ ] Same logged-in account can reopen same link without duplicate-link flag
+
+Proof: `call-access-verified-context-ui-contract` proves the public join view
+captures a stable verified user/session/token snapshot after link verification,
+sends `verified_user_id`, `verified_session_id`, and the current bearer token to
+call-access session issuance, and fails safely with `call_access_conflict` if
+the verified context remains after local logout. Backend route-guard proof keeps
+the authenticated account authoritative for the issued session.
+
+## 7. Personalized Link: Logged-In User, Strong Mismatch
+
+- [ ] Logged-in user opens personalized link with strongly different link data
+- [ ] Strong mismatch is detected when first name differs
+- [ ] Strong mismatch is detected when last name differs
+- [ ] Strong mismatch is detected when first and last name differ
+- [ ] Warning modal is displayed
+- [ ] Warning modal explains link may have been issued for someone else
+- [ ] Warning modal explains link data differs from account data
+- [ ] Warning modal asks for host name
+- [ ] Link data of other person is not displayed
+- [ ] Differing link data is not exposed in clear text
+- [ ] Host name is verified server-side
+- [x] Wrong host name grants no direct access
+- [x] Wrong host name does not reveal foreign data
+- [ ] Wrong host name may lead to lobby / manual review
+- [ ] Correct host name is accepted
+- [ ] Correct host name shows success confirmation
+- [ ] After correct host name, user is asked whether account data should be updated
+- [ ] User can decline update
+- [ ] Declining update leaves logged-in account unchanged
+- [ ] Declining update continues with logged-in account
+- [ ] User can request account update
+- [ ] User must re-enter differing values manually
+- [ ] System does not show differing link values
+- [x] System does not show data from guessed / foreign link
+- [ ] Email confirmation is sent to logged-in account email
+- [ ] Email is not sent to temporary link-account email
+- [ ] Without email confirmation, account data is not updated
+- [ ] With email confirmation, only confirmed data is updated
+- [ ] After update, user remains logged in as original account
+- [ ] Update does not modify temporary foreign account
+- [ ] Update does not modify other registered accounts
+- [ ] Flow is audit-logged
+- [ ] Host-name brute force is rate-limited
+- [ ] Repeated wrong host names trigger lock / review if configured
+- [x] Host-name error messages leak no host data
+
+Proof: `call-access-strong-mismatch-privacy-contract` pins the focused browser
+case in `call-access-join.spec.js`: a logged-in wrong account opens a
+personalized link, the simulated server returns a strong-mismatch wrong-host
+denial, the join/session responses contain no invitee/host/session sentinels,
+the UI renders only the generic forbidden state, no workspace/lobby admission is
+entered, and the active browser session remains unchanged.
+
+Proof: `call-access-strong-mismatch-privacy-contract` creates a personalized
+link for one invitee, authenticates a strongly different logged-in account, and
+proves the backend `/api/call-access/{id}/join` and `/session` routes return
+only generic mismatch/host-name field errors for unverified or wrong host-name
+attempts. The responses contain no target invitee, host, external participant,
+call title, call id, or denied session id, and no call-access session is
+persisted.
+
+## 8. Duplicate Personalized Link / Abuse Detection
+
+- [ ] Personalized link is first opened by account A
+- [ ] Same personalized link is reopened by account A
+- [ ] Reuse by same account does not create false foreign-account flag
+- [ ] Same personalized link is later opened by account B
+- [ ] Use of same personalized link by different logged-in account is detected
+- [ ] Account B is flagged for review
+- [ ] Account A may appear as affected reference in audit log
+- [ ] Flag is created even if account B provides correct host name
+- [ ] Flag is created even if account B does not enter the call
+- [ ] Flag is created when account B reaches warning modal if policy requires it
+- [ ] Concurrent use of same personalized link by two accounts is detected
+- [ ] Race condition on parallel link open creates no inconsistent assignment
+- [ ] Link already used inside call marks later use by other account as suspicious
+- [ ] Temporary account cannot be taken over by second registered account without review
+- [ ] Review flag contains call, link ID, affected accounts, and timestamps
+- [ ] Review flag contains no unnecessary sensitive link data
+- [ ] Admin / reviewer can understand the flag
+- [ ] Abuse detection works after logout / login switch in same browser
+- [ ] Abuse detection works across devices
+- [ ] Abuse detection works across browsers
+
+## 9. Anonymous Join Link: User Logged In
+
+- [ ] Logged-in user opens anonymous join link
+- [ ] Temporary account is not permanently created or is removed
+- [ ] User joins as logged-in user
+- [ ] Logged-in user’s rights are used
+- [ ] Logged-in user receives no rights from anonymous link
+- [ ] Logged-in user can join if own rights allow it
+- [ ] Logged-in user lands in lobby if no direct permission exists
+- [ ] Logged-in system admin can join every active call through anonymous link
+- [ ] Logged-in organization admin can join own organization calls through anonymous link
+- [ ] Logged-in organization admin cannot join foreign organization calls through anonymous link
+- [ ] Logged-in guest-list user can join through anonymous link
+- [ ] Logged-in user not on guest list lands in lobby through anonymous link
+- [ ] Anonymous link does not overwrite account data
+- [ ] Anonymous link does not modify guest list
+- [ ] Anonymous link creates no personalized identity binding
+- [ ] Invalid anonymous link is rejected
+- [ ] Manipulated anonymous link grants no access
+
+## 10. Anonymous Join Link: User Not Logged In
+
+- [x] Not logged-in user opens anonymous join link
+- [x] Temporary anonymous account is created
+- [ ] Temporary anonymous account lands in lobby
+- [ ] Temporary anonymous account receives no registered user rights
+- [ ] Temporary anonymous account receives no organization rights
+- [ ] Temporary anonymous account receives no owner rights
+- [ ] Lobby shows waiting anonymous user according to privacy rules
+- [ ] Host can admit anonymous user
+- [ ] Temporary moderator can admit anonymous user
+- [ ] Admin can admit anonymous user
+- [ ] Unauthorized participant cannot admit anonymous user
+- [ ] After admission, anonymous temporary user enters call
+- [ ] If admitted anonymous user leaves and was not kicked, they can rejoin
+- [ ] Rejoin after admission does not require another approval
+- [ ] If anonymous user was kicked, rejoin requires approval or is blocked
+- [ ] Anonymous temporary user cannot gain rights by changing display name
+- [ ] Multiple anonymous users through same link are separate temporary participants
+- [ ] Anonymous link does not reveal guest list or account data
+- [ ] Anonymous link can be disabled if supported
+- [ ] Disabled anonymous link allows no lobby entry
+
+## 11. Lobby and Admission
+
+- [ ] User without direct permission lands in lobby
+- [ ] Anonymous not logged-in user lands in lobby
+- [ ] Personalized temporary user without direct permission lands in lobby
+- [ ] Logged-in user without direct permission lands in lobby
+- [ ] Lobby entry informs host / authorized moderators
+- [ ] Host sees waiting participant
+- [ ] Temporary moderator sees waiting participant
+- [ ] Organization admin sees waiting participant for own organization call
+- [ ] System admin sees waiting participant
+- [ ] Unauthorized user sees no lobby management controls
+- [x] Host can admit participant
+- [ ] Temporary moderator can admit participant
+- [x] Organization admin can admit participant
+- [x] System admin can admit participant
+- [x] Host can reject participant
+- [ ] Temporary moderator can reject participant
+- [x] Organization admin can reject participant
+- [x] System admin can reject participant
+- [ ] Rejected participant cannot enter call
+- [ ] Admitted participant enters call
+- [x] Admission is stored call-scoped
+- [x] Admission does not apply to other calls
+- [x] Admission does not apply to other organizations
+- [ ] Temporary user admission applies only to same temporary user / link context
+- [x] Concurrent admission by multiple moderators creates no error state
+- [x] Concurrent rejection and admission resolves deterministically
+- [ ] Lobby status updates correctly
+- [ ] Participant is removed from lobby after admission
+- [ ] Participant is removed from lobby after aborting join attempt
+- [x] Participant is not shown twice in lobby
+- [x] Manipulated lobby-admission request without permission is rejected
+
+Proof: `call-access-session-contract` creates a personal access link, persists
+the access/session binding with access ID, call ID, room ID, user ID, and link
+kind, proves the session waits for the bound room until allowed, then enters only
+that bound room. The same contract requests a secondary call ID with the
+access-bound session and proves it does not enter or queue admission for the
+secondary room. It also creates an open anonymous access session with a guest
+name and proves a distinct guest user plus open call-scoped binding is created.
+`realtime-lobby-concurrency-contract` simulates two backend workers against the
+same call-participant row and proves concurrent `lobby/allow` is idempotent,
+late duplicate admission returns `already_allowed` without mutating state, and
+admit/reject races resolve deterministically with rejection winning and no
+queued or admitted handoff left behind.
+`lobby-concurrency-ui.spec.js` drives the browser workspace with duplicate queue
+snapshots, an admitted-plus-stale-queue race snapshot, duplicate room
+participant rows, and final reject-empty state; it proves the lobby badge and
+panel render one queued user, admitted/rejected states leave no stale allow
+controls, and duplicate participant snapshot rows aggregate into one UI row.
+
+## 12. Rejoin, Leave, Kick
+
+- [ ] Admitted temporary user can leave call
+- [ ] Admitted temporary user can reopen same call
+- [ ] Admitted temporary user can rejoin without approval
+- [ ] Rejoin works after browser refresh
+- [ ] Rejoin works after short network interruption
+- [ ] Rejoin works after closing tab and reopening if session remains
+- [ ] Rejoin does not work as another user with same temporary context if account binding is violated
+- [ ] Kicked temporary user cannot directly rejoin
+- [ ] Kicked temporary user lands back in lobby or is blocked
+- [ ] Kicked logged-in user cannot immediately reenter through same link if kick overrides access
+- [ ] Kick state overrides previous admission
+- [ ] Kick state is stored server-side
+- [ ] Kick state is scoped to affected call if intended
+- [ ] Kick state is scoped to affected user / temporary account
+- [ ] Registered authorized user can rejoin after leaving
+- [ ] Admin can rejoin after leaving
+- [ ] Organization admin can rejoin after leaving
+- [ ] Guest-list user can rejoin after leaving
+- [ ] Rejoin after guest-list removal is denied or routed to lobby
+- [ ] Rejoin after admin-role removal uses updated permissions
+- [ ] Rejoin after owner transfer uses updated permissions
+
+## 13. Temporary Moderators
+
+- [ ] Host can assign temporary moderator if supported
+- [ ] Temporary moderator can admit lobby participants
+- [ ] Temporary moderator can reject lobby participants
+- [ ] Temporary moderator can only moderate assigned call
+- [ ] Temporary moderator cannot perform organization-wide admin actions
+- [ ] Temporary moderator cannot transfer owner rights unless allowed
+- [ ] Temporary moderator cannot modify guest list outside permissions
+- [ ] Temporary moderator loses rights after moderation ends
+- [ ] Temporary moderator loses rights after call end if configured
+- [ ] Revoked temporary moderator rights take effect immediately
+- [ ] Manipulated temporary-moderator role in client is rejected server-side
+
+## 14. Privacy and Data Minimization
+
+- [x] Foreign link data is not shown on strong mismatch
+- [ ] Differing data is not shown as comparison list
+- [ ] User must re-enter differing data manually
+- [x] Guessed link reveals no personal data
+- [x] Invalid link reveals no personal data
+- [x] Wrong host name reveals no personal data
+- [ ] Account data is updated only after email confirmation
+- [ ] Email confirmation goes only to logged-in account
+- [ ] Temporary account data is not persisted unnecessarily
+- [ ] Temporary accounts are removed when logged-in user uses anonymous link
+- [ ] Temporary accounts are not merged with wrong registered account
+- [x] Audit logs contain only necessary personal data
+- [x] Frontend state contains no foreign link data
+- [x] API responses contain no foreign link data
+- [x] Browser DevTools / network response contains no foreign link data
+- [x] Error messages contain no foreign link data
+- [ ] Email texts contain no foreign link data unless explicitly safe and necessary
+- [ ] Host-name verification does not allow host enumeration
+- [ ] Rate limits protect sensitive verification paths
+- [x] Privacy-relevant actions are logged
+
+## 15. Security and Manipulation Cases
+
+- [x] Personalized link with modified link ID is rejected
+- [ ] Personalized link with modified call ID is rejected
+- [ ] Anonymous link with modified call ID is rejected
+- [x] Expired link is rejected
+- [ ] Disabled link is rejected
+- [x] Deleted temporary account cannot be revived through old link
+- [ ] API request with forged user ID is rejected
+- [x] API request with forged role parameter is rejected
+- [ ] API request with forged organization parameter is rejected
+- [x] API request with foreign call ID is rejected
+- [x] Owner transfer request without owner/admin right is rejected
+- [x] Lobby admission request without moderator right is rejected
+- [x] Kick request without moderator right is rejected
+- [ ] Account-data update request without email confirmation is rejected
+- [ ] Replay of old email confirmation link is prevented
+- [ ] Email confirmation link is one-time use
+- [ ] Email confirmation link is time-limited
+- [ ] CSRF protection works for account-data change
+- [x] Session fixation during link opening is prevented
+- [x] Login switch during link verification does not cause wrong account binding
+- [x] Logout during link verification causes no data leak
+- [ ] Parallel tabs with different accounts cause no incorrect merge
+- [ ] Permission changes during active call are applied correctly
+- [ ] Owner transfer during active call is applied correctly
+- [ ] Guest-list change during active call is applied correctly
+- [ ] Kick during active call removes user
+- [ ] Deleted call cannot be entered
+- [ ] Ended call cannot be entered
+
+Proof: `call-access-session-fixation-contract` rejects reuse of an existing
+session ID for a new call-access binding, rejects login/account switches between
+verified and authenticated context, rejects wrong-account personalized-link
+issuance, and makes tampered or expired call-access session bindings fail
+session validation. `call-access-session-route-guard-contract` proves the real
+`/api/call-access/{id}/session` route passes authenticated and verified
+user/session context into that guard, rejects wrong logged-in accounts and
+session switches safely, and preserves anonymous personalized/open-link issuance.
+`call-access-verified-context-ui-contract` adds the frontend half of that
+contract by proving the browser join flow forwards the verified context and
+current bearer token into session issuance instead of trusting mutable local
+state. The focused Playwright case `call-access-join.spec.js` covers the
+browser login-switch path after link verification, asserting the session POST
+carries the verified context plus the current bearer, returns a safe conflict,
+does not render foreign response data, and does not replace local session state.
+The focused Playwright logout case clears the verified browser session after
+link verification and proves the join flow fails locally without a
+call-access session POST, workspace navigation, foreign data rendering, or
+foreign session adoption.
+
+## 16. Email Confirmation for Account Data Update
+
+- [ ] Email is triggered only after explicit update request
+- [ ] Email is sent to logged-in account
+- [ ] Email is not sent to temporary account
+- [ ] Email contains secure confirmation link
+- [ ] Confirmation link is account-bound
+- [ ] Confirmation link cannot be used by another logged-in account
+- [ ] Confirmation link is time-limited
+- [ ] Confirmation link is one-time use
+- [ ] Without confirmation, account data remains unchanged
+- [ ] After confirmation, only re-entered data is updated
+- [ ] Confirmation success state is shown
+- [ ] Expired confirmation link updates no data
+- [ ] Already used confirmation link updates no data again
+- [ ] Confirmation is audit-logged
+- [ ] Failed confirmation shows no sensitive data
+- [ ] While confirmation is pending, user can continue with original account
+- [ ] Multiple pending confirmations are handled correctly
+- [ ] Newer change invalidates older confirmation if configured
+- [ ] Race condition between two confirmations resolves deterministically
+
+## 17. Guest List
+
+- [ ] Host can add registered user to guest list
+- [ ] Host can add temporary invited account to guest list
+- [ ] Host can remove guest-list entry
+- [x] User on guest list can directly join
+- [x] User not on guest list cannot directly join
+- [ ] Temporary account on guest list can directly join
+- [ ] Temporary account not on guest list lands in lobby
+- [x] Organization admin does not need guest-list entry for own organization call
+- [x] System admin does not need guest-list entry
+- [x] Guest list is call-scoped
+- [x] Guest list of one call grants no rights to another call
+- [ ] Guest list of one organization grants no rights to another organization
+- [ ] Duplicate guest-list entries are prevented or merged
+- [ ] Removing guest-list entry affects new join attempts immediately
+- [ ] Removing guest-list entry during active call follows product rule
+- [ ] Guest-list changes are audit-logged
+
+## 18. System Admin
+
+- [x] System admin can join call from every organization
+- [ ] System admin can join call without organization if such calls exist
+- [x] System admin can join without guest-list entry
+- [x] System admin can manage lobby
+- [x] System admin can admit participants
+- [x] System admin can reject participants
+- [x] System admin can kick participants
+- [ ] System admin can view / handle review flags if supported
+- [x] System admin rights are never granted to temporary accounts
+- [x] System admin rights cannot be simulated through link data
+- [x] System admin rights remain after owner transfer
+- [x] System admin cannot be degraded through call-owner transfer
+
+## 19. Organization Admin
+
+- [x] Organization admin can join every active call of own organization
+- [x] Organization admin can join own organization call without guest-list entry
+- [x] Organization admin can manage lobby for own organization calls
+- [x] Organization admin can admit participants for own organization calls
+- [x] Organization admin can reject participants for own organization calls
+- [x] Organization admin can kick participants for own organization calls
+- [x] Organization admin cannot join foreign organization calls through this role
+- [x] Organization admin cannot manage lobby of foreign organization
+- [ ] Organization admin rights remain after owner transfer
+- [ ] Organization admin can transfer owner rights if allowed
+- [ ] Organization admin keeps admin rights when transferring ownership
+- [ ] Revoking organization-admin role affects new joins and admin actions immediately
+- [ ] Organization admin rights cannot be expanded through manipulated organization ID
+
+## 20. Normal User
+
+- [x] Normal user can create own call
+- [x] Normal user becomes owner of own call
+- [x] Normal user has admin rights in own call
+- [ ] Normal user can join foreign call only when authorized
+- [x] Normal user can join foreign call when on guest list
+- [x] Normal user cannot join foreign call when not on guest list
+- [x] Normal user cannot manage foreign lobby
+- [x] Normal user can admit participants in own call
+- [x] Normal user can reject participants in own call
+- [x] Normal user can kick participants in own call
+- [x] Normal user loses call-admin rights when transferring ownership
+- [x] Normal user cannot perform owner actions after owner transfer
+- [x] Normal user keeps no hidden admin rights after owner transfer
+- [x] Normal user cannot receive admin rights through anonymous link
+- [x] Normal user cannot receive admin rights through personalized link
+
+Proof: `call-creation-owner-rights-contract` creates calls through the backend
+`POST /api/calls` route as a registered normal user and as a registered admin,
+then verifies the persisted `calls.owner_user_id`, creator room ownership,
+creator `call_participants.call_role = owner`, owner role contexts,
+`can_moderate`, `can_manage_owner`, and own-call update authority. It does not
+exercise owner transfer.
+`call-access-admin-prevention-contract` issues a personalized link for a
+normal user and an anonymous/open link while a normal user context is present,
+then authenticates the issued sessions and proves role, tenant-admin,
+system-admin, moderator, owner-management, and call-admin checks all remain
+false.
+
+## 21. Cross-Organization Cases
+
+- [ ] User from organization A opens personalized link for organization A call
+- [ ] User from organization A opens personalized link for organization B call
+- [ ] User from organization A opens anonymous link for organization B call
+- [x] Organization admin from organization A opens link to organization A call
+- [x] Organization admin from organization A opens link to organization B call
+- [x] Organization admin from organization A receives no org-admin rights in organization B call
+- [ ] User with accounts in multiple organizations is checked in correct call context
+- [ ] Changing active organization in frontend does not change server-side call permission
+- [x] Guest-list entry in organization A does not apply to organization B
+- [ ] Temporary account from organization A invitation receives no rights in organization B
+- [x] Owner rights of organization A call do not apply to organization B call
+- [ ] Review flags are assigned to correct organization / call
+
+Proof: `realtime-call-scope-contract` resolves realtime call context with
+tenant-aware room/call binding, rejects forged same-tenant and foreign-tenant
+room joins, prevents foreign tenant lobby hydration, and proves presence in one
+call room does not imply subscription or moderation rights in another room.
+
+## 22. Multi-Session, Devices, Browsers
+
+- [ ] Logged-in user opens personalized link in browser A
+- [ ] Same user opens same link in browser B
+- [ ] Same user opens same link on another device
+- [x] Different user opens same personalized link on another device
+- [ ] Different active session triggers review flag
+- [ ] Not logged-in user opens same personalized link on another device
+- [ ] Parallel use of same temporary account is handled correctly
+- [ ] Concurrent join attempts create no duplicate participants
+- [ ] Logout in one tab affects link verification in another tab correctly
+- [ ] Login switch during warning modal is handled correctly
+- [ ] Email confirmation in another browser updates correct account
+- [ ] Session expiry while waiting in lobby is handled correctly
+- [x] Session expiry during call creates defined state
+- [ ] Refresh during host-name verification creates defined state
+- [ ] Refresh while email confirmation is pending creates defined state
+
+Proof: `realtime-reconnect-backfill-contract` proves transient auth backend
+errors remain retryable inside a bounded grace window, revoked sessions still
+close as policy violations, requested call reconnect backfill failures return
+retryable 503 diagnostics before websocket upgrade, and successful backfill
+sends authoritative lobby/room snapshots for the call room.
+Browser proof: `npm run test:contract:realtime-reconnect-browser` proves
+transient websocket auth/backfill errors stay in retrying UI state with
+retryable diagnostics and request room snapshot backfill after reconnect.
+Browser E2E proof: `npm run test:e2e:realtime-reconnect-websocket` drives the
+workspace through retryable websocket auth and reconnect-backfill failures in a
+fake browser WebSocket and proves no reload/logout path fires while reconnect
+requests a fresh `room/snapshot/request`.
+
+## 23. Organization Membership Changes After Invitation
+
+- [x] Invited registered user is removed from organization before opening personalized invite link
+- [x] Removed invited user can still open still-valid personalized invite link
+- [x] Removed invited user joins only as call-scoped invited guest
+- [x] Removed invited user does not retain organization-member rights
+- [x] Removed invited user does not retain organization-admin rights
+- [ ] Removed invited user cannot join other organization calls
+- [x] Removed invited user cannot access organization resources
+- [ ] Removed invited user cannot manage call unless separately owner/moderator
+- [x] Removed invited user cannot use stale role data from token/session/cache
+- [x] Removed invited user is blocked if invite was manually invalidated
+- [ ] Removed invited user is blocked if call was deleted
+- [ ] Removed invited user is blocked if call was ended
+- [ ] Removed invited user is blocked or routed according to policy if kicked
+- [ ] User already inside call remains connected after org removal if access was call-scoped
+- [ ] User already inside call immediately loses organization-level privileges after removal
+- [ ] Removed org-admin already inside call loses org-admin controls immediately
+- [ ] Removed org-admin already inside call remains only if explicit call-scoped access exists
+- [ ] Removed user can leave and rejoin same call only while invitation remains valid
+- [ ] Removed user cannot rejoin after invite invalidation
+- [x] Audit log records membership removal
+- [ ] Audit log records permission downgrade
+- [x] Audit log records continued call-scoped access
+- [ ] User invited as org member but later moved to another organization joins only through call-scoped invitation
+- [ ] User invited as org admin but later downgraded to user loses org-admin access but keeps explicit invite access
+- [ ] User invited as normal user but later promoted to org admin receives current org-admin rights if still member
+- [ ] Removed org admin cannot use org-admin rights from stale invite payload
+- [ ] Removed user in lobby loses org-based rights but may remain in lobby through call-scoped invitation
+
+Proof: `call-access-membership-removal-contract` removes tenant, organization,
+and group memberships before opening the personalized link; proves a pre-removal
+normal tenant session with elevated tenant/org rights is rejected after removal,
+including through the locally issued session cache fallback; verifies an
+organization-scoped resource grant is lost with organization membership;
+verifies the link still resolves; issues a call-scoped session; authenticates
+through the call-access fallback without recreating membership; proves
+`tenant_admin` stays false and organization resource access stays denied; and
+confirms the admitted user enters only the bound call room. The focused
+Playwright spec `call-access-join.spec.js` proves the public join/session browser
+path and waiting-for-host state.
+
+## 24. Invite Link Invalidation
+
+- [x] Personalized invite link is manually invalidated before use
+- [ ] Personalized invite link is invalidated after first use
+- [ ] Personalized invite link is invalidated while invitee is in lobby
+- [ ] Personalized invite link is invalidated while invitee is already in call
+- [ ] Anonymous join link is manually invalidated before use
+- [ ] Anonymous join link is invalidated while anonymous guest is in lobby
+- [ ] Anonymous join link is invalidated while anonymous guest is already in call
+- [x] Invalidated link cannot be used for fresh join attempts
+- [ ] Invalidated link cannot be used for rejoin unless product rule allows admitted rejoin
+- [x] Invalidated link does not reveal whether original invitee exists
+- [x] Invalidated link does not reveal guest account data
+- [x] Invalidated link does not recreate deleted temporary accounts
+- [x] Invalidated link state is enforced server-side
+- [ ] Invalidated link state works across browsers
+- [ ] Invalidated link state works across devices
+- [ ] Invalidated link state works across sessions
+- [ ] Invalidated link state survives application restart during CI
+- [x] Rejected invalidated link shows safe invalid-link state
+- [x] Rejected invalidated link does not leak personal data
+- [ ] Stale client-side state cannot join with invalidated link
+
+## 25. Guest Account Lifecycle
+
+- [ ] Guest account is created from personalized calendar invitation
+- [ ] Guest account is deleted when call is deleted
+- [ ] Guest account is deleted or invalidated when invitation is deleted
+- [x] Guest account is deleted or invalidated when invite link is manually invalidated
+- [ ] Guest account is updated, recreated, or invalidated when call is rescheduled according to product rule
+- [ ] Guest account cannot join original call after call was rescheduled and original link invalidated
+- [ ] Guest account cannot join after call was deleted
+- [ ] Guest account cannot join after call was ended
+- [x] Guest account cannot rejoin after cleanup
+- [ ] Guest account cannot be used to infer deleted call data
+- [x] Guest account cleanup does not delete registered user accounts
+- [x] Guest account cleanup does not alter registered user profile data
+- [x] Guest account cleanup does not remove unrelated temporary guests from other calls
+- [x] Guest account cleanup is scoped to affected call / invitation
+- [x] Guest account cleanup is idempotent
+- [x] Guest account cleanup is audit-logged
+- [x] Old guest account cannot be revived through old personalized link
+- [x] Old guest account cannot be revived through stale browser state
+- [ ] Old guest account cannot be revived after application restart
+
+Proof: `demo/video-chat/backend-king-php/tests/call-guest-lifecycle-contract.php`
+now repeats personalized guest cleanup after the account/session were already
+invalidated, verifies the second pass has zero destructive changes, and asserts
+both the first pass and idempotent repeat append sanitized
+`guest_account_cleanup` audit events without raw guest, session, or access-link
+identifiers.
+
+## 26. Call Rescheduling
+
+- [ ] Owner reschedules call before guest opens invite link
+- [ ] Owner reschedules call while guest is in lobby
+- [ ] Owner reschedules call while guest is already inside call
+- [ ] Personalized invite link from old time is invalidated after reschedule if required
+- [ ] New personalized invite link is issued after reschedule if required
+- [ ] Old temporary guest account is deleted, invalidated, or migrated according to product rule
+- [ ] Guest using old link after reschedule cannot join stale call state
+- [ ] Guest using new link after reschedule can join according to current permissions
+- [ ] Registered invited user receives correct behavior when using old link after reschedule
+- [ ] Anonymous join link behavior after reschedule is tested
+- [ ] Lobby entries from old schedule are cleared or migrated according to product rule
+- [ ] Admitted temporary participants from old schedule are cleared or migrated according to product rule
+- [ ] Audit log records reschedule
+- [ ] Audit log records related invite cleanup
+- [ ] Audit log records guest cleanup
+- [ ] Stale links do not join users into wrong call instance
+- [ ] Frontend shows safe and clear state for old links
+
+## 27. Call Deletion
+
+- [ ] Owner deletes call before any guest joins
+- [ ] Owner deletes call while guests are in lobby
+- [ ] Owner deletes call while registered users are inside
+- [ ] Owner deletes call while temporary guests are inside
+- [ ] Owner deletes call while anonymous guests are inside
+- [ ] Deleted call cannot be joined by owner
+- [ ] Deleted call cannot be joined by organization admin
+- [ ] Deleted call cannot be joined by system admin through normal join flow
+- [ ] Deleted call cannot be joined through personalized invite link
+- [ ] Deleted call cannot be joined through anonymous join link
+- [ ] Deleted call cannot be rejoined by previously admitted guest
+- [ ] Deleted call removes or invalidates temporary guest accounts
+- [ ] Deleted call clears lobby entries
+- [ ] Deleted call clears admitted temporary participant state
+- [ ] Deleted call preserves audit log
+- [ ] Deleted call does not delete registered user accounts
+- [ ] Deleted call does not delete unrelated calls
+- [ ] Deleted call does not delete unrelated guests
+- [ ] Users currently in call are disconnected or moved into safe deleted state
+- [ ] Deleted call metadata is not leaked to unauthorized users
+
+## 28. Explicit Call Ending
+
+- [ ] Owner explicitly ends call
+- [ ] Owner leaves call and product treats this as explicit call end
+- [ ] Active registered participants receive ended state
+- [ ] Active temporary guests receive ended state
+- [ ] Active anonymous guests receive ended state
+- [ ] New joins are blocked after explicit end
+- [ ] Rejoins are blocked after explicit end
+- [ ] Personalized invite links are invalidated after explicit end
+- [ ] Anonymous join links are invalidated after explicit end
+- [ ] Temporary guest accounts are deleted or invalidated after explicit end
+- [ ] Lobby entries are cleared after explicit end
+- [ ] Audit log is preserved after explicit end
+- [ ] Organization admin cannot bypass ended-call state through normal join
+- [ ] System admin cannot bypass ended-call state through normal join unless explicit recovery/debug path exists
+- [ ] Late user opening old link sees safe ended-call state
+
+## 29. Implicit Call Ending by Owner Absence
+
+- [ ] Owner loses connection
+- [ ] Owner closes browser tab
+- [ ] Owner browser crashes or context is killed
+- [ ] Owner network is disconnected
+- [ ] Owner is absent for less than 10 minutes equivalent
+- [ ] Owner is absent for 10 minutes equivalent
+- [ ] Owner is absent for 15 minutes equivalent
+- [ ] Owner rejoins before final 5-minute countdown starts
+- [ ] Owner rejoins during final 5-minute countdown
+- [ ] Owner does not rejoin before timer expires
+- [ ] Call ends automatically after 15 minutes owner absence equivalent
+- [ ] Participants are notified when owner absence timer starts if applicable
+- [ ] Participants see visible countdown during last 5 minutes
+- [ ] Countdown starts when 5 minutes remain
+- [ ] Countdown shows correct remaining time
+- [ ] Countdown updates correctly over time
+- [ ] Countdown survives participant refresh
+- [ ] Countdown is synchronized across participants
+- [ ] Countdown does not reveal admin-only data
+- [ ] Countdown disappears if owner rejoins
+- [ ] Call does not end if owner rejoins before timeout
+- [ ] Call ends if owner does not rejoin before timeout
+- [ ] Call-ended state prevents new joins
+- [ ] Call-ended state prevents rejoins
+- [ ] Call-ended state invalidates anonymous join link
+- [ ] Call-ended state invalidates personalized invite links
+- [ ] Call-ended state deletes or invalidates temporary guest accounts
+- [ ] Call-ended state clears lobby entries
+- [ ] Call-ended state preserves audit log
+- [ ] Call-ended state is visible to late users opening old links
+- [ ] Timer is based on server time, not client time
+- [ ] CI test uses fake/test time and does not wait 15 real minutes
+
+## 30. Error and Edge Cases
+
+- [ ] Call does not exist
+- [ ] Call was deleted
+- [ ] Call was ended
+- [ ] Call has not started yet if time-limited
+- [ ] Call has expired if time-limited
+- [ ] Organization does not exist
+- [ ] Organization is disabled
+- [ ] Host no longer exists
+- [ ] Host is disabled
+- [ ] Invited temporary account was deleted
+- [ ] Registered account was disabled
+- [ ] Registered account was deleted
+- [ ] User email is unconfirmed if relevant
+- [ ] Calendar appointment was cancelled
+- [ ] Calendar appointment was moved
+- [ ] Personalized link belongs to another appointment of same host
+- [ ] Personalized link belongs to another call of same host
+- [ ] Host name differs in capitalization
+- [ ] Host name contains special characters
+- [ ] Host name contains spaces / double names
+- [ ] Host name is ambiguous
+- [ ] Host name changed after invitation
+- [ ] First name / last name of logged-in user is missing
+- [ ] First name / last name of temporary link account is missing
+- [ ] Only first name differs
+- [ ] Only last name differs
+- [ ] Email differs but name matches
+- [ ] Address differs but name matches
+- [ ] Street differs and is treated as possible move
+- [ ] Different street is not displayed
+- [ ] Phone number differs if present
+- [ ] Special characters / umlauts in names are normalized correctly
+- [ ] Different spelling with accents is evaluated according to rule
+- [ ] Leading / trailing spaces in names do not cause false strong mismatch
+- [ ] Empty inputs in update form are validated
+- [ ] Invalid email configuration causes no data leaks
+- [ ] Mail sending failure leaves account unchanged
+- [ ] Database error during join leads to safe abort
+- [ ] Network error during join leads to repeatable state
+- [ ] Timeout during lobby admission leads to consistent state
+
+## 31. Audit and Monitoring
+
+- [ ] Call creation is logged
+- [ ] Invitation creation is logged
+- [ ] Personalized link open is logged
+- [ ] Anonymous link open is logged
+- [ ] Temporary account creation is logged
+- [ ] Temporary account removal is logged
+- [ ] Link-account vs logged-in-account comparison is logged
+- [ ] Strong mismatch is logged
+- [ ] Host-name verification is logged
+- [ ] Successful host-name verification is logged
+- [ ] Failed host-name verification is logged
+- [ ] Account-update request is logged
+- [ ] Confirmation email dispatch is logged
+- [ ] Successful email confirmation is logged
+- [ ] Failed email confirmation is logged
+- [ ] Account-data change is logged
+- [ ] Lobby entry is logged
+- [ ] Lobby admission is logged
+- [ ] Lobby rejection is logged
+- [ ] Call join is logged
+- [ ] Call leave is logged
+- [ ] Rejoin is logged
+- [ ] Kick is logged
+- [ ] Owner transfer is logged
+- [ ] Guest-list change is logged
+- [ ] Review flag for duplicate link is logged
+- [ ] Membership removal is logged
+- [ ] Invite invalidation is logged
+- [ ] Call reschedule is logged
+- [ ] Call deletion is logged
+- [ ] Explicit call end is logged
+- [ ] Implicit call end is logged
+- [ ] Owner absence timer start is logged
+- [ ] Owner absence timer cancellation is logged
+- [ ] Audit logs contain time, actor, target, call, and organization
+- [ ] Audit logs contain no unnecessary sensitive link data
+- [ ] Security-relevant events are visible in monitoring
+- [ ] Failed E2E test artifacts include relevant logs
+
+## 32. End-to-End Main Paths
+
+- [ ] New unregistered guest books appointment through calendar, receives personalized link, opens logged out, lands in lobby, is admitted, joins, leaves, rejoins without approval
+- [ ] Registered but logged-out guest books appointment, opens personalized link logged out, temporary account is used, no automatic account takeover
+- [ ] Registered logged-in guest opens own personalized link with matching data, remains logged in, joins as registered user
+- [ ] Registered logged-in guest opens personalized link with light mismatch, remains logged in, joins after permission check
+- [ ] Registered logged-in user opens foreign personalized link with strong mismatch, sees warning modal, enters wrong host name, receives no foreign data
+- [ ] Registered logged-in user opens foreign personalized link with strong mismatch, enters correct host name, declines data update, remains unchanged
+- [ ] Registered logged-in user opens personalized link with strong mismatch, enters correct host name, re-enters data, confirms email, account is updated
+- [ ] Same personalized link is opened by second logged-in account and review flag is created
+- [ ] Logged-in user opens anonymous link and joins as logged-in user with own rights
+- [ ] Not logged-in user opens anonymous link, temporary account is created, user lands in lobby, is admitted, can rejoin
+- [ ] System admin joins foreign active call without invitation
+- [ ] Organization admin joins own organization active call without invitation
+- [ ] Organization admin cannot join foreign organization call through org-admin rights
+- [ ] Normal user on guest list joins foreign call
+- [ ] Normal user without guest-list entry lands in lobby or is denied
+- [ ] User creates own call, becomes owner, transfers ownership, loses call-admin rights
+- [ ] Organization admin creates call, transfers ownership, keeps admin rights
+- [ ] Temporary user is admitted, then kicked, and cannot rejoin without renewed approval
+- [ ] Invited user is removed from organization before opening link, then joins as call-scoped invited guest
+- [ ] Invite link is invalidated before use and cannot be used
+- [ ] Call is rescheduled and stale link no longer grants stale access
+- [ ] Call is deleted and all temporary access is revoked
+- [ ] Owner explicitly ends call and all participants receive ended state
+- [ ] Owner disconnects, final 5-minute countdown is shown, owner does not return, call ends automatically
+- [ ] Owner disconnects, final 5-minute countdown is shown, owner returns, countdown disappears and call remains active
+
+---
+
+# Named Automated Test Checklist
+
+## Test Group: Organization and Role Fixtures
+
+- [ ] `e2e_org_001_create_organization`
+- [ ] `e2e_org_002_register_user_in_organization`
+- [ ] `e2e_org_003_assign_user_role_user`
+- [ ] `e2e_org_004_assign_user_role_admin`
+- [ ] `e2e_org_005_login_normal_user`
+- [ ] `e2e_org_006_login_organization_admin`
+- [ ] `e2e_org_007_logged_out_user_has_no_session`
+- [ ] `e2e_org_008_cross_org_rights_not_leaked`
+- [ ] `e2e_org_009_stale_client_role_ignored`
+- [ ] `e2e_org_010_stale_session_role_revalidated`
+
+## Test Group: Call Creation and Ownership
+
+- [x] `e2e_owner_001_normal_user_creates_call_and_becomes_owner`
+- [x] `e2e_owner_002_admin_user_creates_call_and_becomes_owner`
+- [ ] `e2e_owner_003_owner_can_manage_guest_list`
+- [ ] `e2e_owner_004_owner_can_admit_lobby_participant`
+- [ ] `e2e_owner_005_owner_can_kick_participant`
+- [ ] `e2e_owner_006_normal_user_transfers_owner_and_loses_admin_rights`
+- [ ] `e2e_owner_007_org_admin_transfers_owner_and_keeps_admin_rights`
+- [ ] `e2e_owner_008_new_owner_receives_owner_and_admin_rights`
+- [ ] `e2e_owner_009_exactly_one_current_owner_after_transfer`
+- [ ] `e2e_owner_010_owner_transfer_to_nonexistent_user_rejected`
+- [ ] `e2e_owner_011_owner_transfer_cross_org_rejected_if_forbidden`
+- [ ] `e2e_owner_012_owner_transfer_audit_logged`
+
+Proof: `call-creation-owner-rights-contract` is the backend/API proof for
+`e2e_owner_001` and `e2e_owner_002`: both registered normal-user and admin-user
+creators create their own call through `/api/calls`, become the persisted owner,
+and receive own-call admin/moderation rights. Owner-transfer scenarios remain
+unchecked for the separate owner-transfer lane.
+
+## Test Group: Direct Join Permissions
+
+- [ ] `e2e_join_001_system_admin_can_join_any_active_call`
+- [ ] `e2e_join_002_system_admin_joins_without_guest_list`
+- [ ] `e2e_join_003_org_admin_can_join_own_org_call`
+- [ ] `e2e_join_004_org_admin_cannot_join_foreign_org_call`
+- [ ] `e2e_join_005_guest_list_user_can_join`
+- [ ] `e2e_join_006_user_not_on_guest_list_cannot_direct_join`
+- [ ] `e2e_join_007_owner_can_join_own_call`
+- [ ] `e2e_join_008_disabled_user_cannot_join`
+- [ ] `e2e_join_009_removed_guest_list_entry_revokes_join`
+- [ ] `e2e_join_010_added_guest_list_entry_grants_join`
+- [ ] `e2e_join_011_manipulated_role_rejected`
+- [ ] `e2e_join_012_manipulated_call_id_rejected`
+
+## Test Group: Calendar Invitation
+
+- [ ] `e2e_invite_001_host_creates_calendar_invitation`
+- [ ] `e2e_invite_002_invitee_selects_appointment`
+- [ ] `e2e_invite_003_registered_logged_in_invitee_flow`
+- [ ] `e2e_invite_004_registered_logged_out_invitee_flow`
+- [ ] `e2e_invite_005_unregistered_invitee_creates_temp_account`
+- [ ] `e2e_invite_006_personalized_link_bound_to_temp_account`
+- [ ] `e2e_invite_007_multiple_invitees_get_unique_links`
+- [ ] `e2e_invite_008_cancel_invitation_invalidates_link`
+- [ ] `e2e_invite_009_expired_personalized_link_rejected`
+- [ ] `e2e_invite_010_reopen_same_link_same_context_consistent`
+
+Proof: `call-access-verified-context-ui-contract` pins the focused Playwright
+case `same personalized link in parallel contexts keeps account sessions
+isolated` in `call-access-join.spec.js`. It opens the same personalized link in
+two isolated browser contexts with different authenticated sessions, submits the
+session requests in parallel, proves each POST carries its own bearer plus
+verified user/session snapshot, keeps localStorage/session state isolated after
+one success and one conflict, renders no foreign response data, and guards
+against duplicate join/session request loops.
+
+## Test Group: Personalized Link Logged Out
+
+- [ ] `e2e_personalized_logged_out_001_temp_account_created_from_link`
+- [ ] `e2e_personalized_logged_out_002_existing_account_not_auto_logged_in`
+- [ ] `e2e_personalized_logged_out_003_temp_guest_on_guest_list_direct_join`
+- [ ] `e2e_personalized_logged_out_004_temp_guest_not_on_guest_list_lobby`
+- [ ] `e2e_personalized_logged_out_005_temp_guest_no_registered_rights`
+- [ ] `e2e_personalized_logged_out_006_temp_guest_no_org_rights`
+- [ ] `e2e_personalized_logged_out_007_manipulated_link_rejected`
+- [ ] `e2e_personalized_logged_out_008_invalid_link_safe_error`
+
+## Test Group: Personalized Link Logged In Without Strong Mismatch
+
+- [ ] `e2e_personalized_logged_in_001_logged_in_session_preserved`
+- [ ] `e2e_personalized_logged_in_002_temp_account_does_not_replace_session`
+- [ ] `e2e_personalized_logged_in_003_matching_data_no_warning_modal`
+- [ ] `e2e_personalized_logged_in_004_light_mismatch_no_foreign_link_warning`
+- [ ] `e2e_personalized_logged_in_005_logged_in_account_used_for_permission_check`
+- [ ] `e2e_personalized_logged_in_006_no_auto_account_data_overwrite`
+- [ ] `e2e_personalized_logged_in_007_no_link_data_exposed`
+- [ ] `e2e_personalized_logged_in_008_same_account_reopen_no_duplicate_flag`
+
+## Test Group: Personalized Link Strong Mismatch
+
+- [ ] `e2e_strong_mismatch_001_first_name_mismatch_detected`
+- [ ] `e2e_strong_mismatch_002_last_name_mismatch_detected`
+- [ ] `e2e_strong_mismatch_003_full_name_mismatch_detected`
+- [ ] `e2e_strong_mismatch_004_warning_modal_displayed`
+- [ ] `e2e_strong_mismatch_005_foreign_link_data_not_displayed`
+- [ ] `e2e_strong_mismatch_006_wrong_host_name_no_access`
+- [ ] `e2e_strong_mismatch_007_wrong_host_name_no_data_leak`
+- [ ] `e2e_strong_mismatch_008_correct_host_name_accepted`
+- [ ] `e2e_strong_mismatch_009_decline_account_update_keeps_account_unchanged`
+- [ ] `e2e_strong_mismatch_010_update_requires_manual_reentry`
+- [ ] `e2e_strong_mismatch_011_confirmation_email_sent_to_logged_in_account`
+- [ ] `e2e_strong_mismatch_012_email_not_sent_to_temp_account`
+- [ ] `e2e_strong_mismatch_013_no_update_without_email_confirmation`
+- [ ] `e2e_strong_mismatch_014_confirmed_update_changes_only_confirmed_fields`
+- [ ] `e2e_strong_mismatch_015_rate_limit_host_name_attempts`
+- [ ] `e2e_strong_mismatch_016_audit_logged`
+
+## Test Group: Duplicate Personalized Link
+
+- [ ] `e2e_duplicate_link_001_same_account_reuses_link_no_flag`
+- [ ] `e2e_duplicate_link_002_second_account_uses_link_flag_created`
+- [ ] `e2e_duplicate_link_003_second_account_flag_even_without_join`
+- [ ] `e2e_duplicate_link_004_second_account_flag_even_with_correct_host_name`
+- [ ] `e2e_duplicate_link_005_concurrent_two_accounts_same_link_detected`
+- [ ] `e2e_duplicate_link_006_parallel_open_no_inconsistent_assignment`
+- [ ] `e2e_duplicate_link_007_cross_device_duplicate_detected`
+- [ ] `e2e_duplicate_link_008_cross_browser_duplicate_detected`
+- [ ] `e2e_duplicate_link_009_review_flag_contains_required_metadata`
+- [ ] `e2e_duplicate_link_010_review_flag_avoids_sensitive_data`
+
+## Test Group: Anonymous Link Logged In
+
+- [ ] `e2e_anon_logged_in_001_logged_in_user_uses_own_account`
+- [ ] `e2e_anon_logged_in_002_temp_account_discarded`
+- [ ] `e2e_anon_logged_in_003_logged_in_rights_used`
+- [ ] `e2e_anon_logged_in_004_system_admin_can_join_active_call`
+- [ ] `e2e_anon_logged_in_005_org_admin_can_join_own_org_call`
+- [ ] `e2e_anon_logged_in_006_org_admin_cannot_join_foreign_org_call`
+- [ ] `e2e_anon_logged_in_007_guest_list_user_can_join`
+- [ ] `e2e_anon_logged_in_008_non_guest_user_lands_in_lobby`
+- [ ] `e2e_anon_logged_in_009_anonymous_link_does_not_overwrite_account`
+- [ ] `e2e_anon_logged_in_010_invalid_anonymous_link_rejected`
+
+## Test Group: Anonymous Link Logged Out
+
+- [ ] `e2e_anon_logged_out_001_temp_anonymous_account_created`
+- [ ] `e2e_anon_logged_out_002_temp_anonymous_user_lands_in_lobby`
+- [ ] `e2e_anon_logged_out_003_temp_anonymous_has_no_registered_rights`
+- [ ] `e2e_anon_logged_out_004_host_can_admit_anonymous_guest`
+- [ ] `e2e_anon_logged_out_005_temp_moderator_can_admit_anonymous_guest`
+- [ ] `e2e_anon_logged_out_006_admin_can_admit_anonymous_guest`
+- [ ] `e2e_anon_logged_out_007_unauthorized_user_cannot_admit_guest`
+- [ ] `e2e_anon_logged_out_008_admitted_guest_can_rejoin`
+- [ ] `e2e_anon_logged_out_009_kicked_guest_cannot_direct_rejoin`
+- [ ] `e2e_anon_logged_out_010_multiple_anonymous_guests_are_separate`
+
+## Test Group: Lobby
+
+- [ ] `e2e_lobby_001_unauthorized_user_lands_in_lobby`
+- [ ] `e2e_lobby_002_host_sees_waiting_participant`
+- [ ] `e2e_lobby_003_temp_moderator_sees_waiting_participant`
+- [ ] `e2e_lobby_004_org_admin_sees_waiting_participant_for_own_org`
+- [ ] `e2e_lobby_005_unauthorized_user_no_lobby_controls`
+- [ ] `e2e_lobby_006_host_admits_participant`
+- [ ] `e2e_lobby_007_host_rejects_participant`
+- [ ] `e2e_lobby_008_rejected_participant_cannot_enter`
+- [ ] `e2e_lobby_009_admission_is_call_scoped`
+- [ ] `e2e_lobby_010_concurrent_admission_idempotent`
+- [ ] `e2e_lobby_011_concurrent_admit_reject_deterministic`
+- [ ] `e2e_lobby_012_lobby_state_updates_correctly`
+
+## Test Group: Rejoin and Kick
+
+- [ ] `e2e_rejoin_001_admitted_temp_user_can_rejoin`
+- [ ] `e2e_rejoin_002_rejoin_after_refresh`
+- [ ] `e2e_rejoin_003_rejoin_after_network_interruption`
+- [ ] `e2e_rejoin_004_kicked_temp_user_cannot_direct_rejoin`
+- [ ] `e2e_rejoin_005_kick_overrides_previous_admission`
+- [ ] `e2e_rejoin_006_registered_guest_can_rejoin`
+- [ ] `e2e_rejoin_007_rejoin_after_guest_list_removal_blocked_or_lobby`
+- [ ] `e2e_rejoin_008_rejoin_after_admin_role_removed_uses_new_permissions`
+- [ ] `e2e_rejoin_009_rejoin_after_owner_transfer_uses_new_permissions`
+
+## Test Group: Temporary Moderators
+
+- [ ] `e2e_temp_mod_001_host_assigns_temp_moderator`
+- [ ] `e2e_temp_mod_002_temp_moderator_admits_participant`
+- [ ] `e2e_temp_mod_003_temp_moderator_rejects_participant`
+- [ ] `e2e_temp_mod_004_temp_moderator_limited_to_assigned_call`
+- [ ] `e2e_temp_mod_005_temp_moderator_no_org_admin_actions`
+- [ ] `e2e_temp_mod_006_temp_moderator_rights_revoked_immediately`
+- [ ] `e2e_temp_mod_007_client_side_temp_mod_role_forgery_rejected`
+
+## Test Group: Privacy and Security
+
+- [ ] `e2e_privacy_001_foreign_link_data_not_rendered`
+- [ ] `e2e_privacy_002_foreign_link_data_not_in_api_response`
+- [ ] `e2e_privacy_003_invalid_link_no_personal_data_leak`
+- [ ] `e2e_privacy_004_wrong_host_name_no_personal_data_leak`
+- [ ] `e2e_privacy_005_browser_network_response_no_foreign_data`
+- [ ] `e2e_privacy_006_audit_logs_minimize_sensitive_data`
+- [ ] `e2e_security_001_modified_personalized_link_id_rejected`
+- [ ] `e2e_security_002_modified_call_id_rejected`
+- [ ] `e2e_security_003_forged_user_id_rejected`
+- [ ] `e2e_security_004_forged_role_rejected`
+- [ ] `e2e_security_005_forged_org_id_rejected`
+- [ ] `e2e_security_006_csrf_account_update_protected`
+- [ ] `e2e_security_007_session_fixation_prevented`
+- [ ] `e2e_security_008_parallel_tabs_no_wrong_merge`
+
+## Test Group: Email Confirmation
+
+- [ ] `e2e_email_001_update_request_sends_confirmation_email`
+- [ ] `e2e_email_002_email_sent_to_logged_in_account`
+- [ ] `e2e_email_003_email_not_sent_to_temp_account`
+- [ ] `e2e_email_004_confirmation_link_account_bound`
+- [ ] `e2e_email_005_confirmation_link_one_time_use`
+- [ ] `e2e_email_006_confirmation_link_time_limited`
+- [ ] `e2e_email_007_no_update_without_confirmation`
+- [ ] `e2e_email_008_confirmation_updates_only_reentered_data`
+- [ ] `e2e_email_009_expired_confirmation_link_no_update`
+- [ ] `e2e_email_010_multiple_pending_confirmations_resolved`
+
+## Test Group: Guest List
+
+- [ ] `e2e_guest_list_001_add_registered_user_to_guest_list`
+- [ ] `e2e_guest_list_002_add_temp_account_to_guest_list`
+- [ ] `e2e_guest_list_003_remove_guest_list_entry`
+- [ ] `e2e_guest_list_004_guest_list_user_direct_join`
+- [ ] `e2e_guest_list_005_non_guest_user_no_direct_join`
+- [ ] `e2e_guest_list_006_temp_guest_list_user_direct_join`
+- [ ] `e2e_guest_list_007_guest_list_call_scoped`
+- [ ] `e2e_guest_list_008_guest_list_cross_org_not_valid`
+- [ ] `e2e_guest_list_009_duplicate_guest_entries_handled`
+- [ ] `e2e_guest_list_010_guest_list_changes_audit_logged`
+
+## Test Group: Cross Organization
+
+- [ ] `e2e_cross_org_001_user_a_opens_org_a_link`
+- [ ] `e2e_cross_org_002_user_a_opens_org_b_link`
+- [ ] `e2e_cross_org_003_org_admin_a_opens_org_a_call`
+- [ ] `e2e_cross_org_004_org_admin_a_opens_org_b_call`
+- [ ] `e2e_cross_org_005_org_admin_a_no_admin_rights_in_org_b`
+- [ ] `e2e_cross_org_006_active_org_switch_does_not_change_server_permission`
+- [ ] `e2e_cross_org_007_guest_list_not_cross_org`
+- [ ] `e2e_cross_org_008_owner_rights_not_cross_org`
+- [ ] `e2e_cross_org_009_review_flags_correct_org`
+
+## Test Group: Multi Session and Device
+
+- [ ] `e2e_multi_session_001_same_user_same_link_two_browsers`
+- [ ] `e2e_multi_session_002_same_user_same_link_two_devices`
+- [ ] `e2e_multi_session_003_different_user_same_link_other_device_flags`
+- [ ] `e2e_multi_session_004_concurrent_join_no_duplicate_participants`
+- [ ] `e2e_multi_session_005_login_switch_during_warning_modal_safe`
+- [ ] `e2e_multi_session_006_email_confirmation_other_browser_correct_account`
+- [ ] `e2e_multi_session_007_session_expiry_in_lobby_safe`
+- [ ] `e2e_multi_session_008_session_expiry_in_call_safe`
+- [ ] `e2e_multi_session_009_refresh_during_host_verification_safe`
+- [ ] `e2e_multi_session_010_refresh_during_pending_email_confirmation_safe`
+
+## Test Group: Membership Revocation After Invitation
+
+- [ ] `e2e_membership_001_removed_invited_user_can_use_valid_invite_as_call_guest`
+- [ ] `e2e_membership_002_removed_invited_user_no_org_member_rights`
+- [ ] `e2e_membership_003_removed_invited_admin_no_org_admin_rights`
+- [ ] `e2e_membership_004_removed_invited_user_cannot_join_other_org_calls`
+- [ ] `e2e_membership_005_removed_invited_user_cannot_access_org_resources`
+- [ ] `e2e_membership_006_removed_invited_user_blocked_after_invite_invalidation`
+- [ ] `e2e_membership_007_removed_invited_user_blocked_after_call_deleted`
+- [ ] `e2e_membership_008_removed_invited_user_blocked_after_call_ended`
+- [ ] `e2e_membership_009_removed_invited_user_kick_overrides_invite`
+- [ ] `e2e_membership_010_user_inside_call_remains_if_call_scoped_access`
+- [ ] `e2e_membership_011_user_inside_call_loses_org_privileges_immediately`
+- [ ] `e2e_membership_012_removed_org_admin_inside_call_loses_admin_controls`
+- [ ] `e2e_membership_013_removed_user_rejoin_allowed_only_while_invite_valid`
+- [ ] `e2e_membership_014_membership_removal_audit_logged`
+- [ ] `e2e_membership_015_stale_role_cache_ignored_after_membership_removal`
+
+## Test Group: Invite Invalidation
+
+- [ ] `e2e_invite_invalid_001_personalized_link_invalidated_before_use`
+- [ ] `e2e_invite_invalid_002_personalized_link_invalidated_after_first_use`
+- [ ] `e2e_invite_invalid_003_personalized_link_invalidated_in_lobby`
+- [ ] `e2e_invite_invalid_004_personalized_link_invalidated_in_call`
+- [ ] `e2e_invite_invalid_005_anonymous_link_invalidated_before_use`
+- [ ] `e2e_invite_invalid_006_anonymous_link_invalidated_in_lobby`
+- [ ] `e2e_invite_invalid_007_anonymous_link_invalidated_in_call`
+- [ ] `e2e_invite_invalid_008_invalidated_link_blocks_fresh_join`
+- [ ] `e2e_invite_invalid_009_invalidated_link_blocks_rejoin_if_required`
+- [ ] `e2e_invite_invalid_010_invalidated_link_no_data_leak`
+- [ ] `e2e_invite_invalid_011_invalidated_link_does_not_recreate_temp_account`
+- [ ] `e2e_invite_invalid_012_invalidated_link_survives_app_restart`
+
+## Test Group: Guest Account Lifecycle
+
+- [ ] `e2e_guest_lifecycle_001_temp_guest_created_from_calendar_invite`
+- [ ] `e2e_guest_lifecycle_002_temp_guest_deleted_when_call_deleted`
+- [ ] `e2e_guest_lifecycle_003_temp_guest_deleted_when_invitation_deleted`
+- [ ] `e2e_guest_lifecycle_004_temp_guest_invalidated_when_link_invalidated`
+- [ ] `e2e_guest_lifecycle_005_temp_guest_handled_on_call_reschedule`
+- [ ] `e2e_guest_lifecycle_006_temp_guest_cannot_join_after_call_deleted`
+- [ ] `e2e_guest_lifecycle_007_temp_guest_cannot_join_after_call_ended`
+- [ ] `e2e_guest_lifecycle_008_temp_guest_cannot_rejoin_after_cleanup`
+- [ ] `e2e_guest_lifecycle_009_cleanup_does_not_delete_registered_accounts`
+- [ ] `e2e_guest_lifecycle_010_cleanup_scoped_to_call`
+- [ ] `e2e_guest_lifecycle_011_cleanup_idempotent`
+- [ ] `e2e_guest_lifecycle_012_cleanup_audit_logged`
+
+## Test Group: Call Rescheduling
+
+- [ ] `e2e_reschedule_001_owner_reschedules_before_guest_opens_link`
+- [ ] `e2e_reschedule_002_owner_reschedules_while_guest_in_lobby`
+- [ ] `e2e_reschedule_003_owner_reschedules_while_guest_in_call`
+- [ ] `e2e_reschedule_004_old_personalized_link_invalidated`
+- [ ] `e2e_reschedule_005_new_personalized_link_works`
+- [ ] `e2e_reschedule_006_old_temp_guest_handled_by_product_rule`
+- [ ] `e2e_reschedule_007_old_link_cannot_join_stale_call`
+- [ ] `e2e_reschedule_008_registered_invitee_old_link_behavior`
+- [ ] `e2e_reschedule_009_anonymous_link_behavior_after_reschedule`
+- [ ] `e2e_reschedule_010_lobby_entries_migrated_or_cleared`
+- [ ] `e2e_reschedule_011_admitted_temp_participants_migrated_or_cleared`
+- [ ] `e2e_reschedule_012_reschedule_audit_logged`
+
+## Test Group: Call Deletion
+
+- [ ] `e2e_delete_001_owner_deletes_call_before_guests_join`
+- [ ] `e2e_delete_002_owner_deletes_call_with_guests_in_lobby`
+- [ ] `e2e_delete_003_owner_deletes_call_with_registered_users_inside`
+- [ ] `e2e_delete_004_owner_deletes_call_with_temp_guests_inside`
+- [ ] `e2e_delete_005_owner_deletes_call_with_anonymous_guests_inside`
+- [ ] `e2e_delete_006_deleted_call_blocks_owner_join`
+- [ ] `e2e_delete_007_deleted_call_blocks_org_admin_join`
+- [ ] `e2e_delete_008_deleted_call_blocks_system_admin_normal_join`
+- [ ] `e2e_delete_009_deleted_call_blocks_personalized_link`
+- [ ] `e2e_delete_010_deleted_call_blocks_anonymous_link`
+- [ ] `e2e_delete_011_deleted_call_blocks_admitted_guest_rejoin`
+- [ ] `e2e_delete_012_deleted_call_cleans_temp_guests`
+- [ ] `e2e_delete_013_deleted_call_clears_lobby`
+- [ ] `e2e_delete_014_deleted_call_preserves_audit_log`
+- [ ] `e2e_delete_015_deleted_call_does_not_affect_unrelated_calls`
+
+## Test Group: Explicit Call End
+
+- [ ] `e2e_end_explicit_001_owner_explicitly_ends_call`
+- [ ] `e2e_end_explicit_002_owner_leave_treated_as_call_end_if_configured`
+- [ ] `e2e_end_explicit_003_registered_participants_receive_ended_state`
+- [ ] `e2e_end_explicit_004_temp_guests_receive_ended_state`
+- [ ] `e2e_end_explicit_005_anonymous_guests_receive_ended_state`
+- [ ] `e2e_end_explicit_006_new_joins_blocked_after_end`
+- [ ] `e2e_end_explicit_007_rejoins_blocked_after_end`
+- [ ] `e2e_end_explicit_008_personalized_links_invalidated_after_end`
+- [ ] `e2e_end_explicit_009_anonymous_links_invalidated_after_end`
+- [ ] `e2e_end_explicit_010_temp_guests_cleaned_after_end`
+- [ ] `e2e_end_explicit_011_lobby_cleared_after_end`
+- [ ] `e2e_end_explicit_012_late_old_link_shows_safe_ended_state`
+
+## Test Group: Implicit Call End by Owner Absence
+
+- [ ] `e2e_end_implicit_001_owner_disconnect_starts_absence_timer`
+- [ ] `e2e_end_implicit_002_owner_tab_close_starts_absence_timer`
+- [ ] `e2e_end_implicit_003_owner_process_kill_starts_absence_timer`
+- [ ] `e2e_end_implicit_004_owner_network_loss_starts_absence_timer`
+- [ ] `e2e_end_implicit_005_no_countdown_before_10_min_equivalent`
+- [ ] `e2e_end_implicit_006_countdown_visible_at_10_min_equivalent`
+- [ ] `e2e_end_implicit_007_countdown_updates_over_time`
+- [ ] `e2e_end_implicit_008_countdown_synchronized_across_participants`
+- [ ] `e2e_end_implicit_009_countdown_survives_participant_refresh`
+- [ ] `e2e_end_implicit_010_owner_rejoin_before_countdown_cancels_timer`
+- [ ] `e2e_end_implicit_011_owner_rejoin_during_countdown_cancels_timer`
+- [ ] `e2e_end_implicit_012_owner_absent_15_min_equivalent_ends_call`
+- [ ] `e2e_end_implicit_013_automatic_end_notifies_participants`
+- [ ] `e2e_end_implicit_014_automatic_end_blocks_new_joins`
+- [ ] `e2e_end_implicit_015_automatic_end_blocks_rejoins`
+- [ ] `e2e_end_implicit_016_automatic_end_invalidates_links`
+- [ ] `e2e_end_implicit_017_automatic_end_cleans_guest_accounts`
+- [ ] `e2e_end_implicit_018_timer_uses_server_time`
+- [ ] `e2e_end_implicit_019_ci_uses_test_clock_no_real_15_min_sleep`
+
+## Test Group: King Containers
+
+- [ ] `e2e_king_001_king_can_join_as_owner`
+- [ ] `e2e_king_002_king_can_join_as_registered_user`
+- [ ] `e2e_king_003_king_can_join_as_personalized_guest`
+- [ ] `e2e_king_004_king_can_join_as_anonymous_guest`
+- [ ] `e2e_king_005_king_streams_deterministic_dummy_media`
+- [ ] `e2e_king_006_king_disconnects_gracefully`
+- [ ] `e2e_king_007_king_simulates_abrupt_disconnect`
+- [ ] `e2e_king_008_king_simulates_network_loss`
+- [ ] `e2e_king_009_king_reconnects_same_identity`
+- [ ] `e2e_king_010_king_exposes_call_state`
+- [ ] `e2e_king_011_king_exposes_countdown_state`
+- [ ] `e2e_king_012_king_logs_are_collected_on_failure`
+- [ ] `e2e_king_013_multiple_king_containers_join_same_call`
+- [ ] `e2e_king_014_king_containers_terminate_cleanly`
+
+## Test Group: Audit and Monitoring
+
+- [ ] `e2e_audit_001_call_creation_logged`
+- [ ] `e2e_audit_002_invitation_creation_logged`
+- [ ] `e2e_audit_003_link_open_logged`
+- [ ] `e2e_audit_004_temp_account_creation_logged`
+- [ ] `e2e_audit_005_strong_mismatch_logged`
+- [ ] `e2e_audit_006_host_verification_logged`
+- [ ] `e2e_audit_007_account_update_logged`
+- [ ] `e2e_audit_008_lobby_events_logged`
+- [ ] `e2e_audit_009_join_leave_rejoin_logged`
+- [ ] `e2e_audit_010_kick_logged`
+- [ ] `e2e_audit_011_owner_transfer_logged`
+- [ ] `e2e_audit_012_duplicate_link_flag_logged`
+- [ ] `e2e_audit_013_membership_removal_logged`
+- [ ] `e2e_audit_014_invite_invalidation_logged`
+- [ ] `e2e_audit_015_reschedule_logged`
+- [ ] `e2e_audit_016_delete_logged`
+- [ ] `e2e_audit_017_explicit_end_logged`
+- [ ] `e2e_audit_018_implicit_end_logged`
+- [ ] `e2e_audit_019_owner_absence_timer_logged`
+- [ ] `e2e_audit_020_logs_minimize_sensitive_data`
+
+## Test Group: Main E2E Journeys
+
+- [ ] `e2e_journey_001_unregistered_calendar_guest_lobby_admit_join_leave_rejoin`
+- [ ] `e2e_journey_002_registered_logged_out_invitee_uses_temp_account`
+- [ ] `e2e_journey_003_registered_logged_in_matching_invitee_joins_as_account`
+- [ ] `e2e_journey_004_registered_logged_in_light_mismatch_joins_after_permission_check`
+- [ ] `e2e_journey_005_foreign_personalized_link_wrong_host_no_data_leak`
+- [ ] `e2e_journey_006_foreign_personalized_link_correct_host_decline_update`
+- [ ] `e2e_journey_007_foreign_personalized_link_correct_host_update_confirm_email`
+- [ ] `e2e_journey_008_duplicate_personalized_link_review_flag`
+- [ ] `e2e_journey_009_logged_in_user_anonymous_link_uses_own_rights`
+- [ ] `e2e_journey_010_logged_out_user_anonymous_link_lobby_admit_rejoin`
+- [ ] `e2e_journey_011_system_admin_join_without_invite`
+- [ ] `e2e_journey_012_org_admin_join_own_org_without_invite`
+- [ ] `e2e_journey_013_org_admin_foreign_org_denied`
+- [ ] `e2e_journey_014_normal_guest_list_user_joins_foreign_call`
+- [ ] `e2e_journey_015_normal_non_guest_user_lobby_or_denied`
+- [ ] `e2e_journey_016_normal_user_owner_transfer_loses_admin`
+- [ ] `e2e_journey_017_org_admin_owner_transfer_keeps_admin`
+- [ ] `e2e_journey_018_temp_user_kicked_cannot_rejoin_directly`
+- [ ] `e2e_journey_019_removed_org_member_invite_becomes_call_scoped_guest`
+- [ ] `e2e_journey_020_invalidated_invite_link_denied`
+- [ ] `e2e_journey_021_rescheduled_call_old_link_invalid_new_link_valid`
+- [ ] `e2e_journey_022_deleted_call_revokes_all_temp_access`
+- [ ] `e2e_journey_023_explicit_call_end_revokes_all_join_paths`
+- [ ] `e2e_journey_024_owner_absence_countdown_then_auto_end`
+- [ ] `e2e_journey_025_owner_absence_countdown_then_reconnect_cancels_end`
+
+---
+
+# Definition of Done
+
+- [x] E2E test suite is implemented or extended
+- [x] Playwright or existing E2E framework is configured for CI
+- [ ] CI job runs E2E suite automatically
+- [x] CI starts all required services
+- [x] CI starts media/signaling infrastructure if required
+- [ ] CI starts `king` containers for multi-participant tests
+- [ ] CI collects traces, screenshots, videos, and logs on failure
+- [x] Test data is deterministic
+- [x] Test data is isolated per test or safely reset
+- [ ] Tests cover all critical IAM and call-access flows
+- [ ] Tests cover invitation invalidation
+- [ ] Tests cover guest account cleanup
+- [ ] Tests cover call rescheduling
+- [ ] Tests cover call deletion
+- [ ] Tests cover explicit call ending
+- [ ] Tests cover implicit owner-absence ending
+- [ ] Owner timeout tests do not wait 15 real minutes
+- [ ] Duplicate personalized-link abuse detection is tested
+- [ ] Membership removal after invitation is tested with call-scoped guest access behavior
+- [x] Privacy and data minimization assertions are included
+- [ ] Security manipulation cases are covered
+- [ ] Audit-relevant flows are asserted where audit logs exist
+- [x] Test names are stable and mapped to the checklist
+- [ ] Documentation explains how to run tests locally
+- [ ] Documentation explains how to run tests in CI
+
+Proof: `iam-call-access-seeding.matrix.json` adds deterministic IAM/call-access
+E2E seed principals, calls, access links, and scenario IDs without replacing the
+live backend `call-access-join.spec.js`. `call-access-seed-matrix.spec.js`
+exercises the public join/session browser path against the deterministic matrix,
+checks temporary guests are not elevated to tenant/platform/system admin, and
+asserts the call-access session payload contains no SDP, ICE, media token, or
+TURN credential material.
+
+Merged CI-smoke proof splits `test:e2e:call-access` from the broader chat/layout
+matrix, keeps the live backend call-access spec plus seed-matrix spec in a
+serial `--workers=1` Playwright script, and runs the compose smoke against
+backend/ws/sfu service-DNS origins. Integrated reruns passed
+`npm run test:e2e:call-access`, `npm run test:e2e:matrix`,
+`npm run test:e2e:release-gate`, and `npm run test:contract:iam-call-access`
+with host-PHP `pdo_sqlite` unavailable.
diff --git a/demo/call-app/whiteboard/public/index.html b/demo/call-app/whiteboard/public/index.html
index 38989fa73..3dbcfd1bc 100644
--- a/demo/call-app/whiteboard/public/index.html
+++ b/demo/call-app/whiteboard/public/index.html
@@ -47,6 +47,7 @@
+
normalizeRoomId(routeCallResolve.roomId ||
const activeRoomId = computed(() => normalizeRoomId(serverRoomId.value || desiredRoomId.value));
const activeSocketCallId = computed(() => normalizeSocketCallId(activeCallId.value || routeCallResolve.callId || ''));
const currentUserId = computed(() => (Number.isInteger(sessionState.userId) ? sessionState.userId : 0));
+const backgroundStaticAvatarRender = createBackgroundStaticAvatarRenderState({ callMediaPrefs, currentUserId, peerControlStateByUserId });
const showAdmissionGate = computed(() => {
const gateRoomId = normalizeOptionalRoomId(admissionGateState.roomId);
return gateRoomId !== '' && activeRoomId.value !== gateRoomId;
@@ -1041,6 +1044,7 @@ const mediaStack = createCallWorkspaceMediaStack({
ensureMediaSecuritySession,
evaluateBackgroundFilterGates,
hasRenderableMediaForParticipant,
+ hasStaticAvatarForUserId: backgroundStaticAvatarRender.hasStaticAvatarForUserId,
hintMediaSecuritySync,
isWlvcRuntimePath: (...args) => isWlvcRuntimePath(...args),
lookupMediaNodeForUserId,
@@ -1085,6 +1089,8 @@ const mediaStack = createCallWorkspaceMediaStack({
shouldRecoverMediaSecurityFromFrameError,
shouldSendTransportOnlySfuFrame,
shouldSyncNativeLocalTracksBeforeOffer: (...args) => shouldSyncNativeLocalTracksBeforeOffer(...args),
+ staticAvatarNodeForUserId: backgroundStaticAvatarRender.staticAvatarNodeForUserId,
+ syncControlStateToPeers: (...args) => (typeof syncControlStateToPeers === 'function' ? syncControlStateToPeers(...args) : 0),
stopActivityMonitor: (...args) => stopActivityMonitor(...args),
stopSfuTrackAnnounceTimer,
syncNativePeerConnectionsWithRoster: (...args) => syncNativePeerConnectionsWithRoster(...args),
@@ -1672,6 +1678,7 @@ const participantUiHelpers = createCallWorkspaceParticipantUiHelpers({
aloneIdlePrompt,
apiRequest,
callLayoutState,
+ callMediaPrefs,
callParticipantRoles,
canManageOwnerRole,
canModerate,
@@ -1695,6 +1702,7 @@ const participantUiHelpers = createCallWorkspaceParticipantUiHelpers({
isCompactLayoutViewport,
isCompactMiniStripAbove,
isSocketOnline,
+ hasStaticAvatarForUserId: backgroundStaticAvatarRender.hasStaticAvatarForUserId,
isShellMobileViewport,
layoutModeOptionsFor,
layoutStrategyOptionsFor,
@@ -1949,9 +1957,7 @@ watch(
{ immediate: true },
);
-onBeforeUnmount(() => {
- clearCallAppSidebarControls();
-});
+onBeforeUnmount(() => { backgroundStaticAvatarRender.clearStaticAvatarNodes(); clearCallAppSidebarControls(); });
const chatRuntimeHelpers = createCallWorkspaceChatRuntimeHelpers({
activeCallId,
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/background/BackgroundReplacementUnavailableModal.vue b/demo/video-chat/frontend-vue/src/domain/realtime/background/BackgroundReplacementUnavailableModal.vue
new file mode 100644
index 000000000..184df4f0e
--- /dev/null
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/background/BackgroundReplacementUnavailableModal.vue
@@ -0,0 +1,201 @@
+
+
+
+
+
+
+
+ {{ failureLabel }}
+ {{ statusMessage }}
+
+
+
+ {{ t('calls.workspace.background_use_standard_avatar') }}
+
+
+ {{ t('calls.workspace.background_upload_avatar') }}
+
+
+
+ {{ t('calls.workspace.background_send_unfiltered') }}
+
+
+
+
+
+
+
+
+
+
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/background/avatarFallbackSignal.ts b/demo/video-chat/frontend-vue/src/domain/realtime/background/avatarFallbackSignal.ts
new file mode 100644
index 000000000..f03284bc7
--- /dev/null
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/background/avatarFallbackSignal.ts
@@ -0,0 +1,41 @@
+import { DEFAULT_BACKGROUND_FALLBACK_AVATAR_URL } from '../media/preferences';
+
+export const BACKGROUND_FALLBACK_AVATAR_MODE = 'avatar';
+export const BACKGROUND_FALLBACK_NONE_MODE = 'none';
+
+export function normalizeBackgroundFallbackAvatarUrl(value = '') {
+ const normalized = String(value || '').trim();
+ return normalized === '' ? DEFAULT_BACKGROUND_FALLBACK_AVATAR_URL : normalized;
+}
+
+export function normalizeBackgroundFallbackMode(value = '') {
+ return String(value || '').trim().toLowerCase() === BACKGROUND_FALLBACK_AVATAR_MODE
+ ? BACKGROUND_FALLBACK_AVATAR_MODE
+ : BACKGROUND_FALLBACK_NONE_MODE;
+}
+
+export function createBackgroundFallbackAudioOnlyStream(sourceStream) {
+ if (typeof MediaStream === 'undefined') return sourceStream;
+ const out = new MediaStream();
+ if (!(sourceStream instanceof MediaStream)) return out;
+
+ for (const audioTrack of sourceStream.getAudioTracks()) {
+ if (audioTrack?.readyState === 'ended') continue;
+ out.addTrack(audioTrack);
+ }
+
+ return out;
+}
+
+export function backgroundFallbackControlStateFromPrefs(callMediaPrefs) {
+ const mode = normalizeBackgroundFallbackMode(callMediaPrefs?.backgroundFallbackVideoMode);
+ const avatarUrl = mode === BACKGROUND_FALLBACK_AVATAR_MODE
+ ? normalizeBackgroundFallbackAvatarUrl(callMediaPrefs?.backgroundFallbackAvatarImageUrl)
+ : '';
+
+ return {
+ backgroundFallbackVideoMode: mode,
+ backgroundFallbackAvatarImageUrl: avatarUrl,
+ videoSubstitution: mode === BACKGROUND_FALLBACK_AVATAR_MODE ? BACKGROUND_FALLBACK_AVATAR_MODE : '',
+ };
+}
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/background/backendSelector.ts b/demo/video-chat/frontend-vue/src/domain/realtime/background/backendSelector.ts
deleted file mode 100644
index c857e0c21..000000000
--- a/demo/video-chat/frontend-vue/src/domain/realtime/background/backendSelector.ts
+++ /dev/null
@@ -1,15 +0,0 @@
-export function selectBackgroundFilterBackend() {
- if (typeof window === 'undefined') {
- return {
- backend: 'unsupported',
- supported: false,
- reason: 'no_window',
- };
- }
-
- return {
- backend: 'sinet_wasm',
- supported: true,
- reason: 'ok',
- };
-}
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/background/backendSinetWasm.js b/demo/video-chat/frontend-vue/src/domain/realtime/background/backendSinetWasm.js
deleted file mode 100644
index ab3b2b43b..000000000
--- a/demo/video-chat/frontend-vue/src/domain/realtime/background/backendSinetWasm.js
+++ /dev/null
@@ -1,365 +0,0 @@
-import { shapeForegroundAlpha } from './maskPostprocess';
-
-const VIDEOCHAT_CDN_ORIGIN = String(import.meta.env.VITE_VIDEOCHAT_CDN_ORIGIN || '').replace(/\/+$/, '');
-const SINET_MODEL_WIDTH = 256;
-const SINET_MODEL_HEIGHT = 256;
-const SINET_ASSET_BASE_PATH = '/cdn/vendor/sinet/';
-const SINET_GRAPH_URL = `${VIDEOCHAT_CDN_ORIGIN}${SINET_ASSET_BASE_PATH}sinet-float.onnx`;
-const SINET_EXTERNAL_WEIGHTS_URL = `${VIDEOCHAT_CDN_ORIGIN}${SINET_ASSET_BASE_PATH}sinet.data`;
-const SINET_EXTERNAL_WEIGHTS_PATH = 'sinet.data';
-
-let sinetModelFiles = null;
-let sinetSession = null;
-let ortModule = null;
-let sinetRunChain = Promise.resolve();
-
-function clampBox(box, maxW, maxH) {
- const x = Math.max(0, Math.min(maxW, box.x));
- const y = Math.max(0, Math.min(maxH, box.y));
- const width = Math.max(0, Math.min(maxW - x, box.width));
- const height = Math.max(0, Math.min(maxH - y, box.height));
- return { x, y, width, height };
-}
-
-function estimatePersonBoxFromAlpha(alpha, width, height) {
- const step = 4;
- let minX = width;
- let minY = height;
- let maxX = 0;
- let maxY = 0;
- let hits = 0;
-
- for (let y = 0; y < height; y += step) {
- for (let x = 0; x < width; x += step) {
- const value = alpha[y * width + x] ?? 0;
- if (value < 40) continue;
- hits += 1;
- minX = Math.min(minX, x);
- minY = Math.min(minY, y);
- maxX = Math.max(maxX, x);
- maxY = Math.max(maxY, y);
- }
- }
-
- if (hits <= 0 || maxX <= minX || maxY <= minY) return null;
- const pad = 12;
- return clampBox({
- x: minX - pad,
- y: minY - pad,
- width: maxX - minX + pad * 2,
- height: maxY - minY + pad * 2,
- }, width, height);
-}
-
-function readAlphaControls(opts) {
- return {
- gamma: Math.max(0.4, Math.min(2.5, Number(opts.alphaGamma ?? 0.8))),
- contrast: Math.max(0.25, Math.min(4, Number(opts.maskContrast ?? 0.75))),
- averageRadius: Math.max(0, Math.min(12, Math.round(Number(opts.averageRadius ?? 6)))),
- temporalRise: Math.max(0, Math.min(1, Number(opts.temporalRise ?? 0.7))),
- temporalFall: Math.max(0, Math.min(1, Number(opts.temporalFall ?? 0.6))),
- };
-}
-
-function isVideoFrameReady(video) {
- return video instanceof HTMLVideoElement
- && video.readyState >= 2
- && !video.ended
- && Math.max(0, Number(video.videoWidth) || 0) > 1
- && Math.max(0, Number(video.videoHeight) || 0) > 1;
-}
-
-async function fetchBinaryAsset(path) {
- const response = await fetch(path, { cache: 'force-cache' });
- if (!response.ok) throw new Error(`Failed to load ${path}: HTTP ${response.status}`);
- return new Uint8Array(await response.arrayBuffer());
-}
-
-function configureOrtWasmRuntime(ort) {
- const wasm = ort?.env?.wasm;
- if (!wasm || wasm.__kingSinetConfigured === true) return;
- wasm.proxy = false;
- wasm.numThreads = 1;
- wasm.__kingSinetConfigured = true;
-}
-
-function getOrtModule() {
- if (!ortModule) {
- ortModule = import('onnxruntime-web/wasm')
- .then((ort) => {
- configureOrtWasmRuntime(ort);
- return ort;
- })
- .catch((error) => {
- ortModule = null;
- throw error;
- });
- }
- return ortModule;
-}
-
-function getSinetModelFiles() {
- if (!sinetModelFiles) {
- sinetModelFiles = Promise.all([
- fetchBinaryAsset(SINET_GRAPH_URL),
- fetchBinaryAsset(SINET_EXTERNAL_WEIGHTS_URL),
- ])
- .then(([model, weights]) => ({ model, weights }))
- .catch((error) => {
- sinetModelFiles = null;
- throw error;
- });
- }
- return sinetModelFiles;
-}
-
-function getSinetSession() {
- if (!sinetSession) {
- sinetSession = Promise.all([getSinetModelFiles(), getOrtModule()])
- .then(([{ model, weights }, ort]) => ort.InferenceSession.create(model, {
- executionProviders: ['wasm'],
- graphOptimizationLevel: 'all',
- externalData: [{ path: SINET_EXTERNAL_WEIGHTS_PATH, data: weights }],
- }))
- .catch((error) => {
- sinetSession = null;
- throw error;
- });
- }
- return sinetSession;
-}
-
-function enqueueSinetRun(work) {
- const run = sinetRunChain.then(work, work);
- sinetRunChain = run.catch(() => {});
- return run;
-}
-
-function imageDataToSinetTensor(image, ort) {
- const width = image.width;
- const height = image.height;
- const pixels = width * height;
- const out = new Float32Array(3 * pixels);
- const data = image.data;
-
- for (let i = 0; i < pixels; i += 1) {
- const p = i * 4;
- out[i] = (data[p] ?? 0) / 255;
- out[pixels + i] = (data[p + 1] ?? 0) / 255;
- out[pixels * 2 + i] = (data[p + 2] ?? 0) / 255;
- }
-
- return new ort.Tensor('float32', out, [1, 3, height, width]);
-}
-
-function binaryForegroundAlpha(value, threshold = 0) {
- return Number(value) > Number(threshold) ? 255 : 0;
-}
-
-function singleChannelAlpha(output, pixels) {
- const alpha = new Uint8ClampedArray(pixels);
- let probabilityLike = true;
- const sampleCount = Math.min(pixels, 1024);
-
- for (let i = 0; i < sampleCount; i += 1) {
- const value = output[i] ?? 0;
- if (value < -0.01 || value > 1.01) {
- probabilityLike = false;
- break;
- }
- }
-
- const threshold = probabilityLike ? 0.5 : 0;
- for (let i = 0; i < pixels; i += 1) {
- const value = output[i] ?? 0;
- alpha[i] = binaryForegroundAlpha(value, threshold);
- }
-
- return alpha;
-}
-
-function twoChannelForegroundAlpha(output, pixels) {
- const alpha = new Uint8ClampedArray(pixels);
-
- for (let i = 0; i < pixels; i += 1) {
- const bg = output[i] ?? 0;
- const fg = output[pixels + i] ?? 0;
- alpha[i] = binaryForegroundAlpha(fg, bg);
- }
-
- return alpha;
-}
-
-function sinetForegroundAlpha(output, width, height) {
- const pixels = width * height;
- if (!(output instanceof Float32Array) || output.length < pixels) {
- return new Uint8ClampedArray(pixels);
- }
- if (output.length >= pixels * 2) {
- return twoChannelForegroundAlpha(output, pixels);
- }
- return singleChannelAlpha(output, pixels);
-}
-
-function alphaToFloatMask(alpha) {
- const out = new Float32Array(alpha.length);
- for (let i = 0; i < alpha.length; i += 1) {
- out[i] = (alpha[i] ?? 0) / 255;
- }
- return out;
-}
-
-export async function createSinetWasmSegmentationBackend(opts = {}) {
- if (typeof document === 'undefined') return null;
-
- const detectIntervalMs = Math.max(66, Math.min(1200, Math.round(Number(opts.detectIntervalMs || 140))));
- const alphaControls = readAlphaControls(opts);
- const sampleCanvas = document.createElement('canvas');
- sampleCanvas.width = SINET_MODEL_WIDTH;
- sampleCanvas.height = SINET_MODEL_HEIGHT;
- const sampleCtx = sampleCanvas.getContext('2d', { alpha: false, desynchronized: true, willReadFrequently: true });
- const resultFrameCanvas = document.createElement('canvas');
- resultFrameCanvas.width = 1;
- resultFrameCanvas.height = 1;
- const resultFrameCtx = resultFrameCanvas.getContext('2d', { alpha: false, desynchronized: true });
- if (!sampleCtx || !resultFrameCtx) return null;
-
- await getSinetSession();
-
- let faces = [];
- let pendingMaskValues = null;
- let pendingSampleMs = null;
- let pendingResultFrame = false;
- let lastDetectAt = -Infinity;
- let detectPending = false;
- let previousAlpha = null;
- let disposed = false;
- let runtimeErrorWarned = false;
-
- function resetState() {
- faces = [];
- pendingMaskValues = null;
- pendingSampleMs = null;
- pendingResultFrame = false;
- lastDetectAt = -Infinity;
- detectPending = false;
- previousAlpha = null;
- }
-
- function warnRuntimeError(error) {
- if (runtimeErrorWarned) return;
- runtimeErrorWarned = true;
- console.warn('[BackgroundFilter] SINet WASM segmentation failed', {
- message: error?.message || 'segmentation_failed',
- });
- }
-
- const runSegmentation = async (video, sourceWidth, sourceHeight, targetWidth, targetHeight) => {
- const detectStartedAt = performance.now();
- try {
- if (disposed || !isVideoFrameReady(video)) return;
-
- if (resultFrameCanvas.width !== targetWidth || resultFrameCanvas.height !== targetHeight) {
- resultFrameCanvas.width = targetWidth;
- resultFrameCanvas.height = targetHeight;
- }
- resultFrameCtx.drawImage(video, 0, 0, sourceWidth, sourceHeight, 0, 0, targetWidth, targetHeight);
- sampleCtx.drawImage(video, 0, 0, sourceWidth, sourceHeight, 0, 0, SINET_MODEL_WIDTH, SINET_MODEL_HEIGHT);
-
- const frame = sampleCtx.getImageData(0, 0, SINET_MODEL_WIDTH, SINET_MODEL_HEIGHT);
- const [session, ort] = await Promise.all([getSinetSession(), getOrtModule()]);
- const inputName = session.inputNames[0] || 'image';
- const outputName = session.outputNames[0] || 'mask';
- const inputTensor = imageDataToSinetTensor(frame, ort);
- const output = await enqueueSinetRun(() => session.run({ [inputName]: inputTensor }));
- const tensor = output[outputName] || output[session.outputNames[0]];
- if (!tensor || !(tensor.data instanceof Float32Array)) return;
-
- if (!previousAlpha || previousAlpha.length !== SINET_MODEL_WIDTH * SINET_MODEL_HEIGHT) {
- previousAlpha = new Uint8ClampedArray(SINET_MODEL_WIDTH * SINET_MODEL_HEIGHT);
- }
-
- const alpha = shapeForegroundAlpha(
- sinetForegroundAlpha(tensor.data, SINET_MODEL_WIDTH, SINET_MODEL_HEIGHT),
- SINET_MODEL_WIDTH,
- SINET_MODEL_HEIGHT,
- alphaControls,
- previousAlpha,
- );
- pendingMaskValues = alphaToFloatMask(alpha);
- pendingResultFrame = true;
-
- const box = estimatePersonBoxFromAlpha(alpha, SINET_MODEL_WIDTH, SINET_MODEL_HEIGHT);
- if (box) {
- const scaleX = targetWidth / SINET_MODEL_WIDTH;
- const scaleY = targetHeight / SINET_MODEL_HEIGHT;
- faces = [clampBox({
- x: box.x * scaleX,
- y: box.y * scaleY,
- width: box.width * scaleX,
- height: box.height * scaleY,
- }, targetWidth, targetHeight)];
- } else {
- faces = [];
- }
- } catch (error) {
- warnRuntimeError(error);
- } finally {
- pendingSampleMs = Math.max(0, performance.now() - detectStartedAt);
- detectPending = false;
- }
- };
-
- return {
- kind: 'sinet_wasm',
-
- resetSession() {
- resetState();
- return Promise.resolve();
- },
-
- nextFaces(video, vw, vh, nowMs) {
- if (disposed || !isVideoFrameReady(video)) {
- return { faces: [], detectSampleMs: null, matteMaskBitmap: null, matteMaskValues: null };
- }
-
- const sourceWidth = Math.max(1, Math.round(Number(video.videoWidth) || Number(vw) || 1));
- const sourceHeight = Math.max(1, Math.round(Number(video.videoHeight) || Number(vh) || 1));
- const targetWidth = Math.max(1, Math.round(Number(vw) || sourceWidth));
- const targetHeight = Math.max(1, Math.round(Number(vh) || sourceHeight));
-
- if (!detectPending && nowMs - lastDetectAt >= detectIntervalMs) {
- detectPending = true;
- lastDetectAt = nowMs;
- void runSegmentation(video, sourceWidth, sourceHeight, targetWidth, targetHeight);
- }
-
- const sample = pendingSampleMs;
- pendingSampleMs = null;
- const maskValues = pendingMaskValues;
- pendingMaskValues = null;
- const sourceFrame = pendingResultFrame ? resultFrameCanvas : null;
- pendingResultFrame = false;
-
- return {
- faces,
- detectSampleMs: sample,
- matteMaskBitmap: null,
- matteMaskValues: maskValues,
- matteMaskWidth: SINET_MODEL_WIDTH,
- matteMaskHeight: SINET_MODEL_HEIGHT,
- sourceFrame,
- };
- },
-
- dispose() {
- if (disposed) return;
- disposed = true;
- resetState();
- resultFrameCanvas.width = 1;
- resultFrameCanvas.height = 1;
- sampleCanvas.width = 1;
- sampleCanvas.height = 1;
- },
- };
-}
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/background/backendWorkerSegmenter.js b/demo/video-chat/frontend-vue/src/domain/realtime/background/backendWorkerSegmenter.js
new file mode 100644
index 000000000..d0e835d10
--- /dev/null
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/background/backendWorkerSegmenter.js
@@ -0,0 +1,425 @@
+/**
+ * Worker-based MediaPipe segmentation backend.
+ *
+ * Wraps imageSegmenterWorker.js and exposes the background segmentation
+ * interface expected by stream.js:
+ * { kind, nextFaces(video, vw, vh, nowMs), dispose() }
+ *
+ * Key differences from the old backends:
+ * - Segmentation runs in a dedicated worker thread (no main-thread blocking)
+ * - Uses selfie_multiclass_256x256 + CATEGORY_MASK → full person silhouette
+ * - Returns faces=[] (no face detection; compositor uses the mask directly)
+ * - matteMaskBitmap is a MediaPipe-drawn alpha mask updated on every new inference result
+ *
+ * The returned sourceFrame is matched to the mask result so the compositor can
+ * compose pixels from the same moment in time.
+ */
+
+const VIDEOCHAT_CDN_ORIGIN = String(import.meta.env.VITE_VIDEOCHAT_CDN_ORIGIN || '').replace(/\/+$/, '');
+const MEDIAPIPE_MODEL_BASE_PATH = '/cdn/vendor/mediapipe/models/';
+const MEDIAPIPE_WASM_BASE_PATH = '/wasm/';
+const MODEL_PATH = `${VIDEOCHAT_CDN_ORIGIN}${MEDIAPIPE_MODEL_BASE_PATH}selfie_multiclass_256x256.tflite`;
+const WASM_PATH = `${VIDEOCHAT_CDN_ORIGIN}${MEDIAPIPE_WASM_BASE_PATH}`;
+const INIT_TIMEOUT_MS = 15000;
+const LEASE_WAIT_TIMEOUT_MS = 2000;
+const LEASE_WAIT_POLL_MS = 25;
+const SHARED_BACKEND_IDLE_TTL_MS = 60000;
+
+let sharedBackend = null;
+let sharedBackendPromise = null;
+let sharedBackendIdleTimer = null;
+let activeLeaseOwner = null;
+let leaseCounter = 0;
+
+export async function createWorkerSegmenterBackend(opts = {}) {
+ const modelAssetPath = opts.modelAssetPath || MODEL_PATH;
+ const delegate = opts.delegate || 'GPU';
+ //const detectIntervalMs = Math.max(1, Math.min(1, Math.round(Number(opts.detectIntervalMs || 100))));
+ const wasmPath = opts.wasmPath || WASM_PATH;
+ const workerUrl = new URL('./workers/imageSegmenterWorker.js', import.meta.url);
+ const worker = new Worker(workerUrl, {
+ type: 'module'
+ });
+
+ // Wait for READY before sending INIT
+ await new Promise((resolve, reject) => {
+ const fail = (error) => {
+ clearTimeout(timeout);
+ try { worker.terminate(); } catch { /* ignore */ }
+ reject(error);
+ };
+ const timeout = setTimeout(() => fail(new Error('Worker READY timeout')), 5000);
+ worker.onmessage = (e) => {
+ if (e.data?.type === 'READY') {
+ clearTimeout(timeout);
+ resolve();
+ }
+ };
+ worker.onerror = (e) => {
+ console.error('Worker error before READY:', e);
+ fail(new Error(`Worker error: ${e.message || 'init_failed'}`));
+ };
+ });
+ // Wait for INIT_DONE (or INIT_ERROR / timeout).
+ const ready = await new Promise((resolve, reject) => {
+ const fail = (error) => {
+ clearTimeout(timer);
+ try { worker.terminate(); } catch { /* ignore */ }
+ reject(error);
+ };
+ const timer = setTimeout(() => {
+ fail(new Error('WorkerSegmenter: init timeout'));
+ }, INIT_TIMEOUT_MS);
+
+ worker.onmessage = (event) => {
+ const { type } = event.data;
+ if (type === 'INIT_DONE') {
+ clearTimeout(timer);
+ resolve(event.data.labels || []);
+ } else if (type === 'INIT_ERROR') {
+ fail(new Error(`WorkerSegmenter: ${event.data.error}`));
+ }
+ };
+
+ worker.onerror = (err) => {
+ fail(new Error(`WorkerSegmenter worker error: ${err.message || 'init_failed'}`));
+ };
+
+ worker.postMessage({ type: 'INIT', modelAssetPath, delegate, wasmPath });
+ });
+
+ const pendingFrameCanvas = document.createElement('canvas');
+ pendingFrameCanvas.width = 1;
+ pendingFrameCanvas.height = 1;
+ const pendingFrameCtx = pendingFrameCanvas.getContext('2d', { alpha: false });
+ const resultFrameCanvas = document.createElement('canvas');
+ resultFrameCanvas.width = 1;
+ resultFrameCanvas.height = 1;
+ const resultFrameCtx = resultFrameCanvas.getContext('2d', { alpha: false });
+
+ let labels = Array.isArray(ready) ? ready : [];
+ let pendingFrame = false;
+ let lastDetectAt = -Infinity;
+ let pendingSampleMs = null;
+ let pendingMaskBitmap = null;
+ let pendingMaskValues = null;
+ let pendingMaskWidth = 0;
+ let pendingMaskHeight = 0;
+ let pendingResultFrame = false;
+ let disposed = false;
+ let hasQueuedFrame = false;
+ let queuedFrameParams = null;
+ let sessionId = 0;
+ const resetResolvers = new Map();
+
+ function clearPendingState() {
+ pendingFrame = false;
+ lastDetectAt = -Infinity;
+ pendingSampleMs = null;
+ pendingMaskBitmap?.close?.();
+ pendingMaskBitmap = null;
+ pendingMaskValues = null;
+ pendingMaskWidth = 0;
+ pendingMaskHeight = 0;
+ pendingResultFrame = false;
+ hasQueuedFrame = false;
+ queuedFrameParams = null;
+ }
+
+ function nextSessionId() {
+ sessionId = (sessionId % Number.MAX_SAFE_INTEGER) + 1;
+ return sessionId;
+ }
+
+ function queueLatestFrame(params) {
+ hasQueuedFrame = true;
+ queuedFrameParams = params;
+ }
+
+ function dispatchFrame(params) {
+ if (!params || disposed) return;
+ const { video, sourceWidth, sourceHeight, targetWidth, targetHeight, timestampMs, nowMs } = params;
+ pendingFrame = true;
+ lastDetectAt = nowMs;
+ if (!pendingFrameCtx) {
+ pendingFrame = false;
+ return;
+ }
+ if (pendingFrameCanvas.width !== targetWidth || pendingFrameCanvas.height !== targetHeight) {
+ pendingFrameCanvas.width = targetWidth;
+ pendingFrameCanvas.height = targetHeight;
+ }
+ pendingFrameCtx.drawImage(video, 0, 0, sourceWidth, sourceHeight, 0, 0, targetWidth, targetHeight);
+ const dispatchSessionId = sessionId;
+
+ createImageBitmap(pendingFrameCanvas).then((bitmap) => {
+ if (disposed || dispatchSessionId !== sessionId) {
+ bitmap.close();
+ return;
+ }
+ worker.postMessage(
+ { type: 'SEGMENT_VIDEO', bitmap, sessionId: dispatchSessionId, timestampMs },
+ [bitmap],
+ );
+ }).catch(() => {
+ pendingFrame = false;
+ if (hasQueuedFrame && queuedFrameParams) {
+ const nextParams = queuedFrameParams;
+ hasQueuedFrame = false;
+ queuedFrameParams = null;
+ dispatchFrame(nextParams);
+ }
+ });
+ }
+
+ worker.onmessage = (event) => {
+ const { type } = event.data;
+
+ if (type === 'RESET_DONE') {
+ const resolvedSessionId = Math.max(0, Math.round(Number(event.data.sessionId) || 0));
+ const resolve = resetResolvers.get(resolvedSessionId);
+ if (resolve) {
+ resetResolvers.delete(resolvedSessionId);
+ resolve();
+ }
+ } else if (type === 'SEGMENT_RESULT') {
+ if (Math.max(0, Math.round(Number(event.data.sessionId) || 0)) !== sessionId) {
+ event.data.maskBitmap?.close?.();
+ pendingFrame = false;
+ return;
+ }
+ pendingFrame = false;
+ const { maskBitmap, maskValues, width, height, inferenceMs, inferenceTime } = event.data;
+ const sample = typeof inferenceMs === 'number'
+ ? inferenceMs
+ : typeof inferenceTime === 'number'
+ ? inferenceTime
+ : null;
+ pendingSampleMs = typeof sample === 'number' ? Math.max(0, sample) : null;
+ if (maskBitmap instanceof ImageBitmap && width > 0 && height > 0) {
+ pendingMaskBitmap?.close?.();
+ pendingMaskBitmap = maskBitmap;
+ pendingMaskValues = null;
+ pendingMaskWidth = Math.max(1, Math.round(Number(width) || 1));
+ pendingMaskHeight = Math.max(1, Math.round(Number(height) || 1));
+ } else if (maskValues instanceof Float32Array && width > 0 && height > 0) {
+ pendingMaskBitmap?.close?.();
+ pendingMaskBitmap = null;
+ pendingMaskValues = maskValues;
+ pendingMaskWidth = Math.max(1, Math.round(Number(width) || 1));
+ pendingMaskHeight = Math.max(1, Math.round(Number(height) || 1));
+ }
+ if (pendingSampleMs !== null && resultFrameCtx) {
+ if (resultFrameCanvas.width !== pendingFrameCanvas.width || resultFrameCanvas.height !== pendingFrameCanvas.height) {
+ resultFrameCanvas.width = pendingFrameCanvas.width;
+ resultFrameCanvas.height = pendingFrameCanvas.height;
+ }
+ resultFrameCtx.drawImage(pendingFrameCanvas, 0, 0, resultFrameCanvas.width, resultFrameCanvas.height);
+ pendingResultFrame = true;
+ }
+ if (hasQueuedFrame && queuedFrameParams) {
+ const nextParams = queuedFrameParams;
+ hasQueuedFrame = false;
+ queuedFrameParams = null;
+ dispatchFrame(nextParams);
+ }
+ } else if (type === 'SEGMENT_ERROR') {
+ pendingFrame = false;
+ if (hasQueuedFrame && queuedFrameParams) {
+ const nextParams = queuedFrameParams;
+ hasQueuedFrame = false;
+ queuedFrameParams = null;
+ dispatchFrame(nextParams);
+ }
+ }
+ // SEGMENT_ERROR is logged but we just continue - next frame will retry.
+ };
+
+ return {
+ kind: 'worker-segmenter',
+ labels,
+
+ resetSession() {
+ if (disposed) return Promise.resolve();
+ const nextId = nextSessionId();
+ clearPendingState();
+ return new Promise((resolve) => {
+ const timer = setTimeout(() => {
+ resetResolvers.delete(nextId);
+ resolve();
+ }, 1000);
+ resetResolvers.set(nextId, () => {
+ clearTimeout(timer);
+ resolve();
+ });
+ try {
+ worker.postMessage({ type: 'RESET', sessionId: nextId });
+ } catch {
+ clearTimeout(timer);
+ resetResolvers.delete(nextId);
+ resolve();
+ }
+ });
+ },
+
+ nextFaces(video, vw, vh, nowMs) {
+ if (disposed) return { faces: [], detectSampleMs: null, matteMaskBitmap: null, matteMaskValues: null };
+
+ const sourceWidth = Math.max(1, Math.round(Number(video?.videoWidth) || Number(vw) || 1));
+ const sourceHeight = Math.max(1, Math.round(Number(video?.videoHeight) || Number(vh) || 1));
+ const targetWidth = Math.max(1, Math.round(Number(vw) || sourceWidth));
+ const targetHeight = Math.max(1, Math.round(Number(vh) || sourceHeight));
+
+ const hasFrame = video instanceof HTMLVideoElement
+ && video.readyState >= 2
+ && !video.ended
+ && sourceWidth > 1
+ && sourceHeight > 1;
+
+ if (!hasFrame) {
+ return { faces: [], detectSampleMs: null, matteMaskBitmap: null, matteMaskValues: null };
+ }
+
+ // Dispatch a new frame if the interval has elapsed and the worker is free.
+ //if (!pendingFrame && nowMs - lastDetectAt >= detectIntervalMs) {
+ const timestampMs = Number.isFinite(video.currentTime)
+ ? Math.max(0, Math.round(video.currentTime * 1000))
+ : Math.max(0, Math.round(nowMs));
+ const frameParams = {
+ video,
+ sourceWidth,
+ sourceHeight,
+ targetWidth,
+ targetHeight,
+ timestampMs,
+ nowMs,
+ };
+
+ if (!pendingFrame) {
+ dispatchFrame(frameParams);
+ } else {
+ queueLatestFrame(frameParams);
+ }
+
+ const sample = pendingSampleMs;
+ pendingSampleMs = null;
+ const maskBitmap = pendingMaskBitmap;
+ const maskValues = pendingMaskValues;
+ const maskWidth = pendingMaskWidth;
+ const maskHeight = pendingMaskHeight;
+ pendingMaskBitmap = null;
+ pendingMaskValues = null;
+ pendingMaskWidth = 0;
+ pendingMaskHeight = 0;
+ const sourceFrame = pendingResultFrame ? resultFrameCanvas : null;
+ pendingResultFrame = false;
+
+ return {
+ faces: [],
+ detectSampleMs: sample,
+ matteMaskBitmap: maskBitmap,
+ matteMaskValues: maskValues,
+ matteMaskWidth: maskWidth,
+ matteMaskHeight: maskHeight,
+ sourceFrame,
+ };
+ },
+
+ dispose() {
+ if (disposed) return;
+ disposed = true;
+ try {
+ worker.postMessage({ type: 'CLEANUP' });
+ } catch { /* ignore */ }
+ // Terminate after a short grace period for the cleanup message.
+ setTimeout(() => {
+ try { worker.terminate(); } catch { /* ignore */ }
+ }, 200);
+ pendingFrameCanvas.width = 1;
+ pendingFrameCanvas.height = 1;
+ resultFrameCanvas.width = 1;
+ resultFrameCanvas.height = 1;
+ clearPendingState();
+ for (const resolve of resetResolvers.values()) {
+ try { resolve(); } catch { /* ignore */ }
+ }
+ resetResolvers.clear();
+ },
+ };
+}
+
+function clearSharedBackendIdleTimer() {
+ if (!sharedBackendIdleTimer) return;
+ clearTimeout(sharedBackendIdleTimer);
+ sharedBackendIdleTimer = null;
+}
+
+function scheduleSharedBackendIdleDispose() {
+ clearSharedBackendIdleTimer();
+ sharedBackendIdleTimer = setTimeout(() => {
+ if (activeLeaseOwner) return;
+ sharedBackend?.dispose?.();
+ sharedBackend = null;
+ sharedBackendPromise = null;
+ sharedBackendIdleTimer = null;
+ }, SHARED_BACKEND_IDLE_TTL_MS);
+}
+
+async function waitForActiveLeaseToRelease() {
+ const startedAt = performance.now();
+ while (activeLeaseOwner && performance.now() - startedAt < LEASE_WAIT_TIMEOUT_MS) {
+ await new Promise((resolve) => setTimeout(resolve, LEASE_WAIT_POLL_MS));
+ }
+ if (activeLeaseOwner) {
+ throw new Error(`Worker segmenter already leased by ${activeLeaseOwner}`);
+ }
+}
+
+async function ensureSharedWorkerSegmenterBackend(opts = {}) {
+ clearSharedBackendIdleTimer();
+ if (sharedBackend) return sharedBackend;
+ if (!sharedBackendPromise) {
+ sharedBackendPromise = createWorkerSegmenterBackend(opts)
+ .then((backend) => {
+ sharedBackend = backend;
+ return backend;
+ })
+ .catch((error) => {
+ sharedBackend = null;
+ sharedBackendPromise = null;
+ throw error;
+ });
+ }
+ return sharedBackendPromise;
+}
+
+export async function acquireWorkerSegmenterBackendLease(opts = {}) {
+ await waitForActiveLeaseToRelease();
+ const backend = await ensureSharedWorkerSegmenterBackend(opts);
+ await waitForActiveLeaseToRelease();
+
+ const ownerId = String(opts.ownerId || `background-pipeline-${++leaseCounter}`);
+ activeLeaseOwner = ownerId;
+ await backend.resetSession?.();
+
+ let released = false;
+ return {
+ backend,
+ ownerId,
+ release({ keepWarm = true } = {}) {
+ if (released) return;
+ released = true;
+ if (activeLeaseOwner !== ownerId) return;
+ activeLeaseOwner = null;
+ void backend.resetSession?.();
+ if (keepWarm) {
+ scheduleSharedBackendIdleDispose();
+ } else {
+ clearSharedBackendIdleTimer();
+ backend.dispose?.();
+ sharedBackend = null;
+ sharedBackendPromise = null;
+ }
+ },
+ };
+}
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/background/maskPostprocess.js b/demo/video-chat/frontend-vue/src/domain/realtime/background/maskPostprocess.js
deleted file mode 100644
index a69922b62..000000000
--- a/demo/video-chat/frontend-vue/src/domain/realtime/background/maskPostprocess.js
+++ /dev/null
@@ -1,214 +0,0 @@
-function clampByte(value) {
- return Math.max(0, Math.min(255, Math.round(value)));
-}
-
-function clamp01(value) {
- return Math.max(0, Math.min(1, Number(value) || 0));
-}
-
-function smoothstep(edge0, edge1, value) {
- if (value <= edge0) return 0;
- if (value >= edge1) return 1;
- const t = (value - edge0) / Math.max(1e-6, edge1 - edge0);
- return t * t * (3 - 2 * t);
-}
-
-export const DEFAULT_INNER_CONTRACT_PX = 16;
-export const DEFAULT_INNER_FEATHER_PX = 24;
-
-const INNER_FEATHER_RAMP = [
- { progress: 0.0, alpha: 0.05 },
- { progress: 0.2, alpha: 0.15 },
- { progress: 0.4, alpha: 0.4 },
- { progress: 0.6, alpha: 0.7 },
- { progress: 0.8, alpha: 0.9 },
- { progress: 1.0, alpha: 1.0 },
-];
-
-export function sampleInnerFeatherRamp(progress) {
- const t = clamp01(progress);
- let previous = INNER_FEATHER_RAMP[0];
- for (let index = 1; index < INNER_FEATHER_RAMP.length; index += 1) {
- const next = INNER_FEATHER_RAMP[index];
- if (t > next.progress) {
- previous = next;
- continue;
- }
- const span = Math.max(1e-6, next.progress - previous.progress);
- const localT = (t - previous.progress) / span;
- return previous.alpha + (next.alpha - previous.alpha) * localT;
- }
- return INNER_FEATHER_RAMP[INNER_FEATHER_RAMP.length - 1].alpha;
-}
-
-export function buildInnerDistanceFeatherAlpha(base, width, height, threshold = 110) {
- const sourceWidth = Math.max(1, Math.round(Number(width) || 1));
- const sourceHeight = Math.max(1, Math.round(Number(height) || 1));
- const pixelCount = sourceWidth * sourceHeight;
- if (!base || base.length < pixelCount) return new Uint8ClampedArray(pixelCount);
-
- const maxInset = Math.max(1, Math.floor(Math.min(sourceWidth, sourceHeight) / 3));
- const contractPx = Math.min(DEFAULT_INNER_CONTRACT_PX, maxInset);
- const featherPx = Math.max(1, Math.min(DEFAULT_INNER_FEATHER_PX, maxInset));
- const cutoff = Math.max(0, Math.min(255, Math.round(Number(threshold) || 0)));
- const dist = new Float32Array(pixelCount);
- const inf = sourceWidth + sourceHeight + 1;
- const diagonal = Math.SQRT2;
-
- for (let y = 0; y < sourceHeight; y += 1) {
- for (let x = 0; x < sourceWidth; x += 1) {
- const index = y * sourceWidth + x;
- const value = Number(base[index]) || 0;
- const isImageEdge = x === 0 || y === 0 || x === sourceWidth - 1 || y === sourceHeight - 1;
- dist[index] = value >= cutoff && !isImageEdge ? inf : 0;
- }
- }
-
- for (let y = 0; y < sourceHeight; y += 1) {
- for (let x = 0; x < sourceWidth; x += 1) {
- const index = y * sourceWidth + x;
- let best = dist[index];
- if (x > 0) best = Math.min(best, dist[index - 1] + 1);
- if (y > 0) best = Math.min(best, dist[index - sourceWidth] + 1);
- if (x > 0 && y > 0) best = Math.min(best, dist[index - sourceWidth - 1] + diagonal);
- if (x + 1 < sourceWidth && y > 0) best = Math.min(best, dist[index - sourceWidth + 1] + diagonal);
- dist[index] = best;
- }
- }
-
- for (let y = sourceHeight - 1; y >= 0; y -= 1) {
- for (let x = sourceWidth - 1; x >= 0; x -= 1) {
- const index = y * sourceWidth + x;
- let best = dist[index];
- if (x + 1 < sourceWidth) best = Math.min(best, dist[index + 1] + 1);
- if (y + 1 < sourceHeight) best = Math.min(best, dist[index + sourceWidth] + 1);
- if (x + 1 < sourceWidth && y + 1 < sourceHeight) best = Math.min(best, dist[index + sourceWidth + 1] + diagonal);
- if (x > 0 && y + 1 < sourceHeight) best = Math.min(best, dist[index + sourceWidth - 1] + diagonal);
- dist[index] = best;
- }
- }
-
- const out = new Uint8ClampedArray(pixelCount);
- for (let index = 0; index < pixelCount; index += 1) {
- const distance = dist[index];
- if (distance <= contractPx) {
- out[index] = 0;
- continue;
- }
- const t = Math.min(1, Math.max(0, (distance - contractPx) / featherPx));
- const inside = sampleInnerFeatherRamp(t);
- out[index] = clampByte(inside * 255);
- }
-
- return out;
-}
-
-export function buildInnerDistanceFeatherMaskValues(mask, width, height, threshold = 0.43) {
- const sourceWidth = Math.max(1, Math.round(Number(width) || 1));
- const sourceHeight = Math.max(1, Math.round(Number(height) || 1));
- const pixelCount = sourceWidth * sourceHeight;
- if (!(mask instanceof Float32Array) || mask.length < pixelCount) return mask;
-
- const base = new Uint8ClampedArray(pixelCount);
- for (let index = 0; index < pixelCount; index += 1) {
- base[index] = clampByte(clamp01(mask[index]) * 255);
- }
-
- const alpha = buildInnerDistanceFeatherAlpha(base, sourceWidth, sourceHeight, threshold * 255);
- const out = new Float32Array(pixelCount);
- for (let index = 0; index < pixelCount; index += 1) {
- out[index] = (alpha[index] ?? 0) / 255;
- }
- return out;
-}
-
-function gaussianAverageAlpha(alpha, width, height, radius) {
- if (radius <= 0) return alpha;
-
- const cappedRadius = Math.max(1, Math.min(12, radius));
- const sigma = Math.max(1.25, cappedRadius / 2.2);
- const kernel = new Float32Array(cappedRadius * 2 + 1);
- let kernelSum = 0;
-
- for (let k = -cappedRadius; k <= cappedRadius; k += 1) {
- const weight = Math.exp(-(k * k) / (2 * sigma * sigma));
- kernel[k + cappedRadius] = weight;
- kernelSum += weight;
- }
-
- const tmp = new Float32Array(alpha.length);
- const out = new Uint8ClampedArray(alpha.length);
-
- for (let y = 0; y < height; y += 1) {
- for (let x = 0; x < width; x += 1) {
- let sum = 0;
- let weightSum = 0;
- for (let k = -cappedRadius; k <= cappedRadius; k += 1) {
- const xx = Math.max(0, Math.min(width - 1, x + k));
- const weight = kernel[k + cappedRadius] ?? 0;
- sum += (alpha[y * width + xx] ?? 0) * weight;
- weightSum += weight;
- }
- tmp[y * width + x] = sum / Math.max(1e-6, weightSum || kernelSum);
- }
- }
-
- for (let y = 0; y < height; y += 1) {
- for (let x = 0; x < width; x += 1) {
- let sum = 0;
- let weightSum = 0;
- for (let k = -cappedRadius; k <= cappedRadius; k += 1) {
- const yy = Math.max(0, Math.min(height - 1, y + k));
- const weight = kernel[k + cappedRadius] ?? 0;
- sum += (tmp[yy * width + x] ?? 0) * weight;
- weightSum += weight;
- }
- out[y * width + x] = clampByte(sum / Math.max(1e-6, weightSum || kernelSum));
- }
- }
-
- return out;
-}
-
-export function shapeForegroundAlpha(alpha, width, height, controls, previousAlpha = null) {
- const gamma = Math.max(0.4, Math.min(2.5, Number(controls?.gamma) || 0.8));
- const contrast = Math.max(0.25, Math.min(4, Number(controls?.contrast ?? 0.75)));
- const averageRadius = Math.max(0, Math.min(12, Math.round(Number(controls?.averageRadius ?? 6))));
- const contourHalfWidth = Math.max(0.06, Math.min(0.44, 0.26 / contrast));
- const edgeLow = 0.5 - contourHalfWidth;
- const edgeHigh = 0.5 + contourHalfWidth;
- const averagedInput = gaussianAverageAlpha(alpha, width, height, averageRadius);
- const base = new Uint8ClampedArray(alpha.length);
-
- for (let i = 0; i < alpha.length; i += 1) {
- const raw = (averagedInput[i] ?? 0) / 255;
- const contour = smoothstep(edgeLow, edgeHigh, raw);
- base[i] = clampByte(Math.pow(contour, gamma) * 255);
- }
-
- const out = buildInnerDistanceFeatherAlpha(base, width, height);
- if (!previousAlpha || previousAlpha.length !== out.length) return out;
-
- let maxPrevious = 0;
- let maxTarget = 0;
- for (let i = 0; i < out.length; i += 1) {
- maxPrevious = Math.max(maxPrevious, previousAlpha[i] ?? 0);
- maxTarget = Math.max(maxTarget, out[i] ?? 0);
- }
- if (maxPrevious <= 0 && maxTarget > 0) {
- previousAlpha.set(out);
- return out;
- }
-
- const rise = clamp01(controls?.temporalRise ?? 0.7);
- const fall = clamp01(controls?.temporalFall ?? 0.6);
- for (let i = 0; i < out.length; i += 1) {
- const target = out[i] ?? 0;
- const prev = previousAlpha[i] ?? 0;
- const rate = target >= prev ? rise : fall;
- out[i] = clampByte(prev + (target - prev) * rate);
- }
- previousAlpha.set(out);
-
- return out;
-}
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/background/pipeline/compositorCanvasStage.js b/demo/video-chat/frontend-vue/src/domain/realtime/background/pipeline/compositorCanvasStage.js
deleted file mode 100644
index 90ea891e5..000000000
--- a/demo/video-chat/frontend-vue/src/domain/realtime/background/pipeline/compositorCanvasStage.js
+++ /dev/null
@@ -1,137 +0,0 @@
-import {
- createMaskCanvasTools,
- drawContainImage,
- drawCoverImage,
- loadImageCanvas,
- resolveCanvasColor,
-} from './compositorShared';
-
-function isImageBitmap(value) {
- return typeof ImageBitmap !== 'undefined' && value instanceof ImageBitmap;
-}
-
-export function createCanvasBackgroundCompositorStage({
- canvas,
- getBackgroundColor,
- getBackgroundImageUrl,
- getBlurPx,
- getShowSourceUntilMask,
- video,
-}) {
- const ctx = canvas.getContext('2d', { alpha: true, desynchronized: true });
- if (!ctx) throw new Error('2d compositor unavailable');
-
- const maskTools = createMaskCanvasTools(canvas);
- let backgroundImageCanvas = null;
- let backgroundImageUrl = '';
-
- function setBackgroundImageUrl(url) {
- const nextUrl = String(url || '').trim();
- if (nextUrl === backgroundImageUrl) return;
- backgroundImageUrl = nextUrl;
- backgroundImageCanvas = null;
- if (!backgroundImageUrl) return;
- loadImageCanvas(backgroundImageUrl).then((imageCanvas) => {
- if (backgroundImageUrl !== nextUrl) return;
- backgroundImageCanvas = imageCanvas;
- });
- }
-
- function drawBackground(source, mode, backgroundColor, blurPx) {
- ctx.save();
- ctx.globalCompositeOperation = 'destination-over';
- if (mode === 'replace' && backgroundImageCanvas) {
- ctx.filter = 'none';
- drawCoverImage(ctx, backgroundImageCanvas, canvas.width, canvas.height);
- } else if (backgroundColor) {
- ctx.filter = 'none';
- ctx.fillStyle = resolveCanvasColor(backgroundColor, '#000010');
- ctx.fillRect(0, 0, canvas.width, canvas.height);
- } else if (mode === 'blur') {
- ctx.filter = `blur(${blurPx}px)`;
- drawCoverImage(ctx, source, canvas.width, canvas.height);
- } else {
- ctx.filter = 'none';
- drawContainImage(ctx, source, canvas.width, canvas.height);
- }
- ctx.restore();
- }
-
- function render({
- hasMatteMask,
- maskBitmap = null,
- maskHeight = 0,
- maskUpdated = false,
- maskValues = null,
- maskWidth = 0,
- mode = 'blur',
- sourceFrame = null,
- }) {
- const backgroundColor = String(getBackgroundColor?.() || '').trim();
- setBackgroundImageUrl(getBackgroundImageUrl?.() || '');
- const blurPx = Math.max(1, Math.round(Number(getBlurPx?.() || 3)));
- const foregroundSource = sourceFrame || video;
-
- if (mode === 'off') {
- ctx.save();
- ctx.clearRect(0, 0, canvas.width, canvas.height);
- ctx.globalCompositeOperation = 'source-over';
- ctx.filter = 'none';
- drawContainImage(ctx, video, canvas.width, canvas.height);
- ctx.restore();
- return;
- }
-
- let hasRenderableMask = false;
- if (maskUpdated) {
- hasRenderableMask = isImageBitmap(maskBitmap)
- ? maskTools.drawMaskBitmap(maskBitmap, maskWidth, maskHeight)
- : maskTools.drawMaskValues(maskValues, maskWidth, maskHeight);
- } else {
- hasRenderableMask = Boolean(hasMatteMask);
- }
- maskTools.drawDebugCanvases(foregroundSource);
-
- if (!hasRenderableMask) {
- ctx.save();
- ctx.clearRect(0, 0, canvas.width, canvas.height);
- ctx.globalCompositeOperation = 'source-over';
- if (getShowSourceUntilMask?.() === true) {
- ctx.filter = 'none';
- drawContainImage(ctx, foregroundSource, canvas.width, canvas.height);
- } else if (mode === 'replace' && (backgroundColor || backgroundImageUrl)) {
- ctx.filter = 'none';
- ctx.fillStyle = '#061a4a';
- ctx.fillRect(0, 0, canvas.width, canvas.height);
- } else {
- ctx.filter = `blur(${Math.max(blurPx, 6)}px)`;
- drawCoverImage(ctx, video, canvas.width, canvas.height);
- }
- ctx.restore();
- return;
- }
-
- ctx.save();
- ctx.clearRect(0, 0, canvas.width, canvas.height);
- ctx.globalCompositeOperation = 'source-over';
- ctx.filter = 'none';
- drawContainImage(ctx, foregroundSource, canvas.width, canvas.height);
- ctx.restore();
-
- ctx.save();
- ctx.globalCompositeOperation = 'destination-in';
- ctx.filter = 'none';
- ctx.drawImage(maskTools.maskCanvas, 0, 0, canvas.width, canvas.height);
- ctx.restore();
-
- drawBackground(foregroundSource, mode, backgroundColor, blurPx);
- }
-
- return {
- backend: 'canvas',
- getMatteMaskSnapshot: () => maskTools.getMatteMaskSnapshot(),
- render,
- reset: () => maskTools.clearMask(),
- setBackgroundImageUrl,
- };
-}
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/background/pipeline/compositorShared.js b/demo/video-chat/frontend-vue/src/domain/realtime/background/pipeline/compositorShared.js
deleted file mode 100644
index 86f7294de..000000000
--- a/demo/video-chat/frontend-vue/src/domain/realtime/background/pipeline/compositorShared.js
+++ /dev/null
@@ -1,307 +0,0 @@
-import { buildInnerDistanceFeatherAlpha, buildInnerDistanceFeatherMaskValues } from '../maskPostprocess';
-
-const backgroundCanvasCache = new Map();
-
-export function sourceNaturalSize(source, fallbackWidth, fallbackHeight) {
- return {
- width: Math.max(1, source?.videoWidth || source?.naturalWidth || source?.width || fallbackWidth),
- height: Math.max(1, source?.videoHeight || source?.naturalHeight || source?.height || fallbackHeight),
- };
-}
-
-export function drawCoverImage(ctx, image, width, height) {
- const { width: iw, height: ih } = sourceNaturalSize(image, width, height);
- const scale = Math.max(width / iw, height / ih);
- const dw = iw * scale;
- const dh = ih * scale;
- const dx = (width - dw) * 0.5;
- const dy = (height - dh) * 0.5;
- ctx.drawImage(image, dx, dy, dw, dh);
-}
-
-export function drawContainImage(ctx, image, width, height) {
- const { width: iw, height: ih } = sourceNaturalSize(image, width, height);
- const scale = Math.min(width / iw, height / ih);
- const dw = iw * scale;
- const dh = ih * scale;
- const dx = (width - dw) * 0.5;
- const dy = (height - dh) * 0.5;
- ctx.drawImage(image, dx, dy, dw, dh);
-}
-
-export function resolveCoverUvTransform(image, width, height) {
- const { width: iw, height: ih } = sourceNaturalSize(image, width, height);
- const scale = Math.max(width / iw, height / ih);
- const dw = iw * scale;
- const dh = ih * scale;
- const dx = (width - dw) * 0.5;
- const dy = (height - dh) * 0.5;
- return [
- width / dw,
- height / dh,
- -dx / dw,
- -dy / dh,
- ];
-}
-
-export function resolveCanvasColor(color, fallback = '#000010') {
- const value = String(color || '').trim();
- const cssVariable = value.match(/^var\((--[A-Za-z0-9_-]+)\)$/);
- if (!cssVariable || typeof window === 'undefined' || typeof document === 'undefined') {
- return value || fallback;
- }
- const resolved = window.getComputedStyle(document.documentElement).getPropertyValue(cssVariable[1]).trim();
- return resolved || fallback;
-}
-
-export function colorToVec4(value) {
- const text = resolveCanvasColor(value, '#000010');
- const match = /^#?([0-9a-f]{6})$/i.exec(text);
- if (!match) return [0, 0, 0, 1];
- const hex = match[1];
- return [
- Number.parseInt(hex.slice(0, 2), 16) / 255,
- Number.parseInt(hex.slice(2, 4), 16) / 255,
- Number.parseInt(hex.slice(4, 6), 16) / 255,
- 1,
- ];
-}
-
-export async function loadImageCanvas(url) {
- const src = String(url || '').trim();
- if (!src) return null;
- if (backgroundCanvasCache.has(src)) return backgroundCanvasCache.get(src);
-
- const promise = new Promise((resolve) => {
- const image = new Image();
- image.decoding = 'async';
- image.onload = async () => {
- try {
- await image.decode?.();
- } catch {
- // The onload event already confirmed the image is usable.
- }
- const width = Math.max(1, image.naturalWidth || image.width || 1);
- const height = Math.max(1, image.naturalHeight || image.height || 1);
- const imageCanvas = document.createElement('canvas');
- imageCanvas.width = width;
- imageCanvas.height = height;
- const imageCtx = imageCanvas.getContext('2d', { alpha: false });
- if (!imageCtx) {
- resolve(null);
- return;
- }
- imageCtx.drawImage(image, 0, 0, width, height);
- resolve(imageCanvas);
- };
- image.onerror = () => resolve(null);
- image.src = src;
- });
- backgroundCanvasCache.set(src, promise);
- return promise;
-}
-
-export function resizeCanvas(targetCanvas, width, height) {
- const nextWidth = Math.max(1, Math.round(Number(width) || 1));
- const nextHeight = Math.max(1, Math.round(Number(height) || 1));
- if (targetCanvas.width !== nextWidth || targetCanvas.height !== nextHeight) {
- targetCanvas.width = nextWidth;
- targetCanvas.height = nextHeight;
- return true;
- }
- return false;
-}
-
-export function processMaskForAlpha(mask, width, height) {
- return buildInnerDistanceFeatherMaskValues(mask, width, height);
-}
-
-function isImageBitmap(value) {
- return typeof ImageBitmap !== 'undefined' && value instanceof ImageBitmap;
-}
-
-export function createMaskCanvasTools(canvas) {
- const maskCanvas = document.createElement('canvas');
- const maskLayer = maskCanvas.getContext('2d', {
- alpha: true,
- willReadFrequently: true,
- });
- const maskSourceCanvas = document.createElement('canvas');
- const maskSourceLayer = maskSourceCanvas.getContext('2d', {
- alpha: true,
- willReadFrequently: true,
- });
- let maskSourceImageData = null;
-
- function clearMask() {
- if (!maskLayer) return;
- resizeCanvas(maskCanvas, canvas.width, canvas.height);
- maskLayer.clearRect(0, 0, maskCanvas.width, maskCanvas.height);
- }
-
- function ensureMaskSourceImageData(sourceWidth, sourceHeight) {
- const resizedSource = resizeCanvas(maskSourceCanvas, sourceWidth, sourceHeight);
- if (
- resizedSource
- || !maskSourceImageData
- || maskSourceImageData.width !== sourceWidth
- || maskSourceImageData.height !== sourceHeight
- ) {
- maskSourceImageData = maskSourceLayer.createImageData(sourceWidth, sourceHeight);
- }
- return maskSourceImageData;
- }
-
- function commitSourceAlpha(sourceAlpha, sourceWidth, sourceHeight) {
- if (!maskLayer || !maskSourceLayer || !sourceAlpha || sourceAlpha.length <= 0) {
- clearMask();
- return false;
- }
-
- const imageData = ensureMaskSourceImageData(sourceWidth, sourceHeight);
- const data = imageData.data;
- let maxAlpha = 0;
- for (let pixel = 0; pixel < sourceAlpha.length; pixel += 1) {
- const alpha = sourceAlpha[pixel] ?? 0;
- const offset = pixel * 4;
- data[offset] = 255;
- data[offset + 1] = 255;
- data[offset + 2] = 255;
- data[offset + 3] = alpha;
- if (alpha > maxAlpha) maxAlpha = alpha;
- }
-
- resizeCanvas(maskCanvas, canvas.width, canvas.height);
- maskLayer.clearRect(0, 0, maskCanvas.width, maskCanvas.height);
- if (maxAlpha <= 0) return false;
-
- maskSourceLayer.putImageData(imageData, 0, 0);
- maskLayer.imageSmoothingEnabled = true;
- maskLayer.imageSmoothingQuality = 'high';
- maskLayer.drawImage(maskSourceCanvas, 0, 0, maskCanvas.width, maskCanvas.height);
- return true;
- }
-
- function drawMaskBitmap(maskBitmap, maskWidth, maskHeight) {
- if (!isImageBitmap(maskBitmap) || !maskLayer || !maskSourceLayer) {
- clearMask();
- return false;
- }
-
- const sourceWidth = Math.max(1, Math.round(Number(maskWidth) || maskBitmap.width || 0));
- const sourceHeight = Math.max(1, Math.round(Number(maskHeight) || maskBitmap.height || 0));
- if (sourceWidth <= 1 || sourceHeight <= 1) {
- clearMask();
- return false;
- }
-
- resizeCanvas(maskSourceCanvas, sourceWidth, sourceHeight);
- maskSourceLayer.clearRect(0, 0, sourceWidth, sourceHeight);
- maskSourceLayer.imageSmoothingEnabled = false;
- maskSourceLayer.drawImage(maskBitmap, 0, 0, sourceWidth, sourceHeight);
- const bitmapData = maskSourceLayer.getImageData(0, 0, sourceWidth, sourceHeight);
- const sourceAlpha = new Uint8ClampedArray(sourceWidth * sourceHeight);
- for (let pixel = 0; pixel < sourceAlpha.length; pixel += 1) {
- sourceAlpha[pixel] = bitmapData.data[pixel * 4 + 3] ?? 0;
- }
- const shapedAlpha = buildInnerDistanceFeatherAlpha(sourceAlpha, sourceWidth, sourceHeight);
- return commitSourceAlpha(shapedAlpha, sourceWidth, sourceHeight);
- }
-
- function drawMaskValues(maskValues, maskWidth, maskHeight) {
- if (!(maskValues instanceof Float32Array) || !maskLayer || !maskSourceLayer) {
- clearMask();
- return false;
- }
-
- const sourceWidth = Math.max(1, Math.round(Number(maskWidth) || 0));
- const sourceHeight = Math.max(1, Math.round(Number(maskHeight) || 0));
- const pixelCount = sourceWidth * sourceHeight;
- if (pixelCount <= 1 || maskValues.length < pixelCount) {
- clearMask();
- return false;
- }
-
- const shapedValues = processMaskForAlpha(maskValues, sourceWidth, sourceHeight);
- if (!(shapedValues instanceof Float32Array) || shapedValues.length < pixelCount) {
- clearMask();
- return false;
- }
-
- const sourceAlpha = new Uint8ClampedArray(pixelCount);
- for (let pixel = 0; pixel < pixelCount; pixel += 1) {
- sourceAlpha[pixel] = Math.max(0, Math.min(255, Math.round((Number(shapedValues[pixel]) || 0) * 255)));
- }
- return commitSourceAlpha(sourceAlpha, sourceWidth, sourceHeight);
- }
-
- function drawDebugCanvases(source) {
- const debugRoot = document.getElementById('backgroundPipelineDebugDialog');
- const debugMaskCanvas = debugRoot?.querySelector?.('#maskDebug') || null;
- const debugMaskCtx = debugMaskCanvas?.getContext?.('2d');
- if (!debugRoot || !debugMaskCtx) return;
-
- const width = Math.max(1, maskCanvas.width || debugMaskCanvas.width);
- const height = Math.max(1, maskCanvas.height || debugMaskCanvas.height);
- if (debugMaskCanvas.width !== width || debugMaskCanvas.height !== height) {
- debugMaskCanvas.width = width;
- debugMaskCanvas.height = height;
- }
-
- debugMaskCtx.clearRect(0, 0, width, height);
- debugMaskCtx.drawImage(maskCanvas, 0, 0, width, height);
-
- const maskImage = debugMaskCtx.getImageData(0, 0, width, height);
- const data = maskImage.data;
- let nonZeroPixels = 0;
- for (let i = 0; i < data.length; i += 4) {
- const signal = data[i + 3] ?? 0;
- if (signal > 0) nonZeroPixels += 1;
- data[i] = signal;
- data[i + 1] = signal;
- data[i + 2] = signal;
- data[i + 3] = signal;
- }
- debugMaskCtx.putImageData(maskImage, 0, 0);
-
- if (nonZeroPixels === 0) {
- const previousAlpha = debugMaskCtx.globalAlpha;
- debugMaskCtx.globalAlpha = 0.9;
- debugMaskCtx.fillStyle = resolveCanvasColor('var(--color-error)', '#ef4423');
- debugMaskCtx.font = '12px monospace';
- debugMaskCtx.fillText('mask empty', 8, 18);
- debugMaskCtx.globalAlpha = previousAlpha;
- }
-
- const personOnlyCanvas = debugRoot.querySelector?.('#personDebug') || null;
- const personOnlyCtx = personOnlyCanvas?.getContext?.('2d');
- if (!personOnlyCtx) return;
-
- personOnlyCanvas.width = width;
- personOnlyCanvas.height = height;
- personOnlyCtx.clearRect(0, 0, width, height);
- drawContainImage(personOnlyCtx, source, width, height);
- personOnlyCtx.globalCompositeOperation = 'destination-in';
- personOnlyCtx.drawImage(maskCanvas, 0, 0, width, height);
- personOnlyCtx.globalCompositeOperation = 'source-over';
- }
-
- function getMatteMaskSnapshot() {
- try {
- return maskLayer?.getImageData?.(0, 0, maskCanvas.width, maskCanvas.height) || null;
- } catch {
- return null;
- }
- }
-
- return {
- clearMask,
- drawDebugCanvases,
- drawMaskBitmap,
- drawMaskValues,
- get maskCanvas() {
- return maskCanvas;
- },
- getMatteMaskSnapshot,
- };
-}
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/background/pipeline/compositorStage.js b/demo/video-chat/frontend-vue/src/domain/realtime/background/pipeline/compositorStage.js
index e9d7752b1..55b1e50e6 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/background/pipeline/compositorStage.js
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/background/pipeline/compositorStage.js
@@ -1,7 +1,775 @@
-import { createCanvasBackgroundCompositorStage } from './compositorCanvasStage';
-import { createWebGlBackgroundCompositorStage } from './compositorWebglStage';
+const backgroundCanvasCache = new Map();
+
+function sourceNaturalSize(source, fallbackWidth, fallbackHeight) {
+ return {
+ width: Math.max(1, source?.videoWidth || source?.naturalWidth || source?.width || fallbackWidth),
+ height: Math.max(1, source?.videoHeight || source?.naturalHeight || source?.height || fallbackHeight),
+ };
+}
+
+const WEBGL_VERTEX_SHADER = `
+attribute vec2 aPosition;
+varying vec2 vUv;
+
+void main(void) {
+ vUv = aPosition * 0.5 + 0.5;
+ gl_Position = vec4(aPosition, 0.0, 1.0);
+}
+`;
+
+const WEBGL_FRAGMENT_SHADER = `
+precision highp float;
+
+uniform sampler2D uFrame;
+uniform sampler2D uMask;
+uniform sampler2D uBackground;
+uniform vec2 uOutputSize;
+uniform vec2 uMaskSize;
+uniform vec4 uBackgroundColor;
+uniform vec4 uBackgroundUvTransform;
+uniform float uBlurPx;
+uniform float uMaskFeather;
+uniform float uMaskFlipY;
+uniform float uMaskHigh;
+uniform float uMaskLow;
+uniform int uEffect;
+uniform int uBackgroundMode;
+uniform int uHasMask;
+varying vec2 vUv;
+
+float readMask(vec2 uv) {
+ vec2 maskUv = uMaskFlipY > 0.5 ? vec2(uv.x, 1.0 - uv.y) : uv;
+ vec4 maskColor = texture2D(uMask, maskUv);
+ return maskColor.a < 0.999 ? maskColor.a : maskColor.r;
+}
+
+float featherMask(vec2 uv) {
+ vec2 texel = vec2(max(uMaskFeather, 0.0)) / uMaskSize;
+ float center = readMask(uv);
+ if (uMaskFeather <= 0.0) {
+ return center;
+ }
+
+ float sum = center * 4.0;
+ sum += readMask(uv + texel * vec2(-1.0, 0.0)) * 2.0;
+ sum += readMask(uv + texel * vec2(1.0, 0.0)) * 2.0;
+ sum += readMask(uv + texel * vec2(0.0, -1.0)) * 2.0;
+ sum += readMask(uv + texel * vec2(0.0, 1.0)) * 2.0;
+ sum += readMask(uv + texel * vec2(-1.0, -1.0));
+ sum += readMask(uv + texel * vec2(1.0, -1.0));
+ sum += readMask(uv + texel * vec2(-1.0, 1.0));
+ sum += readMask(uv + texel * vec2(1.0, 1.0));
+ return sum / 16.0;
+}
+
+vec4 readBlurredFrame(vec2 uv) {
+ vec2 texel = vec2(max(uBlurPx, 1.0)) / uOutputSize;
+ vec4 sum = texture2D(uFrame, uv) * 0.20;
+ sum += texture2D(uFrame, uv + texel * vec2(-1.0, 0.0)) * 0.12;
+ sum += texture2D(uFrame, uv + texel * vec2(1.0, 0.0)) * 0.12;
+ sum += texture2D(uFrame, uv + texel * vec2(0.0, -1.0)) * 0.12;
+ sum += texture2D(uFrame, uv + texel * vec2(0.0, 1.0)) * 0.12;
+ sum += texture2D(uFrame, uv + texel * vec2(-0.707, -0.707)) * 0.08;
+ sum += texture2D(uFrame, uv + texel * vec2(0.707, -0.707)) * 0.08;
+ sum += texture2D(uFrame, uv + texel * vec2(-0.707, 0.707)) * 0.08;
+ sum += texture2D(uFrame, uv + texel * vec2(0.707, 0.707)) * 0.08;
+ return sum;
+}
+
+void main(void) {
+ vec4 frame = texture2D(uFrame, vUv);
+ if (uEffect == 0) {
+ gl_FragColor = frame;
+ return;
+ }
+
+ float maskAlpha = uHasMask == 1 ? smoothstep(uMaskLow, uMaskHigh, featherMask(vUv)) : 0.0;
+ vec4 background = uBackgroundColor;
+
+ if (uBackgroundMode == 1) {
+ vec2 backgroundUv = vUv * uBackgroundUvTransform.xy + uBackgroundUvTransform.zw;
+ background = texture2D(uBackground, backgroundUv);
+ } else if (uBackgroundMode == 2) {
+ background = readBlurredFrame(vUv);
+ }
+
+ gl_FragColor = vec4(mix(background.rgb, frame.rgb, maskAlpha), 1.0);
+}
+`;
+
+function drawCoverImage(ctx, image, width, height) {
+ const { width: iw, height: ih } = sourceNaturalSize(image, width, height);
+ const scale = Math.max(width / iw, height / ih);
+ const dw = iw * scale;
+ const dh = ih * scale;
+ const dx = (width - dw) * 0.5;
+ const dy = (height - dh) * 0.5;
+ ctx.drawImage(image, dx, dy, dw, dh);
+}
+
+function drawContainImage(ctx, image, width, height) {
+ const { width: iw, height: ih } = sourceNaturalSize(image, width, height);
+ const scale = Math.min(width / iw, height / ih);
+ const dw = iw * scale;
+ const dh = ih * scale;
+ const dx = (width - dw) * 0.5;
+ const dy = (height - dh) * 0.5;
+ ctx.drawImage(image, dx, dy, dw, dh);
+}
+
+function resolveCoverUvTransform(image, width, height) {
+ const { width: iw, height: ih } = sourceNaturalSize(image, width, height);
+ const scale = Math.max(width / iw, height / ih);
+ const dw = iw * scale;
+ const dh = ih * scale;
+ const dx = (width - dw) * 0.5;
+ const dy = (height - dh) * 0.5;
+ return [
+ width / dw,
+ height / dh,
+ -dx / dw,
+ -dy / dh,
+ ];
+}
+
+function resolveCanvasColor(color, fallback = '#000010') {
+ const value = String(color || '').trim();
+ const cssVariable = value.match(/^var\((--[A-Za-z0-9_-]+)\)$/);
+ if (!cssVariable || typeof window === 'undefined' || typeof document === 'undefined') {
+ return value || fallback;
+ }
+ const resolved = window.getComputedStyle(document.documentElement).getPropertyValue(cssVariable[1]).trim();
+ return resolved || fallback;
+}
+
+function colorToVec4(value) {
+ const text = resolveCanvasColor(value, '#000010');
+ const match = /^#?([0-9a-f]{6})$/i.exec(text);
+ if (!match) return [0, 0, 0, 1];
+ const hex = match[1];
+ return [
+ Number.parseInt(hex.slice(0, 2), 16) / 255,
+ Number.parseInt(hex.slice(2, 4), 16) / 255,
+ Number.parseInt(hex.slice(4, 6), 16) / 255,
+ 1,
+ ];
+}
+
+async function loadImageCanvas(url) {
+ const src = String(url || '').trim();
+ if (!src) return null;
+ if (backgroundCanvasCache.has(src)) return backgroundCanvasCache.get(src);
+
+ const promise = new Promise((resolve) => {
+ const image = new Image();
+ image.decoding = 'async';
+ image.onload = async () => {
+ try {
+ await image.decode?.();
+ } catch {
+ // The onload event already confirmed the image is usable.
+ }
+ const width = Math.max(1, image.naturalWidth || image.width || 1);
+ const height = Math.max(1, image.naturalHeight || image.height || 1);
+ const imageCanvas = document.createElement('canvas');
+ imageCanvas.width = width;
+ imageCanvas.height = height;
+ const imageCtx = imageCanvas.getContext('2d', { alpha: false });
+ if (!imageCtx) {
+ resolve(null);
+ return;
+ }
+ imageCtx.drawImage(image, 0, 0, width, height);
+ resolve(imageCanvas);
+ };
+ image.onerror = () => resolve(null);
+ image.src = src;
+ });
+ backgroundCanvasCache.set(src, promise);
+ return promise;
+}
+
+function resizeCanvas(targetCanvas, width, height) {
+ const nextWidth = Math.max(1, Math.round(Number(width) || 1));
+ const nextHeight = Math.max(1, Math.round(Number(height) || 1));
+ if (targetCanvas.width !== nextWidth || targetCanvas.height !== nextHeight) {
+ targetCanvas.width = nextWidth;
+ targetCanvas.height = nextHeight;
+ return true;
+ }
+ return false;
+}
+
+function processMaskForAlpha(mask, width, height) {
+ if (!(mask instanceof Float32Array)) return mask;
+ const processed = new Float32Array(mask.length);
+ const threshold = 0.5;
+ const blurRadius = 2;
+
+ for (let i = 0; i < mask.length; i += 1) {
+ const value = Number(mask[i]) || 0;
+ processed[i] = value > threshold
+ ? Math.min(1, (value - threshold) / (1 - threshold))
+ : 0;
+ }
+
+ return blurMask(processed, width, height, blurRadius);
+}
+
+function blurMask(mask, width, height, radius) {
+ const output = new Float32Array(mask.length);
+ for (let y = 0; y < height; y += 1) {
+ for (let x = 0; x < width; x += 1) {
+ let sum = 0;
+ let count = 0;
+ for (let ky = -radius; ky <= radius; ky += 1) {
+ for (let kx = -radius; kx <= radius; kx += 1) {
+ const nx = Math.max(0, Math.min(width - 1, x + kx));
+ const ny = Math.max(0, Math.min(height - 1, y + ky));
+ sum += mask[ny * width + nx];
+ count += 1;
+ }
+ }
+ output[y * width + x] = sum / count;
+ }
+ }
+ return output;
+}
+
+function createMaskCanvasTools(canvas) {
+ const maskCanvas = document.createElement('canvas');
+ const maskLayer = maskCanvas.getContext('2d', {
+ alpha: true,
+ willReadFrequently: true,
+ });
+ const maskSourceCanvas = document.createElement('canvas');
+ const maskSourceLayer = maskSourceCanvas.getContext('2d', {
+ alpha: true,
+ willReadFrequently: true,
+ });
+ let maskSourceImageData = null;
+
+ function clearMask() {
+ if (!maskLayer) return;
+ resizeCanvas(maskCanvas, canvas.width, canvas.height);
+ maskLayer.clearRect(0, 0, maskCanvas.width, maskCanvas.height);
+ }
+
+ function drawMaskBitmap(maskBitmap) {
+ if (!(maskBitmap instanceof ImageBitmap) || !maskLayer) {
+ clearMask();
+ return false;
+ }
+
+ resizeCanvas(maskCanvas, canvas.width, canvas.height);
+ maskLayer.clearRect(0, 0, maskCanvas.width, maskCanvas.height);
+ maskLayer.imageSmoothingEnabled = true;
+ maskLayer.imageSmoothingQuality = 'high';
+ maskLayer.drawImage(maskBitmap, 0, 0, maskCanvas.width, maskCanvas.height);
+ return true;
+ }
+
+ function drawMaskValues(maskValues, maskWidth, maskHeight) {
+ if (!(maskValues instanceof Float32Array) || !maskLayer || !maskSourceLayer) {
+ clearMask();
+ return false;
+ }
+
+ const sourceWidth = Math.max(1, Math.round(Number(maskWidth) || 0));
+ const sourceHeight = Math.max(1, Math.round(Number(maskHeight) || 0));
+ const pixelCount = sourceWidth * sourceHeight;
+ if (pixelCount <= 1 || maskValues.length < pixelCount) {
+ clearMask();
+ return false;
+ }
+
+ const resizedSource = resizeCanvas(maskSourceCanvas, sourceWidth, sourceHeight);
+ if (
+ resizedSource
+ || !maskSourceImageData
+ || maskSourceImageData.width !== sourceWidth
+ || maskSourceImageData.height !== sourceHeight
+ ) {
+ maskSourceImageData = maskSourceLayer.createImageData(sourceWidth, sourceHeight);
+ }
+
+ const data = maskSourceImageData.data;
+ let maxAlpha = 0;
+ for (let pixel = 0; pixel < pixelCount; pixel += 1) {
+ const alpha = Math.max(0, Math.min(255, Math.round((Number(maskValues[pixel]) || 0) * 255)));
+ const offset = pixel * 4;
+ data[offset] = 255;
+ data[offset + 1] = 255;
+ data[offset + 2] = 255;
+ data[offset + 3] = alpha;
+ if (alpha > maxAlpha) maxAlpha = alpha;
+ }
+
+ resizeCanvas(maskCanvas, canvas.width, canvas.height);
+ maskLayer.clearRect(0, 0, maskCanvas.width, maskCanvas.height);
+ if (maxAlpha <= 0) return false;
+
+ maskSourceLayer.putImageData(maskSourceImageData, 0, 0);
+ maskLayer.imageSmoothingEnabled = true;
+ maskLayer.imageSmoothingQuality = 'high';
+ maskLayer.drawImage(maskSourceCanvas, 0, 0, maskCanvas.width, maskCanvas.height);
+ return true;
+ }
+
+ function drawDebugCanvases(source) {
+ const debugRoot = document.getElementById('backgroundPipelineDebugDialog');
+ const debugMaskCanvas = debugRoot?.querySelector?.('#maskDebug') || null;
+ const debugMaskCtx = debugMaskCanvas?.getContext?.('2d');
+ if (!debugRoot || !debugMaskCtx) return;
+
+ const width = Math.max(1, maskCanvas.width || debugMaskCanvas.width);
+ const height = Math.max(1, maskCanvas.height || debugMaskCanvas.height);
+ if (debugMaskCanvas.width !== width || debugMaskCanvas.height !== height) {
+ debugMaskCanvas.width = width;
+ debugMaskCanvas.height = height;
+ }
+
+ debugMaskCtx.clearRect(0, 0, width, height);
+ debugMaskCtx.drawImage(maskCanvas, 0, 0, width, height);
+
+ const maskImage = debugMaskCtx.getImageData(0, 0, width, height);
+ const data = maskImage.data;
+ let nonZeroPixels = 0;
+ for (let i = 0; i < data.length; i += 4) {
+ const signal = data[i + 3] ?? 0;
+ if (signal > 0) nonZeroPixels += 1;
+ data[i] = signal;
+ data[i + 1] = signal;
+ data[i + 2] = signal;
+ data[i + 3] = signal;
+ }
+ debugMaskCtx.putImageData(maskImage, 0, 0);
+
+ if (nonZeroPixels === 0) {
+ debugMaskCtx.fillStyle = 'rgba(255, 80, 80, 0.9)';
+ debugMaskCtx.font = '12px monospace';
+ debugMaskCtx.fillText('mask empty', 8, 18);
+ }
+
+ const personOnlyCanvas = debugRoot.querySelector?.('#personDebug') || null;
+ const personOnlyCtx = personOnlyCanvas?.getContext?.('2d');
+ if (!personOnlyCtx) return;
+
+ personOnlyCanvas.width = width;
+ personOnlyCanvas.height = height;
+ personOnlyCtx.clearRect(0, 0, width, height);
+ drawContainImage(personOnlyCtx, source, width, height);
+ personOnlyCtx.globalCompositeOperation = 'destination-in';
+ personOnlyCtx.drawImage(maskCanvas, 0, 0, width, height);
+ personOnlyCtx.globalCompositeOperation = 'source-over';
+ }
+
+ function getMatteMaskSnapshot() {
+ try {
+ return maskLayer?.getImageData?.(0, 0, maskCanvas.width, maskCanvas.height) || null;
+ } catch {
+ return null;
+ }
+ }
+
+ return {
+ clearMask,
+ drawDebugCanvases,
+ drawMaskBitmap,
+ drawMaskValues,
+ get maskCanvas() {
+ return maskCanvas;
+ },
+ getMatteMaskSnapshot,
+ };
+}
+
+function createCanvasBackgroundCompositorStage({
+ canvas,
+ getBackgroundColor,
+ getBackgroundImageUrl,
+ getBlurPx,
+ video,
+}) {
+ const ctx = canvas.getContext('2d', { alpha: true, desynchronized: true });
+ if (!ctx) throw new Error('2d compositor unavailable');
+
+ const maskTools = createMaskCanvasTools(canvas);
+ let backgroundImageCanvas = null;
+ let backgroundImageUrl = '';
+
+ function setBackgroundImageUrl(url) {
+ const nextUrl = String(url || '').trim();
+ if (nextUrl === backgroundImageUrl) return;
+ backgroundImageUrl = nextUrl;
+ backgroundImageCanvas = null;
+ if (!backgroundImageUrl) return;
+ loadImageCanvas(backgroundImageUrl).then((imageCanvas) => {
+ if (backgroundImageUrl !== nextUrl) return;
+ backgroundImageCanvas = imageCanvas;
+ });
+ }
+
+ function drawBackground(source, mode, backgroundColor, blurPx) {
+ ctx.save();
+ ctx.globalCompositeOperation = 'destination-over';
+ if (mode === 'replace' && backgroundImageCanvas) {
+ ctx.filter = 'none';
+ drawCoverImage(ctx, backgroundImageCanvas, canvas.width, canvas.height);
+ } else if (backgroundColor) {
+ ctx.filter = 'none';
+ ctx.fillStyle = resolveCanvasColor(backgroundColor, '#000010');
+ ctx.fillRect(0, 0, canvas.width, canvas.height);
+ } else if (mode === 'blur') {
+ ctx.filter = `blur(${blurPx}px)`;
+ drawCoverImage(ctx, source, canvas.width, canvas.height);
+ } else {
+ ctx.filter = 'none';
+ drawContainImage(ctx, source, canvas.width, canvas.height);
+ }
+ ctx.restore();
+ }
+
+ function render({
+ hasMatteMask,
+ maskBitmap = null,
+ maskHeight = 0,
+ maskUpdated = false,
+ maskValues = null,
+ maskWidth = 0,
+ mode = 'blur',
+ sourceFrame = null,
+ }) {
+ const backgroundColor = String(getBackgroundColor?.() || '').trim();
+ setBackgroundImageUrl(getBackgroundImageUrl?.() || '');
+ const blurPx = Math.max(1, Math.round(Number(getBlurPx?.() || 3)));
+ const foregroundSource = sourceFrame || video;
+
+ if (mode === 'off') {
+ ctx.save();
+ ctx.globalCompositeOperation = 'copy';
+ ctx.filter = 'none';
+ drawContainImage(ctx, video, canvas.width, canvas.height);
+ ctx.restore();
+ return;
+ }
+
+ if (hasMatteMask && !maskUpdated) return;
+
+ let hasRenderableMask = false;
+ if (maskUpdated) {
+ hasRenderableMask = maskBitmap instanceof ImageBitmap
+ ? maskTools.drawMaskBitmap(maskBitmap, maskWidth, maskHeight)
+ : maskTools.drawMaskValues(processMaskForAlpha(maskValues, maskWidth, maskHeight), maskWidth, maskHeight);
+ } else {
+ hasRenderableMask = Boolean(hasMatteMask);
+ }
+
+ maskTools.drawDebugCanvases(foregroundSource);
+
+ if (!hasRenderableMask) {
+ ctx.save();
+ ctx.globalCompositeOperation = 'copy';
+ ctx.filter = mode === 'replace' ? 'none' : `blur(${blurPx}px)`;
+ drawCoverImage(ctx, video, canvas.width, canvas.height);
+ ctx.restore();
+ return;
+ }
+
+ ctx.save();
+ ctx.globalCompositeOperation = 'copy';
+ ctx.filter = 'none';
+ drawContainImage(ctx, foregroundSource, canvas.width, canvas.height);
+ ctx.restore();
+
+ ctx.save();
+ ctx.globalCompositeOperation = 'destination-in';
+ ctx.filter = 'none';
+ ctx.drawImage(maskTools.maskCanvas, 0, 0, canvas.width, canvas.height);
+ ctx.restore();
+
+ drawBackground(foregroundSource, mode, backgroundColor, blurPx);
+ }
+
+ return {
+ backend: 'canvas',
+ getMatteMaskSnapshot: () => maskTools.getMatteMaskSnapshot(),
+ render,
+ reset: () => maskTools.clearMask(),
+ setBackgroundImageUrl,
+ };
+}
+
+function createShader(gl, type, source) {
+ const shader = gl.createShader(type);
+ gl.shaderSource(shader, source);
+ gl.compileShader(shader);
+ if (!gl.getShaderParameter(shader, gl.COMPILE_STATUS)) {
+ const message = gl.getShaderInfoLog(shader) || 'shader_compile_failed';
+ gl.deleteShader(shader);
+ throw new Error(message);
+ }
+ return shader;
+}
+
+function createProgram(gl) {
+ const vertexShader = createShader(gl, gl.VERTEX_SHADER, WEBGL_VERTEX_SHADER);
+ const fragmentShader = createShader(gl, gl.FRAGMENT_SHADER, WEBGL_FRAGMENT_SHADER);
+ const program = gl.createProgram();
+ gl.attachShader(program, vertexShader);
+ gl.attachShader(program, fragmentShader);
+ gl.linkProgram(program);
+ gl.deleteShader(vertexShader);
+ gl.deleteShader(fragmentShader);
+ if (!gl.getProgramParameter(program, gl.LINK_STATUS)) {
+ const message = gl.getProgramInfoLog(program) || 'program_link_failed';
+ gl.deleteProgram(program);
+ throw new Error(message);
+ }
+ return program;
+}
+
+function createTexture(gl) {
+ const texture = gl.createTexture();
+ gl.bindTexture(gl.TEXTURE_2D, texture);
+ gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
+ gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
+ gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
+ gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
+ return texture;
+}
+
+function uploadTexture(gl, texture, unit, source) {
+ gl.activeTexture(gl.TEXTURE0 + unit);
+ gl.bindTexture(gl.TEXTURE_2D, texture);
+ gl.pixelStorei(gl.UNPACK_FLIP_Y_WEBGL, true);
+ gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, source);
+}
+
+function createWebGlBackgroundCompositorStage({
+ canvas,
+ getBackgroundColor,
+ getBackgroundImageUrl,
+ getBlurPx,
+ video,
+}) {
+ console.log('[BackgroundFilter] Using WebGL compositor');
+ const gl = canvas.getContext('webgl2', {
+ alpha: false,
+ desynchronized: true,
+ premultipliedAlpha: false,
+ preserveDrawingBuffer: false,
+ }) || canvas.getContext('webgl', {
+ alpha: false,
+ desynchronized: true,
+ premultipliedAlpha: false,
+ preserveDrawingBuffer: false,
+ });
+ if (!gl) throw new Error('webgl compositor unavailable');
+
+ const program = createProgram(gl);
+ const locations = {
+ aPosition: gl.getAttribLocation(program, 'aPosition'),
+ uBackground: gl.getUniformLocation(program, 'uBackground'),
+ uBackgroundColor: gl.getUniformLocation(program, 'uBackgroundColor'),
+ uBackgroundMode: gl.getUniformLocation(program, 'uBackgroundMode'),
+ uBackgroundUvTransform: gl.getUniformLocation(program, 'uBackgroundUvTransform'),
+ uBlurPx: gl.getUniformLocation(program, 'uBlurPx'),
+ uEffect: gl.getUniformLocation(program, 'uEffect'),
+ uFrame: gl.getUniformLocation(program, 'uFrame'),
+ uHasMask: gl.getUniformLocation(program, 'uHasMask'),
+ uMask: gl.getUniformLocation(program, 'uMask'),
+ uMaskFeather: gl.getUniformLocation(program, 'uMaskFeather'),
+ uMaskFlipY: gl.getUniformLocation(program, 'uMaskFlipY'),
+ uMaskHigh: gl.getUniformLocation(program, 'uMaskHigh'),
+ uMaskLow: gl.getUniformLocation(program, 'uMaskLow'),
+ uMaskSize: gl.getUniformLocation(program, 'uMaskSize'),
+ uOutputSize: gl.getUniformLocation(program, 'uOutputSize'),
+ };
+ const vertexBuffer = gl.createBuffer();
+ gl.bindBuffer(gl.ARRAY_BUFFER, vertexBuffer);
+ gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([
+ -1, -1,
+ 1, -1,
+ -1, 1,
+ -1, 1,
+ 1, -1,
+ 1, 1,
+ ]), gl.STATIC_DRAW);
+
+ const textures = {
+ background: createTexture(gl),
+ frame: createTexture(gl),
+ mask: createTexture(gl),
+ };
+ const maskTools = createMaskCanvasTools(canvas);
+ let backgroundImageCanvas = null;
+ let backgroundImageUrl = '';
+ let hasUploadedMask = false;
+ let latestMaskBitmap = null;
+ let latestMaskValues = null;
+ let latestMaskWidth = 0;
+ let latestMaskHeight = 0;
+ let latestMaskFlipY = 0;
+
+ gl.useProgram(program);
+ gl.uniform1i(locations.uFrame, 0);
+ gl.uniform1i(locations.uMask, 1);
+ gl.uniform1i(locations.uBackground, 2);
+
+ function setBackgroundImageUrl(url) {
+ const nextUrl = String(url || '').trim();
+ if (nextUrl === backgroundImageUrl) return;
+ backgroundImageUrl = nextUrl;
+ backgroundImageCanvas = null;
+ if (!backgroundImageUrl) return;
+ loadImageCanvas(backgroundImageUrl).then((imageCanvas) => {
+ if (backgroundImageUrl !== nextUrl) return;
+ backgroundImageCanvas = imageCanvas;
+ if (backgroundImageCanvas && !gl.isContextLost()) {
+ uploadTexture(gl, textures.background, 2, backgroundImageCanvas);
+ }
+ });
+ }
+
+ function uploadMask({ maskBitmap, maskValues, maskWidth, maskHeight }) {
+ latestMaskBitmap = maskBitmap instanceof ImageBitmap ? maskBitmap : null;
+ latestMaskValues = maskValues instanceof Float32Array ? maskValues : null;
+ latestMaskWidth = Math.max(1, Math.round(Number(maskWidth) || latestMaskBitmap?.width || canvas.width));
+ latestMaskHeight = Math.max(1, Math.round(Number(maskHeight) || latestMaskBitmap?.height || canvas.height));
+
+ if (latestMaskBitmap) {
+ uploadTexture(gl, textures.mask, 1, latestMaskBitmap);
+ latestMaskFlipY = 1;
+ hasUploadedMask = true;
+ if (document.getElementById('backgroundPipelineDebugDialog')) {
+ maskTools.drawMaskBitmap(latestMaskBitmap, latestMaskWidth, latestMaskHeight);
+ }
+ return true;
+ }
+
+ if (latestMaskValues) {
+ const drawn = maskTools.drawMaskValues(
+ processMaskForAlpha(latestMaskValues, latestMaskWidth, latestMaskHeight),
+ latestMaskWidth,
+ latestMaskHeight,
+ );
+ if (!drawn) {
+ hasUploadedMask = false;
+ return false;
+ }
+ uploadTexture(gl, textures.mask, 1, maskTools.maskCanvas);
+ latestMaskFlipY = 0;
+ hasUploadedMask = true;
+ return true;
+ }
+
+ hasUploadedMask = false;
+ latestMaskFlipY = 0;
+ maskTools.clearMask();
+ return false;
+ }
+
+ function ensureMaskCanvasForSnapshot() {
+ if (latestMaskBitmap) {
+ maskTools.drawMaskBitmap(latestMaskBitmap, latestMaskWidth, latestMaskHeight);
+ } else if (latestMaskValues) {
+ maskTools.drawMaskValues(
+ processMaskForAlpha(latestMaskValues, latestMaskWidth, latestMaskHeight),
+ latestMaskWidth,
+ latestMaskHeight,
+ );
+ }
+ }
+
+ function render({
+ hasMatteMask,
+ maskBitmap = null,
+ maskHeight = 0,
+ maskUpdated = false,
+ maskValues = null,
+ maskWidth = 0,
+ mode = 'blur',
+ sourceFrame = null,
+ }) {
+ if (gl.isContextLost()) return;
+
+ const backgroundColor = String(getBackgroundColor?.() || '').trim();
+ setBackgroundImageUrl(getBackgroundImageUrl?.() || '');
+ const blurPx = Math.max(1, Math.round(Number(getBlurPx?.() || 3)));
+ const foregroundSource = sourceFrame || video;
+
+ if (hasMatteMask && !maskUpdated && mode !== 'off') return;
+
+ if (maskUpdated) {
+ uploadMask({ maskBitmap, maskHeight, maskValues, maskWidth });
+ }
+
+ const hasRenderableMask = hasUploadedMask && hasMatteMask;
+
+ gl.viewport(0, 0, canvas.width, canvas.height);
+ gl.useProgram(program);
+ gl.bindBuffer(gl.ARRAY_BUFFER, vertexBuffer);
+ gl.enableVertexAttribArray(locations.aPosition);
+ gl.vertexAttribPointer(locations.aPosition, 2, gl.FLOAT, false, 0, 0);
+ uploadTexture(gl, textures.frame, 0, mode === 'off' ? video : foregroundSource);
+
+ let backgroundMode = 0;
+ let backgroundUvTransform = [1, 1, 0, 0];
+ if (mode === 'replace' && !hasRenderableMask) {
+ backgroundMode = 0;
+ } else if (mode === 'replace' && backgroundImageCanvas) {
+ backgroundMode = 1;
+ backgroundUvTransform = resolveCoverUvTransform(backgroundImageCanvas, canvas.width, canvas.height);
+ uploadTexture(gl, textures.background, 2, backgroundImageCanvas);
+ } else if (mode === 'blur' && !backgroundColor) {
+ backgroundMode = 2;
+ }
+
+ gl.uniform1i(locations.uEffect, mode === 'off' || (mode === 'replace' && !hasRenderableMask) ? 0 : 1);
+ gl.uniform1i(locations.uBackgroundMode, backgroundMode);
+ gl.uniform1i(locations.uHasMask, hasRenderableMask ? 1 : 0);
+ gl.uniform1f(locations.uBlurPx, blurPx);
+ gl.uniform1f(locations.uMaskFeather, 4.35);
+ gl.uniform1f(locations.uMaskFlipY, latestMaskFlipY);
+ gl.uniform1f(locations.uMaskLow, 0.12);
+ gl.uniform1f(locations.uMaskHigh, 0.88);
+ gl.uniform2f(locations.uOutputSize, canvas.width, canvas.height);
+ gl.uniform2f(locations.uMaskSize, latestMaskWidth || canvas.width, latestMaskHeight || canvas.height);
+ gl.uniform4fv(locations.uBackgroundColor, colorToVec4(backgroundColor || '#000000'));
+ gl.uniform4fv(locations.uBackgroundUvTransform, backgroundUvTransform);
+ gl.drawArrays(gl.TRIANGLES, 0, 6);
+
+ if (document.getElementById('backgroundPipelineDebugDialog')) {
+ ensureMaskCanvasForSnapshot();
+ maskTools.drawDebugCanvases(foregroundSource);
+ }
+ }
+
+ function reset() {
+ hasUploadedMask = false;
+ latestMaskBitmap = null;
+ latestMaskValues = null;
+ latestMaskWidth = 0;
+ latestMaskHeight = 0;
+ latestMaskFlipY = 0;
+ maskTools.clearMask();
+ }
+
+ return {
+ backend: 'webgl',
+ getMatteMaskSnapshot() {
+ ensureMaskCanvasForSnapshot();
+ return maskTools.getMatteMaskSnapshot();
+ },
+ render,
+ reset,
+ setBackgroundImageUrl,
+ };
+}
export function createBackgroundCompositorStage(options = {}) {
+ console.log('[BackgroundFilter] Initializing compositor stage', options);
const preferWebGl = options.preferWebGl !== false;
if (preferWebGl) {
try {
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/background/pipeline/compositorWebglStage.js b/demo/video-chat/frontend-vue/src/domain/realtime/background/pipeline/compositorWebglStage.js
deleted file mode 100644
index dcd340957..000000000
--- a/demo/video-chat/frontend-vue/src/domain/realtime/background/pipeline/compositorWebglStage.js
+++ /dev/null
@@ -1,341 +0,0 @@
-import {
- colorToVec4,
- createMaskCanvasTools,
- loadImageCanvas,
- resolveCoverUvTransform,
-} from './compositorShared';
-
-const WEBGL_VERTEX_SHADER = `
-attribute vec2 aPosition;
-varying vec2 vUv;
-
-void main(void) {
- vUv = aPosition * 0.5 + 0.5;
- gl_Position = vec4(aPosition, 0.0, 1.0);
-}
-`;
-
-const WEBGL_FRAGMENT_SHADER = `
-precision highp float;
-
-uniform sampler2D uFrame;
-uniform sampler2D uMask;
-uniform sampler2D uBackground;
-uniform vec2 uOutputSize;
-uniform vec2 uMaskSize;
-uniform vec4 uBackgroundColor;
-uniform vec4 uBackgroundUvTransform;
-uniform float uBlurPx;
-uniform float uMaskFlipY;
-uniform int uEffect;
-uniform int uBackgroundMode;
-uniform int uHasMask;
-varying vec2 vUv;
-
-float readMask(vec2 uv) {
- vec2 maskUv = uMaskFlipY > 0.5 ? vec2(uv.x, 1.0 - uv.y) : uv;
- vec4 maskColor = texture2D(uMask, maskUv);
- return clamp(maskColor.a < 0.999 ? maskColor.a : maskColor.r, 0.0, 1.0);
-}
-
-vec4 readBlurredFrame(vec2 uv) {
- vec2 texel = vec2(max(uBlurPx, 1.0)) / uOutputSize;
- vec4 sum = texture2D(uFrame, uv) * 0.20;
- sum += texture2D(uFrame, uv + texel * vec2(-1.0, 0.0)) * 0.12;
- sum += texture2D(uFrame, uv + texel * vec2(1.0, 0.0)) * 0.12;
- sum += texture2D(uFrame, uv + texel * vec2(0.0, -1.0)) * 0.12;
- sum += texture2D(uFrame, uv + texel * vec2(0.0, 1.0)) * 0.12;
- sum += texture2D(uFrame, uv + texel * vec2(-0.707, -0.707)) * 0.08;
- sum += texture2D(uFrame, uv + texel * vec2(0.707, -0.707)) * 0.08;
- sum += texture2D(uFrame, uv + texel * vec2(-0.707, 0.707)) * 0.08;
- sum += texture2D(uFrame, uv + texel * vec2(0.707, 0.707)) * 0.08;
- return sum;
-}
-
-void main(void) {
- vec4 frame = texture2D(uFrame, vUv);
- if (uEffect == 0) {
- gl_FragColor = frame;
- return;
- }
-
- float maskAlpha = uHasMask == 1 ? readMask(vUv) : 0.0;
- vec4 background = uBackgroundColor;
-
- if (uBackgroundMode == 1) {
- vec2 backgroundUv = vUv * uBackgroundUvTransform.xy + uBackgroundUvTransform.zw;
- background = texture2D(uBackground, backgroundUv);
- } else if (uBackgroundMode == 2) {
- background = readBlurredFrame(vUv);
- }
-
- gl_FragColor = vec4(mix(background.rgb, frame.rgb, maskAlpha), 1.0);
-}
-`;
-
-function isImageBitmap(value) {
- return typeof ImageBitmap !== 'undefined' && value instanceof ImageBitmap;
-}
-
-function createShader(gl, type, source) {
- const shader = gl.createShader(type);
- gl.shaderSource(shader, source);
- gl.compileShader(shader);
- if (!gl.getShaderParameter(shader, gl.COMPILE_STATUS)) {
- const message = gl.getShaderInfoLog(shader) || 'shader_compile_failed';
- gl.deleteShader(shader);
- throw new Error(message);
- }
- return shader;
-}
-
-function createProgram(gl) {
- const vertexShader = createShader(gl, gl.VERTEX_SHADER, WEBGL_VERTEX_SHADER);
- const fragmentShader = createShader(gl, gl.FRAGMENT_SHADER, WEBGL_FRAGMENT_SHADER);
- const program = gl.createProgram();
- gl.attachShader(program, vertexShader);
- gl.attachShader(program, fragmentShader);
- gl.linkProgram(program);
- gl.deleteShader(vertexShader);
- gl.deleteShader(fragmentShader);
- if (!gl.getProgramParameter(program, gl.LINK_STATUS)) {
- const message = gl.getProgramInfoLog(program) || 'program_link_failed';
- gl.deleteProgram(program);
- throw new Error(message);
- }
- return program;
-}
-
-function createTexture(gl) {
- const texture = gl.createTexture();
- gl.bindTexture(gl.TEXTURE_2D, texture);
- gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
- gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
- gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
- gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
- return texture;
-}
-
-function uploadTexture(gl, texture, unit, source) {
- gl.activeTexture(gl.TEXTURE0 + unit);
- gl.bindTexture(gl.TEXTURE_2D, texture);
- gl.pixelStorei(gl.UNPACK_FLIP_Y_WEBGL, true);
- gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, source);
-}
-
-export function createWebGlBackgroundCompositorStage({
- canvas,
- getBackgroundColor,
- getBackgroundImageUrl,
- getBlurPx,
- getShowSourceUntilMask,
- video,
-}) {
- const gl = canvas.getContext('webgl2', {
- alpha: false,
- desynchronized: true,
- premultipliedAlpha: false,
- preserveDrawingBuffer: false,
- }) || canvas.getContext('webgl', {
- alpha: false,
- desynchronized: true,
- premultipliedAlpha: false,
- preserveDrawingBuffer: false,
- });
- if (!gl) throw new Error('webgl compositor unavailable');
-
- const program = createProgram(gl);
- const locations = {
- aPosition: gl.getAttribLocation(program, 'aPosition'),
- uBackground: gl.getUniformLocation(program, 'uBackground'),
- uBackgroundColor: gl.getUniformLocation(program, 'uBackgroundColor'),
- uBackgroundMode: gl.getUniformLocation(program, 'uBackgroundMode'),
- uBackgroundUvTransform: gl.getUniformLocation(program, 'uBackgroundUvTransform'),
- uBlurPx: gl.getUniformLocation(program, 'uBlurPx'),
- uEffect: gl.getUniformLocation(program, 'uEffect'),
- uFrame: gl.getUniformLocation(program, 'uFrame'),
- uHasMask: gl.getUniformLocation(program, 'uHasMask'),
- uMask: gl.getUniformLocation(program, 'uMask'),
- uMaskFlipY: gl.getUniformLocation(program, 'uMaskFlipY'),
- uMaskSize: gl.getUniformLocation(program, 'uMaskSize'),
- uOutputSize: gl.getUniformLocation(program, 'uOutputSize'),
- };
-
- const vertexBuffer = gl.createBuffer();
- gl.bindBuffer(gl.ARRAY_BUFFER, vertexBuffer);
- gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([
- -1, -1,
- 1, -1,
- -1, 1,
- -1, 1,
- 1, -1,
- 1, 1,
- ]), gl.STATIC_DRAW);
-
- const textures = {
- background: createTexture(gl),
- frame: createTexture(gl),
- mask: createTexture(gl),
- };
- const maskTools = createMaskCanvasTools(canvas);
- let backgroundImageCanvas = null;
- let backgroundImageUrl = '';
- let hasUploadedMask = false;
- let latestMaskBitmap = null;
- let latestMaskValues = null;
- let latestMaskWidth = 0;
- let latestMaskHeight = 0;
- let latestMaskFlipY = 0;
-
- gl.useProgram(program);
- gl.uniform1i(locations.uFrame, 0);
- gl.uniform1i(locations.uMask, 1);
- gl.uniform1i(locations.uBackground, 2);
-
- function setBackgroundImageUrl(url) {
- const nextUrl = String(url || '').trim();
- if (nextUrl === backgroundImageUrl) return;
- backgroundImageUrl = nextUrl;
- backgroundImageCanvas = null;
- if (!backgroundImageUrl) return;
- loadImageCanvas(backgroundImageUrl).then((imageCanvas) => {
- if (backgroundImageUrl !== nextUrl) return;
- backgroundImageCanvas = imageCanvas;
- if (backgroundImageCanvas && !gl.isContextLost()) {
- uploadTexture(gl, textures.background, 2, backgroundImageCanvas);
- }
- });
- }
-
- function uploadMask({ maskBitmap, maskValues, maskWidth, maskHeight }) {
- latestMaskBitmap = isImageBitmap(maskBitmap) ? maskBitmap : null;
- latestMaskValues = maskValues instanceof Float32Array ? maskValues : null;
- latestMaskWidth = Math.max(1, Math.round(Number(maskWidth) || latestMaskBitmap?.width || canvas.width));
- latestMaskHeight = Math.max(1, Math.round(Number(maskHeight) || latestMaskBitmap?.height || canvas.height));
-
- if (latestMaskBitmap) {
- const drawn = maskTools.drawMaskBitmap(latestMaskBitmap, latestMaskWidth, latestMaskHeight);
- if (!drawn) {
- hasUploadedMask = false;
- return false;
- }
- uploadTexture(gl, textures.mask, 1, maskTools.maskCanvas);
- latestMaskFlipY = 0;
- hasUploadedMask = true;
- return true;
- }
-
- if (latestMaskValues) {
- const drawn = maskTools.drawMaskValues(
- latestMaskValues,
- latestMaskWidth,
- latestMaskHeight,
- );
- if (!drawn) {
- hasUploadedMask = false;
- return false;
- }
- uploadTexture(gl, textures.mask, 1, maskTools.maskCanvas);
- latestMaskFlipY = 0;
- hasUploadedMask = true;
- return true;
- }
-
- hasUploadedMask = false;
- latestMaskFlipY = 0;
- maskTools.clearMask();
- return false;
- }
-
- function ensureMaskCanvasForSnapshot() {
- if (latestMaskBitmap) {
- maskTools.drawMaskBitmap(latestMaskBitmap, latestMaskWidth, latestMaskHeight);
- } else if (latestMaskValues) {
- maskTools.drawMaskValues(
- latestMaskValues,
- latestMaskWidth,
- latestMaskHeight,
- );
- }
- }
-
- function render({
- hasMatteMask,
- maskBitmap = null,
- maskHeight = 0,
- maskUpdated = false,
- maskValues = null,
- maskWidth = 0,
- mode = 'blur',
- sourceFrame = null,
- }) {
- if (gl.isContextLost()) return;
-
- const requestedBackgroundColor = String(getBackgroundColor?.() || '').trim();
- setBackgroundImageUrl(getBackgroundImageUrl?.() || '');
- const blurPx = Math.max(1, Math.round(Number(getBlurPx?.() || 3)));
- const foregroundSource = sourceFrame || video;
-
- if (maskUpdated) {
- uploadMask({ maskBitmap, maskHeight, maskValues, maskWidth });
- }
-
- const hasRenderableMask = hasUploadedMask && hasMatteMask;
- const showSourceUntilMask = getShowSourceUntilMask?.() === true;
- const warmupPlaceholder = !hasRenderableMask && mode === 'replace' && !showSourceUntilMask;
- const backgroundColor = warmupPlaceholder ? '#061a4a' : requestedBackgroundColor;
- gl.viewport(0, 0, canvas.width, canvas.height);
- gl.useProgram(program);
- gl.bindBuffer(gl.ARRAY_BUFFER, vertexBuffer);
- gl.enableVertexAttribArray(locations.aPosition);
- gl.vertexAttribPointer(locations.aPosition, 2, gl.FLOAT, false, 0, 0);
- uploadTexture(gl, textures.frame, 0, mode === 'off' ? video : foregroundSource);
-
- let backgroundMode = 0;
- let backgroundUvTransform = [1, 1, 0, 0];
- if (mode === 'replace' && backgroundImageCanvas && !warmupPlaceholder) {
- backgroundMode = 1;
- backgroundUvTransform = resolveCoverUvTransform(backgroundImageCanvas, canvas.width, canvas.height);
- uploadTexture(gl, textures.background, 2, backgroundImageCanvas);
- } else if (mode === 'blur' || (mode === 'replace' && !backgroundColor)) {
- backgroundMode = 2;
- }
-
- gl.uniform1i(locations.uEffect, mode === 'off' || (!hasRenderableMask && showSourceUntilMask) ? 0 : 1);
- gl.uniform1i(locations.uBackgroundMode, backgroundMode);
- gl.uniform1i(locations.uHasMask, hasRenderableMask ? 1 : 0);
- gl.uniform1f(locations.uBlurPx, Math.max(blurPx, hasRenderableMask ? 1 : 6));
- gl.uniform1f(locations.uMaskFlipY, latestMaskFlipY);
- gl.uniform2f(locations.uOutputSize, canvas.width, canvas.height);
- gl.uniform2f(locations.uMaskSize, latestMaskWidth || canvas.width, latestMaskHeight || canvas.height);
- gl.uniform4fv(locations.uBackgroundColor, colorToVec4(backgroundColor || '#000000'));
- gl.uniform4fv(locations.uBackgroundUvTransform, backgroundUvTransform);
- gl.drawArrays(gl.TRIANGLES, 0, 6);
-
- if (document.getElementById('backgroundPipelineDebugDialog')) {
- ensureMaskCanvasForSnapshot();
- maskTools.drawDebugCanvases(foregroundSource);
- }
- }
-
- function reset() {
- hasUploadedMask = false;
- latestMaskBitmap = null;
- latestMaskValues = null;
- latestMaskWidth = 0;
- latestMaskHeight = 0;
- latestMaskFlipY = 0;
- maskTools.clearMask();
- }
-
- return {
- backend: 'webgl',
- getMatteMaskSnapshot() {
- ensureMaskCanvasForSnapshot();
- return maskTools.getMatteMaskSnapshot();
- },
- render,
- reset,
- setBackgroundImageUrl,
- };
-}
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/background/staticAvatarRender.ts b/demo/video-chat/frontend-vue/src/domain/realtime/background/staticAvatarRender.ts
new file mode 100644
index 000000000..67737f562
--- /dev/null
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/background/staticAvatarRender.ts
@@ -0,0 +1,80 @@
+import {
+ BACKGROUND_FALLBACK_AVATAR_MODE,
+ normalizeBackgroundFallbackAvatarUrl,
+ normalizeBackgroundFallbackMode,
+} from './avatarFallbackSignal';
+
+export function createBackgroundStaticAvatarRenderState({
+ callMediaPrefs,
+ currentUserId,
+ peerControlStateByUserId,
+} = {}) {
+ const avatarNodesByUserId = new Map();
+
+ function stateForUserId(userId) {
+ const normalizedUserId = Number(userId || 0);
+ if (!Number.isInteger(normalizedUserId) || normalizedUserId <= 0) return null;
+
+ if (normalizedUserId === Number(currentUserId?.value || 0)) {
+ return {
+ mode: normalizeBackgroundFallbackMode(callMediaPrefs?.backgroundFallbackVideoMode),
+ imageUrl: callMediaPrefs?.backgroundFallbackAvatarImageUrl,
+ };
+ }
+
+ const peerState = peerControlStateByUserId?.[normalizedUserId];
+ if (!peerState || typeof peerState !== 'object') return null;
+ return {
+ mode: normalizeBackgroundFallbackMode(peerState.backgroundFallbackVideoMode || peerState.videoSubstitution),
+ imageUrl: peerState.backgroundFallbackAvatarImageUrl,
+ };
+ }
+
+ function staticAvatarUrlForUserId(userId) {
+ const state = stateForUserId(userId);
+ if (!state || state.mode !== BACKGROUND_FALLBACK_AVATAR_MODE) return '';
+ return normalizeBackgroundFallbackAvatarUrl(state.imageUrl);
+ }
+
+ function hasStaticAvatarForUserId(userId) {
+ return staticAvatarUrlForUserId(userId) !== '';
+ }
+
+ function staticAvatarNodeForUserId(userId) {
+ const normalizedUserId = Number(userId || 0);
+ const imageUrl = staticAvatarUrlForUserId(normalizedUserId);
+ if (!Number.isInteger(normalizedUserId) || normalizedUserId <= 0 || imageUrl === '') return null;
+ if (typeof document === 'undefined') return null;
+
+ let node = avatarNodesByUserId.get(normalizedUserId) || null;
+ if (!(node instanceof HTMLImageElement)) {
+ node = document.createElement('img');
+ node.className = 'workspace-static-avatar-media';
+ node.alt = '';
+ node.decoding = 'async';
+ node.loading = 'eager';
+ node.dataset.callStaticAvatar = '1';
+ node.dataset.userId = String(normalizedUserId);
+ avatarNodesByUserId.set(normalizedUserId, node);
+ }
+ if (node.dataset.staticAvatarSrc !== imageUrl) {
+ node.src = imageUrl;
+ node.dataset.staticAvatarSrc = imageUrl;
+ }
+ return node;
+ }
+
+ function clearStaticAvatarNodes() {
+ for (const node of avatarNodesByUserId.values()) {
+ if (node?.parentElement) node.remove();
+ }
+ avatarNodesByUserId.clear();
+ }
+
+ return {
+ clearStaticAvatarNodes,
+ hasStaticAvatarForUserId,
+ staticAvatarNodeForUserId,
+ staticAvatarUrlForUserId,
+ };
+}
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/background/stream.ts b/demo/video-chat/frontend-vue/src/domain/realtime/background/stream.ts
index a6ced7830..24494308e 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/background/stream.ts
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/background/stream.ts
@@ -1,4 +1,4 @@
-import { createSinetWasmSegmentationBackend } from './backendSinetWasm';
+import { acquireWorkerSegmenterBackendLease } from './backendWorkerSegmenter';
import { toNumber } from './math';
import { createBackgroundPipelineController } from './pipeline/controller';
import { createBackgroundCompositorStage } from './pipeline/compositorStage';
@@ -10,22 +10,6 @@ import { BACKGROUND_PIPELINE_STAGE_NAMES, BACKGROUND_PIPELINE_STAGE_STATES } fro
const LONG_RAF_FRAME_MS = 300;
const BACKGROUND_FILTER_READY_TIMEOUT_MS = 500;
-const BACKGROUND_SEGMENTER_INIT_RETRY_MS = 60000;
-const BACKGROUND_SEGMENTER_WARN_THROTTLE_MS = 10000;
-
-let sinetSegmenterUnavailableUntil = 0;
-
-function epochNowMs() {
- return Date.now();
-}
-
-function formatInitFailure(kind, error) {
- return `${kind}: ${error?.message || 'init_failed'}`;
-}
-
-function isSegmenterCoolingDown(unavailableUntil) {
- return unavailableUntil > epochNowMs();
-}
function resolveProcessingSpec(sourceWidth, sourceHeight, sourceFps, maxProcessWidth, maxProcessFps) {
const inW = Math.max(1, Math.round(toNumber(sourceWidth, 1280)));
const inH = Math.max(1, Math.round(toNumber(sourceHeight, 720)));
@@ -52,19 +36,11 @@ function normalizeBackgroundFilterRuntimeConfig(options = {}) {
backgroundImageUrl: String(options.backgroundImageUrl ?? '').trim(),
blurPx: Math.max(1, Math.min(28, Math.round(toNumber(options.blurPx, 3)))),
detectIntervalMs: Math.max(1, Math.min(1200, Math.round(toNumber(options.detectIntervalMs, 1)))),
- alphaGamma: Math.max(0.4, Math.min(2.5, toNumber(options.alphaGamma, 0.8))),
- averageRadius: Math.max(0, Math.min(12, Math.round(toNumber(options.averageRadius, 6)))),
- holeFillRadius: Math.max(0, Math.min(4, Math.round(toNumber(options.holeFillRadius, 0)))),
- maskContrast: Math.max(0.25, Math.min(4, toNumber(options.maskContrast, 0.75))),
- mattePreset: String(options.mattePreset || '').trim(),
mode,
overloadConsecutiveFrames: Math.max(3, Math.min(60, Math.round(toNumber(options.overloadConsecutiveFrames, 12)))),
overloadFrameMs: Math.max(40, Math.min(400, toNumber(options.overloadFrameMs, 90))),
sourceActive: options.sourceActive !== false,
- showSourceUntilMask: options.showSourceUntilMask === true,
statsIntervalMs: Math.max(500, Math.min(5e3, Math.round(toNumber(options.statsIntervalMs, 1e3)))),
- temporalFall: Math.max(0, Math.min(1, toNumber(options.temporalFall, 0.6))),
- temporalRise: Math.max(0, Math.min(1, toNumber(options.temporalRise, 0.7))),
};
}
async function waitForVideoReady(video) {
@@ -113,6 +89,9 @@ async function createBackgroundFilterStreamLegacy(sourceStream, options = {}) {
const maxProcessFps = toNumber(options.maxProcessFps, 24);
const onStats = typeof options.onStats === "function" ? options.onStats : null;
const onOverload = typeof options.onOverload === "function" ? options.onOverload : null;
+ const onSegmentationUnavailable = typeof options.onSegmentationUnavailable === 'function'
+ ? options.onSegmentationUnavailable
+ : null;
let disposed = false;
const video = document.createElement("video");
video.autoplay = true;
@@ -167,11 +146,9 @@ async function createBackgroundFilterStreamLegacy(sourceStream, options = {}) {
const targetFps = Math.max(8, Math.min(30, Math.round(fps)));
let segmentationBackend = null;
let segmentationBackendInitPromise = null;
+ let segmentationBackendLease = null;
let segmentationBackendKind = 'none';
- let segmentationBackendInitFailed = false;
- let segmentationBackendLastFailures = [];
- let lastSegmentationBackendWarningAt = 0;
- let nextSegmentationBackendRetryAt = 0;
+ let segmentationUnavailableNotified = false;
let handle = null;
let resolveReady = () => { };
let sourceStopReason = '';
@@ -198,8 +175,8 @@ async function createBackgroundFilterStreamLegacy(sourceStream, options = {}) {
getBackgroundColor: () => runtimeConfig.backgroundColor,
getBackgroundImageUrl: () => runtimeConfig.backgroundImageUrl,
getBlurPx: () => runtimeConfig.blurPx,
- getMattePreset: () => runtimeConfig.mattePreset,
- getShowSourceUntilMask: () => runtimeConfig.showSourceUntilMask,
+ // default to true for now. Must use runtime capability when available
+ // if not supported (for the 3% of users that can't use WebGL), compositor using canvas 2d ctx is still in place
preferWebGl: true,
video,
});
@@ -240,24 +217,39 @@ async function createBackgroundFilterStreamLegacy(sourceStream, options = {}) {
compositorStage.reset();
}
- function releaseSegmentationBackend() {
- segmentationBackend?.dispose?.();
+ function releaseSegmentationBackend({ keepWarm = true } = {}) {
+ segmentationBackendLease?.release?.({ keepWarm });
+ segmentationBackendLease = null;
segmentationBackend = null;
segmentationBackendKind = 'none';
- segmentationBackendInitFailed = false;
- segmentationBackendLastFailures = [];
- nextSegmentationBackendRetryAt = 0;
}
- function warnSegmentationBackendInitFailure(failures) {
- const nowMs = epochNowMs();
- if (nowMs - lastSegmentationBackendWarningAt < BACKGROUND_SEGMENTER_WARN_THROTTLE_MS) return;
- lastSegmentationBackendWarningAt = nowMs;
- console.warn('[BackgroundFilter] Segmentation backend failed to initialize', {
- selected: segmentationBackendKind,
- requested: 'sinet-wasm',
- failures,
- });
+ function notifySegmentationUnavailable(reason, failures = []) {
+ if (
+ segmentationUnavailableNotified
+ || runtimeConfig.mode === 'off'
+ || !runtimeConfig.sourceActive
+ || disposed
+ ) {
+ return;
+ }
+ segmentationUnavailableNotified = true;
+ if (handle) {
+ handle.active = false;
+ handle.reason = 'segmentation_unavailable';
+ handle.backend = segmentationBackendKind;
+ }
+ if (!onSegmentationUnavailable) return;
+ try {
+ onSegmentationUnavailable({
+ backend: segmentationBackendKind,
+ failures: Array.isArray(failures) ? failures : [],
+ reason: String(reason || 'segmentation_unavailable'),
+ requested: 'worker-segmenter',
+ });
+ } catch {
+ // The call must keep running even if UI diagnostics fail.
+ }
}
async function ensureSegmentationBackend() {
@@ -265,41 +257,27 @@ async function createBackgroundFilterStreamLegacy(sourceStream, options = {}) {
if (segmentationBackendInitPromise) return segmentationBackendInitPromise;
const initFailures = [];
segmentationBackendInitPromise = (async () => {
- segmentationBackendInitFailed = false;
- segmentationBackendLastFailures = [];
- nextSegmentationBackendRetryAt = 0;
- if (!isSegmenterCoolingDown(sinetSegmenterUnavailableUntil)) {
- try {
- segmentationBackend = await createSinetWasmSegmentationBackend({
- alphaGamma: runtimeConfig.alphaGamma,
- averageRadius: runtimeConfig.averageRadius,
- detectIntervalMs: runtimeConfig.detectIntervalMs,
- holeFillRadius: runtimeConfig.holeFillRadius,
- maskContrast: runtimeConfig.maskContrast,
- mattePreset: runtimeConfig.mattePreset,
- temporalFall: runtimeConfig.temporalFall,
- temporalRise: runtimeConfig.temporalRise,
- });
- if (!segmentationBackend) {
- throw new Error('SINet backend unavailable');
- }
- if (disposed || runtimeConfig.mode === 'off' || !runtimeConfig.sourceActive) {
- releaseSegmentationBackend();
- }
- } catch (error) {
- initFailures.push(formatInitFailure('sinet_wasm', error));
- sinetSegmenterUnavailableUntil = epochNowMs() + BACKGROUND_SEGMENTER_INIT_RETRY_MS;
- segmentationBackend = null;
+ try {
+ segmentationBackendLease = await acquireWorkerSegmenterBackendLease({
+ detectIntervalMs: runtimeConfig.detectIntervalMs,
+ ownerId: `background-filter-${performance.now().toFixed(3)}`,
+ });
+ segmentationBackend = segmentationBackendLease.backend;
+ if (disposed || runtimeConfig.mode === 'off' || !runtimeConfig.sourceActive) {
+ releaseSegmentationBackend({ keepWarm: true });
}
- } else {
- initFailures.push('sinet_wasm: cooling_down');
+ } catch (error) {
+ initFailures.push(`worker-segmenter: ${error?.message || 'init_failed'}`);
+ segmentationBackend = null;
}
segmentationBackendKind = segmentationBackend?.kind || 'none';
if (segmentationBackendKind === 'none' && initFailures.length > 0) {
- segmentationBackendInitFailed = true;
- segmentationBackendLastFailures = initFailures.slice();
- nextSegmentationBackendRetryAt = epochNowMs() + BACKGROUND_SEGMENTER_INIT_RETRY_MS;
- warnSegmentationBackendInitFailure(initFailures);
+ console.warn('[BackgroundFilter] Segmentation backend failed to initialize', {
+ selected: segmentationBackendKind,
+ requested: 'worker-segmenter',
+ failures: initFailures,
+ });
+ notifySegmentationUnavailable('init_failed', initFailures);
}
return segmentationBackend;
})().finally(() => {
@@ -314,22 +292,18 @@ async function createBackgroundFilterStreamLegacy(sourceStream, options = {}) {
if (handle) handle.backend = segmentationBackendKind;
}).catch((error) => {
segmentationBackendKind = 'none';
- segmentationBackendInitFailed = true;
- segmentationBackendLastFailures = [error?.message || 'init_failed'];
- nextSegmentationBackendRetryAt = epochNowMs() + BACKGROUND_SEGMENTER_INIT_RETRY_MS;
if (handle) handle.backend = 'none';
- warnSegmentationBackendInitFailure(segmentationBackendLastFailures);
+ console.warn('[BackgroundFilter] Segmentation backend failed to initialize', {
+ selected: segmentationBackendKind,
+ requested: 'worker-segmenter',
+ failures: [error?.message || 'init_failed'],
+ });
+ notifySegmentationUnavailable('init_failed', [error?.message || 'init_failed']);
});
}
function syncPipelineStageStates() {
if (!pipelineController) return;
- const segmenterRequested = runtimeConfig.sourceActive && runtimeConfig.mode !== 'off';
- const segmenterState = segmenterRequested
- ? (segmentationBackendInitFailed && !segmentationBackendInitPromise
- ? BACKGROUND_PIPELINE_STAGE_STATES.FAILED
- : BACKGROUND_PIPELINE_STAGE_STATES.RUNNING)
- : BACKGROUND_PIPELINE_STAGE_STATES.IDLE;
pipelineController.updateStage(
BACKGROUND_PIPELINE_STAGE_NAMES.SOURCE,
runtimeConfig.sourceActive ? BACKGROUND_PIPELINE_STAGE_STATES.RUNNING : BACKGROUND_PIPELINE_STAGE_STATES.PAUSED,
@@ -337,12 +311,10 @@ async function createBackgroundFilterStreamLegacy(sourceStream, options = {}) {
);
pipelineController.updateStage(
BACKGROUND_PIPELINE_STAGE_NAMES.SEGMENTER,
- segmenterState,
- {
- backend: segmentationBackendKind,
- failures: segmentationBackendLastFailures,
- mode: runtimeConfig.mode,
- },
+ runtimeConfig.sourceActive && runtimeConfig.mode !== 'off'
+ ? BACKGROUND_PIPELINE_STAGE_STATES.RUNNING
+ : BACKGROUND_PIPELINE_STAGE_STATES.IDLE,
+ { mode: runtimeConfig.mode },
);
pipelineController.updateStage(
BACKGROUND_PIPELINE_STAGE_NAMES.COMPOSITOR,
@@ -352,37 +324,13 @@ async function createBackgroundFilterStreamLegacy(sourceStream, options = {}) {
}
function applyConfigUpdate(nextOptions = {}) {
- const previousSegmentationConfig = [
- runtimeConfig.alphaGamma,
- runtimeConfig.averageRadius,
- runtimeConfig.detectIntervalMs,
- runtimeConfig.holeFillRadius,
- runtimeConfig.maskContrast,
- runtimeConfig.mattePreset,
- runtimeConfig.temporalFall,
- runtimeConfig.temporalRise,
- ].join('|');
const nextConfig = normalizeBackgroundFilterRuntimeConfig(nextOptions);
if (!Object.prototype.hasOwnProperty.call(nextOptions, 'sourceActive')) {
nextConfig.sourceActive = runtimeConfig.sourceActive;
}
- if (!Object.prototype.hasOwnProperty.call(nextOptions, 'showSourceUntilMask')) {
- nextConfig.showSourceUntilMask = runtimeConfig.showSourceUntilMask;
- }
Object.assign(runtimeConfig, nextConfig);
- const nextSegmentationConfig = [
- runtimeConfig.alphaGamma,
- runtimeConfig.averageRadius,
- runtimeConfig.detectIntervalMs,
- runtimeConfig.holeFillRadius,
- runtimeConfig.maskContrast,
- runtimeConfig.mattePreset,
- runtimeConfig.temporalFall,
- runtimeConfig.temporalRise,
- ].join('|');
- if (segmentationBackend && previousSegmentationConfig !== nextSegmentationConfig) {
- segmenterStage.reset();
- releaseSegmentationBackend();
+ if (runtimeConfig.mode !== 'off') {
+ segmentationUnavailableNotified = false;
}
if (pipelineController) {
pipelineController.emit(BACKGROUND_PIPELINE_MESSAGE_TYPES.CONFIG_UPDATE, {
@@ -393,7 +341,7 @@ async function createBackgroundFilterStreamLegacy(sourceStream, options = {}) {
}
if (runtimeConfig.mode === 'off') {
segmenterStage.reset();
- releaseSegmentationBackend();
+ releaseSegmentationBackend({ keepWarm: true });
}
syncPipelineStageStates();
}
@@ -404,7 +352,7 @@ async function createBackgroundFilterStreamLegacy(sourceStream, options = {}) {
if (!runtimeConfig.sourceActive) {
segmenterStage.reset();
compositorStage.reset();
- releaseSegmentationBackend();
+ releaseSegmentationBackend({ keepWarm: true });
}
if (pipelineController) {
pipelineController.emit(
@@ -429,15 +377,6 @@ async function createBackgroundFilterStreamLegacy(sourceStream, options = {}) {
syncCanvasToSourceFrame(vw, vh);
const now = performance.now();
const effectEnabled = runtimeConfig.mode !== 'off';
- if (
- effectEnabled
- && !segmentationBackend
- && !segmentationBackendInitPromise
- && segmentationBackendInitFailed
- && epochNowMs() >= nextSegmentationBackendRetryAt
- ) {
- warmSegmentationBackend();
- }
//const canRunSegmentation = effectEnabled && now >= overloadCooldownUntil && Boolean(segmentationBackend);
const canRunSegmentation = effectEnabled && Boolean(segmentationBackend);
const segmentationWidth = Math.max(1, Math.round(canvas.width));
@@ -456,6 +395,7 @@ async function createBackgroundFilterStreamLegacy(sourceStream, options = {}) {
underLoad,
})
: (segmenterStage.reset(), segmenterStage.getState());
+ const hasRenderableMatte = effectEnabled && Boolean(segmenterState.hasMatteMask);
compositorStage.render({
hasMatteMask: segmenterState.hasMatteMask,
maskBitmap: segmenterState.maskBitmap,
@@ -463,7 +403,7 @@ async function createBackgroundFilterStreamLegacy(sourceStream, options = {}) {
maskUpdated: segmenterState.maskUpdated,
maskValues: segmenterState.maskValues,
maskWidth: segmenterState.maskWidth,
- mode: runtimeConfig.mode,
+ mode: hasRenderableMatte ? runtimeConfig.mode : 'off',
now,
sourceFrame: segmenterState.sourceFrame,
});
@@ -545,7 +485,7 @@ async function createBackgroundFilterStreamLegacy(sourceStream, options = {}) {
} catch {
}
}
- releaseSegmentationBackend();
+ releaseSegmentationBackend({ keepWarm: true });
compositorStage.reset();
segmenterStage.reset();
pipelineController?.emit(BACKGROUND_PIPELINE_MESSAGE_TYPES.PIPELINE_STOP, {});
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/background/unavailablePrompt.ts b/demo/video-chat/frontend-vue/src/domain/realtime/background/unavailablePrompt.ts
new file mode 100644
index 000000000..8acb8ff11
--- /dev/null
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/background/unavailablePrompt.ts
@@ -0,0 +1,43 @@
+import { openBackgroundReplacementUnavailablePrompt } from '../media/preferences';
+
+function normalizeFailureList(value) {
+ return Array.isArray(value)
+ ? value.map((entry) => String(entry || '').trim()).filter(Boolean).slice(0, 5)
+ : [];
+}
+
+export function handleBackgroundReplacementUnavailable({
+ callMediaPrefs,
+ captureDiagnostic,
+ details = {},
+ refs,
+ runtimeToken,
+ state,
+}) {
+ if (runtimeToken !== state.backgroundRuntimeToken) return;
+ const failures = normalizeFailureList(details?.failures);
+ callMediaPrefs.backgroundFilterActive = false;
+ callMediaPrefs.backgroundFilterReason = 'segmentation_unavailable';
+ callMediaPrefs.backgroundFilterBackend = String(details?.backend || 'none');
+ openBackgroundReplacementUnavailablePrompt({
+ reason: String(details?.reason || 'segmentation_unavailable'),
+ failures,
+ });
+ captureDiagnostic({
+ category: 'media',
+ level: 'warning',
+ eventType: 'local_background_replacement_unavailable',
+ code: 'background_replacement_unavailable',
+ message: 'Local background replacement is unavailable; the user must choose avatar or unfiltered camera video.',
+ payload: {
+ media_runtime_path: refs.mediaRuntimePathRef.value,
+ background_filter_mode: callMediaPrefs.backgroundFilterMode,
+ background_backdrop_mode: callMediaPrefs.backgroundBackdropMode,
+ background_quality_profile: callMediaPrefs.backgroundQualityProfile,
+ requested_backend: String(details?.requested || 'worker-segmenter'),
+ selected_backend: String(details?.backend || 'none'),
+ failures,
+ },
+ immediate: true,
+ });
+}
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/background/workers/imageSegmenterWorker.js b/demo/video-chat/frontend-vue/src/domain/realtime/background/workers/imageSegmenterWorker.js
new file mode 100644
index 000000000..97c1423dd
--- /dev/null
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/background/workers/imageSegmenterWorker.js
@@ -0,0 +1,349 @@
+/**
+ * Web Worker: MediaPipe Tasks-Vision ImageSegmenter
+ *
+ * Uses selfie_multiclass_256x256.tflite with CATEGORY_MASK output to produce
+ * a MediaPipe-drawn foreground alpha mask.
+ *
+ * Protocol:
+ * IN { type: 'INIT', modelAssetPath?, delegate?, wasmPath? }
+ * OUT { type: 'INIT_DONE', labels: string[] }
+ * OUT { type: 'INIT_ERROR', error: string }
+ *
+ * IN { type: 'SEGMENT_VIDEO', bitmap: ImageBitmap, timestampMs: number }
+ * (bitmap is transferred - caller must not reuse it)
+ * OUT { type: 'SEGMENT_RESULT', maskBitmap: ImageBitmap|null, maskValues: Float32Array|null, width, height, inferenceMs }
+ * (maskBitmap or maskValues.buffer is transferred)
+ * OUT { type: 'SEGMENT_ERROR', error: string }
+ *
+ * IN { type: 'CLEANUP' }
+ * OUT { type: 'CLEANUP_DONE' }
+ */
+
+const tasksVisionModulePath = typeof TASKS_VISION_MODULE_PATH === 'string'
+ ? TASKS_VISION_MODULE_PATH
+ : '/cdn/vendor/mediapipe/tasks-vision/vision_bundle.mjs';
+
+async function importStaticModule(modulePath) {
+ const moduleUrl = new URL(modulePath, self.location.origin).href;
+ return import(
+ /* @vite-ignore */
+ moduleUrl
+ );
+}
+
+const { DrawingUtils, ImageSegmenter, FilesetResolver } = await importStaticModule(tasksVisionModulePath);
+const DEFAULT_WASM_PATH = '/wasm';
+const DEFAULT_MODEL_PATH = '/cdn/vendor/mediapipe/models/selfie_multiclass_256x256.tflite';
+
+let segmenter = null;
+let segmenterLabels = [];
+let lastTimestampMs = -1;
+let isInitializing = false;
+let renderCanvas = null;
+
+function buildCategoryAlphaColors() {
+ const colors = [];
+ colors.push([0, 0, 0, 0]);
+ for (let index = 1; index < 256; index += 1) {
+ colors.push([255, 255, 255, 255]);
+ }
+ return colors;
+}
+
+const CATEGORY_ALPHA_COLORS = buildCategoryAlphaColors();
+
+function trimTrailingSlash(value) {
+ return String(value || '').replace(/\/+$/, '');
+}
+
+function buildWasmCandidates(inputPath) {
+ const configured = trimTrailingSlash(inputPath || DEFAULT_WASM_PATH);
+ const sameOrigin = trimTrailingSlash(self.location.origin);
+ const candidates = [
+ configured,
+ `${sameOrigin}/wasm`,
+ `${sameOrigin}/cdn/vendor/mediapipe/wasm`,
+ '/wasm',
+ '/cdn/vendor/mediapipe/wasm',
+ ];
+ return Array.from(new Set(candidates.filter(Boolean)));
+}
+
+function buildModelCandidates(inputPath) {
+ const configured = String(inputPath || DEFAULT_MODEL_PATH);
+ if (/^https?:\/\//i.test(configured) || configured.startsWith('/')) {
+ return Array.from(new Set([
+ configured,
+ DEFAULT_MODEL_PATH,
+ ]));
+ }
+ return [
+ configured,
+ `/cdn/vendor/mediapipe/models/${configured.replace(/^\/+/, '')}`,
+ ];
+}
+
+async function isFetchableBinary(url) {
+ try {
+ const res = await fetch(`${url}?cb=${Date.now()}`, { method: 'GET', cache: 'no-store' });
+ if (!res.ok) return false;
+ const contentType = String(res.headers.get('content-type') || '').toLowerCase();
+ if (contentType.includes('text/html')) return false;
+ return true;
+ } catch {
+ return false;
+ }
+}
+
+async function resolveWasmPath(inputPath) {
+ const candidates = buildWasmCandidates(inputPath);
+ for (const base of candidates) {
+ const probe = `${base}/vision_wasm_internal.js`;
+ if (await isFetchableBinary(probe)) return base;
+ }
+ throw new Error(`No valid Tasks-Vision wasm path found. Tried: ${candidates.join(', ')}`);
+}
+
+async function resolveModelPath(inputPath) {
+ const candidates = buildModelCandidates(inputPath);
+ for (const modelPath of candidates) {
+ if (await isFetchableBinary(modelPath)) return modelPath;
+ }
+ throw new Error(`No valid segmentation model path found. Tried: ${candidates.join(', ')}`);
+}
+
+function stripImportQuery(value) {
+ if (typeof value !== 'string' || value === '') return value;
+ const cleaned = value
+ .replace(/[?&]import(?=(&|$))/g, '')
+ .replace(/[?&]$/, '');
+ return cleaned;
+}
+
+function sanitizeFilesetPaths(fileset) {
+ if (!fileset || typeof fileset !== 'object') return fileset;
+ const keys = Object.keys(fileset);
+ for (const key of keys) {
+ if (!/Path$/i.test(key)) continue;
+ const current = fileset[key];
+ if (typeof current !== 'string') continue;
+ fileset[key] = stripImportQuery(current);
+ }
+ return fileset;
+}
+
+function clamp01(value) {
+ return Math.max(0, Math.min(1, Number(value) || 0));
+}
+
+function confidenceMaskValues(confidenceMasks) {
+ const masks = Array.isArray(confidenceMasks) ? confidenceMasks : [];
+ const firstMask = masks[0] || null;
+ const width = Math.max(1, Math.round(Number(firstMask?.width) || 0));
+ const height = Math.max(1, Math.round(Number(firstMask?.height) || 0));
+ if (!firstMask || width <= 1 || height <= 1) return null;
+
+ const pixelCount = width * height;
+ const confidenceArrays = [];
+ for (let index = 0; index < masks.length; index += 1) {
+ const label = String(segmenterLabels[index] || '').trim().toLowerCase();
+ if (segmenterLabels.length === 0 && masks.length > 1 && index === 0) continue;
+ if (label === 'background') continue;
+ try {
+ const values = masks[index]?.getAsFloat32Array?.();
+ if (values && values.length >= pixelCount) confidenceArrays.push(values);
+ } catch {
+ // Ignore a failed class mask and keep combining the rest.
+ }
+ }
+
+ if (confidenceArrays.length === 0) return null;
+
+ const values = new Float32Array(pixelCount);
+ let maxAlpha = 0;
+ for (let pixel = 0; pixel < pixelCount; pixel += 1) {
+ let alpha = 0;
+ for (const classValues of confidenceArrays) {
+ alpha = Math.max(alpha, clamp01(classValues[pixel] || 0));
+ }
+ maxAlpha = Math.max(maxAlpha, alpha);
+ values[pixel] = alpha;
+ }
+ if (maxAlpha <= 0) return null;
+
+ return {
+ values,
+ width,
+ height,
+ };
+}
+
+function categoryMaskBitmap(categoryMask) {
+ const width = Math.max(1, Math.round(Number(categoryMask?.width) || 0));
+ const height = Math.max(1, Math.round(Number(categoryMask?.height) || 0));
+ if (!categoryMask || width <= 1 || height <= 1) return null;
+
+ try {
+ if (!renderCanvas || renderCanvas.width !== width || renderCanvas.height !== height) {
+ renderCanvas = new OffscreenCanvas(width, height);
+ }
+ renderCanvas.width = width;
+ renderCanvas.height = height;
+
+ const glCtx = renderCanvas.getContext('webgl2');
+ if (!glCtx) return null;
+
+ const drawingUtils = new DrawingUtils(glCtx);
+ drawingUtils.drawCategoryMask(categoryMask, CATEGORY_ALPHA_COLORS, [0, 0, 0, 0]);
+ const bitmap = renderCanvas.transferToImageBitmap();
+ return {
+ bitmap,
+ width,
+ height,
+ };
+ } catch {
+ return null;
+ }
+}
+
+// vision_wasm_internal.js is a classic UMD script that sets self.ModuleFactory.
+// In a type:module worker, the Tasks-Vision browser module skips this side
+// effect. We must manually fetch+eval it in global scope before
+// calling FilesetResolver.forVisionTasks(), otherwise MediaPipe throws
+// "ModuleFactory not set".
+// DO NOT EVER REMOVE THE FOLLOWING FUNCTION OR THE CALL TO IT, or the worker will fail to initialize with a very confusing error.
+async function loadModuleFactory(resolvedWasmPath) {
+ const url = `${resolvedWasmPath}/vision_wasm_internal.js`;
+ const res = await fetch(`${url}?cb=${Date.now()}`, { cache: 'no-store' });
+ if (!res.ok) throw new Error(`Failed to fetch wasm loader: ${res.status} ${url}`);
+ const src = await res.text();
+ (0, eval)(src);
+ if (typeof self.ModuleFactory !== 'function') {
+ throw new Error(`ModuleFactory not set after eval of ${url}`);
+ }
+ console.log('[Worker] ModuleFactory loaded from:', url);
+}
+
+async function initialize({ modelAssetPath, delegate, wasmPath }) {
+ if (isInitializing) return;
+ isInitializing = true;
+ try {
+ const resolvedWasm = await resolveWasmPath(wasmPath || DEFAULT_WASM_PATH);
+ const resolvedModel = await resolveModelPath(modelAssetPath || DEFAULT_MODEL_PATH);
+
+ await loadModuleFactory(resolvedWasm);
+
+ console.log('ModuleFactory before fileset:', typeof self.ModuleFactory);
+ const fileset = sanitizeFilesetPaths(await FilesetResolver.forVisionTasks(resolvedWasm));
+
+ const response = await fetch(resolvedModel);
+ if (!response.ok) {
+ throw new Error(`Failed to fetch model (${response.status}): ${resolvedModel}`);
+ }
+ const modelBuffer = await response.arrayBuffer();
+
+ if (segmenter) {
+ try { segmenter.close(); } catch { /* ignore */ }
+ segmenter = null;
+ }
+ if (!renderCanvas) {
+ renderCanvas = new OffscreenCanvas(1, 1);
+ }
+
+ segmenter = await ImageSegmenter.createFromOptions(fileset, {
+ baseOptions: {
+ modelAssetBuffer: new Uint8Array(modelBuffer),
+ delegate: delegate === 'GPU' ? 'GPU' : 'CPU',
+ },
+ canvas: renderCanvas,
+ runningMode: 'VIDEO',
+ outputCategoryMask: true,
+ outputConfidenceMasks: true,
+ });
+
+ segmenterLabels = segmenter.getLabels();
+ console.log('Segmenter initialized with labels:', segmenterLabels);
+ self.postMessage({ type: 'INIT_DONE', labels: segmenterLabels });
+ } catch (error) {
+ self.postMessage({ type: 'INIT_ERROR', error: error?.message || String(error) });
+ } finally {
+ isInitializing = false;
+ }
+}
+
+self.onmessage = async (event) => {
+ const { type } = event.data;
+
+ if (type === 'INIT') {
+ console.log('Worker received INIT message with config', event.data);
+ await initialize(event.data);
+
+ } else if (type === 'RESET') {
+ // Keep lastTimestampMs monotonic while the ImageSegmenter stays alive.
+ // MediaPipe VIDEO mode rejects lower timestamps even after our session reset.
+ self.postMessage({ type: 'RESET_DONE', sessionId: Math.max(0, Math.round(Number(event.data.sessionId) || 0)) });
+
+ } else if (type === 'SEGMENT_VIDEO' || type === 'SEGMENT_IMAGE') {
+ if (!segmenter) {
+ event.data.bitmap?.close();
+ self.postMessage({ type: 'SEGMENT_ERROR', error: 'Segmenter not initialized' });
+ return;
+ }
+
+ const { bitmap, sessionId, timestampMs } = event.data;
+ const requestSessionId = Math.max(0, Math.round(Number(sessionId) || 0));
+ const ts = timestampMs > lastTimestampMs ? timestampMs : lastTimestampMs + 1;
+ lastTimestampMs = ts;
+
+ const startMs = performance.now();
+ try {
+ segmenter.segmentForVideo(bitmap, ts, (result) => {
+ bitmap.close();
+ const inferenceTime = performance.now() - startMs;
+
+ let maskBitmap = null;
+ let maskValues = null;
+ let width = 0;
+ let height = 0;
+
+ const categoryResult = categoryMaskBitmap(result.categoryMask);
+ if (categoryResult) {
+ maskBitmap = categoryResult.bitmap;
+ width = categoryResult.width;
+ height = categoryResult.height;
+ } else {
+ const fallbackResult = confidenceMaskValues(result.confidenceMasks);
+ if (fallbackResult) {
+ maskValues = fallbackResult.values;
+ width = fallbackResult.width;
+ height = fallbackResult.height;
+ }
+ }
+
+ result.close?.();
+
+ const transfer = maskBitmap
+ ? [maskBitmap]
+ : maskValues
+ ? [maskValues.buffer]
+ : [];
+ self.postMessage(
+ { type: 'SEGMENT_RESULT', mode: 'VIDEO', maskBitmap, maskValues, width, height, inferenceTime, sessionId: requestSessionId },
+ transfer,
+ );
+ });
+ } catch (e) {
+ try { bitmap?.close(); } catch { /* ignore */ }
+ self.postMessage({ type: 'SEGMENT_ERROR', error: e?.message || String(e) });
+ }
+
+ } else if (type === 'CLEANUP') {
+ try { segmenter?.close(); } catch { /* ignore */ }
+ segmenter = null;
+ segmenterLabels = [];
+ renderCanvas = null;
+ lastTimestampMs = -1;
+ self.postMessage({ type: 'CLEANUP_DONE' });
+ }
+};
+self.postMessage({ type: 'READY' });
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppParticipantGrantButton.vue b/demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppParticipantGrantButton.vue
index 8cccfb0f4..d25794cb0 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppParticipantGrantButton.vue
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppParticipantGrantButton.vue
@@ -2,14 +2,18 @@
+ {{ buttonLabel }}
@@ -41,6 +45,11 @@ const props = defineProps({
type: Function,
required: true,
},
+ variant: {
+ type: String,
+ default: 'icon',
+ validator: (value) => ['icon', 'label'].includes(value),
+ },
});
const emit = defineEmits(['grant-updated']);
@@ -80,16 +89,21 @@ const effectiveGrantState = computed(() => {
return localState === 'allowed' || localState === 'denied' ? localState : storedGrantState.value;
});
+const variant = computed(() => (props.variant === 'label' ? 'label' : 'icon'));
const nextGrantState = computed(() => (effectiveGrantState.value === 'allowed' ? 'denied' : 'allowed'));
const buttonIcon = computed(() => (
effectiveGrantState.value === 'allowed'
- ? '/assets/orgas/kingrt/icons/add_to_call.png'
- : '/assets/orgas/kingrt/icons/remove_user.png'
+ ? '/assets/orgas/kingrt/icons/remove_user.png'
+ : '/assets/orgas/kingrt/icons/add_to_call.png'
));
+const buttonLabel = computed(() => {
+ if (pending.value) return 'Saving';
+ return effectiveGrantState.value === 'allowed' ? 'Revoke' : 'Allow';
+});
const buttonTitle = computed(() => (
- effectiveGrantState.value === 'allowed'
- ? 'Revoke Call App access'
- : 'Allow Call App access'
+ !props.canManage
+ ? 'Only the call owner or a moderator can change Call App access'
+ : `${buttonLabel.value} Call App access for ${String(props.row?.displayName || props.row?.display_name || 'participant').trim() || 'participant'}`
));
function emitGrantRealtimeUpdate(grantState) {
@@ -145,6 +159,10 @@ watch(
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppWorkspaceHost.vue b/demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppWorkspaceHost.vue
index 304277001..7d734ef36 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppWorkspaceHost.vue
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppWorkspaceHost.vue
@@ -224,7 +224,8 @@ const accessNoticeLabel = computed(() => {
}
.call-app-workspace-mini-video-slot :deep(video),
-.call-app-workspace-mini-video-slot :deep(canvas) {
+.call-app-workspace-mini-video-slot :deep(canvas),
+.call-app-workspace-mini-video-slot :deep(.workspace-static-avatar-media) {
position: absolute;
inset: 0;
width: 100% !important;
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppsSidebarPanel.css b/demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppsSidebarPanel.css
new file mode 100644
index 000000000..6b473267e
--- /dev/null
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppsSidebarPanel.css
@@ -0,0 +1,379 @@
+.call-apps-sidebar {
+ min-height: 0;
+ overflow-y: auto;
+ overflow-x: hidden;
+ padding: 0 0 56px;
+ display: grid;
+ gap: 1px;
+ align-content: start;
+ container-type: inline-size;
+ direction: rtl;
+}
+
+.call-apps-sidebar > * {
+ direction: var(--app-content-direction);
+}
+
+.call-apps-search {
+ display: flex;
+ flex-direction: row-reverse;
+ align-items: center;
+ justify-content: flex-start;
+ gap: clamp(10px, 4cqi, 20px);
+ padding: clamp(12px, 5cqi, 20px);
+ background: var(--bg-surface-strong);
+}
+
+.call-apps-search-input {
+ height: 34px;
+ min-width: 0;
+ flex: 1 1 auto;
+ background: var(--border-subtle);
+}
+
+.call-apps-search-submit {
+ width: 34px;
+ height: 34px;
+ flex: 0 0 34px;
+}
+
+.call-apps-search-submit img {
+ width: 16px;
+ height: 16px;
+ filter: var(--action-icon-filter);
+}
+
+.call-apps-picker {
+ display: grid;
+ gap: 6px;
+ padding: 10px;
+ background: var(--bg-surface-strong);
+}
+
+.call-apps-picker span {
+ font-size: 11px;
+ color: var(--text-muted);
+}
+
+.call-apps-list {
+ display: grid;
+ gap: 1px;
+ min-height: 0;
+}
+
+.call-apps-list.loading {
+ opacity: 0.7;
+}
+
+.call-apps-list-item {
+ width: 100%;
+ border: 0;
+ background: var(--bg-surface-strong);
+ color: var(--text-primary);
+ min-height: 82px;
+ padding: 12px clamp(12px, 5cqi, 20px);
+ display: grid;
+ grid-template-columns: minmax(0, 1fr);
+ align-items: center;
+ gap: 10px;
+ text-align: left;
+ cursor: pointer;
+}
+
+.call-apps-list-item:hover,
+.call-apps-list-item.active {
+ background: var(--color-border);
+}
+
+.call-apps-item-main {
+ min-width: 0;
+ display: grid;
+ gap: 6px;
+}
+
+.call-apps-item-name {
+ font-size: 13px;
+ font-weight: 800;
+ color: var(--text-primary);
+ overflow: hidden;
+ text-overflow: ellipsis;
+ white-space: nowrap;
+}
+
+.call-apps-item-meta,
+.call-apps-item-side,
+.call-apps-item-state,
+.call-apps-hint {
+ font-size: 11px;
+ color: var(--text-muted);
+}
+
+.call-apps-item-side {
+ min-width: 0;
+ display: flex;
+ flex-wrap: wrap;
+ align-items: center;
+ gap: 6px;
+}
+
+.call-apps-item-state {
+ text-transform: uppercase;
+ letter-spacing: 0;
+ min-width: 0;
+ overflow-wrap: anywhere;
+}
+
+.call-apps-item-action,
+.call-apps-access-default {
+ display: inline-flex;
+ width: fit-content;
+ min-height: 22px;
+ align-items: center;
+ padding: 3px 8px;
+ border: 1px solid var(--color-border);
+ background: var(--color-primary-navy);
+ color: var(--color-heading);
+ font-size: 10px;
+ font-weight: 900;
+ line-height: 12px;
+ text-transform: uppercase;
+}
+
+.call-apps-item-badges {
+ min-width: 0;
+ display: flex;
+ flex-wrap: wrap;
+ gap: 6px;
+}
+
+.call-apps-status-badge {
+ min-height: 20px;
+ padding: 3px 7px;
+ border: 1px solid var(--color-border);
+ background: var(--color-primary-navy);
+ color: var(--color-heading);
+ font-size: 10px;
+ font-weight: 900;
+ line-height: 12px;
+ text-transform: uppercase;
+}
+
+.call-apps-status-badge.state-installed,
+.call-apps-status-badge.state-enabled,
+.call-apps-status-badge.state-healthy {
+ color: var(--color-success);
+}
+
+.call-apps-status-badge.state-disabled,
+.call-apps-status-badge.state-unhealthy {
+ color: var(--color-error);
+}
+
+.call-apps-pagination {
+ display: flex;
+ flex-direction: row;
+ justify-content: flex-end;
+ align-items: center;
+ flex-wrap: wrap;
+ gap: clamp(10px, 4cqi, 20px);
+ background: var(--bg-surface-strong);
+ padding: clamp(12px, 5cqi, 20px);
+}
+
+.call-apps-detail,
+.call-apps-access {
+ background: var(--bg-surface-strong);
+ padding: clamp(12px, 5cqi, 20px);
+ display: grid;
+ gap: clamp(12px, 5cqi, 20px);
+}
+
+.call-apps-detail-head,
+.call-apps-access-head {
+ display: grid;
+ gap: 4px;
+}
+
+.call-apps-detail-head h2,
+.call-apps-access-head h2 {
+ margin: 0;
+ font-size: 14px;
+ line-height: 18px;
+ color: var(--text-primary);
+}
+
+.call-apps-detail-head span,
+.call-apps-access-head span {
+ font-size: 12px;
+ color: var(--text-muted);
+ min-width: 0;
+ overflow-wrap: anywhere;
+}
+
+.call-apps-detail-grid {
+ margin: 0;
+ display: grid;
+ grid-template-columns: minmax(0, 1fr);
+ gap: 8px;
+}
+
+.call-apps-detail-grid div,
+.call-apps-policy {
+ display: grid;
+ gap: 4px;
+}
+
+.call-apps-detail-grid dt,
+.call-apps-policy legend {
+ font-size: 11px;
+ color: var(--text-muted);
+}
+
+.call-apps-detail-grid dd {
+ margin: 0;
+ font-size: 12px;
+ color: var(--text-primary);
+ min-width: 0;
+ overflow: hidden;
+ text-overflow: ellipsis;
+}
+
+.call-apps-policy {
+ min-width: 0;
+ margin: 0;
+ padding: 0;
+ border: 0;
+ gap: 8px;
+}
+
+.call-apps-policy-options {
+ min-width: 0;
+ display: grid;
+ grid-template-columns: minmax(0, 1fr);
+ gap: 8px;
+}
+
+.call-apps-policy-choice {
+ min-width: 0;
+ min-height: 54px;
+ display: grid;
+ grid-template-columns: 18px minmax(0, 1fr);
+ grid-template-rows: auto auto;
+ column-gap: 8px;
+ align-items: center;
+ padding: 8px;
+ border: 1px solid var(--color-border);
+ background: var(--color-primary-navy);
+}
+
+.call-apps-policy-choice.active {
+ border-color: var(--color-cyan-primary);
+}
+
+.call-apps-policy-choice input {
+ grid-row: 1 / span 2;
+}
+
+.call-apps-policy-choice span {
+ min-width: 0;
+ color: var(--text-primary);
+ font-size: 12px;
+ font-weight: 900;
+}
+
+.call-apps-policy-choice small {
+ min-width: 0;
+ color: var(--text-muted);
+ font-size: 11px;
+ line-height: 14px;
+}
+
+.call-apps-empty,
+.call-apps-error,
+.call-apps-notice {
+ padding: 12px 10px;
+ font-size: 12px;
+ background: var(--bg-surface-strong);
+}
+
+.call-apps-empty {
+ color: var(--text-muted);
+}
+
+.call-apps-error {
+ color: var(--color-error);
+}
+
+.call-apps-notice {
+ color: var(--color-success);
+}
+
+.call-apps-access-list {
+ display: grid;
+ gap: 8px;
+}
+
+.call-apps-access-row {
+ min-width: 0;
+ display: grid;
+ grid-template-columns: minmax(0, 1fr);
+ align-items: center;
+ gap: 8px;
+ padding-block: 4px;
+}
+
+.call-apps-access-main {
+ min-width: 0;
+ display: grid;
+ gap: 2px;
+}
+
+.call-apps-access-name {
+ min-width: 0;
+ overflow: hidden;
+ text-overflow: ellipsis;
+ white-space: nowrap;
+ font-size: 12px;
+ font-weight: 800;
+ color: var(--text-primary);
+}
+
+.call-apps-access-state {
+ display: inline-flex;
+ width: fit-content;
+ min-height: 20px;
+ align-items: center;
+ padding: 3px 7px;
+ border: 1px solid var(--color-border);
+ background: var(--color-primary-navy);
+ font-size: 11px;
+ font-weight: 900;
+ text-transform: uppercase;
+ color: var(--text-muted);
+}
+
+.call-apps-access-state.state-allowed {
+ color: var(--color-success);
+}
+
+.call-apps-access-state.state-denied {
+ color: var(--color-warning);
+}
+
+@container (min-width: 380px) {
+ .call-apps-list-item {
+ grid-template-columns: minmax(0, 1fr) auto;
+ }
+
+ .call-apps-detail-grid {
+ grid-template-columns: minmax(0, 1fr) minmax(0, 1fr);
+ }
+
+ .call-apps-policy-options {
+ grid-template-columns: minmax(0, 1fr) minmax(0, 1fr);
+ }
+
+ .call-apps-access-row {
+ grid-template-columns: minmax(0, 1fr) auto;
+ }
+}
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppsSidebarPanel.vue b/demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppsSidebarPanel.vue
index 43d8f84dc..e92b8129e 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppsSidebarPanel.vue
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppsSidebarPanel.vue
@@ -62,7 +62,10 @@
{{ healthStatusLabel(app) }}
- {{ app.app_key }}
+
+ {{ app.app_key }}
+ {{ selectedAppKey === app.app_key ? 'Selected' : 'Select' }}
+
@@ -115,13 +118,21 @@
{{ healthStatusLabel(selectedApp) }}
-
- Default participant access
-
- Blocked by default
- Allowed by default
-
-
+
+ Default participant access
+
+
+
+ Blocked
+ Grant individually
+
+
+
+ Allowed
+ Participants can open
+
+
+
Access
{{ activeSessionName }}
+ {{ activeSessionDefaultAccessLabel }}
{{ participant.displayName }}
- {{ grantStateLabel(participant) }}
+ {{ grantStateLabel(participant) }}
+
+ No call participants are available for this Call App session.
+
@@ -296,6 +312,10 @@ function defaultGrantState() {
return String(activeSessionForAccess.value?.default_app_policy || '') === 'allowed_by_default' ? 'allowed' : 'denied';
}
+const activeSessionDefaultAccessLabel = computed(() => (
+ defaultGrantState() === 'allowed' ? 'Default: allowed' : 'Default: blocked'
+));
+
function grantStateForParticipant(participant) {
const userId = Number(participant?.userId || participant?.user_id || 0);
const sessionId = String(activeSessionForAccess.value?.id || '').trim();
@@ -314,6 +334,10 @@ function grantStateLabel(participant) {
return grantStateForParticipant(participant) === 'allowed' ? 'Allowed' : 'Blocked';
}
+function grantStateClass(participant) {
+ return grantStateForParticipant(participant) === 'allowed' ? 'state-allowed' : 'state-denied';
+}
+
function applyLocalGrantUpdate(event) {
const sessionId = String(event?.sessionId || activeSessionForAccess.value?.id || '').trim();
const userId = Number(event?.userId || 0);
@@ -420,283 +444,4 @@ watch(
);
-
+
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/callApps/callAppPresenceRelay.js b/demo/video-chat/frontend-vue/src/domain/realtime/callApps/callAppPresenceRelay.js
index 07796976b..96b7a2740 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/callApps/callAppPresenceRelay.js
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/callApps/callAppPresenceRelay.js
@@ -22,13 +22,39 @@ export function normalizeCallAppPresenceDisplayName(value) {
return displayName.slice(0, 80);
}
-export function normalizeCallAppPresenceParticipantRows(rows, currentUserId = 0) {
+function normalizeCallAppPresenceGrantState(value) {
+ const state = plainString(value).toLowerCase();
+ return state === 'allowed' || state === 'denied' ? state : '';
+}
+
+function defaultGrantStateForSession(session = {}) {
+ return plainString(session?.default_app_policy || session?.defaultAppPolicy).toLowerCase() === 'allowed_by_default'
+ ? 'allowed'
+ : 'denied';
+}
+
+export function callAppPresenceUserAuthorizedForSession(session = {}, userId = 0) {
+ const normalizedUserId = Number(userId || 0);
+ if (!Number.isInteger(normalizedUserId) || normalizedUserId <= 0) return false;
+
+ const grants = Array.isArray(session?.grants) ? session.grants : [];
+ const grant = grants.find((row) => (
+ plainString(row?.subject_type).toLowerCase() === 'user'
+ && Number(row?.user_id || row?.userId || 0) === normalizedUserId
+ ));
+ const explicitState = normalizeCallAppPresenceGrantState(grant?.grant_state || grant?.grantState);
+ return (explicitState || defaultGrantStateForSession(session)) === 'allowed';
+}
+
+export function normalizeCallAppPresenceParticipantRows(rows, currentUserId = 0, session = null) {
const localUserId = Number(currentUserId || 0);
const seen = new Set();
const participants = [];
for (const row of Array.isArray(rows) ? rows : []) {
const userId = Number(row?.userId || row?.user_id || 0);
if (!Number.isInteger(userId) || userId <= 0 || userId === localUserId || seen.has(userId)) continue;
+ if (row?.isRoomMember === false || row?.is_room_member === false) continue;
+ if (session && !callAppPresenceUserAuthorizedForSession(session, userId)) continue;
seen.add(userId);
participants.push({
userId,
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/callApps/callAppWorkspaceState.js b/demo/video-chat/frontend-vue/src/domain/realtime/callApps/callAppWorkspaceState.js
index 748333fa5..5151b0050 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/callApps/callAppWorkspaceState.js
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/callApps/callAppWorkspaceState.js
@@ -2,7 +2,49 @@ import { computed, ref } from 'vue';
export const CALL_APP_WORKSPACE_LAYOUT_MODE = 'call_app_workspace';
export const CALL_APP_WORKSPACE_MINI_LIMIT = 5;
-const CALL_APP_IFRAME_ORIGIN = String(import.meta.env.VITE_VIDEOCHAT_CALL_APP_ORIGIN || '').trim().replace(/\/+$/, '');
+const CALL_APP_IFRAME_ORIGIN = normalizeConfiguredCallAppOrigin(import.meta.env.VITE_VIDEOCHAT_CALL_APP_ORIGIN);
+
+function normalizeConfiguredCallAppOrigin(value) {
+ const trimmed = String(value || '').trim().replace(/\/+$/, '');
+ if (trimmed === '') return '';
+ const withScheme = /^[a-z][a-z0-9+.-]*:\/\//i.test(trimmed) ? trimmed : `https://${trimmed}`;
+ try {
+ const parsed = new URL(withScheme);
+ parsed.pathname = '';
+ parsed.search = '';
+ parsed.hash = '';
+ return parsed.toString().replace(/\/+$/, '');
+ } catch {
+ return '';
+ }
+}
+
+function callAppOriginForAppKey(appKey) {
+ if (CALL_APP_IFRAME_ORIGIN === '') return '';
+ const hostAppKey = String(appKey || '')
+ .trim()
+ .toLowerCase()
+ .replace(/[^a-z0-9-]+/g, '-')
+ .replace(/^-+|-+$/g, '');
+ if (hostAppKey === '') return CALL_APP_IFRAME_ORIGIN;
+
+ try {
+ const parsed = new URL(CALL_APP_IFRAME_ORIGIN);
+ const parts = parsed.hostname.split('.');
+ if (parts.length >= 3 && ['app', 'apps', 'whiteboard'].includes(parts[0])) {
+ parts[0] = hostAppKey;
+ parsed.hostname = parts.join('.');
+ parsed.pathname = '';
+ parsed.search = '';
+ parsed.hash = '';
+ return parsed.toString().replace(/\/+$/, '');
+ }
+ } catch {
+ return CALL_APP_IFRAME_ORIGIN;
+ }
+
+ return CALL_APP_IFRAME_ORIGIN;
+}
function normalizeSession(raw = {}) {
const session = raw && typeof raw === 'object' ? raw : {};
@@ -64,7 +106,8 @@ export function callAppWorkspaceIframeUrl(session) {
.join('/');
if (appKey === '' || entrypoint === '') return 'about:blank';
const path = `/call-app/${encodeURIComponent(appKey)}/${entrypoint}`;
- return CALL_APP_IFRAME_ORIGIN !== '' ? `${CALL_APP_IFRAME_ORIGIN}${path}` : path;
+ const origin = callAppOriginForAppKey(appKey);
+ return origin !== '' ? `${origin}${path}` : path;
}
export function createCallAppWorkspaceState({
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/callApps/useCallAppCrdtBridge.js b/demo/video-chat/frontend-vue/src/domain/realtime/callApps/useCallAppCrdtBridge.js
index 853c45150..d7fe2d0a2 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/callApps/useCallAppCrdtBridge.js
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/callApps/useCallAppCrdtBridge.js
@@ -2,7 +2,7 @@ import { onBeforeUnmount } from 'vue';
import {
CALL_APP_IFRAME_BRIDGE_PROTOCOL,
CALL_APP_IFRAME_OPAQUE_ORIGIN,
- sanitizeCallAppBridgePayload,
+ cloneSafeCallAppBridgePayload,
} from './useCallAppIframeBridge.js';
import {
callAppDiagnosticElapsedMs,
@@ -13,6 +13,7 @@ import {
import {
CALL_APP_PRESENCE_SIGNAL_TYPE,
CALL_APP_PRESENCE_WINDOW_EVENT,
+ callAppPresenceUserAuthorizedForSession,
createCallAppPresenceSignalPayload,
normalizeCallAppPresenceDisplayName,
normalizeCallAppPresenceParticipantRows,
@@ -24,13 +25,15 @@ import {
function postToIframe(frameWindow, session, type, payload = {}) {
if (!frameWindow || !session) return;
try {
- frameWindow.postMessage(sanitizeCallAppBridgePayload({
+ const message = cloneSafeCallAppBridgePayload({
type,
bridge_protocol: CALL_APP_IFRAME_BRIDGE_PROTOCOL,
app_session_id: String(session?.id || '').trim(),
app_key: String(session?.app_key || '').trim(),
...payload,
- }), '*');
+ });
+ if (!message) return;
+ frameWindow.postMessage(message, '*');
} catch {
// A bridge message must never break the parent call runtime.
}
@@ -172,6 +175,7 @@ export function createCallAppCrdtBridge({
const targetParticipants = normalizeCallAppPresenceParticipantRows(
unrefValue(participants),
Number(unrefValue(currentUserId) || 0),
+ session,
);
let sentCount = 0;
for (const participant of targetParticipants) {
@@ -196,16 +200,18 @@ export function createCallAppCrdtBridge({
function handlePresencePublish(frameWindow, session, message) {
const payloadType = normalizeCallAppPresencePayloadType(message?.payload_type);
const displayName = normalizeCallAppPresenceDisplayName(unrefValue(currentUserDisplayName));
+ const senderAuthorized = callAppPresenceUserAuthorizedForSession(session, Number(unrefValue(currentUserId) || 0));
const payload = normalizeCallAppPresencePayload(payloadType, message?.payload || {}, {
actorId: message?.actor_id || message?.payload?.actor_id,
displayName,
});
- const sentCount = payloadType !== '' && payload ? sendPresenceToPeers(session, payloadType, payload) : 0;
+ const accepted = senderAuthorized && payloadType !== '' && Boolean(payload);
+ const sentCount = accepted ? sendPresenceToPeers(session, payloadType, payload) : 0;
postToIframe(frameWindow, session, 'call_app.presence.published', {
request_id: requestId(message),
result: {
- ok: payloadType !== '' && Boolean(payload),
- state: payloadType !== '' && payload ? 'accepted' : 'ignored',
+ ok: accepted,
+ state: senderAuthorized ? (accepted ? 'accepted' : 'ignored') : 'participant_grant_denied',
persisted: false,
payload_type: payloadType,
sent_count: sentCount,
@@ -217,6 +223,7 @@ export function createCallAppCrdtBridge({
const frameWindow = iframeRef?.value?.contentWindow || null;
const session = activeSession?.value || null;
if (!frameWindow || !session) return;
+ if (!callAppPresenceUserAuthorizedForSession(session, Number(unrefValue(currentUserId) || 0))) return;
const signal = normalizeRemoteCallAppPresenceSignal(event?.detail?.signal || event?.detail?.payload || {});
if (!signal) return;
if (String(signal.app_session_id || '').trim() !== String(session.id || '').trim()) return;
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/callApps/useCallAppIframeBridge.js b/demo/video-chat/frontend-vue/src/domain/realtime/callApps/useCallAppIframeBridge.js
index f53699d4b..9a2f29f64 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/callApps/useCallAppIframeBridge.js
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/callApps/useCallAppIframeBridge.js
@@ -1,35 +1,73 @@
-import { computed, onBeforeUnmount, ref, watch } from 'vue';
+import { computed, isProxy, isRef, onBeforeUnmount, ref, toRaw, unref, watch } from 'vue';
import { emitCallAppDiagnostic } from './callAppDiagnostics.js';
export const CALL_APP_IFRAME_BRIDGE_PROTOCOL = 'king.call_app.iframe.v1';
export const CALL_APP_IFRAME_OPAQUE_ORIGIN = 'null';
-export function sanitizeCallAppBridgePayload(value, depth = 0) {
+function rawBridgeValue(value) {
+ if (isRef(value)) return rawBridgeValue(unref(value));
+ return isProxy(value) ? toRaw(value) : value;
+}
+
+function plainBridgeObject(value) {
+ const prototype = Object.getPrototypeOf(value);
+ return prototype === Object.prototype || prototype === null;
+}
+
+export function sanitizeCallAppBridgePayload(value, depth = 0, seen = new WeakSet()) {
if (depth > 8) return null;
if (value === null || value === undefined) return null;
+ const rawValue = rawBridgeValue(value);
+ if (rawValue !== value) return sanitizeCallAppBridgePayload(rawValue, depth, seen);
const valueType = typeof value;
if (valueType === 'string' || valueType === 'boolean') return value;
if (valueType === 'number') return Number.isFinite(value) ? value : 0;
if (valueType === 'bigint') return value.toString();
if (valueType !== 'object') return null;
if (value instanceof Date) return value.toISOString();
+ if (value instanceof URL) return value.toString();
+ if (value instanceof Error) return { name: value.name, message: value.message };
+ if (ArrayBuffer.isView(value)) return Array.from(value);
+ if (value instanceof ArrayBuffer) return Array.from(new Uint8Array(value));
+ if (seen.has(value)) return null;
+ seen.add(value);
if (Array.isArray(value)) {
- return value.map((item) => sanitizeCallAppBridgePayload(item, depth + 1));
+ return Array.from(value, (item) => sanitizeCallAppBridgePayload(item, depth + 1, seen));
}
+ if (!plainBridgeObject(value)) return null;
const out = {};
for (const key of Object.keys(value)) {
const normalizedKey = String(key || '').trim();
if (normalizedKey === '') continue;
+ if (['__proto__', 'constructor', 'prototype'].includes(normalizedKey)) continue;
try {
- out[normalizedKey] = sanitizeCallAppBridgePayload(value[key], depth + 1);
+ out[normalizedKey] = sanitizeCallAppBridgePayload(value[key], depth + 1, seen);
} catch {
out[normalizedKey] = null;
}
}
+ seen.delete(value);
return out;
}
+export function cloneSafeCallAppBridgePayload(value) {
+ const sanitized = sanitizeCallAppBridgePayload(value);
+ if (typeof structuredClone === 'function') {
+ try {
+ structuredClone(sanitized);
+ return sanitized;
+ } catch {
+ // Fall through to JSON cloning, which preserves the bridge's JSON-only contract.
+ }
+ }
+ try {
+ return JSON.parse(JSON.stringify(sanitized));
+ } catch {
+ return null;
+ }
+}
+
function normalizeLaunchResponse(payload) {
const result = payload?.result && typeof payload.result === 'object' ? payload.result : {};
const context = result.context && typeof result.context === 'object' ? result.context : {};
@@ -103,6 +141,7 @@ export function createCallAppIframeBridge({
const status = ref('idle');
const error = ref('');
let generation = 0;
+ let failedPostGeneration = -1;
const sessionId = computed(() => String(activeSession?.value?.id || '').trim());
const appKey = computed(() => String(activeSession?.value?.app_key || '').trim());
@@ -117,6 +156,7 @@ export function createCallAppIframeBridge({
function resetLaunchState(nextStatus = 'idle') {
generation += 1;
+ failedPostGeneration = -1;
launch.value = null;
status.value = nextStatus;
error.value = '';
@@ -126,15 +166,30 @@ export function createCallAppIframeBridge({
const frameWindow = iframeRef?.value?.contentWindow || null;
const session = activeSession?.value || null;
if (!frameWindow || !session || !launch.value?.token) return false;
+ if (failedPostGeneration === generation) return false;
try {
+ const message = cloneSafeCallAppBridgePayload(
+ safePostMessagePayload(session, launch.value, { participantDisplayName }),
+ );
+ if (!message) {
+ throw new Error('Call App launch message could not be serialized.');
+ }
frameWindow.postMessage(
- sanitizeCallAppBridgePayload(safePostMessagePayload(session, launch.value, { participantDisplayName })),
+ message,
'*',
);
} catch (postError) {
+ failedPostGeneration = generation;
status.value = 'error';
error.value = postError instanceof Error ? postError.message : 'Call App launch message could not be sent.';
+ emitCallAppDiagnostic('call_app_iframe_bridge_error', {
+ session_id: sessionId.value,
+ app_key: appKey.value,
+ iframe_message_type: 'call_app.launch',
+ reason: 'post_message_failed',
+ message: error.value,
+ });
return false;
}
status.value = 'launch_sent';
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/local/localStreamLifecycle.ts b/demo/video-chat/frontend-vue/src/domain/realtime/local/localStreamLifecycle.ts
index 29687f10a..1d053d48a 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/local/localStreamLifecycle.ts
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/local/localStreamLifecycle.ts
@@ -22,9 +22,12 @@ export function unpublishSfuTracksForClient(sfuClient, tracks) {
}
}
-export function stopRetiredLocalStreams(retiredStreams, preservedStreams = []) {
+export function stopRetiredLocalStreams(retiredStreams, preservedStreams = [], options = {}) {
const preserved = new Set();
const preservedTrackIds = new Set();
+ const protectedTrackIds = new Set(Array.isArray(options?.protectedTrackIds) ? options.protectedTrackIds : []);
+ const reason = String(options?.reason || 'retired_local_stream_cleanup');
+ const captureDiagnostic = typeof options?.captureDiagnostic === 'function' ? options.captureDiagnostic : null;
for (const stream of preservedStreams) {
if (stream instanceof MediaStream) {
preserved.add(stream);
@@ -39,6 +42,22 @@ export function stopRetiredLocalStreams(retiredStreams, preservedStreams = []) {
if (preserved.has(stream)) continue;
for (const track of stream.getTracks()) {
if (track?.id && preservedTrackIds.has(track.id)) continue;
+ if (track?.id && protectedTrackIds.has(track.id)) {
+ captureDiagnostic?.({
+ category: 'media',
+ level: 'info',
+ eventType: 'local_media_cleanup_preserved_active_track',
+ code: 'local_media_cleanup_preserved_active_track',
+ message: 'Stale local media cleanup preserved an active camera, microphone, or screen-share track.',
+ payload: {
+ reason,
+ track_id: track.id,
+ track_kind: String(track.kind || ''),
+ media_runtime_path: String(options?.mediaRuntimePath || ''),
+ },
+ });
+ continue;
+ }
try {
track.stop();
} catch {
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/local/mediaOrchestration.ts b/demo/video-chat/frontend-vue/src/domain/realtime/local/mediaOrchestration.ts
index a2c074064..f1c0d5fd9 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/local/mediaOrchestration.ts
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/local/mediaOrchestration.ts
@@ -8,8 +8,11 @@ import {
} from '../media/speakerOutputRouting';
import { applySfuVideoProfileConstraintsToStream, reportSfuLocalCaptureSettings } from './sfuCaptureProfileConstraints';
import { isLocalMediaPermissionDeniedError, LOCAL_MEDIA_PERMISSION_DENIED_RETRY_COOLDOWN_MS } from './localMediaPermissionPolicy';
+import { createBackgroundFallbackAudioOnlyStream } from '../background/avatarFallbackSignal';
+import { handleBackgroundReplacementUnavailable } from '../background/unavailablePrompt';
import { shouldUseReactiveBackgroundPipeline } from '../background/pipeline/featureFlags';
import { buildDisplayMediaOptions, hasGetDisplayMedia, normalizeDisplayMediaError } from './screenShareCapture';
+
export function createLocalMediaOrchestrationHelpers({
backgroundBaselineCollector,
backgroundFilterController,
@@ -34,6 +37,7 @@ export function createLocalMediaOrchestrationHelpers({
shouldSyncNativeLocalTracksBeforeOffer,
syncNativePeerConnectionsWithRoster,
syncNativePeerLocalTracks,
+ syncControlStateToPeers = () => 0,
sendNativeOffer,
} = callbacks;
const captureDiagnostic = typeof captureClientDiagnostic === 'function' ? captureClientDiagnostic : () => {};
@@ -87,6 +91,7 @@ export function createLocalMediaOrchestrationHelpers({
const unpublishSfuTracks = typeof localPublisherCallbacks.unpublishSfuTracks === 'function'
? localPublisherCallbacks.unpublishSfuTracks
: () => {};
+ let lastBackgroundFallbackControlStateKey = '';
let localMediaPermissionRetryAfterMs = 0;
let screenShareStream = null;
let screenShareEndedHandler = null;
@@ -94,6 +99,19 @@ export function createLocalMediaOrchestrationHelpers({
let activeScreenShareTrackId = '';
const hasScreenShareParticipantPublisher = Boolean(startScreenShareParticipant && stopScreenShareParticipant);
+ function syncBackgroundFallbackControlState(force = false) {
+ const mode = String(callMediaPrefs.backgroundFallbackVideoMode || 'none').trim().toLowerCase() === 'avatar'
+ ? 'avatar'
+ : 'none';
+ const avatarUrl = mode === 'avatar'
+ ? String(callMediaPrefs.backgroundFallbackAvatarImageUrl || '').trim()
+ : '';
+ const nextKey = `${mode}:${avatarUrl}`;
+ if (!force && nextKey === lastBackgroundFallbackControlStateKey) return;
+ lastBackgroundFallbackControlStateKey = nextKey;
+ syncControlStateToPeers();
+ }
+
function buildLocalMediaConstraints() {
const cameraDeviceId = String(callMediaPrefs.selectedCameraId || '').trim();
const microphoneDeviceId = String(callMediaPrefs.selectedMicrophoneId || '').trim();
@@ -840,6 +858,16 @@ export function createLocalMediaOrchestrationHelpers({
overloadFrameMs: 48,
overloadConsecutiveFrames: 6,
statsIntervalMs: 1000,
+ onSegmentationUnavailable: (details = {}) => {
+ handleBackgroundReplacementUnavailable({
+ callMediaPrefs,
+ captureDiagnostic,
+ details,
+ refs,
+ runtimeToken,
+ state,
+ });
+ },
onOverload: () => {
if (runtimeToken !== state.backgroundRuntimeToken) return;
resetBackgroundRuntimeMetrics('overload');
@@ -888,6 +916,17 @@ export function createLocalMediaOrchestrationHelpers({
backgroundBaselineCollector.reset();
state.backgroundBaselineCaptured = false;
+ if (String(callMediaPrefs.backgroundFallbackVideoMode || 'none') === 'avatar') {
+ backgroundFilterController.dispose();
+ resetBackgroundRuntimeMetrics('avatar_placeholder');
+ callMediaPrefs.backgroundFilterBackend = 'avatar_placeholder';
+ callMediaPrefs.backgroundFilterActive = false;
+ syncBackgroundFallbackControlState(true);
+ return createBackgroundFallbackAudioOnlyStream(rawStream);
+ }
+
+ syncBackgroundFallbackControlState(false);
+
const options = resolveBackgroundFilterOptions(runtimeToken);
if (!shouldUseReactiveBackgroundPipeline() && options.mode !== 'blur' && options.mode !== 'replace') {
resetBackgroundRuntimeMetrics('off');
@@ -902,22 +941,6 @@ export function createLocalMediaOrchestrationHelpers({
callMediaPrefs.backgroundFilterActive = true;
callMediaPrefs.backgroundFilterReason = result.reason === 'ok_fallback' ? 'ok_fallback' : 'ok';
callMediaPrefs.backgroundFilterBackend = String(result.backend || 'none');
- if (callMediaPrefs.backgroundFilterBackend === 'sinet_unavailable') {
- captureDiagnostic({
- category: 'media',
- level: 'warning',
- eventType: 'local_background_sinet_unavailable',
- code: 'sinet_unavailable',
- message: 'Local background compositor is active, but SINet segmentation did not initialize.',
- payload: {
- media_runtime_path: refs.mediaRuntimePathRef.value,
- background_filter_mode: callMediaPrefs.backgroundFilterMode,
- background_backdrop_mode: callMediaPrefs.backgroundBackdropMode,
- background_quality_profile: callMediaPrefs.backgroundQualityProfile,
- },
- immediate: true,
- });
- }
} else {
callMediaPrefs.backgroundFilterActive = false;
callMediaPrefs.backgroundFilterReason = String(result?.reason || 'setup_failed');
@@ -952,9 +975,38 @@ export function createLocalMediaOrchestrationHelpers({
return Number(generation || 0) === Math.max(0, Number(state.localMediaCaptureGeneration || 0));
}
+ function activeLocalMediaStreamsForCleanup() {
+ const streams = [
+ refs.localStreamRef.value,
+ refs.localRawStreamRef.value,
+ refs.localFilteredStreamRef.value,
+ ];
+ if (screenShareStream instanceof MediaStream) {
+ streams.push(screenShareStream);
+ }
+ return streams.filter((stream) => stream instanceof MediaStream);
+ }
+
function discardStaleLocalMediaCapture(generation, streams = []) {
if (isCurrentLocalMediaCaptureGeneration(generation)) return false;
- stopRetiredLocalStreams(streams, []);
+ stopRetiredLocalStreams(streams, activeLocalMediaStreamsForCleanup(), {
+ reason: 'stale_local_media_capture_discarded',
+ captureDiagnostic,
+ mediaRuntimePath: refs.mediaRuntimePathRef.value,
+ });
+ captureDiagnostic({
+ category: 'media',
+ level: 'info',
+ eventType: 'stale_local_media_capture_discarded',
+ code: 'stale_local_media_capture_discarded',
+ message: 'Stale local media capture was discarded without stopping active camera, microphone, or screen-share tracks.',
+ payload: {
+ media_runtime_path: refs.mediaRuntimePathRef.value,
+ stale_generation: Number(generation || 0),
+ current_generation: Math.max(0, Number(state.localMediaCaptureGeneration || 0)),
+ active_screen_share_track_id: activeScreenShareTrackId,
+ },
+ });
return true;
}
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/local/publisherFrameDispatch.ts b/demo/video-chat/frontend-vue/src/domain/realtime/local/publisherFrameDispatch.ts
index 42d73c2e7..cb6619617 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/local/publisherFrameDispatch.ts
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/local/publisherFrameDispatch.ts
@@ -80,6 +80,7 @@ export async function dispatchPublisherFrame({
captureClientDiagnosticError,
onRequiredSfuUnavailable,
onRequiredSfuFailure,
+ onOptionalSfuFailure,
}) {
const gossipFirst = VIDEOCHAT_MEDIA_CARRIER_CONFIG.gossipPrimary;
const sfuOptional = VIDEOCHAT_MEDIA_CARRIER_CONFIG.sfuSendIsOptional;
@@ -95,19 +96,18 @@ export async function dispatchPublisherFrame({
});
}
- if (VIDEOCHAT_MEDIA_CARRIER_CONFIG.gossipPrimary && gossipPublished) {
- return {
- ok: true,
- gossipPublished,
- sfuSent: false,
- sfuSendOptional: true,
- sfuFallbackSkipped: true,
- postSendBufferedAmount: safeFunction(getSfuClientBufferedAmount, () => 0)(),
- };
- }
-
const sendClient = safeFunction(currentOpenSfuClient, () => null)();
if (!sendClient) {
+ if (gossipFirst && gossipPublished) {
+ return {
+ ok: true,
+ gossipPublished,
+ sfuSent: false,
+ sfuSendOptional: true,
+ sfuMirrorSkipped: true,
+ postSendBufferedAmount: safeFunction(getSfuClientBufferedAmount, () => 0)(),
+ };
+ }
if (!sfuOptional) {
return {
ok: Boolean(safeFunction(onRequiredSfuUnavailable)()),
@@ -184,6 +184,7 @@ export async function dispatchPublisherFrame({
mediaRuntimePath,
failureDetails,
});
+ safeFunction(onOptionalSfuFailure, () => undefined)(failureDetails);
return {
ok: gossipPublished,
gossipPublished,
@@ -256,6 +257,15 @@ export async function dispatchWlvcPublisherFrame({
);
return false;
},
+ onOptionalSfuFailure: (sfuSendFailureDetails) => {
+ safeFunction(paceForcedKeyframeRecovery, () => undefined)();
+ handleWlvcFrameSendFailure(
+ getSfuClientBufferedAmount(),
+ trackId,
+ String(sfuSendFailureDetails?.reason || 'sfu_frame_send_failed'),
+ sfuSendFailureDetails,
+ );
+ },
});
}
@@ -311,6 +321,20 @@ export async function dispatchProtectedBrowserPublisherFrame({
);
return false;
},
+ onOptionalSfuFailure: (sfuSendFailureDetails) => {
+ if (!critical) {
+ reportNonCriticalDrop(String(sfuSendFailureDetails?.reason || 'sfu_browser_thumbnail_frame_send_failed'), {
+ ...(sfuSendFailureDetails || {}),
+ });
+ return;
+ }
+ handleWlvcFrameSendFailure(
+ getSfuClientBufferedAmount(),
+ trackId,
+ String(sfuSendFailureDetails?.reason || 'sfu_browser_encoded_frame_send_failed'),
+ sfuSendFailureDetails,
+ );
+ },
});
}
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/local/screenSharePublisher.js b/demo/video-chat/frontend-vue/src/domain/realtime/local/screenSharePublisher.js
index f48d86b11..f24ce0c97 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/local/screenSharePublisher.js
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/local/screenSharePublisher.js
@@ -431,6 +431,21 @@ export function createScreenShareParticipantPublisher({
if (stopRequested || !isActive()) return false;
if (reconnectTimer !== null || reconnectInFlight) return true;
if (reconnectAttempts >= SCREEN_SHARE_RECONNECT_MAX_ATTEMPTS) {
+ callbacks.captureClientDiagnostic?.({
+ category: 'media',
+ level: 'error',
+ eventType: 'local_screen_share_sfu_reconnect_exhausted',
+ code: 'local_screen_share_sfu_reconnect_exhausted',
+ message: 'Screen sharing media routing exhausted reconnect attempts; only the screen-share capture will be cleaned up.',
+ payload: screenShareDiagnosticsPayload(refs, {
+ reason: String(reason || 'sfu_disconnected'),
+ attempts: reconnectAttempts,
+ max_attempts: SCREEN_SHARE_RECONNECT_MAX_ATTEMPTS,
+ cleanup_scope: 'screen_share_capture_only',
+ publisher_media_source: SCREEN_SHARE_MEDIA_SOURCE,
+ }),
+ immediate: true,
+ });
void stop('disconnected');
return false;
}
@@ -709,6 +724,7 @@ export function createScreenShareParticipantPublisher({
async function stop(reason = 'stopped') {
stopRequested = true;
+ const stoppedAfterReconnectAttempts = reconnectAttempts;
resetReconnectState();
detachEndedHandler();
pipeline.stopLocalEncodingPipeline();
@@ -743,6 +759,8 @@ export function createScreenShareParticipantPublisher({
message: 'Local screen sharing left the call media roster.',
payload: screenShareDiagnosticsPayload(refs, {
reason: String(reason || 'stopped'),
+ cleanup_scope: 'screen_share_capture_only',
+ reconnect_attempts: stoppedAfterReconnectAttempts,
publisher_media_source: SCREEN_SHARE_MEDIA_SOURCE,
}),
});
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/media/preferences.ts b/demo/video-chat/frontend-vue/src/domain/realtime/media/preferences.ts
index c2165f77e..cdbc4a249 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/media/preferences.ts
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/media/preferences.ts
@@ -15,6 +15,7 @@ const CALL_MEDIA_PREFS_OUTGOING_VIDEO_PROFILE_VERSION = 5;
const CALL_MEDIA_DEVICE_REFRESH_CACHE_MS = 30000;
const MOBILE_MEDIA_DEVICE_RELEASE_DELAY_MS = 250;
export const DEFAULT_BACKGROUND_REPLACEMENT_IMAGE_URL = '/assets/orgas/kingrt/social/invitation-preview.png';
+export const DEFAULT_BACKGROUND_FALLBACK_AVATAR_URL = '/assets/orgas/kingrt/avatar-placeholder.svg';
function clampVolume(value) {
const numeric = Number(value);
@@ -208,6 +209,11 @@ export const callMediaPrefs = reactive({
backgroundBlurTransition: persistedPrefs?.backgroundBlurTransition ?? 10,
backgroundApplyOutgoing: persistedPrefs?.backgroundApplyOutgoing ?? true,
backgroundReplacementImageUrl: persistedPrefs?.backgroundReplacementImageUrl || '',
+ backgroundFallbackVideoMode: 'none',
+ backgroundFallbackAvatarImageUrl: '',
+ backgroundReplacementUnavailablePromptOpen: false,
+ backgroundReplacementUnavailableReason: '',
+ backgroundReplacementUnavailableFailures: [],
backgroundMaxProcessWidth: persistedPrefs?.backgroundMaxProcessWidth ?? 960,
backgroundMaxProcessFps: persistedPrefs?.backgroundMaxProcessFps ?? 24,
backgroundFilterActive: false,
@@ -378,7 +384,13 @@ export function setCallMicrophoneVolume(value) {
}
export function setCallBackgroundFilterMode(mode) {
- callMediaPrefs.backgroundFilterMode = toBackgroundFilterMode(mode);
+ const nextMode = toBackgroundFilterMode(mode);
+ callMediaPrefs.backgroundFilterMode = nextMode;
+ if (nextMode !== 'off') {
+ callMediaPrefs.backgroundFallbackVideoMode = 'none';
+ callMediaPrefs.backgroundFallbackAvatarImageUrl = '';
+ callMediaPrefs.backgroundReplacementUnavailablePromptOpen = false;
+ }
persistCallMediaPrefs();
}
@@ -412,6 +424,37 @@ export function setCallBackgroundReplacementImageUrl(value) {
persistCallMediaPrefs();
}
+export function openBackgroundReplacementUnavailablePrompt(details = {}) {
+ callMediaPrefs.backgroundReplacementUnavailablePromptOpen = true;
+ callMediaPrefs.backgroundReplacementUnavailableReason = String(details?.reason || 'segmentation_unavailable');
+ callMediaPrefs.backgroundReplacementUnavailableFailures = Array.isArray(details?.failures)
+ ? details.failures.map((entry) => String(entry || '').trim()).filter(Boolean).slice(0, 5)
+ : [];
+}
+
+export function closeBackgroundReplacementUnavailablePrompt() {
+ callMediaPrefs.backgroundReplacementUnavailablePromptOpen = false;
+}
+
+export function useCallBackgroundFallbackAvatar(imageUrl = DEFAULT_BACKGROUND_FALLBACK_AVATAR_URL) {
+ const avatarUrl = String(imageUrl || '').trim() || DEFAULT_BACKGROUND_FALLBACK_AVATAR_URL;
+ callMediaPrefs.backgroundFallbackVideoMode = 'avatar';
+ callMediaPrefs.backgroundFallbackAvatarImageUrl = avatarUrl;
+ callMediaPrefs.backgroundFilterMode = 'off';
+ callMediaPrefs.backgroundApplyOutgoing = false;
+ closeBackgroundReplacementUnavailablePrompt();
+ persistCallMediaPrefs();
+}
+
+export function clearCallBackgroundFallbackVideo() {
+ callMediaPrefs.backgroundFallbackVideoMode = 'none';
+ callMediaPrefs.backgroundFallbackAvatarImageUrl = '';
+ callMediaPrefs.backgroundFilterMode = 'off';
+ callMediaPrefs.backgroundApplyOutgoing = false;
+ closeBackgroundReplacementUnavailablePrompt();
+ persistCallMediaPrefs();
+}
+
export function isCallBackgroundPresetActive(preset) {
const mode = String(callMediaPrefs.backgroundFilterMode || 'off').trim().toLowerCase();
const applyOutgoing = Boolean(callMediaPrefs.backgroundApplyOutgoing);
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/sfu/frameDecode.ts b/demo/video-chat/frontend-vue/src/domain/realtime/sfu/frameDecode.ts
index d0170590b..e859d98cd 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/sfu/frameDecode.ts
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/sfu/frameDecode.ts
@@ -747,7 +747,7 @@ export function createSfuFrameDecodeHelpers({
function shouldDropRemoteSfuFrameForContinuity(publisherId, peer, frame) {
if (!peer || typeof peer !== 'object') return false;
- const trackKey = sfuFrameTrackStateKey(frame);
+ const trackKey = remoteJitterTrackKey(frame);
ensureRemoteSfuTrackCacheState(peer);
if (!peer.lastSfuFrameSequenceByTrack || typeof peer.lastSfuFrameSequenceByTrack !== 'object') {
peer.lastSfuFrameSequenceByTrack = {};
@@ -1106,6 +1106,7 @@ export function createSfuFrameDecodeHelpers({
return;
}
let peer = peerLookup?.peer || null;
+ const resolvedPublisherId = normalizeSfuPublisherId(peerLookup?.publisherId || publisherId);
if (peerLookup?.matchedBy === 'publisher_user_id') {
captureClientDiagnostic({
category: 'media',
@@ -1115,12 +1116,15 @@ export function createSfuFrameDecodeHelpers({
message: 'SFU frame used a publisher id that did not match the known track publisher key, so the client matched by user id.',
payload: {
frame_publisher_id: publisherId,
- resolved_publisher_id: String(peerLookup.publisherId || ''),
+ resolved_publisher_id: resolvedPublisherId,
publisher_user_id: Number(frame?.publisherUserId || 0),
},
});
}
- peer = updateSfuRemotePeerUserId(peerLookup?.publisherId || publisherId, peer, frame?.publisherUserId, {
+ if (resolvedPublisherId !== publisherId) {
+ frame = { ...frame, publisherId: resolvedPublisherId, publisherIdAlias: publisherId };
+ }
+ peer = updateSfuRemotePeerUserId(resolvedPublisherId, peer, frame?.publisherUserId, {
publisherMediaSource: frame?.publisherMediaSource || frame?.publisher_media_source || '',
});
const publisherUserId = Number(frame?.publisherUserId || 0);
@@ -1139,18 +1143,18 @@ export function createSfuFrameDecodeHelpers({
if (init) {
void init.then((createdPeer) => {
const nextPeer = updateSfuRemotePeerUserId(
- publisherId,
- createdPeer || remotePeersRef.value.get(publisherId),
+ resolvedPublisherId,
+ createdPeer || remotePeersRef.value.get(resolvedPublisherId) || remotePeersRef.value.get(publisherId),
frame?.publisherUserId,
{ publisherMediaSource: frame?.publisherMediaSource || frame?.publisher_media_source || '' }
);
- void decodeSfuFrameForPeer(publisherId, nextPeer, frame);
+ void decodeSfuFrameForPeer(resolvedPublisherId, nextPeer, frame);
});
}
return;
}
- void decodeSfuFrameForPeer(publisherId, peer, frame);
+ void decodeSfuFrameForPeer(resolvedPublisherId, peer, frame);
}
return {
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/sfu/remoteJitterBuffer.ts b/demo/video-chat/frontend-vue/src/domain/realtime/sfu/remoteJitterBuffer.ts
index 5d2dc185e..92147bc5d 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/sfu/remoteJitterBuffer.ts
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/sfu/remoteJitterBuffer.ts
@@ -14,10 +14,17 @@ function normalizeRemoteFrameVideoLayer(value) {
return '';
}
+function normalizeRemoteFrameContinuityCarrier(frame) {
+ const transportPath = String(frame?.transportPath || frame?.transport_path || '').trim().toLowerCase();
+ return transportPath === 'gossip_rtc_datachannel' ? 'gossip' : '';
+}
+
export function remoteJitterTrackKey(frame) {
const trackId = String(frame?.trackId || '').trim() || 'default';
const videoLayer = normalizeRemoteFrameVideoLayer(frame?.videoLayer || frame?.video_layer);
- return videoLayer !== '' ? `${trackId}:${videoLayer}` : trackId;
+ const carrier = normalizeRemoteFrameContinuityCarrier(frame);
+ const baseKey = videoLayer !== '' ? `${trackId}:${videoLayer}` : trackId;
+ return carrier !== '' ? `${baseKey}:${carrier}` : baseKey;
}
export function ensureRemoteJitterBufferState(peer, trackKey) {
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/sfu/remotePeers.ts b/demo/video-chat/frontend-vue/src/domain/realtime/sfu/remotePeers.ts
index 6d0bae98e..a368f4ae7 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/sfu/remotePeers.ts
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/sfu/remotePeers.ts
@@ -203,7 +203,7 @@ export function createSfuRemotePeerHelpers({
}
return {
- publisherId: normalizedPublisherId,
+ publisherId: fallback.publisherId,
peer: fallback.peer,
matchedBy: 'publisher_user_id',
};
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/gossipNeighborLifecycle.ts b/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/gossipNeighborLifecycle.ts
index 0b7c88cd8..aaf9d1c4a 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/gossipNeighborLifecycle.ts
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/gossipNeighborLifecycle.ts
@@ -11,11 +11,53 @@ const LEGACY_GOSSIP_WEBRTC_SIGNAL_KINDS = Object.freeze([
'gossip_webrtc_answer',
'gossip_webrtc_ice',
]);
+const GOSSIP_NEIGHBOR_RENEGOTIATE_DELAY_MS = 25;
+const GOSSIP_NEIGHBOR_RENEGOTIATE_MAX_ATTEMPTS = 8;
function safePeerId(value) {
return String(value || '').trim();
}
+function normalizedSignalingState(pc) {
+ return String(pc?.signalingState || '').trim().toLowerCase();
+}
+
+function signalingStateIsStable(pc) {
+ const signalingState = normalizedSignalingState(pc);
+ return signalingState === 'stable' || signalingState === '';
+}
+
+function shouldDeferOfferSetLocalFailure(error, pc) {
+ const message = String(error?.message || error || '').toLowerCase();
+ return normalizedSignalingState(pc) !== 'closed'
+ && (
+ message.includes('wrong state')
+ || message.includes('have-remote-offer')
+ || message.includes('stable')
+ );
+}
+
+function shouldIgnoreStaleRemoteOfferAnswerFailure(error, pc) {
+ const message = String(error?.message || error || '').toLowerCase();
+ return normalizedSignalingState(pc) !== 'closed'
+ && (
+ message.includes('wrong signalingstate')
+ || message.includes('wrong state')
+ || message.includes('stable')
+ );
+}
+
+function shouldIgnoreStableRollbackRace(error, pc) {
+ const signalingState = normalizedSignalingState(pc);
+ const message = String(error?.message || error || '').toLowerCase();
+ return (signalingState === 'stable' || signalingState === '')
+ && (
+ message.includes('wrong signalingstate')
+ || message.includes('wrong state')
+ || message.includes('stable')
+ );
+}
+
function normalizeSdp(payload) {
const sdpPayload = payload && typeof payload.sdp === 'object' ? payload.sdp : null;
const type = String(sdpPayload?.type || '').trim().toLowerCase();
@@ -93,6 +135,8 @@ export function createGossipNeighborLifecycle({
pendingIce: [],
negotiating: false,
needsRenegotiate: false,
+ queuedRenegotiateAttempts: 0,
+ queuedRenegotiateTimer: null,
};
peers.set(normalizedPeerId, peer);
@@ -118,6 +162,14 @@ export function createGossipNeighborLifecycle({
void negotiatePeer(peer, 'negotiationneeded');
});
+ pc.addEventListener('signalingstatechange', () => {
+ if (!peer.initiator || peer.negotiating || !peer.needsRenegotiate) return;
+ if (!signalingStateIsStable(peer.pc)) return;
+ peer.needsRenegotiate = false;
+ peer.queuedRenegotiateAttempts = 0;
+ scheduleQueuedRenegotiate(peer, 'signaling_stable');
+ });
+
pc.addEventListener('connectionstatechange', () => {
const state = String(pc.connectionState || '').trim().toLowerCase();
onPeerConnectionState(normalizedPeerId, state, 'connectionstatechange');
@@ -160,6 +212,61 @@ export function createGossipNeighborLifecycle({
return peer;
}
+ function clearQueuedRenegotiate(peer) {
+ if (!peer?.queuedRenegotiateTimer) return;
+ clearTimeout(peer.queuedRenegotiateTimer);
+ peer.queuedRenegotiateTimer = null;
+ }
+
+ function scheduleQueuedRenegotiate(peer, reason = 'queued_renegotiate') {
+ if (!peer?.pc || peer.pc.signalingState === 'closed') return false;
+ if (!signalingStateIsStable(peer.pc)) {
+ captureClientDiagnostic({
+ category: 'media',
+ level: 'info',
+ eventType: 'gossip_neighbor_renegotiate_waiting_stable',
+ code: 'gossip_neighbor_renegotiate_waiting_stable',
+ message: 'Dedicated Gossip neighbor renegotiation is waiting for a stable signaling state.',
+ payload: {
+ peer_id: safePeerId(peer.peerId),
+ reason: String(reason || 'queued_renegotiate'),
+ signaling_state: normalizedSignalingState(peer.pc),
+ topology_epoch: topologyEpoch,
+ },
+ });
+ return false;
+ }
+ if (peer.queuedRenegotiateTimer) return true;
+
+ peer.queuedRenegotiateAttempts = Math.max(0, Number(peer.queuedRenegotiateAttempts || 0)) + 1;
+ if (peer.queuedRenegotiateAttempts > GOSSIP_NEIGHBOR_RENEGOTIATE_MAX_ATTEMPTS) {
+ peer.needsRenegotiate = false;
+ captureClientDiagnostic({
+ category: 'media',
+ level: 'warning',
+ eventType: 'gossip_neighbor_renegotiate_quarantined',
+ code: 'gossip_neighbor_renegotiate_quarantined',
+ message: 'Dedicated Gossip neighbor renegotiation was quarantined after repeated queued attempts.',
+ payload: {
+ peer_id: safePeerId(peer.peerId),
+ reason: String(reason || 'queued_renegotiate'),
+ attempt_count: peer.queuedRenegotiateAttempts,
+ signaling_state: String(peer.pc?.signalingState || ''),
+ topology_epoch: topologyEpoch,
+ },
+ });
+ return false;
+ }
+
+ peer.queuedRenegotiateTimer = setTimeout(() => {
+ peer.queuedRenegotiateTimer = null;
+ if (peers.get(safePeerId(peer.peerId)) !== peer) return;
+ if (!peer.pc || peer.pc.signalingState === 'closed') return;
+ void negotiatePeer(peer, reason);
+ }, GOSSIP_NEIGHBOR_RENEGOTIATE_DELAY_MS);
+ return true;
+ }
+
async function negotiatePeer(peer, reason) {
if (!peer?.pc || peer.negotiating) {
if (peer) peer.needsRenegotiate = true;
@@ -167,16 +274,55 @@ export function createGossipNeighborLifecycle({
}
peer.negotiating = true;
try {
- const signalingState = String(peer.pc.signalingState || '').trim().toLowerCase();
- if (signalingState !== 'stable' && signalingState !== '') {
+ if (!signalingStateIsStable(peer.pc)) {
peer.needsRenegotiate = true;
return false;
}
const offer = await peer.pc.createOffer();
- await peer.pc.setLocalDescription(offer);
+ const preSetLocalState = normalizedSignalingState(peer.pc);
+ if (preSetLocalState !== 'stable' && preSetLocalState !== '') {
+ peer.needsRenegotiate = true;
+ captureClientDiagnostic({
+ category: 'media',
+ level: 'info',
+ eventType: 'gossip_neighbor_offer_deferred',
+ code: 'gossip_neighbor_offer_deferred',
+ message: 'Dedicated Gossip neighbor offer was deferred because a remote offer arrived first.',
+ payload: {
+ peer_id: safePeerId(peer.peerId),
+ reason: String(reason || 'offer'),
+ signaling_state: preSetLocalState,
+ topology_epoch: topologyEpoch,
+ },
+ });
+ return false;
+ }
+ try {
+ await peer.pc.setLocalDescription(offer);
+ } catch (error) {
+ if (shouldDeferOfferSetLocalFailure(error, peer.pc)) {
+ peer.needsRenegotiate = true;
+ captureClientDiagnostic({
+ category: 'media',
+ level: 'info',
+ eventType: 'gossip_neighbor_offer_deferred',
+ code: 'gossip_neighbor_offer_deferred',
+ message: 'Dedicated Gossip neighbor offer was deferred because signaling state changed while setting the local offer.',
+ payload: {
+ peer_id: safePeerId(peer.peerId),
+ reason: String(reason || 'offer'),
+ signaling_state: normalizedSignalingState(peer.pc),
+ error: String(error?.message || error || ''),
+ topology_epoch: topologyEpoch,
+ },
+ });
+ return false;
+ }
+ throw error;
+ }
const local = peer.pc.localDescription;
if (!local?.sdp) return false;
- return sendSocketFrame({
+ const sent = sendSocketFrame({
type: 'call/offer',
target_user_id: Number(peer.peerId),
payload: {
@@ -193,6 +339,8 @@ export function createGossipNeighborLifecycle({
},
},
});
+ if (sent) peer.queuedRenegotiateAttempts = 0;
+ return sent;
} catch (error) {
captureClientDiagnostic({
category: 'media',
@@ -210,8 +358,12 @@ export function createGossipNeighborLifecycle({
} finally {
peer.negotiating = false;
if (peer.needsRenegotiate) {
- peer.needsRenegotiate = false;
- void negotiatePeer(peer, 'queued_renegotiate');
+ if (signalingStateIsStable(peer.pc)) {
+ peer.needsRenegotiate = false;
+ scheduleQueuedRenegotiate(peer, 'queued_renegotiate');
+ } else {
+ scheduleQueuedRenegotiate(peer, 'queued_renegotiate');
+ }
}
}
}
@@ -241,15 +393,68 @@ export function createGossipNeighborLifecycle({
if (signalingState === 'have-local-offer') {
const remoteWinsCollision = normalizedPeerId < localPeerId();
if (!remoteWinsCollision) return;
- await peer.pc.setLocalDescription({ type: 'rollback' });
+ try {
+ await peer.pc.setLocalDescription({ type: 'rollback' });
+ } catch (error) {
+ if (!shouldIgnoreStableRollbackRace(error, peer.pc)) throw error;
+ captureClientDiagnostic({
+ category: 'media',
+ level: 'info',
+ eventType: 'gossip_neighbor_offer_stale',
+ code: 'gossip_neighbor_offer_stale',
+ message: 'Dedicated Gossip neighbor rollback raced with stable signaling; continuing with the remote offer.',
+ payload: {
+ peer_id: normalizedPeerId,
+ signaling_state: normalizedSignalingState(peer.pc),
+ error: String(error?.message || error || ''),
+ topology_epoch: topologyEpoch,
+ },
+ });
+ }
} else if (signalingState !== 'stable' && signalingState !== '') {
return;
}
await peer.pc.setRemoteDescription(new RTCSessionDescription(remote));
+ const postRemoteState = normalizedSignalingState(peer.pc);
+ if (postRemoteState !== 'have-remote-offer') {
+ captureClientDiagnostic({
+ category: 'media',
+ level: 'info',
+ eventType: 'gossip_neighbor_offer_stale',
+ code: 'gossip_neighbor_offer_stale',
+ message: 'Dedicated Gossip neighbor offer was ignored because signaling no longer has a pending remote offer.',
+ payload: {
+ peer_id: normalizedPeerId,
+ signaling_state: postRemoteState,
+ topology_epoch: topologyEpoch,
+ },
+ });
+ return;
+ }
await flushPendingIce(peer);
const answer = await peer.pc.createAnswer();
- await peer.pc.setLocalDescription(answer);
+ try {
+ await peer.pc.setLocalDescription(answer);
+ } catch (error) {
+ if (shouldIgnoreStaleRemoteOfferAnswerFailure(error, peer.pc)) {
+ captureClientDiagnostic({
+ category: 'media',
+ level: 'info',
+ eventType: 'gossip_neighbor_offer_stale',
+ code: 'gossip_neighbor_offer_stale',
+ message: 'Dedicated Gossip neighbor answer was skipped because the remote offer was no longer pending.',
+ payload: {
+ peer_id: normalizedPeerId,
+ signaling_state: normalizedSignalingState(peer.pc),
+ error: String(error?.message || error || ''),
+ topology_epoch: topologyEpoch,
+ },
+ });
+ return;
+ }
+ throw error;
+ }
const local = peer.pc.localDescription;
if (!local?.sdp) return;
sendSocketFrame({
@@ -291,6 +496,7 @@ export function createGossipNeighborLifecycle({
if (!peer?.pc || !remote || remote.type !== 'answer') return;
try {
await peer.pc.setRemoteDescription(new RTCSessionDescription(remote));
+ peer.queuedRenegotiateAttempts = 0;
await flushPendingIce(peer);
} catch {}
}
@@ -357,6 +563,7 @@ export function createGossipNeighborLifecycle({
const peer = peers.get(normalizedPeerId);
if (!peer) return false;
peers.delete(normalizedPeerId);
+ clearQueuedRenegotiate(peer);
try {
peer.pc?.close?.();
} catch {}
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/mediaSecurityErrors.ts b/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/mediaSecurityErrors.ts
new file mode 100644
index 000000000..713645e06
--- /dev/null
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/mediaSecurityErrors.ts
@@ -0,0 +1,14 @@
+export function mediaSecurityErrorCode(error) {
+ const code = String(error?.code || '').trim().toLowerCase();
+ if (code === 'participant_set_mismatch') return 'participant_set_mismatch';
+ if (code === 'downgrade_attempt') return 'downgrade_attempt';
+
+ const message = String(error?.message || error || '').trim().toLowerCase();
+ if (message.includes('participant_set_mismatch')) return 'participant_set_mismatch';
+ if (message.includes('downgrade_attempt') || message.includes('downgrade')) return 'downgrade_attempt';
+ if (message.includes('wrong_key_id')) return 'wrong_key_id';
+ if (message.includes('wrong_epoch')) return 'wrong_epoch';
+ if (message.includes('replay_detected')) return 'replay_detected';
+ if (message.includes('malformed_protected_frame')) return 'malformed_protected_frame';
+ return message;
+}
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/mediaSecurityParticipantSet.ts b/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/mediaSecurityParticipantSet.ts
index 656d3616a..6bb31d7d0 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/mediaSecurityParticipantSet.ts
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/mediaSecurityParticipantSet.ts
@@ -10,6 +10,40 @@ export function mediaSecurityParticipantSignatureIds(session) {
.filter((userId, index, userIds) => userId > 0 && userIds.indexOf(userId) === index);
}
+export function mediaSecurityParticipantIdsFromSignature(signature) {
+ return String(signature || '')
+ .split(',')
+ .map((userId) => normalizeUserId(userId))
+ .filter((userId) => userId > 0);
+}
+
+export function mediaSecurityParticipantSetDelta(previousUserIds, nextUserIds) {
+ const previous = new Set(Array.isArray(previousUserIds) ? previousUserIds : []);
+ const next = new Set(Array.isArray(nextUserIds) ? nextUserIds : []);
+ return {
+ added: Array.from(next).filter((userId) => !previous.has(userId)),
+ removed: Array.from(previous).filter((userId) => !next.has(userId)),
+ };
+}
+
+export function shouldForceMediaSecurityRekeyForParticipantSetDelta(delta, requestedForceRekey = false) {
+ if (requestedForceRekey) return true;
+ return Array.isArray(delta?.removed) && delta.removed.length > 0;
+}
+
+export function shouldRecoverMediaSecuritySignalSender({
+ hasRealtimeRoomSync = false,
+ targetUserIds = [],
+ senderUserId = 0,
+} = {}) {
+ const normalizedSenderUserId = normalizeUserId(senderUserId);
+ if (normalizedSenderUserId <= 0) return false;
+ if (hasRealtimeRoomSync !== true) return true;
+ return (Array.isArray(targetUserIds) ? targetUserIds : [])
+ .map((userId) => normalizeUserId(userId))
+ .includes(normalizedSenderUserId);
+}
+
export function mergeMediaSecurityParticipantIds(session, targetUserIds = [], extraUserId = 0) {
return Array.from(new Set([
...mediaSecurityParticipantSignatureIds(session),
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/mediaSecurityRuntime.ts b/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/mediaSecurityRuntime.ts
index 299ef96da..d033e3a8a 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/mediaSecurityRuntime.ts
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/mediaSecurityRuntime.ts
@@ -1,6 +1,13 @@
import { computed } from 'vue';
import { reportNativeAudioBridgeFailure as reportNativeAudioBridgeFailureEvent } from '../../native/audioBridgeFailureReporter';
-import { mergeMediaSecurityParticipantIds } from './mediaSecurityParticipantSet';
+import { mediaSecurityErrorCode } from './mediaSecurityErrors';
+import {
+ mediaSecurityParticipantIdsFromSignature,
+ mediaSecurityParticipantSetDelta,
+ mergeMediaSecurityParticipantIds,
+ shouldForceMediaSecurityRekeyForParticipantSetDelta,
+ shouldRecoverMediaSecuritySignalSender,
+} from './mediaSecurityParticipantSet';
import { createMediaSecuritySfuPublishGate } from './mediaSecuritySfuPublishGate';
export function createCallWorkspaceMediaSecurityRuntime({
@@ -72,20 +79,6 @@ export function createCallWorkspaceMediaSecurityRuntime({
return normalizedUserId;
}
- function mediaSecurityErrorCode(error) {
- const code = String(error?.code || '').trim().toLowerCase();
- if (code === 'participant_set_mismatch') return 'participant_set_mismatch';
- if (code === 'downgrade_attempt') return 'downgrade_attempt';
-
- const message = String(error?.message || error || '').trim().toLowerCase();
- if (message.includes('participant_set_mismatch')) return 'participant_set_mismatch';
- if (message.includes('downgrade_attempt') || message.includes('downgrade')) return 'downgrade_attempt';
- if (message.includes('wrong_key_id')) return 'wrong_key_id';
- if (message.includes('wrong_epoch')) return 'wrong_epoch';
- if (message.includes('malformed_protected_frame')) return 'malformed_protected_frame';
- return message;
- }
-
function remoteMediaSecurityEligibleTargetIds() {
const seen = new Set();
const userIds = [];
@@ -403,6 +396,7 @@ export function createCallWorkspaceMediaSecurityRuntime({
activeRoomId.value,
currentMediaSecurityRuntimePath(),
Number(targetUserId || 0),
+ String(session?.participantSignature || ''),
Number(session?.epoch || 0),
'hello',
].join(':');
@@ -413,28 +407,13 @@ export function createCallWorkspaceMediaSecurityRuntime({
activeRoomId.value,
currentMediaSecurityRuntimePath(),
Number(targetUserId || 0),
+ String(session?.participantSignature || ''),
Number(session?.epoch || 0),
String(session?.senderKeyId || ''),
'sender-key',
].join(':');
}
- function mediaSecurityParticipantIdsFromSignature(signature) {
- return String(signature || '')
- .split(',')
- .map((userId) => Number(userId || 0))
- .filter((userId) => Number.isInteger(userId) && userId > 0);
- }
-
- function mediaSecurityParticipantSetDelta(previousUserIds, nextUserIds) {
- const previous = new Set(Array.isArray(previousUserIds) ? previousUserIds : []);
- const next = new Set(Array.isArray(nextUserIds) ? nextUserIds : []);
- return {
- added: Array.from(next).filter((userId) => !previous.has(userId)),
- removed: Array.from(previous).filter((userId) => !next.has(userId)),
- };
- }
-
function currentSfuSenderKeySignaledTargetIds(targetUserIds = remoteMediaSecurityEligibleTargetIds()) {
const session = ensureMediaSecuritySession();
return (Array.isArray(targetUserIds) ? targetUserIds : [])
@@ -462,11 +441,6 @@ export function createCallWorkspaceMediaSecurityRuntime({
return sfuPublishGate.canProtectCurrentSfuTargets();
}
- function shouldForceRekeyForParticipantSetDelta(delta, requestedForceRekey = false) {
- if (requestedForceRekey) return true;
- return Array.isArray(delta?.removed) && delta.removed.length > 0;
- }
-
function incomingMediaSecurityHelloResponseKey(senderUserId, payloadBody, session) {
const payload = payloadBody && typeof payloadBody === 'object' ? payloadBody : {};
return [
@@ -478,6 +452,7 @@ export function createCallWorkspaceMediaSecurityRuntime({
String(payload.device_id || ''),
String(payload.public_key || ''),
String(payload.hybrid_public_key || ''),
+ String(session?.participantSignature || ''),
Number(session?.epoch || 0),
String(session?.senderKeyId || ''),
'hello-response',
@@ -603,6 +578,10 @@ export function createCallWorkspaceMediaSecurityRuntime({
'sender_key_participant_mismatch',
false,
);
+ requestRemoteMediaSecuritySync(normalizedTargetId, 'sender_key_participant_mismatch', {
+ peer_state: String(peer?.state || ''),
+ peer_has_wrapping_key: Boolean(peer?.wrappingKey),
+ });
await sendMediaSecurityHello(normalizedTargetId, true);
captureClientDiagnostic({
category: 'media',
@@ -696,9 +675,11 @@ export function createCallWorkspaceMediaSecurityRuntime({
const marked = session.markParticipantSet(eligibleTargetUserIds);
const participantDelta = mediaSecurityParticipantSetDelta(previousUserIds, marked.userIds);
mediaSecurityStateVersion.value += 1;
- const shouldRotateSenderKey = shouldForceRekeyForParticipantSetDelta(participantDelta, forceRekey);
- if (shouldRotateSenderKey) {
+ const shouldRotateSenderKey = shouldForceMediaSecurityRekeyForParticipantSetDelta(participantDelta, forceRekey);
+ if (marked.changed || shouldRotateSenderKey) {
clearMediaSecuritySignalCaches();
+ }
+ if (shouldRotateSenderKey) {
const ready = await session.ensureReady();
if (!ready) {
mediaSecurityStateVersion.value += 1;
@@ -801,6 +782,12 @@ export function createCallWorkspaceMediaSecurityRuntime({
&& message === 'malformed_protected_frame';
}
+ function shouldTreatNativeFrameErrorAsDuplicateDrop(direction, error, senderUserId = 0) {
+ const message = String(error?.message || error || '').trim().toLowerCase();
+ return isRemoteNativeFrameError(direction, senderUserId)
+ && message === 'replay_detected';
+ }
+
function shouldTreatNativeFrameErrorAsRecoverableDrop(direction, error, senderUserId = 0) {
return isRemoteNativeFrameError(direction, senderUserId)
&& shouldRecoverMediaSecurityFromFrameError(error);
@@ -973,8 +960,10 @@ export function createCallWorkspaceMediaSecurityRuntime({
? 'native_media_frame_decrypt_failed'
: 'native_media_frame_encrypt_failed';
const bootstrapFrameDrop = shouldTreatNativeFrameErrorAsBootstrapDrop(direction, error, senderUserId);
+ const duplicateFrameDrop = shouldTreatNativeFrameErrorAsDuplicateDrop(direction, error, senderUserId);
const transientFrameDrop = shouldTreatNativeFrameErrorAsTransient(direction, error, senderUserId);
- const recoverableFrameDrop = transientFrameDrop
+ const recoverableFrameDrop = duplicateFrameDrop
+ || transientFrameDrop
|| shouldTreatNativeFrameErrorAsRecoverableDrop(direction, error, senderUserId);
const logKey = [code, direction || 'unknown', senderUserId || 0, trackId || 'n/a', errorMessage].join(':');
const nowMs = Date.now();
@@ -997,6 +986,7 @@ export function createCallWorkspaceMediaSecurityRuntime({
track_id: trackId,
media_runtime_path: mediaRuntimePath.value,
security: nativeAudioSecurityTelemetrySnapshot(),
+ duplicate_frame_drop: duplicateFrameDrop,
recoverable_frame_drop: recoverableFrameDrop,
},
immediate: !recoverableFrameDrop,
@@ -1051,10 +1041,10 @@ export function createCallWorkspaceMediaSecurityRuntime({
normalizedSenderUserId,
));
const participantDelta = mediaSecurityParticipantSetDelta(previousUserIds, marked.userIds);
- const shouldForceRekey = shouldForceRekeyForParticipantSetDelta(participantDelta, false);
+ const shouldForceRekey = shouldForceMediaSecurityRekeyForParticipantSetDelta(participantDelta, false);
if (marked.changed || shouldForceRekey) {
mediaSecurityStateVersion.value += 1;
- if (shouldForceRekey) clearMediaSecuritySignalCaches();
+ clearMediaSecuritySignalCaches();
}
state.mediaSecurityHelloSignalsSent.delete(mediaSecurityHelloSignalKey(normalizedSenderUserId, session));
state.mediaSecuritySenderKeySignalsSent.delete(mediaSecuritySenderKeySignalKey(normalizedSenderUserId, session));
@@ -1075,10 +1065,8 @@ export function createCallWorkspaceMediaSecurityRuntime({
const participantDelta = mediaSecurityParticipantSetDelta(previousUserIds, marked.userIds);
if (marked.changed) {
mediaSecurityStateVersion.value += 1;
- const shouldForceRekey = shouldForceRekeyForParticipantSetDelta(participantDelta, false);
- if (shouldForceRekey) {
- clearMediaSecuritySignalCaches();
- }
+ const shouldForceRekey = shouldForceMediaSecurityRekeyForParticipantSetDelta(participantDelta, false);
+ clearMediaSecuritySignalCaches();
scheduleMediaSecurityParticipantSync('hello_participant_set_changed', shouldForceRekey);
}
const accepted = await session.handleHelloSignal(normalizedSenderUserId, payloadBody || {});
@@ -1110,14 +1098,64 @@ export function createCallWorkspaceMediaSecurityRuntime({
}
if (!accepted && remoteMediaSecurityTargetIds().includes(normalizedSenderUserId)) {
scheduleMediaSecurityParticipantSync('sender_key_pending');
+ } else if (
+ !accepted
+ && shouldRecoverMediaSecuritySignalSender({
+ hasRealtimeRoomSync: hasRealtimeRoomSync.value,
+ targetUserIds: remoteMediaSecurityTargetIds(),
+ senderUserId: normalizedSenderUserId,
+ })
+ ) {
+ requestRoomSnapshot();
+ requestRemoteMediaSecuritySync(normalizedSenderUserId, 'sender_key_pending_snapshot', {
+ signal_type: type,
+ });
}
}
} catch (error) {
mediaDebugLog('[MediaSecurity] signaling failed', error);
const errorCode = mediaSecurityErrorCode(error);
+ const shouldRecoverSignalSender = shouldRecoverMediaSecuritySignalSender({
+ hasRealtimeRoomSync: hasRealtimeRoomSync.value,
+ targetUserIds: remoteMediaSecurityTargetIds(),
+ senderUserId: normalizedSenderUserId,
+ });
+ if (
+ type === 'media-security/sender-key'
+ && errorCode === 'participant_set_mismatch'
+ && shouldRecoverSignalSender
+ ) {
+ state.mediaSecurityHelloSignalsSent.delete(mediaSecurityHelloSignalKey(normalizedSenderUserId, session));
+ state.mediaSecuritySenderKeySignalsSent.delete(mediaSecuritySenderKeySignalKey(normalizedSenderUserId, session));
+ sfuPublishGate.deleteSenderKeySignalTime(mediaSecuritySenderKeySignalKey(normalizedSenderUserId, session));
+ requestRoomSnapshot();
+ requestRemoteMediaSecuritySync(normalizedSenderUserId, 'sender_key_participant_mismatch', {
+ error_code: errorCode,
+ signal_type: type,
+ direction: 'receiver',
+ });
+ await sendMediaSecurityHello(normalizedSenderUserId, true);
+ scheduleMediaSecurityParticipantSync('sender_key_participant_mismatch', false);
+ startMediaSecurityHandshakeWatchdog();
+ captureClientDiagnostic({
+ category: 'media',
+ level: 'warning',
+ eventType: 'media_security_sender_key_participant_mismatch',
+ code: 'media_security_sender_key_participant_mismatch',
+ message: 'Incoming media security sender key was stale for the current participant set; a fresh hello/sync was requested.',
+ payload: {
+ sender_user_id: normalizedSenderUserId,
+ signal_type: type,
+ direction: 'receiver',
+ media_runtime_path: mediaRuntimePath.value,
+ security: session.telemetrySnapshot(currentMediaSecurityRuntimePath()),
+ },
+ });
+ return;
+ }
if (
(errorCode === 'participant_set_mismatch' || errorCode === 'downgrade_attempt')
- && remoteMediaSecurityTargetIds().includes(normalizedSenderUserId)
+ && shouldRecoverSignalSender
) {
const shouldForceRekeyAfterSignalFailure = errorCode === 'downgrade_attempt';
if (errorCode === 'downgrade_attempt') {
@@ -1132,6 +1170,9 @@ export function createCallWorkspaceMediaSecurityRuntime({
error_code: errorCode,
signal_type: type,
});
+ if (errorCode === 'participant_set_mismatch') {
+ await sendMediaSecurityHello(normalizedSenderUserId, true);
+ }
scheduleMediaSecurityParticipantSync('signal_failed_reconnect', shouldForceRekeyAfterSignalFailure);
startMediaSecurityHandshakeWatchdog();
}
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/mediaStack.ts b/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/mediaStack.ts
index 13273bbf7..3ecdb33ec 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/mediaStack.ts
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/mediaStack.ts
@@ -545,6 +545,7 @@ export function createCallWorkspaceMediaStack(options) {
sendSocketFrame: callbacks.sendSocketFrame,
shouldMaintainNativePeerConnections: runtimeHealth.shouldMaintainNativePeerConnections,
shouldSyncNativeLocalTracksBeforeOffer: callbacks.shouldSyncNativeLocalTracksBeforeOffer,
+ syncControlStateToPeers: callbacks.syncControlStateToPeers,
syncNativePeerConnectionsWithRoster: callbacks.syncNativePeerConnectionsWithRoster,
syncNativePeerLocalTracks: callbacks.syncNativePeerLocalTracks,
sendNativeOffer: callbacks.sendNativeOffer,
@@ -572,6 +573,7 @@ export function createCallWorkspaceMediaStack(options) {
refs: {
activeRoomId: refs.activeRoomId,
activeSocketCallId: refs.activeSocketCallId,
+ connectedParticipantUsers: refs.connectedParticipantUsers,
currentUserId: refs.currentUserId,
desiredRoomId: refs.desiredRoomId,
encodeIntervalRef: refs.encodeIntervalRef,
@@ -618,11 +620,13 @@ export function createCallWorkspaceMediaStack(options) {
gridVideoParticipants: () => refs.gridVideoParticipants.value,
gridVideoSlotId: constants.gridVideoSlotId,
hasRenderableMediaForParticipant: callbacks.hasRenderableMediaForParticipant,
+ hasStaticAvatarForUserId: callbacks.hasStaticAvatarForUserId,
lookupMediaNodeForUserId: callbacks.lookupMediaNodeForUserId,
miniVideoParticipants: () => refs.miniVideoParticipants.value,
miniVideoSlotId: constants.miniVideoSlotId,
primaryVideoUserId: () => refs.primaryVideoUserId.value,
remotePeerMediaNode: callbacks.remotePeerMediaNode,
+ staticAvatarNodeForUserId: callbacks.staticAvatarNodeForUserId,
},
refs: {
currentUserId: refs.currentUserId,
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/participantUi.ts b/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/participantUi.ts
index c16396c57..10f80ad11 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/participantUi.ts
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/participantUi.ts
@@ -5,6 +5,13 @@ import { createCallWorkspaceModerationSync } from './moderationSync';
import { createVideoFullscreenToggle } from './videoFullscreenToggle';
import { isScreenShareMediaSource, isScreenShareUserId, screenShareUserIdForOwner } from '../../screenShareIdentity.js';
import { compareLocalizedStrings } from '../../../../support/localeCollation.js';
+import {
+ BACKGROUND_FALLBACK_AVATAR_MODE,
+ BACKGROUND_FALLBACK_NONE_MODE,
+ backgroundFallbackControlStateFromPrefs,
+ normalizeBackgroundFallbackAvatarUrl,
+ normalizeBackgroundFallbackMode,
+} from '../../background/avatarFallbackSignal';
export function createCallWorkspaceParticipantUiHelpers(context) {
const {
@@ -16,6 +23,7 @@ export function createCallWorkspaceParticipantUiHelpers(context) {
aloneIdlePrompt,
apiRequest,
callLayoutState,
+ callMediaPrefs,
callParticipantRoles,
canManageOwnerRole,
canModerate,
@@ -35,6 +43,7 @@ export function createCallWorkspaceParticipantUiHelpers(context) {
isCompactLayoutViewport,
isCompactMiniStripAbove,
isSocketOnline,
+ hasStaticAvatarForUserId = () => false,
layoutStrategyOptionsFor,
lobbyActionState,
lobbyListRef,
@@ -267,6 +276,9 @@ function participantMediaStatus(userId) {
if (!Number.isInteger(normalizedUserId) || normalizedUserId <= 0 || normalizedUserId === currentUserId.value) {
return { show: false, state: 'local', label: '' };
}
+ if (hasStaticAvatarForUserId(normalizedUserId)) {
+ return { show: false, state: 'avatar', label: '' };
+ }
const hasRenderable = typeof participantHasRenderableMedia === 'function'
? participantHasRenderableMedia(normalizedUserId)
@@ -730,15 +742,22 @@ function lobbyRowSnapshot(row) {
};
}
+function defaultPeerControlState() {
+ return {
+ handRaised: false,
+ cameraEnabled: true,
+ micEnabled: true,
+ screenEnabled: false,
+ backgroundFallbackVideoMode: BACKGROUND_FALLBACK_NONE_MODE,
+ backgroundFallbackAvatarImageUrl: '',
+ videoSubstitution: '',
+ };
+}
+
function peerControlSnapshot(userId) {
const normalizedUserId = Number(userId);
if (!Number.isInteger(normalizedUserId) || normalizedUserId <= 0) {
- return {
- handRaised: false,
- cameraEnabled: true,
- micEnabled: true,
- screenEnabled: false,
- };
+ return defaultPeerControlState();
}
if (normalizedUserId === currentUserId.value) {
@@ -747,16 +766,12 @@ function peerControlSnapshot(userId) {
cameraEnabled: controlState.cameraEnabled,
micEnabled: controlState.micEnabled,
screenEnabled: controlState.screenEnabled,
+ ...backgroundFallbackControlStateFromPrefs(callMediaPrefs),
};
}
if (!peerControlStateByUserId[normalizedUserId] || typeof peerControlStateByUserId[normalizedUserId] !== 'object') {
- peerControlStateByUserId[normalizedUserId] = {
- handRaised: false,
- cameraEnabled: true,
- micEnabled: true,
- screenEnabled: false,
- };
+ peerControlStateByUserId[normalizedUserId] = defaultPeerControlState();
}
return peerControlStateByUserId[normalizedUserId];
@@ -768,6 +783,9 @@ function describePeerControlState(userId) {
if (state.handRaised) badges.push('hand');
if (!state.micEnabled) badges.push('mic off');
if (!state.cameraEnabled) badges.push('cam off');
+ if (normalizeBackgroundFallbackMode(state.backgroundFallbackVideoMode || state.videoSubstitution) === BACKGROUND_FALLBACK_AVATAR_MODE) {
+ badges.push('avatar');
+ }
if (state.screenEnabled) badges.push('screen');
return badges.join(' · ');
}
@@ -1110,12 +1128,7 @@ function updatePeerControlState(userId, patch) {
if (normalizedUserId === currentUserId.value) return;
if (!peerControlStateByUserId[normalizedUserId] || typeof peerControlStateByUserId[normalizedUserId] !== 'object') {
- peerControlStateByUserId[normalizedUserId] = {
- handRaised: false,
- cameraEnabled: true,
- micEnabled: true,
- screenEnabled: false,
- };
+ peerControlStateByUserId[normalizedUserId] = defaultPeerControlState();
}
peerControlStateByUserId[normalizedUserId] = {
@@ -1127,12 +1140,7 @@ function updatePeerControlState(userId, patch) {
function resetPeerControlState(userId) {
const normalizedUserId = Number(userId);
if (!Number.isInteger(normalizedUserId) || normalizedUserId <= 0 || normalizedUserId === currentUserId.value) return;
- peerControlStateByUserId[normalizedUserId] = {
- handRaised: false,
- cameraEnabled: true,
- micEnabled: true,
- screenEnabled: false,
- };
+ peerControlStateByUserId[normalizedUserId] = defaultPeerControlState();
}
function applyReactionEvent(payload) {
@@ -1172,11 +1180,20 @@ function applyRemoteControlState(payload, sender) {
if (kind === 'workspace-control-state') {
const state = payload && typeof payload.state === 'object' ? payload.state : {};
const nextScreenEnabled = Boolean(state.screenEnabled);
+ const fallbackMode = normalizeBackgroundFallbackMode(
+ state.backgroundFallbackVideoMode || state.videoSubstitution,
+ );
+ const fallbackAvatarUrl = fallbackMode === BACKGROUND_FALLBACK_AVATAR_MODE
+ ? normalizeBackgroundFallbackAvatarUrl(state.backgroundFallbackAvatarImageUrl)
+ : '';
updatePeerControlState(senderUserId, {
handRaised: Boolean(state.handRaised),
cameraEnabled: state.cameraEnabled !== false,
micEnabled: state.micEnabled !== false,
screenEnabled: nextScreenEnabled,
+ backgroundFallbackVideoMode: fallbackMode,
+ backgroundFallbackAvatarImageUrl: fallbackAvatarUrl,
+ videoSubstitution: fallbackMode === BACKGROUND_FALLBACK_AVATAR_MODE ? BACKGROUND_FALLBACK_AVATAR_MODE : '',
});
if (nextScreenEnabled) {
pinScreenShareParticipant(screenShareUserIdForOwner(senderUserId));
@@ -1184,6 +1201,7 @@ function applyRemoteControlState(payload, sender) {
forgetScreenShareAutoPin(screenShareUserIdForOwner(senderUserId), true);
}
refreshUsersDirectoryPresentation();
+ nextTick(() => renderCallVideoLayout());
return true;
}
@@ -1217,6 +1235,7 @@ function applyRemoteControlState(payload, sender) {
}
function syncControlStateToPeers() {
+ const backgroundFallbackState = backgroundFallbackControlStateFromPrefs(callMediaPrefs);
const peerIds = connectedParticipantUsers.value
.map((row) => row.userId)
.filter((userId) => (
@@ -1240,6 +1259,7 @@ function syncControlStateToPeers() {
cameraEnabled: controlState.cameraEnabled,
micEnabled: controlState.micEnabled,
screenEnabled: controlState.screenEnabled,
+ ...backgroundFallbackState,
},
},
});
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/roomState.ts b/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/roomState.ts
index 2d1aae765..29ee5051e 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/roomState.ts
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/roomState.ts
@@ -311,9 +311,22 @@ export function createCallWorkspaceRoomStateHelpers(context) {
}
}
+ function uniqueLobbyEntriesByUser(entries) {
+ const rows = new Map();
+ for (const entry of Array.isArray(entries) ? entries : []) {
+ const normalized = normalizeLobbyEntry(entry);
+ const userId = Number(normalized?.user_id || 0);
+ if (!Number.isInteger(userId) || userId <= 0) continue;
+ rows.set(userId, normalized);
+ }
+
+ return Array.from(rows.values());
+ }
+
function applyLobbySnapshot(payload) {
const roomId = normalizeRoomId(payload?.room_id || payload?.roomId || activeRoomId.value);
- const admittedRows = Array.isArray(payload?.admitted) ? payload.admitted.map(normalizeLobbyEntry) : [];
+ const admittedRows = uniqueLobbyEntriesByUser(payload?.admitted);
+ const admittedUserIds = new Set(admittedRows.map((entry) => Number(entry?.user_id || 0)));
const admittedCurrentUser = admittedRows.some((entry) => Number(entry?.user_id || 0) === currentUserId.value);
if (roomId !== activeRoomId.value) {
@@ -335,7 +348,8 @@ export function createCallWorkspaceRoomStateHelpers(context) {
.map((entry) => Number(entry?.user_id || 0))
.filter((userId) => Number.isInteger(userId) && userId > 0)
);
- const nextQueueRows = Array.isArray(payload?.queue) ? payload.queue.map(normalizeLobbyEntry) : [];
+ const nextQueueRows = uniqueLobbyEntriesByUser(payload?.queue)
+ .filter((entry) => !admittedUserIds.has(Number(entry?.user_id || 0)));
lobbyQueue.value = nextQueueRows;
lobbyAdmitted.value = admittedRows;
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/socketLifecycle.ts b/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/socketLifecycle.ts
index 641c9fd8a..33333866f 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/socketLifecycle.ts
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/socketLifecycle.ts
@@ -7,6 +7,11 @@ import { applyGossipTopologyFromRoomStatePayload } from './roomStateTopology';
const WEBSOCKET_NEGOTIATION_TIMEOUT_MS = 5 * 60 * 1000;
const MEDIA_SECURITY_SYNC_REQUEST_SIGNAL_TYPE = 'call/media-security-sync-request';
+const RETRYABLE_RECONNECT_BACKFILL_REASONS = Object.freeze([
+ 'access_session_binding_unavailable',
+ 'realtime_backfill_unavailable',
+ 'websocket_reconnect_backfill_unavailable',
+]);
const STALE_TARGET_PRUNING_SIGNAL_TYPES = Object.freeze([
'call/answer',
'call/ice',
@@ -529,12 +534,33 @@ export function createCallWorkspaceSocketHelpers({
}
const transientAuthBackendError = code === 'websocket_auth_temporarily_unavailable'
|| closeReason === 'auth_backend_error';
- if (transientAuthBackendError) {
+ const transientReconnectBackfillError = code === 'websocket_reconnect_backfill_unavailable'
+ || RETRYABLE_RECONNECT_BACKFILL_REASONS.includes(closeReason);
+ const retryableRealtimeReconnectError = transientAuthBackendError || transientReconnectBackfillError;
+ if (retryableRealtimeReconnectError) {
+ const retryReason = transientAuthBackendError
+ ? 'auth_backend_error'
+ : (closeReason || code || 'websocket_reconnect_retryable_error');
state.manualSocketClose = false;
refs.workspaceError.value = '';
refs.workspaceNotice.value = '';
- refs.connectionReason.value = 'auth_backend_error';
+ refs.connectionReason.value = retryReason;
refs.connectionState.value = 'retrying';
+ captureClientDiagnostic({
+ category: 'realtime',
+ level: 'warning',
+ eventType: 'realtime_websocket_retryable_error',
+ code: code || retryReason,
+ message,
+ payload: {
+ retryable: true,
+ retry_reason: retryReason,
+ requested_room_id: refs.desiredRoomId.value,
+ active_call_id: refs.activeSocketCallId.value,
+ details: payload?.details || {},
+ },
+ immediate: true,
+ });
closeSocketLocal();
scheduleReconnect();
return;
@@ -831,6 +857,21 @@ export function createCallWorkspaceSocketHelpers({
if (originIndex >= orderedSocketOrigins.length) {
refs.connectionState.value = 'retrying';
refs.connectionReason.value = 'socket_unreachable';
+ captureClientDiagnostic({
+ category: 'realtime',
+ level: 'warning',
+ eventType: 'realtime_websocket_retryable_error',
+ code: 'websocket_connect_retry_scheduled',
+ message: 'Realtime websocket connection could not be established; retrying without expiring the session.',
+ payload: {
+ retryable: true,
+ retry_reason: refs.connectionReason.value,
+ requested_room_id: refs.desiredRoomId.value,
+ active_call_id: refs.activeSocketCallId.value,
+ origin_count: orderedSocketOrigins.length,
+ },
+ immediate: true,
+ });
finishConnectInFlight();
scheduleReconnect();
return;
@@ -867,6 +908,25 @@ export function createCallWorkspaceSocketHelpers({
connectWithOriginAt(originIndex + 1);
};
+ const captureRetryableReconnectClose = (retryReason, closeEvent = null) => {
+ captureClientDiagnostic({
+ category: 'realtime',
+ level: 'warning',
+ eventType: 'realtime_websocket_retryable_error',
+ code: retryReason || 'websocket_retryable_close',
+ message: 'Realtime websocket closed with a retryable reconnect condition.',
+ payload: {
+ retryable: true,
+ retry_reason: retryReason || 'websocket_retryable_close',
+ close_code: Number(closeEvent?.code || 0),
+ close_reason: String(closeEvent?.reason || '').trim(),
+ requested_room_id: refs.desiredRoomId.value,
+ active_call_id: refs.activeSocketCallId.value,
+ },
+ immediate: true,
+ });
+ };
+
const failOverToNextOrigin = (closeReason = 'failover') => {
if (failedOver) return;
failedOver = true;
@@ -997,9 +1057,19 @@ export function createCallWorkspaceSocketHelpers({
finishConnectInFlight();
return;
}
+ const retryableBackfillClose = RETRYABLE_RECONNECT_BACKFILL_REASONS.includes(closeReason);
+ if (retryableBackfillClose) {
+ refs.connectionState.value = 'retrying';
+ refs.connectionReason.value = closeReason;
+ captureRetryableReconnectClose(closeReason, event);
+ finishConnectInFlight();
+ scheduleReconnect();
+ return;
+ }
if (closeReason === 'auth_backend_error' || event?.code === 1011) {
refs.connectionState.value = 'retrying';
refs.connectionReason.value = closeReason || 'socket_internal_error';
+ captureRetryableReconnectClose(closeReason || 'socket_internal_error', event);
finishConnectInFlight();
scheduleReconnect();
return;
diff --git a/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/videoLayout.ts b/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/videoLayout.ts
index 88f3d3ceb..a4478c748 100644
--- a/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/videoLayout.ts
+++ b/demo/video-chat/frontend-vue/src/domain/realtime/workspace/callWorkspace/videoLayout.ts
@@ -17,11 +17,13 @@ export function createCallWorkspaceVideoLayoutHelpers({
gridVideoParticipants,
gridVideoSlotId,
hasRenderableMediaForParticipant,
+ hasStaticAvatarForUserId = () => false,
lookupMediaNodeForUserId,
miniVideoParticipants,
miniVideoSlotId,
primaryVideoUserId,
remotePeerMediaNode,
+ staticAvatarNodeForUserId = () => null,
} = callbacks;
let deferredVideoLayoutQueued = false;
@@ -50,6 +52,7 @@ export function createCallWorkspaceVideoLayoutHelpers({
function participantHasRenderableMedia(userId) {
refs.mediaRenderVersion.value;
+ if (hasStaticAvatarForUserId(userId)) return true;
return hasRenderableMediaForParticipant({
currentUserId: refs.currentUserId.value,
localFilteredStream: refs.localFilteredStreamRef.value,
@@ -72,6 +75,10 @@ export function createCallWorkspaceVideoLayoutHelpers({
}
function mediaNodeForUserId(userId) {
+ if (hasStaticAvatarForUserId(userId)) {
+ const avatarNode = staticAvatarNodeForUserId(userId);
+ if (avatarNode instanceof HTMLElement) return avatarNode;
+ }
return lookupMediaNodeForUserId({
currentUserId: refs.currentUserId.value,
localVideoElement: refs.localVideoElement.value,
diff --git a/demo/video-chat/frontend-vue/src/modules/localization/callWorkspaceMessages.js b/demo/video-chat/frontend-vue/src/modules/localization/callWorkspaceMessages.js
index 30668524b..01c7c0266 100644
--- a/demo/video-chat/frontend-vue/src/modules/localization/callWorkspaceMessages.js
+++ b/demo/video-chat/frontend-vue/src/modules/localization/callWorkspaceMessages.js
@@ -18,6 +18,14 @@ export const CALL_WORKSPACE_MESSAGES = Object.freeze({
'calls.workspace.attachment_type_not_allowed': 'File type is not allowed.',
'calls.workspace.attachment_upload_failed': 'Could not upload chat attachment.',
'calls.workspace.attachment_upload_timeout': 'Chat attachment upload timed out after {seconds}s. Try again or use a smaller file / faster connection.',
+ 'calls.workspace.background_avatar_read_failed': 'Could not read the selected avatar image.',
+ 'calls.workspace.background_avatar_type_invalid': 'Avatar image must be PNG, JPEG, or WEBP.',
+ 'calls.workspace.background_choice_apply_failed': 'Could not apply the selected background alternative.',
+ 'calls.workspace.background_send_unfiltered': 'Send unfiltered video',
+ 'calls.workspace.background_unavailable_body': 'Background replacement is unavailable in this browser right now. Choose what others should see instead.',
+ 'calls.workspace.background_unavailable_title': 'Background replacement unavailable',
+ 'calls.workspace.background_upload_avatar': 'Upload avatar',
+ 'calls.workspace.background_use_standard_avatar': 'Use standard avatar',
'calls.workspace.chat': 'Chat',
'calls.workspace.chat_reconnecting': 'Realtime chat is reconnecting. The message is still in the composer.',
'calls.workspace.chat_send_offline': 'Could not send chat message while websocket is offline.',
diff --git a/demo/video-chat/frontend-vue/src/modules/localization/englishMessages.js b/demo/video-chat/frontend-vue/src/modules/localization/englishMessages.js
index b52b64686..dcf4244ba 100644
--- a/demo/video-chat/frontend-vue/src/modules/localization/englishMessages.js
+++ b/demo/video-chat/frontend-vue/src/modules/localization/englishMessages.js
@@ -2,6 +2,8 @@ import { CALL_WORKSPACE_MESSAGES } from './callWorkspaceMessages.js';
import { CALENDAR_MESSAGES } from './calendarMessages.js';
import { GOVERNANCE_MESSAGES } from './governanceMessages.js';
import { INFRASTRUCTURE_MESSAGES } from './infrastructureMessages.js';
+import { LOCALIZATION_ADMIN_MESSAGES } from './localizationAdminMessages.js';
+import { MARKETPLACE_MESSAGES } from './marketplaceMessages.js';
import { PUBLIC_MESSAGES } from './publicMessages.js';
import { USERS_OVERVIEW_MESSAGES } from './usersOverviewMessages.js';
@@ -10,6 +12,8 @@ export const ENGLISH_MESSAGES = Object.freeze({
...CALENDAR_MESSAGES,
...GOVERNANCE_MESSAGES,
...INFRASTRUCTURE_MESSAGES,
+ ...LOCALIZATION_ADMIN_MESSAGES,
+ ...MARKETPLACE_MESSAGES,
...PUBLIC_MESSAGES,
...USERS_OVERVIEW_MESSAGES,
'common.back': 'Back',
@@ -780,69 +784,4 @@ export const ENGLISH_MESSAGES = Object.freeze({
'pagination.next': 'Next',
'pagination.previous': 'Previous',
'pagination.total': 'total',
- 'localization.admin.actions': 'Actions',
- 'localization.admin.code': 'Code',
- 'localization.admin.direction': 'Direction',
- 'localization.admin.edit': 'Edit',
- 'localization.admin.editor_count': '{count} translation entries',
- 'localization.admin.editor_title': 'Translation Editor',
- 'localization.admin.key': 'Key',
- 'localization.admin.language': 'Language',
- 'localization.admin.languages_total': 'languages',
- 'localization.admin.load_data_failed': 'Could not load localization data.',
- 'localization.admin.load_locales_failed': 'Could not load locales.',
- 'localization.admin.load_translations_failed': 'Could not load translations.',
- 'localization.admin.locale': 'Locale',
- 'localization.admin.left_language': 'Left language',
- 'localization.admin.namespace': 'Namespace',
- 'localization.admin.no_languages': 'No languages match the current filter.',
- 'localization.admin.no_translation_changes': 'No translation changes to save.',
- 'localization.admin.request_failed': 'Localization request failed.',
- 'localization.admin.right_language': 'Right language',
- 'localization.admin.save_translations': 'Save translations',
- 'localization.admin.save_translations_failed': 'Could not save translations.',
- 'localization.admin.saving_translations': 'Saving translations...',
- 'localization.admin.search_languages': 'Search languages',
- 'localization.admin.title': 'Localization',
- 'localization.admin.translations_saved': 'Saved {count} translations.',
- 'localization.admin.updated': 'Updated',
- 'localization.admin.value': 'Value',
- 'marketplace.actions': 'Actions',
- 'marketplace.add_app': 'Add marketplace app',
- 'marketplace.all_categories': 'All categories',
- 'marketplace.app_created': 'Marketplace app created.',
- 'marketplace.app_deleted': 'Marketplace app deleted.',
- 'marketplace.app_updated': 'Marketplace app updated.',
- 'marketplace.apps_total': 'apps',
- 'marketplace.category': 'Category',
- 'marketplace.category.assistant': 'Assistant',
- 'marketplace.category.avatar': 'Avatar',
- 'marketplace.category.collaboration': 'Collaboration',
- 'marketplace.category.other': 'Other',
- 'marketplace.category.utility': 'Utility',
- 'marketplace.category.whiteboard': 'Whiteboard',
- 'marketplace.category_filter': 'Category filter',
- 'marketplace.close': 'Close',
- 'marketplace.confirm_delete': 'Delete {name}?',
- 'marketplace.delete_app': 'Delete app',
- 'marketplace.delete_failed': 'Could not delete marketplace app.',
- 'marketplace.description': 'Description',
- 'marketplace.description_placeholder': 'Optional notes about the app, feature scope, or integration path.',
- 'marketplace.edit_app': 'Edit app',
- 'marketplace.empty_filter': 'No marketplace apps match the current filter.',
- 'marketplace.form_subtitle': 'Manage callable marketplace entries for video calls.',
- 'marketplace.load_failed': 'Could not load marketplace apps.',
- 'marketplace.loading': 'Loading marketplace apps...',
- 'marketplace.manufacturer': 'Manufacturer',
- 'marketplace.manufacturer_placeholder': 'Intelligent Intern',
- 'marketplace.name': 'Name',
- 'marketplace.no_website': 'No website',
- 'marketplace.save_failed': 'Could not save marketplace app.',
- 'marketplace.search': 'Search marketplace apps',
- 'marketplace.search_placeholder': 'Search by name, manufacturer, or website',
- 'marketplace.this_app': 'this app',
- 'marketplace.title': 'Marketplace',
- 'marketplace.updated': 'Updated {date}',
- 'marketplace.website': 'Website',
- 'marketplace.website_placeholder': 'https://example.com',
});
diff --git a/demo/video-chat/frontend-vue/src/modules/localization/localizationAdminMessages.js b/demo/video-chat/frontend-vue/src/modules/localization/localizationAdminMessages.js
new file mode 100644
index 000000000..6fdeb2dfb
--- /dev/null
+++ b/demo/video-chat/frontend-vue/src/modules/localization/localizationAdminMessages.js
@@ -0,0 +1,29 @@
+export const LOCALIZATION_ADMIN_MESSAGES = Object.freeze({
+ 'localization.admin.actions': 'Actions',
+ 'localization.admin.code': 'Code',
+ 'localization.admin.direction': 'Direction',
+ 'localization.admin.edit': 'Edit',
+ 'localization.admin.editor_count': '{count} translation entries',
+ 'localization.admin.editor_title': 'Translation Editor',
+ 'localization.admin.key': 'Key',
+ 'localization.admin.language': 'Language',
+ 'localization.admin.languages_total': 'languages',
+ 'localization.admin.load_data_failed': 'Could not load localization data.',
+ 'localization.admin.load_locales_failed': 'Could not load locales.',
+ 'localization.admin.load_translations_failed': 'Could not load translations.',
+ 'localization.admin.locale': 'Locale',
+ 'localization.admin.left_language': 'Left language',
+ 'localization.admin.namespace': 'Namespace',
+ 'localization.admin.no_languages': 'No languages match the current filter.',
+ 'localization.admin.no_translation_changes': 'No translation changes to save.',
+ 'localization.admin.request_failed': 'Localization request failed.',
+ 'localization.admin.right_language': 'Right language',
+ 'localization.admin.save_translations': 'Save translations',
+ 'localization.admin.save_translations_failed': 'Could not save translations.',
+ 'localization.admin.saving_translations': 'Saving translations...',
+ 'localization.admin.search_languages': 'Search languages',
+ 'localization.admin.title': 'Localization',
+ 'localization.admin.translations_saved': 'Saved {count} translations.',
+ 'localization.admin.updated': 'Updated',
+ 'localization.admin.value': 'Value',
+});
diff --git a/demo/video-chat/frontend-vue/src/modules/localization/marketplaceMessages.js b/demo/video-chat/frontend-vue/src/modules/localization/marketplaceMessages.js
new file mode 100644
index 000000000..016180e6e
--- /dev/null
+++ b/demo/video-chat/frontend-vue/src/modules/localization/marketplaceMessages.js
@@ -0,0 +1,52 @@
+export const MARKETPLACE_MESSAGES = Object.freeze({
+ 'marketplace.actions': 'Actions',
+ 'marketplace.add_app': 'Add marketplace app',
+ 'marketplace.all_categories': 'All categories',
+ 'marketplace.app_created': 'Marketplace app created.',
+ 'marketplace.app_deleted': 'Marketplace app deleted.',
+ 'marketplace.app_updated': 'Marketplace app updated.',
+ 'marketplace.apps_total': 'apps',
+ 'marketplace.call_app_install.enable': 'Enable for organization',
+ 'marketplace.call_app_install.install': 'Install for organization',
+ 'marketplace.call_app_install.unhealthy': 'Call App catalog is not healthy',
+ 'marketplace.call_app_install.verify': 'Verify organization installation',
+ 'marketplace.call_app_install_failed': 'Could not install {name} for this organization.',
+ 'marketplace.call_app_installed': '{name} installed and enabled for this organization.',
+ 'marketplace.call_app_state': 'Call App: {state}',
+ 'marketplace.call_app_state.disabled': 'disabled for organization',
+ 'marketplace.call_app_state.installed': 'installed for organization',
+ 'marketplace.call_app_state.not_installed': 'not installed',
+ 'marketplace.call_app_state.ordered': 'ordered, not installed',
+ 'marketplace.call_app_state.unhealthy': 'catalog unhealthy',
+ 'marketplace.category': 'Category',
+ 'marketplace.category.assistant': 'Assistant',
+ 'marketplace.category.avatar': 'Avatar',
+ 'marketplace.category.collaboration': 'Collaboration',
+ 'marketplace.category.other': 'Other',
+ 'marketplace.category.utility': 'Utility',
+ 'marketplace.category.whiteboard': 'Whiteboard',
+ 'marketplace.category_filter': 'Category filter',
+ 'marketplace.close': 'Close',
+ 'marketplace.confirm_delete': 'Delete {name}?',
+ 'marketplace.delete_app': 'Delete app',
+ 'marketplace.delete_failed': 'Could not delete marketplace app.',
+ 'marketplace.description': 'Description',
+ 'marketplace.description_placeholder': 'Optional notes about the app, feature scope, or integration path.',
+ 'marketplace.edit_app': 'Edit app',
+ 'marketplace.empty_filter': 'No marketplace apps match the current filter.',
+ 'marketplace.form_subtitle': 'Manage callable marketplace entries for video calls.',
+ 'marketplace.load_failed': 'Could not load marketplace apps.',
+ 'marketplace.loading': 'Loading marketplace apps...',
+ 'marketplace.manufacturer': 'Manufacturer',
+ 'marketplace.manufacturer_placeholder': 'Intelligent Intern',
+ 'marketplace.name': 'Name',
+ 'marketplace.no_website': 'No website',
+ 'marketplace.save_failed': 'Could not save marketplace app.',
+ 'marketplace.search': 'Search marketplace apps',
+ 'marketplace.search_placeholder': 'Search by name, manufacturer, or website',
+ 'marketplace.this_app': 'this app',
+ 'marketplace.title': 'Marketplace',
+ 'marketplace.updated': 'Updated {date}',
+ 'marketplace.website': 'Website',
+ 'marketplace.website_placeholder': 'https://example.com',
+});
diff --git a/demo/video-chat/frontend-vue/src/modules/marketplace/pages/AdminMarketplaceTable.vue b/demo/video-chat/frontend-vue/src/modules/marketplace/pages/AdminMarketplaceTable.vue
index 8d0093b68..f592ed624 100644
--- a/demo/video-chat/frontend-vue/src/modules/marketplace/pages/AdminMarketplaceTable.vue
+++ b/demo/video-chat/frontend-vue/src/modules/marketplace/pages/AdminMarketplaceTable.vue
@@ -10,12 +10,12 @@
-
+
{{ app.name }}
{{ categoryLabel(app.category) }}
- Call App: {{ callAppStateLabel(app) }}
+ {{ t('marketplace.call_app_state', { state: callAppStateLabel(app) }) }}
@@ -34,15 +34,17 @@
v-if="catalogApp(app)"
icon="/assets/orgas/kingrt/icons/add.png"
:title="installTitle(app)"
- :disabled="installingAppId === app.id || !canInstallCallApp(app)"
+ :disabled="installingAppKey === callAppKey(app) || !canInstallCallApp(app)"
@click="$emit('install-call-app', app)"
/>
0) return `marketplace:${appId}`;
+ const catalogKey = callAppKey(app);
+ return catalogKey === '' ? `catalog:${String(app?.name || '')}` : `catalog:${catalogKey}`;
+}
+
+function isCatalogOnly(app) {
+ return app?.catalog_only === true || String(app?.source || '') === 'call_app_catalog';
+}
+
function organizationState(app) {
const catalog = catalogApp(app);
const organization = catalog?.organization;
@@ -134,18 +151,18 @@ function canInstallCallApp(app) {
function callAppStateLabel(app) {
const state = organizationState(app);
- if (state.installed === true) return 'installed for organization';
- if (state.status === 'disabled') return 'disabled for organization';
- if (state.ordered === true) return 'ordered, not installed';
- if (!isCatalogHealthy(app)) return 'catalog unhealthy';
- return 'not installed';
+ if (state.installed === true) return t('marketplace.call_app_state.installed');
+ if (state.status === 'disabled') return t('marketplace.call_app_state.disabled');
+ if (state.ordered === true) return t('marketplace.call_app_state.ordered');
+ if (!isCatalogHealthy(app)) return t('marketplace.call_app_state.unhealthy');
+ return t('marketplace.call_app_state.not_installed');
}
function installTitle(app) {
- if (isInstalled(app)) return 'Verify organization installation';
- if (!isCatalogHealthy(app)) return 'Call App catalog is not healthy';
- if (organizationState(app).status === 'disabled') return 'Enable for organization';
- return 'Install for organization';
+ if (isInstalled(app)) return t('marketplace.call_app_install.verify');
+ if (!isCatalogHealthy(app)) return t('marketplace.call_app_install.unhealthy');
+ if (organizationState(app).status === 'disabled') return t('marketplace.call_app_install.enable');
+ return t('marketplace.call_app_install.install');
}
diff --git a/demo/video-chat/frontend-vue/src/modules/marketplace/pages/AdminMarketplaceView.vue b/demo/video-chat/frontend-vue/src/modules/marketplace/pages/AdminMarketplaceView.vue
index e6ff09183..a755cde81 100644
--- a/demo/video-chat/frontend-vue/src/modules/marketplace/pages/AdminMarketplaceView.vue
+++ b/demo/video-chat/frontend-vue/src/modules/marketplace/pages/AdminMarketplaceView.vue
@@ -24,7 +24,7 @@
v-else
:rows="rows"
:mutating-app-id="mutatingAppId"
- :installing-app-id="installingAppId"
+ :installing-app-key="installingAppKey"
@install-call-app="installCallApp"
@edit-app="openEditApp"
@delete-app="deleteApp"
@@ -134,7 +134,7 @@ const apiRequest = createAdminMarketplaceApi({ router });
const categoryFilter = ref('all');
const notice = ref('');
const mutatingAppId = ref(0);
-const installingAppId = ref(0);
+const installingAppKey = ref('');
const sidePanelForm = useAdminSidePanelForm();
const dialogOpen = sidePanelForm.open;
const formSaving = sidePanelForm.saving;
@@ -253,15 +253,14 @@ async function deleteApp(app) {
}
async function installCallApp(app) {
- const appId = Number(app?.id || 0);
const catalog = app && typeof app === 'object' && app.call_app_catalog && typeof app.call_app_catalog === 'object'
? app.call_app_catalog
: null;
const appKey = String(catalog?.app_key || '').trim();
- if (appId <= 0 || appKey === '') return;
+ if (appKey === '') return;
const label = String(app?.name || appKey).trim() || appKey;
- installingAppId.value = appId;
+ installingAppKey.value = appKey;
error.value = '';
try {
await apiRequest(`/api/marketplace/call-apps/${encodeURIComponent(appKey)}/orders`, {
@@ -274,12 +273,12 @@ async function installCallApp(app) {
config: {},
},
});
- notice.value = `${label} installed and enabled for this organization.`;
+ notice.value = t('marketplace.call_app_installed', { name: label });
await loadRows();
} catch (err) {
- error.value = err instanceof Error ? err.message : `Could not install ${label} for this organization.`;
+ error.value = err instanceof Error ? err.message : t('marketplace.call_app_install_failed', { name: label });
} finally {
- installingAppId.value = 0;
+ installingAppKey.value = '';
}
}
diff --git a/demo/video-chat/frontend-vue/src/support/backendOrigin.ts b/demo/video-chat/frontend-vue/src/support/backendOrigin.ts
index dc834c0fc..8a28761b2 100644
--- a/demo/video-chat/frontend-vue/src/support/backendOrigin.ts
+++ b/demo/video-chat/frontend-vue/src/support/backendOrigin.ts
@@ -51,15 +51,17 @@ function parseBooleanEnv(value, fallback = false) {
const allowInsecureWebSockets = parseBooleanEnv(import.meta.env.VITE_VIDEOCHAT_ALLOW_INSECURE_WS, false);
-function resolveProductionBackendOriginForHost(hostname, protocol) {
+function resolveProductionServiceOriginForHost(hostname, protocol, service) {
const host = String(hostname || '').trim().toLowerCase();
const scheme = String(protocol || '').trim().toLowerCase() === 'https:' ? 'https' : '';
+ const serviceKey = String(service || '').trim().toLowerCase();
+ if (!['api', 'ws', 'sfu'].includes(serviceKey)) return '';
if (scheme !== 'https' || host === '') return '';
if (host === 'app.kingrt.com') {
- return 'https://api.app.kingrt.com';
+ return `https://${serviceKey}.kingrt.com`;
}
- if (host.endsWith('.app.kingrt.com') && !host.startsWith('api.')) {
- return `https://api.${host}`;
+ if (host.endsWith('.kingrt.com') && !host.startsWith(`${serviceKey}.`)) {
+ return `https://${serviceKey}.kingrt.com`;
}
return '';
}
@@ -85,7 +87,7 @@ function detectDefaultBackendOrigin() {
if (typeof window !== 'undefined') {
const protocol = window.location.protocol === 'https:' ? 'https' : 'http';
const host = String(window.location.hostname || 'localhost').trim() || 'localhost';
- const productionOrigin = resolveProductionBackendOriginForHost(host, window.location.protocol);
+ const productionOrigin = resolveProductionServiceOriginForHost(host, window.location.protocol, 'api');
if (productionOrigin !== '') {
return productionOrigin;
}
@@ -107,6 +109,10 @@ function detectDefaultBackendWebSocketOrigin() {
if (typeof window !== 'undefined') {
const protocol = window.location.protocol === 'https:' ? 'https' : 'http';
const host = String(window.location.hostname || 'localhost').trim() || 'localhost';
+ const productionOrigin = resolveProductionServiceOriginForHost(host, window.location.protocol, 'ws');
+ if (productionOrigin !== '') {
+ return productionOrigin;
+ }
return `${protocol}://${host}:${inferredWsPort}`;
}
@@ -131,6 +137,10 @@ function detectDefaultBackendSfuOrigin() {
if (typeof window !== 'undefined') {
const protocol = window.location.protocol === 'https:' ? 'https' : 'http';
const host = String(window.location.hostname || 'localhost').trim() || 'localhost';
+ const productionOrigin = resolveProductionServiceOriginForHost(host, window.location.protocol, 'sfu');
+ if (productionOrigin !== '') {
+ return productionOrigin;
+ }
return `${protocol}://${host}:${inferredSfuPort}`;
}
diff --git a/demo/video-chat/frontend-vue/tests/contract/backend-origin-production-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/backend-origin-production-contract.mjs
index e24ff8d2d..205e97668 100644
--- a/demo/video-chat/frontend-vue/tests/contract/backend-origin-production-contract.mjs
+++ b/demo/video-chat/frontend-vue/tests/contract/backend-origin-production-contract.mjs
@@ -27,7 +27,7 @@ function loadBackendOrigin(runtimeEnv = {}, windowValue = undefined) {
return new Function(
'runtimeEnv',
'window',
- `${source}; return { resolveBackendOrigin, resolveBackendOriginCandidates };`,
+ `${source}; return { resolveBackendOrigin, resolveBackendWebSocketOrigin, resolveBackendSfuOrigin, resolveBackendOriginCandidates };`,
)(runtimeEnv, windowValue);
}
@@ -40,8 +40,18 @@ try {
});
assert.equal(
productionBackend.resolveBackendOrigin(),
- 'https://api.app.kingrt.com',
- 'production app host must use the public API origin, not the frontend origin',
+ 'https://api.kingrt.com',
+ 'production app host must use the service-root API origin, not derive API from app.kingrt.com',
+ );
+ assert.equal(
+ productionBackend.resolveBackendWebSocketOrigin(),
+ 'https://ws.kingrt.com',
+ 'production app host must use the service-root WS origin, not derive WS from app.kingrt.com',
+ );
+ assert.equal(
+ productionBackend.resolveBackendSfuOrigin(),
+ 'https://sfu.kingrt.com',
+ 'production app host must use the service-root SFU origin, not derive SFU from app.kingrt.com',
);
const explicitBackend = loadBackendOrigin({ VITE_VIDEOCHAT_BACKEND_ORIGIN: 'https://custom-api.example.test' }, {
@@ -69,10 +79,18 @@ try {
);
const deployScript = readVideoChat('scripts/deploy.sh');
+ assert.ok(
+ deployScript.includes('DEPLOY_APP_DOMAIN="${DEPLOY_APP_DOMAIN:-app.${DEPLOY_DOMAIN}}"'),
+ 'deploy config must split the frontend app domain from the service base/root domain',
+ );
assert.ok(
deployScript.includes('VIDEOCHAT_V1_BACKEND_ORIGIN=https://\\${API_DOMAIN}'),
'generated production env must build the frontend against the API domain',
);
+ assert.ok(
+ !deployScript.includes('api.${APP_DOMAIN}') && !deployScript.includes('api.${DEPLOY_APP_DOMAIN}'),
+ 'generated service origins must not derive API from the frontend app domain',
+ );
assert.ok(
deployScript.includes('set_env_value VIDEOCHAT_V1_BACKEND_ORIGIN "https://\\${API_DOMAIN}"'),
'production env refresh must keep the API domain backend origin',
diff --git a/demo/video-chat/frontend-vue/tests/contract/background-aspect-preservation-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/background-aspect-preservation-contract.mjs
index 2c5d38f63..01e471275 100644
--- a/demo/video-chat/frontend-vue/tests/contract/background-aspect-preservation-contract.mjs
+++ b/demo/video-chat/frontend-vue/tests/contract/background-aspect-preservation-contract.mjs
@@ -18,17 +18,17 @@ function read(relativePath) {
try {
const packageJson = read('package.json');
const backgroundStream = read('src/domain/realtime/background/stream.ts');
- const compositorShared = read('src/domain/realtime/background/pipeline/compositorShared.js');
- const compositorCanvas = read('src/domain/realtime/background/pipeline/compositorCanvasStage.js');
+ const compositorStage = read('src/domain/realtime/background/pipeline/compositorStage.js');
assert.ok(packageJson.includes('background-aspect-preservation-contract.mjs'), 'package script must expose background aspect contract');
assert.ok(backgroundStream.includes('function resolveVideoSourceDimensions(video, settings = {})'), 'background stream must prefer real video dimensions over track settings');
assert.ok(backgroundStream.includes('const videoWidth = Math.max(0, Math.round(toNumber(video?.videoWidth, 0)));'), 'background stream must read portrait videoWidth after metadata');
assert.ok(backgroundStream.includes('function syncCanvasToSourceFrame(nextSourceWidth, nextSourceHeight)'), 'background stream must resize its output canvas when mobile orientation metadata changes');
assert.ok(backgroundStream.includes('const segmentationWidth = Math.max(1, Math.round(canvas.width));'), 'background segmentation must use the aspect-preserving canvas size');
- assert.ok(compositorShared.includes('function drawContainImage(ctx, image, width, height)'), 'background compositor must have aspect-preserving contain drawing');
- assert.ok(compositorShared.includes('function drawCoverImage(ctx, image, width, height)'), 'background compositor must have aspect-preserving cover drawing');
- assert.ok(!compositorCanvas.includes('ctx.drawImage(video, 0, 0, canvas.width, canvas.height);'), 'background compositor must not stretch mobile portrait video directly into landscape canvas');
+ assert.ok(compositorStage.includes('function drawContainImage(ctx, image, width, height)'), 'background compositor must have aspect-preserving contain drawing');
+ assert.ok(compositorStage.includes('function drawCoverImage(ctx, image, width, height)'), 'background compositor must have aspect-preserving cover drawing');
+ assert.ok(compositorStage.includes('drawContainImage(ctx, foregroundSource, canvas.width, canvas.height);'), 'background compositor must contain foreground frames');
+ assert.ok(!compositorStage.includes('ctx.drawImage(video, 0, 0, canvas.width, canvas.height);'), 'background compositor must not stretch mobile portrait video directly into landscape canvas');
process.stdout.write('[background-aspect-preservation-contract] PASS\n');
} catch (error) {
diff --git a/demo/video-chat/frontend-vue/tests/contract/background-filter-mask-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/background-filter-mask-contract.mjs
index ca921c8bf..28457d5c2 100644
--- a/demo/video-chat/frontend-vue/tests/contract/background-filter-mask-contract.mjs
+++ b/demo/video-chat/frontend-vue/tests/contract/background-filter-mask-contract.mjs
@@ -11,6 +11,10 @@ function readUtf8(relativePath) {
return fs.readFileSync(path.join(frontendRoot, relativePath), 'utf8');
}
+function exists(relativePath) {
+ return fs.existsSync(path.join(frontendRoot, relativePath));
+}
+
function requireContains(source, needle, label) {
assert.ok(source.includes(needle), `${label} missing: ${needle}`);
}
@@ -21,61 +25,69 @@ function requireMissing(source, needle, label) {
try {
const stream = readUtf8('src/domain/realtime/background/stream.ts');
- const maskPostprocess = readUtf8('src/domain/realtime/background/maskPostprocess.js');
- const compositorStage = readUtf8('src/domain/realtime/background/pipeline/compositorStage.js');
- const compositorShared = readUtf8('src/domain/realtime/background/pipeline/compositorShared.js');
- const compositorCanvas = readUtf8('src/domain/realtime/background/pipeline/compositorCanvasStage.js');
- const compositorWebgl = readUtf8('src/domain/realtime/background/pipeline/compositorWebglStage.js');
- const controller = readUtf8('src/domain/realtime/background/controller.ts');
- const segmenter = readUtf8('src/domain/realtime/background/pipeline/segmenterStage.js');
- const sinetBackend = readUtf8('src/domain/realtime/background/backendSinetWasm.js');
+ const compositor = readUtf8('src/domain/realtime/background/pipeline/compositorStage.js');
+ const workerBackend = readUtf8('src/domain/realtime/background/backendWorkerSegmenter.js');
+ const worker = readUtf8('src/domain/realtime/background/workers/imageSegmenterWorker.js');
+ const orchestration = readUtf8('src/domain/realtime/local/mediaOrchestration.ts');
+ const avatarFallback = readUtf8('src/domain/realtime/background/avatarFallbackSignal.ts');
+ const staticAvatarRender = readUtf8('src/domain/realtime/background/staticAvatarRender.ts');
+ const unavailablePrompt = readUtf8('src/domain/realtime/background/unavailablePrompt.ts');
+ const preferences = readUtf8('src/domain/realtime/media/preferences.ts');
+ const modal = readUtf8('src/domain/realtime/background/BackgroundReplacementUnavailableModal.vue');
+ const participantUi = readUtf8('src/domain/realtime/workspace/callWorkspace/participantUi.ts');
+ const videoLayout = readUtf8('src/domain/realtime/workspace/callWorkspace/videoLayout.ts');
+ const workspace = readUtf8('src/domain/realtime/CallWorkspaceView.vue');
+ const template = readUtf8('src/domain/realtime/CallWorkspaceView.template.html');
+
+ assert.equal(exists('src/domain/realtime/background/backendSinetWasm.js'), false, 'production SINet backend must not be revived');
+ assert.equal(exists('src/domain/realtime/background/backendSelector.ts'), false, 'production SINet selector must not be revived');
+ assert.equal(exists('src/domain/realtime/background/maskPostprocess.js'), false, 'deleted SINet matte postprocess must not be revived');
+
+ requireContains(stream, "import { acquireWorkerSegmenterBackendLease } from './backendWorkerSegmenter';", 'Pierre worker segmenter stream');
+ requireContains(stream, "requested: 'worker-segmenter'", 'worker segmenter diagnostics');
+ requireContains(stream, 'function notifySegmentationUnavailable(reason, failures = [])', 'segmentation unavailable callback');
+ requireContains(stream, "handle.reason = 'segmentation_unavailable';", 'segmentation unavailable handle state');
+ requireContains(stream, "mode: hasRenderableMatte ? runtimeConfig.mode : 'off'", 'source-visible warmup and failure rendering');
+ requireContains(stream, 'for (const audioTrack of sourceStream.getAudioTracks()) out.addTrack(audioTrack);', 'audio track preservation');
+ requireMissing(stream, 'createSinetWasmSegmentationBackend', 'SINet production backend');
+ requireMissing(stream, "requested: 'sinet-wasm'", 'SINet diagnostics');
- requireContains(maskPostprocess, 'const DEFAULT_INNER_CONTRACT_PX = 16;', 'background filter contour contraction');
- requireContains(maskPostprocess, 'const DEFAULT_INNER_FEATHER_PX = 24;', 'background filter contour feather');
- requireContains(maskPostprocess, 'function smoothstep(edge0, edge1, value)', 'background filter contour-only smoothing');
- requireContains(maskPostprocess, 'const edgeLow = 0.5 - contourHalfWidth', 'background filter contour low edge');
- requireContains(maskPostprocess, 'function sampleInnerFeatherRamp(progress) {', 'background filter stepped feather ramp');
- requireContains(maskPostprocess, 'function buildInnerDistanceFeatherAlpha(base, width, height, threshold = 110) {', 'shared contour shaping helper');
- requireContains(maskPostprocess, 'const inside = sampleInnerFeatherRamp(t);', 'stepped feather ramp application');
- requireMissing(maskPostprocess, '(raw - 0.5) * contrast + 0.5', 'mask global contrast alpha lift');
+ requireContains(compositor, 'float maskAlpha = uHasMask == 1 ? smoothstep(uMaskLow, uMaskHigh, featherMask(vUv)) : 0.0;', 'contour alpha smoothing');
+ requireContains(compositor, 'gl_FragColor = vec4(mix(background.rgb, frame.rgb, maskAlpha), 1.0);', 'hard foreground/background composite');
+ requireMissing(compositor, 'Math.exp', 'compositor sigmoid or softmax');
+ requireMissing(compositor, 'softmax', 'compositor softmax');
+ requireMissing(compositor, 'sigmoid', 'compositor sigmoid');
- requireContains(compositorShared, 'buildInnerDistanceFeatherAlpha(sourceAlpha, sourceWidth, sourceHeight)', 'bitmap matte contour shaping');
- requireContains(compositorShared, 'return buildInnerDistanceFeatherMaskValues(mask, width, height);', 'value matte contour shaping');
- requireMissing(compositorShared, 'function blurMask(', 'secondary contour blur');
- requireContains(compositorStage, 'createWebGlBackgroundCompositorStage(options)', 'WebGL compositor preference');
- requireContains(compositorStage, 'createCanvasBackgroundCompositorStage(options)', 'canvas compositor fallback');
- requireContains(compositorCanvas, 'getShowSourceUntilMask?.() === true', 'canvas preview warmup source policy');
- requireContains(compositorCanvas, "ctx.fillStyle = '#061a4a';", 'canvas replacement warmup privacy placeholder');
- requireMissing(compositorCanvas, 'if (hasMatteMask && !maskUpdated) return;', 'canvas stale-mask frame freeze');
- requireContains(compositorWebgl, 'float maskAlpha = uHasMask == 1 ? readMask(vUv) : 0.0;', 'WebGL direct shaped alpha mask');
- requireContains(compositorWebgl, "const warmupPlaceholder = !hasRenderableMask && mode === 'replace' && !showSourceUntilMask;", 'WebGL replacement warmup privacy placeholder');
- requireMissing(compositorWebgl, 'smoothstep(uMaskLow, uMaskHigh', 'WebGL shader global mask blending');
- requireMissing(compositorWebgl, 'if (hasMatteMask && !maskUpdated', 'WebGL stale-mask frame freeze');
+ requireContains(workerBackend, "kind: 'worker-segmenter'", 'Pierre worker backend identity');
+ requireContains(workerBackend, "const workerUrl = new URL('./workers/imageSegmenterWorker.js', import.meta.url);", 'worker module boundary');
+ requireContains(worker, 'ImageSegmenter.createFromOptions', 'MediaPipe worker boundary');
+ requireContains(worker, "delegate: delegate === 'GPU' ? 'GPU' : 'CPU'", 'MediaPipe delegate boundary');
+ requireContains(worker, "const glCtx = renderCanvas.getContext('webgl2');", 'MediaPipe category-mask WebGL boundary');
+ requireMissing(worker, 'Math.exp', 'worker softmax/sigmoid fallback');
+ requireMissing(worker, 'softmax', 'worker softmax fallback');
+ requireMissing(worker, 'sigmoid', 'worker sigmoid fallback');
- requireContains(stream, "import { createSinetWasmSegmentationBackend } from './backendSinetWasm';", 'background stream SINet WASM backend');
- requireContains(stream, 'const BACKGROUND_FILTER_READY_TIMEOUT_MS = 500;', 'background stream bounded ready handoff');
- requireContains(stream, 'const ready = new Promise((resolve) => {', 'background stream readiness promise');
- requireContains(stream, 'const readyTimer = setTimeout(', 'background stream readiness timeout');
- requireContains(stream, 'segmentationBackend = await createSinetWasmSegmentationBackend({', 'background stream lazy SINet acquisition');
- requireContains(stream, "requested: 'sinet-wasm'", 'background stream SINet diagnostics name');
- requireContains(stream, 'maskContrast: runtimeConfig.maskContrast,', 'background stream mask contrast controls');
- requireContains(stream, 'averageRadius: runtimeConfig.averageRadius,', 'background stream Gaussian averaging controls');
- requireContains(stream, 'temporalRise: runtimeConfig.temporalRise,', 'background stream temporal rise controls');
- requireContains(stream, 'temporalFall: runtimeConfig.temporalFall,', 'background stream temporal fall controls');
- requireContains(stream, 'getShowSourceUntilMask: () => runtimeConfig.showSourceUntilMask,', 'background stream preview warmup policy');
- requireContains(stream, "Object.prototype.hasOwnProperty.call(nextOptions, 'showSourceUntilMask')", 'background stream preserves preview warmup policy on config update');
- requireMissing(stream, 'acquireWorkerSegmenterBackendLease', 'background stream MediaPipe worker lease');
- requireMissing(stream, 'backendWorkerSegmenter', 'background stream MediaPipe worker backend');
- requireMissing(stream, 'backendMediapipe', 'background stream legacy MediaPipe backend import');
- requireMissing(stream, 'backendTfjs', 'background stream legacy TFJS backend import');
- requireContains(stream, 'ready,', 'background filter stream handle readiness promise');
- requireContains(controller, 'await handle.ready;', 'background filter controller ready handoff');
- requireContains(segmenter, 'latestMaskValues = hasValueMask ? segmentation.matteMaskValues : null;', 'segmenter keeps latest value mask');
- requireContains(sinetBackend, "executionProviders: ['wasm']", 'SINet backend local WASM execution');
- requireContains(sinetBackend, 'pendingMaskValues = alphaToFloatMask(alpha);', 'SINet backend value masks');
- requireContains(sinetBackend, 'function binaryForegroundAlpha(value, threshold = 0)', 'SINet no-softmax foreground classification');
- requireMissing(sinetBackend, 'Math.exp(bg - max)', 'SINet softmax');
- requireMissing(sinetBackend, '1 / (1 + Math.exp', 'SINet sigmoid');
+ requireContains(orchestration, 'handleBackgroundReplacementUnavailable({', 'modal prompt trigger');
+ requireContains(unavailablePrompt, 'openBackgroundReplacementUnavailablePrompt({', 'prompt state update');
+ requireContains(unavailablePrompt, "eventType: 'local_background_replacement_unavailable'", 'field diagnostic');
+ requireContains(orchestration, 'createBackgroundFallbackAudioOnlyStream(rawStream)', 'avatar placeholder audio-only local stream');
+ requireContains(orchestration, "backgroundFilterBackend = 'avatar_placeholder'", 'avatar backend state');
+ requireContains(orchestration, 'syncBackgroundFallbackControlState(true)', 'avatar control-state signal');
+ requireContains(avatarFallback, 'for (const audioTrack of sourceStream.getAudioTracks())', 'avatar fallback audio preservation');
+ requireContains(avatarFallback, 'out.addTrack(audioTrack);', 'avatar fallback audio track copy');
+ requireMissing(avatarFallback, 'captureStream', 'avatar fallback video stream');
+ requireContains(staticAvatarRender, "node.dataset.callStaticAvatar = '1';", 'static avatar render node');
+ requireContains(participantUi, 'backgroundFallbackControlStateFromPrefs(callMediaPrefs)', 'control-state includes static avatar mode');
+ requireContains(videoLayout, 'if (hasStaticAvatarForUserId(userId))', 'video layout static avatar route');
+ requireContains(preferences, 'DEFAULT_BACKGROUND_FALLBACK_AVATAR_URL', 'standard avatar fallback');
+ requireContains(preferences, 'backgroundReplacementUnavailablePromptOpen', 'modal visibility state');
+ requireContains(preferences, 'useCallBackgroundFallbackAvatar', 'avatar choice action');
+ requireContains(preferences, 'clearCallBackgroundFallbackVideo', 'unfiltered choice action');
+ requireContains(modal, 'background_use_standard_avatar', 'standard avatar button');
+ requireContains(modal, 'background_upload_avatar', 'upload avatar button');
+ requireContains(modal, 'background_send_unfiltered', 'unfiltered video button');
+ requireContains(workspace, "import BackgroundReplacementUnavailableModal from './background/BackgroundReplacementUnavailableModal.vue';", 'workspace modal import');
+ requireContains(template, ' ', 'workspace modal mount');
console.log('[background-filter-mask-contract] PASS');
} catch (error) {
diff --git a/demo/video-chat/frontend-vue/tests/contract/background-king-wasm-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/background-king-wasm-contract.mjs
index 9e99d1fd6..9f9828eb1 100644
--- a/demo/video-chat/frontend-vue/tests/contract/background-king-wasm-contract.mjs
+++ b/demo/video-chat/frontend-vue/tests/contract/background-king-wasm-contract.mjs
@@ -21,51 +21,54 @@ function requireMissing(source, needle, label) {
try {
const stream = readUtf8('src/domain/realtime/background/stream.ts');
- const backend = readUtf8('src/domain/realtime/background/backendSinetWasm.js');
- const postprocess = readUtf8('src/domain/realtime/background/maskPostprocess.js');
- const selector = readUtf8('src/domain/realtime/background/backendSelector.ts');
+ const backend = readUtf8('src/domain/realtime/background/backendWorkerSegmenter.js');
+ const worker = readUtf8('src/domain/realtime/background/workers/imageSegmenterWorker.js');
+ const orchestration = readUtf8('src/domain/realtime/local/mediaOrchestration.ts');
+ const avatarSignal = readUtf8('src/domain/realtime/background/avatarFallbackSignal.ts');
+ const videoLayout = readUtf8('src/domain/realtime/workspace/callWorkspace/videoLayout.ts');
+ const unavailablePrompt = readUtf8('src/domain/realtime/background/unavailablePrompt.ts');
+ const modal = readUtf8('src/domain/realtime/background/BackgroundReplacementUnavailableModal.vue');
+ const messages = readUtf8('src/modules/localization/callWorkspaceMessages.js');
- requireContains(stream, "import { createSinetWasmSegmentationBackend } from './backendSinetWasm';", 'production background stream');
- requireContains(stream, 'segmentationBackend = await createSinetWasmSegmentationBackend({', 'production backend construction');
- requireMissing(stream, 'createMediaPipeSegmentationBackend', 'production background stream');
- requireMissing(stream, 'createTfjsSegmentationBackend', 'production background stream');
+ requireContains(stream, "import { acquireWorkerSegmenterBackendLease } from './backendWorkerSegmenter';", 'production background stream');
+ requireContains(stream, 'segmentationBackendLease = await acquireWorkerSegmenterBackendLease({', 'production backend lease');
+ requireContains(stream, 'if (segmentationBackendInitPromise) return segmentationBackendInitPromise;', 'idempotent backend init');
+ requireContains(stream, 'notifySegmentationUnavailable', 'segmentation unavailable notification');
+ requireContains(stream, "mode: hasRenderableMatte ? runtimeConfig.mode : 'off'", 'source-visible warmup/failure mode');
+ requireMissing(stream, 'createSinetWasmSegmentationBackend', 'production stream SINet backend');
+ requireMissing(stream, 'backendSelector', 'production stream backend selector');
- requireContains(backend, "import('onnxruntime-web/wasm')", 'SINet WASM backend runtime');
- requireContains(backend, 'function configureOrtWasmRuntime', 'SINet WASM runtime guard');
- requireContains(backend, 'wasm.proxy = false;', 'SINet WASM runtime must not depend on ORT proxy workers');
- requireContains(backend, 'wasm.numThreads = 1;', 'SINet WASM runtime must not depend on SharedArrayBuffer threading');
- requireContains(backend, 'const SINET_MODEL_WIDTH = 256;', 'SINet model width');
- requireContains(backend, 'const SINET_MODEL_HEIGHT = 256;', 'SINet model height');
- requireContains(backend, 'const SINET_GRAPH_URL = `${VIDEOCHAT_CDN_ORIGIN}${SINET_ASSET_BASE_PATH}sinet-float.onnx`;', 'vendored SINet graph');
- requireContains(backend, 'const SINET_EXTERNAL_WEIGHTS_URL = `${VIDEOCHAT_CDN_ORIGIN}${SINET_ASSET_BASE_PATH}sinet.data`;', 'vendored SINet weights');
- requireContains(backend, "externalData: [{ path: SINET_EXTERNAL_WEIGHTS_PATH, data: weights }]", 'explicit SINet external data mount');
- requireContains(backend, "executionProviders: ['wasm']", 'SINet backend must use local WASM execution');
- requireContains(backend, 'sinetForegroundAlpha', 'SINet foreground conversion');
- requireContains(backend, 'probabilityLike', 'SINet foreground conversion must avoid softmaxing probability outputs');
- requireContains(backend, 'function binaryForegroundAlpha(value, threshold = 0)', 'SINet foreground conversion must use hard foreground classification');
- requireMissing(backend, 'Math.exp(bg - max)', 'SINet foreground conversion');
- requireMissing(backend, 'fgExp / Math.max', 'SINet foreground conversion');
- requireContains(backend, 'shapeForegroundAlpha', 'SINet matte shaping controls');
- requireContains(backend, "kind: 'sinet_wasm'", 'SINet WASM backend identity');
- requireContains(postprocess, 'function gaussianAverageAlpha', 'mask local Gaussian averaging');
- requireContains(postprocess, 'function smoothstep(edge0, edge1, value)', 'mask contour transition');
- requireContains(postprocess, 'const edgeLow = 0.5 - contourHalfWidth', 'mask contour band');
- requireMissing(postprocess, '(raw - 0.5) * contrast + 0.5', 'mask global contrast alpha lift');
- requireContains(postprocess, 'controls?.contrast ?? 0.75', 'mask contrast default');
- requireContains(postprocess, 'Number(controls?.averageRadius ?? 6)', 'wide default Gaussian radius');
- requireContains(backend, 'opts.maskContrast ?? 0.75', 'production mask contrast fallback');
- requireContains(backend, 'opts.averageRadius ?? 6', 'production average radius fallback');
- requireMissing(postprocess, 'blackPoint', 'mask postprocess');
- requireMissing(postprocess, 'whitePoint', 'mask postprocess');
- requireMissing(postprocess, 'threshold ? value : 0', 'mask postprocess');
- requireMissing(postprocess, 'keepDominantComponents', 'mask postprocess');
- requireMissing(postprocess, 'fillEnclosedHoles', 'mask postprocess');
- requireContains(postprocess, 'previousAlpha', 'mask temporal averaging');
- requireContains(selector, "backend: 'sinet_wasm'", 'backend selector');
- requireMissing(selector, 'center_mask_fallback', 'backend selector');
- requireMissing(selector, 'face_detector', 'backend selector');
+ requireContains(backend, 'Worker-based MediaPipe segmentation backend.', 'Pierre worker backend documentation');
+ requireContains(backend, 'SHARED_BACKEND_IDLE_TTL_MS = 60000', 'shared backend warm retention');
+ requireContains(backend, 'acquireWorkerSegmenterBackendLease', 'exclusive backend lease API');
+ requireContains(backend, 'await backend.resetSession?.();', 'lease reset before reuse');
+ requireContains(backend, "kind: 'worker-segmenter'", 'worker backend identity');
+ requireContains(backend, 'queueLatestFrame(frameParams);', 'latest-frame queue under worker pressure');
- console.log('[background-king-wasm-contract] PASS production uses SINet WASM segmentation');
+ requireContains(worker, 'loadModuleFactory(resolvedWasm);', 'MediaPipe wasm factory init');
+ requireContains(worker, 'sanitizeFilesetPaths(await FilesetResolver.forVisionTasks(resolvedWasm))', 'Vite-safe fileset paths');
+ requireContains(worker, 'modelAssetBuffer: new Uint8Array(modelBuffer)', 'local model buffer load');
+ requireContains(worker, 'outputCategoryMask: true', 'category mask output');
+ requireContains(worker, 'outputConfidenceMasks: true', 'confidence mask fallback output');
+ requireMissing(worker, 'cdn.jsdelivr.net', 'worker CDN source');
+ requireMissing(worker, 'unpkg.com', 'worker CDN source');
+
+ requireContains(orchestration, 'onSegmentationUnavailable: (details = {}) => {', 'local media prompt hook');
+ requireContains(orchestration, 'handleBackgroundReplacementUnavailable({', 'prompt handler call');
+ requireContains(unavailablePrompt, 'openBackgroundReplacementUnavailablePrompt({', 'prompt state update');
+ requireContains(unavailablePrompt, "eventType: 'local_background_replacement_unavailable'", 'field diagnostic');
+ requireContains(orchestration, 'createBackgroundFallbackAudioOnlyStream(rawStream)', 'avatar fallback keeps only audio stream');
+ requireContains(orchestration, 'syncBackgroundFallbackControlState(true)', 'avatar fallback sends static state');
+ requireContains(avatarSignal, 'backgroundFallbackControlStateFromPrefs', 'static avatar control-state payload');
+ requireContains(videoLayout, 'staticAvatarNodeForUserId(userId)', 'static avatar tile rendering');
+ requireContains(modal, 'useDefaultAvatar', 'standard avatar action');
+ requireContains(modal, 'handleAvatarFile', 'uploaded avatar action');
+ requireContains(modal, 'sendUnfilteredVideo', 'unfiltered video action');
+ requireContains(modal, 'clearCallBackgroundFallbackVideo();', 'unfiltered choice support');
+ requireContains(messages, 'calls.workspace.background_unavailable_title', 'modal localization title');
+ requireContains(messages, 'calls.workspace.background_send_unfiltered', 'unfiltered localization');
+
+ console.log('[background-king-wasm-contract] PASS production uses Pierre worker pipeline with explicit user alternative');
} catch (error) {
console.error(`[background-king-wasm-contract] FAIL: ${error.message}`);
process.exit(1);
diff --git a/demo/video-chat/frontend-vue/tests/contract/background-regression-matrix-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/background-regression-matrix-contract.mjs
new file mode 100644
index 000000000..1c5cc8e6a
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/contract/background-regression-matrix-contract.mjs
@@ -0,0 +1,181 @@
+import assert from 'node:assert/strict';
+import fs from 'node:fs';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = path.dirname(__filename);
+const frontendRoot = path.resolve(__dirname, '../..');
+
+function readUtf8(relativePath) {
+ return fs.readFileSync(path.join(frontendRoot, relativePath), 'utf8');
+}
+
+function readJson(relativePath) {
+ return JSON.parse(readUtf8(relativePath));
+}
+
+function requireContains(source, needle, label) {
+ assert.ok(source.includes(needle), `${label} missing: ${needle}`);
+}
+
+function requireMissing(source, needle, label) {
+ assert.ok(!source.includes(needle), `${label} must not contain: ${needle}`);
+}
+
+function assertStringArray(value, label) {
+ assert.ok(Array.isArray(value), `${label} must be an array`);
+ assert.ok(value.length > 0, `${label} must not be empty`);
+ for (const item of value) {
+ assert.equal(typeof item, 'string', `${label} entries must be strings`);
+ assert.notEqual(item.trim(), '', `${label} entries must not be blank`);
+ }
+}
+
+function assertFailureShape(fixture) {
+ const failure = fixture.known_failure;
+ assert.equal(failure.id, 'chromium_mediapipe_gpu_service_init_failure');
+ assert.equal(failure.browser_family, 'chromium');
+ assert.equal(failure.phase, 'segmentation_backend_init');
+ assert.equal(failure.classification, 'gpu_service_init_failure');
+
+ const groups = failure.shape.must_match_groups;
+ assert.ok(Array.isArray(groups), 'failure shape must expose matcher groups');
+ assert.deepEqual(
+ groups.map((group) => group.id),
+ ['mediapipe_segmenter_init', 'chromium_gpu_service', 'init_failure'],
+ 'failure shape must distinguish MediaPipe init, Chromium GPU-service, and init-failure signals',
+ );
+ for (const group of groups) {
+ assertStringArray(group.any, `failure shape matcher ${group.id}`);
+ }
+
+ const cpuRisk = failure.shape.cpu_delegate_gpu_touch_risk;
+ assert.equal(cpuRisk.delegate, 'CPU');
+ assert.equal(cpuRisk.treat_as_unsafe_when_gpu_signature_present, true);
+ assert.deepEqual(cpuRisk.local_worker_signals, [
+ 'ImageSegmenter.createFromOptions',
+ "delegate === 'GPU' ? 'GPU' : 'CPU'",
+ 'DrawingUtils',
+ "getContext('webgl2')",
+ ]);
+}
+
+function assertBackendLadder(fixture) {
+ const ladder = fixture.backend_ladder;
+ assert.ok(Array.isArray(ladder), 'backend ladder must be an array');
+ assert.deepEqual(
+ ladder.map((step) => step.backend),
+ ['worker_segmenter', 'user_avatar_placeholder', 'unfiltered_video'],
+ 'background unavailable path must use Pierre worker, then explicit user choice',
+ );
+
+ const [worker, avatar, unfiltered] = ladder;
+ assert.ok(worker.enabled_when.includes('worker_available'), 'MediaPipe must stay scoped to the worker backend');
+ assert.equal(worker.on_init_failure, 'keep_source_visible_then_prompt_user');
+ assert.ok(avatar.enabled_when.includes('user_chooses_standard_or_uploaded_avatar'), 'avatar requires explicit user choice');
+ assert.deepEqual(avatar.required_behavior, [
+ 'signal_static_avatar_once',
+ 'keep_audio_tracks_live',
+ 'do_not_stream_avatar_frames',
+ 'do_not_apply_synthetic_background',
+ ]);
+ assert.ok(unfiltered.enabled_when.includes('user_chooses_unfiltered_video'), 'unfiltered video requires explicit user choice');
+ assert.deepEqual(unfiltered.required_behavior, [
+ 'keep_source_video_visible',
+ 'keep_audio_tracks_live',
+ 'keep_published_media_alive',
+ 'do_not_apply_synthetic_background_over_person',
+ ]);
+}
+
+function assertQuarantine(fixture) {
+ assert.equal(fixture.quarantine.cooldown_ms_min, 60000, 'quarantine cooldown must be at least the current 60s retry window');
+ assert.deepEqual(fixture.quarantine.scope_keys, ['browser_family', 'backend', 'delegate', 'model_source']);
+ assert.deepEqual(fixture.quarantine.idempotency, {
+ single_transition_per_cooldown_window: true,
+ do_not_retry_failed_backend_per_frame: true,
+ do_not_restart_media_tracks: true,
+ do_not_reload_page_or_call: true,
+ same_failure_same_window_keeps_selected_fallback: true,
+ });
+}
+
+function assertDiagnostics(fixture) {
+ assert.deepEqual(fixture.diagnostics_required, [
+ 'selected_backend',
+ 'failed_backend',
+ 'browser_family',
+ 'gpu_availability',
+ 'model_source',
+ 'fallback_reason',
+ 'user_choice_required',
+ ]);
+}
+
+function assertBrowserMatrixSchema(fixture) {
+ const matrix = fixture.browser_matrix_required;
+ assert.deepEqual(
+ matrix.map((entry) => entry.browser),
+ ['Chrome Stable', 'Chromium Ubuntu', 'Firefox'],
+ 'browser regression matrix must preserve the sprint-required browser set',
+ );
+ for (const entry of matrix) {
+ assertStringArray(entry.required_fields, `${entry.browser} required fields`);
+ assert.ok(entry.required_fields.includes('version'), `${entry.browser} must record version`);
+ assert.ok(entry.required_fields.includes('selected_backend'), `${entry.browser} must record selected backend`);
+ assert.ok(entry.required_fields.includes('console_signatures'), `${entry.browser} must record console signatures`);
+ }
+}
+
+function assertCurrentRuntimeBoundaries(fixture) {
+ const stream = readUtf8('src/domain/realtime/background/stream.ts');
+ const workerBackend = readUtf8('src/domain/realtime/background/backendWorkerSegmenter.js');
+ const worker = readUtf8('src/domain/realtime/background/workers/imageSegmenterWorker.js');
+ const modal = readUtf8('src/domain/realtime/background/BackgroundReplacementUnavailableModal.vue');
+ const orchestration = readUtf8('src/domain/realtime/local/mediaOrchestration.ts');
+ const avatarSignal = readUtf8('src/domain/realtime/background/avatarFallbackSignal.ts');
+ const unavailablePrompt = readUtf8('src/domain/realtime/background/unavailablePrompt.ts');
+
+ assert.equal(fixture.current_runtime_baseline.production_default_backend, 'worker-segmenter');
+ assert.equal(fixture.current_runtime_baseline.mediapipe_is_worker_scoped, true);
+ assert.equal(fixture.current_runtime_baseline.segmentation_unavailable_prompts_user, true);
+
+ requireContains(stream, "import { acquireWorkerSegmenterBackendLease } from './backendWorkerSegmenter';", 'current production worker backend');
+ requireContains(stream, 'if (segmentationBackendInitPromise) return segmentationBackendInitPromise;', 'current init idempotency');
+ requireContains(stream, "requested: 'worker-segmenter'", 'current backend diagnostics');
+ requireContains(stream, 'notifySegmentationUnavailable', 'current unavailable prompt hook');
+ requireMissing(stream, 'ImageSegmenter.createFromOptions', 'production stream must not directly instantiate MediaPipe');
+ requireMissing(stream, "delegate === 'GPU' ? 'GPU' : 'CPU'", 'production stream must not switch MediaPipe delegates directly');
+
+ requireContains(workerBackend, "kind: 'worker-segmenter'", 'worker backend identity');
+ requireContains(worker, 'ImageSegmenter.createFromOptions', 'local MediaPipe worker fixture boundary');
+ requireContains(worker, "delegate: delegate === 'GPU' ? 'GPU' : 'CPU'", 'local MediaPipe delegate boundary');
+ requireContains(worker, "const glCtx = renderCanvas.getContext('webgl2');", 'local MediaPipe category-mask WebGL boundary');
+ requireContains(worker, 'new DrawingUtils(glCtx)', 'local MediaPipe DrawingUtils WebGL boundary');
+ requireContains(modal, 'background_use_standard_avatar', 'standard avatar choice');
+ requireContains(modal, 'background_upload_avatar', 'uploaded avatar choice');
+ requireContains(modal, 'background_send_unfiltered', 'unfiltered video choice');
+ requireContains(orchestration, 'handleBackgroundReplacementUnavailable({', 'unavailable prompt handler');
+ requireContains(orchestration, 'createBackgroundFallbackAudioOnlyStream(rawStream)', 'avatar fallback audio-only stream');
+ requireContains(orchestration, 'syncBackgroundFallbackControlState(true)', 'static avatar signal');
+ requireMissing(avatarSignal, 'captureStream', 'avatar fallback frame streaming');
+ requireContains(unavailablePrompt, "eventType: 'local_background_replacement_unavailable'", 'field diagnostic');
+}
+
+try {
+ const fixture = readJson('tests/contract/background-regression-matrix-fixture.json');
+ assert.equal(fixture.fixture_version, 1);
+ assertStringArray(fixture.source_basis, 'source_basis');
+ assertFailureShape(fixture);
+ assertDiagnostics(fixture);
+ assertBackendLadder(fixture);
+ assertQuarantine(fixture);
+ assertBrowserMatrixSchema(fixture);
+ assertCurrentRuntimeBoundaries(fixture);
+
+ console.log('[background-regression-matrix-contract] PASS');
+} catch (error) {
+ console.error(`[background-regression-matrix-contract] FAIL: ${error.message}`);
+ process.exit(1);
+}
diff --git a/demo/video-chat/frontend-vue/tests/contract/background-regression-matrix-fixture.json b/demo/video-chat/frontend-vue/tests/contract/background-regression-matrix-fixture.json
new file mode 100644
index 000000000..0a4ddae1b
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/contract/background-regression-matrix-fixture.json
@@ -0,0 +1,167 @@
+{
+ "fixture_version": 1,
+ "source_basis": [
+ "SPRINT.md BGF-01/BGF-02 local contract text",
+ "src/domain/realtime/background/workers/imageSegmenterWorker.js local MediaPipe worker boundary",
+ "src/domain/realtime/background/stream.ts Pierre worker pipeline",
+ "src/domain/realtime/background/BackgroundReplacementUnavailableModal.vue user alternative"
+ ],
+ "known_failure": {
+ "id": "chromium_mediapipe_gpu_service_init_failure",
+ "browser_family": "chromium",
+ "phase": "segmentation_backend_init",
+ "affected_backend": "mediapipe_gpu",
+ "classification": "gpu_service_init_failure",
+ "shape": {
+ "must_match_groups": [
+ {
+ "id": "mediapipe_segmenter_init",
+ "any": [
+ "MediaPipe",
+ "Tasks-Vision",
+ "ImageSegmenter",
+ "ImageSegmenter.createFromOptions",
+ "FilesetResolver.forVisionTasks",
+ "INIT_ERROR"
+ ]
+ },
+ {
+ "id": "chromium_gpu_service",
+ "any": [
+ "GPU service",
+ "gpu service",
+ "GPU process",
+ "GpuChannel",
+ "ContextResult::kFatalFailure",
+ "Failed to create shared context",
+ "WebGL context lost"
+ ]
+ },
+ {
+ "id": "init_failure",
+ "any": [
+ "createFromOptions",
+ "forVisionTasks",
+ "init_failed",
+ "failed to initialize",
+ "initialization failed"
+ ]
+ }
+ ],
+ "cpu_delegate_gpu_touch_risk": {
+ "delegate": "CPU",
+ "treat_as_unsafe_when_gpu_signature_present": true,
+ "local_worker_signals": [
+ "ImageSegmenter.createFromOptions",
+ "delegate === 'GPU' ? 'GPU' : 'CPU'",
+ "DrawingUtils",
+ "getContext('webgl2')"
+ ]
+ }
+ }
+ },
+ "diagnostics_required": [
+ "selected_backend",
+ "failed_backend",
+ "browser_family",
+ "gpu_availability",
+ "model_source",
+ "fallback_reason",
+ "user_choice_required"
+ ],
+ "backend_ladder": [
+ {
+ "backend": "worker_segmenter",
+ "enabled_when": [
+ "background_replacement_requested",
+ "browser_family_supported",
+ "worker_available",
+ "local_mediapipe_assets_available"
+ ],
+ "on_init_failure": "keep_source_visible_then_prompt_user"
+ },
+ {
+ "backend": "user_avatar_placeholder",
+ "enabled_when": [
+ "worker_segmenter_unavailable",
+ "user_chooses_standard_or_uploaded_avatar"
+ ],
+ "required_behavior": [
+ "signal_static_avatar_once",
+ "keep_audio_tracks_live",
+ "do_not_stream_avatar_frames",
+ "do_not_apply_synthetic_background"
+ ]
+ },
+ {
+ "backend": "unfiltered_video",
+ "enabled_when": [
+ "worker_segmenter_unavailable",
+ "user_chooses_unfiltered_video"
+ ],
+ "required_behavior": [
+ "keep_source_video_visible",
+ "keep_audio_tracks_live",
+ "keep_published_media_alive",
+ "do_not_apply_synthetic_background_over_person"
+ ]
+ }
+ ],
+ "quarantine": {
+ "cooldown_ms_min": 60000,
+ "scope_keys": [
+ "browser_family",
+ "backend",
+ "delegate",
+ "model_source"
+ ],
+ "idempotency": {
+ "single_transition_per_cooldown_window": true,
+ "do_not_retry_failed_backend_per_frame": true,
+ "do_not_restart_media_tracks": true,
+ "do_not_reload_page_or_call": true,
+ "same_failure_same_window_keeps_selected_fallback": true
+ }
+ },
+ "browser_matrix_required": [
+ {
+ "browser": "Chrome Stable",
+ "required_fields": [
+ "version",
+ "os",
+ "gpu_availability",
+ "mediapipe_gpu_result",
+ "mediapipe_cpu_result",
+ "selected_backend",
+ "console_signatures"
+ ]
+ },
+ {
+ "browser": "Chromium Ubuntu",
+ "required_fields": [
+ "version",
+ "os",
+ "gpu_availability",
+ "mediapipe_gpu_result",
+ "mediapipe_cpu_result",
+ "selected_backend",
+ "console_signatures"
+ ]
+ },
+ {
+ "browser": "Firefox",
+ "required_fields": [
+ "version",
+ "os",
+ "gpu_availability",
+ "selected_backend",
+ "console_signatures"
+ ]
+ }
+ ],
+ "current_runtime_baseline": {
+ "production_default_backend": "worker-segmenter",
+ "mediapipe_is_worker_scoped": true,
+ "segmentation_unavailable_prompts_user": true
+ }
+}
diff --git a/demo/video-chat/frontend-vue/tests/contract/background-segmentation-harness-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/background-segmentation-harness-contract.mjs
index 4a788d7e5..3e8740630 100644
--- a/demo/video-chat/frontend-vue/tests/contract/background-segmentation-harness-contract.mjs
+++ b/demo/video-chat/frontend-vue/tests/contract/background-segmentation-harness-contract.mjs
@@ -12,71 +12,40 @@ function readUtf8(relativePath) {
}
function requireFile(relativePath, label) {
- assert.ok(fs.existsSync(path.join(frontendRoot, relativePath)), `${label} missing: ${relativePath}`);
+ const fullPath = path.join(frontendRoot, relativePath);
+ assert.ok(fs.existsSync(fullPath), `${label} missing: ${relativePath}`);
+ assert.ok(fs.statSync(fullPath).size > 0, `${label} must not be empty`);
}
try {
const html = readUtf8('tests/standalone/king-background-segmentation-harness.html');
- const source = readUtf8('tests/standalone/king-background-segmentation-harness.ts');
+ const harness = readUtf8('tests/standalone/king-background-segmentation-harness.ts');
+ const stream = readUtf8('src/domain/realtime/background/stream.ts');
+ const workerBackend = readUtf8('src/domain/realtime/background/backendWorkerSegmenter.js');
+ const worker = readUtf8('src/domain/realtime/background/workers/imageSegmenterWorker.js');
- requireFile('public/cdn/vendor/sinet/sinet-float.onnx', 'vendored SINet ONNX graph');
- requireFile('public/cdn/vendor/sinet/sinet.data', 'vendored SINet ONNX external weights');
- requireFile('public/cdn/vendor/sinet/metadata-float.json', 'vendored SINet metadata');
+ requireFile('public/cdn/vendor/mediapipe/tasks-vision/vision_bundle.mjs', 'vendored MediaPipe Tasks bundle');
+ requireFile('public/cdn/vendor/mediapipe/models/selfie_multiclass_256x256.tflite', 'vendored MediaPipe segmentation model');
+ requireFile('public/wasm/vision_wasm_internal.js', 'vendored MediaPipe wasm loader');
- assert.ok(html.includes('king-background-segmentation-harness.ts'), 'standalone harness must load the TypeScript module');
- assert.ok(source.includes("from '../../src/lib/wasm/wasm-codec'"), 'harness must keep King WASM infra available');
- assert.ok(source.includes("from '../../src/domain/realtime/background/maskPostprocess'"), 'harness must use the same matte postprocess as production');
- assert.ok(source.includes("from 'onnxruntime-web/wasm'"), 'harness must load ONNX Runtime WASM for fast segmentation candidates');
- assert.ok(source.includes('ort.env.wasm.proxy = false;'), 'harness must not depend on ORT proxy workers');
- assert.ok(source.includes('ort.env.wasm.numThreads = 1;'), 'harness must not depend on SharedArrayBuffer threading');
- assert.ok(source.includes('createKingBackgroundMatteRefiner'), 'harness must instantiate the King background segmenter');
- assert.ok(html.includes('value="sinet"'), 'harness must expose SINet as the fast segmentation candidate');
- assert.ok(source.includes("fetchBinaryAsset('/cdn/vendor/sinet/sinet-float.onnx')"), 'harness must load the vendored SINet ONNX graph');
- assert.ok(source.includes("fetchBinaryAsset('/cdn/vendor/sinet/sinet.data')"), 'harness must load the vendored SINet external weights');
- assert.ok(source.includes("externalData: [{ path: 'sinet.data', data: weights }]"), 'harness must mount SINet external weights explicitly');
- assert.ok(source.includes('InferenceSession.create(model'), 'harness must create SINet sessions from fetched model bytes');
- assert.ok(source.includes('sinetForegroundAlpha'), 'harness must convert SINet output to foreground alpha');
- assert.ok(source.includes('probabilityLike'), 'harness must avoid softmaxing probability-like SINet outputs');
- assert.ok(!source.includes('Math.exp(bg - max)'), 'harness must not softmax raw SINet logits');
- assert.ok(!source.includes('fgExp / Math.max'), 'harness must not couple foreground alpha to the background logit');
- assert.ok(source.includes('shapeForegroundAlpha'), 'harness must apply shared Gaussian, contrast, gamma, and temporal mask shaping');
- assert.ok(!html.includes('value="modnet"'), 'harness must not expose MODNet fallback');
- assert.ok(!source.includes('Xenova/modnet'), 'harness must not run MODNet fallback');
- assert.ok(html.includes('id="deviceSelect"'), 'harness must expose device selection');
- assert.ok(html.includes('id="dtypeSelect"'), 'harness must expose dtype selection');
- assert.ok(html.includes('id="alphaGammaInput"'), 'harness must expose alpha gamma shaping');
- assert.ok(html.includes('id="maskContrastInput"'), 'harness must expose mask contrast shaping');
- assert.ok(!html.includes('maskBlackPointInput'), 'harness must not expose black point controls');
- assert.ok(!html.includes('maskWhitePointInput'), 'harness must not expose white point controls');
- assert.ok(html.includes('id="averageRadiusInput"'), 'harness must expose Gaussian radius shaping');
- assert.ok(html.includes('id="temporalRiseInput"'), 'harness must expose temporal rise shaping');
- assert.ok(html.includes('id="temporalFallInput"'), 'harness must expose temporal fall shaping');
- assert.ok(html.includes('id="intervalInput"'), 'harness must expose inference interval control');
- assert.ok(source.includes('function shapeAlpha'), 'harness must shape alpha after model inference');
- assert.ok(source.includes('model.segment(image.data)'), 'harness must keep the King WASM bootstrap comparison path');
- assert.ok(html.includes('Foreground Mask'), 'harness must show a mask pane');
- assert.ok(html.includes('Composited Blur'), 'harness must show a result pane');
- assert.ok(source.includes('const personLayer = new OffscreenCanvas(compositeCanvas.width, compositeCanvas.height);'), 'harness must composite foreground on a separate layer');
- assert.ok(source.includes("personLayerCtx.globalCompositeOperation = 'destination-in';"), 'harness must mask only the foreground layer');
- assert.ok(source.includes('function compositeAlpha'), 'harness must suppress weak mask background before compositing');
- assert.ok(source.includes('Math.pow(normalized, 3.2)'), 'harness exclusion must strongly suppress weak background alpha');
- assert.ok(source.includes('maskImage.data[p + 3] = compositeAlpha(alpha[i] ?? 0, preset);'), 'harness must use compositor alpha instead of raw mask alpha');
- assert.ok(source.includes("compositeCtx.globalCompositeOperation = 'source-over';"), 'harness must keep the blurred background under the masked foreground');
- assert.ok(!html.includes('contrastInput'), 'harness must not expose source-image preprocessing controls');
- assert.ok(!html.includes('alphaThresholdInput'), 'harness must not expose hard mask thresholding');
- assert.ok(!source.includes('preprocessSampleFrame'), 'harness must not preprocess frames before model inference');
- assert.ok(html.includes('value="weak_blur"'), 'harness must expose thin blur');
- assert.ok(html.includes('value="hard_blur"'), 'harness must expose thick blur');
- assert.ok(html.includes('value="exclusion"'), 'harness must expose deep-blue exclusion background');
- assert.ok(!html.includes('Replace matte'), 'harness must not label exclusion as replace matte');
- assert.ok(source.includes("const EXCLUSION_BACKGROUND = '#061a4a';"), 'harness exclusion must use deep blue background');
- assert.ok(source.includes("preset === 'exclusion' ? 'replace' : preset"), 'harness must map exclusion to the existing King matte preset');
- assert.ok(source.includes("compositeCtx.globalCompositeOperation = 'copy';"), 'harness exclusion must replace the whole background canvas');
- assert.ok(source.includes('compositeCtx.fillStyle = EXCLUSION_BACKGROUND;'), 'harness exclusion must fill with deep blue before drawing foreground');
- assert.ok(!source.includes('const excludedBackground = new OffscreenCanvas(compositeCanvas.width, compositeCanvas.height);'), 'harness exclusion must not overlay an inverse blue mask over source video');
- assert.ok(!source.includes('MediaPipe'), 'harness must not use MediaPipe');
- assert.ok(!source.includes('tfjs'), 'harness must not use TFJS');
- assert.ok(!source.includes('@huggingface/transformers'), 'harness must not use Transformers.js');
+ assert.ok(html.includes('king-background-segmentation-harness.ts'), 'standalone harness must stay available for model comparison');
+ assert.ok(harness.includes("from 'onnxruntime-web/wasm'"), 'standalone harness may keep SINet/ONNX experiments isolated from production');
+ assert.ok(harness.includes('createKingBackgroundMatteRefiner'), 'standalone harness must keep King WASM comparison available');
+ assert.ok(!stream.includes('onnxruntime-web/wasm'), 'production stream must not import the standalone ONNX experiment');
+ assert.ok(!stream.includes('/cdn/vendor/sinet/'), 'production stream must not load SINet assets');
+
+ assert.ok(stream.includes('acquireWorkerSegmenterBackendLease'), 'production stream must use the worker segmenter backend');
+ assert.ok(workerBackend.includes('selfie_multiclass_256x256.tflite'), 'worker backend must use the vendored selfie multiclass model');
+ assert.ok(worker.includes('ImageSegmenter.createFromOptions'), 'worker must initialize MediaPipe Tasks ImageSegmenter');
+ assert.ok(worker.includes('outputCategoryMask: true'), 'worker must request category masks');
+ assert.ok(worker.includes('outputConfidenceMasks: true'), 'worker must keep confidence masks as a local worker fallback');
+ assert.ok(!worker.includes('cdn.jsdelivr.net'), 'worker must not fetch runtime from jsDelivr');
+ assert.ok(!worker.includes('unpkg.com'), 'worker must not fetch runtime from unpkg');
+
+ assert.ok(html.includes('Foreground Mask'), 'standalone harness must show a mask pane');
+ assert.ok(html.includes('Composited Blur'), 'standalone harness must show a result pane');
+ assert.ok(html.includes('value="exclusion"'), 'standalone harness must expose deep-blue exclusion background');
+ assert.ok(harness.includes("const EXCLUSION_BACKGROUND = '#061a4a';"), 'standalone harness exclusion must use deep blue background');
console.log('[background-segmentation-harness-contract] PASS');
} catch (error) {
diff --git a/demo/video-chat/frontend-vue/tests/contract/background-sinet-defaults-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/background-sinet-defaults-contract.mjs
index 26de847e6..6931663f0 100644
--- a/demo/video-chat/frontend-vue/tests/contract/background-sinet-defaults-contract.mjs
+++ b/demo/video-chat/frontend-vue/tests/contract/background-sinet-defaults-contract.mjs
@@ -12,54 +12,36 @@ function readUtf8(relativePath) {
}
try {
- const html = readUtf8('tests/standalone/king-background-segmentation-harness.html');
- const harness = readUtf8('tests/standalone/king-background-segmentation-harness.ts');
- const backend = readUtf8('src/domain/realtime/background/backendSinetWasm.js');
- const postprocess = readUtf8('src/domain/realtime/background/maskPostprocess.js');
- const productionCallPaths = [
- readUtf8('src/domain/realtime/local/mediaOrchestration.ts'),
- readUtf8('src/domain/calls/access/joinPreview.ts'),
- readUtf8('src/domain/calls/dashboard/enterCall.ts'),
- readUtf8('src/domain/calls/admin/enterCall.ts'),
- ];
-
- assert.ok(html.includes('SINet fast '), 'standalone default model must expose SINet fast first');
- assert.ok(html.includes('WASM '), 'standalone default device must be WASM first');
- assert.ok(html.includes('SINet float '), 'standalone default dtype must be SINet float');
- assert.ok(html.includes('id="alphaGammaInput" type="number" min="0.4" max="2.5" step="0.05" value="0.8"'), 'standalone default gamma must be 0.8');
- assert.ok(html.includes('id="maskContrastInput" type="number" min="0.25" max="4" step="0.05" value="0.75"'), 'standalone default contrast must be 0.75');
- assert.ok(html.includes('id="averageRadiusInput" type="number" min="0" max="12" step="1" value="6"'), 'standalone default Gaussian radius must be 6');
- assert.ok(html.includes('id="temporalRiseInput" type="number" min="0" max="1" step="0.05" value="0.7"'), 'standalone default temporal rise must be 0.7');
- assert.ok(html.includes('id="temporalFallInput" type="number" min="0" max="1" step="0.05" value="0.6"'), 'standalone default temporal fall must be 0.6');
-
- assert.ok(harness.includes('Number(alphaGammaInput.value || 0.8)'), 'harness fallback gamma must be 0.8');
- assert.ok(harness.includes('Number(maskContrastInput.value || 0.75)'), 'harness fallback contrast must be 0.75');
- assert.ok(harness.includes('Number(averageRadiusInput.value || 6)'), 'harness fallback Gaussian radius must be 6');
- assert.ok(harness.includes('Number(temporalRiseInput.value || 0.7)'), 'harness fallback temporal rise must be 0.7');
- assert.ok(harness.includes('Number(temporalFallInput.value || 0.6)'), 'harness fallback temporal fall must be 0.6');
-
- assert.ok(backend.includes("executionProviders: ['wasm']"), 'production SINet default execution provider must be WASM');
- assert.ok(backend.includes('Number(opts.alphaGamma ?? 0.8)'), 'production default gamma must be 0.8');
- assert.ok(backend.includes('Number(opts.maskContrast ?? 0.75)'), 'production default contrast must be 0.75');
- assert.ok(backend.includes('Number(opts.averageRadius ?? 6)'), 'production default Gaussian radius must be 6');
- assert.ok(backend.includes('Number(opts.temporalRise ?? 0.7)'), 'production default temporal rise must be 0.7');
- assert.ok(backend.includes('Number(opts.temporalFall ?? 0.6)'), 'production default temporal fall must be 0.6');
-
- assert.ok(postprocess.includes('Number(controls?.gamma) || 0.8'), 'shared postprocess default gamma must be 0.8');
- assert.ok(postprocess.includes('Number(controls?.contrast ?? 0.75)'), 'shared postprocess default contrast must be 0.75');
- assert.ok(postprocess.includes('Number(controls?.averageRadius ?? 6)'), 'shared postprocess default Gaussian radius must be 6');
- assert.ok(postprocess.includes('controls?.temporalRise ?? 0.7'), 'shared postprocess default temporal rise must be 0.7');
- assert.ok(postprocess.includes('controls?.temporalFall ?? 0.6'), 'shared postprocess default temporal fall must be 0.6');
-
- for (const source of productionCallPaths) {
- assert.ok(source.includes('alphaGamma: 0.8,'), 'production call path must pass standalone gamma 0.8');
- assert.ok(source.includes('maskContrast: 0.75,'), 'production call path must pass standalone contrast 0.75');
- assert.ok(source.includes('averageRadius: 6,'), 'production call path must pass standalone Gaussian radius 6');
- assert.ok(source.includes('temporalRise: 0.7,'), 'production call path must pass standalone temporal rise 0.7');
- assert.ok(source.includes('temporalFall: 0.6,'), 'production call path must pass standalone temporal fall 0.6');
- }
-
- console.log('[background-sinet-defaults-contract] PASS');
+ const preferences = readUtf8('src/domain/realtime/media/preferences.ts');
+ const orchestration = readUtf8('src/domain/realtime/local/mediaOrchestration.ts');
+ const avatarFallback = readUtf8('src/domain/realtime/background/avatarFallbackSignal.ts');
+ const staticAvatarRender = readUtf8('src/domain/realtime/background/staticAvatarRender.ts');
+ const modal = readUtf8('src/domain/realtime/background/BackgroundReplacementUnavailableModal.vue');
+ const stream = readUtf8('src/domain/realtime/background/stream.ts');
+
+ assert.ok(preferences.includes("export const DEFAULT_BACKGROUND_FALLBACK_AVATAR_URL = '/assets/orgas/kingrt/avatar-placeholder.svg';"), 'standard avatar fallback must be explicit');
+ assert.ok(preferences.includes("backgroundFallbackVideoMode: 'none'"), 'avatar fallback must not be enabled silently');
+ assert.ok(preferences.includes("backgroundReplacementUnavailablePromptOpen: false"), 'unavailable modal must default closed');
+ assert.ok(preferences.includes('useCallBackgroundFallbackAvatar(imageUrl = DEFAULT_BACKGROUND_FALLBACK_AVATAR_URL)'), 'standard avatar action must use the default avatar');
+ assert.ok(preferences.includes('clearCallBackgroundFallbackVideo()'), 'unfiltered action must clear avatar fallback');
+
+ assert.ok(orchestration.includes("String(callMediaPrefs.backgroundFallbackVideoMode || 'none') === 'avatar'"), 'local media must honor avatar fallback mode');
+ assert.ok(orchestration.includes('backgroundFilterController.dispose();'), 'avatar fallback must release background filter pipeline');
+ assert.ok(orchestration.includes("resetBackgroundRuntimeMetrics('avatar_placeholder');"), 'avatar fallback must expose runtime state');
+ assert.ok(orchestration.includes('syncBackgroundFallbackControlState(true)'), 'avatar fallback must publish static control-state once');
+ assert.ok(orchestration.includes('createBackgroundFallbackAudioOnlyStream(rawStream)'), 'avatar fallback must not create a fake video stream');
+ assert.ok(avatarFallback.includes('for (const audioTrack of sourceStream.getAudioTracks())'), 'avatar fallback must preserve audio tracks');
+ assert.ok(!avatarFallback.includes('captureStream'), 'avatar fallback must not stream avatar video frames');
+ assert.ok(staticAvatarRender.includes("node.dataset.callStaticAvatar = '1';"), 'avatar fallback must render as static tile media');
+
+ assert.ok(modal.includes('accept="image/png,image/jpeg,image/webp"'), 'avatar upload must restrict image types');
+ assert.ok(modal.includes('useCallBackgroundFallbackAvatar(DEFAULT_BACKGROUND_FALLBACK_AVATAR_URL)'), 'modal standard avatar button must use default avatar');
+ assert.ok(modal.includes('clearCallBackgroundFallbackVideo();'), 'modal unfiltered button must disable replacement');
+
+ assert.ok(stream.includes("requested: 'worker-segmenter'"), 'production backend remains Pierre worker segmenter');
+ assert.ok(!stream.includes('sinet_wasm'), 'production stream must not default to SINet WASM');
+
+ console.log('[background-sinet-defaults-contract] PASS avatar/unfiltered alternative defaults');
} catch (error) {
console.error(`[background-sinet-defaults-contract] FAIL: ${error.message}`);
process.exit(1);
diff --git a/demo/video-chat/frontend-vue/tests/contract/call-access-link-privacy-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/call-access-link-privacy-contract.mjs
new file mode 100644
index 000000000..ebd3c16b8
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/contract/call-access-link-privacy-contract.mjs
@@ -0,0 +1,56 @@
+import assert from 'node:assert/strict';
+import fs from 'node:fs';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = path.dirname(__filename);
+const frontendRoot = path.resolve(__dirname, '../..');
+const videoChatRoot = path.resolve(frontendRoot, '..');
+const repoRoot = path.resolve(videoChatRoot, '../..');
+
+function readText(relativePath) {
+ return fs.readFileSync(path.join(repoRoot, relativePath), 'utf8');
+}
+
+const joinView = readText('demo/video-chat/frontend-vue/src/domain/calls/access/JoinView.vue');
+const admissionGate = readText('demo/video-chat/frontend-vue/src/domain/calls/access/admissionGate.ts');
+const e2eSpec = readText('demo/video-chat/frontend-vue/tests/e2e/call-access-join.spec.js');
+
+assert.match(
+ admissionGate,
+ /export function safeCallAccessInvalidMessage/,
+ 'call-access UI must expose a safe invalid-link message helper',
+);
+assert.match(
+ joinView,
+ /function resetJoinContextDetails\(\)[\s\S]*state\.callId = ''[\s\S]*state\.roomId = ''[\s\S]*state\.callTitle = ''[\s\S]*state\.guestName = ''/s,
+ 'invalid link states must clear call-specific UI details before rendering',
+);
+assert.match(
+ joinView,
+ /if \(!response\.ok \|\| !payload \|\| payload\.status !== 'ok'\) \{[\s\S]*payload = \{ error: \{ code: 'call_access_validation_failed' \} \};[\s\S]*state\.contextError = localizedApiErrorMessage\(payload,\s*t\('public\.join\.resolve_failed'\)\);[\s\S]*return;[\s\S]*\}/s,
+ 'failed public join resolution must replace backend payloads with a generic invalid-link code',
+);
+assert.match(
+ joinView,
+ /catch \(error\) \{[\s\S]*showSafeInvalidAccessState\(\);[\s\S]*\} finally/s,
+ 'unexpected join resolution errors must render the safe invalid-link state instead of raw exception text',
+);
+assert.match(
+ e2eSpec,
+ /invalid call-access link renders safe state without foreign call data/,
+ 'call-access E2E must cover invalid link privacy',
+);
+assert.match(
+ e2eSpec,
+ /Private Foreign Call[\s\S]*not\.toContainText\(foreignTitle\)[\s\S]*not\.toContainText\(foreignEmail\)/s,
+ 'invalid link E2E must prove foreign call title and email are not rendered',
+);
+assert.match(
+ e2eSpec,
+ /getByRole\('button', \{ name: \/\^Join call\$\/ \}\)\)\.toHaveCount\(0\)/,
+ 'invalid link E2E must prove the join action is not shown',
+);
+
+process.stdout.write('[call-access-link-privacy-contract] PASS\n');
diff --git a/demo/video-chat/frontend-vue/tests/contract/call-access-strong-mismatch-privacy-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/call-access-strong-mismatch-privacy-contract.mjs
new file mode 100644
index 000000000..5b76bf3fa
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/contract/call-access-strong-mismatch-privacy-contract.mjs
@@ -0,0 +1,66 @@
+import assert from 'node:assert/strict';
+import fs from 'node:fs';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+const __dirname = path.dirname(fileURLToPath(import.meta.url));
+const root = path.resolve(__dirname, '..', '..');
+
+function read(relativePath) {
+ return fs.readFileSync(path.join(root, relativePath), 'utf8');
+}
+
+const e2eSpec = read('tests/e2e/call-access-join.spec.js');
+
+assert.match(
+ e2eSpec,
+ /strong personalized-link mismatch wrong host denial gives no access and leaks no foreign person data/,
+ 'public join E2E must cover strong personalized-link mismatch wrong-host denial',
+);
+assert.match(
+ e2eSpec,
+ /foreignNeedles\s*=\s*\[[\s\S]*linkInviteeName[\s\S]*linkInviteeEmail[\s\S]*realHostName[\s\S]*realHostEmail[\s\S]*deniedSessionToken[\s\S]*\]/,
+ 'strong-mismatch E2E must define link invitee, host, and denied-session leak sentinels',
+);
+assert.match(
+ e2eSpec,
+ /expectTextDoesNotContain\(joinBody,\s*foreignNeedles,\s*'strong-mismatch join response'\)/,
+ 'strong-mismatch E2E must prove the join response has no foreign person data',
+);
+assert.match(
+ e2eSpec,
+ /status:\s*403[\s\S]*code:\s*'call_access_forbidden'[\s\S]*mismatch:\s*'strong_personalized_link'[\s\S]*host_name:\s*'wrong_host_name'/,
+ 'strong-mismatch E2E must model a server-side wrong-host-name denial',
+);
+assert.match(
+ e2eSpec,
+ /expectTextDoesNotContain\(sessionBody,\s*foreignNeedles,\s*'strong-mismatch wrong-host denial response'\)/,
+ 'strong-mismatch E2E must prove the denial response has no foreign person data',
+);
+assert.match(
+ e2eSpec,
+ /sessionRequestAuthorization\)\.toBe\(`Bearer \$\{wrongLoggedInSession\.sessionToken\}`\)/,
+ 'strong-mismatch E2E must prove the current logged-in session is authoritative',
+);
+assert.match(
+ e2eSpec,
+ /sessionRequestBody\)\.toEqual\(\{\s*verified_user_id:\s*wrongLoggedInUserId,\s*verified_session_id:\s*wrongLoggedInSession\.sessionId,\s*\}\)/,
+ 'strong-mismatch E2E must prove verified logged-in context is sent to session issuance',
+);
+assert.match(
+ e2eSpec,
+ /not\.toContainText\(\/Call owner has been notified\|Waiting for host\/i\)[\s\S]*expect\(page\.url\(\)\)\.not\.toContain\('\/workspace\/call'\)/,
+ 'strong-mismatch E2E must prove wrong-host denial grants no direct call access',
+);
+assert.match(
+ e2eSpec,
+ /storedSession\.sessionId\)\.toBe\(wrongLoggedInSession\.sessionId\)[\s\S]*storedSession\.sessionToken\)\.toBe\(wrongLoggedInSession\.sessionToken\)[\s\S]*storedSession\.sessionToken\)\.not\.toBe\(deniedSessionToken\)/,
+ 'strong-mismatch E2E must prove denied responses do not bind a foreign session',
+);
+assert.match(
+ e2eSpec,
+ /expect\(joinGetCount\)\.toBe\(1\)[\s\S]*expect\(sessionPostCount\)\.toBe\(1\)/,
+ 'strong-mismatch E2E must guard against reload or duplicate request loops',
+);
+
+console.log('[call-access-strong-mismatch-privacy-contract] PASS');
diff --git a/demo/video-chat/frontend-vue/tests/contract/call-access-verified-context-ui-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/call-access-verified-context-ui-contract.mjs
new file mode 100644
index 000000000..0d891cb14
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/contract/call-access-verified-context-ui-contract.mjs
@@ -0,0 +1,174 @@
+import assert from 'node:assert/strict';
+import fs from 'node:fs';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+const __dirname = path.dirname(fileURLToPath(import.meta.url));
+const root = path.resolve(__dirname, '..', '..');
+
+function read(relativePath) {
+ return fs.readFileSync(path.join(root, relativePath), 'utf8');
+}
+
+const admissionGate = read('src/domain/calls/access/admissionGate.ts');
+const callAccessSession = read('src/domain/calls/access/callAccessSession.ts');
+const joinView = read('src/domain/calls/access/JoinView.vue');
+const authSession = read('src/domain/auth/session.ts');
+const callAccessJoinSpec = read('tests/e2e/call-access-join.spec.js');
+
+assert.match(
+ admissionGate,
+ /export function callAccessVerifiedContextFromSession\([\s\S]*userId[\s\S]*sessionId[\s\S]*sessionToken[\s\S]*return null[\s\S]*return \{\s*userId,\s*sessionId,\s*sessionToken,\s*\}/,
+ 'admission gate must expose a stable verified user/session snapshot helper',
+);
+
+assert.match(
+ joinView,
+ /verifiedAccessContext:\s*null/,
+ 'public join state must carry a stable verified context snapshot',
+);
+assert.match(
+ joinView,
+ /state\.verifiedAccessContext\s*=\s*callAccessVerifiedContextFromSession\(sessionState\)/,
+ 'public join must capture the logged-in context after link verification',
+);
+assert.match(
+ joinView,
+ /loginWithCallAccess\(accessId,\s*\{[\s\S]*verifiedContext:\s*state\.verifiedAccessContext[\s\S]*\}\)/,
+ 'public join must pass the verified context into call-access session issuance',
+);
+
+assert.match(
+ callAccessSession,
+ /body\.verified_user_id\s*=\s*verifiedContext\.userId/,
+ 'call-access session request body must include verified_user_id',
+);
+assert.match(
+ callAccessSession,
+ /body\.verified_session_id\s*=\s*verifiedContext\.sessionId/,
+ 'call-access session request body must include verified_session_id',
+);
+assert.match(
+ callAccessSession,
+ /headers\.authorization\s*=\s*`Bearer \$\{token\}`/,
+ 'call-access session request must send the current session token when present',
+);
+assert.match(
+ callAccessSession,
+ /verifiedContext[\s\S]*sessionState\.sessionToken[\s\S]*status:\s*409[\s\S]*errorCode:\s*'call_access_conflict'/,
+ 'call-access session request must fail safely if verified context exists after local logout',
+);
+assert.match(
+ authSession,
+ /export function applySessionEnvelope\(/,
+ 'auth session envelope application must remain shared after extracting call-access login',
+);
+assert.doesNotMatch(
+ authSession,
+ /export async function loginWithCallAccess/,
+ 'call-access login request logic belongs with public join/access helpers',
+);
+
+assert.match(
+ callAccessJoinSpec,
+ /login switch after verified call-access link fails without rebinding or leaking foreign data/,
+ 'public join E2E must cover login switch after verified link context',
+);
+assert.match(
+ callAccessJoinSpec,
+ /sessionRequestAuthorization\)\.toBe\(`Bearer \$\{switchedSession\.sessionToken\}`\)/,
+ 'login-switch E2E must prove session issuance uses the current bearer token',
+);
+assert.match(
+ callAccessJoinSpec,
+ /sessionRequestBody\)\.toEqual\(\{\s*verified_user_id:\s*2,\s*verified_session_id:\s*verifiedSession\.sessionId,\s*\}\)/,
+ 'login-switch E2E must prove the verified user/session snapshot is sent',
+);
+assert.match(
+ callAccessJoinSpec,
+ /expect\(sessionPayload\?\.error\?\.code\)\.toBe\('call_access_conflict'\)/,
+ 'login-switch E2E must require a safe conflict from the route guard',
+);
+assert.match(
+ callAccessJoinSpec,
+ /not\.toContainText\(foreignTitle\)[\s\S]*not\.toContainText\(foreignEmail\)[\s\S]*not\.toContainText\(rejectedCallAccessToken\)/,
+ 'login-switch E2E must prove foreign call/person/session details are not rendered',
+);
+assert.match(
+ callAccessJoinSpec,
+ /storedSession\.sessionId\)\.toBe\(switchedSession\.sessionId\)[\s\S]*storedSession\.sessionToken\)\.toBe\(switchedSession\.sessionToken\)[\s\S]*storedSession\.sessionToken\)\.not\.toBe\(rejectedCallAccessToken\)/,
+ 'login-switch E2E must prove the failed call-access response does not bind a new session',
+);
+assert.match(
+ callAccessJoinSpec,
+ /expect\(joinGetCount\)\.toBe\(1\)[\s\S]*expect\(sessionPostCount\)\.toBe\(1\)/,
+ 'login-switch E2E must guard against reload or duplicate session POST loops',
+);
+assert.match(
+ callAccessJoinSpec,
+ /logout during verified call-access link context fails closed without leaking or joining/,
+ 'public join E2E must cover logout after verified link context',
+);
+assert.match(
+ callAccessJoinSpec,
+ /const \{ logoutSession, sessionState \} = await import\('\/src\/domain\/auth\/session\.ts'\);[\s\S]*await logoutSession\(\)/,
+ 'logout E2E must exercise the real browser logout/session-clear path',
+);
+assert.match(
+ callAccessJoinSpec,
+ /expect\(sessionPostCount\)\.toBe\(0\)/,
+ 'logout E2E must prove no call-access session request is issued after verified context is logged out',
+);
+assert.match(
+ callAccessJoinSpec,
+ /expect\(page\.url\(\)\)\.not\.toContain\('\/workspace\/call'\)/,
+ 'logout E2E must prove the browser does not enter the workspace',
+);
+assert.match(
+ callAccessJoinSpec,
+ /logout denial must not render \$\{value\}/,
+ 'logout E2E must prove foreign call/invite/host/session data is not rendered',
+);
+assert.match(
+ callAccessJoinSpec,
+ /storedSession\.sessionToken \|\| ''\)\.toBe\(''\)[\s\S]*JSON\.stringify\(storedSession\)\)\.not\.toContain\(rejectedSessionToken\)/,
+ 'logout E2E must prove no foreign call-access session is adopted',
+);
+
+assert.match(
+ callAccessJoinSpec,
+ /same personalized link in parallel contexts keeps account sessions isolated/,
+ 'public join E2E must cover parallel use of one personalized link by two different logged-in accounts',
+);
+assert.match(
+ callAccessJoinSpec,
+ /createPublicJoinPage\(browser, baseURL\)[\s\S]*createPublicJoinPage\(browser, baseURL\)[\s\S]*Promise\.all\(\[[\s\S]*page\.goto\(`\/join\/\$\{accessId\}`\)[\s\S]*page\.goto\(`\/join\/\$\{accessId\}`\)/,
+ 'parallel-account E2E must use separate browser contexts opening the same personalized link concurrently',
+);
+assert.match(
+ callAccessJoinSpec,
+ /requests\.a\.sessionAuthorization\)\.toBe\(`Bearer \$\{accountA\.sessionToken\}`\)[\s\S]*requests\.b\.sessionAuthorization\)\.toBe\(`Bearer \$\{accountB\.sessionToken\}`\)/,
+ 'parallel-account E2E must prove each session POST uses its own current bearer token',
+);
+assert.match(
+ callAccessJoinSpec,
+ /requests\.a\.sessionBody\)\.toEqual\(\{\s*verified_user_id:\s*accountA\.userId,\s*verified_session_id:\s*accountA\.sessionId,\s*\}\)[\s\S]*requests\.b\.sessionBody\)\.toEqual\(\{\s*verified_user_id:\s*accountB\.userId,\s*verified_session_id:\s*accountB\.sessionId,\s*\}\)/,
+ 'parallel-account E2E must prove verified link contexts are not crossed between accounts',
+);
+assert.match(
+ callAccessJoinSpec,
+ /storedA\.sessionToken\)\.toBe\(accountA\.issuedCallAccessToken\)[\s\S]*storedB\.sessionToken\)\.toBe\(accountB\.sessionToken\)[\s\S]*storedB\.sessionToken\)\.not\.toBe\(accountA\.issuedCallAccessToken\)[\s\S]*storedB\.sessionToken\)\.not\.toBe\(accountB\.rejectedCallAccessToken\)/,
+ 'parallel-account E2E must prove localStorage/session state remains isolated after mixed success and conflict responses',
+);
+assert.match(
+ callAccessJoinSpec,
+ /dialogB[\s\S]*not\.toContainText\('Foreign Linked Call Title'\)[\s\S]*foreignNeedlesForB[\s\S]*not\.toContainText\(value\)/,
+ 'parallel-account E2E must prove the rejected second account sees no foreign UI data',
+);
+assert.match(
+ callAccessJoinSpec,
+ /requests\.a\.joinGetCount\)\.toBe\(1\)[\s\S]*requests\.b\.joinGetCount\)\.toBe\(1\)[\s\S]*requests\.a\.sessionPostCount\)\.toBe\(1\)[\s\S]*requests\.b\.sessionPostCount\)\.toBe\(1\)/,
+ 'parallel-account E2E must guard against reload loops or duplicate session POSTs in either context',
+);
+
+console.log('[call-access-verified-context-ui-contract] PASS');
diff --git a/demo/video-chat/frontend-vue/tests/contract/call-app-csp-postmessage-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/call-app-csp-postmessage-contract.mjs
new file mode 100644
index 000000000..fd157d027
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/contract/call-app-csp-postmessage-contract.mjs
@@ -0,0 +1,106 @@
+import assert from 'node:assert/strict';
+import { readFile } from 'node:fs/promises';
+import path from 'node:path';
+
+const root = path.resolve(new URL('../..', import.meta.url).pathname);
+const repoRoot = path.resolve(root, '../../..');
+
+async function read(relativePath) {
+ return readFile(path.join(repoRoot, relativePath), 'utf8');
+}
+
+const [
+ hostSource,
+ workspaceStateSource,
+ iframeBridgeSource,
+ crdtBridgeSource,
+ callAppStaticSource,
+ edgeSource,
+] = await Promise.all([
+ read('demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppWorkspaceHost.vue'),
+ read('demo/video-chat/frontend-vue/src/domain/realtime/callApps/callAppWorkspaceState.js'),
+ read('demo/video-chat/frontend-vue/src/domain/realtime/callApps/useCallAppIframeBridge.js'),
+ read('demo/video-chat/frontend-vue/src/domain/realtime/callApps/useCallAppCrdtBridge.js'),
+ read('demo/video-chat/edge/call_app_static.php'),
+ read('demo/video-chat/edge/edge.php'),
+]);
+
+assert.match(
+ workspaceStateSource,
+ /function normalizeConfiguredCallAppOrigin[\s\S]*https:\/\/\$\{trimmed\}[\s\S]*return parsed\.toString\(\)\.replace\(\/\\\/\+\$\/,\s*''\)/,
+ 'Call App iframe origin must accept the configured whiteboard host even when it is supplied as a bare host',
+);
+
+assert.match(
+ workspaceStateSource,
+ /VITE_VIDEOCHAT_CALL_APP_ORIGIN[\s\S]*normalizeConfiguredCallAppOrigin[\s\S]*\['app', 'apps', 'whiteboard'\]\.includes\(parts\[0\]\)[\s\S]*parts\[0\] = hostAppKey/s,
+ 'Call App iframe URL must use the configured whiteboard host and derive future app subdomains from that host family',
+);
+
+assert.doesNotMatch(
+ hostSource,
+ /\scsp=/,
+ 'Call App iframe host must not use the iframe csp attribute because CSP is enforced by the served response',
+);
+
+assert.match(
+ hostSource,
+ /sandbox="allow-scripts allow-forms allow-pointer-lock allow-downloads"/,
+ 'Call App iframe must keep the sandbox policy that yields an opaque iframe origin',
+);
+
+assert.match(
+ callAppStaticSource,
+ /function videochat_edge_call_app_content_security_policy[\s\S]*"connect-src 'self'"[\s\S]*"img-src 'self' data: blob:"[\s\S]*'frame-ancestors ' \. \$frameAncestor[\s\S]*Content-Security-Policy'\] = videochat_edge_call_app_content_security_policy/s,
+ 'Call App static CSP must be compatible with the whiteboard host and allow only the configured app embedder origin',
+);
+
+assert.match(
+ callAppStaticSource,
+ /Allow-CSP-From'[\s\S]*allowedEmbedderOrigin/,
+ 'Call App static response must advertise the trusted embedder origin for browser Embedded-CSP compatibility',
+);
+
+assert.match(
+ edgeSource,
+ /videochat_edge_serve_call_app_static\([\s\S]*'https:\/\/' \. \$domain/,
+ 'edge must serialize the frontend app origin as the trusted frame ancestor for Call App subdomain responses',
+);
+
+assert.match(
+ iframeBridgeSource,
+ /import \{ computed, isProxy, isRef,[\s\S]*toRaw, unref,[\s\S]*\} from 'vue'/,
+ 'Call App bridge sanitizer must unwrap Vue refs/proxies before postMessage',
+);
+
+assert.match(
+ iframeBridgeSource,
+ /function rawBridgeValue[\s\S]*isRef\(value\)[\s\S]*isProxy\(value\) \? toRaw\(value\) : value/s,
+ 'Call App bridge sanitizer must normalize reactive values to raw cloneable values',
+);
+
+assert.match(
+ iframeBridgeSource,
+ /export function cloneSafeCallAppBridgePayload[\s\S]*structuredClone\(sanitized\)[\s\S]*JSON\.parse\(JSON\.stringify\(sanitized\)\)/s,
+ 'Call App bridge must prove payloads are structured-clone safe and retain a JSON fallback',
+);
+
+assert.match(
+ iframeBridgeSource,
+ /failedPostGeneration[\s\S]*if \(failedPostGeneration === generation\) return false[\s\S]*reason: 'post_message_failed'/s,
+ 'Call App launch postMessage failures must be terminal for the current launch generation and must not retry-loop',
+);
+
+assert.match(
+ crdtBridgeSource,
+ /cloneSafeCallAppBridgePayload[\s\S]*frameWindow\.postMessage\(message, '\*'\)/s,
+ 'Call App CRDT bridge responses must also pass through the clone-safe postMessage serializer',
+);
+
+assert.doesNotMatch(
+ `${hostSource}\n${workspaceStateSource}\n${iframeBridgeSource}\n${crdtBridgeSource}`,
+ /location\.reload|window\.location\.reload|iframeRef\.value\.src\s*=/,
+ 'Call App iframe launch, serialization, and CSP failures must not trigger frontend reload loops',
+);
+
+console.log('[call-app-csp-postmessage-contract] PASS');
diff --git a/demo/video-chat/frontend-vue/tests/contract/call-app-frame-csp-headers-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/call-app-frame-csp-headers-contract.mjs
new file mode 100644
index 000000000..43ea3f820
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/contract/call-app-frame-csp-headers-contract.mjs
@@ -0,0 +1,132 @@
+import assert from 'node:assert/strict';
+import { readFile } from 'node:fs/promises';
+import path from 'node:path';
+
+const root = path.resolve(new URL('../..', import.meta.url).pathname);
+const repoRoot = path.resolve(root, '../../..');
+
+async function read(relativePath) {
+ return readFile(path.join(repoRoot, relativePath), 'utf8');
+}
+
+const [
+ edgeSource,
+ callAppStaticSource,
+ workspaceStateSource,
+ semanticDnsSource,
+ semanticDnsContract,
+ deploySmoke,
+ prodDebug,
+] = await Promise.all([
+ read('demo/video-chat/edge/edge.php'),
+ read('demo/video-chat/edge/call_app_static.php'),
+ read('demo/video-chat/frontend-vue/src/domain/realtime/callApps/callAppWorkspaceState.js'),
+ read('demo/video-chat/backend-king-php/domain/call_apps/call_app_semantic_dns.php'),
+ read('demo/video-chat/backend-king-php/tests/call-app-semantic-dns-contract.php'),
+ read('demo/video-chat/scripts/deploy-smoke.sh'),
+ read('demo/video-chat/scripts/prod-debug.sh'),
+]);
+
+assert.match(
+ callAppStaticSource,
+ /function videochat_edge_call_app_normalize_origin[\s\S]*parse_url\(\$trimmed\)[\s\S]*in_array\(\$scheme, \['http', 'https'\], true\)[\s\S]*return \$origin/s,
+ 'Call App static hosting must normalize trusted embedder origins before serializing CSP headers',
+);
+
+assert.match(
+ callAppStaticSource,
+ /function videochat_edge_call_app_frame_ancestor[\s\S]*return \$normalized !== '' \? \$normalized : "'none'"/,
+ 'Call App CSP must fail closed when the configured embedder origin is invalid',
+);
+
+assert.match(
+ callAppStaticSource,
+ /function videochat_edge_call_app_content_security_policy[\s\S]*"default-src 'self'"[\s\S]*"script-src 'self' 'unsafe-inline'"[\s\S]*"style-src 'self' 'unsafe-inline'"[\s\S]*"connect-src 'self'"[\s\S]*"img-src 'self' data: blob:"[\s\S]*"font-src 'self'"[\s\S]*"base-uri 'none'"[\s\S]*"object-src 'none'"[\s\S]*"frame-src 'none'"[\s\S]*'frame-ancestors ' \. \$frameAncestor/s,
+ 'Call App CSP must be explicit, self-scoped, and frame-ancestor compatible with the parent app',
+);
+
+assert.doesNotMatch(
+ callAppStaticSource,
+ /script-src[^"'\n;]*\*|connect-src[^"'\n;]*\*|frame-ancestors[^"'\n;]*\*/,
+ 'Call App CSP must not weaken script, connect, or frame-ancestor policy to wildcards',
+);
+
+assert.match(
+ callAppStaticSource,
+ /Content-Security-Policy'\] = videochat_edge_call_app_content_security_policy\(\$allowedEmbedderOrigin\)[\s\S]*Allow-CSP-From'\] = \$allowedEmbedderOrigin/s,
+ 'Call App HTML responses must deliver CSP and accept the same normalized embedder origin for Embedded-CSP checks',
+);
+
+assert.match(
+ edgeSource,
+ /\$serveStatic = static function \([\s\S]*use \([\s\S]*\$domain[\s\S]*\$isCallAppAsset[\s\S]*videochat_edge_call_app_content_security_policy\(\$allowedEmbedderOrigin\)[\s\S]*Allow-CSP-From'\] = \$allowedEmbedderOrigin/s,
+ 'edge fallback for packaged /call-app assets must use the same Call App CSP helper and app embedder origin',
+);
+
+assert.match(
+ edgeSource,
+ /videochat_edge_serve_call_app_static\([\s\S]*'https:\/\/' \. \$domain/,
+ 'dedicated Call App host must receive the configured frontend app origin as frame ancestor',
+);
+
+assert.doesNotMatch(
+ `${edgeSource}\n${callAppStaticSource}`,
+ /X-Frame-Options|frame-ancestors 'self'/,
+ 'Call App frame responses must not reintroduce X-Frame-Options or self-only frame ancestors',
+);
+
+assert.match(
+ workspaceStateSource,
+ /normalizeConfiguredCallAppOrigin[\s\S]*VITE_VIDEOCHAT_CALL_APP_ORIGIN[\s\S]*\['app', 'apps', 'whiteboard'\]\.includes\(parts\[0\]\)[\s\S]*parts\[0\] = hostAppKey/s,
+ 'frontend iframe URL generation must stay aligned with the dedicated Call App host family',
+);
+
+assert.match(
+ edgeSource,
+ /if \(\$label === '' \|\| str_contains\(\$label, '\.'\)\) \{[\s\S]*return ''/s,
+ 'edge Call App host routing must reject nested app labels like app.whiteboard.kingrt.com',
+);
+
+assert.match(
+ semanticDnsContract,
+ /array_merge\(\$whiteboard, \['app_key' => 'kanban'\]\)[\s\S]*'whiteboard\.kingrt\.test'[\s\S]*'kanban\.kingrt\.test'/s,
+ 'Semantic-DNS contract must prove future app hosts are derived as app-key.root-domain, not nested below whiteboard',
+);
+
+assert.match(
+ deploySmoke,
+ /expect_response_header_contains[\s\S]*Content-Security-Policy[\s\S]*frame-ancestors https:\/\/\$\{DEPLOY_APP_DOMAIN\}[\s\S]*Allow-CSP-From[\s\S]*https:\/\/\$\{DEPLOY_APP_DOMAIN\}[\s\S]*expect_response_header_absent[\s\S]*X-Frame-Options/s,
+ 'deploy smoke must verify production Whiteboard CSP headers and absence of frame-blocking X-Frame-Options',
+);
+
+assert.match(
+ prodDebug,
+ /call_app_csp_header_proof[\s\S]*\/public\/index\.html[\s\S]*\/call-app\/whiteboard\/public\/index\.html/s,
+ 'prod-debug must provide a read-only proof path for both Whiteboard Call App production entrypoints',
+);
+
+assert.match(
+ prodDebug,
+ /Content-Security-Policy[\s\S]*frame-ancestors https:\/\/\$\{DEPLOY_APP_DOMAIN\}[\s\S]*script-src 'self'[\s\S]*connect-src 'self'[\s\S]*Allow-CSP-From[\s\S]*https:\/\/\$\{DEPLOY_APP_DOMAIN\}[\s\S]*X-Frame-Options/s,
+ 'prod-debug must prove Call App CSP and Embedded-CSP headers are compatible with the configured app embedder',
+);
+
+assert.match(
+ prodDebug,
+ /wildcard_frame_ancestors_pattern[\s\S]*frame-ancestors\[\^;\]\*\\\*[\s\S]*wildcard_frame_src_pattern[\s\S]*frame-src\[\^;\]\*\\\*[\s\S]*wildcard_script_src_pattern[\s\S]*script-src\[\^;\]\*\\\*[\s\S]*wildcard_connect_src_pattern[\s\S]*connect-src\[\^;\]\*\\\*/,
+ 'prod-debug must reject wildcard frame/script/connect CSP directives in production responses',
+);
+
+assert.match(
+ prodDebug,
+ /nested_pattern="https\?:\/\/\[A-Za-z0-9\.-\]\+\\\\\.\$\{escaped_app_domain\}"[\s\S]*must not reference nested \*\.\$\{DEPLOY_APP_DOMAIN\} service origins/s,
+ 'prod-debug must reject nested *.app.kingrt.com service origins in Call App production responses',
+);
+
+assert.doesNotMatch(
+ `${edgeSource}\n${callAppStaticSource}\n${deploySmoke}\n${prodDebug}`,
+ /location\.reload|window\.location\.reload|reload\(\)/,
+ 'edge/deploy Call App frame CSP failures must not be handled through reload loops',
+);
+
+console.log('[call-app-frame-csp-headers-contract] PASS');
diff --git a/demo/video-chat/frontend-vue/tests/contract/call-app-iframe-launch-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/call-app-iframe-launch-contract.mjs
index e3949d2a3..a934efc84 100644
--- a/demo/video-chat/frontend-vue/tests/contract/call-app-iframe-launch-contract.mjs
+++ b/demo/video-chat/frontend-vue/tests/contract/call-app-iframe-launch-contract.mjs
@@ -95,7 +95,7 @@ assert.match(
assert.match(
callAppStaticSource,
- /Content-Security-Policy'[\s\S]*default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'; connect-src 'self'; img-src 'self' data: blob:/,
+ /function videochat_edge_call_app_content_security_policy[\s\S]*"default-src 'self'"[\s\S]*"script-src 'self' 'unsafe-inline'"[\s\S]*"style-src 'self' 'unsafe-inline'"[\s\S]*"connect-src 'self'"[\s\S]*"img-src 'self' data: blob:"[\s\S]*Content-Security-Policy'\] = videochat_edge_call_app_content_security_policy/s,
'Call App static edge must deliver a CSP at least as strong as the iframe csp attribute',
);
@@ -107,7 +107,7 @@ assert.match(
assert.match(
callAppStaticSource,
- /frame-ancestors ' \. \$allowedEmbedderOrigin/,
+ /function videochat_edge_call_app_content_security_policy[\s\S]*'frame-ancestors ' \. \$frameAncestor/,
'Call App static edge must restrict embedders to the trusted app origin',
);
@@ -125,7 +125,7 @@ assert.doesNotMatch(
assert.match(
edgeSource,
- /\$isCallAppAsset = str_starts_with\(\$path, '\/call-app\/'\)[\s\S]*Content-Security-Policy[\s\S]*frame-ancestors 'self'/,
+ /\$isCallAppAsset = str_starts_with\(\$path, '\/call-app\/'\)[\s\S]*videochat_edge_call_app_content_security_policy/,
'edge must enforce Call App CSP on the served response instead of iframe csp',
);
@@ -170,19 +170,19 @@ assert.match(
assert.match(
bridgeSource,
- /safePostMessagePayload[\s\S]*type:\s*['"]call_app\.launch['"][\s\S]*launch_token[\s\S]*sanitizeCallAppBridgePayload\(safePostMessagePayload\(session, launch\.value,[\s\S]*participantDisplayName/s,
+ /safePostMessagePayload[\s\S]*type:\s*['"]call_app\.launch['"][\s\S]*launch_token[\s\S]*cloneSafeCallAppBridgePayload\([\s\S]*safePostMessagePayload\(session, launch\.value,[\s\S]*participantDisplayName/s,
'parent bridge must send the launch token only through the sanitized iframe bridge message',
);
assert.match(
bridgeSource,
- /function sanitizeCallAppBridgePayload[\s\S]*Array\.isArray\(value\)[\s\S]*map\(\(item\) => sanitizeCallAppBridgePayload/s,
- 'parent bridge must convert reactive/proxy arrays into cloneable arrays before postMessage',
+ /function rawBridgeValue[\s\S]*isProxy\(value\)[\s\S]*function sanitizeCallAppBridgePayload[\s\S]*Array\.isArray\(value\)[\s\S]*Array\.from\(value, \(item\) => sanitizeCallAppBridgePayload/s,
+ 'parent bridge must unwrap reactive/proxy values and convert arrays into cloneable arrays before postMessage',
);
assert.match(
bridgeSource,
- /frameWindow\.postMessage\([\s\S]*sanitizeCallAppBridgePayload\(safePostMessagePayload\(session, launch\.value,[\s\S]*'\*'/s,
+ /frameWindow\.postMessage\([\s\S]*message,[\s\S]*'\*'/s,
'parent launch bridge must send only sanitized cloneable payloads',
);
diff --git a/demo/video-chat/frontend-vue/tests/contract/call-app-marketplace-to-call-journey-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/call-app-marketplace-to-call-journey-contract.mjs
index 8bbd27713..895255785 100644
--- a/demo/video-chat/frontend-vue/tests/contract/call-app-marketplace-to-call-journey-contract.mjs
+++ b/demo/video-chat/frontend-vue/tests/contract/call-app-marketplace-to-call-journey-contract.mjs
@@ -24,6 +24,7 @@ const [
catalogStoreSource,
adminMarketplaceSource,
adminMarketplaceTableSource,
+ marketplaceEntitlementTestSource,
crdtBridgeSource,
whiteboardSource,
whiteboardRuntimeSource,
@@ -34,6 +35,7 @@ const [
read('demo/video-chat/frontend-vue/src/stores/callAppsCatalogStore.js'),
read('demo/video-chat/frontend-vue/src/modules/marketplace/pages/AdminMarketplaceView.vue'),
read('demo/video-chat/frontend-vue/src/modules/marketplace/pages/AdminMarketplaceTable.vue'),
+ read('demo/video-chat/backend-king-php/tests/call-app-marketplace-entitlement-contract.php'),
read('demo/video-chat/frontend-vue/src/domain/realtime/callApps/useCallAppCrdtBridge.js'),
read('demo/call-app/whiteboard/public/index.html'),
read('demo/call-app/whiteboard/public/whiteboard.js'),
@@ -58,6 +60,12 @@ assert.match(
'backend journey must prove installed whiteboard availability through Semantic-DNS/MCP discovery',
);
+assert.match(
+ marketplaceEntitlementTestSource,
+ /catalog whiteboard must start not installed for organization[\s\S]*catalog whiteboard must expose add-to-organization action before install[\s\S]*post-install Whiteboard must appear in call availability/s,
+ 'marketplace contract must prove catalog visibility, add-to-organization action, and post-install call availability',
+);
+
assert.match(
lifecycleTestSource,
/non-owner participant must not attach Call App[\s\S]*owner attach should create session[\s\S]*default-allowed participant launch token should return 201/,
@@ -96,14 +104,14 @@ assert.match(
assert.match(
adminMarketplaceSource,
- /\/api\/marketplace\/call-apps\/\$\{encodeURIComponent\(appKey\)\}\/orders[\s\S]*\/api\/marketplace\/call-apps\/\$\{encodeURIComponent\(appKey\)\}\/installations/s,
- 'admin marketplace must order and install Call Apps for the active organization from the real marketplace endpoints',
+ /const appKey = String\(catalog\?\.app_key \|\| ''\)\.trim\(\)[\s\S]*\/api\/marketplace\/call-apps\/\$\{encodeURIComponent\(appKey\)\}\/orders[\s\S]*\/api\/marketplace\/call-apps\/\$\{encodeURIComponent\(appKey\)\}\/installations/s,
+ 'admin marketplace must order and install catalog-only Call Apps for the active organization from the real marketplace endpoints',
);
assert.match(
adminMarketplaceTableSource,
- /catalogApp\(app\)[\s\S]*install-call-app[\s\S]*Verify organization installation[\s\S]*Install for organization/s,
- 'admin marketplace table must expose idempotent catalog-backed organization install actions',
+ /catalogApp\(app\)[\s\S]*install-call-app[\s\S]*isCatalogOnly\(app\)[\s\S]*marketplace\.call_app_install\.verify[\s\S]*marketplace\.call_app_install\.install/s,
+ 'admin marketplace table must expose idempotent catalog-backed organization install actions for catalog-only rows',
);
assert.doesNotMatch(
diff --git a/demo/video-chat/frontend-vue/tests/contract/call-app-production-deploy-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/call-app-production-deploy-contract.mjs
index 75e9baa0a..35af26d96 100644
--- a/demo/video-chat/frontend-vue/tests/contract/call-app-production-deploy-contract.mjs
+++ b/demo/video-chat/frontend-vue/tests/contract/call-app-production-deploy-contract.mjs
@@ -20,6 +20,7 @@ const [
edgeDockerfile,
edgePhp,
deploy,
+ deploySmoke,
deployHetzner,
] = await Promise.all([
read('SPRINT.md'),
@@ -32,6 +33,7 @@ const [
read('demo/video-chat/edge/Dockerfile'),
read('demo/video-chat/edge/edge.php'),
read('demo/video-chat/scripts/deploy.sh'),
+ read('demo/video-chat/scripts/deploy-smoke.sh'),
read('demo/video-chat/scripts/lib/deploy-hetzner.sh'),
]);
@@ -57,8 +59,11 @@ for (const functionName of [
for (const envKey of [
'VIDEOCHAT_DEPLOY_CALL_APP_DOMAIN',
+ 'VIDEOCHAT_DEPLOY_REGISTRY_DOMAIN',
'VIDEOCHAT_DEPLOY_MOTHERNODE_DOMAIN',
'VIDEOCHAT_CALL_APP_PUBLIC_HOST',
+ 'VIDEOCHAT_CALL_APP_PUBLIC_ROOT_DOMAIN',
+ 'VIDEOCHAT_CALL_APP_REGISTRY_HOST',
'VIDEOCHAT_CALL_APP_MOTHERNODE_HOST',
'VIDEOCHAT_CALL_APP_MCP_ENDPOINT',
'VIDEOCHAT_CALL_APP_SEMANTIC_DNS_REGISTER',
@@ -82,19 +87,19 @@ assert.match(
assert.match(
semanticDnsTest,
- /VIDEOCHAT_DEPLOY_CALL_APP_DOMAIN[\s\S]*apps\.kingrt\.test[\s\S]*VIDEOCHAT_DEPLOY_MOTHERNODE_DOMAIN[\s\S]*mother\.kingrt\.test[\s\S]*videochat_call_app_register_runtime_semantic_dns_catalog/s,
- 'backend contract must prove deploy env parsing and runtime Mothernode registration',
+ /VIDEOCHAT_DEPLOY_CALL_APP_DOMAIN[\s\S]*whiteboard\.kingrt\.test[\s\S]*VIDEOCHAT_DEPLOY_REGISTRY_DOMAIN[\s\S]*registry\.kingrt\.test[\s\S]*videochat_call_app_register_runtime_semantic_dns_catalog/s,
+ 'backend contract must prove deploy env parsing and runtime registry registration',
);
assert.match(workspaceState, /VITE_VIDEOCHAT_CALL_APP_ORIGIN[\s\S]*CALL_APP_IFRAME_ORIGIN/s);
assert.ok(
- workspaceState.includes("return CALL_APP_IFRAME_ORIGIN !== '' ?"),
+ workspaceState.includes('function callAppOriginForAppKey'),
'Call App iframe URL must honor the dedicated deployment origin',
);
assert.match(
workspaceState,
- /`\$\{CALL_APP_IFRAME_ORIGIN\}\$\{path\}` : path/,
- 'Call App iframe URL must prefix the sanitized app path with the deployment origin',
+ /parts\[0\] = hostAppKey[\s\S]*return origin !== '' \? `\$\{origin\}\$\{path\}` : path/s,
+ 'Call App iframe URL must resolve whiteboard.kingrt.com and future {app_key}.kingrt.com origins',
);
assert.match(
@@ -123,20 +128,41 @@ assert.match(
assert.match(
deploy,
- /DEPLOY_CALL_APP_DOMAIN[\s\S]*apps\.\$\{DEPLOY_DOMAIN\}[\s\S]*DEPLOY_MOTHERNODE_DOMAIN[\s\S]*mother\.\$\{DEPLOY_DOMAIN\}/,
- 'deploy script must default Call App and Mothernode subdomains from the main domain',
+ /DEPLOY_APP_DOMAIN[\s\S]*app\.\$\{DEPLOY_DOMAIN\}[\s\S]*DEPLOY_CALL_APP_DOMAIN[\s\S]*whiteboard\.\$\{DEPLOY_DOMAIN\}[\s\S]*DEPLOY_REGISTRY_DOMAIN[\s\S]*registry\.\$\{DEPLOY_DOMAIN\}/,
+ 'deploy script must split app.kingrt.com from the kingrt.com service root and default Call App/registry subdomains from the root',
);
assert.match(
deploy,
- /CERTBOT_DOMAINS=\([\s\S]*CALL_APP_DOMAIN[\s\S]*MOTHERNODE_DOMAIN[\s\S]*certbot certonly/s,
- 'deploy script must include Call App and Mothernode domains in the certificate SAN set',
+ /CERTBOT_DOMAINS=\([\s\S]*APP_DOMAIN[\s\S]*CALL_APP_DOMAIN[\s\S]*REGISTRY_DOMAIN[\s\S]*certbot certonly/s,
+ 'deploy script must include app, Call App, and registry domains in the certificate SAN set',
+);
+
+assert.match(
+ deploySmoke,
+ /DEPLOY_APP_DOMAIN="\$\{VIDEOCHAT_DEPLOY_APP_DOMAIN:-app\.\$\{DEPLOY_DOMAIN\}\}"[\s\S]*expect_http_code https-frontend 200 "https:\/\/\$\{DEPLOY_APP_DOMAIN\}\/"/s,
+ 'deploy smoke must probe app.kingrt.com as the frontend when kingrt.com is the service root',
+);
+
+assert.match(
+ deploySmoke,
+ /CALL_APP_DOMAIN=\$\{call_app_q\} REGISTRY_DOMAIN=\$\{registry_q\}[\s\S]*"\$\{CALL_APP_DOMAIN\}" "\$\{REGISTRY_DOMAIN\}"/s,
+ 'deploy smoke must verify Call App and registry certificate SANs',
+);
+
+assert.match(
+ deploySmoke,
+ /expect_http_code call-app-whiteboard-host 200 "https:\/\/\$\{DEPLOY_CALL_APP_DOMAIN\}\/public\/index\.html"[\s\S]*expect_http_code call-app-whiteboard-path 200 "https:\/\/\$\{DEPLOY_CALL_APP_DOMAIN\}\/call-app\/whiteboard\/public\/index\.html"/s,
+ 'deploy smoke must verify the semantic whiteboard.kingrt.com host and packaged Call App path',
);
assert.match(
deployHetzner,
- /VIDEOCHAT_DEPLOY_CALL_APP_DOMAIN[\s\S]*VIDEOCHAT_DEPLOY_MOTHERNODE_DOMAIN[\s\S]*DEPLOY_CALL_APP_DOMAIN[\s\S]*DEPLOY_MOTHERNODE_DOMAIN/s,
- 'Hetzner deploy helper must persist and provision Call App and Mothernode DNS names',
+ /VIDEOCHAT_DEPLOY_APP_DOMAIN[\s\S]*VIDEOCHAT_DEPLOY_CALL_APP_DOMAIN[\s\S]*VIDEOCHAT_DEPLOY_REGISTRY_DOMAIN[\s\S]*DEPLOY_APP_DOMAIN[\s\S]*DEPLOY_CALL_APP_DOMAIN[\s\S]*DEPLOY_REGISTRY_DOMAIN/s,
+ 'Hetzner deploy helper must persist and provision app, Call App, and registry DNS names',
);
+assert.ok(!deploy.includes('cnd.${DEPLOY_DOMAIN}') && !deployHetzner.includes('cnd.${DEPLOY_DOMAIN}'), 'production generation must not provision legacy cnd aliases');
+assert.ok(!deploy.includes('mother.${DEPLOY_DOMAIN}') && !deployHetzner.includes('mother.${DEPLOY_DOMAIN}'), 'production generation must not use mother.kingrt.com as the canonical registry host');
+
console.log('[call-app-production-deploy-contract] PASS');
diff --git a/demo/video-chat/frontend-vue/tests/contract/call-app-sidebar-access-ux-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/call-app-sidebar-access-ux-contract.mjs
new file mode 100644
index 000000000..dbee75380
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/contract/call-app-sidebar-access-ux-contract.mjs
@@ -0,0 +1,73 @@
+import assert from 'node:assert/strict';
+import { readFile } from 'node:fs/promises';
+import path from 'node:path';
+
+const root = path.resolve(new URL('../..', import.meta.url).pathname);
+const repoRoot = path.resolve(root, '../../..');
+
+async function read(relativePath) {
+ return readFile(path.join(repoRoot, relativePath), 'utf8');
+}
+
+const [sidebarSource, sidebarStyles, grantButtonSource] = await Promise.all([
+ read('demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppsSidebarPanel.vue'),
+ read('demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppsSidebarPanel.css'),
+ read('demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppParticipantGrantButton.vue'),
+]);
+const sidebarCombinedSource = `${sidebarSource}\n${sidebarStyles}`;
+
+assert.match(
+ sidebarSource,
+ /call-apps-item-side[\s\S]*call-apps-item-state[\s\S]*call-apps-item-action[\s\S]*Selected[\s\S]*Select/,
+ 'Call Apps list rows must expose a responsive selected/select action area instead of relying on a bare app key',
+);
+
+assert.match(
+ sidebarSource,
+ /fieldset class="call-apps-policy"[\s\S]*Default participant access[\s\S]*type="radio" value="blocked_by_default"[\s\S]*Grant individually[\s\S]*type="radio" value="allowed_by_default"[\s\S]*Participants can open/,
+ 'Call Apps attach flow must make default participant access explicit with inline grant/restrict choices',
+);
+
+assert.match(
+ sidebarSource,
+ /call-apps-access-default[\s\S]*activeSessionDefaultAccessLabel[\s\S]*Default: allowed[\s\S]*Default: blocked/s,
+ 'Call Apps access panel must show the active session default access policy',
+);
+
+assert.match(
+ sidebarSource,
+ /call-apps-access-state" :class="grantStateClass\(participant\)"[\s\S]*variant="label"[\s\S]*@grant-updated="applyLocalGrantUpdate"/,
+ 'Call Apps access rows must combine readable state badges with labeled backend-backed grant buttons',
+);
+
+assert.match(
+ sidebarCombinedSource,
+ /\.call-apps-list-item[\s\S]*grid-template-columns:\s*minmax\(0,\s*1fr\)[\s\S]*\.call-apps-item-side[\s\S]*flex-wrap:\s*wrap[\s\S]*\.call-apps-access-row[\s\S]*grid-template-columns:\s*minmax\(0,\s*1fr\)[\s\S]*@container\s*\(min-width:\s*380px\)[\s\S]*\.call-apps-access-row[\s\S]*grid-template-columns:\s*minmax\(0,\s*1fr\)\s*auto/s,
+ 'Call Apps sidebar list and access actions must stack at narrow widths and expand at wider sidebar widths',
+);
+
+assert.doesNotMatch(
+ sidebarCombinedSource,
+ /call-apps-card|card-heavy|nested-card/i,
+ 'Call Apps sidebar access UX must not introduce nested card UI',
+);
+
+assert.match(
+ grantButtonSource,
+ /variant:\s*\{[\s\S]*validator:\s*\(value\) => \['icon', 'label'\]\.includes\(value\)[\s\S]*variant-label/,
+ 'Call App grant button must support a labeled sidebar variant while preserving the icon variant',
+);
+
+assert.match(
+ grantButtonSource,
+ /effectiveGrantState\.value === 'allowed'[\s\S]*remove_user\.png[\s\S]*add_to_call\.png[\s\S]*effectiveGrantState\.value === 'allowed' \? 'Revoke' : 'Allow'/,
+ 'Call App grant button must show the next action clearly: allowed means Revoke, denied means Allow',
+);
+
+assert.match(
+ grantButtonSource,
+ /Only the call owner or a moderator can change Call App access[\s\S]*buttonLabel\.value\} Call App access for/,
+ 'Call App grant button titles must explain unavailable controls and name the affected participant',
+);
+
+console.log('[call-app-sidebar-access-ux-contract] PASS');
diff --git a/demo/video-chat/frontend-vue/tests/contract/call-app-sidebar-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/call-app-sidebar-contract.mjs
index cb0c1a172..b8c268082 100644
--- a/demo/video-chat/frontend-vue/tests/contract/call-app-sidebar-contract.mjs
+++ b/demo/video-chat/frontend-vue/tests/contract/call-app-sidebar-contract.mjs
@@ -11,6 +11,7 @@ async function read(relativePath) {
const [
sidebarSource,
+ sidebarStyles,
leftSidebarSource,
callWorkspaceSource,
tabsComposableSource,
@@ -19,6 +20,7 @@ const [
sprintSource,
] = await Promise.all([
read('demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppsSidebarPanel.vue'),
+ read('demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppsSidebarPanel.css'),
read('demo/video-chat/frontend-vue/src/layouts/CallWorkspaceLeftSidebar.vue'),
read('demo/video-chat/frontend-vue/src/domain/realtime/CallWorkspaceView.vue'),
read('demo/video-chat/frontend-vue/src/layouts/useCallLeftSidebarTabs.js'),
@@ -26,6 +28,7 @@ const [
read('demo/video-chat/frontend-vue/src/stores/callAppsCatalogStore.js'),
read('SPRINT.md'),
]);
+const sidebarCombinedSource = `${sidebarSource}\n${sidebarStyles}`;
assert.match(
shellSource,
@@ -112,7 +115,7 @@ assert.match(
);
assert.match(
- sidebarSource,
+ sidebarCombinedSource,
/\.call-apps-sidebar[\s\S]*container-type:\s*inline-size[\s\S]*\.call-apps-search[\s\S]*flex-direction:\s*row-reverse[\s\S]*gap:\s*clamp\([\s\S]*padding:\s*clamp\(/s,
'Call Apps sidebar search must keep the submit icon right-aligned with responsive spacing',
);
@@ -142,7 +145,7 @@ assert.match(
);
assert.match(
- sidebarSource,
+ sidebarCombinedSource,
/\.call-apps-pagination[\s\S]*justify-content:\s*flex-end[\s\S]*flex-wrap:\s*wrap[\s\S]*gap:\s*clamp\([\s\S]*padding:\s*clamp\(/s,
'Call Apps sidebar pagination must use right-aligned responsive action spacing',
);
@@ -172,7 +175,7 @@ assert.match(
);
assert.match(
- sidebarSource,
+ sidebarCombinedSource,
/@container\s*\(min-width:\s*380px\)[\s\S]*\.call-apps-list-item[\s\S]*grid-template-columns:\s*minmax\(0,\s*1fr\)\s*auto[\s\S]*\.call-apps-detail-grid[\s\S]*grid-template-columns:\s*minmax\(0,\s*1fr\)\s*minmax\(0,\s*1fr\)/s,
'Call Apps sidebar must adapt list and detail grids at wider container sizes',
);
diff --git a/demo/video-chat/frontend-vue/tests/contract/call-app-whiteboard-cursors-access-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/call-app-whiteboard-cursors-access-contract.mjs
new file mode 100644
index 000000000..f9c49b3ae
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/contract/call-app-whiteboard-cursors-access-contract.mjs
@@ -0,0 +1,119 @@
+import assert from 'node:assert/strict';
+import { readFile } from 'node:fs/promises';
+import path from 'node:path';
+import {
+ callAppPresenceUserAuthorizedForSession,
+ normalizeCallAppPresenceParticipantRows,
+ normalizeCallAppPresencePayload,
+} from '../../src/domain/realtime/callApps/callAppPresenceRelay.js';
+
+const root = path.resolve(new URL('../..', import.meta.url).pathname);
+const repoRoot = path.resolve(root, '../../..');
+
+async function read(relativePath) {
+ return readFile(path.join(repoRoot, relativePath), 'utf8');
+}
+
+const [bridgeSource, relaySource, whiteboardHtml, whiteboardCss, whiteboardRuntime, e2eSource] = await Promise.all([
+ read('demo/video-chat/frontend-vue/src/domain/realtime/callApps/useCallAppCrdtBridge.js'),
+ read('demo/video-chat/frontend-vue/src/domain/realtime/callApps/callAppPresenceRelay.js'),
+ read('demo/call-app/whiteboard/public/index.html'),
+ read('demo/call-app/whiteboard/public/whiteboard.css'),
+ read('demo/call-app/whiteboard/public/whiteboard.js'),
+ read('demo/video-chat/frontend-vue/tests/e2e/call-app-whiteboard.spec.js'),
+]);
+const whiteboardSource = `${whiteboardHtml}\n${whiteboardCss}\n${whiteboardRuntime}`;
+
+const session = {
+ id: 'whiteboard-session-contract',
+ app_key: 'whiteboard',
+ default_app_policy: 'blocked_by_default',
+ grants: [
+ { subject_type: 'user', user_id: 10, grant_state: 'allowed' },
+ { subject_type: 'user', user_id: 20, grant_state: 'allowed' },
+ { subject_type: 'user', user_id: 30, grant_state: 'denied' },
+ ],
+};
+
+assert.equal(callAppPresenceUserAuthorizedForSession(session, 10), true, 'allowed user should be authorized for Call App presence');
+assert.equal(callAppPresenceUserAuthorizedForSession(session, 30), false, 'denied user should not be authorized for Call App presence');
+assert.equal(callAppPresenceUserAuthorizedForSession(session, 0), false, 'anonymous zero user id should not be authorized for user presence');
+
+assert.deepEqual(
+ normalizeCallAppPresenceParticipantRows([
+ { userId: 10, displayName: 'Owner' },
+ { userId: 20, displayName: 'Participant' },
+ { userId: 30, displayName: 'Revoked' },
+ { userId: 40, displayName: 'Disconnected', isRoomMember: false },
+ ], 10, session),
+ [{ userId: 20, displayName: 'Participant' }],
+ 'presence relay should target only other connected users with allowed Call App grants',
+);
+
+assert.equal(
+ normalizeCallAppPresencePayload('cursor.move', { x: 12, y: 34 }, { actorId: 'user_owner', displayName: 'Owner Name' })?.label,
+ 'Owner Name',
+ 'cursor payloads must carry the sender display name as their label',
+);
+
+assert.match(
+ relaySource,
+ /function defaultGrantStateForSession[\s\S]*export function callAppPresenceUserAuthorizedForSession[\s\S]*grant_state/s,
+ 'presence relay must resolve explicit and default participant grant state before targeting peers',
+);
+
+assert.match(
+ bridgeSource,
+ /callAppPresenceUserAuthorizedForSession\(session,\s*Number\(unrefValue\(currentUserId\)[\s\S]*state:\s*senderAuthorized \? \(accepted \? 'accepted' : 'ignored'\) : 'participant_grant_denied'/s,
+ 'parent bridge must reject presence publishes from users without an allowed Call App grant',
+);
+
+assert.match(
+ bridgeSource,
+ /normalizeCallAppPresenceParticipantRows\([\s\S]*unrefValue\(participants\)[\s\S]*Number\(unrefValue\(currentUserId\)[\s\S]*session/s,
+ 'parent bridge must filter cursor fanout targets through active session grants',
+);
+
+assert.match(
+ bridgeSource,
+ /handleRemotePresence[\s\S]*callAppPresenceUserAuthorizedForSession\(session,\s*Number\(unrefValue\(currentUserId\)[\s\S]*return/s,
+ 'parent bridge must not post remote cursors into an iframe after the local participant grant is revoked',
+);
+
+assert.match(
+ whiteboardSource,
+ /function applyAccessState[\s\S]*state\.cursors\.clear\(\)[\s\S]*state\.selections\.clear\(\)/,
+ 'whiteboard runtime must clear remote cursors and selections when read access is lost',
+);
+
+assert.match(
+ whiteboardSource,
+ /function applyPresence[\s\S]*if \(!canRead\(\)\) return;[\s\S]*label:\s*displayNameLabel\(payload\.label \|\| payload\.display_name/s,
+ 'whiteboard runtime must ignore remote cursor updates without read access and render authorized cursors with display-name labels',
+);
+
+assert.match(
+ whiteboardSource,
+ /const cursorOverlay = document\.getElementById\('cursorOverlay'\)[\s\S]*function syncCursorOverlay[\s\S]*remote-cursor-label[\s\S]*label\.textContent = displayNameLabel\(cursor\.label \|\| cursor\.display_name\)[\s\S]*cursorOverlay\.replaceChildren\(\.\.\.labels\)/s,
+ 'whiteboard runtime must render authorized remote cursor names into the DOM overlay, not only into canvas pixels',
+);
+
+assert.match(
+ whiteboardSource,
+ /function removePresenceForActor[\s\S]*state\.cursors\.delete\(normalizedActorId\)[\s\S]*state\.selections\.delete\(normalizedActorId\)[\s\S]*message\.type === 'call_app\.presence\.leave'/s,
+ 'whiteboard runtime must remove stale remote cursor and selection presence when a remote participant leaves',
+);
+
+assert.match(
+ e2eSource,
+ /injectRemoteCursor\('participant'[\s\S]*label: 'Reviewer'[\s\S]*injectRemoteCursor\('participant'[\s\S]*label: 'Facilitator'[\s\S]*remote-cursor-label'\)\)\.toHaveCount\(3\)[\s\S]*leaveRemoteCursor\('participant', 'user_reviewer_e2e'\)[\s\S]*\['Owner', 'Facilitator'\]/s,
+ 'whiteboard E2E must prove multiple named remote cursors render and a remote leave removes only the leaving cursor',
+);
+
+assert.match(
+ e2eSource,
+ /participantLaunchCountBeforeRemoteCursors[\s\S]*launchCount\.participant[\s\S]*participantFrameSrcBeforeRemoteCursors[\s\S]*revoke\('participant'\)[\s\S]*remote-cursor-label'\)\)\.toHaveCount\(0\)[\s\S]*toBe\(participantLaunchCountBeforeRemoteCursors\)[\s\S]*participantFrameSrcBeforeRemoteCursors/s,
+ 'whiteboard E2E must prove cursor cleanup after revoke does not reload the iframe',
+);
+
+console.log('[call-app-whiteboard-cursors-access-contract] PASS');
diff --git a/demo/video-chat/frontend-vue/tests/contract/call-app-whiteboard-install-browser-proof-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/call-app-whiteboard-install-browser-proof-contract.mjs
new file mode 100644
index 000000000..7bb52d55f
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/contract/call-app-whiteboard-install-browser-proof-contract.mjs
@@ -0,0 +1,157 @@
+import assert from 'node:assert/strict';
+import { readFile } from 'node:fs/promises';
+import path from 'node:path';
+
+const root = path.resolve(new URL('../..', import.meta.url).pathname);
+const repoRoot = path.resolve(root, '../../..');
+
+async function read(relativePath) {
+ return readFile(path.join(repoRoot, relativePath), 'utf8');
+}
+
+function assertContains(source, needle, message) {
+ assert.ok(source.includes(needle), message);
+}
+
+const [
+ e2eSource,
+ marketplaceViewSource,
+ marketplaceTableSource,
+ sidebarSource,
+ sidebarStyles,
+ grantButtonSource,
+ packageJsonSource,
+] = await Promise.all([
+ read('demo/video-chat/frontend-vue/tests/e2e/call-app-whiteboard-install-sidebar.spec.js'),
+ read('demo/video-chat/frontend-vue/src/modules/marketplace/pages/AdminMarketplaceView.vue'),
+ read('demo/video-chat/frontend-vue/src/modules/marketplace/pages/AdminMarketplaceTable.vue'),
+ read('demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppsSidebarPanel.vue'),
+ read('demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppsSidebarPanel.css'),
+ read('demo/video-chat/frontend-vue/src/domain/realtime/callApps/CallAppParticipantGrantButton.vue'),
+ read('demo/video-chat/frontend-vue/package.json'),
+]);
+
+const packageJson = JSON.parse(packageJsonSource);
+const sidebarCombinedSource = `${sidebarSource}\n${sidebarStyles}`;
+
+assert.match(
+ e2eSource,
+ /Install for organization[\s\S]*Whiteboard installed and enabled for this organization/,
+ 'browser proof must start from a user-visible Whiteboard organization install action and installed state',
+);
+
+assert.match(
+ e2eSource,
+ /POST \/api\/marketplace\/call-apps\/whiteboard\/orders[\s\S]*POST \/api\/marketplace\/call-apps\/whiteboard\/installations/,
+ 'browser proof must place an order before installing Whiteboard through marketplace endpoints',
+);
+
+assert.match(
+ e2eSource,
+ /GET \/api\/calls\/\$\{CALL_ID\}\/call-apps\/available[\s\S]*POST \/api\/calls\/\$\{CALL_ID\}\/call-app-sessions/,
+ 'browser proof must verify installed Whiteboard appears through call app availability before attach',
+);
+
+assert.match(
+ e2eSource,
+ /\/api\/call-app-sessions\/session-whiteboard-install-proof\/participant-grants[\s\S]*retired_launch_tokens/,
+ 'browser proof must exercise backend-authoritative grant mutation and token retirement signal',
+);
+
+assert.match(
+ e2eSource,
+ /default_app_policy:\s*'allowed_by_default'[\s\S]*grant_state:\s*'denied'/,
+ 'browser proof must cover explicit default access and participant revoke payloads',
+);
+
+assert.match(
+ e2eSource,
+ /setViewportSize\(\{\s*width:\s*360[\s\S]*scrollWidth <= element\.clientWidth[\s\S]*gridTemplateColumns/,
+ 'browser proof must keep narrow sidebar responsiveness assertions',
+);
+
+assert.match(
+ e2eSource,
+ /Installed[\s\S]*Enabled[\s\S]*Healthy[\s\S]*Select[\s\S]*Default: allowed[\s\S]*Revoke[\s\S]*Blocked[\s\S]*Allow/,
+ 'browser proof must assert installed app availability and usable access control state transitions',
+);
+
+assert.match(
+ e2eSource,
+ /readWhiteboardAssets[\s\S]*whiteboardFrame[\s\S]*call_app\.launch[\s\S]*call_app\.crdt\.bootstrap\.response[\s\S]*call_app\.crdt\.ops\.response/s,
+ 'browser proof must run the real Whiteboard iframe beside the sidebar host instead of using a placeholder',
+);
+
+assert.match(
+ e2eSource,
+ /showRemoteCursor\('Owner'\)[\s\S]*remote-cursor-label'\)\)\.toHaveText\('Owner'\)[\s\S]*call-apps-access-row\[data-user-id="2"\][\s\S]*remote-cursor-label'\)\)\.toHaveText\('Owner'\)/s,
+ 'browser proof must show Whiteboard cursor labels do not collide with sidebar participant grant controls',
+);
+
+assert.match(
+ e2eSource,
+ /launchCountBeforeGrantToggle[\s\S]*frameSrcBeforeGrantToggle[\s\S]*whiteboardLaunchCount[\s\S]*toBe\(launchCountBeforeGrantToggle\)[\s\S]*whiteboardFrameSrc[\s\S]*frameSrcBeforeGrantToggle/s,
+ 'browser proof must prove sidebar access control updates do not reload the Call App iframe',
+);
+
+assert.doesNotMatch(
+ e2eSource,
+ /sqlite|PDO|INSERT\s+INTO|videochat_bootstrap|manual\s+DB/i,
+ 'browser proof must not rely on manual database edits or backend storage shortcuts',
+);
+
+assert.match(
+ marketplaceViewSource,
+ /\/api\/marketplace\/call-apps\/\$\{encodeURIComponent\(appKey\)\}\/orders[\s\S]*\/api\/marketplace\/call-apps\/\$\{encodeURIComponent\(appKey\)\}\/installations/,
+ 'marketplace UI must keep using backend order and installation endpoints for Call App install',
+);
+
+assert.match(
+ marketplaceTableSource,
+ /callAppStateLabel[\s\S]*marketplace\.call_app_state\.installed[\s\S]*marketplace\.call_app_state\.not_installed[\s\S]*installTitle[\s\S]*marketplace\.call_app_install\.install/s,
+ 'marketplace table must expose install and installed states for Call Apps',
+);
+
+assertContains(
+ sidebarSource,
+ 'loadAvailableApps',
+ 'Call Apps sidebar must use backend availability before showing installed apps',
+);
+
+assertContains(
+ sidebarSource,
+ 'attachSelectedApp',
+ 'Call Apps sidebar must create backend sessions before showing access controls',
+);
+
+assertContains(
+ sidebarSource,
+ 'Call App participant access',
+ 'Call Apps sidebar must show participant access controls after session attach',
+);
+
+assert.match(
+ sidebarCombinedSource,
+ /\.call-apps-list-item[\s\S]*grid-template-columns:\s*minmax\(0,\s*1fr\)[\s\S]*@container\s*\(min-width:\s*380px\)[\s\S]*\.call-apps-list-item[\s\S]*grid-template-columns:\s*minmax\(0,\s*1fr\)\s*auto/s,
+ 'Call Apps sidebar must remain responsive at narrow and wider sidebar widths',
+);
+
+assert.match(
+ grantButtonSource,
+ /\/api\/call-app-sessions\/\$\{encodeURIComponent\(sessionId\.value\)\}\/participant-grants[\s\S]*grant_state/,
+ 'Call App grant controls must persist participant access through the backend grant endpoint',
+);
+
+assertContains(
+ packageJson.scripts['test:e2e:call-app-whiteboard'],
+ 'tests/e2e/call-app-whiteboard-install-sidebar.spec.js',
+ 'focused Whiteboard E2E script must include install-to-sidebar browser proof',
+);
+
+assertContains(
+ packageJson.scripts['test:contract:call-apps'],
+ 'tests/contract/call-app-whiteboard-install-browser-proof-contract.mjs',
+ 'Call Apps contract suite must include the Whiteboard install browser proof contract',
+);
+
+console.log('[call-app-whiteboard-install-browser-proof-contract] PASS');
diff --git a/demo/video-chat/frontend-vue/tests/contract/call-app-whiteboard-runtime-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/call-app-whiteboard-runtime-contract.mjs
index 054050f06..61100ea97 100644
--- a/demo/video-chat/frontend-vue/tests/contract/call-app-whiteboard-runtime-contract.mjs
+++ b/demo/video-chat/frontend-vue/tests/contract/call-app-whiteboard-runtime-contract.mjs
@@ -34,6 +34,7 @@ const [
]);
const whiteboardSource = `${iframeSource}\n${stylesheetSource}\n${runtimeSource}`;
+const packageJson = JSON.parse(packageJsonSource);
assert.equal(
manifest.status,
@@ -47,6 +48,18 @@ assert.match(
'whiteboard runtime must render a fixed-format canvas workspace',
);
+assert.match(
+ whiteboardSource,
+ /id="cursorOverlay" class="cursor-overlay"[\s\S]*\.remote-cursor-label[\s\S]*function syncCursorOverlay[\s\S]*label\.textContent = displayNameLabel\(cursor\.label \|\| cursor\.display_name\)/,
+ 'whiteboard runtime must render remote cursor display names in an accessible overlay tied to presence labels',
+);
+
+assert.match(
+ whiteboardSource,
+ /function removePresenceForActor[\s\S]*message\.type === 'call_app\.presence\.leave'/,
+ 'whiteboard runtime must support remote presence leave cleanup without iframe reload',
+);
+
for (const tool of ['select', 'pen', 'highlighter', 'line', 'rect', 'ellipse', 'text', 'sticky', 'delete']) {
assert.match(
iframeSource,
@@ -202,8 +215,8 @@ assert.match(
);
assert.match(
- packageJsonSource,
- /"test:e2e:call-app-whiteboard":\s*"playwright test tests\/e2e\/call-app-whiteboard\.spec\.js"/,
+ packageJson.scripts['test:e2e:call-app-whiteboard'] || '',
+ /^playwright test(?: .*)?tests\/e2e\/call-app-whiteboard\.spec\.js(?: .*)?$/,
'package scripts must expose the Whiteboard Call App browser E2E proof',
);
diff --git a/demo/video-chat/frontend-vue/tests/contract/foreground-reconnect-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/foreground-reconnect-contract.mjs
index 14dad25f4..f03d55e9f 100644
--- a/demo/video-chat/frontend-vue/tests/contract/foreground-reconnect-contract.mjs
+++ b/demo/video-chat/frontend-vue/tests/contract/foreground-reconnect-contract.mjs
@@ -63,11 +63,11 @@ try {
'workspace realtime must not group auth_backend_error with policy blocks',
);
- const realtimeWebsocket = fs.readFileSync(path.join(repoRoot, 'demo/video-chat/backend-king-php/http/module_realtime_websocket.php'), 'utf8');
+ const realtimeWebsocketReconnect = fs.readFileSync(path.join(repoRoot, 'demo/video-chat/backend-king-php/http/module_realtime_websocket_reconnect.php'), 'utf8');
const router = fs.readFileSync(path.join(repoRoot, 'demo/video-chat/backend-king-php/http/router.php'), 'utf8');
const authSession = fs.readFileSync(path.join(repoRoot, 'demo/video-chat/backend-king-php/http/module_auth_session.php'), 'utf8');
- assert.match(realtimeWebsocket, /websocket_auth_temporarily_unavailable/, 'backend websocket liveness must not label transient auth backend errors as invalid sessions');
- assert.match(realtimeWebsocket, /Session validation is temporarily unavailable for realtime commands\./, 'backend websocket liveness must send a retryable auth backend message');
+ assert.match(realtimeWebsocketReconnect, /websocket_auth_temporarily_unavailable/, 'backend websocket liveness must not label transient auth backend errors as invalid sessions');
+ assert.match(realtimeWebsocketReconnect, /Session validation is temporarily unavailable for realtime commands\./, 'backend websocket liveness must send a retryable auth backend message');
assert.match(router, /authentication backend error transport=%s exception=%s message=%s/, 'backend router must log swallowed auth backend exceptions');
assert.match(authSession, /session probe failed exception=%s message=%s/, 'session probe must log swallowed auth backend exceptions');
diff --git a/demo/video-chat/frontend-vue/tests/contract/gossip-dedicated-neighbor-lifecycle-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/gossip-dedicated-neighbor-lifecycle-contract.mjs
index 6544ff54c..1fbcf7b1a 100644
--- a/demo/video-chat/frontend-vue/tests/contract/gossip-dedicated-neighbor-lifecycle-contract.mjs
+++ b/demo/video-chat/frontend-vue/tests/contract/gossip-dedicated-neighbor-lifecycle-contract.mjs
@@ -51,6 +51,43 @@ assert(
&& /void handleIce\(senderPeerId,\s*payload\)/.test(lifecycle),
'gossip_neighbor signaling must be consumed by the dedicated lifecycle, not by native media signaling',
)
+assert(
+ /function scheduleQueuedRenegotiate\(peer,\s*reason = 'queued_renegotiate'\)/.test(lifecycle)
+ && /queuedRenegotiateTimer/.test(lifecycle)
+ && /GOSSIP_NEIGHBOR_RENEGOTIATE_MAX_ATTEMPTS/.test(lifecycle)
+ && /gossip_neighbor_renegotiate_quarantined/.test(lifecycle)
+ && /scheduleQueuedRenegotiate\(peer,\s*'queued_renegotiate'\)/.test(lifecycle)
+ && !/void negotiatePeer\(peer,\s*'queued_renegotiate'\)/.test(lifecycle),
+ 'queued Gossip neighbor renegotiation must be deduped and bounded instead of recursively calling negotiatePeer from finally',
+)
+assert(
+ (
+ /const preSetLocalState = String\(peer\.pc\.signalingState \|\| ''\)\.trim\(\)\.toLowerCase\(\)/.test(lifecycle)
+ || /const preSetLocalState = normalizedSignalingState\(peer\.pc\)/.test(lifecycle)
+ )
+ && /preSetLocalState !== 'stable'/.test(lifecycle)
+ && /shouldDeferOfferSetLocalFailure\(error,\s*peer\.pc\)/.test(lifecycle)
+ && /gossip_neighbor_offer_deferred/.test(lifecycle)
+ && /await peer\.pc\.setLocalDescription\(offer\)/.test(lifecycle),
+ 'Gossip neighbor offer creation must defer both pre-set and setLocalDescription have-remote-offer glare',
+)
+assert(
+ /function shouldIgnoreStaleRemoteOfferAnswerFailure\(error,\s*pc\)/.test(lifecycle)
+ && /const postRemoteState = normalizedSignalingState\(peer\.pc\)/.test(lifecycle)
+ && /postRemoteState !== 'have-remote-offer'/.test(lifecycle)
+ && /gossip_neighbor_offer_stale/.test(lifecycle),
+ 'remote Gossip offers that become stale while answering must be treated idempotently instead of failing the neighbor link',
+)
+assert(
+ /addEventListener\('signalingstatechange'/.test(lifecycle)
+ && /gossip_neighbor_renegotiate_waiting_stable/.test(lifecycle)
+ && /peer\.queuedRenegotiateAttempts = 0;[\s\S]*scheduleQueuedRenegotiate\(peer,\s*'signaling_stable'\)/.test(lifecycle),
+ 'queued Gossip neighbor renegotiation must wait for stable signaling instead of burning the quarantine budget',
+)
+assert(
+ /function closePeer\(peerId,\s*reason = 'retired'\)[\s\S]*clearQueuedRenegotiate\(peer\)[\s\S]*peer\.pc\?\.close\?\.\(\)/.test(lifecycle),
+ 'closing a Gossip neighbor must clear pending queued renegotiation timers before closing the peer connection',
+)
assert(
/import \{ createGossipNeighborLifecycle \} from '\.\/gossipNeighborLifecycle'/.test(dataLane)
&& /const assignedGossipNeighborIds = new Set\(\)/.test(dataLane)
@@ -74,6 +111,10 @@ assert(
packageJson.includes('gossip-dedicated-neighbor-lifecycle-contract.mjs'),
'gossip contract suite must include the dedicated neighbor lifecycle contract',
)
+assert(
+ packageJson.includes('gossip-neighbor-renegotiate-stack-contract.mjs'),
+ 'gossip contract suite must include the production stack-overflow renegotiation proof',
+)
assert(
/- \[x\] GSP-04 Dedicated bounded neighbor lifecycle/.test(sprint),
'SPRINT.md must mark GSP-04 complete when dedicated neighbor lifecycle proof exists',
diff --git a/demo/video-chat/frontend-vue/tests/contract/gossip-media-carrier-integration-smoke-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/gossip-media-carrier-integration-smoke-contract.mjs
index 125e69342..a35b10e1d 100644
--- a/demo/video-chat/frontend-vue/tests/contract/gossip-media-carrier-integration-smoke-contract.mjs
+++ b/demo/video-chat/frontend-vue/tests/contract/gossip-media-carrier-integration-smoke-contract.mjs
@@ -134,6 +134,9 @@ function publisherHarness(overrides = {}) {
order.push('required_sfu_failure')
return false
},
+ onOptionalSfuFailure: (failureDetails) => {
+ order.push(`optional_sfu_failure:${String(failureDetails?.reason || '')}`)
+ },
...overrides,
},
}
@@ -180,9 +183,23 @@ harness = publisherHarness({
}),
})
result = await gossipPrimaryDispatch.dispatchPublisherFrame(harness.args)
-assert(result.ok === true && result.gossipPublished === true && result.sfuSent === false, 'gossip_primary must not mirror a successfully published Gossip frame into SFU')
-assert(result.sfuFallbackSkipped === true, 'gossip_primary must expose that SFU fallback was skipped after successful Gossip publication')
-assert(harness.order.join(',') === 'gossip', 'gossip_primary with an open SFU socket must still avoid SFU send when Gossip publication succeeds')
+assert(result.ok === true && result.gossipPublished === true && result.sfuSent === true, 'gossip_primary must mirror a successfully published Gossip frame into SFU when the SFU socket is open')
+assert(result.sfuSendOptional === true, 'gossip_primary SFU mirroring must stay optional')
+assert(harness.order.join(',') === 'gossip,sfu', 'gossip_primary with an open SFU socket must publish Gossip first and then mirror to SFU')
+
+harness = publisherHarness({
+ currentOpenSfuClient: () => ({
+ sendEncodedFrame: async () => {
+ harness.order.push('sfu')
+ return false
+ },
+ getLastSendFailure: () => ({ reason: 'contract_optional_sfu_failure' }),
+ }),
+})
+result = await gossipPrimaryDispatch.dispatchPublisherFrame(harness.args)
+assert(result.ok === true && result.gossipPublished === true && result.sfuSent === false, 'gossip_primary must keep Gossip live when optional SFU mirror send fails')
+assert(harness.order.join(',') === 'gossip,sfu,optional_sfu_failure:contract_optional_sfu_failure', 'gossip_primary optional SFU failure must still notify backpressure after Gossip publication')
+assert(harness.diagnostics.some((event) => event?.eventType === 'sfu_optional_send_failed_after_gossip_publish'), 'gossip_primary optional SFU failure must be diagnosed without blocking Gossip')
harness = publisherHarness({
publishLocalEncodedFrameToGossip: () => {
@@ -233,7 +250,7 @@ harness = publisherHarness({
})
result = await sfuMirrorDispatch.dispatchPublisherFrame(harness.args)
assert(result.ok === true && result.sfuSent === false && result.gossipPublished === true, 'sfu_mirror must keep Gossip mirror publication when SFU send fails after encode')
-assert(harness.order.join(',') === 'sfu,gossip', 'sfu_mirror SFU failure must still mirror to Gossip after the failed SFU attempt')
+assert(harness.order.join(',') === 'sfu,gossip,optional_sfu_failure:contract_sfu_failure', 'sfu_mirror SFU failure must still mirror to Gossip and notify backpressure after the failed SFU attempt')
assert(harness.diagnostics.some((event) => event?.eventType === 'sfu_optional_send_failed_after_gossip_publish'), 'sfu_mirror SFU failure must be diagnostic after Gossip mirror publication')
const unhealthySfu = readyAggregate({
diff --git a/demo/video-chat/frontend-vue/tests/contract/gossip-neighbor-renegotiate-stack-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/gossip-neighbor-renegotiate-stack-contract.mjs
new file mode 100644
index 000000000..95cd768c7
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/contract/gossip-neighbor-renegotiate-stack-contract.mjs
@@ -0,0 +1,571 @@
+import assert from 'node:assert/strict'
+import path from 'node:path'
+import { fileURLToPath } from 'node:url'
+import { loadViteSsrModule } from './viteSsrLoader.mjs'
+
+const __dirname = path.dirname(fileURLToPath(import.meta.url))
+const frontendRoot = path.resolve(__dirname, '../..')
+
+const previousRtcPeerConnection = globalThis.RTCPeerConnection
+const previousRtcSessionDescription = globalThis.RTCSessionDescription
+
+function delay(ms) {
+ return new Promise((resolve) => setTimeout(resolve, ms))
+}
+
+class ReentrantOfferFailurePeerConnection {
+ static instances = []
+
+ constructor() {
+ this.listeners = new Map()
+ this.signalingState = 'stable'
+ this.connectionState = 'new'
+ this.localDescription = null
+ this.remoteDescription = null
+ this.createOfferCalls = 0
+ this.closed = false
+ ReentrantOfferFailurePeerConnection.instances.push(this)
+ }
+
+ addEventListener(type, listener) {
+ this.listeners.set(type, listener)
+ }
+
+ createOffer() {
+ this.createOfferCalls += 1
+ this.listeners.get('negotiationneeded')?.({ type: 'negotiationneeded' })
+ throw new Error('sync createOffer failure')
+ }
+
+ async setLocalDescription(description) {
+ this.localDescription = description
+ }
+
+ async setRemoteDescription(description) {
+ this.remoteDescription = description
+ }
+
+ async createAnswer() {
+ return { type: 'answer', sdp: 'v=0\r\n' }
+ }
+
+ async addIceCandidate() {}
+
+ close() {
+ this.closed = true
+ this.signalingState = 'closed'
+ }
+}
+
+class StableDeferredPeerConnection {
+ static instances = []
+
+ constructor() {
+ this.listeners = new Map()
+ this.signalingState = 'stable'
+ this.connectionState = 'new'
+ this.localDescription = null
+ this.remoteDescription = null
+ this.createOfferCalls = 0
+ this.closed = false
+ StableDeferredPeerConnection.instances.push(this)
+ }
+
+ addEventListener(type, listener) {
+ this.listeners.set(type, listener)
+ }
+
+ async createOffer() {
+ this.createOfferCalls += 1
+ if (this.createOfferCalls === 1) {
+ this.signalingState = 'have-remote-offer'
+ }
+ return { type: 'offer', sdp: 'v=0\r\n' }
+ }
+
+ async setLocalDescription(description) {
+ this.localDescription = description
+ if (description?.type === 'offer') this.signalingState = 'have-local-offer'
+ }
+
+ async setRemoteDescription(description) {
+ this.remoteDescription = description
+ }
+
+ async createAnswer() {
+ return { type: 'answer', sdp: 'v=0\r\n' }
+ }
+
+ async addIceCandidate() {}
+
+ releaseStableSignaling() {
+ this.signalingState = 'stable'
+ this.listeners.get('signalingstatechange')?.({ type: 'signalingstatechange' })
+ }
+
+ close() {
+ this.closed = true
+ this.signalingState = 'closed'
+ }
+}
+
+class SetLocalGlareDeferredPeerConnection {
+ static instances = []
+
+ constructor() {
+ this.listeners = new Map()
+ this.signalingState = 'stable'
+ this.connectionState = 'new'
+ this.localDescription = null
+ this.remoteDescription = null
+ this.createOfferCalls = 0
+ this.setLocalOfferCalls = 0
+ this.closed = false
+ SetLocalGlareDeferredPeerConnection.instances.push(this)
+ }
+
+ addEventListener(type, listener) {
+ this.listeners.set(type, listener)
+ }
+
+ async createOffer() {
+ this.createOfferCalls += 1
+ return { type: 'offer', sdp: 'v=0\r\n' }
+ }
+
+ async setLocalDescription(description) {
+ if (description?.type === 'offer') {
+ this.setLocalOfferCalls += 1
+ if (this.setLocalOfferCalls === 1) {
+ this.signalingState = 'have-remote-offer'
+ throw new Error(
+ "Failed to execute 'setLocalDescription' on 'RTCPeerConnection': Failed to set local offer sdp: Called in wrong state: have-remote-offer",
+ )
+ }
+ this.signalingState = 'have-local-offer'
+ }
+ this.localDescription = description
+ }
+
+ async setRemoteDescription(description) {
+ this.remoteDescription = description
+ }
+
+ async createAnswer() {
+ return { type: 'answer', sdp: 'v=0\r\n' }
+ }
+
+ async addIceCandidate() {}
+
+ releaseStableSignaling() {
+ this.signalingState = 'stable'
+ this.listeners.get('signalingstatechange')?.({ type: 'signalingstatechange' })
+ }
+
+ close() {
+ this.closed = true
+ this.signalingState = 'closed'
+ }
+}
+
+class StableAfterRemoteOfferPeerConnection {
+ static instances = []
+
+ constructor() {
+ this.listeners = new Map()
+ this.signalingState = 'stable'
+ this.connectionState = 'new'
+ this.localDescription = null
+ this.remoteDescription = null
+ this.createAnswerCalls = 0
+ this.setLocalAnswerCalls = 0
+ this.closed = false
+ StableAfterRemoteOfferPeerConnection.instances.push(this)
+ }
+
+ addEventListener(type, listener) {
+ this.listeners.set(type, listener)
+ }
+
+ async createOffer() {
+ return { type: 'offer', sdp: 'v=0\r\n' }
+ }
+
+ async setLocalDescription(description) {
+ if (description?.type === 'answer') {
+ this.setLocalAnswerCalls += 1
+ throw new Error(
+ "Failed to execute 'setLocalDescription' on 'RTCPeerConnection': Called in wrong signalingState: stable",
+ )
+ }
+ this.localDescription = description
+ }
+
+ async setRemoteDescription(description) {
+ this.remoteDescription = description
+ this.signalingState = 'stable'
+ }
+
+ async createAnswer() {
+ this.createAnswerCalls += 1
+ return { type: 'answer', sdp: 'v=0\r\n' }
+ }
+
+ async addIceCandidate() {}
+
+ close() {
+ this.closed = true
+ this.signalingState = 'closed'
+ }
+}
+
+class RollbackStableRacePeerConnection {
+ static instances = []
+
+ constructor() {
+ this.listeners = new Map()
+ this.signalingState = 'stable'
+ this.connectionState = 'new'
+ this.localDescription = null
+ this.remoteDescription = null
+ this.createOfferCalls = 0
+ this.createAnswerCalls = 0
+ this.rollbackCalls = 0
+ this.closed = false
+ RollbackStableRacePeerConnection.instances.push(this)
+ }
+
+ addEventListener(type, listener) {
+ this.listeners.set(type, listener)
+ }
+
+ async createOffer() {
+ this.createOfferCalls += 1
+ return { type: 'offer', sdp: 'v=0\r\n' }
+ }
+
+ async setLocalDescription(description) {
+ if (description?.type === 'offer') {
+ this.localDescription = description
+ this.signalingState = 'have-local-offer'
+ return
+ }
+ if (description?.type === 'rollback') {
+ this.rollbackCalls += 1
+ this.signalingState = 'stable'
+ throw new Error(
+ "Failed to execute 'setLocalDescription' on 'RTCPeerConnection': Called in wrong signalingState: stable",
+ )
+ }
+ if (description?.type === 'answer') {
+ this.localDescription = description
+ this.signalingState = 'stable'
+ }
+ }
+
+ async setRemoteDescription(description) {
+ this.remoteDescription = description
+ if (description?.type === 'offer') this.signalingState = 'have-remote-offer'
+ if (description?.type === 'answer') this.signalingState = 'stable'
+ }
+
+ async createAnswer() {
+ this.createAnswerCalls += 1
+ return { type: 'answer', sdp: 'v=0\r\n' }
+ }
+
+ async addIceCandidate() {}
+
+ close() {
+ this.closed = true
+ this.signalingState = 'closed'
+ }
+}
+
+try {
+ globalThis.RTCSessionDescription = function RTCSessionDescription(description) {
+ return description
+ }
+ globalThis.RTCPeerConnection = ReentrantOfferFailurePeerConnection
+
+ const { createGossipNeighborLifecycle } = await loadViteSsrModule(
+ frontendRoot,
+ '/src/domain/realtime/workspace/callWorkspace/gossipNeighborLifecycle.ts',
+ )
+ const diagnostics = []
+ const lifecycle = createGossipNeighborLifecycle({
+ callbacks: {
+ activeCallId: () => 'call-prod-reentry',
+ activeRoomId: () => 'room-prod-reentry',
+ captureClientDiagnostic: (event) => diagnostics.push(event),
+ currentUserId: () => 1001,
+ getDataTransport: () => ({
+ bindPeerConnection: () => {},
+ close: () => {},
+ }),
+ sendSocketFrame: () => false,
+ },
+ })
+
+ lifecycle.applyAssignedNeighbors(
+ { topology_epoch: 42, admitted_peers: [{ peer_id: 1002 }] },
+ new Set(['1002']),
+ )
+
+ await delay(350)
+
+ const peerConnection = ReentrantOfferFailurePeerConnection.instances[0]
+ assert.ok(peerConnection, 'assigned gossip neighbor must create a peer connection')
+ assert.equal(
+ peerConnection.createOfferCalls,
+ 9,
+ 'reentrant negotiation must be capped at one initial offer plus eight queued retries',
+ )
+ assert.equal(
+ diagnostics.filter((event) => event?.eventType === 'gossip_neighbor_renegotiate_quarantined').length,
+ 1,
+ 'reentrant negotiation failures must be quarantined once after the bounded retry budget is exhausted',
+ )
+ assert.ok(
+ diagnostics.some((event) => event?.eventType === 'gossip_neighbor_offer_failed'),
+ 'synchronous offer failures must be captured as client diagnostics',
+ )
+
+ lifecycle.closePeer('1002', 'contract_cleanup')
+ const callsAtClose = peerConnection.createOfferCalls
+ await delay(75)
+ assert.equal(
+ peerConnection.createOfferCalls,
+ callsAtClose,
+ 'closing a gossip neighbor must cancel any queued renegotiation timer',
+ )
+
+ globalThis.RTCPeerConnection = StableDeferredPeerConnection
+ const stableDiagnostics = []
+ const sentFrames = []
+ const stableLifecycle = createGossipNeighborLifecycle({
+ callbacks: {
+ activeCallId: () => 'call-prod-stable-wait',
+ activeRoomId: () => 'room-prod-stable-wait',
+ captureClientDiagnostic: (event) => stableDiagnostics.push(event),
+ currentUserId: () => 2001,
+ getDataTransport: () => ({
+ bindPeerConnection: () => {},
+ close: () => {},
+ }),
+ sendSocketFrame: (frame) => {
+ sentFrames.push(frame)
+ return true
+ },
+ },
+ })
+
+ stableLifecycle.applyAssignedNeighbors(
+ { topology_epoch: 43, admitted_peers: [{ peer_id: 2002 }] },
+ new Set(['2002']),
+ )
+
+ await delay(125)
+ const stablePeerConnection = StableDeferredPeerConnection.instances[0]
+ assert.equal(
+ stablePeerConnection.createOfferCalls,
+ 1,
+ 'non-stable signaling must not spin queued gossip renegotiation attempts',
+ )
+ assert.equal(
+ stableDiagnostics.filter((event) => event?.eventType === 'gossip_neighbor_renegotiate_quarantined').length,
+ 0,
+ 'non-stable signaling deferral must not burn the quarantine budget',
+ )
+ assert.ok(
+ stableDiagnostics.some((event) => event?.eventType === 'gossip_neighbor_renegotiate_waiting_stable'),
+ 'non-stable signaling deferral must emit a stable-wait diagnostic',
+ )
+ stablePeerConnection.releaseStableSignaling()
+ await delay(75)
+ assert.equal(
+ stablePeerConnection.createOfferCalls,
+ 2,
+ 'stable signaling transition must release the queued gossip renegotiation exactly once',
+ )
+ assert.equal(sentFrames.length, 1, 'stable signaling release must send the deferred gossip offer')
+
+ globalThis.RTCPeerConnection = SetLocalGlareDeferredPeerConnection
+ const glareDiagnostics = []
+ const glareSentFrames = []
+ const glareLifecycle = createGossipNeighborLifecycle({
+ callbacks: {
+ activeCallId: () => 'call-prod-set-local-glare',
+ activeRoomId: () => 'room-prod-set-local-glare',
+ captureClientDiagnostic: (event) => glareDiagnostics.push(event),
+ currentUserId: () => 3001,
+ getDataTransport: () => ({
+ bindPeerConnection: () => {},
+ close: () => {},
+ }),
+ sendSocketFrame: (frame) => {
+ glareSentFrames.push(frame)
+ return true
+ },
+ },
+ })
+
+ glareLifecycle.applyAssignedNeighbors(
+ { topology_epoch: 44, admitted_peers: [{ peer_id: 3002 }] },
+ new Set(['3002']),
+ )
+
+ await delay(75)
+ const glarePeerConnection = SetLocalGlareDeferredPeerConnection.instances[0]
+ assert.equal(
+ glarePeerConnection.createOfferCalls,
+ 1,
+ 'setLocalDescription wrong-state glare must not spin before signaling returns to stable',
+ )
+ assert.equal(
+ glareDiagnostics.filter((event) => event?.eventType === 'gossip_neighbor_offer_failed').length,
+ 0,
+ 'setLocalDescription wrong-state glare must be deferred rather than reported as a dead offer failure',
+ )
+ assert.ok(
+ glareDiagnostics.some((event) => event?.eventType === 'gossip_neighbor_offer_deferred'),
+ 'setLocalDescription wrong-state glare must emit a deferred offer diagnostic',
+ )
+ glarePeerConnection.releaseStableSignaling()
+ await delay(75)
+ assert.equal(
+ glarePeerConnection.setLocalOfferCalls,
+ 2,
+ 'stable signaling after setLocalDescription glare must retry the local offer once',
+ )
+ assert.equal(glareSentFrames.length, 1, 'setLocalDescription glare recovery must send the deferred gossip offer')
+
+ globalThis.RTCPeerConnection = StableAfterRemoteOfferPeerConnection
+ const staleRemoteDiagnostics = []
+ const staleRemoteSentFrames = []
+ const staleRemoteLifecycle = createGossipNeighborLifecycle({
+ callbacks: {
+ activeCallId: () => 'call-prod-stale-remote-offer',
+ activeRoomId: () => 'room-prod-stale-remote-offer',
+ captureClientDiagnostic: (event) => staleRemoteDiagnostics.push(event),
+ currentUserId: () => 4001,
+ getDataTransport: () => ({
+ bindPeerConnection: () => {},
+ close: () => {},
+ }),
+ sendSocketFrame: (frame) => {
+ staleRemoteSentFrames.push(frame)
+ return true
+ },
+ },
+ })
+
+ staleRemoteLifecycle.applyAssignedNeighbors(
+ { topology_epoch: 45, admitted_peers: [{ peer_id: 4002 }] },
+ new Set(),
+ )
+ staleRemoteLifecycle.handleGossipNeighborSignal('call/offer', '4002', {
+ kind: 'gossip_neighbor_offer',
+ runtime_path: 'gossip_primary_neighbor',
+ sdp: { type: 'offer', sdp: 'v=0\r\n' },
+ })
+
+ await delay(75)
+ const staleRemotePeerConnection = StableAfterRemoteOfferPeerConnection.instances[0]
+ assert.ok(staleRemotePeerConnection, 'remote gossip offer must create a non-initiator peer connection')
+ assert.equal(
+ staleRemotePeerConnection.createAnswerCalls,
+ 0,
+ 'stale remote offer that leaves signaling stable must not create an invalid answer',
+ )
+ assert.equal(
+ staleRemotePeerConnection.setLocalAnswerCalls,
+ 0,
+ 'stale remote offer that leaves signaling stable must not call setLocalDescription(answer)',
+ )
+ assert.equal(
+ staleRemoteSentFrames.length,
+ 0,
+ 'stale remote offer must not emit a gossip answer frame',
+ )
+ assert.equal(
+ staleRemoteDiagnostics.filter((event) => event?.eventType === 'gossip_neighbor_offer_handle_failed').length,
+ 0,
+ 'stable-state stale remote offers must not be reported as offer-handle failures',
+ )
+ assert.ok(
+ staleRemoteDiagnostics.some((event) => event?.eventType === 'gossip_neighbor_offer_stale'),
+ 'stable-state stale remote offers must emit an idempotent stale-offer diagnostic',
+ )
+
+ globalThis.RTCPeerConnection = RollbackStableRacePeerConnection
+ const rollbackRaceDiagnostics = []
+ const rollbackRaceSentFrames = []
+ const rollbackRaceLifecycle = createGossipNeighborLifecycle({
+ callbacks: {
+ activeCallId: () => 'call-prod-rollback-stable-race',
+ activeRoomId: () => 'room-prod-rollback-stable-race',
+ captureClientDiagnostic: (event) => rollbackRaceDiagnostics.push(event),
+ currentUserId: () => 5002,
+ getDataTransport: () => ({
+ bindPeerConnection: () => {},
+ close: () => {},
+ }),
+ sendSocketFrame: (frame) => {
+ rollbackRaceSentFrames.push(frame)
+ return true
+ },
+ },
+ })
+
+ rollbackRaceLifecycle.applyAssignedNeighbors(
+ { topology_epoch: 46, admitted_peers: [{ peer_id: 5001 }] },
+ new Set(['5001']),
+ )
+ await delay(75)
+ rollbackRaceLifecycle.handleGossipNeighborSignal('call/offer', '5001', {
+ kind: 'gossip_neighbor_offer',
+ runtime_path: 'gossip_primary_neighbor',
+ sdp: { type: 'offer', sdp: 'v=0\r\n' },
+ })
+
+ await delay(75)
+ const rollbackRacePeerConnection = RollbackStableRacePeerConnection.instances[0]
+ assert.ok(rollbackRacePeerConnection, 'rollback-stable race must reuse the assigned initiator peer connection')
+ assert.equal(
+ rollbackRacePeerConnection.rollbackCalls,
+ 1,
+ 'remote-wins glare must attempt rollback before accepting the remote offer',
+ )
+ assert.equal(
+ rollbackRacePeerConnection.createAnswerCalls,
+ 1,
+ 'stable rollback race must continue into a valid answer instead of abandoning the remote offer',
+ )
+ assert.ok(
+ rollbackRaceSentFrames.some((frame) => frame?.payload?.kind === 'gossip_neighbor_answer'),
+ 'stable rollback race must send a gossip neighbor answer',
+ )
+ assert.equal(
+ rollbackRaceDiagnostics.filter((event) => event?.eventType === 'gossip_neighbor_offer_handle_failed').length,
+ 0,
+ 'stable rollback race must not be reported as an offer-handle failure',
+ )
+ assert.ok(
+ rollbackRaceDiagnostics.some((event) => event?.eventType === 'gossip_neighbor_offer_stale'),
+ 'stable rollback race must emit an idempotent stale-offer diagnostic',
+ )
+
+ console.log('[gossip-neighbor-renegotiate-stack-contract] PASS')
+} finally {
+ if (previousRtcPeerConnection === undefined) {
+ delete globalThis.RTCPeerConnection
+ } else {
+ globalThis.RTCPeerConnection = previousRtcPeerConnection
+ }
+ if (previousRtcSessionDescription === undefined) {
+ delete globalThis.RTCSessionDescription
+ } else {
+ globalThis.RTCSessionDescription = previousRtcSessionDescription
+ }
+}
diff --git a/demo/video-chat/frontend-vue/tests/contract/gossip-publisher-pipeline-decoupling-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/gossip-publisher-pipeline-decoupling-contract.mjs
index 8f2f98493..cd7e267b4 100644
--- a/demo/video-chat/frontend-vue/tests/contract/gossip-publisher-pipeline-decoupling-contract.mjs
+++ b/demo/video-chat/frontend-vue/tests/contract/gossip-publisher-pipeline-decoupling-contract.mjs
@@ -56,8 +56,9 @@ assert(
'optional SFU failure must return success based on Gossip publication',
)
assert(
- /VIDEOCHAT_MEDIA_CARRIER_CONFIG\.gossipPrimary && gossipPublished[\s\S]*sfuFallbackSkipped:\s*true/.test(helper),
- 'gossip_primary must skip optional SFU fallback after successful Gossip publication',
+ /gossipFirst && gossipPublished[\s\S]*sfuMirrorSkipped:\s*true/.test(helper)
+ && /const sent = await sendClient\.sendEncodedFrame\(frame\)/.test(helper),
+ 'gossip_primary must publish Gossip first and mirror to SFU when an open SFU client is available',
)
assert(
/publisherRequiresSfuBeforeEncode\(\) && !currentOpenSfuClient\(\)/.test(publisherPipeline)
diff --git a/demo/video-chat/frontend-vue/tests/contract/gossip-sfu-dual-carrier-continuity-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/gossip-sfu-dual-carrier-continuity-contract.mjs
new file mode 100644
index 000000000..3f1bf8e8c
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/contract/gossip-sfu-dual-carrier-continuity-contract.mjs
@@ -0,0 +1,60 @@
+import assert from 'node:assert/strict';
+import fs from 'node:fs';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+const __dirname = path.dirname(fileURLToPath(import.meta.url));
+const frontendRoot = path.resolve(__dirname, '../..');
+
+function read(relativePath) {
+ return fs.readFileSync(path.join(frontendRoot, relativePath), 'utf8');
+}
+
+const remoteJitterBuffer = read('src/domain/realtime/sfu/remoteJitterBuffer.ts');
+const frameDecode = read('src/domain/realtime/sfu/frameDecode.ts');
+const mediaSecurityRuntime = read('src/domain/realtime/workspace/callWorkspace/mediaSecurityRuntime.ts');
+const mediaSecurityErrors = read('src/domain/realtime/workspace/callWorkspace/mediaSecurityErrors.ts');
+
+assert.match(
+ remoteJitterBuffer,
+ /function normalizeRemoteFrameContinuityCarrier\(frame\)[\s\S]*transportPath === 'gossip_rtc_datachannel' \? 'gossip' : ''/,
+ 'remote receiver jitter keys must scope Gossip continuity without renaming the existing SFU sequence domain',
+);
+
+assert.match(
+ remoteJitterBuffer,
+ /const baseKey = videoLayer !== '' \? `\$\{trackId\}:\$\{videoLayer\}` : trackId;[\s\S]*return carrier !== '' \? `\$\{baseKey\}:\$\{carrier\}` : baseKey;/,
+ 'remote jitter track key must suffix Gossip frames so they cannot be made stale by SFU mirror sequence numbers',
+);
+
+assert.match(
+ frameDecode,
+ /function shouldDropRemoteSfuFrameForContinuity\(publisherId, peer, frame\)[\s\S]*const trackKey = remoteJitterTrackKey\(frame\);/,
+ 'remote continuity state must use the carrier-scoped jitter key',
+);
+
+assert.match(
+ frameDecode,
+ /function renderDecodedSfuFrame\(peer, decoded, frame = null\)[\s\S]*const trackKey = sfuFrameTrackStateKey\(frame\);/,
+ 'render caches must remain shared by media track and not split canvases by transport carrier',
+);
+
+assert.match(
+ mediaSecurityRuntime,
+ /function shouldTreatNativeFrameErrorAsDuplicateDrop\(direction, error, senderUserId = 0\)[\s\S]*message === 'replay_detected'/,
+ 'native protected audio replay must be treated as duplicate carrier delivery, not a hard media-security failure',
+);
+
+assert.match(
+ mediaSecurityRuntime,
+ /const shouldRecoverReceiver = shouldRecoverMediaSecurityFromFrameError\(error\) \|\| transientFrameDrop;/,
+ 'native replay duplicate drops must not trigger media-security resync loops',
+);
+
+assert.match(
+ mediaSecurityErrors,
+ /message\.includes\('replay_detected'\)/,
+ 'media-security error normalization must expose replay_detected explicitly',
+);
+
+console.log('[gossip-sfu-dual-carrier-continuity-contract] PASS');
diff --git a/demo/video-chat/frontend-vue/tests/contract/iam-call-access-e2e-foundation-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/iam-call-access-e2e-foundation-contract.mjs
new file mode 100644
index 000000000..903da05c0
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/contract/iam-call-access-e2e-foundation-contract.mjs
@@ -0,0 +1,168 @@
+import assert from 'node:assert/strict';
+import fs from 'node:fs';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = path.dirname(__filename);
+const frontendRoot = path.resolve(__dirname, '../..');
+const videoChatRoot = path.resolve(frontendRoot, '..');
+const repoRoot = path.resolve(videoChatRoot, '../..');
+
+function readText(relativePath) {
+ return fs.readFileSync(path.join(repoRoot, relativePath), 'utf8');
+}
+
+function readJson(relativePath) {
+ return JSON.parse(readText(relativePath));
+}
+
+const packageJson = readJson('demo/video-chat/frontend-vue/package.json');
+const matrix = readJson('demo/video-chat/contracts/v1/ui-parity-acceptance.matrix.json');
+const e2eSpec = readText('demo/video-chat/frontend-vue/tests/e2e/call-access-join.spec.js');
+const seedMatrixSpec = readText('demo/video-chat/frontend-vue/tests/e2e/call-access-seed-matrix.spec.js');
+const seedMatrixHelper = readText('demo/video-chat/frontend-vue/tests/e2e/helpers/callAccessSeedMatrix.js');
+const backendContract = readText('demo/video-chat/backend-king-php/tests/call-access-membership-removal-contract.php');
+const smoke = readText('demo/video-chat/scripts/smoke.sh');
+const auth = readText('demo/video-chat/backend-king-php/support/auth.php');
+const authCache = readText('demo/video-chat/backend-king-php/support/auth_session_cache.php');
+const tenantContext = readText('demo/video-chat/backend-king-php/support/tenant_context.php');
+const callAccessPublic = readText('demo/video-chat/backend-king-php/domain/calls/call_access_public.php');
+
+const scripts = packageJson.scripts || {};
+const callAccessScript = String(scripts['test:e2e:call-access'] || '');
+const matrixScript = String(scripts['test:e2e:matrix'] || '');
+
+assert.match(
+ callAccessScript,
+ /playwright test tests\/e2e\/call-access-join\.spec\.js/,
+ 'package script must keep the live backend Call Access Playwright spec',
+);
+assert.match(
+ callAccessScript,
+ /tests\/e2e\/call-access-seed-matrix\.spec\.js/,
+ 'package script must include additive deterministic Call Access seed-matrix coverage',
+);
+assert.match(
+ callAccessScript,
+ /--workers=1/,
+ 'call-access E2E script must run serially to avoid live backend access-link contention',
+);
+assert.match(
+ String(scripts['test:contract:iam-call-access'] || ''),
+ /iam-call-access-e2e-foundation-contract\.mjs/,
+ 'package script must expose the IAM Call Access contract gate',
+);
+assert.match(
+ String(scripts['test:contract:iam-call-access'] || ''),
+ /\.\.\/backend-king-php\/tests\/call-access-membership-removal-contract\.sh/,
+ 'IAM Call Access contract gate must include the backend membership-removal proof',
+);
+assert.doesNotMatch(
+ matrixScript,
+ /tests\/e2e\/call-access-join\.spec\.js/,
+ 'broader compose E2E matrix must not execute the live Call Access join spec with host-style backend origin',
+);
+
+const uiParityPaths = new Set(matrix.commands?.['frontend:e2e:ui-parity']?.paths || []);
+const matrixPaths = new Set(matrix.commands?.['frontend:e2e:matrix']?.paths || []);
+const callAccessPaths = new Set(matrix.commands?.['frontend:e2e:call-access']?.paths || []);
+const requiredSpecs = new Set(matrix.release_gate?.required_ui_parity_specs || []);
+assert.ok(
+ uiParityPaths.has('frontend-vue/tests/e2e/call-access-join.spec.js'),
+ 'UI parity matrix must list the Call Access join spec',
+);
+assert.ok(
+ !matrixPaths.has('frontend-vue/tests/e2e/call-access-join.spec.js'),
+ 'chat/layout compose matrix must not list the live Call Access join spec',
+);
+assert.ok(
+ callAccessPaths.has('frontend-vue/tests/e2e/call-access-join.spec.js'),
+ 'focused Call Access command must list the live backend Call Access join spec',
+);
+assert.ok(
+ callAccessPaths.has('frontend-vue/tests/e2e/call-access-seed-matrix.spec.js'),
+ 'focused Call Access command must list the deterministic seed-matrix spec',
+);
+assert.ok(
+ requiredSpecs.has('frontend-vue/tests/e2e/call-access-join.spec.js'),
+ 'release gate must pin the Call Access join spec as required coverage',
+);
+
+assert.match(e2eSpec, /\/api\/call-access\/\$\{accessId\}\/join/, 'E2E spec must observe the public join resolution request');
+assert.match(e2eSpec, /\/api\/call-access\/\$\{accessId\}\/session/, 'E2E spec must observe the public call-access session request');
+assert.match(e2eSpec, /nativeAudioTransferHarness\.js/, 'E2E spec must keep using the live backend harness');
+assert.match(e2eSpec, /createInvitedCallViaApi[\s\S]*createPersonalAccessJoinPath/s, 'E2E spec must keep live API call and access-link creation');
+assert.match(e2eSpec, /tenant_admin[\s\S]*false/, 'E2E spec must assert the session does not gain tenant-admin rights');
+assert.match(
+ seedMatrixSpec,
+ /temporary_personalized_guest[\s\S]*temporary_anonymous_guest[\s\S]*tenant_admin[\s\S]*false/s,
+ 'seed-matrix spec must prove temporary guests do not receive tenant/system admin rights',
+);
+assert.match(
+ seedMatrixHelper,
+ /VIDEOCHAT_CALL_ACCESS_SEED_MATRIX_JSON/,
+ 'seed-matrix helper must support compose smoke injection when contracts/v1 is outside the frontend container mount',
+);
+assert.match(
+ backendContract,
+ /videochat_tenant_user_is_member\(\$pdo, \$invitedUserId, \$tenantId\)[\s\S]*membership removal/s,
+ 'backend contract must prove losing tenant membership remains effective',
+);
+assert.match(
+ backendContract,
+ /videochat_resolve_call_access_public\(\$pdo, \$accessId\)[\s\S]*remain resolvable/s,
+ 'backend contract must prove explicit call-scoped links remain resolvable',
+);
+assert.match(
+ backendContract,
+ /tenant_admin[\s\S]*false/,
+ 'backend contract must prove call-scoped fallback does not restore tenant admin rights',
+);
+assert.match(
+ smoke,
+ /call-access-membership-removal-contract\.sh/,
+ 'smoke gate must include the backend call-access membership-removal contract',
+);
+assert.match(
+ smoke,
+ /VITE_VIDEOCHAT_BACKEND_ORIGIN='http:\/\/videochat-backend-v1:18080'[\s\S]*npm run test:e2e:call-access/s,
+ 'compose smoke must run the focused Call Access E2E command against the backend service DNS origin',
+);
+assert.match(
+ smoke,
+ /npm run test:e2e:call-access -- --reporter=list --workers=1/,
+ 'compose smoke must serialize the live Call Access E2E command to avoid fresh-compose SQLite write contention',
+);
+assert.match(
+ smoke,
+ /VIDEOCHAT_CALL_ACCESS_SEED_MATRIX_JSON=\$\{call_access_seed_matrix_json\}/,
+ 'compose smoke must inject the deterministic Call Access seed matrix into the frontend container',
+);
+assert.match(
+ smoke,
+ /VITE_VIDEOCHAT_WS_ORIGIN='http:\/\/videochat-backend-ws-v1:18080'[\s\S]*VITE_VIDEOCHAT_ALLOW_INSECURE_WS='1'[\s\S]*npm run test:e2e:call-access/s,
+ 'compose smoke must provide service-DNS websocket origin for the live Call Access lobby path',
+);
+assert.match(
+ smoke,
+ /VITE_VIDEOCHAT_BACKEND_ORIGIN='http:\/\/127\.0\.0\.1:\$\{compose_backend_port\}'[\s\S]*npm run test:e2e:matrix/s,
+ 'compose smoke must keep the broader chat/layout matrix on the host-style backend origin',
+);
+assert.match(
+ auth + authCache,
+ /videochat_tenant_context_for_call_access_session/,
+ 'auth paths must fall back to call-scoped tenant context for access sessions',
+);
+assert.match(
+ tenantContext,
+ /membership_id,[\s\S]*0 AS membership_id,[\s\S]*'member' AS membership_role/s,
+ 'call-scoped tenant fallback must be least-privilege and must not invent membership ids',
+);
+assert.match(
+ callAccessPublic,
+ /videochat_fetch_active_user_for_call_access\([\s\S]*false[\s\S]*\);/,
+ 'public call-access resolution must allow explicit invitation lookup without active tenant membership',
+);
+
+process.stdout.write('[iam-call-access-e2e-foundation-contract] PASS\n');
diff --git a/demo/video-chat/frontend-vue/tests/contract/media-reconnect-release-smoke-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/media-reconnect-release-smoke-contract.mjs
new file mode 100644
index 000000000..7a9d2ae54
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/contract/media-reconnect-release-smoke-contract.mjs
@@ -0,0 +1,50 @@
+import assert from 'node:assert/strict';
+import fs from 'node:fs';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = path.dirname(__filename);
+const frontendRoot = path.resolve(__dirname, '../..');
+const repoRoot = path.resolve(frontendRoot, '../../..');
+
+function read(relativePath) {
+ return fs.readFileSync(path.join(repoRoot, relativePath), 'utf8');
+}
+
+const packageJson = JSON.parse(read('demo/video-chat/frontend-vue/package.json'));
+const releaseSmokeCommand = String(packageJson.scripts?.['test:contract:media-reconnect-release-smoke'] || '');
+const smoke = read('demo/video-chat/scripts/smoke.sh');
+
+assert.match(
+ releaseSmokeCommand,
+ /node tests\/contract\/media-reconnect-screenshare-stability-contract\.mjs/,
+ 'release smoke must include the static reconnect/screenshare lifecycle contract',
+);
+assert.match(
+ releaseSmokeCommand,
+ /node tests\/contract\/media-reconnect-screenshare-browser-smoke-contract\.mjs/,
+ 'release smoke must include the fake-browser stale capture smoke',
+);
+assert.match(
+ releaseSmokeCommand,
+ /node tests\/contract\/media-reconnect-release-smoke-contract\.mjs/,
+ 'release smoke must pin this deploy-facing hook contract',
+);
+assert.doesNotMatch(
+ releaseSmokeCommand,
+ /\b(playwright|vite|dev-server|getUserMedia|getDisplayMedia)\b/,
+ 'release smoke must remain a deterministic node contract and not require real devices or a browser',
+);
+assert.match(
+ smoke,
+ /npm run test:contract:media-reconnect-release-smoke/,
+ 'main smoke gate must run the media reconnect/screenshare release smoke before deploy',
+);
+assert.match(
+ smoke,
+ /frontend contract: media reconnect screenshare release smoke/,
+ 'smoke output must name the media reconnect/screenshare release gate clearly',
+);
+
+process.stdout.write('[media-reconnect-release-smoke-contract] PASS\n');
diff --git a/demo/video-chat/frontend-vue/tests/contract/media-reconnect-screenshare-browser-smoke-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/media-reconnect-screenshare-browser-smoke-contract.mjs
new file mode 100644
index 000000000..e8be53f44
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/contract/media-reconnect-screenshare-browser-smoke-contract.mjs
@@ -0,0 +1,117 @@
+import assert from 'node:assert/strict';
+import fs from 'node:fs';
+import path from 'node:path';
+import vm from 'node:vm';
+import { fileURLToPath } from 'node:url';
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = path.dirname(__filename);
+const root = path.resolve(__dirname, '../..');
+
+function read(relativePath) {
+ return fs.readFileSync(path.join(root, relativePath), 'utf8');
+}
+
+class FakeMediaTrack {
+ constructor(id, kind) {
+ this.id = id;
+ this.kind = kind;
+ this.readyState = 'live';
+ this.stopCount = 0;
+ }
+
+ stop() {
+ this.stopCount += 1;
+ this.readyState = 'ended';
+ }
+}
+
+class FakeMediaStream {
+ constructor(tracks = []) {
+ this.tracks = tracks;
+ }
+
+ getTracks() {
+ return [...this.tracks];
+ }
+
+ getAudioTracks() {
+ return this.tracks.filter((track) => track.kind === 'audio');
+ }
+
+ getVideoTracks() {
+ return this.tracks.filter((track) => track.kind === 'video');
+ }
+}
+
+globalThis.MediaStream = FakeMediaStream;
+
+const lifecycleSource = `${read('src/domain/realtime/local/localStreamLifecycle.ts')
+ .replaceAll('export function ', 'function ')}
+globalThis.__localStreamLifecycle = { stopRetiredLocalStreams };`;
+const sandbox = {
+ globalThis: {},
+ MediaStream: FakeMediaStream,
+};
+vm.runInNewContext(lifecycleSource, sandbox, { filename: 'localStreamLifecycle.ts' });
+const { stopRetiredLocalStreams } = sandbox.globalThis.__localStreamLifecycle;
+
+const cameraTrack = new FakeMediaTrack('active-camera', 'video');
+const audioTrack = new FakeMediaTrack('active-microphone', 'audio');
+const screenTrack = new FakeMediaTrack('active-screen-share', 'video');
+const retiredCameraTrack = new FakeMediaTrack('retired-camera', 'video');
+const retiredAudioTrack = new FakeMediaTrack('retired-microphone', 'audio');
+const retiredScreenTrack = new FakeMediaTrack('retired-screen-share', 'video');
+
+const activeCameraAudioStream = new FakeMediaStream([cameraTrack, audioTrack]);
+const activeScreenShareStream = new FakeMediaStream([screenTrack]);
+const staleReconnectStream = new FakeMediaStream([
+ cameraTrack,
+ audioTrack,
+ screenTrack,
+ retiredCameraTrack,
+ retiredAudioTrack,
+ retiredScreenTrack,
+]);
+
+const diagnostics = [];
+stopRetiredLocalStreams(
+ [staleReconnectStream],
+ [activeCameraAudioStream, activeScreenShareStream],
+ {
+ reason: 'stale_local_media_capture_discarded',
+ mediaRuntimePath: 'wlvc_sfu',
+ captureDiagnostic: (entry) => diagnostics.push(entry),
+ },
+);
+
+for (const track of [cameraTrack, audioTrack, screenTrack]) {
+ assert.equal(track.readyState, 'live', `${track.id} must stay live after stale reconnect cleanup`);
+ assert.equal(track.stopCount, 0, `${track.id} must not be stopped by stale reconnect cleanup`);
+}
+
+for (const track of [retiredCameraTrack, retiredAudioTrack, retiredScreenTrack]) {
+ assert.equal(track.readyState, 'ended', `${track.id} must be stopped during stale reconnect cleanup`);
+ assert.equal(track.stopCount, 1, `${track.id} must be stopped exactly once`);
+}
+
+const mediaOrchestration = read('src/domain/realtime/local/mediaOrchestration.ts');
+assert.match(
+ mediaOrchestration,
+ /eventType: 'stale_local_media_capture_discarded'[\s\S]*active_screen_share_track_id: activeScreenShareTrackId/,
+ 'stale reconnect cleanup diagnostics must identify the screen-share state separately from cleanup',
+);
+
+const screenSharePublisher = read('src/domain/realtime/local/screenSharePublisher.js');
+assert.match(
+ screenSharePublisher,
+ /const stoppedAfterReconnectAttempts = reconnectAttempts;[\s\S]*eventType: 'local_screen_share_participant_stopped'[\s\S]*cleanup_scope: 'screen_share_capture_only'[\s\S]*reconnect_attempts: stoppedAfterReconnectAttempts/s,
+ 'screen-share stop diagnostics must keep reconnect attempts and screen-share-only cleanup scope',
+);
+assert.match(
+ screenSharePublisher,
+ /eventType: 'local_screen_share_sfu_reconnect_exhausted'[\s\S]*cleanup_scope: 'screen_share_capture_only'/s,
+ 'screen-share reconnect exhaustion diagnostics must not look like camera/audio cleanup',
+);
+
+process.stdout.write('[media-reconnect-screenshare-browser-smoke-contract] PASS\n');
diff --git a/demo/video-chat/frontend-vue/tests/contract/media-reconnect-screenshare-stability-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/media-reconnect-screenshare-stability-contract.mjs
new file mode 100644
index 000000000..f8b667d60
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/contract/media-reconnect-screenshare-stability-contract.mjs
@@ -0,0 +1,54 @@
+import assert from 'node:assert/strict';
+import fs from 'node:fs';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = path.dirname(__filename);
+const root = path.resolve(__dirname, '../..');
+
+function read(relativePath) {
+ return fs.readFileSync(path.join(root, relativePath), 'utf8');
+}
+
+const mediaOrchestration = read('src/domain/realtime/local/mediaOrchestration.ts');
+const localStreamLifecycle = read('src/domain/realtime/local/localStreamLifecycle.ts');
+const screenSharePublisher = read('src/domain/realtime/local/screenSharePublisher.js');
+
+assert.match(
+ localStreamLifecycle,
+ /export function stopRetiredLocalStreams\(retiredStreams, preservedStreams = \[\], options = \{\}\)/,
+ 'retired stream cleanup must accept explicit cleanup options',
+);
+assert.match(
+ localStreamLifecycle,
+ /protectedTrackIds[\s\S]*local_media_cleanup_preserved_active_track[\s\S]*track\.stop\(\)/,
+ 'retired stream cleanup must preserve protected active tracks before stopping stale tracks',
+);
+assert.match(
+ mediaOrchestration,
+ /function activeLocalMediaStreamsForCleanup\(\)[\s\S]*refs\.localStreamRef\.value[\s\S]*refs\.localRawStreamRef\.value[\s\S]*refs\.localFilteredStreamRef\.value[\s\S]*screenShareStream/s,
+ 'stale local capture cleanup must preserve current camera, microphone, and screen-share streams',
+);
+assert.match(
+ mediaOrchestration,
+ /function discardStaleLocalMediaCapture[\s\S]*stopRetiredLocalStreams\(streams, activeLocalMediaStreamsForCleanup\(\), \{[\s\S]*reason: 'stale_local_media_capture_discarded'[\s\S]*stale_local_media_capture_discarded/s,
+ 'stale local capture cleanup must use active-stream preservation and emit a reconnect-safe diagnostic',
+);
+assert.match(
+ mediaOrchestration,
+ /message: 'Stale local media capture was discarded without stopping active camera, microphone, or screen-share tracks\.'/,
+ 'stale capture diagnostic must distinguish reconnect cleanup from active media shutdown',
+);
+assert.match(
+ screenSharePublisher,
+ /local_screen_share_sfu_reconnect_exhausted[\s\S]*cleanup_scope: 'screen_share_capture_only'/,
+ 'screen-share reconnect exhaustion must report screen-share-only cleanup scope',
+);
+assert.match(
+ screenSharePublisher,
+ /const stoppedAfterReconnectAttempts = reconnectAttempts;[\s\S]*cleanup_scope: 'screen_share_capture_only'[\s\S]*reconnect_attempts: stoppedAfterReconnectAttempts/s,
+ 'screen-share stop diagnostics must preserve reconnect attempt context before cleanup resets',
+);
+
+process.stdout.write('[media-reconnect-screenshare-stability-contract] PASS\n');
diff --git a/demo/video-chat/frontend-vue/tests/contract/media-security-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/media-security-contract.mjs
index 953b52d78..070721fea 100644
--- a/demo/video-chat/frontend-vue/tests/contract/media-security-contract.mjs
+++ b/demo/video-chat/frontend-vue/tests/contract/media-security-contract.mjs
@@ -413,6 +413,7 @@ try {
);
const mediaSecurityRuntimeSource = read('../../src/domain/realtime/workspace/callWorkspace/mediaSecurityRuntime.ts');
+ const mediaSecurityErrorsSource = read('../../src/domain/realtime/workspace/callWorkspace/mediaSecurityErrors.ts');
const runtimeConfigSource = read('../../src/domain/realtime/workspace/callWorkspace/runtimeConfig.ts');
const orchestrationSource = read('../../src/domain/realtime/workspace/callWorkspace/orchestration.ts');
const publisherPipelineSource = read('../../src/domain/realtime/local/publisherPipeline.ts');
@@ -446,19 +447,34 @@ try {
assert.match(mediaSecurityRuntimeSource, /function mediaSecurityHandshakeRetryTimeoutMsForAttempt\(retryAttempt\)/, 'handshake retry watchdog must derive timeout from retry attempt');
assert.match(mediaSecurityRuntimeSource, /state\.mediaSecurityHandshakeRetryCountByUserId\.set\(normalizedTargetId, retryAttempt \+ 1\);/, 'handshake retry watchdog must advance retry attempt after each timeout');
assert.match(mediaSecurityRuntimeSource, /state\.mediaSecurityHandshakeRetryCountByUserId\.delete\(normalizedSenderUserId\);/, 'handshake retry watchdog must reset retry attempts when sender-key is accepted');
- assert.match(mediaSecurityRuntimeSource, /function mediaSecurityErrorCode\(error\)[\s\S]*message\.includes\('participant_set_mismatch'\)[\s\S]*return 'participant_set_mismatch';/m, 'media-security runtime must normalize verbose participant-set mismatch errors into a stable recovery code');
+ assert.match(mediaSecurityRuntimeSource, /from '\.\/mediaSecurityErrors'/, 'media-security runtime must keep error-code normalization extracted out of the oversized runtime module');
+ assert.match(mediaSecurityErrorsSource, /export function mediaSecurityErrorCode\(error\)[\s\S]*message\.includes\('participant_set_mismatch'\)[\s\S]*return 'participant_set_mismatch';/m, 'media-security errors helper must normalize verbose participant-set mismatch errors into a stable recovery code');
assert.match(securitySource, /isParticipantSetMismatchError\(error\)[\s\S]*\.includes\('participant_set_mismatch'\)/m, 'media-security core must detect wrapped participant-set mismatch messages emitted by browser runtimes');
- assert.match(mediaSecurityRuntimeSource, /message\.includes\('participant_set_mismatch'\)/, 'media-security recovery must treat participant-set churn as a reconnectable key path');
+ assert.match(mediaSecurityErrorsSource, /message\.includes\('participant_set_mismatch'\)/, 'media-security recovery must treat participant-set churn as a reconnectable key path');
assert.match(mediaSecurityRuntimeSource, /clearMediaSecuritySignalCaches\(\);[\s\S]*requestRoomSnapshot\(\);[\s\S]*startMediaSecurityHandshakeWatchdog\(\);[\s\S]*scheduleMediaSecurityParticipantSync\('sync_participant_set_recover'/m, 'participant-set mismatch recovery must clear stale signal caches, refresh the authoritative room snapshot, restart the handshake watchdog, and schedule a follow-up sync');
assert.match(mediaSecurityRuntimeSource, /eventType: 'media_security_participant_set_recover'[\s\S]*code: 'media_security_participant_set_recover'/m, 'participant-set mismatch recovery must emit a durable diagnostic event for deploy log triage');
assert.match(mediaSecurityRuntimeSource, /function queueMediaSecuritySyncAfterInFlight\(forceRekey = false\)[\s\S]*state\.mediaSecuritySyncPending = true;[\s\S]*state\.mediaSecuritySyncPendingForceRekey = Boolean\(state\.mediaSecuritySyncPendingForceRekey \|\| forceRekey\);/m, 'media-security runtime must not drop participant-set syncs that arrive while a join rekey is already in flight');
assert.match(mediaSecurityRuntimeSource, /if \(state\.mediaSecuritySyncInFlight\) \{[\s\S]*queueMediaSecuritySyncAfterInFlight\(forceRekey\);[\s\S]*return;[\s\S]*\}/m, 'media-security runtime must queue in-flight sync churn instead of returning permanently stale participant transcripts');
assert.match(mediaSecurityRuntimeSource, /scheduleMediaSecurityParticipantSync\('pending_after_inflight', shouldForceRekey\);/, 'queued media-security churn must run a follow-up sync after the active rekey completes');
+ assert.match(mediaSecurityRuntimeSource, /const shouldRotateSenderKey = shouldForceMediaSecurityRekeyForParticipantSetDelta\(participantDelta, forceRekey\);[\s\S]*if \(marked\.changed \|\| shouldRotateSenderKey\) \{[\s\S]*clearMediaSecuritySignalCaches\(\);[\s\S]*if \(shouldRotateSenderKey\) \{/m, 'participant-set additions must invalidate hello/sender-key signal caches without forcing a sender-key epoch rotation');
+ assert.match(mediaSecurityRuntimeSource, /if \(type === 'call\/media-security-sync-request'\) \{[\s\S]*if \(marked\.changed \|\| shouldForceRekey\) \{[\s\S]*clearMediaSecuritySignalCaches\(\);/m, 'remote sync requests must clear stale signal caches after participant-set additions before replaying hello');
+ assert.match(mediaSecurityRuntimeSource, /if \(type === 'media-security\/hello'\) \{[\s\S]*if \(marked\.changed\) \{[\s\S]*clearMediaSecuritySignalCaches\(\);[\s\S]*scheduleMediaSecurityParticipantSync\('hello_participant_set_changed', shouldForceRekey\);/m, 'incoming hello participant-set additions must refresh cached hello/sender-key signals without forcing a rekey storm');
assert.match(mediaSecurityRuntimeSource, /scheduleMediaSecurityParticipantSync\([\s\S]*'sender_key_participant_mismatch',[\s\S]*false,[\s\S]*\);/m, 'sender-key participant mismatch must wait for a fresh hello instead of force-rekeying every stale transcript');
+ assert.match(mediaSecurityRuntimeSource, /requestRemoteMediaSecuritySync\(normalizedTargetId, 'sender_key_participant_mismatch'/, 'sender-key participant mismatch must prompt the remote peer to refresh hello/sender-key state');
assert.match(mediaSecurityRuntimeSource, /requestRoomSnapshot\(\);[\s\S]*scheduleMediaSecurityParticipantSync\([\s\S]*'sender_key_participant_mismatch',[\s\S]*false,[\s\S]*\);/m, 'sender-key participant mismatch must refresh the authoritative room snapshot before non-forced transcript recovery');
+ assert.match(mediaSecurityRuntimeSource, /type === 'media-security\/sender-key'[\s\S]*errorCode === 'participant_set_mismatch'[\s\S]*requestRemoteMediaSecuritySync\(normalizedSenderUserId, 'sender_key_participant_mismatch'[\s\S]*await sendMediaSecurityHello\(normalizedSenderUserId, true\);[\s\S]*scheduleMediaSecurityParticipantSync\('sender_key_participant_mismatch', false\);[\s\S]*eventType: 'media_security_sender_key_participant_mismatch'/m, 'incoming stale sender-key participant mismatches must use sender-key recovery instead of the generic hard signal-failure path');
+ {
+ const senderKeyMismatchRecoveryIndex = mediaSecurityRuntimeSource.indexOf("type === 'media-security/sender-key'\n && errorCode === 'participant_set_mismatch'");
+ const senderKeyMismatchReturnIndex = mediaSecurityRuntimeSource.indexOf(' return;', senderKeyMismatchRecoveryIndex);
+ const signalFailedIndex = mediaSecurityRuntimeSource.indexOf("captureClientDiagnosticError('media_security_signal_failed'", senderKeyMismatchRecoveryIndex);
+ assert.ok(senderKeyMismatchRecoveryIndex >= 0, 'sender-key participant mismatch receive recovery block must exist');
+ assert.ok(senderKeyMismatchReturnIndex > senderKeyMismatchRecoveryIndex, 'sender-key participant mismatch receive recovery must return after scheduling recovery');
+ assert.ok(signalFailedIndex < 0 || senderKeyMismatchReturnIndex < signalFailedIndex, 'sender-key participant mismatch receive recovery must not fall through to media_security_signal_failed');
+ }
assert.match(mediaSecurityRuntimeSource, /scheduleMediaSecurityParticipantSync\('hello_participant_set_changed', shouldForceRekey\);/, 'incoming hello participant-set additions must not force a fresh local sender-key epoch for already signaled receivers');
assert.match(mediaSecurityRuntimeSource, /const shouldForceRekeyAfterSignalFailure = errorCode === 'downgrade_attempt';/m, 'participant-set signal failures must not create a forced rekey storm while waiting for a fresh hello');
assert.match(mediaSecurityRuntimeSource, /if \(errorCode === 'downgrade_attempt'\) \{[\s\S]*session\.markPeerRemoved\?\.\(normalizedSenderUserId\);[\s\S]*\}/m, 'downgrade-attempt recovery must clear stale peer key state before rebuilding the handshake');
+ assert.match(mediaSecurityRuntimeSource, /if \(errorCode === 'participant_set_mismatch'\) \{[\s\S]*await sendMediaSecurityHello\(normalizedSenderUserId, true\);[\s\S]*\}/m, 'participant-set mismatch recovery must immediately send a fresh hello so the publisher can rebuild the wrapping transcript');
assert.match(mediaSecurityRuntimeSource, /scheduleMediaSecurityParticipantSync\('signal_failed_reconnect', shouldForceRekeyAfterSignalFailure\);/, 'media-security signal failures must trigger a reconnect-style participant sync and force rekey after participant-set churn or downgrade attempts');
assert.match(mediaSecurityRuntimeSource, /requestRoomSnapshot\(\);[\s\S]*scheduleMediaSecurityParticipantSync\('signal_failed_reconnect', shouldForceRekeyAfterSignalFailure\);/, 'media-security signal failures must refresh the authoritative room snapshot before reconnect-style sync');
assert.match(mediaSecurityRuntimeSource, /const resyncDelayMs = shouldSettleParticipantChurn \? settleMs : 0;[\s\S]*setTimeout\(\(\) => \{[\s\S]*syncMediaSecurityWithParticipants\(shouldForceRekey\);[\s\S]*\}, resyncDelayMs\);/m, 'participant-set sync must use the SFU target settle window instead of fanning out immediate rekeys for join churn');
@@ -471,11 +487,16 @@ try {
assert.match(mediaSecurityRuntimeSource, /requestRemoteMediaSecuritySync\(normalizedSenderUserId, 'signal_failed_reconnect'/, 'media-security signal failures must send an authoritative remote sync request to the sender');
assert.match(mediaSecurityRuntimeSource, /function hintMediaSecuritySync\(reason = 'unspecified'[\s\S]*const targetUserIds = remoteMediaSecurityEligibleTargetIds\(\);[\s\S]*for \(const targetUserId of targetUserIds\) \{[\s\S]*requestRemoteMediaSecuritySync\(targetUserId, 'media_security_sync_hint'/m, 'SFU publish security-gate hints must prompt remote peers to refresh their sender-key handshake, not only retry the local sender key');
assert.match(mediaSecurityRuntimeSource, /function shouldForceReplyToIncomingMediaSecurityHello\(senderUserId, payloadBody, session\)[\s\S]*incomingMediaSecurityHelloResponseKey\(senderUserId, payloadBody, session\)[\s\S]*state\.mediaSecurityHelloSignalsSent\.add\(key\);/m, 'accepted remote hello responses must be deduped by incoming hello identity to avoid broker replay echo loops');
+ assert.match(mediaSecurityRuntimeSource, /function mediaSecurityHelloSignalKey\(targetUserId, session\)[\s\S]*String\(session\?\.participantSignature \|\| ''\)[\s\S]*'hello'/m, 'media-security hello dedupe keys must include the participant signature so participant-set churn refreshes hellos');
+ assert.match(mediaSecurityRuntimeSource, /function mediaSecuritySenderKeySignalKey\(targetUserId, session\)[\s\S]*String\(session\?\.participantSignature \|\| ''\)[\s\S]*'sender-key'/m, 'media-security sender-key dedupe keys must include the participant signature so SFU publish gates reopen only after current-set sender keys');
+ assert.match(mediaSecurityRuntimeSource, /function incomingMediaSecurityHelloResponseKey\(senderUserId, payloadBody, session\)[\s\S]*String\(session\?\.participantSignature \|\| ''\)[\s\S]*'hello-response'/m, 'incoming hello response dedupe must force one fresh reply after local participant-set churn');
assert.match(mediaSecurityRuntimeSource, /const forceReply = shouldForceReplyToIncomingMediaSecurityHello\([\s\S]*normalizedSenderUserId,[\s\S]*payloadBody \|\| \{\},[\s\S]*session,[\s\S]*\);[\s\S]*await sendMediaSecurityHello\(normalizedSenderUserId, forceReply\);[\s\S]*await sendMediaSecuritySenderKey\(normalizedSenderUserId, forceReply\);/m, 'accepted remote hello must force exactly one fresh response per unique hello so reconnecting peers can unwrap sender keys without flooding the broker');
assert.doesNotMatch(mediaSecurityRuntimeSource, /if \(accepted\) \{[\s\S]*await sendMediaSecurityHello\(normalizedSenderUserId, true\);[\s\S]*await sendMediaSecuritySenderKey\(normalizedSenderUserId, true\);/m, 'accepted remote hello must not force-answer every broker replay');
assert.match(mediaSecurityParticipantSetSource, /export function mergeMediaSecurityParticipantIds\(session, targetUserIds = \[\], extraUserId = 0\)[\s\S]*mediaSecurityParticipantSignatureIds\(session\)[\s\S]*targetUserIds[\s\S]*extraUserId[\s\S]*\.sort\(\(left, right\) => left - right\);/m, 'media-security participant merge must preserve the existing signed participant set while adding current targets and the hello sender');
+ assert.match(mediaSecurityParticipantSetSource, /export function shouldRecoverMediaSecuritySignalSender\([\s\S]*hasRealtimeRoomSync !== true[\s\S]*return true;[\s\S]*\.includes\(normalizedSenderUserId\);/m, 'media-security signal recovery must accept pre-snapshot reconnect senders while still requiring target membership after room sync');
assert.match(mediaSecurityRuntimeSource, /const marked = session\.markParticipantSet\(mergeMediaSecurityParticipantIds\([\s\S]*session,[\s\S]*remoteMediaSecurityTargetIds\(\),[\s\S]*normalizedSenderUserId,[\s\S]*\)\);[\s\S]*if \(remoteMediaSecurityTargetIds\(\)\.includes\(normalizedSenderUserId\)\) \{[\s\S]*scheduleMediaSecurityParticipantSync\('hello_accepted'\);[\s\S]*\}/m, 'media-security runtime must merge remote hello senders into the current signed participant set and only schedule follow-up sync once the sender is in the current remote target set');
assert.match(mediaSecurityRuntimeSource, /const accepted = await session\.handleSenderKeySignal\(normalizedSenderUserId, payloadBody \|\| \{\}\);[\s\S]*if \(!accepted && remoteMediaSecurityTargetIds\(\)\.includes\(normalizedSenderUserId\)\) \{[\s\S]*scheduleMediaSecurityParticipantSync\('sender_key_pending'\);[\s\S]*\}/m, 'media-security runtime must defer sender-key recovery sync until the sender is present in the current remote participant target set');
+ assert.match(mediaSecurityRuntimeSource, /shouldRecoverMediaSecuritySignalSender\(\{[\s\S]*hasRealtimeRoomSync: hasRealtimeRoomSync\.value,[\s\S]*targetUserIds: remoteMediaSecurityTargetIds\(\),[\s\S]*senderUserId: normalizedSenderUserId,[\s\S]*\}\)/m, 'participant-set signal failures before the authoritative snapshot must still request snapshot-backed recovery');
assert.doesNotMatch(mediaSecurityRuntimeSource, /elapsed=\$\{Date\.now\(\) - helloSentAt\}ms — force-retrying Hello/, 'participant sync must not hide the join race behind a multi-second inline Hello retry loop');
assert.match(mediaSecurityRuntimeSource, /if \(!signal\) \{[\s\S]*const shouldRefreshHello = helloSentAt <= 0 \|\| \(Date\.now\(\) - helloSentAt\) >= 750;[\s\S]*await sendMediaSecurityHello\(normalizedTargetId, true\);/m, 'missing sender-key peer wrapping context must refresh hello quickly instead of waiting for the multi-second watchdog');
assert.match(mediaSecurityRuntimeSource, /if \(!signal\) \{[\s\S]*requestRemoteMediaSecuritySync\(normalizedTargetId, 'sender_key_not_ready'/m, 'missing sender-key peer wrapping context must also ask the remote peer to refresh hello/sender-key state');
diff --git a/demo/video-chat/frontend-vue/tests/contract/mediapipe-cdn-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/mediapipe-cdn-contract.mjs
index 36675d264..70261c895 100644
--- a/demo/video-chat/frontend-vue/tests/contract/mediapipe-cdn-contract.mjs
+++ b/demo/video-chat/frontend-vue/tests/contract/mediapipe-cdn-contract.mjs
@@ -7,7 +7,7 @@ const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
const frontendRoot = path.resolve(__dirname, '../..');
const repoVideoChatRoot = path.resolve(frontendRoot, '..');
-const sinetVendorDir = path.join(frontendRoot, 'public/cdn/vendor/sinet');
+const mediaPipeVendorDir = path.join(frontendRoot, 'public/cdn/vendor/mediapipe');
function readUtf8(relativePath) {
return fs.readFileSync(path.join(frontendRoot, relativePath), 'utf8');
@@ -22,71 +22,46 @@ function assertFile(filePath) {
assert.ok(fs.statSync(filePath).size > 0, `${filePath} must not be empty`);
}
-function assertNoJsdelivrAssetSource(source, label) {
+function assertNoExternalAssetSource(source, label) {
assert.ok(!source.includes('cdn.jsdelivr.net'), `${label} must not load jsDelivr assets`);
assert.ok(!source.includes('unpkg.com'), `${label} must not load unpkg assets`);
}
try {
- requireMissingPath('src/domain/realtime/background/backendMediapipe.ts', 'MediaPipe background backend');
+ requireMissingPath('src/domain/realtime/background/backendMediapipe.ts', 'legacy MediaPipe main-thread backend');
requireMissingPath('src/domain/realtime/background/backendTfjs.ts', 'TFJS background backend');
- requireMissingPath('src/domain/realtime/background/backendWorkerSegmenter.js', 'MediaPipe worker segmenter backend');
- requireMissingPath('src/domain/realtime/background/workers/imageSegmenterWorker.js', 'MediaPipe ImageSegmenter worker');
+ requireMissingPath('src/domain/realtime/background/backendSinetWasm.js', 'production SINet backend');
+ requireMissingPath('src/domain/realtime/background/backendSelector.ts', 'production SINet selector');
- const sinetBackend = readUtf8('src/domain/realtime/background/backendSinetWasm.js');
- const maskPostprocess = readUtf8('src/domain/realtime/background/maskPostprocess.js');
+ const workerBackend = readUtf8('src/domain/realtime/background/backendWorkerSegmenter.js');
+ const worker = readUtf8('src/domain/realtime/background/workers/imageSegmenterWorker.js');
const stream = readUtf8('src/domain/realtime/background/stream.ts');
- const selector = readUtf8('src/domain/realtime/background/backendSelector.ts');
const featureFlags = readUtf8('src/domain/realtime/background/pipeline/featureFlags.js');
const packageJson = readUtf8('package.json');
- assertNoJsdelivrAssetSource(sinetBackend, 'SINet WASM backend');
- assert.ok(sinetBackend.includes("import('onnxruntime-web/wasm')"), 'SINet backend must import the ONNX Runtime WASM build');
- assert.ok(sinetBackend.includes("executionProviders: ['wasm']"), 'SINet backend must force the WASM execution provider');
- assert.ok(sinetBackend.includes('wasm.proxy = false'), 'SINet backend must avoid ORT worker proxy mode for deterministic init');
- assert.ok(sinetBackend.includes('wasm.numThreads = 1'), 'SINet backend must use single-threaded WASM by default');
- assert.ok(sinetBackend.includes('/cdn/vendor/sinet/'), 'SINet backend must load vendored SINet assets');
- assert.ok(sinetBackend.includes('externalData: [{ path: SINET_EXTERNAL_WEIGHTS_PATH, data: weights }]'), 'SINet backend must mount external ONNX weights');
- assert.ok(sinetBackend.includes("kind: 'sinet_wasm'"), 'SINet backend must expose its backend kind');
- assert.ok(sinetBackend.includes('matteMaskValues'), 'SINet backend must return value masks for the shared compositor');
- assert.ok(sinetBackend.includes('function binaryForegroundAlpha(value, threshold = 0)'), 'SINet fallback must classify raw mask output without sigmoid');
- assert.ok(sinetBackend.includes('alpha[i] = binaryForegroundAlpha(fg, bg);'), 'SINet fallback must use foreground-vs-background argmax for two-channel outputs');
- assert.ok(sinetBackend.includes('const threshold = probabilityLike ? 0.5 : 0;'), 'SINet fallback must threshold single-channel masks without sigmoid');
- assert.ok(!sinetBackend.includes('Math.exp(bg - max)'), 'SINet fallback must not softmax background and foreground logits');
- assert.ok(!sinetBackend.includes('fgExp / Math.max'), 'SINet fallback must not couple foreground alpha to the background logit');
- assert.ok(!sinetBackend.includes('foregroundLogitToProbability'), 'SINet fallback must not run raw logits through sigmoid');
- assert.ok(!sinetBackend.includes('1 / (1 + Math.exp'), 'SINet fallback must not use sigmoid alpha mapping');
- assert.ok(!sinetBackend.includes('@mediapipe'), 'SINet backend must not depend on MediaPipe');
+ assertNoExternalAssetSource(workerBackend, 'worker segmenter backend');
+ assertNoExternalAssetSource(worker, 'ImageSegmenter worker');
+ assert.ok(workerBackend.includes("const MODEL_PATH = `${VIDEOCHAT_CDN_ORIGIN}${MEDIAPIPE_MODEL_BASE_PATH}selfie_multiclass_256x256.tflite`;"), 'worker backend must load the vendored model through the configured CDN origin');
+ assert.ok(workerBackend.includes("const WASM_PATH = `${VIDEOCHAT_CDN_ORIGIN}${MEDIAPIPE_WASM_BASE_PATH}`;"), 'worker backend must load wasm through the configured CDN origin');
+ assert.ok(worker.includes("const DEFAULT_MODEL_PATH = '/cdn/vendor/mediapipe/models/selfie_multiclass_256x256.tflite';"), 'worker default model path must be local CDN');
+ assert.ok(worker.includes("const DEFAULT_WASM_PATH = '/wasm';"), 'worker default wasm path must be local');
+ assert.ok(worker.includes('loadModuleFactory(resolvedWasm);'), 'worker must load the vendored wasm factory explicitly');
+ assert.ok(worker.includes('modelAssetBuffer: new Uint8Array(modelBuffer)'), 'worker must pass a local model buffer to MediaPipe');
+ assert.ok(stream.includes('backendWorkerSegmenter'), 'production stream must use Pierre worker segmenter');
+ assert.ok(!stream.includes('backendSinetWasm'), 'production stream must not use SINet as fallback');
+ assert.ok(!featureFlags.includes('VITE_VIDEOCHAT_WORKER_SEGMENTER'), 'worker segmenter must be the production path, not a dead toggle');
+ assert.ok(packageJson.includes('mediapipe-cdn-contract.mjs'), 'CDN contract must remain executable in CI');
- assert.ok(maskPostprocess.includes('function smoothstep(edge0, edge1, value)'), 'Mask postprocess must smooth only the contour transition');
- assert.ok(maskPostprocess.includes('const edgeLow = 0.5 - contourHalfWidth'), 'Mask postprocess must keep hard background outside the contour band');
- assert.ok(!maskPostprocess.includes('(raw - 0.5) * contrast + 0.5'), 'Mask postprocess must not raise background alpha through global contrast shaping');
-
- assert.ok(stream.includes("import { createSinetWasmSegmentationBackend } from './backendSinetWasm';"), 'Background stream must import the SINet WASM backend');
- assert.ok(stream.includes('BACKGROUND_SEGMENTER_INIT_RETRY_MS'), 'Background stream must back off repeated segmenter init failures');
- assert.ok(!stream.includes('backendWorkerSegmenter'), 'production background stream must not use the MediaPipe worker fallback');
- assert.ok(!stream.includes('MediaPipe'), 'production background stream must not reference MediaPipe');
- assert.ok(!stream.includes('TensorFlow'), 'production background stream must not reference TensorFlow');
- assert.ok(!stream.includes('tfjs'), 'production background stream must not reference TFJS');
- assert.ok(selector.includes("backend: 'sinet_wasm'"), 'backend selector must report SINet WASM');
- assert.ok(!featureFlags.includes('VITE_VIDEOCHAT_WORKER_SEGMENTER'), 'production background feature flags must not expose legacy worker segmenter toggles');
- assert.ok(!featureFlags.includes('WORKER_SEGMENTER'), 'production background feature flags must not retain dead worker segmenter exports');
- assert.ok(!packageJson.includes('@mediapipe/tasks-vision'), 'frontend package must not depend on MediaPipe Tasks');
- assert.ok(packageJson.includes('mediapipe-cdn-contract.mjs'), 'legacy CDN contract name must remain executable in CI');
-
- assertFile(path.join(sinetVendorDir, 'metadata-float.json'));
- assertFile(path.join(sinetVendorDir, 'sinet-float.onnx'));
- assertFile(path.join(sinetVendorDir, 'sinet.data'));
- const sinetMetadata = JSON.parse(fs.readFileSync(path.join(sinetVendorDir, 'metadata-float.json'), 'utf8'));
- assert.equal(sinetMetadata.model_id, 'sinet');
- assert.equal(sinetMetadata.runtime, 'onnx');
+ assertFile(path.join(mediaPipeVendorDir, 'tasks-vision/vision_bundle.mjs'));
+ assertFile(path.join(mediaPipeVendorDir, 'models/selfie_multiclass_256x256.tflite'));
+ assertFile(path.join(frontendRoot, 'public/wasm/vision_wasm_internal.js'));
const edge = fs.readFileSync(path.join(repoVideoChatRoot, 'edge/edge.php'), 'utf8');
assert.ok(edge.includes('VIDEOCHAT_EDGE_CDN_DOMAIN'), 'King edge must recognize the CDN host');
assert.ok(edge.includes('Access-Control-Allow-Origin'), 'King edge must emit CORS for CDN assets');
assert.ok(edge.includes("'wasm' => 'application/wasm'"), 'King edge must serve wasm with the correct MIME type');
- console.log('[mediapipe-cdn-contract] PASS legacy MediaPipe/TFJS background fallback removed');
+ console.log('[mediapipe-cdn-contract] PASS worker MediaPipe assets are local/CDN-hosted');
} catch (error) {
console.error(`[mediapipe-cdn-contract] FAIL: ${error.message}`);
process.exit(1);
diff --git a/demo/video-chat/frontend-vue/tests/contract/prod-debug-observability-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/prod-debug-observability-contract.mjs
new file mode 100644
index 000000000..8c2ad0cd9
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/contract/prod-debug-observability-contract.mjs
@@ -0,0 +1,103 @@
+import assert from 'node:assert/strict';
+import fs from 'node:fs';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = path.dirname(__filename);
+const frontendRoot = path.resolve(__dirname, '../..');
+const videoChatRoot = path.resolve(frontendRoot, '..');
+const repoRoot = path.resolve(videoChatRoot, '../..');
+
+function readText(relativePath) {
+ return fs.readFileSync(path.join(repoRoot, relativePath), 'utf8');
+}
+
+const scriptPath = 'demo/video-chat/scripts/prod-debug.sh';
+const script = readText(scriptPath);
+const readme = readText('README.md');
+
+assert.match(script, /^#!\/usr\/bin\/env bash/, 'prod-debug must be a bash operator script');
+assert.match(script, /mode: read-only production diagnostics/, 'prod-debug must declare its read-only mode');
+assert.match(script, /no deploy, restart, DB write, DNS change, or admin action/, 'prod-debug must state forbidden production mutations');
+assert.match(script, /LOCAL_ENV_FILE=.*\.env\.local/, 'prod-debug must use existing .env.local as its local source');
+assert.match(script, /redact_stream\(\)/, 'prod-debug must redact output');
+assert.match(script, /TOKEN\|SECRET\|PASSWORD\|PASS\|KEY\|CREDENTIAL\|COOKIE\|SESSION/, 'prod-debug redaction must cover token/password-like values');
+assert.match(script, /REDACTED_MEDIA_PAYLOAD/, 'prod-debug must redact media payload-like fields before printing logs');
+assert.match(script, /VIDEOCHAT_PROD_DEBUG_DRY_RUN/, 'prod-debug must expose a dry-run path for local proof without network or SSH');
+
+for (const endpoint of [
+ '/api/runtime',
+ '/api/version',
+ '/api/marketplace/call-apps',
+ '/public/index.html',
+ '/call-app/whiteboard/public/index.html',
+]) {
+ assert.ok(script.includes(endpoint), `prod-debug must inspect ${endpoint}`);
+}
+
+for (const label of [
+ 'lobby websocket',
+ 'sfu websocket',
+ 'marketplace apps',
+ 'call-app host',
+ 'Call-App CSP Header Proof',
+ 'call-app whiteboard host CSP',
+ 'call-app whiteboard path CSP',
+ 'filtered recent logs',
+ 'media reconnect',
+ 'screen-share reconnect exhaustion',
+ 'stale local media capture discard',
+ 'audio/video track loss',
+ 'SFU reconnect and websocket transport',
+ 'Call App frame and CSP errors',
+]) {
+ assert.ok(script.includes(label), `prod-debug must include ${label}`);
+}
+
+assert.match(
+ script,
+ /Content-Security-Policy[\s\S]*Allow-CSP-From[\s\S]*X-Frame-Options[\s\S]*nested \*\.\$\{DEPLOY_APP_DOMAIN\} service origins/,
+ 'prod-debug must prove Whiteboard Call App CSP, Embedded-CSP, frame-option absence, and nested-origin absence',
+);
+
+assert.match(script, /docker compose[\s\S]* ps/, 'remote probe must inspect compose container status');
+assert.match(script, /\$\{COMPOSE\[@\]\}" logs --no-color --tail/, 'remote probe must collect bounded recent container logs');
+assert.match(script, /filter_recent_logs\(\)/, 'remote log filtering must label each investigation category');
+assert.match(script, /stale_local_media_capture_discarded/, 'remote log scan must include stale local media capture discard diagnostics');
+assert.match(script, /local_screen_share_sfu_reconnect_exhausted/, 'remote log scan must include screen-share SFU reconnect exhaustion diagnostics');
+assert.match(script, /\(audio\|video\).*track/, 'remote log scan must include audio/video track-loss terms');
+assert.match(script, /Content-Security-Policy\|Allow-CSP-From\|frame-ancestors\|postMessage/, 'remote log scan must include Call App frame and CSP diagnostics');
+
+const forbiddenPatterns = [
+ /\bcurl\b[^\n]*\s-X\s*(POST|PUT|PATCH|DELETE)\b/i,
+ /\bdocker\s+compose\b[^\n]*(\bup\b|\bdown\b|\brestart\b|\brm\b|\bkill\b|\bpull\b|\bpush\b|\bexec\b)/i,
+ /\bdocker\b[^\n]*(\brun\b|\brestart\b|\brm\b|\bkill\b|\bpull\b|\bpush\b|\bexec\b)/i,
+ /\b(rsync|scp)\b/,
+ /\b(certbot|hcloud|terraform|kubectl)\b/,
+ /\bdeploy\.sh\b/,
+ /\bsqlite3\b[^\n]*(INSERT|UPDATE|DELETE|REPLACE|DROP|CREATE|ALTER|VACUUM)/i,
+];
+
+for (const pattern of forbiddenPatterns) {
+ assert.doesNotMatch(script, pattern, `prod-debug must remain read-only; forbidden pattern ${pattern}`);
+}
+
+assert.match(
+ readme,
+ /demo\/video-chat\/scripts\/prod-debug\.sh/,
+ 'README must expose the production debug command',
+);
+assert.match(
+ readme,
+ /read-only[\s\S]*media reconnect[\s\S]*screen-share[\s\S]*stale local media capture[\s\S]*audio\/video track loss[\s\S]*SFU reconnect[\s\S]*Call App frame\/CSP[\s\S]*prod-debug\.sh[\s\S]*does not deploy,\s*restart,\s*write DB data,\s*change DNS,\s*or use admin actions/i,
+ 'README must document prod-debug as read-only and non-mutating',
+);
+
+assert.match(
+ readme,
+ /Whiteboard Call App CSP\/`Allow-CSP-From` frame headers[\s\S]*\/public\/index\.html[\s\S]*\/call-app\/whiteboard\/public\/index\.html[\s\S]*absence[\s\S]*of `X-Frame-Options`[\s\S]*absence of nested `\*\.app\.kingrt\.com` service origins/s,
+ 'README must document the read-only Call App frame-header proof',
+);
+
+process.stdout.write('[prod-debug-observability-contract] PASS\n');
diff --git a/demo/video-chat/frontend-vue/tests/contract/public-pages-localization-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/public-pages-localization-contract.mjs
index 4c6e9c884..2c8525a56 100644
--- a/demo/video-chat/frontend-vue/tests/contract/public-pages-localization-contract.mjs
+++ b/demo/video-chat/frontend-vue/tests/contract/public-pages-localization-contract.mjs
@@ -55,7 +55,7 @@ const appointmentApiSource = await readFile(path.join(root, 'src/domain/calls/ap
const appointmentBookingModalSource = await readFile(path.join(root, 'src/domain/calls/appointment/AppointmentBookingModal.vue'), 'utf8');
const appointmentBookingModalCssSource = await readFile(path.join(root, 'src/domain/calls/appointment/AppointmentBookingModal.css'), 'utf8');
const joinViewSource = await readFile(path.join(root, 'src/domain/calls/access/JoinView.vue'), 'utf8');
-const sessionSource = await readFile(path.join(root, 'src/domain/auth/session.ts'), 'utf8');
+const callAccessSessionSource = await readFile(path.join(root, 'src/domain/calls/access/callAccessSession.ts'), 'utf8');
const englishMessagesSource = await readFile(path.join(root, 'src/modules/localization/englishMessages.js'), 'utf8');
const publicMessagesSource = await readFile(path.join(root, 'src/modules/localization/publicMessages.js'), 'utf8');
const fallbackMessagesSource = `${englishMessagesSource}\n${publicMessagesSource}`;
@@ -87,7 +87,7 @@ assert.match(joinViewSource, /localizedApiErrorMessage\(\{ error: \{ code: 'call
assert.match(joinViewSource, /localizedApiErrorMessage\(errorPayload,\s*t\('public\.join\.start_session_failed'\)\)/, 'public join session errors must resolve through stable codes');
assert.doesNotMatch(joinViewSource, /result\.message/, 'public join session errors must not display pre-localized backend messages directly');
assert.doesNotMatch(joinViewSource, /payload\?\.error\?\.message/, 'public join view must not display backend English error messages directly');
-assert.match(sessionSource, /errorCode:\s*errorCodeFromPayload\(payload\)/, 'call access session login must expose backend error codes for public localization');
+assert.match(callAccessSessionSource, /errorCode:\s*errorCodeFromPayload\(payload\)/, 'call access session login must expose backend error codes for public localization');
for (const key of [
'errors.api.call_access_expired',
'errors.api.call_access_not_found',
diff --git a/demo/video-chat/frontend-vue/tests/contract/realtime-reconnect-browser-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/realtime-reconnect-browser-contract.mjs
new file mode 100644
index 000000000..432c8df0f
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/contract/realtime-reconnect-browser-contract.mjs
@@ -0,0 +1,159 @@
+import assert from 'node:assert/strict';
+import fs from 'node:fs';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+function fail(message) {
+ throw new Error(`[realtime-reconnect-browser-contract] FAIL: ${message}`);
+}
+
+function read(root, relativePath) {
+ return fs.readFileSync(path.join(root, relativePath), 'utf8');
+}
+
+function section(source, start, end, label) {
+ const startIndex = source.indexOf(start);
+ assert.notEqual(startIndex, -1, `${label} start missing`);
+ const endIndex = source.indexOf(end, startIndex + start.length);
+ assert.notEqual(endIndex, -1, `${label} end missing`);
+ return source.slice(startIndex, endIndex);
+}
+
+function assertIncludes(source, needle, message) {
+ assert.ok(source.includes(needle), message);
+}
+
+function assertOrder(source, first, second, message) {
+ const firstIndex = source.indexOf(first);
+ const secondIndex = source.indexOf(second);
+ assert.ok(firstIndex >= 0, `${message}: first anchor missing`);
+ assert.ok(secondIndex >= 0, `${message}: second anchor missing`);
+ assert.ok(firstIndex < secondIndex, message);
+}
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = path.dirname(__filename);
+const root = path.resolve(__dirname, '../..');
+
+try {
+ const packageJson = JSON.parse(read(root, 'package.json'));
+ assert.equal(
+ packageJson.scripts['test:contract:realtime-reconnect-browser'],
+ 'node tests/contract/realtime-reconnect-browser-contract.mjs',
+ 'package script must expose the focused browser reconnect contract',
+ );
+
+ const socketLifecycle = read(root, 'src/domain/realtime/workspace/callWorkspace/socketLifecycle.ts');
+ const workspace = read(root, 'src/domain/realtime/CallWorkspaceView.vue');
+
+ assertIncludes(
+ socketLifecycle,
+ "'websocket_reconnect_backfill_unavailable'",
+ 'browser lifecycle must know the retryable reconnect backfill code from the backend contract',
+ );
+ assert.match(
+ socketLifecycle,
+ /const transientAuthBackendError = code === 'websocket_auth_temporarily_unavailable'[\s\S]*\|\| closeReason === 'auth_backend_error';/,
+ 'auth backend websocket errors must remain retryable in the workspace lifecycle',
+ );
+ assert.match(
+ socketLifecycle,
+ /const transientReconnectBackfillError = code === 'websocket_reconnect_backfill_unavailable'[\s\S]*RETRYABLE_RECONNECT_BACKFILL_REASONS\.includes\(closeReason\);/,
+ 'reconnect backfill failures must be classified with retryable auth/backend failures',
+ );
+
+ const retryableSystemErrorBlock = section(
+ socketLifecycle,
+ 'if (retryableRealtimeReconnectError) {',
+ " if (code === 'websocket_session_invalidated'",
+ 'retryable system/error handler',
+ );
+ assertIncludes(retryableSystemErrorBlock, 'state.manualSocketClose = false;', 'retryable websocket errors must not become manual closes');
+ assertIncludes(retryableSystemErrorBlock, "refs.connectionState.value = 'retrying';", 'retryable websocket errors must leave the UI in retrying state');
+ assertIncludes(retryableSystemErrorBlock, "eventType: 'realtime_websocket_retryable_error'", 'retryable websocket errors must emit a diagnostic');
+ assertIncludes(retryableSystemErrorBlock, 'retryable: true', 'retryable diagnostic must carry retryable=true');
+ assertIncludes(retryableSystemErrorBlock, 'requested_room_id: refs.desiredRoomId.value', 'retryable diagnostic must include requested call room scope');
+ assertIncludes(retryableSystemErrorBlock, 'active_call_id: refs.activeSocketCallId.value', 'retryable diagnostic must include call scope');
+ assertIncludes(retryableSystemErrorBlock, 'closeSocketLocal();', 'retryable websocket errors may recycle the socket');
+ assertIncludes(retryableSystemErrorBlock, 'scheduleReconnect();', 'retryable websocket errors must schedule reconnect');
+ assert.doesNotMatch(
+ retryableSystemErrorBlock,
+ /connectionState\.value = 'expired'|connectionState\.value = 'blocked'|manualSocketClose = true|location\.reload|window\.location\.reload|logoutSession|router\.replace/,
+ 'retryable auth/backfill errors must not trigger logout, reload, blocked, or expired UI paths',
+ );
+
+ const closeHandlerBlock = section(
+ socketLifecycle,
+ "socket.addEventListener('close', (event) => {",
+ ' negotiationTimer = setTimeout',
+ 'websocket close handler',
+ );
+ assertIncludes(closeHandlerBlock, 'const retryableBackfillClose = RETRYABLE_RECONNECT_BACKFILL_REASONS.includes(closeReason);', 'close handler must classify retryable backfill closes');
+ assert.match(
+ closeHandlerBlock,
+ /if \(retryableBackfillClose\) \{[\s\S]*refs\.connectionState\.value = 'retrying';[\s\S]*captureRetryableReconnectClose\(closeReason, event\);[\s\S]*scheduleReconnect\(\);[\s\S]*return;/,
+ 'retryable backfill close must diagnose and retry instead of ending the call',
+ );
+ assert.match(
+ closeHandlerBlock,
+ /if \(closeReason === 'auth_backend_error' \|\| event\?\.code === 1011\) \{[\s\S]*refs\.connectionState\.value = 'retrying';[\s\S]*captureRetryableReconnectClose\(closeReason \|\| 'socket_internal_error', event\);[\s\S]*scheduleReconnect\(\);/,
+ 'auth backend/internal closes must remain retryable and diagnostic',
+ );
+
+ const originExhaustedBlock = section(
+ socketLifecycle,
+ 'if (originIndex >= orderedSocketOrigins.length) {',
+ ' const socketOrigin = orderedSocketOrigins[originIndex] || \'\';',
+ 'websocket origin exhaustion handler',
+ );
+ assertIncludes(originExhaustedBlock, "refs.connectionState.value = 'retrying';", 'pre-upgrade websocket failures must leave the UI retrying');
+ assertIncludes(originExhaustedBlock, "refs.connectionReason.value = 'socket_unreachable';", 'pre-upgrade websocket failures must keep a retry reason');
+ assertIncludes(originExhaustedBlock, "eventType: 'realtime_websocket_retryable_error'", 'pre-upgrade websocket failures must emit retryable diagnostics');
+ assertIncludes(originExhaustedBlock, "code: 'websocket_connect_retry_scheduled'", 'pre-upgrade websocket diagnostic must use a stable retry code');
+ assertIncludes(originExhaustedBlock, 'retryable: true', 'pre-upgrade websocket diagnostic must carry retryable=true');
+ assertIncludes(originExhaustedBlock, 'scheduleReconnect();', 'pre-upgrade websocket failures must retry instead of expiring the session');
+ assert.doesNotMatch(
+ originExhaustedBlock,
+ /connectionState\.value = 'expired'|connectionState\.value = 'blocked'|manualSocketClose = true|location\.reload|window\.location\.reload|logoutSession|router\.replace/,
+ 'pre-upgrade websocket failures must not trigger logout, reload, blocked, or expired UI paths',
+ );
+
+ const openHandlerBlock = section(
+ socketLifecycle,
+ "socket.addEventListener('open', () => {",
+ " socket.addEventListener('message', handleSocketMessage);",
+ 'websocket open handler',
+ );
+ assertOrder(
+ openHandlerBlock,
+ "refs.connectionState.value = 'online';",
+ 'requestRoomSnapshot();',
+ 'successful reconnect must request authoritative room snapshot after the socket is online',
+ );
+ assertIncludes(openHandlerBlock, 'reconnect: isReconnectOpen', 'open diagnostic must identify reconnect opens');
+
+ const welcomeBlock = section(
+ socketLifecycle,
+ "if (type === 'system/welcome') {",
+ " if (type === 'room/snapshot') {",
+ 'system welcome handler',
+ );
+ assertIncludes(welcomeBlock, 'requestRoomSnapshot();', 'system welcome must request room snapshot backfill');
+ assertIncludes(
+ workspace,
+ "sendSocketFrame({ type: 'room/snapshot/request' })",
+ 'workspace snapshot request must use the room/snapshot/request websocket command',
+ );
+ assert.doesNotMatch(
+ socketLifecycle,
+ /location\.reload|window\.location\.reload|logoutSession|router\.replace/,
+ 'socket lifecycle must not handle reconnect auth/backfill failures through reload or logout navigation',
+ );
+
+ process.stdout.write('[realtime-reconnect-browser-contract] PASS\n');
+} catch (error) {
+ if (error instanceof Error) {
+ fail(error.message);
+ }
+ fail('unknown failure');
+}
diff --git a/demo/video-chat/frontend-vue/tests/contract/sfu-protected-browser-encoder-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/sfu-protected-browser-encoder-contract.mjs
index daf19c5f4..4858bae1a 100644
--- a/demo/video-chat/frontend-vue/tests/contract/sfu-protected-browser-encoder-contract.mjs
+++ b/demo/video-chat/frontend-vue/tests/contract/sfu-protected-browser-encoder-contract.mjs
@@ -144,7 +144,7 @@ try {
requireContains(mediaSecuritySfuPublishGate, 'return sentAtMs > 0 && (nowMs - sentAtMs) >= propagationMs;', 'SFU publish gate must hold protected frames until sender-key signals can reach receivers');
requireContains(mediaSecuritySfuPublishGate, 'currentSfuSenderKeySignalsCoverTargets(targetUserIds)', 'SFU publish gate must require full current-target sender-key coverage before propagation readiness');
requireContains(mediaSecurityRuntime, 'state.mediaSecuritySenderKeySignalsSent.has(mediaSecuritySenderKeySignalKey(userId, session))', 'SFU publish gate must use sender-key signal receipts from the current media-security session');
- requireContains(mediaSecurityRuntime, 'shouldForceRekeyForParticipantSetDelta(participantDelta, forceRekey)', 'participant joins must not force a global sender-key cache reset while existing receivers can still decrypt video');
+ requireContains(mediaSecurityRuntime, 'shouldForceMediaSecurityRekeyForParticipantSetDelta(participantDelta, forceRekey)', 'participant joins must not force a sender-key epoch rotation while existing receivers can still decrypt video');
requireContains(publisherPipeline, "protected_media_fallback: 'transport_only_until_sender_key_ready'", 'RGBA fallback publisher must keep sending frames while sender keys settle');
requireContains(publisherPipeline, "protected_media_fallback: 'transport_only_after_protect_unavailable'", 'RGBA fallback publisher must keep sending frames after preferred media-security protection is temporarily unavailable');
requireContains(mediaSecurityTargets, 'return targetUserIds;', 'SFU media-security target set must come from connected remote participants, not delayed publisher discovery');
diff --git a/demo/video-chat/frontend-vue/tests/contract/sfu-video-recovery-timing-contract.mjs b/demo/video-chat/frontend-vue/tests/contract/sfu-video-recovery-timing-contract.mjs
index e44dbd8eb..f35e5c87f 100644
--- a/demo/video-chat/frontend-vue/tests/contract/sfu-video-recovery-timing-contract.mjs
+++ b/demo/video-chat/frontend-vue/tests/contract/sfu-video-recovery-timing-contract.mjs
@@ -101,7 +101,11 @@ try {
requireContains(frameDecode, 'resizeCanvasPreservingFrame(peer.decodedCanvas, nextWidth, nextHeight);', 'decoder reconfigure does not clear the remote canvas before the next keyframe');
requireContains(remotePeers, "mediaConnectionState: 'connecting'", 'new SFU peers start in connecting media state');
requireContains(remotePeers, 'function findSfuRemotePeerEntryByPeer', 'remote peer owner lookup for publisher rollover');
- requireContains(remotePeers, 'publisherId: normalizedPublisherId', 'publisher alias lookup adopts the current frame publisher id');
+ requireContains(remotePeers, 'publisherId: fallback.publisherId', 'publisher alias lookup keeps the existing track publisher key');
+ requireContains(frameDecode, 'const resolvedPublisherId = normalizeSfuPublisherId(peerLookup?.publisherId || publisherId);', 'frame decode resolves publisher alias before continuity and recovery');
+ requireContains(frameDecode, 'resolved_publisher_id: resolvedPublisherId', 'alias diagnostics expose the canonical publisher key');
+ requireContains(frameDecode, 'frame = { ...frame, publisherId: resolvedPublisherId, publisherIdAlias: publisherId };', 'alias frames are decoded under the existing publisher key');
+ requireContains(frameDecode, 'void decodeSfuFrameForPeer(resolvedPublisherId, peer, frame);', 'remote frame decode uses resolved publisher key for keyframe recovery');
requireContains(remotePeers, 'function resetSfuRemotePeerMediaContinuity', 'remote peer continuity reset helper');
requireContains(remotePeers, "'publisher_id_rollover'", 'publisher id rollover resets remote continuity');
requireContains(remotePeers, "'track_set_rollover'", 'track rollover resets remote continuity');
@@ -111,7 +115,7 @@ try {
requireContains(remotePeers, 'nextSfuSocketRestartAllowedAtMs: 0', 'remote peer continuity starts without a restart backoff gate');
requireContains(remotePeers, 'Keep the last visible frame while waiting for the rollover keyframe.', 'rollover preserves the visible remote frame');
requireNotContains(remotePeers, 'clearDecodedCanvas(peer);', 'rollover clearing the displayed remote canvas');
- requireContains(remotePeers, 'setSfuRemotePeer(normalizedPublisherId, updatedPeer, resolvedPreviousPublisherId)', 'frame alias adoption moves peer to new publisher id');
+ requireContains(remotePeers, 'setSfuRemotePeer(normalizedPublisherId, updatedPeer, resolvedPreviousPublisherId)', 'genuine publisher rollover can still move peer ownership');
requireContains(mediaStack, 'bumpMediaRenderVersion,', 'runtime health and frame decode receive media render invalidation');
requireContains(sfuClient, 'private markPublisherFrameReceived(msg: SfuClientMessage', 'SFU client tracks publisher frame freshness');
requireContains(sfuClient, "if (stringField(msg?.type) !== 'sfu/frame') return", 'publisher frame tracker keys off normalized SFU frame messages');
diff --git a/demo/video-chat/frontend-vue/tests/e2e/call-access-join.spec.js b/demo/video-chat/frontend-vue/tests/e2e/call-access-join.spec.js
new file mode 100644
index 000000000..6478ab16b
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/e2e/call-access-join.spec.js
@@ -0,0 +1,1030 @@
+import { test, expect } from '@playwright/test';
+
+import {
+ adminCredentials,
+ createAuthenticatedPage,
+ createInvitedCallViaApi,
+ createPersonalAccessJoinPath,
+ installMediaDeviceShim,
+} from './helpers/nativeAudioTransferHarness.js';
+
+const sessionStorageKey = 'ii_videocall_v1_session';
+
+function accessIdFromJoinPath(joinPath) {
+ const match = String(joinPath || '').match(/\/join\/([a-f0-9-]{36})(?:[/?#].*)?$/i);
+ return match ? match[1].toLowerCase() : '';
+}
+
+function parseJsonPostData(request) {
+ try {
+ return JSON.parse(request.postData() || '{}');
+ } catch {
+ return null;
+ }
+}
+
+function expectTextDoesNotContain(text, values, label) {
+ const lowerText = String(text || '').toLowerCase();
+ for (const value of values) {
+ const needle = String(value || '').trim().toLowerCase();
+ if (needle === '') continue;
+ expect(lowerText, `${label} must not contain ${value}`).not.toContain(needle);
+ }
+}
+
+async function createPublicJoinPage(browser, baseURL) {
+ const context = await browser.newContext({ baseURL, permissions: ['camera', 'microphone'] });
+ await installMediaDeviceShim(context);
+ const page = await context.newPage();
+ return { context, page };
+}
+
+test('personal call-access link starts a call-scoped session and waits for host admission', async ({ browser }) => {
+ test.setTimeout(90_000);
+ const baseURL = test.info().project.use.baseURL || 'http://127.0.0.1:4174';
+ const participantUserId = 2;
+ const callTitle = `E2E Call Access ${Date.now()}`;
+
+ const { context: adminContext, storedSession: adminSession } = await createAuthenticatedPage(
+ browser,
+ baseURL,
+ adminCredentials,
+ );
+ const { context: publicContext, page } = await createPublicJoinPage(browser, baseURL);
+
+ try {
+ const callId = await createInvitedCallViaApi({
+ sessionToken: adminSession.sessionToken,
+ title: callTitle,
+ participantUserId,
+ });
+ const joinPath = await createPersonalAccessJoinPath({
+ callId,
+ sessionToken: adminSession.sessionToken,
+ participantUserId,
+ });
+ const accessId = accessIdFromJoinPath(joinPath);
+ expect(accessId, 'join path must contain the backend-issued access id').not.toBe('');
+
+ const joinResponsePromise = page.waitForResponse((response) => (
+ response.url().includes(`/api/call-access/${accessId}/join`)
+ && response.request().method() === 'GET'
+ ));
+ await page.goto(joinPath);
+ const joinResponse = await joinResponsePromise;
+ expect(joinResponse.status()).toBe(200);
+ const joinPayload = await joinResponse.json();
+ expect(joinPayload?.status).toBe('ok');
+ expect(joinPayload?.result?.link_kind).toBe('personal');
+ expect(joinPayload?.result?.call?.id).toBe(callId);
+
+ const joinDialog = page.getByRole('dialog', { name: 'Join video call' });
+ await expect(joinDialog).toBeVisible({ timeout: 20_000 });
+ await expect(joinDialog).toContainText(callTitle);
+ await expect(joinDialog).toContainText('Personalized link');
+
+ const sessionResponsePromise = page.waitForResponse((response) => (
+ response.url().includes(`/api/call-access/${accessId}/session`)
+ && response.request().method() === 'POST'
+ ));
+ await joinDialog.getByRole('button', { name: /^Join call$/ }).click();
+ const sessionResponse = await sessionResponsePromise;
+ expect(sessionResponse.status()).toBe(200);
+ const sessionPayload = await sessionResponse.json();
+ expect(sessionPayload?.status).toBe('ok');
+ expect(sessionPayload?.result?.user?.id).toBe(participantUserId);
+ expect(sessionPayload?.result?.call?.id).toBe(callId);
+ expect(sessionPayload?.result?.tenant?.permissions?.tenant_admin ?? false).toBe(false);
+
+ await expect(joinDialog).toContainText(/Call owner has been notified|Waiting for host/i, { timeout: 20_000 });
+
+ const storedSession = await page.evaluate((key) => {
+ try {
+ return JSON.parse(localStorage.getItem(key) || '{}');
+ } catch {
+ return {};
+ }
+ }, sessionStorageKey);
+ expect(storedSession.sessionToken).toBe(sessionPayload?.result?.session?.token);
+ expect(storedSession.sessionId).toBe(sessionPayload?.result?.session?.id);
+ } finally {
+ await Promise.allSettled([
+ adminContext.close(),
+ publicContext.close(),
+ ]);
+ }
+});
+
+test('invalid call-access link renders safe state without foreign call data', async ({ browser }) => {
+ const baseURL = test.info().project.use.baseURL || 'http://127.0.0.1:4174';
+ const foreignTitle = `Private Foreign Call ${Date.now()}`;
+ const foreignEmail = `private-${Date.now()}@example.invalid`;
+ const guessedAccessId = '11111111-1111-4111-8111-111111111111';
+
+ const { context, page } = await createPublicJoinPage(browser, baseURL);
+
+ try {
+ await page.route('**/api/call-access/*/join', async (route) => {
+ await route.fulfill({
+ status: 404,
+ contentType: 'application/json',
+ body: JSON.stringify({
+ status: 'error',
+ error: {
+ code: 'call_access_not_found',
+ message: `No access for ${foreignEmail}`,
+ },
+ result: {
+ call: {
+ id: 'foreign-call-id',
+ title: foreignTitle,
+ owner: {
+ email: foreignEmail,
+ name: 'Private Owner',
+ },
+ },
+ },
+ }),
+ });
+ });
+
+ await page.goto(`/join/${guessedAccessId}`);
+
+ const joinDialog = page.getByRole('dialog', { name: 'Join video call' });
+ await expect(joinDialog).toBeVisible();
+ await expect(joinDialog).toContainText(/call link is invalid|call access id is invalid/i);
+ await expect(joinDialog).not.toContainText(foreignTitle);
+ await expect(joinDialog).not.toContainText(foreignEmail);
+ await expect(joinDialog.getByRole('button', { name: /^Join call$/ })).toHaveCount(0);
+ } finally {
+ await context.close();
+ }
+});
+
+test('login switch after verified call-access link fails without rebinding or leaking foreign data', async ({ browser }) => {
+ const baseURL = test.info().project.use.baseURL || 'http://127.0.0.1:4174';
+ const accessId = '22222222-2222-4222-8222-222222222222';
+ const callId = 'call-access-login-switch-call';
+ const callTitle = 'Verified Link Call';
+ const foreignTitle = 'Foreign Switched Account Call';
+ const foreignEmail = 'foreign-switch@example.invalid';
+ const rejectedCallAccessToken = 'sess_foreign_call_access_should_not_bind';
+ const verifiedSession = {
+ sessionId: 'sess_verified_standard',
+ sessionToken: 'sess_verified_standard',
+ expiresAt: '2026-09-01T10:00:00Z',
+ };
+ const switchedSession = {
+ sessionId: 'sess_current_admin',
+ sessionToken: 'sess_current_admin',
+ expiresAt: '2026-09-01T10:00:00Z',
+ };
+
+ const { context, page } = await createPublicJoinPage(browser, baseURL);
+ let sessionStateRequestAuthorization = '';
+ let joinGetCount = 0;
+ let sessionPostCount = 0;
+ let sessionRequestAuthorization = '';
+ let sessionRequestBody = null;
+
+ try {
+ await context.addInitScript(({ key, session }) => {
+ localStorage.setItem(key, JSON.stringify(session));
+ }, { key: sessionStorageKey, session: verifiedSession });
+
+ await page.route('**/api/auth/session-state', async (route) => {
+ sessionStateRequestAuthorization = route.request().headers().authorization || '';
+ await route.fulfill({
+ status: 200,
+ contentType: 'application/json',
+ body: JSON.stringify({
+ status: 'ok',
+ result: { state: 'authenticated' },
+ session: {
+ id: verifiedSession.sessionId,
+ token: verifiedSession.sessionToken,
+ expires_at: verifiedSession.expiresAt,
+ },
+ user: {
+ id: 2,
+ email: 'user@intelligent-intern.com',
+ display_name: 'Standard Verified User',
+ role: 'user',
+ status: 'active',
+ },
+ tenant: {
+ id: 1,
+ uuid: 'tenant-1',
+ label: 'Intelligent Intern',
+ role: 'member',
+ permissions: { tenant_admin: false },
+ },
+ }),
+ });
+ });
+
+ await page.route(`**/api/call-access/${accessId}/join`, async (route) => {
+ joinGetCount += 1;
+ await route.fulfill({
+ status: 200,
+ contentType: 'application/json',
+ body: JSON.stringify({
+ status: 'ok',
+ result: {
+ link_kind: 'personal',
+ call: {
+ id: callId,
+ room_id: 'lobby',
+ title: callTitle,
+ },
+ access_link: {
+ id: accessId,
+ target_user_id: 2,
+ },
+ },
+ }),
+ });
+ });
+
+ await page.route(`**/api/call-access/${accessId}/session`, async (route) => {
+ sessionPostCount += 1;
+ sessionRequestAuthorization = route.request().headers().authorization || '';
+ sessionRequestBody = parseJsonPostData(route.request());
+ await route.fulfill({
+ status: 409,
+ contentType: 'application/json',
+ body: JSON.stringify({
+ status: 'error',
+ error: {
+ code: 'call_access_conflict',
+ message: `Rejected switched account for ${foreignEmail}`,
+ },
+ result: {
+ session: {
+ id: rejectedCallAccessToken,
+ token: rejectedCallAccessToken,
+ expires_at: '2026-09-01T10:05:00Z',
+ },
+ user: {
+ id: 99,
+ email: foreignEmail,
+ display_name: 'Foreign Switched User',
+ role: 'user',
+ },
+ call: {
+ id: 'foreign-call-id',
+ title: foreignTitle,
+ },
+ },
+ }),
+ });
+ });
+
+ const joinResponsePromise = page.waitForResponse((response) => (
+ response.url().includes(`/api/call-access/${accessId}/join`)
+ && response.request().method() === 'GET'
+ ));
+ await page.goto(`/join/${accessId}`);
+ const joinResponse = await joinResponsePromise;
+ expect(joinResponse.status()).toBe(200);
+ expect(sessionStateRequestAuthorization).toBe(`Bearer ${verifiedSession.sessionToken}`);
+
+ const joinDialog = page.getByRole('dialog', { name: 'Join video call' });
+ await expect(joinDialog).toBeVisible({ timeout: 20_000 });
+ await expect(joinDialog).toContainText(callTitle);
+ await expect(joinDialog).toContainText('Personalized link');
+
+ const switchedSnapshot = await page.evaluate(async ({ key, session }) => {
+ const { sessionState } = await import('/src/domain/auth/session.ts');
+ sessionState.role = 'admin';
+ sessionState.displayName = 'Switched Admin';
+ sessionState.email = 'admin@intelligent-intern.com';
+ sessionState.userId = 1;
+ sessionState.accountType = 'account';
+ sessionState.isGuest = false;
+ sessionState.sessionId = session.sessionId;
+ sessionState.sessionToken = session.sessionToken;
+ sessionState.expiresAt = session.expiresAt;
+ sessionState.recovered = true;
+ localStorage.setItem(key, JSON.stringify(session));
+ return {
+ userId: sessionState.userId,
+ sessionId: sessionState.sessionId,
+ sessionToken: sessionState.sessionToken,
+ };
+ }, { key: sessionStorageKey, session: switchedSession });
+ expect(switchedSnapshot).toEqual({
+ userId: 1,
+ sessionId: switchedSession.sessionId,
+ sessionToken: switchedSession.sessionToken,
+ });
+
+ const sessionResponsePromise = page.waitForResponse((response) => (
+ response.url().includes(`/api/call-access/${accessId}/session`)
+ && response.request().method() === 'POST'
+ ));
+ await joinDialog.getByRole('button', { name: /^Join call$/ }).click();
+ const sessionResponse = await sessionResponsePromise;
+ expect(sessionResponse.status()).toBe(409);
+ const sessionPayload = await sessionResponse.json();
+ expect(sessionPayload?.error?.code).toBe('call_access_conflict');
+
+ expect(sessionPostCount).toBe(1);
+ expect(sessionRequestAuthorization).toBe(`Bearer ${switchedSession.sessionToken}`);
+ expect(sessionRequestBody).toEqual({
+ verified_user_id: 2,
+ verified_session_id: verifiedSession.sessionId,
+ });
+
+ await expect(joinDialog).toContainText('This call link cannot be used for the current call state.');
+ await expect(joinDialog).not.toContainText(foreignTitle);
+ await expect(joinDialog).not.toContainText(foreignEmail);
+ await expect(joinDialog).not.toContainText(rejectedCallAccessToken);
+
+ const storedSession = await page.evaluate((key) => {
+ try {
+ return JSON.parse(localStorage.getItem(key) || '{}');
+ } catch {
+ return {};
+ }
+ }, sessionStorageKey);
+ expect(storedSession.sessionId).toBe(switchedSession.sessionId);
+ expect(storedSession.sessionToken).toBe(switchedSession.sessionToken);
+ expect(storedSession.sessionToken).not.toBe(rejectedCallAccessToken);
+ expect(page.url()).toContain(`/join/${accessId}`);
+
+ await page.waitForTimeout(300);
+ expect(joinGetCount).toBe(1);
+ expect(sessionPostCount).toBe(1);
+ } finally {
+ await context.close();
+ }
+});
+
+test('logout during verified call-access link context fails closed without leaking or joining', async ({ browser }) => {
+ const baseURL = test.info().project.use.baseURL || 'http://127.0.0.1:4174';
+ const accessId = '55555555-5555-4555-8555-555555555555';
+ const callId = 'call-access-logout-verified-call';
+ const safeCallTitle = 'Verified Logout Link Call';
+ const foreignTitle = 'Foreign Logout Session Call';
+ const foreignInviteEmail = 'foreign-logout-invitee@example.invalid';
+ const foreignHostName = 'Private Logout Host';
+ const foreignHostEmail = 'private-logout-host@example.invalid';
+ const rejectedSessionToken = 'sess_logout_foreign_should_not_bind';
+ const verifiedSession = {
+ sessionId: 'sess_verified_before_logout',
+ sessionToken: 'sess_verified_before_logout',
+ expiresAt: '2026-09-01T10:00:00Z',
+ };
+ const foreignNeedles = [
+ foreignTitle,
+ foreignInviteEmail,
+ foreignHostName,
+ foreignHostEmail,
+ rejectedSessionToken,
+ ];
+
+ const { context, page } = await createPublicJoinPage(browser, baseURL);
+ let sessionStateRequestAuthorization = '';
+ let joinGetCount = 0;
+ let sessionPostCount = 0;
+ let logoutPostCount = 0;
+ const navigations = [];
+
+ try {
+ await context.addInitScript(({ key, session }) => {
+ localStorage.setItem(key, JSON.stringify(session));
+ }, { key: sessionStorageKey, session: verifiedSession });
+
+ page.on('framenavigated', (frame) => {
+ if (frame === page.mainFrame()) navigations.push(frame.url());
+ });
+
+ await page.route('**/api/auth/session-state', async (route) => {
+ sessionStateRequestAuthorization = route.request().headers().authorization || '';
+ await route.fulfill({
+ status: 200,
+ contentType: 'application/json',
+ body: JSON.stringify({
+ status: 'ok',
+ result: { state: 'authenticated' },
+ session: {
+ id: verifiedSession.sessionId,
+ token: verifiedSession.sessionToken,
+ expires_at: verifiedSession.expiresAt,
+ },
+ user: {
+ id: 2,
+ email: 'verified-logout-user@example.invalid',
+ display_name: 'Verified Logout User',
+ role: 'user',
+ status: 'active',
+ },
+ tenant: {
+ id: 1,
+ uuid: 'tenant-1',
+ label: 'Intelligent Intern',
+ role: 'member',
+ permissions: { tenant_admin: false },
+ },
+ }),
+ });
+ });
+
+ await page.route('**/api/auth/logout', async (route) => {
+ logoutPostCount += 1;
+ await route.fulfill({
+ status: 200,
+ contentType: 'application/json',
+ body: JSON.stringify({ status: 'ok', result: { post_logout_landing_url: '' } }),
+ });
+ });
+
+ await page.route(`**/api/call-access/${accessId}/join`, async (route) => {
+ joinGetCount += 1;
+ await route.fulfill({
+ status: 200,
+ contentType: 'application/json',
+ body: JSON.stringify({
+ status: 'ok',
+ result: {
+ state: 'resolved',
+ access_link: { id: accessId, target_user_id: 2 },
+ link_kind: 'personal',
+ call: {
+ id: callId,
+ room_id: 'lobby',
+ title: safeCallTitle,
+ },
+ target_hint: { participant_email: null },
+ join_path: `/join/${accessId}`,
+ },
+ }),
+ });
+ });
+
+ await page.route(`**/api/call-access/${accessId}/session`, async (route) => {
+ sessionPostCount += 1;
+ await route.fulfill({
+ status: 200,
+ contentType: 'application/json',
+ body: JSON.stringify({
+ status: 'ok',
+ result: {
+ session: {
+ id: rejectedSessionToken,
+ token: rejectedSessionToken,
+ expires_at: '2026-09-01T10:05:00Z',
+ },
+ user: {
+ id: 99,
+ email: foreignInviteEmail,
+ display_name: 'Foreign Logout Invitee',
+ role: 'user',
+ },
+ call: {
+ id: 'foreign-logout-call-id',
+ title: foreignTitle,
+ owner: {
+ display_name: foreignHostName,
+ email: foreignHostEmail,
+ },
+ },
+ },
+ }),
+ });
+ });
+
+ const joinResponsePromise = page.waitForResponse((response) => (
+ response.url().includes(`/api/call-access/${accessId}/join`)
+ && response.request().method() === 'GET'
+ ));
+ await page.goto(`/join/${accessId}`);
+ const joinResponse = await joinResponsePromise;
+ expect(joinResponse.status()).toBe(200);
+ expect(sessionStateRequestAuthorization).toBe(`Bearer ${verifiedSession.sessionToken}`);
+
+ const joinDialog = page.getByRole('dialog', { name: 'Join video call' });
+ await expect(joinDialog).toBeVisible({ timeout: 20_000 });
+ await expect(joinDialog).toContainText(safeCallTitle);
+ await expect(joinDialog).toContainText('Personalized link');
+
+ const logoutResult = await page.evaluate(async () => {
+ const { logoutSession, sessionState } = await import('/src/domain/auth/session.ts');
+ const result = await logoutSession();
+ return {
+ result,
+ userId: sessionState.userId,
+ sessionId: sessionState.sessionId,
+ sessionToken: sessionState.sessionToken,
+ };
+ });
+ expect(logoutResult.sessionId).toBe('');
+ expect(logoutResult.sessionToken).toBe('');
+ expect(logoutResult.userId).toBe(0);
+ expect(logoutPostCount).toBe(1);
+
+ await joinDialog.getByRole('button', { name: /^Join call$/ }).click();
+ await expect(joinDialog).toContainText('This call link cannot be used for the current call state.');
+ await page.waitForTimeout(300);
+
+ expect(sessionPostCount).toBe(0);
+ expect(joinGetCount).toBe(1);
+ expect(page.url()).toContain(`/join/${accessId}`);
+ expect(page.url()).not.toContain('/workspace/call');
+ expect(navigations.filter((url) => url.includes('/workspace/call'))).toEqual([]);
+
+ for (const value of foreignNeedles) {
+ await expect(joinDialog, `logout denial must not render ${value}`).not.toContainText(value);
+ }
+
+ const storedSession = await page.evaluate((key) => {
+ try {
+ return JSON.parse(localStorage.getItem(key) || '{}');
+ } catch {
+ return {};
+ }
+ }, sessionStorageKey);
+ expect(storedSession.sessionId || '').toBe('');
+ expect(storedSession.sessionToken || '').toBe('');
+ expect(JSON.stringify(storedSession)).not.toContain(rejectedSessionToken);
+ } finally {
+ await context.close();
+ }
+});
+
+test('same personalized link in parallel contexts keeps account sessions isolated', async ({ browser }) => {
+ const baseURL = test.info().project.use.baseURL || 'http://127.0.0.1:4174';
+ const accessId = '44444444-4444-4444-8444-444444444444';
+ const callId = 'parallel-account-isolation-call';
+ const callTitle = 'Parallel Account Isolation Call';
+ const accountA = {
+ userId: 2,
+ email: 'parallel-a@example.invalid',
+ displayName: 'Parallel Account A',
+ sessionId: 'sess_parallel_account_a',
+ sessionToken: 'sess_parallel_account_a',
+ issuedCallAccessToken: 'sess_call_access_account_a',
+ };
+ const accountB = {
+ userId: 3,
+ email: 'parallel-b@example.invalid',
+ displayName: 'Parallel Account B',
+ sessionId: 'sess_parallel_account_b',
+ sessionToken: 'sess_parallel_account_b',
+ rejectedCallAccessToken: 'sess_call_access_account_b_rejected',
+ };
+ const foreignNeedlesForA = [
+ accountB.email,
+ accountB.displayName,
+ accountB.sessionId,
+ accountB.sessionToken,
+ accountB.rejectedCallAccessToken,
+ ];
+ const foreignNeedlesForB = [
+ accountA.email,
+ accountA.displayName,
+ accountA.sessionId,
+ accountA.sessionToken,
+ accountA.issuedCallAccessToken,
+ ];
+
+ const accountAPage = await createPublicJoinPage(browser, baseURL);
+ const accountBPage = await createPublicJoinPage(browser, baseURL);
+ const requests = {
+ a: {
+ sessionStateAuthorization: '',
+ sessionAuthorization: '',
+ sessionBody: null,
+ joinGetCount: 0,
+ sessionPostCount: 0,
+ },
+ b: {
+ sessionStateAuthorization: '',
+ sessionAuthorization: '',
+ sessionBody: null,
+ joinGetCount: 0,
+ sessionPostCount: 0,
+ },
+ };
+
+ async function seedStoredSession(context, account) {
+ await context.addInitScript(({ key, session }) => {
+ localStorage.setItem(key, JSON.stringify(session));
+ }, {
+ key: sessionStorageKey,
+ session: {
+ sessionId: account.sessionId,
+ sessionToken: account.sessionToken,
+ expiresAt: '2026-09-01T10:00:00Z',
+ },
+ });
+ }
+
+ async function installRoutes(page, account, requestLog, { acceptSession }) {
+ await page.route('**/api/auth/session-state', async (route) => {
+ requestLog.sessionStateAuthorization = route.request().headers().authorization || '';
+ await route.fulfill({
+ status: 200,
+ contentType: 'application/json',
+ body: JSON.stringify({
+ status: 'ok',
+ result: { state: 'authenticated' },
+ session: {
+ id: account.sessionId,
+ token: account.sessionToken,
+ expires_at: '2026-09-01T10:00:00Z',
+ },
+ user: {
+ id: account.userId,
+ email: account.email,
+ display_name: account.displayName,
+ role: 'user',
+ status: 'active',
+ },
+ tenant: {
+ id: 1,
+ uuid: 'tenant-1',
+ label: 'Intelligent Intern',
+ role: 'member',
+ permissions: { tenant_admin: false },
+ },
+ }),
+ });
+ });
+
+ await page.route(`**/api/call-access/${accessId}/join`, async (route) => {
+ requestLog.joinGetCount += 1;
+ await route.fulfill({
+ status: 200,
+ contentType: 'application/json',
+ body: JSON.stringify({
+ status: 'ok',
+ result: {
+ link_kind: 'personal',
+ call: {
+ id: callId,
+ room_id: 'lobby',
+ title: callTitle,
+ },
+ access_link: {
+ id: accessId,
+ target_user_id: accountA.userId,
+ },
+ },
+ }),
+ });
+ });
+
+ await page.route(`**/api/call-access/${accessId}/session`, async (route) => {
+ requestLog.sessionPostCount += 1;
+ requestLog.sessionAuthorization = route.request().headers().authorization || '';
+ requestLog.sessionBody = parseJsonPostData(route.request());
+ if (acceptSession) {
+ await route.fulfill({
+ status: 200,
+ contentType: 'application/json',
+ body: JSON.stringify({
+ status: 'ok',
+ result: {
+ session: {
+ id: accountA.issuedCallAccessToken,
+ token: accountA.issuedCallAccessToken,
+ expires_at: '2026-09-01T10:05:00Z',
+ },
+ user: {
+ id: accountA.userId,
+ email: accountA.email,
+ display_name: accountA.displayName,
+ role: 'user',
+ status: 'active',
+ },
+ tenant: {
+ id: 1,
+ uuid: 'tenant-1',
+ label: 'Intelligent Intern',
+ role: 'member',
+ permissions: { tenant_admin: false },
+ },
+ call: {
+ id: callId,
+ room_id: 'lobby',
+ title: callTitle,
+ },
+ },
+ }),
+ });
+ return;
+ }
+
+ await route.fulfill({
+ status: 409,
+ contentType: 'application/json',
+ body: JSON.stringify({
+ status: 'error',
+ error: {
+ code: 'call_access_conflict',
+ message: `Duplicate personalized link use by ${accountA.email}`,
+ },
+ result: {
+ session: {
+ id: accountB.rejectedCallAccessToken,
+ token: accountB.rejectedCallAccessToken,
+ expires_at: '2026-09-01T10:05:00Z',
+ },
+ user: {
+ id: accountA.userId,
+ email: accountA.email,
+ display_name: accountA.displayName,
+ role: 'user',
+ },
+ call: {
+ id: 'foreign-linked-call',
+ title: 'Foreign Linked Call Title',
+ },
+ },
+ }),
+ });
+ });
+ }
+
+ try {
+ await Promise.all([
+ seedStoredSession(accountAPage.context, accountA),
+ seedStoredSession(accountBPage.context, accountB),
+ ]);
+ await Promise.all([
+ installRoutes(accountAPage.page, accountA, requests.a, { acceptSession: true }),
+ installRoutes(accountBPage.page, accountB, requests.b, { acceptSession: false }),
+ ]);
+
+ const joinResponseA = accountAPage.page.waitForResponse((response) => (
+ response.url().includes(`/api/call-access/${accessId}/join`)
+ && response.request().method() === 'GET'
+ ));
+ const joinResponseB = accountBPage.page.waitForResponse((response) => (
+ response.url().includes(`/api/call-access/${accessId}/join`)
+ && response.request().method() === 'GET'
+ ));
+ await Promise.all([
+ accountAPage.page.goto(`/join/${accessId}`),
+ accountBPage.page.goto(`/join/${accessId}`),
+ ]);
+ expect((await joinResponseA).status()).toBe(200);
+ expect((await joinResponseB).status()).toBe(200);
+ expect(requests.a.sessionStateAuthorization).toBe(`Bearer ${accountA.sessionToken}`);
+ expect(requests.b.sessionStateAuthorization).toBe(`Bearer ${accountB.sessionToken}`);
+
+ const dialogA = accountAPage.page.getByRole('dialog', { name: 'Join video call' });
+ const dialogB = accountBPage.page.getByRole('dialog', { name: 'Join video call' });
+ await expect(dialogA).toBeVisible({ timeout: 20_000 });
+ await expect(dialogB).toBeVisible({ timeout: 20_000 });
+ await expect(dialogA).toContainText(callTitle);
+ await expect(dialogB).toContainText(callTitle);
+ for (const value of foreignNeedlesForA) {
+ await expect(dialogA, `account A dialog must not render ${value}`).not.toContainText(value);
+ }
+ for (const value of foreignNeedlesForB) {
+ await expect(dialogB, `account B dialog must not render ${value}`).not.toContainText(value);
+ }
+
+ const sessionResponseA = accountAPage.page.waitForResponse((response) => (
+ response.url().includes(`/api/call-access/${accessId}/session`)
+ && response.request().method() === 'POST'
+ ));
+ const sessionResponseB = accountBPage.page.waitForResponse((response) => (
+ response.url().includes(`/api/call-access/${accessId}/session`)
+ && response.request().method() === 'POST'
+ ));
+ await Promise.all([
+ dialogA.getByRole('button', { name: /^Join call$/ }).click(),
+ dialogB.getByRole('button', { name: /^Join call$/ }).click(),
+ ]);
+ const [responseA, responseB] = await Promise.all([sessionResponseA, sessionResponseB]);
+ expect(responseA.status()).toBe(200);
+ expect(responseB.status()).toBe(409);
+
+ expect(requests.a.sessionAuthorization).toBe(`Bearer ${accountA.sessionToken}`);
+ expect(requests.b.sessionAuthorization).toBe(`Bearer ${accountB.sessionToken}`);
+ expect(requests.a.sessionBody).toEqual({
+ verified_user_id: accountA.userId,
+ verified_session_id: accountA.sessionId,
+ });
+ expect(requests.b.sessionBody).toEqual({
+ verified_user_id: accountB.userId,
+ verified_session_id: accountB.sessionId,
+ });
+
+ await expect(dialogA).toContainText(/Call owner has been notified|Waiting for host/i, { timeout: 20_000 });
+ await expect(dialogB).toContainText('This call link cannot be used for the current call state.');
+ await expect(dialogB).not.toContainText('Foreign Linked Call Title');
+ for (const value of foreignNeedlesForB) {
+ await expect(dialogB, `account B conflict must not render ${value}`).not.toContainText(value);
+ }
+
+ const [storedA, storedB] = await Promise.all([
+ accountAPage.page.evaluate((key) => JSON.parse(localStorage.getItem(key) || '{}'), sessionStorageKey),
+ accountBPage.page.evaluate((key) => JSON.parse(localStorage.getItem(key) || '{}'), sessionStorageKey),
+ ]);
+ expect(storedA.sessionId).toBe(accountA.issuedCallAccessToken);
+ expect(storedA.sessionToken).toBe(accountA.issuedCallAccessToken);
+ expect(storedB.sessionId).toBe(accountB.sessionId);
+ expect(storedB.sessionToken).toBe(accountB.sessionToken);
+ expect(storedB.sessionToken).not.toBe(accountA.issuedCallAccessToken);
+ expect(storedB.sessionToken).not.toBe(accountB.rejectedCallAccessToken);
+
+ expect(accountAPage.page.url()).toContain(`/join/${accessId}`);
+ expect(accountBPage.page.url()).toContain(`/join/${accessId}`);
+ expect(accountBPage.page.url()).not.toContain('/workspace/call');
+ expect(requests.a.joinGetCount).toBe(1);
+ expect(requests.b.joinGetCount).toBe(1);
+ expect(requests.a.sessionPostCount).toBe(1);
+ expect(requests.b.sessionPostCount).toBe(1);
+ } finally {
+ await Promise.allSettled([
+ accountAPage.context.close(),
+ accountBPage.context.close(),
+ ]);
+ }
+});
+
+test('strong personalized-link mismatch wrong host denial gives no access and leaks no foreign person data', async ({ browser }) => {
+ const baseURL = test.info().project.use.baseURL || 'http://127.0.0.1:4174';
+ const accessId = '33333333-3333-4333-8333-333333333333';
+ const callId = 'strong-mismatch-call';
+ const safeCallTitle = 'Strong Mismatch Waiting Room';
+ const wrongHostName = 'Definitely Wrong Host';
+ const linkInviteeName = 'Foreign Link Invitee';
+ const linkInviteeEmail = 'foreign-link-invitee@example.invalid';
+ const realHostName = 'Private Foreign Host';
+ const realHostEmail = 'private-host@example.invalid';
+ const deniedSessionToken = 'sess_denied_strong_mismatch_should_not_bind';
+ const wrongLoggedInUserId = 3;
+ const wrongLoggedInSession = {
+ sessionId: 'sess_wrong_logged_in_user',
+ sessionToken: 'sess_wrong_logged_in_user',
+ expiresAt: '2026-09-01T10:00:00Z',
+ };
+ const foreignNeedles = [
+ linkInviteeName,
+ linkInviteeEmail,
+ realHostName,
+ realHostEmail,
+ deniedSessionToken,
+ ];
+
+ const { context, page } = await createPublicJoinPage(browser, baseURL);
+ let sessionStateRequestAuthorization = '';
+ let joinGetCount = 0;
+ let sessionPostCount = 0;
+ let sessionRequestAuthorization = '';
+ let sessionRequestBody = null;
+
+ try {
+ await context.addInitScript(({ key, session }) => {
+ localStorage.setItem(key, JSON.stringify(session));
+ }, { key: sessionStorageKey, session: wrongLoggedInSession });
+
+ await page.route('**/api/auth/session-state', async (route) => {
+ sessionStateRequestAuthorization = route.request().headers().authorization || '';
+ await route.fulfill({
+ status: 200,
+ contentType: 'application/json',
+ body: JSON.stringify({
+ status: 'ok',
+ result: { state: 'authenticated' },
+ session: {
+ id: wrongLoggedInSession.sessionId,
+ token: wrongLoggedInSession.sessionToken,
+ expires_at: wrongLoggedInSession.expiresAt,
+ },
+ user: {
+ id: wrongLoggedInUserId,
+ email: 'wrong-current-user@example.invalid',
+ display_name: 'Wrong Current User',
+ role: 'user',
+ status: 'active',
+ },
+ tenant: {
+ id: 1,
+ uuid: 'tenant-1',
+ label: 'Intelligent Intern',
+ role: 'member',
+ permissions: { tenant_admin: false },
+ },
+ }),
+ });
+ });
+
+ await page.route(`**/api/call-access/${accessId}/join`, async (route) => {
+ joinGetCount += 1;
+ await route.fulfill({
+ status: 200,
+ contentType: 'application/json',
+ body: JSON.stringify({
+ status: 'ok',
+ result: {
+ state: 'resolved',
+ access_link: { id: accessId },
+ link_kind: 'personal',
+ call: {
+ id: callId,
+ room_id: 'lobby',
+ title: safeCallTitle,
+ },
+ target_hint: { participant_email: null },
+ join_path: `/join/${accessId}`,
+ },
+ }),
+ });
+ });
+
+ await page.route(`**/api/call-access/${accessId}/session`, async (route) => {
+ sessionPostCount += 1;
+ sessionRequestAuthorization = route.request().headers().authorization || '';
+ sessionRequestBody = parseJsonPostData(route.request());
+ await route.fulfill({
+ status: 403,
+ contentType: 'application/json',
+ body: JSON.stringify({
+ status: 'error',
+ error: {
+ code: 'call_access_forbidden',
+ message: 'Call access link is not available for your session.',
+ details: {
+ mismatch: 'strong_personalized_link',
+ fields: {
+ host_name: 'wrong_host_name',
+ },
+ },
+ },
+ }),
+ });
+ });
+
+ const joinResponsePromise = page.waitForResponse((response) => (
+ response.url().includes(`/api/call-access/${accessId}/join`)
+ && response.request().method() === 'GET'
+ ));
+ await page.goto(`/join/${accessId}`);
+ const joinResponse = await joinResponsePromise;
+ expect(joinResponse.status()).toBe(200);
+ expect(sessionStateRequestAuthorization).toBe(`Bearer ${wrongLoggedInSession.sessionToken}`);
+ const joinBody = await joinResponse.text();
+ expectTextDoesNotContain(joinBody, foreignNeedles, 'strong-mismatch join response');
+
+ const joinDialog = page.getByRole('dialog', { name: 'Join video call' });
+ await expect(joinDialog).toBeVisible({ timeout: 20_000 });
+ await expect(joinDialog).toContainText(safeCallTitle);
+ await expect(joinDialog).toContainText('Personalized link');
+ for (const value of foreignNeedles) {
+ await expect(joinDialog, `dialog must not render ${value}`).not.toContainText(value);
+ }
+
+ const sessionResponsePromise = page.waitForResponse((response) => (
+ response.url().includes(`/api/call-access/${accessId}/session`)
+ && response.request().method() === 'POST'
+ ));
+ await joinDialog.getByRole('button', { name: /^Join call$/ }).click();
+ const sessionResponse = await sessionResponsePromise;
+ expect(sessionResponse.status()).toBe(403);
+ const sessionBody = await sessionResponse.text();
+ expectTextDoesNotContain(sessionBody, foreignNeedles, 'strong-mismatch wrong-host denial response');
+ const sessionPayload = JSON.parse(sessionBody);
+ expect(sessionPayload?.error?.code).toBe('call_access_forbidden');
+ expect(sessionPayload?.error?.details?.mismatch).toBe('strong_personalized_link');
+ expect(sessionPayload?.error?.details?.fields?.host_name).toBe('wrong_host_name');
+
+ expect(sessionPostCount).toBe(1);
+ expect(sessionRequestAuthorization).toBe(`Bearer ${wrongLoggedInSession.sessionToken}`);
+ expect(sessionRequestBody).toEqual({
+ verified_user_id: wrongLoggedInUserId,
+ verified_session_id: wrongLoggedInSession.sessionId,
+ });
+
+ await expect(joinDialog).toContainText('This call link is not available for your session.');
+ await expect(joinDialog).not.toContainText(/Call owner has been notified|Waiting for host/i);
+ for (const value of [...foreignNeedles, wrongHostName]) {
+ await expect(joinDialog, `dialog denial must not render ${value}`).not.toContainText(value);
+ }
+ expect(page.url()).toContain(`/join/${accessId}`);
+ expect(page.url()).not.toContain('/workspace/call');
+
+ const storedSession = await page.evaluate((key) => {
+ try {
+ return JSON.parse(localStorage.getItem(key) || '{}');
+ } catch {
+ return {};
+ }
+ }, sessionStorageKey);
+ expect(storedSession.sessionId).toBe(wrongLoggedInSession.sessionId);
+ expect(storedSession.sessionToken).toBe(wrongLoggedInSession.sessionToken);
+ expect(storedSession.sessionToken).not.toBe(deniedSessionToken);
+
+ await page.waitForTimeout(300);
+ expect(joinGetCount).toBe(1);
+ expect(sessionPostCount).toBe(1);
+ } finally {
+ await context.close();
+ }
+});
diff --git a/demo/video-chat/frontend-vue/tests/e2e/call-access-seed-matrix.spec.js b/demo/video-chat/frontend-vue/tests/e2e/call-access-seed-matrix.spec.js
new file mode 100644
index 000000000..7bf312d36
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/e2e/call-access-seed-matrix.spec.js
@@ -0,0 +1,108 @@
+import { test, expect } from '@playwright/test';
+
+import {
+ accessIdFromJoinPath,
+ createCallAccessMatrixPage,
+ getSeedAccessLink,
+ getSeedCall,
+ getSeedScenario,
+ getSeedUser,
+ seedUserKeys,
+ sessionStorageKey,
+ tenantSnapshotForSeedUser,
+} from './helpers/callAccessSeedMatrix.js';
+
+test('IAM call-access seed matrix covers required principals without temporary admin elevation', () => {
+ expect(seedUserKeys()).toEqual(expect.arrayContaining([
+ 'system_admin',
+ 'alpha_org_admin',
+ 'beta_org_admin',
+ 'alpha_call_owner',
+ 'alpha_normal_user',
+ 'registered_guest',
+ 'removed_invited_member',
+ 'temporary_personalized_guest',
+ 'temporary_anonymous_guest',
+ ]));
+
+ const systemAdminScenario = getSeedScenario('system_admin_join_any_organization_call_without_guest_list');
+ expect(systemAdminScenario.call_keys).toEqual(expect.arrayContaining(['alpha_active', 'beta_active', 'tenantless_active']));
+ expect(systemAdminScenario.expected.guest_list_required).toBe(false);
+ expect(systemAdminScenario.expected.can_manage_lobby).toBe(true);
+ expect(systemAdminScenario.expected.platform_admin).toBe(true);
+
+ for (const userKey of ['temporary_personalized_guest', 'temporary_anonymous_guest']) {
+ const user = getSeedUser(userKey);
+ const tenant = tenantSnapshotForSeedUser(userKey, 'alpha_active');
+ expect(user.temporary).toBe(true);
+ expect(user.role).toBe('user');
+ expect(user.system_admin).toBe(false);
+ expect(tenant?.permissions?.platform_admin ?? false).toBe(false);
+ expect(tenant?.permissions?.tenant_admin ?? false).toBe(false);
+ }
+});
+
+test('personal call-access matrix seed starts a call-scoped session and waits for host admission', async ({ browser }) => {
+ test.setTimeout(60_000);
+ const baseURL = test.info().project.use.baseURL || 'http://127.0.0.1:4174';
+ const scenario = getSeedScenario('call_scoped_removed_member_personal_waits_for_host');
+ const link = getSeedAccessLink(scenario.link_key);
+ const call = getSeedCall(link.call_key);
+ const participant = getSeedUser(link.target_user_key);
+ const accessId = accessIdFromJoinPath(link.join_path);
+
+ expect(accessId, 'join path must contain the backend-issued access id').not.toBe('');
+
+ const { context, page } = await createCallAccessMatrixPage(browser, baseURL, {
+ scenarioKey: scenario.key,
+ });
+ try {
+ const joinResponsePromise = page.waitForResponse((response) => (
+ response.url().includes(`/api/call-access/${accessId}/join`)
+ && response.request().method() === 'GET'
+ ));
+ await page.goto(link.join_path);
+ const joinResponse = await joinResponsePromise;
+ expect(joinResponse.status()).toBe(200);
+ const joinPayload = await joinResponse.json();
+ expect(joinPayload?.status).toBe('ok');
+ expect(joinPayload?.result?.link_kind).toBe('personal');
+ expect(joinPayload?.result?.call?.id).toBe(call.id);
+ expect(joinPayload?.result?.target_user?.id).toBe(participant.id);
+
+ const joinDialog = page.getByRole('dialog', { name: 'Join video call' });
+ await expect(joinDialog).toBeVisible({ timeout: 20_000 });
+ await expect(joinDialog).toContainText(call.title);
+ await expect(joinDialog).toContainText('Personalized link');
+
+ const sessionResponsePromise = page.waitForResponse((response) => (
+ response.url().includes(`/api/call-access/${accessId}/session`)
+ && response.request().method() === 'POST'
+ ));
+ await joinDialog.getByRole('button', { name: /^Join call$/ }).click();
+ const sessionResponse = await sessionResponsePromise;
+ expect(sessionResponse.status()).toBe(200);
+ const sessionPayload = await sessionResponse.json();
+ expect(sessionPayload?.status).toBe('ok');
+ expect(sessionPayload?.result?.user?.id).toBe(participant.id);
+ expect(sessionPayload?.result?.call?.id).toBe(call.id);
+ expect(sessionPayload?.result?.tenant?.permissions?.tenant_admin ?? false).toBe(false);
+ expect(JSON.stringify(sessionPayload)).not.toMatch(/\b(?:sdp|ice|candidate|media_token|turn_credential)\b/i);
+
+ await expect(joinDialog).toContainText(/Call owner has been notified|Waiting for host/i, { timeout: 20_000 });
+ const socketFrames = await page.evaluate(() => window.__iamCallAccessSocketFrames || []);
+ expect(socketFrames.some((frame) => frame?.type === 'lobby/queue/join')).toBe(true);
+
+ const storedSession = await page.evaluate((key) => {
+ try {
+ return JSON.parse(localStorage.getItem(key) || '{}');
+ } catch {
+ return {};
+ }
+ }, sessionStorageKey);
+ expect(storedSession.sessionToken).toBe(sessionPayload?.result?.session?.token);
+ expect(storedSession.sessionId).toBe(sessionPayload?.result?.session?.id);
+ } finally {
+ await context.close();
+ }
+});
diff --git a/demo/video-chat/frontend-vue/tests/e2e/call-app-whiteboard-install-sidebar.spec.js b/demo/video-chat/frontend-vue/tests/e2e/call-app-whiteboard-install-sidebar.spec.js
new file mode 100644
index 000000000..bf5438ce9
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/e2e/call-app-whiteboard-install-sidebar.spec.js
@@ -0,0 +1,660 @@
+import { expect, test } from '@playwright/test';
+import { readFile } from 'node:fs/promises';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+const CALL_ID = 'call-whiteboard-install-proof';
+const __dirname = path.dirname(fileURLToPath(import.meta.url));
+const whiteboardPublicRoot = path.resolve(__dirname, '../../../../call-app/whiteboard/public');
+const bridgeProtocol = 'king.call_app.iframe.v1';
+
+async function readWhiteboardAssets() {
+ const [html, css, js] = await Promise.all([
+ readFile(path.join(whiteboardPublicRoot, 'index.html'), 'utf8'),
+ readFile(path.join(whiteboardPublicRoot, 'whiteboard.css'), 'utf8'),
+ readFile(path.join(whiteboardPublicRoot, 'whiteboard.js'), 'utf8'),
+ ]);
+ return { html, css, js };
+}
+
+function jsonResponse(payload, status = 200) {
+ return {
+ status,
+ headers: {
+ 'cache-control': 'no-store',
+ 'content-type': 'application/json',
+ },
+ body: JSON.stringify(payload),
+ };
+}
+
+function hostHtml(baseURL) {
+ const whiteboardEntry = `${baseURL.replace(/\/+$/, '')}/__whiteboard_install_sidebar/whiteboard/index.html`;
+ return `
+
+
+
+ Whiteboard Install Sidebar Proof
+
+
+
+
+
+ Marketplace
+ Whiteboard is not installed.
+ Install for organization
+
+
+
+
+
+
+ `;
+}
+
+async function installRoutes(page, baseURL, assets) {
+ const server = {
+ installed: false,
+ session: null,
+ requests: [],
+ };
+ await page.route('**/__whiteboard_install_sidebar/host.html', async (route) => {
+ await route.fulfill({
+ status: 200,
+ headers: { 'content-type': 'text/html; charset=utf-8' },
+ body: hostHtml(baseURL),
+ });
+ });
+ await page.route('**/__whiteboard_install_sidebar/whiteboard/**', async (route) => {
+ const url = new URL(route.request().url());
+ const pathname = url.pathname;
+ const headers = { 'cache-control': 'no-store' };
+ if (pathname.endsWith('/index.html')) {
+ await route.fulfill({
+ status: 200,
+ headers: { ...headers, 'content-type': 'text/html; charset=utf-8' },
+ body: assets.html,
+ });
+ return;
+ }
+ if (pathname.endsWith('/whiteboard.css')) {
+ await route.fulfill({
+ status: 200,
+ headers: { ...headers, 'content-type': 'text/css; charset=utf-8' },
+ body: assets.css,
+ });
+ return;
+ }
+ if (pathname.endsWith('/whiteboard.js')) {
+ await route.fulfill({
+ status: 200,
+ headers: { ...headers, 'content-type': 'text/javascript; charset=utf-8' },
+ body: assets.js,
+ });
+ return;
+ }
+ await route.fulfill({ status: 404, body: 'missing whiteboard sidebar fixture' });
+ });
+ await page.route('**/api/**', async (route) => {
+ const request = route.request();
+ const url = new URL(request.url());
+ const method = request.method();
+ const body = request.postDataJSON?.() || null;
+ server.requests.push({ method, pathname: url.pathname, body });
+
+ if (method === 'POST' && url.pathname === '/api/marketplace/call-apps/whiteboard/orders') {
+ await route.fulfill(jsonResponse({ status: 'success', result: { order_id: 'order-whiteboard-install-proof' } }, 201));
+ return;
+ }
+ if (method === 'POST' && url.pathname === '/api/marketplace/call-apps/whiteboard/installations') {
+ server.installed = true;
+ await route.fulfill(jsonResponse({ status: 'success', result: { installation_id: 'install-whiteboard-proof', status: 'enabled' } }, 201));
+ return;
+ }
+ if (method === 'GET' && url.pathname === `/api/calls/${CALL_ID}/call-apps/available`) {
+ await route.fulfill(jsonResponse({
+ status: 'success',
+ result: {
+ apps: server.installed ? [{
+ app_key: 'whiteboard',
+ name: 'Whiteboard',
+ category: 'whiteboard',
+ version: '0.1.0',
+ availability: { installed: true, enabled: true, healthy: true },
+ installation: { status: 'enabled', default_app_policy: 'blocked_by_default' },
+ }] : [],
+ pagination: { page: 1, page_count: 1, has_prev: false, has_next: false, total: server.installed ? 1 : 0 },
+ },
+ }));
+ return;
+ }
+ if (method === 'POST' && url.pathname === `/api/calls/${CALL_ID}/call-app-sessions`) {
+ server.session = {
+ id: 'session-whiteboard-install-proof',
+ call_id: CALL_ID,
+ app_key: 'whiteboard',
+ status: 'active',
+ default_app_policy: body.default_app_policy,
+ app: { name: 'Whiteboard', category: 'whiteboard' },
+ grants: [
+ { subject_type: 'user', user_id: 1, grant_state: 'allowed' },
+ { subject_type: 'user', user_id: 2, grant_state: 'allowed' },
+ ],
+ };
+ await route.fulfill(jsonResponse({ status: 'success', result: server.session }, 201));
+ return;
+ }
+ if (method === 'PATCH' && url.pathname === '/api/call-app-sessions/session-whiteboard-install-proof/participant-grants') {
+ const grant = body.grants?.[0] || {};
+ server.session.grants = server.session.grants.map((row) => (
+ Number(row.user_id) === Number(grant.user_id) ? { ...row, grant_state: grant.grant_state } : row
+ ));
+ await route.fulfill(jsonResponse({
+ status: 'success',
+ result: {
+ session: server.session,
+ changed_grants: [{ ...grant, retired_launch_tokens: grant.grant_state === 'denied' ? 1 : 0 }],
+ },
+ }));
+ return;
+ }
+
+ await route.fulfill(jsonResponse({ status: 'error', error: { code: 'unexpected_request' } }, 404));
+ });
+ return server;
+}
+
+test('Whiteboard install appears in Call Apps sidebar with usable access controls', async ({ page }) => {
+ const assets = await readWhiteboardAssets();
+ const baseURL = test.info().project.use.baseURL || 'http://127.0.0.1:4174';
+ const server = await installRoutes(page, baseURL, assets);
+
+ await page.setViewportSize({ width: 360, height: 760 });
+ await page.goto(`${baseURL.replace(/\/+$/, '')}/__whiteboard_install_sidebar/host.html`, { waitUntil: 'domcontentloaded' });
+
+ await page.getByRole('button', { name: 'Install for organization' }).click();
+ await expect(page.getByText('Whiteboard installed and enabled for this organization.')).toBeVisible();
+
+ await page.getByRole('button', { name: 'Call Apps' }).click();
+ const whiteboardRow = page.locator('.call-apps-list-item').filter({ hasText: 'Whiteboard' });
+ await expect(whiteboardRow).toBeVisible();
+ await expect(whiteboardRow.getByText('Installed', { exact: true })).toBeVisible();
+ await expect(whiteboardRow.getByText('Enabled', { exact: true })).toBeVisible();
+ await expect(whiteboardRow.getByText('Healthy', { exact: true })).toBeVisible();
+ await expect(whiteboardRow.getByText('Select', { exact: true })).toBeVisible();
+
+ const sidebarNoOverflow = await page.locator('.sidebar').evaluate((element) => element.scrollWidth <= element.clientWidth);
+ expect(sidebarNoOverflow).toBe(true);
+ const narrowColumns = await page.locator('.call-apps-list-item').evaluate((element) => getComputedStyle(element).gridTemplateColumns);
+ expect(narrowColumns.trim().split(/\s+/)).toHaveLength(1);
+
+ await page.locator('input[name="defaultAccess"][value="allowed_by_default"]').check();
+ await page.getByRole('button', { name: 'Add to call' }).click();
+ await expect(page.getByRole('heading', { name: 'Access' })).toBeVisible();
+ const whiteboardFrame = page.frameLocator('iframe[name="whiteboard"]');
+ await expect(whiteboardFrame.locator('#modeBadge')).toHaveText('Editor');
+ await expect.poll(() => page.evaluate(() => window.whiteboardInstallSidebarProof.state.whiteboardReady)).toBe(true);
+ await page.evaluate(() => window.whiteboardInstallSidebarProof.showRemoteCursor('Owner'));
+ await expect(whiteboardFrame.locator('.remote-cursor-label')).toHaveText('Owner');
+ const launchCountBeforeGrantToggle = await page.evaluate(() => window.whiteboardInstallSidebarProof.state.whiteboardLaunchCount);
+ const frameSrcBeforeGrantToggle = await page.evaluate(() => window.whiteboardInstallSidebarProof.state.whiteboardFrameSrc);
+ await expect(page.getByText('Default: allowed')).toBeVisible();
+ await expect(page.locator('.call-apps-access-row[data-user-id="1"]')).toContainText('Owner');
+ await expect(page.locator('.call-apps-access-row[data-user-id="1"]')).toContainText('Allowed');
+ await expect(page.locator('.call-apps-access-row[data-user-id="2"]')).toContainText('Participant');
+ await expect(page.locator('.call-apps-access-row[data-user-id="2"]')).toContainText('Revoke');
+
+ await page.locator('.call-apps-access-row[data-user-id="2"] .call-apps-grant-action').click();
+ await expect(page.locator('.call-apps-access-row[data-user-id="2"]')).toContainText('Blocked');
+ await expect(page.locator('.call-apps-access-row[data-user-id="2"]')).toContainText('Allow');
+ await expect(whiteboardFrame.locator('.remote-cursor-label')).toHaveText('Owner');
+ await expect.poll(() => page.evaluate(() => window.whiteboardInstallSidebarProof.state.whiteboardLaunchCount))
+ .toBe(launchCountBeforeGrantToggle);
+ expect(await page.evaluate(() => window.whiteboardInstallSidebarProof.state.whiteboardFrameSrc)).toBe(frameSrcBeforeGrantToggle);
+
+ expect(server.requests.map((entry) => `${entry.method} ${entry.pathname}`)).toEqual(expect.arrayContaining([
+ 'POST /api/marketplace/call-apps/whiteboard/orders',
+ 'POST /api/marketplace/call-apps/whiteboard/installations',
+ `GET /api/calls/${CALL_ID}/call-apps/available`,
+ `POST /api/calls/${CALL_ID}/call-app-sessions`,
+ 'PATCH /api/call-app-sessions/session-whiteboard-install-proof/participant-grants',
+ ]));
+ expect(server.requests.find((entry) => entry.pathname === `/api/calls/${CALL_ID}/call-app-sessions`)?.body)
+ .toMatchObject({ app_key: 'whiteboard', default_app_policy: 'allowed_by_default' });
+ expect(server.requests.find((entry) => entry.pathname === '/api/call-app-sessions/session-whiteboard-install-proof/participant-grants')?.body)
+ .toMatchObject({ grants: [{ subject_type: 'user', user_id: 2, grant_state: 'denied' }] });
+});
diff --git a/demo/video-chat/frontend-vue/tests/e2e/call-app-whiteboard.spec.js b/demo/video-chat/frontend-vue/tests/e2e/call-app-whiteboard.spec.js
index da271dc95..49efab911 100644
--- a/demo/video-chat/frontend-vue/tests/e2e/call-app-whiteboard.spec.js
+++ b/demo/video-chat/frontend-vue/tests/e2e/call-app-whiteboard.spec.js
@@ -137,6 +137,7 @@ function hostHtml(baseURL) {
logicalClock: 0,
ops: [],
presence: [],
+ presenceDeliveries: [],
snapshot: null,
snapshotClock: 0,
appendAttempts: [],
@@ -307,6 +308,7 @@ function hostHtml(baseURL) {
actor_id: entry.actor_id,
payload_type: entry.payload_type,
})),
+ presenceDeliveries: state.presenceDeliveries.map((entry) => ({ ...entry })),
snapshotClock: state.snapshotClock,
appendAttempts: state.appendAttempts.slice(),
deniedAppendCount: state.deniedAppendCount,
@@ -333,6 +335,33 @@ function hostHtml(baseURL) {
});
audit(alias + ' revoked');
},
+ injectRemoteCursor(alias, cursor) {
+ postTo(alias, 'call_app.presence.update', {
+ actor_id: String(cursor.actorId || ''),
+ payload_type: 'cursor.move',
+ payload: {
+ actor_id: String(cursor.actorId || ''),
+ display_name: String(cursor.label || ''),
+ label: String(cursor.label || ''),
+ x: Number(cursor.x || 0),
+ y: Number(cursor.y || 0),
+ color: String(cursor.color || '#1582bf'),
+ },
+ });
+ state.presenceDeliveries.push({
+ from: String(cursor.label || 'remote'),
+ to: alias,
+ actor_id: String(cursor.actorId || ''),
+ payload_type: 'cursor.move',
+ label: String(cursor.label || ''),
+ });
+ },
+ leaveRemoteCursor(alias, actorId) {
+ postTo(alias, 'call_app.presence.leave', {
+ actor_id: String(actorId || ''),
+ });
+ audit('remote cursor left: ' + actorId);
+ },
reload(alias) {
state.ready[alias] = false;
const frame = document.getElementById(participants[alias].frameId);
@@ -457,6 +486,13 @@ function hostHtml(baseURL) {
payload_type: String(message.payload_type || ''),
payload: message.payload || {},
});
+ state.presenceDeliveries.push({
+ from: participant.alias,
+ to: target.alias,
+ actor_id: participant.actorId,
+ payload_type: String(message.payload_type || ''),
+ label: String(message.payload?.label || message.payload?.display_name || ''),
+ });
}
}
});
@@ -495,6 +531,29 @@ async function nonWhitePixelCount(page, frameName) {
});
}
+async function canvasRegionNonWhiteCount(page, frameName, region) {
+ const frame = page.frame({ name: frameName });
+ expect(frame).toBeTruthy();
+ return frame.evaluate((sampleRegion) => {
+ const canvas = document.getElementById('board');
+ const context = canvas.getContext('2d');
+ const x = Math.max(0, Math.min(canvas.width - 1, Math.floor(sampleRegion.x)));
+ const y = Math.max(0, Math.min(canvas.height - 1, Math.floor(sampleRegion.y)));
+ const width = Math.max(1, Math.min(canvas.width - x, Math.floor(sampleRegion.width)));
+ const height = Math.max(1, Math.min(canvas.height - y, Math.floor(sampleRegion.height)));
+ const pixels = context.getImageData(x, y, width, height).data;
+ let count = 0;
+ for (let index = 0; index < pixels.length; index += 4) {
+ const red = pixels[index];
+ const green = pixels[index + 1];
+ const blue = pixels[index + 2];
+ const alpha = pixels[index + 3];
+ if (alpha > 0 && !(red > 245 && green > 245 && blue > 245)) count += 1;
+ }
+ return count;
+ }, region);
+}
+
test('Whiteboard Call App journey covers collaboration, presence, replay, snapshot, and revocation', async ({ page }) => {
const assets = await readWhiteboardAssets();
const baseURL = test.info().project.use.baseURL || 'http://127.0.0.1:4174';
@@ -531,8 +590,67 @@ test('Whiteboard Call App journey covers collaboration, presence, replay, snapsh
const ownerCanvasBox = await ownerFrame.locator('#board').boundingBox();
expect(ownerCanvasBox).toBeTruthy();
+ const cursorScreenPoint = {
+ x: ownerCanvasBox.x + 520,
+ y: ownerCanvasBox.y + 92,
+ };
+ const cursorBoardPoint = {
+ x: ((cursorScreenPoint.x - ownerCanvasBox.x) / ownerCanvasBox.width) * 1600,
+ y: ((cursorScreenPoint.y - ownerCanvasBox.y) / ownerCanvasBox.height) * 900,
+ };
+ const cursorRegion = {
+ x: cursorBoardPoint.x + 20,
+ y: cursorBoardPoint.y + 6,
+ width: 190,
+ height: 36,
+ };
+ const cursorRegionBefore = await canvasRegionNonWhiteCount(page, 'participant', cursorRegion);
const presenceBeforeCursorBurst = await page.evaluate(() => window.whiteboardHarness.state.presence.length);
await page.waitForTimeout(650);
+ await page.mouse.move(cursorScreenPoint.x, cursorScreenPoint.y);
+ await expect.poll(() => canvasRegionNonWhiteCount(page, 'participant', cursorRegion))
+ .toBeGreaterThan(cursorRegionBefore + 40);
+ await expect(participantFrame.locator('.remote-cursor-label')).toHaveText('Owner');
+ const cursorRegionWithOwner = await canvasRegionNonWhiteCount(page, 'participant', cursorRegion);
+ expect(cursorRegionWithOwner).toBeGreaterThan(cursorRegionBefore + 40);
+ await expect.poll(() => page.evaluate(() => window.whiteboardHarness.state.presenceDeliveries))
+ .toEqual(expect.arrayContaining([
+ expect.objectContaining({
+ from: 'owner',
+ to: 'participant',
+ payload_type: 'cursor.move',
+ label: 'Owner',
+ }),
+ ]));
+ await expect(participantFrame.locator('.remote-cursor-label')).toHaveText('Owner');
+ const participantLaunchCountBeforeRemoteCursors = await page.evaluate(() => window.whiteboardHarness.state.launchCount.participant);
+ const participantFrameSrcBeforeRemoteCursors = await page.locator('#participantFrame').evaluate((frame) => frame.src);
+ await page.evaluate(() => {
+ window.whiteboardHarness.injectRemoteCursor('participant', {
+ actorId: 'user_reviewer_e2e',
+ label: 'Reviewer',
+ x: 760,
+ y: 176,
+ color: '#00652f',
+ });
+ window.whiteboardHarness.injectRemoteCursor('participant', {
+ actorId: 'user_facilitator_e2e',
+ label: 'Facilitator',
+ x: 920,
+ y: 226,
+ color: '#f47221',
+ });
+ });
+ await expect(participantFrame.locator('.remote-cursor-label')).toHaveCount(3);
+ await expect.poll(() => participantFrame.locator('.remote-cursor-label').allTextContents())
+ .toEqual(['Owner', 'Reviewer', 'Facilitator']);
+ await page.evaluate(() => window.whiteboardHarness.leaveRemoteCursor('participant', 'user_reviewer_e2e'));
+ await expect(participantFrame.locator('.remote-cursor-label')).toHaveCount(2);
+ await expect.poll(() => participantFrame.locator('.remote-cursor-label').allTextContents())
+ .toEqual(['Owner', 'Facilitator']);
+ await expect.poll(() => page.evaluate(() => window.whiteboardHarness.state.launchCount.participant))
+ .toBe(participantLaunchCountBeforeRemoteCursors);
+ expect(await page.locator('#participantFrame').evaluate((frame) => frame.src)).toBe(participantFrameSrcBeforeRemoteCursors);
for (let index = 0; index < 8; index += 1) {
await page.mouse.move(ownerCanvasBox.x + 350 + index * 12, ownerCanvasBox.y + 120 + index * 4);
}
@@ -583,6 +701,23 @@ test('Whiteboard Call App journey covers collaboration, presence, replay, snapsh
await page.evaluate(() => window.whiteboardHarness.revoke('participant'));
await expect(participantFrame.locator('#modeBadge')).toHaveText('No access');
await expect(participantFrame.locator('[data-tool="pen"]')).toBeDisabled();
+ await expect(participantFrame.locator('.remote-cursor-label')).toHaveCount(0);
+ await expect.poll(() => page.evaluate(() => window.whiteboardHarness.state.launchCount.participant))
+ .toBe(participantLaunchCountBeforeRemoteCursors);
+ expect(await page.locator('#participantFrame').evaluate((frame) => frame.src)).toBe(participantFrameSrcBeforeRemoteCursors);
+ await expect.poll(() => canvasRegionNonWhiteCount(page, 'participant', cursorRegion))
+ .toBeLessThanOrEqual(cursorRegionBefore + 12);
+ const participantPresenceDeliveriesAfterRevoke = await page.evaluate(() => (
+ window.whiteboardHarness.state.presenceDeliveries.filter((entry) => entry.to === 'participant').length
+ ));
+ await page.waitForTimeout(650);
+ await page.mouse.move(cursorScreenPoint.x + 64, cursorScreenPoint.y + 24);
+ await page.waitForTimeout(200);
+ await expect.poll(() => page.evaluate(() => (
+ window.whiteboardHarness.state.presenceDeliveries.filter((entry) => entry.to === 'participant').length
+ ))).toBe(participantPresenceDeliveriesAfterRevoke);
+ const cursorRegionAfterRevoke = await canvasRegionNonWhiteCount(page, 'participant', cursorRegion);
+ expect(cursorRegionAfterRevoke).toBeLessThanOrEqual(cursorRegionBefore + 12);
await drawStroke(page, 'iframe[name="participant"]');
await expect.poll(() => page.evaluate(() => window.whiteboardHarness.state.ops.length))
diff --git a/demo/video-chat/frontend-vue/tests/e2e/helpers/callAccessSeedMatrix.js b/demo/video-chat/frontend-vue/tests/e2e/helpers/callAccessSeedMatrix.js
new file mode 100644
index 000000000..a77c706ce
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/e2e/helpers/callAccessSeedMatrix.js
@@ -0,0 +1,636 @@
+import fs from 'node:fs';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+export const backendOrigin = process.env.VITE_VIDEOCHAT_BACKEND_ORIGIN || 'http://127.0.0.1:18080';
+export const sessionStorageKey = 'ii_videocall_v1_session';
+
+const helperDir = path.dirname(fileURLToPath(import.meta.url));
+export const callAccessSeedMatrixPath = path.resolve(
+ helperDir,
+ '../../../../contracts/v1/iam-call-access-seeding.matrix.json',
+);
+
+function readSeedMatrix() {
+ const inlineMatrix = String(process.env.VIDEOCHAT_CALL_ACCESS_SEED_MATRIX_JSON || '').trim();
+ if (inlineMatrix !== '') {
+ return JSON.parse(inlineMatrix);
+ }
+ return JSON.parse(fs.readFileSync(callAccessSeedMatrixPath, 'utf8'));
+}
+
+function clone(value) {
+ return JSON.parse(JSON.stringify(value));
+}
+
+function byKey(rows, label) {
+ const index = new Map();
+ for (const row of Array.isArray(rows) ? rows : []) {
+ const key = String(row?.key || '').trim();
+ if (key === '') throw new Error(`${label} matrix row is missing key.`);
+ if (index.has(key)) throw new Error(`${label} matrix row key is duplicated: ${key}`);
+ index.set(key, row);
+ }
+ return index;
+}
+
+export const iamCallAccessSeedMatrix = Object.freeze(readSeedMatrix());
+
+const tenantIndex = byKey(iamCallAccessSeedMatrix.tenants, 'tenant');
+const userIndex = byKey(iamCallAccessSeedMatrix.users, 'user');
+const callIndex = byKey(iamCallAccessSeedMatrix.calls, 'call');
+const accessLinkIndex = byKey(iamCallAccessSeedMatrix.access_links, 'access link');
+const scenarioIndex = byKey(iamCallAccessSeedMatrix.scenarios, 'scenario');
+
+function requiredRow(index, key, label) {
+ const normalizedKey = String(key || '').trim();
+ const row = index.get(normalizedKey);
+ if (!row) throw new Error(`Unknown ${label} matrix key: ${normalizedKey}`);
+ return row;
+}
+
+export function getSeedTenant(key) {
+ return clone(requiredRow(tenantIndex, key, 'tenant'));
+}
+
+export function getSeedUser(key) {
+ return clone(requiredRow(userIndex, key, 'user'));
+}
+
+export function getSeedCall(key) {
+ return clone(requiredRow(callIndex, key, 'call'));
+}
+
+export function getSeedAccessLink(key) {
+ return clone(requiredRow(accessLinkIndex, key, 'access link'));
+}
+
+export function getSeedScenario(key) {
+ return clone(requiredRow(scenarioIndex, key, 'scenario'));
+}
+
+export function accessIdFromJoinPath(joinPath) {
+ const match = String(joinPath || '').match(/\/join\/([a-f0-9-]{36})(?:[/?#].*)?$/i);
+ return match ? match[1].toLowerCase() : '';
+}
+
+export function seedUserKeys() {
+ return [...userIndex.keys()];
+}
+
+export function seedScenarioKeys() {
+ return [...scenarioIndex.keys()];
+}
+
+function tenantForCall(call) {
+ const tenantKey = typeof call?.tenant_key === 'string' ? call.tenant_key : '';
+ return tenantKey === '' ? null : requiredRow(tenantIndex, tenantKey, 'tenant');
+}
+
+function membershipForTenant(user, tenantKey) {
+ return (Array.isArray(user?.memberships) ? user.memberships : [])
+ .find((membership) => String(membership?.tenant_key || '') === tenantKey) || null;
+}
+
+function permissionsFor(user, membershipRole) {
+ const normalizedRole = String(membershipRole || 'member').trim().toLowerCase();
+ const isTenantAdmin = normalizedRole === 'owner' || normalizedRole === 'admin';
+ const isPlatformAdmin = user?.system_admin === true || String(user?.role || '').trim().toLowerCase() === 'admin';
+ const elevated = isTenantAdmin || isPlatformAdmin;
+ return {
+ platform_admin: isPlatformAdmin,
+ tenant_admin: elevated,
+ manage_users: elevated,
+ manage_organizations: elevated,
+ manage_groups: elevated,
+ manage_permission_grants: elevated,
+ edit_themes: elevated,
+ export_import: elevated,
+ manage_lobby: elevated,
+ admit_participants: elevated,
+ reject_participants: elevated,
+ kick_participants: elevated,
+ };
+}
+
+export function tenantSnapshotForSeedUser(userKey, callKey) {
+ const user = requiredRow(userIndex, userKey, 'user');
+ const call = requiredRow(callIndex, callKey, 'call');
+ return clone(tenantSnapshotFor(user, call));
+}
+
+function tenantSnapshotFor(user, call) {
+ const tenant = tenantForCall(call);
+ if (!tenant) return null;
+ const tenantKey = String(call.tenant_key || '');
+ const membership = membershipForTenant(user, tenantKey);
+ const role = String(membership?.role || 'member').trim().toLowerCase() || 'member';
+ return {
+ id: tenant.id,
+ tenant_id: tenant.id,
+ uuid: tenant.uuid,
+ public_id: tenant.uuid,
+ slug: tenant.slug,
+ label: tenant.label,
+ role,
+ membership_id: membership ? Number(user.id) * 100 + Number(tenant.id) : 0,
+ permissions: permissionsFor(user, role),
+ };
+}
+
+function userPayload(user, tenant = null, overrides = {}) {
+ return {
+ id: user.id,
+ email: user.email,
+ display_name: overrides.displayName || user.display_name,
+ role: user.role,
+ status: 'active',
+ time_format: '24h',
+ date_format: 'dmy_dot',
+ theme: 'dark',
+ locale: 'en',
+ direction: 'ltr',
+ supported_locales: ['en'],
+ avatar_path: null,
+ post_logout_landing_url: '',
+ account_type: user.account_type,
+ is_guest: Boolean(user.is_guest),
+ tenant,
+ };
+}
+
+function ownerPayload(call) {
+ const owner = requiredRow(userIndex, call.owner_user_key, 'user');
+ return {
+ user_id: owner.id,
+ display_name: owner.display_name,
+ email: owner.email,
+ };
+}
+
+function participantPayload(user, callRole = 'participant', inviteState = 'allowed') {
+ return {
+ user_id: user.id,
+ display_name: user.display_name,
+ email: user.email,
+ call_role: callRole,
+ invite_state: inviteState,
+ joined_at: null,
+ connected_at: null,
+ };
+}
+
+function callPayload(call, viewerUser = null, inviteState = 'pending') {
+ const owner = requiredRow(userIndex, call.owner_user_key, 'user');
+ const guestUsers = (Array.isArray(call.guest_list_user_keys) ? call.guest_list_user_keys : [])
+ .map((key) => requiredRow(userIndex, key, 'user'));
+ const internal = [
+ participantPayload(owner, 'owner', 'allowed'),
+ ...guestUsers.map((user) => participantPayload(user, 'participant', 'allowed')),
+ ];
+ if (viewerUser && !internal.some((participant) => Number(participant.user_id) === Number(viewerUser.id))) {
+ internal.push(participantPayload(viewerUser, 'participant', inviteState));
+ }
+
+ return {
+ id: call.id,
+ room_id: call.room_id,
+ title: call.title,
+ status: call.status,
+ starts_at: call.starts_at,
+ ends_at: call.ends_at,
+ owner: ownerPayload(call),
+ participants: {
+ total: internal.length,
+ internal,
+ external: [],
+ },
+ my_participation: viewerUser ? {
+ call_role: Number(viewerUser.id) === Number(owner.id) ? 'owner' : 'participant',
+ invite_state: inviteState,
+ } : null,
+ };
+}
+
+function accessLinkPayload(link, call, targetUser = null) {
+ const tenant = tenantForCall(call);
+ return {
+ id: link.id,
+ call_id: call.id,
+ room_id: call.room_id,
+ tenant_id: tenant?.id || null,
+ link_kind: link.link_kind,
+ participant_user_id: targetUser?.id || null,
+ participant_email: targetUser?.email || null,
+ created_by_user_id: ownerPayload(call).user_id,
+ created_at: '2026-05-08T10:00:00.000Z',
+ expires_at: '2030-01-01T00:00:00.000Z',
+ consumed_at: null,
+ last_used_at: null,
+ };
+}
+
+function seedSessionIdForUser(user) {
+ return `sess_iam_seed_${String(user.key || user.id).replace(/[^a-z0-9_]+/gi, '_')}`;
+}
+
+function callAccessSessionId(link, user) {
+ return `sess_iam_call_access_${String(link.key).replace(/[^a-z0-9_]+/gi, '_')}_${String(user.key).replace(/[^a-z0-9_]+/gi, '_')}`;
+}
+
+export function storedSessionForSeedUser(userKey, callKey = 'alpha_active') {
+ const user = requiredRow(userIndex, userKey, 'user');
+ const call = requiredRow(callIndex, callKey, 'call');
+ return {
+ role: user.role,
+ displayName: user.display_name,
+ email: user.email,
+ userId: user.id,
+ avatarPath: null,
+ timeFormat: '24h',
+ theme: 'dark',
+ status: 'active',
+ sessionId: seedSessionIdForUser(user),
+ sessionToken: seedSessionIdForUser(user),
+ expiresAt: '2030-01-01T00:00:00.000Z',
+ tenant: tenantSnapshotFor(user, call),
+ };
+}
+
+export async function installStoredSeedSession(context, userKey, callKey = 'alpha_active') {
+ await context.addInitScript(
+ ({ key, value }) => {
+ localStorage.setItem(key, JSON.stringify(value));
+ },
+ { key: sessionStorageKey, value: storedSessionForSeedUser(userKey, callKey) },
+ );
+}
+
+function jsonHeaders() {
+ return {
+ 'access-control-allow-origin': '*',
+ 'access-control-allow-credentials': 'true',
+ 'access-control-allow-headers': 'content-type, authorization, x-session-id',
+ 'access-control-allow-methods': 'GET, POST, PATCH, DELETE, OPTIONS',
+ 'access-control-expose-headers': 'content-disposition, content-type',
+ 'access-control-max-age': '86400',
+ 'content-type': 'application/json; charset=utf-8',
+ };
+}
+
+async function fulfillJson(route, status, payload) {
+ await route.fulfill({
+ status,
+ headers: jsonHeaders(),
+ json: payload,
+ });
+}
+
+function parseJsonBody(request) {
+ const raw = String(request.postData() || '').trim();
+ if (raw === '') return {};
+ try {
+ const parsed = JSON.parse(raw);
+ return parsed && typeof parsed === 'object' ? parsed : {};
+ } catch {
+ return {};
+ }
+}
+
+function bearerToken(request) {
+ const authorization = String(request.headers().authorization || '').trim();
+ const match = authorization.match(/^Bearer\s+(.+)$/i);
+ if (match) return match[1].trim();
+ return String(request.headers()['x-session-id'] || '').trim();
+}
+
+function seededSessionRecordFromToken(token) {
+ for (const user of userIndex.values()) {
+ const sessionId = seedSessionIdForUser(user);
+ if (sessionId === token) {
+ const firstCall = [...callIndex.values()].find((call) => call.tenant_key) || [...callIndex.values()][0];
+ return {
+ session: {
+ id: sessionId,
+ token: sessionId,
+ token_type: 'session_id',
+ issued_at: '2026-05-08T10:00:00.000Z',
+ expires_at: '2030-01-01T00:00:00.000Z',
+ },
+ user,
+ call: firstCall,
+ tenant: tenantSnapshotFor(user, firstCall),
+ };
+ }
+ }
+ return null;
+}
+
+function sessionStatePayload(record) {
+ const tenant = record.tenant || tenantSnapshotFor(record.user, record.call);
+ return {
+ status: 'ok',
+ result: { state: 'authenticated' },
+ session: record.session,
+ user: userPayload(record.user, tenant),
+ tenant,
+ time: '2026-05-08T10:00:00.000Z',
+ };
+}
+
+function targetUserForAccessLink(link, requestBody = {}) {
+ if (link.link_kind === 'open') {
+ const anonymousKey = String(link.anonymous_user_key || 'temporary_anonymous_guest');
+ const anonymousUser = requiredRow(userIndex, anonymousKey, 'user');
+ const guestName = String(requestBody.guest_name || '').trim();
+ return {
+ ...anonymousUser,
+ display_name: guestName || anonymousUser.display_name,
+ };
+ }
+ return requiredRow(userIndex, link.target_user_key, 'user');
+}
+
+function resolveAccessLinkById(accessId) {
+ const normalizedAccessId = String(accessId || '').trim().toLowerCase();
+ return [...accessLinkIndex.values()].find((link) => String(link.id).toLowerCase() === normalizedAccessId) || null;
+}
+
+export async function installCallAccessSeedRoutes(context) {
+ const issuedSessions = new Map();
+
+ await context.route('**/api/**', async (route) => {
+ const request = route.request();
+ if (request.method() === 'OPTIONS') {
+ await route.fulfill({ status: 204, headers: jsonHeaders() });
+ return;
+ }
+
+ const url = new URL(request.url());
+ const joinMatch = url.pathname.match(/^\/api\/call-access\/([a-f0-9-]{36})\/join$/i);
+ if (joinMatch && request.method() === 'GET') {
+ const link = resolveAccessLinkById(joinMatch[1]);
+ if (!link) {
+ await fulfillJson(route, 404, {
+ status: 'error',
+ error: { code: 'call_access_not_found', message: 'Call access link does not exist.' },
+ });
+ return;
+ }
+ const call = requiredRow(callIndex, link.call_key, 'call');
+ const targetUser = link.link_kind === 'personal' ? requiredRow(userIndex, link.target_user_key, 'user') : null;
+ await fulfillJson(route, 200, {
+ status: 'ok',
+ result: {
+ state: 'resolved',
+ access_link: accessLinkPayload(link, call, targetUser),
+ link_kind: link.link_kind,
+ call: callPayload(call, targetUser, link.requires_admission ? 'pending' : 'allowed'),
+ target_user: targetUser ? userPayload(targetUser, tenantSnapshotFor(targetUser, call)) : null,
+ target_hint: { participant_email: targetUser?.email || null },
+ join_path: link.join_path,
+ },
+ time: '2026-05-08T10:00:00.000Z',
+ });
+ return;
+ }
+
+ const sessionMatch = url.pathname.match(/^\/api\/call-access\/([a-f0-9-]{36})\/session$/i);
+ if (sessionMatch && request.method() === 'POST') {
+ const link = resolveAccessLinkById(sessionMatch[1]);
+ if (!link) {
+ await fulfillJson(route, 404, {
+ status: 'error',
+ error: { code: 'call_access_not_found', message: 'Call access link does not exist.' },
+ });
+ return;
+ }
+ const body = parseJsonBody(request);
+ if (link.link_kind === 'open' && String(body.guest_name || '').trim() === '') {
+ await fulfillJson(route, 422, {
+ status: 'error',
+ error: { code: 'call_access_validation_failed', message: 'Guest name is required.' },
+ });
+ return;
+ }
+ const call = requiredRow(callIndex, link.call_key, 'call');
+ const targetUser = targetUserForAccessLink(link, body);
+ const tenant = tenantSnapshotFor(targetUser, call);
+ const sessionId = callAccessSessionId(link, targetUser);
+ const session = {
+ id: sessionId,
+ token: sessionId,
+ token_type: 'session_id',
+ issued_at: '2026-05-08T10:00:00.000Z',
+ expires_at: '2030-01-01T00:00:00.000Z',
+ expires_in_seconds: 43200,
+ };
+ issuedSessions.set(session.token, { session, user: targetUser, call, tenant, link });
+ await fulfillJson(route, 200, {
+ status: 'ok',
+ result: {
+ state: 'session_started',
+ session,
+ user: userPayload(targetUser, tenant),
+ tenant,
+ access_link: accessLinkPayload(link, call, targetUser),
+ link_kind: link.link_kind,
+ call: callPayload(call, targetUser, link.requires_admission ? 'pending' : 'allowed'),
+ join_path: link.join_path,
+ },
+ time: '2026-05-08T10:00:00.000Z',
+ });
+ return;
+ }
+
+ if (url.pathname === '/api/auth/session-state' || url.pathname === '/api/auth/session') {
+ const token = bearerToken(request);
+ const record = issuedSessions.get(token) || seededSessionRecordFromToken(token);
+ if (!record) {
+ await fulfillJson(route, 401, {
+ status: 'error',
+ error: { code: 'auth_failed', message: 'A valid session token is required.' },
+ });
+ return;
+ }
+ await fulfillJson(route, 200, sessionStatePayload(record));
+ return;
+ }
+
+ const resolveMatch = url.pathname.match(/^\/api\/calls\/resolve\/([^/]+)$/);
+ if (resolveMatch && request.method() === 'GET') {
+ const callRef = decodeURIComponent(resolveMatch[1] || '');
+ const call = [...callIndex.values()].find((row) => row.id === callRef || row.room_id === callRef);
+ if (!call) {
+ await fulfillJson(route, 404, {
+ status: 'error',
+ error: { code: 'calls_not_found', message: 'Call does not exist.' },
+ });
+ return;
+ }
+ await fulfillJson(route, 200, {
+ status: 'ok',
+ result: {
+ state: 'resolved',
+ resolved_as: 'call',
+ call: callPayload(call),
+ },
+ time: '2026-05-08T10:00:00.000Z',
+ });
+ return;
+ }
+
+ const callMatch = url.pathname.match(/^\/api\/calls\/([^/]+)$/);
+ if (callMatch && request.method() === 'GET') {
+ const callId = decodeURIComponent(callMatch[1] || '');
+ const call = [...callIndex.values()].find((row) => row.id === callId || row.room_id === callId);
+ if (!call) {
+ await fulfillJson(route, 404, {
+ status: 'error',
+ error: { code: 'calls_not_found', message: 'Call does not exist.' },
+ });
+ return;
+ }
+ await fulfillJson(route, 200, {
+ status: 'ok',
+ call: callPayload(call),
+ time: '2026-05-08T10:00:00.000Z',
+ });
+ return;
+ }
+
+ await fulfillJson(route, 404, {
+ status: 'error',
+ error: { code: 'not_found', message: `Missing IAM call-access seed route: ${url.pathname}` },
+ });
+ });
+}
+
+export async function installCallAccessFakeRealtime(context, { linkKey }) {
+ const link = requiredRow(accessLinkIndex, linkKey, 'access link');
+ const call = requiredRow(callIndex, link.call_key, 'call');
+ await context.addInitScript(({ roomId, callId, requiresAdmission }) => {
+ const listenersSymbol = Symbol('listeners');
+
+ window.__iamCallAccessSocketFrames = [];
+ window.__iamCallAccessSocketEvents = [];
+ window.__iamCallAccessSockets = [];
+
+ class FakeWebSocket {
+ static CONNECTING = 0;
+ static OPEN = 1;
+ static CLOSING = 2;
+ static CLOSED = 3;
+
+ constructor(url) {
+ this.url = String(url || '');
+ this.readyState = FakeWebSocket.CONNECTING;
+ this[listenersSymbol] = {};
+ window.__iamCallAccessSockets.push(this);
+ setTimeout(() => {
+ if (this.readyState === FakeWebSocket.CLOSED) return;
+ this.readyState = FakeWebSocket.OPEN;
+ this.dispatch('open', {});
+ this.emit({
+ type: 'system/welcome',
+ active_room_id: roomId,
+ admission: {
+ requires_admission: Boolean(requiresAdmission),
+ pending_room_id: roomId,
+ call_id: callId,
+ },
+ });
+ }, 0);
+ }
+
+ addEventListener(type, callback) {
+ if (!this[listenersSymbol][type]) this[listenersSymbol][type] = [];
+ this[listenersSymbol][type].push(callback);
+ if (type === 'open' && this.readyState === FakeWebSocket.OPEN) {
+ setTimeout(() => callback({}), 0);
+ }
+ }
+
+ removeEventListener(type, callback) {
+ this[listenersSymbol][type] = (this[listenersSymbol][type] || [])
+ .filter((registered) => registered !== callback);
+ }
+
+ dispatch(type, event) {
+ for (const callback of this[listenersSymbol][type] || []) callback(event);
+ }
+
+ emit(payload) {
+ window.__iamCallAccessSocketEvents.push(payload);
+ this.dispatch('message', { data: JSON.stringify(payload) });
+ }
+
+ send(data) {
+ let payload = null;
+ try {
+ payload = JSON.parse(String(data || '{}'));
+ } catch {
+ payload = { type: 'invalid_json' };
+ }
+ window.__iamCallAccessSocketFrames.push(payload);
+ if (payload.type === 'lobby/queue/join') {
+ setTimeout(() => {
+ this.emit({
+ type: 'lobby/snapshot',
+ room_id: roomId,
+ call_id: callId,
+ pending: [],
+ admitted: [],
+ rejected: [],
+ });
+ }, 0);
+ }
+ }
+
+ close(code = 1000, reason = 'test_close') {
+ if (this.readyState === FakeWebSocket.CLOSED) return;
+ this.readyState = FakeWebSocket.CLOSED;
+ this.dispatch('close', { code, reason });
+ }
+ }
+
+ window.WebSocket = FakeWebSocket;
+ }, {
+ roomId: call.room_id,
+ callId: call.id,
+ requiresAdmission: link.requires_admission !== false,
+ });
+}
+
+export async function installCallAccessMediaDeviceShim(context) {
+ await context.addInitScript(() => {
+ Object.defineProperty(navigator, 'mediaDevices', {
+ configurable: true,
+ value: {
+ ...(navigator.mediaDevices || {}),
+ getUserMedia: async () => new MediaStream(),
+ enumerateDevices: async () => [
+ { kind: 'audioinput', deviceId: 'iam-audio', label: 'IAM matrix microphone', groupId: 'iam-call-access' },
+ { kind: 'videoinput', deviceId: 'iam-video', label: 'IAM matrix camera', groupId: 'iam-call-access' },
+ { kind: 'audiooutput', deviceId: 'iam-speaker', label: 'IAM matrix speaker', groupId: 'iam-call-access' },
+ ],
+ getSupportedConstraints: () => ({ audio: true, video: true, deviceId: true }),
+ addEventListener: () => {},
+ removeEventListener: () => {},
+ },
+ });
+ });
+}
+
+export async function createCallAccessMatrixPage(browser, baseURL, { scenarioKey }) {
+ const scenario = requiredRow(scenarioIndex, scenarioKey, 'scenario');
+ const linkKey = String(scenario.link_key || '').trim();
+ if (linkKey === '') throw new Error(`Scenario ${scenarioKey} is not bound to a call-access link.`);
+
+ const context = await browser.newContext({ baseURL, permissions: ['camera', 'microphone'] });
+ await installCallAccessSeedRoutes(context);
+ await installCallAccessMediaDeviceShim(context);
+ await installCallAccessFakeRealtime(context, { linkKey });
+ const page = await context.newPage();
+ return { context, page, scenario: clone(scenario) };
+}
diff --git a/demo/video-chat/frontend-vue/tests/e2e/helpers/nativeAudioTransferHarness.js b/demo/video-chat/frontend-vue/tests/e2e/helpers/nativeAudioTransferHarness.js
index 1ace04045..941c01ebc 100644
--- a/demo/video-chat/frontend-vue/tests/e2e/helpers/nativeAudioTransferHarness.js
+++ b/demo/video-chat/frontend-vue/tests/e2e/helpers/nativeAudioTransferHarness.js
@@ -132,7 +132,7 @@ async function installSocketInstrumentation(context) {
});
}
-async function installMediaDeviceShim(context, {
+export async function installMediaDeviceShim(context, {
audioFrequency = 440,
videoWidth = 320,
videoHeight = 240,
diff --git a/demo/video-chat/frontend-vue/tests/e2e/helpers/videochatMatrixHarness.js b/demo/video-chat/frontend-vue/tests/e2e/helpers/videochatMatrixHarness.js
index 22d26c8af..40667254b 100644
--- a/demo/video-chat/frontend-vue/tests/e2e/helpers/videochatMatrixHarness.js
+++ b/demo/video-chat/frontend-vue/tests/e2e/helpers/videochatMatrixHarness.js
@@ -244,6 +244,15 @@ export async function installMatrixApiRoutes(context, user) {
return;
}
+ if (url.pathname === '/api/user/client-diagnostics' && request.method() === 'POST') {
+ await route.fulfill({
+ status: 200,
+ headers: { ...corsHeaders(), 'content-type': 'application/json; charset=utf-8' },
+ json: { status: 'ok', result: { accepted: true } },
+ });
+ return;
+ }
+
if (url.pathname === `/api/calls/resolve/${matrixCallRef}`) {
await route.fulfill({
status: 200,
@@ -385,6 +394,10 @@ export async function installFakeMediaAndRealtime(context, user) {
window.__matrixSocketFrames = [];
window.__matrixSocketEvents = [];
+ window.__matrixSocketLifecycle = [];
+ window.__matrixSocketConnectFailures = [];
+ window.__matrixFetchCalls = [];
+ window.__matrixPageLifecycleEvents = [];
window.__matrixAttachmentStore = {};
window.__matrixLastChatMessage = null;
window.__matrixSockets = [];
@@ -405,9 +418,10 @@ export async function installFakeMediaAndRealtime(context, user) {
const nativeFetch = window.fetch.bind(window);
window.fetch = async (...args) => {
- const response = await nativeFetch(...args);
const url = String(args[0]?.url || args[0] || '');
- const method = String(args[1]?.method || 'GET').toUpperCase();
+ const method = String(args[1]?.method || args[0]?.method || 'GET').toUpperCase();
+ window.__matrixFetchCalls.push({ url, method, time: Date.now() });
+ const response = await nativeFetch(...args);
if (method === 'POST' && url.includes('/chat/attachments')) {
try {
const payload = await response.clone().json();
@@ -420,6 +434,13 @@ export async function installFakeMediaAndRealtime(context, user) {
return response;
};
+ window.addEventListener('beforeunload', () => {
+ window.__matrixPageLifecycleEvents.push({ type: 'beforeunload', time: Date.now() });
+ });
+ window.addEventListener('pagehide', () => {
+ window.__matrixPageLifecycleEvents.push({ type: 'pagehide', time: Date.now() });
+ });
+
Object.defineProperty(navigator, 'mediaDevices', {
configurable: true,
value: {
@@ -475,8 +496,22 @@ export async function installFakeMediaAndRealtime(context, user) {
this.readyState = FakeWebSocket.CONNECTING;
this[listenersSymbol] = {};
window.__matrixSockets.push(this);
+ window.__matrixSocketLifecycle.push({ type: 'construct', url, time: Date.now() });
+ const connectFailure = window.__matrixSocketConnectFailures.shift();
+ if (connectFailure) {
+ setTimeout(() => {
+ if (this.readyState !== FakeWebSocket.CONNECTING) return;
+ this.readyState = FakeWebSocket.CLOSED;
+ const code = Number(connectFailure.code || 1011);
+ const reason = String(connectFailure.reason || 'websocket_reconnect_backfill_unavailable');
+ window.__matrixSocketLifecycle.push({ type: 'connect-failure', code, reason, url: this.url, time: Date.now() });
+ this.dispatch('close', { code, reason });
+ }, Math.max(0, Number(connectFailure.delayMs || 0)));
+ return;
+ }
setTimeout(() => {
this.readyState = FakeWebSocket.OPEN;
+ window.__matrixSocketLifecycle.push({ type: 'open', url: this.url, time: Date.now() });
this.dispatch('open', {});
const welcome = {
type: 'system/welcome',
@@ -634,11 +669,37 @@ export async function installFakeMediaAndRealtime(context, user) {
close(code = 1000, reason = 'test_close') {
if (this.readyState === FakeWebSocket.CLOSED) return;
this.readyState = FakeWebSocket.CLOSED;
+ window.__matrixSocketLifecycle.push({ type: 'close', code, reason, url: this.url, time: Date.now() });
this.dispatch('close', { code, reason });
}
}
window.__matrixEmit = dispatchToOpenSockets;
+ window.__matrixEmitToLatestSocket = (payload) => {
+ const openSocket = [...window.__matrixSockets]
+ .reverse()
+ .find((socket) => socket.readyState === FakeWebSocket.OPEN && String(socket.url || '').includes('/ws?'));
+ if (!openSocket) return false;
+ openSocket.emit(payload);
+ return true;
+ };
+ window.__matrixEmitRetryableAuthError = () => window.__matrixEmitToLatestSocket({
+ type: 'system/error',
+ code: 'websocket_auth_temporarily_unavailable',
+ message: 'Session validation is temporarily unavailable for realtime commands.',
+ details: {
+ reason: 'auth_backend_error',
+ retryable: true,
+ close: { close_code: 1011, close_reason: 'auth_backend_error' },
+ },
+ time: new Date().toISOString(),
+ });
+ window.__matrixQueueSocketConnectFailure = (failure = {}) => {
+ const reason = String(failure.reason || 'websocket_reconnect_backfill_unavailable');
+ const code = Number(failure.code || 1011);
+ window.__matrixSocketConnectFailures.push({ code, reason, delayMs: Number(failure.delayMs || 0) });
+ return window.__matrixSocketConnectFailures.length;
+ };
window.__matrixForceSocketClose = (code = 1006, reason = 'network_drop') => {
const openSocket = [...window.__matrixSockets]
.reverse()
diff --git a/demo/video-chat/frontend-vue/tests/e2e/lobby-concurrency-ui.spec.js b/demo/video-chat/frontend-vue/tests/e2e/lobby-concurrency-ui.spec.js
new file mode 100644
index 000000000..cf7048bee
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/e2e/lobby-concurrency-ui.spec.js
@@ -0,0 +1,174 @@
+import { test, expect } from '@playwright/test';
+import {
+ createMatrixPage,
+ matrixCallId,
+ matrixRoomId,
+ matrixUsers,
+ openMatrixWorkspace,
+} from './helpers/videochatMatrixHarness.js';
+
+const waitingUserId = 20;
+const waitingUserName = 'Waiting User';
+
+function lobbyEntry(overrides = {}) {
+ return {
+ user_id: waitingUserId,
+ display_name: waitingUserName,
+ role: 'user',
+ requested_unix_ms: 1_780_600_000_000,
+ requested_at: '2026-05-08T10:00:00.000Z',
+ ...overrides,
+ };
+}
+
+function admittedEntry(overrides = {}) {
+ return {
+ user_id: waitingUserId,
+ display_name: waitingUserName,
+ role: 'user',
+ admitted_unix_ms: 1_780_600_001_000,
+ admitted_at: '2026-05-08T10:00:01.000Z',
+ admitted_by: {
+ user_id: matrixUsers.admin.id,
+ display_name: matrixUsers.admin.displayName,
+ role: matrixUsers.admin.role,
+ },
+ ...overrides,
+ };
+}
+
+function lobbySnapshot({ queue = [], admitted = [], reason = 'test_lobby_concurrency' }) {
+ return {
+ type: 'lobby/snapshot',
+ room_id: matrixRoomId,
+ queue,
+ queue_count: queue.length,
+ admitted,
+ admitted_count: admitted.length,
+ reason,
+ server_unix_ms: Date.now(),
+ time: new Date().toISOString(),
+ };
+}
+
+function participantRow({ connectionId, userId, displayName, role = 'user', callRole = 'participant' }) {
+ return {
+ connection_id: connectionId,
+ room_id: matrixRoomId,
+ user: {
+ id: userId,
+ display_name: displayName,
+ role,
+ call_role: callRole,
+ },
+ connected_at: '2026-05-08T10:00:02.000Z',
+ };
+}
+
+async function emitMatrixEvent(page, payload) {
+ await page.evaluate((eventPayload) => {
+ window.__matrixEmit(eventPayload);
+ }, payload);
+}
+
+async function openLobbyPanel(page) {
+ await page.locator('button.tab-lobby').click();
+ const lobbyPanel = page.locator('.panel-lobby.active');
+ await expect(lobbyPanel).toBeVisible();
+ return lobbyPanel;
+}
+
+async function openUsersPanel(page) {
+ await page.getByRole('tab', { name: 'Users' }).click();
+ const usersPanel = page.locator('.panel-users.active');
+ await expect(usersPanel).toBeVisible();
+ return usersPanel;
+}
+
+async function setWorkspaceParticipants(page, participants) {
+ await page.evaluate((rows) => {
+ const setup = document.querySelector('.workspace-call-view')?.__vueParentComponent?.setupState;
+ if (!setup) throw new Error('Call workspace setup state is not available.');
+ setup.participantsRaw = rows;
+ }, participants);
+}
+
+test('concurrent lobby snapshots do not duplicate queue rows, participants, or stale controls', async ({ browser, baseURL }) => {
+ const { context, page } = await createMatrixPage(browser, baseURL, matrixUsers.admin);
+ try {
+ await openMatrixWorkspace(page);
+
+ await emitMatrixEvent(page, lobbySnapshot({
+ reason: 'concurrent_duplicate_queue',
+ queue: [
+ lobbyEntry({ requested_unix_ms: 1_780_600_000_000 }),
+ lobbyEntry({ requested_unix_ms: 1_780_600_000_100 }),
+ ],
+ }));
+
+ const lobbyPanel = await openLobbyPanel(page);
+ await expect(lobbyPanel.locator('.user-row', { hasText: waitingUserName })).toHaveCount(1);
+ await expect(page.locator('.tab-lobby .tab-notice-badge')).toHaveText('1');
+
+ const allowButton = lobbyPanel.locator('button[title="Allow user"]');
+ const removeButton = lobbyPanel.locator('button[title="Remove user"]');
+ await expect(allowButton).toHaveCount(1);
+ await expect(removeButton).toHaveCount(1);
+ await expect(allowButton).toBeEnabled();
+ await expect(removeButton).toBeEnabled();
+
+ await allowButton.click();
+ await expect.poll(() => page.evaluate(() => (
+ (window.__matrixSocketFrames || []).filter((frame) => frame?.type === 'lobby/allow').length
+ ))).toBe(1);
+ await expect(allowButton).toBeDisabled();
+
+ await emitMatrixEvent(page, lobbySnapshot({
+ reason: 'concurrent_admitted_wins_over_stale_queue',
+ queue: [
+ lobbyEntry({ requested_unix_ms: 1_780_600_000_200 }),
+ lobbyEntry({ requested_unix_ms: 1_780_600_000_300 }),
+ ],
+ admitted: [
+ admittedEntry({ admitted_unix_ms: 1_780_600_001_000 }),
+ admittedEntry({ admitted_unix_ms: 1_780_600_001_050 }),
+ ],
+ }));
+
+ await expect(lobbyPanel.locator('.user-row', { hasText: waitingUserName })).toHaveCount(0);
+ await expect(lobbyPanel.locator('button[title="Allow user"]')).toHaveCount(0);
+ await expect(lobbyPanel.locator('.user-list-empty')).toBeVisible();
+ await expect(page.locator('.tab-lobby .tab-notice-badge')).toHaveCount(0);
+
+ const usersPanel = await openUsersPanel(page);
+ await setWorkspaceParticipants(page, [
+ participantRow({
+ connectionId: 'conn-admin',
+ userId: matrixUsers.admin.id,
+ displayName: matrixUsers.admin.displayName,
+ role: matrixUsers.admin.role,
+ callRole: matrixUsers.admin.callRole,
+ }),
+ participantRow({
+ connectionId: 'conn-user',
+ userId: matrixUsers.user.id,
+ displayName: matrixUsers.user.displayName,
+ role: matrixUsers.user.role,
+ callRole: matrixUsers.user.callRole,
+ }),
+ participantRow({ connectionId: 'conn-waiting-a', userId: waitingUserId, displayName: waitingUserName }),
+ participantRow({ connectionId: 'conn-waiting-b', userId: waitingUserId, displayName: waitingUserName }),
+ ]);
+
+ await expect(usersPanel.locator('.user-row', { hasText: waitingUserName })).toHaveCount(1);
+ await expect(usersPanel.locator('.user-row', { hasText: waitingUserName }).locator('button[title="Remove from lobby"]')).toBeDisabled();
+
+ await openLobbyPanel(page);
+ await emitMatrixEvent(page, lobbySnapshot({ reason: 'reject_final_empty' }));
+ await expect(lobbyPanel.locator('.user-row', { hasText: waitingUserName })).toHaveCount(0);
+ await expect(lobbyPanel.locator('button[title="Allow user"]')).toHaveCount(0);
+ await expect(lobbyPanel.locator('.user-list-empty')).toBeVisible();
+ } finally {
+ await context.close();
+ }
+});
diff --git a/demo/video-chat/frontend-vue/tests/e2e/realtime-reconnect-websocket.spec.js b/demo/video-chat/frontend-vue/tests/e2e/realtime-reconnect-websocket.spec.js
new file mode 100644
index 000000000..884cb5e8d
--- /dev/null
+++ b/demo/video-chat/frontend-vue/tests/e2e/realtime-reconnect-websocket.spec.js
@@ -0,0 +1,151 @@
+import { test, expect } from '@playwright/test';
+import {
+ createMatrixPage,
+ matrixCallRef,
+ matrixUsers,
+ sessionStorageKey,
+} from './helpers/videochatMatrixHarness.js';
+
+async function openMatrixWorkspaceWithRealtimeSocket(page) {
+ await page.goto(`/workspace/call/${matrixCallRef}`);
+ await page.waitForSelector('.workspace-call-view');
+ await page.waitForFunction(() => {
+ const setup = document.querySelector('.workspace-call-view')?.__vueParentComponent?.setupState;
+ return setup?.connectionState === 'online';
+ });
+ await page.waitForFunction(() => (
+ (window.__matrixSocketFrames || []).some((frame) => frame?.type === 'room/snapshot/request')
+ ));
+}
+
+async function reconnectProbe(page) {
+ return page.evaluate((storageKey) => {
+ const setup = document.querySelector('.workspace-call-view')?.__vueParentComponent?.setupState;
+ const frames = Array.isArray(window.__matrixSocketFrames) ? window.__matrixSocketFrames : [];
+ const sockets = Array.isArray(window.__matrixSockets) ? window.__matrixSockets : [];
+ const fetchCalls = Array.isArray(window.__matrixFetchCalls) ? window.__matrixFetchCalls : [];
+ const lifecycle = Array.isArray(window.__matrixSocketLifecycle) ? window.__matrixSocketLifecycle : [];
+ const pageLifecycle = Array.isArray(window.__matrixPageLifecycleEvents) ? window.__matrixPageLifecycleEvents : [];
+
+ return {
+ connectionState: String(setup?.connectionState || ''),
+ connectionReason: String(setup?.connectionReason || ''),
+ currentUrl: window.location.href,
+ storedSessionPresent: Boolean(localStorage.getItem(storageKey)),
+ socketCount: sockets.filter((socket) => String(socket?.url || '').includes('/ws?')).length,
+ snapshotRequests: frames.filter((frame) => frame?.type === 'room/snapshot/request').length,
+ fetchCalls,
+ lifecycle,
+ pageLifecycle,
+ };
+ }, sessionStorageKey);
+}
+
+async function waitForReconnectBackfill(page, previous) {
+ await page.waitForFunction((before) => {
+ const setup = document.querySelector('.workspace-call-view')?.__vueParentComponent?.setupState;
+ const frames = Array.isArray(window.__matrixSocketFrames) ? window.__matrixSocketFrames : [];
+ const sockets = Array.isArray(window.__matrixSockets) ? window.__matrixSockets : [];
+ const snapshotRequests = frames.filter((frame) => frame?.type === 'room/snapshot/request').length;
+ const socketCount = sockets.filter((socket) => String(socket?.url || '').includes('/ws?')).length;
+ return setup?.connectionState === 'online'
+ && socketCount > before.socketCount
+ && snapshotRequests > before.snapshotRequests;
+ }, previous, { timeout: 12_000 });
+}
+
+function expectNoLogoutOrReload(probe, observedNavigationUrls, logoutRequests, initialUrl) {
+ expect(probe.currentUrl).toBe(initialUrl);
+ expect(probe.storedSessionPresent).toBe(true);
+ expect(observedNavigationUrls).toEqual([]);
+ expect(logoutRequests).toEqual([]);
+ expect(probe.fetchCalls.filter((call) => String(call.url || '').includes('/api/auth/logout'))).toEqual([]);
+ expect(probe.pageLifecycle).toEqual([]);
+ expect(probe.connectionState).toBe('online');
+}
+
+test('retryable websocket auth error keeps the browser session and requests room snapshot after reconnect', async ({ browser }) => {
+ test.setTimeout(90_000);
+ const baseURL = test.info().project.use.baseURL || 'http://127.0.0.1:4174';
+ const admin = await createMatrixPage(browser, baseURL, matrixUsers.admin);
+ const observedNavigationUrls = [];
+ const logoutRequests = [];
+
+ try {
+ await openMatrixWorkspaceWithRealtimeSocket(admin.page);
+ const initialUrl = admin.page.url();
+ admin.page.on('framenavigated', (frame) => {
+ if (frame === admin.page.mainFrame()) observedNavigationUrls.push(frame.url());
+ });
+ admin.page.on('request', (request) => {
+ if (request.url().includes('/api/auth/logout')) logoutRequests.push(request.url());
+ });
+
+ const before = await reconnectProbe(admin.page);
+ expect(before.snapshotRequests).toBeGreaterThan(0);
+
+ const emitted = await admin.page.evaluate(() => window.__matrixEmitRetryableAuthError());
+ expect(emitted).toBe(true);
+ await admin.page.waitForFunction(() => {
+ const setup = document.querySelector('.workspace-call-view')?.__vueParentComponent?.setupState;
+ const lifecycle = window.__matrixSocketLifecycle || [];
+ return setup?.connectionState === 'retrying'
+ || lifecycle.some((event) => event?.type === 'close' && event?.reason === 'client_close');
+ });
+
+ await waitForReconnectBackfill(admin.page, before);
+ const after = await reconnectProbe(admin.page);
+ expect(after.snapshotRequests).toBeGreaterThan(before.snapshotRequests);
+ expectNoLogoutOrReload(after, observedNavigationUrls, logoutRequests, initialUrl);
+ } finally {
+ await Promise.allSettled([admin.context.close()]);
+ }
+});
+
+test('retryable websocket backfill handshake failure retries and backfills the room snapshot without logout', async ({ browser }) => {
+ test.setTimeout(90_000);
+ const baseURL = test.info().project.use.baseURL || 'http://127.0.0.1:4174';
+ const admin = await createMatrixPage(browser, baseURL, matrixUsers.admin);
+ const observedNavigationUrls = [];
+ const logoutRequests = [];
+
+ try {
+ await openMatrixWorkspaceWithRealtimeSocket(admin.page);
+ const initialUrl = admin.page.url();
+ admin.page.on('framenavigated', (frame) => {
+ if (frame === admin.page.mainFrame()) observedNavigationUrls.push(frame.url());
+ });
+ admin.page.on('request', (request) => {
+ if (request.url().includes('/api/auth/logout')) logoutRequests.push(request.url());
+ });
+
+ const before = await reconnectProbe(admin.page);
+ expect(before.snapshotRequests).toBeGreaterThan(0);
+
+ const queued = await admin.page.evaluate(() => window.__matrixQueueSocketConnectFailure({
+ code: 1011,
+ reason: 'websocket_reconnect_backfill_unavailable',
+ }));
+ expect(queued).toBe(1);
+ const dropped = await admin.page.evaluate(() => window.__matrixForceSocketClose(1006, 'network_drop'));
+ expect(dropped).toBe(true);
+
+ await admin.page.waitForFunction(() => (
+ (window.__matrixSocketLifecycle || []).some((event) => (
+ event?.type === 'connect-failure'
+ && event?.reason === 'websocket_reconnect_backfill_unavailable'
+ ))
+ ), null, { timeout: 10_000 });
+ await waitForReconnectBackfill(admin.page, before);
+
+ const after = await reconnectProbe(admin.page);
+ expect(after.snapshotRequests).toBeGreaterThan(before.snapshotRequests);
+ expect(after.lifecycle.some((event) => (
+ event?.type === 'connect-failure'
+ && event?.reason === 'websocket_reconnect_backfill_unavailable'
+ ))).toBe(true);
+ expectNoLogoutOrReload(after, observedNavigationUrls, logoutRequests, initialUrl);
+ } finally {
+ await Promise.allSettled([admin.context.close()]);
+ }
+});
diff --git a/demo/video-chat/scripts/deploy-smoke.sh b/demo/video-chat/scripts/deploy-smoke.sh
index 1aac66a88..83d0c3e59 100755
--- a/demo/video-chat/scripts/deploy-smoke.sh
+++ b/demo/video-chat/scripts/deploy-smoke.sh
@@ -82,6 +82,45 @@ expect_http_code() {
log "${label}: HTTP ${code}"
}
+expect_response_header_contains() {
+ local label="$1" url="$2" header_name="$3" expected_value="$4"
+ local headers code header_value
+ headers="$(mktemp)"
+ code="$(curl -sS --max-time "${TIMEOUT}" -o /dev/null -D "${headers}" -w '%{http_code}' "${url}" || true)"
+ if [[ "${code}" != "200" ]]; then
+ cat "${headers}" >&2 || true
+ rm -f "${headers}"
+ fail "${label}: expected HTTP 200 while checking ${header_name}, got ${code:-none}"
+ fi
+ header_value="$(awk -v name="${header_name}" 'BEGIN { lower = tolower(name) ":" } tolower($0) ~ "^" lower { sub("^[^:]*:[[:space:]]*", "", $0); gsub("\r", "", $0); print $0; exit }' "${headers}")"
+ if [[ -z "${header_value}" || "${header_value}" != *"${expected_value}"* ]]; then
+ cat "${headers}" >&2 || true
+ rm -f "${headers}"
+ fail "${label}: expected ${header_name} to contain ${expected_value}"
+ fi
+ rm -f "${headers}"
+ log "${label}: ${header_name} contains ${expected_value}"
+}
+
+expect_response_header_absent() {
+ local label="$1" url="$2" header_name="$3"
+ local headers code
+ headers="$(mktemp)"
+ code="$(curl -sS --max-time "${TIMEOUT}" -o /dev/null -D "${headers}" -w '%{http_code}' "${url}" || true)"
+ if [[ "${code}" != "200" ]]; then
+ cat "${headers}" >&2 || true
+ rm -f "${headers}"
+ fail "${label}: expected HTTP 200 while checking ${header_name}, got ${code:-none}"
+ fi
+ if awk -v name="${header_name}" 'BEGIN { lower = tolower(name) ":" } tolower($0) ~ "^" lower { found = 1 } END { exit found ? 0 : 1 }' "${headers}"; then
+ cat "${headers}" >&2 || true
+ rm -f "${headers}"
+ fail "${label}: ${header_name} must not be present"
+ fi
+ rm -f "${headers}"
+ log "${label}: ${header_name} absent"
+}
+
public_get_json() {
local label="$1" url="$2" output code
output="$(mktemp)"
@@ -165,8 +204,9 @@ assert_public_localization_payload() {
check_https_redirect() {
local headers code location
+ local frontend_domain="${DEPLOY_APP_DOMAIN:-${DEPLOY_DOMAIN}}"
headers="$(mktemp)"
- code="$(curl -sS --max-time "${TIMEOUT}" -D "${headers}" -o /dev/null -w '%{http_code}' "http://${DEPLOY_DOMAIN}/" || true)"
+ code="$(curl -sS --max-time "${TIMEOUT}" -D "${headers}" -o /dev/null -w '%{http_code}' "http://${frontend_domain}/" || true)"
case "${code}" in
301|308) ;;
*)
@@ -178,7 +218,7 @@ check_https_redirect() {
location="$(awk 'tolower($1) == "location:" {print $2; exit}' "${headers}" | tr -d '\r')"
rm -f "${headers}"
- [[ "${location}" == "https://${DEPLOY_DOMAIN}/"* ]] || fail "http redirect location mismatch: ${location:-missing}"
+ [[ "${location}" == "https://${frontend_domain}/"* ]] || fail "http redirect location mismatch: ${location:-missing}"
log "http redirect: ${code} -> ${location}"
}
@@ -239,23 +279,27 @@ verify_remote_certbot_hook() {
[[ -n "${DEPLOY_HOST}" ]] || fail "VIDEOCHAT_DEPLOY_HOST is required for certbot renewal-hook smoke"
require_cmd ssh
- local ssh_dest sudo_value domain_q api_q ws_q sfu_q turn_q cdn_q
+ local ssh_dest sudo_value domain_q app_q api_q ws_q sfu_q turn_q cdn_q call_app_q registry_q mothernode_q
ssh_dest="${DEPLOY_USER}@${DEPLOY_HOST}"
sudo_value=""
[[ "${DEPLOY_USER}" != "root" ]] && sudo_value="sudo"
domain_q="$(shell_quote "${DEPLOY_DOMAIN}")"
+ app_q="$(shell_quote "${DEPLOY_APP_DOMAIN}")"
api_q="$(shell_quote "${DEPLOY_API_DOMAIN}")"
ws_q="$(shell_quote "${DEPLOY_WS_DOMAIN}")"
sfu_q="$(shell_quote "${DEPLOY_SFU_DOMAIN}")"
turn_q="$(shell_quote "${DEPLOY_TURN_DOMAIN}")"
cdn_q="$(shell_quote "${DEPLOY_CDN_DOMAIN}")"
+ call_app_q="$(shell_quote "${DEPLOY_CALL_APP_DOMAIN}")"
+ registry_q="$(shell_quote "${DEPLOY_REGISTRY_DOMAIN}")"
+ mothernode_q="$(shell_quote "${DEPLOY_MOTHERNODE_DOMAIN}")"
local ssh_args=(-p "${DEPLOY_SSH_PORT}" -o BatchMode=yes -o StrictHostKeyChecking=accept-new)
if [[ -n "${VIDEOCHAT_DEPLOY_SSH_KEY:-}" ]]; then
ssh_args+=(-i "${VIDEOCHAT_DEPLOY_SSH_KEY}")
fi
- ssh "${ssh_args[@]}" "${ssh_dest}" "SUDO=$(shell_quote "${sudo_value}") DOMAIN=${domain_q} API_DOMAIN=${api_q} WS_DOMAIN=${ws_q} SFU_DOMAIN=${sfu_q} TURN_DOMAIN=${turn_q} CDN_DOMAIN=${cdn_q} bash -s" <<'REMOTE'
+ ssh "${ssh_args[@]}" "${ssh_dest}" "SUDO=$(shell_quote "${sudo_value}") DOMAIN=${domain_q} APP_DOMAIN=${app_q} API_DOMAIN=${api_q} WS_DOMAIN=${ws_q} SFU_DOMAIN=${sfu_q} TURN_DOMAIN=${turn_q} CDN_DOMAIN=${cdn_q} CALL_APP_DOMAIN=${call_app_q} REGISTRY_DOMAIN=${registry_q} MOTHERNODE_DOMAIN=${mothernode_q} bash -s" <<'REMOTE'
set -euo pipefail
HOOK=/etc/letsencrypt/renewal-hooks/deploy/king-videochat-restart.sh
${SUDO} test -x "${HOOK}"
@@ -263,9 +307,12 @@ ${SUDO} grep -Fq 'docker compose' "${HOOK}"
${SUDO} grep -Fq 'restart || true' "${HOOK}"
cert_output="$(${SUDO} certbot certificates -d "${DOMAIN}" 2>&1)"
printf '%s\n' "${cert_output}" | grep -Fq "Certificate Name: ${DOMAIN}"
-for expected in "${DOMAIN}" "${API_DOMAIN}" "${WS_DOMAIN}" "${SFU_DOMAIN}" "${TURN_DOMAIN}" "${CDN_DOMAIN}"; do
+for expected in "${DOMAIN}" "${APP_DOMAIN}" "${API_DOMAIN}" "${WS_DOMAIN}" "${SFU_DOMAIN}" "${TURN_DOMAIN}" "${CDN_DOMAIN}" "${CALL_APP_DOMAIN}" "${REGISTRY_DOMAIN}"; do
printf '%s\n' "${cert_output}" | grep -Fq "${expected}"
done
+if [ -n "${MOTHERNODE_DOMAIN}" ] && [ "${MOTHERNODE_DOMAIN}" != "${REGISTRY_DOMAIN}" ]; then
+ printf '%s\n' "${cert_output}" | grep -Fq "${MOTHERNODE_DOMAIN}"
+fi
REMOTE
log "certbot renewal hook and certificate SANs verified on ${ssh_dest}"
@@ -704,17 +751,30 @@ DEPLOY_DOMAIN="${VIDEOCHAT_DEPLOY_DOMAIN:-${VIDEOCHAT_V1_PUBLIC_HOST:-}}"
DEPLOY_USER="${VIDEOCHAT_DEPLOY_USER:-root}"
DEPLOY_SSH_PORT="${VIDEOCHAT_DEPLOY_SSH_PORT:-22}"
DEPLOY_HOST="${VIDEOCHAT_DEPLOY_HOST:-}"
+DEPLOY_APP_DOMAIN="${VIDEOCHAT_DEPLOY_APP_DOMAIN:-app.${DEPLOY_DOMAIN}}"
DEPLOY_API_DOMAIN="${VIDEOCHAT_DEPLOY_API_DOMAIN:-api.${DEPLOY_DOMAIN}}"
DEPLOY_WS_DOMAIN="${VIDEOCHAT_DEPLOY_WS_DOMAIN:-ws.${DEPLOY_DOMAIN}}"
DEPLOY_SFU_DOMAIN="${VIDEOCHAT_DEPLOY_SFU_DOMAIN:-sfu.${DEPLOY_DOMAIN}}"
DEPLOY_TURN_DOMAIN="${VIDEOCHAT_DEPLOY_TURN_DOMAIN:-turn.${DEPLOY_DOMAIN}}"
DEPLOY_CDN_DOMAIN="${VIDEOCHAT_DEPLOY_CDN_DOMAIN:-cdn.${DEPLOY_DOMAIN}}"
+DEPLOY_CALL_APP_DOMAIN="${VIDEOCHAT_DEPLOY_CALL_APP_DOMAIN:-whiteboard.${DEPLOY_DOMAIN}}"
+DEPLOY_REGISTRY_DOMAIN="${VIDEOCHAT_DEPLOY_REGISTRY_DOMAIN:-${VIDEOCHAT_DEPLOY_MOTHERNODE_DOMAIN:-registry.${DEPLOY_DOMAIN}}}"
+DEPLOY_MOTHERNODE_DOMAIN="${VIDEOCHAT_DEPLOY_MOTHERNODE_DOMAIN:-${DEPLOY_REGISTRY_DOMAIN}}"
check_https_redirect
-expect_http_code https-frontend 200 "https://${DEPLOY_DOMAIN}/"
+expect_http_code https-frontend 200 "https://${DEPLOY_APP_DOMAIN}/"
expect_http_code cdn-mediapipe-tasks-model 200 "https://${DEPLOY_CDN_DOMAIN}/cdn/vendor/mediapipe/models/selfie_multiclass_256x256.tflite"
expect_http_code cdn-mediapipe-wasm-loader 200 "https://${DEPLOY_CDN_DOMAIN}/cdn/vendor/mediapipe/selfie_segmentation/selfie_segmentation_solution_simd_wasm_bin.js"
expect_http_code cdn-tensorflow-fallback-loader 200 "https://${DEPLOY_CDN_DOMAIN}/cdn/vendor/tensorflow/tfjs-core/tf-core.min.js"
+expect_http_code call-app-whiteboard-host 200 "https://${DEPLOY_CALL_APP_DOMAIN}/public/index.html"
+expect_http_code call-app-whiteboard-path 200 "https://${DEPLOY_CALL_APP_DOMAIN}/call-app/whiteboard/public/index.html"
+for call_app_url in "https://${DEPLOY_CALL_APP_DOMAIN}/public/index.html" "https://${DEPLOY_CALL_APP_DOMAIN}/call-app/whiteboard/public/index.html"; do
+ expect_response_header_contains call-app-frame-ancestor "${call_app_url}" Content-Security-Policy "frame-ancestors https://${DEPLOY_APP_DOMAIN}"
+ expect_response_header_contains call-app-script-csp "${call_app_url}" Content-Security-Policy "script-src 'self'"
+ expect_response_header_contains call-app-connect-csp "${call_app_url}" Content-Security-Policy "connect-src 'self'"
+ expect_response_header_contains call-app-embedded-csp "${call_app_url}" Allow-CSP-From "https://${DEPLOY_APP_DOMAIN}"
+ expect_response_header_absent call-app-x-frame-options "${call_app_url}" X-Frame-Options
+done
health_payload="$(public_get_json "api health" "https://${DEPLOY_API_DOMAIN}/health")"
printf '%s' "${health_payload}" | assert_public_health_payload
diff --git a/demo/video-chat/scripts/deploy.sh b/demo/video-chat/scripts/deploy.sh
index 4e7e063c7..e185abc00 100755
--- a/demo/video-chat/scripts/deploy.sh
+++ b/demo/video-chat/scripts/deploy.sh
@@ -10,7 +10,7 @@ usage() {
cat <<'USAGE'
Usage:
VIDEOCHAT_DEPLOY_HOST= \
- VIDEOCHAT_DEPLOY_DOMAIN= \
+ VIDEOCHAT_DEPLOY_DOMAIN= \
VIDEOCHAT_DEPLOY_EMAIL= \
demo/video-chat/scripts/deploy.sh [wizard|deploy|prepare|public-http|http-preview|status|credentials|certonly|sync]
@@ -24,7 +24,7 @@ Local state:
Required remote environment:
VIDEOCHAT_DEPLOY_HOST SSH target host/IP.
- VIDEOCHAT_DEPLOY_DOMAIN Public DNS name. Its A/AAAA record must point to the server.
+ VIDEOCHAT_DEPLOY_DOMAIN Service base/root DNS name. Its A/AAAA record must point to the server.
VIDEOCHAT_DEPLOY_EMAIL Let's Encrypt registration/notification email.
Optional environment:
@@ -51,15 +51,18 @@ Optional environment:
Known hosts file for refresh, default: ~/.ssh/known_hosts.
VIDEOCHAT_DEPLOY_REMOTE_LOCALE
Locale for remote shell commands, default: C.UTF-8.
+ VIDEOCHAT_DEPLOY_APP_DOMAIN Frontend app host, default: app..
VIDEOCHAT_DEPLOY_API_DOMAIN API host, default: api..
VIDEOCHAT_DEPLOY_WS_DOMAIN Lobby websocket host, default: ws..
VIDEOCHAT_DEPLOY_SFU_DOMAIN SFU websocket host, default: sfu..
VIDEOCHAT_DEPLOY_TURN_DOMAIN TURN host, default: turn..
VIDEOCHAT_DEPLOY_CDN_DOMAIN Static/CDN asset host, default: cdn..
VIDEOCHAT_DEPLOY_CALL_APP_DOMAIN
- Call App iframe host, default: apps..
+ Whiteboard Call App iframe host, default: whiteboard..
+ VIDEOCHAT_DEPLOY_REGISTRY_DOMAIN
+ Registry/Mothernode host, default: registry..
VIDEOCHAT_DEPLOY_MOTHERNODE_DOMAIN
- Call App Mothernode host, default: mother..
+ Legacy env name for the registry host, default: registry..
VIDEOCHAT_DEPLOY_EXTERNAL_DOMAINS
Optional comma-separated hostnames to route to an
external HTTP upstream through the King edge.
@@ -181,40 +184,48 @@ refresh_deploy_config() {
DEPLOY_REFRESH_KNOWN_HOSTS="1"
fi
DEPLOY_REMOTE_LOCALE="${VIDEOCHAT_DEPLOY_REMOTE_LOCALE:-C.UTF-8}"
+ DEPLOY_APP_DOMAIN="${VIDEOCHAT_DEPLOY_APP_DOMAIN:-}"
DEPLOY_API_DOMAIN="${VIDEOCHAT_DEPLOY_API_DOMAIN:-}"
DEPLOY_WS_DOMAIN="${VIDEOCHAT_DEPLOY_WS_DOMAIN:-}"
DEPLOY_SFU_DOMAIN="${VIDEOCHAT_DEPLOY_SFU_DOMAIN:-}"
DEPLOY_TURN_DOMAIN="${VIDEOCHAT_DEPLOY_TURN_DOMAIN:-}"
DEPLOY_CDN_DOMAIN="${VIDEOCHAT_DEPLOY_CDN_DOMAIN:-}"
DEPLOY_CALL_APP_DOMAIN="${VIDEOCHAT_DEPLOY_CALL_APP_DOMAIN:-}"
+ DEPLOY_REGISTRY_DOMAIN="${VIDEOCHAT_DEPLOY_REGISTRY_DOMAIN:-}"
DEPLOY_MOTHERNODE_DOMAIN="${VIDEOCHAT_DEPLOY_MOTHERNODE_DOMAIN:-}"
DEPLOY_EXTERNAL_DOMAINS="${VIDEOCHAT_DEPLOY_EXTERNAL_DOMAINS:-}"
DEPLOY_EXTERNAL_UPSTREAM="${VIDEOCHAT_DEPLOY_EXTERNAL_UPSTREAM:-}"
if [[ -n "${DEPLOY_DOMAIN}" ]]; then
+ DEPLOY_APP_DOMAIN="${DEPLOY_APP_DOMAIN:-app.${DEPLOY_DOMAIN}}"
DEPLOY_API_DOMAIN="${DEPLOY_API_DOMAIN:-api.${DEPLOY_DOMAIN}}"
DEPLOY_WS_DOMAIN="${DEPLOY_WS_DOMAIN:-ws.${DEPLOY_DOMAIN}}"
DEPLOY_SFU_DOMAIN="${DEPLOY_SFU_DOMAIN:-sfu.${DEPLOY_DOMAIN}}"
DEPLOY_TURN_DOMAIN="${DEPLOY_TURN_DOMAIN:-turn.${DEPLOY_DOMAIN}}"
DEPLOY_CDN_DOMAIN="${DEPLOY_CDN_DOMAIN:-cdn.${DEPLOY_DOMAIN}}"
- DEPLOY_CALL_APP_DOMAIN="${DEPLOY_CALL_APP_DOMAIN:-apps.${DEPLOY_DOMAIN}}"
- DEPLOY_MOTHERNODE_DOMAIN="${DEPLOY_MOTHERNODE_DOMAIN:-mother.${DEPLOY_DOMAIN}}"
+ DEPLOY_CALL_APP_DOMAIN="${DEPLOY_CALL_APP_DOMAIN:-whiteboard.${DEPLOY_DOMAIN}}"
+ DEPLOY_REGISTRY_DOMAIN="${DEPLOY_REGISTRY_DOMAIN:-registry.${DEPLOY_DOMAIN}}"
+ DEPLOY_MOTHERNODE_DOMAIN="${DEPLOY_MOTHERNODE_DOMAIN:-${DEPLOY_REGISTRY_DOMAIN}}"
fi
+ DEPLOY_REGISTRY_DOMAIN="${DEPLOY_REGISTRY_DOMAIN:-${DEPLOY_MOTHERNODE_DOMAIN}}"
+ DEPLOY_MOTHERNODE_DOMAIN="${DEPLOY_MOTHERNODE_DOMAIN:-${DEPLOY_REGISTRY_DOMAIN}}"
DEPLOY_VUE_ALLOWED_HOSTS="${VIDEOCHAT_DEPLOY_VUE_ALLOWED_HOSTS:-}"
if [[ -z "${DEPLOY_VUE_ALLOWED_HOSTS}" && -n "${DEPLOY_DOMAIN}" ]]; then
- DEPLOY_VUE_ALLOWED_HOSTS="${DEPLOY_DOMAIN},${DEPLOY_API_DOMAIN},${DEPLOY_WS_DOMAIN},${DEPLOY_SFU_DOMAIN},${DEPLOY_TURN_DOMAIN},${DEPLOY_CDN_DOMAIN},${DEPLOY_CALL_APP_DOMAIN},${DEPLOY_MOTHERNODE_DOMAIN}"
+ DEPLOY_VUE_ALLOWED_HOSTS="${DEPLOY_DOMAIN},${DEPLOY_APP_DOMAIN},${DEPLOY_API_DOMAIN},${DEPLOY_WS_DOMAIN},${DEPLOY_SFU_DOMAIN},${DEPLOY_TURN_DOMAIN},${DEPLOY_CDN_DOMAIN},${DEPLOY_CALL_APP_DOMAIN},${DEPLOY_REGISTRY_DOMAIN}"
fi
if [[ -n "${DEPLOY_EXTERNAL_DOMAINS}" ]]; then
DEPLOY_VUE_ALLOWED_HOSTS="${DEPLOY_VUE_ALLOWED_HOSTS:+${DEPLOY_VUE_ALLOWED_HOSTS},}${DEPLOY_EXTERNAL_DOMAINS}"
fi
+ export VIDEOCHAT_DEPLOY_APP_DOMAIN="${DEPLOY_APP_DOMAIN}"
export VIDEOCHAT_DEPLOY_API_DOMAIN="${DEPLOY_API_DOMAIN}"
export VIDEOCHAT_DEPLOY_WS_DOMAIN="${DEPLOY_WS_DOMAIN}"
export VIDEOCHAT_DEPLOY_SFU_DOMAIN="${DEPLOY_SFU_DOMAIN}"
export VIDEOCHAT_DEPLOY_TURN_DOMAIN="${DEPLOY_TURN_DOMAIN}"
export VIDEOCHAT_DEPLOY_CDN_DOMAIN="${DEPLOY_CDN_DOMAIN}"
export VIDEOCHAT_DEPLOY_CALL_APP_DOMAIN="${DEPLOY_CALL_APP_DOMAIN}"
+ export VIDEOCHAT_DEPLOY_REGISTRY_DOMAIN="${DEPLOY_REGISTRY_DOMAIN}"
export VIDEOCHAT_DEPLOY_MOTHERNODE_DOMAIN="${DEPLOY_MOTHERNODE_DOMAIN}"
export VIDEOCHAT_DEPLOY_EXTERNAL_DOMAINS="${DEPLOY_EXTERNAL_DOMAINS}"
export VIDEOCHAT_DEPLOY_EXTERNAL_UPSTREAM="${DEPLOY_EXTERNAL_UPSTREAM}"
@@ -244,11 +255,9 @@ deploy_refresh_known_hosts_enabled() {
deploy_dns_targets() {
local target seen=""
- local legacy_cdn_domain=""
local external_domains=()
- [[ -n "${DEPLOY_DOMAIN}" ]] && legacy_cdn_domain="cnd.${DEPLOY_DOMAIN}"
IFS=',' read -r -a external_domains <<< "${DEPLOY_EXTERNAL_DOMAINS}"
- for target in "${DEPLOY_DOMAIN}" "${DEPLOY_API_DOMAIN}" "${DEPLOY_WS_DOMAIN}" "${DEPLOY_SFU_DOMAIN}" "${DEPLOY_TURN_DOMAIN}" "${DEPLOY_CDN_DOMAIN}" "${DEPLOY_CALL_APP_DOMAIN}" "${DEPLOY_MOTHERNODE_DOMAIN}" "${legacy_cdn_domain}" "${external_domains[@]}"; do
+ for target in "${DEPLOY_DOMAIN}" "${DEPLOY_APP_DOMAIN}" "${DEPLOY_API_DOMAIN}" "${DEPLOY_WS_DOMAIN}" "${DEPLOY_SFU_DOMAIN}" "${DEPLOY_TURN_DOMAIN}" "${DEPLOY_CDN_DOMAIN}" "${DEPLOY_CALL_APP_DOMAIN}" "${DEPLOY_REGISTRY_DOMAIN}" "${external_domains[@]}"; do
target="${target//[[:space:]]/}"
[[ -n "${target}" ]] || continue
case " ${seen} " in
@@ -326,12 +335,14 @@ persist_current_deploy_config() {
local_env_upsert VIDEOCHAT_DEPLOY_RSYNC_DELETE "${DEPLOY_RSYNC_DELETE}"
local_env_upsert VIDEOCHAT_DEPLOY_REFRESH_KNOWN_HOSTS "${DEPLOY_REFRESH_KNOWN_HOSTS:-0}"
local_env_upsert VIDEOCHAT_DEPLOY_REMOTE_LOCALE "${DEPLOY_REMOTE_LOCALE}"
+ local_env_upsert VIDEOCHAT_DEPLOY_APP_DOMAIN "${DEPLOY_APP_DOMAIN}"
local_env_upsert VIDEOCHAT_DEPLOY_API_DOMAIN "${DEPLOY_API_DOMAIN}"
local_env_upsert VIDEOCHAT_DEPLOY_WS_DOMAIN "${DEPLOY_WS_DOMAIN}"
local_env_upsert VIDEOCHAT_DEPLOY_SFU_DOMAIN "${DEPLOY_SFU_DOMAIN}"
local_env_upsert VIDEOCHAT_DEPLOY_TURN_DOMAIN "${DEPLOY_TURN_DOMAIN}"
local_env_upsert VIDEOCHAT_DEPLOY_CDN_DOMAIN "${DEPLOY_CDN_DOMAIN}"
local_env_upsert VIDEOCHAT_DEPLOY_CALL_APP_DOMAIN "${DEPLOY_CALL_APP_DOMAIN}"
+ local_env_upsert VIDEOCHAT_DEPLOY_REGISTRY_DOMAIN "${DEPLOY_REGISTRY_DOMAIN}"
local_env_upsert VIDEOCHAT_DEPLOY_MOTHERNODE_DOMAIN "${DEPLOY_MOTHERNODE_DOMAIN}"
local_env_upsert VIDEOCHAT_DEPLOY_EXTERNAL_DOMAINS "${DEPLOY_EXTERNAL_DOMAINS}"
local_env_upsert VIDEOCHAT_DEPLOY_EXTERNAL_UPSTREAM "${DEPLOY_EXTERNAL_UPSTREAM}"
@@ -538,9 +549,10 @@ sync_checkout() {
}
certbot_standalone() {
- local deploy_path_q domain_q email_q api_domain_q ws_domain_q sfu_domain_q turn_domain_q cdn_domain_q call_app_domain_q mothernode_domain_q legacy_cdn_domain_q external_domains_q
+ local deploy_path_q domain_q app_domain_q email_q api_domain_q ws_domain_q sfu_domain_q turn_domain_q cdn_domain_q call_app_domain_q registry_domain_q mothernode_domain_q external_domains_q
deploy_path_q="$(shell_quote "${DEPLOY_PATH}")"
domain_q="$(shell_quote "${DEPLOY_DOMAIN}")"
+ app_domain_q="$(shell_quote "${DEPLOY_APP_DOMAIN}")"
email_q="$(shell_quote "${DEPLOY_EMAIL}")"
api_domain_q="$(shell_quote "${DEPLOY_API_DOMAIN}")"
ws_domain_q="$(shell_quote "${DEPLOY_WS_DOMAIN}")"
@@ -548,8 +560,8 @@ certbot_standalone() {
turn_domain_q="$(shell_quote "${DEPLOY_TURN_DOMAIN}")"
cdn_domain_q="$(shell_quote "${DEPLOY_CDN_DOMAIN}")"
call_app_domain_q="$(shell_quote "${DEPLOY_CALL_APP_DOMAIN}")"
+ registry_domain_q="$(shell_quote "${DEPLOY_REGISTRY_DOMAIN}")"
mothernode_domain_q="$(shell_quote "${DEPLOY_MOTHERNODE_DOMAIN}")"
- legacy_cdn_domain_q="$(shell_quote "cnd.${DEPLOY_DOMAIN}")"
external_domains_q="$(shell_quote "${DEPLOY_EXTERNAL_DOMAINS}")"
log "Obtaining/renewing Let's Encrypt cert for ${DEPLOY_DOMAIN}"
@@ -558,6 +570,7 @@ set -euo pipefail
SUDO="$(sudo_prefix)"
DEPLOY_PATH=${deploy_path_q}
DOMAIN=${domain_q}
+APP_DOMAIN=${app_domain_q}
EMAIL=${email_q}
API_DOMAIN=${api_domain_q}
WS_DOMAIN=${ws_domain_q}
@@ -565,8 +578,8 @@ SFU_DOMAIN=${sfu_domain_q}
TURN_DOMAIN=${turn_domain_q}
CDN_DOMAIN=${cdn_domain_q}
CALL_APP_DOMAIN=${call_app_domain_q}
+REGISTRY_DOMAIN=${registry_domain_q}
MOTHERNODE_DOMAIN=${mothernode_domain_q}
-LEGACY_CDN_DOMAIN=${legacy_cdn_domain_q}
EXTERNAL_DOMAINS=${external_domains_q}
VIDEOCHAT_DIR="\${DEPLOY_PATH}/demo/video-chat"
FRONTEND_WAS_RUNNING=0
@@ -631,16 +644,17 @@ fi
CERTBOT_DOMAINS=(
-d "\${DOMAIN}"
+ -d "\${APP_DOMAIN}"
-d "\${API_DOMAIN}"
-d "\${WS_DOMAIN}"
-d "\${SFU_DOMAIN}"
-d "\${TURN_DOMAIN}"
-d "\${CDN_DOMAIN}"
-d "\${CALL_APP_DOMAIN}"
- -d "\${MOTHERNODE_DOMAIN}"
+ -d "\${REGISTRY_DOMAIN}"
)
-if [ -n "\${LEGACY_CDN_DOMAIN}" ] && [ "\${LEGACY_CDN_DOMAIN}" != "\${CDN_DOMAIN}" ]; then
- CERTBOT_DOMAINS+=(-d "\${LEGACY_CDN_DOMAIN}")
+if [ -n "\${MOTHERNODE_DOMAIN}" ] && [ "\${MOTHERNODE_DOMAIN}" != "\${REGISTRY_DOMAIN}" ]; then
+ CERTBOT_DOMAINS+=(-d "\${MOTHERNODE_DOMAIN}")
fi
IFS=',' read -r -a EXTRA_CERT_DOMAINS <<< "\${EXTERNAL_DOMAINS}"
for cert_domain in "\${EXTRA_CERT_DOMAINS[@]}"; do
@@ -669,18 +683,20 @@ REMOTE
}
write_remote_runtime_files() {
- local deploy_path_q domain_q api_domain_q ws_domain_q sfu_domain_q turn_domain_q cdn_domain_q call_app_domain_q mothernode_domain_q external_domains_q external_upstream_q turn_external_ip_q admin_q user_q turn_q vue_allowed_hosts_q
+ local deploy_path_q domain_q app_domain_q api_domain_q ws_domain_q sfu_domain_q turn_domain_q cdn_domain_q call_app_domain_q registry_domain_q mothernode_domain_q external_domains_q external_upstream_q turn_external_ip_q admin_q user_q turn_q vue_allowed_hosts_q
local infra_provider_q infra_cluster_q infra_node_roles_q infra_local_node_name_q infra_local_public_ip_q infra_hcloud_token_q infra_hcloud_api_base_q
local otel_enable_q otel_endpoint_q otel_protocol_q otel_metrics_q otel_logs_q
local allow_insecure_ws_q
deploy_path_q="$(shell_quote "${DEPLOY_PATH}")"
domain_q="$(shell_quote "${DEPLOY_DOMAIN}")"
+ app_domain_q="$(shell_quote "${DEPLOY_APP_DOMAIN}")"
api_domain_q="$(shell_quote "${DEPLOY_API_DOMAIN}")"
ws_domain_q="$(shell_quote "${DEPLOY_WS_DOMAIN}")"
sfu_domain_q="$(shell_quote "${DEPLOY_SFU_DOMAIN}")"
turn_domain_q="$(shell_quote "${DEPLOY_TURN_DOMAIN}")"
cdn_domain_q="$(shell_quote "${DEPLOY_CDN_DOMAIN}")"
call_app_domain_q="$(shell_quote "${DEPLOY_CALL_APP_DOMAIN}")"
+ registry_domain_q="$(shell_quote "${DEPLOY_REGISTRY_DOMAIN}")"
mothernode_domain_q="$(shell_quote "${DEPLOY_MOTHERNODE_DOMAIN}")"
external_domains_q="$(shell_quote "${DEPLOY_EXTERNAL_DOMAINS}")"
external_upstream_q="$(shell_quote "${DEPLOY_EXTERNAL_UPSTREAM}")"
@@ -709,12 +725,14 @@ set -euo pipefail
SUDO="$(sudo_prefix)"
DEPLOY_PATH=${deploy_path_q}
DOMAIN=${domain_q}
+APP_DOMAIN=${app_domain_q}
API_DOMAIN=${api_domain_q}
WS_DOMAIN=${ws_domain_q}
SFU_DOMAIN=${sfu_domain_q}
TURN_DOMAIN=${turn_domain_q}
CDN_DOMAIN=${cdn_domain_q}
CALL_APP_DOMAIN=${call_app_domain_q}
+REGISTRY_DOMAIN=${registry_domain_q}
MOTHERNODE_DOMAIN=${mothernode_domain_q}
EXTERNAL_DOMAINS=${external_domains_q}
EXTERNAL_UPSTREAM=${external_upstream_q}
@@ -764,7 +782,7 @@ ASSET_VERSION="\$(date -u +%Y%m%d%H%M%S)"
\${SUDO}tee "\${LOCAL_ENV}" >/dev/null </dev/null
-printf 'HTTP preview frontend: http://%s:5176\\n' "\${DOMAIN}"
+printf 'HTTP preview frontend: http://%s:5176\\n' "\${APP_DOMAIN}"
REMOTE
}
diff --git a/demo/video-chat/scripts/lib/deploy-hetzner.sh b/demo/video-chat/scripts/lib/deploy-hetzner.sh
index fd8bf6080..7e5e61a25 100644
--- a/demo/video-chat/scripts/lib/deploy-hetzner.sh
+++ b/demo/video-chat/scripts/lib/deploy-hetzner.sh
@@ -109,12 +109,14 @@ persist_wizard_env() {
VIDEOCHAT_DEPLOY_REFRESH_KNOWN_HOSTS
VIDEOCHAT_DEPLOY_KNOWN_HOSTS_FILE
VIDEOCHAT_DEPLOY_REMOTE_LOCALE
+ VIDEOCHAT_DEPLOY_APP_DOMAIN
VIDEOCHAT_DEPLOY_API_DOMAIN
VIDEOCHAT_DEPLOY_WS_DOMAIN
VIDEOCHAT_DEPLOY_SFU_DOMAIN
VIDEOCHAT_DEPLOY_TURN_DOMAIN
VIDEOCHAT_DEPLOY_CDN_DOMAIN
VIDEOCHAT_DEPLOY_CALL_APP_DOMAIN
+ VIDEOCHAT_DEPLOY_REGISTRY_DOMAIN
VIDEOCHAT_DEPLOY_MOTHERNODE_DOMAIN
VIDEOCHAT_DEPLOY_VUE_ALLOWED_HOSTS
VIDEOCHAT_DEPLOY_ADMIN_PASSWORD
@@ -319,7 +321,7 @@ resolved_ips_for_domain() {
wait_for_dns_to_server() {
local timeout="${VIDEOCHAT_DEPLOY_DNS_WAIT_SECONDS:-900}" deadline resolved target target_resolved all_ok
- local targets=("${DEPLOY_DOMAIN}" "${DEPLOY_API_DOMAIN}" "${DEPLOY_WS_DOMAIN}" "${DEPLOY_SFU_DOMAIN}" "${DEPLOY_TURN_DOMAIN}" "${DEPLOY_CDN_DOMAIN}" "${DEPLOY_CALL_APP_DOMAIN}" "${DEPLOY_MOTHERNODE_DOMAIN}")
+ local targets=("${DEPLOY_DOMAIN}" "${DEPLOY_APP_DOMAIN}" "${DEPLOY_API_DOMAIN}" "${DEPLOY_WS_DOMAIN}" "${DEPLOY_SFU_DOMAIN}" "${DEPLOY_TURN_DOMAIN}" "${DEPLOY_CDN_DOMAIN}" "${DEPLOY_CALL_APP_DOMAIN}" "${DEPLOY_REGISTRY_DOMAIN}")
if ! command -v getent >/dev/null 2>&1; then
log "WARN: getent is missing locally; skipping DNS wait"
return 0
@@ -403,9 +405,8 @@ hcloud_set_dns_a_record() {
}
hcloud_set_videochat_subdomain_records() {
- local target seen="" legacy_cdn_domain=""
- [[ -n "${DEPLOY_DOMAIN:-}" ]] && legacy_cdn_domain="cnd.${DEPLOY_DOMAIN}"
- for target in "${DEPLOY_API_DOMAIN}" "${DEPLOY_WS_DOMAIN}" "${DEPLOY_SFU_DOMAIN}" "${DEPLOY_TURN_DOMAIN}" "${DEPLOY_CDN_DOMAIN}" "${DEPLOY_CALL_APP_DOMAIN}" "${DEPLOY_MOTHERNODE_DOMAIN}" "${legacy_cdn_domain}"; do
+ local target seen=""
+ for target in "${DEPLOY_APP_DOMAIN}" "${DEPLOY_API_DOMAIN}" "${DEPLOY_WS_DOMAIN}" "${DEPLOY_SFU_DOMAIN}" "${DEPLOY_TURN_DOMAIN}" "${DEPLOY_CDN_DOMAIN}" "${DEPLOY_CALL_APP_DOMAIN}" "${DEPLOY_REGISTRY_DOMAIN}"; do
[[ -n "${target}" ]] || continue
case " ${seen} " in
*" ${target} "*) continue ;;
@@ -433,7 +434,7 @@ run_hcloud_dns_step() {
hcloud_set_dns_a_record || true
hcloud_set_videochat_subdomain_records
else
- log "Manual DNS required: set A ${DEPLOY_DOMAIN}, ${DEPLOY_API_DOMAIN}, ${DEPLOY_WS_DOMAIN}, ${DEPLOY_SFU_DOMAIN}, ${DEPLOY_TURN_DOMAIN}, ${DEPLOY_CDN_DOMAIN}, ${DEPLOY_CALL_APP_DOMAIN}, ${DEPLOY_MOTHERNODE_DOMAIN} -> ${DEPLOY_PUBLIC_IP}"
+ log "Manual DNS required: set A ${DEPLOY_DOMAIN}, ${DEPLOY_APP_DOMAIN}, ${DEPLOY_API_DOMAIN}, ${DEPLOY_WS_DOMAIN}, ${DEPLOY_SFU_DOMAIN}, ${DEPLOY_TURN_DOMAIN}, ${DEPLOY_CDN_DOMAIN}, ${DEPLOY_CALL_APP_DOMAIN}, ${DEPLOY_REGISTRY_DOMAIN} -> ${DEPLOY_PUBLIC_IP}"
fi
wait_for_dns_to_server
diff --git a/demo/video-chat/scripts/prod-debug.sh b/demo/video-chat/scripts/prod-debug.sh
new file mode 100755
index 000000000..d837c275a
--- /dev/null
+++ b/demo/video-chat/scripts/prod-debug.sh
@@ -0,0 +1,350 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
+VIDEOCHAT_DIR="$(cd -- "${SCRIPT_DIR}/.." && pwd)"
+LOCAL_ENV_FILE="${VIDEOCHAT_DIR}/.env.local"
+TIMEOUT="${VIDEOCHAT_PROD_DEBUG_TIMEOUT:-10}"
+LOG_LINES="${VIDEOCHAT_PROD_DEBUG_LOG_LINES:-240}"
+
+log() {
+ printf '[videochat-prod-debug] %s\n' "$*"
+}
+
+fail() {
+ printf '[videochat-prod-debug] ERROR: %s\n' "$*" >&2
+ exit 1
+}
+
+require_cmd() {
+ command -v "$1" >/dev/null 2>&1 || fail "missing required command: $1"
+}
+
+shell_quote() {
+ printf '%q' "$1"
+}
+
+regex_escape() {
+ printf '%s' "$1" | sed -E 's/[][(){}.^$?+*|\\]/\\&/g'
+}
+
+redact_stream() {
+ sed -E \
+ -e 's/(authorization:[[:space:]]*bearer[[:space:]]+)[^[:space:]]+/\1[REDACTED]/Ig' \
+ -e 's/(bearer[[:space:]]+)[A-Za-z0-9._~+\/=-]+/\1[REDACTED]/Ig' \
+ -e 's/([A-Za-z_][A-Za-z0-9_]*(TOKEN|SECRET|PASSWORD|PASS|KEY|CREDENTIAL|COOKIE|SESSION)[A-Za-z0-9_]*=)[^[:space:]]+/\1[REDACTED]/Ig' \
+ -e 's/("(token|secret|password|pass|key|credential|cookie|session)[^"]*"[[:space:]]*:[[:space:]]*")[^"]+/\1[REDACTED]/Ig' \
+ -e 's/(([?&][^=&[:space:]]*(token|secret|password|pass|key|credential|cookie|session)[^=&[:space:]]*=))[^&[:space:]]+/\1[REDACTED]/Ig' \
+ -e 's/("(media_)?(payload|frame|frame_data|image_data|encoded|binary|bytes)"[[:space:]]*:[[:space:]]*")[^"]+/\1[REDACTED_MEDIA_PAYLOAD]/Ig' \
+ -e 's/(data:(image|video|audio)\/[A-Za-z0-9.+-]+;base64,)[A-Za-z0-9+\/=._~-]+/\1[REDACTED_MEDIA_PAYLOAD]/Ig' \
+ -e 's/([A-Za-z_][A-Za-z0-9_]*(MEDIA_PAYLOAD|FRAME_DATA|IMAGE_DATA|ENCODED_FRAME|BINARY_FRAME)[A-Za-z0-9_]*=)[^[:space:]]+/\1[REDACTED_MEDIA_PAYLOAD]/Ig'
+}
+
+load_local_env() {
+ [[ -f "${LOCAL_ENV_FILE}" ]] || return 0
+ local preserved_names=(
+ VIDEOCHAT_DEPLOY_DOMAIN DEPLOY_DOMAIN
+ VIDEOCHAT_DEPLOY_APP_DOMAIN DEPLOY_APP_DOMAIN
+ VIDEOCHAT_DEPLOY_API_DOMAIN DEPLOY_API_DOMAIN
+ VIDEOCHAT_DEPLOY_WS_DOMAIN DEPLOY_WS_DOMAIN
+ VIDEOCHAT_DEPLOY_SFU_DOMAIN DEPLOY_SFU_DOMAIN
+ VIDEOCHAT_DEPLOY_CDN_DOMAIN DEPLOY_CDN_DOMAIN
+ VIDEOCHAT_DEPLOY_CALL_APP_DOMAIN DEPLOY_CALL_APP_DOMAIN
+ VIDEOCHAT_DEPLOY_REGISTRY_DOMAIN DEPLOY_REGISTRY_DOMAIN
+ VIDEOCHAT_DEPLOY_HOST DEPLOY_HOST
+ VIDEOCHAT_DEPLOY_USER DEPLOY_USER
+ VIDEOCHAT_DEPLOY_SSH_PORT DEPLOY_SSH_PORT
+ VIDEOCHAT_DEPLOY_PATH DEPLOY_PATH
+ VIDEOCHAT_PROD_DEBUG_DRY_RUN VIDEOCHAT_PROD_DEBUG_SKIP_REMOTE
+ )
+ local name
+ declare -A preserved_values=()
+ for name in "${preserved_names[@]}"; do
+ if [[ -n "${!name+x}" ]]; then
+ preserved_values["${name}"]="${!name}"
+ fi
+ done
+ set -a
+ # shellcheck source=/dev/null
+ source "${LOCAL_ENV_FILE}"
+ set +a
+ if [[ -n "${preserved_values[VIDEOCHAT_DEPLOY_DOMAIN]+x}" || -n "${preserved_values[DEPLOY_DOMAIN]+x}" ]]; then
+ local service_prefix
+ for service_prefix in APP API WS SFU CDN CALL_APP REGISTRY; do
+ if [[ -z "${preserved_values[VIDEOCHAT_DEPLOY_${service_prefix}_DOMAIN]+x}" && -z "${preserved_values[DEPLOY_${service_prefix}_DOMAIN]+x}" ]]; then
+ unset "VIDEOCHAT_DEPLOY_${service_prefix}_DOMAIN" "DEPLOY_${service_prefix}_DOMAIN"
+ fi
+ done
+ fi
+ for name in "${!preserved_values[@]}"; do
+ printf -v "${name}" '%s' "${preserved_values[${name}]}"
+ export "${name}"
+ done
+}
+
+normalize_domains() {
+ DEPLOY_DOMAIN="${VIDEOCHAT_DEPLOY_DOMAIN:-${DEPLOY_DOMAIN:-kingrt.com}}"
+ DEPLOY_APP_DOMAIN="${VIDEOCHAT_DEPLOY_APP_DOMAIN:-${DEPLOY_APP_DOMAIN:-app.${DEPLOY_DOMAIN}}}"
+ DEPLOY_API_DOMAIN="${VIDEOCHAT_DEPLOY_API_DOMAIN:-${DEPLOY_API_DOMAIN:-api.${DEPLOY_DOMAIN}}}"
+ DEPLOY_WS_DOMAIN="${VIDEOCHAT_DEPLOY_WS_DOMAIN:-${DEPLOY_WS_DOMAIN:-ws.${DEPLOY_DOMAIN}}}"
+ DEPLOY_SFU_DOMAIN="${VIDEOCHAT_DEPLOY_SFU_DOMAIN:-${DEPLOY_SFU_DOMAIN:-sfu.${DEPLOY_DOMAIN}}}"
+ DEPLOY_CDN_DOMAIN="${VIDEOCHAT_DEPLOY_CDN_DOMAIN:-${DEPLOY_CDN_DOMAIN:-cdn.${DEPLOY_DOMAIN}}}"
+ DEPLOY_CALL_APP_DOMAIN="${VIDEOCHAT_DEPLOY_CALL_APP_DOMAIN:-${DEPLOY_CALL_APP_DOMAIN:-whiteboard.${DEPLOY_DOMAIN}}}"
+ DEPLOY_REGISTRY_DOMAIN="${VIDEOCHAT_DEPLOY_REGISTRY_DOMAIN:-${DEPLOY_REGISTRY_DOMAIN:-registry.${DEPLOY_DOMAIN}}}"
+ DEPLOY_HOST="${VIDEOCHAT_DEPLOY_HOST:-${DEPLOY_HOST:-}}"
+ DEPLOY_USER="${VIDEOCHAT_DEPLOY_USER:-${DEPLOY_USER:-root}}"
+ DEPLOY_SSH_PORT="${VIDEOCHAT_DEPLOY_SSH_PORT:-${DEPLOY_SSH_PORT:-22}}"
+ DEPLOY_PATH="${VIDEOCHAT_DEPLOY_PATH:-${DEPLOY_PATH:-/opt/king-videochat}}"
+}
+
+section() {
+ printf '\n## %s\n' "$1"
+}
+
+http_probe() {
+ local label="$1" url="$2" method="${3:-GET}" body code
+ if [[ "${VIDEOCHAT_PROD_DEBUG_DRY_RUN:-0}" == "1" ]]; then
+ printf '%-28s DRY-RUN %s %s\n' "${label}" "${method}" "${url}" | redact_stream
+ return 0
+ fi
+ body="$(mktemp)"
+ if [[ "${method}" == "HEAD" ]]; then
+ code="$(curl -sS -I --max-time "${TIMEOUT}" -o "${body}" -w '%{http_code}' "${url}" || true)"
+ else
+ code="$(curl -sS --max-time "${TIMEOUT}" -o "${body}" -w '%{http_code}' -X "${method}" "${url}" || true)"
+ fi
+ printf '%-28s %s %s\n' "${label}" "${code:-000}" "${url}" | redact_stream
+ if [[ "${method}" == "GET" && -s "${body}" ]]; then
+ head -c 2000 "${body}" | redact_stream
+ printf '\n'
+ fi
+ rm -f "${body}"
+}
+
+header_value() {
+ local header_name="$1" headers="$2"
+ awk -v name="${header_name}" 'BEGIN { lower = tolower(name) ":" } tolower($0) ~ "^" lower { sub("^[^:]*:[[:space:]]*", "", $0); gsub("\r", "", $0); print $0; exit }' "${headers}"
+}
+
+call_app_frame_header_probe() {
+ local label="$1" url="$2" headers body code csp allow_csp_from nested_pattern escaped_app_domain
+ local wildcard_frame_ancestors_pattern wildcard_frame_src_pattern wildcard_script_src_pattern wildcard_connect_src_pattern
+ if [[ "${VIDEOCHAT_PROD_DEBUG_DRY_RUN:-0}" == "1" ]]; then
+ log "${label}: DRY-RUN ${url}; CSP frame-ancestors https://${DEPLOY_APP_DOMAIN}; Allow-CSP-From https://${DEPLOY_APP_DOMAIN}; X-Frame-Options absent; no nested *.${DEPLOY_APP_DOMAIN} origins"
+ return 0
+ fi
+ headers="$(mktemp)"
+ body="$(mktemp)"
+ code="$(curl -sS --max-time "${TIMEOUT}" -D "${headers}" -o "${body}" -w '%{http_code}' "${url}" || true)"
+ if [[ "${code}" != "200" ]]; then
+ printf '[videochat-prod-debug] %s headers:\n' "${label}" >&2
+ cat "${headers}" >&2 || true
+ printf '[videochat-prod-debug] %s body:\n' "${label}" >&2
+ head -c 2000 "${body}" >&2 || true
+ printf '\n' >&2
+ rm -f "${headers}" "${body}"
+ fail "${label}: expected HTTP 200, got ${code:-none}"
+ fi
+
+ csp="$(header_value Content-Security-Policy "${headers}")"
+ allow_csp_from="$(header_value Allow-CSP-From "${headers}")"
+ if [[ "${csp}" != *"frame-ancestors https://${DEPLOY_APP_DOMAIN}"* ]]; then
+ cat "${headers}" >&2 || true
+ rm -f "${headers}" "${body}"
+ fail "${label}: Content-Security-Policy must allow frame ancestor https://${DEPLOY_APP_DOMAIN}"
+ fi
+ if [[ "${csp}" != *"script-src 'self'"* ]]; then
+ cat "${headers}" >&2 || true
+ rm -f "${headers}" "${body}"
+ fail "${label}: Content-Security-Policy must keep script-src self-scoped"
+ fi
+ if [[ "${csp}" != *"connect-src 'self'"* ]]; then
+ cat "${headers}" >&2 || true
+ rm -f "${headers}" "${body}"
+ fail "${label}: Content-Security-Policy must keep connect-src self-scoped"
+ fi
+ wildcard_frame_ancestors_pattern='(^|[[:space:];])frame-ancestors[^;]*\*'
+ wildcard_frame_src_pattern='(^|[[:space:];])frame-src[^;]*\*'
+ wildcard_script_src_pattern='(^|[[:space:];])script-src[^;]*\*'
+ wildcard_connect_src_pattern='(^|[[:space:];])connect-src[^;]*\*'
+ if [[ "${csp}" =~ ${wildcard_frame_ancestors_pattern} || "${csp}" =~ ${wildcard_frame_src_pattern} || "${csp}" =~ ${wildcard_script_src_pattern} || "${csp}" =~ ${wildcard_connect_src_pattern} ]]; then
+ cat "${headers}" >&2 || true
+ rm -f "${headers}" "${body}"
+ fail "${label}: Content-Security-Policy must not use wildcard frame/script/connect directives"
+ fi
+ if [[ "${allow_csp_from}" != "https://${DEPLOY_APP_DOMAIN}" ]]; then
+ cat "${headers}" >&2 || true
+ rm -f "${headers}" "${body}"
+ fail "${label}: Allow-CSP-From must equal https://${DEPLOY_APP_DOMAIN}"
+ fi
+ if [[ -n "$(header_value X-Frame-Options "${headers}")" ]]; then
+ cat "${headers}" >&2 || true
+ rm -f "${headers}" "${body}"
+ fail "${label}: X-Frame-Options must be absent for Call App iframe responses"
+ fi
+
+ escaped_app_domain="$(regex_escape "${DEPLOY_APP_DOMAIN}")"
+ nested_pattern="https?://[A-Za-z0-9.-]+\\.${escaped_app_domain}"
+ if grep -Eia "${nested_pattern}" "${headers}" "${body}" >/dev/null; then
+ printf '[videochat-prod-debug] %s nested app-domain origin matches:\n' "${label}" >&2
+ grep -Eia "${nested_pattern}" "${headers}" "${body}" >&2 || true
+ rm -f "${headers}" "${body}"
+ fail "${label}: must not reference nested *.${DEPLOY_APP_DOMAIN} service origins"
+ fi
+
+ rm -f "${headers}" "${body}"
+ log "${label}: HTTP ${code}; CSP frame-ancestors https://${DEPLOY_APP_DOMAIN}; Allow-CSP-From https://${DEPLOY_APP_DOMAIN}; X-Frame-Options absent; no nested *.${DEPLOY_APP_DOMAIN} origins"
+}
+
+call_app_csp_header_proof() {
+ section "Call-App CSP Header Proof"
+ call_app_frame_header_probe "call-app whiteboard host CSP" "https://${DEPLOY_CALL_APP_DOMAIN}/public/index.html"
+ call_app_frame_header_probe "call-app whiteboard path CSP" "https://${DEPLOY_CALL_APP_DOMAIN}/call-app/whiteboard/public/index.html"
+}
+
+websocket_probe() {
+ local label="$1" url="$2" headers body code upgrade curl_url
+ if [[ "${VIDEOCHAT_PROD_DEBUG_DRY_RUN:-0}" == "1" ]]; then
+ printf '%-28s DRY-RUN websocket %s\n' "${label}" "${url}" | redact_stream
+ return 0
+ fi
+ curl_url="${url/wss:\/\//https://}"
+ headers="$(mktemp)"
+ body="$(mktemp)"
+ code="$(
+ curl -sS --http1.1 --max-time "${TIMEOUT}" \
+ -D "${headers}" \
+ -o "${body}" \
+ -w '%{http_code}' \
+ -H 'Connection: Upgrade' \
+ -H 'Upgrade: websocket' \
+ -H 'Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==' \
+ -H 'Sec-WebSocket-Version: 13' \
+ "${curl_url}" || true
+ )"
+ upgrade="$(awk 'tolower($1) == "upgrade:" {print $2; exit}' "${headers}" | tr -d '\r')"
+ printf '%-28s %s upgrade=%s %s\n' "${label}" "${code:-000}" "${upgrade:-none}" "${url}" | redact_stream
+ rm -f "${headers}" "${body}"
+}
+
+ssh_args() {
+ local args=(-p "${DEPLOY_SSH_PORT}" -o BatchMode=yes -o ConnectTimeout="${TIMEOUT}")
+ if [[ -n "${VIDEOCHAT_DEPLOY_SSH_KEY:-}" ]]; then
+ args+=(-i "${VIDEOCHAT_DEPLOY_SSH_KEY}")
+ fi
+ printf '%s\n' "${args[@]}"
+}
+
+remote_readonly_probe() {
+ if [[ "${VIDEOCHAT_PROD_DEBUG_DRY_RUN:-0}" == "1" ]]; then
+ section "Remote Containers And Recent Diagnostics"
+ log "DRY-RUN: remote SSH probes skipped; read-only compose ps/logs would execute when enabled"
+ return 0
+ fi
+
+ case "${VIDEOCHAT_PROD_DEBUG_SKIP_REMOTE:-0}" in
+ 1|true|TRUE|yes|YES)
+ log "remote SSH probes skipped; VIDEOCHAT_PROD_DEBUG_SKIP_REMOTE=1"
+ return 0
+ ;;
+ esac
+
+ [[ -n "${DEPLOY_HOST}" ]] || {
+ log "remote SSH probes skipped; VIDEOCHAT_DEPLOY_HOST is not set"
+ return 0
+ }
+
+ local deploy_path_q log_lines_q ssh_dest
+ deploy_path_q="$(shell_quote "${DEPLOY_PATH}")"
+ log_lines_q="$(shell_quote "${LOG_LINES}")"
+ ssh_dest="${DEPLOY_USER}@${DEPLOY_HOST}"
+
+ section "Remote Containers And Recent Diagnostics"
+ log "SSH target: ${DEPLOY_USER}@${DEPLOY_HOST}:${DEPLOY_SSH_PORT}, deploy path: ${DEPLOY_PATH}"
+ mapfile -t SSH_ARGS < <(ssh_args)
+ ssh "${SSH_ARGS[@]}" "${ssh_dest}" "bash -s" <&1 \
+ | grep -Eai "\${pattern}" \
+ | tail -n "\${LOG_LINES}" \
+ | redact_stream || true
+}
+if [ ! -d "\${VIDEOCHAT_DIR}" ]; then
+ echo "remote checkout missing: \${VIDEOCHAT_DIR}"
+ exit 0
+fi
+cd "\${VIDEOCHAT_DIR}"
+if [ -f docker-compose.deploy.local.yml ]; then
+ COMPOSE=(docker compose --env-file .env --env-file .env.local -f docker-compose.v1.yml -f docker-compose.deploy.local.yml --profile edge --profile turn)
+elif [ -f docker-compose.v1.yml ]; then
+ COMPOSE=(docker compose --env-file .env -f docker-compose.v1.yml --profile edge --profile turn)
+else
+ echo "docker compose files not found"
+ exit 0
+fi
+echo "# docker compose ps"
+"\${COMPOSE[@]}" ps 2>&1 | redact_stream || true
+filter_recent_logs "call health and runtime status" 'call|lobby|diagnostic|health|runtime|room|presence|error|warn|fail'
+filter_recent_logs "media reconnect" 'media[_ -]?reconnect|reconnect.*media|foreground[_ -]?reconnect|local[_ -]?media.*reconnect|stale_local_media_capture_discarded'
+filter_recent_logs "screen-share reconnect exhaustion" 'local_screen_share_sfu_reconnect_exhausted|screen[_ -]?share.*(reconnect|exhaust|stopped|disconnect)|screen_share.*reconnect'
+filter_recent_logs "stale local media capture discard" 'stale_local_media_capture_discarded|local_media_cleanup_preserved_active_track|stale.*local.*media.*discard'
+filter_recent_logs "audio/video track loss" '(audio|video).*(track|capture).*(ended|lost|stop|stopped|mute|failed|error)|track.*(lost|ended|stopped)|getUserMedia|devicechange|NotReadableError|NotAllowedError'
+filter_recent_logs "SFU reconnect and websocket transport" 'sfu.*(reconnect|disconnect|websocket|ws|close|closed|error|fail)|local_screen_share_sfu_reconnect|sfu_reconnect|websocket.*sfu'
+filter_recent_logs "Call App frame and CSP errors" 'call[_ -]?app.*(frame|iframe|csp|postmessage|postMessage|sandbox|origin|launch|error|fail)|Content-Security-Policy|Allow-CSP-From|frame-ancestors|postMessage'
+REMOTE
+}
+
+main() {
+ require_cmd curl
+ require_cmd sed
+ load_local_env
+ normalize_domains
+
+ section "Read Only Contract"
+ log "mode: read-only production diagnostics; no deploy, restart, DB write, DNS change, or admin action"
+ log "env source: ${LOCAL_ENV_FILE} if present; values are used only for domains and SSH target"
+
+ section "Domains"
+ printf 'root=%s\napp=%s\napi=%s\nws=%s\nsfu=%s\ncdn=%s\ncall_app=%s\nregistry=%s\n' \
+ "${DEPLOY_DOMAIN}" "${DEPLOY_APP_DOMAIN}" "${DEPLOY_API_DOMAIN}" "${DEPLOY_WS_DOMAIN}" \
+ "${DEPLOY_SFU_DOMAIN}" "${DEPLOY_CDN_DOMAIN}" "${DEPLOY_CALL_APP_DOMAIN}" "${DEPLOY_REGISTRY_DOMAIN}" \
+ | redact_stream
+
+ section "Public Runtime And Asset Version"
+ http_probe "api runtime" "https://${DEPLOY_API_DOMAIN}/api/runtime"
+ http_probe "api version" "https://${DEPLOY_API_DOMAIN}/api/version"
+ http_probe "app shell" "https://${DEPLOY_APP_DOMAIN}/" HEAD
+ http_probe "cdn asset root" "https://${DEPLOY_CDN_DOMAIN}/" HEAD
+
+ section "API, WS, SFU, Marketplace, Call-App Reachability"
+ http_probe "marketplace apps" "https://${DEPLOY_API_DOMAIN}/api/marketplace/call-apps"
+ http_probe "call-app host" "https://${DEPLOY_CALL_APP_DOMAIN}/" HEAD
+ http_probe "registry host" "https://${DEPLOY_REGISTRY_DOMAIN}/" HEAD
+ websocket_probe "lobby websocket" "wss://${DEPLOY_WS_DOMAIN}/ws"
+ websocket_probe "sfu websocket" "wss://${DEPLOY_SFU_DOMAIN}/sfu"
+
+ call_app_csp_header_proof
+
+ remote_readonly_probe
+}
+
+main "$@"
diff --git a/demo/video-chat/scripts/smoke.sh b/demo/video-chat/scripts/smoke.sh
index fbf20b44a..f074bb944 100755
--- a/demo/video-chat/scripts/smoke.sh
+++ b/demo/video-chat/scripts/smoke.sh
@@ -26,6 +26,19 @@ run_step() {
log "OK: ${step}"
}
+should_run_guest_cleanup_sqlite_proof() {
+ case "${VIDEOCHAT_SMOKE_RUN_GUEST_CLEANUP_SQLITE_PROOF:-auto}" in
+ 1|true|TRUE|yes|YES)
+ return 0
+ ;;
+ 0|false|FALSE|no|NO)
+ return 1
+ ;;
+ esac
+
+ [[ "${VIDEOCHAT_SMOKE_COMPOSE_ONLY:-0}" == "1" && "${VIDEOCHAT_SMOKE_REQUIRE_COMPOSE:-0}" == "1" ]]
+}
+
validate_tcp_port() {
local candidate="${1:-}"
if [[ ! "${candidate}" =~ ^[0-9]+$ ]]; then
@@ -537,8 +550,47 @@ compose_smoke() {
}
'
+ if [[ "${VIDEOCHAT_SMOKE_SKIP_BACKEND_CALL_ACCESS_SESSION_CONTRACT:-0}" != "1" ]]; then
+ log "compose backend call-access session contract gate"
+ VIDEOCHAT_V1_BACKEND_PORT="${compose_backend_port}" \
+ VIDEOCHAT_V1_BACKEND_WS_PORT="${compose_backend_ws_port}" \
+ VIDEOCHAT_V1_BACKEND_SFU_PORT="${compose_backend_sfu_port}" \
+ VIDEOCHAT_V1_FRONTEND_PORT="${compose_frontend_port}" \
+ VIDEOCHAT_V1_BACKEND_ORIGIN="http://127.0.0.1:${compose_backend_port}" \
+ VIDEOCHAT_V1_BACKEND_PHP_IMAGE="${compose_backend_php_image}" \
+ "${compose_cmd[@]}" exec -T videochat-backend-v1 sh -lc "\
+ cd \"\${VIDEOCHAT_SMOKE_BACKEND_WORKDIR:-/app}\" && \
+ tests/call-access-session-contract.sh"
+ fi
+
+ if [[ "${VIDEOCHAT_SMOKE_SKIP_FRONTEND_CALL_ACCESS_E2E:-0}" != "1" ]]; then
+ local call_access_seed_matrix_json
+ call_access_seed_matrix_json="$(tr -d '\n' < "${ROOT_DIR}/contracts/v1/iam-call-access-seeding.matrix.json")"
+ log "compose frontend Playwright call-access gate"
+ VIDEOCHAT_V1_BACKEND_PORT="${compose_backend_port}" \
+ VIDEOCHAT_V1_BACKEND_WS_PORT="${compose_backend_ws_port}" \
+ VIDEOCHAT_V1_BACKEND_SFU_PORT="${compose_backend_sfu_port}" \
+ VIDEOCHAT_V1_FRONTEND_PORT="${compose_frontend_port}" \
+ VIDEOCHAT_V1_BACKEND_ORIGIN="http://127.0.0.1:${compose_backend_port}" \
+ VIDEOCHAT_V1_BACKEND_PHP_IMAGE="${compose_backend_php_image}" \
+ "${compose_cmd[@]}" exec -T \
+ -e "VIDEOCHAT_CALL_ACCESS_SEED_MATRIX_JSON=${call_access_seed_matrix_json}" \
+ videochat-frontend-v1 sh -lc "\
+ cd \"\${VIDEOCHAT_SMOKE_FRONTEND_WORKDIR:-/app}\" && \
+ PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH=/usr/bin/chromium-browser \
+ PLAYWRIGHT_FRONTEND_PORT=4174 \
+ VITE_VIDEOCHAT_BACKEND_ORIGIN='http://videochat-backend-v1:18080' \
+ VITE_VIDEOCHAT_BACKEND_PORT='18080' \
+ VITE_VIDEOCHAT_WS_ORIGIN='http://videochat-backend-ws-v1:18080' \
+ VITE_VIDEOCHAT_WS_PORT='18080' \
+ VITE_VIDEOCHAT_SFU_ORIGIN='http://videochat-backend-sfu-v1:18080' \
+ VITE_VIDEOCHAT_SFU_PORT='18080' \
+ VITE_VIDEOCHAT_ALLOW_INSECURE_WS='1' \
+ npm run test:e2e:call-access -- --reporter=list --workers=1"
+ fi
+
if [[ "${VIDEOCHAT_SMOKE_SKIP_FRONTEND_E2E_MATRIX:-0}" != "1" ]]; then
- log "compose frontend Playwright matrix gate"
+ log "compose frontend Playwright chat/layout matrix gate"
VIDEOCHAT_V1_BACKEND_PORT="${compose_backend_port}" \
VIDEOCHAT_V1_BACKEND_WS_PORT="${compose_backend_ws_port}" \
VIDEOCHAT_V1_BACKEND_SFU_PORT="${compose_backend_sfu_port}" \
@@ -568,6 +620,11 @@ run_step "deployment baseline: multi-node runtime architecture" bash -lc "'${ROO
run_step "deployment baseline: ops hardening" bash -lc "'${ROOT_DIR}/scripts/check-ops-hardening.sh'"
run_step "deployment baseline: production endpoint smoke syntax" bash -lc "bash -n '${ROOT_DIR}/scripts/deploy-smoke.sh'"
run_step "compose stack boot + migration/auth sanity" compose_smoke
+if should_run_guest_cleanup_sqlite_proof; then
+ run_step "backend contract: guest cleanup Docker SQLite proof" bash -lc "'${BACKEND_DIR}/tests/call-guest-cleanup-sqlite-proof.sh'"
+else
+ log "SKIP: guest cleanup Docker SQLite proof not selected; set VIDEOCHAT_SMOKE_RUN_GUEST_CLEANUP_SQLITE_PROOF=1 to run outside required compose-only smoke"
+fi
if [[ "${VIDEOCHAT_SMOKE_COMPOSE_ONLY:-0}" == "1" ]]; then
log "Compose-only smoke checks passed."
@@ -673,6 +730,7 @@ run_step "backend contract: chat fanout" bash -lc "'${BACKEND_DIR}/tests/realtim
run_step "backend contract: reaction stream throttle" bash -lc "'${BACKEND_DIR}/tests/realtime-reaction-contract.sh'"
run_step "backend contract: invite redeem" bash -lc "'${BACKEND_DIR}/tests/invite-code-redeem-contract.sh'"
run_step "backend contract: call-access session binding" bash -lc "'${BACKEND_DIR}/tests/call-access-session-contract.sh'"
+run_step "backend contract: call-access membership removal" bash -lc "'${BACKEND_DIR}/tests/call-access-membership-removal-contract.sh'"
run_step "backend contract: call signaling bootstrap" bash -lc "'${BACKEND_DIR}/tests/realtime-signaling-contract.sh'"
run_step "backend contract: SFU room binding and relay" bash -lc "'${BACKEND_DIR}/tests/realtime-sfu-contract.sh'"
@@ -688,6 +746,12 @@ run_step "frontend contract: media security frame path" bash -lc "
npm run test:contract:media-security
"
+run_step "frontend contract: media reconnect screenshare release smoke" bash -lc "
+ set -euo pipefail
+ cd '${FRONTEND_DIR}'
+ npm run test:contract:media-reconnect-release-smoke
+"
+
run_step "frontend contract: MediaPipe CDN assets" bash -lc "
set -euo pipefail
cd '${FRONTEND_DIR}'