Sync lets a web application publish rendered frames to native video tools and receive native multichannel audio on the same computer. The browser SDK supports direct RGBA bytes, Canvas 2D, WebGL2, WebGPU, and bounded audio-source reads. The audio research, protocol, limitations, and qualification record live in Audio input.
The published SDK package is @noisefactor/sync 0.3.0. Exact source
1972af1ce3f0d14054f3693e250c668aff536884
passed the
cross-platform CI matrix.
Its public release
was published from that source on September 14, 2026.
The SDK and native companion have separate versions.
Install the SDK 0.3.0 release tarball in your application:
npm install https://github.com/noisefactorllc/sync/releases/download/sdk-v0.3.0/noisefactor-sync-0.3.0.tgzYou can then import from @noisefactor/sync.
This command installs the GitHub release asset directly.
It does not require an npm account.
For direct browser imports, download the modules ZIP.
Extract modules/ into your application's static assets.
Import index.js from that directory.
Keep the complete directory because its files use relative imports.
The release page includes SHA-256 checksums for both downloads.
Build the distribution from the repository root:
npm run package:sdkThe command creates these versioned files:
dist/sdk/0.3.0/modules/
dist/sdk/0.3.0/noisefactor-sync-0.3.0.tgz
dist/sdk/0.3.0/SHA256SUMS
Install the tarball from the path that applies to your application:
npm install /absolute/path/to/sync/dist/sdk/0.3.0/noisefactor-sync-0.3.0.tgzYou can then import from @noisefactor/sync.
You can also copy modules/ into your application's static assets.
Import index.js from the copied directory when you use this vendored option.
Keep the complete module directory. Its files use relative imports between the client, protocol, diagnostics, and queue modules.
The SDK does not install or update the native Sync companion.
Check the download page for available installers.
Native preview 0.2.68 includes the audio extension, but clients must still check
the connected companion's capabilities instead of relying on its product or
protocol version. A compatible runtime advertises a selected and available
audio provider whose direction is receive.
For source testing, build and run the daemon from the same source checkout. Follow the native build instructions.
Serve this response header from the top-level application:
Permissions-Policy: loopback-network=(self)A cross-origin parent must name the child origin in its response header:
Permissions-Policy: loopback-network=(self "https://visuals.example")The parent must also add allow="loopback-network" to the application iframe.
Call pair() directly from a deliberate user action.
Sync shows the application name and exact origin before it creates a token.
import { SyncBridgeClient } from '@noisefactor/sync';
connectButton.addEventListener('click', async () => {
const pairingClient = new SyncBridgeClient();
const { token } = await pairingClient.pair('My visual app');
pairingClient.close();
const client = new SyncBridgeClient({ token });
await client.connect();
});The pairing client does not store the token. It does not put the token in a URL or a WebSocket subprotocol. The application owns any token persistence. Create a new client with the approved token.
Each successful pair() rotates the single stored token for that origin.
Treat the returned token as the origin's latest credential and propagate it to
every audio and video integration before either opens a future control
connection. Already authenticated sessions remain open; rotation affects later
authentication attempts. Pairing-store replacement,
control authentication.
probe() and connect() are passive operations.
They return or throw a permission result instead of starting pairing.
The default daemon endpoint is http://127.0.0.1:53979.
The endpoint option also accepts an explicit IPv4 or IPv6 loopback URL with a port.
It rejects remote hosts, credentials, paths, queries, and fragments.
SDK 0.3.0 adds native audio-source discovery and bounded PCM reads. Check the companion capability before offering the device picker, and let the user choose a source before opening it:
const audio = new SyncBridgeClient({ token });
try {
const welcome = await audio.connect();
const supportsAudio = welcome.capabilities.providers.some(provider =>
provider.id === 'audio' && provider.direction === 'receive' &&
provider.available && provider.selected);
if (!supportsAudio) throw new Error('Install a Sync companion with audio input');
const sources = await audio.listAudioSources();
const selectedSourceId = await chooseAudioSource(sources);
const format = await audio.openAudioSource(selectedSourceId);
try {
const packet = await audio.readAudioSource(selectedSourceId);
consumeAudio(packet.planes, format, packet.firstFrame, packet.droppedFrames);
} finally {
await audio.closeAudioSource(selectedSourceId);
}
} finally {
audio.close();
}Each client owns one capture. Use separate clients for separate audio sources
and for video output. readAudioSource() is a bounded pull operation; avoid a
busy polling loop. Reset queued browser audio when firstFrame stops following
the preceding cursor or droppedFrames changes.
Audio requires fresh origin pairing after the companion restarts, even when a
stored token remains valid for video. When an operation reports the daemon code
audio_pairing_required, ask the user to start pairing again. The opened format
reports the channel count and sample rate that the native backend actually
provided. The API does not add channels missing from that backend and does not
provide sample-accurate audio/video synchronization. See the
audio contract and qualification matrix.
On Linux, follow the documented
pw-jack service setup; installing
the package alone does not make an already-running service load PipeWire's JACK
compatibility library.
Each sender needs this descriptor:
const descriptor = {
width: 1280,
height: 720,
format: 'rgba8unorm',
colorSpace: 'srgb',
alphaMode: 'opaque',
fps: 60,
};The width and height must be integers from 1 through 4096.
The frame payload must not exceed 64 MiB.
The format value is rgba8unorm.
The colorSpace value is srgb or display-p3.
The alphaMode value is opaque, straight, or premultiplied.
The fps value must be positive and finite.
Call configure() before the first submission.
Call it again after each output-size or color change.
sender.configure(descriptor);
sender.submit(source, performance.now());The timestamp uses the same clock as performance.now().
The SDK encodes Math.round((performance.timeOrigin + timestamp) * 1000) microseconds.
submit() never waits for the native receiver.
It returns true when the queue accepts the source.
It returns false after a queue, transport, or backpressure drop.
Read sender.stats for local accepted, sent, busy, backpressure, and failed counters.
Local sent means that the browser passed the frame to its WebSocket.
Native accepted means that the daemon providers accepted the frame.
Neither counter proves that a receiving application showed the frame.
The application owns the render loop and each source resource.
The export queue only borrows a source during enqueue().
Each queue emits top-down RGBA bytes in a reusable Uint8Array.
The frame callback borrows these bytes until it returns.
createRgbaSender() creates an RgbaExportQueue for you.
It uses maxBufferedFrames: 2 when you do not supply a buffer limit.
const sender = await client.createRgbaSender('My RGBA output');
sender.configure(descriptor);
sender.submit({
width: descriptor.width,
height: descriptor.height,
rowStride: descriptor.width * 4,
data: rgbaBytes,
}, performance.now());data can be an ArrayBuffer or an array view.
rowStride must contain at least four bytes for each pixel.
The queue copies each used row before submit() returns.
The caller can then change or release the source bytes.
Create the queue with the configured source canvas:
const exportQueue = new CanvasExportQueue({ canvas });
const sender = await client.createSender('My canvas', {
exportQueue,
maxBufferedFrames: 2,
});
sender.configure(descriptor);
function render(timestamp) {
drawCanvas(timestamp);
sender.submit(canvas, timestamp);
requestAnimationFrame(render);
}The source canvas dimensions must equal the descriptor dimensions. The queue copies through a separate 2D canvas and emits top-down RGBA bytes. It applies the selected alpha mode to the copied output. This queue performs a synchronous 2D readback for each accepted submission. Measure its render-loop cost at your target size and frame rate.
Create the queue with the WebGL2 context:
const exportQueue = new WebGL2ExportQueue({ gl, slots: 3 });
const sender = await client.createSender('My WebGL output', {
exportQueue,
maxBufferedFrames: 2,
});
sender.configure(descriptor);
function render(timestamp) {
renderWebGL();
sender.submit(null, timestamp);
requestAnimationFrame(render);
}The slots value can be an integer from 1 through 8.
The default is 3.
Pass null to read the default framebuffer.
Its drawing-buffer dimensions must equal the descriptor dimensions.
You can also pass a complete framebuffer object with a color attachment.
The queue restores the framebuffer, read-buffer, pixel-pack buffer, and pack state after each submission.
It flips the WebGL rows into top-down order.
It does not convert color spaces or alpha representation.
Set the descriptor to match the bytes in the WebGL source.
Treat a webglcontextlost event as a sender loss.
Stop the sender and release your renderer resources.
The source texture must include GPUTextureUsage.COPY_SRC.
It must be a single 2D texture without multisampling.
Its dimensions must equal the descriptor dimensions.
The queue accepts rgba8unorm, rgba8unorm-srgb, bgra8unorm, and bgra8unorm-srgb textures.
It converts BGRA data to RGBA during readback.
It does not convert color spaces or alpha representation.
Set the descriptor to match the bytes in the WebGPU texture.
context.configure({
device,
format,
alphaMode: 'opaque',
usage: GPUTextureUsage.RENDER_ATTACHMENT | GPUTextureUsage.COPY_SRC,
});
const exportQueue = new WebGPUExportQueue({ device, slots: 3 });
const sender = await client.createSender('My WebGPU output', {
exportQueue,
maxBufferedFrames: 2,
});
sender.configure(descriptor);
function render(timestamp) {
const texture = context.getCurrentTexture();
renderWebGPU(texture);
sender.submit(texture, timestamp);
requestAnimationFrame(render);
}The slots value can be an integer from 1 through 8.
The default is 3.
The device must support the padded readback buffer size.
Treat device.lost and uncaptured errors as sender failures.
SyncFrameSink accepts a WebSocket, an export queue, one buffer limit, and a clock.
Most applications can use SyncBridgeClient.createSender() instead.
A custom queue exposes these members:
configure(descriptor)
enqueue(source, timestamp, onFrame, sequence)
poll()
available
close({ backendLost })
enqueue() returns true only when it accepts the source.
It calls onFrame(frame, timestamp, sequence) when bytes are ready.
GPU queues call the callback during poll().
The sender calls poll() from each submit() call.
Set exactly one pressure limit when you call createSender().
Use a positive maxBufferedFrames value or a positive maxBufferedBytes value.
The queue can also reject work while all readback slots are busy.
await sender.getStats() returns native counters:
{
accepted,
dropped,
rejected,
failed,
lastSequence,
lastPresentationTimeUs,
checksum,
}The six counters are safe nonnegative JavaScript integers. The checksum is a 16-digit lowercase hexadecimal string.
createDiagnosticSnapshot({ client, sender, descriptor, error }) combines safe client and native fields.
It includes SDK, daemon, protocol, provider, frame, and counter information.
It excludes credentials, error messages, and error causes.
Stop the caller-owned render loop before you close the sender. Then close the sender and wait for native cleanup:
cancelAnimationFrame(animationFrame);
try {
sender.close();
await sender.closed;
} finally {
client.close();
}sender.closed rejects with SyncSenderLostError after an unexpected data-socket loss.
The error can include the WebSocket close code and a bounded reason.
Pass { backendLost: true } to sender.close() when the renderer backend cannot release queue resources safely.
Release all other resources that your application owns.
SDK version 0.3.0 exports:
SyncBridgeClientandSYNC_DEFAULT_ENDPOINTSyncFrameSinkRgbaExportQueueCanvasExportQueueWebGL2ExportQueueWebGPUExportQueuecreateDiagnosticSnapshotSYNC_SDK_VERSIONencodeFrameV1anddecodeFrameHeaderV1PIXEL_FORMAT,COLOR_SPACE, andALPHA_MODESYNC_ERROR_CODESyncBridgeErrorand the typed Sync error subclasses
The TypeScript declarations also export AudioSourceFormat, AudioSource, and
AudioPacket for the audio methods on SyncBridgeClient.
SYNC_SDK_VERSION is 0.3.0.
The SDK version and the companion product version are independent.
One sender can appear through each selected and available provider.
| Platform | Native outputs |
|---|---|
| macOS | Syphon, optional NDI, and Sync Camera |
| Windows | Spout, optional NDI, and Sync Camera on Windows 11 |
| Ubuntu 24.04 | V4L2 Sync Camera and optional NDI |
NDI needs an operator-installed runtime. The browser SDK does not select or install native providers.
Client, control, pairing, and sender-lifecycle errors extend SyncBridgeError.
These errors include a stable code.
Queue and adapter validation can also throw standard TypeError and RangeError objects.
Handle these groups in the application UI:
| Error group | Application response |
|---|---|
| Permission required or denied | Ask the user to use the explicit Connect control or browser settings. |
| Pairing denied, busy, or timed out | Keep output stopped and let the user start a new attempt. |
| Authentication | Remove the stored token and require a new explicit pairing action. |
| Capability | Show that no selected native send provider is available. |
| Sender lost | Stop the render submission and release renderer resources. |
| Protocol or configuration | Report the exact stable error code and fix the integration. |
Create the client and sender in browser-only lifecycle code.
Do not access browser globals during server-side rendering.
Keep one sender controller outside the component render function.
Submit after the framework completes the renderer's frame.
On resize, resize the source first and then call sender.configure().
On component cleanup, stop animation, close the sender, await sender.closed, and close the client.
The protocol accepts canonical HTTPS origins, loopback HTTP origins, and lowercase app:// origins.
Read the origin rules before you use a custom application scheme.
Read protocol v1 before you implement the wire protocol directly. Use the SDK when your environment supports browser modules. The complete example includes Canvas 2D, WebGL2, WebGPU, diagnostics, resize, loss, stop, and restart behavior.