Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ indent_style = space
indent_size = 4
switch_case_indent = true

[demo/throttle/enable]
[{demo/throttle/enable,test/smoke/clients/docker/*}]
indent_style = space
indent_size = 4
switch_case_indent = true
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/alert.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ on:
- Release Swift Lib
- release-winget
- rpm-repo
- Smoke
- TS
- Video
types:
Expand All @@ -64,7 +65,7 @@ jobs:
github.event.workflow_run.event != 'pull_request_target' &&
github.event.workflow_run.event != 'merge_group' &&
(github.event.workflow_run.event == 'schedule' ||
!contains(fromJSON('["Interop", "Nightly", "TS", "Video"]'), github.event.workflow_run.name))
!contains(fromJSON('["Interop", "Nightly", "Smoke", "TS", "Video"]'), github.event.workflow_run.name))
runs-on: ubuntu-latest

steps:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/interop.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name: Interop

# Cross-language interop test built from THIS checkout (not the published
# packages, unlike the moq-dev/smoke repo). Stands up a relay and runs the
# packages, unlike smoke.yml). Stands up a relay and runs the
# publish x subscribe matrix across rust / python / go / browser / native-js / c,
# then browser-to-browser media QA. TS validation has its own workflow.
on:
Expand Down
7 changes: 5 additions & 2 deletions .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,10 @@ jobs:
path: ${{ matrix.artifact-path }}
retention-days: 45

# One writer owns gh-pages, including future Criterion regression history.
# gh-pages is the one Pages site, including future Criterion regression
# history. smoke.yml also commits its relay report there under relays/; each
# writer touches only its own directory, deploys the whole branch, and holds
# the benchmark-pages group so they never race.
# Explicit Pages deployment works with GITHUB_TOKEN; a branch push alone does not.
benchmark-site:
name: Benchmark site
Expand All @@ -223,7 +226,7 @@ jobs:
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
# Only this main-branch publisher pushes the shared history branch.
# Only main-branch publishers push the shared history branch.
persist-credentials: true
- uses: DeterminateSystems/nix-installer-action@1d87d45818068401a10cf16bdc5f00b24994a83f # main
with:
Expand Down
520 changes: 520 additions & 0 deletions .github/workflows/smoke.yml

Large diffs are not rendered by default.

6 changes: 3 additions & 3 deletions quest/m1/smoke-moxygen.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@

## Goal

Waiting on upstream Moxygen. moq-dev/smoke's Moxygen lane passes all six
Waiting on upstream Moxygen. test/smoke's Moxygen lane passes all six
cases through moq-relay on WebTransport and raw QUIC at draft-16, and its
`continue-on-error` exception is removed.

Check by running `just moxygen` in moq-dev/smoke, or by reading the latest
Moxygen job there. Delete this quest once the lane passes and the exception
Check by running `just test smoke moxygen`, or by reading the latest
Moxygen job in the Smoke workflow. Delete this quest once the lane passes and the exception
is gone.

## Plan
Expand Down
2 changes: 1 addition & 1 deletion quest/m2/release/catalog-readers.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
## Goal

The latest published Go and JavaScript clients read live media from the
current Rust publisher in moq-dev/smoke's Nix channel. Incompatible optional
current Rust publisher in test/smoke's Nix channel. Incompatible optional
archive metadata does not make an otherwise playable catalog unreadable.

## Plan
Expand Down
4 changes: 2 additions & 2 deletions test/interop/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
Cross-language interop test that builds every client from **this checkout**
and runs them against each other.

This is the in-tree companion to [moq-dev/smoke](https://github.com/moq-dev/smoke).
That repo installs each client from its public package registry (crates.io, PyPI,
This is the in-tree companion to [`test/smoke`](../smoke).
That suite installs each client from its public package registry (crates.io, PyPI,
npm, ...) to catch *packaging* breakage in a release. This one builds each client
from the workspace source (`cargo`, `bun`, `uv`, `cc`) to catch *interop*
regressions before anything is published. No apt/brew/npm/PyPI, and no
Expand Down
4 changes: 2 additions & 2 deletions test/interop/interop.sh
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/usr/bin/env bash
# Cross-language media interop test against THIS checkout.
#
# Unlike the standalone moq-dev/smoke repo (which installs each client from its
# Unlike test/smoke (which installs each client from its
# public registry to catch packaging breakage), this builds every client from
# the workspace source. It proves the code in the tree interoperates across
# implementations before anything is published: a relay built from rs/moq-relay,
Expand Down Expand Up @@ -69,7 +69,7 @@ TAIL_ONLY=0
PROFILE="${INTEROP_PROFILE:-debug}"

# Binaries under test. Built from source below unless overridden to point at a
# prebuilt (mirrors the standalone moq-dev/smoke repo's RELAY_BIN/MOQ_BIN escape hatch).
# prebuilt (mirrors test/smoke's RELAY_BIN/MOQ_BIN escape hatch).
RELAY="${RELAY_BIN:-}"
MOQ="${MOQ_BIN:-}"

Expand Down
12 changes: 11 additions & 1 deletion test/justfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# in the root `just check`.
#
# Two cross-language interop tests exist:
# - https://github.com/moq-dev/smoke installs each client from its PUBLIC
# - `just test smoke` (smoke/) installs each client from its PUBLIC
# package registry to catch packaging breakage in a release.
# - `just test interop` (interop/, below) builds every client from THIS checkout
# to catch interop regressions before anything is published.
Expand Down Expand Up @@ -51,6 +51,16 @@ interop *args:
wire-compat *args:
./interop/compat.sh "$@"

# Installs each client from its public registry, inside test/smoke's own Nix
# shell rather than this repo's toolchain. The first argument picks a smoke
# recipe and the rest pass through, e.g. `just test smoke full` or `just test
# smoke relays --only moq-dev-rs`. With none, the Rust-only matrix. See smoke/README.md.

# Published-package interop smoke.
[positional-arguments]
smoke *args:
nix develop ./smoke --command just --justfile smoke/justfile "$@"

# Paint a short-side transcode rung from a coded portrait through moq-watch.
portrait:
./portrait/run.sh
Expand Down
22 changes: 22 additions & 0 deletions test/smoke/.envrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Automatically load the Nix development shell when entering this directory.
# Requires direnv: https://direnv.net/
#
# To enable:
# 1. Install direnv
# 2. Run: direnv allow
#
# This installs every client toolchain (ffmpeg, uv, go, bun, Chromium, ...).
if ! has nix_direnv_version || ! nix_direnv_version 3.1.0; then
source_url "https://raw.githubusercontent.com/nix-community/nix-direnv/3.1.0/direnvrc" "sha256-yMJ2OVMzrFaDPn7q8nCBZFRYpL/f0RcHzhmw/i6btJM="
fi
use_flake

# Reclaim stale Nix store paths in the background, at most once per week.
# nix-direnv pins the current dev shell, so only older versions are freed.
if has nix-collect-garbage; then
_gc_stamp="$PWD/.direnv/nix-gc-stamp"
if [ ! -f "$_gc_stamp" ] || [ -n "$(find "$_gc_stamp" -mtime +7 2>/dev/null)" ]; then
touch "$_gc_stamp"
(nix-collect-garbage --delete-older-than 7d >/dev/null 2>&1 &)
fi
fi
36 changes: 36 additions & 0 deletions test/smoke/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Per-client build/install artifacts. Everything here is fetched from a public
# package registry at test time, never committed.
clients/js/node_modules/
clients/js/dist-vite/
clients/js/dist-esbuild/
clients/js/artifacts/
clients/js-native/node_modules/
clients/token/js/node_modules/
clients/go/bin/
clients/swift/.build/
clients/kotlin/.gradle/
clients/kotlin/build/
clients/cloudflare/target/
*.log

# No committed PACKAGE lock files: every run must re-resolve the moq packages
# under test to their LATEST published versions. freshness.sh enforces this.
# (flake.lock is allowed: it pins the dev toolchain, not the moq packages, and
# the moq "nix" channel references the moq flake ad-hoc so it's never locked.)
go.sum
clients/js/bun.lock
clients/js/bun.lockb
clients/js-native/bun.lock
clients/js-native/bun.lockb
clients/token/js/bun.lock
clients/token/js/bun.lockb
Cargo.lock
uv.lock
package-lock.json
yarn.lock
pnpm-lock.yaml
clients/swift/Package.resolved
clients/kotlin/gradle.lockfile

# direnv / nix-direnv working dir.
.direnv/
30 changes: 30 additions & 0 deletions test/smoke/AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
Cross-language interop smoke for the published MoQ packages.
`test/interop` builds every client from this checkout; this suite installs the public artifacts instead and checks that they talk.

# Required

- Install the published artifact each cell claims to test.
- MoQ package versions stay at latest: `@moq/*` on the `latest` tag, PyPI `moq-rs`, `moq.dev/moq`, `moqdev/*` images, and GitHub release tarballs. `cloudflare.sh` and `moxygen.sh` build from unpinned Git HEAD.
- No package lock files under `test/smoke/` (`go.sum`, `bun.lock`, `Cargo.lock`, `uv.lock`, and the rest of the list in `.gitignore`). `flake.lock` pins the dev toolchain. The moq flake is not a flake input; `just nix-channel` builds `github:moq-dev/moq#moq-relay` and `#moq` with `--refresh`.
- npm `playwright` is the one version pin, and it matches the flake's `PLAYWRIGHT_VERSION`. Bump the toolchain with `nix flake update` and the pin together.
- `moq-relay` and `moq` come from a channel (PATH, `RELAY_BIN`/`MOQ_BIN`, `just nix-channel`, or `clients/docker`). The harness leaves installing them to that channel.
- A red cell is the result. `mark_broken` drops that package's cells and lets the rest of the matrix run. Retries belong on transient registry and release downloads. A client that failed to exchange media gets no retry, no longer timeout, and no skip.
- A published-package bug is fixed at its source, then goes green here on the next release.
- Hang media (`smoke.sh`), Cloudflare, and moxygen stay on separate lanes.
- In `relays.sh`, only `--required` relays fail the run. The default required relay is moq-dev's own.
- Wire new lanes into `.github/workflows/smoke.yml`. It runs on pull requests that touch `test/smoke/`, and nightly.
- New dependencies use the newest stable version, except the pins `freshness.sh` enforces.

# Development

This directory has its own flake, separate from moq's, so its toolchain floats on its own schedule.
direnv loads it here. Otherwise: `nix develop ./test/smoke --command ...`, or `just test smoke <recipe>` from the root.
The shell carries client toolchains. The moq binaries under test come from a channel.

```bash
just check # shfmt, shellcheck, actionlint, freshness
just smoke # rust-only matrix; flags pass through to smoke.sh
just full # cross-language matrix
```

`just check` does not run the matrix. Run the lane you changed (`just smoke`, `just token`, `just cloudflare`, `just moxygen`, `just relays`).
65 changes: 65 additions & 0 deletions test/smoke/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# moq smoke

### [Latest smoke results](https://moq-dev.github.io/moq/relays/)

Nightly publish/subscribe results for the published clients through public MoQ relays. One row per relay, sorted by name ignoring case, with a passing pair count per transport (WebTransport, QUIC, WebSocket). An empty cell means that relay has no endpoint there. Hover a count for the failure, click it for the logs. The page keeps the last 60 runs and updates from `main`.

This suite installs the published moq packages and checks that they exchange media. [`test/interop`](../interop) is its in-tree companion: it builds each client from this checkout instead, to catch interop regressions before anything is published. A missing wheel, a stale Homebrew formula, a broken `.deb`, or an export lost in packaging shows up as a red cell. A cell passes when the subscriber receives a non-empty frame.

## Lanes

| Lane | Run | Coverage |
| --- | --- | --- |
| Packages | `just smoke`, `just full` | Published clients through a local `moq-relay`. `smoke` is Rust only. `full` is the cross-language matrix. |
| Public relays | `just relays` | Rust publisher and subscriber through every relay in the [moq-interop-runner](https://github.com/englishm/moq-interop-runner) registry. CI publishes with Rust and the browser (`js-web`) and subscribes with Rust, `js-web`, and `js-bun`. On those relays `js-web` and `js-bun` run only on WebTransport (`@moq/web-transport` has no raw QUIC mode); a client that does not belong is omitted. WebSocket is its own column for the relays in `WEBSOCKET_KEYS` (`moq-dev-rs`): each of their `http(s)` endpoints also runs as `ws(s)`, and every other relay keeps that fallback off. A relay that refuses anonymous root broadcasts is dialed on the path `publish_path` names (`stitcher-moq` publishes under `/moq-test`). That run is what the [results page](https://moq-dev.github.io/moq/relays/) shows. Only `--required` relays fail the run; the default is moq-dev's `cdn.moq.dev`, including its WebSocket endpoint. |
| Cloudflare | `just cloudflare` | Relays built from the default branches of [cloudflare/moq-rs](https://github.com/cloudflare/moq-rs) and moq-dev/moq, driven by a Cloudflare client over WebTransport and raw QUIC. |
| moxygen | `just moxygen` | Meta's [`moxygen`](https://github.com/facebookexperimental/moxygen) interop client through the latest moq-dev relay. Needs Linux Docker. |
| Tokens | `just token`, `just token-full` | `moq auth` and npm [`@moq/auth`](https://www.npmjs.com/package/@moq/auth) mint and verify each other's JWTs, and each verifier rejects a tampered token. |

Hang media, Cloudflare, and moxygen each have their own lane.

CI runs the matrix nightly and on pull requests that touch `test/smoke/`: [`.github/workflows/smoke.yml`](../../.github/workflows/smoke.yml).

## Clients

`moq-relay` and `moq` come from `PATH` or from `RELAY_BIN` and `MOQ_BIN`. Every other client installs from its registry on each run.

| Client | Role | Install |
| --- | --- | --- |
| Rust | publish, subscribe | cargo, Homebrew, apt, `nix build github:moq-dev/moq`, or [`moqdev/*`](https://hub.docker.com/u/moqdev) images |
| Python | publish, subscribe | PyPI [`moq-rs`](https://pypi.org/project/moq-rs/) |
| Go | publish, subscribe | [`moq.dev/moq`](https://pkg.go.dev/moq.dev/moq) |
| Browser | publish, subscribe | npm [`@moq/watch`](https://www.npmjs.com/package/@moq/watch) + [`@moq/publish`](https://www.npmjs.com/package/@moq/publish), via vite, esbuild, or [jsDelivr](https://www.jsdelivr.com/) |
| Native JS | subscribe | npm [`@moq/net`](https://www.npmjs.com/package/@moq/net) + [`@moq/hang`](https://www.npmjs.com/package/@moq/hang) under node and bun |
| Swift | subscribe | SPM [`moq-dev/moq-swift`](https://github.com/moq-dev/moq-swift), macOS |
| Kotlin | subscribe | Maven Central [`dev.moq:moq`](https://central.sonatype.com/artifact/dev.moq/moq) |
| C | subscribe | [`libmoq`](https://github.com/moq-dev/moq/releases) release tarball, linked by hand, pkg-config, or CMake |
| GStreamer | subscribe | [`moq-gst`](https://github.com/moq-dev/moq/releases?q=moq-gst) plugin against system GStreamer |

## Run

This directory's own Nix flake carries the client toolchains, separate from moq's (direnv loads `.envrc` here). Put `moq-relay` and `moq` on `PATH` (cargo, Homebrew, or apt) before the lanes below.

```bash
cd test/smoke
nix develop
cargo install moq-relay moq-cli # or brew / apt
just full
just relays
just cloudflare
just moxygen
just token
just check # shfmt, shellcheck, actionlint, freshness
```

`just nix-channel` is a run, not an install. It builds `github:moq-dev/moq#moq-relay` and `#moq` with `nix build --refresh --no-link`, then runs `smoke.sh` with `RELAY_BIN` and `MOQ_BIN` pointed at those outputs. Nothing is left on `PATH`, so a later `just full` does not see those binaries. The Docker images work the same way: point `RELAY_BIN` and `MOQ_BIN` at the wrappers in [`clients/docker`](clients/docker). Those wrappers are not an install.

`just check` lints the harness. Run the lane you changed.

## Versions

Packages resolve to latest on every run: `@moq/*` on the `latest` tag, PyPI `moq-rs`, `moq.dev/moq`, `moqdev/*` images, and GitHub release tarballs. `cloudflare.sh` and `moxygen.sh` build from unpinned Git HEAD. `test/smoke/` commits no package lock files. `flake.lock` pins the dev toolchain.

npm `playwright` is pinned to the flake's `PLAYWRIGHT_VERSION`. `just freshness` fails when that pin drifts or a lock file is committed. Bump the toolchain with `nix flake update` and the pin together.

A broken package fails only its own cells.
16 changes: 16 additions & 0 deletions test/smoke/clients/c/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Builds the C smoke subscriber against the prebuilt libmoq using the CMake
# package config it ships (lib/cmake/moq/moq-config.cmake). This is the CMake
# consumer path: find_package locates the moq::moq imported target, which
# carries the include dir, the static archive, and the per-platform system
# libraries. A broken config (wrong paths, missing target or system libs) fails
# configure/build here -- breakage the hand-rolled cc build can't surface.
#
# smoke.sh points CMAKE_PREFIX_PATH at the extracted tarball root, so this is
# never built standalone; it's driven by c_build_cmake.
cmake_minimum_required(VERSION 3.16)
project(moq_c_smoke C)

find_package(moq REQUIRED CONFIG)

add_executable(smoke subscribe.c)
target_link_libraries(smoke PRIVATE moq::moq)
Loading
Loading