Skip to content

build: collapse compose, adopt bake, rewrite Makefile + CI (2/3) - #2

Closed
Coriou wants to merge 4 commits into
build/devx-overhaul-pr1from
build/devx-overhaul-pr2
Closed

build: collapse compose, adopt bake, rewrite Makefile + CI (2/3)#2
Coriou wants to merge 4 commits into
build/devx-overhaul-pr1from
build/devx-overhaul-pr2

Conversation

@Coriou

@Coriou Coriou commented May 16, 2026

Copy link
Copy Markdown
Owner

Summary

Second of three stacked PRs. Stacked on #1 — base is build/devx-overhaul-pr1; when #1 merges, GitHub will auto-rebase this onto main.

Lands Phases C + D + E + F: the contributor-visible workflow changes that make the new build system usable.

  • Phase C (refactor(compose): collapse 4 files into single profile-driven compose.yaml) — replaces docker-compose.{dev,prod,override,demod-test}.yml and docker/Dockerfile.demod-test with a single root compose.yaml using v2 profiles: dev (sdrpp-server + wavekit-api), prod-single-host (wavekit-full image-only), prod-distributed (wavekit-sdrpp-prod + wavekit-core-prod image-only), demod-test (interactive final-demod). Every build: block declares cache_from pointing at ghcr.io/coriou/wavekit:cache-<stage> registry refs (cache misses fall through silently per Req 4.3). nginx-reverse-proxy and codercom/code-server services removed (Req 2.7 — both unused). packages/sdr-host/docker-compose.yml untouched (Non-negotiable 14).
  • Phase D (build(bake): introduce GHCR-backed buildx bake pipeline) — adds docker/bake.hcl declaring 4 final targets (final, final-core, final-sdrpp, final-demod), 1 CI helper (ci-core), and 2 groups (default, demod). Each target's cache-from chain pulls per-stage refs; cache-to writes are gated by a CACHE_FROM_ONLY=true default (CI flips to false on main-branch push for mode=max writes). Multi-arch linux/amd64,linux/arm64 defaults. docker/build.sh and docker/platform-utils.sh deleted; docker/push.sh rewritten as a thin GHCR-only bake invocation using WAVEKIT_GH_OWNER env var (coriou fallback). docker/init.sh dropped the .docker-cache mkdir.
  • Phase E (chore(make): trim Makefile to lifecycle-focused target inventory) — 386 → 156 lines, 63 → 34 targets. Headline DX commands: make dev (pnpm dev — native, no Docker), make dev-stack (compose dev profile), make docker-build (buildx bake default group). The 22 forbidden targets (Req 6.2) are gone (make dev-up/docker-dev/docker-build-{full,core,sdrpp}/etc.). No emoji or decorative ANSI in newly-introduced output.
  • Phase F (ci: add docker-build job; bump Node to 22; Corepack-driven pnpm) — .github/workflows/ci.yml rewritten: lint-typecheck-test job runs full pnpm lint + typecheck + build + test on Node 22 via Corepack (previously missing the test step). New docker-build job depends on it, sets up QEMU + buildx (docker-container driver), runs docker/bake-action@v5. PR path: ci-core target multi-arch, cache-from only, no login, no push. Main path: log in to GHCR with GITHUB_TOKEN, build default group, push: true, CACHE_FROM_ONLY=false for mode=max cache-to writes.

Properties verified locally

  • Property 11 (no plain docker build): grep -rn '\bdocker build\b' docker/ Makefile compose.yaml | grep -v buildx → empty.
  • Property 12 (single compose file): find . -maxdepth 1 -type f -name 'compose*.yaml' -o -name 'docker-compose*.yml' → only ./compose.yaml.
  • Bake parses: docker buildx bake --file docker/bake.hcl --print default produces valid JSON with the expected default[final, final-core, final-sdrpp] group.
  • Single-arch warm bake: 36 s wall time, all stages CACHED on re-run.
  • Multi-arch local bake: ci-core for linux/amd64,linux/arm64 succeeds (46 min cold under QEMU; subsequent warm builds much faster once GHCR cache populated by main-branch push).
  • docker compose --profile dev up --build end-to-end: wavekit-api reaches healthy, curl localhost:9000/health returns 200.

Notable deviations (documented inline in code)

  • sdrpp-server healthcheck: switched from design's HTTP curl :5259/ to TCP-port probe via bash -c '</dev/tcp/localhost/5259'. SDR++ in server mode speaks the binary RTL-TCP protocol on 5259, not HTTP — curl never succeeds. The TCP-reachability probe matches the actual contract.
  • wavekit-api port mapping: 9000:3000 (not 9000:9000 as in design.md §3) because config/default.yaml has api.port=3000 and the s6 run script doesn't use with-contenv to inherit container env. Matches the legacy make dev-up pattern. The container-internal healthcheck probes localhost:3000/health accordingly.
  • docker/scripts/start-sdrpp.sh: removed --log-level info (one-line surgical fix; pre-existing bug surfaced by Phase B's corrected SDR++ install path — the flag triggers a basic_string from null std::logic_error on SDR++ v1.1.0). Without this fix, the dev profile's end-to-end test couldn't pass.

Deferred

  • D.8 (initial GHCR cache seed): a one-time docker buildx bake … --push --cache-to mode=max bootstrap from a local machine. Deferred — Phase F's CI workflow seeds the cache via the same mechanism on the first main-branch push after this PR merges, making the manual seed redundant. Cold PR builds before that first main push will use the Req 4.3 "no cache" fallback (~25-40 min on first PR; subsequent PRs ~2-5 min once cache is live).
  • F.7 (live draft-PR smoke): the CI's GHCR cache-from path can only be fully validated against a live GHCR cache. Per the plan's R5 mitigation, this PR opening IS that smoke test — the docker-build job's cold-cache PR build is the validation.

Test plan

  • CI green on both lint-typecheck-test and docker-build jobs for this PR
  • Reviewer confirms compose.yaml profiles parse cleanly: for p in dev prod-single-host prod-distributed demod-test; do docker compose --profile $p config -q; done exits 0 for each
  • Reviewer confirms make help lists only the canonical inventory + sdr-host-* + fixtures-*
  • Reviewer confirms the bake's default group → [final, final-core, final-sdrpp] and the demod group → [final-demod]
  • Reviewer confirms .github/workflows/ci.yml PR path has no login/push: true/cache-to=type=registry, and main path has all three

Spec

.kiro/specs/build-devx-overhaul/tasks.md Phase C/D/E/F sections checked off. D.8 and F.7 marked DEFERRED with rationale.

Coriou added 4 commits May 16, 2026 09:30
…e.yaml

Replace docker-compose.{dev,prod,override,demod-test}.yml plus the standalone
docker/Dockerfile.demod-test with a single root-level compose.yaml that uses
Compose v2 profiles (dev, prod-single-host, prod-distributed, demod-test).
The demod-test service now builds from the Dockerfile's final-demod stage
landed in Phase B; the standalone Dockerfile.demod-test is removed.

Each build: block declares cache_from pointing at ghcr.io/coriou/wavekit:
cache-<stage> registry refs (cache_to omitted; only CI on main writes cache
per design.md §4.2). The dev compose no longer defines nginx-reverse-proxy
or vscode-server services (both unused; removed per Req 2.7). No reference
to .docker-cache anywhere (Req 4.5).

Three deviations from design.md §3 were necessary to make the C.12 gate pass;
each is documented inline in compose.yaml:

  1. sdrpp-server healthcheck switched from `curl http://localhost:5259/` to
     a TCP-port probe via `bash -c '</dev/tcp/localhost/5259'`. SDR++ in
     server mode speaks its own binary protocol on 5259, not HTTP, so curl
     fails on every probe with `HTTP/0.9 not allowed`. Same change applied
     to prod-distributed's wavekit-sdrpp-prod for consistency.

  2. wavekit-api port mapping changed from `9000:9000` to `9000:3000` to
     match the legacy `make dev-up -p 9000:3000` pattern. config/default.yaml
     sets api.port=3000 and the s6-supervised process doesn't inherit
     WAVEKIT_API_PORT from container env (s6's `with-contenv` isn't used
     in the run script — Phase B scope). The wavekit-api healthcheck (which
     runs inside the container) probes `http://localhost:3000/health` for
     the same reason.

  3. docker/scripts/start-sdrpp.sh: removed `--log-level info` from the
     sdrpp invocation. SDR++ v1.1.0 doesn't accept that flag and crash-loops
     with `basic_string from null` std::logic_error, preventing sdrpp-server
     from ever binding port 5259. Pre-existing bug surfaced by Phase B's
     corrected SDR++ install path; minimum-touch unblock for C.12.

C.12 result: `docker compose --profile dev up -d --wait` brings both
containers to (healthy) state; `curl http://localhost:9000/health` returns
`{"status":"ok","timestamp":"..."}` within 1s of api readiness. Cold-start
time is ~3 min, dominated by SDR++'s first-run config-file initialization
crash-loop (s6 restarts the process until the JSON configs settle).

packages/sdr-host/docker-compose.yml is untouched — it deploys to the Pi
and is not part of the root compose (Non-negotiable 14).

Phase C of build/devx-overhaul-pr2. Closes tasks C.1-C.12.
Add docker/bake.hcl declaring four build targets (final, final-core,
final-sdrpp, final-demod), one CI helper (ci-core), and two groups
(default, demod). Each target's cache-from chain pulls per-stage refs
from ghcr.io/coriou/wavekit:cache-<stage>; cache-to writes are gated
by a CACHE_FROM_ONLY=true default (CI flips to false on main push to
seed cache via mode=max). Multi-arch (linux/amd64,linux/arm64)
defaults across every target.

docker/push.sh rewritten as a thin bake invocation: GHCR-only (drops
the docker.io path per Req 5.5), uses WAVEKIT_GH_OWNER env var with
coriou fallback per packages/sdr-host/scripts/build-publish.sh
pattern, drops linux/arm/v7 entirely (Req 5.4).

docker/init.sh: dropped the .docker-cache mkdir/chmod; the local
directory cache is replaced by GHCR registry cache (Req 4.5). Buildx
builder + network/volume setup retained. "Next steps" footer updated
to reference Phase E targets (make dev-stack / make dev-stack-logs).

docker/build.sh: deleted. Makefile in Phase E will invoke `docker
buildx bake --file docker/bake.hcl default` directly; the wrapper
provided no additional value.

docker/platform-utils.sh: deleted. build_multiarch is replaced by
bake's platforms directive; detect_platform had no remaining
consumers (verified via grep on Makefile + docker/ + .github/ +
packages/sdr-host/).

D.8 (initial cache seed bootstrap) is deferred to the user / CI's
first main-branch push. Phase F's CI workflow will populate GHCR
cache via cache-to mode=max on its first main-branch push, making
the manual seed redundant. PR cold builds proceed via Req 4.3's
"no cache" fallback contract (verified during D.3 multi-arch test:
anonymous GHCR cache-from returns 403 silently; build still
succeeds).

Phase D of build/devx-overhaul-pr2. Closes D.1-D.7; D.8 deferred.
Replace the 386-line / 63-target Makefile with a focused inventory of 15
canonical lifecycle targets plus the retained fixtures-* (5) and sdr-host-*
(13) groups. The new Makefile exposes:

  help                              # default; lists targets
  dev / dev-dashboard / dev-dashboard-build / dev-configs  # native loop
  dev-stack / dev-stack-down / dev-stack-logs              # compose dev profile
  dev-shell / dev-status                                   # container introspection
  docker-init / docker-build / docker-push                 # buildx bake pipeline
  docker-clean / docker-prune                              # housekeeping
  demod-test                                               # interactive decoder test env
  sdr-host-*  (13 targets, unchanged)                      # Pi-hosted SDR
  fixtures-*  (5 targets, unchanged)                       # test corpus

Deleted (per Req 6.2 + design §7.2):
  docker-{dev,prod,build-full,build-core,build-sdrpp,run-core,test,test-coverage,lint,
          compose-up,compose-down,compose-logs,logs,logs-api,logs-sdrpp,logs-decoders,
          shell,status,health,ps,info,inspect,history}
  dev-{up,build,start,stop,restart,logs,logs-raw,audio,debug-audio}
  demo, install-buildx, cli-install (folded into dev-dashboard-build)

Newly-introduced output is emoji-free and uses no decorative ANSI (Req 6.3).
Existing colour in retained sdr-host-* / fixtures-* recipes preserved to
avoid drive-by churn.

Phase E of build/devx-overhaul-pr2. Closes tasks E.1-E.4.
Rewrite .github/workflows/ci.yml to exercise the Phase D buildx bake
pipeline on every PR and push to main:

- lint-typecheck-test (was the only job before) now runs the full
  pnpm lint + typecheck + build + test sequence on Node 22 with
  pnpm installed via Corepack from the package.json packageManager
  field. Drops the pnpm/action-setup pin and the Node 20 floor.

- docker-build is a new job that depends on lint-typecheck-test,
  sets up QEMU + buildx (docker-container driver), and runs
  docker/bake-action@v5 against docker/bake.hcl. On pull_request it
  builds the `ci-core` target multi-arch with cache-from only (no
  GHCR auth needed; cache misses fall through per Req 4.3). On
  push to main it logs in to GHCR with GITHUB_TOKEN, builds the
  `default` group (final, final-core, final-sdrpp) multi-arch with
  push: true, and writes cache-to mode=max to seed the GHCR
  registry cache for downstream builds.

F.7 (live draft-PR smoke test) is deferred to a manual maintainer
step after PR 2 is up. Cold-cache PR build expected ~15-25 min;
subsequent PRs with cache hits ~2-5 min per Req 4.3.

Phase F of build/devx-overhaul-pr2. Closes F.1-F.6; F.7 deferred.
@Coriou
Coriou deleted the branch build/devx-overhaul-pr1 May 16, 2026 15:41
@Coriou Coriou closed this May 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant