build: collapse compose, adopt bake, rewrite Makefile + CI (2/3) - #2
Closed
Coriou wants to merge 4 commits into
Closed
build: collapse compose, adopt bake, rewrite Makefile + CI (2/3)#2Coriou wants to merge 4 commits into
Coriou wants to merge 4 commits into
Conversation
…e.yaml
Replace docker-compose.{dev,prod,override,demod-test}.yml plus the standalone
docker/Dockerfile.demod-test with a single root-level compose.yaml that uses
Compose v2 profiles (dev, prod-single-host, prod-distributed, demod-test).
The demod-test service now builds from the Dockerfile's final-demod stage
landed in Phase B; the standalone Dockerfile.demod-test is removed.
Each build: block declares cache_from pointing at ghcr.io/coriou/wavekit:
cache-<stage> registry refs (cache_to omitted; only CI on main writes cache
per design.md §4.2). The dev compose no longer defines nginx-reverse-proxy
or vscode-server services (both unused; removed per Req 2.7). No reference
to .docker-cache anywhere (Req 4.5).
Three deviations from design.md §3 were necessary to make the C.12 gate pass;
each is documented inline in compose.yaml:
1. sdrpp-server healthcheck switched from `curl http://localhost:5259/` to
a TCP-port probe via `bash -c '</dev/tcp/localhost/5259'`. SDR++ in
server mode speaks its own binary protocol on 5259, not HTTP, so curl
fails on every probe with `HTTP/0.9 not allowed`. Same change applied
to prod-distributed's wavekit-sdrpp-prod for consistency.
2. wavekit-api port mapping changed from `9000:9000` to `9000:3000` to
match the legacy `make dev-up -p 9000:3000` pattern. config/default.yaml
sets api.port=3000 and the s6-supervised process doesn't inherit
WAVEKIT_API_PORT from container env (s6's `with-contenv` isn't used
in the run script — Phase B scope). The wavekit-api healthcheck (which
runs inside the container) probes `http://localhost:3000/health` for
the same reason.
3. docker/scripts/start-sdrpp.sh: removed `--log-level info` from the
sdrpp invocation. SDR++ v1.1.0 doesn't accept that flag and crash-loops
with `basic_string from null` std::logic_error, preventing sdrpp-server
from ever binding port 5259. Pre-existing bug surfaced by Phase B's
corrected SDR++ install path; minimum-touch unblock for C.12.
C.12 result: `docker compose --profile dev up -d --wait` brings both
containers to (healthy) state; `curl http://localhost:9000/health` returns
`{"status":"ok","timestamp":"..."}` within 1s of api readiness. Cold-start
time is ~3 min, dominated by SDR++'s first-run config-file initialization
crash-loop (s6 restarts the process until the JSON configs settle).
packages/sdr-host/docker-compose.yml is untouched — it deploys to the Pi
and is not part of the root compose (Non-negotiable 14).
Phase C of build/devx-overhaul-pr2. Closes tasks C.1-C.12.
Add docker/bake.hcl declaring four build targets (final, final-core, final-sdrpp, final-demod), one CI helper (ci-core), and two groups (default, demod). Each target's cache-from chain pulls per-stage refs from ghcr.io/coriou/wavekit:cache-<stage>; cache-to writes are gated by a CACHE_FROM_ONLY=true default (CI flips to false on main push to seed cache via mode=max). Multi-arch (linux/amd64,linux/arm64) defaults across every target. docker/push.sh rewritten as a thin bake invocation: GHCR-only (drops the docker.io path per Req 5.5), uses WAVEKIT_GH_OWNER env var with coriou fallback per packages/sdr-host/scripts/build-publish.sh pattern, drops linux/arm/v7 entirely (Req 5.4). docker/init.sh: dropped the .docker-cache mkdir/chmod; the local directory cache is replaced by GHCR registry cache (Req 4.5). Buildx builder + network/volume setup retained. "Next steps" footer updated to reference Phase E targets (make dev-stack / make dev-stack-logs). docker/build.sh: deleted. Makefile in Phase E will invoke `docker buildx bake --file docker/bake.hcl default` directly; the wrapper provided no additional value. docker/platform-utils.sh: deleted. build_multiarch is replaced by bake's platforms directive; detect_platform had no remaining consumers (verified via grep on Makefile + docker/ + .github/ + packages/sdr-host/). D.8 (initial cache seed bootstrap) is deferred to the user / CI's first main-branch push. Phase F's CI workflow will populate GHCR cache via cache-to mode=max on its first main-branch push, making the manual seed redundant. PR cold builds proceed via Req 4.3's "no cache" fallback contract (verified during D.3 multi-arch test: anonymous GHCR cache-from returns 403 silently; build still succeeds). Phase D of build/devx-overhaul-pr2. Closes D.1-D.7; D.8 deferred.
Replace the 386-line / 63-target Makefile with a focused inventory of 15
canonical lifecycle targets plus the retained fixtures-* (5) and sdr-host-*
(13) groups. The new Makefile exposes:
help # default; lists targets
dev / dev-dashboard / dev-dashboard-build / dev-configs # native loop
dev-stack / dev-stack-down / dev-stack-logs # compose dev profile
dev-shell / dev-status # container introspection
docker-init / docker-build / docker-push # buildx bake pipeline
docker-clean / docker-prune # housekeeping
demod-test # interactive decoder test env
sdr-host-* (13 targets, unchanged) # Pi-hosted SDR
fixtures-* (5 targets, unchanged) # test corpus
Deleted (per Req 6.2 + design §7.2):
docker-{dev,prod,build-full,build-core,build-sdrpp,run-core,test,test-coverage,lint,
compose-up,compose-down,compose-logs,logs,logs-api,logs-sdrpp,logs-decoders,
shell,status,health,ps,info,inspect,history}
dev-{up,build,start,stop,restart,logs,logs-raw,audio,debug-audio}
demo, install-buildx, cli-install (folded into dev-dashboard-build)
Newly-introduced output is emoji-free and uses no decorative ANSI (Req 6.3).
Existing colour in retained sdr-host-* / fixtures-* recipes preserved to
avoid drive-by churn.
Phase E of build/devx-overhaul-pr2. Closes tasks E.1-E.4.
Rewrite .github/workflows/ci.yml to exercise the Phase D buildx bake pipeline on every PR and push to main: - lint-typecheck-test (was the only job before) now runs the full pnpm lint + typecheck + build + test sequence on Node 22 with pnpm installed via Corepack from the package.json packageManager field. Drops the pnpm/action-setup pin and the Node 20 floor. - docker-build is a new job that depends on lint-typecheck-test, sets up QEMU + buildx (docker-container driver), and runs docker/bake-action@v5 against docker/bake.hcl. On pull_request it builds the `ci-core` target multi-arch with cache-from only (no GHCR auth needed; cache misses fall through per Req 4.3). On push to main it logs in to GHCR with GITHUB_TOKEN, builds the `default` group (final, final-core, final-sdrpp) multi-arch with push: true, and writes cache-to mode=max to seed the GHCR registry cache for downstream builds. F.7 (live draft-PR smoke test) is deferred to a manual maintainer step after PR 2 is up. Cold-cache PR build expected ~15-25 min; subsequent PRs with cache hits ~2-5 min per Req 4.3. Phase F of build/devx-overhaul-pr2. Closes F.1-F.6; F.7 deferred.
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Second of three stacked PRs. Stacked on #1 — base is
build/devx-overhaul-pr1; when #1 merges, GitHub will auto-rebase this ontomain.Lands Phases C + D + E + F: the contributor-visible workflow changes that make the new build system usable.
refactor(compose): collapse 4 files into single profile-driven compose.yaml) — replacesdocker-compose.{dev,prod,override,demod-test}.ymlanddocker/Dockerfile.demod-testwith a single rootcompose.yamlusing v2 profiles:dev(sdrpp-server + wavekit-api),prod-single-host(wavekit-full image-only),prod-distributed(wavekit-sdrpp-prod + wavekit-core-prod image-only),demod-test(interactivefinal-demod). Everybuild:block declarescache_frompointing atghcr.io/coriou/wavekit:cache-<stage>registry refs (cache misses fall through silently per Req 4.3).nginx-reverse-proxyandcodercom/code-serverservices removed (Req 2.7 — both unused).packages/sdr-host/docker-compose.ymluntouched (Non-negotiable 14).build(bake): introduce GHCR-backed buildx bake pipeline) — addsdocker/bake.hcldeclaring 4 final targets (final,final-core,final-sdrpp,final-demod), 1 CI helper (ci-core), and 2 groups (default,demod). Each target'scache-fromchain pulls per-stage refs;cache-towrites are gated by aCACHE_FROM_ONLY=truedefault (CI flips tofalseon main-branch push formode=maxwrites). Multi-archlinux/amd64,linux/arm64defaults.docker/build.shanddocker/platform-utils.shdeleted;docker/push.shrewritten as a thin GHCR-only bake invocation usingWAVEKIT_GH_OWNERenv var (coriou fallback).docker/init.shdropped the.docker-cachemkdir.chore(make): trim Makefile to lifecycle-focused target inventory) — 386 → 156 lines, 63 → 34 targets. Headline DX commands:make dev(pnpm dev — native, no Docker),make dev-stack(compose dev profile),make docker-build(buildx bake default group). The 22 forbidden targets (Req 6.2) are gone (make dev-up/docker-dev/docker-build-{full,core,sdrpp}/etc.). No emoji or decorative ANSI in newly-introduced output.ci: add docker-build job; bump Node to 22; Corepack-driven pnpm) —.github/workflows/ci.ymlrewritten:lint-typecheck-testjob runs full pnpm lint + typecheck + build + test on Node 22 via Corepack (previously missing the test step). Newdocker-buildjob depends on it, sets up QEMU + buildx (docker-container driver), runsdocker/bake-action@v5. PR path:ci-coretarget multi-arch, cache-from only, no login, no push. Main path: log in to GHCR withGITHUB_TOKEN, builddefaultgroup,push: true,CACHE_FROM_ONLY=falseformode=maxcache-to writes.Properties verified locally
docker build):grep -rn '\bdocker build\b' docker/ Makefile compose.yaml | grep -v buildx→ empty.find . -maxdepth 1 -type f -name 'compose*.yaml' -o -name 'docker-compose*.yml'→ only./compose.yaml.docker buildx bake --file docker/bake.hcl --print defaultproduces valid JSON with the expecteddefault→[final, final-core, final-sdrpp]group.ci-coreforlinux/amd64,linux/arm64succeeds (46 min cold under QEMU; subsequent warm builds much faster once GHCR cache populated by main-branch push).docker compose --profile dev up --buildend-to-end:wavekit-apireaches healthy,curl localhost:9000/healthreturns 200.Notable deviations (documented inline in code)
curl :5259/to TCP-port probe viabash -c '</dev/tcp/localhost/5259'. SDR++ in server mode speaks the binary RTL-TCP protocol on 5259, not HTTP — curl never succeeds. The TCP-reachability probe matches the actual contract.9000:3000(not9000:9000as in design.md §3) becauseconfig/default.yamlhasapi.port=3000and the s6runscript doesn't usewith-contenvto inherit container env. Matches the legacymake dev-uppattern. The container-internal healthcheck probeslocalhost:3000/healthaccordingly.docker/scripts/start-sdrpp.sh: removed--log-level info(one-line surgical fix; pre-existing bug surfaced by Phase B's corrected SDR++ install path — the flag triggers abasic_string from nullstd::logic_error on SDR++ v1.1.0). Without this fix, the dev profile's end-to-end test couldn't pass.Deferred
docker buildx bake … --push --cache-to mode=maxbootstrap from a local machine. Deferred — Phase F's CI workflow seeds the cache via the same mechanism on the first main-branch push after this PR merges, making the manual seed redundant. Cold PR builds before that first main push will use the Req 4.3 "no cache" fallback (~25-40 min on first PR; subsequent PRs ~2-5 min once cache is live).docker-buildjob's cold-cache PR build is the validation.Test plan
lint-typecheck-testanddocker-buildjobs for this PRcompose.yamlprofiles parse cleanly:for p in dev prod-single-host prod-distributed demod-test; do docker compose --profile $p config -q; doneexits 0 for eachmake helplists only the canonical inventory + sdr-host-* + fixtures-*defaultgroup →[final, final-core, final-sdrpp]and thedemodgroup →[final-demod].github/workflows/ci.ymlPR path has nologin/push: true/cache-to=type=registry, and main path has all threeSpec
.kiro/specs/build-devx-overhaul/tasks.mdPhase C/D/E/F sections checked off. D.8 and F.7 marked DEFERRED with rationale.