Skip to content

build: s6 tree partition + Dockerfile refactor (1/3) - #1

Merged
Coriou merged 4 commits into
mainfrom
build/devx-overhaul-pr1
May 16, 2026
Merged

build: s6 tree partition + Dockerfile refactor (1/3)#1
Coriou merged 4 commits into
mainfrom
build/devx-overhaul-pr1

Conversation

@Coriou

@Coriou Coriou commented May 16, 2026

Copy link
Copy Markdown
Owner

Summary

First of three stacked PRs implementing the build & DX overhaul spec at .kiro/specs/build-devx-overhaul/. This PR lands Phases A + B: the s6 service-tree partition and the top-to-bottom Dockerfile refactor.

  • Phase A (refactor(s6): partition sdrpp-server into separate overlay tree) — moves the sdrpp-server service definition, its services/contents.d/ and user/contents.d/ registrations out of the canonical docker/overlay/s6-overlay/ tree into a sibling docker/overlay/s6-overlay-sdrpp/ tree. Also deletes the canonical tree's wavekit-api → sdrpp-server hard dep (loosened per design §5; SDR-source reachability is now solely SourceManager's exponential-backoff contract). All 6 file moves recorded as R100 (git history preserved).
  • Phase B (3 commits) — Dockerfile rewrite from a 737-line monolith into 20 narrow, named stages: thin base-build, 12 decoder *-build stages each FROM base-build with own apt deps, single-install node-build, then final-base (shared by final/final-core) plus final-sdrpp and final-demod. All 14 upstream decoder/lib refs pinned via ARG <NAME>_REF=<sha> at top of file; checkout pattern is git fetch --depth 1 origin "${REF}" && git checkout --detach FETCH_HEAD. final-core is now a one-line LABEL mode=core stage with zero rm -rf post-hoc surgery on the s6 tree.

The third commit on this PR (build(docker): exclude **/node_modules from build context; restore Req 3.6) fixes a .dockerignore defect that previously required a pnpm config set node-linker hoisted override in node-build to work around two issues:

  1. .dockerignore's bare node_modules pattern only matched the context-root directory, so cli/node_modules and packages/*/node_modules entered the build context and clobbered pnpm's fresh symlinks during COPY . ..
  2. .npmrc's enable-global-virtual-store=true put the virtual store at ~/.local/share/pnpm/store/v10/links/ which collided with this stage's BuildKit cache mount target.

Fixing both root causes (**/node_modules in dockerignore; --config.enable-global-virtual-store=false on the install) lifted the hoisted-linker override entirely. .npmrc's node-linker=isolated is now the single source of truth (Req 3.6 satisfied strictly).

Properties verified locally (Non-negotiables 7, 8, 9)

  • Property 8: docker run --rm wavekit:dev-core find /etc/s6-overlay -iname '*sdrpp*' → empty. ls /usr/local/bin /usr/bin | grep -i sdrpp → empty.
  • Property 9: docker run --rm wavekit:dev-core ls /etc/s6-overlay/s6-rc.d/wavekit-api/dependencies.d/ → exactly wavekit-init.
  • Property 8 inverse: docker run --rm wavekit:dev find /etc/s6-overlay -name sdrpp-server -type d → exactly one match.
  • All 4 final targets (final, final-core, final-sdrpp, final-demod) build clean via docker buildx build --target <t> --load ..
  • dsd-fme ncurses linkage on amd64 resolves via Debian's libncurses6 package (the hardcoded COPY --from=dsd-fme-build /usr/lib/x86_64-linux-gnu/libncurses* line was removed entirely per Req 3.13's preferred path).

R2 / R3 notes

  • R2 (ncurses): hardcoded x86_64-linux-gnu COPY deleted; verified ldd /usr/local/bin/dsd-fme on amd64 resolves to the runtime-base libncurses6. Debian's libncurses6 is multi-arch so the same install pattern works on arm64; full arm64 build was started but not completed end-to-end (QEMU wall-cost). If arm64 ever shows a missing symbol the fix is a TARGETARCH-conditional COPY.
  • R3 (node-linker): see commit 65bc16b body for the full diagnostic chain. Final state: zero pnpm config set calls in the Dockerfile; isolated linker honored from .npmrc.

Test plan

  • CI's lint-typecheck-test job green (Node 22 + Corepack)
  • CI's docker-build job green on ci-core target (multi-arch; cache-from only on PR — first run will be cold, ~20-30 min, subsequent ~2-5 min once cache seeded)
  • Reviewer reads through the new Dockerfile section-by-section against the spec's stage inventory (design §2.1)
  • Reviewer confirms docker/overlay/s6-overlay/ is sdrpp-free and docker/overlay/s6-overlay-sdrpp/ contains the 6 expected files
  • Reviewer confirms final-core has no rm -rf targeting /etc/s6-overlay/ (mode partition achieved by additive COPY only)

Follow-ons (separate PRs)

  • PR 2 (#TBD): compose collapse + buildx bake + Makefile + CI
  • PR 3 (#TBD): sdr-host Corepack + docs + dockerignore audit

Spec

  • .kiro/specs/build-devx-overhaul/requirements.md — 11 user stories, 51 acceptance criteria
  • .kiro/specs/build-devx-overhaul/design.md — 14 anti-patterns retired, 20-stage Dockerfile structure, s6 contract change, 12 correctness properties
  • .kiro/specs/build-devx-overhaul/tasks.md — 60-task implementation checklist (A.1–B.2.12 checked off in this PR)

Coriou added 4 commits May 16, 2026 01:56
Kiro spec at .kiro/specs/build-devx-overhaul/ defines 11 user stories, 51
acceptance criteria, a 20-stage Dockerfile structure, compose topology, GHCR
cache strategy, s6 contract change, 12 correctness properties, risk register,
and a 60-task dependency-sequenced implementation checklist.

Plan at docs/superpowers/plans/2026-05-16-build-devx-overhaul.md sequences 9
phases (A-I) across 3 PR boundaries, defines 8 verification gates, and flags
10 execution-time risks.

This commit lands the contract; subsequent commits on this branch implement
Phases A and B (s6 tree partition + Dockerfile refactor) per the spec.
Move the sdrpp-server service definition, its services/contents.d/
registration, and its user/contents.d/ registration out of the canonical
docker/overlay/s6-overlay/ tree and into a sibling
docker/overlay/s6-overlay-sdrpp/ tree. Delete the canonical tree's
wavekit-api -> sdrpp-server hard dep; wavekit-api now depends only on
wavekit-init.

Phase A of build/devx-overhaul; sets up the Phase B Dockerfile to COPY
the canonical tree into final-base (no sdrpp residue) and additively COPY
the sdrpp tree into the final stage only. See
.kiro/specs/build-devx-overhaul/{requirements,design,tasks}.md and
docs/superpowers/plans/2026-05-16-build-devx-overhaul.md.
Replaces the 737-line monolithic Dockerfile with 20 stages per the Build & DX
Overhaul spec (Phase B). Each decoder builds on a thin `base-build` stage and
installs only its own apt deps; upstream sources are pinned to SHAs at the
top of the file. The 4 final targets (`final`, `final-core`, `final-sdrpp`,
`final-demod`) build cleanly via `docker buildx build --target <t> --load`.

Stage inventory:
  - base-build / runtime-base (toolchain + runtime libs)
  - 12 decoder builders: sdrpp, dsd-fme, multimon-ng, rtl433, acarsdec,
    ais-catcher, direwolf, dumpvdl2, readsb, soapy-rtltcp, csdr, lora
  - node-build (single pnpm install)
  - 5 finals: final-base (common ancestor), final (+SDR++), final-core
    (no additions), final-sdrpp (SDR++ only), final-demod (interactive)

Key changes:
  - Pinned ARG <NAME>_REF for every upstream (initial values captured via
    `git ls-remote`; bumps go through PR review). Re-uses lora-build's
    existing pinned-checkout pattern across all decoders.
  - `runtime-base` selects s6-overlay via TARGETARCH (was BUILDPLATFORM —
    wrong for cross-arch builds). Drops the /var/lib/apt cache mount,
    dedupes libsox-fmt-all, removes unused tini.
  - `final-base` consolidates everything `final` and `final-core` shared.
    `final-core` is a one-line stage; the mode partition is now achieved
    by additive COPY from `docker/overlay/s6-overlay-sdrpp/` (Phase A) into
    `final`, not by `rm -rf` post-hoc (Req 3.10).
  - Hardcoded `COPY /usr/lib/x86_64-linux-gnu/libncurses*` deleted — the
    runtime `libncurses6` apt package satisfies dsd-fme's linkage on
    both amd64 and arm64 (Req 3.13).
  - Old `FROM final AS default` stage removed; bake-driven default group
    in Phase D supersedes it.

Notes / mitigations:
  - R3 (node-build): a pure-isolated single install couldn't survive both
    (a) the pnpm store cache mount dropping the global virtual store path
    after the layer commits, and (b) `cli/node_modules` and
    `packages/*/node_modules` leaking into the build context from the host
    (.dockerignore only excludes root-level `node_modules`). Per the Phase
    B brief, set `node-linker hoisted` BEFORE the single install. Project
    .npmrc default (`isolated`) is unchanged so `pnpm dev` is unaffected.
    Follow-up: fix .dockerignore to exclude `**/node_modules` so the
    override can be lifted (Req 3.6).
  - SDR++ install paths corrected to `/usr/bin/sdrpp` and `/usr/lib/sdrpp/`
    (current upstream installs there, not `/usr/local/`; the old COPY
    silently no-op'd). SDR++ runtime libs (libglfw3, libvolk2.5, libglvnd0,
    libopengl0) added only to `final` and `final-sdrpp` so `final-core`
    stays slim.
  - `final-sdrpp` reconstructed: the previous `COPY .../s6-rc.d/base ...`
    referenced a directory renamed to `wavekit-init/` in commit 7d18546.
    Now uses per-file COPYs of the canonical structural pieces
    (services/type, user/type, user/contents.d/services) plus the
    sdrpp-server service from the sibling overlay.
  - dsd-fme verification widened from `--version` to `command -v` because
    the pinned HEAD no longer accepts `--version`. Binary presence on PATH
    is the relevant smoke contract.

Gate B verification (PASS):
  - final-core has no sdrpp residue (Property 8)
  - wavekit-api dependencies: only `wavekit-init` (Property 9)
  - final has the sdrpp-server tree (additive overlay landed)
  - `node -e "require('fastify'); ..."` returns OK in final-core
  - 15 top-of-file ARGs (1 S6_OVERLAY_VERSION + 14 decoder refs)
  - touched-src warm build keeps every decoder *-build stage CACHED

Builds all 4 targets via buildx --load successfully.

Refs: .kiro/specs/build-devx-overhaul/{requirements,design,tasks}.md
      docs/superpowers/plans/2026-05-16-build-devx-overhaul.md
…q 3.6

Phase B's node-build originally needed a `pnpm config set node-linker
hoisted` reconfigure to work around two independent issues:

1. .dockerignore's bare `node_modules` pattern only matches the
   context-root directory, so cli/node_modules and packages/*/node_modules
   entered the build context and clobbered pnpm's fresh symlinks during
   COPY . .

2. .npmrc's `enable-global-virtual-store=true` puts the virtual store at
   ~/.local/share/pnpm/store/v10/links/, which collides with this stage's
   BuildKit cache mount target — symlinks dangle once the cache mount
   unmounts at layer commit.

Fixing both root causes restores Requirement 3.6 strictly (.npmrc
`node-linker=isolated` is the single source of truth for package layout):

- .dockerignore: `node_modules` -> `**/node_modules`
- Dockerfile: pass `--config.enable-global-virtual-store=false` to the
  install invocation (and the matching prune step), dropping the prior
  `pnpm config set node-linker hoisted` reconfigure entirely.

The override is a per-invocation flag; .npmrc remains untouched so `pnpm
dev` outside Docker uses the project default unchanged.
@Coriou
Coriou merged commit 4dac7ab into main May 16, 2026
2 checks passed
@Coriou
Coriou deleted the build/devx-overhaul-pr1 branch May 16, 2026 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant