Package and verify actual production runtime consumers

.agents/inbox/2026-10-27-runtime-images-need-installed-consumer-proof.md

@@ -0,0 +1,46 @@
+# Runtime pruning needs actual consumer and persistence evidence
+
+Date: 2026-10-27. Issue #185. Searches of the symptom/security indices found
+consumer-verified dependency and startup lessons, but no runtime-pruning contract.
+
+## Evidence
+
+`docker run --rm --entrypoint node -i graphdone-audit-patched-api:20261023
+--input-type=module < tests/runtime/image-inventory.mjs` failed on installed
+Vite/Vitest/compiler/lint dependencies. A selected production install removed
+them, but the first real compiled API launch failed `ERR_MODULE_NOT_FOUND` for
+`graphql-tag`. The dependency had been supplied by the excluded web workspace.
+Declaring the existing locked version in server restored the real consumer.
+
+Review also found a remaining npm-based startup shell call, proposed UID drift,
+an SDK root entry that is intentionally absent, and auth volumes targeting a
+directory different from the process's store. The final source removes the
+redundant shell call, preserves runtime identities, validates SDK subpaths and
+points production volumes at the actual store. Three final image contracts pass
+real API/MCP/database readback and account recovery after container replacement.
+Host server 211/211, harness 192/192, types and lint pass; final CI remains required.
+
+The full Trivy 0.76.1/dated-DB inventories show 0 npm findings in all images, 0 OS
+findings in the Alpine variants, and 228 residual Bookworm OS entries after the
+tzdata update. A raw critical MiniZip entry has a specific Debian applicability
+explanation; this does not justify dismissing the rest. Full provenance,
+commands, negative controls and limits are in the
+[runtime packaging record](../../docs/quality/RUNTIME_IMAGE_PACKAGING_2026-10-27.md).
+
+## Cause and corrective rule
+
+A successful monorepo build can hide undeclared imports supplied by another
+workspace. Copying a pruned root node_modules alone can lose nested dependencies,
+native binaries, process identity or data ownership. Install selected production
+consumers from the lock, preserve workspace-local trees, compile native code
+against the runtime libc, and execute the actual compiled consumers. Test the
+real data mount across container replacement; in-memory SQLite is insufficient.
+Use the image's default production configuration for missing-secret tests.
+
+## Limits
+
+Present packages are not necessarily reachable surfaces. Scanner output and
+package counts do not establish exploitability or full-image security clearance.
+Retain raw findings and use primary distro evidence for scoped applicability.
+No deployment, volume migration/deletion, identity change or protection change
+is authorized by an image inventory or this lesson.

.agents/issues/INDEX.md

@@ -2,6 +2,7 @@
 
 | Symptom | Canonical lesson and regression |
 | --- | --- |
+| Pruned runtime starts failing imports, retains dev tools or loses auth across container replacement | [Installed consumer and persistence proof](../inbox/2026-10-27-runtime-images-need-installed-consumer-proof.md); `tests/runtime/run-image.mjs` |
 | A toolchain upgrade changes test counts, loses matcher types or drops unimported coverage | [Preserve discovery and consumer behavior](../inbox/2026-10-27-toolchain-upgrades-must-preserve-test-contracts.md); [migration evidence](../../docs/quality/TOOLCHAIN_MIGRATION_2026-10-27.md) |
 | OAuth links an unverified email, changes accounts after a provider email change, or leaves an orphan user after a link failure | [Provider proof and isolated atomic linking](../inbox/2026-10-27-oauth-linking-needs-provider-proof-and-atomicity.md); `oauth-identity`, `oauth-profiles` and actual `oauth-strategies` tests |
 | Login and reset links cross-use, or a reset/reactivation revives old sessions | [Bind purpose and session revision at the write](../inbox/2026-10-27-email-purpose-and-revision-are-auth-boundaries.md); real SQLite/HTTP/WebSocket `auth-link-boundary.test.ts` |

.agents/issues/security-audit-index.md

@@ -14,3 +14,5 @@
 | A fresh audit reports Zod high severity after an earlier green scan; bounded collections still walk rejected elements | [Zod consumer and bounded-validation lesson](../inbox/2026-10-23-zod-constrained-collections.md); recheck the resolved SDK dependency and use an isolated audit cache without inferring why older metadata differed |
 | Browser login lockout is mistaken for server throttling; one-use email token succeeds concurrently | [Remaining authentication work](../../docs/quality/SECURITY_SCOPE_2026-10-23.md#remaining-authentication-work) |
 | A federation upgrade removes UUID findings but breaks Neo4j schema generation; an override leaves the old lock resolution | [Validate federation consumers and installed UUID paths](../inbox/2026-10-27-federation-consumers-bound-uuid-upgrades.md); `tests/lib/dependencies/uuid.test.mjs` |
+
+Runtime image pruning and actual OS/npm inventories: [2026-10-27 consumer/persistence lesson](../inbox/2026-10-27-runtime-images-need-installed-consumer-proof.md), [image provenance and residuals](../../docs/quality/RUNTIME_IMAGE_PACKAGING_2026-10-27.md).

.github/workflows/ci.yml

@@ -338,11 +338,83 @@ jobs:
             test-artifacts/
           retention-days: 7
 
+  runtime-images:
+    name: Runtime Images (${{ matrix.name }})
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    permissions:
+      contents: read
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - name: api
+            dockerfile: packages/server/Dockerfile
+            target: runner
+            uid: '0'
+          - name: generic
+            dockerfile: deployment/Dockerfile
+            target: runner
+            uid: '1001'
+          - name: self-host
+            dockerfile: deployment/self-host/Dockerfile
+            target: api
+            uid: '1000'
+    services:
+      neo4j:
+        image: neo4j:5.26.12
+        env:
+          NEO4J_AUTH: neo4j/graphdone_runtime_fixture
+          NEO4J_PLUGINS: '["apoc"]'
+          NEO4J_dbms_security_procedures_unrestricted: 'apoc.*'
+        options: >-
+          --health-cmd "cypher-shell -u neo4j -p graphdone_runtime_fixture 'RETURN 1'"
+          --health-interval 10s --health-timeout 5s --health-retries 30
+        ports:
+          - 7687:7687
+    env:
+      GRAPHDONE_DISPOSABLE_NEO4J: '1'
+      NEO4J_URI: bolt://127.0.0.1:7687
+      NEO4J_USER: neo4j
+      NEO4J_PASSWORD: graphdone_runtime_fixture
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: ${{ env.NODE_VERSION }}
+      - name: Build the final runtime image
+        run: |
+          mkdir -p test-artifacts/runtime
+          docker build --pull --target '${{ matrix.target }}' -f '${{ matrix.dockerfile }}' --iidfile test-artifacts/runtime/image-id .
+      - name: Verify installed packages, production startup, API, MCP and volume recovery
+        shell: bash
+        run: node tests/runtime/run-image.mjs "$(cat test-artifacts/runtime/image-id)" '${{ matrix.uid }}' | tee test-artifacts/runtime/contract.jsonl
+      - name: Record final image OS and npm inventory
+        shell: bash
+        run: |
+          docker save "$(cat test-artifacts/runtime/image-id)" -o "$RUNNER_TEMP/runtime-image.tar"
+          mkdir -p "$RUNNER_TEMP/runtime-trivy-cache"
+          docker run --rm --user "$(id -u):$(id -g)" \
+            --mount "type=bind,source=$RUNNER_TEMP/runtime-image.tar,target=/image.tar,readonly" \
+            --mount "type=bind,source=$RUNNER_TEMP/runtime-trivy-cache,target=/cache" \
+            aquasec/trivy@sha256:790fd66d1697395d8693d1acaa1ea702406a20f109b6fc7bac8db623eca6c06c \
+            image --input /image.tar --cache-dir /cache --scanners vuln --list-all-pkgs --format json \
+            > test-artifacts/runtime/trivy.json
+          cp "$RUNNER_TEMP/runtime-trivy-cache/db/metadata.json" test-artifacts/runtime/trivy-db.json
+          node tests/runtime/check-scan.mjs test-artifacts/runtime/trivy.json
+      - name: Retain image provenance and contract results
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: runtime-image-${{ matrix.name }}-${{ github.sha }}
+          path: test-artifacts/runtime/
+          retention-days: 7
+
   # ── Overall status ─────────────────────────────────────────────────────────
   ci-success:
     name: CI Success
     runs-on: ubuntu-latest
-    needs: [quality, unit-tests, mcp-contract, build, security-scan, smoke-gate]
+    needs: [quality, unit-tests, mcp-contract, build, security-scan, smoke-gate, runtime-images]
     if: always()
     steps:
       - name: Verify required jobs passed
@@ -356,7 +428,8 @@ jobs:
              && "${{ needs.mcp-contract.result }}" == "success" \
              && "${{ needs.build.result }}" == "success" \
              && "${{ needs.security-scan.result }}" == "success" \
-             && "${{ needs.smoke-gate.result }}" == "success" ]]; then
+             && "${{ needs.smoke-gate.result }}" == "success" \
+             && "${{ needs.runtime-images.result }}" == "success" ]]; then
             echo "✅ CI green — lint, types, every unit suite, the build, and the user smoke gate all passed."
           else
             echo "❌ CI failed — see the failing job above."

deployment/Dockerfile

@@ -1,50 +1,40 @@
-# Multi-stage build for production
-FROM node:22-alpine AS base
-
-# Install dependencies only when needed
-FROM base AS deps
-RUN apk add --no-cache libc6-compat
+FROM node:22-alpine AS build
+RUN apk add --no-cache python3 make g++
 WORKDIR /app
-
-# Install dependencies based on the preferred package manager
-COPY package.json package-lock.json turbo.json ./
+COPY package.json package-lock.json turbo.json tsconfig.json ./
 COPY packages/core/package.json ./packages/core/
 COPY packages/server/package.json ./packages/server/
-COPY packages/web/package.json ./packages/web/
 COPY packages/mcp-server/package.json ./packages/mcp-server/
-
-RUN npm ci --legacy-peer-deps
-
-# Build the source code
-FROM base AS builder
-WORKDIR /app
-COPY --from=deps /app/node_modules ./node_modules
-COPY --from=deps /app/packages ./packages
-COPY . .
-
-# Build all packages
+COPY packages/web/package.json ./packages/web/
+RUN npm ci --legacy-peer-deps --no-audit --no-fund && npm rebuild sqlite3 --build-from-source
+COPY packages ./packages
 RUN npm run build
+RUN find packages/core/dist packages/server/dist packages/mcp-server/dist -type f \( -name '*.test.*' -o -name '*.spec.*' \) -delete
 
-# Production image, copy all the files and run the app
-FROM base AS runner
+FROM node:22-alpine AS production-deps
 WORKDIR /app
+COPY package.json package-lock.json ./
+COPY packages/core/package.json ./packages/core/
+COPY packages/server/package.json ./packages/server/
+COPY packages/mcp-server/package.json ./packages/mcp-server/
+COPY packages/web/package.json ./packages/web/
+RUN npm ci --legacy-peer-deps --omit=dev --omit=optional --ignore-scripts --no-audit --no-fund --workspace=@graphdone/core --workspace=@graphdone/server --workspace=@graphdone/mcp-server --include-workspace-root=true
+COPY --from=build /app/node_modules/sqlite3/build/Release/node_sqlite3.node ./node_modules/sqlite3/build/Release/node_sqlite3.node
 
+FROM node:22-alpine AS runner
+RUN apk add --no-cache curl && rm -rf /usr/local/lib/node_modules/npm /usr/local/lib/node_modules/corepack /opt/yarn* /usr/local/bin/npm /usr/local/bin/npx /usr/local/bin/corepack /usr/local/bin/yarn /usr/local/bin/yarnpkg /usr/local/bin/pnpm /usr/local/bin/pnpx
+RUN addgroup --system --gid 1001 nodejs && adduser --system --uid 1001 graphdone
+WORKDIR /app
 ENV NODE_ENV=production
-
-# Create non-root user
-RUN addgroup --system --gid 1001 nodejs
-RUN adduser --system --uid 1001 graphdone
-
-# Copy built application
-COPY --from=builder --chown=graphdone:nodejs /app/package.json ./package.json
-COPY --from=builder --chown=graphdone:nodejs /app/packages/core ./packages/core
-COPY --from=builder --chown=graphdone:nodejs /app/packages/server ./packages/server
-COPY --from=builder --chown=graphdone:nodejs /app/packages/web/dist ./web
-COPY --from=deps --chown=graphdone:nodejs /app/node_modules ./node_modules
-
+COPY --from=production-deps /app/package.json ./package.json
+COPY --from=production-deps /app/node_modules ./node_modules
+COPY --from=production-deps /app/packages ./packages
+COPY --from=build /app/packages/core/dist ./packages/core/dist
+COPY --from=build /app/packages/server/dist ./packages/server/dist
+COPY --from=build /app/packages/mcp-server/dist ./packages/mcp-server/dist
+COPY --from=build /app/packages/web/dist ./web
+RUN mkdir -p /app/packages/server/data && chown graphdone:nodejs /app/packages/server/data
 USER graphdone
-
-EXPOSE 4127
-
 WORKDIR /app/packages/server
+EXPOSE 4127
 CMD ["node", "dist/index.js"]

deployment/docker-compose.http.yml

@@ -66,7 +66,7 @@ services:
     volumes:
       - ../packages/server/.env:/app/.env
       - logs:/app/logs
-      - sqlite_auth_data:/app/data
+      - sqlite_auth_data:/app/packages/server/data
       # SSL certificates (uncomment when using HTTPS)
       # - ./certs:/app/certs:ro
     healthcheck:

deployment/docker-compose.yml

@@ -68,7 +68,7 @@ services:
     volumes:
       - ../packages/server/.env:/app/.env
       - logs:/app/logs
-      - sqlite_auth_data:/app/data
+      - sqlite_auth_data:/app/packages/server/data
       - ./certs:/app/certs:ro  # Mount SSL certificates
     healthcheck:
       test: ["CMD", "curl", "-k", "-f", "https://localhost:4128/health"]

deployment/self-host/Dockerfile

@@ -2,17 +2,36 @@ FROM node:22-bookworm-slim AS build
 RUN apt-get update && apt-get install -y --no-install-recommends python3 make g++ ca-certificates && rm -rf /var/lib/apt/lists/*
 WORKDIR /app
 COPY package.json package-lock.json turbo.json tsconfig.json ./
-COPY packages ./packages
+COPY packages/core/package.json ./packages/core/
+COPY packages/server/package.json ./packages/server/
+COPY packages/mcp-server/package.json ./packages/mcp-server/
+COPY packages/web/package.json ./packages/web/
 # Compile the native binding against this image's libc instead of accepting a
 # prebuilt SQLite binary that may require a newer glibc than the runtime image.
 RUN npm ci --legacy-peer-deps --no-audit --no-fund && npm rebuild sqlite3 --build-from-source
-RUN npm run build -w @graphdone/core && npm run build -w @graphdone/server
+COPY packages ./packages
+RUN npm run build -w @graphdone/core && npm run build -w @graphdone/server && npm run build -w @graphdone/mcp-server
 RUN VITE_GRAPHQL_URL=/api/graphql npm run build -w @graphdone/web
+RUN find packages/core/dist packages/server/dist packages/mcp-server/dist -type f \( -name '*.test.*' -o -name '*.spec.*' \) -delete
+FROM node:22-bookworm-slim AS production-deps
+WORKDIR /app
+COPY package.json package-lock.json ./
+COPY packages/core/package.json ./packages/core/
+COPY packages/server/package.json ./packages/server/
+COPY packages/mcp-server/package.json ./packages/mcp-server/
+COPY packages/web/package.json ./packages/web/
+RUN npm ci --legacy-peer-deps --omit=dev --omit=optional --ignore-scripts --no-audit --no-fund --workspace=@graphdone/core --workspace=@graphdone/server --workspace=@graphdone/mcp-server --include-workspace-root=true
+COPY --from=build /app/node_modules/sqlite3/build/Release/node_sqlite3.node ./node_modules/sqlite3/build/Release/node_sqlite3.node
 FROM node:22-bookworm-slim AS api
+ENV NODE_ENV=production
+RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --only-upgrade tzdata && rm -rf /var/lib/apt/lists/* /usr/local/lib/node_modules/npm /usr/local/lib/node_modules/corepack /opt/yarn* /usr/local/bin/npm /usr/local/bin/npx /usr/local/bin/corepack /usr/local/bin/yarn /usr/local/bin/yarnpkg /usr/local/bin/pnpm /usr/local/bin/pnpx
 WORKDIR /app
-COPY --from=build /app/node_modules ./node_modules
-COPY --from=build /app/packages/core ./packages/core
-COPY --from=build /app/packages/server ./packages/server
+COPY --from=production-deps /app/package.json ./package.json
+COPY --from=production-deps /app/node_modules ./node_modules
+COPY --from=production-deps /app/packages ./packages
+COPY --from=build /app/packages/core/dist ./packages/core/dist
+COPY --from=build /app/packages/server/dist ./packages/server/dist
+COPY --from=build /app/packages/mcp-server/dist ./packages/mcp-server/dist
 RUN mkdir -p /app/packages/server/data && chown -R node:node /app/packages/server/data
 WORKDIR /app/packages/server
 USER node