Run the Manager locally (make dev) via a NodeApi abstraction + DummyApi

[runleveldev]

Rebased onto current main (which now includes #347, #351, and #352). This PR is a single commit with no upstream-PR duplication.

Why

Supersedes #329. That PR made the Manager runnable on localhost but did so in ways flagged in review: a mock that bypassed the job-runner, in-band signaling (apiUrl === 'local'), a SQLite-specific migration, and dev-login provisioning that clobbered the Docker Compose bootstrap.

This reworks the same goal — run the Manager standalone and create containers without Proxmox — but addresses the root causes and keeps the real provisioning code path intact. It also lays the groundwork for additional production-capable node providers behind a single NodeApi abstraction (starting with the dummy provider).

What changed

1. NodeApi abstraction (ProxmoxApi + new DummyApi)

• Node.api() now dispatches on a new nodeType column:
  • proxmox (default) → authenticated ProxmoxApi (unchanged behavior).
  • dummy → new DummyApi (utils/dummy-api.js) implementing the same method surface, simulating the Proxmox calls.
• No HTTP-handler short-circuit. Container creation still flows POST /containers → Job → job-runner.js → bin/create-container.js → node.api(). The full creation path is exercised; only the hypervisor calls are faked. Running the job-runner is required (and make dev does so).
• DummyApi.clusterResources('lxc') derives the running set from the DB so the live container-status resolver (Container status API: compute live status instead of static DB column #352) reports dummy containers as running; the dev bootstrap gives the dummy node placeholder credentials so the resolver routes through node.api().

2. Provider-agnostic-ish node selection

Node selection only excludes nodes that can't provision (it picks a dummy node, or a node with full apiUrl/tokenId/secret), so a half-configured node is never selected and then failed in node.api(). The DELETE path likewise routes VM teardown through node.api(). Node.api() validates apiUrl alongside the credentials.

3. nodeType column (Postgres-safe)

New migration 20260605000000-add-node-type-to-nodes adds nodeType (STRING(50), default 'proxmox'). Existing rows backfill to proxmox via the column default; Node.bulkCreate (node import) also applies it, so production node selection is unchanged.

4. Root-cause fix for the erroneous UNIQUE on Containers.nodeId (via patch-package)

A v6 backport of sequelize/sequelize#17583 (patches/sequelize+6.37.8.patch) so SQLite's changeColumn/describeTable no longer marks members of a composite index as unique. Wired via patch-package (postinstall). Replaces the SQLite-specific migration.

5. Local dev environment out of /auth/dev

New bin/dev-bootstrap.js seeds a localhost site, a dummy node, and a localhost external domain. It no-ops if any Site already exists, so it never collides with the Docker Compose bootstrap. The /auth/dev login route is left clean. The groups + push-notification seeders are made idempotent so re-running them under make dev doesn't violate unique constraints.

6. SQL query logging gated by LOG_LEVEL=trace

Sequelize query logging (noisy — the job-runner polls the Jobs table every tick) is gated behind LOG_LEVEL=trace, in both dev and production; off otherwise. Pass it through: make dev LOG_LEVEL=trace.

7. @vscode/sqlite3 instead of the unmaintained sqlite3

The dev SQLite driver is the actively-maintained @vscode/sqlite3 fork, wired in via Sequelize v6 dialectModule. It builds from source (no prebuilt-binary glibc mismatches that broke make dev on LTS distros) and is a devDependency required lazily only when the sqlite dialect is selected — so production (Postgres) never installs or loads it (npm ci --omit=dev skips the native build entirely).

8. make dev

Lives in create-a-container/Makefile (the root Makefile delegates). Self-contained: installs server + client deps with npm install --no-package-lock (so it never rewrites the lockfile), runs migrations, seeds the localhost site + dummy node, builds the client, then runs server (nodemon) + job-runner + client watch together. No .env is needed — the server's defaults (SQLite at data/database.sqlite, an auto-generated session secret, dev login when not in production) are sufficient. db:migrate is fixed to call sequelize-cli.

9. Docs

New "Run the Manager Locally (make dev)" section in the development-workflow doc; README + release-pipeline pointers updated; LOG_LEVEL documented in example.env.

How this addresses the #329 review comments

Review comment	Resolution
Mocks are a bad idea; prefer a NodeApi interface + DummyApi, with the job-runner still in the loop	DummyApi implements the ProxmoxApi surface; creation runs through the job-runner and bin/create-container.js unchanged
In-band signaling (apiUrl === 'local'); want a dedicated field	Dedicated nodeType column drives Node.api()
Dev-login defaults break the docker-compose bootstrap	Provisioning moved out of /auth/dev into bin/dev-bootstrap.js, which no-ops if any Site exists
Don't merge SQLite-specific migrations (Postgres is the only supported DB)	No SQLite migration; root cause fixed via patch-package backport of sequelize#17583
Redundant .gitignore entry	make dev uses the default data/database.sqlite (already ignored by create-a-container/data/.gitignore)

Also addressed the Copilot review on this PR (provisionable-node filter + clearer error, Node.api() apiUrl validation, conditional LOG_LEVEL forwarding in the Makefile, larger non-reused DummyApi VMID space, real locally-administered MAC prefix, patch-package moved so --omit=dev installs don't fail).

Testing

• Migrations run clean on fresh SQLite (incl. Container status API: compute live status instead of static DB column #352's remove-container-status); Containers has no column-level UNIQUE on nodeId.
• End-to-end: bin/create-container.js against DummyApi lands the container, and the live status resolver reports it running (and creating/failed for the other states); DELETE routes through node.api() and removes the row.
• npm ci (dev) and npm ci --omit=dev (production, no @vscode/sqlite3, no native build) both succeed.
• make dev runs server + job-runner + client watch and does not modify the lockfile.

Closes #329

[Copilot]

Pull request overview

This PR makes the Manager runnable standalone on localhost via make dev, adding a provider abstraction (Node.api() dispatching by nodeType) and a DummyApi so container creation can exercise the real job-runner path without needing Proxmox.

Changes:

• Add nodeType with Node.api() dispatch (proxmox vs dummy) and implement DummyApi to simulate hypervisor calls.
• Introduce make dev + bin/dev-bootstrap.js to bootstrap a local SQLite dev environment and run server + job-runner together.
• Add a patch-package backport patch for Sequelize SQLite composite-index uniqueness behavior and gate Sequelize SQL logging behind LOG_LEVEL=trace.

Reviewed changes

Copilot reviewed 14 out of 16 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
README.md	Updates docs pointer to highlight make dev workflow.
mie-opensource-landing/docs/developers/development-workflow.md	Adds a dedicated make dev section and reorganizes Docker-stack guidance.
Makefile	Adds dev target to install, migrate, bootstrap, build, and run server + job-runner.
create-a-container/utils/dummy-api.js	Introduces DummyApi implementing the NodeApi/ProxmoxApi method surface for local runs.
create-a-container/routers/api/v1/containers.js	Makes node selection provider-agnostic and routes delete teardown through node.api().
create-a-container/patches/sequelize+6.37.8.patch	Adds a patch-package patch for Sequelize v6 SQLite query-interface uniqueness handling.
create-a-container/package.json	Adds postinstall patch-package hook and updates db migration script to sequelize-cli.
create-a-container/package-lock.json	Locks patch-package and its transitive dependencies.
create-a-container/models/node.js	Adds nodeType field and Node.api() dispatch to DummyApi vs ProxmoxApi.
create-a-container/migrations/20260605000000-add-node-type-to-nodes.js	Adds nodeType column with default proxmox.
create-a-container/example.env	Documents LOG_LEVEL behavior for SQL query logging.
create-a-container/config/config.js	Gates Sequelize SQL logging behind LOG_LEVEL=trace (including production).
create-a-container/bin/dev-bootstrap.js	Seeds an empty DB with a localhost site, dummy node, and localhost external domain.

Files not reviewed (1)

• create-a-container/package-lock.json: Generated file

[cmyers-mieweb]

LGTM