Skip to content

feat(installer): add Spark express vLLM profile option - #8512

Merged
cv merged 32 commits into
mainfrom
feat/spark-express-vllm-e2e
Aug 10, 2026
Merged

feat(installer): add Spark express vLLM profile option#8512
cv merged 32 commits into
mainfrom
feat/spark-express-vllm-e2e

Conversation

@ericksoa

@ericksoa ericksoa commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Summary

Completes the accepted DGX Spark Express option 2 capability in #8379:

  • Adds the fixed catalog-backed vLLM profile choice and physical qualification target.
  • Publishes authenticated host-local vLLM on exactly 127.0.0.1:8000 and the current private openshell-docker bridge gateway on port 8000.
  • Keeps the sandbox route at https://inference.local while host validation remains on loopback.
  • Pins every authenticated managed-vLLM Docker operation to the physical default daemon.
  • Revalidates exact sandbox and container identities before live-target cleanup.

Type of Change

  • Code change with doc updates

Quality Gates

  • Tests added or updated for changed behavior
  • Docs updated for user-facing behavior changes
  • Sensitive paths changed (onboarding, inference, Docker publication, and E2E cleanup)
  • Sensitive-path review completed; private-publication and Docker-daemon findings are resolved

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: docs-updated
  • Evidence:
    • docs/get-started/prerequisites.mdx
    • docs/inference/choose-local-inference-server.mdx
    • docs/inference/set-up-vllm-on-two-dgx-sparks.mdx
    • docs/inference/set-up-vllm.mdx
    • docs/reference/platform-support.mdx
    • docs/resources/prompt-assets/dgx-spark.md
    • docs/resources/starter-prompt.md
    • test/e2e/README.md
  • Agent: Codex Desktop

Verification

  • Normal pre-commit, commit-msg, and pre-push hooks
  • Changed-file lane: 311 files and 3,950 tests passed
  • Focused daemon and private-bridge lane: 102 tests passed
  • Follow-up lifecycle lane: 8 tests passed
  • Non-default agent configuration synchronization: 25 tests passed
  • CLI type-check and Biome
  • Repository architecture checks: 1,647 files, 4,919 edges, 0 cycles
  • Starter-prompt, agent-variant, route, link, and shell-command documentation checks
  • Fern build: 0 errors and 2 hidden-page warnings
  • Documentation follows the writing guide and controlled word list
  • Final focused inference diagnostics: 99 tests passed
  • Final CLI type-check and build identity: 64cf09fd3
  • Physical spark-express-vllm target: OpenShell 0.0.99 passed in 5m58s; final source tree with OpenShell 0.0.101 passed in 5m46s
  • Test-conditional growth guardrail after the final test-only refactor
  • Current-base CI repair: llama.cpp qualification/shared-runtime tests, 27 passed; stale-recovery tests, 5 passed

DGX Spark evidence

  • Host: DGX Spark with NVIDIA GB10 and 124,610 MiB RAM.
  • Final hardware-tested source commit: 28cbe0e85088a22b4031298f39b812fbc0627b9d; final rebased PR head 7f1d958562bbc2915704d551f97b1132991c62f2 has the identical Git tree 544ea15a168b5091c9b21437fb7036d4f600dfa8.
  • OpenShell 0.0.101 final-tree target: 1/1 passed in 5m46s after vLLM readiness in 4m30s.
  • Managed vLLM published exactly 127.0.0.1:8000 and 172.18.0.1:8000; no wildcard or additional binding was present.
  • Authenticated sandbox chat, unrelated-egress denial, and identity-guarded cleanup passed.
  • The earlier OpenShell 0.0.99 target and manual source-checkout onboarding also passed; four dashboard probes and two clean recovery cycles returned HTTP 200.
  • Full evidence and limitations: feat(installer): add Spark express vLLM profile option #8512 (comment)

Host-specific SSH finding

This machine has a pre-existing custom wildcard Host * SSH stanza that applies Compression yes, ControlMaster auto, a socket ControlPath, and ControlPersist 600 to the OpenShell sandbox host. These are machine-specific settings, not DGX Spark defaults. With them active, both OpenShell 0.0.99 and 0.0.101 failed the primary dashboard forward. A temporary first-match Host sandbox override set compression, multiplexing, and persistence off; the final OpenShell 0.0.101 target then passed onboarding, runtime checks, inference, denial, and cleanup. The original SSH file was restored byte-for-byte.

One earlier failed diagnostic relay attempt left stale OpenShell PID state until the exact record was moved aside. Track that behavior separately with OpenShell. It does not establish a DGX Spark requirement or block this PR's source-checkout qualification.

Signed-off-by: Aaron Erickson aerickson@nvidia.com
Signed-off-by: Carlos Villela cvillela@nvidia.com

Summary by CodeRabbit

  • New Features

    • DGX Spark Express now offers managed vLLM with automatic profile selection or a fixed Qwen3.6 35B-A3B NVFP4 profile.
    • Fixed profiles include guided validation, protected settings, and offline model-cache support.
  • Bug Fixes

    • Improved host-local vLLM connectivity, recovery, authentication, and Docker networking.
    • Restricted published inference ports to validated loopback and private bridge addresses.
  • Documentation

    • Updated setup, prerequisites, platform support, and troubleshooting guidance for Spark inference profiles.
    • Added DGX Spark Express qualification and safety-test documentation.

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
@ericksoa ericksoa self-assigned this Aug 7, 2026
@copy-pr-bot

copy-pr-bot Bot commented Aug 7, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 0c0484e1-a826-4415-8135-44d43d429931

📥 Commits

Reviewing files that changed from the base of the PR and between ea39cc4 and 7f1d958.

📒 Files selected for processing (45)
  • ci/platform-matrix.json
  • docs/get-started/prerequisites.mdx
  • docs/inference/choose-local-inference-server.mdx
  • docs/inference/set-up-vllm-on-two-dgx-sparks.mdx
  • docs/inference/set-up-vllm.mdx
  • docs/reference/platform-support.mdx
  • docs/resources/prompt-assets/dgx-spark.md
  • docs/resources/starter-prompt.md
  • scripts/checks/run-llama-cpp-dgx-spark-qualification.mts
  • scripts/install.sh
  • src/lib/agent/onboard.test.ts
  • src/lib/agent/onboard.ts
  • src/lib/inference/local-vllm-auth.test.ts
  • src/lib/inference/local.test.ts
  • src/lib/inference/local.ts
  • src/lib/inference/serving/host-local-vllm-selection.test.ts
  • src/lib/inference/serving/host-local-vllm-selection.ts
  • src/lib/inference/serving/vllm-host-local-lifecycle.test.ts
  • src/lib/inference/serving/vllm-host-local-lifecycle.ts
  • src/lib/inference/serving/vllm-host-local-network.test.ts
  • src/lib/inference/serving/vllm-host-local-network.ts
  • src/lib/inference/serving/vllm-managed-support.ts
  • src/lib/inference/vllm.test.ts
  • src/lib/inference/vllm.ts
  • src/lib/onboard.ts
  • src/lib/onboard/command.test.ts
  • src/lib/onboard/command.ts
  • src/lib/onboard/config-sync.test.ts
  • src/lib/onboard/config-sync.ts
  • src/lib/onboard/inference-providers/types.ts
  • src/lib/onboard/openclaw-setup.test.ts
  • src/lib/onboard/openclaw-setup.ts
  • src/lib/onboard/setup-inference.ts
  • src/lib/onboard/setup-nim-vllm.test.ts
  • src/lib/onboard/setup-nim-vllm.ts
  • test/e2e/README.md
  • test/e2e/live/inference-routing-helpers.ts
  • test/e2e/live/spark-express-vllm.test.ts
  • test/e2e/mock-parity.json
  • test/e2e/support/spark-express-vllm-safety.test.ts
  • test/e2e/support/spark-express-vllm-safety.ts
  • test/helpers/installer-express-prompt-pty-harness.ts
  • test/install-express-prompt.test.ts
  • test/llama-cpp-dgx-spark-qualification-runner.test.ts
  • test/starter-prompt-docs.test.ts

📝 Walkthrough

Walkthrough

DGX Spark Express now supports automatic managed-vLLM selection and a fixed catalog-backed vLLM profile. Managed host-local vLLM uses validated bridge networking and separate inference and validation endpoints. Onboarding, sandbox sync, documentation, and physical E2E qualification were updated.

Changes

DGX Spark vLLM onboarding and qualification

Layer / File(s) Summary
Express profile selection and onboarding
scripts/install.sh, src/lib/onboard/command.ts, docs/..., test/install-express-prompt.test.ts
Express installation now selects managed or fixed vLLM profiles. Fixed-profile overrides are rejected, and profile provenance is preserved during resume.
Managed vLLM bridge networking and validation
src/lib/inference/..., docs/inference/..., src/lib/onboard/setup-nim-vllm.ts
Managed vLLM resolves a private Docker bridge gateway, publishes loopback and bridge bindings, uses a dedicated Docker environment, and separates inference from validation URLs.
Noninteractive sandbox configuration sync
src/lib/onboard/config-sync.ts, src/lib/agent/onboard.ts, src/lib/onboard/openclaw-setup.ts
Sandbox configuration scripts use shared noninteractive arguments with --no-tty and bash -s.
Physical Spark qualification and safety validation
test/e2e/live/spark-express-vllm.test.ts, test/e2e/support/spark-express-vllm-safety.ts, scripts/checks/run-llama-cpp-dgx-spark-qualification.mts
Physical qualification validates local Docker use, exact resource identity, catalog-derived container configuration, inference, egress denial, diagnostics, and cleanup.

Estimated code review effort: 5 (Critical) | ~90 minutes

Possibly related issues

Possibly related PRs

Suggested labels: feature, area: inference, provider: vllm, platform: dgx-spark, area: onboarding

Suggested reviewers: apurvvkumaria, cv, jyaunches

✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/spark-express-vllm-e2e

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

@github-code-quality

github-code-quality Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 7f1d958 in the feat/spark-express-v... branch remains at 96%, unchanged from commit ea39cc4 in the main branch.


Updated August 10, 2026 02:08 UTC

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Completed · low confidence · 0 blockers · 0 warnings · 0 suggestions
  • Model comparison: normalized findings match; normalized terminology decisions differ; normalized E2E selections differ; severity counts match.
5 terminology differences from the second opinion

Advisory only. These are normalized differences from the primary terminology receipt.

  • fixed catalog-backed vLLM profile at docs/resources/prompt-assets/dgx-spark.md:14: primary classified it as justified; the second opinion classified it as define.
  • local-model profile at src/lib/onboard/command.ts:39: selected only by the second-opinion lane as established.
  • host-local vLLM at src/lib/inference/serving/vllm-host-local-network.ts:15: selected only by the second-opinion lane as established.
  • bounded host publication at docs/inference/choose-local-inference-server.mdx:133: selected only by the second-opinion lane as define.
  • exact private IPv4 gateway at docs/inference/choose-local-inference-server.mdx:116: selected only by the second-opinion lane as define.
1 additional E2E selection from the second opinion

Advisory only. The primary lane did not select these E2E jobs or targets.

  • spark-install: The completed second-opinion lane identified E2E coverage that the primary lane omitted.

Second-opinion terminology and E2E selections are advisory. Live E2E does not run automatically for pull requests.

3 semantic terminology decisions

Terminology decisions are advisory. They affect the assessment only when a separate finding identifies concrete semantic impact.

  • justified — automatic serving-profile selection at docs/resources/prompt-assets/dgx-spark.md:13: Keep this term. The option text and documentation state the option-1 and option-2 distinction.
  • justified — fixed catalog-backed vLLM profile at docs/resources/prompt-assets/dgx-spark.md:14: Keep this term. The modifiers identify catalog ownership and fixed behavior that affect configuration.
  • define — private bridge gateway at docs/inference/choose-local-inference-server.mdx:145: Keep the term with the surrounding definition of one private IPv4 Gateway value and exact binding requirement.

E2E guidance

Advisory only. A maintainer can dispatch the default E2E suite against this exact revision.

Recommended E2E: cloud-inference, cloud-onboard, managed-image-multiarch-startup, managed-image-protected-runtime, security-posture, full-e2e, hermes-e2e, inference-routing, network-policy, onboard-repair, onboard-resume

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

@cv cv left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Product scope and security review are incomplete. This draft adds a second DGX Spark Express option, fixed vLLM profile, installer behavior, public documentation, and a physical qualification target without a linked accepted issue or design decision. Record the decision that defines ownership, lifecycle, compatibility, security, and hardware validation for this supported surface, and complete the required sensitive-path review and exact DGX Spark evidence before marking the PR ready. Refresh onto current main and rerun all required checks afterward.

@copy-pr-bot

copy-pr-bot Bot commented Aug 8, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@cv

cv commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

DGX Spark hardware evidence — final source tree

Tested on a physical DGX Spark with NVIDIA GB10 and 124,610 MiB RAM. The final hardware run used source commit 28cbe0e85088a22b4031298f39b812fbc0627b9d. The rebased PR head 7f1d958562bbc2915704d551f97b1132991c62f2 has the identical Git tree 544ea15a168b5091c9b21437fb7036d4f600dfa8.

OpenShell 0.0.101 passing scope

  • The protected spark-express-vllm target passed 1/1 in 5m46s.
  • Express option 2 selected local-model-profile.vllm.spark.v1 and recipe vllm.qwen3-6-35b-a3b-nvfp4.spark-single.v1.
  • vLLM became ready after 4m30s.
  • The runtime used model revision 491c2f1ea524c639598bf8fa787a93fed5a6fbce and image nvcr.io/nvidia/vllm@sha256:9204569b17ee4c0eff75194b8e6e458479c8aee18953b5ab9cf359fcdac659e2.
  • Managed vLLM published only 127.0.0.1:8000 and 172.18.0.1:8000.
  • Authenticated sandbox chat passed. Unrelated HTTPS CONNECT was denied with curl exit 56 and proxy HTTP 403.
  • Cleanup rechecked the full sandbox and managed-container identities before removal. No test sandbox, vLLM container, dashboard listener, inference listener, or OpenShell forward remained.
  • The original SSH configuration was restored byte-for-byte. The test-preparation gateway service was returned to its prior stopped state. OpenShell 0.0.101 remains installed.

The earlier OpenShell 0.0.99 target passed 1/1 in 5m58s. A separate source-checkout onboarding on that version passed all eight phases, returned exactly SPARK_FINAL_OK, produced four dashboard HTTP 200 responses, and passed two clean recovery cycles.

Host-specific SSH finding

This machine has a pre-existing custom wildcard Host * SSH stanza. For the OpenShell sandbox host, its effective settings are Compression yes, ControlMaster auto, a socket ControlPath, and ControlPersist 600. These are machine-specific settings, not DGX Spark defaults.

With those settings active, the OpenShell 0.0.101 run completed vLLM readiness and sandbox creation, but the primary dashboard forward was refused. Onboarding exited 1, the target stopped before inference assertions, and its exact-identity cleanup passed.

A temporary first-match Host sandbox override set Compression no, ControlMaster no, and ControlPersist no. The otherwise identical OpenShell 0.0.101 target then passed onboarding, exact runtime checks, inference, denial, and cleanup. This shows that 0.0.101 alone does not remove the host-specific interference; disabling connection sharing and compression for the OpenShell sandbox host circumvents it.

One earlier failed diagnostic relay attempt left a stale OpenShell PID record until that exact record was moved aside. Clean stops removed their records. Track that stale-record behavior separately with OpenShell; it does not establish a DGX Spark requirement or block this PR.

cv and others added 3 commits August 9, 2026 18:14
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
@cv
cv marked this pull request as ready for review August 10, 2026 02:16
@cv
cv merged commit 05d607c into main Aug 10, 2026
60 of 67 checks passed
@cv
cv deleted the feat/spark-express-vllm-e2e branch August 10, 2026 02:21
@github-actions github-actions Bot added the v0.0.106 Release target label Aug 10, 2026
@cv cv mentioned this pull request Aug 10, 2026
23 tasks
cv added a commit that referenced this pull request Aug 10, 2026
<!-- markdownlint-disable MD041 -->
## Summary

Make CLI tests independent of scheduler timing and host OpenShell
installation state. Increase CLI coverage from 8 to 12 shards; the
measured run model lowers the slowest shard from 586.3 seconds to 344.3
seconds while issue #8669 tracks the underlying slow tests.

## Related Issue

Relates #8669.
Follow-up to #8512.

## Changes

- Inject a deterministic monotonic clock into the stale
deadline-generation lifecycle-lock test.
- Keep the stale-rebuild fixture on supported OpenShell 0.0.101 binaries
and directly verify that rebuild does not invoke the installer.
- Give the legacy Hermes compatibility fixture setup a 30-second hook
timeout so an overloaded coverage shard does not fail Vitest’s default
10-second timeout.
- Run PR and main CLI coverage with 12 shards, merge 12 reports, and
report failures from every shard.
- Exercise shard 12 as valid input, reject shard 13, and verify
failed-shard links for shard 12.

## Type of Change

- [x] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates

- [x] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [ ] Tests not applicable — justification:
- [ ] Docs updated for user-facing behavior changes
- [x] Docs not applicable — justification: This changes internal CI
parallelism and deterministic test fixtures only; no supported product
surface changes.
- [ ] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Documentation Writer Review

- [x] Documentation writer subagent reviewed the completed changes
- Result: `no-docs-needed`
- Evidence:
- The workflow changes only increase internal CLI coverage parallelism
and update failed-shard validation.
- The test changes make existing CI and lifecycle fixtures
deterministic, including direct verification that rebuild does not
invoke the OpenShell installer.
- The legacy Hermes compatibility change only extends the heavy
test-fixture setup timeout; it does not change production behavior.
- No production command, configuration, user workflow, output contract,
or supported product surface changes.
- Agent: Codex Desktop
<!-- docs-review-head-sha: 01dc6e2 -->
<!-- docs-review-agents-blob-sha: c4923a3 -->

## DGX Station Hardware Evidence

- [ ] Tested on DGX Station
- Tested commit: Not applicable
- Station profile/scenario: Not applicable;
`scripts/prepare-dgx-station-host.sh` is unchanged.
- Result: Not applicable
- Supporting evidence: Not applicable

## Verification

- [x] PR description includes a `Signed-off-by:` line and every commit
appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run validate:pr` passed after refreshing `origin/main` when hooks
were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — the two formerly flaky files passed
five runs, 225 test executions; the updated installer probe passed five
additional runs; the workflow contract passed 5/5; the source-shape
budget passed. The legacy Hermes compatibility suite passed 18/18 tests
in each of five normal runs and 18/18 once with V8 coverage; that
coverage command exited only on unrelated whole-repository coverage
thresholds.
- [ ] Applicable broad gate passed — `npm run check` passed every
pre-commit check. Its CLI coverage lane completed 23,741 passing tests
and found 27 unrelated DGX Spark host-state failures: an installed user
service, a group-writable Codex workspace parent, and ARM64 Python
executed in an amd64 container. An isolated HOME cleared all 24
service-related failures.
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only)
- [ ] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

---
Signed-off-by: Carlos Villela <cvillela@nvidia.com>

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Tests**
  * Expanded CLI coverage testing from 8 to 12 parallel shards.
  * Updated coverage validation and reporting to include all 12 shards.
* Improved test reliability with deterministic timing and extended setup
timeouts.
* Verified rebuilds avoid unnecessary installation attempts and related
messaging.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

v0.0.106 Release target

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants