Skip to content

feat(remote-ssh): carry SSH transports over ProxyCommand processes - #3591

Merged
bobleer merged 5 commits into
GCWing:mainfrom
Raing5Days:feat/ssh-proxy-command
Nov 2, 2026
Merged

bobleer merged 5 commits into
GCWing:mainfrom
Raing5Days:feat/ssh-proxy-command

Conversation

@Raing5Days

@Raing5Days Raing5Days commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds OpenSSH-style ProxyCommand support to the remote SSH service (first batch, backend only), so SSH remote workspaces can follow Coder-style platform configs (coder config-ssh) where the SSH transport is a local process's stdin/stdout and the workspace itself runs no sshd.

  • Data model: optional proxyCommand threaded through SSHConnectionConfig, SSHConfigEntry, SavedConnection, and SavedConnectionRecord with #[serde(default)]; older payloads and connections.json files stay loadable, and connection_params_equal treats a changed command as config drift.
  • Transport: the proxy command runs through process_manager::create_tokio_command (cmd.exe /C with raw args on Windows, sh -c elsewhere) after %h/%p/%r/%% expansion; the SSH handshake reuses the existing establish_stream_session, so handshake timeouts, known-hosts verification, and authentication are unchanged.
  • Lifecycle: the child process is owned for the whole session (unpublished-handshake guard, published ActiveConnection, disconnect/reconnect/test paths all kill it). Failures report the proxy exit code plus the last stderr lines instead of a bare handshake error. A reconnect keeps a stale proxy alive only while its old transport still serves in-flight work.
  • ~/.ssh/config: Include is now expanded before parsing (recursive, glob, ~, cycle- and depth-guarded), so hosts that only exist in included files (Coder, AtomGit, devenv) are listed and resolved. In this release ProxyCommand is exclusive with ProxyJump and Docker container targets; combinations are rejected with explicit errors.

No UI yet by design: the connection-dialog field lands in a follow-up batch. The field already flows end to end from ~/.ssh/config resolution, saved configs, and wire payloads.

Type and Areas

Type: Feature

Areas: Rust services (services-integrations remote_ssh / remote_persistence), legacy migration adapters

Motivation / Impact

OpenBitFun could only connect over direct TCP (plus ProxyJump chains), and ssh_config 0.1 neither understands ProxyCommand nor expands Include, so Coder-style hosts were silently treated as normal direct hosts and included Host blocks were invisible. With this change a saved config (or ~/.ssh/config host) carrying ProxyCommand connects through the proxy process, and included Host blocks show up in the host list. Pod rebuilds on such platforms no longer require re-provisioning an sshd.

Verification

  • cargo test -p openbitfun-services-integrations --no-default-features --features remote-ssh-concrete --lib remote_ssh:: — 159 passed / 0 failed (includes new ProxyCommand token/lifecycle/exclusivity tests and Include loader tests)
  • cargo test -p openbitfun-services-integrations --no-default-features --features remote-ssh --test remote_ssh_contracts — 18 passed / 0 failed; same target under --features remote-ssh-concrete — 17 passed / 0 failed
  • cargo test -p openbitfun-services-integrations --no-default-features --features remote-persistence --lib remote_persistence::tests:: — 4 passed / 0 failed
  • cargo clippy -p openbitfun-services-integrations --no-default-features --features remote-ssh-concrete --lib --tests — no warnings in the new code
  • cargo check -p openbitfun-legacy-migration-adapters --lib --tests — passed
  • node scripts/check-core-boundaries.mjs — passed
  • node scripts/format-changed-rust.mjs — clean
  • Not yet exercised against a live Coder workspace (needs the UI batch plus a real deployment); the process lifecycle is covered by real child-process unit tests instead.

CI follow-up

  • Fixed the missing SSHConnectionConfig.proxy_command initializer in the PortForward tool and the desktop local-Docker test fixture. PortForward now preserves ProxyCommand from the resolved SSH config rather than silently falling back to direct TCP.
  • cargo test --locked -p openbitfun-core --no-default-features --features agent-runtime,git,remote-workspace --lib agentic::tools::implementations::port_forward_tool::tests - 4 passed / 0 failed.
  • cargo test --locked -p openbitfun-services-integrations --no-default-features --features remote-ssh --test remote_ssh_contracts - 18 passed / 0 failed.
  • pnpm run fmt:rs and git diff --cached --check - passed. Existing unused/dead-code compiler warnings remain.
  • This follow-up was verified locally on Windows. Full desktop builds, Linux/macOS builds, and live SSH/Coder, Remote Control, Peer Device, and Detached Dispatch scenarios were not exercised locally; hosted CI is pending for the new head.

Reviewer Notes

  • Remote scenarios: this is the Remote Workspace path. The persisted/wire shapes only gain an optional field, so Detached Dispatch and Peer Device hosts on older builds keep reading records written by newer ones and vice versa. The proxy command executes on whichever host establishes the SSH session, matching OpenSSH semantics.
  • Compatibility: the new field is optional on read and omitted on write when absent; no persisted shape is repurposed or narrowed.
  • Deliberate exclusions (explicit errors, not silent behavior): ProxyCommand + ProxyJump, ProxyCommand + container targets, WSL + ProxyCommand.
  • Host keys: the target stays the ~/.ssh/config alias in OpenBitFun's known-hosts store; Coder workspaces that regenerate host keys will surface the app's own host-key verification on reconnect. StrictHostKeyChecking remains unparsed.

Generated with OpenBitFun

Add OpenSSH-style ProxyCommand support to the remote SSH service, first
batch (backend only):

- Thread an optional proxyCommand through SSHConnectionConfig,
  SSHConfigEntry, SavedConnection, and SavedConnectionRecord with serde
  defaults so older payloads and connections.json stay loadable, and
  include it in connection drift detection.
- Spawn the ProxyCommand through the shared process manager (cmd.exe /C
  with raw args on Windows, sh -c elsewhere), expand %h/%p/%r/%% tokens,
  drain stderr into the log plus a bounded tail, and carry the SSH
  handshake over the joined stdout/stdin via the existing
  establish_stream_session path.
- Own the child process for the whole session lifetime: unpublished
  handshakes, published connections, and disconnect/reconnect paths all
  kill it, and a reconnect keeps a stale proxy alive only while its old
  transport still serves in-flight work. Failures report the proxy exit
  code and stderr tail instead of a bare handshake error.
- Merge ~/.ssh/config with its Include tree (recursive, glob, cycle and
  depth guarded) before parsing, making Coder/AtomGit style hosts
  visible; ProxyCommand stays exclusive with ProxyJump and container
  targets, rejected with explicit errors for now.

The connection dialog field for editing proxyCommand lands in a
follow-up batch; ~/.ssh/config hosts already carry it end to end.
@Raing5Days
Raing5Days force-pushed the feat/ssh-proxy-command branch from 8e827bc to a9171a3 Compare November 1, 2026 13:44
@Raing5Days

Copy link
Copy Markdown
Contributor Author

已 rebase 到 2644c2ed3(origin/main,2026-10-31),零冲突,force-with-lease 推送为 a9171a36a。重跑后的 CI 结果:

  • ✅ Rust Build Check(windows / macos / ubuntu)、CLI Tests(windows / ubuntu)、Impact-selected Linux(x64 / arm64)、Frontend Build、DSH Profile Packaging、Build Impact、Repository Object Sizes、Shell Deploy Scripts
  • ❌ CLI Tests (macos-15) → daemon::control::tests::daemon_owner_outlives_endpoint_until_cleanup_finishes
  • ❌ Rust / CLI Validation → 只是 needs: [cli-test] 的聚合门,被上一条连带

macOS 这条与本 PR 无关,三条证据:

  1. 失败断言在 src/apps/cli/src/daemon/control.rs:448:9,由 d1a0eb1ff(fix(cli): hold daemon ownership through runtime cleanup)引入,正好落在本次 rebase 的窗口内(旧 base 没有这个测试);
  2. 本 PR 不触碰 src/apps/cli 下任何文件;
  3. 今早 gcwing/project-resources-workspace-icons 在同一行、同一断言上挂过一次(run 41186323098,2026-10-31T09:03Z),36 分钟后同分支下一轮 run 41187513369 即绿 —— macOS 上租约释放的偶发竞态,不是确定性失败。

外部贡献者没有本仓库 admin,gh run rerun 被拒(Must have admin rights to Repository),无法只重跑该 job,因此用 close → reopen 触发一次新的全量 CI。

@Raing5Days Raing5Days closed this Nov 1, 2026
@Raing5Days Raing5Days reopened this Nov 1, 2026
@GCWing
GCWing requested a review from bobleer November 2, 2026 01:46
- Replace the ssh_config 0.1 crate (pairwise tokenizer, equality-only Host
  matching, last value wins) with a line-based resolver over the
  Include-merged text: Host wildcards and negation, first value wins,
  whole-line ProxyCommand, and ProxyCommand none.
- Restore the parent Host/Match context after each Include, detect cycles
  on the recursion stack only, cap depth at 16, accept Include=, and load
  the config on the blocking pool.
- Supervise ProxyCommand through ProcessTreeChild: sh -c "exec ..." in its
  own process group on Unix, a kill-on-close Job on Windows. Tear down the
  whole tree on drop, failure, and disconnect; on reconnect keep a displaced
  proxy only until its old transport drains instead of leaking it.
- Reject %h/%r substitutions containing shell metacharacters
  (CVE-2023-51385) before anything runs.
- Fail ProxyJump hops whose alias resolves to a ProxyCommand explicitly.
- Advertise ssh_proxy_command_v1 from desktop hosts and document it.
- Send the unpublished-session goodbye before tearing down proxy children
  and drop a duplicated cfg attribute.
@bobleer

bobleer commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator

Pushed 34f8a8b to fix the review findings:

  1. ssh_config parsing (critical). The ssh_config 0.1 crate reads keys and values as alternating tokens, so ProxyCommand /usr/bin/coder ssh --stdio %h became ProxyCommand=/usr/bin/coder and every key after it in the block was misread. It also matches Host patterns by exact string only and keeps the last value. The crate is removed. A line-based resolver (ssh_config_loader::resolve_host_settings) now reads the Include-merged text and handles */?/! patterns, first value wins, a ProxyCommand value that spans the whole line, ProxyCommand none, Key=value, and Match all. get_ssh_config and list_ssh_config_hosts both use it. The host list skips pure-pattern blocks and lists each alias once. Tests cover Coder's real form (--global-config "/x y" ... --ssh-host-prefix coder. %h) and check that keys after it keep their values.
  2. Reconnect leak. std::mem::forget(child) is gone. A displaced proxy is passed to a task holding a Weak of the old handle. The task tears down and reaps the whole process tree once the last user of that handle is gone or the proxy exits.
  3. Kill semantics. ProxyCommandChild is built on services-core::process_tree::ProcessTreeChild. On Unix it runs sh -c "exec <cmd>" in its own process group (process_group(0)) and kills the whole group. On Windows it runs in a Job with JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE. No new dependencies or features. The Unix tests now check that the proxy and a forked grandchild are gone after drop, and after shutdown when the shell exits first.
  4. Shell injection. Before %h/%r are expanded, the host or user must match [A-Za-z0-9._\-@:%\[\]], otherwise the connection fails with an explicit error (same rule as OpenSSH 9.6, CVE-2023-51385). Tests added.
  5. Include. After each Include, the parent's current Host/Match line is written again (Host * at top level). Cycle detection only looks at the current include stack, so re-including a file from another block works. Depth limit is 16. Include = x and Include=x are accepted. File and glob IO runs in spawn_blocking. The module doc now describes the actual behaviour.
  6. ProxyJump hop with ProxyCommand. Connecting now fails with an explicit unsupported error instead of dialing the hop directly. Desktop hosts advertise a new ssh_proxy_command_v1 capability (CLI does not, same as wsl_workspaces_v1). The generated registry and TS are regenerated, and the capability is documented in docs/architecture/remote-workspace-transport.md. There is no UI gating yet.
  7. Removed the duplicated cfg on release_verify. UnpublishedSshSession now moves proxy children into the goodbye task and shuts them down after the goodbye.

Verification (macOS):

  • --lib remote_ssh:: (remote-ssh-concrete): 207 passed
  • remote_ssh_contracts: 18 passed with remote-ssh, 17 with remote-ssh-concrete
  • openbitfun-product-domains remote_surface: passed
  • capabilities:check, check:core-boundaries, fmt:rs: pass
  • Windows cross-check could not run locally: ring has no Windows C sysroot here. The Windows paths reuse the existing ProcessTreeChild Job code.

Not covered by a unit test: the ProxyJump hop rejection, because it reads $HOME/.ssh/config. Only local scenarios were exercised; remote workspace, peer, and dispatch flows were not tested.

- Run Windows proxies through cmd.exe /d /v:off /s /c with one outer
  quote pair so quoted programs and arguments survive.
- Accumulate IdentityFile/CertificateFile and prefer the block naming the
  host for single-value consumers.
- Keep ProxyCommand verbatim; strip inline comments on Host lines (and on
  ProxyCommand on Windows).
- Let ProxyJump and ProxyCommand compete: first obtained wins.
- Report found for any matched setting, including Host * defaults.
- Pass the includer's Host/Match context into nested Includes.
- Warn, and name the ignored ProxyCommand on dial failure, when a profile
  without proxyCommand targets an alias that has one; correct the docs.
@bobleer

bobleer commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator

Pushed the second-review fixes:

  1. Windows now runs cmd.exe /d /v:off /s /c "<command>" (same pattern as external_editor.rs), so Coder's "C:\x\coder.exe" --global-config "C:\...\coderv2" ssh --stdio %h keeps its quotes, and /d keeps AutoRun output out of the stream. Added a test for the argument string and a Windows-only test with a quoted program and two quoted arguments. The Windows test was not run locally.
  2. IdentityFile/CertificateFile now accumulate in order. Single-value consumers take the first value from a block that names the host exactly, else the first value overall.
  3. ProxyCommand is kept verbatim, so "a b" ssh "%h" survives.
  4. ProxyJump and ProxyCommand now compete as in OpenSSH: whichever is obtained first wins (none counts), and the other is dropped. Tests added.
  5. found is true whenever any setting matched, including Host * defaults only.
  6. Nested Includes inherit the includer's Host/Match context instead of a global Host *.
  7. Trailing # comments are stripped on Host lines, and on ProxyCommand only on Windows (POSIX sh already treats them as comments).
  8. The docs now say the desktop dialog does not send proxyCommand yet. When a profile without proxyCommand targets an ~/.ssh/config alias that has one, the host logs a warning, and a failed direct dial names the ignored ProxyCommand. The connection itself is not blocked, so profiles that dialed directly before keep working.

Tests (macOS): --lib remote_ssh:: 213 passed; remote_ssh_contracts 18 passed (remote-ssh) and 17 passed (remote-ssh-concrete); fmt:rs clean.

# Conflicts:
#	src/crates/contracts/product-domains/src/generated/remote-surface-registry.json
#	src/web-ui/src/infrastructure/api/generated/remoteSurface.ts
@bobleer
bobleer merged commit 724bf2f into GCWing:main Nov 2, 2026
14 checks passed
bobleer added a commit to Raing5Days/OpenBitFun that referenced this pull request Nov 2, 2026
Regenerate the remote surface registry artifacts after GCWing#3677 and GCWing#3591.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants