feat: add a ssh proxy that can connect to running sessions by eikek · Pull Request #4586 · SwissDataScienceCenter/renku

eikek · 2026-09-04T10:08:07Z

Adds support for SSH in sessions that use images built from code.
Currently not supported on apps, jobs or on remote clusters or sessions on CSCS/slurm.

/deploy extra-values=sshProxy.enabled=true,dataService.imageBuilders.enabled=true,dataService.imageBuilders.strategyName=renku-buildpacks-v3,dataService.imageBuilders.outputImagePrefix=harbor.dev.renku.ch/renku-build/,dataService.imageBuilders.nodeSelector.renku.io/node-purpose=user,dataService.imageBuilders.tolerations[0].effect=NoSchedule,dataService.imageBuilders.tolerations[0].key=renku.io/dedicated,dataService.imageBuilders.tolerations[0].operator=Equal,dataService.imageBuilders.tolerations[0].value=user,sshProxy.domain=ssh.ci-renku-4586.dev.renku.ch

RenkuBot · 2026-09-04T10:09:02Z

You can access the deployment of this PR at https://ci-renku-4586.dev.renku.ch

SalimKayal

Ok a few things on the helm side.

SalimKayal

oups, approved instead of requesting changes, sorry.

sgaist

Just some suggestion around labeling this new backend that should allow for improved handling in the near future.

SalimKayal · 2026-10-08T10:17:59Z

+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+  name: allow-external-to-ssh-proxy
+  labels:
+    app: {{ template "renku.name" . }}
+    chart: {{ template "renku.chart" . }}
+    release: {{ .Release.Name }}
+    heritage: {{ .Release.Service }}
+spec:
+  podSelector:
+    matchLabels:
+      app: renku-ssh-proxy
+      release: {{ .Release.Name }}
+  policyTypes:
+    - Ingress
+  ingress:
+    - from:
+        - ipBlock:
+            cidr: 0.0.0.0/0
+---


Do we still need this with the external-dns annotation ?

I think your comment is on the wrong file. I think you are talking about the annotation of the service, right?
This part of the service, right?

"external-dns.kubernetes.io/hostname": {{ include "renku.sshDomain" . | quote }}

That is how the load balancer type service gets assigned a hostname for ingress in azure I believe. But that annotation is general and not azure specific. It just depends on the networking/ingress controller of your cluster. In some cases the controller may do nothing with it. I opened a PR to clean some of this up. I changed a few things around this - take a look there. #4680

What I did in my cleanup is changed the values file so that you can disable the load balancer service. Then you have to setup the ingress outside of the helm chart. Also I added the option to specify additional annotations.

…, default to port 22 (#4675) ## Summary Cleanup of the SSH proxy: the proxy-to-session target is now fixed rather than configurable, the hop-2 keygen is gated behind `sshProxy.enabled`, the proxy runs two hardened replicas, and the public entry defaults to port **22** so `ssh <session>@ssh.<host>` works with no `-p`. Stacked on #4586. ## Motivation and context - The hop-2 target is fixed by the amalthea contract. The session Service port is `80` and the login user is `renku`. Exposing `sshProxy.targetPort`/`targetUser` in values invites an operator to change a value that silently breaks every session; the proxy's own fallback was also stale at `2222`. - The `platform-init` hook generated the two hop-2 Secrets unconditionally, even when the proxy is disabled. - A single replica with no PodDisruptionBudget is not HA for an internet-facing entry point. - The entry defaulted to `2222`, so the copy-paste command needed a non-standard port. data-services already omits `:22` from the `ssh://` URL, so defaulting to 22 makes the plain command correct. ## Behavior - `ssh <session>@ssh.<host>` connects on the default port 22; `sshProxy.service.port` remains operator-overridable. - With `sshProxy.enabled=false`, the hop-2 key Secrets are no longer generated. - Two proxy replicas are spread across nodes and a PDB keeps one available during node drains. - The external ingress NetworkPolicy admits only the proxy's pod port (2222) instead of all ports. ## Changes - `renku-ssh-proxy/src/config.rs`: hop-2 default target port `2222 → 80`. - `helm-chart/renku/templates/ssh-proxy/deployment.yaml`: drop the `RENKU_SSH_PROXY_TARGET_PORT`/`_TARGET_USER` env; default to preferred hostname pod anti-affinity (overridable via `sshProxy.affinity`). - `helm-chart/renku/templates/ssh-proxy/pdb.yaml` (new): `policy/v1` PDB, `maxUnavailable: 50%`, created only when enabled and `replicaCount > 1`. - `helm-chart/renku/values.yaml`: remove `targetPort`/`targetUser`; `replicaCount: 1 → 2`; `service.port: 2222 → 22`. - `helm-chart/renku/templates/network-policies.yaml`: pin `allow-external-to-ssh-proxy` to pod port `2222`. - `helm-chart/renku/templates/setup-job-platform-init.yaml` + `scripts/platform-init/platform-init.py`: pass `SSH_PROXY_ENABLED` into the hook and guard `ensure_ssh_proxy_session_keys` on it. ## Notes - The NetworkPolicy pins the **pod** port `2222`, which assumes the CNI evaluates ingress policy post-DNAT (Calico/Cilium do). Verify on the first cluster that enables the proxy; a pre-DNAT CNI would need the Service port as well.

olevski · 2026-10-13T16:05:26Z

We should see if we can change the ssh command shown in the ui. ssh://... does not work in all places. For example I was testing with positron - like a reskinned vscode that R users like. And the ssh:// prefix does not work there.

* Deleted old references to apps * resolved broken links and how to configure frontends * Run Prettier * resolve links * resolve broken link * resolve first link * resolve broken links

* chore: add renku cli docs * address comments

- allow to connect to a fixed target host - include nix testing vm - authenticate with fixed public key - Add dockerfile and workflow for build/publish - Add building ssh-proxy image to chartpress

* Enable local port-forwarding * Do not allow forwarding outside the session host * Don't break the session if port-forwarding fails * Don't drop connection when data-service connection fails

* chore(helm): drop unused internal-authn mount from ssh-proxy * fix(helm): generate ssh-proxy host key when unset * feat(ssh-proxy-session): add ssh-proxy-session-keys generator image * feat(ssh-proxy-session): generate static proxy-to-session keys via hook job * refactor(ssh-proxy-session): generate proxy-to-session keys in platform-init

* feat(ssh-proxy-session): add ssh-proxy-session-keys generator image * feat(ssh-proxy-session): generate static proxy-to-session keys via hook job * refactor(ssh-proxy-session): generate proxy-to-session keys in platform-init * feat(ssh-proxy-session): load session host/auth keys from config * feat(ssh-proxy-session): pin session host key and use publickey auth * feat(ssh-proxy-session): mount session host/auth keys in ssh-proxy * chore(ssh-proxy-session): devshell and dev VM support for session keys * feat(ssh-proxy-session): pass session key secret names to data-services * refactor(ssh-proxy): remove b64 decode fallback from load_private_key_from_file * squashme: Update helm-chart/renku/templates/data-service/session-env-secret.yaml --------- Co-authored-by: Tasko Olevski <16360283+olevski@users.noreply.github.com>

* feat(ssh-proxy-session): add ssh-proxy-session-keys generator image * feat(ssh-proxy-session): generate static proxy-to-session keys via hook job * refactor(ssh-proxy-session): generate proxy-to-session keys in platform-init * feat(ssh-proxy-session): load session host/auth keys from config * refactor(ssh-proxy): remove b64 decode fallback from load_private_key_from_file * feat(ssh-proxy): set explicit SSH auth and keepalive limits * feat(ssh-proxy): harden proxy pod security context * feat(ssh-proxy): restrict proxy egress to required peers * fix(ssh-proxy): disable unused ServiceAccount token projection * squashme: Apply batched suggestions from code review Co-authored-by: Samuel Gaist <samuel.gaist@idiap.ch> * Update helm-chart/renku/values.yaml --------- Co-authored-by: Tasko Olevski <16360283+olevski@users.noreply.github.com> Co-authored-by: Samuel Gaist <samuel.gaist@idiap.ch>

* feat(ssh-proxy-session): add ssh-proxy-session-keys generator image * feat(ssh-proxy-session): generate static proxy-to-session keys via hook job * refactor(ssh-proxy-session): generate proxy-to-session keys in platform-init * feat(ssh-proxy-session): load session host/auth keys from config * refactor(ssh-proxy): remove b64 decode fallback from load_private_key_from_file * ci(ssh-proxy): audit Cargo.lock for known vulnerabilities * ci(ssh-proxy): track cargo dependencies with dependabot * docs(ssh-proxy): document the dependency audit cadence

- also pass some information into the data service

…, default to port 22 (#4675) ## Summary Cleanup of the SSH proxy: the proxy-to-session target is now fixed rather than configurable, the hop-2 keygen is gated behind `sshProxy.enabled`, the proxy runs two hardened replicas, and the public entry defaults to port **22** so `ssh <session>@ssh.<host>` works with no `-p`. Stacked on #4586. ## Motivation and context - The hop-2 target is fixed by the amalthea contract. The session Service port is `80` and the login user is `renku`. Exposing `sshProxy.targetPort`/`targetUser` in values invites an operator to change a value that silently breaks every session; the proxy's own fallback was also stale at `2222`. - The `platform-init` hook generated the two hop-2 Secrets unconditionally, even when the proxy is disabled. - A single replica with no PodDisruptionBudget is not HA for an internet-facing entry point. - The entry defaulted to `2222`, so the copy-paste command needed a non-standard port. data-services already omits `:22` from the `ssh://` URL, so defaulting to 22 makes the plain command correct. ## Behavior - `ssh <session>@ssh.<host>` connects on the default port 22; `sshProxy.service.port` remains operator-overridable. - With `sshProxy.enabled=false`, the hop-2 key Secrets are no longer generated. - Two proxy replicas are spread across nodes and a PDB keeps one available during node drains. - The external ingress NetworkPolicy admits only the proxy's pod port (2222) instead of all ports. ## Changes - `renku-ssh-proxy/src/config.rs`: hop-2 default target port `2222 → 80`. - `helm-chart/renku/templates/ssh-proxy/deployment.yaml`: drop the `RENKU_SSH_PROXY_TARGET_PORT`/`_TARGET_USER` env; default to preferred hostname pod anti-affinity (overridable via `sshProxy.affinity`). - `helm-chart/renku/templates/ssh-proxy/pdb.yaml` (new): `policy/v1` PDB, `maxUnavailable: 50%`, created only when enabled and `replicaCount > 1`. - `helm-chart/renku/values.yaml`: remove `targetPort`/`targetUser`; `replicaCount: 1 → 2`; `service.port: 2222 → 22`. - `helm-chart/renku/templates/network-policies.yaml`: pin `allow-external-to-ssh-proxy` to pod port `2222`. - `helm-chart/renku/templates/setup-job-platform-init.yaml` + `scripts/platform-init/platform-init.py`: pass `SSH_PROXY_ENABLED` into the hook and guard `ensure_ssh_proxy_session_keys` on it. ## Notes - The NetworkPolicy pins the **pod** port `2222`, which assumes the CNI evaluates ingress policy post-DNAT (Calico/Cilium do). Verify on the first cluster that enables the proxy; a pre-DNAT CNI would need the Service port as well.

olevski

I tested in this deployment. Things look good to me. There is some small followup issues and work. But we can get those in a separate PR into release-2.22.0. This PR is already large enough.

leafty

Would it be possible to have the UI display the server fingerprint or include it in the ssh command? SSH being Trust-on-first-use is bad for security and it would be nice to avoid people trusting SSH servers just by habit.

leafty

Would it be possible to have the UI display the server fingerprint or include it in the ssh command? SSH being Trust-on-first-use is bad for security and it would be nice to avoid people trusting SSH servers just by habit.

lorenzo-cavazzi

Lgtm! 🚀

P.S. I agree with the suggestion from @leafty

eikek force-pushed the build/renku-ssh-proxy branch from 73e8c3c to 64a3457 Compare September 9, 2026 07:28

SalimKayal approved these changes Sep 10, 2026

View reviewed changes

SalimKayal requested changes Sep 10, 2026

View reviewed changes

eikek force-pushed the build/renku-ssh-proxy branch from e52784e to c701fef Compare September 10, 2026 15:59

eikek requested a review from SalimKayal September 11, 2026 12:32

eikek force-pushed the build/renku-ssh-proxy branch from 18a26c4 to 4c04672 Compare September 11, 2026 12:40

Panaetius reviewed Sep 17, 2026

View reviewed changes

sgaist requested changes Sep 21, 2026

View reviewed changes

Comment thread helm-chart/renku/templates/ssh-proxy/deployment.yaml

Comment thread helm-chart/renku/templates/ssh-proxy/deployment.yaml

Comment thread helm-chart/renku/templates/network-policies.yaml Outdated

olevski requested changes Sep 24, 2026

View reviewed changes

Comment thread helm-chart/renku/values.yaml

Comment thread helm-chart/renku/values.yaml Outdated

Comment thread helm-chart/renku/values.yaml Outdated

Comment thread helm-chart/renku/templates/ssh-proxy/secret.yaml Outdated

Comment thread renku-ssh-proxy/src/config.rs Outdated

SalimKayal force-pushed the build/renku-ssh-proxy branch from bc5afee to cf0d8e4 Compare September 28, 2026 12:24

SalimKayal added this pull request to stack #4645 September 29, 2026 16:50

olevski removed this pull request from stack #4645 October 2, 2026 07:07

eikek force-pushed the build/renku-ssh-proxy branch from d314861 to f307963 Compare October 2, 2026 13:44

eikek mentioned this pull request Oct 2, 2026

ssh-proxy: provide the internal user to connect to a session instance with the authorization response #4652

Open

eikek force-pushed the build/renku-ssh-proxy branch from f307963 to b0dad9e Compare October 2, 2026 14:00

SalimKayal mentioned this pull request Oct 7, 2026

refactor(ssh-proxy): fix the proxy-to-session target, gate the keygen, default to port 22 #4675

Merged

SalimKayal reviewed Oct 8, 2026

View reviewed changes

olevski requested review from SalimKayal, olevski and sgaist October 12, 2026 20:44

olevski changed the base branch from master to release-2.22.0 October 12, 2026 20:45

olevski mentioned this pull request Oct 12, 2026

chore: address rust comments for ssh proxy #4688

Merged

bethcg and others added 5 commits October 14, 2026 15:58

docs: clean outdated app references (#4635)

25bbf23

* Deleted old references to apps * resolved broken links and how to configure frontends * Run Prettier * resolve links * resolve broken link * resolve first link * resolve broken links

chore: add renku cli docs (#4630)

1b03fc1

* chore: add renku cli docs * address comments

Initial implementation of renku-ssh-proxy

7cbafc7

- allow to connect to a fixed target host - include nix testing vm - authenticate with fixed public key - Add dockerfile and workflow for build/publish - Add building ssh-proxy image to chartpress

Authorize ssh access using data services (#4613)

45bbe96

* Enable local port-forwarding * Do not allow forwarding outside the session host * Don't break the session if port-forwarding fails * Don't drop connection when data-service connection fails

SalimKayal and others added 17 commits October 14, 2026 15:58

refactor(ssh-proxy): use secret service for authz (#4643)

f1e20e4

Add ssh subdomain annotation

c27bc69

Fix whitespace in ssh-proxy deployment.yaml

d23378c

Fix release label in ssh-proxy pod

5f224b8

Add feature flag for ssh-proxy deployment

8e5f8b8

- also pass some information into the data service

Move ssh session keys into default data-service environment

db82d6d

Update build pack images to 0.11.0

d7426d7

chore: expose SSH_ENABLED variable the the UI pod (#4673)

89173ef

feat(helm-chart): add label for networkpolicy selector

3c01721

chore: bump renku-ui to 4.34.0 (#4677)

7084aec

chore: address rust comments (#4688)

b744c09

chore: helm chart cleanup (#4680)

6ab2029

feat: update data services to 0.88.0

0b5abda

olevski force-pushed the build/renku-ssh-proxy branch from 07b9a35 to 0b5abda Compare October 14, 2026 13:59

olevski requested a review from Panaetius October 14, 2026 13:59

chore: update rust dependencies

7bc4db8

olevski approved these changes Oct 14, 2026

View reviewed changes

olevski marked this pull request as ready for review October 14, 2026 15:19

olevski requested review from a team as code owners October 14, 2026 15:19

leafty reviewed Oct 15, 2026

View reviewed changes

lorenzo-cavazzi approved these changes Oct 16, 2026

View reviewed changes

Conversation

eikek commented Sep 4, 2026 • edited by olevski Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RenkuBot commented Sep 4, 2026

Uh oh!

SalimKayal left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SalimKayal left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sgaist left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SalimKayal Oct 8, 2026

Choose a reason for hiding this comment

Uh oh!

olevski Oct 12, 2026

Choose a reason for hiding this comment

Uh oh!

olevski Oct 12, 2026

Choose a reason for hiding this comment

Uh oh!

olevski commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

olevski left a comment

Choose a reason for hiding this comment

Uh oh!

leafty left a comment

Choose a reason for hiding this comment

Uh oh!

leafty left a comment

Choose a reason for hiding this comment

Uh oh!

lorenzo-cavazzi left a comment

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

eikek commented Sep 4, 2026 •

edited by olevski

Loading

olevski commented Oct 13, 2026 •

edited

Loading