Scw runner lifecycle improvements - #97
Open
Puneetha-Ramachandra wants to merge 19 commits into
Open
Conversation
- Added ansible.cfg, site.yml, group_vars/all.yml, and inventory/hosts.yml. - Added kernelspace role for compiling RISC-V kernel modules, depmod, and sysctl configs. - Added userspace role for containerd, kubelet, node_exporter, prometheus-agent, and watchdogs. - Configured userspace role to download RISC-V kubernetes binaries from GitLab. - Verified both roles against riscv-runner-1 with 100% success.
- Added setup_runner_ansible() in scw.py to invoke ansible-playbook. - Added --ansible flag to scw runner setup and scw runner reinstall. - Added --kernelspace-only and --userspace-only mutually exclusive group to scw runner setup. - Fixed scw_client.default_region and added import subprocess in scw.py. - Fixed containerd config generation and gcc-14 alternatives in Ansible roles.
- Added SHA256 checksum verification for node_exporter and prometheus downloads. - Deployed containerd config via Jinja2 template atomically instead of sed. - Switched scw.py setup and reinstall wholesale to Ansible (removed --ansible flag). - Unified scw runner setup to reboot and wait for k8s node in all setup paths. - Added explanatory comment for scw_client.default_region.
- Moved top-level ansible/ folder to runner/ansible/ as requested in PR review. - Updated scw.py to invoke runner/ansible/site.yml.
…llback, and pass Cockpit & probe extra-vars - Removed ~1,000 lines of legacy SETUP_SCRIPT multiline bash string and setup_runner() function from scw.py. - Updated cmd_runner_create to call setup_runner_ansible(). - Passed cockpit_metrics_push_url, cockpit_metrics_token, and github_probe_token as extra-vars in setup_runner_ansible(). - Added fallback from riscv64-linux-gnu-readelf to readelf for native RISC-V targets in kernelspace role.
…ndant inventory/hosts.yml - Deleted ~860 lines of legacy SETUP_SCRIPT multiline bash string from scw.py. - Deleted redundant runner/ansible/inventory/hosts.yml inventory file and removed inventory entry from ansible.cfg. - Scaleway API (via scw.py) serves as the dynamic source of truth for runner IPs.
…ce, and maintainability) - Security: Pass extra-vars to ansible via secure temporary JSON file (0600 perms) to prevent secret exposure in /proc/cmdline. - Security: Support SCW_SSH_KEY_PATH environment variable fallback for custom SSH keys. - Lifecycle: Fix --kernelspace-only timeout by using wait_ssh_up instead of wait_k8s_node. - Performance: Add cache_valid_time: 3600 to kernelspace apt update task. - Performance: Combine kernel module build and install into a single make invocation. - Performance: Increase SSH ControlPersist to 300s in ansible.cfg. - Performance: Set gather_subset: min in site.yml for faster fact gathering. - Maintainability: Add userspace role meta/main.yml declaring dependency on kernelspace. - Maintainability: Add runner/ansible/README.md documentation.
…error cleanup, delete resilience, and cockpit token cleanup)
- Fix cmd_runner_reboot: Only cordon node, do not delete node from k8s during server reboot so kubelet reconnects automatically.
- Fix cordon_k8s_node: Add 300s max drain timeout to prevent infinite loops on stuck terminating pods.
- Fix cmd_runner_create: Wrap server creation in try...except block to delete partially provisioned server on failure.
- Fix cmd_runner_delete: Wrap Control Plane connection in try...except block so offline CP does not block baremetal server deletion.
- Fix cmd_runner_delete: Delete Cockpit metrics push token ({runner}-metrics-token) on runner deletion.
…obe token tasks for 1:1 parity - Add crictl (v1.35.0) installation task to /usr/local/bin. - Add CNI plugins (v1.4.0) installation task to /opt/cni/bin. - Add containerd-watchdog script, service, and timer templates and handler. - Add /etc/raw-github-probe/token token file task when github_probe_token is provided. - Achieve 100% functional equivalence with legacy bash runner setup scripts.
…checksums via get_url - Use ansible.builtin.get_url with checksum validation for downloading node_exporter and prometheus archives to /tmp. - Extract archives locally via unarchive module. - Ensures strict SHA256 artifact verification.
…er_kubeadm to prevent idle connection reset - Re-establish fresh SSH connection to Control Plane host in setup_runner_kubeadm. - Prevents ConnectionResetError (Errno 104) on long-running OS reinstall / Ansible runs where the initial SSH connection drops.
…erver deletion on setup failure, and cockpit token deletion) - Revert 300s drain timeout in cordon_k8s_node to avoid killing active user workflows. - Revert server.delete on setup failure in _do_runner_create to preserve baremetal instances for recovery. - Revert explicit Cockpit token deletion in cmd_runner_delete as token creation auto-recreates existing tokens.
…ifecycle changes - Keep PR strictly focused on Ansible setup migration. - Remove --kernelspace-only / --userspace-only CLI flags. - Revert cmd_runner_reboot, cmd_runner_delete, and SSH key fallback to main baseline.
- Add --kernelspace-only and --userspace-only CLI arguments to scw runner setup for targeted Ansible role execution. - Update cmd_runner_reboot to only cordon the node on reboot so kubelet reconnects automatically. - Add resilience to cmd_runner_delete with try-except fallback when Control Plane connection fails. - Add SCW_SSH_KEY_PATH environment variable fallback.
Puneetha-Ramachandra
changed the base branch from
main
to
ansible-runner-setup
August 26, 2026 05:35
…y support - Update cmd_runner_reboot to only cordon the node on reboot so kubelet reconnects automatically. - Add resilience to cmd_runner_delete with try-except fallback when Control Plane connection fails. - Add SCW_SSH_KEY_PATH environment variable fallback.
Merged
# Conflicts: # scripts/scw.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.