Listener deadlock when minRunners > 0 and config changes are applied

[RiccardoAtzori91]

Description

When minRunners > 0 is configured on an AutoScalingRunnerSet and a config change is applied (via Helm upgrade, ArgoCD sync, etc.), the controller enters a deadlock state where the new AutoscalingListener cannot be created.

Steps to Reproduce

1. Configure a runner scale set with minRunners: 1 (or higher)
2. Wait for idle runners to be created
3. Apply any config change that triggers listener recreation (e.g., change minRunners from 1 to 3, modify env vars, change volumes, etc.)
4. Observe the controller logs

Expected Behavior

The controller should gracefully drain idle runners and recreate the listener with the new configuration, or apply the config change without requiring a full drain.

Actual Behavior

The controller logs show:

"Creating a new AutoscalingListener is waiting for the running and pending runners to finish. Waiting for the running and pending runners to finish:", "running":1, "pending":0

The controller:

1. Deletes the old AutoscalingListener
2. Waits for all runners to reach 0 before creating the new listener
3. But minRunners > 0 keeps idle runners alive indefinitely
4. Deadlock: new listener never gets created, runners remain idle forever

Workaround

Manually delete the EphemeralRunnerSet:

kubectl delete ephemeralrunnersets -n <namespace> --all

This unblocks the controller, which then creates a new ERS and listener with the correct configuration.

Environment

• ARC version: 0.14.2 (latest as of 2026-07-11)
• Controller image: ghcr.io/actions/gha-runner-scale-set-controller:0.14.2
• Kubernetes: 1.30+
• Deployment method: Helm via ArgoCD

Additional Context

This is a fundamental design issue: the controller requires all runners to be drained before applying config changes, but minRunners prevents the drain from completing. The only safe pattern currently is to always keep minRunners: 0, which defeats the purpose of having warm runners for faster job startup.

A potential fix could be:

• Allow the controller to force-terminate idle runners during config transitions
• Or apply config changes in-place without requiring a full listener recreation
• Or add a timeout after which idle runners are terminated to unblock the transition

[github-actions]

Hello! Thank you for filing an issue.

The maintainers will triage your issue shortly.

In the meantime, please take a look at the troubleshooting guide for bug reports.

If this is a feature request, please review our contribution guidelines.