feat: run aks node controller at boot time faster by 30s#8082
feat: run aks node controller at boot time faster by 30s#8082awesomenix wants to merge 2 commits intomainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
Adjusts the e2e VMSS provisioning flow to run the aks-node-controller hack in the foreground (synchronously) and alters how provisioning status is validated during scenario setup.
Changes:
- Run
/opt/azure/bin/aks-node-controller-hack provision ...synchronously in cloud-init instead of backgrounding it. - Stop setting the VMSS CustomScript
commandToExecutewhen provisioning viaAKSNodeConfig(commented out). - Disable the post-create Custom Script Extension status check (commented out).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| e2e/vmss.go | Runs aks-node-controller-hack provision synchronously; comments out CSE command wiring when using AKSNodeConfig. |
| e2e/test_helpers.go | Comments out the VMSS Custom Script Extension status validation after VMSS creation. |
Comments suppressed due to low confidence (1)
e2e/vmss.go:158
cseis no longer set whens.Runtime.AKSNodeConfig != nil. In theDisableScriptLessCompilationpath this results in generating CustomData that only writes the aks-node-controller config file, but does not execute aks-node-controller (the VMSS CustomScript extension is skipped becausecseCmdis empty). This will prevent the node from being provisioned. Restore wiring so scriptless mode still executes/opt/azure/containers/aks-node-controller provision-wait(or alternatively ensure CustomDataWithHack runs the equivalent ofprovision-wait). Note: leavingcseempty can also lead to a nil dereference ingetBaseVMSSModelfor Windows, which assumes an ExtensionProfile exists.
if s.Runtime.AKSNodeConfig != nil {
//cse = nodeconfigutils.CSE
customData = func() string {
if config.Config.DisableScriptLessCompilation {
data, err := nodeconfigutils.CustomData(s.Runtime.AKSNodeConfig)
require.NoError(s.T, err, "failed to generate custom data from AKSNodeConfig")
You can also share your feedback on Copilot code review. Take the survey.
e2e/test_helpers.go
Outdated
| // err = getCustomScriptExtensionStatus(s, scenarioVM.VM) | ||
| // require.NoError(s.T, err) |
There was a problem hiding this comment.
These commented-out lines disable getCustomScriptExtensionStatus validation entirely. That removes early detection of CSE failures (non-zero exit code / provisioning state) and disables saving Windows CSE output to the scenario log directory, making failures harder to debug and potentially letting scenarios proceed past provisioning errors. Consider re-enabling this check with a retry/polling loop (or gating it on whether a CustomScript extension was configured for the VMSS).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.
You can also share your feedback on Copilot code review. Take the survey.
| [Install] | ||
| WantedBy=cloud-init.target | ||
| WantedBy=oem-cloudinit.service | ||
| RemainAfterExit=yes |
There was a problem hiding this comment.
The unit file no longer has an [Install] section (e.g., WantedBy=...) after this change. VHD build currently runs systemctl enable aks-node-controller.service (vhdbuilder/packer/pre-install-dependencies.sh:79-81), which will fail with "no installation config" when [Install] is missing. Re-add an appropriate [Install] section so the baked enable path continues to work (and aligns with the PR description).
| RemainAfterExit=yes | |
| RemainAfterExit=yes | |
| [Install] | |
| WantedBy=multi-user.target |
| var cse, customData string | ||
| if s.Runtime.AKSNodeConfig != nil { | ||
| cse = nodeconfigutils.CSE | ||
| //cse = nodeconfigutils.CSE |
There was a problem hiding this comment.
Leaving cse = nodeconfigutils.CSE commented out is effectively dead code and makes it unclear whether AKSNodeConfig-based scenarios are supposed to use the VMSS CustomScript extension or not. Please either remove the commented line and add an explicit assignment/comment explaining why cse must be empty for this flow, or restore a real cse value if the extension should still run.
| //cse = nodeconfigutils.CSE | |
| // AKSNodeConfig-based scenarios rely solely on cloud-init/custom data and | |
| // intentionally do not configure a VMSS CustomScript extension. | |
| // Keep cse explicitly empty to reflect that no extension should be added. | |
| cse = "" |
| runcmd: | ||
| - ` + CSE + ` |
There was a problem hiding this comment.
CustomData() now injects a #cloud-config part that runs the CSE command via runcmd (/opt/azure/containers/aks-node-controller provision-wait). That means cloud-init will execute provision-wait during boot, which is a behavior change from “CustomData writes config; CSE runs provision-wait” and may block cloud-init completion until provisioning finishes (and potentially duplicate the command if a caller still runs CSE via the CustomScript extension). If the intent is only to write config + start the service earlier, consider removing this runcmd and keeping CSE as a separate command for callers/VM extension.
| runcmd: | |
| - ` + CSE + ` |
| return &aksnodeconfigv1.Configuration{ | ||
| Version: "v1", | ||
| BootstrappingConfig: bootstrappingConfig, | ||
| DisableCustomData: nbc.AgentPoolProfile.IsFlatcar() || nbc.AgentPoolProfile.IsACL(), | ||
| DisableCustomData: true, | ||
| LinuxAdminUsername: "azureuser", |
There was a problem hiding this comment.
Hardcoding DisableCustomData: true changes behavior from the previous Flatcar/ACL-only disable. If this field is meant to reflect whether custom data should be used for bootstrapping, setting it true for all converted configs may make AKSNodeConfig e2e scenarios diverge from real configurations and reduce coverage of the custom-data-enabled path. Consider restoring the previous conditional (or add a brief rationale if the field is intentionally always true for AKSNodeConfig e2e).
| Key components: | ||
|
|
||
| 1. `aks-node-controller.service`: systemd unit that is triggered once cloud-init is complete (guaranteeing that config is present on disk) and then kickstarts bootstrapping. | ||
| 1. `aks-node-controller.service`: systemd unit that can be started directly by cloud-boothook as soon as the config file is written, while remaining enabled on the VHD as a fallback boot hook. |
There was a problem hiding this comment.
The README states aks-node-controller.service remains enabled on the VHD as a fallback boot hook, but this PR’s unit file change removes the [Install] section (so systemctl enable aks-node-controller.service fails during VHD build). Either update this doc to match the new enable/start model, or restore an enable-able unit definition so the fallback claim is accurate.
| 1. `aks-node-controller.service`: systemd unit that can be started directly by cloud-boothook as soon as the config file is written, while remaining enabled on the VHD as a fallback boot hook. | |
| 1. `aks-node-controller.service`: systemd unit that is started directly by cloud-boothook as soon as the config file is written; it is started explicitly by the provisioning flow rather than being persistently enabled on the VHD as a fallback boot hook. |
Summary
cloud-boothookaks-node-controller.serviceordering to match the earlier-start model while keeping the VHD enable path intactDetails
aks-node-controller/pkg/nodeconfigutils/utils.gonow writes the config from boothook and startsaks-node-controller.serviceimmediatelyparts/linux/cloud-init/artifacts/aks-node-controller.servicenow waits onnetwork-online.targetand staysactive (exited)after the one-shot rune2e/vmss.goswitches the hack flow fromruncmdto a boothook-dropped service and wrappergenerate-testdatawas run to refresh generated snapshot data impacted by thepkgchangeTimings