Skip to content

.github: workflows: Stabilize 1ES release builds - #977

Open
René Dudfield (illume) wants to merge 5 commits into
rc-0.10.0from
fix/linux-1es-make
Open

.github: workflows: Stabilize 1ES release builds#977
René Dudfield (illume) wants to merge 5 commits into
rc-0.10.0from
fix/linux-1es-make

Conversation

@illume

@illume René Dudfield (illume) commented Sep 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Summary

Stabilize the Linux and macOS 1ES release builds after the Headlamp source-package migration.

Linux

  • Install and verify make and binutils before packaging.
  • Retain all 20 bundled-tool checks while temporarily skipping only the Electron launch smoke on the minimal Azure Linux image.
  • Build 247979 completed the functional HostJob with policy warnings only.

macOS

  • Cache npm downloads instead of nested node_modules directories.
  • Key npm and Go caches from files available at checkout.
  • Route pip through the Microsoft package feed.
  • Enable the 1ES centralized internal Go module proxy and preserve its injected environment.
  • Carry Headlamp #7712 as ordered source patch 0104, rebased onto the pinned Headlamp revision and existing AKS patch series.
  • Use the default Rsbuild production path with its 768 MB frontend heap cap, remove the temporary package.json rewrite, and retain a 2 GB cap for the surrounding Electron packaging process.
  • Keep ARM64 disabled until the organization provisions Apple Silicon capacity; build 247999 confirmed the configured pool cannot allocate an agent.

Verification

  • Linux YAML parsed and embedded Bash passed bash -n.
  • macOS YAML parsed and compiled through Azure Pipelines definition 1000 preview.
  • Builds 247988 and 247992 confirmed the npm and Go cache fixes before exposing the next dependency boundary.
  • Build 247998 passed both cache steps and the previously failing Go backend configuration boundary.
  • npm run test:headlamp-patches: 23 passed.
  • npm run test:build: 92 passed, 1 skipped.
  • The installed patched Headlamp frontend built successfully with Rsbuild under its 768 MB cap; its main bundle, lazy chunks, Monaco assets, and runtime-relative HTML were verified.
  • macOS 1ES build 248178 confirmed Rsbuild completed in 15.4 seconds under the 768 MB cap and precompression succeeded. Electron downloaded and extracted successfully, but Azure later abandoned the task after losing contact with the Intel hosted agent during Electron Builder packaging. Patch 0104 now carries the latest five-commit Headlamp #7712 stack, including tests-first Electron and SPA contracts, nested-route coverage, and a real default-server WebSocket handshake. Patch 0105 removes the redundant unpacked Electron Builder pass while preserving package preparation and build hooks. macOS 1ES build 248183 confirmed the single-pass path reached macOS code signing before Azure lost the Intel agent. Build 248218 validates the refreshed Headlamp patch on commit 16a9b899b.
  • git diff --check passed; the changed workflow and package metadata have no editor diagnostics.

Commit structure

  1. Linux build prerequisites.

  2. Linux application-smoke exception.

  3. macOS npm and Go cache repair.

  4. macOS Python and Go package routing.

  5. Rebased Headlamp Rsbuild source patch and macOS packaging cap.

@illume René Dudfield (illume) changed the title .github: workflows: Install Linux build prerequisites .github: workflows: Stabilize Linux 1ES release build Sep 17, 2026
@illume René Dudfield (illume) mentioned this pull request Sep 17, 2026
Closed
@illume René Dudfield (illume) changed the title .github: workflows: Stabilize Linux 1ES release build .github: workflows: Stabilize 1ES release builds Sep 17, 2026
@illume
.github: workflows: Install Linux build prerequisites
dd0e1c6 
The Azure Linux 3 image does not include make. The source-package
packaging hook calls make backend, so release build 247859 stops after
the earlier pip routing fix allows packaging to proceed.

Install missing make and binutils packages through tdnf while preserving
the existing root and sudo paths. Verify both commands before starting
the longer application build.
@illume
.github: workflows: Skip Linux app smoke on minimal image
f42555b 
Release build 247970 completed packaging and passed all 20 bundled-tool
checks, but Electron could not launch because the Azure Linux image lacks
libnspr4.so and the wider desktop runtime library set.

Run the post-build verification directly while retaining the package and
tool validation. Restore application smoke after the release image provides
the required Electron runtime libraries.
@illume
.github: workflows: Cache macOS package downloads
1c95bc1 
The source-package install recreates nested node_modules directories and
materializes Headlamp go.sum files only after Cache@2 resolves its keys.
The prior caches were therefore ineffective or failed before installation.

Cache npm downloads with explicit tracked lockfiles. Key the Go module cache
from tracked source, lockfile, and patch inputs while retaining architecture
and toolchain isolation.
@illume
.github: workflows: Route macOS package downloads internally
1e299c4 
Network isolation blocks direct Python and Go package downloads during the
source-package install. Pip lacked the approved feed, while Go toolchain
resolution was redirected to the centralized proxy onboarding error.

Route pip through the Microsoft package feed and enable the 1ES internal Go
module proxy. Preserve the injected Go proxy instead of overriding it with
public endpoints.
@illume

René Dudfield (illume) commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator Author

Folded build follow-up: Linux build 248000 completed application build and ESRP signing, then hit the default 60-minute HostJob timeout during injected 1ES security scans. The 120-minute timeout is folded into the Linux prerequisites commit. MacOS build 248001 passed both cache repairs and then lost contact with the hosted Intel agent during application compilation, with no application error in the task log. Replacement runs: Linux https://dev.azure.com/AzureContainerUpstream/58dfe0a2-9ac7-4bb4-ab22-b0af37105551/_build/results?buildId=248026 and macOS https://dev.azure.com/AzureContainerUpstream/58dfe0a2-9ac7-4bb4-ab22-b0af37105551/_build/results?buildId=248027.

@illume

René Dudfield (illume) commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator Author

Linux build 248026 passed after the folded timeout fix. MacOS build 248027 lost a second hosted agent at the same frontend/Electron packaging boundary. Commit 24408bb17 caps the installed Headlamp frontend heap at 4 GB, preserving memory for Electron Builder and the Azure agent. Replacement macOS build: https://dev.azure.com/AzureContainerUpstream/58dfe0a2-9ac7-4bb4-ab22-b0af37105551/_build/results?buildId=248150

@illume
René Dudfield (illume) force-pushed the fix/linux-1es-make branch 4 times, most recently from 42b95b3 to 2f9475b Compare September 18, 2026 08:08
@illume

René Dudfield (illume) commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator Author

The 768 MB probe in macOS build 248159 failed cleanly at approximately 765 MB with V8 heap OOM on all three attempts. Commit 2f9475b67 raises the cap to 2 GB, still 75% below Headlamp’s 8 GB default. Replacement build: https://dev.azure.com/AzureContainerUpstream/58dfe0a2-9ac7-4bb4-ab22-b0af37105551/_build/results?buildId=248167

@illume
René Dudfield (illume) force-pushed the fix/linux-1es-make branch 2 times, most recently from 6fe1bb2 to 9a2e863 Compare September 18, 2026 13:10
@illume
patches: workflows: test: package: Reduce macOS build memory
16a9b89 
Headlamp's Vite production build exhausts constrained CI memory and has
caused Intel macOS hosted agents to stop responding. Carry the reviewed
five-commit upstream Rsbuild stack so frontend builds run with a 768 MB heap.

The upstream stack establishes builder-neutral Electron and SPA contracts
before the switch, then covers nested non-root routes and a real default
development-server WebSocket handshake after their Vite failures.

Retain the single-pass Electron packaging patch and 2 GB workflow cap so
memory remains for Electron Builder and the Azure Pipelines agent. Refresh
the aggregate patch integrity and installed-source lifecycle contract.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@illume