Skip to content

ci: change runner to solx-linux-amd64-self-hosted#375

Merged
hedgar2017 merged 8 commits into
mainfrom
rename-self-hosted-runner-label
May 27, 2026
Merged

ci: change runner to solx-linux-amd64-self-hosted#375
hedgar2017 merged 8 commits into
mainfrom
rename-self-hosted-runner-label

Conversation

@nebasuke
Copy link
Copy Markdown
Member

@nebasuke nebasuke commented Apr 27, 2026

  • Switch the runner from solx-linux-amd64 (per-run) to solx-linux-amd64-self-hosted (persistent) in the three workflows that target it: integration-tests.yaml, coverage.yaml, sanitizer.yaml.
  • In integration-tests.yaml, share the solc build between the PR checkout and the temp-solx-main checkout via symlink when their solc submodule SHAs match — same pattern as the existing LLVM-sharing step.
  • Refactor the LLVM share step to use the same output+skip pattern (steps.share-llvm.outputs.shared) as the new solc step instead of building inline, so the two share/build pairs are symmetric.
  • Clear stale dirs/symlinks (rm -rf before ln -snf) in both share steps so a previous run's main-build output on the persistent self-hosted workspace can't be reused.

Runs

Integration tests:

@nebasuke nebasuke changed the title ci: rename self-hosted runner label to solx-linux-amd64-self-hosted ci: Change self-hosted runner to solx-linux-amd64-self-hosted Apr 27, 2026
@nebasuke nebasuke added ci:coverage Trigger coverage workflow on PR ci:sanitizer Trigger sanitizer workflow on PR ci:integration Trigger integration tests workflow on PR question Further information is requested and removed ci:coverage Trigger coverage workflow on PR ci:sanitizer Trigger sanitizer workflow on PR ci:integration Trigger integration tests workflow on PR labels Apr 27, 2026
@nebasuke nebasuke force-pushed the rename-self-hosted-runner-label branch from 047336d to 38015a7 Compare April 28, 2026 10:22
@nebasuke nebasuke added ci:coverage Trigger coverage workflow on PR ci:sanitizer Trigger sanitizer workflow on PR ci:integration Trigger integration tests workflow on PR and removed ci:sanitizer Trigger sanitizer workflow on PR ci:coverage Trigger coverage workflow on PR ci:integration Trigger integration tests workflow on PR labels Apr 28, 2026
@nebasuke nebasuke changed the title ci: Change self-hosted runner to solx-linux-amd64-self-hosted ci: Change runner to solx-linux-amd64-self-hosted Apr 28, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 28, 2026

📊 solx Tester Report

➡️ Download

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 28, 2026

📊 Hardhat Projects Report

➡️ Download

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 28, 2026

📊 Foundry Projects Report

➡️ Download

@nebasuke nebasuke added ci:coverage Trigger coverage workflow on PR and removed question Further information is requested ci:sanitizer Trigger sanitizer workflow on PR ci:integration Trigger integration tests workflow on PR labels Apr 28, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 28, 2026

Coverage Summary

Crate Line Coverage Function Coverage
solx 🟢 94.7% 🟡 68.0%
solx-benchmark-converter 🔴 0.0% 🔴 0.0%
solx-codegen-evm 🟢 88.2% 🟢 80.4%
solx-compiler-downloader 🔴 0.0% 🔴 0.0%
solx-core 🟢 96.0% 🟢 88.0%
solx-dev 🔴 2.5% 🔴 3.4%
solx-evm-assembly 🟡 73.8% 🟡 64.8%
solx-solc-test-adapter 🔴 1.7% 🔴 2.1%
solx-standard-json 🟢 95.1% 🟢 89.4%
solx-tester 🔴 37.8% 🔴 35.7%
solx-utils 🟢 83.0% 🟢 80.6%
solx-yul 🟡 78.9% 🟡 71.3%
Total 🟡 56.2% 🔴 48.7%

Codecov Report | HTML Report | Workflow Run

@nebasuke nebasuke added ci:sanitizer Trigger sanitizer workflow on PR ci:integration Trigger integration tests workflow on PR labels Apr 28, 2026
@nebasuke nebasuke force-pushed the rename-self-hosted-runner-label branch from 4b3dfaf to 194d2f6 Compare May 6, 2026 21:52
@nebasuke nebasuke marked this pull request as ready for review May 6, 2026 21:53
@nebasuke nebasuke requested a review from a team May 6, 2026 22:56
@hedgar2017 hedgar2017 changed the title ci: Change runner to solx-linux-amd64-self-hosted ci: change runner to solx-linux-amd64-self-hosted May 7, 2026
@hedgar2017 hedgar2017 requested review from Copilot May 7, 2026 13:44
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates CI workflows that run on the solx Linux AMD64 runner to use a persistent self-hosted runner, and adds/refactors build-artifact sharing between the PR checkout and a secondary temp-solx-main checkout to reduce redundant builds while avoiding stale reuse.

Changes:

  • Switch runs-on from solx-linux-amd64 to solx-linux-amd64-self-hosted in the sanitizer, coverage, and integration workflows.
  • In integration-tests.yaml, refactor LLVM sharing to an output-driven “share then conditional build” pattern, and add an equivalent solc sharing step via symlinks when submodule SHAs match.
  • Add cleanup (rm -rf before ln -snf) in sharing steps to prevent stale paths/symlinks from being reused on a persistent workspace.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
.github/workflows/sanitizer.yaml Switch sanitizer workflow to the persistent self-hosted runner label.
.github/workflows/coverage.yaml Switch coverage workflow to the persistent self-hosted runner label.
.github/workflows/integration-tests.yaml Switch to self-hosted runner; refactor LLVM sharing/build flow; add solc sharing/build flow with stale-path cleanup.

Comment thread .github/workflows/integration-tests.yaml
Copy link
Copy Markdown
Contributor

@hedgar2017 hedgar2017 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nebasuke — stepping back from the per-line comments for a moment, the three problems flagged here (the two stale-symlink fallback paths, plus the build-script-hash dimension that share-solc misses) all come from the same place: the workflow is reimplementing a SHA-keyed mirror in bash, in parallel to the cache-key logic the build-llvm and build-solc composite actions already own.

Worth considering folding the mirror in as a mirror-from input on those actions instead. The workflow then collapses to two ordinary uses: calls per artifact:

- name: Build LLVM (PR)
  uses: ./.github/actions/build-llvm
  with: { build-type: Release, enable-assertions: 'false' }

- name: Build LLVM (main)
  uses: ./.github/actions/build-llvm
  continue-on-error: true
  with:
    working-dir: temp-solx-main
    mirror-from: '.'
    build-type: Release
    enable-assertions: 'false'

- name: Build solc (PR)
  uses: ./.github/actions/build-solc

- name: Build solc (main)
  uses: ./.github/actions/build-solc
  continue-on-error: true
  with:
    working-dir: temp-solx-main/solx-solidity
    mirror-from: solx-solidity

Inside the action, a single new step gated on inputs.mirror-from != '', falling through to the existing cache-restore + build chain when it can't mirror:

- name: Mirror artifact from sibling checkout
  id: mirror
  if: inputs.mirror-from != ''
  shell: bash
  run: |
    SRC="${{ inputs.mirror-from }}/target-llvm/target-final"
    DST="${{ inputs.working-dir }}/target-llvm/target-final"
    SRC_KEY=$(<compute the same cache key, with mirror-from's SHAs>)
    [ "$SRC_KEY" = "${{ steps.llvm-cache.outputs.key }}" ] && [ -d "$SRC" ] || {
      echo "mirrored=false" >> "$GITHUB_OUTPUT"; exit 0
    }
    rm -rf "$DST"
    mkdir -p "$(dirname "$DST")"
    cp -al "$SRC" "$DST"
    echo "mirrored=true" >> "$GITHUB_OUTPUT"

Existing build steps' if: extends to ... && steps.mirror.outputs.mirrored != 'true'.

Two things this gets that the surgical fixes won't:

  • Cleanup is unconditional and adjacent to the writerm -rf "$DST" sits one line above cp -al, so it can't drift out of the SHA-match branch the way the current rm -rf did. Submodule-internal stale state and Copilot's broader "stale temp-solx-main" concern collapse into the same fix.
  • The mirror predicate inherits the action's full cache-key dimensions for free — boost version, MLIR flag, build-script hash, sanitizer — so the share decision can't drift from the cache decision as keys grow.

Also cp -al instead of ln -snf. Hardlinks under cmake/cargo's rename-on-write conventions don't have the through-write surface symlinks do, and the consumers here are read-only.

Bigger than a surgical fix on this PR — happy to take it as a follow-up if you'd rather close #375 with the rm -rf cleanup and the script-hash check in the share predicate (per the inline comments).

Comment thread .github/workflows/integration-tests.yaml
Comment thread .github/workflows/integration-tests.yaml
@nebasuke nebasuke force-pushed the rename-self-hosted-runner-label branch 2 times, most recently from b975657 to 0be720f Compare May 15, 2026 14:51
@nebasuke
Copy link
Copy Markdown
Member Author

Thanks — going with the surgical path on this PR. Filed the mirror-from refactor as #448 to track separately.

What landed here (path B):

  • aa3b16d wipes temp-solx-main wholesale before the optional Checkout main. That's your "delete temp-solx-main before the optional checkout" variant, and it closes both stale-symlink concerns (LLVM and solc submodule-internal) plus Copilot's broader stale-checkout flag in one step.
  • 484673c + da1d918 add the build-script-hash dimension to both share predicates, mirroring the cache-key composition in build-{llvm,solc}/action.yml.
  • 0be720f extracts the find | sort -z | xargs -0 cat | sha256sum | cut pipeline into a named hash_tree shell function — it's computed twice per step (PR vs main) and the inline one-liner obscured what was being compared.

Why not path A on this PR: the architectural win (mirror predicate inheriting the action's cache-key dimensions for free, hardlinks instead of symlinks, cleanup adjacent to the write) is real and worth doing, but it rewrites the composite actions — bigger blast radius than the three defects this PR is fixing. Easier to review the surgical patch against the inline comments and ship that, then refactor cleanly under #448.

@nebasuke nebasuke force-pushed the rename-self-hosted-runner-label branch from 0be720f to a1b1ec1 Compare May 20, 2026 13:16
@nebasuke nebasuke requested a review from hedgar2017 May 20, 2026 13:16
nebasuke added 8 commits May 27, 2026 11:21
Single-job probe on solx-linux-amd64-self-hosted with no container and
no checkout, so we can read user/id, workspace path ownership, and
mount info. Triggered only when this file is touched.

Temporary; remove once the runner is reachable.
Mirror the existing 'Share LLVM with main checkout' pattern for solc.
When the solx-solidity SHA on the PR matches main (the typical case for
PRs that don't bump the submodule), symlink solx-solidity/build and
solx-solidity/boost from the PR checkout into temp-solx-main and skip
the redundant Build solc (main) step.

Observed cost saving: ~2 minutes per integration-tests run, where
Build solc (main) was hitting the artifact cache differently than
Build solc (PR) and falling through to a ccache-assisted rebuild
even with identical cache keys.
Remove the diagnostic probe workflow now that the self-hosted runner is
validated, refactor the LLVM share step to the same output+skip pattern
used by the new solc share step, and clear stale dirs/symlinks before
relinking so a persistent self-hosted workspace can't reuse a previous
run's main-build output.
actions/checkout's `git clean -ffdx` doesn't recurse into submodules, so
on the persistent self-hosted workspace a prior run can leave stale
solx-solidity/{build,boost} dirs or symlinks (pointing at the PR's
freshly-built artifacts) inside temp-solx-main. Subsequent SHA-mismatch
runs would extract through those symlinks via `actions/cache/restore`,
silently corrupting the PR's build output and skewing benchmarks.

Wholesale `rm -rf temp-solx-main` before the optional checkout closes
this and the symmetric stale-checkout concern in one stroke.
build-solc/action.yml keys its artifact cache on submodule SHA *plus*
sha256 of `solx-dev/src/solc`. The share predicate matched only on SHA,
so a PR that edits `solx-dev/src/solc/*` without bumping the
solx-solidity submodule pin would silently share its own solc binary as
the "main" baseline, skewing the benchmark with no signal.

Compute the same script hash on both sides and require both checks to
pass before symlinking. Falling back to a real `Build solc (main)` is
strictly better than a silent share when inputs diverge.
Mirror the share-solc fix: `solx-dev llvm build` is driven by
`solx-dev/src/llvm/*`, so a PR that edits those scripts without bumping
the solx-llvm submodule pin would silently share its own LLVM build as
the "main" baseline. Compute and compare the script hash on both sides
before symlinking, falling back to a real LLVM build for main when they
diverge.
Pull the dense `find | sort -z | xargs -0 cat | sha256sum | cut` pipeline
into a named shell function. The hash is computed twice per step (PR vs
main) and the original one-liner obscured what was being compared.
@hedgar2017 hedgar2017 force-pushed the rename-self-hosted-runner-label branch from a1b1ec1 to 283cb6a Compare May 27, 2026 03:21
@hedgar2017 hedgar2017 enabled auto-merge May 27, 2026 03:21
@hedgar2017 hedgar2017 added this pull request to the merge queue May 27, 2026
Merged via the queue into main with commit 88b5096 May 27, 2026
41 of 42 checks passed
@hedgar2017 hedgar2017 deleted the rename-self-hosted-runner-label branch May 27, 2026 04:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:coverage Trigger coverage workflow on PR ci:integration Trigger integration tests workflow on PR ci:sanitizer Trigger sanitizer workflow on PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants