Skip to content

Speed up CI: preinstalled Python, earlier snapshot pickup, configuration cache #2021

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
igorwojda merged 1 commit into from
Oct 13, 2026
Merged

Speed up CI: preinstalled Python, earlier snapshot pickup, configuration cache #2021

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 45 additions & 18 deletions .github/actions/get-published-konsist-snapshot/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,21 @@
name: Get Published Konsist Snapshot
description: >
Waits for the Publish workflow run of the current commit (and event) to complete and outputs the unique version of
the Konsist snapshot it published to the Central snapshots repository. Requires 'actions: read' permission.
Waits for the Publish workflow run of the current commit (and event) to upload the 'konsist-snapshot-version' artifact
and outputs the unique version of the Konsist snapshot it published to the Central snapshots repository.
Requires 'actions: read' permission.

inputs:
github-token:
description: Token used to read workflow runs and artifacts
required: true
timeout-minutes:
description: How long to wait for the Publish workflow run to complete
description: How long to wait for the Publish workflow run to publish the snapshot
default: '30'
initial-delay-seconds:
description: >
How long after the Publish workflow run start to begin polling for the artifact. Publish runs upload it
after 81-113s (October 2026), so polling earlier only uses up the GITHUB_TOKEN rate limit.
default: '75'

outputs:
version:
Expand All @@ -19,45 +25,66 @@ outputs:
runs:
using: composite
steps:
- name: Wait For Publish Workflow Run
- name: Wait For Published Snapshot Version
id: published-snapshot
shell: bash
env:
GH_TOKEN: ${{ inputs.github-token }}
TIMEOUT_MINUTES: ${{ inputs.timeout-minutes }}
INITIAL_DELAY_SECONDS: ${{ inputs.initial-delay-seconds }}
HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
EVENT_NAME: ${{ github.event_name }}
run: |
RUNS_URL="repos/$GITHUB_REPOSITORY/actions/runs?head_sha=$HEAD_SHA&event=$EVENT_NAME&per_page=100"
# Latest run of the Publish workflow (runs are sorted from newest)
RUN_FILTER='[.workflow_runs[] | select(.path | startswith(".github/workflows/publish.yml"))][0] // {} | "\(.id) \(.status) \(.conclusion)"'
RUN_FILTER='[.workflow_runs[] | select(.path | startswith(".github/workflows/publish.yml"))][0] // {} | "\(.id) \(.status) \(.conclusion) \(.run_started_at)"'
DEADLINE=$(( $(date +%s) + TIMEOUT_MINUTES * 60 ))
INITIAL_DELAY_DONE=false

# The artifact is available as soon as it is uploaded, about a minute before the run is marked as completed
while true; do
read -r RUN_ID STATUS CONCLUSION < <(gh api "$RUNS_URL" --jq "$RUN_FILTER")
read -r RUN_ID STATUS CONCLUSION RUN_STARTED_AT < <(gh api "$RUNS_URL" --jq "$RUN_FILTER")
RUN_URL="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$RUN_ID"

if [ "$RUN_ID" != "null" ]; then
# Skip polling while the snapshot is being published (counted from the Publish workflow run start)
if [ "$INITIAL_DELAY_DONE" = false ] && [ "$RUN_STARTED_AT" != "null" ]; then
INITIAL_DELAY_DONE=true
DELAY=$(( $(date -d "$RUN_STARTED_AT" +%s) + INITIAL_DELAY_SECONDS - $(date +%s) ))

if [ "$STATUS" != "completed" ] && [ "$DELAY" -gt 0 ]; then
echo "Waiting ${DELAY}s before polling Publish workflow run $RUN_URL..."
sleep "$DELAY"
continue
fi
fi

ARTIFACT_ID=$(gh api "repos/$GITHUB_REPOSITORY/actions/runs/$RUN_ID/artifacts?name=konsist-snapshot-version" --jq '.artifacts[0].id // empty')

if [ -n "$ARTIFACT_ID" ]; then
break
fi
fi

if [ "$STATUS" = "completed" ]; then
break
echo "::error::Publish workflow run $RUN_URL concluded with '$CONCLUSION' without publishing the snapshot."
exit 1
fi

if [ "$(date +%s)" -ge "$DEADLINE" ]; then
echo "::error::Publish workflow run for $HEAD_SHA ($EVENT_NAME) did not complete in $TIMEOUT_MINUTES minutes."
echo "::error::Publish workflow run for $HEAD_SHA ($EVENT_NAME) did not publish the snapshot in $TIMEOUT_MINUTES minutes."
exit 1
fi

echo "Waiting for Publish workflow run (run: $RUN_ID, status: $STATUS)..."
sleep 20
sleep 5
done

RUN_URL="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$RUN_ID"

if [ "$CONCLUSION" != "success" ]; then
echo "::error::Publish workflow run $RUN_URL concluded with '$CONCLUSION'."
exit 1
fi

gh run download "$RUN_ID" --repo "$GITHUB_REPOSITORY" --name konsist-snapshot-version --dir "$RUNNER_TEMP/konsist-snapshot-version"
VERSION=$(tr -d '\r\n' < "$RUNNER_TEMP/konsist-snapshot-version/konsist-snapshot-version.txt")
ARTIFACT_DIR="$RUNNER_TEMP/konsist-snapshot-version"
mkdir -p "$ARTIFACT_DIR"
gh api "repos/$GITHUB_REPOSITORY/actions/artifacts/$ARTIFACT_ID/zip" > "$ARTIFACT_DIR/artifact.zip"
unzip -o -q "$ARTIFACT_DIR/artifact.zip" -d "$ARTIFACT_DIR"
VERSION=$(tr -d '\r\n' < "$ARTIFACT_DIR/konsist-snapshot-version.txt")

echo "Konsist snapshot $VERSION published by $RUN_URL"
echo "version=$VERSION" >> "$GITHUB_OUTPUT"
2 changes: 2 additions & 0 deletions .github/workflows/check-ktdoc-snippets.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ jobs:

- name: Set up Gradle
uses: gradle/actions/setup-gradle@v6
with:
cache-encryption-key: ${{ secrets.GRADLE_ENCRYPTION_KEY }}

- name: Run Python Script for All Files
run: python3 misc/scripts/check_ktdoc_snippets.py -all
24 changes: 12 additions & 12 deletions .github/workflows/check-starter-projects.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Add Konsist Snapshot Repository
run: python3 misc/scripts/replace_konsist_version/add_konsist_snapshot_repository_to_config_file.py samples/starter-projects/android-gradle-groovy-junit-4/settings.gradle
Expand Down Expand Up @@ -145,7 +145,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Add Konsist Snapshot Repository
run: python3 misc/scripts/replace_konsist_version/add_konsist_snapshot_repository_to_config_file.py samples/starter-projects/android-gradle-groovy-junit-5/settings.gradle
Expand Down Expand Up @@ -183,7 +183,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Add Konsist Snapshot Repository
run: python3 misc/scripts/replace_konsist_version/add_konsist_snapshot_repository_to_config_file.py samples/starter-projects/android-gradle-groovy-kotest/settings.gradle
Expand Down Expand Up @@ -221,7 +221,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Add Konsist Snapshot Repository
run: python3 misc/scripts/replace_konsist_version/add_konsist_snapshot_repository_to_config_file.py samples/starter-projects/android-gradle-kotlin-junit-4/settings.gradle.kts
Expand Down Expand Up @@ -258,7 +258,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Add Konsist Snapshot Repository
run: python3 misc/scripts/replace_konsist_version/add_konsist_snapshot_repository_to_config_file.py samples/starter-projects/android-gradle-kotlin-junit-5/settings.gradle.kts
Expand Down Expand Up @@ -295,7 +295,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Add Konsist Snapshot Repository
run: python3 misc/scripts/replace_konsist_version/add_konsist_snapshot_repository_to_config_file.py samples/starter-projects/android-gradle-kotlin-kotest/settings.gradle.kts
Expand Down Expand Up @@ -332,7 +332,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Add Konsist Snapshot Repository
run: python3 misc/scripts/replace_konsist_version/add_konsist_snapshot_repository_to_config_file.py samples/starter-projects/spring-gradle-kotlin-junit-5/build.gradle.kts
Expand Down Expand Up @@ -369,7 +369,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Add Konsist Snapshot Repository
run: python3 misc/scripts/replace_konsist_version/add_konsist_snapshot_repository_to_config_file.py samples/starter-projects/spring-gradle-kotlin-kotest/build.gradle.kts
Expand Down Expand Up @@ -406,7 +406,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Add Konsist Snapshot Repository
run: python3 misc/scripts/replace_konsist_version/add_konsist_snapshot_repository_to_config_file.py samples/starter-projects/spring-gradle-groovy-junit-5/build.gradle
Expand Down Expand Up @@ -444,7 +444,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Add Konsist Snapshot Repository
run: python3 misc/scripts/replace_konsist_version/add_konsist_snapshot_repository_to_config_file.py samples/starter-projects/spring-gradle-groovy-kotest/build.gradle
Expand Down Expand Up @@ -481,7 +481,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Add Konsist Snapshot Repository
run: python3 misc/scripts/replace_konsist_version/add_konsist_snapshot_repository_to_config_file.py samples/starter-projects/spring-maven-junit5/pom.xml
Expand Down Expand Up @@ -518,7 +518,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Add Konsist Snapshot Repository
run: python3 misc/scripts/replace_konsist_version/add_konsist_snapshot_repository_to_config_file.py samples/starter-projects/spring-maven-kotest/pom.xml
Expand Down
34 changes: 17 additions & 17 deletions .github/workflows/check-test-projects.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Use Konsist Snapshot From Maven Central
run: python misc/scripts/replace_konsist_version/use_konsist_snapshot_in_test_project.py misc/projects/test-projects/konsist-declaration-tester ${{ needs.published-snapshot.outputs.version }}
Expand Down Expand Up @@ -95,7 +95,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Use Konsist Snapshot From Maven Central
run: python misc/scripts/replace_konsist_version/use_konsist_snapshot_in_test_project.py misc/projects/test-projects/konsist-declaration-tester ${{ needs.published-snapshot.outputs.version }}
Expand Down Expand Up @@ -132,7 +132,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Use Konsist Snapshot From Maven Central
run: python misc/scripts/replace_konsist_version/use_konsist_snapshot_in_test_project.py misc/projects/test-projects/konsist-declaration-tester ${{ needs.published-snapshot.outputs.version }}
Expand Down Expand Up @@ -171,7 +171,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Use Konsist Snapshot From Maven Central
run: python misc/scripts/replace_konsist_version/use_konsist_snapshot_in_test_project.py misc/projects/test-projects/konsist-compiler-configuration-extension-storage-tester ${{ needs.published-snapshot.outputs.version }}
Expand Down Expand Up @@ -208,7 +208,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Use Konsist Snapshot From Maven Central
run: python misc/scripts/replace_konsist_version/use_konsist_snapshot_in_test_project.py misc/projects/test-projects/konsist-kotlin-compiler-classpath-conflict-tester ${{ needs.published-snapshot.outputs.version }}
Expand Down Expand Up @@ -245,7 +245,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Use Konsist Snapshot From Maven Central
run: python misc/scripts/replace_konsist_version/use_konsist_snapshot_in_test_project.py misc/projects/test-projects/konsist-path-tester ${{ needs.published-snapshot.outputs.version }}
Expand Down Expand Up @@ -286,7 +286,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Use Konsist Snapshot From Maven Central
run: python misc/scripts/replace_konsist_version/use_konsist_snapshot_in_test_project.py misc/projects/test-projects/konsist-path-tester ${{ needs.published-snapshot.outputs.version }}
Expand Down Expand Up @@ -323,7 +323,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Use Konsist Snapshot From Maven Central
run: python misc/scripts/replace_konsist_version/use_konsist_snapshot_in_test_project.py misc/projects/test-projects/konsist-path-tester ${{ needs.published-snapshot.outputs.version }}
Expand Down Expand Up @@ -358,7 +358,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Use Konsist Snapshot From Maven Central
run: python misc/scripts/replace_konsist_version/use_konsist_snapshot_in_test_project.py misc/projects/test-projects/konsist-root-path-gradle-tester ${{ needs.published-snapshot.outputs.version }}
Expand Down Expand Up @@ -403,7 +403,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Use Konsist Snapshot From Maven Central
run: python misc/scripts/replace_konsist_version/use_konsist_snapshot_in_test_project.py misc/projects/test-projects/konsist-root-path-gradle-tester ${{ needs.published-snapshot.outputs.version }}
Expand Down Expand Up @@ -443,7 +443,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Use Konsist Snapshot From Maven Central
run: python misc/scripts/replace_konsist_version/use_konsist_snapshot_in_test_project.py misc/projects/test-projects/konsist-root-path-gradle-tester ${{ needs.published-snapshot.outputs.version }}
Expand Down Expand Up @@ -481,7 +481,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Use Konsist Snapshot From Maven Central
run: python misc/scripts/replace_konsist_version/use_konsist_snapshot_in_test_project.py misc/projects/test-projects/konsist-root-path-nested-gradle-wrapper-tester ${{ needs.published-snapshot.outputs.version }}
Expand Down Expand Up @@ -526,7 +526,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Use Konsist Snapshot From Maven Central
run: python misc/scripts/replace_konsist_version/use_konsist_snapshot_in_test_project.py misc/projects/test-projects/konsist-root-path-nested-gradle-wrapper-tester ${{ needs.published-snapshot.outputs.version }}
Expand Down Expand Up @@ -566,7 +566,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Use Konsist Snapshot From Maven Central
run: python misc/scripts/replace_konsist_version/use_konsist_snapshot_in_test_project.py misc/projects/test-projects/konsist-root-path-nested-gradle-wrapper-tester ${{ needs.published-snapshot.outputs.version }}
Expand Down Expand Up @@ -604,7 +604,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Add Konsist Snapshot Repository
run: python misc/scripts/replace_konsist_version/add_konsist_snapshot_repository_to_config_file.py misc/projects/test-projects/konsist-root-path-maven-tester/pom.xml
Expand Down Expand Up @@ -648,7 +648,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Add Konsist Snapshot Repository
run: python misc/scripts/replace_konsist_version/add_konsist_snapshot_repository_to_config_file.py misc/projects/test-projects/konsist-root-path-maven-tester/pom.xml
Expand Down Expand Up @@ -688,7 +688,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v7
with:
python-version: '3.15.0'
python-version: '3.x'

- name: Add Konsist Snapshot Repository
run: python misc/scripts/replace_konsist_version/add_konsist_snapshot_repository_to_config_file.py misc/projects/test-projects/konsist-root-path-maven-tester/pom.xml
Expand Down
Loading
Loading