-
Notifications
You must be signed in to change notification settings - Fork 37
Add SLO workflow for the Java SDK #644
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,146 @@ | ||
| #!/usr/bin/env bash | ||
| # | ||
| # Builds the Docker image for the YDB Java SDK SLO workload. | ||
| # | ||
| # The script assembles a temporary build context containing two checkouts | ||
| # side by side — the SDK source tree and the ydb-java-examples checkout — | ||
| # and feeds that context to `docker build` using the Dockerfile shipped | ||
| # inside `ydb-java-examples/slo/`. | ||
| # | ||
| # The Dockerfile takes care of building the SDK from source, installing it | ||
| # into an in-image local Maven repository, and then building the workload | ||
| # against that exact SDK version. So the script does not need any Maven / | ||
| # JDK setup on the host; only `docker` and standard POSIX tools. | ||
| # | ||
| # If the initial build fails and `--fallback-image` is provided, the script | ||
| # tags the fallback image as `--tag` and exits successfully. This mirrors | ||
| # the behaviour of the equivalent script in `ydb-go-sdk` and is used by the | ||
| # baseline build, where we want to keep going even if the historical commit | ||
| # can't be built any more. | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| usage() { | ||
| cat >&2 <<'EOF' | ||
| Usage: | ||
| build-slo-image.sh \ | ||
| --sdk <path> \ | ||
| --examples <path> \ | ||
| --tag <docker-tag> \ | ||
| [--fallback-image <docker-tag>] | ||
|
|
||
| Options: | ||
| --sdk Path to the ydb-java-sdk checkout to build against. | ||
| --examples Path to the ydb-java-examples checkout that owns the | ||
| SLO workload sources (must contain slo/Dockerfile). | ||
| --tag Docker tag to assign to the built image | ||
| (e.g. ydb-app-current). | ||
| --fallback-image If the initial Docker build fails, tag this image as | ||
| --tag and exit successfully. Useful for the baseline | ||
| build, which may be unable to compile a historical | ||
| SDK commit. | ||
| EOF | ||
| } | ||
|
|
||
| die() { | ||
| echo "ERROR: $*" >&2 | ||
| exit 1 | ||
| } | ||
|
|
||
| sdk_dir="" | ||
| examples_dir="" | ||
| tag="" | ||
| fallback_image="" | ||
|
|
||
| while [[ $# -gt 0 ]]; do | ||
| case "$1" in | ||
| --sdk) | ||
| sdk_dir="${2:-}" | ||
| shift 2 | ||
| ;; | ||
| --examples) | ||
| examples_dir="${2:-}" | ||
| shift 2 | ||
| ;; | ||
| --tag) | ||
| tag="${2:-}" | ||
| shift 2 | ||
| ;; | ||
| --fallback-image) | ||
| fallback_image="${2:-}" | ||
| shift 2 | ||
| ;; | ||
| -h|--help) | ||
| usage | ||
| exit 0 | ||
| ;; | ||
| *) | ||
| die "Unknown argument: $1 (use --help)" | ||
| ;; | ||
| esac | ||
| done | ||
|
|
||
| if [[ -z "$sdk_dir" || -z "$examples_dir" || -z "$tag" ]]; then | ||
| usage | ||
| die "Incomplete argument set" | ||
| fi | ||
|
|
||
| [[ -d "$sdk_dir" ]] || die "--sdk does not exist: $sdk_dir" | ||
| [[ -d "$examples_dir" ]] || die "--examples does not exist: $examples_dir" | ||
|
|
||
| sdk_dir="$(cd "$sdk_dir" && pwd)" | ||
| examples_dir="$(cd "$examples_dir" && pwd)" | ||
|
|
||
| dockerfile="${examples_dir}/slo/Dockerfile" | ||
| [[ -f "$dockerfile" ]] || die "Dockerfile not found: $dockerfile" | ||
|
|
||
| # Assemble a build context that contains the two checkouts side by side. | ||
| # We use hard links where possible to avoid copying large amounts of data; | ||
| # `cp -al` falls back to a regular copy when hard links aren't supported | ||
| # (e.g. across filesystems on the GitHub runner cache). | ||
| context_dir="$(mktemp -d)" | ||
| trap 'rm -rf "$context_dir"' EXIT | ||
|
|
||
| echo "Assembling build context in $context_dir" | ||
| echo " ydb-java-sdk: $sdk_dir" | ||
| echo " ydb-java-examples: $examples_dir" | ||
| echo " tag: $tag" | ||
|
|
||
| copy_tree() { | ||
| local src="$1" | ||
| local dst="$2" | ||
| if cp -al "$src" "$dst" 2>/dev/null; then | ||
| return 0 | ||
| fi | ||
| cp -a "$src" "$dst" | ||
| } | ||
|
|
||
| copy_tree "$sdk_dir" "$context_dir/ydb-java-sdk" | ||
| copy_tree "$examples_dir" "$context_dir/ydb-java-examples" | ||
|
|
||
| # Drop any leftover .git directories from the build context so we don't ship | ||
| # them into image layers and don't confuse Maven plugins that probe for git. | ||
| find "$context_dir" -maxdepth 3 -type d -name '.git' -prune -exec rm -rf {} + | ||
|
|
||
| set +e | ||
| docker build \ | ||
| --platform linux/amd64 \ | ||
| -t "$tag" \ | ||
| -f "$dockerfile" \ | ||
| "$context_dir" | ||
| exit_code=$? | ||
| set -e | ||
|
|
||
| if [[ $exit_code -eq 0 ]]; then | ||
| echo "Docker image $tag built successfully" | ||
| exit 0 | ||
| fi | ||
|
|
||
| echo "Docker build for $tag failed (exit code $exit_code)" >&2 | ||
|
|
||
| if [[ -z "$fallback_image" ]]; then | ||
| die "Docker build failed and --fallback-image is not set" | ||
| fi | ||
|
|
||
| echo "Falling back to image $fallback_image, tagging as $tag" | ||
| docker tag "$fallback_image" "$tag" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| name: slo-report | ||
|
|
||
| on: | ||
| workflow_run: | ||
| workflows: ["SLO"] | ||
| types: | ||
| - completed | ||
|
|
||
| jobs: | ||
| publish-slo-report: | ||
| runs-on: ubuntu-latest | ||
| name: Publish YDB SLO Report | ||
| permissions: | ||
| checks: write | ||
| contents: read | ||
| pull-requests: write | ||
| if: github.event.workflow_run.conclusion == 'success' | ||
| steps: | ||
| - name: Publish YDB SLO Report | ||
| uses: ydb-platform/ydb-slo-action/report@v2 | ||
| with: | ||
| github_token: ${{ secrets.GITHUB_TOKEN }} | ||
| github_run_id: ${{ github.event.workflow_run.id }} | ||
|
|
||
| remove-slo-label: | ||
| needs: publish-slo-report | ||
| if: always() && github.event.workflow_run.event == 'pull_request' | ||
| runs-on: ubuntu-latest | ||
| name: Remove SLO Label | ||
| permissions: | ||
| pull-requests: write | ||
| steps: | ||
| - name: Remove SLO label from PR | ||
| uses: actions/github-script@v7 | ||
| with: | ||
| script: | | ||
| const pullRequests = context.payload.workflow_run.pull_requests; | ||
| if (!pullRequests || pullRequests.length === 0) { | ||
| console.log('No pull requests associated with this workflow run'); | ||
| return; | ||
| } | ||
| for (const pr of pullRequests) { | ||
| try { | ||
| await github.rest.issues.removeLabel({ | ||
| owner: context.repo.owner, | ||
| repo: context.repo.repo, | ||
| issue_number: pr.number, | ||
| name: 'SLO' | ||
| }); | ||
| console.log(`Removed SLO label from PR #${pr.number}`); | ||
| } catch (error) { | ||
| if (error.status === 404) { | ||
| console.log(`SLO label not found on PR #${pr.number}, skipping`); | ||
| } else { | ||
| throw error; | ||
| } | ||
| } | ||
| } | ||
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,172 @@ | ||||||||||||||||||||||
| name: SLO | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| on: | ||||||||||||||||||||||
| push: | ||||||||||||||||||||||
| branches: | ||||||||||||||||||||||
| - master | ||||||||||||||||||||||
| - develop | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| pull_request: | ||||||||||||||||||||||
| types: [opened, reopened, synchronize, labeled] | ||||||||||||||||||||||
| branches: | ||||||||||||||||||||||
| - master | ||||||||||||||||||||||
| - develop | ||||||||||||||||||||||
|
alex268 marked this conversation as resolved.
|
||||||||||||||||||||||
|
|
||||||||||||||||||||||
| workflow_dispatch: | ||||||||||||||||||||||
| inputs: | ||||||||||||||||||||||
| github_issue: | ||||||||||||||||||||||
| description: "GitHub issue number where the SLO results will be reported" | ||||||||||||||||||||||
| required: true | ||||||||||||||||||||||
| baseline_ref: | ||||||||||||||||||||||
| description: "Baseline commit/branch/tag to compare against (leave empty to auto-detect merge-base with master)" | ||||||||||||||||||||||
| required: false | ||||||||||||||||||||||
| slo_workload_duration_seconds: | ||||||||||||||||||||||
| description: "Duration of the SLO workload in seconds" | ||||||||||||||||||||||
| required: false | ||||||||||||||||||||||
| default: "600" | ||||||||||||||||||||||
| slo_workload_read_max_rps: | ||||||||||||||||||||||
| description: "Maximum read RPS for the SLO workload" | ||||||||||||||||||||||
| required: false | ||||||||||||||||||||||
| default: "1000" | ||||||||||||||||||||||
| slo_workload_write_max_rps: | ||||||||||||||||||||||
| description: "Maximum write RPS for the SLO workload" | ||||||||||||||||||||||
| required: false | ||||||||||||||||||||||
| default: "100" | ||||||||||||||||||||||
| examples_ref: | ||||||||||||||||||||||
| description: "Branch/tag/SHA of ydb-java-examples to use for the workload sources" | ||||||||||||||||||||||
| required: false | ||||||||||||||||||||||
| default: "master" | ||||||||||||||||||||||
|
|
||||||||||||||||||||||
| jobs: | ||||||||||||||||||||||
| ydb-slo-action: | ||||||||||||||||||||||
| if: contains(github.event.pull_request.labels.*.name, 'SLO') || github.event_name == 'workflow_dispatch' || github.event_name == 'push' | ||||||||||||||||||||||
|
||||||||||||||||||||||
| if: contains(github.event.pull_request.labels.*.name, 'SLO') || github.event_name == 'workflow_dispatch' || github.event_name == 'push' | |
| if: github.event_name == 'workflow_dispatch' || github.event_name == 'push' || (github.event_name == 'pull_request' && contains(github.event.pull_request.labels.*.name, 'SLO')) |
Copilot
AI
Apr 26, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
curl for downloading yq does not use -f/--fail, so a 404/redirect-to-HTML could silently write an invalid binary and only fail later in a less obvious way. Use curl -fLo (consistent with the buildx/compose downloads below) to fail fast on HTTP errors.
| sudo curl -L "https://github.com/mikefarah/yq/releases/download/${YQ_VERSION}/yq_linux_amd64" \ | |
| -o /usr/local/bin/yq | |
| sudo curl -fLo /usr/local/bin/yq \ | |
| "https://github.com/mikefarah/yq/releases/download/${YQ_VERSION}/yq_linux_amd64" |
Copilot
AI
Apr 26, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Potential bash injection: ${{ inputs.baseline_ref }} is interpolated directly into the shell script inside double quotes. A workflow_dispatch input containing a " or newline could break quoting and inject shell syntax. Safer pattern is to pass the input via env: and read it as a normal shell variable (still quoted), or use ${{ toJSON(inputs.baseline_ref) }} style escaping when embedding into bash.
| run: | | |
| set -euo pipefail | |
| if [[ -n "${{ inputs.baseline_ref }}" ]]; then | |
| BASELINE="${{ inputs.baseline_ref }}" | |
| env: | |
| INPUT_BASELINE_REF: ${{ inputs.baseline_ref }} | |
| run: | | |
| set -euo pipefail | |
| if [[ -n "${INPUT_BASELINE_REF}" ]]; then | |
| BASELINE="${INPUT_BASELINE_REF}" |
Copilot
AI
Apr 26, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
github_issue is only provided for workflow_dispatch, but this workflow also runs on pull_request/push. For those events, ${{ github.event.inputs.github_issue }} will be empty, which is likely to break ydb-slo-action/init@v2 (and makes push runs impossible to report anywhere). Consider deriving it from the PR number when event_name == 'pull_request', and either require an explicit input (or skip reporting) on push runs.
| uses: ydb-platform/ydb-slo-action/init@v2 | |
| timeout-minutes: 30 | |
| with: | |
| github_issue: ${{ github.event.inputs.github_issue }} | |
| if: ${{ github.event_name == 'workflow_dispatch' || github.event_name == 'pull_request' }} | |
| uses: ydb-platform/ydb-slo-action/init@v2 | |
| timeout-minutes: 30 | |
| with: | |
| github_issue: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.github_issue || github.event.pull_request.number }} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This workflow triggers on every successful
SLOrun, includingpushandworkflow_dispatch. For non-PR runs there may be no associated PR to comment on, andydb-slo-action/report@v2may fail or post nowhere. Consider gatingpublish-slo-reportongithub.event.workflow_run.event == 'pull_request'(or otherwise handling the non-PR reporting path explicitly).There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for testing only