Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
146 changes: 146 additions & 0 deletions .github/scripts/build-slo-image.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
#!/usr/bin/env bash
#
# Builds the Docker image for the YDB Java SDK SLO workload.
#
# The script assembles a temporary build context containing two checkouts
# side by side — the SDK source tree and the ydb-java-examples checkout —
# and feeds that context to `docker build` using the Dockerfile shipped
# inside `ydb-java-examples/slo/`.
#
# The Dockerfile takes care of building the SDK from source, installing it
# into an in-image local Maven repository, and then building the workload
# against that exact SDK version. So the script does not need any Maven /
# JDK setup on the host; only `docker` and standard POSIX tools.
#
# If the initial build fails and `--fallback-image` is provided, the script
# tags the fallback image as `--tag` and exits successfully. This mirrors
# the behaviour of the equivalent script in `ydb-go-sdk` and is used by the
# baseline build, where we want to keep going even if the historical commit
# can't be built any more.

set -euo pipefail

usage() {
cat >&2 <<'EOF'
Usage:
build-slo-image.sh \
--sdk <path> \
--examples <path> \
--tag <docker-tag> \
[--fallback-image <docker-tag>]

Options:
--sdk Path to the ydb-java-sdk checkout to build against.
--examples Path to the ydb-java-examples checkout that owns the
SLO workload sources (must contain slo/Dockerfile).
--tag Docker tag to assign to the built image
(e.g. ydb-app-current).
--fallback-image If the initial Docker build fails, tag this image as
--tag and exit successfully. Useful for the baseline
build, which may be unable to compile a historical
SDK commit.
EOF
}

die() {
echo "ERROR: $*" >&2
exit 1
}

sdk_dir=""
examples_dir=""
tag=""
fallback_image=""

while [[ $# -gt 0 ]]; do
case "$1" in
--sdk)
sdk_dir="${2:-}"
shift 2
;;
--examples)
examples_dir="${2:-}"
shift 2
;;
--tag)
tag="${2:-}"
shift 2
;;
--fallback-image)
fallback_image="${2:-}"
shift 2
;;
-h|--help)
usage
exit 0
;;
*)
die "Unknown argument: $1 (use --help)"
;;
esac
done

if [[ -z "$sdk_dir" || -z "$examples_dir" || -z "$tag" ]]; then
usage
die "Incomplete argument set"
fi

[[ -d "$sdk_dir" ]] || die "--sdk does not exist: $sdk_dir"
[[ -d "$examples_dir" ]] || die "--examples does not exist: $examples_dir"

sdk_dir="$(cd "$sdk_dir" && pwd)"
examples_dir="$(cd "$examples_dir" && pwd)"

dockerfile="${examples_dir}/slo/Dockerfile"
[[ -f "$dockerfile" ]] || die "Dockerfile not found: $dockerfile"

# Assemble a build context that contains the two checkouts side by side.
# We use hard links where possible to avoid copying large amounts of data;
# `cp -al` falls back to a regular copy when hard links aren't supported
# (e.g. across filesystems on the GitHub runner cache).
context_dir="$(mktemp -d)"
trap 'rm -rf "$context_dir"' EXIT

echo "Assembling build context in $context_dir"
echo " ydb-java-sdk: $sdk_dir"
echo " ydb-java-examples: $examples_dir"
echo " tag: $tag"

copy_tree() {
local src="$1"
local dst="$2"
if cp -al "$src" "$dst" 2>/dev/null; then
return 0
fi
cp -a "$src" "$dst"
}

copy_tree "$sdk_dir" "$context_dir/ydb-java-sdk"
copy_tree "$examples_dir" "$context_dir/ydb-java-examples"

# Drop any leftover .git directories from the build context so we don't ship
# them into image layers and don't confuse Maven plugins that probe for git.
find "$context_dir" -maxdepth 3 -type d -name '.git' -prune -exec rm -rf {} +

set +e
docker build \
--platform linux/amd64 \
-t "$tag" \
-f "$dockerfile" \
"$context_dir"
exit_code=$?
set -e

if [[ $exit_code -eq 0 ]]; then
echo "Docker image $tag built successfully"
exit 0
fi

echo "Docker build for $tag failed (exit code $exit_code)" >&2

if [[ -z "$fallback_image" ]]; then
die "Docker build failed and --fallback-image is not set"
fi

echo "Falling back to image $fallback_image, tagging as $tag"
docker tag "$fallback_image" "$tag"
58 changes: 58 additions & 0 deletions .github/workflows/slo-report.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
name: slo-report

on:
workflow_run:
workflows: ["SLO"]
types:
- completed

jobs:
publish-slo-report:
runs-on: ubuntu-latest
name: Publish YDB SLO Report
permissions:
checks: write
contents: read
pull-requests: write
if: github.event.workflow_run.conclusion == 'success'
steps:
- name: Publish YDB SLO Report
uses: ydb-platform/ydb-slo-action/report@v2
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
github_run_id: ${{ github.event.workflow_run.id }}
Comment on lines +10 to +23
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This workflow triggers on every successful SLO run, including push and workflow_dispatch. For non-PR runs there may be no associated PR to comment on, and ydb-slo-action/report@v2 may fail or post nowhere. Consider gating publish-slo-report on github.event.workflow_run.event == 'pull_request' (or otherwise handling the non-PR reporting path explicitly).

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for testing only


remove-slo-label:
needs: publish-slo-report
if: always() && github.event.workflow_run.event == 'pull_request'
runs-on: ubuntu-latest
name: Remove SLO Label
permissions:
pull-requests: write
steps:
- name: Remove SLO label from PR
uses: actions/github-script@v7
with:
script: |
const pullRequests = context.payload.workflow_run.pull_requests;
if (!pullRequests || pullRequests.length === 0) {
console.log('No pull requests associated with this workflow run');
return;
}
for (const pr of pullRequests) {
try {
await github.rest.issues.removeLabel({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: pr.number,
name: 'SLO'
});
console.log(`Removed SLO label from PR #${pr.number}`);
} catch (error) {
if (error.status === 404) {
console.log(`SLO label not found on PR #${pr.number}, skipping`);
} else {
throw error;
}
}
}
172 changes: 172 additions & 0 deletions .github/workflows/slo.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
name: SLO

on:
push:
branches:
- master
- develop

pull_request:
types: [opened, reopened, synchronize, labeled]
branches:
- master
- develop
Comment thread
alex268 marked this conversation as resolved.

workflow_dispatch:
inputs:
github_issue:
description: "GitHub issue number where the SLO results will be reported"
required: true
baseline_ref:
description: "Baseline commit/branch/tag to compare against (leave empty to auto-detect merge-base with master)"
required: false
slo_workload_duration_seconds:
description: "Duration of the SLO workload in seconds"
required: false
default: "600"
slo_workload_read_max_rps:
description: "Maximum read RPS for the SLO workload"
required: false
default: "1000"
slo_workload_write_max_rps:
description: "Maximum write RPS for the SLO workload"
required: false
default: "100"
examples_ref:
description: "Branch/tag/SHA of ydb-java-examples to use for the workload sources"
required: false
default: "master"

jobs:
ydb-slo-action:
if: contains(github.event.pull_request.labels.*.name, 'SLO') || github.event_name == 'workflow_dispatch' || github.event_name == 'push'
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The job-level if: evaluates contains(github.event.pull_request.labels.*.name, 'SLO') even on push and workflow_dispatch events where github.event.pull_request is null. This can cause an expression evaluation error and skip/break the workflow. Guard the label check behind an github.event_name == 'pull_request' condition (or restructure the expression so it only references github.event.pull_request for PR events).

Suggested change
if: contains(github.event.pull_request.labels.*.name, 'SLO') || github.event_name == 'workflow_dispatch' || github.event_name == 'push'
if: github.event_name == 'workflow_dispatch' || github.event_name == 'push' || (github.event_name == 'pull_request' && contains(github.event.pull_request.labels.*.name, 'SLO'))

Copilot uses AI. Check for mistakes.

name: Run YDB SLO Tests
runs-on: ubuntu-latest

permissions:
contents: read

strategy:
fail-fast: false
matrix:
sdk:
- name: java-query-kv
command: ""

concurrency:
group: slo-${{ github.ref }}-${{ matrix.sdk.name }}
cancel-in-progress: true

steps:
- name: Install dependencies
run: |
set -euxo pipefail
YQ_VERSION=v4.48.2
BUILDX_VERSION=0.30.1
COMPOSE_VERSION=2.40.3

sudo curl -L "https://github.com/mikefarah/yq/releases/download/${YQ_VERSION}/yq_linux_amd64" \
-o /usr/local/bin/yq
Comment on lines +69 to +70
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

curl for downloading yq does not use -f/--fail, so a 404/redirect-to-HTML could silently write an invalid binary and only fail later in a less obvious way. Use curl -fLo (consistent with the buildx/compose downloads below) to fail fast on HTTP errors.

Suggested change
sudo curl -L "https://github.com/mikefarah/yq/releases/download/${YQ_VERSION}/yq_linux_amd64" \
-o /usr/local/bin/yq
sudo curl -fLo /usr/local/bin/yq \
"https://github.com/mikefarah/yq/releases/download/${YQ_VERSION}/yq_linux_amd64"

Copilot uses AI. Check for mistakes.
sudo chmod +x /usr/local/bin/yq

sudo mkdir -p /usr/local/lib/docker/cli-plugins

sudo curl -fLo /usr/local/lib/docker/cli-plugins/docker-buildx \
"https://github.com/docker/buildx/releases/download/v${BUILDX_VERSION}/buildx-v${BUILDX_VERSION}.linux-amd64"
sudo chmod +x /usr/local/lib/docker/cli-plugins/docker-buildx

sudo curl -fLo /usr/local/lib/docker/cli-plugins/docker-compose \
"https://github.com/docker/compose/releases/download/v${COMPOSE_VERSION}/docker-compose-linux-x86_64"
sudo chmod +x /usr/local/lib/docker/cli-plugins/docker-compose

yq --version
docker --version
docker buildx version
docker compose version

- name: Checkout current SDK version
uses: actions/checkout@v5
with:
path: sdk-current
fetch-depth: 0

- name: Determine baseline commit
id: baseline
working-directory: sdk-current
run: |
set -euo pipefail
if [[ -n "${{ inputs.baseline_ref }}" ]]; then
BASELINE="${{ inputs.baseline_ref }}"
Comment on lines +97 to +100
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential bash injection: ${{ inputs.baseline_ref }} is interpolated directly into the shell script inside double quotes. A workflow_dispatch input containing a " or newline could break quoting and inject shell syntax. Safer pattern is to pass the input via env: and read it as a normal shell variable (still quoted), or use ${{ toJSON(inputs.baseline_ref) }} style escaping when embedding into bash.

Suggested change
run: |
set -euo pipefail
if [[ -n "${{ inputs.baseline_ref }}" ]]; then
BASELINE="${{ inputs.baseline_ref }}"
env:
INPUT_BASELINE_REF: ${{ inputs.baseline_ref }}
run: |
set -euo pipefail
if [[ -n "${INPUT_BASELINE_REF}" ]]; then
BASELINE="${INPUT_BASELINE_REF}"

Copilot uses AI. Check for mistakes.
else
BASELINE=$(git merge-base HEAD origin/master)
fi
echo "sha=${BASELINE}" >> "$GITHUB_OUTPUT"

if git merge-base --is-ancestor "${BASELINE}" origin/master && \
[ "$(git rev-parse origin/master)" = "${BASELINE}" ]; then
BASELINE_REF="master"
else
BRANCH=$(git branch -r --contains "${BASELINE}" | grep -v HEAD | head -1 | sed 's|.*/||' || echo "")
if [ -n "${BRANCH}" ]; then
BASELINE_REF="${BRANCH}@${BASELINE:0:7}"
else
BASELINE_REF="${BASELINE:0:7}"
fi
fi
echo "ref=${BASELINE_REF}" >> "$GITHUB_OUTPUT"

- name: Checkout baseline SDK version
uses: actions/checkout@v5
with:
ref: ${{ steps.baseline.outputs.sha }}
path: sdk-baseline
fetch-depth: 1

- name: Checkout ydb-java-examples
uses: actions/checkout@v5
with:
repository: ydb-platform/ydb-java-examples
ref: ${{ inputs.examples_ref || 'master' }}
path: examples

- name: Build current workload image
run: |
set -euxo pipefail
chmod +x sdk-current/.github/scripts/build-slo-image.sh
sdk-current/.github/scripts/build-slo-image.sh \
--sdk "${GITHUB_WORKSPACE}/sdk-current" \
--examples "${GITHUB_WORKSPACE}/examples" \
--tag ydb-app-current

- name: Build baseline workload image
run: |
set -euxo pipefail
# Reuse the build script from the current checkout — it doesn't
# depend on SDK contents, only on the layout it produces.
sdk-current/.github/scripts/build-slo-image.sh \
--sdk "${GITHUB_WORKSPACE}/sdk-baseline" \
--examples "${GITHUB_WORKSPACE}/examples" \
--tag ydb-app-baseline \
--fallback-image ydb-app-current

- name: Run SLO Tests
uses: ydb-platform/ydb-slo-action/init@v2
timeout-minutes: 30
with:
github_issue: ${{ github.event.inputs.github_issue }}
Comment on lines +154 to +157
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

github_issue is only provided for workflow_dispatch, but this workflow also runs on pull_request/push. For those events, ${{ github.event.inputs.github_issue }} will be empty, which is likely to break ydb-slo-action/init@v2 (and makes push runs impossible to report anywhere). Consider deriving it from the PR number when event_name == 'pull_request', and either require an explicit input (or skip reporting) on push runs.

Suggested change
uses: ydb-platform/ydb-slo-action/init@v2
timeout-minutes: 30
with:
github_issue: ${{ github.event.inputs.github_issue }}
if: ${{ github.event_name == 'workflow_dispatch' || github.event_name == 'pull_request' }}
uses: ydb-platform/ydb-slo-action/init@v2
timeout-minutes: 30
with:
github_issue: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.github_issue || github.event.pull_request.number }}

Copilot uses AI. Check for mistakes.
github_token: ${{ secrets.GITHUB_TOKEN }}
workload_name: ${{ matrix.sdk.name }}
workload_duration: ${{ inputs.slo_workload_duration_seconds || '600' }}
workload_current_ref: ${{ github.head_ref || github.ref_name }}
workload_current_image: ydb-app-current
workload_current_command: >-
${{ matrix.sdk.command }}
--read-rps ${{ inputs.slo_workload_read_max_rps || '1000' }}
--write-rps ${{ inputs.slo_workload_write_max_rps || '100' }}
workload_baseline_ref: ${{ steps.baseline.outputs.ref }}
workload_baseline_image: ydb-app-baseline
workload_baseline_command: >-
${{ matrix.sdk.command }}
--read-rps ${{ inputs.slo_workload_read_max_rps || '1000' }}
--write-rps ${{ inputs.slo_workload_write_max_rps || '100' }}