Skip to content

test: various fixes to package testing #229

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
patrick-stephens merged 8 commits into from
Mar 19, 2026
Merged

test: various fixes to package testing #229

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
f4b04a4
test: fix incorrect env var usage for versioning
patrick-stephens Mar 19, 2026
4d52dc0
fix: update install script to cope with delays in page fetching
patrick-stephens Mar 19, 2026
640184b
test: reduce debug output for tests
patrick-stephens Mar 19, 2026
9af1a5d
test: fix incorrect env var usage for versioning
patrick-stephens Mar 19, 2026
012882a
test: provide helpful output
patrick-stephens Mar 19, 2026
739a91f
test: increase curl timeout
patrick-stephens Mar 19, 2026
1884edc
fix: review comments
patrick-stephens Mar 19, 2026
89f3891
fix: resolve incorrect tag usage for PRs
patrick-stephens Mar 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .github/workflows/build.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -378,6 +378,18 @@ jobs:
registry-password: ${{ secrets.GITHUB_TOKEN }}
output-file: image-debian-sbom.spdx

- name: Generate JSON schema from container
timeout-minutes: 1
# We don't want to fail the release if this fails since it's not critical and can be debugged after the fact if needed
continue-on-error: true
run: |
echo "Generating JSON schema from container image"
docker run --rm -t "${{ env.UBI_IMAGE_NAME }}:${{ env.TAG }}" -J > output/telemetry-forge-ubi-schema.json
docker run --rm -t "${{ env.DISTROLESS_IMAGE_NAME }}:${{ env.TAG }}" -J > output/telemetry-forge-debian-schema.json
# We use the UBI image schema as the main one since it has more metadata and we can link to the distroless one for reference
cp output/telemetry-forge-ubi-schema.json output/telemetry-forge-agent-schema.json
shell: bash

- name: Save image as tarball
run: |
mkdir -p output
Expand Down
7 changes: 7 additions & 0 deletions .github/workflows/call-test-containers.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,13 @@ jobs:
run: curl -sSfL https://github.com/shenwei356/rush/releases/download/v0.7.0/rush_linux_amd64.tar.gz | sudo tar xzf - -C /usr/local/bin
shell: bash

- name: Ensure we use the right version during a release
if: startsWith(github.ref, 'refs/tags/v')
Copy link

@cubic-dev-ai cubic-dev-ai bot Mar 19, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The tag-only if condition prevents setting version env vars for non-tag runs, so container tests can validate against the wrong fallback version instead of inputs.image-tag.

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. At .github/workflows/call-test-containers.yaml, line 90:

<comment>The tag-only `if` condition prevents setting version env vars for non-tag runs, so container tests can validate against the wrong fallback version instead of `inputs.image-tag`.</comment>

<file context>
@@ -88,6 +86,13 @@ jobs:
         shell: bash
 
+      - name: Ensure we use the right version during a release
+        if: startsWith(github.ref, 'refs/tags/v')
+        run: |
+          echo "TELEMETRY_FORGE_AGENT_VERSION=${{ inputs.image-tag }}" >> $GITHUB_ENV
</file context>
Suggested change
if: startsWith(github.ref, 'refs/tags/v')
if: ${{ always() }}
Fix with Cubic

run: |
echo "TELEMETRY_FORGE_AGENT_VERSION=${{ inputs.image-tag }}" >> $GITHUB_ENV
echo "FLUENTDO_AGENT_VERSION=${{ inputs.image-tag }}" >> $GITHUB_ENV
shell: bash

- name: Run container integration tests
run: ./testing/bats/run-container-integration-tests.sh
shell: bash
Expand Down
29 changes: 25 additions & 4 deletions install.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -368,12 +368,33 @@ fetch_available_versions() {
log_debug "Fetching page $page_count"
fi

local versions_response
versions_response=$(curl -s -L "$url" 2>/dev/null || echo "")
local versions_response=""
local fetch_attempts=0
local max_fetch_attempts=5
local fetch_delay=10

while [ $fetch_attempts -lt $max_fetch_attempts ]; do
fetch_attempts=$((fetch_attempts + 1))
versions_response=$(curl --max-time 60 -s -L "$url" 2>/dev/null || echo "")

if [ -z "$versions_response" ]; then
log_debug "curl returned empty response (attempt $fetch_attempts/$max_fetch_attempts)"
elif echo "$versions_response" | grep -qi "just a moment"; then
# Cloudflare or similar CDN challenge page indicating not ready yet
log_debug "Received loading/challenge page (attempt $fetch_attempts/$max_fetch_attempts)"
versions_response=""
else
break
fi

if [ $fetch_attempts -lt $max_fetch_attempts ]; then
log_warning "Server not ready, retrying in ${fetch_delay}s (attempt $fetch_attempts/$max_fetch_attempts)..."
sleep "$fetch_delay"
fi
done

if [ -z "$versions_response" ]; then
log_error "Failed to fetch versions from $url"
log_debug "curl returned empty response"
log_error "Failed to fetch versions from $url after $max_fetch_attempts attempts"
return 1
fi

Expand Down
2 changes: 2 additions & 0 deletions testing/bats/run-package-functional-tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,8 @@ echo "INFO: running test container 'bats/test/$DISTRO'"
-v "$DOWNLOAD_DIR:/downloads:ro" \
-e FLUENT_BIT_BINARY="$FLUENT_BIT_BINARY" \
-e TELEMETRY_FORGE_AGENT_PACKAGE_INSTALLED=true \
-e TELEMETRY_FORGE_AGENT_VERSION="$TELEMETRY_FORGE_AGENT_VERSION" \
-e TELEMETRY_FORGE_AGENT_URL="$TELEMETRY_FORGE_AGENT_URL" \
"bats/test/$DISTRO"

echo "INFO: All tests complete"
4 changes: 2 additions & 2 deletions testing/bats/tests/functional/common/simple.bats
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,13 +24,13 @@ function teardown() {
[[ "$TELEMETRY_FORGE_AGENT_VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]
}

@test "verify version" {
@test "verify reporting $TELEMETRY_FORGE_AGENT_VERSION version" {
run "$FLUENT_BIT_BINARY" --version
[ "$status" -eq 0 ]
[[ "$output" =~ Telemetry\ Forge\ Agent\ v$TELEMETRY_FORGE_AGENT_VERSION ]]
}

@test "verify help" {
@test "verify help works for $FLUENT_BIT_BINARY" {
run "$FLUENT_BIT_BINARY" --help
[ "$status" -eq 0 ]
}
Expand Down
2 changes: 1 addition & 1 deletion testing/bats/tests/integration/linux/integration-install-script.bats
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ setupFile() {

# Test that we can fetch the top-level index
@test "integration: can access index at $TELEMETRY_FORGE_AGENT_URL/index.html" {
response=$(curl -s -o /dev/null -w "%{http_code}" "$TELEMETRY_FORGE_AGENT_URL/index.html")
response=$(curl --max-time 60 -s -o /dev/null -w "%{http_code}" "$TELEMETRY_FORGE_AGENT_URL/index.html")
[ "$response" = "200" ]
}

Expand Down
12 changes: 6 additions & 6 deletions testing/bats/tests/integration/linux/integration-install-targets.bats
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ setupFile() {
local install_script="${repo_root}/install.sh"
assert_file_exist "$install_script"
run ${CONTAINER_RUNTIME:-docker} run --rm -t -v "${install_script}:/install.sh:ro" \
almalinux:9 /bin/sh -c 'yum install -y epel-release && \
almalinux:9 /bin/sh -c 'yum install -yq epel-release && \
/install.sh --debug'
assert_success
assert_output --partial 'Telemetry Forge Agent installation completed successfully!'
Expand All @@ -56,7 +56,7 @@ setupFile() {
local install_script="${repo_root}/install.sh"
assert_file_exist "$install_script"
run ${CONTAINER_RUNTIME:-docker} run --rm -t -v "${install_script}:/install.sh:ro" \
almalinux:10 /bin/sh -c 'yum install -y epel-release && \
almalinux:10 /bin/sh -c 'yum install -yq epel-release && \
/install.sh --debug'
assert_success
assert_output --partial 'Telemetry Forge Agent installation completed successfully!'
Expand All @@ -68,7 +68,7 @@ setupFile() {
local install_script="${repo_root}/install.sh"
assert_file_exist "$install_script"
run ${CONTAINER_RUNTIME:-docker} run --rm -t -v "${install_script}:/install.sh:ro" \
ubuntu:22.04 /bin/sh -c 'apt-get update && apt-get install -y curl && /install.sh --debug'
ubuntu:22.04 /bin/sh -c 'apt-get update -qq && apt-get install -qqy curl && /install.sh --debug'
assert_success
assert_output --partial 'Telemetry Forge Agent installation completed successfully!'
refute_output --partial '[ERROR]'
Expand All @@ -79,7 +79,7 @@ setupFile() {
local install_script="${repo_root}/install.sh"
assert_file_exist "$install_script"
run ${CONTAINER_RUNTIME:-docker} run --rm -t -v "${install_script}:/install.sh:ro" \
ubuntu:24.04 /bin/sh -c 'apt-get update && apt-get install -y curl && /install.sh --debug'
ubuntu:24.04 /bin/sh -c 'apt-get update -qq && apt-get install -qqy curl && /install.sh --debug'
assert_success
assert_output --partial 'Telemetry Forge Agent installation completed successfully!'
refute_output --partial '[ERROR]'
Expand All @@ -90,7 +90,7 @@ setupFile() {
local install_script="${repo_root}/install.sh"
assert_file_exist "$install_script"
run ${CONTAINER_RUNTIME:-docker} run --rm -t -v "${install_script}:/install.sh:ro" \
debian:bookworm /bin/sh -c 'apt-get update && apt-get install -y curl && /install.sh --debug'
debian:bookworm /bin/sh -c 'apt-get update -qq && apt-get install -qqy curl && /install.sh --debug'
assert_success
assert_output --partial 'Telemetry Forge Agent installation completed successfully!'
refute_output --partial '[ERROR]'
Expand All @@ -101,7 +101,7 @@ setupFile() {
local install_script="${repo_root}/install.sh"
assert_file_exist "$install_script"
run ${CONTAINER_RUNTIME:-docker} run --rm -t -v "${install_script}:/install.sh:ro" \
debian:trixie /bin/sh -c 'apt-get update && apt-get install -y curl && /install.sh --debug'
debian:trixie /bin/sh -c 'apt-get update -qq && apt-get install -qqy curl && /install.sh --debug'
assert_success
assert_output --partial 'Telemetry Forge Agent installation completed successfully!'
refute_output --partial '[ERROR]'
Expand Down
Loading