Skip to content

test: various fixes to package testing#229

Merged
patrick-stephens merged 8 commits intomainfrom
fixup_testing_mar2026
Mar 19, 2026
Merged

test: various fixes to package testing#229
patrick-stephens merged 8 commits intomainfrom
fixup_testing_mar2026

Conversation

@patrick-stephens
Copy link
Contributor

@patrick-stephens patrick-stephens commented Mar 19, 2026

Resolves various test failures seen during recent builds and releases.

  • Version value was not being passed through correctly in some cases
  • Install script can fail if index page is still working on providing versions
  • Too much debug from installing helper things on some platforms
  • Curl hitting timeouts as well for accessing the index page

Summary by cubic

Stabilizes package and container tests by fixing version env propagation and installer retries, and corrects tag handling for PR builds. Also generates JSON schema artifacts from container images during build (non-blocking).

  • Bug Fixes

    • Install script now retries version index fetches (up to 5 attempts with 10s backoff) and ignores CDN “just a moment” pages before failing.
    • Use a 60s curl timeout for index checks in the installer and integration tests.
    • Propagate version env correctly: set TELEMETRY_FORGE_AGENT_VERSION (and legacy FLUENTDO_AGENT_VERSION) from the image tag in the workflow, default version to tag in test runners, and fix tag usage for PR builds.
  • Test Updates

    • Reduce noise in container installs (yum -yq, apt-get -qqy).
    • Pass TELEMETRY_FORGE_AGENT_VERSION and TELEMETRY_FORGE_AGENT_URL into functional test containers.
    • Clarify Bats test names to include version/binary for easier debugging.

Written for commit 89f3891. Summary will update on new commits.

Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
@patrick-stephens patrick-stephens requested a review from a team as a code owner March 19, 2026 16:00
@patrick-stephens patrick-stephens added the build-linux Option to enable building of Linux packages only for a PR to test label Mar 19, 2026
Copy link

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 7 files

Confidence score: 3/5

  • There is a concrete reliability risk in install.sh: without curl timeouts, the current retry loop can still block indefinitely before retries, which can leave installs hanging for users.
  • In testing/bats/tests/integration/linux/integration-install-script.bats, using --connect-timeout alone does not cover slow response bodies, so tests may miss the same hang condition once a connection is established.
  • This is a moderate, user-impacting timeout behavior issue (with high confidence), so merge risk is not low until total request timeouts are applied consistently.
  • Pay close attention to install.sh and testing/bats/tests/integration/linux/integration-install-script.bats - timeout handling should cover the full request lifecycle, not just connection setup.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="testing/bats/tests/integration/linux/integration-install-script.bats">

<violation number="1" location="testing/bats/tests/integration/linux/integration-install-script.bats:38">
P2: `--connect-timeout` only covers TCP connection setup here; it will not help when `/index.html` is slow to return after the connection is established. Use a total request timeout instead.</violation>
</file>

<file name="install.sh">

<violation number="1" location="install.sh:378">
P2: Add curl timeouts here; otherwise this retry loop can still hang forever before any retry happens.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
@patrick-stephens
Copy link
Contributor Author

@cubic-dev-ai please re-review

@cubic-dev-ai
Copy link

cubic-dev-ai bot commented Mar 19, 2026

@cubic-dev-ai please re-review

@patrick-stephens I have started the AI code review. It will take a few minutes to complete.

Copy link

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 7 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

@patrick-stephens patrick-stephens removed the build-linux Option to enable building of Linux packages only for a PR to test label Mar 19, 2026
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
@patrick-stephens patrick-stephens merged commit ef02525 into main Mar 19, 2026
14 checks passed
@patrick-stephens patrick-stephens deleted the fixup_testing_mar2026 branch March 19, 2026 16:49
Copy link

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 3 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name=".github/workflows/call-test-containers.yaml">

<violation number="1" location=".github/workflows/call-test-containers.yaml:90">
P2: The tag-only `if` condition prevents setting version env vars for non-tag runs, so container tests can validate against the wrong fallback version instead of `inputs.image-tag`.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

shell: bash

- name: Ensure we use the right version during a release
if: startsWith(github.ref, 'refs/tags/v')
Copy link

@cubic-dev-ai cubic-dev-ai bot Mar 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The tag-only if condition prevents setting version env vars for non-tag runs, so container tests can validate against the wrong fallback version instead of inputs.image-tag.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At .github/workflows/call-test-containers.yaml, line 90:

<comment>The tag-only `if` condition prevents setting version env vars for non-tag runs, so container tests can validate against the wrong fallback version instead of `inputs.image-tag`.</comment>

<file context>
@@ -88,6 +86,13 @@ jobs:
         shell: bash
 
+      - name: Ensure we use the right version during a release
+        if: startsWith(github.ref, 'refs/tags/v')
+        run: |
+          echo "TELEMETRY_FORGE_AGENT_VERSION=${{ inputs.image-tag }}" >> $GITHUB_ENV
</file context>
Suggested change
if: startsWith(github.ref, 'refs/tags/v')
if: ${{ always() }}
Fix with Cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant