Skip to content

CI: matrixed ui-smoke-tests job across Java 17,26 × context paths × samples#167

Merged
vharseko merged 3 commits intoOpenIdentityPlatform:masterfrom
vharseko:copilot/matrix-ui-smoke-tests
May 6, 2026
Merged

CI: matrixed ui-smoke-tests job across Java 17,26 × context paths × samples#167
vharseko merged 3 commits intoOpenIdentityPlatform:masterfrom
vharseko:copilot/matrix-ui-smoke-tests

Conversation

@vharseko
Copy link
Copy Markdown
Member

@vharseko vharseko commented May 6, 2026

The Playwright UI smoke tests previously ran inline in build-maven only on Linux, only against the default context path, and only against a vanilla OpenIDM (no samples). Coverage needed to expand across Java versions, context paths, and sample projects without bloating the build job.

Changes to .github/workflows/build.yml

  • Removed the three Linux-only UI steps from build-maven (UI Smoke Tests (Playwright), Start OpenIDM with custom context path /myidm, and the second Playwright run). Test on Unix / Test on Windows startup checks are kept, so per-OS startup validation is unaffected.
  • Added a new ui-smoke-tests job (needs: build-maven, fail-fast: false) with a 4 × 2 × 3 = 24-combination matrix:
    • java: [ '17', '21', '25', '26' ]
    • context_path: [ "", "/myidm" ]
    • samples: [ "", "samples/getting-started", "samples/workflow" ]
    • include entries map each axis value to a slash-free *_label for artifact naming.
  • Job steps: setup-java for the matrix Java → download-artifact of ubuntu-latest-${{ matrix.java }} (built upstream, JRE/bytecode aligned) → unzip → cached Playwright browsers → unified OpenIDM startup that conditionally injects OPENIDM_OPTS (custom context path) and -p <sample> → Playwright run with OPENIDM_CONTEXT_PATH / OPENIDM_SAMPLE env → failure-only artifact upload under failure-ui-java<J>-<context_label>-<samples_label>.
  • Security: pinned actions/download-artifact to v5 to avoid GHSA arbitrary-file-write affecting >= 4.0.0, < 4.1.3.

Unified startup snippet

- name: Start OpenIDM (context_path='${{ matrix.context_path }}', samples='${{ matrix.samples }}')
  run: |
    OPTS=""
    if [ -n "${{ matrix.context_path }}" ]; then
      OPTS="-Dlogback.configurationFile=conf/logging-config.groovy -Dopenidm.context.path=${{ matrix.context_path }}"
    fi
    ARGS=""
    if [ -n "${{ matrix.samples }}" ]; then
      ARGS="-p ${{ matrix.samples }}"
    fi
    OPENIDM_OPTS="$OPTS" openidm/startup.sh $ARGS &
    timeout 3m bash -c 'until grep -q "OpenIDM ready" openidm/logs/openidm0.log.0 ; do sleep 5; done' || cat openidm/logs/openidm0.log.0
    grep -q "OpenIDM ready" openidm/logs/openidm0.log.0
    ! grep "ERROR"  openidm/logs/openidm0.log.0
    ! grep "SEVERE" openidm/logs/openidm0.log.0

Follow-ups worth considering

  • The e2e/ suite reads OPENIDM_CONTEXT_PATH (defaults to /openidm) but does not yet branch on OPENIDM_SAMPLE; sample-specific assertions / skips can be added incrementally.
  • 24 runs per PR is non-trivial; if cost becomes an issue, restrict the full matrix to push on protected branches and trim it for pull_request.

@vharseko vharseko requested a review from maximthomas May 6, 2026 14:25
@vharseko vharseko changed the title CI: matrixed ui-smoke-tests job across Java 17/21/25/26 × context paths × samples CI: matrixed ui-smoke-tests job across Java 17,26 × context paths × samples May 6, 2026
@vharseko vharseko merged commit e337f4c into OpenIdentityPlatform:master May 6, 2026
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants