Skip to content

Add weekly disconnected FAR E2E periodic job for medik8s/system-tests#80494

Open
ugreener wants to merge 1 commit into
openshift:mainfrom
ugreener:fix/far-disconnected-periodic-job
Open

Add weekly disconnected FAR E2E periodic job for medik8s/system-tests#80494
ugreener wants to merge 1 commit into
openshift:mainfrom
ugreener:fix/far-disconnected-periodic-job

Conversation

@ugreener

@ugreener ugreener commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Add a weekly disconnected E2E test job for the FAR (fence-agents-remediation) operator in the medik8s/system-tests Prow config. The job runs every Sunday at 10:00 UTC on an air-gapped AWS cluster, mirroring operator images via the medik8s-disconnected-catalogsource step before installing and testing FAR.

Changes

  • New file: ci-operator/config/medik8s/system-tests/medik8s-system-tests-main__4.22-disconnected.yaml
    • Periodic job e2e-far-weekly-aws-disconnected using openshift-e2e-aws-disconnected workflow
    • Uses medik8s-disconnected-catalogsource for image mirroring in air-gapped environment
    • Uses medik8s-operator-subscribe for OLM installation from mirrored catalog
    • Cron: 0 10 * * 0 (staggered from connected FAR at 04:00, connected SBR at 06:00)

Jira: RHWA-1038

Summary by CodeRabbit

This PR extends the OpenShift CI system-tests configuration for the medik8s/system-tests repository by adding a new periodic disconnected E2E job to validate the FAR (fence-agents-remediation) operator in air-gapped environments.

What’s being changed

  • Adds ci-operator/config/medik8s/system-tests/medik8s-system-tests-main__4.22-disconnected.yaml
  • Introduces a new periodic job: e2e-far-weekly-aws-disconnected

How it works

  • Runs weekly on Sundays at 10:00 UTC (0 10 * * 0) to avoid overlap with related periodic coverage
  • Uses the openshift-e2e-aws-disconnected workflow appropriate for disconnected AWS clusters
  • Installs/validates the FAR operator by:
    • Mirroring required operator images into the disconnected environment using medik8s-disconnected-catalogsource
    • Subscribing/installing the operator via OLM from the mirrored catalog using medik8s-operator-subscribe
  • Sets FAR-focused environment/configuration (including ECO_TEST_FEATURES wiring) and targets the intended OpenShift test refs/channel

Why it matters

  • Ensures the FAR operator can be successfully installed and exercised in environments where external registries are unreachable, aligning with RHWA-1038 requirements for disconnected validation.

CI workflow/testing automation notes

  • The author requested /pj-rehearse auto-ack twice to rehearse the job changes in CI prior to merging, with automatic acknowledgement after successful completion.

@ugreener

Copy link
Copy Markdown
Contributor Author

/pj-rehearse auto-ack

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@ugreener: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@coderabbitai

coderabbitai Bot commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@ugreener, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 6 minutes and 13 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: d0bad979-df36-42d1-a8d3-aa08fa303335

📥 Commits

Reviewing files that changed from the base of the PR and between 40d3dfd and 3f38f42.

⛔ Files ignored due to path filters (1)
  • ci-operator/jobs/medik8s/system-tests/medik8s-system-tests-main-periodics.yaml is excluded by !ci-operator/jobs/**
📒 Files selected for processing (1)
  • ci-operator/config/medik8s/system-tests/medik8s-system-tests-main__4.22-disconnected.yaml

Walkthrough

A new ci-operator YAML configuration file is added for the Medik8s system-tests repository targeting OpenShift 4.22 in disconnected mode. It specifies the UPI installer base image, nightly release candidate, wildcard resource requests, and a scheduled weekly E2E AWS disconnected test job.

Changes

Medik8s 4.22 Disconnected CI Configuration

Layer / File(s) Summary
4.22 disconnected system-test CI config
ci-operator/config/medik8s/system-tests/medik8s-system-tests-main__4.22-disconnected.yaml
New file defining the full ci-operator configuration: UPI installer base image, nightly 4.22 release candidate targeting, wildcard resource request defaults, and a scheduled e2e-far-weekly-aws-disconnected test job with environment variables and workflow invocation.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

🚥 Pre-merge checks | ✅ 15
✅ Passed checks (15 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately and specifically describes the main change: adding a weekly disconnected FAR E2E periodic job for medik8s/system-tests.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed PR adds a CI configuration file (YAML), not Ginkgo tests. The check for "Stable and Deterministic Test Names" applies to Ginkgo test definitions (It(), Describe(), etc.), not CI job configurations....
Test Structure And Quality ✅ Passed PR adds only CI/CD configuration (YAML file), not Ginkgo test code. Custom check for Ginkgo test quality is not applicable to configuration files.
Microshift Test Compatibility ✅ Passed No new Ginkgo e2e tests are added in this PR. The change is a CI configuration file (YAML) that defines a periodic job, not test code, so the MicroShift compatibility check is not applicable.
Single Node Openshift (Sno) Test Compatibility ✅ Passed PR only adds a Prow CI configuration YAML file without any new Ginkgo e2e test code (It, Describe, Context, When). SNO compatibility check does not apply.
Topology-Aware Scheduling Compatibility ✅ Passed The PR adds only a CI configuration file (ci-operator config YAML) for Prow CI/CD, not deployment manifests, operator code, or controllers. Topology-aware scheduling checks apply to production work...
Ote Binary Stdout Contract ✅ Passed PR adds only YAML CI configuration, not test binary code. OTE Stdout Contract check applies only to Go test binaries with process-level stdout writes; YAML configs are not subject to this check.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed This PR adds only a Prow CI configuration YAML file, not Ginkgo e2e test code. The check is inapplicable as it specifically targets new Ginkgo tests (It(), Describe(), etc.).
No-Weak-Crypto ✅ Passed The PR adds only a YAML CI configuration file (51 lines) defining test job settings. No cryptographic code, weak crypto algorithms (MD5, SHA1, DES, RC4, 3DES, Blowfish, ECB), custom crypto implemen...
Container-Privileges ✅ Passed The new YAML configuration file contains no privileged container settings (privileged: true, hostPID, hostNetwork, hostIPC, SYS_ADMIN, runAsUser: 0, or allowPrivilegeEscalation: true).
No-Sensitive-Data-In-Logs ✅ Passed File contains only standard CI configuration values. Environment variables reference test infrastructure domains (devcluster.openshift.com) and non-sensitive operator/package names already used thr...

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci openshift-ci Bot requested review from clobrano and razo7 June 14, 2026 06:59
@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 14, 2026
@openshift-merge-bot openshift-merge-bot Bot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Jun 14, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@ugreener: no rehearsable tests are affected by this change

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
ci-operator/config/medik8s/system-tests/medik8s-system-tests-main__4.22-disconnected.yaml (1)

1-51: ⚠️ Potential issue | 🔴 Critical

Rename this file to follow the variant periodic configuration pattern and ensure generated job files are created.

Per CLAUDE.md guidance, files containing only periodic tests with cron: or interval: scheduling must use the __periodics.yaml suffix pattern. This file should be renamed from medik8s-system-tests-main__4.22-disconnected.yaml to medik8s-system-tests-main-4.22-disconnected__periodics.yaml to match the documented convention for release-specific periodic configurations.

Additionally, verify that running make update generates the corresponding periodic job file in ci-operator/jobs/medik8s/system-tests/. The periodic job for e2e-far-weekly-aws-disconnected should appear in a separate -periodics.yaml file.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/config/medik8s/system-tests/medik8s-system-tests-main__4.22-disconnected.yaml`
around lines 1 - 51, The configuration file contains a periodic test
(e2e-far-weekly-aws-disconnected) with cron scheduling but does not follow the
required naming convention for periodic configurations. Rename the file from
medik8s-system-tests-main__4.22-disconnected.yaml to
medik8s-system-tests-main-4.22-disconnected__periodics.yaml to match the
documented convention where the variant segment uses a hyphen instead of double
underscore and __periodics is appended before the file extension. After
renaming, run make update to verify that the corresponding periodic job file is
generated in the ci-operator/jobs/medik8s/system-tests/ directory with the
appropriate -periodics.yaml suffix.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@ci-operator/config/medik8s/system-tests/medik8s-system-tests-main__4.22-disconnected.yaml`:
- Around line 38-40: The inline test step contains an env block that declares
ECO_TEST_FEATURES by name only without a value, which is not the correct
ci-operator pattern. Environment variables are automatically propagated through
the test hierarchy from workflows and test definitions to their subordinate
stages and steps. Remove the entire env block (containing the name:
ECO_TEST_FEATURES declaration and from: src) from the test step configuration,
or if the variable must be explicitly passed to the container, replace it with a
proper dependencies mechanism reference instead of the bare name declaration.

---

Outside diff comments:
In
`@ci-operator/config/medik8s/system-tests/medik8s-system-tests-main__4.22-disconnected.yaml`:
- Around line 1-51: The configuration file contains a periodic test
(e2e-far-weekly-aws-disconnected) with cron scheduling but does not follow the
required naming convention for periodic configurations. Rename the file from
medik8s-system-tests-main__4.22-disconnected.yaml to
medik8s-system-tests-main-4.22-disconnected__periodics.yaml to match the
documented convention where the variant segment uses a hyphen instead of double
underscore and __periodics is appended before the file extension. After
renaming, run make update to verify that the corresponding periodic job file is
generated in the ci-operator/jobs/medik8s/system-tests/ directory with the
appropriate -periodics.yaml suffix.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 573e1ca3-1341-4a9a-9e03-1b257b2c0f5d

📥 Commits

Reviewing files that changed from the base of the PR and between 9b83f98 and 8c8880b.

📒 Files selected for processing (1)
  • ci-operator/config/medik8s/system-tests/medik8s-system-tests-main__4.22-disconnected.yaml

@ugreener ugreener changed the title Add FAR disconnected periodic job config for system-tests Add weekly disconnected FAR E2E periodic job for medik8s/system-tests Jun 14, 2026
@ugreener ugreener force-pushed the fix/far-disconnected-periodic-job branch 2 times, most recently from f7a1566 to c60a92d Compare June 14, 2026 07:25
@openshift-merge-bot openshift-merge-bot Bot removed the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Jun 14, 2026
@ugreener ugreener force-pushed the fix/far-disconnected-periodic-job branch from c60a92d to 40d3dfd Compare June 14, 2026 07:36
@ugreener

Copy link
Copy Markdown
Contributor Author

/pj-rehearse auto-ack

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@ugreener: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@ugreener, pj-rehearse: unable prepare a candidate for rehearsal; rehearsals will not be run. This could be due to a branch that needs to be rebased. ERROR:

couldn't checkout base SHA 0d7e49ceb83604e6f1236687ba4b7eaeab263e73: error checking out "0d7e49ceb83604e6f1236687ba4b7eaeab263e73": exit status 128 fatal: unable to read tree (0d7e49ceb83604e6f1236687ba4b7eaeab263e73)

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@ugreener, pj-rehearse: unable to determine affected jobs. This could be due to a branch that needs to be rebased. ERROR:

couldn't prepare candidate: couldn't checkout base SHA 0d7e49ceb83604e6f1236687ba4b7eaeab263e73: error checking out "0d7e49ceb83604e6f1236687ba4b7eaeab263e73": exit status 128 fatal: unable to read tree (0d7e49ceb83604e6f1236687ba4b7eaeab263e73)

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@ugreener

Copy link
Copy Markdown
Contributor Author

/pj-rehearse periodic-ci-medik8s-system-tests-main-4.22-disconnected-e2e-far-weekly-aws-disconnected

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@ugreener: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

Add weekly disconnected E2E test job for the FAR (fence-agents-remediation)
operator. The job mirrors operator images to a private registry using the
medik8s-disconnected-catalogsource step (PR openshift#79687), then installs FAR via
OLM and runs the E2E suite on an air-gapped AWS cluster.

Schedule: Sunday 10:00 UTC (staggered after connected FAR at 04:00,
connected SBR at 06:00, and disconnected SBR at 08:00).

Jira: RHWA-1038

Co-Authored-By: Claude <noreply@anthropic.com>
@ugreener ugreener force-pushed the fix/far-disconnected-periodic-job branch from 40d3dfd to 3f38f42 Compare June 14, 2026 07:53
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

[REHEARSALNOTIFIER]
@ugreener: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name Repo Type Reason
periodic-ci-medik8s-system-tests-main-4.22-disconnected-e2e-far-weekly-aws-disconnected N/A periodic Periodic changed
Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@ugreener

Copy link
Copy Markdown
Contributor Author

/pj-rehearse periodic-ci-medik8s-system-tests-main-4.22-disconnected-e2e-far-weekly-aws-disconnected

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@ugreener: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-ci

openshift-ci Bot commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

@ugreener: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/rehearse/periodic-ci-medik8s-system-tests-main-4.22-disconnected-e2e-far-weekly-aws-disconnected 3f38f42 link unknown /pj-rehearse periodic-ci-medik8s-system-tests-main-4.22-disconnected-e2e-far-weekly-aws-disconnected

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@razo7 razo7 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Configuration is correct and follows established medik8s CI patterns.
There is an infrastructure issue (SSH key format in cluster profile), not a config problem, that I would like to see fixed before merging the PR (and use the PR to test it first).

I won't block the PR completely, so please unblock it if we can't wait.
/hold

- as: e2e-far-weekly-aws-disconnected
capabilities:
- intranet
cron: 0 10 * * 0

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like the rehearsal failure (mirror-images-check-registry-service) is caused by an SSH private key format issue in the medik8s-aws cluster profile:

Load key "/var/run/.../ssh-privatekey": invalid format
Permission denied (publickey,...)

@openshift-ci openshift-ci Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 15, 2026
@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jun 15, 2026
@openshift-ci

openshift-ci Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: razo7, ugreener

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants