Skip to content

feat: parameterise biossetting to test #54

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
simontesar merged 5 commits into from
Sep 28, 2026
Merged

feat: parameterise biossetting to test #54

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
0c95cae
doc: update BYOBMC
simontesar Sep 25, 2026
9a5db20
fix: remove some explicit timeouts
simontesar Sep 25, 2026
41739e4
refactor: remove 'absent' pattern in favour of null
simontesar Sep 25, 2026
a798423
feat: 06: parameterise biossetting to test
simontesar Sep 21, 2026
0f086b0
refactor: remove 'absent' pattern in favour of null
simontesar Sep 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 91 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,15 +36,103 @@ model: "Standard PC (Q35 + ICH9, 2009)"
firmwareVersion: "1.0.0"
powerState: "On"
biosVersion: "1.0.0"
biosSettingNoRebootValue: "+1-555-0100"
biosSettingNoRebootKey: "AssetTag"
biosSettingNoRebootValue: "compat-test-06"
biosSettingRebootValue: "Bios"
bootOrder: ["Hdd", "Pxe", "Cd"]
bmcSettingKey: "EmailAlert.1.Address"
bmcSettingValue: "alerts@example.com"
```

Adjust the credentials and expectations to your respective values and point the tests to it:

```bash
make test/02-discovery VALUES=/path/to/new/file.yaml
make ASSERT_TIMEOUT=2m VALUES=/path/to/new/file.yaml test/01-bmc-registration
```

You will probably encounter an error like:
```shell
chainsaw test --values /home/user/src/metal-operator-test-framework/values-t160.yaml --parallel 1 --assert-timeout 2m tests/01-bmc-registration
Version: 0.2.15
Loading default configuration...
- Using test file: chainsaw-test
- TestDirs [tests/01-bmc-registration]
- Quiet false
- SkipDelete false
- FailFast false
- Namespace ''
- FastNamespaceDeletion false
- FullName false
- IncludeTestRegex ''
- ExcludeTestRegex ''
- ApplyTimeout 5s
- AssertTimeout 2m0s
- CleanupTimeout 30s
- DeleteTimeout 15s
- ErrorTimeout 30s
- ExecTimeout 5s
- DeletionPropagationPolicy Background
- Parallel 1
- Values [/home/user/src/metal-operator-test-framework/values-t160.yaml]
- Template true
- NoCluster false
- PauseOnFailure false
Loading tests...
- 01-bmc-registration (tests/01-bmc-registration)
Loading values...
Running tests...
=== RUN chainsaw
=== PAUSE chainsaw
=== CONT chainsaw
=== RUN chainsaw/01-bmc-registration
| 06:44:56 | 01-bmc-registration | @chainsaw | CREATE | OK | v1/Namespace @ chainsaw-elegant-puma
| 06:44:56 | 01-bmc-registration | register-bmc | TRY | BEGIN |
| 06:44:56 | 01-bmc-registration | register-bmc | APPLY | RUN | metal.ironcore.dev/v1alpha1/BMCSecret @ compatibility-chainsaw-elegant-puma
| 06:44:56 | 01-bmc-registration | register-bmc | CREATE | OK | metal.ironcore.dev/v1alpha1/BMCSecret @ compatibility-chainsaw-elegant-puma
| 06:44:56 | 01-bmc-registration | register-bmc | APPLY | DONE | metal.ironcore.dev/v1alpha1/BMCSecret @ compatibility-chainsaw-elegant-puma
| 06:44:56 | 01-bmc-registration | register-bmc | APPLY | RUN | metal.ironcore.dev/v1alpha1/BMC @ compatibility-chainsaw-elegant-puma
| 06:44:56 | 01-bmc-registration | register-bmc | CREATE | OK | metal.ironcore.dev/v1alpha1/BMC @ compatibility-chainsaw-elegant-puma
| 06:44:56 | 01-bmc-registration | register-bmc | APPLY | DONE | metal.ironcore.dev/v1alpha1/BMC @ compatibility-chainsaw-elegant-puma
| 06:44:56 | 01-bmc-registration | register-bmc | TRY | END |
| 06:44:56 | 01-bmc-registration | assert-bmc-enabled | TRY | BEGIN |
| 06:44:56 | 01-bmc-registration | assert-bmc-enabled | ASSERT | RUN | metal.ironcore.dev/v1alpha1/BMC @ compatibility-chainsaw-elegant-puma
| 06:46:56 | 01-bmc-registration | assert-bmc-enabled | ASSERT | ERROR | metal.ironcore.dev/v1alpha1/BMC @ compatibility-chainsaw-elegant-puma
=== ERROR
-------------------------------------------------------------------
metal.ironcore.dev/v1alpha1/BMC/compatibility-chainsaw-elegant-puma
-------------------------------------------------------------------
* status.firmwareVersion: Invalid value: "7.30.10.50": Expected value: "1.0.0"
* status.model: Invalid value: "16G Monolithic": Expected value: "Standard PC (Q35 + ICH9, 2009)"

--- expected
+++ actual
@@ -3,8 +3,8 @@
metadata:
name: compatibility-chainsaw-elegant-puma
status:
- firmwareVersion: 1.0.0
- model: Standard PC (Q35 + ICH9, 2009)
+ firmwareVersion: 7.30.10.50
+ model: 16G Monolithic
powerState: "On"
state: Enabled
| 06:46:56 | 01-bmc-registration | assert-bmc-enabled | TRY | END |
| 06:46:56 | 01-bmc-registration | register-bmc | CLEANUP | BEGIN |
| 06:46:56 | 01-bmc-registration | register-bmc | DELETE | OK | metal.ironcore.dev/v1alpha1/BMC @ compatibility-chainsaw-elegant-puma
| 06:46:57 | 01-bmc-registration | register-bmc | DELETE | OK | metal.ironcore.dev/v1alpha1/BMCSecret @ compatibility-chainsaw-elegant-puma
| 06:46:57 | 01-bmc-registration | register-bmc | CLEANUP | END |
| 06:46:57 | 01-bmc-registration | @chainsaw | CLEANUP | BEGIN |
| 06:46:57 | 01-bmc-registration | @chainsaw | DELETE | OK | v1/Namespace @ chainsaw-elegant-puma
| 06:47:02 | 01-bmc-registration | @chainsaw | CLEANUP | END |
--- FAIL: chainsaw (125.32s)
--- FAIL: chainsaw/01-bmc-registration (125.32s)
FAIL
Tests Summary...
- Passed tests 0
- Failed tests 1
- Skipped tests 0
Done with failures.
Error: some tests failed
make: *** [Makefile:59: test/01-bmc-registration] Error 1
```

A mismatch like this means the expectations in your `VALUES` file need to be adjusted to reality.
3 changes: 1 addition & 2 deletions tests/06-biossettings-noreboot/biossettings-noreboot.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,4 @@ spec:
settingsFlow:
- name: admin-phone
priority: 1
settings:
AdminPhone: ($values.biosSettingNoRebootValue)
settings: (from_items([[($values.biosSettingNoRebootKey), ($values.biosSettingNoRebootValue)]]))
3 changes: 1 addition & 2 deletions tests/06-biossettings-noreboot/chainsaw-test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,6 @@ spec:
- name: assert-biossettings-applied
try:
- assert:
timeout: 10m
resource:
apiVersion: metal.ironcore.dev/v1alpha1
kind: BIOSSettings
Expand All @@ -66,4 +65,4 @@ spec:
metadata:
name: (join('-', ['compatibility', $namespace, 'noreboot']))
spec:
(serverMaintenanceRef || 'absent'): 'absent'
(serverMaintenanceRef == null): true
3 changes: 1 addition & 2 deletions tests/07-biossettings-reboot/chainsaw-test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,6 @@ spec:
- name: assert-biossettings-applied
try:
- assert:
timeout: 30m
resource:
apiVersion: metal.ironcore.dev/v1alpha1
kind: BIOSSettings
Expand All @@ -56,6 +55,6 @@ spec:
metadata:
name: (join('-', ['compatibility', $namespace, 'system-0']))
spec:
(serverMaintenanceRef || 'absent'): 'absent'
(serverMaintenanceRef == null): true
status:
state: Available
8 changes: 4 additions & 4 deletions tests/10-bmcversion/chainsaw-test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,18 +72,18 @@ spec:
metadata:
name: (join('-', [$bmcName, 'bmc']))
spec:
(serverMaintenanceRefs || 'absent'): 'absent'
(serverMaintenanceRefs == null): true
status:
(upgradeTask || 'absent'): 'absent'
(conditions || 'absent'): 'absent'
(upgradeTask == null): true
(conditions == null): true
- assert:
resource:
apiVersion: metal.ironcore.dev/v1alpha1
kind: Server
metadata:
name: (join('-', [$bmcName, 'system-0']))
spec:
(serverMaintenanceRef || 'absent'): 'absent'
(serverMaintenanceRef == null): true
status:
state: Available
- assert:
Expand Down
Loading