dcuc: update dasharo-sdk to v1.7.0

[m-iwanicki]

v1.5.0 used up around 10 GB of space.
Not sure what's wrong with workflow, when used manually it works at it always worked.

At least one problem is with missing wget (also no curl). We could sidestep it by running as root and installing it then su to coreboot user.
Or keep using the same sdk version as before in workflow but use v1.7.0 in dcuc.

[macpijan]

I guess there are some relevant changes in the v1.7.0 container in terms of users, would need to check that. Maybe the default USER set in the end is different than before?

[philipanda]

works for me.
I wonder how the same container has wget on my system, but does not have it in GH CI.

Details

fgolas in ~/3mdeb/dcu on update-dasharo-sdk ● λ bash test/approve-ci
download protectli_vp46xx_v1.2.0.rom
Saving './data/dl/protectli_vp46xx_v1.2.0.rom'
HTTP response 200  [https://dl.3mdeb.com/open-source-firmware/Dasharo/protectli_vault_cml/v1.2.0/protectli_vp46xx_v1.2.0.rom]
./data/dl/protectli_ 100% [=======================================================================================================>]   16.00M   13.75MB/s
                          [Files: 1  Bytes: 16.00M [10.04MB/s] Redirects: 0  Todo: 0  Errors: 0                                    ]
download novacustom_nv4x_adl_v1.6.0.rom
HTTP response 302  [https://3mdeb.com/open-source-firmware/Dasharo/novacustom_nv4x_adl/v1.6.0/novacustom_nv4x_adl_v1.6.0.rom]
Enqueue https://dl.3mdeb.com/open-source-firmware/Dasharo/novacustom_nv4x_adl/v1.6.0/novacustom_nv4x_adl_v1.6.0.rom
Saving './data/dl/novacustom_nv4x_adl_v1.6.0.rom'
HTTP response 200  [https://dl.3mdeb.com/open-source-firmware/Dasharo/novacustom_nv4x_adl/v1.6.0/novacustom_nv4x_adl_v1.6.0.rom]
./data/dl/novacustom 100% [=======================================================================================================>]   32.00M   25.94MB/s
                          [Files: 1  Bytes: 32.00M [17.85MB/s] Redirects: 1  Todo: 0  Errors: 0                                    ]
download protectli_vault_kbl_v1.0.14.rom
HTTP response 302  [https://3mdeb.com/open-source-firmware/Dasharo/protectli_vault_kbl/v1.0.14/protectli_vault_kbl_v1.0.14.rom]
Enqueue https://dl.3mdeb.com/open-source-firmware/Dasharo/protectli_vault_kbl/v1.0.14/protectli_vault_kbl_v1.0.14.rom
Saving './data/dl/protectli_vault_kbl_v1.0.14.rom'
HTTP response 200  [https://dl.3mdeb.com/open-source-firmware/Dasharo/protectli_vault_kbl/v1.0.14/protectli_vault_kbl_v1.0.14.rom]
./data/dl/protectli_ 100% [=======================================================================================================>]    8.00M    --.-KB/s
                          [Files: 1  Bytes: 8.00M [9.85MB/s] Redirects: 1  Todo: 0  Errors: 0                                      ]
download bootsplash.bmp
Saving './data/dl/bootsplash.bmp'
HTTP response 200  [https://raw.githubusercontent.com/Dasharo/dasharo-blobs/main/dasharo/bootsplash.bmp]
./data/dl/bootsplash 100% [=======================================================================================================>]  541.45K    --.-KB/s
                          [Files: 1  Bytes: 541.45K [1.25MB/s] Redirects: 0  Todo: 0  Errors: 0                                    ]
download dummy.pdf
Saving './data/dl/dummy.pdf'
HTTP response 200  [https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf]
./data/dl/dummy.pdf  100% [=======================================================================================================>]   12.95K    --.-KB/s
                          [Files: 1  Bytes: 12.95K [100.41KB/s] Redirects: 0  Todo: 0  Errors: 0                                   ]
download novacustom_v540tu_mtl_v0.9.0.rom
HTTP response 301 Moved Permanently [http://dl.3mdeb.com/open-source-firmware/Dasharo/novacustom_v54x_mtl/v0.9.0/novacustom_v54x_mtl_v0.9.0.rom]
Enqueue https://dl.3mdeb.com/open-source-firmware/Dasharo/novacustom_v54x_mtl/v0.9.0/novacustom_v54x_mtl_v0.9.0.rom
Saving './data/dl/novacustom_v540tu_mtl_v0.9.0.rom'
HTTP response 200  [https://dl.3mdeb.com/open-source-firmware/Dasharo/novacustom_v54x_mtl/v0.9.0/novacustom_v54x_mtl_v0.9.0.rom]
./data/dl/novacustom 100% [=======================================================================================================>]   32.00M   28.83MB/s
                          [Files: 1  Bytes: 32.00M [20.04MB/s] Redirects: 1  Todo: 0  Errors: 0                                    ]
download gbe.bin
Saving './data/dl/gbe.bin'
HTTP response 200  [https://raw.githubusercontent.com/Dasharo/dasharo-blobs/main/novacustom/v5x0tu/gbe.bin]
./data/dl/gbe.bin    100% [=======================================================================================================>]    8.00K    --.-KB/s
                          [Files: 1  Bytes: 8.00K [27.97KB/s] Redirects: 0  Todo: 0  Errors: 0                                     ]
Setting up test data...
▌ describe   help message
  approved   ../dcuc --help
▌ describe   Try to replace logo in binary supporting that
  approved   ../dcuc logo ./data/work/protectli_vp46xx_v1.2.0.rom  -l ./data/work/bootsplash.bmp
▌ describe   Try to replace logo in binary NOT supporting that
  approved   ../dcuc logo ./data/work/protectli_vault_kbl_v1.0.14.rom -l ./data/work/bootsplash.bmp
▌ describe   Try to replace serial in binary supporting that
  approved   ../dcuc smbios ./data/work/protectli_vp46xx_v1.2.0.rom  -s D07229051
▌ describe   Try to replace serial in binary NOT supporting that
  approved   ../dcuc smbios ./data/work/novacustom_nv4x_adl_v1.6.0.rom -s D07229051
▌ describe   Try to replace UUID in binary supporting that
  approved   ../dcuc smbios ./data/work/protectli_vp46xx_v1.2.0.rom  -u 96bcfa1a-42b4-6717-a44c-d8bbc18cbea4
▌ describe   Try to replace UUID in binary NOT supporting that
  approved   ../dcuc smbios ./data/work/novacustom_nv4x_adl_v1.6.0.rom -u 96bcfa1a-42b4-6717-a44c-d8bbc18cbea4
▌ describe   Try to replace both serial and UUID in binary supporting that
  approved   ../dcuc smbios ./data/work/protectli_vp46xx_v1.2.0.rom  -s D07229051 -u 96bcfa1a-42b4-6717-a44c-d8bbc18cbea4
▌ describe   Try to replace both serial and UUID in binary NOT supporting that
  approved   ../dcuc smbios ./data/work/novacustom_nv4x_adl_v1.6.0.rom -s D07229051 -u 96bcfa1a-42b4-6717-a44c-d8bbc18cbea4
▌ describe   Try to replace logo using unsupported format (PDF)
  approved   ../dcuc logo ./data/work/protectli_vp46xx_v1.2.0.rom  -l ./data/work/dummy.pdf
▌ describe   Try to replace UUID using incorrect format
  approved   ../dcuc smbios ./data/work/protectli_vp46xx_v1.2.0.rom  -u dead-beef
▌ describe   Get variable from uninitialized store
  approved   ../dcuc variable ./data/work/novacustom_nv4x_adl_v1.6.0.rom --get SerialRedirection
▌ describe   Initialize variable store in a release binary
  approved   ../dcuc variable ./data/work/novacustom_nv4x_adl_v1.6.0.rom --set SerialRedirection --value Enabled
▌ describe   Set supported bool variable to Disabled
  approved   ../dcuc variable ./data/work/SerialRedirectionDisabled_novacustom_nv4x_adl_v1.6.0.rom --set SerialRedirection --value Disabled
▌ describe   Get supported bool variable (expecting Disabled)
  approved   ../dcuc variable ./data/work/SerialRedirectionDisabled_novacustom_nv4x_adl_v1.6.0.rom --get SerialRedirection
▌ describe   Set supported bool variable to Enabled
  approved   ../dcuc variable ./data/work/SerialRedirectionEnabled_novacustom_nv4x_adl_v1.6.0.rom --set SerialRedirection --value Enabled
▌ describe   Get supported bool variable (expecting Enabled)
  approved   ../dcuc variable ./data/work/SerialRedirectionEnabled_novacustom_nv4x_adl_v1.6.0.rom --get SerialRedirection
▌ describe   Set supported bool variable to invalid value
  approved   ../dcuc variable ./data/work/SerialRedirectionEnabled_novacustom_nv4x_adl_v1.6.0.rom --set SerialRedirection --value EnabledInvalid
▌ describe   Set MeMode to Enabled
  approved   ../dcuc variable ./data/work/MeModeEnabled_novacustom_nv4x_adl_v1.6.0.rom --set MeMode --value Enabled
▌ describe   Get MeMode (expecting Enabled)
  approved   ../dcuc variable ./data/work/MeModeEnabled_novacustom_nv4x_adl_v1.6.0.rom --get MeMode
▌ describe   Set MeMode to Disabled (Soft)
  approved   ../dcuc variable ./data/work/MeModeDisabledSoft_novacustom_nv4x_adl_v1.6.0.rom --set MeMode --value "Disabled (Soft)"
▌ describe   Get MeMode (expecting Disabled (Soft))
  approved   ../dcuc variable ./data/work/MeModeDisabledSoft_novacustom_nv4x_adl_v1.6.0.rom --get MeMode
▌ describe   Set MeMode to Disabled (HAP)
  approved   ../dcuc variable ./data/work/MeModeDisabledHAP_novacustom_nv4x_adl_v1.6.0.rom --set MeMode --value "Disabled (HAP)"
▌ describe   Get MeMode (expecting Disabled (HAP))
  approved   ../dcuc variable ./data/work/MeModeDisabledHAP_novacustom_nv4x_adl_v1.6.0.rom --get MeMode
▌ describe   Set MeMode to InvalidMode
  approved   ../dcuc variable ./data/work/MeModeDisabledHAP_novacustom_nv4x_adl_v1.6.0.rom --set MeMode --value InvalidMode
▌ describe   Set FanCurveOption to Silent
  approved   ../dcuc variable ./data/work/FanCurveSilent_novacustom_nv4x_adl_v1.6.0.rom --set FanCurveOption --value Silent
▌ describe   Get FanCurveMode (expecting Silent)
  approved   ../dcuc variable ./data/work/FanCurveSilent_novacustom_nv4x_adl_v1.6.0.rom --get FanCurveOption
▌ describe   Set FanCurveOption to Performance
  approved   ../dcuc variable ./data/work/FanCurvePerformance_novacustom_nv4x_adl_v1.6.0.rom --set FanCurveOption --value Performance
▌ describe   Get FanCurveMode (expecting Performance)
  approved   ../dcuc variable ./data/work/FanCurvePerformance_novacustom_nv4x_adl_v1.6.0.rom --get FanCurveOption
▌ describe   Set FanCurveMode to InvalidMode
  approved   ../dcuc variable ./data/work/FanCurvePerormance_novacustom_nv4x_adl_v1.6.0.rom --set FanCurveOption --value InvalidMode
▌ describe   Set unsupported variable
  approved   ../dcuc variable ./data/work/novacustom_nv4x_adl_v1.6.0.rom --set SerialRedirectionInvalid --value Enabled
▌ describe   Set valid MAC
  approved   ../dcuc mac ./data/work/novacustom_v540tu_mtl_v0.9.0_gbe.rom --set 12:34:56:78:9A:BC
▌ describe   Set invalid MAC
  approved   ../dcuc mac ./data/work/novacustom_v540tu_mtl_v0.9.0_gbe.rom --set 00:00:00:00:00:00
▌ describe   Get MAC (should be set by 'Set valid MAC' and not set by 'Set invalid MAC')
  approved   ../dcuc mac ./data/work/novacustom_v540tu_mtl_v0.9.0_gbe.rom
▌ describe   Get MAC Bad checksums
  approved   ../dcuc mac ./data/work/novacustom_v540tu_mtl_v0.9.0.rom
▌ describe   Set MAC Bad checksums
  approved   ../dcuc mac ./data/work/novacustom_v540tu_mtl_v0.9.0.rom --set 12:34:56:78:9A:BC
▌ describe   MAC Unsupported binary
  approved   ../dcuc mac ./data/work/protectli_vp46xx_v1.2.0.rom
▌ exit       ./test/approve finished successfully

I see, approve-ci runs it locally, not in the SDK. It was done this way because we want to avoid multilayer conteainers. CI has to run a container, and we dont want to run a container that will then start dasharo-sdk via dcuc. So we assumed dasharo-sdk will contain all the tools needed to download the file dependencies, but it doesn't. we need to update the CI to first load the deps using another container, and only then run only the tests in dasharo-sdk.

I think some hacks could be done to make sure it works on the newest dasharo-sdk, but maybe it's better to just include wget and file in it? Dasharo/dasharo-sdk#16

not reviewed thoroughly

[miczyg1]

I guess there are some relevant changes in the v1.7.0 container in terms of users, would need to check that. Maybe the default USER set in the end is different than before?

Yes, there were problems with user ID in the newer container. It caused me a lto of failures in our coreboot CI. I believe now the CI user is 1001, while the container had 1000

Here is what I discovered: Dasharo/coreboot@00b34cf (look at commit message)