FIX: Report why a cloud property has no value, and make a broken cloud example go red - #73
Merged
Merged
Conversation
…sts that fail when a cloud example is broken The hardware profile cloud engine threw away the reason the cloud service gives when a resource key is not entitled to a property, so the TAC and native model examples printed nine PHP warnings per lookup and the useless line 'Unknown ()'. The engine now carries each aspect level value and its companion nullreason into every profile, and the example helper reports the reason instead of reading a missing array key. A new unit suite runs the engine and the example helper against fixed cloud responses, so it needs no resource key and no network connection. It runs on every build, which is what makes a broken cloud example show as red. The repository's run-unit-tests script now runs it rather than printing that there are no unit tests. The example tests no longer assert only that some output was produced. They assert the headings the examples print, that no line reads as a programming fault, and that every device line says something useful. The failure to match test now checks what the example printed instead of asserting true. Resource key environment variables move onto the '_51DEGREES_RESOURCE_KEY' convention, with the previous names still read as a fallback so an existing setup keeps working, and a missing key now names the variable that was wanted.
UACHCloudTests and UACHCloudTests_PHP5 call ResourceKeys::find but never imported the class, so PHP looked for it in the tests namespace and the nightly run of this pull request failed with Class "fiftyone\pipeline\devicedetection\tests\ResourceKeys" not found. The other two test files that use it already carry the import.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Three faults, all found by running the cloud examples against a resource
key that has the hardware aspect but is not entitled to the hardware
properties themselves, which is what a customer on a free or partial
subscription has.
1. A missing entitlement was handled badly. The cloud service answers
correctly and says why it has no value, for example
"hardwarevendornullreason": "HardwareVendor is a paid feature. You need a licence key to retrieve data.", butHardwareProfileCloudread onlyhardware.profilesand threw the reasons away. Each profile therefore hadno
hardwarevendorkey at all, soExampleUtils::getHumanReadableread amissing array key and then read two properties off null.
2. Nothing on the dashboard would go red.
ci/run-unit-tests.ps1said"There are no unit tests", so the only tests that touched the cloud
examples were in the integration job, which is skipped whenever the
resource key or the CSV asset is absent. The example tests that did run
asserted only
assertGreaterThan(0, count($output)), which passes whenevery value printed is
Unknown (), andtestFailureToMatchdiscarded theexample's output and asserted
true.3. Environment variable names did not follow the convention. The
examples and tests read
resource_key,AcceptChPlatformKeyand similar,so the keys exported centrally by common-ci
steps/set-resource-keys.ps1under the
_51DEGREES_RESOURCE_KEYprefix never reached them, and a testthat could not find a key did not say which variable it wanted.
What changed
src/HardwareProfileCloud.phpnow builds the aspect level values fromthe cloud response, pairing each null value with its companion
<name>nullreason, and adds them to every profile that does not alreadycarry that property. This matches what the base
CloudEnginein thepipeline library already does for other aspects. A response with no
hardwaresection no longer raises warnings.ExampleUtils::getHumanReadabledistinguishes three cases, being a value,no value with a reason, and a property that is not in the results at all,
and names the property in the last case. It raises no warnings.
ExampleUtils::getProfilesandgetNoProfilesMessagelet the TAC andnative model examples say plainly that no profiles came back, rather than
ending with an uncaught exception when the key has no hardware aspect.
tests/HardwareProfileCloudTest.phpruns the engine andthe example helper against fixed cloud responses through
tests/classes/FakeCloudRequestEngine.php, so it needs no resource key,no licence key and no network connection.
ci/run-unit-tests.ps1runs it,which means a broken cloud example now shows as red on every build.
tests/ExampleTests.phpasserts the headings the examples print, that noline reads as a programming fault, and that every device line says
something useful.
testFailureToMatchasserts what the example printed._51DEGREES_RESOURCE_KEYconvention, in
phpunit.xml,tests/classes/Constants.phpandci/run-integration-tests.ps1. The previous names are still read as afallback through the new
tests/classes/ResourceKeys.php, so an existingsetup keeps working, and a missing key now names the variable that was
wanted. The query string parameter the web examples accept keeps the name
resource_key, because that is part of the URL rather than theenvironment, and it now has its own constant.
Before
Running
php examples/cloud/tacLookupConsole.phpwith a resource key thathas the hardware aspect but no entitlement to the hardware properties, on
PHP 8.5.1 against
cloud.51degrees.com. Sixty three PHP warnings acrossthe two lookups, three per property per profile, from
ExampleUtils.phplines 100, 102 and 110.
The exit code was 0, so nothing anywhere reported a problem.
After
Same key, same command, same machine. No warnings, and the reason the
service gave is printed.
The tests now go red
The new unit suite, against the code as it was before this change (the old
HardwareProfileCloud, with everything else from this branch in place):Against this branch:
The two deprecations come from
mustache/mustache, a dependency of thepipeline core JavaScript builder, and are not raised by this repository.
The strengthened example tests, run against the example code as it was
before this change with the not entitled key:
Against this branch, same key:
phpstan analysereports no errors.Not fixed here, and why
The TLS failure message belongs to another repository. Running the same
example on a machine whose PHP has no CA bundle configured gives this,
which does not mention TLS at all:
HttpClient::makeCloudRequestcallssubstr()on the result ofcurl_execwithout checking forfalse. The real error, fromcurl_error, isSSL certificate OpenSSL verify result: unable to get local issuer certificate (20). That file is in51Degrees/pipeline-php-cloudrequestengine,
not here, so it is raised there as an issue rather than patched in this
pull request.
The organisation secret still has its old name. The workflows now read
secrets._51DEGREES_RESOURCE_KEY_SUPERand fall back tosecrets.SUPER_RESOURCE_KEY, so nothing breaks either way. Renaming theorganisation secret to
_51DEGREES_RESOURCE_KEY_SUPERwould letcommon-ci's central
set-resource-keys.ps1export it, after which thefallback and the legacy variable names can be deleted. That rename needs
organisation access and is not done here.
Integration tests are still gated on the CSV asset.
ci/run-integration-tests.ps1skips every integration test, cloud onesincluded, when
tests/51Degrees.csvis missing. The cloud tests do notneed that file. Splitting the gate is worth doing but is left out of this
change to keep it reviewable.
Produced with AI assistance. Every claim above was checked by running the
commands shown, and the output is real rather than illustrative. It needs
human review before merging.