Skip to content

FIX: Report why a cloud property has no value, and make a broken cloud example go red - #73

Merged
Automation51D merged 2 commits into
mainfrom
fix/cloud-example-failures
Sep 6, 2026
Merged

FIX: Report why a cloud property has no value, and make a broken cloud example go red#73
Automation51D merged 2 commits into
mainfrom
fix/cloud-example-failures

Conversation

@jwrosewell

Copy link
Copy Markdown
Contributor

Why

Three faults, all found by running the cloud examples against a resource
key that has the hardware aspect but is not entitled to the hardware
properties themselves, which is what a customer on a free or partial
subscription has.

1. A missing entitlement was handled badly. The cloud service answers
correctly and says why it has no value, for example
"hardwarevendornullreason": "HardwareVendor is a paid feature. You need a licence key to retrieve data.", but HardwareProfileCloud read only
hardware.profiles and threw the reasons away. Each profile therefore had
no hardwarevendor key at all, so ExampleUtils::getHumanReadable read a
missing array key and then read two properties off null.

2. Nothing on the dashboard would go red. ci/run-unit-tests.ps1 said
"There are no unit tests", so the only tests that touched the cloud
examples were in the integration job, which is skipped whenever the
resource key or the CSV asset is absent. The example tests that did run
asserted only assertGreaterThan(0, count($output)), which passes when
every value printed is Unknown (), and testFailureToMatch discarded the
example's output and asserted true.

3. Environment variable names did not follow the convention. The
examples and tests read resource_key, AcceptChPlatformKey and similar,
so the keys exported centrally by common-ci
steps/set-resource-keys.ps1
under the _51DEGREES_RESOURCE_KEY prefix never reached them, and a test
that could not find a key did not say which variable it wanted.

What changed

  • src/HardwareProfileCloud.php now builds the aspect level values from
    the cloud response, pairing each null value with its companion
    <name>nullreason, and adds them to every profile that does not already
    carry that property. This matches what the base CloudEngine in the
    pipeline library already does for other aspects. A response with no
    hardware section no longer raises warnings.
  • ExampleUtils::getHumanReadable distinguishes three cases, being a value,
    no value with a reason, and a property that is not in the results at all,
    and names the property in the last case. It raises no warnings.
  • ExampleUtils::getProfiles and getNoProfilesMessage let the TAC and
    native model examples say plainly that no profiles came back, rather than
    ending with an uncaught exception when the key has no hardware aspect.
  • New unit suite tests/HardwareProfileCloudTest.php runs the engine and
    the example helper against fixed cloud responses through
    tests/classes/FakeCloudRequestEngine.php, so it needs no resource key,
    no licence key and no network connection. ci/run-unit-tests.ps1 runs it,
    which means a broken cloud example now shows as red on every build.
  • tests/ExampleTests.php asserts the headings the examples print, that no
    line reads as a programming fault, and that every device line says
    something useful. testFailureToMatch asserts what the example printed.
  • Resource key variables move onto the _51DEGREES_RESOURCE_KEY
    convention, in phpunit.xml, tests/classes/Constants.php and
    ci/run-integration-tests.ps1. The previous names are still read as a
    fallback through the new tests/classes/ResourceKeys.php, so an existing
    setup keeps working, and a missing key now names the variable that was
    wanted. The query string parameter the web examples accept keeps the name
    resource_key, because that is part of the URL rather than the
    environment, and it now has its own constant.

Before

Running php examples/cloud/tacLookupConsole.php with a resource key that
has the hardware aspect but no entitlement to the hardware properties, on
PHP 8.5.1 against cloud.51degrees.com. Sixty three PHP warnings across
the two lookups, three per property per profile, from ExampleUtils.php
lines 100, 102 and 110.

This example shows the details of devices associated with a given 'Type Allocation Code' or 'TAC'.
More background information on TACs can be found through various online sources such as Wikipedia: https://en.wikipedia.org/wiki/Type_Allocation_Code
----------------------------------------
Which devices are associated with the TAC '35925406'?
PHP Warning:  Undefined array key "hardwarevendor" in ...\examples\cloud\classes\ExampleUtils.php on line 100
PHP Warning:  Attempt to read property "hasValue" on null in ...\examples\cloud\classes\ExampleUtils.php on line 102
PHP Warning:  Attempt to read property "noValueMessage" on null in ...\examples\cloud\classes\ExampleUtils.php on line 110
PHP Warning:  Undefined array key "hardwarename" in ...\examples\cloud\classes\ExampleUtils.php on line 100
PHP Warning:  Attempt to read property "hasValue" on null in ...\examples\cloud\classes\ExampleUtils.php on line 102
PHP Warning:  Attempt to read property "noValueMessage" on null in ...\examples\cloud\classes\ExampleUtils.php on line 110
PHP Warning:  Undefined array key "hardwaremodel" in ...\examples\cloud\classes\ExampleUtils.php on line 100
PHP Warning:  Attempt to read property "hasValue" on null in ...\examples\cloud\classes\ExampleUtils.php on line 102
PHP Warning:  Attempt to read property "noValueMessage" on null in ...\examples\cloud\classes\ExampleUtils.php on line 110
	Unknown () Unknown () (Unknown ())

The exit code was 0, so nothing anywhere reported a problem.

After

Same key, same command, same machine. No warnings, and the reason the
service gave is printed.

This example shows the details of devices associated with a given 'Type Allocation Code' or 'TAC'.
More background information on TACs can be found through various online sources such as Wikipedia: https://en.wikipedia.org/wiki/Type_Allocation_Code
----------------------------------------
Which devices are associated with the TAC '35925406'?
	Unknown (HardwareVendor is a paid feature. You need a licence key to retrieve data. Visit https://51degrees.com/pricing for details) Unknown (HardwareName is a paid feature. You need a licence key to retrieve data. Visit https://51degrees.com/pricing for details) (Unknown (HardwareModel is a paid feature. You need a licence key to retrieve data. Visit https://51degrees.com/pricing for details))
Which devices are associated with the TAC '86386802'?
	Unknown (HardwareVendor is a paid feature. ... ) Unknown (HardwareName is a paid feature. ... ) (Unknown (HardwareModel is a paid feature. ... ))
	[six more profiles, same shape]

The tests now go red

The new unit suite, against the code as it was before this change (the old
HardwareProfileCloud, with everything else from this branch in place):

$ phpunit --fail-on-warning tests/HardwareProfileCloudTest.php
Tests: 5, Assertions: 14, Failures: 2, Warnings: 3, Deprecations: 2.
exit 1

Against this branch:

$ phpunit --fail-on-warning --testsuite Unit
D....                                                               5 / 5 (100%)
Tests: 5, Assertions: 22, Deprecations: 2.
exit 0

The two deprecations come from mustache/mustache, a dependency of the
pipeline core JavaScript builder, and are not raised by this repository.

The strengthened example tests, run against the example code as it was
before this change with the not entitled key:

$ phpunit --fail-on-warning tests/ExampleTests.php
Tests: 4, Assertions: 35, Failures: 2, Warnings: 5, Deprecations: 5.
exit 1

Against this branch, same key:

$ phpunit --fail-on-warning --display-warnings tests/ExampleTests.php
DDDD                                                                4 / 4 (100%)
Tests: 4, Assertions: 53, Deprecations: 5.
exit 0

phpstan analyse reports no errors.

Not fixed here, and why

The TLS failure message belongs to another repository. Running the same
example on a machine whose PHP has no CA bundle configured gives this,
which does not mention TLS at all:

PHP Fatal error:  Uncaught TypeError: substr(): Argument #1 ($string) must be of type string, false given in vendor\51degrees\fiftyone.pipeline.cloudrequestengine\src\HttpClient.php:86

HttpClient::makeCloudRequest calls substr() on the result of
curl_exec without checking for false. The real error, from
curl_error, is SSL certificate OpenSSL verify result: unable to get local issuer certificate (20). That file is in
51Degrees/pipeline-php-cloudrequestengine,
not here, so it is raised there as an issue rather than patched in this
pull request.

The organisation secret still has its old name. The workflows now read
secrets._51DEGREES_RESOURCE_KEY_SUPER and fall back to
secrets.SUPER_RESOURCE_KEY, so nothing breaks either way. Renaming the
organisation secret to _51DEGREES_RESOURCE_KEY_SUPER would let
common-ci's central set-resource-keys.ps1 export it, after which the
fallback and the legacy variable names can be deleted. That rename needs
organisation access and is not done here.

Integration tests are still gated on the CSV asset.
ci/run-integration-tests.ps1 skips every integration test, cloud ones
included, when tests/51Degrees.csv is missing. The cloud tests do not
need that file. Splitting the gate is worth doing but is left out of this
change to keep it reviewable.


Produced with AI assistance. Every claim above was checked by running the
commands shown, and the output is real rather than illustrative. It needs
human review before merging.

…sts that fail when a cloud example is broken

The hardware profile cloud engine threw away the reason the cloud service
gives when a resource key is not entitled to a property, so the TAC and
native model examples printed nine PHP warnings per lookup and the useless
line 'Unknown ()'. The engine now carries each aspect level value and its
companion nullreason into every profile, and the example helper reports
the reason instead of reading a missing array key.

A new unit suite runs the engine and the example helper against fixed
cloud responses, so it needs no resource key and no network connection.
It runs on every build, which is what makes a broken cloud example show as
red. The repository's run-unit-tests script now runs it rather than
printing that there are no unit tests.

The example tests no longer assert only that some output was produced.
They assert the headings the examples print, that no line reads as a
programming fault, and that every device line says something useful. The
failure to match test now checks what the example printed instead of
asserting true.

Resource key environment variables move onto the '_51DEGREES_RESOURCE_KEY'
convention, with the previous names still read as a fallback so an
existing setup keeps working, and a missing key now names the variable
that was wanted.
UACHCloudTests and UACHCloudTests_PHP5 call ResourceKeys::find but never imported the class, so PHP looked for it in the tests namespace and the nightly run of this pull request failed with Class "fiftyone\pipeline\devicedetection\tests\ResourceKeys" not found. The other two test files that use it already carry the import.
@Automation51D
Automation51D merged commit 64bb70f into main Sep 6, 2026
1 check passed
@Automation51D
Automation51D deleted the fix/cloud-example-failures branch September 6, 2026 06:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants