Digest initialization failure is silently ignored in three hash functions, producing all-zero and meaningless digests

[djbclark]

Worst-case impact. HashPubKey()'s output isn't just a hash — it's the actual TLS trust-on-first-use identity: SavePublicKey() writes trusted keys to ppkeys/<user>-<digest>.pub, and lastseen maps IP → trusted digest (Address2Hostkey()). When the digest silently collapses to an all-zero constant instead of erroring, every key hashed during the failure window collapses to the same identity, not merely a bad one.

I checked SavePublicKey(): it refuses to overwrite a file that already exists for a given identity ("already exists, not rewriting"), so this isn't a silent-overwrite bug — it's first-come-first-served. Whichever key gets trusted first under the all-zero identity occupies that slot permanently; any different key that later also hashes to zero (guaranteed, since it's a constant rather than a rare collision) will be looked up under that same slot and treated as a match for it. During a live failure window this means a host trusted first under the collided identity becomes indistinguishable from any other host whose key also failed to hash the same way — and since neither HashFile() nor HashPubKey() logged anything before this fix, there is no forensic trail explaining why two different keys were ever treated as the same identity.

Caveat, so this isn't overstated: the demonstrated trigger (CryptoDeInitialize() unloading OpenSSL 3's default provider, then something hashes again afterward) is an internal call-ordering condition, not something a remote attacker can force directly today. So this is a genuine silent integrity hole in the trust model, not a demonstrated remote exploit — the severity is "this should never have been able to fail silently and indistinguishably," not "here is a working attack chain."

---

Component: libutils · Affects: master (0c0620d), present since 2019 · Severity: normal

I'd normally file this against project CFE in the Jira tracker, but I can't
right now — both routes are broken on my end: the API returns 401 on
/rest/api/3/myself with a well-formed token, and the web UI shows me as
logged in (I see a Create button) but any actual action behaves as if I'm
logged out. Opening this here instead so it isn't stuck waiting on that; happy
to move/link it into Jira myself once access is sorted, or if a maintainer
can do that from your end in the meantime, much appreciated.

---

libntech's libutils/hash.c calls EVP_DigestInit() or EVP_DigestInit_ex() in six
places. Three of them do not handle failure, each in a different way, and none of
the three logs anything:

• HashFile() (via the static HashFile_Stream()) and HashPubKey() zero their
  output digest up front and fill it in only on success. On failure the caller
  receives an all-zero digest that is indistinguishable from a real one. Both
  functions return void, so the digest buffer is the only channel and it
  carries no success indication.

• HashNew() ignores the return value entirely and then calls EVP_DigestUpdate()
  and EVP_DigestFinal_ex() on an uninitialized context. It returns a non-NULL
  Hash whose digest is meaningless.

The affected code, on master (0c0620d):

• HashNew
• HashFile_Stream
• HashPubKey

This is an oversight rather than a decision. All three functions arrived in
libntech in the same commit, f277970 (2019-10-03, "Added hash functions from
libpromises"). At that commit HashString() already had the else-branch that logs
LOG_LEVEL_ERR in exactly this case, while HashFile_Stream() and HashPubKey() went
straight to freeing the context with no else at all. The asymmetry has been there
for six years and is still present on master.

The three siblings in the same file that already handle it correctly:

• HashString
• HashNewFromDescriptor
• HashNewFromKey

Impact. The most serious of the three is HashPubKey(), which computes the digest
of a host's public key. In cfengine/core it feeds libpromises/lastseen.c,
libpromises/crypto.c, libenv/sysinfo.c, cf-execd/cf-execd-runner.c, and is
included by libcfnet/tls_generic.c and libcfnet/client_protocol.c. A public key
digest that silently becomes a constant is a host identity that collides across
every host that hits the failure. HashFile() has 12 call sites in core.

Reachability. This is not theoretical. On OpenSSL 3, CryptoDeInitialize() unloads
the default provider, after which every subsequent EVP_DigestInit() fails.
Anything that hashes after that point is silently affected. That is how this was
found.

Reproduction. Call any of the three functions, then call CryptoDeInitialize(),
then call it again. Observed on the second call, against unmodified master:

HashFile     -> all-zero digest, nothing logged
HashPubKey   -> all-zero digest, nothing logged
HashNew      -> non-NULL Hash, meaningless digest, nothing logged

Proposed fix. Each of the three is modelled on its own in-file sibling above:
HashFile_Stream() and HashPubKey() gain the else-branch that HashString() already
has, and HashNew() logs and returns NULL as HashNewFromDescriptor() already does.
HashNew() already returns NULL on four other paths and has no callers in
cfengine/core, so failing closed there breaks nothing. Modelling each fix on its
own sibling keeps the change an oversight repair rather than a new opinion.
21 insertions, 3 deletions, one file. Builds clean; libntech's hash_test passes
(6 tests).

Making failure detectable by the callers of HashFile() and HashPubKey(), rather
than merely visible in the log, would mean changing the return types of two void
functions with callers across both repositories, and is deliberately left out.

No unit test is included: forcing EVP_DigestInit() to fail requires
CryptoDeInitialize(), which lives in cfengine/core's libpromises and is not
available to libntech's own tests, and a libntech-level test could only assert
the all-zero digest that is returned either way.

Pull request: now open upstream at
#291 (review thread and diff
also available on my fork: djbclark#1)

Diff against upstream master:
master...djbclark:libntech:silent-digest-failure

Commit:
djbclark@dc85a6f

[djbclark]

Correction to the "Worst-case impact" paragraph above — it overstates the trust impact, and I'd rather say so than leave it standing.

That paragraph asserts that HashPubKey()'s output "isn't just a hash — it's the actual TLS trust-on-first-use identity: SavePublicKey() writes trusted keys to ppkeys/<user>-<digest>.pub, and lastseen maps IP → trusted digest." That is wrong, and I checked it by grep rather than by reading.

HashPubKey() is not on the peer-facing TOFU path at all. TLS TOFU, SavePublicKey() on the network paths, and lastseen writes for remote hosts all go through KeyPrintableHash() / KeyNew() → HashNewFromKey() (libcfnet/tls_generic.c, libcfnet/client_protocol.c, cf-serverd/server_tls.c, cf-serverd/server_classic.c). HashNewFromKey() has returned NULL on EVP_DigestInit_ex failure since long before this report — it already fails closed. There are zero HashPubKey calls in libcfnet or cf-serverd.

What HashPubKey()'s digest actually reaches, in cfengine/core, is self-identity bookkeeping: TrustKey/GetPubkeyDigest (cf-key, a local operator tool), PolicyHubUpdateKeys (the hub's own key, at start-up), sys.key_digest inventory, cf-execd's mail header, and the localhost branch of Address2Hostkey. Only that last one is a lastseen path, and only for localhost.

So the first-come-first-served analysis of SavePublicKey() is accurate as a description of that function, but it is not reachable from this defect the way the paragraph implies.

What the impact actually is, stated as precisely as I can: even in the worst live failure — the provider missing for the whole process, or FIPS rejecting the algorithm — an all-zero digest is a colliding lookup handle, not a bypassed cryptographic gate. The SHA=0000… slot holds the first writer's real public key, and impersonating that identity still requires the matching private key; TLS possession-of-key checks are untouched. What an operator gets is misattributed inventory and first-writer-wins confusion in ppkeys — an integrity and diagnostics defect, not an authentication bypass. The existing caveat about this not being remotely forceable stands and is, if anything, understated.

I'd still argue it should be fixed, for the reason the original caveat gives: this should never have been able to fail silently and indistinguishably. But "normal" severity is right, and I don't want the trust-model framing above to be the thing a reader carries away.

One other correction, which affects the PR rather than this issue: the claim that no libntech unit test is possible because forcing EVP_DigestInit() to fail needs cfengine/core's CryptoDeInitialize() is also false. It can be done from libntech alone by draining the OpenSSL 3 providers before any digest operation runs, and #291 now carries that test. Details in a comment there.

Both errors were mine, and both were caught by review before a maintainer had to spend time on them.

[djbclark]

Now tracked in Jira as CFE-4717.

The report above says I'd normally file this against CFE but couldn't — the API was returning 401 on a well-formed token and the web UI behaved as though logged out. That's resolved: the missing piece was permission within the project, not the Atlassian account itself. Thanks to @nickanderson for sorting it out over Matrix.

CFE-4717 carries the corrected version of this report, not the original:

• the "Worst-case impact" paragraph's trust-model framing is dropped, per my correction comment above — HashPubKey() is not on the peer-facing TOFU path, and the impact is misattributed inventory and first-writer-wins confusion in ppkeys, not an authentication bypass;
• the "no unit test is possible" claim is dropped too, since Handle digest initialization failure when hashing #291 now carries tests/unit/hash_init_fail_test.c, verified failing 3-of-3 against the unfixed file.

Happy to close this issue in favour of the Jira ticket if that's what you'd prefer — leaving it open for now so the correction above stays visible.

[larsewi]

Thanks for the report @djbclark. Let's close this in favor of the ticket CFE-4717.