Repository navigation
The consumer close() that stranded its client, and a main that could not pass its own gate - #4
Merged
Conversation
…not pass its own gate
TWO REPAIRS, AND THE SECOND WAS FOUND WHILE VERIFYING THE FIRST.
RdKafkaConsumerClient::close() called KafkaConsumer::close() and then released
the property. Releasing the property is exactly right and was always there; it
did nothing, because by the time it ran the handle was already gone.
ext-rdkafka 6.0.5, kafka_consumer.c:531-542, is the whole of that method:
rd_kafka_consumer_close(intern->rk);
intern->rk = NULL;
No rd_kafka_destroy(), and the free handler at kafka_consumer.c:53-64 destroys
the handle only `if (intern->rk)` — which close() has just nulled. So the PHP
object is freed and the rd_kafka_t is not. Measured on PHP 8.5.8 / ext-rdkafka
6.0.5 / librdkafka 2.15.1, no broker: four OS threads stranded per closed
consumer, still there after five seconds of polling; zero when the reference is
dropped. Unconditional, not a race.
It now calls unsubscribe(), which leaves the consumer group, is safe on a
consumer that never subscribed, and lets the drop destroy the client. The test
counts rd_kafka_thread_cnt() rather than asserting a WeakReference goes null,
because the natural test is GREEN on this bug: the PHP object really is freed and
the leak is underneath it, in C. Checked both ways — it fails against the old
body with "Failed asserting that 12 is identical to 0".
dworkers found it the hard way: the same call had been added there, deliberately
and with a careful comment, to stop a segfault at PHP shutdown after a green
suite. It was the cause of it.
AND main COULD NOT PASS `composer check`, in three separate ways, which is why CI
has been red on every PHP version since 26.09.6:
- The version is carried in FOUR places and 26.09.6 moved one. Version::VERSION
said 26.09.6, the CHANGELOG heading said 26.09.7, the README badge said
26.09.5 and the verbatim listing in docs/versioning.md said 26.09.5 — so the
code inside the tag v26.09.7 reported itself as 26.09.6. All four now say
26.09.8.
- pint --test was red in three files from the 26.09.7 commit: an import added
out of alphabetical order in SecurityAutoConfiguration, and the two security
cache manifests recompiled beside it.
- phpstan had three errors in InMemoryJwksProviderTest, all from one missing
@return array<string, mixed> on its fixture helper.
`composer check` now passes: pint, phpstan clean, 3808 tests, deptrac 0/0.
NOT FIXED HERE, BECAUSE IT CANNOT BE. The Release workflow has failed for
26.09.5, 26.09.6 and 26.09.7 at its own preflight, which is doing exactly what
its comment says it is for: the split action exits 0 even when its push fails, so
the gate refuses rather than producing a matrix of green jobs and zero published
packages. ACCESS_TOKEN is not set — `gh secret list` is empty — and the mirror
repositories do not exist: fireflyframework/firefly-kernel, firefly-container and
firefly-eda-kafka all answer "Could not resolve to a Repository". No split
package has ever been published. That needs an org PAT and the repositories,
which is docs/publishing.md steps 6-7 and a person.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The leak
RdKafkaConsumerClient::close()calledKafkaConsumer::close()and then released the property. Releasing the property is exactly right and was always there; it did nothing, because by the time it ran the handle was already gone.ext-rdkafka 6.0.5,
kafka_consumer.c:531-542, is the whole of that method:No
rd_kafka_destroy(), and the free handler atkafka_consumer.c:53-64destroys the handle onlyif (intern->rk)— whichclose()has just nulled. The PHP object is freed and therd_kafka_tis not.Measured on PHP 8.5.8 / ext-rdkafka 6.0.5 / librdkafka 2.15.1, no broker: four OS threads stranded per closed consumer, still there after five seconds of polling; zero when the reference is dropped. Unconditional, not a race.
It now calls
unsubscribe()— which leaves the consumer group, is safe on a consumer that never subscribed, and lets the drop destroy the client (librdkafka logsDestroying cgrp).The test counts
rd_kafka_thread_cnt()rather than asserting aWeakReferencegoes null, because the natural test is green on this bug: the PHP object really is freed and the leak is underneath it, in C. Checked both ways — it fails against the old body withFailed asserting that 12 is identical to 0.dworkers found it the hard way: the same call had been added there, deliberately and with a careful comment, to stop a segfault at PHP shutdown after a green suite. It was the cause of it.
And
maincould not passcomposer checkThree separate reasons, which is why CI has been red on every PHP version since 26.09.6:
Version::VERSIONsaid26.09.6, the CHANGELOG heading said26.09.7, the README badge said26.09.5, the verbatim listing indocs/versioning.mdsaid26.09.5— so the code inside the tagv26.09.7reported itself as26.09.6. All four now say26.09.8.pint --testwas red in three files from the 26.09.7 commit: an import out of alphabetical order inSecurityAutoConfiguration, and the two security cache manifests recompiled beside it.phpstanhad three errors inInMemoryJwksProviderTest, all from one missing@return array<string, mixed>.composer checknow passes: pint, phpstan clean, 3808 tests, deptrac 0/0.Not fixed here, because it cannot be
The Release workflow has failed for 26.09.5, 26.09.6 and 26.09.7 at its preflight, which is doing exactly what its comment says: the split action exits 0 even when its push fails, so the gate refuses rather than producing a matrix of green jobs and zero published packages.
ACCESS_TOKENis not set (gh secret listis empty) and the mirror repositories do not exist —fireflyframework/firefly-kernel,firefly-containerandfirefly-eda-kafkaall answer Could not resolve to a Repository. No split package has ever been published. That needs an org PAT and the repositories:docs/publishing.mdsteps 6-7.