[DNM] dt: bump franz-bench and client-swarm to current versions#30580
Draft
nguyen-andrew wants to merge 2 commits into
Draft
[DNM] dt: bump franz-bench and client-swarm to current versions#30580nguyen-andrew wants to merge 2 commits into
nguyen-andrew wants to merge 2 commits into
Conversation
Brings the franz-go bench example consumed by franzgo_bench.py up to
the v1.20.x line used by kgo-verifier and transform-verifier. The
bench progress output format ("X.XX MiB/s; Y.YYk records/s") and CLI
flags are unchanged between v1.5.0 and v1.20.7, so no wrapper changes
in franzgo_examples.py are needed.
NOTE: the SHA pins the client-swarm bump-rdkafka-0.39 branch HEAD for now; this will be updated to a real client-swarm main-branch commit once the PR is merged. Bumps client-swarm to rdkafka 0.39 (bundled librdkafka 2.12) to align with the broader non-Java client modernization for Kafka 4.x. rdkafka 0.39 pulls in librdkafka-sys 4.10+, whose build uses bindgen which requires libclang at compile time, so the client-swarm Docker stage now installs libclang-dev. The bump causes test_topic_recreation_while_producing to fail due to a behavior gap in librdkafka: after a topic is deleted and recreated, the producer's per-partition leader_epoch cache isn't reset, so the new topic's fresh leader_epoch is rejected as stale and the producer keeps sending to the previous-incarnation's leader. Earlier librdkafka tore down per-partition state on topic deletion; v2.10 changed that path to keep the state in place across delete+recreate, and under librdkafka's defaults (topic.metadata.refresh.interval.ms = 5min, topic.metadata.propagation.max.ms = 30s) the producer doesn't reconverge within the test's 30s wait_until. This commit works around the gap by passing 2-second values for those two settings via the producer properties, so the cache reconverges within the test's 30s window.
Member
Author
|
/ci-repeat 1 |
Collaborator
Member
Author
|
/ci-repeat 1 |
Collaborator
Retry command for Build#84848please wait until all jobs are finished before running the slash command |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of the effort to bring our test clients up to Kafka 4.x support
(ENG-1185):
v1.5.0→v1.20.7.0.37.0→0.39.0(bundled librdkafka 2.12),plus a workaround in
topic_creation_test.pyfor a librdkafkaleader_epoch behavior gap that the bump exposes.
See the commit messages for more details.
[DNM] because the client-swarm SHA currently pins the
bump-rdkafka-0.39branch on
redpanda-data/client-swarm. The client-swarm bump's commitwill be amended with a real main-branch SHA once that branch lands; this
PR shouldn't merge before that.
Backports Required
Release Notes