Skip to content

[fix][broker]Producer with AUTO_PRODUCE schema failed to reconnect, which caused by schema incompatible#25437

Merged
poorbarcode merged 2 commits intoapache:masterfrom
poorbarcode:fix/replication_schema_compatibility_issue
Apr 1, 2026
Merged

[fix][broker]Producer with AUTO_PRODUCE schema failed to reconnect, which caused by schema incompatible#25437
poorbarcode merged 2 commits intoapache:masterfrom
poorbarcode:fix/replication_schema_compatibility_issue

Conversation

@poorbarcode
Copy link
Copy Markdown
Contributor

Motivation

Background

  • Under schema-validation-enforced settings
  • Create a producer with AUTO_PRODUCE schema
    • The producer started.
  • Registers a new schema to the topic by Admin
  • The producer reconnects
    • (Highlight)Issue: the producer could reconnect with stale AutoProduceBytesSchema (BYTES), which then became incompatible with the schema already registered on the cluster.

You can reproduce the issue with the new test testReplicationReconnectWithSchemaValidationEnforced

Modifications

The goal is to make replication recover automatically: when an IncompatibleSchemaException occurs, refresh the topic schema and retry producer creation so the replication pipeline can resume.

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository: x

@poorbarcode poorbarcode added this to the 5.0.0 milestone Mar 30, 2026
@poorbarcode poorbarcode self-assigned this Mar 30, 2026
@poorbarcode poorbarcode added type/bug The PR fixed a bug or issue reported a bug release/4.1.4 release/4.0.10 labels Mar 30, 2026
@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Mar 30, 2026
@lhotari lhotari changed the title [fix][broker]Producer with AUTO_PRODECE schema failed to reconnect, which caused by schema incompatible [fix][broker]Producer with AUTO_PRODUCE schema failed to reconnect, which caused by schema incompatible Mar 31, 2026
@poorbarcode poorbarcode merged commit fc27fe7 into apache:master Apr 1, 2026
83 of 85 checks passed
poorbarcode added a commit that referenced this pull request Apr 1, 2026
…hich caused by schema incompatible (#25437)

(cherry picked from commit fc27fe7)
poorbarcode added a commit that referenced this pull request Apr 1, 2026
…hich caused by schema incompatible (#25437)

(cherry picked from commit fc27fe7)
priyanshu-ctds pushed a commit to datastax/pulsar that referenced this pull request Apr 1, 2026
…hich caused by schema incompatible (apache#25437)

(cherry picked from commit fc27fe7)
(cherry picked from commit 3d33e48)
priyanshu-ctds pushed a commit to datastax/pulsar that referenced this pull request Apr 1, 2026
…hich caused by schema incompatible (apache#25437)

(cherry picked from commit fc27fe7)
(cherry picked from commit 3d33e48)
priyanshu-ctds pushed a commit to datastax/pulsar that referenced this pull request Apr 2, 2026
…hich caused by schema incompatible (apache#25437)

(cherry picked from commit fc27fe7)
(cherry picked from commit 3d33e48)
priyanshu-ctds pushed a commit to datastax/pulsar that referenced this pull request Apr 3, 2026
…hich caused by schema incompatible (apache#25437)

(cherry picked from commit fc27fe7)
(cherry picked from commit 3d33e48)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants