Added support of direct write mode#641
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #641 +/- ##
============================================
+ Coverage 70.87% 71.07% +0.20%
- Complexity 3314 3341 +27
============================================
Files 374 377 +3
Lines 15699 15810 +111
Branches 1650 1662 +12
============================================
+ Hits 11126 11237 +111
+ Misses 3931 3927 -4
- Partials 642 646 +4 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
Adds “direct write” support for topic writers by allowing write sessions to target a preferred node (derived from partition location), and refactors the internal stream abstraction to support both real and pre-failed streams.
Changes:
- Introduce
WriteStreamFactoryto construct regular or direct-write streams (by partitionId or producerId). - Refactor
TopicStreaminto an interface with concrete implementationsTopicStreamBaseandTopicStreamFail. - Add/adjust tests validating direct write behavior and the new stream base class usage.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| topic/src/main/java/tech/ydb/topic/write/impl/WriteStreamFactory.java | New factory implementing direct-write selection + partition/node probing logic. |
| topic/src/main/java/tech/ydb/topic/write/impl/WriteStream.java | Switch to TopicStreamBase and add Fail stream implementation. |
| topic/src/main/java/tech/ydb/topic/write/impl/WriteSession.java | Wire WriteStreamFactory into retryable write session creation. |
| topic/src/main/java/tech/ydb/topic/settings/WriterSettings.java | Add direct write flag and builder setter. |
| topic/src/main/java/tech/ydb/topic/impl/TopicStream.java | Convert from abstract class to interface. |
| topic/src/main/java/tech/ydb/topic/impl/TopicStreamBase.java | New concrete base implementation wrapping grpc stream behavior. |
| topic/src/main/java/tech/ydb/topic/impl/TopicStreamFail.java | New stream implementation that fails immediately with a fixed status. |
| topic/src/main/java/tech/ydb/topic/impl/GrpcTopicRpc.java | Add writeSession(id, directWriteNodeId) with direct-mode request settings. |
| topic/src/main/java/tech/ydb/topic/TopicRpc.java | Extend API with default overload for direct-write node targeting + proto type import cleanup. |
| topic/src/test/java/tech/ydb/topic/write/impl/WriteStreamFactoryTest.java | New tests for direct write factory scenarios and failures. |
| topic/src/test/java/tech/ydb/topic/impl/TopicStreamTest.java | Update test to use TopicStreamBase. |
| topic/src/test/java/tech/ydb/topic/impl/TopicRetryableStreamTest.java | Update mocks/types to reflect new TopicStream interface/base. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Overall: the refactoring from TopicStream (abstract class) to TopicStream (interface) + TopicStreamBase + TopicStreamFail is well-structured. The TopicStreamFail pattern integrates cleanly with the retry mechanism in TopicRetryableStream — when direct write setup fails, the failure status flows through the existing retry logic, so transient errors (e.g. UNAVAILABLE from describeTopic) get retried while non-retryable errors (e.g. BAD_REQUEST for missing partition) stop the writer. The test coverage for WriteStreamFactory is thorough and covers the important success and failure paths for both partition-id-based and producer-id-based direct write modes.
See inline comments for potential issues found during review.
There was a problem hiding this comment.
Overall: the refactoring from TopicStream (abstract class) to TopicStream (interface) + TopicStreamBase + TopicStreamFail is well-structured. The TopicStreamFail pattern integrates cleanly with the retry mechanism in TopicRetryableStream — when direct write setup fails, the failure status flows through the existing retry logic, so transient errors (e.g. UNAVAILABLE from describeTopic) get retried while non-retryable errors (e.g. BAD_REQUEST for missing partition) stop the writer. The test coverage for WriteStreamFactory is thorough and covers the important success and failure paths for both partition-id-based and producer-id-based direct write modes.
See inline comments for potential issues found during review.
There was a problem hiding this comment.
I reviewed the changes in context of the surrounding codebase. The PR restructures the write-side stream infrastructure — splitting TopicStream into an interface + base implementation + fail implementation, extracting StreamFactory from WriteSession into a standalone WriteStreamFactory, and adding direct write mode support. Overall the code flow is correct and test coverage is good. I found no critical issues in the changes.
A few non-critical observations below in inline comments.
No description provided.