Conversation
|
Skipping CI for Draft Pull Request. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Code Review
This pull request improves the handling of handshake events to unblock idle tables and optimizes barrier tracking in the maintainer to prevent memory growth during DDL-heavy workloads by waiting for the DDL dispatcher's report. The feedback identifies an issue where the deferredStatus flag is incorrectly placed, potentially leading to spurious warning logs. Additionally, it is suggested to expand the barrier deferral logic to include InfluenceType_All (syncpoints) to further mitigate memory growth issues and to update log messages accordingly.
| if event == nil { | ||
| if !needACK { | ||
| deferredStatus = true | ||
| continue | ||
| } |
There was a problem hiding this comment.
The deferredStatus flag is currently only set when event == nil. However, handleOneStatus can return a non-nil event with needACK == false (for example, in handleBlockState at lines 432 and 466 when an event is deferred due to pending schedule events or scheduling not being ready). In these cases, deferredStatus remains false, which will trigger a spurious "no dispatcher status to send" warning log at line 156 if no other statuses are processed in the request.
Consider setting deferredStatus = true and continuing the loop whenever needACK is false to correctly handle all deferred status paths.
| if event == nil { | |
| if !needACK { | |
| deferredStatus = true | |
| continue | |
| } | |
| if !needACK { | |
| deferredStatus = true | |
| continue | |
| } | |
| if event == nil { |
| normalBlockEventIncludesDDLSpan(blockState.BlockTables) { | ||
| log.Debug("wait ddl dispatcher report before tracking normal block event", |
There was a problem hiding this comment.
If the helper function is updated to include InfluenceType_All (syncpoints), the log message and function call should be updated to reflect that it now covers more than just 'normal' block events.
| normalBlockEventIncludesDDLSpan(blockState.BlockTables) { | |
| log.Debug("wait ddl dispatcher report before tracking normal block event", | |
| blockEventIncludesDDLSpan(blockState.BlockTables) { | |
| log.Debug("wait ddl dispatcher report before tracking block event", |
| func normalBlockEventIncludesDDLSpan(blockedTables *heartbeatpb.InfluencedTables) bool { | ||
| if blockedTables == nil || blockedTables.InfluenceType != heartbeatpb.InfluenceType_Normal { | ||
| return false | ||
| } | ||
| for _, tableID := range blockedTables.TableIDs { | ||
| if tableID == common.DDLSpanTableID { | ||
| return true | ||
| } | ||
| } | ||
| return false | ||
| } |
There was a problem hiding this comment.
The current implementation only defers InfluenceType_Normal events that include the DDL span. However, InfluenceType_All events (such as syncpoints) also conceptually include the DDL span and are coordinated by the DDL dispatcher. If there is a large backlog of syncpoints, table dispatchers reporting them before the DDL dispatcher will still create BarrierEvent objects, which could lead to the memory growth issue this PR aims to address.
Consider renaming this function to blockEventIncludesDDLSpan and including InfluenceType_All in the check.
func blockEventIncludesDDLSpan(blockedTables *heartbeatpb.InfluencedTables) bool {
if blockedTables == nil {
return false
}
if blockedTables.InfluenceType == heartbeatpb.InfluenceType_All {
return true
}
if blockedTables.InfluenceType != heartbeatpb.InfluenceType_Normal {
return false
}
for _, tableID := range blockedTables.TableIDs {
if tableID == common.DDLSpanTableID {
return true
}
}
return false
}|
[FORMAT CHECKER NOTIFICATION] Notice: To remove the 📖 For more info, you can check the "Contribute Code" section in the development guide. |
What problem does this PR solve?
Issue Number: close #xxx
What is changed and how it works?
Check List
Tests
Questions
Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?
Release note