[server] Add request queue size metric per processor#3232
Open
zuston wants to merge 1 commit intoapache:mainfrom
Open
[server] Add request queue size metric per processor#3232zuston wants to merge 1 commit intoapache:mainfrom
zuston wants to merge 1 commit intoapache:mainfrom
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds per-RPC-processor visibility into request queue depth by exporting a requestQueueSize gauge scoped by processor index, enabling operators to pinpoint queue congestion to a specific processor rather than relying on only aggregated queue metrics.
Changes:
- Register a per-processor
requestQueueSizegauge under arequest_processor_indexmetric group keyed byprocessor_index. - Extend
RequestsMetricswith a helper to register gauges in a keyed child metric group. - Document the new coordinator/tabletserver per-processor queue size metric in the observability metrics guide.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| website/docs/maintenance/observability/monitor-metrics.md | Documents the new request_processor_index-scoped requestQueueSize gauge for coordinator and tabletserver. |
| fluss-rpc/src/main/java/org/apache/fluss/rpc/netty/server/RequestsMetrics.java | Adds an overload to register a gauge in a keyed child metric group (key/value). |
| fluss-rpc/src/main/java/org/apache/fluss/rpc/netty/server/RequestProcessorPool.java | Registers requestQueueSize per processor (keyed by processor_index) in addition to the existing aggregated gauge. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+64
to
+68
| requestsMetrics.gauge( | ||
| PROCESSOR_INDEX, | ||
| String.valueOf(i), | ||
| MetricNames.REQUEST_QUEUE_SIZE, | ||
| requestChannel::requestsCount); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Currently, each connection is bound to a dedicated RPC processor. Based on the aggregated RPC queue metrics, we cannot determine whether high latency is caused by a specific processor queue being blocked.
When latency is high, I want to identify whether a particular processor’s queue is congested or stuck.
Brief change log
Tests
API and Format
Documentation