Add P/D metrics aggregation endpoint to PD master - #1556
Closed
sufubao wants to merge 1 commit into
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PD 分离部署时,master 的
/metrics只包含自身指标,监控需要额外配置各个 P/D 节点地址。本改动新增/pd/metrics,从注册表自动发现并采集所有 P/D 节点,支持role=prefill|decode筛选,适用于多 P、多 D,节点扩缩容无需更新 Prometheus 目标地址。指标按指标族合并并添加
pd_role、pd_node标签,保留各节点的原始数值;原有/metrics行为不变。采集使用并发上限和超时,失败节点不影响成功节点,并通过lightllm_pd_scrape_success和lightllm_pd_registered_nodes暴露采集状态及注册节点数。部分或全部节点失败时仍返回 HTTP 200,因此 P/D 告警需使用节点采集状态,而不是仅依赖 Prometheus 的up。附带 Prometheus 配置与 Grafana 查询说明。单元测试文件已按要求移除,PR 仅包含实现和文档。
验证:移除前 22 项接口及采集测试通过;同步最新 upstream/main 后 Black(120 列)、Python 编译和
git diff --check通过。尚未进行实际 P/D 服务联调。