Skip to content

Add P/D metrics aggregation endpoint to PD master - #1556

Closed
sufubao wants to merge 1 commit into
ModelTC:mainfrom
sufubao:pd-master-metrics
Closed

Add P/D metrics aggregation endpoint to PD master#1556
sufubao wants to merge 1 commit into
ModelTC:mainfrom
sufubao:pd-master-metrics

Conversation

@sufubao

@sufubao sufubao commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

PD 分离部署时,master 的 /metrics 只包含自身指标,监控需要额外配置各个 P/D 节点地址。本改动新增 /pd/metrics,从注册表自动发现并采集所有 P/D 节点,支持 role=prefill|decode 筛选,适用于多 P、多 D,节点扩缩容无需更新 Prometheus 目标地址。

指标按指标族合并并添加 pd_rolepd_node 标签,保留各节点的原始数值;原有 /metrics 行为不变。采集使用并发上限和超时,失败节点不影响成功节点,并通过 lightllm_pd_scrape_successlightllm_pd_registered_nodes 暴露采集状态及注册节点数。部分或全部节点失败时仍返回 HTTP 200,因此 P/D 告警需使用节点采集状态,而不是仅依赖 Prometheus 的 up

附带 Prometheus 配置与 Grafana 查询说明。单元测试文件已按要求移除,PR 仅包含实现和文档。

验证:移除前 22 项接口及采集测试通过;同步最新 upstream/main 后 Black(120 列)、Python 编译和 git diff --check 通过。尚未进行实际 P/D 服务联调。

@sufubao sufubao closed this Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant