[Feature] Unified OpenTelemetry host and process monitoring for Linux and Windows

[mike-realuptime]

Search before asking

• I had searched in the issues and found no similar feature requirement.

Description

SkyWalking currently provides infrastructure-monitoring examples that are primarily Linux-oriented and based on node-exporter metrics. In that architecture, the OpenTelemetry Collector is mainly used as a transport layer between the exporter and SkyWalking OAP.

This proposal introduces an OpenTelemetry-native infrastructure-monitoring option for both Linux and Windows using only otelcol-contrib on the monitored host.

The same Collector would be responsible for:

• Host CPU, memory, paging, filesystem, disk and network metrics.
• Platform-specific metrics such as Linux swap/file descriptors and Windows pagefile/handles.
• Process discovery and filtering.
• Process-name normalization.
• Low-cardinality process aggregation.
• OTLP transport to SkyWalking OAP.

This removes the requirement to deploy and maintain additional host-side exporters such as:

• node-exporter
• process-exporter
• Windows-specific standalone exporters

The goal is to provide a consistent infrastructure-monitoring model:

A host is represented as a SkyWalking service in either Layer.OS_LINUX or Layer.OS_WINDOWS.
A normalized process group is represented as an instance of its corresponding host service.
Host and process dashboards follow the same navigation and visual structure on Linux and Windows.
Platform-specific differences are preserved where the underlying operating-system semantics are different.
Previous architecture

The existing Linux-oriented approach generally follows this model:

node-exporter
↓
OpenTelemetry Collector
↓
SkyWalking OAP
↓
Horizon UI

Adding process monitoring typically requires another component:

node-exporter
process-exporter
↓
OpenTelemetry Collector
↓
SkyWalking OAP
↓
Horizon UI

In this architecture, the Collector is mainly a transport or translation component.

Proposed architecture

otelcol-contrib
├── hostmetrics host scrapers
├── hostmetrics process scraper
├── Windows performance counters where required
├── process normalization
├── process aggregation
└── OTLP exporter
↓
SkyWalking OAP
↓
Horizon UI

This reduces the host-side monitoring stack to one service and one configuration.

Proposed host model

Each monitored host is created as a SkyWalking service:

Layer.OS_LINUX
└── Linux host

Layer.OS_WINDOWS
└── Windows host

The host identity is provided through a stable resource attribute:

node_identifier_host_name

Separate job_name values distinguish Linux and Windows host and process pipelines.

Proposed process model

Processes are represented as instances belonging to the corresponding host service:

Host service
├── nginx
├── java
├── postgres
├── sqlservr
├── w3wp
└── dotnet

Process instances are created using the normalized identity:

node_identifier_host_name + process_name

PID is intentionally not part of the SkyWalking entity identity.

Multiple PIDs with the same normalized executable name are aggregated into one process group. This avoids creating a new SkyWalking instance for every operating-system PID.

Examples of normalization include:

sqlservr.exe → sqlservr
w3wp.exe → w3wp
python3.12 → python
python.exe → python
java.exe → java

The Collector removes high-cardinality and potentially sensitive attributes before export, including:

process.pid
process.parent_pid
process.command
process.command_line
process.executable.path
process.owner

Process collection can also use an allowlist so that only operationally relevant processes are monitored.

Host metrics

The proposed MAL rules provide a consistent operational set of host metrics.

• Common Linux and Windows metrics
• CPU utilization.
• CPU utilization by state.
• Normalized CPU utilization.
• Logical CPU count.
• Memory used.
• Memory available.
• Memory total.
• Filesystem utilization by mount point or volume.
• Disk read throughput.
• Disk write throughput.
• Network receive throughput.
• Network transmit throughput.
• Linux-specific metrics
• Native system load averages.
• Swap usage.
• Buffer/cache memory.
• File descriptors.
• TCP and socket metrics where available.
• Windows-specific metrics
• Processor Queue Length converted into 1, 5 and 15-minute exponentially weighted load averages by the OpenTelemetry hostmetrics load scraper.
• Windows pagefile usage.
• Global allocated handles through the windowsperfcounters receiver.
• Per-process open handles.

The proposal does not force Linux and Windows metrics to have identical low-level semantics. Instead, it provides equivalent operational concepts where possible and retains OS-specific rules where required.

Process metrics

The default process-monitoring profile would expose a deliberately small, operationally useful set of metrics:

• Number of grouped processes/PIDs.
• Total thread count.
• Total CPU percentage.
• User CPU percentage.
• System CPU percentage.
• Resident physical memory in bytes.
• Percentage of host physical memory.
• Open file descriptors on Linux.
• Open handles on Windows.
• Uptime of the oldest process in the group.

Process virtual-memory size is intentionally excluded from the default dashboards. Summing virtual address spaces across process groups can produce extremely large and misleading values, especially for multi-process and 64-bit applications.

Advanced process metrics such as disk operations, context switches, paging faults and signals would remain outside the default profile. They could be enabled by users who specifically require them.

Horizon UI improvements

The proposed Horizon templates extend the standard OS_LINUX and OS_WINDOWS experience without requiring frontend code changes.

Host dashboard

The templates add or align panels for:

Normalized CPU utilization.
CPU utilization by state.
CPU load over 1, 5 and 15 minutes.
Physical memory.
Linux swap or Windows pagefile.
Filesystem utilization by mount point or Windows volume.
Disk throughput.
Network throughput.
Linux file descriptors or Windows allocated handles.
Platform-specific network status metrics where available.
Process overview

Each host dashboard includes:

Top process groups by total, user and system CPU.
Top process groups by resident memory.
Top process groups by percentage of host memory.
Top process groups by thread count.
Top process groups by open file descriptors or handles.
Top process groups by number of grouped PIDs.
Process drill-down

Horizon's instance navigation is enabled and presented as Processes.

Selecting a process group displays:

Current CPU utilization.
Current resident memory.
Percentage of host memory.
Number of grouped PIDs.
Thread count.
File descriptors or handles.
Uptime of the oldest process.
Historical CPU, memory, process count, thread, handle and uptime charts.

This uses Horizon's existing template capabilities and SkyWalking's existing service/instance model.

Metrics simplification and cardinality control

The proposal reduces metric complexity through the following rules:

• Use a curated default metric set.
• Do not include PID in the SkyWalking instance identity.
• Normalize executable names before aggregation.
• Aggregate all PIDs sharing the same normalized process name.
• Remove unnecessary process resource attributes before export.
• Use process allowlists where appropriate.
• Keep host and process pipelines separate using different job_name values.
• Preserve Linux and Windows as separate layers while presenting a consistent UI.
• Avoid collecting metrics that are not used by the default dashboards.
• Avoid creating false equivalence between OS-specific metrics.

Proposed contribution

The implementation would consist only of configuration and template assets.

1. SkyWalking MAL rules
   OpenTelemetry hostmetrics rules for Layer.OS_LINUX.
   OpenTelemetry hostmetrics rules for Layer.OS_WINDOWS.
   OpenTelemetry process rules for Linux.
   OpenTelemetry process rules for Windows.
   Consistent metric naming and units.
   Service and instance mappings for host-to-process drill-down.

2. OpenTelemetry Collector example configurations
   Linux hostmetrics receiver and pipeline.
   Windows hostmetrics receiver and pipeline.
   Linux process receiver and aggregation pipeline.
   Windows process receiver and aggregation pipeline.
   Windows performance-counter example for global handles.
   Process allowlist examples.
   Process-name normalization examples.
   Removal of PID and other high-cardinality attributes.
   No credentials or environment-specific endpoints.

3. Horizon UI templates
   Updated OS_LINUX template.
   Updated OS_WINDOWS template.
   Host-level infrastructure dashboards.
   Top-process widgets.
   Process instance navigation.
   Process drill-down dashboards.
   Equivalent layouts for Linux and Windows where the available metrics permit it.

No product-code changes

This feature does not require changes to:

SkyWalking OAP runtime code.
SkyWalking storage schema.
MAL engine.
MQE engine.
GraphQL API.
Horizon frontend components.
SkyWalking agents.
OTLP protocol handling.
Service or instance entity models.

The proposal only contributes:

MAL rule files
OpenTelemetry Collector configuration examples
Horizon template JSON files
Documentation

The implementation uses capabilities already available in SkyWalking OAP, Horizon UI and otelcol-contrib.

The existing node-exporter integration does not need to be removed. The OpenTelemetry-native configuration can initially be introduced as an alternative and potentially become the recommended approach after community validation.

Benefits

1. One monitoring agent for Linux and Windows.
2. No separate node-exporter requirement.
3. No separate process-exporter requirement.
4. Lower deployment and upgrade complexity.
5. Lower operational overhead.
6. Consistent OTLP-based ingestion.
7. Consistent host and process navigation.
8. Process monitoring available on Windows and Linux.
9. Controlled process cardinality.
10. Reduced exposure of process metadata.
11. Easier packaging for virtual machines, physical servers and cloud instances.
12. Easier creation of common dashboards and alerts.
13. No changes to SkyWalking product code.

Validation

A working prototype has been validated using:

SkyWalking OAP 10.4.
Horizon UI 0.6.
otelcol-contrib 0.159.
Linux hostmetrics and process scrapers.
Windows hostmetrics, process and performance-counter receivers.
Host-to-process drill-down through SkyWalking service instances.

The exact target versions and repository locations can be adjusted according to the maintainers' guidance.

Screenshots

.

Use case

Unified infrastructure monitoring

An organization operates both Linux and Windows servers and wants to monitor them through the same SkyWalking deployment and the same host-side telemetry agent.

Reduced host-agent footprint

A user wants to avoid installing and maintaining node-exporter, process-exporter and additional Windows exporters alongside the OpenTelemetry Collector.

Process-aware host troubleshooting

When a host experiences high CPU or memory consumption, the operator can identify the responsible normalized process group directly from the host dashboard and drill down into its history.

Critical process monitoring

Operators can verify whether important process groups such as nginx, postgres, java, sqlservr, w3wp or dotnet are present and consuming expected resources.

Low-cardinality process monitoring

Applications that create multiple worker PIDs can be monitored as a single logical process group without creating one SkyWalking instance per PID.

Windows infrastructure visibility

Windows users receive host, pagefile, filesystem, disk, network, handles and process visibility comparable to the existing Linux infrastructure-monitoring experience.

Consistent alerting and automation

The normalized metrics can be consumed by SkyWalking alarms, MQE queries and other integrations without requiring separate exporter-specific metric models.

Simpler onboarding

A new host only requires an otelcol-contrib installation, an OS-specific configuration and an OTLP destination.

Related issues

#13658
Issue #13658 proposed Linux process monitoring using process-exporter.

This proposal generalizes and simplifies that earlier use case by:

Replacing the process-exporter dependency with the OpenTelemetry hostmetrics process scraper.
Adding Windows process monitoring.
Adding host and process configuration for both operating systems.
Introducing low-cardinality process-name aggregation.
Adding Horizon Top Processes and process drill-down templates.
Using one OpenTelemetry Collector as the host-side monitoring component.

This proposal may be considered an evolution or replacement of the architecture described in #13658.

Are you willing to submit a pull request to implement this on your own?

• Yes I am willing to submit a pull request on my own!

Code of Conduct

• I agree to follow this project's Code of Conduct

[wu-sheng]

From my understanding, this kind of unification makes things more complex. The original pre-aggregation logic exists only via MAL; now, it has to be separated into two parts: OTel config and MAL scripts.
Also, this could break the existing deployment until the user noticed they have to change their OTEL collector.
Our current placement of the OTEL collector is just to leverage its ecosystem and forward the original metrics, all analysis logic, downsampling, pre-aggregation should be in OAP + MAL only.