Add OpenTelemetry-native host and process monitoring

[mike-realuptime]

Summary

This PR adds OpenTelemetry Collector hostmetrics support for Linux and Windows infrastructure monitoring, including host-level metrics and logical process monitoring.

Key changes:

• Adds Linux and Windows hostmetrics mappings while preserving existing SkyWalking metric semantics.
• Adds process-hostmetrics-linux and process-hostmetrics-windows.
• Aggregates processes by normalized process name in the Collector before OTLP export.
• Normalizes Windows native hostmetrics fields, including state -> mode and process.handles -> process.open_handles.
• Enables the new process MAL rules by default.
• Preserves existing node-exporter contracts and fixes semantic differences for TCP state, CPU mode names, and cached memory.
• Adds deployable Linux and Windows Collector reference configurations.
• Extends MAL fixtures and E2E coverage, including raw Collector assertions and Windows OTLP fixture validation.

The final test suite is green across MAL runtime tests, backend build, Linux host/process E2E, Windows OTLP fixture coverage, raw Collector assertions, and the existing node-exporter regression E2E.

No Horizon/UI changes are included in this PR.

[Copilot]

🟡 Changes recommended

Several YAML files have an incomplete Apache 2.0 license header (missing the leading “Licensed to the Apache Software Foundation (ASF)…” line).

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR adds OpenTelemetry-native host and process monitoring support (Linux + Windows) by extending existing MAL rules to accept either exporter-based metrics or OTel hostmetrics, and introduces a new E2E case that validates the OTel hostmetrics + pre-aggregated process pipeline end-to-end.

Changes:

• Extend Linux vm.yaml and Windows windows.yaml MAL rules to map both exporter and OTel hostmetrics sources into the same canonical SkyWalking meter targets (avoiding rule ownership collisions).
• Add new process MAL rules (process-hostmetrics-{linux,windows}.yaml) modeling normalized process groups as logical SkyWalking instances.
• Add a new E2E case (vm/otel-hostmetrics) including collector config, compose wiring, and workflow integration.

File summaries

File	Description
test/e2e-v2/cases/vm/prometheus-node-exporter/otel-rules/vm.yaml	Extends Linux VM MAL to accept both node-exporter and OTel hostmetrics under vm-monitoring.
test/e2e-v2/cases/vm/otel-hostmetrics/otel-rules/windows.yaml	Adds Windows VM MAL rules for the OTel-hostmetrics E2E case.
test/e2e-v2/cases/vm/otel-hostmetrics/otel-rules/vm.yaml	Adds Linux VM MAL rules for the OTel-hostmetrics E2E case.
test/e2e-v2/cases/vm/otel-hostmetrics/otel-rules/process-hostmetrics-windows.yaml	Adds Windows process MAL rules mapping normalized process groups to instances.
test/e2e-v2/cases/vm/otel-hostmetrics/otel-rules/process-hostmetrics-linux.yaml	Adds Linux process MAL rules mapping normalized process groups to instances.
test/e2e-v2/cases/vm/otel-hostmetrics/otel-collector-config.yaml	Adds an otelcol-contrib config for host + process collection, normalization, grouping, and aggregation.
test/e2e-v2/cases/vm/otel-hostmetrics/expected/process-count-3.yml	Adds an assertion verifying grouped process count aggregation (=3).
test/e2e-v2/cases/vm/otel-hostmetrics/entrypoint.sh	Starts 3 sleep processes and runs the collector in the E2E container.
test/e2e-v2/cases/vm/otel-hostmetrics/e2e.yaml	Defines the new E2E verification steps for hostmetrics + process metrics.
test/e2e-v2/cases/vm/otel-hostmetrics/Dockerfile.otelcol	Builds a minimal image bundling otelcol-contrib plus the E2E entrypoint.
test/e2e-v2/cases/vm/otel-hostmetrics/docker-compose.yml	Wires OAP + BanyanDB + collector container; mounts MAL rules and collector config.
oap-server/server-starter/src/main/resources/otel-rules/windows.yaml	Updates production Windows infrastructure MAL to accept both windows-exporter and OTel hostmetrics sources.
oap-server/server-starter/src/main/resources/otel-rules/vm.yaml	Updates production Linux infrastructure MAL to accept both node-exporter and OTel hostmetrics sources.
oap-server/server-starter/src/main/resources/otel-rules/process-hostmetrics-windows.yaml	Adds production Windows process MAL rules for normalized process groups.
oap-server/server-starter/src/main/resources/otel-rules/process-hostmetrics-linux.yaml	Adds production Linux process MAL rules for normalized process groups.
.github/workflows/skywalking.yaml	Adds the new “VM OpenTelemetry Hostmetrics” E2E job entry and pins otelcol-contrib version via env.

Review details

• Files reviewed: 16/16 changed files
• Comments generated: 3
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[wu-sheng]

Thanks for the contribution. I reviewed the current head and reproduced the Collector path with otelcol-contrib:0.158.0. I think the following should be addressed before merge:

1. Fix the MAL unit-test data. All current unit-test jobs fail with 12 EMPTY results. This is not an existing exporter-output regression: the new VM/Windows hostmetrics-only expressions have no matching input in vm.data.yaml and windows.data.yaml. Please extend those fixtures, and add runtime fixtures for the two new process MAL files (which are currently compile-only).

2. The advertised Collector-side process aggregation does not occur. Every scrape still exports three process.count=1 datapoints for the three sleep processes, rather than one process.count=3 datapoint. The datapoints have slightly different timestamps, and aggregate_on_attributes includes the timestamp in its grouping key. MAL then sums the three values within the report, so the final OAP value is correctly 3 and repeated reports are averaged rather than accumulated. Please either remove the ineffective Collector aggregation and document that MAL performs the value aggregation, or implement actual pre-OTLP aggregation and assert the raw Collector output.

3. The native Windows path does not match the MAL inputs. windows.yaml filters/groups system_cpu_time by mode, but hostmetrics emits state; idle is therefore included and the per-mode result collapses. process-hostmetrics-windows.yaml expects process_open_handles, while native Windows hostmetrics emits process.handles (process_handles in MAL). No Windows normalization configuration or Windows test is included.

4. The process MAL rules are not enabled by the default OAP configuration. enabledOtelMetricsRules contains vm and windows, but neither process-hostmetrics-linux nor process-hostmetrics-windows. The E2E masks this by explicitly enabling the Linux process rule. Please either enable them by default or document the required opt-in configuration.

5. The existing node-exporter TopN-with-attributes assertion is regressed. The rewritten cpu_load1 expression removes the previous decorate({ me -> me.attr0 = me.layer.name() }), while the E2E still queries top_n(..., attr0='OS_LINUX'). Restore the decoration or intentionally update the contract and assertion.

6. Several mappings described as equivalent have different semantics:

   • tcp_curr_estab maps only state=established, whereas Tcp_CurrEstab includes ESTABLISHED and CLOSE-WAIT.
   • OTel CPU states interrupt and wait are copied unchanged, while the existing node-exporter label values are irq and iowait.
   • OTel/gopsutil cached memory includes SReclaimable, while node_memory_Cached_bytes does not, so memory_buff_cache changes meaning depending on its source.

7. process.count inherits the {threads} unit. copy_metric copies the descriptor from process.threads; the exact runtime output reports process.count Unit: {threads}. Please set the unit explicitly to {process}.

8. The production/reference configuration and Windows coverage are missing. The only Collector configuration added is explicitly Linux E2E configuration, despite the PR advertising Linux and Windows production support. Please add deployable reference documentation/configuration and Windows OTLP fixture coverage.

The process_memory_utilization * 100 expression is intentional fixed-point scaling for UI precision, so I do not consider it an output bug. Its comment should describe the basis-point scaling, and the corresponding UI expression should divide by 100.

[mike-realuptime]

Thank you very much for the detailed review.

I have addressed the points you raised and pushed an updated version of the PR.

This review has been a particularly valuable learning experience for me. Several of the issues you pointed out helped me better understand both SkyWalking's metric contracts and the OpenTelemetry Collector processing model. I sincerely appreciate the time you spent reproducing the setup and explaining the problems so precisely.

The changes are summarized below:

1. Extended the Linux VM and Windows MAL fixtures with the missing hostmetrics inputs and added runtime fixtures for both new process MAL rules.

2. Implemented real Collector-side logical process aggregation before OTLP export. The E2E now verifies the raw post-processor Collector output and confirms that three sleep processes become exactly one process.count=3 datapoint before OAP/MAL processing.

3. Added the required Windows normalization:

   • system.cpu.time: state -> mode
   • process.handles -> process.open_handles

4. Added process-hostmetrics-linux and process-hostmetrics-windows to the default enabledOtelMetricsRules.

5. Restored the historical cpu_load1 decoration so the TopN attr0=OS_LINUX contract is preserved.

6. Corrected the semantic mappings for:

   • Tcp_CurrEstab
   • CPU interrupt -> irq
   • CPU wait -> iowait
   • cached-memory semantics

7. process.count is now explicitly created with unit {process}.

8. Added deployable Linux and Windows Collector reference configurations and Windows OTLP host/process E2E coverage.

The final validation is green, including:

• MAL runtime tests
• backend build
• Linux hostmetrics E2E
• Linux process E2E
• raw Collector aggregation/normalization assertions
• Windows OTLP host/process E2E coverage
• existing node-exporter regression E2E

Thank you again for the thorough review and for the opportunity to improve both the implementation and my understanding of SkyWalking's architecture.

[wu-sheng]

[P1] Preserve the existing Linux CPU metric scale

Collector Contrib 0.158.0 defaults system.cpu.utilization to averaging across cores. The new MAL expressions treat this value as summed CPU utilization, changing the existing meter_vm_cpu_total_percentage and meter_vm_cpu_average_used semantics. Four fully busy cores must report 400%, but this configuration produces 100%. The new normalized metric then divides again and reports 25% instead of 100%.

Please retain per-core attributes or configure sum aggregation, update the corresponding E2E configuration, and assert these values on a multi-core input. This enhancement should preserve the existing metric scale.

[wu-sheng]

[P2] Avoid overlapping CPU states in the new Windows normalized metric

Native Windows system CPU time already includes interrupt time. Summing every non-idle state adds interrupt again, inflating the new meter_win_cpu_norm_percentage and potentially exceeding 100%.

Please calculate this new normalized metric from non-overlapping user and system time. The existing total-CPU rule already sums all non-idle modes; this finding concerns the new normalized metric, rather than a regression in the existing exporter path.

[wu-sheng]

[P3] Remove or implement the advertised network-connections metric

The supported-metrics table lists meter_vm_network_connections, but no corresponding MAL rule is included. Users following this documentation cannot query that metric. Please remove the entry or provide the rule that produces it.