Feature: Detailed Memory Subsystem Telemetry (Swap, ZRAM, HugePages, Shmem, and Kernel Slab Breakdown)

[OrDinaD]

Problem Description / Motivation

In enterprise Linux environments and high-density container platforms (e.g., Kubernetes nodes running latency-sensitive databases, caching tiers, JVMs, or memory-constrained workloads), knowing only coarse-grained MemTotal, MemAvailable, and MemUsed is inadequate for capacity planning, troubleshooting OOM incidents, and detecting memory leaks.

Modern Linux systems rely on complex memory tiering and subsystems:

• ZRAM / Zswap / Swap: Compressed in-RAM block devices (zram0) and swap usage are critical indicators of memory pressure. Currently, web_htop has zero visibility into swap activity or compressed memory ratios.
• HugePages (Transparent HugePages & HugeTLB): Systems running enterprise workloads rely on 2MB/1GB hugepages (AnonHugePages, HugePages_Total, HugePages_Free). Failing to allocate hugepages causes silent fallback to 4KB pages and severe TLB miss penalties.
• Shared Memory & IPC: Shmem and Mapped pages, tmpfs mounts, and shared memory segments can pin significant portions of RAM without appearing in process RSS.
• Kernel Slab & Buffers: SReclaimable, SUnreclaim, Dirty, and Writeback pages are vital to distinguish between memory that can be quickly freed under pressure versus locked kernel buffers.

Current Limitations in the Codebase

1. Primitive Memory Metrics Model:
   • common/include/common/models/memory_metrics.hpp: struct MemoryMetrics only tracks total_bytes, available_bytes, used_bytes, and used_percent.
   • server/src/collectors/metrics_collector.cpp (MetricsCollector::Memory): Reads /proc/meminfo, extracts only MemTotal and MemAvailable, and discards the remaining ~50 kernel metrics parsed by KeyValues().
2. Missing Swap & Subsystem Allocations:
   • Fields such as SwapTotal, SwapFree, Zram, Shmem, Buffers, Cached, Slab, SReclaimable, SUnreclaim, AnonHugePages, HugePages_Total, HugePages_Free, Dirty, and Writeback are completely ignored.
3. No Per-Process Memory Breakdown:
   • ParseProcess in linux_samples.cpp parses /proc/[pid]/stat field 23 (RSS pages), ignoring /proc/[pid]/status or /proc/[pid]/statm (Shared, Text, Data, Lib, Swap). A process consuming gigabytes of shared memory or swap is reported with incomplete memory footprints.
4. Lack of Diagnostics & Prometheus Exposure:
   • /diagnostics and /exporter endpoints in reactor.cpp do not export detailed memory breakdowns or swap saturation gauges.

Proposed Technical Architecture & Implementation Plan

1. Model & Protocol Enhancements (common/include/common/models/memory_metrics.hpp)

Extend MemoryMetrics with structured memory breakdown fields:

struct MemoryMetrics
{
    TimeStamp timestamp{};
    Bytes total_bytes{};
    Bytes available_bytes{};
    Bytes used_bytes{};
    Percentage used_percent{};

    // Swap & Compressed Memory
    Bytes swap_total_bytes{};
    Bytes swap_free_bytes{};
    Bytes swap_used_bytes{};
    Percentage swap_used_percent{};

    // Kernel & Cache breakdown
    Bytes buffers_bytes{};
    Bytes cached_bytes{};
    Bytes shared_bytes{};          // Shmem
    Bytes slab_reclaimable_bytes{};
    Bytes slab_unreclaimable_bytes{};
    Bytes dirty_bytes{};
    Bytes writeback_bytes{};

    // Hugepages breakdown
    Bytes anon_hugepages_bytes{};
    std::uint64_t hugepages_total{};
    std::uint64_t hugepages_free{};
    std::uint64_t hugepages_reserved{};
    Bytes hugepage_size_bytes{};
    
    // Optional ZRAM stats (if /sys/block/zram0 exists)
    std::optional<Bytes> zram_orig_data_size;
    std::optional<Bytes> zram_compr_data_size;
    std::optional<Bytes> zram_mem_used_total;
};

Update serialization and deserialization in common/src/common/models/memory_metrics.cpp with backward-compatible JSON schema mappings.

2. Collector Implementation (server/src/collectors/metrics_collector.cpp)

• In MetricsCollector::Memory, expand the KeyValues() lookup:
  • SwapTotal, SwapFree, Cached, Buffers, Shmem, SReclaimable, SUnreclaim, Dirty, Writeback, AnonHugePages, HugePages_Total, HugePages_Free, Hugepagesize.
• Query /sys/block/zram<id>/mm_stat or /sys/block/zram<id>/disksize via LinuxSource::Sys when present to compute compression ratios (orig_data_size / compr_data_size).
• Gracefully handle systems where HugePages or ZRAM are unconfigured or unsupported without marking the collector state as unavailable.

3. Exporter & UI Integration

• In reactor.cpp (Diagnostics(bool prometheus)), expose:
  • web_htop_memory_swap_used_bytes
  • web_htop_memory_hugepages_free
  • web_htop_memory_slab_bytes
  • web_htop_memory_zram_compression_ratio
• In client/src/ui/terminal_ui.cpp, update memory gauge and add an advanced memory breakdown panel displaying Cache/Buffer, Shmem, HugeTLB, and ZRAM/Swap status.

Acceptance Criteria

• MemoryMetrics contains fields for swap, shared memory, slab reclaimable/unreclaimable, dirty/writeback, hugepages, and zram.
• Serialization and deserialization unit tests in tests/common/test_serialization.cpp cover all new fields with both complete and missing/legacy payloads.
• FakeSource in tests/core/test_core.cpp is updated with complete /proc/meminfo and /sys/block/zram0 fixtures verifying correct calculations and edge cases (e.g. 0 swap, disabled hugepages).
• No allocations or string conversions are added in the critical loop beyond the existing parsing budget.
• Prometheus metrics on /exporter report the new memory statistics accurately.