Rework the memory reduction thread around explicit reclaim helpers#40777
Open
benhillis wants to merge 1 commit into
Open
Rework the memory reduction thread around explicit reclaim helpers#40777benhillis wants to merge 1 commit into
benhillis wants to merge 1 commit into
Conversation
This was referenced Jun 11, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
This PR refactors the WSL2 mini-init memory reduction thread to use explicit helper functions for procfs reads and to base “idle” detection on aggregate non-idle CPU time, enabling gradual reclaim/drop-cache/compaction policies to be expressed more clearly while aiming to preserve existing idle-gated behavior.
Changes:
- Introduces
ReadProcFileandReadAggregateCpuTimesto read procfs reliably and compute busy-vs-idle CPU deltas across intervals. - Adds helpers to compute reclaimable cache bytes (
/proc/meminfo) and free memory bytes (sysinfo) and uses them to drive gradual reclaim, drop_caches, and compaction policies. - Replaces the prior ring-buffer/user-time-only logic with a stateful, tick-based policy loop using explicit thresholds and hysteresis.
benhillis
force-pushed
the
benhill/mem-reclaim-1-helpers
branch
from
12dd190 to
51f1c8e
Compare
benhillis
force-pushed
the
benhill/mem-reclaim-1-helpers
branch
from
51f1c8e to
4730acd
Compare
benhillis
force-pushed
the
benhill/mem-reclaim-1-helpers
branch
from
4730acd to
8e6f845
Compare
Replace the ring-buffer idle detector and user-CPU-only sampling in the mini-init memory reduction thread with a clearer, helper-based design: - Sample aggregate non-idle CPU time (user, system, irq, softirq, steal) so kernel-bound work keeps the VM out of the idle state, instead of looking at user time alone. - ReadProcFile reads a full procfs snapshot into a caller buffer (close-on-exec, partial-read safe); GetReclaimableCacheBytes / GetFreeMemoryBytes read the relevant counters through it. - Gradual mode reclaims cold page cache (cgroup memory.reclaim) above a fixed floor while CPU-idle, with a hysteresis margin so it does not churn near the floor. - DropCache mode stays gated on sustained CPU idle, drops once, and re-drops only after the reclaimable cache grows meaningfully. - Compaction is gated on free-memory growth so it runs only when there are newly-freed pages worth coalescing. RequestCgroupReclaim performs the memory.reclaim write best-effort: it treats the kernel's expected EAGAIN (some, but not all, pages evicted) as success without logging, and never throws so a transient write error cannot tear down the long-lived reduction thread. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
benhillis
force-pushed
the
benhill/mem-reclaim-1-helpers
branch
from
8e6f845 to
3ed82ea
Compare
Comment on lines
+4026
to
+4030
| if (!State.HavePreviousSample) | ||
| { | ||
| std::this_thread::sleep_for(std::chrono::seconds(1)); | ||
| Stop = GetUserCpuTime(); | ||
| THROW_LAST_ERROR_IF(Stop == -1); | ||
| if ((Stop - Start) < IdleThreshold) | ||
| { | ||
| THROW_LAST_ERROR_IF(WriteToFile("/proc/sys/vm/compact_memory", "1\n") < 0); | ||
| } | ||
| State.PreviousBusy = Busy; | ||
| State.PreviousIdle = Idle; | ||
| State.HavePreviousSample = true; |
Comment on lines
3722
to
+3726
| struct sysinfo Info = {}; | ||
| THROW_LAST_ERROR_IF(sysinfo(&Info) < 0); | ||
| return (Info.totalram - Info.freeram) * Info.mem_unit; | ||
| if (sysinfo(&Info) < 0) | ||
| { | ||
| LOG_ERROR("sysinfo failed {}", errno); | ||
| return -1; |
Comment on lines
+3745
to
+3748
| // Gradual: reclaimable cache below this floor is always retained; only the excess above it (beyond a | ||
| // hysteresis margin) is reclaimed. | ||
| constexpr long long c_floorBaseBytes = 128ll * 1024 * 1024; | ||
| constexpr long long c_gradualHysteresisBytes = 128ll * 1024 * 1024; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First in a series of incremental changes reworking the WSL2 guest memory reduction thread. Each change is submitted one PR at a time; the next will follow once this merges. This PR is behavior-preserving relative to the existing idle-gated reclaim and introduces no default change.
What this does
Replaces the ring-buffer idle detector and user-CPU-only sampling in the mini-init memory reduction thread with a clearer, helper-based design:
ReadProcFilereads a full procfs snapshot into a caller buffer (close-on-exec, partial-read safe);GetReclaimableCacheBytes/GetFreeMemoryBytesread the relevant counters through it.memory.reclaim) above a fixed floor while CPU-idle, with a hysteresis margin so it does not churn near the floor.RequestCgroupReclaimperforms thememory.reclaimwrite best-effort: it treats the kernel's expectedEAGAIN(some, but not all, pages evicted) as success without logging, and never throws, so a transient write error cannot tear down the long-lived reduction thread.Series (submitted sequentially)