Price the premise: what does QuestDB cost under an enforced memory cap?

[mairas]

Why this exists

The DuckDB/Parquet history provider plan (see the tracking issue) is justified entirely
by one number: QuestDB's 365.6 MB resident memory on halpi.hurma. Document review
established that the number does not mean what the plan assumed, and that the cheapest
alternative to a nine-unit build was never tested. This issue prices the premise.

Until it is answered, nobody can say whether the outcome is a three-repo project or a
kernel cmdline edit.

What is already known

• JVM_PREPEND=-Xmx192m caps QuestDB's Java heap. The heap is not the problem.
• The RSS is mostly anonymous, not reclaimable. Measured 2026-08-21 after a
  five-minute warm-up: 420,320 kB RSS = 340,288 kB anonymous (81%) + 80,032 kB
  file-backed (19%). config.yml's "Most of QuestDB's memory is off-heap" refers to
  off-heap native allocation (Cairo column buffers, ILP buffers), which is anonymous —
  not to mapped files. So the comparison against DuckDB's 146 MB anonymous heap is
  roughly like-for-like and the memory driver is sound. An earlier claim to the contrary
  is retracted in the tracking issue.
• That 420 MB figure came from a run with no -Xmx: JVM_PREPEND had been replaced
  by NMT flags, so the JVM defaulted to a quarter of 4 GB (NMT showed heap 1,065 MB
  reserved / 327 MB committed). Today's 365.6 MB is the capped figure.
• halpi.hurma boots with cgroup_disable=memory. /sys/fs/cgroup/cgroup.controllers
  lists cpuset cpu io pids — no memory. So mem_limit: 768m in QuestDB's compose is
  accepted and ignored, and QuestDB has never run under an enforced ceiling here.
• HaLOS builds its own image with pi-gen and controls the kernel cmdline.

Questions to answer

1. How much of QuestDB's RSS is reclaimable? Answered — 81% anonymous,
   19% file-backed. No further measurement needed.
2. What does QuestDB cost under an enforced cap? Enable the memory controller, apply
   a real limit at 256 MB and 384 MB, and measure RSS, CPU and history-API latency under
   the same three-window method used for the existing baseline (300 s windows, 180 s
   settle, cgroup cpu.stat, /proc/<pid>/io, /proc/diskstats, half-window means
   checked for steady state).
3. Does capping break it? config.yml warns that too small a heap sends the JVM into
   continuous full GC, which presents as a container that starts and never becomes
   healthy — 128m was already past that edge on a three-table database. A cap that makes
   QuestDB unhealthy is a failed configuration, not a win.
4. What does enabling the memory controller cost everything else? It is a global
   kernel change affecting every container on the device, not just QuestDB.

Method notes

• Enabling the memory controller requires a kernel cmdline change and a reboot. On a
  device in use, that needs scheduling — do not do it as a side effect of a benchmark.
• Run the comparison with sqhp enabled and recording, not against an idle QuestDB. The
  mapped-page footprint depends on what is being read and written.
• Report dispersion, not just means. The existing baseline reported none, and its own
  numbers show the noise floor is high enough to swallow the +0.15 CPU point figure the
  plan leaned on.

What the answers decide

• If a capped QuestDB lands near the projected DuckDB steady state, the plan's driver is
  gone and the outcome is a configuration change to ship.
• If most of the 365.6 MB is file-backed and reclaimable, the memory recovered by
  switching is much smaller than the headline, and the success criteria need restating in
  absolute terms against the device's real memory budget with the full marine stack
  running.
• If a capped QuestDB is materially worse on CPU or latency, or cannot stay healthy, the
  premise survives and the plan can be rebuilt against the review's structural findings.

Related

Tracking issue for the plan, and its nine unit issues, which are closed as superseded
pending this answer.