[MP][Maru] Maru CXL shared L1 backend for MP mode by seohui-XCENA · Pull Request #20 · xcena-dev/LMCache

seohui-XCENA · 2026-07-03T08:38:02Z

내부 공유용 초안. upstream 제출 전 테스트 trim + 영문화 예정.

디자인 문서:

https://metisx.atlassian.net/wiki/x/HgCjdw : maru l1 design
https://metisx.atlassian.net/wiki/x/KQCtdw : L1Manager API 관점 - maru가 구현해야 할 것

요약

MP mode의 L1 tier를 CPU DRAM 대신 cross-instance 공유 CXL 풀(Maru)로 교체한다.
그 위의 LMCache 동작 — L1↔L2 티어링(write-through / discard-evict / promote-on-miss),
컨트롤러, L2 어댑터, eviction — 은 기존 그대로 maru L1 위에서 돈다.

핵심 설계 결정:

maru 로직은 MaruL1Manager 한 클래스에 격리하고, 기존 스택은 L1ManagerInterface
Protocol seam으로 그 위에서 무변경 동작시킨다.

Control(LMCache) / Directory(MaruServer) 분리: store/evict/promote/pin 결정은 LMCache의 기존
컨트롤러가, MaruServer는 그 결정을 실행·기록하는 passive 공유 디렉터리(key→region/offset +
pin_count).
공유 매체: 여러 인스턴스가 같은 CXL 물리 메모리를 mmap → zero-copy, P2P/전송 없음.

설계 문서: docs/design/v1/distributed/maru_l1.md (이 PR에 포함) — sibling + Protocol seam, API 계약, 상태머신 불변식, TTL sweeper·crash-recovery 전제.

통합 방식 (sibling + 구조적 Protocol)

StorageManager가 config에 따라 L1을 한 줄로 선택:
MaruL1Manager(cfg) if maru_config else L1Manager(cfg) — 둘 다 L1ManagerInterface 만족.
l1_protocol.py(신규): 구조적 typing.Protocol L1ManagerInterface(17 메서드, @runtime_checkable).
L1Manager·MaruL1Manager 둘 다 상속 없이 구조적으로 만족.
기존 컨트롤러 3개(Store/Prefetch/L1Eviction)는 l1_manager 파라미터 타입만 L1ManagerInterface로
확장(런타임 동작 동일).

코어 무침습

dev와 byte-identical (무변경): l1_manager.py, l1_memory_manager.py, internal_api.py,
모든 컨트롤러 로직, L2 어댑터.

실제로 편집되는 건 배선점뿐이고 전부 additive/behavior-neutral:

컨트롤러 시그니처(타입 확장) — store/prefetch/eviction_controller.py
storage_manager.py — L1 선택 분기 + maru-only register_kv_layout wrapper
config.py — maru_config 필드 + CLI + startup guard
엔진 훅 1곳 — lmcache_driven_transfer.py의 register_kv_layout(default 백엔드엔 no-op)
serde_wrapper.py — L2 어댑터 l1_manager 타입을 Protocol로 위젠(behavior-neutral)

커밋 구성 (C1–C11)

C	내용
C1	`L1ManagerInterface` protocol + maru L1 config
C2	`MaruMemoryAllocator` (CXL-backed allocator)
C3	`MaruL1Manager` RPC control 표면 (read/write/delete/lifecycle)
C4	L1 리스너 발화 + `touch_keys` real화
C5	`finish_write_and_reserve_read` (L2→L1 promote, temporary/retained)
C6	orphan write/read-pin 회수 TTL sweeper
C7	컨트롤러 `l1_manager` 파라미터 → `L1ManagerInterface` 위젠
C8	StorageManager L1 선택 분기 + `register_kv_layout` seam
C9	미지원 조합 거부 (gds/devdax/L2-RDMA/skip_l1/p2p/engine)
C10	`register_kv_cache`에 `register_kv_layout` 엔진 훅 배선 (+deferred 훅 테스트)
C11	event_bus parity + 제어 통합 테스트 + conformance 스위트

테스트

테스트 10파일(maru_fakes in-memory 하네스 포함): manager 단위, Protocol 정합(test_l1_protocol),
config guard, single-region, 제어 통합(기존 티어링 컨트롤러 위에서 maru 구동), register_kv_layout,
memory_allocator, conformance(같은 계약을 기존 L1Manager와 MaruL1Manager에 동시 실행 → 드리프트 차단).
전체 maru 스위트 gpu1(CUDA_VISIBLE_DEVICES=1)에서 통과.
⚠️ 지금은 구현단계 테스트라 단계별 검증이 많음 → upstream PR 전 최소 필요분으로 trim 예정.

범위 · 알려진 한계

single-model / single-object-group only: 첫 init_layout에서 pool을 단일 layout으로 고정 →
다른 모델·하이브리드(num_object_groups>1)는 fail-fast ValueError. 같은 모델 멀티-인스턴스는 정상.
LRU는 per-instance 로컬 뷰: policy가 이 인스턴스의 리스너 이벤트로만 채워짐(다른 노드 recency는 모름).
orphan 회수 = 프로세스 생존 시에만: TTL sweeper는 데몬 스레드라 "흐름만 끊긴" orphan은 회수하지만,
client crash(프로세스 사망) 회수는 이 PR 범위 밖 — maru-side 전제에 의존한다:
(1) MaruServer가 client별 pin을 추적해 disconnect 시 일괄 해제, (2) 죽은 owner의 region이 RM으로 반납
(region owner-release)되며 미등록 write page 회수. 둘 다 maru-side(MaruServer) 책임.

남은 것

upstream PR 전 테스트 trim (구현단계 step-by-step → 최소분).
Phase 2: L1Manager에서 L1StateBackend seam 추출 → 하나의 L1Manager가 local/maru backend를
모두 서빙(MaruL1Manager 삭제). sibling 머지 후 별도 작업.

PR 초안

What this PR does / why we need it:

This PR adds Maru, a CXL shared-memory KV cache engine, as a shared L1 tier for MP mode — the shared-L1 counterpart to the in-process Maru backend from LMCache#2705. Today each MP server has a private, node-local L1 (pinned DRAM / Device-DAX / GDS). With Maru, MP servers on separate nodes mmap one CXL pool and read each other's entries zero-copy: a node that never stored a key still hits it if a peer did, with no network transfer.

Control stays entirely with LMCache. Maru is a sibling L1 manager (MaruL1Manager) behind a new structural L1ManagerInterface Protocol; the stock controllers, L1↔L2 tiering, eviction, and L2 adapters run unchanged on top. The L1 core (l1_manager.py, l1_memory_manager.py, internal_api.py) is byte-identical to dev; the remaining edits are additive wiring only.

Design doc: docs/design/v1/distributed/maru_l1.md · Maru: docs / github

Benchmark:

setup

Hardware: 1× NVIDIA RTX PRO 6000 Blackwell Server Edition (96 GB), Intel Xeon 6730P (Granite Rapids), 1 TB DRAM; CXL pool via Device-DAX (/dev/dax9.0, 500 GB), served by maru-server + maru-resource-manager
Versions: vLLM 0.24.0 · LMCache v0.1.dev1942 · Maru 0.1.0 · model Qwen/Qwen3-8B (bf16 KV)
Workload: lmcache bench engine, long-doc-qa — one exported config replayed unchanged against all three setups: 40 GB KV volume → 29 documents × 10k tokens (tokens_per_gb_kvcache=7281), 1 query/doc, tile order, 4 in-flight, 128 output tokens (ignore_eos), seed 42
vLLM identical in all runs: --gpu-memory-utilization 0.5 (GPU KV ≈ 30 GiB ≈ 220k tokens < 290k-token workload, so the GPU cache alone cannot hold the working set), default prefix caching, chunk size 256
LMCache server (MP mode, lmcache_driven): B --l1-size-gb 60 (DRAM L1, default lazy allocator) · B′ same + --no-l1-use-lazy (preallocated MixedMemoryAllocator) · C --maru-server-url maru://localhost:5555 --maru-pool-size-gb 60 --l1-size-gb 0 (Maru CXL L1); same 60 GB capacity, no eviction in any run
All cached runs had identical cache behavior — warm-up pass all-miss, measured pass 100% hit (289,536 / 289,536 lookup tokens, from Prometheus counters) — so the deltas between B/B′/C isolate the L1 medium and allocator, not hit-rate differences

Metric	A: vLLM alone	B: MP + DRAM L1 (lazy alloc, default)	B′: MP + DRAM L1 (mixed alloc)	C: MP + Maru L1
Mean TTFT (ms)	1404.0	215.7 (−84.6%)	178.3 (−87.3%)	184.6 (−86.9%)
P50 TTFT (ms)	1244.7	206.5	175.6	181.3
P90 TTFT (ms)	2022.2	234.0	197.7	204.8
P99 TTFT (ms)	2032.7	376.2	263.2	283.0
Benchmark duration (s)	30.13	17.73	16.69	16.58
Input throughput (tok/s)	9,639	16,380	17,395	17,516
Output throughput (tok/s)	123.2	209.4	222.4	223.9
Mean decode speed (tok/s)	53.1	66.3	66.7	66.6
Mean request latency (ms)	3,905	2,150	2,101	2,109

All runs: 29/29 requests successful, 290,396 input tokens.
B′ = B with --no-l1-use-lazy (preallocated pinned-DRAM MixedMemoryAllocator instead of the default LazyMemoryAllocator).

Maru CXL L1 vs DRAM L1: with identical 100% hit rates

Grafana (LMCache MP dashboard) during the runs — L1 usage filling over the warm-up pass and 100%-hit read phase:

DRAM L1 (run B)	Maru L1 (run C)

Special notes for your reviewers:

Control / directory split. LMCache decides everything (store / evict / promote / pin); the passive MaruServer is only a shared directory (key → region/offset) plus a cross-node pin_count. Page reclamation is driven by LMCache eviction, so the allocator's free is a no-op.
Why a sibling, not an allocator swap. A private tier keeps membership and locks in a local dict; a shared tier needs them authoritative in the shared directory (cross-node reads and pins). MaruMemoryAllocator supplies the CXL medium, MaruL1Manager the shared control. (The MP Coordinator is unrelated: it plans fleet-wide L2 eviction on logical keys, not L1 physical-page state.) A later L1StateBackend seam to shrink the sibling is possible follow-up, out of scope here.
Startup guards reject unsupported combos up front: gds/devdax, skip_l1 store policy, registered/RDMA-type L2 (copy-type only), p2p, and engine_driven/auto transfer mode.

If applicable:

this PR contains user facing changes - docs added (docs/source/mp/configuration.rst + design doc)
this PR contains unit tests

jooho-XCENA

docs/coding_standards.md 기준 리뷰입니다. error 항목만 코멘트로 답니다 (warning/info는 별도 공유).

[error] 신규 maru L1 백엔드에 in-repo 설계 문서가 없고, PR 설명이 인용하는 설계 정본이 dangling 참조임

PR 설명은 temp_docs/mp/design/maru_l1_manager.md · temp_docs/mp/design/maru_l1_api_mapping.md를 설계 정본으로 인용하지만, temp_docs/는 PR 트리·브랜치 히스토리 어디에도 없습니다 (git ls-tree -r HEAD | grep temp_docs → 없음). 이 PR은 maru_l1_manager.py(1,057줄) · maru_memory_allocator.py(243줄) · l1_protocol.py(113줄)를 추가하면서 docs/를 건드리지 않아, coding_standards §5.1(non-trivial 신규 기능은 docs/design/<path>/ 설계 문서 필수)에 걸리고 리뷰어가 설계 정합 검토(리뷰 Step 2)를 할 수 없습니다.

수정 제안: 설계 정본을 docs/design/v1/distributed/(예: maru_l1.md — MaruL1Manager / MaruMemoryAllocator / L1ManagerInterface seam / orphan sweeper·crash-recovery 전제 포함)로 이 PR에 추가하고, PR 설명의 참조를 in-repo 경로로 갱신해 주세요.

kihwan-XCENA

주호님이 위에 리뷰로 언급하신 것 처럼 _pending_write와 _pending_read가 동시에 존재 할 수 있는 상황이 되어도 괜찮을까요?
하나의 KV Cache가 _pending_write 및 _pending_read에 동시에 저장되는 경우가 있는 것 같습니다.

seohui-XCENA · 2026-07-07T04:48:13Z

[error] 설계 문서 (in-repo) — 수정했습니다 (df00f4a6).

docs/design/v1/distributed/maru_l1.md를 이 PR에 추가했습니다 — sibling + Protocol seam, MaruL1Manager API 계약, pending-read/write 상태머신 불변식, TTL sweeper, crash-recovery 전제 포함(리뷰용 요약, 계약 중심). PR 설명의 temp_docs/... 참조도 이 in-repo 경로로 갱신했습니다.

seohui-XCENA · 2026-07-07T04:48:14Z

@kihwan-XCENA 좋은 지적 감사합니다 — 실제로 가능한 버그였습니다(주호님 리뷰의 mid-write reserve_read 건과 동일 뿌리).

reserve_read가 mid-write(_pending_write) 키를 KEY_NOT_READABLE로 pre-mark만 하고 pin/stage엔 그대로 넘겨서, peer가 그 키를 디렉터리에 등록해 둔 경우 batch_pin 성공 → _pending_read에도 추가돼 이중 스테이징이 됐습니다. 48581006에서 mid-write 키를 pin/stage 대상에서 제외해 한 키는 _pending_write/_pending_read 중 한 곳에만 존재하도록 불변식을 복원했고, 회귀 테스트도 추가했습니다.

- l1_protocol.py: structural runtime_checkable Protocol mirroring L1Manager's 17-method surface, with per-method listener/lock contract docstrings - config.py: MaruL1Config + maru_config field, __post_init__ DRAM-clamp skip, maru CLI args (--maru-server-url/--maru-pool-size-gb/--maru-instance-id), parse_args_to_config wiring - tests: interface<->L1Manager method-set + signature conformance, maru config parsing Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

- thin wrapper over external maru_lmcache.CxlMemoryAdapter, lazy-imported - two-phase init_layout: build MaruHandler + CxlMemoryAdapter on first layout (single-model; layout mismatch rejected) - free/batched_free no-op (page lifecycle owned by MaruServer); abort_alloc discards an allocated-but-unregistered page - MaruL1Config -> maru.MaruConfig mapping; maru-only get_by_location / create_store_handle / handler surface for MaruL1Manager - tests use mocked maru runtime (no CXL required) Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

…ifecycle) - sibling of L1Manager over the maru shared CXL pool: membership/read protection live in the MaruServer directory (pin_count), locally only in-flight staging (_pending_read refcount, _pending_write) - reserve_read: per-key independent pins (1+extra_count via one RPC), rollback on partial pin / retrieve failure / unresolvable page - reserve_write mode=new: local staged check + batch_exists dedup (cross-instance), all-or-nothing OOM; finish_write: batch_store, dup-skip is success, definitive False reclaims the page, unknown server state never recycles - delete: staged keys and pinned keys refuse with KEY_IS_LOCKED (exists() disambiguates the handler's pinned/missing conflation) - clear(force=False) keeps locked staging (stock parity); close drains - PARITY/MARU provenance comments; RPC reply length guards - tests: stateful fake maru runtime with fault-injection knobs, failure-path coverage, and a conformance suite parametrized over both L1Manager (CUDA) and MaruL1Manager Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

- reserve_read/reserve_write/finish_write/finish_read/delete/clear now fire the on_l1_keys_* events (feeds the eviction LRU and the store controller), reporting only the keys that actually succeeded - finish_read: temporary reads (local staging, never directory-pinned) are reclaimed via abort_alloc at refcount zero and fire deleted_by_manager; normal reads still unpin -- _drain_staging mirrors the same branch - touch_keys: no-op -> fires on_l1_keys_accessed (unsynchronized, like stock) - event_bus (mp_observability) publish deferred: observability-only, no tiering-control impact - tests: RecordingListener fake; maru firing edge cases (hit-only, store-fail, temporary reclaim, removed-only, clear) + cross-backend conformance lifecycle test parametrized over stock and maru Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

- promote branches on the staged is_temporary flag (Decision A): temporary (default prefetch) moves the loaded page straight to read staging with no batch_store and no pin -- finish_read reclaims it at refcount zero; retained (prefetch_policy: retain) registers via batch_store then re-resolves the authoritative page with pins so a dup-skip that auto-freed our page still yields the winning shared page - fires on_l1_keys_finish_write_and_reserve_read, never on_l1_keys_write_finished (the latter would make the store controller re-store the key to L2) - extract _pin_retrieve_stage (shared by reserve_read + retained promote) and _store_staged (shared by finish_write + retained promote); reserve_read and finish_write refactored onto them, behavior unchanged - document the load-failure cleanup gap (finish_write->delete on failed keys publishes then removes; caller-side batch-abort is the future fix) - tests: temporary/retained promote, dup-skip re-resolve, extra_count pins, wrong-state/unstaged guards, store-failure, anti re-store event check; cross-backend conformance for retained promote, temporary drop, promote event Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

- daemon sweeper (started in __init__, stopped in close) scans staging under the manager lock and reclaims entries whose TTL has elapsed: an expired write page is returned to the owner (abort_alloc); an expired read releases its pins (a temporary read reclaims its private page instead) - _PendingRead/_PendingWrite carry a monotonic deadline (default never); set at reserve/promote and refreshed on overlapping reserve (mirrors a stock re-lock extending the TTL). abandonment is a time judgement -- a refcount says how many holds exist, not whether they will ever be released - no listener fires on sweep: a late finish_read/unsafe_read then sees KEY_NOT_EXIST and recomputes (same failure path as a stock TTL expiry), and firing across the daemon thread would be a novel hazard for stock listeners - tests: sweeper lifecycle, expired write/read/temporary reclaim, live staging left intact, overlapping reserve refreshes the deadline, late finish is KEY_NOT_EXIST Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

- store/prefetch/eviction controllers type their l1_manager param as the structural L1ManagerInterface instead of the concrete L1Manager, so either L1Manager or MaruL1Manager can drive them; runtime behavior unchanged - the controllers call only Protocol methods (reserve/finish read+write, finish_write_and_reserve_read, delete, is_key_evictable, get_memory_usage, register_listener), so L1Manager still satisfies the param structurally Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

…_layout - StorageManager.__init__ selects the L1 backend: MaruL1Manager when memory_config.maru_config is set, else the stock L1Manager; typed as the shared L1ManagerInterface so the controllers drive either unchanged - new StorageManager.register_kv_layout: maru-gated (isinstance) forward that brings up the CXL pool once the layout is known, rejecting >1 object group (single-model maru limit); a no-op for stock. Engine call site lands later - get_l1_memory_desc is now L1MemoryDesc | None (maru has no single registerable region); the l1_memory_desc property raises if accessed while None (its only consumer, p2p, is rejected at startup for maru) - widen SerdeL2AdapterWrapper l1_manager param to L1ManagerInterface (like the controllers); collect its reserve_write buffers in one pass so the success path types as list[MemoryObj] - tests: maru vs stock selection, register_kv_layout forward / multi-group reject / stock no-op Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

…/p2p/engine) - validate_storage_manager_config: a maru L1 config rejects the other L1 backends (gds-l1-path, l1-devdax-path), store_policy=skip_l1, and L2 adapters that require a single registerable region (nixl / mooncake-rdma); copy-type L2 and the default store policy pass - l1_exposes_single_memory_region returns False for maru (the shared CXL pool has no single registerable region), so the existing p2p startup guard fires; its message now lists maru too - run_http_server rejects maru paired with a non-lmcache_driven transfer mode (engine-driven/auto assume engine-side buffers the shared pool lacks) - tests: maru + gds/devdax/skip_l1/registered-L2 raise; maru + copy-L2 and the default policy pass; l1_exposes False for maru Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

…ol bring-up - after the KV layout is resolved in register_kv_cache, call StorageManager.register_kv_layout with the layout desc, the storage MemoryFormat (KV_MLA_FMT / KV_2LTD via is_mla), chunk size, and object-group count; a no-op for the stock backend, idempotent for maru across instances - on failure (e.g. maru rejecting >1 object group) close the just-built cache context and re-raise so the instance is not left half-registered - runtime test deferred: importing this module needs a c_ops rebuild (built .so lacks execute_object_group_transfer); lands with the C9 G6 test post-rebuild Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

…ister_kv_layout) - G6 (C9): run_http_server rejects maru paired with a non-lmcache_driven transfer mode - C10: register_kv_cache forwards to StorageManager.register_kv_layout with the right MemoryFormat (KV_MLA_FMT / KV_2LTD via is_mla); on a rejected layout it closes the cache context and leaves the instance unregistered; stock registers normally - both were blocked by a stale c_ops build (missing execute_object_group_transfer); unblocked after the rebuild Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

- MaruL1Manager now publishes the same L1 events to the shared event bus as stock L1Manager, alongside the listener notifications: L1_READ_RESERVED, L1_READ_FINISHED (+ L1_KEYS_EVICTED for freed temporaries), L1_WRITE_RESERVED, L1_WRITE_FINISHED, L1_WRITE_FINISHED_AND_READ_RESERVED, and L1_KEYS_EVICTED for delete/clear; touch_keys stays listener-only (matches stock) - add get_event_bus() in __init__ and a _publish helper mirroring stock - closes the observability gap deferred in C4 -- maru L1 ops now surface in the mp_observability dashboards - tests: write/read/delete lifecycle events, promote publishes the promote event (never L1_WRITE_FINISHED), temporary finish_read publishes evicted, touch_keys publishes nothing Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

…stack - a StorageManager harness backed by the maru fakes (fake CXL pool + MaruServer directory) + a mock L2, driving the real Store/Prefetch/Eviction controllers over MaruL1Manager - covers store -> maru directory registration, prefetch of directory-resident keys as a full L1 hit (maru reserve_read), and watermark eviction driving MaruL1Manager.delete on the shared directory - L1<->L2 byte movement (write-through, promote) is stock controller logic that needs real-memory-backed pages and is covered by the stock StorageManager tests, so it is not re-asserted here; extra_count and cross-node PINNED are covered at the manager level (C3/C5) Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

…acts) - add cross-backend conformance cases for the shared error paths: reserve_read of a mid-write key is KEY_NOT_READABLE (not a miss), finish_read/finish_write on an unstaged key is KEY_NOT_EXIST, delete of a missing key is KEY_NOT_EXIST - clear(force=True) is intentionally not conformance-tested: stock clears all objects while maru only drops local staging (the shared directory belongs to other instances) -- a legitimate divergence, not drift Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

- get_memory_usage total = owned pool + CXL device free (cxl_pool.free_size from get_stats), so the eviction watermark tracks whole-device fill instead of just this instance's owned pool (which evicts prematurely while the device still has room) - cache the last-known free and reuse it when a get_stats RPC omits cxl_pool (transient timeout / older server) so total never momentarily collapses to the owned pool and fires a spurious eviction Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

- Track server pins on _PendingRead.pinned separately from refcount, so a temporary stage that absorbs an overlapping reserve_read's pins releases them; previously those pins leaked and left the page un-evictable on MaruServer. - Exclude mid-write keys from reserve_read's pin/stage step: a key mid-write on this instance stays KEY_NOT_READABLE even when a peer has registered it, instead of being double-staged in both _pending_write and _pending_read (which stranded the in-flight write and failed its promote). - Add regression tests for both paths. Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

- Add docstrings to allocate, batched_allocate, close, and memcheck. - Note the RuntimeError raised when called before init_layout(), and the out-of-memory -> None contract that reserve_write's all-or-nothing handling depends on. Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

- Add docs/design/v1/distributed/maru_l1.md: the sibling + Protocol seam, the MaruL1Manager API contracts, the pending-read/pending-write state-machine invariants, the TTL sweeper, and the crash-recovery premises. - Document the Maru CXL shared L1 tier in the MP configuration guide and the architecture index; point the deprecated in-process maru page to it. Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

- Add MaruL1Config.auto_expand (default True) with a --maru-auto-expand / --no-maru-auto-expand CLI flag, threaded into MaruConfig via the allocator. - Branch get_memory_usage on it: auto_expand on keeps the device-fill watermark (owned pool + CXL device free); off anchors total to the owned pool, so a hard-capped pool evicts before it is exhausted instead of OOMing while the device still has free space the pool cannot grow into. - Extend the fake handler with a cxl_free knob and add a watermark test. Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

- Rewrite docs/design/v1/distributed/maru_l1.md to the l2_adapters house style (Overview + components + operation flow + configuration + limits); document the auto_expand knob and the single-device region bound, and drop the internals-heavy framing. - Correct MaruMemoryAllocator docstrings: page reclamation is driven by LMCache eviction via MaruL1Manager (delete / abort_alloc), not the allocator, so free/batched_free are no-ops. Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

- Merge the maru-specific test files into one unit file (test_maru_l1_manager.py): fold in the allocator, config/startup-guard, and control-integration tests, and drop cases already covered by the shared L1Manager conformance suite (notably the notification tests: ~14 -> 4). - Absorb test_l1_protocol.py (structural conformance + shared helpers) into test_l1_manager_conformance.py. - Delete the now-empty test_maru_memory_allocator.py, test_l1_config_maru.py, test_maru_integration.py, and test_l1_protocol.py. Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

- Add --maru-auto-expand / --no-maru-auto-expand to the Maru CXL Shared L1 argument table. - Tighten the single-CXL-device pool-size note and align the tier wording with the design doc (cross-node). Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

…gram Restore the word dropped when MaruL1Manager was added to the EvictionController line, for consistency with the rest of index.rst. Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

- Add the required --l1-size-gb 0 to the quickstart command (ignored under Maru but the parser requires it). - Note that the example assumes a running MaruServer and that higher-level launchers derive --maru-server-url. Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

- lmcache_mp.l1_memory_usage_bytes / l1_usage_ratio are registered in L1Manager.__init__, but the maru path builds MaruL1Manager instead of L1Manager (mutually exclusive), so both gauges silently vanished whenever maru is the L1 backend - mirror the stock registration in MaruL1Manager.__init__: same meter and gauge names, singleton dispatch via _gauge_registered/_gauge_target, and a self-contained ratio helper (no reach into the stock private helper) - conftest: stub maru's register_gauge for distributed tests -- OTel honors only the first registration of a gauge name, so a test-built MaruL1Manager could otherwise claim the name and starve the stock gauge assertions in test_l1_l2_state_metrics.py - test: both gauges registered at construction and their callbacks read the live manager, not the zero fallback Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

seohui-XCENA force-pushed the maru-mp-l1 branch 3 times, most recently from 2477191 to ce54689 Compare July 3, 2026 10:19

jooho-XCENA reviewed Jul 6, 2026

View reviewed changes

Comment thread lmcache/v1/distributed/maru_l1_manager.py

Comment thread lmcache/v1/distributed/maru_l1_manager.py Outdated

Comment thread lmcache/v1/distributed/memory_manager/maru_memory_allocator.py

kihwan-XCENA reviewed Jul 7, 2026

View reviewed changes

seohui-XCENA force-pushed the maru-mp-l1 branch from c5c0638 to df00f4a Compare July 7, 2026 04:38

seohui-XCENA added 20 commits July 7, 2026 17:52

seohui-XCENA force-pushed the maru-mp-l1 branch from 7930d5c to 8572c18 Compare July 7, 2026 08:57

seohui-XCENA added 4 commits July 7, 2026 20:22

[MP][Maru] Keep "watermark-triggered" wording in the architecture dia…

0da8b6b

…gram Restore the word dropped when MaruL1Manager was added to the EvictionController line, for consistency with the rest of index.rst. Signed-off-by: seohui-XCENA <seohui.son@xcena.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[MP][Maru] Maru CXL shared L1 backend for MP mode#20

[MP][Maru] Maru CXL shared L1 backend for MP mode#20
seohui-XCENA wants to merge 25 commits into
devfrom
maru-mp-l1

seohui-XCENA commented Jul 3, 2026 •

edited

Loading

Uh oh!

jooho-XCENA left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kihwan-XCENA left a comment

Uh oh!

seohui-XCENA commented Jul 7, 2026

Uh oh!

seohui-XCENA commented Jul 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

seohui-XCENA commented Jul 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

요약

통합 방식 (sibling + 구조적 Protocol)

코어 무침습

커밋 구성 (C1–C11)

테스트

범위 · 알려진 한계

남은 것

PR 초안

Uh oh!

jooho-XCENA left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kihwan-XCENA left a comment

Choose a reason for hiding this comment

Uh oh!

seohui-XCENA commented Jul 7, 2026

Uh oh!

seohui-XCENA commented Jul 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

seohui-XCENA commented Jul 3, 2026 •

edited

Loading