Python API

MaruHandler

The main client interface for Maru. Handles storing, retrieving, and deleting KV cache entries in CXL shared memory via MaruServer.

from maru import MaruConfig, MaruHandler

Usage

config = MaruConfig(server_url="tcp://localhost:5555", pool_size=1024 * 1024 * 100)

with MaruHandler(config) as handler:
    data = b"hello world"

    # Store: alloc → write → register
    handle = handler.alloc(size=len(data))
    handle.buf[:len(data)] = data
    handler.store(key=12345, handle=handle)

    # Retrieve (zero-copy memoryview into CXL memory)
    result = handler.retrieve(key=12345)

    # Check existence
    handler.exists(key=12345)

    # Delete
    handler.delete(key=12345)

Key Behaviors

  • Auto-connect: Connects automatically on init when auto_connect=True (default)

  • Auto-expand: Automatically allocates new memory regions when current ones are full

  • Deduplication: Skips store if the key already exists

  • Zero-copy retrieve: Returns memoryview pointing directly into mmap region

  • Thread-safe: Read operations (retrieve, exists) are lock-free; write operations (store, delete) are serialized by an internal write lock

Methods

class maru_handler.MaruHandler(config: MaruConfig | None = None)[source]

Bases: object

Initialize MaruHandler.

Parameters:

config – Configuration object. If None, uses defaults.

connect() → bool[source]

Connect to the server and request a memory allocation.

Returns:

True if successful

close() → None[source]

Close the connection and return all allocations.

Sets _closing event to reject new operations, then acquires _write_lock to wait for in-flight writes before teardown.

alloc(size: int) → AllocHandle[source]

Allocate a page and return a handle with a writable mmap memoryview.

The caller writes directly to handle.buf, then passes the handle to store(key, handle=handle) to register without copying.

Parameters:

size – Required bytes (must be <= chunk_size)

Returns:

AllocHandle with writable memoryview

Raises:

  • RuntimeError – If not connected or closing

  • ValueError – If size exceeds chunk_size or allocation fails

free(handle: AllocHandle) → None[source]

Free a page previously obtained via alloc().

Can be called before store() (discard) or after (eviction).

Parameters:

handle – AllocHandle from alloc()

Raises:

ValueError – If handle is not tracked (already freed or invalid)

store(key: str, info: MemoryInfo | memoryview | None = None, prefix: bytes | None = None, *, data: memoryview | None = None, handle: AllocHandle | None = None) → bool[source]

Store data to the KV cache.

If handle is provided (zero-copy path), data is already written to the mmap region via alloc() and only register_kv is performed. Otherwise, allocate + memcpy + register are performed in one call.

Parameters:

  • key – The chunk key string

  • info – MemoryInfo or memoryview with data

  • prefix – Optional bytes to prepend (e.g., serialized metadata header)

  • data – memoryview with data (preferred, keyword-only)

  • handle – AllocHandle from alloc() for zero-copy store

Returns:

True if successful

retrieve(key: str) → MemoryInfo | None[source]

Retrieve a zero-copy MemoryInfo from the KV cache.

Returns a MemoryInfo with a memoryview slice of the mmap region. Works for both owned (RW) and shared (RO) regions.

WARNING: The returned memoryview is only valid while the region remains mapped. Do not use after calling close().

Parameters:

key – The chunk key string

Returns:

MemoryInfo with memoryview, or None if not found

exists(key: str) → bool[source]

Check if a key exists.

Parameters:

key – The chunk key string

Returns:

True if exists

delete(key: str) → bool[source]

Delete a key and free the corresponding page.

Parameters:

key – The chunk key string

Returns:

True if deleted

healthcheck() → bool[source]

Check if the handler and MaruServer are healthy.

Verifies local connection state and sends a heartbeat RPC to confirm the MaruServer is responsive.

Returns:

True if connected and server responded to heartbeat

get_stats() → dict[source]

Get server statistics.

batch_retrieve(keys: list[str]) → list[MemoryInfo | None][source]

Retrieve multiple values as MemoryInfo in batch.

Uses a single batch RPC call for lookup, returns zero-copy memoryview slices for both owned (RW) and shared (RO) regions.

WARNING: Returned memoryviews are only valid while regions remain mapped.

On RPC failure, allocated pages are freed but data already written to those pages is not zeroed. This is safe because the pages are never registered with the server and will be overwritten on reuse.

Parameters:

keys – List of chunk key strings

Returns:

List of MemoryInfo (None for keys not found)

batch_store(keys: list[str], infos: list[MemoryInfo | memoryview], prefixes: list[bytes | None] | None = None) → list[bool][source]

Store multiple key-value pairs in batch.

Uses a single batch RPC call for registration.

Parameters:

  • keys – List of chunk key strings

  • infos – List of MemoryInfo or memoryview with data

  • prefixes – Optional list of prefix bytes per entry

Returns:

List of booleans indicating success for each key

batch_exists(keys: list[str]) → list[bool][source]

Check if multiple keys exist.

Uses a single batch RPC call instead of N individual calls.

Parameters:

keys – List of chunk key strings

Returns:

List of booleans indicating existence for each key

property pool_handle: _MockModule.MaruHandle | None

Get initial pool handle (backward compat).

property allocator: PagedMemoryAllocator | None

Get the first region’s allocator (backward compat).

property owned_region_manager: OwnedRegionManager | None

Get the owned region manager.

property instance_id: str

Get instance ID.

property connected: bool

Check if connected.

AllocHandle

Handle returned by alloc() for zero-copy writes. The caller writes data directly to buf (a writable memoryview into CXL shared memory), then passes the handle to store() to register the key.

# 1. Allocate a page
handle = handler.alloc(size=1024 * 1024)

# 2. Write directly to CXL memory (zero-copy)
handle.buf[:len(data)] = data

# 3. Register the key (only metadata is sent)
handler.store(key=42, handle=handle)

Property

Type

Description

buf

memoryview

Writable view into the allocated CXL memory page

size

int

Requested allocation size in bytes

region_id

int

Region ID of the allocated page

page_index

int

Page index within the region
class maru_handler.memory.AllocHandle(buf: memoryview, _region_id: int, _page_index: int, _size: int)[source]

Bases: object

Handle returned by MaruHandler.alloc() for zero-copy writes.

Caller writes directly to buf (an mmap memoryview), then passes this handle to store(key, handle=handle) to register without copy.

property region_id: int

Region ID of the allocated page.

property page_index: int

Page index within the region.

property size: int

Requested allocation size in bytes.

MemoryInfo

Zero-copy view into CXL shared memory, returned by retrieve(). Wraps a memoryview pointing directly into an mmap’d CXL region.

result = handler.retrieve(key=42)
if result is not None:
    # result.view is a memoryview into CXL shared memory
    data = bytes(result.view)
    length = len(result.view)

Property

Type

Description

view

memoryview

Zero-copy view into the CXL memory region (read-only for cross-instance retrieves)
class maru_handler.memory.MemoryInfo(view: memoryview)[source]

Bases: object

Zero-copy data descriptor using memoryview.

Interface type between LMCache (MemoryObj) and Maru (mmap region). Supports both RW and RO regions via a single memoryview field.

PUT: connector creates MemoryInfo(view=memory_obj.byte_array)

handler writes via memoryview slice assignment (buf[off:off+n] = view)

GET: handler returns MemoryInfo(view=memoryview_slice)

connector creates tensor via torch.frombuffer(info.view)

Data size is available via len(view) or view.nbytes.