> For the complete documentation index, see [llms.txt](https://docs.rumi.systems/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.rumi.systems/performance/benchmark-suite/modules/storage-module.md).

# Storage Module

The Storage module benchmarks Rumi's ODS (Object Data Store) performance, measuring object operations in a transactional context.

## Overview

The ODS provides:

* In-memory object storage
* Transactional semantics
* Indexed access
* Persistence integration

This benchmark measures the overhead of object operations within the Rumi engine.

## Test Programs

### ESMember

**Class**: `com.neeve.perf.ods.ESMember`

Benchmarks object store operations using Event Sourcing HA policy.

## Operations Benchmarked

The benchmark measures:

* Object creation
* Object updates
* Object queries (by key)
* Object iteration
* Index lookups

## Command-Line Parameters

### General Parameters

| Short | Long                   | Default  | Description                   |
| ----- | ---------------------- | -------- | ----------------------------- |
| `-c`  | `--count`              | 10000000 | Number of commits to perform  |
| `-t`  | `--warmupTime`         | 2        | Warmup time in seconds        |
| `-r`  | `--rate`               | -1       | Commit rate (-1 = unlimited)  |
| `-n`  | `--numPerCommit`       | 1        | Number of messages per commit |
| `-j`  | `--affinity`           | null     | CPU affinity mask             |
| `-a`  | `--noLatencyWrites`    | false    | Don't write latencies to file |
| `-b`  | `--printIntervalStats` | false    | Print interval stats          |
| `-h`  | `--help`               | false    | Print help                    |

### Persistence Parameters

| Short | Long                       | Default        | Description                        |
| ----- | -------------------------- | -------------- | ---------------------------------- |
| `-m`  | `--logMode`                | rw             | Log open mode (rw, rws, rwd)       |
| `-i`  | `--initialLogLength`       | 1              | Preallocated log length (GB)       |
| `-z`  | `--zeroOutInitial`         | false          | Zero out preallocated length       |
| `-a`  | `--writeBufferSize`        | 8192           | Write buffer size (bytes)          |
| `-b`  | `--readBufferSize`         | 8192           | Read buffer size (bytes)           |
| `-f`  | `--flushOnCommit`          | false          | Flush on every commit              |
| `-m`  | `--flushUsingMappedMemory` | false          | Use memory-mapped flush            |
| `-d`  | `--detached`               | false          | Enable detached writes             |
| `-q`  | `--queueDepth`             | 1024           | Queue depth for detached writes    |
| `-l`  | `--publisherClaimStrategy` | SingleThreaded | Disruptor publisher claim strategy |
| `-w`  | `--writerWaitStrategy`     | Yielding       | Disruptor writer wait strategy     |
| `-x`  | `--writerAffinity`         | \[0]           | Disruptor writer thread affinity   |
| `-p`  | `--pageSize`               | 4096           | Page size for disk I/O             |

## Running Benchmarks

### Basic Store Operations Test

```bash
$JAVA_HOME/bin/java -cp "libs/*" com.neeve.perf.ods.ESMember \
  -c 10000000 \
  -r -1 \
  -n 1
```

### With Persistence Enabled

```bash
$JAVA_HOME/bin/java -cp "libs/*" com.neeve.perf.ods.ESMember \
  -c 10000000 \
  -r -1 \
  -i 10 \
  -f true \
  -d true
```

## Key Parameters

### count

Number of commits to perform:

```bash
-c 10000000  # 10M commits
```

### numPerCommit

Number of messages/operations per commit:

```bash
-n 1  # One operation per commit (default)
-n 100  # Batch 100 operations per commit
```

### detached

Enable detached persistence:

```bash
-d true  # Persistence in separate thread
```

More efficient for high-throughput scenarios.

## Interpreting Results

### Typical Results (Linux x86-64)

| Operation              | Hot Cache     | Cold Cache     |
| ---------------------- | ------------- | -------------- |
| Object Create          | \~500ns       | \~2µs          |
| Object Update          | \~300ns       | \~1.5µs        |
| Object Query (indexed) | \~200ns       | \~1µs          |
| Object Iteration       | \~50ns/object | \~100ns/object |

### Performance Characteristics

1. **Hot Cache Performance**:
   * Sub-microsecond operations
   * Minimal GC impact
   * Excellent for high-frequency access
2. **Memory Efficiency**:
   * Compact object representation
   * Efficient index structures
   * Zero-copy where possible
3. **Transactional Overhead**:
   * Minimal overhead within transactions
   * Batch operations for efficiency

## Comparison with AEP Module

The [AEP Module](/performance/benchmark-suite/modules/aep-module.md) includes store operations:

* **Storage Module**: Measures pure object operation overhead
* **AEP Module**: Store operations within message processing
* **AEP Impact**: Store operations add < 1µs to end-to-end latency

**Store operations are a tiny fraction of end-to-end latency**

## Best Practices

### For High Performance

1. **Use Indexes Wisely**:
   * Index frequently-queried fields
   * Avoid over-indexing (impacts updates)
   * Choose appropriate index types
2. **Batch Operations**:
   * Group related updates in single transaction
   * Reduces transaction overhead
3. **Memory Management**:
   * Size heap appropriately for object count
   * Monitor GC behavior
   * Use off-heap storage for very large datasets

### For Large Datasets

```java
// GOOD: Batch updates in single transaction
transaction(() -> {
    for (Order order : orders) {
        orderStore.update(order);
    }
});

// BAD: Separate transaction per update
for (Order order : orders) {
    transaction(() -> {
        orderStore.update(order);  // Transaction overhead per update!
    });
}
```

## Object Store Features

### Indexing

Automatic index maintenance:

```java
@Store
@Indexes({
    @Index(fields = {"customerId"}),
    @Index(fields = {"orderDate", "customerId"})
})
public interface Order {
    String getOrderId();
    String getCustomerId();
    LocalDate getOrderDate();
    // ...
}
```

### Transactions

ACID semantics:

* **Atomicity**: All or nothing
* **Consistency**: Constraints enforced
* **Isolation**: Serializable transactions
* **Durability**: Persisted if enabled

### Replication

Automatic state replication in clustered deployments:

* State changes replicated to backup
* Backup maintains identical state
* Transparent failover

## Next Steps

* Review [AEP Module](/performance/benchmark-suite/modules/aep-module.md) to see store usage in end-to-end context
* Review [Persistence Module](/performance/benchmark-suite/modules/persistence-module.md) for storage layer benchmarks
* Return to [Benchmark Suite](/performance/benchmark-suite.md) for other modules


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.rumi.systems/performance/benchmark-suite/modules/storage-module.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.