> For the complete documentation index, see [llms.txt](https://docs.rumi.systems/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.rumi.systems/performance/benchmark-suite.md).

# Benchmark Suite

The Rumi Performance Benchmark Suite is a collection of benchmarking tools that measure the performance of individual Rumi runtime components. These tools serve both as performance measurement utilities and as sample code demonstrating Rumi best practices.

## What's in the Suite

The benchmark suite contains 7 modules, each focusing on a specific Rumi component:

| Module                                                                        | Component       | Description                                       |
| ----------------------------------------------------------------------------- | --------------- | ------------------------------------------------- |
| [Time](/performance/benchmark-suite/modules/time-module.md)                   | Time API        | Measures overhead of Rumi's time API              |
| [Serialization](/performance/benchmark-suite/modules/serialization-module.md) | Encoding (ADM)  | Message serialization/deserialization performance |
| [Link](/performance/benchmark-suite/modules/link-module.md)                   | Link            | Cluster replication link throughput and latency   |
| [Messaging](/performance/benchmark-suite/modules/messaging-module.md)         | Messaging (SMA) | Pub/sub messaging layer performance               |
| [Persistence](/performance/benchmark-suite/modules/persistence-module.md)     | Persistence     | Message and data persistence performance          |
| [Storage](/performance/benchmark-suite/modules/storage-module.md)             | Store (ODS)     | Object store operation performance                |
| [AEP](/performance/benchmark-suite/modules/aep-module.md)                     | Engine (AEP)    | **End-to-end canonical benchmark**                |

## Relationship to Canonical Benchmark

The **AEP Module** contains the [Canonical Benchmark](/performance/canonical-benchmark.md) - the official end-to-end performance test used to measure Rumi's published performance metrics. While the canonical benchmark measures the complete Receive-Process-Send flow, the other modules isolate and test individual components.

## Source Repository

The benchmark suite source code is available at:

* **Repository**: [github.com/rumidata/nvx-rumi-perf](https://github.com/rumidata/nvx-rumi-perf)
* **Organization**: Multi-module Maven project
* **License**: Embedded (no separate license required to run)

## Releases

### Versioning

A benchmark suite release is published for each Rumi release. The benchmark version matches the Rumi version it tests.

### Distribution Naming

Distributions are named as follows:

**Linux and macOS:**

```
nvx-rumi-perf-dist-{version}-{arch}.tar.gz
```

**Windows:**

```
nvx-rumi-perf-dist-{version}-{arch}.zip
```

Valid architectures (`arch`):

* `linux-x86-64` - Linux (recommended for performance testing)
* `osx-x86-64` - macOS (development only)
* `win-x86-64` - Windows (development only)

**Example**: The Linux distribution for version 4.0.4 is named:

```
nvx-rumi-perf-dist-4.0.4-linux-x86-64.tar.gz
```

**Important**: Only Linux distributions include Rumi native libraries required for zero-garbage operation. Windows and macOS distributions can be used for development, but Linux is required for full performance optimization.

### Download

Download distributions from the Neeve artifact repository:

**Linux:**

```bash
wget https://nexus.n5corp.com/repository/maven-public/com/neeve/nvx-rumi-perf-dist/4.0.4/nvx-rumi-perf-dist-4.0.4-linux-x86-64.tar.gz
```

**macOS:**

```bash
wget https://nexus.n5corp.com/repository/maven-public/com/neeve/nvx-rumi-perf-dist/4.0.4/nvx-rumi-perf-dist-4.0.4-osx-x86-64.tar.gz
```

**Windows:**

```bash
wget https://nexus.n5corp.com/repository/maven-public/com/neeve/nvx-rumi-perf-dist/4.0.4/nvx-rumi-perf-dist-4.0.4-win-x86-64.zip
```

Replace `4.0.4` with your desired version.

## Running Benchmarks

### 1. Extract Distribution

**Linux/macOS:**

```bash
tar xvf nvx-rumi-perf-dist-{version}-{arch}.tar.gz
cd nvx-rumi-perf-{version}
```

**Windows:**

```cmd
unzip nvx-rumi-perf-dist-{version}-win-x86-64.zip
cd nvx-rumi-perf-{version}
```

This creates:

```
nvx-rumi-perf-{version}/
├── conf/      # Configuration files
└── libs/      # All JARs and dependencies
```

### 2. Run a Benchmark

**With Java 8:**

*Linux/macOS:*

```bash
$JAVA_HOME/bin/java -cp "libs/*" {BenchmarkClass} {parameters}
```

*Windows:*

```cmd
%JAVA_HOME%\bin\java -cp "libs\*" {BenchmarkClass} {parameters}
```

**With Java 11 or later:**

*Linux/macOS:*

```bash
$JAVA_HOME/bin/java \
  --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED \
  --add-opens=java.base/sun.nio.ch=ALL-UNNAMED \
  --add-opens=java.base/java.lang=ALL-UNNAMED \
  --add-opens=java.base/java.nio=ALL-UNNAMED \
  --add-opens=java.base/java.io=ALL-UNNAMED \
  --add-opens=java.management/sun.management=ALL-UNNAMED \
  --illegal-access=warn \
  -cp "libs/*" {BenchmarkClass} {parameters}
```

*Windows:*

```cmd
%JAVA_HOME%\bin\java ^
  --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED ^
  --add-opens=java.base/sun.nio.ch=ALL-UNNAMED ^
  --add-opens=java.base/java.lang=ALL-UNNAMED ^
  --add-opens=java.base/java.nio=ALL-UNNAMED ^
  --add-opens=java.base/java.io=ALL-UNNAMED ^
  --add-opens=java.management/sun.management=ALL-UNNAMED ^
  --illegal-access=warn ^
  -cp "libs\*" {BenchmarkClass} {parameters}
```

### Example - Serialization Benchmark

**With Java 8:**

```bash
$JAVA_HOME/bin/java -cp "libs/*" com.neeve.perf.serialization.Driver --provider protobuf.random
```

**With Java 11+:**

```bash
$JAVA_HOME/bin/java \
  --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED \
  --add-opens=java.base/sun.nio.ch=ALL-UNNAMED \
  --add-opens=java.base/java.lang=ALL-UNNAMED \
  --add-opens=java.base/java.nio=ALL-UNNAMED \
  --add-opens=java.base/java.io=ALL-UNNAMED \
  --add-opens=java.management/sun.management=ALL-UNNAMED \
  --illegal-access=warn \
  -cp "libs/*" com.neeve.perf.serialization.Driver --provider protobuf.random
```

## Next Steps

* **Module Documentation**: See the [Modules](/performance/benchmark-suite/modules.md) section for detailed information on each benchmark module, including program classes, parameters, and interpreting results
* **Canonical Benchmark**: Learn about the [official end-to-end benchmark](/performance/canonical-benchmark.md)
* **Build from Source**: See the [nvx-rumi-perf repository](https://github.com/rumidata/nvx-rumi-perf) for build instructions


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.rumi.systems/performance/benchmark-suite.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.