> For the complete documentation index, see [llms.txt](https://docs.rumi.systems/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.rumi.systems/performance/benchmark-suite.md).

# Benchmark Suite

The Rumi Performance Benchmark Suite is a collection of benchmarking tools that measure the performance of individual Rumi runtime components. These tools serve both as performance measurement utilities and as sample code demonstrating Rumi best practices.

## What's in the Suite

The benchmark suite contains 7 modules, each focusing on a specific Rumi component:

| Module                                                                        | Component       | Description                                       |
| ----------------------------------------------------------------------------- | --------------- | ------------------------------------------------- |
| [Time](/performance/benchmark-suite/modules/time-module.md)                   | Time API        | Measures overhead of Rumi's time API              |
| [Serialization](/performance/benchmark-suite/modules/serialization-module.md) | Encoding (ADM)  | Message serialization/deserialization performance |
| [Link](/performance/benchmark-suite/modules/link-module.md)                   | Link            | Cluster replication link throughput and latency   |
| [Messaging](/performance/benchmark-suite/modules/messaging-module.md)         | Messaging (SMA) | Pub/sub messaging layer performance               |
| [Persistence](/performance/benchmark-suite/modules/persistence-module.md)     | Persistence     | Message and data persistence performance          |
| [Storage](/performance/benchmark-suite/modules/storage-module.md)             | Store (ODS)     | Object store operation performance                |
| [AEP](/performance/benchmark-suite/modules/aep-module.md)                     | Engine (AEP)    | **End-to-end canonical benchmark**                |

## Relationship to Canonical Benchmark

The **AEP Module** contains the [Canonical Benchmark](/performance/canonical-benchmark.md) - the official end-to-end performance test used to measure Rumi's published performance metrics. While the canonical benchmark measures the complete Receive-Process-Send flow, the other modules isolate and test individual components.

## Source Repository

The benchmark suite source code is available at:

* **Repository**: [github.com/rumidata/nvx-rumi-perf](https://github.com/rumidata/nvx-rumi-perf)
* **Organization**: Multi-module Maven project
* **License**: Embedded (no separate license required to run)

## Releases

### Versioning

A benchmark suite release is published for each Rumi release. The benchmark version matches the Rumi version it tests.

### Distribution Naming

Distributions are named as follows:

**Linux and macOS:**

```
nvx-rumi-perf-dist-{version}-{arch}.tar.gz
```

**Windows:**

```
nvx-rumi-perf-dist-{version}-{arch}.zip
```

Valid architectures (`arch`):

* `linux-x86-64` - Linux (recommended for performance testing)
* `osx-x86-64` - macOS (development only)
* `win-x86-64` - Windows (development only)

**Example**: The Linux distribution for version 4.0.4 is named:

```
nvx-rumi-perf-dist-4.0.4-linux-x86-64.tar.gz
```

**Important**: Only Linux distributions include Rumi native libraries required for zero-garbage operation. Windows and macOS distributions can be used for development, but Linux is required for full performance optimization.

### Download

Download distributions from the Neeve artifact repository:

**Linux:**

```bash
wget https://nexus.n5corp.com/repository/maven-public/com/neeve/nvx-rumi-perf-dist/4.0.4/nvx-rumi-perf-dist-4.0.4-linux-x86-64.tar.gz
```

**macOS:**

```bash
wget https://nexus.n5corp.com/repository/maven-public/com/neeve/nvx-rumi-perf-dist/4.0.4/nvx-rumi-perf-dist-4.0.4-osx-x86-64.tar.gz
```

**Windows:**

```bash
wget https://nexus.n5corp.com/repository/maven-public/com/neeve/nvx-rumi-perf-dist/4.0.4/nvx-rumi-perf-dist-4.0.4-win-x86-64.zip
```

Replace `4.0.4` with your desired version.

## Running Benchmarks

### 1. Extract Distribution

**Linux/macOS:**

```bash
tar xvf nvx-rumi-perf-dist-{version}-{arch}.tar.gz
cd nvx-rumi-perf-{version}
```

**Windows:**

```cmd
unzip nvx-rumi-perf-dist-{version}-win-x86-64.zip
cd nvx-rumi-perf-{version}
```

This creates:

```
nvx-rumi-perf-{version}/
├── conf/      # Configuration files
└── libs/      # All JARs and dependencies
```

### 2. Run a Benchmark

**With Java 8:**

*Linux/macOS:*

```bash
$JAVA_HOME/bin/java -cp "libs/*" {BenchmarkClass} {parameters}
```

*Windows:*

```cmd
%JAVA_HOME%\bin\java -cp "libs\*" {BenchmarkClass} {parameters}
```

**With Java 11 or later:**

*Linux/macOS:*

```bash
$JAVA_HOME/bin/java \
  --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED \
  --add-opens=java.base/sun.nio.ch=ALL-UNNAMED \
  --add-opens=java.base/java.lang=ALL-UNNAMED \
  --add-opens=java.base/java.nio=ALL-UNNAMED \
  --add-opens=java.base/java.io=ALL-UNNAMED \
  --add-opens=java.management/sun.management=ALL-UNNAMED \
  --illegal-access=warn \
  -cp "libs/*" {BenchmarkClass} {parameters}
```

*Windows:*

```cmd
%JAVA_HOME%\bin\java ^
  --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED ^
  --add-opens=java.base/sun.nio.ch=ALL-UNNAMED ^
  --add-opens=java.base/java.lang=ALL-UNNAMED ^
  --add-opens=java.base/java.nio=ALL-UNNAMED ^
  --add-opens=java.base/java.io=ALL-UNNAMED ^
  --add-opens=java.management/sun.management=ALL-UNNAMED ^
  --illegal-access=warn ^
  -cp "libs\*" {BenchmarkClass} {parameters}
```

### Example - Serialization Benchmark

**With Java 8:**

```bash
$JAVA_HOME/bin/java -cp "libs/*" com.neeve.perf.serialization.Driver --provider protobuf.random
```

**With Java 11+:**

```bash
$JAVA_HOME/bin/java \
  --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED \
  --add-opens=java.base/sun.nio.ch=ALL-UNNAMED \
  --add-opens=java.base/java.lang=ALL-UNNAMED \
  --add-opens=java.base/java.nio=ALL-UNNAMED \
  --add-opens=java.base/java.io=ALL-UNNAMED \
  --add-opens=java.management/sun.management=ALL-UNNAMED \
  --illegal-access=warn \
  -cp "libs/*" com.neeve.perf.serialization.Driver --provider protobuf.random
```

## Next Steps

* **Module Documentation**: See the [Modules](/performance/benchmark-suite/modules.md) section for detailed information on each benchmark module, including program classes, parameters, and interpreting results
* **Canonical Benchmark**: Learn about the [official end-to-end benchmark](/performance/canonical-benchmark.md)
* **Build from Source**: See the [nvx-rumi-perf repository](https://github.com/rumidata/nvx-rumi-perf) for build instructions