Benchmark Suite

The Rumi Performance Benchmark Suite is a collection of benchmarking tools that measure the performance of individual Rumi runtime components. These tools serve both as performance measurement utilities and as sample code demonstrating Rumi best practices.

What's in the Suite

The benchmark suite contains 7 modules, each focusing on a specific Rumi component:

Module	Component	Description
Time	Time API	Measures overhead of Rumi's time API
Serialization	Encoding (ADM)	Message serialization/deserialization performance
Link	Link	Cluster replication link throughput and latency
Messaging	Messaging (SMA)	Pub/sub messaging layer performance
Persistence	Persistence	Message and data persistence performance
Storage	Store (ODS)	Object store operation performance
AEP	Engine (AEP)	End-to-end canonical benchmark

Relationship to Canonical Benchmark

The AEP Module contains the Canonical Benchmark - the official end-to-end performance test used to measure Rumi's published performance metrics. While the canonical benchmark measures the complete Receive-Process-Send flow, the other modules isolate and test individual components.

Source Repository

The benchmark suite source code is available at:

• Repository: github.com/rumidata/nvx-rumi-perf

• Organization: Multi-module Maven project

• License: Embedded (no separate license required to run)

Releases

Versioning

A benchmark suite release is published for each Rumi release. The benchmark version matches the Rumi version it tests.

Distribution Naming

Distributions are named as follows:

Linux and macOS:

nvx-rumi-perf-dist-{version}-{arch}.tar.gz

Windows:

nvx-rumi-perf-dist-{version}-{arch}.zip

Valid architectures (arch):

• linux-x86-64 - Linux (recommended for performance testing)

• osx-x86-64 - macOS (development only)

• win-x86-64 - Windows (development only)

Example: The Linux distribution for version 4.0.4 is named:

nvx-rumi-perf-dist-4.0.4-linux-x86-64.tar.gz

Important: Only Linux distributions include Rumi native libraries required for zero-garbage operation. Windows and macOS distributions can be used for development, but Linux is required for full performance optimization.

Download

Download distributions from the Neeve artifact repository:

Linux:

wget https://nexus.n5corp.com/repository/maven-public/com/neeve/nvx-rumi-perf-dist/4.0.4/nvx-rumi-perf-dist-4.0.4-linux-x86-64.tar.gz

macOS:

wget https://nexus.n5corp.com/repository/maven-public/com/neeve/nvx-rumi-perf-dist/4.0.4/nvx-rumi-perf-dist-4.0.4-osx-x86-64.tar.gz

Windows:

wget https://nexus.n5corp.com/repository/maven-public/com/neeve/nvx-rumi-perf-dist/4.0.4/nvx-rumi-perf-dist-4.0.4-win-x86-64.zip

Replace 4.0.4 with your desired version.

Running Benchmarks

1. Extract Distribution

Linux/macOS:

tar xvf nvx-rumi-perf-dist-{version}-{arch}.tar.gz
cd nvx-rumi-perf-{version}

Windows:

unzip nvx-rumi-perf-dist-{version}-win-x86-64.zip
cd nvx-rumi-perf-{version}

This creates:

nvx-rumi-perf-{version}/
├── conf/      # Configuration files
└── libs/      # All JARs and dependencies

2. Run a Benchmark

With Java 8:

Linux/macOS:

$JAVA_HOME/bin/java -cp "libs/*" {BenchmarkClass} {parameters}

Windows:

%JAVA_HOME%\bin\java -cp "libs\*" {BenchmarkClass} {parameters}

With Java 11 or later:

Linux/macOS:

$JAVA_HOME/bin/java \
  --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED \
  --add-opens=java.base/sun.nio.ch=ALL-UNNAMED \
  --add-opens=java.base/java.lang=ALL-UNNAMED \
  --add-opens=java.base/java.nio=ALL-UNNAMED \
  --add-opens=java.base/java.io=ALL-UNNAMED \
  --add-opens=java.management/sun.management=ALL-UNNAMED \
  --illegal-access=warn \
  -cp "libs/*" {BenchmarkClass} {parameters}

Windows:

%JAVA_HOME%\bin\java ^
  --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED ^
  --add-opens=java.base/sun.nio.ch=ALL-UNNAMED ^
  --add-opens=java.base/java.lang=ALL-UNNAMED ^
  --add-opens=java.base/java.nio=ALL-UNNAMED ^
  --add-opens=java.base/java.io=ALL-UNNAMED ^
  --add-opens=java.management/sun.management=ALL-UNNAMED ^
  --illegal-access=warn ^
  -cp "libs\*" {BenchmarkClass} {parameters}

Example - Serialization Benchmark

With Java 8:

$JAVA_HOME/bin/java -cp "libs/*" com.neeve.perf.serialization.Driver --provider protobuf.random

With Java 11+:

$JAVA_HOME/bin/java \
  --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED \
  --add-opens=java.base/sun.nio.ch=ALL-UNNAMED \
  --add-opens=java.base/java.lang=ALL-UNNAMED \
  --add-opens=java.base/java.nio=ALL-UNNAMED \
  --add-opens=java.base/java.io=ALL-UNNAMED \
  --add-opens=java.management/sun.management=ALL-UNNAMED \
  --illegal-access=warn \
  -cp "libs/*" com.neeve.perf.serialization.Driver --provider protobuf.random

Next Steps

• Module Documentation: See the Modules section for detailed information on each benchmark module, including program classes, parameters, and interpreting results

• Canonical Benchmark: Learn about the official end-to-end benchmark

• Build from Source: See the nvx-rumi-perf repository for build instructions