> For the complete documentation index, see [llms.txt](https://docs.rumi.systems/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.rumi.systems/performance/benchmark-suite/modules/link-module.md).

# Link Module

The Link module benchmarks cluster replication link latency, measuring the performance of the low-level transport used for replicating state between cluster members.

## Overview

The Link layer provides the foundation for cluster replication in Rumi, offering:

* Direct point-to-point communication between cluster members
* Minimal overhead transport for state replication
* Support for various network protocols (TCP, UDP, InfiniBand, etc.)

This benchmark measures the raw performance capabilities of the replication link layer.

## Test Programs

The Link module provides multiple test program variants:

### Streaming Tests

Measure unidirectional throughput:

**Blocking Variant**:

* Sender: `com.neeve.perf.link.BlockingStreamingSender`
* Receiver: `com.neeve.perf.link.BlockingStreamingReceiver`

**Non-Blocking Variant**:

* Receiver: `com.neeve.perf.link.NonBlockingStreamingReceiver` (no sender variant)

**RDMA Variant**:

* Sender: `com.neeve.perf.link.RdmaStreamingSender`
* Receiver: `com.neeve.perf.link.RdmaStreamingReceiver`

The sender continuously sends messages to the receiver as fast as possible, measuring maximum sustained throughput.

### Ping-Pong Tests

Measure round-trip latency:

**Blocking Variant**:

* Sender: `com.neeve.perf.link.BlockingPingPongSender`
* Receiver: `com.neeve.perf.link.BlockingPingPongReceiver`

**Non-Blocking Variant**:

* Sender: `com.neeve.perf.link.NonBlockingPingPongSender`
* Receiver: `com.neeve.perf.link.NonBlockingPingPongReceiver`

The sender sends a message and waits for a response from the receiver, measuring round-trip time.

## Command-Line Parameters

### Blocking Streaming Sender

| Short | Long                         | Default   | Description                                                             |
| ----- | ---------------------------- | --------- | ----------------------------------------------------------------------- |
| `-d`  | `--descriptor`               | -         | Connection descriptor (e.g., tcp\://192.168.1.7:12000\&tcpnodelay=true) |
| `-m`  | `--messageSize`              | 256       | Message size in bytes                                                   |
| `-b`  | `--bufferSize`               | 256       | Write buffer size                                                       |
| `-t`  | `--testCount`                | 100000000 | Number of messages to send                                              |
| `-r`  | `--testRate`                 | 10000000  | Send rate (messages/sec)                                                |
| `-w`  | `--warmupTime`               | 2         | Warmup time in seconds                                                  |
| `-c`  | `--cpuAffinityMask`          | -         | CPU affinity mask                                                       |
| `-i`  | `--printIntervalStats`       | false     | Output periodic interval stats                                          |
| `-f`  | `--dontWriteLatenciesToFile` | false     | Suppress latency file output                                            |

### Blocking Streaming Receiver

| Short | Long                | Default | Description                         |
| ----- | ------------------- | ------- | ----------------------------------- |
| `-d`  | `--descriptor`      | -       | Connection descriptor               |
| `-m`  | `--messageSize`     | 256     | Message size in bytes               |
| `-c`  | `--cpuAffinityMask` | -       | CPU affinity mask                   |
| `-s`  | `--stats`           | false   | Output incremental throughput stats |

### Blocking Ping-Pong Sender

| Short | Long                         | Default | Description                    |
| ----- | ---------------------------- | ------- | ------------------------------ |
| `-d`  | `--descriptor`               | -       | Connection descriptor          |
| `-m`  | `--messageSize`              | 256     | Message size in bytes          |
| `-c`  | `--testCount`                | 300000  | Number of messages to send     |
| `-r`  | `--testRate`                 | 10000   | Send rate (messages/sec)       |
| `-a`  | `--cpuAffinityMask`          | -       | CPU affinity mask              |
| `-p`  | `--spinRead`                 | false   | Spin or block on read          |
| `-o`  | `--oneWayLatency`            | false   | Calculate one-way latency      |
| `-i`  | `--printIntervalStats`       | false   | Output periodic interval stats |
| `-f`  | `--dontWriteLatenciesToFile` | false   | Suppress latency file output   |

### Blocking Ping-Pong Receiver

| Short | Long                | Default | Description                         |
| ----- | ------------------- | ------- | ----------------------------------- |
| `-d`  | `--descriptor`      | -       | Connection descriptor               |
| `-m`  | `--messageSize`     | 256     | Message size in bytes               |
| `-c`  | `--cpuAffinityMask` | -       | CPU affinity mask                   |
| `-p`  | `--spinRead`        | false   | Spin or block on read               |
| `-s`  | `--stats`           | false   | Output incremental throughput stats |

## Use Cases

### Streaming Test

**Purpose**: Measure maximum throughput **Setup**: Two machines connected via high-speed network **Use Case**: Validate network configuration and capacity

### Ping-Pong Test

**Purpose**: Measure minimum latency **Setup**: Two machines connected via low-latency network **Use Case**: Validate network tuning and baseline latency

## Running Benchmarks

### Streaming Throughput Test

**On Receiver Machine**:

```bash
$JAVA_HOME/bin/java -cp "libs/*" com.neeve.perf.link.BlockingStreamingReceiver \
  -d "tcp://0.0.0.0:5000&tcpnodelay=true" \
  -m 256 \
  -s true
```

**On Sender Machine**:

```bash
$JAVA_HOME/bin/java -cp "libs/*" com.neeve.perf.link.BlockingStreamingSender \
  -d "tcp://receiver.example.com:5000&tcpnodelay=true" \
  -m 256 \
  -t 10000000 \
  -r 10000000
```

### Ping-Pong Latency Test

**On Receiver Machine**:

```bash
$JAVA_HOME/bin/java -cp "libs/*" com.neeve.perf.link.BlockingPingPongReceiver \
  -d "tcp://0.0.0.0:5000&tcpnodelay=true" \
  -m 256 \
  -s true
```

**On Sender Machine**:

```bash
$JAVA_HOME/bin/java -cp "libs/*" com.neeve.perf.link.BlockingPingPongSender \
  -d "tcp://receiver.example.com:5000&tcpnodelay=true" \
  -m 256 \
  -c 100000 \
  -r 10000
```

## Interpreting Results

### Throughput Results

```
Messages sent: 10000000
Duration: 45.2 seconds
Throughput: 221,238 messages/second
Bandwidth: 452 Mbps
```

### Latency Results

```
Round-trip latency statistics:
  50th percentile: 42.5 µs
  99th percentile: 58.3 µs
  99.9th percentile: 82.1 µs
```

## Network Configurations

### TCP over 10GbE

**Typical Results**:

* Throughput: 1-2M messages/second
* Latency: 15-25µs round-trip

### TCP over InfiniBand

**Typical Results**:

* Throughput: 2-4M messages/second
* Latency: 8-15µs round-trip

### RDMA over InfiniBand

**Typical Results**:

* Throughput: 5-10M messages/second
* Latency: 2-5µs round-trip

## Comparison with Higher Layers

Link layer provides the foundation for cluster replication:

* **Link Layer**: \~10µs (raw replication transport)
* **Messaging Layer**: \~15µs (adds SMA abstractions)
* **AEP Engine**: \~27µs (adds transactions, persistence, full clustering with consensus)

## Next Steps

* Review [Messaging Module](/performance/benchmark-suite/modules/messaging-module.md) for SMA layer benchmarks
* Review [AEP Module](/performance/benchmark-suite/modules/aep-module.md) for end-to-end benchmarks
* Return to [Benchmark Suite](/performance/benchmark-suite.md) for other modules


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.rumi.systems/performance/benchmark-suite/modules/link-module.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.