> For the complete documentation index, see [llms.txt](https://docs.rumi.systems/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.rumi.systems/rumi-management/guides/monitor/customizing-dashboards.md).

# Customizing Dashboards

This guide covers creating custom Grafana panels and dashboards for Rumi monitoring.

## Overview

While the canned dashboards cover common monitoring scenarios, you can create custom dashboards to focus on specific metrics, combine data from multiple sources, or add alerting.

## Creating Custom Panels

### Querying InfluxDB

All Rumi telemetry data is stored in InfluxDB under the `rumi_heartbeat_rp` retention policy. To create a custom panel:

1. Add a new panel to a dashboard.
2. Select the InfluxDB data source.
3. Write a query targeting the appropriate measurement.

### Query Example

To chart process CPU load for a specific container:

```sql
SELECT mean("process_cpu_load") FROM "rumi_heartbeat_rp"."application"
WHERE "system_name" =~ /^$system_name$/ AND "vm_name" =~ /^$vm_name$/
GROUP BY time($__interval) fill(null)
```

### Using Template Variables

Reference the existing template variables (`$system_name`, `$vm_name`, `$app_name`, `$host_name`) in your queries to make custom panels filterable using the same dropdowns as the canned dashboards.

## Creating Custom Dashboards

To create a new dashboard:

1. Navigate to **Dashboards** > **New Dashboard**.
2. Add panels querying the InfluxDB measurements.
3. Add template variables for filtering (see [Setting Up Dashboards](/rumi-management/guides/monitor/setting-up-dashboards.md) for variable query syntax).
4. Save the dashboard.

## Available Measurements

For a complete reference of all available InfluxDB measurements, tags, and fields, see the [InfluxDB Measurements Reference](/rumi-management/reference/agent/influxdb-measurements.md).

Key measurements for custom dashboards:

| Measurement                           | Content                                        |
| ------------------------------------- | ---------------------------------------------- |
| `application`                         | Container and application-level system metrics |
| `application.store`                   | State store operations and hit rates           |
| `application.messages`                | Message counts and throughput                  |
| `application.messages.type`           | Per-message-type counts                        |
| `application.messages.type.latencies` | Per-message-type latency statistics            |
| `application.transaction`             | Transaction counts and durations               |
| `application.transaction.latencies`   | Transaction latency percentiles                |
| `application.disruptor`               | Ring buffer utilization                        |
| `application.disruptor.latencies`     | Disruptor processing latencies                 |
| `application.user.gauge`              | Custom application gauges                      |
| `application.user.counter`            | Custom application counters                    |
| `application.bus`                     | Per-bus messaging statistics                   |
| `system`                              | Container-level system metrics                 |
| `system.gc`                           | Garbage collection statistics                  |
| `system.disk`                         | Disk I/O and space utilization                 |
| `system.threads`                      | Thread-level statistics                        |

## Alerting

Grafana supports alerting on panel queries. You can configure alerts on custom panels to notify when:

* Latency exceeds a threshold.
* Memory usage approaches limits.
* Error rates spike.
* Custom application counters cross thresholds.

See the [Grafana alerting documentation](https://grafana.com/docs/grafana/latest/alerting/) for details on configuring alert rules and notification channels.

## Best Practices

* **Save customizations separately** — Avoid modifying canned dashboards directly. Create new dashboards or copy panels to preserve the ability to upgrade canned dashboards.
* **Use consistent naming** — Follow the tag naming conventions used by the canned dashboards for consistency.
* **Leverage template variables** — Use the same template variable names so filtering works consistently across dashboards.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.rumi.systems/rumi-management/guides/monitor/customizing-dashboards.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.