Admin Over SMA

Since 3.10

Overview

In addition to the standard direct (TCP based) monitoring capabilities, a Rumi container can be configured to allow administrative applications to monitor and manage it over messaging. When Admin over SMA is enabled, containers emit heartbeat, trace and lifecycle events over defined messaging channels for consumption by listening clients. Administrative and monitoring tools use discovery to discover available containers. Once an Admin Client discovers a container it connects to the configured message bus and subscribes to its monitoring channels. At a high level, all that is required to enable Admin over SMA is to set the following properties:

nv.discovery.descriptor=<discovery-provider>
nv.server.admin.transports=sma
nv.server.admin.bus.descriptor=<bus-connection-descriptor>

As long as both the admin client and the container are using the same discovery descriptor and message bus, they will be able to communicate with one another.

Warning: When enabling Admin over SMA it is important to note that by default admin clients using SMA will attempt to connect to discovered containers. If older version containers or containers not configured for Admin over SMA are advertising themselves over the admin client's discovery address the container will never respond and the admin client will see connection timeouts. To avoid this you may:

• Use passive monitoring on the client side to avoid attempts to ping the container

• Use separate container discovery for containers that support Admin over SMA vs. those that don't

Admin Channels

The table below lists the bus channels used for administration over SMA:

Name	Key	QOS	ID	Description
Command and Control
container-request	container-admin/${xvmName}/request	BestEffort	30000	Command and control requests
container-response	container-admin/${adminClientId}/response	BestEffort	30001	Command and control responses (to issuing admin client)
Monitoring
container-heartbeat	container-admin/${xvmName}/heartbeat	BestEffort	30002	container heartbeats
container-trace	container-admin/${xvmName}/trace	BestEffort	30003	Log trace records
container-event	container-admin/${xvmName}/event	BestEffort	30004	Lifecycle and alert events

By default, all admin channels start with container-admin channel prefix. This prefix can be changed by setting the environment property nv.admin.sma.channelKeyPrefix which can be useful for cases where it is desirable to more granularly partition admin traffic. Note that each channel is configured with a topic level of either xvmName or adminClientId. The xvmName topic level allows clients to listen in to a particular container, and the adminClientId allows a container to send responses targeted to a particular admin client.

Request and Response Channels

When command and control is enabled, these channels allow admin clients to issue commands to a container. Such commands include container control commands such as triggering thread dumps or reloading applications and also include invocation of application-defined commands.

Heartbeats

Container heartbeats are periodically emitted over the heartbeats channel when they are enabled for the container.

Heartbeats are emitted using the SrvMonHeartbeatMessage type defined in the com.neeve.server.mon package. While this package contains additional lifecycle related messages types used by direct monitoring clients, only SrvMonHeartbeatMessage are emitted over the heartbeats channel.

Trace

When enabled, SrvMonTraceRecords are emitted over the trace channel allowing admin clients to view trace output as it happens.

Trace records are emitted using SrvMonTraceRecord message type that can be found in the com.neeve.server.mon.cnc package.

Events

The events channel is used to emit events such as lifecycle or alert events which are defined in the com.neeve.server.mon.lifecycle and com.neeve.server.mon.alert packages.

Configuration

Discovery

Admin clients should not attempt to issue commands until a container has been discovered via the discovery provider (see Understanding Discovery). When an admin client detects that a container is no longer discoverable it should stop issuing commands over SMA.

Configuring Admin Client Connectivity

Administrative tools, such as the AdminTool, must be configured to enable SMA as a transport along with the admin bus connection information. As admin clients won't always be configured via DDL, configuration is done via system/environment properties to configure connectivity to the containers monitoring container.

Properties Based Configuration with Bus Descriptor:

nv.server.admin.transports=sma
nv.server.admin.sma.bus.descriptor=solace://solhost:55555&topic_starts_with_channel=false&SESSION_VPN_NAME=default&use_default_queue_name=false&use_default_queue_name_as_default_client_id=true&topic_starts_with_channel=false&usejni=true&single_session=true

Alternatively, it is possible to configure the bus descriptor in decomposed form, which can be useful in cases where configuration properties are more overridden across environments.

Properties Based Configuration for Clients:

nv.server.admin.transports=sma
nv.server.admin.sma.bus.provider=solace
nv.server.admin.sma.bus.address=solhost
nv.server.admin.sma.bus.port=55555
nv.server.admin.sma.bus.properties.SESSION_VPN_NAME=default
nv.server.admin.sma.bus.properties.use_default_queue_name=false
nv.server.admin.sma.bus.properties.use_default_queue_name_as_default_client_id
nv.server.admin.sma.bus.properties.topic_starts_with_channel=false
nv.server.admin.sma.bus.properties.usejni=true
nv.server.admin.sma.bus.properties.single_session=true

Configuring Containers

Containers can be configured using the same environment properties as clients by setting the properties in the DDL environment section:

<env>
  <nv>
    <container>
      <admin>
        <transports>sma</transports>
        <sma>
          <bus>
            <descriptor>solace://solhost:55555&topic_starts_with_channel=false&SESSION_VPN_NAME=default&use_default_queue_name=false&use_default_queue_name_as_default_client_id=true&topic_starts_with_channel=false&usejni=true&single_session=true</descriptor>
          </bus>
        </sma>
      </admin>
    </container>
  </nv>
</env>

Advanced Container Bus Configuration

Using environment based configuration is the simplest way of configuring Admin over SMA for a container. In cases where bus configuration is not being injected by deployment tools, it is possible to use the <container> <admin> element in DDL to enable admin over SMA and reference a DDL defined bus definition.

Container Admin over SMA Configuration

To configure a container to use a bus named 'container-admin' the container's <admin> <sma> element can be used:

<containers>
  <container name="order-processing-1" template="container-template">
    <admin>
      <transports>
        <sma enabled="true">
          <busName>container-admin</busName>
        </sma>
      </transports>
    </admin>
    <heartbeats enabled="true" interval="5s"/>
  </container>
</containers>

Admin Bus Configuration

The following bus definition can then be configured for use by the container.

<buses>
  <bus name="container-admin">
    <provider>solace</provider>
    <address>solhost</address>
    <port>55555</port>
  </bus>
</buses>

Admin channels (container-request, container-response, container-heartbeat, container-event, and container-trace) should not be configured for the bus, they are automatically created by the container when it is started.

With the above configuration, the order-processing-1 container will create a connection to solace://solhost:55555 with a username of order-processing-1. It will use the following topics:

Subscribe:

• container-request channel: container-admin/order-processing-1/request

Publish:

• container-response channel: container-admin/${adminClientId}/response (where the adminClientId is substituted with that of the sending client when a response is sent)

• container-heartbeats channel: container-admin/order-processing-1/heartbeat (when heartbeats are enabled)

• container-trace channel: container-admin/order-processing-1/trace (when trace emission is enabled)

• container-event channel: container-admin/order-processing-1/event

Tip: Enabling container heartbeats isn't strictly necessary, but in most monitoring scenarios it is desirable.

Passive Monitoring

It is possible to use Admin over SMA in a purely passive monitoring capacity by setting the property:

nv.server.admin.passivemonitoringonly=true

With the above configuration setting clients will throw an exception if an attempt is made to send commands to a container, and containers will not issue subscriptions on the container-request channel.

Related Topics

• container Stats and Heartbeats - Configure heartbeat emission

• rumi admin - Command-line administrative interface

• Implementing Command Handlers - Define custom commands

Next Steps

1. Configure discovery provider for container and admin clients

2. Set up message bus for admin channels

3. Enable Admin Over SMA in container configuration

4. Configure admin tools to use SMA transport

5. Test admin operations via messaging