Understanding Message Serialization

Most Rumi applications never think about how a message becomes bytes. You declare a message in a model, call sendMessage(), and a handler on another microservice receives the same message back as a Java object. The bus binding does the encoding on the way out and the decoding on the way in.

This page is for the cases where that is not enough:

• You are integrating an application that does not run on Rumi, and it has to produce or consume Rumi messages directly.

• You are looking at bytes in a wire sniffer, a broker's message browser, or a log, and you need to know what they mean.

• You are writing a custom bus binding, or a tool that reads a message log.

Serialization and Deserialization

Rumi messages are ordinary Java objects. A message declared in an ADM model generates a class implementing MessageView, and that interface carries the full serialization surface.

Two things travel on the wire for every message:

• The message payload, the encoded fields of the message itself.

• The message metadata, a small fixed-layout header that tells the receiver how to interpret the payload, and which channel the message was sent on.

The payload alone is not enough to reconstruct a message. The receiver needs the metadata to know which encoding was used and which generated factory and type to hand the bytes to. How the two are carried is a property of the binding: the Solace binding puts the payload in the message body and the metadata in an x-sma-metadata property; the JMS binding does the same over a BytesMessage.

Engine-Independent Serializers

com.neeve.sma.MessageView exposes serialization that does not require an engine, a binding, or any running Rumi infrastructure. Given a message object you can turn it into bytes, and given bytes you can turn them back into a message.

The methods come in families, one per destination type:

Destination	Serialize	Deserialize
byte[]	serializeToByteArray(), serializeTo(byte[], int)	deserializeFromByteArray(byte[]), deserializeFrom(byte[], int, int)
ByteBuffer	serializeToByteBuffer(), serializeTo(ByteBuffer)	deserializeFromByteBuffer(ByteBuffer), deserializeFrom(ByteBuffer)
IOBuffer	serializeToIOBuffer(boolean), serializeTo(IOBuffer, int)	deserializeFrom(IOBuffer, int, int)
IOElasticBuffer	serializeTo(IOElasticBuffer, int)	deserializeFrom(IOElasticBuffer, int, int)
PktPacket	serializeToPacket(), serializeTo(PktPacket)	deserializeFromPacket(PktPacket), deserializeFrom(PktPacket)
Native address	serializeTo(long, int)	deserializeFrom(long, int, int)
JSON	serializeToJson()	deserializeFromJson(String)

Note the offset-into-existing-buffer overloads, serializeTo(byte[] array, int offset) and friends. These write into a buffer you already own rather than allocating a new one, and they return the number of bytes written. If you are assembling a larger frame, or you are on a path where allocation matters, these are the ones to reach for; the serializeToXxx() forms allocate.

serializeToJson() and deserializeFromJson() are a convenience for debugging and interoperability, and they work whatever encoding the message uses. JSON is not an encoding type in Rumi: there is no JSON code generator, and no message reports a JSON encoding on the wire. If you have seen JSON listed as an encoding, that is X Platform.

See the MessageView Javadoc for exact signatures and per-method semantics.

Message Metadata

MessageMetadata is the header that accompanies every message. It has a fixed binary layout, which is what makes it readable by an application that is not running Rumi.

Message Encoding Type

The encoding type is a single byte identifying how the payload was encoded. Rumi generates and sends two encodings:

Value	Constant	Encoding
4	ENCODING_TYPE_PROTOBUF	Protobuf
6	ENCODING_TYPE_QUARK	Quark

Choose between them with the encodingType model directive, which accepts protobuf or quark. See Choosing an Encoding Type.

encodingType also accepts xbuf and xbuf2. Both are aliases for the Protobuf generator, kept so that models carried over from X Platform still build. They produce Protobuf-encoded messages that report encoding type 4, not 3 or 7, so do not read those names as selecting a distinct wire encoding.

A receiver is more permissive than a sender. The Protobuf message factory also accepts encoding types 3 (Xbuf) and 7 (Xbuf2) on the wire, so a Rumi service can consume messages produced by an X Platform peer. Metadata carrying those values is therefore normal on a mixed estate; it just cannot originate from Rumi.

Field Reference

Field	Meaning
Version	Metadata wire format version. Determines the layout of everything after it. Which versions a binding can send varies; see below.
Encoding type	How the payload is encoded. See the table above.
View factory	Id of the ADM-generated factory that can create the message.
View type	Id of the message type within that factory. V2 and later. On V1 metadata this reads as 0.
Sender	Id of the sending member. 0 when unspecified.
Flow	Message flow id, used for ordering and duplicate detection. 0 when unspecified.
Sno	Message sequence number within the flow. 0 for an unsequenced message.
Channel id	Id of the channel the message was sent on, or a non-positive value if no id was sent.
Channel name length	Length in bytes of the channel name that follows, or -1 if no name was sent.
Channel name	The channel name, if present. Variable length.
Request id	Correlation id for request-reply. V3 only.
Requestor id length	Length in bytes of the requestor id that follows, or -1 if absent. V3 only.
Requestor id	Identifies the requestor a reply should be routed back to. V3 only. Variable length.

The view factory and view type together identify the message class. This is why a V1 receiver has to introspect the payload to work out what it is holding, while a V2 receiver can dispatch straight from the metadata.

Channel id and channel name are the inputs to inbound channel resolution, and a message does not necessarily carry both. See Inbound Channel Resolution for what the receiver does with them, and why a channel id of -1 is normal rather than an error.

Which Version Is Sent

The version is selected per bus by the sma_metadata_version property on the message bus descriptor, whose default is 2. So unless a bus says otherwise, V2 goes out.

Set it to 3 for the correlation fields that request-reply needs. Nothing else in the layout changes, so this is a per-bus decision about whether the extra ten bytes of fixed header earn their place.

MessageMetadata.VERSION is V3. That constant is the newest version the code understands, not the version a binding sends. Do not read it as a default.

Which versions a binding can send is a property of the binding, not of the platform. All of them read the same version, but they do not all implement all three:

Binding	V1	V2	V3
Solace	yes	yes	yes
JMS	no	yes	yes
Kafka	no	yes	yes
MQTT	no	yes	yes
Loopback	no	yes	yes
Ether	no	no	always V3

Two entries in that table are worth reading twice:

• Solace is the only binding that sends V1, and only for Protobuf-encoded messages: asking for V1 with any other encoding throws V1 SMA metadata is only supported with Protobuf encoding type. If you need to produce V1 for an old consumer, Solace is the binding that can do it.

• Ether ignores sma_metadata_version entirely and always writes V3. Setting the property on an Ether bus has no effect.

Asking a binding for a version it does not implement is an error at send time rather than a silent downgrade, so a misconfigured bus fails loudly.

None of this applies to a binding running in raw mode, which writes no SMA metadata at all: the payload goes on the wire on its own and the receiver is responsible for knowing what it is. Raw mode is available on the Solace, JMS, Kafka, MQTT and loopback bindings.

V1 Wire Layout

Fixed portion is 24 bytes, followed by the variable-length channel name. Sent only by the Solace binding, and only with Protobuf encoding; parsed by every binding.

Offset	Size	Field
0	1	Version (1)
1	1	Encoding type
2	2	View factory
4	4	Sender
8	4	Flow
12	8	Sno
20	2	Channel id
22	2	Channel name length (-1 if absent)
24	n	Channel name

V2 Wire Layout

V2 inserts the view type after the view factory, shifting everything below it by two bytes. The fixed portion is 26 bytes.

Offset	Size	Field
0	1	Version (2)
1	1	Encoding type
2	2	View factory
4	2	View type
6	4	Sender
10	4	Flow
14	8	Sno
22	2	Channel id
24	2	Channel name length (-1 if absent)
26	n	Channel name

V3 Wire Layout

V3 is the newest version, sent when a bus sets sma_metadata_version=3. It adds the correlation fields that make request-reply possible on the wire: without a request id there is nothing to match a reply against.

The fixed portion is 36 bytes, and unlike V1 and V2 there are two variable-length trailers.

Offset	Size	Field
0	1	Version (3)
1	1	Encoding type
2	2	View factory
4	2	View type
6	4	Sender
10	4	Flow
14	8	Sno
22	2	Channel id
24	8	Request id
32	2	Channel name length (-1 if absent)
34	2	Requestor id length (-1 if absent)
36	n	Channel name, then requestor id

Because there are two trailers, the total serialized length is the 36-byte fixed portion plus both lengths. A reader that assumes a single variable field will mis-parse V3.

These offsets correspond to the V1_MESSAGE_*_POS, V2_MESSAGE_*_POS and V3_MESSAGE_*_POS constants in MessageMetadata, and the fixed lengths to FIXED_WIRE_LENGTH_V1 (24), FIXED_WIRE_LENGTH_V2 (26) and FIXED_WIRE_LENGTH_V3 (36).

Read the version byte at offset 0 first and branch on it. Do not hard-code a single version: sma_metadata_version is a bus descriptor property rather than a property of any one binding, so it can differ per bus within a single service, and an X Platform peer may still send V1.

Sending Messages from External Applications

An application that does not run on Rumi can still exchange messages with one. It has to do by hand what the binding would otherwise do: serialize the payload, build the metadata, and put both where the binding expects to find them.

The mechanics are per-binding, and each binding's page carries a worked example:

• Solace Binding: Sending and Receiving from External Applications, using x-sma-metadata on a BytesMessage.

• JMS Binding: Sending and Receiving from External Applications.

The field semantics those examples rely on are the ones documented above.

Related Topics

• Messaging Model - channels, keys, and inbound channel resolution

• Choosing an Encoding Type

• Sending Messages