tlt

The Transaction Log Tool (rumi tlt) is an interactive utility for querying and analyzing Rumi transaction logs.

Overview

The Transaction Log Tool provides capabilities for browsing and querying the recovery transaction logs and inbound/outbound transaction logs recorded by a Rumi application. The terms 'log' and 'transaction log' are used interchangeable herein for the sake of brevity.

Key capabilities:

• Browse transaction logs - Navigate through log entries

• Query with SQL-like syntax - Use XPQL (Rumi Query Language) to query logs

• Create indexes - Build indexes for faster queries

• Export results - Dump results to console, CSV, or HTML

• Archive logs - Package logs with tool for portable analysis

• Compare logs - Identify differences between log files

Usage

Syntax:

rumi tlt [log-path]

Parameters:

• [log-path] - Optional path to transaction log or directory (opens all logs in directory)

Examples:

# Launch tool in current directory
rumi tlt

# Open specific log
rumi tlt /path/to/application.log

# Open all logs in directory
rumi tlt /path/to/logs/

Interactive Tool Features

The tool supports a subset of ReadLine functionality with familiar unix-style command line operations:

• Command history

• Tab command completion

• Command scripting

Basic Usage Example

Opening logs and browsing:

> open rdat
<10,4604,MY-PC> 20181118-14:15:53:144 (inf)...[RogLog->'processor-1'] Live transaction log file is 'processor-1.log' [C:\dev\nvx-app-talon-starter\target\testbed\rdat\processor-1-1]...
<10,4604,MY-PC> 20181118-14:15:53:514 (inf)...Log open (size=15825952, entries=59793) took 361 milliseconds.
<10,4604,MY-PC> 20181118-14:15:53:572 (inf)...[RogLog->'processor-1'] Scavenging old log files....
<10,4604,MY-PC> 20181118-14:15:53:572 (inf)...[RogLog->'processor-1'] ....scavenged 0 files (0 failed).
reader opened with Lazy deserialization.
rdat/processor-1>

Displaying entries:

rdat/processor-1> next 1
I=InboundMsgsInTxn?
O=OutboundMsgsInTxn?
D=PossibleDup?
S=CommitStart?
E=CommitEnd?
Type |                          Class |                             ObjectId |                             ParentId |   TxnId | StblTxId|  InSeq# | OutSeq# | MsgSeq# | ChkPnt# | I | O | D | S | E
   P |                        AepFlow | 2c708c60-eb73-11e8-880f-9eb6348a2af3 |                                 null | 0000001 | 0000001 | 0000000 | 0000000 | 0000000 | 0000001 | F | F | F | F | F
   P |                     Repository | 2c7399a1-eb73-11e8-880f-9eb6348a2af3 | 2c708c60-eb73-11e8-880f-9eb6348a2af3 | 0000001 | 0000001 | 0000000 | 0000000 | 0000000 | 0000001 | F | F | F | F | F

Operating Modes

The Transaction Log Tool operates in two modes:

Browse Mode

In browse mode the prompt indicates the log that is being browsed. Navigate logs using:

• next - Display next entry

• nexttxn - Display next transaction

• skip / skiptxn - Skip entries/transactions

• rewind - Return to start of log

• switch - Switch between logs

• dump - Dump entire log to console or file

Format options:

• Default: Brief tabular format

• -d flag: Detailed JSON format

• -c flag: CSV format

• .html extension: HTML table format

Query Mode

In query mode all opened logs can be queried with SQL-like syntax using XPQL (Rumi Query Language).

Key features:

• SQL-like SELECT, WHERE, GROUP BY, ORDER BY, LIMIT clauses

• Create indexes for faster queries

• Export query results to various formats

• Aggregate functions (COUNT, MIN, MAX, SUM, AVG)

Example query session:

query>SELECT Customer.firstName, Entry.transactionId, timestamp, entryType from logs WHERE firstName = 'Bob'
Query executed in 90ms. Use 'next' or 'dump' to display results
query>next 10
|  Customer.firstName|       transactionId|                  timestamp|        entryType |
+--------------------+--------------------+---------------------------+------------------+
|                 Bob|                   1|2014-01-27 05:28:41.832 PST|             'Put'|
|                 Bob|                   2|2014-01-27 05:28:41.835 PST|          'Update'|
No more results

Switch to query mode with:

> switch query

For detailed information on query syntax, see Querying Transaction Logs.

Scripting

The tool supports scripting for automation and batch processing.

Init Scripts

Starting with 1.9, the Transaction Log Tool accepts a -i command line option that is interpreted as a comma separated list of initial scripts to run prior to entering the main command processing loop in interactive mode, or before running a script specified by the -s option.

Init scripts are useful in cases where an application would always issue a set of commands at startup.

You can also set init scripts via -Dnv.tlt.initScripts system property.

Batch Mode

Launch the tool in non-interactive mode by supplying a script using the -s <scriptname> option:

rumi tlt -s my-script.txt

Script format:

• Lines starting with // are comments

• Blank lines are ignored

• Scripts should end with exit or bye command

• Each line is processed as a command

Script Command

Run a script from within the tool:

> script my-analysis.txt

Properties (Variables)

The tool supports two types of properties:

Configuration Properties

Settings that configure tool behavior. These are persistent across tool invocations for a given user.

Common configuration properties:

Property	Default	Description
timestampFormat	yyyy-MM-dd HH:mm:ss.SSS z	Timestamp display format
timestampTZ	DEFAULT	Timezone for timestamps (DEFAULT uses system timezone)
displayMetadata	On	Show metadata with entries (On/Off/Only)
filterUnsetFields	Off	Filter unset field values in detailed output
jsonStyle	Default	JSON output style (Default/Minimal/SingleLine/Custom)
selectPreviewCount	0	Number of results to preview after query execution
deleteExtractedArchivesDirOnExit	off	Clean up extracted archives on exit

Runtime Properties

Union of environment and system properties available to scripts.

Setting Properties

Two ways to set properties:

> set <propName> <propValue>
> <propName>=<propValue>

Listing Properties

> get message
> get -a             # List all properties
> get -a os.         # Filter properties starting with "os."

Property Substitution

Scripts and commands support variable substitution:

> message = "Hello World"
> echo ${message}
Hello World
> echo ${user.name} says "${message2, "Default Message"}"
Bob says "Default Message"

Syntax: ${propertyName} or ${propertyName, defaultValue}

Customizing Output

Transaction log entries can be displayed in various formats.

Tabular Format

Default textual table format.

Browse mode - Compact view showing relationships and transaction boundaries:

Type |                          Class |                             ObjectId |                             ParentId |   TxnId | StblTxId|  InSeq# | OutSeq# | MsgSeq# | ChkPnt# | I | O | D | E
   P |                       Customer | f5162d70-bfd8-11e4-8390-c48508440345 |                                 null | 0000001 | 0000000 | 0000000 | 0000000 | 0000000 | 0000000 | F | F | F | T

Query mode - Shows selected fields:

query>SELECT firstName, lastName, customerId from logs
|           firstName|            lastName|          customerId|
+--------------------+--------------------+--------------------+
|Bob                 |null                |                   1|

CSV Format

Enabled with -c option or when dumping to .csv file. Useful for:

• Exporting to other tools

• Viewing truncated data

• Spreadsheet analysis

> next -c
Type,Class,ObjectId,ParentId,TxnId,StblTxId,InSeq#,OutSeq#,MsgSeq#,ChkPnt#,ContainsInbound,ContainsOutbound,PossibleDup,CommitEnd
Put,Customer,f5162d70-bfd8-11e4-8390-c48508440345,null,1,0,0,0,0,0,false,false,false,true

HTML Format

Output to .html file with dump command creates HTML table. Unlike tabular mode, strings aren't truncated.

> dump myResults.html

Detailed (JSON) Format

Use -d flag to display entries in JSON format:

> next -d
{
  "entryType" : "Put",
  "checkpointVersion" : 0,
  "className" : "com.neeve.rog.log.test.unit.messages.proto.Customer",
  "commitEnd" : true,
  "contentEncodingType" : "Protobuf",
  "entrySize" : 195,
  "filePosition" : 9,
  "id" : "f5162d70-bfd8-11e4-8390-c48508440345",
  "logName" : "testLog",
  "payloadSize" : 21,
  "simpleClassName" : "Customer",
  "stableTransactionId" : 0,
  "timestamp" : "02/28/2015 10:05:27.367PM PST",
  "transactionId" : 1,
  "metadata" : { ... },
  "object" : { ... }
}

Control what's displayed with displayMetadata property:

• On - Show metadata and object (default)

• Off - Show only object

• Only - Show only metadata

Filter unset fields with filterUnsetFields property:

> set filterUnsetFields On

JSON Style Formats

Control JSON formatting with jsonStyle property:

Style	Description
Default	Jackson default pretty printer (indented with 2 spaces)
Minimal	Single line, compact
SingleLine	Single line with spaces after colons
Custom	Custom PrettyPrinter class via nv.json.customPrettyPrinter property

Dates and Times

Timestamp Display

Timestamps are formatted using timestampFormat and timestampTZ properties.

View current settings:

> get -a timestamp
|       Property|                    Value|                         Description|
+---------------+-------------------------+------------------------------------+
|timestampFormat|yyyy-MM-dd HH:mm:ss.SSS z|The default timestamp format to use |
|timestampTZ    |DEFAULT                  |The timezone in which to display    |
|               |                         |timestamps. A value of 'DEFAULT'    |
|               |                         |uses the current system timezone    |

Change settings:

> set timestampFormat "MM/dd/yyyy HH:mm:ss.SSS z"
> set timestampTZ UTC
> reset timestampTZ     # Reset to default

Timestamp Input

TLT accepts timestamps in queries in multiple formats. See Querying Transaction Logs for detailed timestamp parsing information.

Running From Archives

The archive command bundles a log with the binaries needed to launch the tool into an executable jar, making it easy to analyze older archived logs.

Create archive:

> archive mylog.log

Run archive:

java -jar archive.jar

Specify extraction location:

java -jar archive.jar -e /path/to/extraction-dir

Archived contents will be extracted and opened automatically.

Common Commands

File Operations

open

Open transaction logs for browsing or querying.

Usage:
  open [-h] [-r] [-m] <log>

       [-r|--repair] Repair log (requires writable mode)
       [-m|--mode] <r|rwd|rws|rw> Open mode (default='r')
       [-p|--properties] Comma separated prop=value configuration properties
       log: Path to log file or directory
       [additionalLogs] Additional logs or directories

Examples:

> open OrderProcessingApp          # Open specific log
> open .                           # Open all logs in working directory
> open /logs/backupLogs            # Open all logs in directory

close

Close open logs and queries.

Usage:
  close [-h] [<target>]

       [target] Name of log or directory to close

Examples:

> close OrderProcessingApp         # Close specific log
> close                            # Close all logs

switch

Switch between browse and query modes or between logs.

Usage:
  switch [-h] [-l] [<target>]

       [-l|--list] Display list of options
       [target] Log name for browse mode, 'query' for query mode, 'browse' for browse mode

Examples:

> switch query                     # Switch to query mode
> switch browse                    # Switch to browse mode
> switch processor-1               # Switch to processor-1 log
> switch -l                        # List available logs

list

List various objects (logs, types, indexes, commands, schema).

Usage:
  LIST [-h] [-v] <type> [<filter>]

       [-v|--verbose] Show extra details if available
       type: <schema|open|logs|commands|indexes|types>
       [filter] Optional case sensitive filter

Examples:

> list logs                        # List open logs
> list types                       # List types in logs
> list indexes                     # List indexes on open logs
> list commands                    # List available commands

Browsing Commands

next

Display next entries from log or query results.

Usage:
  next [-h] [-r] [-d] [-c] [<count>]

       [-r|--raw] Write raw body
       [-d|--detailed] Detailed dump (JSON format)
       [-c|--csv] CSV format for brief output
       [count] Number of entries (default=1)

Examples:

> next                            # Show next entry
> next 10                         # Show next 10 entries
> next -d                         # Show next entry in detail
> next -c                         # Show next entry in CSV

nexttxn

Display next application transaction (browse mode only).

Usage:
  nexttxn [-h] [-d] [-c] [-o]

       [-d|--detailed] Detailed dump
       [-c|--csv] CSV format
       [-o|--sorted] DEPRECATED

skip

Skip entries or query results.

Usage:
  skip [-h] [<count>]

       [count] Number of entries to skip (default=1)

skiptxn

Skip transactions (browse mode only).

Usage:
  skiptxn [-h] [<count>]

       [count] Number of transactions to skip (default=1)

rewind

Reset to beginning of log or query results.

Usage:
  rewind [-h]

dump

Dump log contents or query results to console or file.

Usage:
  dump [-h] [-d] [-r] [-g] [-o] [-s] [-x] [-n] [-f] [-p] [<dumpFile>]

       [-d|--detailed] Detailed dump (JSON)
       [-r|--raw] Raw format
       [-g|--group] Group by transaction
       [-o|--sorted] DEPRECATED
       [-s|--state] Build and dump application state graph
       [-x|--skipEntries] Skip displaying entries
       [-n|--noOrphan] Skip entries without transaction id
       [-f|--force] Force overwrite of existing file
       [-p|--packets] Dump packets instead of entries (browse mode)
       [dumpFile] Output file (.csv for CSV, .html for HTML)

Examples:

> dump                            # Dump all to console
> dump results.csv                # Dump to CSV file
> dump results.html               # Dump to HTML file
> dump -d details.txt             # Dump detailed (JSON) to file

tail

Tail entries in current log (browse mode only).

Usage:
  tail [-h] [-t] [-s] [-o] [-d] [-f] [-i] [-p] [-c] [-n]

       [-t|--transactions] Tail completed transactions
       [-s|--stats] Display aggregate log stats
       [-o|--sorted] DEPRECATED
       [-d|--detailed] Detailed dump
       [-f|--follow] Continually follow log file
       [-i|--interval] Check interval in seconds (default=1.0)
       [-p|--noorphan] Skip orphans
       [-c|--csv] CSV format
       [-n|--count] Number of lines (default=10)

Examples:

> tail                            # Show last 10 entries
> tail -n 20                      # Show last 20 entries
> tail -f                         # Follow log continuously
> tail -t                         # Tail transactions

stats

Display log file statistics (browse mode only).

Usage:
  stats [-h] [-a]

       [-a|--all] Read to end and display all stats

Query Commands

SELECT / select

Execute SQL-like queries against open logs.

Usage:
  SELECT [-h]

       query: FROM <logs> WHERE <conditions>

Examples:

SELECT * FROM logs WHERE firstName = 'Bob'
SELECT Customer.firstName, timestamp FROM logs WHERE Entry.entryType = 'Put'
SELECT COUNT(*), lastName FROM logs GROUP BY lastName
SELECT * FROM logs ORDER BY timestamp DESC LIMIT 10

See Querying Transaction Logs for comprehensive query syntax documentation.

CREATE / create

Create indexes on log fields for faster queries.

Usage:
  create [-h]

       indexArgs: <optional:UNIQUE> INDEX <name> ON <log_name|logs>(<indexField>)

Examples:

CREATE UNIQUE INDEX customer_u1 ON some_repo(com/foo/bar/Customer.id)
CREATE INDEX customer_n1 ON another_repo(Customer.address.zipcode)
CREATE UNIQUE INDEX metadata_id ON repo(Metadata.id)

DROP / drop

Drop an index.

Usage:
  drop [-h]

       name: DROP INDEX <indexName>

Example:

DROP INDEX customer_u1

DESCRIBE / describe

Describe fields for a given object type.

Usage:
  DESCRIBE [-h] <type>

       type: Type name (fully qualified if ambiguous)

Examples:

describe Customer
describe com/mycompany/Customer
describe Customer.address
describe entry
describe metadata

SCHEMA / schema / PACKAGE / package

Set default package for resolving class references.

Usage:
  SCHEMA [-h] [-x] [<packageName>]

       [-x|--clear] Clear current default schema
       [packageName] Package name (e.g. com.foo or com/foo)

Examples:

SCHEMA com.foo.bar
SCHEMA com/foo/bar
SCHEMA -x                          -- Clear schema

COUNT / count

Get count of query results.

Usage:
  COUNT [-h]

Analysis Commands

compare

Compare two logs.

Usage:
  compare [-h] [-v] [-c] [-m] [-s] [-l] <log1Name> <log2Name> [<outputFile>]

       [-v|--verbose] Dump debugging info
       [-c|--csv] CSV format for diffs
       [-m|--metadata] Consider platform metadata
       [-s|--state] Compare rebuilt state (state replication logs)
       [-l|--limit] Number of divergent entries to show (default=5)
       log1Name: First log to compare
       log2Name: Second log to compare
       [outputFile] Output file (.csv, .html, or text)

Example:

> compare log1 log2 differences.html

rewrite / writelog

Rewrite query results as new transaction log.

Usage:
  rewrite [-h] <outputFolder>

       outputFolder: Output folder for rewritten logs

WARNING: Recovering from a rewritten transaction log is inherently unsafe. Removing events from the recovery stream may result in an unrecoverable log or recovery with inconsistent state. Use extreme caution in production.

archive

Create executable jar archive of transaction logs.

Usage:
  archive [-h] [-m] [-n] [-a] <log>

       [-m|--includeMessageLogs] Include inbound/outbound message logs
       [-n|--name] Archive name (default: log name or 'archive')
       [-a|--additional] Additional files/directories to include
       log: Log to archive
       [additionalLogs] Additional logs to include

Examples:

> archive mylog.log
> archive -m mylog.log             # Include message logs
> archive -n my-archive mylog.log

Factory Commands

factory

Register a factory with the Rumi runtime.

Usage:
  factory [-h] <factoryClass>

       factoryClass: Fully qualified factory class name

factories

Register multiple factories from a file.

Usage:
  factories [-h] <factoryFile>

       factoryFile: File containing factory class names

Utility Commands

help

Display help information.

Usage:
  help [-h] [<command>]

       [command] Show help for specific command

Examples:

> help                             # General help
> help next                        # Help for 'next' command
> help SELECT                      # Help for SELECT query

script

Run a command script.

Usage:
  script [-h] [<script>]

       [script] Script file to execute

history

Display command history.

Usage:
  history [-h] [-c] [<n>]

       [-c|--clear] Clear command history
       [n] Show last n lines (default=1000)

echo

Display a message or toggle echo setting.

Usage:
  echo [-h] [<value>]

       [value] Message to display or on/off to toggle echo

ansi

Enable or disable ANSI output.

Usage:
  ansi [-h] [<value>]

       [value] <off|on> Turn ANSI on or off

stacktraces

Toggle exception stacktrace display.

Usage:
  stacktraces [-h] <enabled>

       enabled: <off|on|false|true>

get

Get configuration or environment property.

Usage:
  get [-h] [-a] [<propName>]

       [-a|--all] List all properties
       [propName] Property name (or filter with -a)

set

Set configuration or environment property.

Usage:
  set [-h] [<propName>] [<propValue>]

       [propName] Property name
       [propValue] Property value (omit to clear)

reset

Reset property to default value.

Usage:
  reset [-h] <propName>

       propName: Property to reset ('*' resets all)

bye / exit / quit

Exit the tool.

Usage:
  bye [-h]

Use Cases

Debugging Application Issues

Query logs to find problematic transactions:

-- Find all errors
SELECT * FROM logs WHERE Entry.entryType = 'ERROR'

-- Find specific customer updates
SELECT Customer.* FROM logs
WHERE Customer.id = '12345'
AND Entry.entryType IN ('Put', 'Update')
ORDER BY timestamp

-- Find transactions in time window
SELECT * FROM logs
WHERE Entry.timestamp BETWEEN 'yesterday at 3pm' AND 'yesterday at 5pm'

Message Flow Tracing

Track message flow across applications:

-- Find message by ID across all logs
SELECT x_repository_name, * FROM logs
WHERE messageId = 'ABC123'

-- Trace order processing
SELECT timestamp, Entry.entryType, Order.* FROM logs
WHERE Order.orderId = 12345
ORDER BY timestamp

Performance Analysis

Analyze transaction patterns and performance:

-- Count transactions by type
SELECT Entry.entryType, COUNT(*) FROM logs
GROUP BY Entry.entryType

-- Find large messages
SELECT Entry.payloadSize, Entry.className FROM logs
WHERE Entry.payloadSize > 10000
ORDER BY Entry.payloadSize DESC
LIMIT 20

-- Transactions per customer
SELECT Customer.id, COUNT(*) as txnCount FROM logs
GROUP BY Customer.id
ORDER BY txnCount DESC

State Reconstruction

Rebuild and analyze application state:

# Dump rebuilt state
> switch mylog
> dump -s state-dump.txt

Log Comparison

Compare logs from different environments or time periods:

> compare prod-log dev-log comparison.html

Archiving for Analysis

Create portable archives for offline analysis:

> archive -m -n incident-2024-01-04 processor-1.log

Then share incident-2024-01-04.jar for analysis:

java -jar incident-2024-01-04.jar

Prerequisites

• Java 11+ (provided by Rumi CLI)

• Access to transaction log files

• For querying: Application's message/entity factory classes

Tips

1. Create indexes for frequently queried fields to improve performance

2. Use relative timestamps for quick queries: 'ten minutes ago' is easier than exact timestamps

3. Start broad, refine - Begin with wide queries then add WHERE clauses to narrow results

4. Export results to CSV/HTML for sharing or further analysis in spreadsheet tools

5. Use scripts for repeatable analysis workflows

6. Set timestampFormat to match your preferred format for easier reading

7. Filter unset fields in JSON output for cleaner detailed views

Related Documentation

• Querying Transaction Logs - Comprehensive query syntax and examples

• Heartbeats Dump Tool (hbdump) - Extract and analyze heartbeat messages

• Cloud Local Commands - Manage running applications

---

Last updated: 2025-01-04