> For the complete documentation index, see [llms.txt](https://docs.rumi.systems/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.rumi.systems/rumi-cli/commands/tools/tlt.md). # tlt The Transaction Log Tool (`rumi tlt`) is an interactive utility for querying and analyzing Rumi transaction logs. ## Overview The Transaction Log Tool provides capabilities for browsing and querying the recovery transaction logs and inbound/outbound transaction logs recorded by a Rumi application. The terms 'log' and 'transaction log' are used interchangeable herein for the sake of brevity. **Key capabilities:** * **Browse transaction logs** - Navigate through log entries * **Query with SQL-like syntax** - Use XPQL (Rumi Query Language) to query logs * **Create indexes** - Build indexes for faster queries * **Export results** - Dump results to console, CSV, or HTML * **Archive logs** - Package logs with tool for portable analysis * **Compare logs** - Identify differences between log files ## Usage **Syntax:** ```bash rumi tlt [log-path] ``` **Parameters:** * `[log-path]` - Optional path to transaction log or directory (opens all logs in directory) **Examples:** ```bash # Launch tool in current directory rumi tlt # Open specific log rumi tlt /path/to/application.log # Open all logs in directory rumi tlt /path/to/logs/ ``` ## Interactive Tool Features The tool supports a subset of [ReadLine](http://cnswww.cns.cwru.edu/php/chet/readline/rltop.html) functionality with familiar unix-style command line operations: * Command history * Tab command completion * Command scripting ### Basic Usage Example Opening logs and browsing: ``` > open rdat <10,4604,MY-PC> 20181118-14:15:53:144 (inf)...[RogLog->'processor-1'] Live transaction log file is 'processor-1.log' [C:\dev\nvx-app-talon-starter\target\testbed\rdat\processor-1-1]... <10,4604,MY-PC> 20181118-14:15:53:514 (inf)...Log open (size=15825952, entries=59793) took 361 milliseconds. <10,4604,MY-PC> 20181118-14:15:53:572 (inf)...[RogLog->'processor-1'] Scavenging old log files.... <10,4604,MY-PC> 20181118-14:15:53:572 (inf)...[RogLog->'processor-1'] ....scavenged 0 files (0 failed). reader opened with Lazy deserialization. rdat/processor-1> ``` Displaying entries: ``` rdat/processor-1> next 1 I=InboundMsgsInTxn? O=OutboundMsgsInTxn? D=PossibleDup? S=CommitStart? E=CommitEnd? Type | Class | ObjectId | ParentId | TxnId | StblTxId| InSeq# | OutSeq# | MsgSeq# | ChkPnt# | I | O | D | S | E P | AepFlow | 2c708c60-eb73-11e8-880f-9eb6348a2af3 | null | 0000001 | 0000001 | 0000000 | 0000000 | 0000000 | 0000001 | F | F | F | F | F P | Repository | 2c7399a1-eb73-11e8-880f-9eb6348a2af3 | 2c708c60-eb73-11e8-880f-9eb6348a2af3 | 0000001 | 0000001 | 0000000 | 0000000 | 0000000 | 0000001 | F | F | F | F | F ``` ## Operating Modes The Transaction Log Tool operates in two modes: ### Browse Mode In browse mode the prompt indicates the log that is being browsed. Navigate logs using: * `next` - Display next entry * `nexttxn` - Display next transaction * `skip` / `skiptxn` - Skip entries/transactions * `rewind` - Return to start of log * `switch` - Switch between logs * `dump` - Dump entire log to console or file **Format options:** * Default: Brief tabular format * `-d` flag: Detailed JSON format * `-c` flag: CSV format * `.html` extension: HTML table format ### Query Mode In query mode all opened logs can be queried with SQL-like syntax using XPQL (Rumi Query Language). **Key features:** * SQL-like SELECT, WHERE, GROUP BY, ORDER BY, LIMIT clauses * Create indexes for faster queries * Export query results to various formats * Aggregate functions (COUNT, MIN, MAX, SUM, AVG) **Example query session:** ```sql query>SELECT Customer.firstName, Entry.transactionId, timestamp, entryType from logs WHERE firstName = 'Bob' Query executed in 90ms. Use 'next' or 'dump' to display results query>next 10 | Customer.firstName| transactionId| timestamp| entryType | +--------------------+--------------------+---------------------------+------------------+ | Bob| 1|2014-01-27 05:28:41.832 PST| 'Put'| | Bob| 2|2014-01-27 05:28:41.835 PST| 'Update'| No more results ``` Switch to query mode with: ``` > switch query ``` For detailed information on query syntax, see [Querying Transaction Logs](/rumi-core/guides/operating-applications/analysis-and-troubleshooting/querying-transaction-logs.md). ## Scripting The tool supports scripting for automation and batch processing. ### Init Scripts Starting with 1.9, the Transaction Log Tool accepts a `-i` command line option that is interpreted as a comma separated list of initial scripts to run prior to entering the main command processing loop in interactive mode, or before running a script specified by the `-s` option. Init scripts are useful in cases where an application would always issue a set of commands at startup. You can also set init scripts via `-Dnv.tlt.initScripts` system property. ### Batch Mode Launch the tool in non-interactive mode by supplying a script using the `-s ` option: ```bash rumi tlt -s my-script.txt ``` **Script format:** * Lines starting with `//` are comments * Blank lines are ignored * Scripts should end with `exit` or `bye` command * Each line is processed as a command ### Script Command Run a script from within the tool: ``` > script my-analysis.txt ``` ## Properties (Variables) The tool supports two types of properties: ### Configuration Properties Settings that configure tool behavior. These are persistent across tool invocations for a given user. **Common configuration properties:** | Property | Default | Description | | ---------------------------------- | --------------------------- | ------------------------------------------------------ | | `timestampFormat` | `yyyy-MM-dd HH:mm:ss.SSS z` | Timestamp display format | | `timestampTZ` | `DEFAULT` | Timezone for timestamps (DEFAULT uses system timezone) | | `displayMetadata` | `On` | Show metadata with entries (On/Off/Only) | | `filterUnsetFields` | `Off` | Filter unset field values in detailed output | | `jsonStyle` | `Default` | JSON output style (Default/Minimal/SingleLine/Custom) | | `selectPreviewCount` | `0` | Number of results to preview after query execution | | `deleteExtractedArchivesDirOnExit` | `off` | Clean up extracted archives on exit | ### Runtime Properties Union of environment and system properties available to scripts. ### Setting Properties Two ways to set properties: ``` > set > = ``` ### Listing Properties ``` > get message > get -a # List all properties > get -a os. # Filter properties starting with "os." ``` ### Property Substitution Scripts and commands support variable substitution: ``` > message = "Hello World" > echo ${message} Hello World > echo ${user.name} says "${message2, "Default Message"}" Bob says "Default Message" ``` Syntax: `${propertyName}` or `${propertyName, defaultValue}` ## Customizing Output Transaction log entries can be displayed in various formats. ### Tabular Format Default textual table format. **Browse mode** - Compact view showing relationships and transaction boundaries: ``` Type | Class | ObjectId | ParentId | TxnId | StblTxId| InSeq# | OutSeq# | MsgSeq# | ChkPnt# | I | O | D | E P | Customer | f5162d70-bfd8-11e4-8390-c48508440345 | null | 0000001 | 0000000 | 0000000 | 0000000 | 0000000 | 0000000 | F | F | F | T ``` **Query mode** - Shows selected fields: ``` query>SELECT firstName, lastName, customerId from logs | firstName| lastName| customerId| +--------------------+--------------------+--------------------+ |Bob |null | 1| ``` ### CSV Format Enabled with `-c` option or when dumping to `.csv` file. Useful for: * Exporting to other tools * Viewing truncated data * Spreadsheet analysis ``` > next -c Type,Class,ObjectId,ParentId,TxnId,StblTxId,InSeq#,OutSeq#,MsgSeq#,ChkPnt#,ContainsInbound,ContainsOutbound,PossibleDup,CommitEnd Put,Customer,f5162d70-bfd8-11e4-8390-c48508440345,null,1,0,0,0,0,0,false,false,false,true ``` ### HTML Format Output to `.html` file with `dump` command creates HTML table. Unlike tabular mode, strings aren't truncated. ``` > dump myResults.html ``` ### Detailed (JSON) Format Use `-d` flag to display entries in JSON format: ``` > next -d { "entryType" : "Put", "checkpointVersion" : 0, "className" : "com.neeve.rog.log.test.unit.messages.proto.Customer", "commitEnd" : true, "contentEncodingType" : "Protobuf", "entrySize" : 195, "filePosition" : 9, "id" : "f5162d70-bfd8-11e4-8390-c48508440345", "logName" : "testLog", "payloadSize" : 21, "simpleClassName" : "Customer", "stableTransactionId" : 0, "timestamp" : "02/28/2015 10:05:27.367PM PST", "transactionId" : 1, "metadata" : { ... }, "object" : { ... } } ``` Control what's displayed with `displayMetadata` property: * `On` - Show metadata and object (default) * `Off` - Show only object * `Only` - Show only metadata Filter unset fields with `filterUnsetFields` property: ``` > set filterUnsetFields On ``` ### JSON Style Formats Control JSON formatting with `jsonStyle` property: | Style | Description | | ------------ | --------------------------------------------------------------------- | | `Default` | Jackson default pretty printer (indented with 2 spaces) | | `Minimal` | Single line, compact | | `SingleLine` | Single line with spaces after colons | | `Custom` | Custom PrettyPrinter class via `nv.json.customPrettyPrinter` property | ## Dates and Times ### Timestamp Display Timestamps are formatted using `timestampFormat` and `timestampTZ` properties. View current settings: ``` > get -a timestamp | Property| Value| Description| +---------------+-------------------------+------------------------------------+ |timestampFormat|yyyy-MM-dd HH:mm:ss.SSS z|The default timestamp format to use | |timestampTZ |DEFAULT |The timezone in which to display | | | |timestamps. A value of 'DEFAULT' | | | |uses the current system timezone | ``` Change settings: ``` > set timestampFormat "MM/dd/yyyy HH:mm:ss.SSS z" > set timestampTZ UTC > reset timestampTZ # Reset to default ``` ### Timestamp Input TLT accepts timestamps in queries in multiple formats. See [Querying Transaction Logs](/rumi-core/guides/operating-applications/analysis-and-troubleshooting/querying-transaction-logs.md#dates-and-timestamps) for detailed timestamp parsing information. ## Running From Archives The `archive` command bundles a log with the binaries needed to launch the tool into an executable jar, making it easy to analyze older archived logs. **Create archive:** ``` > archive mylog.log ``` **Run archive:** ```bash java -jar archive.jar ``` **Specify extraction location:** ```bash java -jar archive.jar -e /path/to/extraction-dir ``` Archived contents will be extracted and opened automatically. ## Common Commands ### File Operations #### open Open transaction logs for browsing or querying. ``` Usage: open [-h] [-r] [-m] [-r|--repair] Repair log (requires writable mode) [-m|--mode] Open mode (default='r') [-p|--properties] Comma separated prop=value configuration properties log: Path to log file or directory [additionalLogs] Additional logs or directories ``` **Examples:** ``` > open OrderProcessingApp # Open specific log > open . # Open all logs in working directory > open /logs/backupLogs # Open all logs in directory ``` #### close Close open logs and queries. ``` Usage: close [-h] [] [target] Name of log or directory to close ``` **Examples:** ``` > close OrderProcessingApp # Close specific log > close # Close all logs ``` #### switch Switch between browse and query modes or between logs. ``` Usage: switch [-h] [-l] [] [-l|--list] Display list of options [target] Log name for browse mode, 'query' for query mode, 'browse' for browse mode ``` **Examples:** ``` > switch query # Switch to query mode > switch browse # Switch to browse mode > switch processor-1 # Switch to processor-1 log > switch -l # List available logs ``` #### list List various objects (logs, types, indexes, commands, schema). ``` Usage: LIST [-h] [-v] [] [-v|--verbose] Show extra details if available type: [filter] Optional case sensitive filter ``` **Examples:** ``` > list logs # List open logs > list types # List types in logs > list indexes # List indexes on open logs > list commands # List available commands ``` ### Browsing Commands #### next Display next entries from log or query results. ``` Usage: next [-h] [-r] [-d] [-c] [] [-r|--raw] Write raw body [-d|--detailed] Detailed dump (JSON format) [-c|--csv] CSV format for brief output [count] Number of entries (default=1) ``` **Examples:** ``` > next # Show next entry > next 10 # Show next 10 entries > next -d # Show next entry in detail > next -c # Show next entry in CSV ``` #### nexttxn Display next application transaction (browse mode only). ``` Usage: nexttxn [-h] [-d] [-c] [-o] [-d|--detailed] Detailed dump [-c|--csv] CSV format [-o|--sorted] DEPRECATED ``` #### skip Skip entries or query results. ``` Usage: skip [-h] [] [count] Number of entries to skip (default=1) ``` #### skiptxn Skip transactions (browse mode only). ``` Usage: skiptxn [-h] [] [count] Number of transactions to skip (default=1) ``` #### rewind Reset to beginning of log or query results. ``` Usage: rewind [-h] ``` #### dump Dump log contents or query results to console or file. ``` Usage: dump [-h] [-d] [-r] [-g] [-o] [-s] [-x] [-n] [-f] [-p] [] [-d|--detailed] Detailed dump (JSON) [-r|--raw] Raw format [-g|--group] Group by transaction [-o|--sorted] DEPRECATED [-s|--state] Build and dump application state graph [-x|--skipEntries] Skip displaying entries [-n|--noOrphan] Skip entries without transaction id [-f|--force] Force overwrite of existing file [-p|--packets] Dump packets instead of entries (browse mode) [dumpFile] Output file (.csv for CSV, .html for HTML) ``` **Examples:** ``` > dump # Dump all to console > dump results.csv # Dump to CSV file > dump results.html # Dump to HTML file > dump -d details.txt # Dump detailed (JSON) to file ``` #### tail Tail entries in current log (browse mode only). ``` Usage: tail [-h] [-t] [-s] [-o] [-d] [-f] [-i] [-p] [-c] [-n] [-t|--transactions] Tail completed transactions [-s|--stats] Display aggregate log stats [-o|--sorted] DEPRECATED [-d|--detailed] Detailed dump [-f|--follow] Continually follow log file [-i|--interval] Check interval in seconds (default=1.0) [-p|--noorphan] Skip orphans [-c|--csv] CSV format [-n|--count] Number of lines (default=10) ``` **Examples:** ``` > tail # Show last 10 entries > tail -n 20 # Show last 20 entries > tail -f # Follow log continuously > tail -t # Tail transactions ``` #### stats Display log file statistics (browse mode only). ``` Usage: stats [-h] [-a] [-a|--all] Read to end and display all stats ``` ### Query Commands #### SELECT / select Execute SQL-like queries against open logs. ``` Usage: SELECT [-h] query: FROM WHERE ``` **Examples:** ```sql SELECT * FROM logs WHERE firstName = 'Bob' SELECT Customer.firstName, timestamp FROM logs WHERE Entry.entryType = 'Put' SELECT COUNT(*), lastName FROM logs GROUP BY lastName SELECT * FROM logs ORDER BY timestamp DESC LIMIT 10 ``` See [Querying Transaction Logs](/rumi-core/guides/operating-applications/analysis-and-troubleshooting/querying-transaction-logs.md) for comprehensive query syntax documentation. #### CREATE / create Create indexes on log fields for faster queries. ``` Usage: create [-h] indexArgs: INDEX ON () ``` **Examples:** ```sql CREATE UNIQUE INDEX customer_u1 ON some_repo(com/foo/bar/Customer.id) CREATE INDEX customer_n1 ON another_repo(Customer.address.zipcode) CREATE UNIQUE INDEX metadata_id ON repo(Metadata.id) ``` #### DROP / drop Drop an index. ``` Usage: drop [-h] name: DROP INDEX ``` **Example:** ```sql DROP INDEX customer_u1 ``` #### DESCRIBE / describe Describe fields for a given object type. ``` Usage: DESCRIBE [-h] type: Type name (fully qualified if ambiguous) ``` **Examples:** ``` describe Customer describe com/mycompany/Customer describe Customer.address describe entry describe metadata ``` #### SCHEMA / schema / PACKAGE / package Set default package for resolving class references. ``` Usage: SCHEMA [-h] [-x] [] [-x|--clear] Clear current default schema [packageName] Package name (e.g. com.foo or com/foo) ``` **Examples:** ```sql SCHEMA com.foo.bar SCHEMA com/foo/bar SCHEMA -x -- Clear schema ``` #### COUNT / count Get count of query results. ``` Usage: COUNT [-h] ``` ### Analysis Commands #### compare Compare two logs. ``` Usage: compare [-h] [-v] [-c] [-m] [-s] [-l] [] [-v|--verbose] Dump debugging info [-c|--csv] CSV format for diffs [-m|--metadata] Consider platform metadata [-s|--state] Compare rebuilt state (state replication logs) [-l|--limit] Number of divergent entries to show (default=5) log1Name: First log to compare log2Name: Second log to compare [outputFile] Output file (.csv, .html, or text) ``` **Example:** ``` > compare log1 log2 differences.html ``` #### rewrite / writelog Rewrite query results as new transaction log. ``` Usage: rewrite [-h] outputFolder: Output folder for rewritten logs ``` {% hint style="warning" %} **WARNING**: Recovering from a rewritten transaction log is inherently unsafe. Removing events from the recovery stream may result in an unrecoverable log or recovery with inconsistent state. Use extreme caution in production. {% endhint %} #### archive Create executable jar archive of transaction logs. ``` Usage: archive [-h] [-m] [-n] [-a] [-m|--includeMessageLogs] Include inbound/outbound message logs [-n|--name] Archive name (default: log name or 'archive') [-a|--additional] Additional files/directories to include log: Log to archive [additionalLogs] Additional logs to include ``` **Examples:** ``` > archive mylog.log > archive -m mylog.log # Include message logs > archive -n my-archive mylog.log ``` ### Factory Commands #### factory Register a factory with the Rumi runtime. ``` Usage: factory [-h] factoryClass: Fully qualified factory class name ``` #### factories Register multiple factories from a file. ``` Usage: factories [-h] factoryFile: File containing factory class names ``` ### Utility Commands #### help Display help information. ``` Usage: help [-h] [] [command] Show help for specific command ``` **Examples:** ``` > help # General help > help next # Help for 'next' command > help SELECT # Help for SELECT query ``` #### script Run a command script. ``` Usage: script [-h] [