Version 3.2.1
Log cursors in Java

WiredTiger cursors provide access to data from a variety of sources, and one of these sources is the records in the transaction log files. Log files may not be present in every WiredTiger database, only databases that have been configured for logging using the log configuration for wiredtiger.open. In databases with log files, a log cursor provides access to the log records. Although log cursors are read-only, applications may store records in the log using Session.log_printf. While the log cursor is open automatic log file archiving, even if enabled, will not reclaim any log files.

Each physical WiredTiger log record represents one or more operations in the database. When a log record represents more than a single operation in the database, all of the operations in a log record will be part of the same transaction, however, there is no corresponding guarantee that all operations in a transaction must appear in the same log record.

The following examples are taken from the complete example program ex_log.java.

To open a log cursor on the database:

cursor = session.open_cursor("log:", null, null);

A log cursor's key is a unique log record identifier, plus a int operation counter within that log record. When a log record reflects something log record that is not a transaction, such as the start of a checkpoint, the operation counter returned for the key will be zero. When the log record maps to a transaction, the first log record returned, with an operation counter of zero, will be the entire log record. Then the Cursor.next call will step into the transaction and return the first individual operation within that transaction and each additional individual operation, adding one to the operation counter for each one. A transaction with a single operation will return two records related to that transaction.

The unique log record identifier maps to a WT_LSN data structure, which has two fields: WT_LSN::file, the log file identifier, and WT_LSN::offset, the offset of the log record in the log file.

Here is an example of getting the log cursor's key:

lsn.file = cursor.getKeyInt();
lsn.offset = cursor.getKeyInt();
opcount = cursor.getKeyInt();

The log cursor's value is comprised of six fields:

  • a long transaction ID (set for commit records only, otherwise 0),
  • a int record type
  • a int operation type (set for commit records only, otherwise 0)
  • a int file id (if applicable, otherwise 0)
  • the operation key (commit records only, otherwise empty)
  • the operation value

The transaction ID may not be unique across recovery, that is, closing and reopening the database may result in transaction IDs smaller than previously seen transaction IDs.

The log record and log operation types are taken from log_types; typically, the only record or operation type applications are concerned with is WT_LOGREC_MESSAGE, which is a log record generated by the application.

The file ID may not be unique across recovery, that is, closing and reopening the database may result in file IDs changing. Additionally, there is currently no way to map file IDs to file names or higher-level objects.

Here is an example of getting the log cursor's value:

txnid = cursor.getValueLong();
rectype = cursor.getValueInt();
optype = cursor.getValueInt();
fileid = cursor.getValueInt();
logrec_key = cursor.getValueByteArray();
logrec_value = cursor.getValueByteArray();

For clarity, imagine a set of three log records:

  • the first recording an internal operation, say a checkpoint start,
  • the second committing a transaction with three operations,
  • the third committing a transaction with a single operation.

The log cursor's Cursor.next call will return a total of seven records. Here's an example of what it would look like:

  1. LSN=[1,1000], operation counter=0, transaction ID=0, value of checkpoint start
  2. LSN=[1,2000], operation counter=0, transaction ID=30, all operations of transaction 30
  3. LSN=[1,2000], operation counter=1, transaction ID=30, first operation of transaction 30
  4. LSN=[1,2000], operation counter=2, transaction ID=30, second operation of transaction 30
  5. LSN=[1,2000], operation counter=3, transaction ID=30, third operation of transaction 30
  6. LSN=[1,3000], operation counter=0, transaction ID=31, all operations of transaction 31
  7. LSN=[1,3000], operation counter=1, transaction ID=31, only operation of transaction 31

The first time the log cursor will return a key with a unique log ID, no transaction ID, and an operation counter of 0. The next six returns from the log cursor will have a common log ID, a common transaction ID, and operation counters starting at 0, returning the whole record and then starting at 1 and ending at 5 for each of the five individual operations. The next return from the log cursor will again have a unique log ID, a unique transaction ID, and an operation counter of 0. And the final return from the log cursor will have an operation counter of 1.

Here's a more complete example of walking the log and displaying the results:

static void
print_record(Lsn lsn, int opcount,
int rectype, int optype, long txnid, int fileid,
byte[] key, byte[] value)
{
System.out.print(
"LSN [" + lsn.file + "][" + lsn.offset + "]." + opcount +
": record type " + rectype + " optype " + optype +
" txnid " + txnid + " fileid " + fileid);
System.out.println(" key size " + key.length +
" value size " + value.length);
if (rectype == wiredtiger.WT_LOGREC_MESSAGE)
System.out.println("Application Record: " + new String(value));
}
/*
* simple_walk_log --
* A simple walk of the log.
*/
static int
simple_walk_log(Session session, int count_min)
throws WiredTigerException
{
Cursor cursor;
Lsn lsn = new Lsn();
byte[] logrec_key, logrec_value;
long txnid;
int fileid, opcount, optype, rectype;
int count, ret;
cursor = session.open_cursor("log:", null, null);
count = 0;
while ((ret = cursor.next()) == 0) {
count++;
lsn.file = cursor.getKeyInt();
lsn.offset = cursor.getKeyInt();
opcount = cursor.getKeyInt();
txnid = cursor.getValueLong();
rectype = cursor.getValueInt();
optype = cursor.getValueInt();
fileid = cursor.getValueInt();
logrec_key = cursor.getValueByteArray();
logrec_value = cursor.getValueByteArray();
print_record(lsn, opcount,
rectype, optype, txnid, fileid, logrec_key, logrec_value);
}
if (ret == wiredtiger.WT_NOTFOUND)
ret = 0;
ret = cursor.close();
if (count < count_min) {
System.err.println("Expected minimum " + count_min +
" records, found " + count);
return (1);
}
return (ret);
}

The log cursor's key can be used to search for specific records in the log (assuming the record still exists and has not been archived), by setting the key and calling Cursor.search. However, it is not possible to search for a specific operation within a log record, and the key's operation counter is ignored when the key is set. The result of a search for a log record with more than one operation is always the first operation in the log record.

Here is an example of setting the log cursor's key:

cursor.putKeyInt(lsnsave.file);
cursor.putKeyInt(lsnsave.offset);
cursor.putKeyInt(0);

Log cursors are read-only, however applications can insert their own log records using Session.log_printf. Here is an example of adding an application record into the database log:

ret = session.log_printf("Wrote " + record_count + " records");
count_min++;