Version 11.2.0
All Classes Functions Variables Typedefs Enumerations Enumerator Modules Pages
Data Sources

WiredTiger provides access to data from a variety of sources. At the lowest level, data may be stored in a file using a tree structure. A relational schema supporting tables, indices and column groups is layered on top of file. Additional sources include LSM trees and statistics, and applications can further extend the supported types by implementing the WT_DATA_SOURCE interface.

Common operations on all data sources are performed using WT_CURSOR handles. See Cursor operations for a description of how to use cursors.

Builtin data sources

The following are the builtin basic cursor types:

URITypeNotes
table:<table name>[<projection>]table cursortable key, table value(s) with optional projection of columns
colgroup:<table name>:<column group name>column group cursortable key, column group value(s)
index:<table name>:<index name>[<projection>]index cursorkey=index key, value=table value(s) with optional projection of columns
join:table:<table name>[<projection>]join cursorkey=table key, value=table value(s) with optional projection of columns

Some administrative tasks can be accomplished using the following special cursor types that give access to data managed by WiredTiger:

URITypeNotes
backup:backup cursorkey=string, see Backups for details
backup:exportexport cursor that generates a text file WiredTiger.export. The file contains metadata for all objects in the database. It can be used in the import process as the value for metadata_file configuration optionkey=string, see Export tables using backup cursor for details
backup:query_idbackup cursor that only returns block incremental idskey=string, see Backups for details
log:log cursorkey=(long fileID, long offset, int seqno),
value=(uint64_t txnid, uint32_t rectype,
uint32_t optype, uint32_t fileid,
WT_ITEM key, WT_ITEM value)
,
see Log cursors for details
metadata:[create]metadata cursor (optionally only returning configuration strings for WT_SESSION::create if create is appended)key=string, value=string,
see Reading WiredTiger Metadata for details
statistics:[<data source URI>]database, data source, join or session statistics cursorkey=int id,
value=(string description, string value, uint64_t value),
see Statistics Data for details

Advanced applications may also open the following low-level cursor types:

URITypeNotes
file:<file name>file cursorfile key, file value(s)
lsm:<name>LSM cursor (key=LSM key, value=LSM value)LSM key, LSM value,
see Log-Structured Merge Trees

Raw Files

WiredTiger's schema layer can be bypassed by opening cursors with a "file:" URI, using the name of the underlying file. This can be useful for seeing the contents of a column group or index without reading all of the columns from the table.

For example, if an index becomes inconsistent with its primary, a file cursor can read from the index without errors (even though some of the keys that are returned may not exist in the primary).

Table Index data

When an index is created for a table, records are inserted into the index whenever the table is updated. These records use a different key to the primary table, as specified when the index is created with the WT_SESSION::create method.

A cursor opened on an index has the specified index columns as its key, accessed by WT_CURSOR::set_key and WT_CURSOR::get_key. The value columns default to returning the value columns from the table, but this can be overridden by configuring a projection cursor (see Projections), which can access the table key columns or a subset of the value columns.

Statistics Data

Statistics cursors can be used to retrieve run-time statistics about a WiredTiger database as well as statistics for individual data sources. The statistics are at two levels: per-database and per-individual data source. Database-wide statistics are retrieved with the "statistics:" URI; individual data source statistics are available by specifying "statistics:<data source URI>". Statistics about a join cursor can be retrieved by specifying "statistics:join" and supplying the join cursor as an argument in the SESSION::open_cursor call. Statistics about a session can be retrieved by specifying "statistics:session".

The statistic key is an integer from the list of keys in Statistics Keys. Statistics cursors return three values from the WT_CURSOR::get_value call: a printable description of the statistic, a printable version of the entry's value, and the entry's unsigned 64-bit integral value, respectively.

The cursor's statistics values are loaded when the cursor is opened and remain unchanged for the life of the cursor, unless the cursor is reset by calling the WT_CURSOR::reset method, which reloads the values. A reset call on session statistics cursor would set the statistics values to zero.

The following is an example of printing run-time statistics about the WiredTiger engine:

error_check(session->open_cursor(session, "statistics:", NULL, NULL, &cursor));
print_cursor(cursor);
error_check(cursor->close(cursor));

The following is an example of printing statistics about a table:

error_check(session->open_cursor(session, "statistics:table:access", NULL, NULL, &cursor));
print_cursor(cursor);
error_check(cursor->close(cursor));

The following is an example of printing statistics about a join cursor:

error_check(session->open_cursor(session, "statistics:join", join_cursor, NULL, &stat_cursor));
print_cursor(stat_cursor);
error_check(stat_cursor->close(stat_cursor));

The following is an example of printing statistics about a session cursor:

error_check(session->open_cursor(session, "statistics:session", NULL, NULL, &stat_cursor));
print_cursor(stat_cursor);
error_check(stat_cursor->close(stat_cursor));

These four examples can use a common display routine that iterates through the statistics until the cursor returns the end of the list.

void
print_cursor(WT_CURSOR *cursor)
{
const char *desc, *pvalue;
int64_t value;
int ret;
while ((ret = cursor->next(cursor)) == 0) {
error_check(cursor->get_value(cursor, &desc, &pvalue, &value));
if (value != 0)
printf("%s=%s\n", desc, pvalue);
}
scan_end_check(ret == WT_NOTFOUND);
}

Individual statistics values can be retrieved by searching for the corresponding key, as shown in the following example:

WT_CURSOR *cursor;
const char *desc, *pvalue;
int64_t value;
error_check(session->open_cursor(session, "statistics:table:access", NULL, NULL, &cursor));
error_check(cursor->search(cursor));
error_check(cursor->get_value(cursor, &desc, &pvalue, &value));
printf("%s=%s\n", desc, pvalue);
error_check(cursor->close(cursor));

See Performance monitoring with statistics for more examples of how statistics can be used.

WT_SESSION::open_cursor
int open_cursor(WT_SESSION *session, const char *uri, WT_CURSOR *to_dup, const char *config, WT_CURSOR **cursorp)
Open a new cursor on a data source or duplicate an existing cursor.
WT_CURSOR
A WT_CURSOR handle is the interface to a cursor.
Definition: wiredtiger.in:199
WT_CURSOR::search
int search(WT_CURSOR *cursor)
Return the record matching the key.
WT_CURSOR::next
int next(WT_CURSOR *cursor)
Return the next record.
WT_CURSOR::get_value
int get_value(WT_CURSOR *cursor,...)
Get the value for the current record.
WT_CURSOR::close
int close(WT_CURSOR *cursor)
Close the cursor.
WT_STAT_DSRC_BTREE_OVERFLOW
#define WT_STAT_DSRC_BTREE_OVERFLOW
btree: overflow pages, only reported if tree_walk or all statistics are enabled
Definition: wiredtiger.in:7043
WT_CURSOR::set_key
void set_key(WT_CURSOR *cursor,...)
Set the key for the next operation.
WT_NOTFOUND
#define WT_NOTFOUND
Item not found.
Definition: wiredtiger.in:4001