Cursor operations

Common operations in WiredTiger are performed using WT_CURSOR handles. A cursor includes:

• a position within a data source
• getter/setters for key and value fields
• encoding of fields to store in the data source
• methods to navigate within and iterate through the data

Opening a cursor

Cursors are created using the WT_SESSION::open_cursor method. For example, from the program ex_cursor.c:

    error_check(session->open_cursor(session, "table:world", NULL, NULL, &cursor));

Another example from the same program:

    error_check(
      session->open_cursor(session, "table:world(country,population)", NULL, NULL, &cursor));

In addition to traditional data sources, cursors in WiredTiger are used to access projections and even created data sources such as the run-time statistics:

    error_check(session->open_cursor(session, "statistics:", NULL, NULL, &cursor));

See Cursors for more information on available cursor types.

Closing a cursor

Cursors remain open until either WT_CURSOR::close is called or the cursor's session is closed, which may either be in WT_SESSION::close or WT_CONNECTION::close.

Positioning a cursor

Cursors may be positioned at the beginning of the data source, the end of the data source, at an exact key within the data source, and near a key within the data source.

To invalidate the position of a cursor so that subsequent iterations start from the beginning or end of the data source, use the WT_CURSOR::reset method:

int
cursor_reset(WT_CURSOR *cursor)
{
    return (cursor->reset(cursor));
}

To move a cursor forward in the data source, use the cursor WT_CURSOR::next method:

int
cursor_forward_scan(WT_CURSOR *cursor)
{
    const char *key, *value;
    int ret;
 
    while ((ret = cursor->next(cursor)) == 0) {
        error_check(cursor->get_key(cursor, &key));
        error_check(cursor->get_value(cursor, &value));
    }
    scan_end_check(ret == WT_NOTFOUND);
 
    return (0);
}

If the WT_CURSOR::next method is called on a cursor without a position in the data source, it is positioned at the beginning of the data source.

To move a cursor backward in the data source, use the cursor WT_CURSOR::prev method:

int
cursor_reverse_scan(WT_CURSOR *cursor)
{
    const char *key, *value;
    int ret;
 
    while ((ret = cursor->prev(cursor)) == 0) {
        error_check(cursor->get_key(cursor, &key));
        error_check(cursor->get_value(cursor, &value));
    }
    scan_end_check(ret == WT_NOTFOUND);
 
    return (0);
}

If the WT_CURSOR::prev method is called on a cursor without a position in the data source, it is positioned at the end of the data source.

To position a cursor at a specific location in the data source, use the WT_CURSOR::search method:

int
cursor_search(WT_CURSOR *cursor)
{
    const char *value;
 
    cursor->set_key(cursor, "foo");
 
    error_check(cursor->search(cursor));
    error_check(cursor->get_value(cursor, &value));
 
    return (0);
}

To position a cursor at or near a location in the data source, use the WT_CURSOR::search_near method:

int
cursor_search_near(WT_CURSOR *cursor)
{
    const char *key, *value;
    int exact;
 
    cursor->set_key(cursor, "foo");
 
    error_check(cursor->search_near(cursor, &exact));
    switch (exact) {
    case -1: /* Returned key smaller than search key */
        error_check(cursor->get_key(cursor, &key));
        break;
    case 0: /* Exact match found */
        break;
    case 1: /* Returned key larger than search key */
        error_check(cursor->get_key(cursor, &key));
        break;
    }
    error_check(cursor->get_value(cursor, &value));
 
    return (0);
}

After a transaction is successfully committed, cursors in the session retain their position, as well as any currently set keys or values they may have. If a transaction is rolled back for any reason, cursors in the session are reset (as if the WT_CURSOR::reset method was called), discarding any cursor position as well as any currently set keys or values.

Cursors can be configured to move to a random position with WT_CURSOR::next is called, see Cursor random for details.

Inserting, removing and updating

To insert new data, and optionally update existing data, using a cursor, use the WT_CURSOR::insert method:

int
cursor_insert(WT_CURSOR *cursor)
{
    cursor->set_key(cursor, "foo");
    cursor->set_value(cursor, "bar");
 
    return (cursor->insert(cursor));
}

To update existing data using a cursor, use the WT_CURSOR::update method:

int
cursor_update(WT_CURSOR *cursor)
{
    cursor->set_key(cursor, "foo");
    cursor->set_value(cursor, "newbar");
 
    return (cursor->update(cursor));
}

The WT_SESSION::open_cursor overwrite configuration is true by default, causing WT_CURSOR::insert and WT_CURSOR::update to ignore the current state of the record, and these methods will succeed regardless of whether or not the record previously exists. When an application configures overwrite to false, WT_CURSOR::insert will fail with WT_DUPLICATE_KEY if the record previously exists and WT_CURSOR::update will fail with WT_NOTFOUND if the record does not previously exist.

To remove existing data using a cursor, use the WT_CURSOR::remove method:

int
cursor_remove(WT_CURSOR *cursor)
{
    cursor->set_key(cursor, "foo");
    return (cursor->remove(cursor));
}

Returning the largest key

The WT_SESSION::largest_key gets the largest key in a table regardless of any underlying visibility considerations.

        const char *largest_key;
        error_check(cursor->largest_key(cursor));
        error_check(cursor->get_key(cursor, &largest_key));

Any following prev or next calls will behave as if they were invoked on an unpositioned cursor whether the largest key call is successful or not.

Cursor position after error

After any cursor handle method failure, the cursor's position is undetermined. For cursor operations that expect a key to be set before the operation begins (including WT_CURSOR::search, WT_CURSOR::insert, WT_CURSOR::update and WT_CURSOR::remove), the application's key and value will not be cleared by an error.

Applications that cannot re-position the cursor after failure must duplicate the cursor by calling WT_SESSION::open_cursor and passing the cursor as the to_dup parameter before calling a cursor method that will attempt to re-position the cursor. Cursor duplication is not supported for the backup, config and statistics cursor types.

Cursor key/value memory scoping

When applications pass a pointer (either to a WT_ITEM or a string), to WT_CURSOR::set_key or WT_CURSOR::set_value, WiredTiger does not copy the memory referenced by the pointer. For this reason, the application must keep the referenced memory unchanged and valid until the next operation that successfully positions the cursor, modifies the underlying data, or the cursor is reset or closed (discarding its resources). The operations that position the cursor are WT_CURSOR::next, WT_CURSOR::prev, WT_CURSOR::search and WT_CURSOR::search_near; the operations that modify the underlying data are WT_CURSOR::insert, WT_CURSOR::update and WT_CURSOR::remove.

If a cursor operation fails (for example, due to a WT_NOTFOUND error), it may be retried without calling WT_CURSOR::set_key or WT_CURSOR::set_value again. That is, the cursor may still reference the application-supplied memory until the cursor is successfully positioned, underlying data is modified, or the cursor is closed or reset.

Any pointers returned by WT_CURSOR::get_key or WT_CURSOR::get_value are only valid until a subsequent cursor call that successfully positions the cursor, modifies the underlying data, or the cursor is reset or closed. These pointers may reference private WiredTiger data structures that may not be modified or freed by the application. If a longer scope is required, the application must make a copy of the memory before the cursor is re-used, closed or reset.

Apply bounds to the cursor

A cursor may have a set of bounds applied to it, this restricts the key range that the cursor operates within.

Suppose we have a table with keys A to Z. We can restrict the cursor to only operate over keys L->Q. If the cursor walks past key Q, the cursor will enter an unpositioned state having walked off the "end" of the table from its perspective. Additionally should the cursor perform a search_near, it will only search within the key range of L->Q. This can provide an application with a performance boost when operating within tables that have a large number of deleted or invisible records. This is because the key comparison with the bound occurs before we determine if the value associated with the key is visible or not. The cursor can have a lower bound set or an upper bound set, or both.

int
cursor_bound(WT_CURSOR *cursor)
{
    cursor->set_key(cursor, "A");
    error_check(cursor->bound(cursor, "action=set,bound=lower"));
    return (0);
}

Both or either one of the lower and upper bounds can be set, if a bound is not set then the usual start or end of the file applies. Given our previous example had we set a lower bound of L and no upper bound then our key range would be L->Z. When setting the bounds, the relevant bound is set prior with cursor->set_key. This key does not have to exist or be visible in order for bounds setting to succeed.

Cursor operations next, prev, search, search near, modify, insert, update, and remove support bounds. When performing a traversal such as cursor->next or cursor->prev, the cursor will only return keys that are within the bounds that have been set. For example, calling cursor->next on an unpositioned range bounded cursor will position it on the lower bound rather than the start of the key range.

If search is called on a key that's not within bounds, this will result in returning a WT_NOTFOUND. Calling cursor->search_near with a key outside the range will always return a key within the range unless no keys are present. cursor->search_near can benefit from range bounded cursors, once the cursor traverses outside of the bounded range the search_near logic can early exit reducing the overall number of keys visited.

Clearing the bounds can be done through resetting the cursor with cursor->reset or by passing a clear config to the bounds API call. This will unconditionally clear both bounds.

Bounds API is supported with row-store and variable length column store. Bounds API is not supported with fixed length column store.

The bounds API supports these cursor types:

• File cursor
• Table cursor
• Datastore cursor
• Dump cursor
• Index cursor