Cursor operations

Common operations in WiredTiger are performed using WT_CURSOR handles.

A cursor includes:

• a position within a data source
• getter/setters for key and value fields
• encoding of fields to store in the data source
• methods to navigate within and iterate through the data

Opening a cursor

Cursors are created using the WT_SESSION::open_cursor method. For example, from the program ex_cursor.c:

        ret = session->open_cursor(session, "table:world", NULL, NULL, &cursor);

Another example from the same program:

        ret = session->open_cursor(session,
            "table:world(country,population)", NULL, NULL, &cursor);

In addition to traditional data sources, cursors in WiredTiger are used to access projections and even created data sources such as the run-time statistics:

        ret = session->open_cursor(session, "statistics:", NULL, NULL, &cursor);

See Cursors for more information on available cursor types.

Closing a cursor

Cursors remain open until either WT_CURSOR::close is called or the cursor's session is closed, which may either be in WT_SESSION::close or WT_CONNECTION::close.

Positioning a cursor

Cursors may be positioned at the beginning of the data source, the end of the data source, at an exact key within the data source, and near a key within the data source.

To invalidate the position of a cursor so that subsequent iterations start from the beginning or end of the data source, use the WT_CURSOR::reset method:

int
cursor_reset(WT_CURSOR *cursor)
{
        return (cursor->reset(cursor));
}

To move a cursor forward in the data source, use the cursor WT_CURSOR::next method:

int
cursor_forward_scan(WT_CURSOR *cursor)
{
        const char *key, *value;
        int ret;

        while ((ret = cursor->next(cursor)) == 0) {
                ret = cursor->get_key(cursor, &key);
                ret = cursor->get_value(cursor, &value);
        }
        return (ret);
}

If the WT_CURSOR::next method is called on a cursor without a position in the data source, it is positioned at the beginning of the data source.

To move a cursor backward in the data source, use the cursor WT_CURSOR::prev method:

int
cursor_reverse_scan(WT_CURSOR *cursor)
{
        const char *key, *value;
        int ret;

        while ((ret = cursor->prev(cursor)) == 0) {
                ret = cursor->get_key(cursor, &key);
                ret = cursor->get_value(cursor, &value);
        }
        return (ret);
}

If the WT_CURSOR::prev method is called on a cursor without a position in the data source, it is positioned at the end of the data source.

To position a cursor at a specific location in the data source, use the WT_CURSOR::search method:

int
cursor_search(WT_CURSOR *cursor)
{
        const char *value;
        int ret;

        cursor->set_key(cursor, "foo");

        if ((ret = cursor->search(cursor)) != 0)
                ret = cursor->get_value(cursor, &value);

        return (ret);
}

To position a cursor at or near a location in the data source, use the WT_CURSOR::search_near method:

int
cursor_search_near(WT_CURSOR *cursor)
{
        const char *key, *value;
        int exact, ret;

        cursor->set_key(cursor, "foo");

        if ((ret = cursor->search_near(cursor, &exact)) == 0) {
                switch (exact) {
                case -1:        /* Returned key smaller than search key */
                        ret = cursor->get_key(cursor, &key);
                        break;
                case 0:         /* Exact match found */
                        break;
                case 1:         /* Returned key larger than search key */
                        ret = cursor->get_key(cursor, &key);
                        break;
                }

                ret = cursor->get_value(cursor, &value);
        }

        return (ret);
}

Cursor positions do not survive transactions: cursors that are open during WT_SESSION::begin_transaction, WT_SESSION::commit_transaction or WT_SESSION::rollback_transaction will lose their position as if WT_CURSOR::reset was called.

Cursors can be configured to move to a random position with WT_CURSOR::next is called, see Cursor random for details.

Inserting and updating

To insert new data, and optionally update existing data, using a cursor, use the WT_CURSOR::insert method:

int
cursor_insert(WT_CURSOR *cursor)
{
        cursor->set_key(cursor, "foo");
        cursor->set_value(cursor, "bar");

        return (cursor->insert(cursor));
}

To update existing data using a cursor, use the WT_CURSOR::update method:

int
cursor_update(WT_CURSOR *cursor)
{
        cursor->set_key(cursor, "foo");
        cursor->set_value(cursor, "newbar");

        return (cursor->update(cursor));
}

To remove existing data using a cursor, use the WT_CURSOR::remove method:

int
cursor_remove(WT_CURSOR *cursor)
{
        cursor->set_key(cursor, "foo");
        return (cursor->remove(cursor));
}

The WT_SESSION::open_cursor overwrite configuration is true by default, causing WT_CURSOR::insert, WT_CURSOR::remove and WT_CURSOR::update to ignore the current state of the record, and these methods will succeed regardless of whether or not the record previously exists.

When an application configures overwrite to false, WT_CURSOR::insert will fail with WT_DUPLICATE_KEY if the record previously exists, and WT_CURSOR::update and WT_CURSOR::remove will fail with WT_NOTFOUND if the record does not previously exist.

Cursor position after error

After any cursor handle method failure, the cursor's position is undetermined. For cursor operations that expect a key to be set before the operation begins (including WT_CURSOR::search, WT_CURSOR::insert, WT_CURSOR::update and WT_CURSOR::remove), the application's key and value will not be cleared by an error.

Applications that cannot re-position the cursor after failure must duplicate the cursor by calling WT_SESSION::open_cursor and passing the cursor as the to_dup parameter before calling a cursor method that will attempt to re-position the cursor. Cursor duplication is not supported for the backup, config and statistics cursor types.

Cursor key/value memory scoping

When applications pass pointers to WT_CURSOR::set_key or WT_CURSOR::set_value, which can be a WT_ITEM or a string, the application is required to keep the memory valid until the next operation that successfully positions the cursor. These operations are WT_CURSOR::remove, WT_CURSOR::search, WT_CURSOR::search_near and WT_CURSOR::update, but do not include WT_CURSOR::insert, as it does not position the cursor.

If such an operation fails (for example, due to a WT_DEADLOCK error), it may be retried without calling WT_CURSOR::set_key or WT_CURSOR::set_value again. That is, the cursor may still reference the application-supplied memory until it is successfully positioned.

Any pointers returned by WT_CURSOR::get_key or WT_CURSOR::get_value are only valid until the cursor is positioned. These pointers may reference private WiredTiger data structures that must not be modified or freed by the application. If a longer scope is required, the application must make a copy of the memory before the cursor is positioned.

The comments in this example code explain when the application can safely modify memory passed to WT_CURSOR::set_key or WT_CURSOR::set_value:

                (void)snprintf(keybuf, sizeof(keybuf), "%s", op->key);
                cursor->set_key(cursor, keybuf);
                (void)snprintf(valuebuf, sizeof(valuebuf), "%s", op->value);
                cursor->set_value(cursor, valuebuf);

                /*
                 * The application must keep the key and value memory valid
                 * until the next operation that positions the cursor.
                 * Modifying either the key or value buffers is not permitted.
                 */

                /* Apply the operation (insert, update, search or remove). */
                if ((ret = op->apply(cursor)) != 0) {
                        fprintf(stderr, "Error performing the operation: %s\n",
                            wiredtiger_strerror(ret));
                        return (ret);
                }

                /*
                 * Except for WT_CURSOR::insert, the cursor has been positioned
                 * and no longer references application memory, so application
                 * buffers can be safely overwritten.
                 */
                if (op->apply != cursor->insert) {
                        strcpy(keybuf, "no key");
                        strcpy(valuebuf, "no value");
                }

                /*
                 * Check that get_key/value behave as expected after the
                 * operation.
                 */
                if ((ret = cursor->get_key(cursor, &key)) != 0 ||
                    (op->apply != cursor->remove &&
                    (ret = cursor->get_value(cursor, &value)) != 0)) {
                        fprintf(stderr, "Error in get_key/value: %s\n",
                            wiredtiger_strerror(ret));
                        return (ret);
                }

                /*
                 * Except for WT_CURSOR::insert (which does not position the
                 * cursor), the application now has pointers to memory owned
                 * by the cursor.  Modifying the memory referenced by either
                 * key or value is not permitted.
                 */

                /* Check that the cursor's key and value are what we expect. */
                if (op->apply != cursor->insert)
                        if (key == keybuf ||
                            (op->apply != cursor->remove &&
                            value == valuebuf)) {
                                fprintf(stderr,
                                    "Cursor points at application memory!\n");
                                return (EINVAL);
                        }

                if (strcmp(key, op->key) != 0 ||
                    (op->apply != cursor->remove &&
                    strcmp(value, op->value) != 0)) {
                        fprintf(stderr, "Unexpected key / value!\n");
                        return (EINVAL);
                }