In previous WiredTiger releases, there were several cases where configuration strings were treated in a case-sensitive manner (for example, it was possible to specify "True", "true" or even "tRuE" as a boolean value). For consistency, in this release all WiredTiger configuration strings are case-sensitive, and only "true" will be accepted.
In previous releases of WiredTiger, a statistics cursor made a snapshot of the relevant statistics before the first statistics value was read, and those values remained unchanged from that point on, no matter how the cursor was used. In the WiredTiger 2.5.2 release, calling the WT_CURSOR::reset method to reset the cursor refreshes the statistics returned by the cursor.
There are no special upgrade steps required.
Opening a statistics cursor on a data source without a checkpoint
name no longer aggregates statistics for all open checkpoints on that data source. The returned statistics will only be for the in-memory version of the data source.
For applications that read and write from ordinary tables (without specifying a checkpoint
), there will be no change. Applications that opened cursors on checkpoints and relied on their statistics being aggregated into a single statistics cursor will need to open statistics cursors on each checkpoint.
The statistic for the number of bytes written to the log minus the overhead of log record headers and padding was renamed to make the intent clearer. Any applications using the old name will need to be updated.
Collators, compressors and extractors can now be disabled with an explicit "none"
value as an alternative to using an empty string. Any applications using the name "none"
for a collator, compressor or extractor will need to be updated.
internal_item_max
and leaf_item_max
configuration strings are now deprecated in favor of the internal_key_max
, leaf_key_max
, and leaf_value_max
configuration strings. See Page and overflow key/value sizes for more information. WT_DEADLOCK
error return has been deprecated in favor of WT_ROLLBACK to clarify that WT_SESSION::rollback_transaction should be called; no program changes are required. In previous versions of WiredTiger, all cursors in a session were reset at transaction boundaries (WT_SESSION::begin_transaction, WT_SESSION::commit_transaction and WT_SESSION::rollback_transaction). Now they are only reset by WT_SESSION::rollback_transaction.
This change means that a cursor can be used to iterate through a table and perform transactional updates based on the visited records without the cursor losing its position.
Applications relying on this behavior (for example, to avoid pinning the underlying cursor resources), should reset cursors explicitly when the position is no longer required.
NULL
. lsm_manager
option group. To disable merges in all LSM trees pass lsm_manger=
(merge=false) to wiredtiger_open. –with-spinlock=pthread_adaptive
at configuration time. lsm=
(merge_threads) configuration option has been replaced by the W::wiredtiger_open lsm_manager=
(worker_thread_max) option. The new version specifies a set of LSM threads that are shared across all LSM trees in a database, the older configuration was per LSM table. The eviction_workers
configuration setting has been replaced by eviction=
(threads_min) and eviction=
(threads_max) settings.
There is also a semantic change because eviction_workers
used to configure additional threads whereas the new settings configure the total number of threads involved with eviction.
There are no special upgrade steps required.
In the 2.2.1 release, the order that configuration strings are parsed and override earlier values changed. Applications using the Wiredtiger.config
file or WIREDTIGER_CONFIG
environment variable may need to change. The old order:
Wiredtiger.config
WIREDTIGER_CONFIG
In the new order the user's configuration settings override:
Wiredtiger.config
WIREDTIGER_CONFIG
os_cache_dirty_max
off for LSM In some earlier versions of WiredTiger, creating an LSM table automatically configured os_cache_dirty_max
, causing additional system calls that slowed some workloads. Applications that benefit from this setting should set it explicitly in WT_SESSION::create.
transaction_sync
setting change transaction_sync
configuration setting has changed from a string value to a structure with two fields: method
, which corresponds to the previous value (dsync
, fsync
or none
), and enabled
, which determines whether there is a sync on every transaction commit by default. Applications can enable or disable sync for a specific transaction with the sync
setting to WT_SESSION::begin_transaction. In the 2.2.0 release, prefix compression default to false
. Applications that benefit from prefix compression will need to explicitly set prefix_compression=true
when creating tables.
–enable-verbose
in the configure command to be able to use verbose messages. shared_cache
configuration option group have changed. The option that was named enable
is no longer available. To enable a shared_cache
it is compulsory to name the pool being shared. We are now also enforcing that only one of cache_size
and shared_cache
are specified in the wiredtiger_open configuration string. In the 2.1 release of WiredTiger WT_ITEM::size type has changed from uint32_t
to size_t
. Applications may require modifications to resolve compile-time errors.
In the 2.1 release of WiredTiger, the behavior of the compress_raw callback has changed so that it will only be retried if it returns EAGAIN
. If it returns zero and sets result_slots
to zero, WiredTiger will assume that raw compression has failed and will fall back to calling WT_COMPRESSOR::compress.
transaction_sync
configuration setting default value has changed from "dsync" to "fsync". This is due to enhancements to the group commit implementation in WiredTiger - which mean that greater throughput can be achieved with explicit "fsync" calls than by enabling "dsync" on a file handle. Applications that don't execute concurrent transactions may see better throughput with transaction_sync set to "dsync". The underlying file format is unchanged in 2.0
lsm_
now belong to the lsm
configuration group. If you are explicitly configuring any of the following options, you should review the WT_SESSION::create documentation for details of the updated syntax: lsm_auto_throttle, lsm_bloom, lsm_bloom_config, lsm_bloom_bit_count, lsm_bloom_hash_count, lsm_bloom_oldest, lsm_chunk_max, lsm_chunk_size, lsm_merge_max and lsm_merge_threads. The underlying file formats changed in the 1.6.6 release; tables and files should be dumped and re-loaded into a new database.
In previous releases, the trigger
configuration string to the WT_SESSION::compact method specified a requirement to initiate compaction; in the 1.6.6 release, this configuration string has been removed, and compaction will be attempted if it seems likely at least 10% of the file can be recovered. Applications may require modifications to resolve run-time errors.
In previous releases, the wiredtiger_open function took a statistics
configuration, which defaulted to false; when set to true, additional, generally performance-expensive statistics were maintained by the database, above and beyond a default set of statistics. In version 1.6.6, the statistics
configuration is a list which may be set to "all", "fast" or "none". When set to "none", no statistics are maintained by the database; when set to "fast", only relatively performance-inexpensive statistics are maintained, and when set to "all", all statistics are maintained, regardless of cost.
In previous releases, the wiredtiger_open function took a statistics_log
configuration which logged the performance-inexpensive database statistics to a file. In version 1.6.6, the statistics_log
configuration logs whatever statistics are configured for the database. If the database is configured with statistics
to "none", no statistics will be logged to the file; if the database is configured with "all" or "fast", the corresponding statistics will be logged to the file.
In previous releases, the WT_SESSION::cursor method took statistics_clear
and a statistics_fast
configurations. The statistics_clear
configuration defaulted to false; when set to true, statistics counters were reset after they were gathered by the cursor. The statistics_fast
configuration defaulted to true; when set to true, the cursor only gathered performance-inexpensive statistics for the cursor, and when set to false, the cursor gathered all available statistics, regardless of cost.
In version 1.6.6, these two configuration booleans have been replaced with a new configuration list statistics
, which may be set from the values "clear", "fast" and "all". When "fast" is configured, only relatively performance-inexpensive statistics are gathered, and when "all" is configured, all statistics are gathered, regardless of cost. When "clear" is configured, statistics counters are reset after they are gathered.
Additionally, in version 1.6.6, statistics cursors must be configured to agree with the database statistics configuration; when the database statistics are configured to "none", attempts to open a statistics cursor will fail; when the database statistics are configured to "fast", a statistics cursor must also be configured to "fast"; when the database statistics are configured to "all", a statistics cursor may be configured to either "fast" or "all". Opening a statistics cursor without configuring either "fast" or "all" will configure the cursor to be the same as the current database configuration.
Applications may require modifications to resolve run-time errors; application statistics configuration and cursors should be reviewed to confirm they are configured for the desired behavior;
Add a new WT_EVENT_HANDLER::handle_close callback that WiredTiger will call any time it automatically closes an application session or cursor handle.
Additionally add a WT_SESSION parameter into the existing WT_EVENT_HANDLER::handle_error, WT_EVENT_HANDLER::handle_message and WT_EVENT_HANDLER::handle_progress callback functions.
In previous releases, the WT_CURSOR::insert ended positioned at the inserted record. To minimize the cursor resources held by applications inserting many records, the WT_CURSOR::insert method has been changed to end without any position. Application insert cursors should be reviewed to confirm they do not attempt to iterate after an insert.
In previous releases, the default statistics_fast
configuration to the WT_SESSION::open_cursor method was false
; in the 1.6.5 release, the default statistics_fast configuration is true
. Applications opening statistics cursors should be reviewed to confirm they have the correct behavior.
sync
configuration key to wiredtiger_open has been renamed checkpoint_sync
. The underlying file formats changed in the 1.6.4 release; tables and files should be dumped and re-loaded into a new database.
wt
utility's load
command has been changed to overwrite existing data, by default, and the -o
flag to the load
command (overwrite existing data) has been replaced with the -n
flag (do not overwrite existing data). Applications requiring the previous default behavior of not overwriting existing data should add the -n
option to their command line configuration; applications previously using the -o
option on their command line configurations should remove it. In previous releases, the WT_SESSION::open_cursor overwrite
configuration string behaved inconsistently across Btree and LSM data sources. In Btree, overwrite
was false
by default and was limited to the WT_CURSOR::insert method, changing an insert to succeed regardless of whether or not the record previously existed. In LSM trees, overwrite
was true
by default, and applied to the WT_CURSOR::insert, WT_CURSOR::remove and WT_CURSOR::update methods, configuring all three methods to ignore the existing state of the record.
In the 1.6.3 release, the overwrite
configuration is consistent across both Btree and LSM tree data sources. For performance reasons, the default is the behavior previously described for LSM trees: in other words, overwrite
is true
by default, causing WT_CURSOR::insert, WT_CURSOR::remove and WT_CURSOR::update to ignore the current state of the record, and these methods will succeed regardless of whether or not the record previously exists. When an application configures overwrite
to false
, WT_CURSOR::insert will fail with WT_DUPLICATE_KEY if the record previously exists, and WT_CURSOR::update and WT_CURSOR::remove will fail with WT_NOTFOUND if the record does not previously exist.
This is a potentially serious API change that will not be detected by compilation. Application cursors should be reviewed to confirm they are configured for the desired behavior.
transactional
configuration transactional
configuration key has been removed from wiredtiger_open. Any application setting it should simply remove it, no change in application behavior is needed. New functionality was added to the list of WiredTiger extension methods; applications using the extension methods will require recompilation.
The "source"
configuration key has been removed from WT_SESSION::create. Normal applications should not have been using it, and there were a number of bugs associated with it.
uncompressed
, which means blocks that are compressed will no longer also include a checksum, by default. Applications using compression insufficient for the purposes of corrupted block identification should change their file checksum configuration to on
. In the 1.6.1 release, the default for the WT_SESSION::create configuration string allocation_size
changed from 512B to 4KB, and the default for the configuration string internal_page_max
changed from 2KB to 4KB. Applications wanting to create files with smaller allocation or internal page sizes will need to set those configuration values explicitly.
In the 1.6.1 release, an explicit shared_cache=(enable=boolean) option was added to the wiredtiger_open configuration options. Existing applications that use shared cache functionality will need to add the enable option to the configuration string. The default value for the option is false.
split_pct
argument to the WT_COMPRESSOR::compress_raw function changed type from u_int
to int
, applications may require modification to avoid compiler warnings. An undocumented feature where configuration string case was ignored has been removed, and all configuration strings are now case-dependent. Applications may require modifications to resolve run-time errors.
The following changes are only applicable to applications loading extensions and/or using the WiredTiger extension functions described in WT_EXTENSION_API.
The signature of wiredtiger_extension_init has changed from (WT_SESSION *session, WT_EXTENSION_API *api)
to (WT_CONNECTION *connection)
. As no WT_EXTENSION_API handle reference is passed to the function, the WT_CONNECTION::get_extension_api has been added to support retrieval of the extension API. Applications may require modifications.
The type of all configuration arguments to extension methods has changed from "const char *"
to "WT_CONFIG_ARG *"
, and the WT_EXTENSION::config method added to support configuration parsing; applications may require modifications.
The undocumented wiredtiger_XXX
defines for WT_EXTENSION_API extension methods have been removed from the wiredtiger_ext.h
include file; applications should instead use the method handles referenced by the WT_EXTENSION_API handle to call extension functions.
The extension API methods have all changed to require an additional parameter, the WT_EXTENSION_API method handle; applications may require modifications.
The following changes are only applicable to applications providing new implementations of the WiredTiger WT_DATA_SOURCE class.
The WT_DATA_SOURCE class has three new methods: WT_DATA_SOURCE::compact, WT_DATA_SOURCE::salvage, and WT_DATA_SOURCE::verify; applications may require modifications to resolve compile errors.
The owner
argument to the WT_DATA_SOURCE::open_cursor method has been removed; applications may require modifications to resolve compile errors.
exclusive
argument to the WT_DATA_SOURCE::create method has been removed; applications may require modifications to resolve compile errors. statistics
configuration string to the wiredtiger_open function. A new member, WT_COMPRESSOR::compress_raw, was added to the WT_COMPRESSOR extension API. Applications using the WT_COMPRESSOR extension API should add a NULL as the second field of that structure.
The WT_SESSION::create method's checksum
configuration string has been changed from a boolean type to a string type. Applications using the checksum configuration string should change a value of true
to the string on
, and a value of false
to the string off
or the string uncompressed
.
The installed WiredTiger extension library names changed to limit namespace pollution:
Library | Previous Name | New Name |
---|---|---|
Bzip2 compression | bzip2_compress.a | libwiredtiger_bzip2.a |
bzip2_compress.la | libwiredtiger_bzip2.la | |
bzip2_compress.so | libwiredtiger_bzip2.so | |
Snappy compression | snappy_compress.a | libwiredtiger_snappy.a |
snappy_compress.la | libwiredtiger_snappy.la | |
snappy_compress.so | libwiredtiger_snappy.so | |
No-op compression | nop_compress.a | No longer installed |
nop_compress.la | No longer installed | |
nop_compress.so | No longer installed | |
Reverse order collator | reverse_collator.a | No longer installed |
reverse_collator.la | No longer installed | |
reverse_collator.so | No longer installed |
The built-in compression name arguments to the WT_SESSION:create block_compressor
configuration string changed for consistency:
Extension | Previous Name | New Name |
---|---|---|
Bzip2 compression | "bzip2_compress" | "bzip2" |
Snappy compression | "snappy_compress" | "snappy" |
The checkpoint functionality supported by WT_SESSION::checkpoint and the snapshot functionality supported by WT_SESSION::sync have been merged into a single piece of functionality.
WT_SESSION.checkpoint
The WT_SESSION::checkpoint method's snapshot
configuration string has been renamed to name
. The name assigned to checkpoints without a specified name
configuration is now "WiredTigerCheckpoint"
.
WT_SESSION.drop
In releases before 1.3, the WT_SESSION::drop method was used to delete snapshots. In 1.3, the functionality of deleting snapshots has been moved to the WT_SESSION::checkpoint method, specifically, snapshots are discarded using the WT_SESSION::checkpoint method's drop
configuration string.
WT_SESSION.sync
The WT_SESSION::sync method has been removed from the 1.3 release; the functionality of creating an object snapshot has moved to the WT_SESSION::checkpoint method, specifically, creating a snapshot of a one or more objects is done using the WT_SESSION::checkpoint method's target
configuration string.
wt drop -s
The -s
option to the drop
command for the wt
command line utility has been removed, and object snapshots may no longer be removed from the command line.
-s
options to the dump
and list
commands for the wt
command line utility have been renamed to be -c
. In releases before 1.3, the WT_SESSION::open_cursor method could duplicate cursors that were not positioned in an object; in 1.3, a cursor must be positioned in order to be duplicated.
In releases before 1.3, ending a transaction by calling the WT_SESSION::commit_transaction or WT_SESSION::rollback_transaction methods implicitly closed all open cursors; in 1.3, the cursors remain open, but are reset (discarding their positions and cursor values). This means applications must change to either close cursors explicitly, or rely on an eventual WT_SESSION::close or WT_CONNECTION::close methods to implicitly close open cursors.
In releases before 1.3, the default isolation level for transaction was snapshot
, and the default isolation level for non-transaction operations was read-uncommitted
; in 1.3, the default isolation level for all operations is read-committed
.
The default can be overridden for a session using the isolation
setting in WT_CONNECTION::open_cursor.
In releases before 1.3, the WT_SESSION::truncate method required cursors used for truncation of a cursor range to reference existing keys in the object; in 1.3, the WT_SESSION::truncate method has been changed to allow cursors to reference any valid key in the object's name space so applications may discard portions of the object name space without knowing exactly what records the object contains.
In releases before 1.3, the WT_CURSOR::equals method returned zero/non-zero to indicate cursor equality; in 1.3, the WT_CURSOR::equals method has been replaced with WT_CURSOR::compare, which compares two cursors and returns a cursor comparison status (less than 0, equal to 0, or greater than 0) depending on the cursors' key order.