Capacity tuning

In some cases, it can be helpful to constrain the overall I/O bandwidth generated by the database. This can be beneficial when resources are shared, for example, in cloud or virtual environments.

The total bandwidth capacity is configured by setting the io_capacity configuration string when calling the wiredtiger_open function. The capacity can be adjusted with WT_CONNECTION::reconfigure.

An example of setting a capacity limit to 40MB per second:

    error_check(wiredtiger_open(home, NULL, "create,io_capacity=(total=40MB)", &conn));

When a total capacity is set the volume of system reads and writes totaled will not exceed the given I/O capacity. If a read or write is scheduled and would overflow the capacity, the issuing thread will sleep to guarantee the capacity ceiling. The policy used is fair to all threads, and gives some weight to both readers and writers to try to ensure that each session can make progress when bandwidth resources are limited.

System reads and writes do not directly translate to disk I/O operations. These operations go through the operating system cache. To ensure the steady flow of data to the disk, setting a capacity also enables an additional thread that monitors the writes performed for each file. For each file that has sufficient data written to it, a call to an asynchronous fsync will be made. This call normally queues the flush in the operating system, though there is no guarantee about when it will actually occur. On Windows, there is no equivalent support for asynchronously scheduling writes to disk, so this extra "sync" thread is not active.

When a total capacity is not set, or equivalently, when it is set to 0, there are no capacity constraints on the database, and pauses will never be inserted before I/O is done, nor are extra asynchronous fsync calls performed.