ConcurrentMergeScheduler
A MergeScheduler that runs each merge using a separate thread.
Specify the max number of threads that may run at once, and the maximum number of simultaneous merges with .setMaxMergesAndThreads.
If the number of merges exceeds the max number of threads then the largest merges are paused until one of the smaller merges completes.
If more than .getMaxMergeCount merges are requested then this class will forcefully throttle the incoming threads by pausing until one more merges complete.
This class sets defaults based on Java's view of the cpu count, and it assumes a solid state disk (or similar). If you have a spinning disk and want to maximize performance, use .setDefaultMaxMergesAndThreads.
Types
This executor provides intra-merge threads for parallel execution of merge tasks. It provides a limited number of threads to execute merge tasks. In particular, if the number of mergeThreads is equal to maxThreadCount, then the executor will execute the merge task in the calling thread.
Runs a merge thread to execute a single merge, then exits.
Properties
Functions
Used for testing
Turn off auto IO throttling.
Turn on dynamic IO throttling, to adaptively rate limit writes bytes/sec to the minimal rate necessary so merges do not fall behind. By default this is disabled and writes are not rate-limited.
Get the per-merge IO throttle rate for forced merges.
Provides an executor for parallelism during a single merge operation. By default, the method returns a SameThreadExecutorService where all intra-merge actions occur in their calling thread.
IndexWriter calls this on init.
Run the merges provided by MergeSource.getNextMerge.
Returns the number of merge threads that are alive, ignoring the calling thread if it is a merge thread. Note that this number is .mergeThreads size.
Removes the calling thread from the active merge threads.
Sets max merges and threads to proper defaults for rotational or non-rotational storage.
Set the per-merge IO throttle rate for forced merges (default: Double.POSITIVE_INFINITY).
Expert: directly set the maximum number of merge threads and simultaneous merges allowed.
Used for testing
Wraps the incoming Directory so that we can merge-throttle it using [ ].