TieredMergePolicy
Merges segments of approximately equal size, subject to an allowed number of segments per tier. This is similar to LogByteSizeMergePolicy, except this merge policy is able to merge non-adjacent segment. This merge policy also does not over-merge (i.e. cascade merges).
For normal merging, this policy first computes a "budget" of how many segments are allowed to be in the index. If the index is over-budget, then the policy sorts segments by decreasing size (pro-rating by percent deletes), and then finds the least-cost merge. Merge cost is measured by a combination of the "skew" of the merge (size of largest segment divided by smallest segment), total merge size and percent deletes reclaimed, so that merges with lower skew, smaller size and those reclaiming more deletes, are favored.
If a merge will produce a segment that's larger than .setMaxMergedSegmentMB, then the policy will merge fewer segments (down to 1 at once, if that one has deletions) to keep the segment size under budget.
NOTE: this policy freely merges non-adjacent segments; if this is a problem, use [ ].
NOTE: This policy always merges by byte size of the segments, always pro-rates by percent deletes
NOTE Starting with Lucene 7.5, if you call IndexWriter.forceMerge with this (default) merge policy, if .setMaxMergedSegmentMB is in conflict with maxNumSegments passed to IndexWriter.forceMerge then maxNumSegments wins. For example, if your index has 50 1 GB segments, and you have .setMaxMergedSegmentMB at 1024 (1 GB), and you call forceMerge(10), the two settings are clearly in conflict. TieredMergePolicy will choose to break the .setMaxMergedSegmentMB constraint and try to merge down to at most ten segments, each up to 5 * 1.25 GB in size (since an extra 25% buffer increase in the expected segment size is targeted).
findForcedDeletesMerges should never produce segments greater than maxSegmentSize.
NOTE: This policy returns natural merges whose size is below the .setFloorSegmentMB for .findFullFlushMerges.
Properties
Returns the current deletesPctAllowed setting.
Returns the current forceMergeDeletesPctAllowed setting.
Returns the current segmentsPerTier setting.
Returns the target search concurrency.
Functions
Determine what set of merge operations is necessary in order to expunge all deletes from the index.
Determine what set of merge operations is necessary in order to merge to <= the specified segment count. IndexWriter calls this when its IndexWriter.forceMerge method is called. This call is always synchronized on the IndexWriter instance so only one thread at a time will call this method.
Identifies merges that we want to execute (synchronously) on commit. By default, this will return .findMerges whose segments are all less than the .maxFullFlushMergeSize.
Determine what set of merge operations are now necessary on the index. IndexWriter calls this whenever there is a change to the segments. This call is always synchronized on the IndexWriter instance so only one thread at a time will call this method.
Returns true if the segment represented by the given CodecReader should be kept even if it's fully deleted. This is useful for testing of for instance if the merge policy implements retention policies for soft deletes.
Return the maximum size of segments to be included in full-flush merges by the default implementation of .findFullFlushMerges.
Returns the number of deletes that a merge would claim on the given segment. This method will by default return the sum of the del count on disk and the pending delete count. Yet, subclasses that wrap merge readers might modify this to reflect deletes that are carried over to the target segment in the case of soft deletes.
Sets the maximum percentage of doc id space taken by deleted docs. The denominator includes both active and deleted documents. Lower values make the index more space efficient at the expense of increased CPU and I/O activity. Values must be between 5 and 50. Default value is
Segments smaller than this size are merged more aggressively:
When forceMergeDeletes is called, we only merge away a segment if its delete percentage is over this threshold. Default is 10%.
Maximum sized segment to produce during normal merging. This setting is approximate: the estimate of the merged segment size is made by summing sizes of to-be-merged segments (compensating for percent deleted docs). Default is 5 GB.
Sets the allowed number of segments per tier. Smaller values mean more merging but fewer segments.
Sets the target search concurrency. This prevents creating segments that are bigger than maxDoc/targetSearchConcurrency, which in turn makes the work parallelizable into targetSearchConcurrency slices of similar doc counts. It also makes merging less aggressive, as higher values result in indices that do less merging and have more segments
Return the byte size of the provided SegmentCommitInfo, prorated by percentage of non-deleted documents.
Returns true if a new segment (regardless of its origin) should use the compound file format. The default implementation returns true iff the size of the given mergedInfo is less or equal to .getMaxCFSSegmentSizeMB and the size is less or equal to the TotalIndexSize * .getNoCFSRatio otherwise false.