common/org.gnit.lucenekmp.analysis.miscellaneous/LimitTokenCountFilter

LimitTokenCountFilter

class LimitTokenCountFilter : TokenFilter

This TokenFilter limits the number of tokens while indexing. It is a replacement for the maximum field length setting inside org.gnit.lucenekmp.index.IndexWriter.

By default, this filter ignores any tokens in the wrapped `TokenStream` once the limit has been reached, which can result in `reset()` being called prior to `incrementToken()` returning `false`. For most `TokenStream` implementations this should be acceptable, and faster then consuming the full stream. If you are wrapping a `TokenStream` which requires that the full stream of tokens be exhausted in order to function properly, use the [LimitTokenCountFilter] `consumeAllTokens` option.

Constructors

LimitTokenCountFilter

constructor(in: TokenStream, maxTokenCount: Int)

Build a filter that only accepts tokens up to a maximum number. This filter will not consume any tokens beyond the maxTokenCount limit

constructor(in: TokenStream, maxTokenCount: Int, consumeAllTokens: Boolean)

Build an filter that limits the maximum number of tokens per field.

Properties

attributeClassesIterator

val attributeClassesIterator: Iterator<Any>

attributeFactory

val attributeFactory: AttributeFactory

attributeImplsIterator

val attributeImplsIterator: Iterator<AttributeImpl>

Functions

fun <T : Attribute> addAttribute(attClass: KClass<T>): T

addAttributeImpl

fun addAttributeImpl(att: AttributeImpl)

fun captureState(): AttributeSource.State?

clearAttributes

fun clearAttributes()

cloneAttributes

fun cloneAttributes(): AttributeSource

open override fun close()

fun copyTo(target: AttributeSource)

open override fun end()

fun endAttributes()

open operator override fun equals(obj: Any?): Boolean

fun <T : Attribute> getAttribute(attClass: KClass<T>): T?

fun hasAttribute(attClass: KClass<out Attribute>): Boolean

fun hasAttributes(): Boolean

open override fun hashCode(): Int

open override fun incrementToken(): Boolean

reflectAsString

fun reflectAsString(prependAttClass: Boolean): String

fun reflectWith(reflector: AttributeReflector)

removeAllAttributes

fun removeAllAttributes()

open override fun reset()

fun restoreState(state: AttributeSource.State?)

open override fun toString(): String

open override fun unwrap(): TokenStream