LimitTokenCountAnalyzer

constructor(delegate: Analyzer, maxTokenCount: Int)

Build an analyzer that limits the maximum number of tokens per field. This analyzer will not consume any tokens beyond the maxTokenCount limit

See also


constructor(delegate: Analyzer, maxTokenCount: Int, consumeAllTokens: Boolean)

Build an analyzer that limits the maximum number of tokens per field.

Parameters

delegate

the analyzer to wrap

maxTokenCount

max number of tokens to produce

consumeAllTokens

whether all tokens from the delegate should be consumed even if maxTokenCount is reached.