ConcatenateGraphFilter
class ConcatenateGraphFilter(inputTokenStream: TokenStream, tokenSeparator: Char?, preservePositionIncrements: Boolean, maxGraphExpansions: Int) : TokenStream
Concatenates/Joins every incoming token with a separator into one output token for every path through the token stream (which is a graph). In simple cases this yields one token, but in the presence of any tokens with a zero positionIncrmeent (e.g. synonyms) it will be more. This filter uses the token bytes, position increment, and position length of the incoming stream. Other attributes are not used or manipulated.
Constructors
Link copied to clipboard
constructor(inputTokenStream: TokenStream, tokenSeparator: Char?, preservePositionIncrements: Boolean, maxGraphExpansions: Int)
Creates a token stream to convert input to a token stream of accepted strings by its token stream graph.
constructor(inputTokenStream: TokenStream, preserveSep: Boolean, preservePositionIncrements: Boolean, maxGraphExpansions: Int)
Calls ConcatenateGraphFilter
Types
Functions
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Converts the tokenStream to an automaton, treating the transition labels as utf-8. Does not close it.
Converts the tokenStream to an automaton. Does not close it.