ConcatenateGraphFilter

class ConcatenateGraphFilter(inputTokenStream: TokenStream, tokenSeparator: Char?, preservePositionIncrements: Boolean, maxGraphExpansions: Int) : TokenStream

Concatenates/Joins every incoming token with a separator into one output token for every path through the token stream (which is a graph). In simple cases this yields one token, but in the presence of any tokens with a zero positionIncrmeent (e.g. synonyms) it will be more. This filter uses the token bytes, position increment, and position length of the incoming stream. Other attributes are not used or manipulated.

Constructors

Link copied to clipboard
constructor(inputTokenStream: TokenStream, tokenSeparator: Char?, preservePositionIncrements: Boolean, maxGraphExpansions: Int)
constructor(inputTokenStream: TokenStream)

Creates a token stream to convert input to a token stream of accepted strings by its token stream graph.

constructor(inputTokenStream: TokenStream, preserveSep: Boolean, preservePositionIncrements: Boolean, maxGraphExpansions: Int)

Types

Link copied to clipboard

Attribute providing access to the term builder and UTF-16 conversion

Link copied to clipboard
object Companion

Properties

Functions

Link copied to clipboard
fun <T : Attribute> addAttribute(attClass: KClass<T>): T
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
open override fun close()
Link copied to clipboard
fun copyTo(target: AttributeSource)
Link copied to clipboard
open override fun end()
Link copied to clipboard
Link copied to clipboard
open operator override fun equals(obj: Any?): Boolean
Link copied to clipboard
fun <T : Attribute> getAttribute(attClass: KClass<T>): T?
Link copied to clipboard
fun hasAttribute(attClass: KClass<out Attribute>): Boolean
Link copied to clipboard
Link copied to clipboard
open override fun hashCode(): Int
Link copied to clipboard
open override fun incrementToken(): Boolean
Link copied to clipboard
fun reflectAsString(prependAttClass: Boolean): String
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
open override fun reset()
Link copied to clipboard
Link copied to clipboard

Converts the tokenStream to an automaton, treating the transition labels as utf-8. Does not close it.

fun toAutomaton(unicodeAware: Boolean): Automaton

Converts the tokenStream to an automaton. Does not close it.

Link copied to clipboard
open override fun toString(): String