common/org.gnit.lucenekmp.analysis.email/UAX29URLEmailAnalyzer

UAX29URLEmailAnalyzer

class UAX29URLEmailAnalyzer : StopwordAnalyzerBase

Filters UAX29URLEmailTokenizer with org.gnit.lucenekmp.analysis.LowerCaseFilter and org.gnit.lucenekmp.analysis.StopFilter, using a list of English stop words.

Since

3.6.0

Constructors

UAX29URLEmailAnalyzer

constructor(stopWords: CharArraySet)

Builds an analyzer with the given stop words.

constructor()

Builds an analyzer with the default stop words (STOP_WORDS_SET).

constructor(stopwords: Reader)

Builds an analyzer with the stop words from the given reader.

Types

object Companion

Properties

val reuseStrategy: Analyzer.ReuseStrategy

val stopwords: CharArraySet

var storedValue: CloseableThreadLocal<Any>?

Functions

open override fun close()

getMaxTokenLength

fun getMaxTokenLength(): Int

open fun getOffsetGap(fieldName: String?): Int

getPositionIncrementGap

open fun getPositionIncrementGap(fieldName: String?): Int

fun normalize(fieldName: String, text: String): BytesRef

setMaxTokenLength

fun setMaxTokenLength(length: Int)

Set the max allowed token length. Tokens larger than this will be chopped up at this token length and emitted as multiple tokens. If you need to skip such large tokens, you could increase this max length, and then use LengthFilter to remove long tokens. The default is UAX29URLEmailAnalyzer.DEFAULT_MAX_TOKEN_LENGTH.

fun tokenStream(fieldName: String, text: String): TokenStream

fun tokenStream(fieldName: String, reader: Reader): TokenStream