common/org.gnit.lucenekmp.analysis.cjk/CJKAnalyzer

CJKAnalyzer

class CJKAnalyzer : StopwordAnalyzerBase

An org.gnit.lucenekmp.analysis.Analyzer that tokenizes text with StandardTokenizer, normalizes content with CJKWidthFilter, folds case with LowerCaseFilter, forms bigrams of CJK with CJKBigramFilter, and filters stopwords with StopFilter

Since

3.1

Constructors

constructor()

Builds an analyzer which removes words in getDefaultStopSet.

constructor(stopwords: CharArraySet)

Builds an analyzer with the given stop words

Types

object Companion

Properties

val reuseStrategy: Analyzer.ReuseStrategy

val stopwords: CharArraySet

var storedValue: CloseableThreadLocal<Any>?

Functions

open override fun close()

open fun getOffsetGap(fieldName: String?): Int

getPositionIncrementGap

open fun getPositionIncrementGap(fieldName: String?): Int

fun normalize(fieldName: String, text: String): BytesRef

fun tokenStream(fieldName: String, text: String): TokenStream

fun tokenStream(fieldName: String, reader: Reader): TokenStream