common/org.gnit.lucenekmp.analysis.core/WhitespaceTokenizerFactory

WhitespaceTokenizerFactory

class WhitespaceTokenizerFactory : TokenizerFactory

Factory for WhitespaceTokenizer.

<fieldType name="text_ws" class="solr.TextField" positionIncrementGap="100">
<analyzer>
     <tokenizer class="solr.WhitespaceTokenizerFactory" rule="unicode"  maxTokenLen="256"/>
</analyzer>
</fieldType>

Options:

rule: either "java" for [WhitespaceTokenizer] or "unicode" for [UnicodeWhitespaceTokenizer]
maxTokenLen: max token length, should be greater than 0 and less than MAX_TOKEN_LENGTH_LIMIT (1024*1024). It is rare to need to change this else [ CharTokenizer]::DEFAULT_MAX_TOKEN_LEN

Since

3.1

Constructors

WhitespaceTokenizerFactory

constructor(args: MutableMap<String, String>)

Creates a new WhitespaceTokenizerFactory

constructor()

Default ctor for compatibility with SPI

Types

object Companion

SPI name

Properties

val classArg: String

isExplicitLuceneMatchVersion

var isExplicitLuceneMatchVersion: Boolean

lateinit var originalArgs: Map<String, String>

Functions

fun create(): Tokenizer

open override fun create(factory: AttributeFactory): Tokenizer

fun get(args: MutableMap<String, String>, name: String): String?

fun get(args: MutableMap<String, String>, name: String, defaultVal: String): String

fun get(args: MutableMap<String, String>, name: String, allowedValues: MutableCollection<String>, defaultVal: String?, caseSensitive: Boolean): String?

fun getChar(args: MutableMap<String, String>, name: String, defaultValue: Char): Char

fun getSet(args: MutableMap<String, String>, name: String): MutableSet<String>?

fun require(args: MutableMap<String, String>, name: String): String

fun require(args: MutableMap<String, String>, name: String, allowedValues: MutableCollection<String>, caseSensitive: Boolean): String

fun requireChar(args: MutableMap<String, String>, name: String): Char