Just like .getPositionIncrementGap, except for Token offsets instead. By default this returns 1. This method is only called if the field produced at least one token for indexing.

getPositionIncrementGap

open fun getPositionIncrementGap(fieldName: String?): Int

Invoked before indexing a IndexableField instance if terms have already been added to that field. This allows custom analyzers to place an automatic position increment gap between IndexbleField instances using the same field name. The default value position increment gap is

normalize

fun normalize(fieldName: String, text: String): BytesRef

Normalize a string down to the representation that it would have in the index.

tokenStream

fun tokenStream(fieldName: String, text: String): TokenStream

Returns a TokenStream suitable for fieldName, tokenizing the contents of text.

fun tokenStream(fieldName: String, reader: Reader): TokenStream

Returns a TokenStream suitable for fieldName, tokenizing the contents of reader.