create

Creates a TokenStream of the specified input using the default attribute factory.


abstract fun create(factory: AttributeFactory): Tokenizer

Creates a TokenStream of the specified input using the given AttributeFactory