tokenStream
Returns a TokenStream suitable for fieldName, tokenizing the contents of reader.
This method uses .createComponents to obtain an instance of [ ]. It returns the sink of the components and stores the components internally. Subsequent calls to this method will reuse the previously stored components after resetting them through TokenStreamComponents.setReader.
NOTE: After calling this method, the consumer must follow the workflow described in TokenStream to properly consume its contents. See the Analysis package documentation for some examples demonstrating this.
NOTE: If your data is available as a String, use .tokenStream which reuses a StringReader-like instance internally.
Return
TokenStream for iterating the analyzed content of reader
Parameters
the name of the field the created TokenStream is used for
the reader the streams source reads from
See also
.tokenStream
Throws
if the Analyzer is closed.
Returns a TokenStream suitable for fieldName, tokenizing the contents of text.
This method uses .createComponents to obtain an instance of [ ]. It returns the sink of the components and stores the components internally. Subsequent calls to this method will reuse the previously stored components after resetting them through TokenStreamComponents.setReader.
NOTE: After calling this method, the consumer must follow the workflow described in TokenStream to properly consume its contents. See the Analysis package documentation for some examples demonstrating this.
Return
TokenStream for iterating the analyzed content of reader
Parameters
the name of the field the created TokenStream is used for
the String the streams source reads from
See also
.tokenStream
Throws
if the Analyzer is closed.