FastCharStream
An efficient implementation of JavaCC's CharStream interface.
Note that this does not do line-number counting, but instead keeps track of the character position of the token in the input, as required by Lucene's [ ] API.
Properties
Functions
Backs up the input stream by amount steps. Lexer calls this method if it had already read some characters, but could not use them to match a (longer) token. So, they will be used again as the prefix of the next token and it is the implementation's responsibility to do this right.
Returns the next character that marks the beginning of the next token. All characters must remain in the buffer between two successive calls to this method to implement backup correctly.
Returns a string made up of characters from the marked token beginning to the current buffer position. Implementations have the choice of returning anything that they want to. For example, for efficiency, one might decide to just return null, which is a valid implementation.