antlr
Class TokenBuffer
public class TokenBuffer
extends java.lang.Object
A Stream of Token objects fed to the parser from a Tokenizer that can
be rewound via mark()/rewind() methods.
A dynamic array is used to buffer up all the input tokens. Normally,
"k" tokens are stored in the buffer. More tokens may be stored during
guess mode (testing syntactic predicate), or when LT(i>k) is referenced.
Consumption of tokens is deferred. In other words, reading the next
token is not done by conume(), but deferred until needed by LA or LT.
int | LA(int i) - Get a lookahead token value
|
Token | LT(int i) - Get a lookahead token
|
void | consume() - Mark another token for deferred consumption
|
TokenStream | getInput() - return the Tokenizer (needed by ParseView)
|
int | mark() - Return an integer marker that can be used to rewind the buffer to
its current state.
|
void | reset() - Reset the input buffer to empty state
|
void | rewind(int mark) - Rewind the token buffer to a marker.
|
markerOffset
(package private) int markerOffset
nMarkers
(package private) int nMarkers
numToConsume
(package private) int numToConsume
TokenBuffer
public TokenBuffer(TokenStream input_)
Create a token buffer
consume
public final void consume()
Mark another token for deferred consumption
getInput
public TokenStream getInput()
return the Tokenizer (needed by ParseView)
mark
public final int mark()
Return an integer marker that can be used to rewind the buffer to
its current state.
reset
public final void reset()
Reset the input buffer to empty state
rewind
public final void rewind(int mark)
Rewind the token buffer to a marker.
mark
- Marker returned previously from mark()