org.codehaus.groovy.syntax.lexer
Class LexerBase

java.lang.Object
  extended byorg.codehaus.groovy.syntax.lexer.LexerBase
All Implemented Interfaces:
Lexer
Direct Known Subclasses:
GroovyLexerBase, GStringLexer, TextLexerBase

public class LexerBase
extends Object
implements Lexer

A base class for all other lexers.

Author:
Bob Mcwhirter, James Strachan, John Wilson, Chris Poirier

Field Summary
protected  Lexer delegate
           
protected  Lexer source
           
protected  int startColumn
           
protected  int startLine
           
 
Constructor Summary
LexerBase()
          Initializes the LexerBase.
 
Method Summary
 char consume()
          Eats a character from the input stream.
 void delegate(Lexer to)
          Delegates our duties to another Lexer.
 int getColumn()
          Returns the current column within that line.
 Lexer getDelegate()
          Gets the lexer that is actually doing the nextToken() work, if it isn't us.
 int getLine()
          Returns the current line number.
 Lexer getSource()
          Gets the lexer from which this lexer is obtaining characters.
protected  int getStartColumn()
          Returns the starting column of the current token.
protected  int getStartLine()
          Returns the starting line of the current token.
 boolean isDelegated()
          Returns true if we are delegated to another lexer.
 boolean isExternallySourced()
          Returns true if we are obtaining our characters from another lexer.
 char la()
          Returns the next character, without consuming it.
 char la(int k)
          Returns the next kth character, without consuming any.
protected  void mark()
          Saves information about the current position, for tracking token extents.
 Token nextToken()
          Finds and returns (consuming) the next token from the underlying stream.
protected  void readEOL()
          Synonym for readEOL(null).
protected  boolean readEOL(StringBuffer destination)
          Reads an end-of-line marker and writes the text into the specified buffer, if supplied.
 void reset()
          Resets a lexer for reuse.
 void setSource(Lexer source)
          Sets the source lexer.
protected  Token symbol(int type)
          Creates a new symbol token.
protected  Token symbol(int type, int columnOffset)
          Creates a new symbol token, and allows you to alter the starting column.
protected  Token tokenizeEOL()
          Process an end-of-line marker and returns a NEWLINE token.
 void undelegate()
          Retakes responsibility for our duties.
protected  Token undelegatedNextToken()
          Does undelegated nextToken() operations.
protected  void unexpected(char c, char[] expected, int offset)
          Creates and throws a new UnexpectedCharacterException.
protected  void unexpected(char c, int offset)
          Synonym for unexpected( c, null, offset ).
protected  void unexpected(char c, int offset, String message)
          Creates and throws a new UnexpectedCharacterException.
 void unsetSource()
          Unsets the source lexer.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

startLine

protected int startLine

startColumn

protected int startColumn

delegate

protected Lexer delegate

source

protected Lexer source
Constructor Detail

LexerBase

public LexerBase()
Initializes the LexerBase.

Method Detail

getDelegate

public Lexer getDelegate()
Gets the lexer that is actually doing the nextToken() work, if it isn't us.

Specified by:
getDelegate in interface Lexer

getSource

public Lexer getSource()
Gets the lexer from which this lexer is obtaining characters.

Specified by:
getSource in interface Lexer

nextToken

public Token nextToken()
                throws ReadException,
                       LexerException
Finds and returns (consuming) the next token from the underlying stream. Returns null when out of tokens. This implementation correctly handles delegation, and subclasses implement undelegatedNextToken(), which will be called by this routine when appropriate.

Specified by:
nextToken in interface Lexer
Throws:
ReadException
LexerException

undelegatedNextToken

protected Token undelegatedNextToken()
                              throws ReadException,
                                     LexerException
Does undelegated nextToken() operations. You supply your lexer-specific nextToken() code by overriding this method.

Throws:
ReadException
LexerException

tokenizeEOL

protected Token tokenizeEOL()
                     throws LexerException,
                            ReadException
Process an end-of-line marker and returns a NEWLINE token. Returns null if not at an end-of-line.

Throws:
LexerException
ReadException

readEOL

protected boolean readEOL(StringBuffer destination)
                   throws LexerException,
                          ReadException
Reads an end-of-line marker and writes the text into the specified buffer, if supplied.

Throws:
LexerException
ReadException

readEOL

protected void readEOL()
                throws LexerException,
                       ReadException
Synonym for readEOL(null).

Throws:
LexerException
ReadException

reset

public void reset()
Resets a lexer for reuse.

Specified by:
reset in interface Lexer

delegate

public void delegate(Lexer to)
Delegates our duties to another Lexer.

Specified by:
delegate in interface Lexer

undelegate

public void undelegate()
Retakes responsibility for our duties.

Specified by:
undelegate in interface Lexer

setSource

public void setSource(Lexer source)
Sets the source lexer.

Specified by:
setSource in interface Lexer

unsetSource

public void unsetSource()
Unsets the source lexer.

Specified by:
unsetSource in interface Lexer

isDelegated

public boolean isDelegated()
Returns true if we are delegated to another lexer.

Specified by:
isDelegated in interface Lexer

isExternallySourced

public boolean isExternallySourced()
Returns true if we are obtaining our characters from another lexer.

Specified by:
isExternallySourced in interface Lexer

unexpected

protected void unexpected(char c,
                          int offset,
                          String message)
                   throws UnexpectedCharacterException
Creates and throws a new UnexpectedCharacterException.

Throws:
UnexpectedCharacterException

unexpected

protected void unexpected(char c,
                          char[] expected,
                          int offset)
                   throws UnexpectedCharacterException
Creates and throws a new UnexpectedCharacterException.

Throws:
UnexpectedCharacterException

unexpected

protected void unexpected(char c,
                          int offset)
                   throws UnexpectedCharacterException
Synonym for unexpected( c, null, offset ).

Throws:
UnexpectedCharacterException

symbol

protected Token symbol(int type,
                       int columnOffset)
Creates a new symbol token, and allows you to alter the starting column.


symbol

protected Token symbol(int type)
Creates a new symbol token.


getLine

public int getLine()
Returns the current line number.

Specified by:
getLine in interface Lexer

getColumn

public int getColumn()
Returns the current column within that line.

Specified by:
getColumn in interface Lexer

mark

protected void mark()
Saves information about the current position, for tracking token extents.


getStartLine

protected int getStartLine()
Returns the starting line of the current token.


getStartColumn

protected int getStartColumn()
Returns the starting column of the current token.


la

public char la()
        throws LexerException,
               ReadException
Returns the next character, without consuming it.

Specified by:
la in interface Lexer
Throws:
LexerException
ReadException

la

public char la(int k)
        throws LexerException,
               ReadException
Returns the next kth character, without consuming any.

Specified by:
la in interface Lexer
Throws:
LexerException
ReadException

consume

public char consume()
             throws LexerException,
                    ReadException
Eats a character from the input stream.

Specified by:
consume in interface Lexer
Throws:
LexerException
ReadException


Copyright © 2003-2005 The Codehaus. All Rights Reserved.