|
||||||||||
PREV NEXT | FRAMES NO FRAMES |
Packages that use Tokenizer | |
---|---|
jfun.parsec | Provides classes and interfaces for parser combinator logic and basic parsers. |
jfun.parsec.tokens | Provides some predefined tokens and Tokenizer implementations. |
Uses of Tokenizer in jfun.parsec |
---|
Classes in jfun.parsec that implement Tokenizer | |
---|---|
class |
ConstTokenizer
This tokenizer will always return the same token regardless of the input range. |
Methods in jfun.parsec that return Tokenizer | |
---|---|
static Tokenizer |
ConstTokenizer.instance(java.lang.Object t)
Creates the Tokenizer instance. |
Methods in jfun.parsec with parameters of type Tokenizer | |
---|---|
static Parser<Tok> |
Lexers.lexer(Parser<?> s,
Tokenizer tn)
Transform the recognized character range of scanner s to a token object with a Tokenizer. |
static Parser<Tok> |
Scanners.lexer(Parser<?> s,
Tokenizer tn)
Deprecated. |
static Parser<Tok> |
Lexers.lexer(Parser<?> s,
Tokenizer tn,
java.lang.String err)
Transform the recognized character range of scanner s to a token object with a Tokenizer. |
static Parser<Tok> |
Scanners.lexer(Parser<?> s,
Tokenizer tn,
java.lang.String err)
Deprecated. |
static Parser<Tok> |
Lexers.lexer(java.lang.String name,
Parser<?> s,
Tokenizer tn)
Transform the recognized character range of scanner s to a token object with a Tokenizer. |
static Parser<Tok> |
Scanners.lexer(java.lang.String name,
Parser<?> s,
Tokenizer tn)
Deprecated. |
static Parser<Tok> |
Lexers.lexer(java.lang.String name,
Parser<?> s,
Tokenizer tn,
java.lang.String err)
Transform the recognized character range of scanner s to a token object with a Tokenizer. |
static Parser<Tok> |
Scanners.lexer(java.lang.String name,
Parser<?> s,
Tokenizer tn,
java.lang.String err)
Deprecated. |
Uses of Tokenizer in jfun.parsec.tokens |
---|
Classes in jfun.parsec.tokens that implement Tokenizer | |
---|---|
class |
TokenString
Deprecated. |
Methods in jfun.parsec.tokens that return Tokenizer | ||
---|---|---|
static Tokenizer |
Tokenizers.forChar()
Creates a tokenizer that's gonna tokenize a single quoted character literal possibly with escape character '\' |
|
static Tokenizer |
Tokenizers.forDecimal()
Get the Tokenizer object that creates a decimal number literal token. |
|
static Tokenizer |
Tokenizers.forDecLong()
Get the Tokenizer object that interprets the recognized character range as a decimal integer and translate it to a long value. |
|
static Tokenizer |
Tokenizers.forHexLong()
Get the Tokenizer object that interprets the recognized character range as a hex integer and translate it to a long value. |
|
static Tokenizer |
Tokenizers.forInteger()
Get the Tokenizer object that creates an integer literal token. |
|
static Tokenizer |
Tokenizers.forOctLong()
Get the Tokenizer object that interprets the recognized character range as a oct integer and translate it to a long value. |
|
static Tokenizer |
Tokenizers.forQuotedString(char open,
char close)
Get the Tokenizer object that converts a string literal quoted by a pair of opening and closing characters. |
|
static Tokenizer |
Tokenizers.forQuotedString(java.lang.String open,
java.lang.String close)
Get the Tokenizer object that converts a string literal quoted by a pair of opening and closing strings. |
|
static Tokenizer |
Tokenizers.forReservedWord()
Get the Tokenizer object that creates a reserved word token. |
|
static Tokenizer |
Tokenizers.forSimpleStringLiteral()
Get the Tokenizer object that converts a string literal quoted by '"' to a string object. |
|
static Tokenizer |
Tokenizers.forSqlStringLiteral()
Get the Tokenizer object that converts a sql string literal quoted by single quote to a string object. |
|
static Tokenizer |
Tokenizers.forString()
Get the Tokenizer object that creates a string object. |
|
static
|
Tokenizers.forTypedToken(T type)
Create a tokenizer that tokenizes the recognized character range to a TypedToken object. |
|
static Tokenizer |
Tokenizers.forWord()
Get the Tokenizer object that creates a word token. |
|
static Tokenizer |
TokenLong.getDecTokenizer()
Deprecated. creates a Tokenizer instance that can parse a decimal integer string. |
|
static Tokenizer |
TokenStringLiteral.getDoubleQuoteTokenizer()
Deprecated. creates a Tokenizer instance() that parsers a double quoted string literal that is possiblly escaped by '\' |
|
static Tokenizer |
TokenLong.getHexTokenizer()
Deprecated. creates a Tokenizer instance that can parse a hex integer string. |
|
static Tokenizer |
TokenLong.getOctTokenizer()
Deprecated. creates a Tokenizer instance that can parse a octal integer string. |
|
static Tokenizer |
TokenStringLiteral.getSqlTokenizer()
Deprecated. creates a Tokenizer instance() that parsers a single quoted string literal. |
|
static Tokenizer |
TokenCharLiteral.getTokenizer()
Deprecated. Creates a tokenizer that's gonna tokenize a single quoted character literal possibly with escape character '\' |
|
static Tokenizer |
TokenDecimal.getTokenizer()
Deprecated. Creates a Tokenizer that's gonna tokenize any valid decimal literal string to a TokenDecimal object. |
|
static Tokenizer |
TokenInteger.getTokenizer()
Deprecated. Creates a Tokenizer that's gonna tokenize any valid decimal literal string to a TokenDecimal object. |
|
static Tokenizer |
TokenString.getTokenizer()
Deprecated. Get the Tokenizer instance. |
|
static Tokenizer |
TokenWord.getTokenizer()
Deprecated. gets an instance of TokenWord that parses a input range to a TokenWord. |
|
static Tokenizer |
TokenQuoted.getTokenizer(char open,
char close)
Deprecated. Use Tokenizers.forQuotedString(char, char) instead. |
|
static Tokenizer |
MyToken.getTokenizer(int k)
Deprecated. Create a tokenizer that's gonna return a MyToken object with a certain kind number. |
|
static Tokenizer |
TokenQuoted.getTokenizer(java.lang.String open,
java.lang.String close)
Deprecated. Use Tokenizers.forQuotedString(String, String) instead. |
|
||||||||||
PREV NEXT | FRAMES NO FRAMES |