org.modeshape.graph.query.parse
Class SqlQueryParser.SqlTokenizer

java.lang.Object
  org.modeshape.graph.query.parse.SqlQueryParser.SqlTokenizer

All Implemented Interfaces:

TokenStream.Tokenizer

Enclosing class:

SqlQueryParser

public static class SqlQueryParser.SqlTokenizer
extends Object
implements TokenStream.Tokenizer

A TokenStream.Tokenizer implementation that parses words, quoted phrases, comments, and symbols. Words are delimited by whitespace and consist only of alpha-number characters plus the underscore character. Quoted phrases are delimited by single-quote and double-quote characters (which may be escaped within the quote). Comments are the characters starting with '/*' and ending with '*/', or starting with '--' and ending with the next line terminator (or the end of the content).

Field Summary

static int	COMMENT The token type for tokens that consist of all the characters between "/*" and "*/" or between "--" and the next line terminator (e.g., '\n', '\r' or "\r\n")
static int	OTHER The token type for tokens that consist of other characters.
static int	QUOTED_STRING The token type for tokens that consist of all the characters within single-quotes, double-quotes, or square brackets.
static int	SYMBOL The token type for tokens that consist of an individual "symbol" character.
static int	WORD The token type for tokens that represent an unquoted string containing a character sequence made up of non-whitespace and non-symbol characters.

Constructor Summary

SqlQueryParser.SqlTokenizer(boolean useComments)

Method Summary

void	tokenize(TokenStream.CharacterStream input, TokenStream.Tokens tokens) Process the supplied characters and construct the appropriate TokenStream.Token objects.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

WORD

public static final int WORD

The token type for tokens that represent an unquoted string containing a character sequence made up of non-whitespace and non-symbol characters.

See Also:

Constant Field Values

SYMBOL

public static final int SYMBOL

The token type for tokens that consist of an individual "symbol" character. The set of characters includes: []<>=-+(),

See Also:

Constant Field Values

OTHER

public static final int OTHER

The token type for tokens that consist of other characters.

See Also:

Constant Field Values

QUOTED_STRING

public static final int QUOTED_STRING

The token type for tokens that consist of all the characters within single-quotes, double-quotes, or square brackets.

See Also:

Constant Field Values

COMMENT

public static final int COMMENT

The token type for tokens that consist of all the characters between "/*" and "*/" or between "--" and the next line terminator (e.g., '\n', '\r' or "\r\n")

See Also:

Constant Field Values

Constructor Detail

SqlQueryParser.SqlTokenizer

public SqlQueryParser.SqlTokenizer(boolean useComments)

Method Detail

tokenize

public void tokenize(TokenStream.CharacterStream input,
                     TokenStream.Tokens tokens)
              throws ParsingException

Process the supplied characters and construct the appropriate TokenStream.Token objects.

Specified by:

tokenize in interface TokenStream.Tokenizer

Parameters:

input - the character input stream; never null

tokens - the factory for TokenStream.Token objects, which records the order in which the tokens are created

Throws:

ParsingException - if there is an error while processing the character stream (e.g., a quote is not closed, etc.)

See Also:

org.modeshape.common.text.TokenStream.Tokenizer#tokenize(CharacterStream, Tokens)