org.modeshape.sequencer.ddl
Class DdlTokenStream.DdlTokenizer

java.lang.Object
  extended by org.modeshape.sequencer.ddl.DdlTokenStream.DdlTokenizer
All Implemented Interfaces:
TokenStream.Tokenizer
Enclosing class:
DdlTokenStream

public static class DdlTokenStream.DdlTokenizer
extends Object
implements TokenStream.Tokenizer


Field Summary
static int COMMENT
          The token type for tokens that consist of all the characters between "/*" and "*/", between "//" and the next line terminator (e.g., '\n', '\r' or "\r\n"), or between "--" and the next line terminator (e.g., '\n', '\r' or "\r\n").
static int DECIMAL
          The token type for tokens that consist of an individual '.' character.
static int DOUBLE_QUOTED_STRING
          The token type for tokens that consist of all the characters within double-quotes.
static int KEYWORD
          The token type for tokens that represent key words or reserved words for a given DDL dialect.
static String PARSER_ID
           
static int SINGLE_QUOTED_STRING
          The token type for tokens that consist of all the characters within single-quotes.
static int STATEMENT_KEY
          The token type for tokens that represent the start of a DDL statement.
static int SYMBOL
          The token type for tokens that consist of an individual "symbol" character.
static int WORD
          The token type for tokens that represent an unquoted string containing a character sequence made up of non-whitespace and non-symbol characters.
 
Constructor Summary
DdlTokenStream.DdlTokenizer(boolean useComments)
           
 
Method Summary
 boolean includeComments()
           
 void tokenize(TokenStream.CharacterStream input, TokenStream.Tokens tokens)
          Process the supplied characters and construct the appropriate TokenStream.Token objects.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

PARSER_ID

public static final String PARSER_ID
See Also:
Constant Field Values

WORD

public static final int WORD
The token type for tokens that represent an unquoted string containing a character sequence made up of non-whitespace and non-symbol characters.

See Also:
Constant Field Values

SYMBOL

public static final int SYMBOL
The token type for tokens that consist of an individual "symbol" character. The set of characters includes: -(){}*,;+%?$[]!<>|=:

See Also:
Constant Field Values

DECIMAL

public static final int DECIMAL
The token type for tokens that consist of an individual '.' character.

See Also:
Constant Field Values

SINGLE_QUOTED_STRING

public static final int SINGLE_QUOTED_STRING
The token type for tokens that consist of all the characters within single-quotes. Single quote characters are included if they are preceded (escaped) by a '\' character.

See Also:
Constant Field Values

DOUBLE_QUOTED_STRING

public static final int DOUBLE_QUOTED_STRING
The token type for tokens that consist of all the characters within double-quotes. Double quote characters are included if they are preceded (escaped) by a '\' character.

See Also:
Constant Field Values

COMMENT

public static final int COMMENT
The token type for tokens that consist of all the characters between "/*" and "*/", between "//" and the next line terminator (e.g., '\n', '\r' or "\r\n"), or between "--" and the next line terminator (e.g., '\n', '\r' or "\r\n").

See Also:
Constant Field Values

KEYWORD

public static final int KEYWORD
The token type for tokens that represent key words or reserved words for a given DDL dialect.

Examples would be: "CREATE", "TABLE", "ALTER", "SCHEMA", "DROP", etc...

see DdlConstants for the default SQL 92 representations.

See Also:
Constant Field Values

STATEMENT_KEY

public static final int STATEMENT_KEY
The token type for tokens that represent the start of a DDL statement.

Examples would be: {"CREATE", "TABLE"} {"CREATE", "OR", "REPLACE", "VIEW"}

see DdlConstants for the default SQL 92 representations.

See Also:
Constant Field Values
Constructor Detail

DdlTokenStream.DdlTokenizer

public DdlTokenStream.DdlTokenizer(boolean useComments)
Method Detail

includeComments

public boolean includeComments()
Returns:
useComments

tokenize

public void tokenize(TokenStream.CharacterStream input,
                     TokenStream.Tokens tokens)
              throws ParsingException
Process the supplied characters and construct the appropriate TokenStream.Token objects.

Specified by:
tokenize in interface TokenStream.Tokenizer
Parameters:
input - the character input stream; never null
tokens - the factory for TokenStream.Token objects, which records the order in which the tokens are created
Throws:
ParsingException - if there is an error while processing the character stream (e.g., a quote is not closed, etc.)
See Also:
org.modeshape.common.text.TokenStream.Tokenizer#tokenize(CharacterStream, Tokens)


Copyright © 2008-2010 JBoss, a division of Red Hat. All Rights Reserved.