lestes::lang::cplus::lex::pp_token

lestes::lang::cplus::lex::pp_token Class Reference

Preprocessor token. More...

#include <pp_token.hh>

Inheritance diagram for lestes::lang::cplus::lex::pp_token:

Inheritance graph
[legend]
List of all members.

Public Types

enum  token_types {
  TOK_FILE_END, TOK_TERMINATOR, TOK_LEFT_BRACKET, TOK_RIGHT_BRACKET,
  TOK_LEFT_BRACE, TOK_RIGHT_BRACE, TOK_LEFT_PAR, TOK_RIGHT_PAR,
  TOK_SEMICOLON, TOK_COLON, TOK_QMARK, TOK_DOT,
  TOK_PLUS, TOK_STAR, TOK_PERCENT, TOK_SLASH,
  TOK_HAT, TOK_AMP, TOK_VBAR, TOK_TILDE,
  TOK_EMARK, TOK_EQ, TOK_LT, TOK_GT,
  TOK_COMMA, TOK_MINUS, TOK_DOT_DOT_DOT, TOK_COLON_COLON,
  TOK_DOT_STAR, TOK_PLUS_EQ, TOK_MINUS_EQ, TOK_STAR_EQ,
  TOK_SLASH_EQ, TOK_PERCENT_EQ, TOK_HAT_EQ, TOK_AMP_EQ,
  TOK_VBAR_EQ, TOK_LT_LT, TOK_GT_GT, TOK_LT_LT_EQ,
  TOK_GT_GT_EQ, TOK_EQ_EQ, TOK_EMARK_EQ, TOK_LT_EQ,
  TOK_GT_EQ, TOK_AMP_AMP, TOK_VBAR_VBAR, TOK_PLUS_PLUS,
  TOK_MINUS_MINUS, TOK_MINUS_GT_STAR, TOK_MINUS_GT, TOK_ASM,
  TOK_AUTO, TOK_BOOL, TOK_BREAK, TOK_CASE,
  TOK_CATCH, TOK_CHAR, TOK_CLASS, TOK_CONST,
  TOK_CONST_CAST, TOK_CONTINUE, TOK_DEFAULT, TOK_DELETE,
  TOK_DO, TOK_DOUBLE, TOK_DYNAMIC_CAST, TOK_ELSE,
  TOK_ENUM, TOK_EXPLICIT, TOK_EXPORT, TOK_EXTERN,
  TOK_FLOAT, TOK_FOR, TOK_FRIEND, TOK_GOTO,
  TOK_IF, TOK_INLINE, TOK_INT, TOK_LONG,
  TOK_MUTABLE, TOK_NAMESPACE, TOK_NEW, TOK_OPERATOR,
  TOK_PRIVATE, TOK_PROTECTED, TOK_PUBLIC, TOK_REGISTER,
  TOK_REINTERPRET_CAST, TOK_RETURN, TOK_SHORT, TOK_SIGNED,
  TOK_SIZEOF, TOK_STATIC, TOK_STATIC_CAST, TOK_STRUCT,
  TOK_SWITCH, TOK_TEMPLATE, TOK_THIS, TOK_THROW,
  TOK_TRY, TOK_TYPEDEF, TOK_TYPEID, TOK_TYPENAME,
  TOK_UNION, TOK_UNSIGNED, TOK_USING, TOK_VIRTUAL,
  TOK_VOID, TOK_VOLATILE, TOK_WCHAR, TOK_WHILE,
  TOK_BOOL_LIT, TOK_CHAR_LIT, TOK_WCHAR_LIT, TOK_STRING_LIT,
  TOK_WSTRING_LIT, TOK_NUMBER_LIT, TOK_IDENT, TOK_HASH,
  TOK_HASH_HASH, TOK_BLANK, TOK_LINE_END, TOK_OTHER,
  TOK_INCLUDE, TOK_DEFINE, TOK_UNDEF, TOK_LINE,
  TOK_PRAGMA, TOK_ERROR, TOK_DEFINED, TOK_IFDEF,
  TOK_IFNDEF, TOK_ELIF, TOK_ENDIF, TOK_INCL_HCHAR,
  TOK_INCL_QCHAR, TOK_INCL_SIG, TOK_LAST, TOK_AVOID_WARNING
}
 Token type constants. More...
enum  { TOKEN_TYPE_COUNT = TOK_AVOID_WARNING }

Public Member Functions

bool is_name (void) const
 Token can be macro name.
bool is_alternative (void) const
 Token has alternative spelling.
bool is_valued (void) const
 Token has value.
bool equals (const ptr< pp_token > &other) const
 Tests equality.
bool congruent (const ptr< pp_token > &other) const
 Tests congruence.
lstring description_get (void) const
 Returns description.
ucn_string spelling_get (void) const
 Returns token spelling.
ptr< taboo_macros > taboo_get (void) const
 Returns taboo macros.
ptr< pp_token > clone (const location_type &a_location) const
 Returns copy of token with different location.
ptr< pp_token > clone (const ptr< taboo_macros > &a_taboo) const
 Returns copy of token with different taboo macros.

Static Public Member Functions

static ptr< pp_token > create (const location_type &a_location, const type_type &a_type)
 Returns new token, initializes with location and type.
static ptr< pp_token > create_alternative (const location_type &a_location, const type_type &a_type)
 Returns new alternative token, initializes with location and type.
static ptr< pp_token > create (const location_type &a_location, const type_type &a_type, const value_type &a_value)
 Returns new token, initializes with location, type and value.
static ptr< pp_token > terminator (void)
 Returns the terminator token.

Protected Member Functions

 pp_token (const location_type &a_location, const type_type &a_type, const value_type &a_value, bool a_alternative, const ptr< taboo_macros > &a_taboo)
 Creates new token, initializes with appropriate values.
virtual void gc_mark (void)
 Marks the object.

Private Types

enum  pp_token_flags {
  FLG_NONE = 0x0, FLG_INTERNAL = 0x1, FLG_EXTERNAL = 0x2, FLG_NAME = 0x4,
  FLG_VARIANT = 0x08
}
 Token flags. More...
typedef lc_host_uint_least16 flags_type
 Type of flags.

Private Member Functions

 pp_token (const pp_token &)
 Hides copy constructor.
pp_token & operator= (const pp_token &)
 Hides assignment operator.

Static Private Member Functions

static void initialize (void)
 Initializes static structures.

Private Attributes

bool alternative
 Token has alternative spelling.
srp< taboo_macros > taboo
 Taboo macros for token.

Static Private Attributes

static bool initialized
 Initialization flag.
static lstring the_description [TOKEN_TYPE_COUNT]
 Descriptions of tokens.
static ucn_string the_spelling [2][TOKEN_TYPE_COUNT]
 Spelling of tokens.
static ptr< token_value > the_value [TOKEN_TYPE_COUNT]
 Values of tokens.
static flags_type the_flags [TOKEN_TYPE_COUNT]
 Flags of tokens.
static ptr< pp_token > terminator_instance = terminator_instance
 The terminator token instance.

Detailed Description

Preprocessor token.

Represents preprocessor token. Each token has type and location, literals and names also have value. Token type properties are represented by internal flags.

Member Typedef Documentation

typedef lc_host_uint_least16 lestes::lang::cplus::lex::pp_token::flags_type [private]

Type of flags.

Member Enumeration Documentation

enum lestes::lang::cplus::lex::pp_token::token_types

Token type constants.

Enumerator:
TOK_FILE_END 
TOK_TERMINATOR 
TOK_LEFT_BRACKET 
TOK_RIGHT_BRACKET 
TOK_LEFT_BRACE 
TOK_RIGHT_BRACE 
TOK_LEFT_PAR 
TOK_RIGHT_PAR 
TOK_SEMICOLON 
TOK_COLON 
TOK_QMARK 
TOK_DOT 
TOK_PLUS 
TOK_STAR 
TOK_PERCENT 
TOK_SLASH 
TOK_HAT 
TOK_AMP 
TOK_VBAR 
TOK_TILDE 
TOK_EMARK 
TOK_EQ 
TOK_LT 
TOK_GT 
TOK_COMMA 
TOK_MINUS 
TOK_DOT_DOT_DOT 
TOK_COLON_COLON 
TOK_DOT_STAR 
TOK_PLUS_EQ 
TOK_MINUS_EQ 
TOK_STAR_EQ 
TOK_SLASH_EQ 
TOK_PERCENT_EQ 
TOK_HAT_EQ 
TOK_AMP_EQ 
TOK_VBAR_EQ 
TOK_LT_LT 
TOK_GT_GT 
TOK_LT_LT_EQ 
TOK_GT_GT_EQ 
TOK_EQ_EQ 
TOK_EMARK_EQ 
TOK_LT_EQ 
TOK_GT_EQ 
TOK_AMP_AMP 
TOK_VBAR_VBAR 
TOK_PLUS_PLUS 
TOK_MINUS_MINUS 
TOK_MINUS_GT_STAR 
TOK_MINUS_GT 
TOK_ASM 
TOK_AUTO 
TOK_BOOL 
TOK_BREAK 
TOK_CASE 
TOK_CATCH 
TOK_CHAR 
TOK_CLASS 
TOK_CONST 
TOK_CONST_CAST 
TOK_CONTINUE 
TOK_DEFAULT 
TOK_DELETE 
TOK_DO 
TOK_DOUBLE 
TOK_DYNAMIC_CAST 
TOK_ELSE 
TOK_ENUM 
TOK_EXPLICIT 
TOK_EXPORT 
TOK_EXTERN 
TOK_FLOAT 
TOK_FOR 
TOK_FRIEND 
TOK_GOTO 
TOK_IF 
TOK_INLINE 
TOK_INT 
TOK_LONG 
TOK_MUTABLE 
TOK_NAMESPACE 
TOK_NEW 
TOK_OPERATOR 
TOK_PRIVATE 
TOK_PROTECTED 
TOK_PUBLIC 
TOK_REGISTER 
TOK_REINTERPRET_CAST 
TOK_RETURN 
TOK_SHORT 
TOK_SIGNED 
TOK_SIZEOF 
TOK_STATIC 
TOK_STATIC_CAST 
TOK_STRUCT 
TOK_SWITCH 
TOK_TEMPLATE 
TOK_THIS 
TOK_THROW 
TOK_TRY 
TOK_TYPEDEF 
TOK_TYPEID 
TOK_TYPENAME 
TOK_UNION 
TOK_UNSIGNED 
TOK_USING 
TOK_VIRTUAL 
TOK_VOID 
TOK_VOLATILE 
TOK_WCHAR 
TOK_WHILE 
TOK_BOOL_LIT 
TOK_CHAR_LIT 
TOK_WCHAR_LIT 
TOK_STRING_LIT 
TOK_WSTRING_LIT 
TOK_NUMBER_LIT 
TOK_IDENT 
TOK_HASH 
TOK_HASH_HASH 
TOK_BLANK 
TOK_LINE_END 
TOK_OTHER 
TOK_INCLUDE 
TOK_DEFINE 
TOK_UNDEF 
TOK_LINE 
TOK_PRAGMA 
TOK_ERROR 
TOK_DEFINED 
TOK_IFDEF 
TOK_IFNDEF 
TOK_ELIF 
TOK_ENDIF 
TOK_INCL_HCHAR 
TOK_INCL_QCHAR 
TOK_INCL_SIG 
TOK_LAST 
TOK_AVOID_WARNING 

anonymous enum

Enumerator:
TOKEN_TYPE_COUNT  Number of token types.

enum lestes::lang::cplus::lex::pp_token::pp_token_flags [private]

Token flags.

Enumerator:
FLG_NONE  Empty flags.
FLG_INTERNAL  Token has internally initialized value.
FLG_EXTERNAL  Token has externally initialized value.
FLG_NAME  Token is a name.
FLG_VARIANT  Token has two variants of spelling.

Constructor & Destructor Documentation

lestes::lang::cplus::lex::pp_token::pp_token ( const location_type &  a_location,
const type_type &  a_type,
const value_type &  a_value,
bool  a_alternative,
const ptr< taboo_macros > &  a_taboo 
) [protected]

Creates new token, initializes with appropriate values.

Creates new token, initializes with location, token type, token value and flags.

Precondition:
a_taboo != NULL
Postcondition:
is_equal(location_get(),a_location)

is_equal(type_get(),a_type)

is_equal(value_get(),a_value)

Parameters:
a_location The initial location.
a_type The type.
a_value The value.
a_alternative The alternative spelling flag.
a_taboo The taboo.

lestes::lang::cplus::lex::pp_token::pp_token ( const pp_token &   )  [private]

Hides copy constructor.

Member Function Documentation

bool lestes::lang::cplus::lex::pp_token::is_name ( void   )  const

Token can be macro name.

Tests if token is name, that is identifier, keyword, true or false.

Returns:
true If token is a name.

bool lestes::lang::cplus::lex::pp_token::is_alternative ( void   )  const

Token has alternative spelling.

Tests if token has alternative spelling.

Returns:
true If token has alternative spelling.

bool lestes::lang::cplus::lex::pp_token::is_valued ( void   )  const

Token has value.

Tests if token has value.

Returns:
true If token has alternative spelling.

bool lestes::lang::cplus::lex::pp_token::equals ( const ptr< pp_token > &  other  )  const

Tests equality.

Tests equality of the token.

Parameters:
other The token to compare with.
Returns:
true If both tokens are equal.

bool lestes::lang::cplus::lex::pp_token::congruent ( const ptr< pp_token > &  other  )  const

Tests congruence.

Tests congruence of the token, that is equivalence considering only type, value and spelling.

Parameters:
other The token to compare with.
Returns:
true If both tokens are congruent.

lstring lestes::lang::cplus::lex::pp_token::description_get ( void   )  const

Returns description.

Returns internal token description dependent on token type, for debugging.

Returns:
The token description.

ucn_string lestes::lang::cplus::lex::pp_token::spelling_get ( void   )  const

Returns token spelling.

Returns token spelling, reflecting the source.

Returns:
String representation of the token.

ptr< taboo_macros > lestes::lang::cplus::lex::pp_token::taboo_get ( void   )  const

Returns taboo macros.

Returns taboo macros for the token.

Returns:
The taboo macros for the token.

ptr< pp_token > lestes::lang::cplus::lex::pp_token::clone ( const location_type &  a_location  )  const

Returns copy of token with different location.

Returns copy of this token, with different location.

Precondition:
a_location != NULL
Parameters:
a_location The new location
Returns:
New token which holds the same values and new location.

ptr< pp_token > lestes::lang::cplus::lex::pp_token::clone ( const ptr< taboo_macros > &  a_taboo  )  const

Returns copy of token with different taboo macros.

Returns copy of this token, with different taboo macros.

Precondition:
a_taboo != NULL

The token is name.

Parameters:
a_taboo The new taboo macros.
Returns:
New token which holds the same values and new taboo macros.

ptr< pp_token > lestes::lang::cplus::lex::pp_token::create ( const location_type &  a_location,
const type_type &  a_type 
) [static]

Returns new token, initializes with location and type.

Returns new token, initializes with location, token type.

Precondition:
The type does not need value or value is initialized internally.
Postcondition:
is_equal(location_get(),a_location)

is_equal(type_get(),a_type)

Parameters:
a_location The location.
a_type The token type.
Returns:
New token initialized with location and token type.

ptr< pp_token > lestes::lang::cplus::lex::pp_token::create_alternative ( const location_type &  a_location,
const type_type &  a_type 
) [static]

Returns new alternative token, initializes with location and type.

Returns new token with alternative spelling, initializes with location and token type.

Precondition:
The type does not need value or value is initialized internally.

Alternative spelling is used only when available.

Postcondition:
is_equal(location_get(),a_location)

is_equal(type_get(),a_type)

Parameters:
a_location The location.
a_type The token type.
Returns:
New token initialized with location and token type.

ptr< pp_token > lestes::lang::cplus::lex::pp_token::create ( const location_type &  a_location,
const type_type &  a_type,
const value_type &  a_value 
) [static]

Returns new token, initializes with location, type and value.

Returns new token, initializes with location, token type and token value.

Precondition:
The token type needs an external value.
Postcondition:
is_equal(location_get(),a_location)

is_equal(type_get(),a_type)

is_equal(value_get(),a_value)

Parameters:
a_location The initial location.
a_type The initial token type.
a_value The initial token value.
Returns:
New token initialized with location, token type and token value.

ptr< pp_token > lestes::lang::cplus::lex::pp_token::terminator ( void   )  [static]

Returns the terminator token.

void lestes::lang::cplus::lex::pp_token::gc_mark ( void   )  [protected, virtual]

Marks the object.

Marks the object.

Reimplemented from lestes::lang::cplus::lex::basic_token< lc_host_uint_least16, ptr< source_location >, ptr< token_value > >.

pp_token& lestes::lang::cplus::lex::pp_token::operator= ( const pp_token &   )  [private]

Hides assignment operator.

void lestes::lang::cplus::lex::pp_token::initialize ( void   )  [static, private]

Initializes static structures.

Initializes helper static structures containing properties of existing token types. The initialization is invoked automatically prior to use of any instance.

Member Data Documentation

bool lestes::lang::cplus::lex::pp_token::alternative [private]

Token has alternative spelling.

srp<taboo_macros> lestes::lang::cplus::lex::pp_token::taboo [private]

Taboo macros for token.

bool lestes::lang::cplus::lex::pp_token::initialized [static, private]

Initialization flag.

Multiple initialization guard.

lstring lestes::lang::cplus::lex::pp_token::the_description [static, private]

Descriptions of tokens.

List of token descriptions.

ucn_string lestes::lang::cplus::lex::pp_token::the_spelling [static, private]

Spelling of tokens.

List of token spellings.

ptr< token_value > lestes::lang::cplus::lex::pp_token::the_value [static, private]

Values of tokens.

List of token internally initialized values.

pp_token::flags_type lestes::lang::cplus::lex::pp_token::the_flags [static, private]

Flags of tokens.

List of token flags.

ptr< pp_token > lestes::lang::cplus::lex::pp_token::terminator_instance = terminator_instance [static, private]

The terminator token instance.

Terminator token instance.

The documentation for this class was generated from the following files:
Generated on Mon Feb 12 18:43:49 2007 for lestes by doxygen 1.5.1-20070107