lestes::lang::cplus::lex::ucn_token_buffer Class Reference

Token buffer. More...

#include <ucn_token_buffer.hh>

Inheritance diagram for lestes::lang::cplus::lex::ucn_token_buffer:

Inheritance graph
[legend]
List of all members.

Public Types

typedef buffer_type::size_type size_type
 Type for size of the buffer.

Public Member Functions

void add_back (const ptr< ucn_token > &item)
 Adds single item to the end.
void advance (size_type len)
 Discards items from the beginning.
ptr< ucn_tokenpeek_front (void) const
 Returns item at the beginning.
ptr< token_valueextract_until (ucn stop)
 Extracts value until stop character without interpreting.
ptr< token_valueextract_ordinary (size_type len)
 Extracts value without interpreting.
ptr< token_valueextract_simple_ucn (size_type len, bool identifier)
 Extracts identifiers and numbers with ucn.
ptr< token_valueextract_invalid_ucn (size_type len, bool identifier)
 Extracts identifiers and numbers with bad ucn.
ptr< token_valueextract_ucn_literal (size_type len)
 Extracts literal with escaped ucn.
ptr< token_valueextract_bad_literal (size_type len)
 Extracts malformed literal.
size_type length (void) const
 Returns length of the buffer.

Static Public Member Functions

static ptr< ucn_token_buffercreate (const ptr< line_control > &a_lines)
 Returns new buffer.

Protected Member Functions

 ucn_token_buffer (const ptr< line_control > &a_lines)
 Creates empty buffer.
virtual void gc_mark (void)
 Marks the object.

Private Types

typedef list< srp< ucn_token > > buffer_type
 Type of buffer to hold stored data.

Private Member Functions

 ucn_token_buffer (const ucn_token_buffer &)
 Hides copy constructor.
ucn_token_bufferoperator= (const ucn_token_buffer &)
 Hides assignment operator.

Private Attributes

srp< buffer_typebuffer
 Buffer to hold stored data.
srp< line_controllines
 Line control to transform locations.

Detailed Description

Token buffer.

Represents flexible buffer holding ucn_token items. Items are added mostly one by one at the end and afterwards removed from the beginning in longer runs, forming literals and identifiers.


Member Typedef Documentation

typedef list< srp<ucn_token> > lestes::lang::cplus::lex::ucn_token_buffer::buffer_type [private]

Type of buffer to hold stored data.

typedef buffer_type::size_type lestes::lang::cplus::lex::ucn_token_buffer::size_type

Type for size of the buffer.

Author:
TMA


Constructor & Destructor Documentation

lestes::lang::cplus::lex::ucn_token_buffer::ucn_token_buffer ( const ptr< line_control > &  a_lines  )  [protected]

Creates empty buffer.

Creates empty ucn token buffer.

Postcondition:
length() == 0
Precondition:
a_lines != NULL
Parameters:
a_lines The associated line control.

lestes::lang::cplus::lex::ucn_token_buffer::ucn_token_buffer ( const ucn_token_buffer  )  [private]

Hides copy constructor.


Member Function Documentation

void lestes::lang::cplus::lex::ucn_token_buffer::add_back ( const ptr< ucn_token > &  item  ) 

Adds single item to the end.

Adds single item to the end of the buffer.

Parameters:
item The item to be added.

void lestes::lang::cplus::lex::ucn_token_buffer::advance ( ucn_token_buffer::size_type  len  ) 

Discards items from the beginning.

Discards len items from the beginning. Used for skipping unimportant values.

Precondition:
len <= length()
Parameters:
len The length of the discarded sequence.

ptr< ucn_token > lestes::lang::cplus::lex::ucn_token_buffer::peek_front ( void   )  const

Returns item at the beginning.

Returns item currently at the beginning of the buffer.

Precondition:
buffer.size() != 0
Returns:
The item at the beginning of the buffer.

ptr< token_value > lestes::lang::cplus::lex::ucn_token_buffer::extract_until ( ucn  stop  ) 

Extracts value until stop character without interpreting.

Removes items from the beginning of the buffer until the stop value is encountered. That value is not removed. Returns token value representing the removed items. Performs no scanning for ucn or ucn escape sequences. Used for hchar and qchar sequences.

Precondition:
The stop value is present in the sequence.
Parameters:
stop The value to stop scanning the sequence.
Returns:
The representation of the items' values.

ptr< token_value > lestes::lang::cplus::lex::ucn_token_buffer::extract_ordinary ( ucn_token_buffer::size_type  len  ) 

Extracts value without interpreting.

Removes len items from the beginning of the buffer. Returns token value representing the removed items. Performs no scanning for ucn or ucn escape sequences. Used for identifiers with no ucn in any form and literals without escaped ucn.

Precondition:
len <= length()
Parameters:
len The length of the removed sequence.
Returns:
The representation of the items' values.

ptr< token_value > lestes::lang::cplus::lex::ucn_token_buffer::extract_simple_ucn ( ucn_token_buffer::size_type  len,
bool  identifier 
)

Extracts identifiers and numbers with ucn.

Removes len items from the beginning of the buffer. Returns token value representing the removed items. Performs scanning for ucn escape sequences, issues errors for ucn invalid in identifiers. Useful for identifiers and preprocessing numbers containing escaped or translated ucn characters.

Precondition:
len <= length()

Only contains backslash in ucn escape sequences.

The contained ucn escape sequences are well-formed.

Parameters:
len The length of the removed sequence.
identifier Flag set to true if extracting identifier.
Returns:
The representation of the items' values, with converted ucn escape sequences.

ptr< token_value > lestes::lang::cplus::lex::ucn_token_buffer::extract_invalid_ucn ( ucn_token_buffer::size_type  len,
bool  identifier 
)

Extracts identifiers and numbers with bad ucn.

Removes len items from the beginning of the buffer. Returns token value representing the removed items. Performs scanning for ucn escape sequences, issues errors for invalid ucn and ucn invalid in identifiers. Used for identifiers and preprocessing numbers containing translated or (possibly unterminated) escaped ucn characters.

Precondition:
len <= length()

Only contains backslash in ucn escape sequences.

Backslash is always followed by `U' or `u'.

Parameters:
len The length of the removed sequence.
identifier Flag set to true if extracting identifier.
Returns:
The representation of the items' values, with converted ucn escape sequences.

ptr< token_value > lestes::lang::cplus::lex::ucn_token_buffer::extract_ucn_literal ( ucn_token_buffer::size_type  len  ) 

Extracts literal with escaped ucn.

Removes len items from the beginning of the buffer. Returns token value representing the removed items. Performs scanning for ucn escape sequences, issues errors for invalid ucn. Useful for well-formed string and character literals.

Precondition:
len <= length()

Only contains well-formed escape sequences.

The contained ucn escape sequences are well-formed.

Parameters:
len The length of the removed sequence.
Returns:
The representation of the items' values, with converted ucn escape sequences.

ptr< token_value > lestes::lang::cplus::lex::ucn_token_buffer::extract_bad_literal ( ucn_token_buffer::size_type  len  ) 

Extracts malformed literal.

Removes len items from the beginning of the buffer. Returns token value representing the removed items. Performs scanning for escape sequences, issues errors for invalid. Used for character and string literals with ill-formed escape sequences.

Precondition:
len <= length()

The sequence does not end with backslash.

Parameters:
len The length of the removed sequence.
Returns:
The representation of the items' values, with converted ucn escape sequences.

ucn_token_buffer::size_type lestes::lang::cplus::lex::ucn_token_buffer::length ( void   )  const

Returns length of the buffer.

Returns length of the buffer.

Returns:
The current length of the buffer.

ptr< ucn_token_buffer > lestes::lang::cplus::lex::ucn_token_buffer::create ( const ptr< line_control > &  a_lines  )  [static]

Returns new buffer.

Retruns new empty ucn token buffer.

Postcondition:
length() == 0
Parameters:
a_lines The associated line control.
Returns:
The new ucn token buffer.

void lestes::lang::cplus::lex::ucn_token_buffer::gc_mark ( void   )  [protected, virtual]

Marks the object.

Marks the object.

Reimplemented from lestes::std::mem::keystone.

ucn_token_buffer& lestes::lang::cplus::lex::ucn_token_buffer::operator= ( const ucn_token_buffer  )  [private]

Hides assignment operator.


Member Data Documentation

srp<buffer_type> lestes::lang::cplus::lex::ucn_token_buffer::buffer [private]

Buffer to hold stored data.

srp<line_control> lestes::lang::cplus::lex::ucn_token_buffer::lines [private]

Line control to transform locations.


The documentation for this class was generated from the following files:
Generated on Mon Feb 12 18:43:55 2007 for lestes by doxygen 1.5.1-20070107