Lexer

Description

The Lexer is the most fundamental concept. A Lexer has a member function, collect, that accepts a token definition container Def, and a the name of the lexer state the token definitions of the lexer component need to be added to (a string). It doesn't return anything (return type is void). Each Lexer can represent a specific pattern or algorithm, or it can be a more complex lexer component formed as a composition of other Lexer's. Additionally, a Lexer exposes a member add_actions, that accepts the token definition container Def, while returning nothing (again, the returned type is void).

Notation

l: A Lexer.
L: A Lexer type.
Def: A token definition container type.
State: A type used to represent lexer state names.

Valid Expressions

In the expressions below, the behavior of the lexer component, l, is left unspecified in the base Lexer concept. These are specified in subsequent, more refined concepts and by the actual models thereof.

For any Lexer the following expressions must be valid:

Expression	Semantics	Return type
`l.collect(def, state, targetstate)`	Add all token definitions provided by this Lexer instance to the lexer state `state` of the token definition container `def`. After matching this token, the lexer should be switched into the state `targetstate` (optional)	`void`
`l.add_actions(def)`	Add all semantic actions provided by this Lexer instance to the token definition container `def`.	`void`

Type Expressions

Expression	Description
`traits::is_lexer<L>::type`	Metafunction that evaluates to `mpl::true_` if a certain type, `L` is a Lexer, `mpl::false_` otherwise (See MPL Boolean Constant).

Postcondition

Upon return from l.collect the following post conditions should hold:

On return, def holds all token definitions defined in the Lexer, l. This includes all Lexer's contained inside l.

Upon return from l.add_actions the following post conditions should hold:

On return, def holds all semantic actions correctly associated with the corresponding token definitions as defined in the Lexer, l. This includes all semantic actions defined by the Lexer's contained inside l.

Models

All lexer components in Spirit.Lex are models of the Lexer concept.