A simple, but customizable lexer, used for the three stage interpreter pipeline

• Lexer: public interface to the end user. This class will call other composite classes (for example call LexeralAnalysis)
• LexeralAnalysis: performs the lexeral analysis.
• TokenData: data container for a token
• Token: typedef shared pointer of TokenData
• TokenList: Container for tokens, inherits std::vector.
• TokenType: Types of tokens available (such as operators, newlines, function indentifiers, etc.)

As the lexer needs input to correctly parse tokens, the language characteristics need to be defined. These characteristics are defined in the LanguageDefinition.h header file.