tokenizes the entire program line by line More...
#include <lexer.h>
Public Member Functions | |
lexer (const std::string &textfile, const std::string &filepath, const int &linenumber) | |
constructor | |
std::vector< token > | __RUN__LEXER___ () |
tokenizes the program | |
Private Member Functions | |
void | advance () |
advances the lexer to the next character to tokenize in the textfile being read | |
token | generate_identifier () |
generates an identifier or keyword based on the sequence of characters being read | |
void | generate_tokens () |
matches the current character to it's corresponding token type | |
token | generate_number () |
generates a number data type | |
token | generate_string () |
generates a string data type | |
Private Attributes | |
std::string | textFile |
file data in string format | |
std::string | filePath |
path of the file | |
std::vector< token > | tokenObjVector |
stores the tokens of the current line in the program | |
int | lnNum |
current line number in the file | |
std::string | current_char |
the current char in the textfile being read | |
int | current_index |
the current index of the current char in the textfile being read | |
tokenizes the entire program line by line