Parser¶
What does this do?¶
The Parser takes the array (list) of tokens generated by the Lexer and turns it into an Abstract Syntax Tree (AST). An AST is a visual representation of how code generation might be completed. This means it takes the ordered list of tokens and turns it into something a Generator can understand and create code out of.
Input & Output¶
The input is a list of tokens generated by the Lexer. These tokens can also be displayed in a human-friendly format of a table. The output is an Abstract Syntax Tree (AST), using Python classes. The following is an example of input and output of the Parser.
Input:
| Token ID | Token Value | Position |
|---|---|---|
| WORD | 1, 1 | |
| FUNC | > | 1, 6 |
| STRING_DEF | “ | 1, 7 |
| WORD | Somewhere | 1, 8 |
| SPACE | 1, 16 | |
| WORD | over | 1, 17 |
| SPACE | 1, 21 | |
| WORD | the | 1 22 |
| SPACE | 1, 25 | |
| WORD | rainbow! | 1 26 |
| END_STRING_DEF | “ | 1, 34 |
| ENDLINE | ; | 1, 35 |
Output:
<ASTNode(l) FUNCTION: [
<ASTValue NAME: "print">,
<ASTNode(1) ARGUMENTS: [
<ASTValue STRING: "Somewhere over the rainbow!">
]>
]>