Mini project for compiler course

• 📦 INT3402
  • Table of Contents
  • 📄 Lexical Analyzer
    • 📜 Introduction
    • 🔧 Prerequisites
    • 🗄️ Data File
    • ⚙️ Run
  • 📄 Parser
    • Introduction
    • 🔧 Prerequisites
    • 🗄️ Data File
    • ⚙️ Run

This is a lexical analyzer for a subset of C language (VC) implemented using Python 3. The lexical analyzer is able to recognize tokens, comments and throw errors for invalid tokens. The language definition is defined in VC Language Definition.

• Python 3.10 (you can also try other versions, it might work).
• A data file containing information about the Deterministic finite automata (DFA) used in the lexical analyzer, the format of the file is defined in 🗄️ Data File.
• A source code file written in VC language.

• The data file is in json format but the extension is .dat, there's a sample data file in the root directory of this project. The data file contains the following fields:
  • keywords: a list of keywords in target the language.
  • special_literals: a list of special literals used in target the language.
  • separators: a list of characters used to separate tokens in target the language.
  • terminal_types: a list of types of tokens in target the language.
  • nodes: a list of nodes in the DFA, this includes:
    • the key of each node is the name of the node.
    • children is a list of children of the node, each child is a map from a list of characters to the name of the child node
    • if the node is the starting node, it will include a field start with value true.
    • if the node is terminal, it will include a field terminal with value true and a field terminal_type with the type of the token from terminal_types, else it will have a field terminal with value false.
• There is also a sample source file for you to use in the root directory of this project.

To run the lexical analyzer, run the following command in the terminal (default value for data_file is dfa.dat):

python lexer.py <source_code_file> [data_file]

To see more information about the command, run the following command in the terminal:

python lexer.py -h

You can also run this online on Repl.it.

This is a parser for a subset of C language (VC) implemented using Python 3. The parser is able to recognize the syntax of the source code and throw errors for invalid syntax. The language definition is defined in VC Language Definition.

The same as 📄 Lexical Analyzer.

• The data file is in a human-readable format with .dat extension.
• The first line in the data file is a list of terminal types in the language (if that terminal type is used directly in the grammar). The terminal types mentioned here may be LITERAL or IDENTIFIER in the language. The terminal types are separated by a vertical bar |. Caution: terminal types are not terminal symbols, they represent a group of terminal symbols.
• The grammar is written from the second line. Each line represents a production rule. The production rule is written in the following format: non-terminal -> production. The non-terminal is the left-hand side of the production rule, and the production is the right-hand side of the production rule. The production is a list of terminal types and non-terminals separated by a space. If there are multiple productions for a non-terminal, they are separated by a vertical bar |. Each symbol in a production is separated by a space. - The first non-terminal in the grammar is the start symbol.

To run the parser, run the following command in the terminal (default value for parser_data is grammar.dat, default value for lexer_data is dfa.dat):

python vcparser.py <source_code_file> [lexer_data] [parser_data]

To see more information about the command, run the following command in the terminal:

python vcparser.py -h

You can also run this online on Repl.it.