Building a language parser using ANTLR4 and ANTLRWorks 2

  • First, get ANTLR4 and ANTLRWorks 2 from their respective sites:  http://www.antlr.org/download.html and http://tunnelvisionlabs.com/products/demo/antlrworks.
  • Next, design a grammar. The name of the language must be saved as the filename, ending with the extension g4. For example, if the language is Java, the file should be named Java.g4. Once you have a grammar designed, start up ANTLRWorks 2.
  • For the Windows platform, this means running antlrworks264.exe or antlrworks2.exe from the bin directory after unzipping ANTLRWorks 2.
  • Test the grammar - load up ANTLRWorks 2, and load your grammar.
  • Generate the recognizer using Run -> Generate Recognizer. Note that you must first select the file.
  • After the recognizer is output to a directory, all the files can be loaded into an IDE. In addition, the ANTLR runtime jar must be added as a reference of the project.
  • Once the recognizer is generated, to test it in a Java IDE, using Java as the grammar, and Java as the target language, a code example follows. This assumes that ExtractInterfaceListener extends/inherits from JavaBaseListener, override the methods interited from the JavaBaseListener class in order to do your parsing, or other stuff. In my example below, I created an additional constructor (or added a parameter to the existing constructor) for ExtractInterfaceListener to put in the name of a file I wanted to extract data to.
  • To parse a file, merely follow the example below. To parse a string, pass the string as a parameter into ANTLRInputStream and invoke the appropriate rule. For example, if the parameter is a string literal, and you want it tested/parsed as a literal, replace the call to parser.compilationUnit() as a call to parser.literal() instead;
package cx.ath.journeyman.JavaTranslator;

import java.io.FileInputStream;
import java.io.IOException;

import org.antlr.v4.runtime.ANTLRInputStream;
import org.antlr.v4.runtime.CommonTokenStream;
import org.antlr.v4.runtime.tree.ParseTree;
import org.antlr.v4.runtime.tree.ParseTreeWalker;

import cx.ath.journeyman.JavaTranslator.generated.JavaLexer;
import cx.ath.journeyman.JavaTranslator.generated.JavaParser;

public class JavaTranslator {

    /**
     * @param args
     */
    public static void main(String[] args) {
        String Directory = "C:\\temp\\";
        String filename = "Main.java";
        String fullFilename = Directory + filename;
        try {
            FileInputStream fis = new FileInputStream(fullFilename);
            ANTLRInputStream input = new ANTLRInputStream(fis);
            JavaLexer lexer = new JavaLexer(input);
            CommonTokenStream tokens = new CommonTokenStream(lexer);
            JavaParser parser = new JavaParser(tokens);
            ParseTree tree = parser.compilationUnit(); // parse
            System.out.println(tree.toStringTree());
            ParseTreeWalker walker = new ParseTreeWalker(); // create standard
                                                            // walker
            ExtractInterfaceListener extractor = new ExtractInterfaceListener(
                    parser, "c:\\temp\\" + filename);
            walker.walk(extractor, tree); // initiate walk of tree with listener
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }

}
Published Mon, 3 Jun 2013 @ 9:09 PM by chuacw
Related articles: , ,

Leave a Comment

(required) 
(required) 
(optional)
(required) 
Enter the following code to ensure that your comment reaches the intended party:
Enter the numbers you see in the image: