Antlr4 walkthrough with Maven project for Java via Intellij Idea

ANTLR is a parser generator that allows you to create a parser for grammar description in one of the main programming languages. It is written in java itself and works great with Java.

Walkthrough:

1) Install Oracle Java JDK and Intellij Idea, (you can skip this step if they are already installed), and run Intellij Idea

2) File-Setting-Plugins

Enter ANTLR in the search field and install the ANTLR v4 grammar plugin. You may need additional search across all repositories.

3) For Maven project add to pom.xml or create a new project.
in dependencies

org.antlrantlr4-runtime4.7

and in plugins

org.antlrantlr4-maven-plugin4.7antlr4

Details https://github.com/antlr/antlr4/blob/master/doc/java-target.md

4) Next, we will create and manually add a grammar file with the extension .g4. The file name must match the word after grammar in the first line. It is composed approximately like this: we take what needs to be parsed and break it into separate tokens. For tokens, we describe tokens, for example, all English letters [a-zA-Z] ;, all numbers [0-9], etc. .. For the example, we took the contents of the example from the official site for the file Hello.g4

// Define a grammar called Hello
grammar Hello;
r  : 'hello' ID ;         // match keyword hello followed by an identifier
ID : [a-z]+ ;             // match lower-case identifiers
WS : [ \t\r\n]+ -> skip ; // skip spaces, tabs, newlines

5) Next, right-click on the second line of the file that starts with r and select the menu item Test Rule r

Grammar checker windows will open below. In this case, the plugin shows an error, most likely due to the fact that this is a test case, but a parser is generated. You can read about it here https://github.com/antlr/antlr4/issues/118 , and for now, ignore it. But in real projects, one should pay more attention to these errors.

6) We click on the grammar file with the right mouse button, select the Configute ANTLR Recoqnizer menu item and generate a parser

After that, a success message will appear in the lower right corner

7) Next, right-click on the file again and select the Configute ANTLR menu item,

and the window for configuring file generation comes out

In this window, enter data about the destination folder and programming language, in our case Java, whether a visitor or listener is needed, as well as other required information, and click OK

And ANTLR then generates files for recognition. However, although the output directory is specified, a new gen folder is often created in the root of the project, and java does not recognize these files.

In order for java to see these files, the folder must either be marked with the right mouse button “Mark Directory As” on “Generated Sources Root” on the gen folder.

And it should turn out like this:

8) ANTLR generated the following classes:

The HelloParser.java class is a description of the parser class, that is, the parser that corresponds to the Hello grammar:

 public class HelloParser extends Parser { ... }

The HelloLexer.java class is a description of the lexer class, or lexical analyzer, corresponding to the HelloInit grammar:

 public class HelloLexer extends Lexer { ... }

Hello.tokens, HelloLexer.tokens are helper classes that contain information about tokens HelloListener.java, HelloBaseListener.java, HelloBaseVisitor, HelloVisitor are classes that contain descriptions of methods that allow you to perform certain actions when traversing the syntax tree

9) After that, add the HelloWalker class (although this class is optional, this code can be changed and added to Main to display information)

  public class HelloWalker extends HelloBaseListener {
    public void enterR(HelloParser.RContext ctx ) {
        System.out.println( "Entering R : " + ctx.ID().getText() );
    }
    public void exitR(HelloParser.RContext ctx ) {
        System.out.println( "Exiting R" );
    }
}

10) And, finally, the Main class - the entry point to the program

  public class Main {
    public static void main( String[] args) throws Exception
    {
       HelloLexer lexer = new HelloLexer(CharStreams.fromString("hello world"));
        CommonTokenStream tokens = new CommonTokenStream(lexer);
        HelloParser parser = new HelloParser(tokens);
        ParseTree tree = parser.r();
        ParseTreeWalker walker = new ParseTreeWalker();
        walker.walk(new HelloWalker(), tree);
    }
}

11) Run the main method, and get the successfully worked out parsing at the output in the console

Entering R : world
Exiting R

→ The project code is posted here