Listener vs Visitor battle at antlr4 stadium

Visit or listen? A matter of taste - no more. Or not?
Background .

Having examined the source text, a tree was formed at the output:



The tree itself does not have any sense, it is “Wooden”, the result (analysis) of this tree has meaning and any value. For those who are not ready to strain and write self-made sledges for descending from a tree (for example, me), the ability to get an analyzer is almost free for antlr4.

1. Visitor

Classic is a behavioral design pattern. When traversing nodes, a method is determined that processes the current type of node, after which the method is called, and here specifically development begins, namely, analysis of the incoming subtree.

2. Listener

The innovation that appeared in the fourth version. The behavior of this class is far from classical (Observer or Publish / Subscribe). In the classic version, there is a manager who notifies subscribers about the occurrence of events. The behavior of the listener in question is more like the work of an inspector. Before checking the node, the inspector makes a note “I am checking the X node”, then the descendants of the node are crawled, after the crawl, which can be done “Conclusion on the results of the crawl of the X node”.

Practice

For a better understanding of what is happening, we turn to the author of the recognizer pragprog.com/book/tpantlr2/the-definitive-antlr-4-reference . In the book “The Definitive ANTLR 4 Reference” in Section 8.2 Translating JSON to XML, the author translates using a Listener.

The examples in the book are based on JAVA, I am not familiar with JAVA, but they are translated into C # without pain (this is what best practice cloning means).

To prepare the listener, we need VS, C # proj and JASON.g4 with approximately the same content

grammar JASON;
json: object
	| array
	;
object
	: '{' pair ( ',' pair )* '}'	# AnObject
	| '{' '}'						# EmptyObject
	;
	pair: STRING':'value;
array
	: '['value(','value)*']'	# ArrayOfValues
	| '['']'					# EmptyArray
	;
value
	: STRING	# String
	| NUMBER	# Atom
	| object	# ObjectValue
	| array	# ArrayValue
	| 'true'	# Atom
	| 'false'	# Atom
	| 'null'	# Atom
	;
STRING
	: '"' ( ESC | ~["\\] )* '"';
	fragment ESC: '\\'(["\\/bfnrt]|UNICODE);
	fragment UNICODE:'u'HEX HEX HEX HEX;
	fragment HEX:[0-9a-fA-F];
NUMBER
	: '-'? INT'.'INT EXP? //1.35,1.35E-9,0.3,-4.5
	| '-'? INT EXP //1e10-3e4
	| '-'? INT //-3,45
	;
	fragment INT: '0'|[1-9][0-9]*;//noleadingzeros
	fragment EXP: [Ee][+\-]? INT;//\-since-means"range"inside[...]
WS : [\t\n\r]+->skip;

This is a grammar to recognize JASON. In the file properties you need to set Generate Listener and Generate Visitor (this is still useful). The result of the Listener's work in the original example from the book is xml text, this does not suit me, I will get XElement (anyway, the xml text will need to be translated into something, although the plus of the text is that it does not clamp into the framework of using specific classes).

The algorithm is simple: maybe antlr4 uses downward parsing (in our case, from the root to the nodes), then the xml will be formed in the same way, creating an ancestor element to which descendants will be added.

Listener Example:

      class XmlListener : JASONBaseListener
	{
		#region fields
		ParseTreeProperty xml = new ParseTreeProperty();
		#endregion
		#region result
		public XElement Root { get; private set; }
		#endregion
		#region xml api
		XElement GetXml( IParseTree ctx )
		{
			return xml.Get( ctx );
		}
		/// 

		/// поиск родительского xml элемента
		/// 
		XElement GetParentXml( IParseTree ctx )
		{
			var parent = ctx.Parent;
			XElement result = GetXml( parent );
			if ( result == null )
				result = GetParentXml( parent );
			return result;
		}
		void SetXml( IParseTree ctx, XElement e )
		{
			xml.Put( ctx, e );
		}
		#endregion
		#region listener
		public override void ExitString( JASONParser.StringContext context )
		{
			var value = GetStringValue( context.STRING() );
			AddValue( context, value );
		}
		public override void ExitAtom( JASONParser.AtomContext context )
		{
			var value = context.GetText();
			AddValue( context, value );
		}
		public override void EnterPair( JASONParser.PairContext context )
		{
			var name = GetStringValue( context.STRING() );
			XElement element = new XElement( name );
			XElement ParentElement = GetParentXml( context );
			ParentElement.Add( element );
			SetXml( context, element );
		}
		public override void EnterJson( JASONParser.JsonContext context )
		{
			Root = new XElement( "JSON" );
			SetXml( context, Root);
		}
		#endregion
		#region private
		private string GetStringValue( ITerminalNode terminal )
		{
			return terminal.GetText().Trim( '"' );
		}
		private void AddValue( ANTLR_CSV.JASONParser.ValueContext context, string value )
		{
			var parent = GetParentXml( context );
			if ( context.Parent.RuleIndex == JASONParser.RULE_array )
			{
				XElement element = new XElement( "elemnt" );
				element.Value = value;
				parent.Add( element );
				SetXml( context, element );
			}
			else
				parent.Value = value;
		}
		#endregion
	}

EnterJson corresponds to the entry into the node described in the grammar like this:

json: object
	| array
	;

ExitString corresponds to the exit from the node described in the grammar like this:

STRING	# String

Unlike the original example, I do not use all the charms of Enter and Exit. That is, ParseTreeProperty recognized to store pairs [subtree, value], it is probably better to replace it with a regular dictionary (it definitely won’t be worse).

Visitor Example:

class XmlVisitor : JASONBaseVisitor
	{
		#region fields
		private XElement _result;
		ParseTreeProperty xml = new ParseTreeProperty();
		#endregion
		#region xml api
		XElement GetXml( IParseTree ctx )
		{
			return xml.Get( ctx );
		}
		XElement GetParentXml( IParseTree ctx )
		{
			var parent = ctx.Parent;
			XElement result = GetXml( parent );
			if ( result == null )
				result = GetParentXml( parent );
			return result;
		}
		void SetXml( IParseTree ctx, XElement e )
		{
			xml.Put( ctx, e );
		}
		#endregion
		#region visitor
		/// 

		/// значение по умолчанию - создаваемое дерево xml
		/// 
		protected override XElement DefaultResult { get { return _result; } }
		public override XElement VisitJson( JASONParser.JsonContext context )
		{
			_result = new XElement( "JSON" );
			SetXml( context, _result );
			return VisitChildren( context );
		}
		public override XElement VisitString( JASONParser.StringContext context )
		{
			var value = GetStringValue( context.STRING() );
			AddValue( context, value );
			return DefaultResult;
		}
		public override XElement VisitAtom( JASONParser.AtomContext context )
		{
			var value = context.GetText();
			AddValue( context, value );
			return DefaultResult;
		}
		public override XElement VisitPair( JASONParser.PairContext context )
		{
			var name = GetStringValue( context.STRING() );
			XElement element = new XElement( name );
			XElement ParentElement = GetParentXml( context );
			ParentElement.Add( element );
			SetXml( context, element );
			return VisitChildren( context );
		}
		#endregion
		#region private
		private string GetStringValue( ITerminalNode terminal )
		{
			return terminal.GetText().Trim( '"' );
		}
		private void AddValue( ANTLR_CSV.JASONParser.ValueContext context, string value )
		{
			var parent = GetParentXml( context );
			if ( context.Parent.RuleIndex == JASONParser.RULE_array )
			{
				XElement element = new XElement( "elemnt" );
				element.Value = value;
				parent.Add( element );
				SetXml( context, element );
			}
			else
				parent.Value = value;
		}
		#endregion
	}

As the saying goes, “Find 10 differences,” the first difference is VisitJson, managing visits without calling VisitChildren (context), visiting descendants ceases, and therefore a detour. Each of the visiting methods should return a value, that is, there is always the result of the visit, and this is convenient:

var result = visitor.Visit( tree ); 

When working with a listener:

walker.Walk( listener, tree );
var result = listener.Root;

In the original example, without a Listener it would be rather tight, there is not much difference for this solution, but I vote in favor of the solution on the Visitor.

Well, so that you could try it yourself:

    private static IParseTree CreateTree()
		{
			StringBuilder sb = new StringBuilder();
			sb.AppendLine( "{" );
			sb.AppendLine( "\"description\":\"Animaginary server config file\"," );
			sb.AppendLine( "\"count\":500," );
			sb.AppendLine( "\"logs\":{\"level\":\"verbose\",\"dir\":\"/var/log\"}," );
			sb.AppendLine( "\"host\":\"antlr.org\"," );
			sb.AppendLine( "\"admin\":[\"parrt\",\"tombu\"]," );
			sb.AppendLine( "\"aliases\":[]" );
			sb.AppendLine( "}" );
			AntlrInputStream input = new AntlrInputStream( sb.ToString() );
			JASONLexer lexer = new JASONLexer( input );
			CommonTokenStream tokens = new CommonTokenStream( lexer );
			JASONParser parser = new JASONParser( tokens );
			IParseTree tree = parser.json();
			return tree;
		}

An example of JSON text is taken almost without changes from the original example:

    private static void ListenerXml()
		{
			IParseTree tree = CreateTree();
			ParseTreeWalker walker = new ParseTreeWalker();
			XmlListener listener = new XmlListener();
			walker.Walk( listener, tree );
			var result = listener.Root;
		}
     private static void VisitorXml()
		{
			IParseTree tree = CreateTree();
			XmlVisitor visitor = new XmlVisitor();
			var result = visitor.Visit( tree );
		}

Well, the result of the execution:

Animaginary server config file500verbose/var/logantlr.orgparrttombu

Oddly enough, but both methods produced the same thing.

Next

PS Listener vs Visitor - 0: 1