Yate: Yandex.Mail has switched to a new template engine

Some time ago we wrote that a new interface appeared in Yandex.Mail that uses data template in the browser. Few large services dared to do this, but even now we consider such a solution to be the most successful. It not only accelerated the operation of the interface, but also allows you to save user traffic and more efficiently spend server processor time.

Then we used XSL as a template engine, and the data was transmitted in XML format. Having transferred the project to a new interface, we began to look for other ways to speed up the Yandex.Mail interface.

Recently, we transferred all Mail to the JS template engine and JSON data.

Reasons for abandoning XML / XSL
We love XSL very much, it is expressive, it is really convenient to write templates for large services with complex connections, a large number of reusable blocks and the ability to redefine or redefine blocks. But he also has a number of significant drawbacks:

• It doesn’t develop.
  If you look at the browser crackers, the last mention of XSL mentioned by the developers dates back to 2004 (the problem in Chrome in the graphs below is especially obvious).
  
• A large number of problems in implementations for various browsers
  First, implementations, as always, differ in IE and not in IE. There are a couple of very unpleasant problems in the Opera, such as ignoring the first xsl:call-templateand complex predicates, and sometimes loading the image before transformation. Chrome requires different from other browsers xsl:output.
• Difficult to debug
  The most popular way to diagnose problems is to comment out pieces of the template.
• It is difficult to profile the
  XSL processor in the browser - a black box, and talking about which designs work faster and which ones slower is possible only on the basis of indirect judgments obtained on synthetic tests .
• It is difficult to expand.
  If you use xslt on the server, then you can still connect exslt , but in the browser you have to get a little tricky with standard tools in order to process strings in a non-standard way.
• It is difficult to process XML data on the server.
  If you need to change the output before entering the client, you have to use E4X - yes, we have spidermonkey . E4X is actually not very convenient and certainly not very fast compared to the JSON modification.

Choosing a new template engine
We tried different template engines (both in semantics and syntax), including: yate , handlebars , jade , dust and ajaxslt .

Having json output and all the necessary data to form a list of letters, we tried to use each of the above template engines.

Since the page with the list of letters uses the largest number of all available mail components (folders, tags, collectors, filters, threads, and the letters themselves), we were immediately able to check each template engine under the maximum load on the speed of work / rendering. We also took into account the convenience of development and the simplicity of code support in the future.

Almost immediately, it became obvious that imperative template engines (handlebars, jade, dust) could not be compared with declarative ones in terms of ease of development, their code turned into a mess, which was more and more difficult to understand. In addition, we are accustomed to the flexibility of XSL and a strong distinction between data and display logic, so it was more convenient for us to work with templating engines of similar semantics.

Thus, yate and ajaxslt were in the final. But, despite all the efforts of our developers, ajaxslt could not get closer to yate's performance and required more than 500 ms for conversion.

The final test for the speed of rendering a list of 200 letters looked like this:



Yate
Yate- A project created and developed by the front-end architect of the Post. This template engine has already successfully established itself on several ready-made, but not yet submitted Yandex projects. Yate is very similar to XSL: with a lighter and js-like syntax, it uses the same paradigms (match, apply), predicates are supported, and there is also jpath - an analog of XPath for navigating through JSON. Templates are compiled into regular javascript and can be used both on the client and on the server.

This is what the same template looks like on XSL and on yate:



The speed of the interface is of great importance to us, so we were pleased when yate showed very good results of the speed of rendering templates - it turned out that the processor implemented in JavaScript runs faster than the XSL processor built into the browser. In addition, given that with each new browser version, JS is becoming faster and more functional, the gap will only widen.

Just starting to use yate, we won in the speed of transformation of 40% or more, depending on the browser. In the case of Chrome, the gain was ... 1200%!

But it seemed to us that this is not enough.
After all, a compiled template is a regular JS-code, which is easy to analyze with a profiler available in any browser:



And this means that it can be investigated and optimized, thereby achieving acceleration of its work.

As a result, we played back a couple of milliseconds, finding and optimizing the heaviest templates, and also looked at what operations in the compiled template are performed most often, and tweaked the compiler so that it immediately generates the code optimal for such operations:



It’s worth noting that we are doing all the research was performed on the maximum number of messages per page - 200. By default, users 30 letters on the page and in such volumes standardization runs even faster:



it is important to simplify and debugging - if something went wrong, dostat but inserted debuggerin the compiled template and see what the problem is. You can also use the built-in functionlogthat displays data on which the template is superimposed into the browser console.

Another important parameter is the amount of data that the user loads. In yate, it is possible to break patterns into modules and specify dependencies between them. Thus, modules can be loaded as needed, and during transformation, indicate which module is needed to generate a particular piece of HTML. It is also easy to add functions to yate, for example, for working with strings or dates. To do this, it is enough to declare the function, implement it in JavaScript and use: Result Yandex.Mail was transferred to the new template engine in a month and a half.









Some services used in Mail have not yet moved to JSON, so for them we use the XML4JSON E4X processor. Since serious architectural changes took place, the Disk, which has remained on XSL / XML, now opens only with page reloading. But we have achieved our goal. The measurements showed that we not only didn’t “sag” in the time of loading the Mail:



But we also significantly reduced the time for going through the pages inside the Mail:



We also received a flexible, expandable and controlled by us solution, which we intend to develop and improve.

DIY
We will definitely write how to use yate in your projects. And if you want to try yate now, all you have to do is npm install yate.

You can use as documentationpresentation or wiki about syntax and jpath . You may also need an XPath analogue for JSON ( jpath or no.path ) and syntax highlighting for vim or Sublime Text . If you already have XSL templates, then you can evaluate how yate will look with xsl2yate .