"Histone" - a new open-source cross-platform template engine


    MegaFon is a dynamic technology company whose work is not limited solely to the provision of communication services. For example, among our assets there are a large number of Internet sites where customers receive various information and services. As part of the project to improve and develop our web projects, we, among other things, create products that can be useful to other developers. Today we would like to introduce the first of them - the Histone template engine, which is an open source project distributed under the Apache Software License 2.0. But first things first.

    What is Histone?

    Many of you use various template engines when creating web applications. The situation is when from the data presented in a certain format, you need to generate some HTML-code using a template that defines the rules for the conversion, is ubiquitous. Several years ago, template engines were actively used to generate HTML code on the server ( Smarty , FreeMarker , Velocity ), today more and more often there is a need to generate HTML code directly in the browser. Examples of such template engines include TrimPath templates , Mustache , Google Closure Templates, and so on.

    With all the variety of solutions created for working on different platforms, there are not many template engines that could be run both on the server and in a web browser. Each of them has both its advantages and significant disadvantages. Some of these disadvantages, in our opinion, are critical, which does not allow us to fully enjoy all the advantages of the template engine that works both on the server and in the web browser.

    The Histone project is an attempt to create a new open-source cross-platform template engine in implementations for Java, JavaScript and PHP. Before starting to develop our own solution, we studied the advantages and disadvantages of existing cross-platform template engines. We analyzed only popular template engines that are widely used to build modern web applications, and did not consider solutions that do not have at least two of the three implementations of interest to us (Java, JavaScript, PHP).
    Google closure templates- The source code of the template is compiled in JavaScript, which speeds up its processing.- To use the template, it must be pre-processed using a special utility. This greatly complicates the development of browser-based applications, since any changes to the template must be recompiled.

    - The syntax of the templates is too complicated, which will be understood by programmers, but will inevitably cause difficulties for template designers.

    - A complex mechanism for creating your own extensions. It does not allow to increase the functionality of the JavaScript version, which makes the extension mechanism inapplicable in the case of cross-platform use of templates.

    - No PHP version
    Mustache- Exists in all possible variations for many programming languages.- Unconventional syntax difficult for designers to understand.

    - The template language lacks a number of important designs, which makes the use of this template engine impossible (at least non-trivial) in the conditions of some projects.

    - The extension mechanism is very poorly documented.

    After analyzing existing products, our team decided to create a new template engine that would have the best sides of the above products, as well as correct those disadvantages and shortcomings that are critical for us. As a result, a list of the main ideas that formed the basis of the development was formed:

    • One template - many implementations of the template engine. The syntax of the template is independent of the programming language that is used to process it. This is necessary so that the same templates can be processed both on the server and in the browser and get the same result.
    • Traditional and intuitive syntax. Constructions processed by the template engine are placed between the delimiters {{and}}, everything else falls into the result of processing the template “as is”.
    • Inability to execute code directly from the template. Many template engines can execute the code of the corresponding programming language directly from the template. This does not allow the use of such a template with code inserts of a specific language in implementations in other languages. Therefore, such a possibility in Histone is completely excluded.
    • Easy to expand. Regardless of the programming language used to process the template, the provided API allows you to add a new function or property to use it in the template in order to meet the needs of any project.
    • Ability to use a single data source for templates processed on different platforms. The universal data format for which Histone has everything necessary to support is JSON. Regardless of which set of data types the implementation language has, the template language allows you to work with a subset of the data types defined for the JSON format.
    • Providing the template with data about the environment in which it is running. The template is provided with specific information about the platform on which it is processed. For example, in the case of processing in the browser, information about the screen resolution or browser window size is available to the template, and in the case of processing on the server, the template can receive data about the operating system or server request headers.
    • Cross-platform and easy porting. Implementations for JavaScript, Java, and PHP are available out of the box. Refusal to use specific external libraries along with a logical organization of the code allows porting the implementation of Histone to any other programming language, for example, C ++, Ruby, Objective-C, ActionScript, etc.
    • Performance. Unlike Google Closure Templates, Histone does not convert the template code into the source code of the programming language in which it is planned to be processed. Instead, it is possible with the help of a special utility included in the product to compile the source code of the template into a form that will be executed bypassing the stage of its parsing and parsing (the most expensive operation). In addition, at this stage, those parts of the template are executed that are independent of functions, properties, and variables defined from the outside.


    As part of the template engine, three interrelated components can be distinguished:

    • Lexical analyzer. Parses the template string into tokens (sequences of characters of the original string, combined into groups). Created using the regex engine built into the implementation language.
    • Syntactical analyzer. It parses a pattern string, the result of which is an abstract syntax tree that reflects the syntactic structure of the input sequence. Created using the recursive descent algorithm.
    • Interpreter. Executes the instructions of the abstract syntax tree, during which the result of processing the template is generated. The most important part of the template engine, which performs arithmetic operations, function calls, reading properties, etc.

    Obviously, the use of a lexical analyzer based on regular expressions and a recursive descent parser is not the most effective approach to building such systems, but this is justified by the fact that our primary task was to ensure the maximum speed of the template interpretation process. Since in real conditions the work with the source code of the templates will occur only during the development of the project, while the web application in public access can use abstract syntax trees of the templates instead of their source code, thereby completely eliminating lexical analysis and parsing.

    Template Syntax

    A template is a text document with inserts of special template tags whose contents will be executed by the template engine during template processing. Template tags are constructions of a template language enclosed between {{and}} characters. Fragments of a text document outside these characters will be transferred to the result of processing the template unchanged.

    Syntax Basics
    Comments{{* a comment *}}
    Processing prevention{{% {{2 * 2}}%}}
    Display property or variable values{{myValue}}
    {{myObject ["foo"]. bar}}
    Calling Functions and Macros{{loadText ("file.txt")}}
    {{global.min (10, 20, 30)}}
    {{myMacro ("foo", "bar")}}
    Method call{{"String" .toUpperCase ()}}
    {{123.isNumber ()}}

    Data types
    Value not defined{{undefined}}
    No value{{null}}
    The numbers{{10}}
    Arrays{{[1, 2, 3]}}
    {{[“foo”: “bar”]}}

    Addition{{10 + 2}}
    Subtraction{{10 - 2}}
    Multiplication{{10 * 2}}
    Remainder of the division{{10 mod 2}}
    Unary minus{{-10}}
    Equally{{10 is 10}}
    Not equal{{10 isNot 2}}
    More{{10> 2}}
    Smaller{{2 <10}}
    Greater than or equal{{10> = 2}}
    Less than or equal{{2 <= 10}}
    Logical "OR"{{true or false}}
    Logical "And"{{true and false}}
    Logical "NOT"{{not true}}
    Ternary operator{{true? true: false}}

    Variable declaration{{var myVar = 10}}

    {{var myVar}}
    {{/ var}}
    Condition{{if true}}
    {{elseif 42}}
    {{/ if}}
    Cycle{{for value in values}}
    {{/ for}}

    {{for key: value in values}}
    {{/ for}}
    Macros{{macro myMacro (arg1, arg2)}}
    {{/ macro}}
    Import Template{{import "common.tpl"}}
    Function or macro call{{call myProc ("arg1", "arg2")}}
    {{/ call}}


    In this article, we tried to briefly describe the main features of the new open source cross-platform template engine. You have become familiar with the internal structure and basic syntax of templates. If you are interested in a detailed description of the technologies on which Histone works, we will be able to tell you this in the next post, in which we will examine how the lexical analyzer we use works.

    You can take an active part in the development of the project and apply it in your resources. Our team will be happy to answer all your questions.


    Also popular now: