Babel and handlebars, time to make friends

I think many people know such a package as Babel, or PyBabel.
A great package for localization, which is based on gettext, like everything else (at least known to me) in the modern world.

There was a task to prepare the site for future localization, all templates are made in Handlebars, this template engine is first and foremost a reference point that you need to make friends with it. The question is who.

I must make a reservation in advance that I had no restrictions on the choice of technologies.
We ultimately use the full set for the static build - ruby ​​(compass), node (coffee, grunt, requirejs), python (backend and the basis of everything and everything), shell scripts, in general, there is no limit.

By the way, if anyone is interested I can describe in detail about the build of a separate post, there requirejs + scss + all of the above, at the moment there are about 1000+ files included in the build + deployment on heroku and non-heroku with one button. In my opinion, the whole process is interesting, but I don’t know what to focus on.

Ie, in fact, all that needed to be found was - who is able to:
a) Follow the patterns of handlebars
b) Be able to work on the same principle as Babel - t. The string is the key, not the constant, when everything is stored in separate files.
c) Prepare a .po file from the lines found.

Then the hands are already untied and there are enough possibilities.

Having spent several hours searching for a finding was disappointing, no one did this for some reason.
I categorically did not want to believe in it, but everything that existed was “not that” and “not so.”

What did existing solutions do and know how?
They with varying success pulled out regeksami lines. They didn’t know how to work with ngettext, or didn’t know how to work according to the principle “string = key”, but only constants.

All this was necessary to eliminate.
The task was immediately broken into points, just take a moment and describe the entire chain that I wanted to receive.

1) Define the text of the translation in the templates, the principle of "string = key"
So much more convenient to work than when the key = the name of a constant. The price of the error is also less, the maximum is whoever will receive the text in English, instead of translation

2) Send the text to babel, as well as the line number
This is necessary, because when babel updates will not be able to determine what has changed, if it will not have line numbers
3) Build rows of .po files, in fact it is the task babel -a
4) From .po files after the translation to make .json files and transfer to someone already on the client side. Conversion - po2json task
5) Make helper-s for handlebars, both single-line and block with ngettext support
Again, diversity for the sake of everyday convenience
6) Both options should be able to work with parameters
7) Both options should send text for translation “to the top ", To someone, who holds those .json files
8) The same someone should substitute the parameters obtained from the template into the final line.

At the 8th point, Jed was chosen , an excellent library with built-in sprintf, which immediately solved the problem of convenient parameter transfer.

The first point was the key, it was impossible to spy it anywhere.
For a long time I considered the regexps and pretended my own - I did not like everything. Crooked, not enough, unreliable.
I considered template code compiled into javascript even longer. From there it was easy to get the lines.
But the very idea of ​​this approach was terrifying.
In addition - it did not solve the problem that you need to know where the line was met in the original template.

I had to look at the source files of handlebars, and here came an insight - his lexer was written with the help of jison and the parser jisona, using the vocabulary directly of the handlebars, sewn inside.
And by calling the Handlebars.parse (template) method, you can get a JSON template structure.
It would seem that everything is great - but there are no line numbers.

But happiness was close and it was necessary to dig in the same direction - but already in the assembled handlebars, in the code of the generated parser, it remained to find the right place to write two lines and voila, the output is the template structure, where for each block the initial line and the final line are indicated. All this is in the parser itself, it simply did not pass out at the right time in the right place.

Then there was the matter of technology, to put everything together, to tie in to babel as an extractor.
A function in python (extractor) that calls the script.loads the patched handlebars.js node.js.
Further, this script recursively goes through the structure, collects the string and returns to the extractor in the desired format. The whole logic of the babel remains unchanged. The string extraction logic is also native to handlebars.js. All this can not but rejoice.

By itself, this post would not make sense if there were no subsequent lines. I think that at least the time spent searching for the right direction for exporting rows will save someone.
- pip install pybabel-hbs

The package is installed along with patched handlebars, and the source code contains examples of the implementation of helper (though on coffee)
The very use in templates turned out to be ideal for me personally, 4 helpers on githaba, the thickest here, for clarity

{{#ntrans num_to_check_aganst param_1="something" num=num_to_check_against}}
    Some text to be translated with %(param_1)s and %(num)s
    Some plural text to be translated with %(param_1)s and %(num)s

In the config babel, you need to add

[hbs: path/to/project/**.hbs] 

For work it is necessary that node.js be in the environment.
The plans include optimization so that node.js does not start for each file separately each time, but once for the entire lifetime of the main babel process.
This is implemented in version 0.2.0, while the post was moderated.
By itself, the acceleration turned out to be quite good, a pack of 150 templates began to be processed in ~ a minute.
Before that, it took a couple of minutes, which was completely unacceptable.
But a minute for such a cola too much.

Essentially (another 4 times), it turned out to be optimized due to the fact that initially transferred data to node.js from python via pexpect, now (version 0.2.1) I transfer only the file name and already node.js reads it. (here of course the minus is that both open the file in turn)
Now processing 150 files takes less than 15 seconds. There is still room for speeding up the process, but for the time being it suits me.
Given the fact that you often do not need to run - good enough result.

- The moral of this story is that, before spending hours writing and testing regular expressions, looking for devious ways to parse any language - it’s worth seeing how it parses itself. Maybe it's easier.
Ultimately, the whole problem was solved by several lines inscribed in handlebars.js itself, and the fact that today is offered on the Internet is ... at least not convenient.

Also popular now: