June 16, 2019 at 12:45Application localization and RTL support. Report Yandex.Taxi
When localizing the service, it is important to carefully consider the coordination of transfers between each other. The head of the Yandex.Taxi client Android development team, Alexander Bonel, spoke about the practices and tools that simplify localization. In the second part of the report, Sasha shared the experience of supporting RTL language in the application: what is good and what does not work for Android out of the box, what problems arise due to RTL support and how to minimize them in the future.
- In my report, I want to tell you what basic ideas and practices we use in the development teams of mobile applications Taxis to solve issues related to the localization and updating of translations in our applications. Then I’ll tell you how we implemented support for working in the drawing mode from right to left in the application.
I'll start with localization. First of all, I would like to clarify the terminology. According to my observations, a large number of people believe that localization is limited only to translations, although in fact it also solves the problems associated with formatting numbers, dates, time, working with text and symbols, legal aspects, delivering content to the consumer in such a way that he had semantic value for him and was not perceived ambiguously. And much more.(Communication with the audience about who has how many languages is supported in applications - Ed.)
I want to tell you a story that happened with us in 2014. We then did a major redesign of the Taxi application. In one of the next releases, we gave users the opportunity to indicate the entrance number, as well as look at this information on the trip information screen and, most importantly, on the order rating screen, which we showed to all users, because we definitely wanted to get a rating for the trip .
At that time, we were rather frivolous about the localization process and translations. For the most part, the whole process was manual. Updating the texts in the necessary XML files, adding new translations was carried out manually. And the human factor played a role. The developer determined the next transfer not in the default locale. This led to the fact that for users with a system locale different from the Russian language, if they asked the entrance, the application crashed at the end of the trip. And people could not make another order.
At that time, we had two languages supported in the application: Russian and English. And we worked only in Russia in three cities: Moscow, Yekaterinburg, St. Petersburg.
We currently operate in 16 countries and are translated into 18 languages. I think you understand the whole problem, if we were in that state at that time. We already realized then that something needed to be changed.
In Yandex, most mobile application developers use the service developed internally to solve problems related to localization. It has a very convenient UI interface, it is possible to define your project, set a set of keys, a set of languages into which you need to translate the project. And after the translations appear, they can be uploaded in any convenient format. He also has very good integration with our internal services. There is versioning. But for me personally, as a developer, the most important thing is that he has an API, because this already suggests that all the manual work associated with translations can be automated. Which we did. We have developed a plugin for Gradle. In fact, updating the translation in the application now comes down to the fact that the developer performs a single task:
In a first approximation, he goes to a localization service. Checks for up-to-date translations, uploads them to the project, puts them into the necessary files. But in addition to this, he also verifies the status of translations in the localization service in relation to what is now in the project.
The next time git status is executed, the developer sees in his working directory which files have changed.
He can also look in the generated HTML report about which key for which value in which language has changed.
And he can also see what problems are currently present in the translations of his project.
The plugin is configurable. We can determine the number of languages that our application supports. It is very important if the localizers start a new language, and it inadvertently seeps into the project, having not passed through proofreading and proofreading.
There is also a specific remapping feature. I will talk about it a bit later in more detail when I talk about RTL.
What problems does this plugin analyze now? The problem that we encountered in 2014 is commonplace, lack of translation. Now, in order to add a new line, a new translation to the project, it is not enough for the developer to simply define it in the XML file.
Firstly, if he does this not in the default locale, then his newly wound key will simply disappear during the next synchronization, and he will know about it at the compilation stage.
If he determines the key in the default file, but forgets to do it in the localization service, then the next time updatetranslations is executed, he will receive an error from the plug-in, which will tell him: “This key is in your project, but it is not in the localization service, it is there need to start. "
The second typical problem is the lack of translation. We get a list of links to those keys for which there is no translation, and, as a rule, we refer them to our manager so that they later set up communication with translators. The third point often occurs when they decided to reconsider the use of format arguments in some way, and, for example, instead of an integer format, they began to use the string format.
But at the same time, in some of the translations they forgot to change the number on the line.
And the fourth problem is also caused by the human factor, mainly in translation. If Unicode characters are used in the translation, it is very easy to get confused in the positioning of characters and instead of an inextricable space to get a hyphenation to another line - this is what we are trying to get rid of.
This is all about localization.
Now I want to talk about the experience of implementing support for rendering from right to left in the application using the example of Israel. We launched in 2018 under the brand name Yango in Israel. As it turned out, in Israel people read differently than you and I do. They read from right to left.
Where do you want to start? With Android, in principle, right-to-left rendering support is implemented out of the box quite well.
It starts with the fact that in the application manifest, in the Application element, you need to declare the supportRtl = ”true” attribute. But I want to please you: if you integrate the Facebook SDK for social authorization in your applications, the caring Facebook developers have already done this for you. And if you do not validate the manifest when merging, then this attribute with the value true will fit into your application, and it will already be able to in Rtl. This is something to think about.
I think that most of those sitting in the audience probably have minimal support for Android 4.4. Someone is more fortunate - this is 5.0 and higher. Someone could be less fortunate, and they support 4.0, or, God forbid, 2.3. In this case, you will have big problems, because full-fledged Rtl support in Android has appeared since version 4.2. Therefore, with minSdk 17th, you will simplify your life by an order of magnitude.
Transferring a project to work with Rtl begins with the refactoring tool in Android Studio - Add RTL Support Where Possible. We must pay tribute, it works quite well. It goes through the XML files of your markup, looks at the presence of attributes with left and right values for gravity, for paddings, margins, and replaces them with start and end. If you want to see how your application works by default in right-to-left mode, you don’t need to have content with the so-called RTL strong characters in Hebrew or Arabic. You just need to enable Force RTL Layout Direction in the developer’s settings.
What are the options for customizing the UI and rendering text Android provides for RTL?
The first point is the layoutDirection attribute, which uses the described Force RTL Layout Direction option. She has only four possible meanings. That is, it is by default inherited from the parent. We can say that the layout is drawn based on the selected locale. You can clearly say that it is drawn from right to left or from left to right.
The second element that many people most likely bypass out of habit when they engage in text alignment is textAlignment. Many still continue to use gravity. Do not use gravity to align text, but use textAlignment.
The third attribute is the text rendering direction, textDirection. It has a fairly flexible configuration. It allows you to determine how the text is rendered depending on which first strong character is in the text, whether it is from the Latin strong LTR character set or from Hebrew strong RTL. But if you fully support RTL in the application, you do not need to bypass it, because ignoring such a funny artifact occurs.
Either this is an easter egg, or is it such a side effect in the implementation of editText: your hint at editText starts to disperse. Therefore, in any case, you must set some value for him.
If you need to observe a certain rendering of some graphic resources or markup in RTL mode, then Android provides the ability to set qualifier ldrtl in your daddy into which you can put any resource, and it will be drawn in the RTL rendering mode as you specified.
Sometimes there is a need for runtime to draw the text coming from the backend as is, how it came, ignoring the mode in which your application is currently working. In fact, in text renderer this is done due to the fact that the text is framed by the so-called Bidi Unicode control characters. The logic for working with these characters in itself contains the BidiFormatter class, providing only one unicodeWrap method. It takes CharSequence as input, and returns a string framed by these characters. There is an exhaustive number of them, and, in principle, it should solve most of your problems.
But if there is something so specific, w3.org has a good article on how to use these symbols.
What problems did we get when we started RTL support in our application? First of all, it is a conflict of localization standards.
BCP47 is a newer localization standard, which in particular has changed the codes of some locales. And the fact is that there is a problem with the locale Java class. It works by default in backward compatibility mode, and converts locale codes from BCP to it to ISO. That is, we saw this when the iw code instead of the he code began to go to our backend in the Accept-Language header.
If you remember the slide with the configuration of our plugin, remapping was needed just for this - in order to translate content from he to iw.
If you need to use the BCP format, this is fully implemented in the Apache Cordova project. They have a Globalization class and an implementation of localeToBcp47Language.
Another point that “delivers” with RTL support in the application is animations. Here, the benefit is, in fact, only animations along the X axis are affected, and perhaps the only solution or approach that will help you avoid more problems is the ability to reconsider the absolute animations in the direction of relative animations.
One problem that is perhaps solved only by reflection, because it is rigidly implemented in the component itself is the hint alignment of TextInputLayout. The fact is that if hint is specified in RTL strong characters in Hebrew, it will be clearly aligned to the right, even though the content that you enter in edit text can be from Latin characters. And vice versa. If your hint is in Latin, it will be clearly aligned to the left. This can only be solved with reflection.
Ultimately, after you have implemented RTL in your application, it makes sense to reconsider the severities for RTL-issues that Lint analyzes. There are only four of them. If you do not support RTL in your application, but explicitly use RTL specific attributes somewhere, then this is RtlEnabled (Lint will tell you about this).
If you support RTL in the application, but minSdkVersion of your application is below 17, then you must also continue to use attributes of the left and right type, you cannot use only the start and end attributes in your markup. This is what RtlCompat parses. RtlSymmetry swears when it sees asymmetric paddings. In general, my opinion is that aapt, when processing resources, should crash if it comes across symmetric paddings. If you need some kind of asymmetric accesses, use margins as a full-fledged layout parameter.
Finally, the fourth problem is the hard-coded RTL. This is just that in the attributes only the values of left and right are indicated. Essentially, this negates the refactoring tool Add RTL Support Where Possible. It all comes from Lint on entering RTL. An analysis implementation is also available in AOSP in the RtlDetector class. That is, if imagination allows you, you can come up with something of your own by looking at how the analysis is going on now.
My report is drawing to a close. I want to give a few theses on building the correct localization in our application, which we have come to over time.
First of all, at the start of the project, think about internationalization. I didn’t really mention it. This is not at all the same as localization. In fact, internationalization makes localization possible in the application, in the product. This works out of the box in Java due to the fact that it has full Unicode support, and in Android - due to its rich resource system. Perhaps the only thing that can happen here is our fixation as developers on the theme that "I write the application only for Russian, which means I will only check it with the Russian language." And when you start using the Armenian, Georgian, Hebrew or Arabic languages in the application, the picture looks completely different and breaks down about your usual paradigm.
Second moment. Consider embedding localization in your CI. It is very disappointing to learn about the problems associated with translations during the testing phase of the release candidate, for which you could prepare for several weeks. That is, we are now running this task for every pool request so that we can be sure in advance that everything is fine with translations and that we can freeze the new code into the main branch of the project.
The third point. Respect the work of translators and your budget. First, delete unused resources because people can continue to spend time and money translating strings that you ultimately do not use in your project. Secondly, follow the translations that are in your repository in the localization service, but which are not in your code.
Ideally, if the API of your localization service provides information about when a particular key was started, and you can use heuristics like “If the key was opened more than a month ago,” check its presence in the latest revision of the master brunch. If he is not there, most likely, this is a clear candidate to remove it and so that the localizers do not spend time on it.
A more or less universal candidate for a silver bullet is to store strings on the backend, and not on the client. At the very least, this will give you the opportunity to always keep them relevant on the application side.
We have a very diverse user audience, everyone perceives the world and application content in their own way. Our task as developers is to help keep this perception. My report on this is over. Thanks a lot!
- In my report, I want to tell you what basic ideas and practices we use in the development teams of mobile applications Taxis to solve issues related to the localization and updating of translations in our applications. Then I’ll tell you how we implemented support for working in the drawing mode from right to left in the application.
I'll start with localization. First of all, I would like to clarify the terminology. According to my observations, a large number of people believe that localization is limited only to translations, although in fact it also solves the problems associated with formatting numbers, dates, time, working with text and symbols, legal aspects, delivering content to the consumer in such a way that he had semantic value for him and was not perceived ambiguously. And much more.(Communication with the audience about who has how many languages is supported in applications - Ed.)
I want to tell you a story that happened with us in 2014. We then did a major redesign of the Taxi application. In one of the next releases, we gave users the opportunity to indicate the entrance number, as well as look at this information on the trip information screen and, most importantly, on the order rating screen, which we showed to all users, because we definitely wanted to get a rating for the trip .
At that time, we were rather frivolous about the localization process and translations. For the most part, the whole process was manual. Updating the texts in the necessary XML files, adding new translations was carried out manually. And the human factor played a role. The developer determined the next transfer not in the default locale. This led to the fact that for users with a system locale different from the Russian language, if they asked the entrance, the application crashed at the end of the trip. And people could not make another order.
At that time, we had two languages supported in the application: Russian and English. And we worked only in Russia in three cities: Moscow, Yekaterinburg, St. Petersburg.
We currently operate in 16 countries and are translated into 18 languages. I think you understand the whole problem, if we were in that state at that time. We already realized then that something needed to be changed.
In Yandex, most mobile application developers use the service developed internally to solve problems related to localization. It has a very convenient UI interface, it is possible to define your project, set a set of keys, a set of languages into which you need to translate the project. And after the translations appear, they can be uploaded in any convenient format. He also has very good integration with our internal services. There is versioning. But for me personally, as a developer, the most important thing is that he has an API, because this already suggests that all the manual work associated with translations can be automated. Which we did. We have developed a plugin for Gradle. In fact, updating the translation in the application now comes down to the fact that the developer performs a single task:
In a first approximation, he goes to a localization service. Checks for up-to-date translations, uploads them to the project, puts them into the necessary files. But in addition to this, he also verifies the status of translations in the localization service in relation to what is now in the project.
The next time git status is executed, the developer sees in his working directory which files have changed.
He can also look in the generated HTML report about which key for which value in which language has changed.
And he can also see what problems are currently present in the translations of his project.
The plugin is configurable. We can determine the number of languages that our application supports. It is very important if the localizers start a new language, and it inadvertently seeps into the project, having not passed through proofreading and proofreading.
There is also a specific remapping feature. I will talk about it a bit later in more detail when I talk about RTL.
What problems does this plugin analyze now? The problem that we encountered in 2014 is commonplace, lack of translation. Now, in order to add a new line, a new translation to the project, it is not enough for the developer to simply define it in the XML file.
Firstly, if he does this not in the default locale, then his newly wound key will simply disappear during the next synchronization, and he will know about it at the compilation stage.
If he determines the key in the default file, but forgets to do it in the localization service, then the next time updatetranslations is executed, he will receive an error from the plug-in, which will tell him: “This key is in your project, but it is not in the localization service, it is there need to start. "
The second typical problem is the lack of translation. We get a list of links to those keys for which there is no translation, and, as a rule, we refer them to our manager so that they later set up communication with translators. The third point often occurs when they decided to reconsider the use of format arguments in some way, and, for example, instead of an integer format, they began to use the string format.
But at the same time, in some of the translations they forgot to change the number on the line.
And the fourth problem is also caused by the human factor, mainly in translation. If Unicode characters are used in the translation, it is very easy to get confused in the positioning of characters and instead of an inextricable space to get a hyphenation to another line - this is what we are trying to get rid of.
This is all about localization.
Now I want to talk about the experience of implementing support for rendering from right to left in the application using the example of Israel. We launched in 2018 under the brand name Yango in Israel. As it turned out, in Israel people read differently than you and I do. They read from right to left.
Where do you want to start? With Android, in principle, right-to-left rendering support is implemented out of the box quite well.
It starts with the fact that in the application manifest, in the Application element, you need to declare the supportRtl = ”true” attribute. But I want to please you: if you integrate the Facebook SDK for social authorization in your applications, the caring Facebook developers have already done this for you. And if you do not validate the manifest when merging, then this attribute with the value true will fit into your application, and it will already be able to in Rtl. This is something to think about.
I think that most of those sitting in the audience probably have minimal support for Android 4.4. Someone is more fortunate - this is 5.0 and higher. Someone could be less fortunate, and they support 4.0, or, God forbid, 2.3. In this case, you will have big problems, because full-fledged Rtl support in Android has appeared since version 4.2. Therefore, with minSdk 17th, you will simplify your life by an order of magnitude.
Transferring a project to work with Rtl begins with the refactoring tool in Android Studio - Add RTL Support Where Possible. We must pay tribute, it works quite well. It goes through the XML files of your markup, looks at the presence of attributes with left and right values for gravity, for paddings, margins, and replaces them with start and end. If you want to see how your application works by default in right-to-left mode, you don’t need to have content with the so-called RTL strong characters in Hebrew or Arabic. You just need to enable Force RTL Layout Direction in the developer’s settings.
What are the options for customizing the UI and rendering text Android provides for RTL?
The first point is the layoutDirection attribute, which uses the described Force RTL Layout Direction option. She has only four possible meanings. That is, it is by default inherited from the parent. We can say that the layout is drawn based on the selected locale. You can clearly say that it is drawn from right to left or from left to right.
The second element that many people most likely bypass out of habit when they engage in text alignment is textAlignment. Many still continue to use gravity. Do not use gravity to align text, but use textAlignment.
The third attribute is the text rendering direction, textDirection. It has a fairly flexible configuration. It allows you to determine how the text is rendered depending on which first strong character is in the text, whether it is from the Latin strong LTR character set or from Hebrew strong RTL. But if you fully support RTL in the application, you do not need to bypass it, because ignoring such a funny artifact occurs.
Either this is an easter egg, or is it such a side effect in the implementation of editText: your hint at editText starts to disperse. Therefore, in any case, you must set some value for him.
Link from the slide
If you need to observe a certain rendering of some graphic resources or markup in RTL mode, then Android provides the ability to set qualifier ldrtl in your daddy into which you can put any resource, and it will be drawn in the RTL rendering mode as you specified.
Sometimes there is a need for runtime to draw the text coming from the backend as is, how it came, ignoring the mode in which your application is currently working. In fact, in text renderer this is done due to the fact that the text is framed by the so-called Bidi Unicode control characters. The logic for working with these characters in itself contains the BidiFormatter class, providing only one unicodeWrap method. It takes CharSequence as input, and returns a string framed by these characters. There is an exhaustive number of them, and, in principle, it should solve most of your problems.
But if there is something so specific, w3.org has a good article on how to use these symbols.
What problems did we get when we started RTL support in our application? First of all, it is a conflict of localization standards.
BCP47 is a newer localization standard, which in particular has changed the codes of some locales. And the fact is that there is a problem with the locale Java class. It works by default in backward compatibility mode, and converts locale codes from BCP to it to ISO. That is, we saw this when the iw code instead of the he code began to go to our backend in the Accept-Language header.
Link from the slide
If you remember the slide with the configuration of our plugin, remapping was needed just for this - in order to translate content from he to iw.
If you need to use the BCP format, this is fully implemented in the Apache Cordova project. They have a Globalization class and an implementation of localeToBcp47Language.
Another point that “delivers” with RTL support in the application is animations. Here, the benefit is, in fact, only animations along the X axis are affected, and perhaps the only solution or approach that will help you avoid more problems is the ability to reconsider the absolute animations in the direction of relative animations.
One problem that is perhaps solved only by reflection, because it is rigidly implemented in the component itself is the hint alignment of TextInputLayout. The fact is that if hint is specified in RTL strong characters in Hebrew, it will be clearly aligned to the right, even though the content that you enter in edit text can be from Latin characters. And vice versa. If your hint is in Latin, it will be clearly aligned to the left. This can only be solved with reflection.
Ultimately, after you have implemented RTL in your application, it makes sense to reconsider the severities for RTL-issues that Lint analyzes. There are only four of them. If you do not support RTL in your application, but explicitly use RTL specific attributes somewhere, then this is RtlEnabled (Lint will tell you about this).
Link from the slide
If you support RTL in the application, but minSdkVersion of your application is below 17, then you must also continue to use attributes of the left and right type, you cannot use only the start and end attributes in your markup. This is what RtlCompat parses. RtlSymmetry swears when it sees asymmetric paddings. In general, my opinion is that aapt, when processing resources, should crash if it comes across symmetric paddings. If you need some kind of asymmetric accesses, use margins as a full-fledged layout parameter.
Finally, the fourth problem is the hard-coded RTL. This is just that in the attributes only the values of left and right are indicated. Essentially, this negates the refactoring tool Add RTL Support Where Possible. It all comes from Lint on entering RTL. An analysis implementation is also available in AOSP in the RtlDetector class. That is, if imagination allows you, you can come up with something of your own by looking at how the analysis is going on now.
My report is drawing to a close. I want to give a few theses on building the correct localization in our application, which we have come to over time.
First of all, at the start of the project, think about internationalization. I didn’t really mention it. This is not at all the same as localization. In fact, internationalization makes localization possible in the application, in the product. This works out of the box in Java due to the fact that it has full Unicode support, and in Android - due to its rich resource system. Perhaps the only thing that can happen here is our fixation as developers on the theme that "I write the application only for Russian, which means I will only check it with the Russian language." And when you start using the Armenian, Georgian, Hebrew or Arabic languages in the application, the picture looks completely different and breaks down about your usual paradigm.
Second moment. Consider embedding localization in your CI. It is very disappointing to learn about the problems associated with translations during the testing phase of the release candidate, for which you could prepare for several weeks. That is, we are now running this task for every pool request so that we can be sure in advance that everything is fine with translations and that we can freeze the new code into the main branch of the project.
The third point. Respect the work of translators and your budget. First, delete unused resources because people can continue to spend time and money translating strings that you ultimately do not use in your project. Secondly, follow the translations that are in your repository in the localization service, but which are not in your code.
Ideally, if the API of your localization service provides information about when a particular key was started, and you can use heuristics like “If the key was opened more than a month ago,” check its presence in the latest revision of the master brunch. If he is not there, most likely, this is a clear candidate to remove it and so that the localizers do not spend time on it.
A more or less universal candidate for a silver bullet is to store strings on the backend, and not on the client. At the very least, this will give you the opportunity to always keep them relevant on the application side.
We have a very diverse user audience, everyone perceives the world and application content in their own way. Our task as developers is to help keep this perception. My report on this is over. Thanks a lot!