20 projects, 20 languages, deadline yesterday
Imagine: you have 7 development teams totaling more than 100 people. They simultaneously saw 13 applications. Work is carried out in 20 repositories.
All applications need to be translated. Some in 6 languages, some in 20. And some in 13, but this is a completely different set of languages, it is not included in the previous 20.
Everyone has a different stack, as a result of different line formats: js, json, ts, yaml or yml. And some still keep their texts in the database.
You work on Agile: daily value delivery, two-week sprints. DoR includes all necessary translations. Well, of course, translations were needed yesterday in order to have time to test.
There is a department of technical writers. Who is a technical writer? This is a person who writes external documentation, sometimes internal. Writes all kinds of texts that users or partners can see: front-end texts, message texts, API responses, errors. Accompanies the development process to be immersed in technology and business logic. And provides timely delivery of translations to the application.
There is also the position of copywriter-translator and localization manager. This is a person who creates all texts in English, and also monitors the consistency of translations, appoints translators, and solves all related problems.
Attention, the question is: how many technical specifications, copywriters and localization managers are necessary so as not to hinder development and not hurt the entire technical department?
In our case, we managed 4 technical services and 1 copywriter-localization manager. The delivery of transfers on average fits within one business day and never exceeds three business days. I hope you find it interesting.
6 years ago we worked in Google sheets and a DB. That is, if during the development process lines appeared for translation, we copied them into a tablet, and then sent them by mail to the translation. When the translation was ready, it was manually poured into the database. The only plus of this solution is that you do not need to re-lay the application to see new lines. But if there is a mistake in the translations, it will not work to roll back. No translation memory, no glossaries. The consistency of translations is achieved by the gaze method.
The first version to automate this process looked like this: when a developer had lines, he added them to a new branch in a special repository for translations. Then, in the same branch, a pipeline was launched, which, by API, sent all the diff lines for translation. True, the translations were supposed to go back to the database already, but it failed to load the lines from the external resource to the internal database via the API.
What did such integration give? A step was removed where the technical writer needs to collect everything into one table, manually send, and then divide the received translations by application and by the number of languages. In this case, the lines were immediately sent for translation as part of a project of the same name with the application for which they were intended. As a result, the technical writer received a set of archives for each of the applications for which work was carried out. This significantly reduced the share of manual labor. In addition, the translation memory was implemented on the provider side. But this solution also retained a number of drawbacks: storing rows in the database did not allow full-fledged row management on our side and, as before, implied a large share of manual work.
The following integration brought a lot of suffering to the developers. It seems to me that those who caught it still have their eyes twitching at the word “localization”. This was the first integration with Serge and Smartcat.
Here it is important to tell what Serge and Smartcat are.
Serge is a utility that supports git. She knows how to get the necessary lines from the branch, send them for translation, and then return the translation for the same lines to the same branch only. We also need a plugin that will call the API of the CAT system in which we translate. The plugin should receive new lines from Serge and return ready translations to Serge.
Smartcat- This is a CAT-system, with the support of a glossary, translation memory, placeholders. Also, Smartcat aggregates and simplifies the process of settlements with freelancers, supports the connection of translation vendors.
At this step, we transferred the rows from the database to the project repository. Now the strings had to be sent directly from the application repository and returned there.
It was supposed that this would work like this: the developer knows from which branch he created his feature branch, and the diff in the resource files between these two branches is exactly what needs to be translated. When a developer has a set of lines for translation, he starts a job in his branch with the Serge config. Serge calculates diff, extracts new lines, calls the plugin and sends the lines for translation. When the translations are ready, the developer calls the following job: he deploys the Serge instance created in the previous step, receives the finished translations and commits them to the source branch.
The solution turned out to be unstable: Serge was not intended to be deployed from scratch every time the pipeline was launched, the developers did not want to think about diffs between the branches, and the Smartcat plugin was in urgent need of updating and improvement. The process of delivering new lines could take hours. And, alas, it was not always successful.
Theoretically, all stages of the process were automated, in fact, the maintenance, calculating the diff before the start of the pipeline and troubleshooting took longer than doing the same task completely manually.
By August 2018, we have launched the current version of integration. We have a localization server. On the server for each repository there is a separate instance of Serge. Serge scans all branches in the repository, sends new lines for translation and commits finished translations to the original branches. In the current integration, everything is fast and stable. After creating a branch for translations, the lines appear in Smartcat within 5-6 minutes. After confirming the transfers, the commit occurs similarly, within the same 5-6 minutes. The delivery time for translations is limited only by the human factor: the workload of translators, the difference in time zones and so on.
In the following articles I will tell you how to configure Serge-Smartcat-Gitlab integration from scratch, and how we solved various non-standard tasks.
Continuation:
20 projects, 20 languages, deadline yesterday. Part 2
20 projects, 20 languages, deadline yesterday. Part 3
All applications need to be translated. Some in 6 languages, some in 20. And some in 13, but this is a completely different set of languages, it is not included in the previous 20.
Everyone has a different stack, as a result of different line formats: js, json, ts, yaml or yml. And some still keep their texts in the database.
You work on Agile: daily value delivery, two-week sprints. DoR includes all necessary translations. Well, of course, translations were needed yesterday in order to have time to test.
There is a department of technical writers. Who is a technical writer? This is a person who writes external documentation, sometimes internal. Writes all kinds of texts that users or partners can see: front-end texts, message texts, API responses, errors. Accompanies the development process to be immersed in technology and business logic. And provides timely delivery of translations to the application.
There is also the position of copywriter-translator and localization manager. This is a person who creates all texts in English, and also monitors the consistency of translations, appoints translators, and solves all related problems.
Attention, the question is: how many technical specifications, copywriters and localization managers are necessary so as not to hinder development and not hurt the entire technical department?
In our case, we managed 4 technical services and 1 copywriter-localization manager. The delivery of transfers on average fits within one business day and never exceeds three business days. I hope you find it interesting.
How did we come to this?
6 years ago we worked in Google sheets and a DB. That is, if during the development process lines appeared for translation, we copied them into a tablet, and then sent them by mail to the translation. When the translation was ready, it was manually poured into the database. The only plus of this solution is that you do not need to re-lay the application to see new lines. But if there is a mistake in the translations, it will not work to roll back. No translation memory, no glossaries. The consistency of translations is achieved by the gaze method.
First try
The first version to automate this process looked like this: when a developer had lines, he added them to a new branch in a special repository for translations. Then, in the same branch, a pipeline was launched, which, by API, sent all the diff lines for translation. True, the translations were supposed to go back to the database already, but it failed to load the lines from the external resource to the internal database via the API.
What did such integration give? A step was removed where the technical writer needs to collect everything into one table, manually send, and then divide the received translations by application and by the number of languages. In this case, the lines were immediately sent for translation as part of a project of the same name with the application for which they were intended. As a result, the technical writer received a set of archives for each of the applications for which work was carried out. This significantly reduced the share of manual labor. In addition, the translation memory was implemented on the provider side. But this solution also retained a number of drawbacks: storing rows in the database did not allow full-fledged row management on our side and, as before, implied a large share of manual work.
Pain and continuous localization
The following integration brought a lot of suffering to the developers. It seems to me that those who caught it still have their eyes twitching at the word “localization”. This was the first integration with Serge and Smartcat.
Here it is important to tell what Serge and Smartcat are.
Serge is a utility that supports git. She knows how to get the necessary lines from the branch, send them for translation, and then return the translation for the same lines to the same branch only. We also need a plugin that will call the API of the CAT system in which we translate. The plugin should receive new lines from Serge and return ready translations to Serge.
Smartcat- This is a CAT-system, with the support of a glossary, translation memory, placeholders. Also, Smartcat aggregates and simplifies the process of settlements with freelancers, supports the connection of translation vendors.
At this step, we transferred the rows from the database to the project repository. Now the strings had to be sent directly from the application repository and returned there.
It was supposed that this would work like this: the developer knows from which branch he created his feature branch, and the diff in the resource files between these two branches is exactly what needs to be translated. When a developer has a set of lines for translation, he starts a job in his branch with the Serge config. Serge calculates diff, extracts new lines, calls the plugin and sends the lines for translation. When the translations are ready, the developer calls the following job: he deploys the Serge instance created in the previous step, receives the finished translations and commits them to the source branch.
The solution turned out to be unstable: Serge was not intended to be deployed from scratch every time the pipeline was launched, the developers did not want to think about diffs between the branches, and the Smartcat plugin was in urgent need of updating and improvement. The process of delivering new lines could take hours. And, alas, it was not always successful.
Theoretically, all stages of the process were automated, in fact, the maintenance, calculating the diff before the start of the pipeline and troubleshooting took longer than doing the same task completely manually.
Light at the end of the tunnel
By August 2018, we have launched the current version of integration. We have a localization server. On the server for each repository there is a separate instance of Serge. Serge scans all branches in the repository, sends new lines for translation and commits finished translations to the original branches. In the current integration, everything is fast and stable. After creating a branch for translations, the lines appear in Smartcat within 5-6 minutes. After confirming the transfers, the commit occurs similarly, within the same 5-6 minutes. The delivery time for translations is limited only by the human factor: the workload of translators, the difference in time zones and so on.
In the following articles I will tell you how to configure Serge-Smartcat-Gitlab integration from scratch, and how we solved various non-standard tasks.
Continuation:
20 projects, 20 languages, deadline yesterday. Part 2
20 projects, 20 languages, deadline yesterday. Part 3