Coursera in Russian: about achievements and awards

    It seems that we have just announced the official launch of the “ Translate Coursera ” project, and our volunteers have already crossed the line of the first million translated words. We thank all participants - you are real good fellows. For you, we have prepared a special page where we collected congratulations and infographics in honor of such an event.

    As you know, the project is based on SmartCAT - ABBYY Language Services own cloud platform for translation automation. It allows all Coursera translators to work with modern technologies for translation in a convenient interface: participants can use Translation Memory, automated support for the integrity of terminology, machine translation, view video clips of lectures. We are constantly updating this technological foundation so that the resource becomes more convenient and better helps our participants with the translation of lectures.

    We think not only about convenience, but also about how to interest volunteers. Recently, the project has earned a system of awards - achievements. They reflect the success of participants in the translation of Coursera. Our developers shared how this system works and how it works. This is what we will tell you today.

    Achievements are issued to the user in the process of his work on the platform and are a form of intangible reward for good results. For participants, they look like medals and are displayed next to the avatar. They are issued for certain successes in translation. For example, the Specialist badge can be obtained by translating a certain part of the course (1, 7, ... 60, 100 %%). This is a multi-level achievement - the more you translate, the higher the level. There are also one-level ones: the achievement after delivery is not subject to exchange.

    Full list:
    Number One - to receive this honorable award, you need to take first place at least once.
    Leader - Achievement is earned with getting into the top ten translators.
    Expert - stands out for being active in voting: the more you evaluate the work of other participants, the higher the expert’s skills, and therefore the more valuable the reward. The first-level expert is awarded for evaluating five translations.
    Specialist - a reward in several levels, which can be obtained for the translation of a certain percentage of the course. For the first level, it is enough to transfer 1% of the entire course - and this is more than it seems. For each course for which this award is introduced, you can get your own Specialist.
    Finalist - Achievement earned by translators whose work has become the best and chosen for placement on Coursera. The first level can be obtained in one translation, which was included in the final version of the subtitles for the Coursera website.

    Technically, the reward is reduced to two program actions:
    1. Calculation for the user of a certain value - “progress” in the achievement;
    2. Upon reaching certain threshold values ​​of “progress”, a reward is issued or its level is increased.

    The award must be visible:
    1. Immediately after receiving it in the working window of the application where the translation is performed;
    2. All rewards received are displayed in the user profile;
    3. And they are all visible in the ranking of participants.

    It is important to remember that the value, on the basis of which the next level of achievement is awarded, can increase or decrease over time. For example, the Leader award is given for getting into the top ten of the rating, and it is obvious that once you get into the top 10, you can eventually drop out. We proceed from the fact that once a reward or its level is assigned to the user and can only change in the direction of increase.

    What does the reward system do most of the time? Periodically recounts the amount of progress in the achievement, and when a threshold is exceeded, it generates an event of issuing a new achievement or its new level.

    Our developers decided that it is best to place the code responsible for calculating the progress, closer to the assembly of achievements and away from the assemblies responsible for the subject area. Which they did.

    As a result, the code responsible for the achievements was located in two assemblies:
    1. Achieve.Module - contains logic that does not depend on the subject area, the specific award and the conditions for its assignment. This assembly can be used "as is" in any other project.
    2. Achieve.Configuration.Module - contains information about the conditions for assigning achievements and algorithms for calculating progress. The localization of calculation algorithms in this assembly made it possible to minimize changes in the assembly of the subject area due to the implementation of achievements. In particular, the refinement of the domain assembly was reduced to declaring the public interfaces of event handlers, for example, IRatingUpdatedHandler, and calling their methods in the appropriate places.

    If a little more, then:
    1. The domain assembly declares the public interface of the ISomeEventHandler event handler and implements a dependency on
      to the services that should generate this event;
    2. The Achieve.Configuration.Module assembly refers to the domain assembly and implements the ISomeEventHandler as many times as necessary, registering the actions in the Dependency Injection container;
    3. The domain assembly calls ISomeEventHandler.Handle (some event params) after saving the changes to the database;
    4. ISomeEventHandler.Handle (), implemented in Achieve.Configuration.Module, uses any domain services to read the information needed to recalculate progress;
    5. ISomeEventHandler.Handle () calculates the progress and, if necessary, generates events for awarding new awards or new levels.

    In other words, the calculation of progress is separated from the assemblies of the subject area using the mechanism of events. And in order to avoid problems associated with “regular” .net events, they are implemented based on a combination of interfaces and Dependency Injection.

    For example, the code of the module that registers in the DI container all the classes necessary for calculating the award Finalist :
    using System.Collections.Generic;
    using AbbyyLS.SmartCAT.BL.Crowd.Interfaces;
    using Ninject;
    using Ninject.Activation;
    using Ninject.Modules;
    namespace AbbyyLS.Achieve
    	class AchieveFinalistModule : NinjectModule
    		public override void Load()
    		private AchievmentConfig CreateFinalistAchievmentConfig(IContext context)
    			var result = new AchievmentConfig
    				AchievmentName = AchievmentNames.Finalist,
    				LevelSteps = new List>
    					new Dictionary{{ProgressNames.ConfirmedVariantsCount, 1}},
    					new Dictionary{{ProgressNames.ConfirmedVariantsCount, 5}},
    					new Dictionary{{ProgressNames.ConfirmedVariantsCount, 25}},
    					new Dictionary{{ProgressNames.ConfirmedVariantsCount, 75}},
    					new Dictionary{{ProgressNames.ConfirmedVariantsCount, 150}},
    				ProgressSteps = new List>(),
    				Ventures = new[] { KnownVentures.Coursera }
    			return result;

    So, we know that with some actions of the user, the progress is recalculated and the events of awarding or raising their level are generated. What should happen at the first start of the system after the implementation of achievements? Obviously, it was necessary to calculate the progress of each user without any events.

    This could be done using methods that are invoked in event handlers that affect achievements. But such methods are sharpened by calculating the progress for one user and can use some data that is transferred to the ISomeEventHandler.Handle (...) method from the domain assembly side, which cannot be obtained when the application starts. For example, ISegmentTranslatedHandler.Handle (userId, segmentId). That is, it is necessary to implement the algorithm of its initial calculation separately from the progress conversion algorithms. In our case, in the Achieve.Module assembly, we announced the IInitialProgressCalculator interface, the actions of which are implemented and registered in the DI container in the Achieve.Configuration.Module assembly.

    Competitive initialization
    One of the most unexpected surprises at the testing stage was connected with the initial calculation of progress. We did not take into account that its initialization can occur simultaneously on several servers of our application. Accordingly, if a user received a reward or its level at the stage of the initial calculation of progress, then a notification about this was sent immediately from all application servers. The problem was solved with the help of an exclusive lock on the initial calculation through the database: the server, which did not get exclusive access to the initial calculation, passed it.

    Multiple recalculation of progress in event processing
    At the testing stage, another interesting story emerged: it turned out that an event that requires the same calculation can be triggered en masse. Obviously, executing an event handler with identical parameters multiple times simultaneously is not the best tactic. Therefore, we again used the exclusive locking mechanism through the database - this time to execute an event handler with specific values ​​for the event parameters. The lock was not implemented everywhere, but only where the problem was fixed.

    Push notifications and periodic polls
    Information about the rewards earned is immediately displayed in the application’s working window thanks to SignalR, the server-side push notification system for the web client. Since we already used it to send other notifications, adding events of achievements was not difficult. Of course, the notification of the receipt of the next award must be shown on all pages. However, some of them are static web pages, and implementing SignalR there would be too expensive and therefore impractical. Therefore, we decided to use a periodic server polling for this: it checks whether new achievements or their levels have appeared since the last request, and only then gives a command to the notification system.

    Data storage
    Achieve.Module stores the following data:
    1. Achievements configuration - threshold values ​​of progress at which a reward is assigned to a participant;
    2. The current value of progress for all achievements;
    3. The maximum level reached by the user for each award;
    4. Unviewed Achievement Assignment Events.

    Localization of texts
    We are a linguistic company, and therefore could not ignore this issue. When constructing the texts that the participant sees along with receiving the award, there are nuances associated with the use of numerals in the Russian language: 1 translation, 2 translations, 5 translations, 11 translations, 21 translations. (As we know, in English everything is much simpler and comes down to the difference between singular and plural: 1 translation, 2 translations.)

    Therefore, we just introduced different text patterns for different numerals:
    • 1, 21, xxx1: translation;
    • 2, 22, xxx2: translation (same for 3 and 4);
    • all other numbers, including 5-20: transfers.

    An example of what a template looks like for localization of achievement text.

    A method for autotests
    Since the most important part of achievement logic - initial calculation of progress - is performed once, when writing tests (especially integration ones), there is a need for an API method that will allow you to reproduce this action many times. To do this, our developers made the Recalculate () method, which causes the same recalculation of progress as when the application was first launched after the introduction of the award system.

    Example code of an event handler that causes a recount of progress:
    using System;
    using System.Collections.Generic;
    using System.Threading.Tasks;
    using AbbyyLS.Core.Helpers;
    using AbbyyLS.SmartCAT.BL.Crowd;
    using Ninject;
    using NLog;
    namespace AbbyyLS.Achieve
    	class VotedVariantCountChangedHandler : ITranslationVotedHandler
    		private static readonly Logger Log = LogManager.GetCurrentClassLogger();
    		public IAchieveService AchieveService { private get; set; }
    		private readonly ITranslationVariantRepository _translationVariantRepository;
    		public VotedVariantCountChangedHandler(ITranslationVariantRepository translationVariantRepository)
    			_translationVariantRepository = translationVariantRepository;
    		public void TranslationVoted(Guid accountId, Guid userId, Guid variantId)
    			Task.Run(() => translationVotedTask(accountId, userId));
    		private void translationVotedTask(Guid accountId, Guid userId)
    			Log.Debug("Achieve event VotedVariantCountChangedHandler.TranslationVoted start");
    				using (new TimerLog(t => Log.Trace("Achieve event VotedVariantCountChangedHandler.TranslationVoted complete in {0} ms", t)))
    					var votedSegmentCount = 
    						(int) _translationVariantRepository.GetVotedTranslationsCount(userId, accountId);
    					AchieveService.SetProgress(accountId, userId, new Dictionary
    						{ProgressNames.VotedVariantCount, votedSegmentCount}
    			catch (Exception ex)
    				Log.ErrorException(string.Format("VotedVariantCountChangedHandler.TranslationVotedTask accountId : {0} userId: {1}", accountId, userId), ex);

    Actually, that's all for today. To see how it works in practice, join the translation of Coursera into Russian here - we will be glad to see you. And all questions, suggestions and suggestions send to or leave them in the comments here.

    Also popular now: