State final certification in schools - not everything is like the exam

We have already told our readers how the ABBYY TestReader solution helped the exam stand on its own feet. A few years ago, it seemed to us quite logical (and at first even simple) using the experience gained, to process the results of the State final certification of 9th grade (GIA 9th grade).

So what did we have back in 2008? An established and generally reliably functioning machine for processing the USE results, which operates on the basis of the ABBYY TestReader Network form input software.

What do we want? Reapply our technology at GIA-9 and become even more investment attractive .

It would seem easier? But, having already embarked on this path, we found several differences between the exams, and this fundamentally determined the further development of events.

The fact is that the Unified State Examination is conducted centrally by the Ministry of Education and Science and Rosobrnadzor according to a single scenario, the forms with answers are processed the same way. In this case, the costs are divided between the federal and regional authorities as follows: the center provides technology and services, and the regions, in fact, organize an exam. The conduct of the GIA-9 was entirely given to the regions, which means that it falls on their budget and is carried out as decided by the leadership of the region. The GIA in Russia unites only that the exams are held according to a single schedule and for the same tasks.

As you can imagine, the regions of Russia are very different in terms of the amount of available financing, and some, obviously, are trying to save. As a result of this, “piecewise” automation is observed: someone is still boilingconducts an exam in the old-fashioned way: students write by hand, and examiners “by hand” check. The same regions that at their own peril and risk are trying to translate the exam into a test form on their own, face many problems. One of them is the need for a large amount of printing blank forms and assignments (officially they are called KIM - control and measuring material) for students. If for the Unified State Examination the forms are ordered centrally at the expense of the federal budget, then the regional authorities are doing this within the framework of the State Tax Inspectorate.

In the best case, each subject orders a print at a printing house; at worst, it prints forms using a regular printer, a copier, or maybe even a risograph .

Since for each region, first of all, it was considered the opportunity to save money and make things simpler, offering color forms for the GIA was pointless. Therefore, we proposed black and white forms that are easily replicated on various devices.



And now let's imagine that we are a subject of the Russian Federation. We need to provide blanks for up to ten thousand schoolchildren, each of whom passes mandatory exams in Russian and mathematics and other subjects. Each student needs a set of two answer forms and one form with assignments for each exam. Total, we need to print ... count, if you're interested. When you count, do not forget that these students study in different schools, which in turn are located in different cities, villages and ... islands. Therefore, when you type, you have to figure out which school the number of forms in Russian, mathematics and other subjects to take to. And if in the village of "Tmutarakan" there is an unfortunate person who suddenly wants to hand over the literature, then it would be nice to provide him with an appropriate kit. A, Well, do not forget about the reserve for worried students who inadvertently ruined the form. Well, counted? So we spat on this disastrous business.

Instead, we decided to add a special printing station to our TestReader, which will think it will count and print for us. It was supposed to become a new component of the program complex for the introduction of forms, intended for mass high-speed printing of machine-readable forms, as well as forms with tasks.

How did we do the printing station

This system was supposed to:

1. To be simple enough for operators in the regions to deal with it.
2. To be reliable - otherwise the exam will fail.
3. Be able to quickly print exam materials on laser printers
4. Provide materials to all PES (points of the exam) in the required amount
5. Minimize printing costs
6. Keep Information Security
7. You will have to print in the slowest graphic mode so as not to disrupt the formatting of the forms.

At the preparatory stage, we talked a lot: with specialists from the Federal Testing Center, with people in the regions who provide exams. At some point, it seemed to us that we were tired of all talking :). But how else to find an option that is suitable for all regions? We came up with different ideas, then searched for flaws in them and came up with new ideas. In the process, such “memory cards" appeared .



At some point, we had an understanding of what functions a print station should have, and we started designing. According to the plan, 90% of the functionality should have been related to the planning and collection of sets, and only 10% should be taken up directly by automatic printing.

Already at this stage, we realized that the development would have to be ordered by outsourcers. Examinations should be carried out within strictly defined deadlines, and our developers did not have enough resources - after all, exactly at the same deadlines we need to prepare modernized software for the USE. We wanted to entrust writing code to a longtime partner of ABBYY - Novosibirsk company ATAPY . When starting to work with outsourcers, we took into account the following:

1. They do not understand the subject area (that is, in state examinations).
2. We already knew that they could write suitable code, since they had already collaborated with them several times.
3. They are located in Novosibirsk, so there is a time difference.

To outsourcers better cope with our order, we:

1. We made the most detailed technical specifications, which included ready-made program algorithms. In fact, we didn’t need programmers from the outsourcer, but encoders, who would do everything exactly as we needed.
2. Themselves completely drew absolutely all the windows and dialogs of the program and described all the actions that should occur when you click on each of the "buttons".
   
   So, for example, the main window should look like:
   
   
3. We made a common complaint database with outsourcers. I must say that at first it caused dissatisfaction with both our programmers (we still left a small part of the development to ourselves) and our Novosibirsk colleagues. However, we insisted and subsequently made sure that we were right.
   When we worked on this project (recall, it was several years ago), cloud solutions in this area were not yet so widespread, therefore we used our own internal development - a clone of the database that we use in ABBYY.
4. We left behind ourselves and developed on our own modules that were supposed to be responsible for the interaction of the printing station with the "environment" - large external systems (databases, etc.). These modules are called “adapters”. Thus, we tried to protect the outsourcers as much as possible from interaction with adjacent systems.
   As a result, our partners did everything well, and we were satisfied with the result.

After the printing station was ready, it was tested, and then technical documents were written for it: operational documentation (what functions are in the program) and the corresponding instruction.

Do you want technical details?

We have them.

The development language is C #. The system database is MS SQL Server 2000/2008 (since it is also used in the USE). Platform - Microsoft .NET 2.0. An external visual component TreeList from DevExpress was purchased for the print station. Everything works on the platform of Windows XP, 2000. Cryptographic protection from the Aladdin company is installed.

The productivity of the printing station on inexpensive computers of that time with Pentium 4, Celeron processors (namely, they were installed in 2008 by most users of our solution) is up to 15-20 pages per minute (in graphical mode!). This is the maximum speed: if the printer is old and slow, it will certainly be less.

What does it look like from the point of view of the user

The subject of the Russian Federation conducting GIA uploads to the system all the necessary data for students: for this, he uses the existing databases for each of the subjects of the Russian Federation with lists of schools. The system sees in which community, city and district the school is located and how many students (in this case, ninth graders) are studying in it. And if there is no such data, then you will have to enter this data: set by default, extrapolate data about 11-graders, enter manually or in any other way. So we determine the number of ninth graders. Then we introduce errors, reserves, reserves, etc. - we complete the planning stage - choose the printing method - Start! Oh, and we’d be glad, but printing is also not so simple.

The region is given the opportunity to choose a printing method from the following main options:

1. We print the whole set in the printing house (answer and task forms)
2. We print only answer forms in the printing house, and the task is printed on printers (for example, in the capital of a region or regional information processing centers).
3. We print the entire kit on printers in the capital of the region.
4. We print tasks on a printer in the capital, and the forms of answers in the field immediately before the exam
5. We print the entire kit locally on printers just before the exam
6. Other combinations that are convenient for the region are possible, but they are rarely used.

If it is more or less understandable with printing in printers and printers in the capital of the region (armed guards, fingerprint entry), then it is worth clarifying additionally about “local printing”. Since it is required to preserve the confidentiality of the tasks before the exam, in the capital of the region, “conditional” printing of sets into files that are encoded with a password is performed. These files are sent to schools in advance, and after issuing the password, immediately before the exam they are printed on any laser printer available.

Passed the exam. Next, ABBYY TestReader scanning technologies enter the scene. Here we have kept the examination processing scheme as much as possible: the forms filled out by students are scanned, recognized, verified, etc. Then, without interacting with federal centers, exam scores are set right on the spot (another difference from the USE).

Summary and Lessons

• At the moment, 50 of the 83 constituent entities of the Russian Federation have chosen our technology for carrying out GIA in their regions.
• The project was completely investment on our part, and it quickly paid off.
• Since the launch of the project in September 2009, the station practically did not require improvements
• From start to finish, the project took 1.5 years: 10 months - analytics (of which 2 months - preparation of a detailed ToR with pictures), 5 months - development, the rest - testing and documentation.
  
  Team composition:
  From ABBYY : two analysts (the project took them about 50% of their working time), project manager, 1 programmer (1 working month), distribution assembly specialist (2 weeks), tester (1 month), technical writer (3 weeks).
  From the side of the outsourcer : project manager (our order took about 20% of his working time), programmer (5 months), tester (2 months), interface designer (2 weeks).
  
  
• We did not use special systems of collective work, therefore, correspondence by e-mail and telephone conversations took a lot of time. And we will not advise anyone to use Outlook and Excel as a Task-manager.
• As often happens when working with outsourcers, we were not able to avoid delaying the development time: we initially planned to cope in 3 months - we did it in 5.
• But on the other hand, the idea with the most detailed technical task, including all interface elements, worked 100% - outsourcers did everything exactly as we saw it. It turned out well.
• The idea with adapters was also successful. It allowed two teams of developers to work independently and to connect parts of the program at the final stage. We will use adapters for future system updates for the exam.
• And finally - GIA turned out to be an even more interesting and complex project than the USE. We liked.