120 servers in 30 days: tenders and other nuances of working with government agencies
We already talked about how Lenovo devices help hurricane hunters and graphic designers . And today we want to share the experience of integrating our solutions into a serious Russian state information system. At the end of 2013, in a very short time - less than a month, we put 120 servers in the medical organizations of the Krasnodar Territory. All of them have been working safely in polyclinics and hospitals so far, but the most interesting is the process of going through all the classical formalities. So this is a story not only about good equipment, but also about how to defeat the system: win a tender, bring a large batch of servers to a large region in a month, configure them and fully satisfy the customer and inspection authorities. Details under the cut.
It all started with the fact that the Ministry of Health of the Russian Federation in 2011 approved a program to modernize healthcare. On its basis, the regions defended their local programs, including those related to informatization. In the case of the Krasnodar Territory, it was required to create the basic information infrastructure of hospitals - lay local networks, connect communication channels, purchase network, server and computer equipment, office equipment. And then, on the basis of this infrastructure, introduce a medical information system for maintaining electronic medical records and medical records, information from which should be consolidated at the regional and federal levels. The deadline for all work was determined on December 31, 2013.
And everything would be fine if in the middle of the project the doctors from Krasnodar did not have to face an unscrupulous supplier. The winner of the tender brought equipment that does not meet the requirements of the state contract, with lower characteristics. By law, the customer is not just unable, but has no right to accept such a technique. And, of course, representatives of the Krasnodar Territory refused it: so began a lawsuit, which ended in favor of the region, but only in the fall of 2013, when only a couple of months remained before the completion of the modernization program. It was urgent again to announce a new tender and look for a new, very fast server supplier. Otherwise, the money allocated for their purchase would have to be returned to the federal budget,
I must say that the tender is a tricky thing. On the one hand, it’s just an auction in which the winner is the one who offers the equipment that suits the stated requirements at the lowest price. On the other hand, customers do not invent requirements, but first they study the market - they look at all devices suitable for the task from all more or less serious manufacturers. As a result, the terms of reference require something averaged with the boundaries “from” and “to”, into which the equipment of several companies falls. In addition, the contract is often drawn up so that warranty service is shifted from the manufacturer to the supplier. It is this approach that allows you to cut off obviously low-quality equipment that someone wants to quickly sell and immediately forget about it.
The terms of reference for the new tender was drawn up taking into account all these points. Not only Lenovo equipment, but also other vendors were suitable for the requirements. But the Moscow company SmartCity won the auction precisely with Lenovo servers. The decisive factor was Lenovo's superior customer specifications and the ability to meet deadlines: close deals before December 31. That is, bring equipment to the region, go through a check-in at the customer’s place, deliver it to hospitals, install software, update, start up, check work, sign transfer certificates and still manage to get money. Frankly, we haven’t done anything like this before, but now we know that even our almost impossible tasks are impossible for us and our partners.
Of course, Lenovo and SmartCity did not undertake the task of informing all hospitals and clinics in the Krasnodar Territory. There were several tenders in the framework of the regional program - for the creation of local computer networks, for the supply of computer equipment, for software, but our task was to supply the servers, without which everything else, in fact, cannot work. Here again, it is appropriate to recall the deadlines: if they had not had time with the servers until December 31, everything else would have become irrelevant.
At that time, the customer already had experience buying servers. As part of the same program, at its very start in 2011, about 70 Aquarius servers were purchased for medical institutions of the Krasnodar Territory. Therefore, the customer was able to very accurately identify their needs.
Firstly, in the region’s hospitals and clinics there was practically no information infrastructure. Only scattered pieces of networks and autonomous jobs. At the same time, the option to buy a good computer for each doctor and a set of necessary software was not even considered; there was not enough funding for such a decision. Plus, do not forget about the requirements for working with personal data. Their local storage is expensive.
In fact, there was exactly enough money to buy one server for one hospital. So the basic infrastructure loomed as follows: a server is installed in each medical institution, a virtualization system, a terminal server, an application server and a database server are deployed on it. And already thin clients installed at the workplace of each doctor work with this. Funds for the system software were not allocated, since initially it was supposed to use only open software.
Krasnodar Territory is quite large. Inside it, 44 municipalities and medical institutions are different: from small district dental clinics to huge regional hospitals with a large number of buildings. The spread in the number of jobs per institution turned out to be very significant: from 5 to 300 or more. Therefore, from the very beginning, all points were divided into six types according to this criterion. The choice of equipment characteristics depended on the number of users (and the allocated budget).
Typical server hardware. The operating system used is Linux CentOS. Each server is virtualized based on the Citrix XenServer hypervisor, on the basis of which virtual terminal servers NXSERVER, MySQL database server and application server of medical information system MIS KPS "SAMSON" are deployed.
Actually, it was precisely to ensure the possibility of working with IIAs that the modernization program was conceived. The hospital information is tied to the medical information system. It creates invoices for payment for services, which are then handed over to insurance companies. Stopping the program means the risk that the medical institution will not receive money for the treatment provided to patients in a timely manner.
For 120 servers brought to the Krasnodar Territory at the moment there were only 2 warranty calls. Both are associated with data storage systems: one at the start-up phase, and the second about six months after installation, that is, in mid-2014. For its part, the supplier turned to Lenovo service centers, and we resolved the problems as soon as possible. But users, that is, doctors, did not even notice anything - their work did not interrupt, we did not receive complaints about the quality.
For the past year and a half, 120 Lenovo servers have been operating safely in hospitals and clinics in the Krasnodar Territory. And, we are sure, they will work for a long time, especially since the possibility of functional expansion was laid down initially. You can install new hard drives, increase memory, but the customer has not yet had such a need.
The success of the campaign was ensured by the experience and coordinated work of the employees of the Lenovo Russian office, our partner, and all kinds of assistance from the IT specialists of hospitals. In a word, exactly as it should be.
Our previous posts:
» ThinkServer RD650: Lenovo's next-generation server anatomy
Federal program
It all started with the fact that the Ministry of Health of the Russian Federation in 2011 approved a program to modernize healthcare. On its basis, the regions defended their local programs, including those related to informatization. In the case of the Krasnodar Territory, it was required to create the basic information infrastructure of hospitals - lay local networks, connect communication channels, purchase network, server and computer equipment, office equipment. And then, on the basis of this infrastructure, introduce a medical information system for maintaining electronic medical records and medical records, information from which should be consolidated at the regional and federal levels. The deadline for all work was determined on December 31, 2013.
And everything would be fine if in the middle of the project the doctors from Krasnodar did not have to face an unscrupulous supplier. The winner of the tender brought equipment that does not meet the requirements of the state contract, with lower characteristics. By law, the customer is not just unable, but has no right to accept such a technique. And, of course, representatives of the Krasnodar Territory refused it: so began a lawsuit, which ended in favor of the region, but only in the fall of 2013, when only a couple of months remained before the completion of the modernization program. It was urgent again to announce a new tender and look for a new, very fast server supplier. Otherwise, the money allocated for their purchase would have to be returned to the federal budget,
Nuances and tenders
I must say that the tender is a tricky thing. On the one hand, it’s just an auction in which the winner is the one who offers the equipment that suits the stated requirements at the lowest price. On the other hand, customers do not invent requirements, but first they study the market - they look at all devices suitable for the task from all more or less serious manufacturers. As a result, the terms of reference require something averaged with the boundaries “from” and “to”, into which the equipment of several companies falls. In addition, the contract is often drawn up so that warranty service is shifted from the manufacturer to the supplier. It is this approach that allows you to cut off obviously low-quality equipment that someone wants to quickly sell and immediately forget about it.
The terms of reference for the new tender was drawn up taking into account all these points. Not only Lenovo equipment, but also other vendors were suitable for the requirements. But the Moscow company SmartCity won the auction precisely with Lenovo servers. The decisive factor was Lenovo's superior customer specifications and the ability to meet deadlines: close deals before December 31. That is, bring equipment to the region, go through a check-in at the customer’s place, deliver it to hospitals, install software, update, start up, check work, sign transfer certificates and still manage to get money. Frankly, we haven’t done anything like this before, but now we know that even our almost impossible tasks are impossible for us and our partners.
Task
Of course, Lenovo and SmartCity did not undertake the task of informing all hospitals and clinics in the Krasnodar Territory. There were several tenders in the framework of the regional program - for the creation of local computer networks, for the supply of computer equipment, for software, but our task was to supply the servers, without which everything else, in fact, cannot work. Here again, it is appropriate to recall the deadlines: if they had not had time with the servers until December 31, everything else would have become irrelevant.
At that time, the customer already had experience buying servers. As part of the same program, at its very start in 2011, about 70 Aquarius servers were purchased for medical institutions of the Krasnodar Territory. Therefore, the customer was able to very accurately identify their needs.
Firstly, in the region’s hospitals and clinics there was practically no information infrastructure. Only scattered pieces of networks and autonomous jobs. At the same time, the option to buy a good computer for each doctor and a set of necessary software was not even considered; there was not enough funding for such a decision. Plus, do not forget about the requirements for working with personal data. Their local storage is expensive.
In fact, there was exactly enough money to buy one server for one hospital. So the basic infrastructure loomed as follows: a server is installed in each medical institution, a virtualization system, a terminal server, an application server and a database server are deployed on it. And already thin clients installed at the workplace of each doctor work with this. Funds for the system software were not allocated, since initially it was supposed to use only open software.
Decision
Krasnodar Territory is quite large. Inside it, 44 municipalities and medical institutions are different: from small district dental clinics to huge regional hospitals with a large number of buildings. The spread in the number of jobs per institution turned out to be very significant: from 5 to 300 or more. Therefore, from the very beginning, all points were divided into six types according to this criterion. The choice of equipment characteristics depended on the number of users (and the allocated budget).
- Type 1 (over 200 users) and Type 2 (from 100 to 200 users). Under this contract, servers were not supplied to these institutions, they were already purchased earlier.
- Type 3 (50 to 100 users): ThinkServer RD540 (2 x Xeon E5-2620v2)SpecificationsTwo Xeon E5-2620v2 processors, 6 ECC Low Voltage RDIMM DDR3 memory modules with a capacity of 8 GB each with a clock frequency of 1600 MHz, 4 hard drives each 300 GB (2.5 ", 15K) with a 6 Gb / s SAS interface, hot-swappable, 1 LSI 9240-8i controller, 2 80 PLUS Gold 800 W hot-swappable power supplies, 1 BMC Management Controller with iKVM function.
- Type 4 (up to 50 users): ThinkServer RD540 (1 x Xeon E5-2603v2)SpecificationsOne Xeon processor E5-2603v2, 4 memory modules ECC Low Voltage RDIMM DDR3 capacity of 8 GB with a clock frequency of 1600 MHz, 4 hard drives of 500 GB (2.5 ", 7.2K) with SATA 6 Gb / s, hot-swappable , 1 LSI 9240-8i controller, 2 80 PLUS Gold 800 W hot-swappable power supplies, 1 BMC Management Controller with iKVM function.
- Type 5 (organizations using type 1-3 servers): Lenovo-EMC 70BN9001WW px12-400r , 12TB (4x3TB).
- Type 6 (organizations using type 3-4 servers): Lenovo-EMC 70BJ9007WW px4-300r , 8TB (4x2TB).
Typical server hardware. The operating system used is Linux CentOS. Each server is virtualized based on the Citrix XenServer hypervisor, on the basis of which virtual terminal servers NXSERVER, MySQL database server and application server of medical information system MIS KPS "SAMSON" are deployed.
Actually, it was precisely to ensure the possibility of working with IIAs that the modernization program was conceived. The hospital information is tied to the medical information system. It creates invoices for payment for services, which are then handed over to insurance companies. Stopping the program means the risk that the medical institution will not receive money for the treatment provided to patients in a timely manner.
For 120 servers brought to the Krasnodar Territory at the moment there were only 2 warranty calls. Both are associated with data storage systems: one at the start-up phase, and the second about six months after installation, that is, in mid-2014. For its part, the supplier turned to Lenovo service centers, and we resolved the problems as soon as possible. But users, that is, doctors, did not even notice anything - their work did not interrupt, we did not receive complaints about the quality.
What is the result?
For the past year and a half, 120 Lenovo servers have been operating safely in hospitals and clinics in the Krasnodar Territory. And, we are sure, they will work for a long time, especially since the possibility of functional expansion was laid down initially. You can install new hard drives, increase memory, but the customer has not yet had such a need.
The success of the campaign was ensured by the experience and coordinated work of the employees of the Lenovo Russian office, our partner, and all kinds of assistance from the IT specialists of hospitals. In a word, exactly as it should be.
Our previous posts:
» ThinkServer RD650: Lenovo's next-generation server anatomy