Measuring the quality of mobile Internet: where we shared the container of pink Samsung Galaxy S II

    About six months ago, a container of Samsung Galaxy S II phones in bright pink came to us , which we immediately sent to our employees. Further, a lot of interesting things happened to them: someone drove them in a car, someone carried them in a case, and some assembled from them high-tech devices of this type:


    Device for "information intelligence"

    Why did we need all this? It's pretty simple. The fact is that in 2012, my unit was tasked with setting up regular collection of statistics on the quality of mobile Internet throughout the entire Beeline network.

    At the beginning of the project, the intro was: quite often, the client calls the call center, says that he has problems with the mobile Internet (low speed, no connection, etc.), but at the same time, from the point of view of the network infrastructure, everything looks fine. In the general case, if there are several dozen such situations, you can send a mobile measuring laboratory to the place, but each of its visits is, in fact, a nailing with a microscope.

    We have been given the task of constantly taking measurements of the quality of mobile Internet in those places where there are clusters of subscribers in order to have a representative picture of statistics.

    Why do we need such measurements if there is internal monitoring of the network infrastructure? It's simple: you always need to look at the network not only from the point of view of infrastructure, but also from the point of view of subscribers. That is, to simulate their normal use of mobile Internet.

    The task of my team was to organize data collection and provide a variety of tools for their subsequent analysis and use. We are an encapsulated object in the huge structure of the company - we have been given the task, the output must be given a stream of reliable data. What will happen to him next is a question for other units.

    Finding a solution


    World experience has suggested that the most effective option in such cases is to give customers the opportunity to collect information and notify about problems. Some foreign operators offer mobile applications that essentially log all the technical parameters and actions on the subscriber’s device and share statistics with the operator. This is a good solution, which I immediately had to give up for three simple reasons:
    • The code of business conduct of our company does not include the collection of such data, it is simply prohibited by ethical standards.
    • The implementation for those who give voluntary consent (for example, paid or “ideological” beta testers from among subscribers) unfortunately lies not only in the technical field, but requires the involvement of a large number of participants, including lawyers, since it affects the contract and personal data.
    • One of the operators has already launched a similar system, but, according to our data, it was not representative, and the interest of subscribers in it, let’s say, faded.

    Our solution was extremely simple and effective: to distribute devices to company employees. A container of pink phones appeared on the scene here.

    Our company employs several hundred people in every major city, we constantly move to places of subscriber congestion (since we ourselves are), and if necessary, we can go with such a test phone to any place where there is a suspicion of problems.

    It turns out sufficient representativeness (this is the most important) and good regularity. Looking ahead, I’ll say that over the past six months we have made about 4 million measurements, which would be simply impossible with other means.

    So, for starters, we determined the quality indicators that need to be measured. It should be not only speed. As a result, we look at Accessibility, Retainability, Access Time, Speed ​​(MeanDataRate) and Latency (Delay, RTT). That is, what together determines the comfort of the subscriber and, for example, the ability to check mail in a matter of seconds. They can be obtained using the mobile OS using specialized software for a smartphone.

    Then we found, customized and implemented a mobile Android application that runs a test every 5 minutes. If at the time of the start of measurement the smartphone is registered in the 3G network, a 1.5 megabyte picture is downloaded. If, at the time of the start of the measurement, the smartphone is registered in the 2G network, the picture is only 200 kilobytes. The picture itself lies on a server in the Moscow data center, and is not cached on our media servers or other types of proxies. Regardless of the region of Russia, downloading occurs from a centralized resource using the http protocol, which provides the same conditions and allows you to evaluate the quality of the mobile Internet service in real subscription conditions.
    The measurement results are regularly sent (https) to the technology web server (also located in the Moscow data center) and stored in a database for further processing, creating reports, graphs and other analytical information. If the smartphone has left the access zone or lost an Internet connection, the data is written to the memory card and wait for the next communication session to be sent.

    Project progress


    Of course, the project, which was called Mobile Quality Analyzer (MQA), could not do without surprises. An important point was the choice of smartphone. As you know, a sufficiently large computing power is required to calculate the selected quality indicators. It turned out that many modems (radio modules) of budget smartphones simply cut speed when they rest against their ceiling. Tests have shown that the cheapest model available at that time on the market that does not distort the measurement results is Samsung Galaxy S II (supports HSDPA, HSPA +, HSUPA technologies). Here we began to search for them in sufficient quantities - and found it at one retail chain, which with great pleasure made us a discount on the unsold pink model. Fortunately, we didn’t care what color the future device would be.

    We distributed the phones by region according to the number of broadcasting sectors of our network in order to have the desired representativeness.

    For beta testing, they handed out phones to those who travel in a mobile measuring laboratory.

    When conducting combat measurements, the usual picture was the presence of three phones in the passenger compartment of a car serving base stations with connected chargers, or a special bag, or a miracle device, as in the picture at the top of the topic. It was very important that all possible measurement errors (of which there are quite a lot) were smoothed out due to many tests. Cities with a large number of subscribers per day accounted for several hundred measurements - this excluded all the "phases of the moon."

    The first results of measurements were disassembled by hands, emergency situations were calculated and we looked at why they were happening. I had to introduce special data processing, which allows finding and discarding distorted results, for example, not taking data into account when the battery is low (less than 3%).

    All phones were preconfigured as follows: 2G / 3G network auto-enable was turned on, GPS was turned off (initially it was planned to work with it, but it very quickly drains the battery), Wi-Fi turned off. The mobile application on the smartphone also ensures that in no case is any data channel other than mobile used — otherwise the results would come with errors. Also, the mobile application is able to monitor updates itself and receive remote configuration settings.

    Measurements


    Here is a table of measurements from January 1, 2013 to mid-summer:



    Architecture





    The servers have the RedHat OS installed. Apache + TomCat was chosen as the Web server.
    On the MQA Gate server, in addition to https, there is protection against DDoS attacks, plus all phones are authorized by IMEI.
    Oracle 11g was chosen as the database.
    Which is nice, the minimum cost.

    We deployed with the contractor in the cloud, first on a test server in Germany, then in the Moscow data center according to our standards, while virtualizing all parts of the processing system.

    Why is that? Because we did not know whether the project would be successful or not, and could not immediately predict the necessary computing power. It has now become clear that the project is more than successful, and will expand. Horizontal scaling is very easy here thanks to the capabilities of virtual environments.

    What is the result? Our team has implemented a project that, according to our data, has no analogues with other Russian operators. Now, the technical departments of the company have accurate data on the quality of mobile Internet from the point of view of subscribers.

    From my own experience, it is very interesting to do such large-scale projects, roughly speaking, sitting in my office room. We didn’t even go to the fields - all the time we worked either with the application, or with the organization of the whole process, or with the server part. From the point of view of an IT project, there is a lot of useful experience, especially in terms of organizing a large number of people and working with statistics. From a simply human point of view, it’s nice that we have developed a tool that will allow us to really improve the network over the next couple of years (although we are not involved in planning and optimizing the network).

    Also popular now: