How we collected several hundred beta testers and why one gift iPad went to the Reichstag

How fast time flies. Last year, we just finished the virtual PBX RingCloud and tried to make a good beta test before launching the solution in production. I think we did it. Under the cut - infographics, distribution of iPads to testers and contented photos (even from Berlin against the backdrop of the Reichstag !!!).

On August 19, 2014, we announced the start of beta testing of our virtual PBX.

The purpose of this beta test was to collect information about errors and shortcomings that were made during the development process. Beta testing was conducted in three areas that interested the company in terms of quality performance.

Usability testing

· Testing SIP interaction;

· Testing security solutions.

We searched for testers through social networks, advertisements and articles on specialized sites. The prize in each nomination was declared Apple iPad 3. The winner was the one who discovered the most errors.

We recorded 272 registrations in the system for testing. But, in the end, 117 testers were registered in the system. For participants, an action plan was developed according to which they tested our virtual PBX.

Testers worked remotely. Especially for participating in beta testing, we set up our system for receiving complaints about malfunctions. All that was required of the participants was to send a letter to us at a special mailing address.

The beta end date was September 5, 2014, and the results were announced on September 12.

In each nomination we were sent errors and system malfunctions. For usability, we were sent 20 testers, for connecting -6 people and for security - 2.

As a result, we received 228 errors, 98 of which were accepted for work.



In the Usability nomination, a real struggle unfolded between the participants, right up to the very last day of testing. Most errors were sent to us: Ekaterina Artomonova, Ryzhikov Denis and Alexander Bogomolov.

Between them, our development team chose the winner. It was difficult to determine the winner. All participants approached testing creatively, showed a creative approach to the analysis of the interface and helped us to make the interface even more simple and understandable. As a result of testing, we handed the Apple iPad to Ekaterina Artomonova, who made the most significant contribution to improving the usability of our virtual PBX.



In the course of testing the security of our system, many aspects were checked, many hacking attempts were made by our beta testers, for which many thanks to them. The most active participants in this nomination were: Stanislav Eremin and Loktionov Lev.

I would like to single out bruteforcing and SQL-injection attacks from the whole variety of tested vulnerabilities, although they did not lead to a result, but still made it clear to us that we were moving in the right direction. For this type of attack, I would especially like to thank Stanislav Eremin, who approached the issue very professionally and also pointed out a lot of XSS vulnerabilities, registration security problems and CSRF vulnerabilities, for which we handed him an Apple iPad. The most interesting thing is that during the beta test Stanislav moved to Europe to implement a new project, but we kept our word and delivered a gift to Berlin!

Because there are no boundaries for a virtual PBX RingCloud . Thanks for attention!