gmaksimenko May 13, 2013 at 18:25

How did I choose Android for testing

From the sandbox

Task:

For production needs, the following two questions arose: how to determine the effectiveness of an existing set of devices for testing an application for Android and how to correctly choose the next device for testing.

Theory:

After a little thought, I started digging the Internet in search of a solution to this problem, but I did not find a ready-made solution (it was about six months ago). But there was enough theory and description of experience in this direction.

The three most useful links with descriptions of strategies for choosing devices for testing are located at the end of the article.

All the strategies described there can be reduced to a general theory, which is concluded in the following three points:

Use existing statistics (it is important to use current devices).
Take into account the parameters and characteristics of the devices.
Parameters that can be taken into account:
- Screen size
- PPI (pixels per inch)
- screen resolution
- Android version
- performance (CPU, RAM)
- design features, such as the type of data input (keyboard, touch, trackball, and so on), the presence of cameras ...
- browser
- manufacturer
- supported services (geolocation)
- price
Consider budget

Further, depending on the project, everyone chooses for himself the determining factors or their combinations.

The options that almost all bloggers take into account are the Android version, PPI, and screen resolution. For web-projects, along with the listed parameters, the browser is important. For heavy native applications, performance is no less important ... and so on ...

The most logical and simple strategy seemed to me the theory of popular and cheap (popular and cheap).

Practice. Stage 1:

Armed with knowledge, I took action.
The project, for which the acquired knowledge was applied, uses approximately 1000 models of mobile devices daily more than 2 million times. Moreover, there were statistics on the number of visits for each device.

The device data collection phase was the most time-consuming. I collected information on the Internet about 123 popular devices, supplementing the document with Android, CPU, screen sizes, resolutions and PPI for each device. This work took about a week.

It turned out that 123 out of 1000 devices cover 80% of visits, and I decided to stop on this.

Data Analysis Phasewas the most interesting. Noting the existing devices (there were 7) for testing in the table, I proceeded with the groups according to various parameters.

Grouping by major.minor version of Android showed that for testing all groups are covered from 4.2 to 2.2 (76% of visits) and only devices with Android version 2.1 (2.95% of visits) and 1.x (0.28% of visits) are not covered. In general, the result is not bad.

Analysis of permissions showed that the second most popular resolution of 480x320 with 18.61% of visits was never checked. At the same time, the most popular group with a resolution of 800x480 (27.56%) is covered by 3 devices.

Analysis by Android version and screen resolutiongave more interesting results. The three most popular groups 2.3 and 800x480 (15.30%), 2.3 and 480x320 (11.83%), 4.1 and 1280x720 (10.75%) are not covered at all. Moreover, the total coverage of a set of devices for testing was only 8% of visits.

As a result, a number of proposals were put forward that could significantly increase the coverage of visits by the main parameters of the devices.

On this my research in this area stopped.

Six months later ...

I recently returned to this issue while working on a new project. To be honest, the prospect of collecting information about devices did not please me.

One colleague saved me a ton of time by sharing the choosedevice.com link . It turned out to be a turnkey automated solution. This project already collected information on more than 200 devices, but only on 4 parameters: api level (os version), screen size, PPI and screen resolution. It can be seen that this project is still young and is now in beta testing.

Actions. Stage 2:

Here is the experience gained when working with this service:

I went to the site using my google account.
Noted devices used for a new project.
Made export of mobile devices from google analytics.
Uploaded these statistics in choosedevice.

All this took 10 minutes, and I got general statistics on mobile devices.

Next, I used the search bar to group devices.

The system calculated the coverage of user visits by the selected parameter and issued recommended devices.

You can also group by two parameters. Below is the result of the service on ppi / resolution.

Conclusion:

An Android device really needs to be selected wisely. ChooseDevice will help to do this efficiently.

Useful links:

stephanierieger.com/strategies-for-choosing-test-devices
devblog.xing.com/qa/how-to-choose-the-right-mobile-test-devices
mobilephonedevelopment.com/archives/1600

ChooseDevice:

choosedevice.com

Tags: