Mobile apps and their testers: all you need to know

    imageHello to you, Habr! My name is Maxim and I work in the QA department of Trinity Digital . In the field of quality assurance, I have more than two years, I love mobile applications, their complexity and dynamism. In this article I tried to make a relatively small list of tools, sources of information and skills that a mobile application tester should always have with him in our 2k17.

    If you break an article into parts, it will look like this:

    • Sources of information for the most successful testing
    • Tools to simplify the life of a tester
    • Hint
    • Application Delivery and Analysis
    • Where to grow further if you grasp Zen

    // Alarma - below is a paragraph for managers and connoisseurs of statistics

    In this article I will not talk about what iOS and Android are, but one cannot help but say what important role mobile platforms play in our lives. If we turn to the statistics on sales of PCs and smartphones , we can see that the number of mobile phones is growing every year, but PCs are less and less in demand. However, do not spread polemics about the death of any of the platforms. As Paul Adams said in an excellent article , every business should find its perfect balance between mobile and stationary type of work with information. In the meantime, managers ran away to solve business issues, I will continue.
    // The paragraph for managers is over

    Sources of information for testing


    In a sense, the tester should be a small specialist in each individual area, like a penknife. To always be in the know, you need to enlist certain sources of information.

    • These are pages of strict guidelines from Apple and Google for publishing in application stores. In general, the rules have much in common and are quite trivial, but, as they say, the devil is in the details. It is these details that you should always keep track of. After all, if your application works with user data, makes money transactions or simply provides content for children, then on the day of release get ready to see the deviation due to a small legal little thing that you, as a tester, should have thought about with the manager and the developer.

    image

    • It is also worth watching the designer guidelines from the same Apple and Google . Of course, design review is the task of the designer, however, the knowledge gained in these guidelines will allow you to better understand what the end user expects of you, allow you to better understand the philosophy of the platform, and also, subsequently, make it possible to conduct high-quality UI and UX tests.
    • Be sure to look at all the technical conferences of Apple and Google, because it is on them that you can find out where each platform is moving, what new tools it provides for both users and developers. You should be interested in any tools, because each of them will either create the final product, or it will be the goal of your testing.
    • It is just necessary to read as many books as possible about testing and software development, various blogs on Habré and other sources [your advertisement could be here]

    Tools to simplify the life of a tester


    The life of a tester is diverse and multifaceted. In order not to drown in the flow of information and be as effective as possible, there are many techniques and tools.

    • Conducting test cases and ticket storage. One of the final exhausts of any testing is tickets in special programs. Each bug not only wants you to find it, but also documents it. For this, a huge number of tools of various kinds of complexity and charge were created. Among the most popular can be distinguished Jira , redmine , TestTarantula , TestLink , TestCollab , QACoverage . The main problem of these services is that they are very similar to the stars in the sky - there are an infinite number of them and they are all similar from the outside, but it is worth taking a closer look, and their difference leaves no doubt. We also use Rapid Reporter for research testing.. With it, you can quickly record all the “roughness” that you noticed when traveling through the application.

    image

    • In order not to forget anything during the testing process, it is worth using one of the smart cards and heuristic cheat sheets . It is important to note that both of them only help you not to miss something important. If any of the points seems unnecessary for a specific project, then you can always create your own heuristics or maps.
    • A bit of mathematics and logic - if you are faced with the task of testing 4 different input fields + their various combinations between each other, then the PairWise 'inga method comes to the rescue . With this tool, you can always create a set of unique and self-sufficient test cases.
    • Be sure to arm yourself with tools for logging, screen recording and a screenshot. Both wired OS tools and programs from third-party developers are suitable.
    • A little automation of the routine, namely the generation of random information , the creation of mail , a basic check on the correctness of the naming of your ticket.
    • A little more automation with the help of “monkeys” - an easy-to-use mechanism for piercing the entire application. There are versions for both ios and android . However, it is worth noting that the monkey gives a small exhaust, it will not work out with some fundamental crashes with it. Relatively successful, these monkeys help to solve problems associated with the load on the application (how it will behave if you press a button 1000 times).
    • Make friends with Android Studio and Xcode, because in them you can not only perform whitewashing testing, but also fully work with debugger and breakpoints.
    • Network manipulation - most modern applications are tightly tied to working with the network, so it will be quite reasonable to use tools such as
      • Network Link Conditioner is a convenient and extremely simple tool for establishing the required connection. Need to simulate poor communication on a real device? You are welcome. I would like to test how the application will behave when a part of the data packets is lost? No problem.
      • For android, things are a bit more complicated. You cannot make such settings on the device itself, so you will either need to use the device emulator with a special setting , or use the Network Link Conditioner (see paragraph above) on a real iOS device + and set Hotspot mode from it. The crutch method is enough, but it is guaranteed to allow you to check the behavior of a real android device without foil and Faraday cages .
      • It is also worth arming yourself with Fiddler or Charles data proxy services - both are beautiful in their own way. If you need to look at the sent / received application data via the network, then there is no better tool. Each of them will perform the function man-in-the-middle, which will allow you to not only view the entire contents of the data packets, but also make it possible to feel like a young hacker.
        PS I also heard about WireShark , but leave it to you as homework;)
    • Simulators and Emulators are the tester’s best friends with a small device farm. If you encounter a bug that appears only on some exotic form of resolution (see Chinese smartphones), then you can help AVD and Genymotion for Android, as well as Simulator in Xcode for iPhone. It is important to note that simulators and emulators will not be able to replace you with real devices, in no case do not assume that successful tests on simulators = successful tests on real devices.

    image


    • The latter type of tools is absolutely indispensable in case you need to test the API. Since many mobile applications are a large software layer of server requests and processing the received responses, it would be reasonable for us, as testers, to check what the server sends from its API methods. Such products as Paw , Insomnia , SoapUI and, my personal favorite, Postman can come to the rescue . At first, all you may need from each of them is the form of a different request to the server (Get, Put, Post, Delete) and the response received. If you get a valid answer to the request, you can breathe calmly and go for tea continue testing further.

    Hint


    In any kind of testing, there are historically established bottle necks and knowing which you can find most of the errors in the shortest possible time. Also here I tried to describe some working points that will simplify the search for errors.

    • Always test the application in portrait and landscape modes. Sometimes developers forget to turn off landscape support or simply skip the layout of any of the necessary elements. It's not very nice to know when your favorite program crashes at the slightest rotation of the screen.
    • Be sure to look at how the application behaves after going to sleep, minimizing, returning to active mode and abruptly leaving active mode (phone call). For Android, there is even a special option in the settings - Do not keep activities ( DNKA ). Be sure to test your application with DNKA on and off, this significantly reduces the time of activity checks. Hint - when running tickets for bugs that were found using DNKA, don’t be too lazy to indicate that you used this tool, it will be easier for the developer to reproduce your error.
    • A huge number of applications use OS services. These include a camera, phone book, microphone, etc. In Android, the user gives access to these services when the application is downloaded, and in iOS (and also in Android starting from version 6.0) permissions are given after the program is launched. All this means that we need to check not only the text of requests for services (you must admit that the causeless request for access to the camera looks strange), we also need to check the operation of the system if permission has not been given. As a life example, I always recall how the application for Internet calls, codenamed “N”, crashed if you did not give it access to the microphone.
    • Notifications / notifications - how much in these words. When dealing with notifications, you always need to remember that they can be either local or server (that is, tied to a network connection). In addition, notifications, ideally, should lead to the screen referred to in the notification. Unfortunately, it is very common practice when clicking on a notification like “Hey, you got a message from Vasya”, you get to the main screen of the application, and not to the much-desired message from Vasily.
    • I already wrote above that there are special tools for tests related to the network, but I wanted to remind and indicate once again that checking how the application reacts to different types of communication and transitions from one state to another is extremely important. Most users of mobile devices do not sit in a comfortable office near a wifi router, but walk along the street or ride the subway. Keeping this in mind you can always find the crashes of your application where the lazy just would not notice them.
    • It is extremely important to see how the application behaves if you quickly switch between two screens (remember the monkey test above). From experience, I’ll say that Android has an extremely huge number of errors precisely because of the fact that during a quick transition between two screens the necessary fields do not have time to fill up with data and, as a result, lead to crash.
    • I can not say about certificates. Mobile application developers use different certificates to sign a developed product (store certificate and test). Most of the time you will see an application that has been signed with a certificate for the test, but when it comes to verifying the version with a different certificate - do not be lazy to conduct a full regression of the application, because more than once it turned out that a fully working application started to crash at startup due to certificate change.
    • Change the language and time of the device (especially time). Never assume that all users will be synchronized with the network and will automatically switch to the correct time zone. If your application somehow needs to change over time (for example, food delivery in 30 minutes), then you have endless scope for activity.
    • Always communicate with managers and developers. A few minutes of informal communication about the new assembly will open up a lot of new things for you, and at the same time simplify your life. Also, before starting to check the build, I recommend asking the developers - so that they check in your place what they already watched and why they are sure that this should work without fail. The main thing - do not fall for stories about fault tolerance and maximum stability; nature has not yet produced such products.
    • Try to correctly predict the time for testing, and, just as importantly, be able to stick to it. Find your pace of work, go into it and gradually improve. In the end, learn to stop and switch between tasks.

    image

    Application Delivery and Analysis


    Delivery of applications to test devices is a very important part of testing, because you will not be able to check the latest actual build until it falls into your hands. It is to solve this problem that various services come to the rescue. Below I will share those that we managed not only to try in practice, but also to find the ideal place in the development process for each of them.

    • TestFlight and Beta testing in the Play Market is a great tool for checking the performance of builds after rolling on old versions of the application that you have already released. These distribution methods mean that most of the application has already been tested, so it makes no sense to use them in the early stages of the project.
    • HockeyApp , AppBlade , Appaloosa , TestFairy are convenient tools for distributing builds for both iOS and Android. Each of them supports the ability to download past builds, and some even allow you to collect analytics on usage and crashes. These tools can be used both at the earliest stages and before release. It should also be noted that many enterprise customers stop distributing their applications through just such services, so don’t be surprised if some of your products never reach the App Store and Play Market, but remain within one of these services.
    • Google Drive, Yandex Disk, [substitute your own] - the services from the second paragraph can be replaced with conventional cloud storage, but in this case, management and version control will fall on your shoulders. If you feel organized enough crazy , then you can try to build all the processes on the cloud drives. We at Trinity Digital use these services when you need to backup a specific version of an application, or when testing at very early stages of development.

    An equally important tool is the analysis of applications, in the end whoever is warned is armed. That is why before the release of the product (and preferably before the start of development) it is worth deciding on those services that will best keep your finger on the pulse of your application. Here I also continued the topic of sources of information from the last chapter.

    • CrashLytics , Bugsee , AppSee and FireBase - give a very detailed analysis of the use of your applications, almost instantly report crashes, and, of course, are full of various graphs and charts. Each of them, of course, has its own unique chips. For example, Bugsee allows the tester to create a ticket with an error record directly from a mobile device with an open application, and FireBase was originally created as a means of quickly deploying a backend. Given all these features and features, you can choose a tool that is more suitable for you.
    • Reviews in the App Store and Play Market - unlike our colleagues from the world of the web, we can always look at the real opinions of our users. Will you write about positive things as much as negative ones? Unlikely. Will negative feedback always be helpful? I doubt it. Will the time spent on reviews be worth it? Of course, if you want to make a really high-quality product that is interesting to users. However, it is worth pointing out that if the whole team is not configured to work with reviews, then your initiative alone will not be enough. Cultivate and instill a love of reviews and work with them - this will definitely pay off in the long run.


    image

    • Implement feedback forms in each application - this way the user can always contact your team without spoiling the rating in the application store + it will be easier for the user to navigate the history of email correspondence, rather than a faceless story.
    • Рано или поздно вам обязательно потребуется организовывать ферму девайсов. Не буду вдаваться в подробности, но, если коротко, то вам обязательно нужно будет учитывать такие факторы как популярность девайсов в вашем регионе, популярность девайсов у аудитории вашего приложения, недостающие ОС и размеры экранов, которые вызывают проблемы больше всего, тенденции моды на ближайшие два года (ибо девайсы вы не на месяц себе берете), и, конечно же, нужно всегда держать в голове, что у Android смартфонов есть огромное количество производителей, каждый из которых считает своим долгом — внедрить кастомную прошивку с максимальным количеством конфликтов для внешних библиотек. На помощь в этой непростой задаче вам должны приходить ваши аналитики, а также такие сервисы, как

      • In the case of Android - AppBrain , Apteligent data , slightly outdated statistics from OpenSignal , statistics from AnTuTu (do not forget that they have a benchmark service, so there will always be more productive devices in their picture of the world), detailed statistics from Google itself
      • For iOS - statistics from Apple itself , from the previously mentioned Apteligent data , and modest statistics from David Smith

    Where to grow further


    We all start with manual testing, but rarely does anyone stay in the same position for many years. So where does the tester go when all the tools from the list have already been studied and you want something new? In fact, each tester eventually moves along one of the following paths - development, autotests, manager, DevOps. Whichever way you like, they all share some qualities that should be present in every self-respecting professional.

    • Systematic approach - any self-respecting tester should systematically approach problems. Turn to mathematical analysis and statistics as often as possible, they will give greater scope for professional growth. This also includes risk management, prioritization and analytics.
    • Communication - the tester is not worth a penny if he can not correctly talk about the results of his test. Tighten both native language and foreign. The higher your communication skills, the faster and better bugs will close.
    • Team management - sooner or later you will have to manage a certain number of people from your field of activity. Start watching how teams are organized in other companies, why some approaches are worth using, and some can be skipped.
    • Automation - do not think that automation is just writing a mountain of code and a complete replacement for manual labor. You can remain a manual tester, but know how to quickly fill in 50 different fields with python, which you are so lazy to fill in yourself each time. Is it automation? Of course, although basic. Do not be afraid of automation, but master it in every possible way, albeit in small steps.
    • Trends - be sure to look at the applications of competitors, winners of various nominations from Apple and Google , follow the trends in social networks. Maybe your primary goal is quality, but knowing the problems of competitors, you can avoid problems at home.
    • Programming - learn to program. This will not only help you with automation, but also allow you to talk with developers in the same language. I will not describe how important it is to know programming languages, just keep in mind that this is an important skill, without which it will only become more difficult.

    Conclusion


    As I said at the very beginning of the article, I tried to make a small list of everything that is necessary for a beginner mobile tester. Now, looking around the article with my gaze, I understand that the whole article perfectly reflects the testing itself as a phenomenon. At first it may seem that it is tiny, but with each new step it is overgrown with new details.

    As a final word, I want to wish you more devices to the farm, fewer final edits and maximum pleasure from the process.

    PS and the afterword will be parted by why why when testing it is important to remain attentive not only to the smartphone screen:

    image

    Also popular now: