What the headache of Android DevOps engineer hurts / JUG Ru Group's blog

It so happened that DevOps tools are usually illustrated using the CI / CD as an example of some large-scale web service. Partly it happened for historical reasons, partly wonderful books like Google SRE Book played a role .

To hell, let's look at something really new . At Mobius 2017, Jing Li from Viacom comes to us with the talk “Android meets Docker” .

On the eve of the conference, we managed to find a few minutes in his busy schedule and ask a couple of questions. In this interview, Jing talks about DevOps in mobile development, provides examples of tasks, and gives concrete recommendations for improving your DevOps process.

- Hi. Tell Habra about yourself. Where do you work, what do you do?

- Previously, he was a developer at Sony, Nokia, eBay and Viacom (they are the owners of MTV, Paramount Pictures and so on). At the moment, I'm preparing for the next big adventure, relaxing at home and working on my open source projects (https://github.com/thyrlian ). Information technologies are developing very fast now, every day you need to learn too much new. Therefore, after work, I try to engage in machine learning and virtual reality.

- How did you get into mobile development? Do you do DevOps on an ongoing basis or was it some kind of one-time job?

- Just like children like to play game consoles, I became addicted to my mobile phone since school, when the grass was greener and the screens were monochrome. It is clear that after graduation, I immediately started mobile development. In mobile development, DevOps happens every minute, especially when someone uploads changes to the code. Just like in normal development, DevOps requires a lot of work to write code, the right architecture and complex configurations. Given the combination of these three aspects, it is difficult to say that this is some kind of one-time job. This is an ongoing process, in which you can track your progress by many specific signs:

• Speed - the time of your reaction should be such that the developers do not have to wait for an answer for too long;
• Stability - no one wants to work with a crumbling system, this is a waste of time;
• Reliability - the results should be quite high quality and reliable, no false positives;
• Consistency - the results should not differ between several attempts to start, or between different machines;
• Flexibility - using modular systems allows you to write code once, and run it everywhere - (Jenkins, CircleCI, etc.);
• Unification - you need to get rid of language barriers and use one universal language for gluing different parts of the system;
• Scalability - with the possibility of parallel tasks, it is necessary to choose the right ways to resolve engineering compromises;
• Reproducibility - never lose control of your version control system configurations.

Often I see how people repeatedly press the “restart assembly” button in the hope that the assembly will turn green. This is due precisely to problems with stability, reliability and consistency.

- How exactly does DevOps on mobile platforms differ from what happens when developing web services (such as Google) or desktop applications? Is there anything special about him?

- Toolchains for traditional development: backend, frontend, desktop - they are quite functional and run natively. On mobile phones, you need to choose between launching on a computer or on a mobile platform. Which, in turn, can be either a physical device or a simulator. In addition to the environment, there are dependencies on various third-party services. The most common part is publishing: Google Play, the App Store, or some of the publishing services like HockeyApp or Fabric. For each of these services, there are console utilities that can be used in automation. Alternatively, you can use tools like Fastlane. Often you need to take care of side tasks. One of the main tasks is design. For example, you need to decide how to automatically import and convert designer resources in a simple way. If the application works in several countries, you may encounter the task of synchronizing localizations from collective translation services such as Crowdin or POEditor. This is not all a headache - for example, we wanted to monitor application performance after publication. Or here's how to get user ratings and reviews? How to display in the form of graphs on a large monitor hanging on the wall? All this data is obtained from external services, integrating them with each other is not at all simple. How to get ratings and user reviews? How to display in the form of graphs on a large monitor hanging on the wall? All this data is obtained from external services, integrating them with each other is not at all simple. How to get ratings and user reviews? How to display in the form of graphs on a large monitor hanging on the wall? All this data is obtained from external services, integrating them with each other is not at all simple.

“You always have interesting stories left.” Tell me about any problem that you had to devopit for a mobile platform? Something off topic is better to keep the intrigue.

- In those days when Google had not yet released Android 6.0 Marshmallow, the system during installation asked the user to provide appropriate permissions. If the new version of the application wanted extended rights, the application could not be automatically updated - at first the system asked the user for permission for this extended set of rights. Once it happened in our team - we added some kind of library, which suddenly quietly added some additional permissions. This instantly brought down the update rate of our application. It became clear that in Continuous Integration you need to add a check that would signal about changes in the list of permissions. Unfortunately, ready-made solutions did not exist at that time. The first thing that came to mind was to write a unit test that programmatically checks all this, but writing such a test did not work (you cannot get all the permissions). The second attempt is to parseAndroidManifest.xml, and, of course, this venture was also doomed to failure, because the permissions of third-party libraries lie in their own manifests and are managed by the manifest collector. Finally, I decided to analyze the generated APK, extract the full list of permissions from it, and then compare it with the previous, previously saved, results (https://github.com/thyrlian/NoNewPermissionForAndroid ). This approach worked fine, but as you can see, it is not intuitive and requires additional work. By the way, starting with Android Studio 3.0, the built-in APK Analyzer provides similar functionality out of the box (https://developer.android.com/studio/build/apk-analyzer.html ). The ecosystem is getting better and better.

- In Russia there is a culture of highly specialized teams: separate development, separate testing, etc. Tell me, what tools should a sysadmin learn to better interact with mobile developers and help them more efficiently? 

- Sysadmins seem like wizards to mobile developers. From the experience of my previous work, a good wizard in a mobile team looks like this:

• He is not just a console master, he is also well versed in the project. For example, Gradle is the de facto official build tool for Android, so it's wise to choose Groovy as the main language for all build scripts. But on iOS CocoaPods and Fastlane are very popular, so it’s worth choosing Ruby already.
• Sysadmins will not be able to guard the peace of the team 24/7. Team members themselves must understand how to break through typical assembly problems, especially through small but urgent problems. When you don’t need to constantly wait for someone to come and solve your problem, not only your efficiency grows, but your job satisfaction also increases.
• "Mobile developers" are called "mobile developers" because they are well versed in mobile phones. They don’t need to be well versed in databases or anything else - in fact, they usually don’t understand. But as soon as the mobile application starts communicating with the server via the API, if problems arise, you need to debug all this, and for the mobile developer this debugging is a big pain. Do not even try to teach them complex SQL queries or how to grab logs with a huge command in the console. It is better to arrange for them a short introductory course on using Kibana or Grafana, into which you yourself have driven frequently used queries.
• On the other hand, the admin must clearly know the needs of the mobile developer and understand the goal that he wants to achieve. You may have tuned Sonar rules for Java backend thousands of times, but these standard rules are not suitable for Android. Android developers need their own custom settings. Before giving them the turnkey solution, carefully check that this solution is specifically tailored for mobile development.

- What technologies should a developer learn in order to build a good infrastructure and the right DevOps process? How to write mobile applications so that it is more convenient to devopit them ?

- You need to learn at least one scripting language, because in a typical build environment you cannot rely solely on Java / Objective-C. Assembly functionality needs to be turned into modules, small fragments that not only simplify refactoring, but also make it easier to adapt to various conditions (for example, when switching from Jenkins pipelines to Cricle CI, or if you need to jump from Circle CI to Travis CI). Mobile development is rapidly evolving, so you should not rely on the use of any specific third-party solutions, because they can become obsolete at any second. Lay on official tools and public APIs from Apple and Google, and tools created on their basis. And most importantly, think broadly, consider solutions that are different from your current configuration. For instance,

- OK thanks! Meet me at Mobius 2017!