How to stop being afraid of Proguard and start living


    Hello, I'm an Android developer and I'm no longer afraid of ProGuard ...


    Usually, this utility is remembered when faced with dalvik dex-limit issue or with the requirement to improve the security of the application. Unfortunately, setting up Proguard correctly is far from the first time. I often watched how many, having broken a project, turned off Proguard and turned on Mulditex support, and each time they were a little sad about this, because Proguard helps both to reduce the size of the application and improve its performance.


    As a result, I decided to write an article in which I can put all the useful information that I learned during several years of working with Proguard and which could help both the very beginners and those who already know something.


    What is it about


    Proguard is an open-source utility for optimizing and obfuscating Java code. This tool processes already compiled Java code, so it should work with any JVM language. More precisely, the language itself is indifferent to Proguard, only baytcode is important. All proguard bytecode manipulations can be divided into 3 main categories: Code shrinking , Optimization and Obfuscation .


    Code shrinking


    Yes, a rather strange idea to write code, and then delete it, but this is the reality of Android development. This, of course, is not about handwritten code (although it happens), but about tons of dead weight, which is brought by libraries of all sorts. Guava , Apache Commons , Google Play Services and other guys can inflate the size of the apk file from 500kb to a couple of tens of megabytes. Sometimes it goes so far that the program cannot compile due to exceeding the Dalvik methods limit . Proguard will help remove all unused code and reduce the size of the application back to several megabytes.


    Optimization


    In addition to removing unnecessary code, Proguard can optimize the remaining code. In its arsenal there is control flow analysis, data-flow analysis, partial evaluation, static single assignment, global value numbering, liveness analysis. Proguard can perform peephole optimizations, reduce the number of allocations of variables, simplify tail recursions and much more ( wiki ). In addition to such common operations, Proguard has optimizations that are useful specifically for the Android platform, for example, replacing enum classes with int-s, removing logging instructions.


    Obfuscation


    Finally, Proguard can turn all your code into an unreadable mess by renaming all classes, methods, and fields into sets of random (in fact, not entirely random) letters. This is a very useful option, since anyone can decompile your apk-file, and not everyone will have enough patience to understand the obfuscated code.


    Principle of operation


    Proguard works in 3 steps in the sequence described above: code shrinking → optimization → obfuscation . Each step is optional.


    Optimization step in case of Android SDK is disabled by default.


    For Proguard to work, you need to provide 3 components:


    • Your compiled code is an archive with the classfiles of your program and all the libraries you use (jar, aar, apk, war, zip, etc.). Proguard only modifies the already compiled code and has nothing to do with the source code.
    • Configuration file (s) - file (s) containing all the rules, options and settings with which you want to start processing.
    • Library jars (aar, apks, ...) are the classes of the platform on which your program runs. In the case of Android, this android.jar. These archives are needed only for the correct analysis of your code, they will not be modified (this does not make sense, because it android.jaris in the phone, we do not have access to it).

    Picture from Jeb Ware presentation, link at the end of the article
    (Picture from the presentation of Jeb Ware, link at the end of the article)


    Using library classes and your config files, Proguard determines all entry points to your program ( seeds ). In other words, it defines those classes and methods that can be called externally and which cannot be touched. Then, starting with the detected seeds , Proguard recursively walks around all your code, ticking the “usable” box with everything that I can reach. The rest of the code will be removed. In the config, you must specify at least one entry point. For a standard java program, this is a function main. In Android, there is no single point of entry into the program; instead, we have standard components (Activity, Service, etc.) that are created and called by the system. Fortunately, we don’t need to specify anything here, the Android SDK will create the necessary config for us.


    Related files


    After finding all the input points, Proguard writes them to a file seeds.txt.


    All the code that Proguard considered unnecessary is written to a file usage.txt. This is a rather strange name for a file containing a remote code, it would be more correct to call it unusage.txt, but we have what we have, just remember about it.


    In the obfuscation step, a file will be created mapping.txtcontaining the <original class name of the method | field | pairs> the <obfuscated class name of the method | field | pair. This file is useful when you need to deobfuse program, for example, read stacktrace. Manually mapping files back is not required, the Android SDK has utilities retraceand proguarduithat will help. Moreover, if you use Fabric Crashlytics, then their gradle plugin can independently find and load this file into the console, so you don’t have to worry about it.


    In the case of Android, these files are usually found in app/build/output/mapping/<product-flavor-name>/.


    Proguard also creates a file dump.txtthat contains everything that Proguard put into the final archive. He never came to me, but perhaps he will be useful to someone.


    How are things in Android


    Android Gradle Plugin can run Proguard on its own. All you need to do is enable this option and specify config files.


    buildTypes {
    <...>
    release {
        minifyEnabled true
        proguardFiles 'proguard-rules.pro', getDefaultProguardFile('proguard-android.txt')
    }
}

    minifyEnabled true - turn on Proguard at the build stage


    proguardFiles- list of config files. Rules from all config files will be added to the general list in the order they appear.


    proguard-rules.pro - this is our config file with project-specific rules


    getDefaultProguardFile('proguard-android.txt')- a function that returns a standard config file for Android applications. He lies inAndroidSDK/tools/proguard


    In fact, the Android SDK has two config files: proguard-android.txtand proguard-android-optimize.txt. In the first of these, there is an option -dontoptimizethat turns off all optimizations. If you want to enable optimization, use the second config.


    In addition to these standard Android SDK configurations (aapt), the rule set for resources is automatically generated: aapt checks all xml files (including the manifest) to find all activations, services, views, etc. and generate for them the necessary rules. Generated rules can be found in app/build/intermediates/proguard-rules/<flavor>/aapt_rules.txt. You do not need to specify it yourself, Android Gradle Plugin will add these rules automatically.


    Picture from Jeb Ware presentation, link at the end of the article
    (Picture from the presentation of Jeb Ware, link at the end of the article)


    Configs


    Setting up Proguard is the most basic part of working with it and at the same time the most difficult. Incorrect config can easily break the compilation of the application and the application itself in runtime. All available configuration options are documented in detail on off. site.


    Among all the options, I would single out 3 of the most important groups:


    • keep rules - All possible entry points to the program. Rules telling Proguard which classes or parts of classes should be kept unchanged or which modifications are allowed for specific classes.
    • optimization tuning - indicate which optimization is permissible, how many optimization cycles need to be done.
    • work with warnings, errors and debugging

    Keep rules


    This is a set of options designed to protect your code from the merciless Proguard. In its most general form, this rule looks like this:


    -keep[,modifier,...]class_specification

    keep - the most common of these options (there are others), telling Proguard to save the class itself and all its members (class members): fields and methods.


    class_specification- a template pointing to class (s) or parts thereof (class members). The general view of the template is very large, it can be viewed off. documentation . You can refer to it, but in general, you can just remember that we have the opportunity to make a template from the following components:


    • select all classes with a specific name, package
    • select all classes that inherit / implement certain classes / interfaces
    • select all classes with specific modifiers and / or specific annotations
    • select all methods with a specific name, modifiers, arguments and return value
    • select all fields with a specific name, modifiers, a specific type.

    • it is possible to use wildcards


      Once again, this is not a strict description of the template, it is rather a list of the features that we have. Here are some examples:



    -keep public class com.example.MyActivity
    keep class com.example.MyActivity


    -keep public class * extends android.app.Activity
    save all public classes that inherit android.app.Activity


    -keep public class*extendsandroid.view.View{ 
      public <init>(android.content.Context); 
      public <init>(android.content.Context, android.util.AttributeSet); 
      public <init>(android.content.Context, android.util.AttributeSet, int); 
      public void set*(...); 
}

    find all public classes that inherit android.view.Viewand save in them 3 constructors with certain parameters + all public methods, with a modifier void, any arguments and a name starting with set. All other parts of the class can be modified.


    -keep class com.habr.** { *; }
    save all classes and all their contents in a package com.habr


    modifiers - addition to keep-rule:


    • includedescriptorclasses - in addition to the specified class / method / field, you must save all classes that are found in their descriptors.
    • includecode - The contents of the method pointed to by this particular rule cannot be touched either.
    • allowshrinking- the classes pointed to by this rule are not input points (seeds) and can be removed, but only if they are not used in the program itself. However, if after the codeshrinking this code remains (due to the fact that someone uses it), this code cannot be optimized / obfuscated.
    • allowoptimization - the classes pointed to by this rule can only be optimized, but not deleted or obfuscated.
    • allowobfuscation - the classes pointed to by this rule can only be obfuscated, but not deleted or optimized.

    Besides keep, there are a few more options:


    -keepclassmembers - indicates that it is necessary to save class members, if the class itself is preserved after code shrinking.


    -keepclasseswithmembers- indicates that it is necessary to save the classes, the contents of which fall under the specified template. For example, it will -keepclasseswithmembers class * { public <init>(android.content.Context); }save all classes that have a public constructor with one type argument Context.


    -keepnames- abbreviation for -keep,allowshrinking.


    -keepclassmembernames- abbreviation for -keepclassmembers,allowshrinking.


    keepclasseswithmembernames- abbreviation for -keepclasseswithmembers,allowshrinking.


    Optimization tuning


    The most important option here is the flag -dontoptimize. If present, no optimization will be performed and all other optimization options will be ignored.


    There are many optimization options, but the following are the most useful to me:


    -optimizations optimization_filter- listing all the ways you want to use. It is better to use the set that is specified in proguard-android-optimize.txtor its subset. A list of all optimizations can be found here .


    -optimizationpasses n- number of optimization cycles. Several cycles can improve the result. At the same time, Proguard is smart enough to stop the cycles if it sees that the result has not improved since last time.


    -assumenosideeffects class_specification- indicates that this method has no side effects and only returns some value. Proguard will delete calls to this method if it finds that the result it returns is not used. Perhaps the most common use of this option is to remove all debug logs:-assumenosideeffects class android.util.Log { public static int d(...); }


    -allowaccessmodification- show all that is hidden :) Excellent option, allowing you to get rid of a heap of artificial accessor-methods for nested classes. Only works in conjunction with-repackageclasses


    -repackageclasses- allows you to move all classes into one specified package. This applies more to obfuscation, but at the same time, gives good results in optimization.


    Other useful options


    -dontwarn and -dontnote


    Proguard is very smart and always reports suspicious places while analyzing the code, sometimes notes, sometimes warnings. If your build is not going to when Proguard is enabled, be sure to read all the logs it produces, it will write what went wrong and, most likely, even tell you how to fix it. After reading all the messages, you either correct the problem, or ignore the message of one of these options, if you are sure that there is no problem.


    For example, it happens that some java-library uses platform-classes that are not in android.jar and Proguard will warn about it. If you are sure that this library works fine in the Android environment, you can disable this warning.-dontwarn java.lang.management.**


    -whyareyoukeeping class_specification - A useful option that prints the reason why Proguard decided not to touch this class / method.


    -verbose - print more detailed logs and exceptions


    -printconfiguration - print the full list of options from all config files that were used, including the rules from the libraries and generated via aapt.


    -keepattributes SourceFile, LineNumberTable- saves meta-information (file names, line numbering) to be able to debug the code in the IDE and get a meaningful stacktrace. Be sure to add this option.


    Practice


    It usually happens like this: turn on Proguard and it breaks the whole project to you giving a ton of errors. Many in this step turn off Proguard and try not to return to it. I will try to give some tips to make this transition process easier.


    Decide on starting entry points


    If you are an Android developer, everything is very simple here - just choose one of the two standard configs from the Android SDK: proguard-android.txtor proguard-android-optimize.txt, they will take care of everything that should remain intact.


    Check all libraries


    Recently, more and more libraries are being distributed with ready-made proguard-configs. Proguard is able to look inside the archive, find the library config and add it to the remaining options. Check each library that you use for the presence of such a config.


    (the contents of the aar file of one of the libraries)
    (the contents of the aar file of one of the libraries)


    If you use Google Play Services, then the plugin com.google.gms.google-serviceswill select the config you need yourself.


    If the authors of the library do not pack the config into the archive, perhaps they took care and wrote the rules on their website, the repository page or in the README file. Try to find the config for the version of the library you are using.


    If you can’t find ready-made rules anywhere, you will have to read the logs and solve the problem individually. Most likely, you need to add keep rules for the library code that is broken. Or ignore errors if they do not interfere with the program.


    Inspect your code


    You can see which code can be sent under the knife, but you should carefully look at all the places where reflection is used one way or another:


    • Class.forName (...) (the documentation promises that Proguard can identify such code, however, there have been cases, it is worth checking)
    • Modelki / entity-classes, which are used in seselizatsii, mapping. All classes, the names of the fields (sometimes the classes themselves) which are important to keep (Gson, RealmIO, etc.)
    • calls to native libraries via JNI

    Tests


    If a class / method is used only in tests and nowhere else, Proguard will remove this code. This is a common situation if you have TDD :) For this case, I have a separate config where I add classes that are not yet integrated into the project, are not used anywhere, but which need to be tested.


    In addition to the Android Gradle Plugin proguardFilesinstructions, there is still testProguardFiles. This instruction is needed to specify the configs that will be applied to the test application that is generated to test your application when you run instrumentation tests. This is usually used to achieve the same optimization / obfuscation in both apk-files so that there is no desynchronization between them. Link .


    APK Analyzer


    Android Studio has such a great tool. You can open it either through Find Action -> Analyze APK, or by opening the apk file itself in Android Studio. Analyzer shows a lot of useful information, but now we are interested in the code. To see what was eventually packed into an APK file, you need to select a file.classes.dex



    By default, you will be shown exactly the result code that has passed the shrinking and optimisation steps. However, you can click on the Load Proguard mappings ... button , add seeds.txtand usage.txtto see the code that has been deleted.



    If for some reason Proguard has modified the code you need, find it in the Analyzer and right-click it to select Generate Proguard Keep Rule . Analyzer will generate you a choice of several variants of the rules, from the most general to the most specific, select ONE of them.




    For authors of libraries


    If you are doing the Android library, you can add a proguard-config to your clients as follows:


    buildTypes {
    release {
        consumerProguardFiles'proguard-rules.pro'
    }
}

    In my opinion, it is better not to be zealous with optimizing and obfuscating your library, but to provide this opportunity to your customers. A good tone is to add to the config what clients will still have to add if they include Proguard. However, if you still want to add security, it is obvious that you need to protect the entire pulic API of your library from Proguard, including descriptors and signatures.


    R8, DexGuard and Redex


    R8 is a new tool from Google in exchange for the current Proguard. Wait, do not try to forget everything that you just read in the article, just consider it as a new Proguard. Google promises to keep the entire public api, so that all configs will work as before. The project is still in beta, but you can try it yourself.


    DexGuard is a paid utility from the developers of Proguard. It can be used together or instead of Proguard. It is argued that DexGuard can do everything that Proguard can, but better. Unfortunately, I did not have a chance to try it, if someone has experience, please share.


    Redex is another dex optimizer from Facebook. It is reported that with it you can achieve up to 25% increase in productivity and reduce the size of the application by applying the tool to the code already processed by Proguard.


    Instead of conclusion


    Do not be afraid to use Proguard, do not be lazy and spend some time on the setup. By this you will reduce its size, increase the speed of work, and add the loyalty of your users. At the same time, try to create effective Proguard-configs, do not write "carpet" rules, otherwise an angry Jake Wharton will come to you and scold you.



    Resources


    Proguard website . There is also information about DexGuard.
    Various examples of rules
    R8
    Record a presentation How Proguard Works with DroidCon
    Record a presentation Effective ProGuard keep rules for smaller applications (Google I / O '18)
    How to enable and configure Proguard for Android Redex
    Wiki
    Page

    Tags:

    Also popular now: