“65K methods is enough for everyone” or how to deal with the limit of DEX methods in Android

It happened all of a sudden. You just wrote the code for your application for android, you liked it, and you enjoyed the process. You added a cool library to get extra features and write simpler code. But instead of a working application, the output is a terrifying inscription:

Unable to execute dex: method ID not in [0, 0xffff]: 65536
Conversion to Dalvik format failed: Unable to execute dex: method ID not in [0, 0xffff]: 65536

And you are in a stupor, you are unable to create a DEX file for the APK. You have no idea what this is and how to fix it. And whatever you do, it will lead you to the most logical state: PANIC .

Meet the enemy


After you have tried all your usual tricks (in the following order: cleaning the project, restarting the IDE, building via the command line, restarting the laptop, calling a friend and even a ten-minute break in the hope that the problem will disappear by itself), you are forced to admit the cruel truth : the problem is still here. Therefore, it would be nice to know what caused it.

There are many posts on the Internet about this problem: here is one of them , one more , and one more ; Oh, and this one and this one too . So what happened? Essentially, it looks like you are faced with something that is often (and mistakenly) called the Dalvik 65K methods limit. Briefly (thanks to fadden ):

You can refer to a very large number of methods in a DEX file, but you can only call the first 65536, because this is all the memory that you have for the method invocation instruction.
[...] the number of methods you can reference is limited, not the number of methods you define. In other words, if your DEX file contains only a few methods, but together they call 70,000 different externally defined methods, you will exceed the limit.

What we have? Your application has too many methods written by you or enclosed in library JAR files. For this reason, the dx tool cannot write the addresses of some methods simply because they do not fit into the field specified for that in the DEX file (which in turn contains compiled Java classes). And that is why the problem should be called the DEX 65K methods limit.

And yes, you understood correctly. This problem will not disappear even when the android switches to the new ART runtime, until Google decides to “fix” the DEX format or replace it with something else.



Let's count


You are probably surprised how you managed to bury more than 65,000 methods in your precious APK. Or maybe not. In any case, there must be a way to count these methods and determine their source.

The header of the DEX file already contains information about the number of method references (for clarity, this number represents unique method references, not the sum of calls for each method). But we also want to see which package adds an exorbitant number of these methods.

The best tool I found was made by Mihai Parparita , who wrote a simple bash script called dex-method-counts. The script is very fast and produces XML, which compares the name of the package with the number of its methods. Another solution belongs to Jake Wharton, it gives exactly the same result, but requires much more time due to its recursive nature (Jake also wrote an interesting article on this topic).

Now we have a good tool in our hands, so why don’t we test it on a small test application? Let's call it SampleApp. Here is the code that is contained in our simple Activity:

package com.whyme.example;
import android.app.Activity;
import android.os.Bundle;
public class SampleActivity extends Activity {
  @Override
  public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    dummyMethod();
  }
  private void dummyMethod() {
    System.out.println(“Oh gosh.”);
  }
}


I also included Google Play Services 5.0.77 in the application, although I do not use this library at all. The meaning of my action will be clarified in a few lines below. Now let's analyze the output of dex-method-counts (compressed, for simplicity):

Read in 20332 method IDs.
: 20332
   : 2
   android: 834
     […]
   com: 18802
     google: 18788
       […]
     whyme: 14
       example: 14
   dalvik: 2
   system: 2
   java: 624
     […]
   javax: 5
     […]
   org: 63
     apache: 24
       […]
     json: 39


Wait. WHAT? I just defined one method in my Activity and already have 20,000 methods in my DEX file?



The culprit is obvious.

Google Play Services: Mixed Feelings


There is a non-zero chance that you are familiar with Google Play Services . Google’s smart move to support the API of many of its services, up to Android 2.3 Gingerbread, with a new release every six weeks.

However, everything has a price, which in this case appeared in the form of a HUGE number of methods that Play Services carries with it. We are talking about about 19,000 methods. And with a limit of 65536, this means that a third of all the methods that we can include are already exhausted. Like this.

Now, before we move on to the “solution” of this problem, I think a little thought will be appropriate. Some developers, both inside and outside of Google, share a common point of view, which says: “If you are faced with a restriction, then you are a bad, very poor developer, you deserve to be punished (and you must burn in hell).” From a more impartial point of view, this means that you recklessly turned on too many libraries, either because you are too lazy, or because your application does too much. Personally, I do not share this point of view. I believe that these are just excuses for two simple things:

1. I do not want to acknowledge the problem / I do not want to fix it.
2. I did not go beyond the limit, so I am a cool developer, and you are doing something wrong.

Maybe the limit was set for the following reasons: “This limit will determine the line between a good and a bad programmer for all future generations. May it be so." But something, I doubt it.

Throw away all unnecessary


In a world where justice triumphs, Google has realized the depth of the problem and decided to reduce the damage it does by dividing the giant Play Services into modules that can be included depending on the functionality that your application uses. For example, if my application is not related to games, it makes absolutely no sense to include all the Play Games API in it. This is the most reasonable move that can be taken and which can be carried out.

However, Google does not fully understand the problem, so we’d better find another way to solve it on our own. As it was said, we will throw away parts of google-play-services.jar, those parts of the API that we definitely will not need.

There are a couple of ways to do this. You can do it manually, but remember that you have to do it every six weeks.

You can rely on the JarJar library , which allows you to remove the desired packages from the jar file. Clear and simple.

If you, like me, prefer the good old command line, then you can use the script that I use for my projects, called (you won’t believe) strip_play_services.sh . You can control the operation of the script using the strip.conf configuration file that comes with the kit. Using it, you can select the Play Services Library modules to be disabled. Essentially, the script does the following:

1. Extracts content from google-play-services.jar
2. Checks if strip.conf exists, and if it exists, then uses it for configuration; if not, it creates a new one, writing all the modules from the Play Services Library there
3. Based on strip.conf, it removes unnecessary modules from library
4. Repacking the remaining modules in google-play-services-STRIPPED.jar file

Here is a simple configuration of strip.conf file:

actions = true
ads = true
analytics = true
appindexing = true
appstate = true
auth = true
cast = true
common = true
drive = false
dynamic = true
games = false
gcm = true
identity = true
internal = true
location = false
maps = false
panorama = false
plus = true
security = true
tagmanager = true
wallet = false
wearable = true


And that’s all. Now, for an example, let's calculate the number of methods of our test application again and see what has changed:

Read in 11216 method IDs.
: 11216
  [...]


This is already something. By choosing the right configuration based on the needs of your application, you can control the “thickness” of Google Play Services and leave more room for methods that you (possibly) really need.

But you also have to be very careful. The Play Services Library is built very well (in terms of modularity), and you can safely remove the “games” module without affecting the “maps” module. But if you remove the module that is used by others (internal, simply put), then nothing will work. Use trial and error!



Is there any other way?


Of course have. Launching ProGuard for your application can help, as it removes unnecessary methods from your code and in addition reduces the size of the APK. But do not expect a big reduction, this will not happen.

Another solution is to create a second DEX file that contains part of your code that you will access through interfaces / reflections and which you will load through custom ClassLoader ( more ). This solution may be more effective in some cases. However, this path is very nontrivial.

I found another solution to my good friend Dario Marcato . He created a Gradle task to remove unnecessary modules. If you are using Gradle, then be sure to give it a try! .

Afterword


Is this a translation of the original article by Sebastiano Gottardo , [DEX] Sky's the limit? No, 65K methods is . If the translation seems “clumsy” to you or if you find spelling / syntactic / factual errors in the text, please write to me in the PM.

Also popular now: