Dex Redux

In October of 2014 a friend and past colleague of mine, Mustafa Ali, wrote a great Medium article to help solve a problem that many Android developers have encountered at one point in their career. The Dex method issue in short is a limit to the amount of executable methods allowed in Android's Dalvik Executable file (the one used to execute Android code). Shortly after writing his guide Google did the unthinkable... they released a support library. The library creates multiple dex files to help apps that are large get around the single dex file limit issue. Though this is a great solution for large apps Google suggests to use Proguard to strip your app of unused code or to reduce your dependency on libraries as a whole. The former is much easier than the latter.

I have to be totally transparent and say that I am not as familiar with the depth of the dexing process. I had initially planned on learning more about this and then writing up a large post however due to a family emergency I have not really had the time I wanted to devote to this. I am writing this in hopes to bring more exposure to the problem and maybe get some people who have a great understanding to help contribute to this post.

Finally The problem

So the other day while integrating two feature branches into our development branch. The following error occurred on our build server:

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':application:transformClassesWithDexForNormalDebug'.
com.android.build.api.transform.TransformException: com.android.ide.common.process.ProcessException: java.util.concurrent.ExecutionException: com.android.dex.DexException: Too many classes in --main-dex-list, main dex capacity exceeded

As I am not 100% familiar with what was going on I googled to see if this was something that others have been seeing. To my surprise there was only one result and that was a Google Issue Ticket. There isn't a ton of info in there, some people are suggesting that Google has broken a filter they have that creates the main-dex-list, which is a file that seems to be used to know which classes that should be included in the dex (I could be totally wrong about this).

Google acknowledged the issue, suggested a work around (which I have tried but haven't seen work) and opened another ticket to account for the actual solution. Because I couldn't get their solution to work I decided to test out a solution suggested by Alex Lipov, who was the one who created the Google Issue. Alex has a blog that talks through what he has found while working on this issue. Oddly enough he wrote this blog December of 2014 but didn't create the Google Tracker ticket till April 7th of 2016.

His updated solution is to access the CreateManifestKeepList class which is responsible for what gets kept for the dex. Then he modifies the accessibility of the class so it won't throw any exceptions while being modified. He basically makes the ImmutableMap mutable. It is a drastic approach but it works.

This is where the real fun starts. After utilizing his fix, I removed it so I could prove to myself that the fix is in fact real. When I removed his code from my Gradle build file...it still compiled without any issues. So I am interested in answering the following:

  1. What is the purpose of the main-dex-list?
  2. Why after removing the workaround am I not seeing the bug anymore?

Once I get back to real life I will pick back up and continue understanding the issue but I figured I would put this out and see if the community had any ideas what was going on. I would love to hear if you have had the same issue and how you solved it or if you have any extra info!

Elliott Chenger