Gradle 4.1 Task Dependencies

While migrating to Gradle 4.1 and Android Gradle Plugin 3.0 we discovered an issue that wasn’t documented. I would love to share this issue with you and explain how we fixed the problem.

The Problem

My team develops libraries that get integrated into the apps we have at Under Armour and, because of that, most of the time we are working on AARs that are consumed by those apps. In our deploy script we run a series of tasks which, up until Gradle 4.1, all ran sequentially (even though you could enable parallel tasks prior to 4). So here is a rough idea of what our CI server runs when we merge code in:

./gradlew clean assemble artifactoryPublish

The problem we found after upgrading to 4.1 is that our tasks were now being run in parallel and our artifactoryPublish task would run before the end of the assemble task. This as you can imagine adds some complication, unless you can get the finished AARs from an alternative reality where the build is already done.

This is now parallelized due to the Worker API which was introduced in Gradle 4.1. I won't delve too deeply, but this new API was made with Android in mind and allows work from a task and build level to be parallelized with minimal effort (from the app developers side). The effort is mostly on the plugin developer, which I am not so I am going to refrain from saying more. Although I will note there is a great talk given by Paul Merlin and Gary Hale on this topic I would suggest watching.

After looking a bit more into the Worker API, I found that not only will this allow in-task parallel work, but it will also allow tasks with no relationship or dependencies to run in parallel as well:

(From the Writing Custom Tasks Guide): Of course, any tasks that are dependent on this task (and any subsequent task actions of this task) will not begin executing until all of the asynchronous work completes. However, other independent tasks that have no relationship to this task can begin executing immediately.

I was unaware of this initially, so after about two hours of experimentation we found that even though there is no way to turn the feature off there are two options for getting around the issue.

The Solution

Solution 1: Run tasks one at a time

Instead of running:

./gradlew clean assemble artifactoryPublish

We could do the following:

./gradlew clean 
./gradlew assemble 
./gradlew artifactoryPublish

Though I think this looks ugly, it gets the job done. Many people will scoff at this, there are times when the quick-and-dirty helps us get through a gauntlet unscathed. Most of the time I disregard these types of solutions, but in a time of need it is still valid.

Now, the reason that I say Solution 1 is viable is that I suspect many people in the Android community are not Gradle experts. I have spent 10-15% of my work time over the last year learning more about Gradle and I feel confident in my knowledge.. I suspect however, that I have still probably only touched 10 to 20% of Gradle.

Solution 2: Create task dependencies in your build gradle files

If you get into a situation where tasks are running out of order due to the parallelization of tasks in 4.1., think about adding task specific dependencies. In order to enforce the dependency between the assemble task and the artifactoryPublish task you can put the following line in your build.grade:

artifactoryPublish.dependsOn assemble

Note: If you have multiple modules in your gradle project you can cascade this to each of them using the subprojects configuration block in your project build.gradle.

The dependsOn method is part of the Task API and allows us to tell our artifactoryPublish that we depend on the assemble task to finish before it can run. Creating this dependency will now stop the assemble task from running before it should and causing build issues.

There have been many things that have changed with the Android plugin and the team at Google have done a great job documenting almost all of these changes. However, there are always things that are overlooked or may be under documented. I hope if you ran into task parallelization issues that this helped you get through the issue and I look forward to hearing other people's experiences.