Kotlin vs. Java: compilation speed

As all Android developers already know, Google recently announced official support for Kotlin on Android. Many risks associated with the use of this wonderful language in Android projects have been removed. But relevant, especially for very large projects, such as Badoo, is the question of build speed. I was glad to find that there are already studies on this topic on the network, and I want to share a translation of one of them.

So, if you are translating an application from Java to Kotlin, will it compile longer?

An earlier article discussed converting an entire Android application from Java to Kotlin . The code on Kotlin was less, and it was more convenient to maintain than on Java, so I came to the conclusion that it was worth it. But some developers do not want to try Kotlin, fearing that it might compile more slowly than Java. And this concern is fair: no one wants to waste time converting the code if as a result the assembly lasts longer. So let's look at the duration of compiling the App Lock application before and after converting to Kotlin. I will not compare the speed of Kotlin and Java line by line, but instead try to answer the question of whether converting the entire code base from one language to another will affect the overall duration of the assembly.

How I tested the build time

I wrote shell scripts for repeated runs of Gradle builds in different scenarios. All tests were performed sequentially ten times. Before each new scenario, the project was cleaned up. For scripts using the Gradle daemon , the latter stopped before starting the benchmark.

All benchmarks were carried out on a machine with an Intel Core i7–6700 operating at a frequency of 3.4 GHz, equipped with 32 GB of DDR4 memory, as well as a Samsung 850 Pro SSD drive. The source code was compiled using Gradle 2.14.1.

Tests

I wanted to run benchmarks for several common usage scenarios: clean builds with / without the Gradle daemon, incremental builds without changing files, incremental builds with a modified file.

The Java App Lock codebase contained 5491 methods and 12,371 lines of code. After converting to Kotlin, the number of methods decreased to 4987, and the number of lines to 8564. During the conversion to the architecture, no major changes were made, so measuring the compilation time before and after the conversion should give a clear idea of ​​the difference in build time between Java and Kotlin .

Clean builds without the Gradle daemon

This is the worst case scenario in terms of build time for both languages: starting a clean build with a cold start. For this test, I disabled the Gradle daemon.

This is how long all ten builds took:

Ten consecutive clean builds without the Gradle daemon

The average build time for Java is [the numbers are corrected according to the original data of the author - approx. translator] 24.5 seconds, Kotlin - 32.4 seconds: an increase of 32%. Not a good start for Kotlin, but most people compile their code in other scenarios.

Most often, we compile the same code base several times as we make changes to it. It is for this scenario that the Gradle daemon was developed, so let's turn it on and see what happens.

Clean builds with the Gradle daemon enabled

One of the problems with JIT compilers like the JVM is that they spend time compiling the code that runs in them, so that as it executes, the performance of the process increases. But if you stop the JVM process, then the performance gain is lost. Each time you build Java code, you usually have to start and stop the JVM. As a result, he does the same job anew each time. To solve this problem, Gradle comes with a daemon that continues to function between builds and helps maintain the performance gains provided by JIT compilation. You can enable the daemon using the Gradle command --daemonentered on the command line, or by adding org.gradle.daemon=trueto the file gradle.properties.

Here is the result of running the same build series, but with the Gradle daemon turned on:

Ten consecutive builds with the Gradle daemon enabled

As you can see, the first run takes about the same amount of time as in a script without a daemon. In subsequent builds, performance increases up to the fourth run. In this scenario, it is more advisable to estimate the average build time after the third run, when the daemon has already warmed up. In this case, a clean build in Java takes an average of 14.1 seconds, and on Kotlin it takes 16.5 seconds: an increase of 13%.

Kotlin is catching up with Java, but still lagging behind. However, regardless of the language used, the Gradle daemon reduces build time by more than 40%. If you are not using it yet, then it's time to start.

So, complete builds on Kotlin run a little slower than on Java. But usually we only compile after making changes to just a few lines of code, so incremental builds should show different performance. Let's find out if Kotlin can catch up with Java where it matters.

Incremental builds

Using incremental compilation is one of the most important features of a compiler to improve performance. In a normal assembly, all the project source files are recompiled, and in an incremental build, it keeps track of which files have changed since the previous build, and as a result, only these files and those that depend on them are recompiled. This can have a very strong effect on compilation time, especially in large projects.
Incremental assemblies appeared in Kotlin 1.0.2 , they can be included by adding kotlin.incremental=trueto the file gradle.properties, or through the command line .

So, how will the compilation time of Kotlin change compared to Java when using incremental compilation?

Here are the benchmark results, provided there are no changes to the files:

Ten consecutive incremental assemblies without modifying files

Now we test incremental compilation provided that one source file is modified. To do this, I changed the Java file and its Kotlin equivalent before each assembly. In this benchmark, this is a file related to the user interface, other files do not depend on it:

Ten consecutive incremental builds with one single modified file

Finally, let's look at the results of incremental compilation with one modified source file, which is imported into many other project files:

Ten consecutive incremental assemblies subject to a single key file change

As you can see, the Gradle daemon still has to warm up for two to three runs, but after that both languages ​​become very close in performance. If there are no changes to the files, it takes 4.6 seconds for a heated assembly, while Kotlin takes 4.5 seconds. If we change the file, but it is not used by other files, then Java takes 7 seconds to complete the heated assembly, and Kotlin takes 6.1 seconds. Finally, if the modified file is imported into many other project files, then when the Gradle daemon is warmed up, incremental Java assembly takes 7.1 seconds, and Kotlin takes an average of 6 seconds.

Conclusion

We measured performance under several different scenarios to see if Kotlin could compete with Java in compilation time. With clean builds, which are relatively rare, Java outperforms Kotlin by 10–15%. But most often, developers perform partial assemblies, in which a large gain in time is achieved through incremental compilation. Thanks to the working Gradle daemon and the incremental compilation enabled, Kotlin is not inferior, or even slightly superior to Java. An impressive result that I did not expect. I express my respect to the Kotlin development team for creating a language that not only has great features, but also compiles so quickly.

If you have not tried Kotlin so far for fear of increasing compilation time, then you can no longer worry: it compiles as fast as Java.

The raw data that I collected while running the benchmarks is here .