Speeding up Julia launch with PackageCompiler.jl

By going to the official website of the Julia programming language , you can see the statement: "Julia is fast!". However, in practice, new users are faced with the problem of slow loading of modules, especially graphic ones [ 1 , 2 ]. Dialectically, the reason for the slow start is the use of JIT compilation, the use of which ensures high language performance. If desired, the features of JIT can be found in other articles . This article will discuss a more practical task - how to speed up the launch of modules in Julia using PackageCompiler.jl .

This article uses julia v1.1.0 . In addition, the method was tested on julia v1.0.3 .
To evaluate the runtime, a trivial script was used:

@time using Plots
x = 0:0.01:10000
y = @. sin(π*x)
@time plot(x,y) # Первый, медленный запуск
@time plot(x,y) # Второй, быстрый запуск

By running this script on stock julia, the following values ​​are obtained:

  2.804964 seconds (5.03 M allocations:x 291.243 MiB, 5.35% gc time)
 13.546407 seconds (45.64 M allocations: 2.232 GiB, 9.00% gc time)
  0.013167 seconds (2.14 k allocations: 7.788 MiB)

As you can see, the first execution of the team plotlasts an incredible 14 seconds. Following him is already 0.01 s.

Let's try to improve these indicators. To get started, install the PackageCompiler module:

import Pkg
Pkg.add("PackageCompiler")

Next, we test the assembly of the module dependency Plots:

import Pkg
Pkg.add("Arpack")
Pkg.build("Arpack")

If the execution of the last command was interrupted with an error
ERROR: LoadError: LibraryProduct(...) is not satisfied, cannot generate deps.jl!
, then you became a victim of Issue # 5 . The problem is associated with the use of external dynamic libraries and will have to be solved after the integration of BinaryProvider . As a temporary solution, on the advice of blegat , we will assemble the library ourselves:

git clone https://github.com/opencollab/arpack-ng.git /tmp/arpack && cd /tmp/arpack
git checkout 3.5.0
bash bootstrap
./configure --enable-mpi --enable-shared
make
cp SRC/.libs/libarpack.so.2.0.0 ~/.julia/packages/Arpack/UiiMc/deps/usr/lib/

After performing these manipulations, the error should disappear:

import Pkg
Pkg.build("Arpack")

It’s time to compile the package Plots:

import Pkg
Pkg.add("Plots")
using PackageCompiler
compile_package("Plots")

Compilation lasts long enough with a lot of text in the console. Pay attention to the last line of the log:

┌ Info: Not replacing system image.
└ You can start julia with `julia -J /home/user/.julia/packages/PackageCompiler/oT98U/sysimg/sys.so` at a posix shell to load the compiled files.
"/home/user/.julia/packages/PackageCompiler/oT98U/sysimg/sys.so"

It indicates the location of the compiled image and a hint on how to start it.
Run the test script using this image:

julia -J /home/user/.julia/packages/PackageCompiler/oT98U/sysimg/sys.so /tmp/test.jl

  0.000251 seconds (501 allocations: 26.766 KiB)
  0.028105 seconds (22.55 k allocations: 8.847 MiB)
  0.017211 seconds (2.14 k allocations: 7.788 MiB, 18.16% gc time)

Thus, the time of the first start of the function plotwas reduced from 14 s to 0.03 s.

This method can be used to export images of several libraries, modify the system image, and compile static programs. To learn more about PackageCompiler.jl, refer to the description in the project repository .