Pillow-SIMD

2.5x faster operations compared to Pillow and 10 times faster than ImageMagick

Pillow-SIMD is a fork follower of the Pillow image library (which itself is a fork of the PIL library, now deceased). “Follower” means that the project does not become independent, but will be updated with Pillow and have the same version numbering, only with a suffix. I hope to release Pillow-SIMD versions more or less expeditiously immediately after the release of Pillow versions.

Why SIMD

There are several ways to improve image processing performance (and all other things, probably, too).

1. Better algorithms can be used that give the same result.
2. A faster implementation of the existing algorithm can be made.
3. You can connect more computing resources to solve the same problem: additional CPU cores, GPUs.

The coolest thing is when you can use a faster algorithm, as when in Pillow 2.7 the Gaussian convolution-based blur was replaced by a sequence of box filters. Unfortunately, the number of such tricks is very limited. The idea of ​​using more computing resources is also very tempting. But unfortunately, often they are either not there, or they cost extra money (as is the case with rented servers). Using GPUs for calculations is generally a non-trivial task associated with selecting a specific hardware and properly configuring drivers. The most reliable way remains - to try to make the existing code work faster on the existing hardware. And here SIMD instructions fit perfectly.

SIMD means: “single instruction, multiple data”. In classical programs, we take operands, perform an operation, and save the result. In the case of SIMD, we immediately take a bunch of operands, do the same action all at once, and save a bunch of results. For a processor, this is easier than performing the same steps several times. There are a huge number of processor command extensions with SIMD instructions, for example: MMX, SSE-SSE4, AVX, AVX2, AVX512, NEON.

In the current version, Pillow-SIMD can be compiled using the SSE4 (default) or AVX2 extensions.

Project status

Pillow-SIMD is suitable for production. Various versions of Pillow-SIMD have been running Uploadcare servers for over a year now . Uploadcare is a service for storing and processing user-generated content and the main sponsor of Pillow-SIMD.

Currently, the following operations are accelerated in the SIMD version:

• Resize (convolution-based resampling): SSE4, AVX2
• Gaussian Blur and Box Filters: SSE4

Performance

The numbers indicate the number of megapixels of the original image processed per second. For example, if you resize an image of 7712 × 4352 in 0.5 seconds, the performance will be 67.1 Mpx / s.

Already in the process of editing, I realized that I feel confusion in the megapixel for ImageMagick 10 ^ 6 pixels, and in the megapixel for Pillow - 2 ^ 20. But this does not greatly affect the overall picture.

Tested:

• ImageMagick 6.9.3-8 Q8 x86_64
• Pillow 3.2.0
• Pillow-SIMD 3.2.0.post2

Pillow is always faster than ImageMagick, and Pillow-SIMD is faster than Pillow about 2-2.5 times for the SSE4 version. Basically, the AVX2 version is 10-15 times faster than ImageMagick.

Tests were performed on Ubuntu 14.04 64-bit, running on an Intel Core i5 4258U processor with AVX2. All tests used only one processor core.

ImageMagick performance was measured by the convert command-line utility with -verboseand arguments -bench. The selected filters exactly match the existing filters in Pillow:

• PIL.Image.BILINEAR == Triangle
• PIL.Image.BICUBIC == Catrom
• PIL.Image.LANCZOS == Lanczos

For testing, such scripts were used .

Why is Pillow so fast

There are no tricks here; high-quality resize and blur methods were used for the tests. The results almost pixel by pixel coincide with a small error. The difference is only in the efficiency of the algorithms themselves. In Pillow 2.7, resampling was rewritten using pre-computed coefficients, less use of floating-point numbers and transposition, effectively using the processor cache.

Why Pillow-SIMD is even faster

Of course, due to the use of SIMD commands. But I still have a few thoughts on how to improve this result.

• Efficient memory handling At the moment, each pixel is loaded into the SSE register separately from the memory, while it is possible to read 4 pixels at a time in one SSE register.
• Calculations on integers Despite the fact that modern processors work very effectively with floating-point numbers, there are two reasons to believe that working with integers will be more efficient: operations on integers are algorithmically simpler; to work with them does not require additional covert.
• Aligning data in memory loading and unloading data from SIMD registers is faster if the addresses in the memory being exchanged are aligned.

Why not pour changes back into Pillow

In short, it is very difficult. Pillow supports a large number of architectures, not only x86. But even on x86 Pillow for some platforms is distributed in the form of compiled executable files. To be able to use the SIMD-instructions in the code, you need to pass to the compiler arguments, authorizing the use of the most advanced regulations that we want to use: -mavx2. After that, you need to check the processor capabilities at runtime and enable one or another code branch depending on them. The problem is that such arguments automatically compose code hidden under preprocessor conditions.if (__AVX2__)and below, which may have no runtime checks. The saddest thing is that such code is really found, at least when compiling GCC, and executable files without explicit use of AVX2, but compiled with -mavx2, start to crash. Of course, you can collect different versions of the library with different compiler options and dynamically connect them, but this [see beginning of this paragraph].

Installation

The good news is that to install the SSE4 version it’s enough to write as usual pip install pillow-simd, and if your processor is capable of SSE4 (I think the probability of this is about 95%), everything will go fine. Remember to remove the original Pillow package.

If you want to build the AVX2 version, you need to pass additional flags to the compiler. The easiest way to do this is by setting an environment variable CCduring installation and compilation.

$ pip uninstall -y pillow-simd ; CC="cc -mavx2" pip install pillow-simd

Sometimes it happens that not only you, but also other packages that you use depend on Pillow. And even if these packages do not really need fast resampling, they still install Pillow without a SIMD, which can be imported first. For this, such a hack can be useful when installing from Github:

$ pip install -e git+https://github.com/uploadcare/[email protected]#egg=pillow

Then, during the installation of another Pillow-dependent package, another version of Pillow will not be installed:

$ pip install xhtml2pdf -e git+https://github.com/uploadcare/[email protected]#egg=pillow