# Pillow 2.7 - Significantly Improved Quality and Productivity

- Transfer

On January 1, 2015, a new version of the Pillow 2.7 image library was released on schedule . Since many changes to it were made by the Uploadcare team , we are pleased to present you an extended version of the release notes for this version.

To begin with, remember how it all began. Pillow is a friendly fork (as its authors call it) of the popular PIL library, Python Imaging Library. The latest version of PIL 1.1.7 was released in 2009 and mainly contained bug fixes. Initially, Pillow was conceived as a project only to tidy the assembly of PIL, and the developers recommended sending all bugs that were not related to the assembly to the original PIL. But time passed, PIL was rapidly becoming obsolete, bugs did not decrease, and Python 3 loomed on the horizon. Therefore, with Pillow 2.0, everything has changed. “Pillow 2.0.0 adds support for Python 3 and includes many bug fixes from around the Internet” reads the project descriptionon PyPI. And since then it started. Every three months there were versions with a huge number of bug fixes and other improvements from various developers. The most significant innovation during this time was perhaps support for WebP and JPEG2000 formats. Now it's time for the next big step.

The functions for resizing images

One of the problems in PIL, and then in Pillow, was that for resizing using a bilinear and bicubic filter, the affine transform method was used , which uses the same number of pixels in the source image to form one final pixel (2x2 pixels for bilinear, 4x4 for bicubic) and fixed filter size. This led to unsatisfactory results for image reduction, which practically did not differ from the nearest neighbor method .

On the left is the nearest neighbor method, on the right is a bicubic filter of affine transformations. The first sample is a decrease of 5.8 times, there are practically no differences. The second one is 1.8 times, the differences are minimal, a short flight of stairs is visible on sharp diagonal lines.

At the same time for the filter

Starting with Pillow 2.7.0, a high-quality convolution-based algorithm is used for all three filters.

On the left is a bicubic filter based on affine transformations, on the right is convolution. Convolution definitely wins.

If before you used some tricks to improve the quality when using a bilinear or bicubic filter (for example, reducing the image in a few steps or preliminary blurring), now they are not necessary.

A new constant has

When the method

Of course, the old constant is left for backward compatibility and is an alias for the new one. Joke for Linguists: Antialias is alias now.

Oddly enough, the quality of reconciliations was also not all right. In previous versions, there was a bug due to which the quality of the Lanczos filter with increasing was almost the same as that of the filter

On the left is the result of a 4.3-fold increase in the previous version, on the right is Pillow 2.7.0. Pictures on the left are both more blurry and pixelated.

The bicubic filter, implemented for affine transformations, gave a sharp, slightly pixelated image when enlarged. The bicubic filter implemented for convolutions is slightly softer.

On the left is the result of a 4.3-fold increase in the previous version, on the right is Pillow 2.7.0. Pictures on the left are more pixelated (have more tangible pixel borders). At the same time, the diagonal lines in the first example are sharper and less susceptible to the step effect. Both that and another - influence of parameter "a" in the bicubic equation . Both effects can only be avoided with a better Lanczos filter.

In the general case, convolution is a more costly algorithm to reduce, because unlike affine transformations , it takes into account all the pixels of the original image. Because of this, the net performance of bilinear and bicubic filters may be lower than before. On the other hand, if you were previously satisfied with the quality of the bilinear and bicubic filters to reduce, perhaps you should consider using a

At the same time, one of the significant improvements of Pillow 2.7.0 is that the performance of convolutions for reduction was increased on average by 2 times compared to the previous version and even compared to ImageMagick. The convolution increase performance for the filter

Because most likely you did not use anything other than

### The default filter for

In Pillow 2.5, the default filter for has

The new method

Performance techniques

These three methods are united by the fact that in them pixels are taken from rows and placed in columns. Such a memory access pattern is very inefficient for large images, because the data manages to be forced out of the processor cache in one pass and must be reloaded from memory for the next pass.

In the new version, the image is divided into logical squares of 128 × 128 pixels in size, and pixel operations are performed sequentially within each square. This allows you to significantly reduce the distance that the processor travels on each line, as a result of which the data does not have time to be forced out of the cache (the memory required for one square is 64Kb).

The implementation has

There was a bug in previous versions of Pillow that caused the blur radius (Gaussian standard deviation) to actually set its diameter. Therefore, for example, to blur the image by a radius of 5, you had to specify a value of 10. The error was fixed, and now the value of the radius is interpreted in the same way as in the rest of the software.

If before that you used Gaussian blur with a certain radius, you need to divide its value by two.

The calculation time of the box filter is constant relative to its radius and depends only on the size of the input image. Because the new implementation of Gaussian blur is based on a box filter; its calculation is also independent of the blur radius.

For a radius of 1 pixel, the new implementation works 5 times faster, for a radius of 10 - 18 times, for a radius of 50 - already 85 times. Your iOS 8-style designer should be pleased.

Theoretically, with Gaussian blur, all points of the source with certain coefficients should participate in the calculation of each point of the final image. In practice, the coefficients of points beyond 3 × standard deviation are so small that there is no point in taking them into account.

The previous implementation took into account only pixels within a radius of 2 × standard deviation for each final pixel. This was not enough, so the quality was worse in comparison with other implementations of Gaussian blur.

Despite the fact that the new implementation is only a mathematical approximation, it does not contain such a bug.

On the left, the result of blurring with a radius of 5 in the previous version (taking into account the bug with doubling the radius), on the right - in the new one. The sharp edges of objects are visible on the left.

All these changes are already working on our servers. Thanks to them, we have improved the quality and speed of the API for processing images on the fly . We also implemented a quick bluer operation . But that is not all. We are preparing the next big step for Pillow, which we will announce a little later.

To begin with, remember how it all began. Pillow is a friendly fork (as its authors call it) of the popular PIL library, Python Imaging Library. The latest version of PIL 1.1.7 was released in 2009 and mainly contained bug fixes. Initially, Pillow was conceived as a project only to tidy the assembly of PIL, and the developers recommended sending all bugs that were not related to the assembly to the original PIL. But time passed, PIL was rapidly becoming obsolete, bugs did not decrease, and Python 3 loomed on the horizon. Therefore, with Pillow 2.0, everything has changed. “Pillow 2.0.0 adds support for Python 3 and includes many bug fixes from around the Internet” reads the project descriptionon PyPI. And since then it started. Every three months there were versions with a huge number of bug fixes and other improvements from various developers. The most significant innovation during this time was perhaps support for WebP and JPEG2000 formats. Now it's time for the next big step.

## Image Resize Filters

The functions for resizing images

`Image.resize()`

and `Image.thumbnail()`

, as one of the arguments, take the filter used for resizing - `resample`

. The options are `NEAREST`

, `BILINEAR`

, `BICUBIC`

and `ANTIALIAS`

. The behavior of almost every one of them has changed in the new version.### Image reduction with bilinear and bicubic filters

One of the problems in PIL, and then in Pillow, was that for resizing using a bilinear and bicubic filter, the affine transform method was used , which uses the same number of pixels in the source image to form one final pixel (2x2 pixels for bilinear, 4x4 for bicubic) and fixed filter size. This led to unsatisfactory results for image reduction, which practically did not differ from the nearest neighbor method .

On the left is the nearest neighbor method, on the right is a bicubic filter of affine transformations. The first sample is a decrease of 5.8 times, there are practically no differences. The second one is 1.8 times, the differences are minimal, a short flight of stairs is visible on sharp diagonal lines.

At the same time for the filter

`ANTIALIAS`

a high-quality convolution-based algorithm was used , which gave equally good results both for decreasing and for increasing. Starting with Pillow 2.7.0, a high-quality convolution-based algorithm is used for all three filters.

On the left is a bicubic filter based on affine transformations, on the right is convolution. Convolution definitely wins.

If before you used some tricks to improve the quality when using a bilinear or bicubic filter (for example, reducing the image in a few steps or preliminary blurring), now they are not necessary.

### Antialias renamed to Lanczos

A new constant has

`Image.LANCZOS`

been added in return `Image.ANTIALIAS`

. When the method

`ANTIALIAS`

was first introduced, it was the only high-quality convolution-based method. And his name reflected this fact. Now that all methods are based on convolution, they have all become “smoothing”. And the real name of the filter that was used earlier for this constant is the Lanczos filter. Of course, the old constant is left for backward compatibility and is an alias for the new one. Joke for Linguists: Antialias is alias now.

### Lanczos filter quality with increasing

Oddly enough, the quality of reconciliations was also not all right. In previous versions, there was a bug due to which the quality of the Lanczos filter with increasing was almost the same as that of the filter

`BILINEAR`

. This bug has been fixed. On the left is the result of a 4.3-fold increase in the previous version, on the right is Pillow 2.7.0. Pictures on the left are both more blurry and pixelated.

### Bicubic filter quality with increasing

The bicubic filter, implemented for affine transformations, gave a sharp, slightly pixelated image when enlarged. The bicubic filter implemented for convolutions is slightly softer.

On the left is the result of a 4.3-fold increase in the previous version, on the right is Pillow 2.7.0. Pictures on the left are more pixelated (have more tangible pixel borders). At the same time, the diagonal lines in the first example are sharper and less susceptible to the step effect. Both that and another - influence of parameter "a" in the bicubic equation . Both effects can only be avoided with a better Lanczos filter.

### Resize Performance

In the general case, convolution is a more costly algorithm to reduce, because unlike affine transformations , it takes into account all the pixels of the original image. Because of this, the net performance of bilinear and bicubic filters may be lower than before. On the other hand, if you were previously satisfied with the quality of the bilinear and bicubic filters to reduce, perhaps you should consider using a

`NEAREST`

filter that gave almost the same result. This will significantly increase productivity.At the same time, one of the significant improvements of Pillow 2.7.0 is that the performance of convolutions for reduction was increased on average by 2 times compared to the previous version and even compared to ImageMagick. The convolution increase performance for the filter

`BILINEAR`

turned out to be one and a half times faster, for `BICUBIC`

four times faster , and for `LANCZOS`

remained at the same level. Because most likely you did not use anything other than

`LANCZOS`

(former`ANTIALIAS`

), then the performance with a decrease for you should increase on average twice. If the use of Lanczos was a necessary measure for you because of the low quality of the other filters, now you can go, for example, to a bilinear filter. This will increase productivity by about a factor of 2 to decrease, and by about 30% to increase.### The default filter for `Image.thumbnail()`

In Pillow 2.5, the default filter for has

`Image.thumbnail()`

been changed from `NEAREST`

to `ANTIALIAS`

. This filter was chosen for the reason that was repeatedly voiced above - the low quality of the remaining filters. In Pillow 2.7.0, the default filter is again changed, this time to `BICUBIC`

, because it is a little faster. In fact, Lanczos does not give any advantages after using the method `Image.draft()`

inside `Image.thumbnail()`

, which reduces the image using the library `libjpeg`

and uses supersampling rather than convolution for this .## Transpose Images

The new method

`Image.TRANSPOSE`

has been added for the function `Image.transpose()`

, in addition to the existing ones `FLIP_LEFT_RIGHT`

, `FLIP_TOP_BOTTOM`

, `ROTATE_90`

, `ROTATE_180`

, `ROTATE_270`

. `TRANSPOSE`

Is an algebraic transposition, i.e. reflection of the image relative to its main diagonal. Performance techniques

`ROTATE_90`

, `ROTATE_270`

and `TRANSPOSE`

it has been significantly increased for larger images that do not fit in the CPU cache. These three methods are united by the fact that in them pixels are taken from rows and placed in columns. Such a memory access pattern is very inefficient for large images, because the data manages to be forced out of the processor cache in one pass and must be reloaded from memory for the next pass.

In the new version, the image is divided into logical squares of 128 × 128 pixels in size, and pixel operations are performed sequentially within each square. This allows you to significantly reduce the distance that the processor travels on each line, as a result of which the data does not have time to be forced out of the cache (the memory required for one square is 64Kb).

## Gaussian Blur and Contour Sharpness

The implementation has

`ImageFilter.GaussianBlur`

been replaced by the consistent use of box filters. The new implementation is based on the article Theoretical foundations of Gaussian convolution by extended box filtering from the Mathematical Image Analysis Group. Since the implementation `ImageFilter.UnsharpMask`

is based on Gaussian blur, everything described in this section also applies to it.### Blur radius

There was a bug in previous versions of Pillow that caused the blur radius (Gaussian standard deviation) to actually set its diameter. Therefore, for example, to blur the image by a radius of 5, you had to specify a value of 10. The error was fixed, and now the value of the radius is interpreted in the same way as in the rest of the software.

If before that you used Gaussian blur with a certain radius, you need to divide its value by two.

### Blur performance

The calculation time of the box filter is constant relative to its radius and depends only on the size of the input image. Because the new implementation of Gaussian blur is based on a box filter; its calculation is also independent of the blur radius.

For a radius of 1 pixel, the new implementation works 5 times faster, for a radius of 10 - 18 times, for a radius of 50 - already 85 times. Your iOS 8-style designer should be pleased.

### Blur quality

Theoretically, with Gaussian blur, all points of the source with certain coefficients should participate in the calculation of each point of the final image. In practice, the coefficients of points beyond 3 × standard deviation are so small that there is no point in taking them into account.

The previous implementation took into account only pixels within a radius of 2 × standard deviation for each final pixel. This was not enough, so the quality was worse in comparison with other implementations of Gaussian blur.

Despite the fact that the new implementation is only a mathematical approximation, it does not contain such a bug.

On the left, the result of blurring with a radius of 5 in the previous version (taking into account the bug with doubling the radius), on the right - in the new one. The sharp edges of objects are visible on the left.

All these changes are already working on our servers. Thanks to them, we have improved the quality and speed of the API for processing images on the fly . We also implemented a quick bluer operation . But that is not all. We are preparing the next big step for Pillow, which we will announce a little later.