August 20, 2009 at 10:21 pmCleaning the image from noise, some methods
If you saw a picture that is obtained in modern digital cameras without processing, then you know that it looks just awful. It is filled with noise. Even when you download a picture to a computer and it has already undergone internal processing in the camera, if you enlarge it and look at individual pixels, you can see how courageously digital algorithms fight noise and lose in this unequal war.
Some algorithms completely erase small details; Nokia cell phones are famous for this. In some cases, the details remained, but they are surrounded by colored islands of complex shape, this can be seen in Sony cameras. Well and so on - each method has its own problems.
What are the means to remove this noise, and which do not violate other people's patents? Hope this short review will be helpful.
1. Go to the brightness-color coordinates.
This conversion can be done in many ways: HSV, L * a * b, etc. For some reasons, which we will not delve into:
- the human eye is much less sensitive to details of color information than luminance
- noise in the color component, on the contrary, is much higher than in luminance.
Therefore, simple filtering of the color component + reverse restoration usually does the picture is much better.
2. The median filter.
A good simple way to clear the picture of noise is the median filter Im_new (x, y) = median {dx = -1..1, dy = -1..1} Im (x + dx, y + dy).
This method has many variations, I will give only a few:
2.1 Step 1: calculate M1 = median (C, Cnorth, Csouth); M2 = median (C, Ceast, Cwest); M3 = median (C, Cne, Csw); M4 = median (C, Cnw, Csw); here Cnort, Cne, ... Cnw - eight neighboring pixels from a 3x3 neighborhood, C - central pixel
Step 2 - calculate Ma = median (C, M1, M2); Mb = median (C, M3, M4);
Step 3 - calculate Csmooth = median (C, Ma, Mb);
Step 4 - replace C with Csmooth.
2.2 Step 1: sort the pixels from the 3x3 neighborhood in ascending order, P [1] ... P [9].
Step 2: If the central pixel is P [1] - replace it with P [2], if the central pixel is P [9] - replace it with P [8], in other cases, leave it unchanged.
This direction is used by Kodak, as well as most scanners and fax machines.
3. Filters that control the amount of correction
This method first suggests smoothing the picture somehow roughly, for example using a low-pass filter, bilateral filter, or something else. And then this procedure is done
Im_new (x, y) = Im (x, y) + S (Im (x, y) -Im_smooth (x, y), threshold).
The transmitter function S can be arranged in different ways, for example:
S (x, threshold) = x, if -thresholdthreshold; S (x, threshold) = - threshold if x <-threshold. If you select a threshold approximately equal to the amount of noise, then all the noise will disappear, and the details and small objects will remain clear.
4. Bilateral filter
A very interesting filter, invented in 2003. For descriptions I send to the Internet.
Here is a pretty good article: scien.stanford.edu/class/psych221/projects/06/imagescaling/bilati.html
An interesting variation of bilateral filter is also T-filter:
Step 1: Find all the pixels in the neighborhood whose values are different from the original pixel no more than a given threshold.
Step 2: Average these found pixels and save the value.
5. Filters using the spectral representation of the signal
This is how Photoshop works, for example. The essence of the idea is to make a Fourier transform in the vicinity of each pixel, then erase the high frequencies and do the inverse transform.
Instead of the Fourier transform, other orthogonal bases are also used, sometimes quite intricate. In fact, this is a whole family of methods.
6. Filters that highlight the dominant direction.
These filters at each point first find the dominant direction (direction of the brightness gradient), and then average the signal only in the perpendicular direction. Thus, lines and small details remain clear. Good variations of this algorithm also take into account the values of the matrix of second derivatives.
This is a whole family of algorithms, descriptions of which can also be found on the Internet.
7. Local classification of fragments.
These filters work especially well with special images, such as text, starry sky, etc.
First, a database of typical elements of such an image is compiled, for example, several hundred fragments of NxN pixels that are already cleared of noise.
The algorithm works like this: the neighborhood of each pixel is compared with these fragments and the one that is most similar is selected. Then the value of the original pixel in the dirty picture is replaced with the value of a similar pixel located in the same place on a clean fragment.
8. I will cite the “simple” method at the end, which can also be used in some cases.
Step 1: Zoom Out (Using Some Smart Downscaling Algorithm)
Step 2: Increase it back (using some smart Upscaling algorithm)
The fact is that Upscaling / Downscaling algorithms are very powerful (Lanczos filter, fractal methods, etc.), so the result is quite satisfactory. The same method can be used as a simple, but quite effective compression.
Some algorithms completely erase small details; Nokia cell phones are famous for this. In some cases, the details remained, but they are surrounded by colored islands of complex shape, this can be seen in Sony cameras. Well and so on - each method has its own problems.
What are the means to remove this noise, and which do not violate other people's patents? Hope this short review will be helpful.
1. Go to the brightness-color coordinates.
This conversion can be done in many ways: HSV, L * a * b, etc. For some reasons, which we will not delve into:
- the human eye is much less sensitive to details of color information than luminance
- noise in the color component, on the contrary, is much higher than in luminance.
Therefore, simple filtering of the color component + reverse restoration usually does the picture is much better.
2. The median filter.
A good simple way to clear the picture of noise is the median filter Im_new (x, y) = median {dx = -1..1, dy = -1..1} Im (x + dx, y + dy).
This method has many variations, I will give only a few:
2.1 Step 1: calculate M1 = median (C, Cnorth, Csouth); M2 = median (C, Ceast, Cwest); M3 = median (C, Cne, Csw); M4 = median (C, Cnw, Csw); here Cnort, Cne, ... Cnw - eight neighboring pixels from a 3x3 neighborhood, C - central pixel
Step 2 - calculate Ma = median (C, M1, M2); Mb = median (C, M3, M4);
Step 3 - calculate Csmooth = median (C, Ma, Mb);
Step 4 - replace C with Csmooth.
2.2 Step 1: sort the pixels from the 3x3 neighborhood in ascending order, P [1] ... P [9].
Step 2: If the central pixel is P [1] - replace it with P [2], if the central pixel is P [9] - replace it with P [8], in other cases, leave it unchanged.
This direction is used by Kodak, as well as most scanners and fax machines.
3. Filters that control the amount of correction
This method first suggests smoothing the picture somehow roughly, for example using a low-pass filter, bilateral filter, or something else. And then this procedure is done
Im_new (x, y) = Im (x, y) + S (Im (x, y) -Im_smooth (x, y), threshold).
The transmitter function S can be arranged in different ways, for example:
S (x, threshold) = x, if -threshold
4. Bilateral filter
A very interesting filter, invented in 2003. For descriptions I send to the Internet.
Here is a pretty good article: scien.stanford.edu/class/psych221/projects/06/imagescaling/bilati.html
An interesting variation of bilateral filter is also T-filter:
Step 1: Find all the pixels in the neighborhood whose values are different from the original pixel no more than a given threshold.
Step 2: Average these found pixels and save the value.
5. Filters using the spectral representation of the signal
This is how Photoshop works, for example. The essence of the idea is to make a Fourier transform in the vicinity of each pixel, then erase the high frequencies and do the inverse transform.
Instead of the Fourier transform, other orthogonal bases are also used, sometimes quite intricate. In fact, this is a whole family of methods.
6. Filters that highlight the dominant direction.
These filters at each point first find the dominant direction (direction of the brightness gradient), and then average the signal only in the perpendicular direction. Thus, lines and small details remain clear. Good variations of this algorithm also take into account the values of the matrix of second derivatives.
This is a whole family of algorithms, descriptions of which can also be found on the Internet.
7. Local classification of fragments.
These filters work especially well with special images, such as text, starry sky, etc.
First, a database of typical elements of such an image is compiled, for example, several hundred fragments of NxN pixels that are already cleared of noise.
The algorithm works like this: the neighborhood of each pixel is compared with these fragments and the one that is most similar is selected. Then the value of the original pixel in the dirty picture is replaced with the value of a similar pixel located in the same place on a clean fragment.
8. I will cite the “simple” method at the end, which can also be used in some cases.
Step 1: Zoom Out (Using Some Smart Downscaling Algorithm)
Step 2: Increase it back (using some smart Upscaling algorithm)
The fact is that Upscaling / Downscaling algorithms are very powerful (Lanczos filter, fractal methods, etc.), so the result is quite satisfactory. The same method can be used as a simple, but quite effective compression.