February 21, 2014 at 11:52Using the Poisson Transform for Seamless Image Overlay
In the problems of machine vision and automated image processing, the problem of seamless image overlay is often encountered. For clarity, I’ll immediately give an example.
By seamless image overlay (blending) we mean a method that allows you to insert part of one image into another so that there are no noticeable seams at the borders of the inserted image. This method, as it were, adjusts the inserted part for the rest of the image. In fact, to obtain the result, only the gradient field of the inserted part and the pixel values of the original image at the border of the processed area are used.
Let us have images A, on which image B is superimposed in a certain place.
For seamless blending, the gradient of the image A in the image area B should be equal to the gradient of the image B. Color recovery in the area B will be based on the gradient in the area B and the color data of the image A on the border of the area B.
This means that to obtain the resulting image, you need to restore image by its gradient, i.e. solve the Poisson problem.
Below is a solution to the problem for a monotonous image. For RGB images, the method is applied to each channel.
Define the border of the inserted region as follows:
And calculate the gradient of the overlay image:
Now all the data is available and we can compose the Poisson equation:
where (x, y) is the coordinate of the current pixel, N is the number of neighboring pixels not including borders (not more than 4), (dx, dy) are the coordinates of neighboring pixels, can take values from the set {(-1, 0), (1 , 0), (0, -1), (0, 1)}
Thus, an equation is compiled for each unknown pixel. As a result, we get a system with M unknowns, where M is the number of pixels of the inserted image. It remains only to solve this system. There are a lot of methods for solving this, but the Jacobi method and the Gauss-Seidel method are usually used for the solution.
Often, in a superimposed area, a gradient field is mixed with the original image. For example, take the maximum value of the gradients A or B at a certain point. It’s best to feel the difference between the original and the combined gradient using an example.
Image Gradient Recovery B:
Recovery by the maximum gradient of images A and B:
Obviously, it is advisable to use image recovery from a combined gradient for objects that have a transparent structure (water, rainbow, clouds, etc.).
And here's how a rainbow overlay would look without a combination of gradients
By seamless image overlay (blending) we mean a method that allows you to insert part of one image into another so that there are no noticeable seams at the borders of the inserted image. This method, as it were, adjusts the inserted part for the rest of the image. In fact, to obtain the result, only the gradient field of the inserted part and the pixel values of the original image at the border of the processed area are used.
Formulation of the problem
Let us have images A, on which image B is superimposed in a certain place.
For seamless blending, the gradient of the image A in the image area B should be equal to the gradient of the image B. Color recovery in the area B will be based on the gradient in the area B and the color data of the image A on the border of the area B.
This means that to obtain the resulting image, you need to restore image by its gradient, i.e. solve the Poisson problem.
Below is a solution to the problem for a monotonous image. For RGB images, the method is applied to each channel.
Decision
Define the border of the inserted region as follows:
And calculate the gradient of the overlay image:
Now all the data is available and we can compose the Poisson equation:
where (x, y) is the coordinate of the current pixel, N is the number of neighboring pixels not including borders (not more than 4), (dx, dy) are the coordinates of neighboring pixels, can take values from the set {(-1, 0), (1 , 0), (0, -1), (0, 1)}
Thus, an equation is compiled for each unknown pixel. As a result, we get a system with M unknowns, where M is the number of pixels of the inserted image. It remains only to solve this system. There are a lot of methods for solving this, but the Jacobi method and the Gauss-Seidel method are usually used for the solution.
Often, in a superimposed area, a gradient field is mixed with the original image. For example, take the maximum value of the gradients A or B at a certain point. It’s best to feel the difference between the original and the combined gradient using an example.
Image Gradient Recovery B:
Recovery by the maximum gradient of images A and B:
Obviously, it is advisable to use image recovery from a combined gradient for objects that have a transparent structure (water, rainbow, clouds, etc.).
And here's how a rainbow overlay would look without a combination of gradients
Application of Poisson Blending Transformation
- Replacing faces / inserting new objects into the frame
- Overlay objects in a frame that have a transparent structure
- Image smoothing when creating panoramas
- Smoothing images when creating 3D models of objects
References
- Original article
- A bit of theory, application and examples
- SLAE solution using the Jacobi method and the Gauss - Seidel method + sources
- Program + Java sources
- Photos taken using Blend Me