Simple motion recognition algorithm

When I started working on a master's thesis on "Analysis of the spatial structure of dynamic images", I ran into the problem that it is very difficult to find any ready-made examples of recognition algorithms for patterns and moving objects. Everywhere, both in literature and on the Internet, there is only a bare theory. The purpose of this article is to fill in this gap.

So, for the experiment, I shot a short video from the apartment window, divided it into frames and saved a couple of frames in the form of pictures:





To determine the fact of movement, I decided so far to go the simple way: turn the images into a matrix, subtract the first from the second matrix (element by element). Here is the C # code that does this processing:

        /// 

        /// Создать матрицу
        /// 
        /// Имя открываемого файла
        private ImageMatrix crate_matrix(string file_name)
        {
            Bitmap picture = new Bitmap(file_name);
            ImageMatrix res = new ImageMatrix();
            using (var wrapper = new ImageWrapper(picture,true))
            {
                res.matrix = new int[wrapper.Width, wrapper.Height];
                for (int i = 0; i < wrapper.Width; i++)
                {
                    for (int j = 0; j < wrapper.Height; j++)
                    {
                        Color color = wrapper[i, j];
                        res.matrix[i, j] = (color.R + color.G + color.B) / 
3;
                    }
                }
                res.height = wrapper.Height;
                res.width = wrapper.Width;
                res.picture = picture;
            }     
            return res;
        }
        private void tsmiDifference_Click(object sender, EventArgs e)
        {
            ImageMatrix matrix1 = crate_matrix("D:\\3\\1.png");
            ImageMatrix matrix2 = crate_matrix("D:\\3\\2.png");
            Bitmap picture = new Bitmap(matrix1.picture);
            using (var wrapper = new ImageWrapper(picture, true))
            {
                for (int i = 0; i < wrapper.Width; i++)
                {
                    for (int j = 1; j < wrapper.Height; j++)
                    {
                        int light1 = matrix1.matrix[i, j];
                        int light2 = matrix2.matrix[i, j];
                        int light = Math.Abs(light2-light1);
                        wrapper[i, j] = Color.FromArgb(light, light, light);
                    }
                }
                pbImage.Image = picture;
            }
        }

And here is what happened during the execution of this algorithm:



On the resulting matrix, we see the sealed outline of the car and some other contours in the background, which, most likely, arose due to hand shake and rain that had occurred at the time of shooting. Rain is not visible in the photo, but it is clearly visible in the video.

In order to improve the quality of recognition, you can enter a certain threshold value in the program:

if (light <50) light = 0; else light = 255;

The result will then be completely different:



As you can see, by simply subtracting the matrices and using the threshold value, we were able to determine a moving object in a dynamic picture. What to do with the received data further? In fact, what we have now is a binary picture. There are some spots in it that define a moving object. Noise, as we see, our algorithm compartment. In the future, you can determine the coordinates of the detected areas of motion. True, we have several moving objects - each spot when trying to determine its coordinates will be a separate object. But if we process several matrices in this way, then we can notice that several objects move synchronously, which allows us to attribute them to a single object.

However, the algorithm given here is not the only way to determine motion. If you like my article, then I will continue the topic and talk about other algorithms for motion recognition and pattern recognition.