December 24, 2011 at 23:29Viola Jones on its own skin, part 1. Setting up a project for OpenCV
Hello my friends! Dear Habrausers and Habrachitateli, I suggest you plunge into the jungle of developing a program for recognizing emotions based on the materials of my previous articles .
At the “input” of the automatic emotion recognition apparatus there should be a color or black-and-white image or video stream loaded into the module, and at the “output” there should be an array of rectangles described around faces and facial features, a textual conclusion about the presence of a particular emotion on the found faces .
The module consists of three main stages:
1. Preliminary preparation of the image for more convenient recognition of faces and facial features;
2. Recognition of faces and facial features by the developed modified Viola-Jones algorithm ;
3. Confirmation of the detected emotion, displaying relevant information on the screen.
The planned program module, I had previously called «EmotionRecognitionTool v 0.1 by SkyNoName», which implements the algorithm Viola-Jones with a modification thereof, for demonstration purposes will be written in the IDE (Integrated Development Environment, the IDE) the Microsoft the Visual Studio © 2010 language C or C ++ using the latest OpenCV (Open Source Computer Vision library) computer vision library ( but I had version 2.2 installed earlier), as well as videoInput version 0.1995 from theo watson user . I will try to develop a module under Windows 7 Ultimate and Ubuntu 10.10 operating systems usingcross-platform build system of the Cmake module .
In fact, the choice of funds is not limited in any way. You can make your own applications, for example, in PHP using GD , using the cross-platform toolkit Qt and other programming languages. But I chose what is at hand at the moment ...
First, it's worth downloading, paying (;)) and installing Microsoft Visual Studio from the official application download page .
Then download the OpenCV library . By the way, a new library release has recently been released with many additions and changes, namely version 2.3.1, but more on that later. For minimal development, you can choose either 2.3or 2.2 on the official download page .
Also, we need the Cmake build system, distributed with the Creative commons license (cc by-nd 3.0) . Download .
When working with a video stream, we need a free-distributed VideoInput module , it can be taken from Git (you can also use the code from ready-made solutions fdlib and stasm ) for the interaction of input and output from cameras to cameras.
There should be no problems installing Visual Studio. However, as with the installation of OpenCV, the truth is there is one point - you need to add some paths to the compiled dll files in the environment variable.
For version 2.2 for win32, this is done during installation: the checkbox is checked on the item Add OpenCV to the system PATH for current user.
For the version “2.3.1 superpack”, the archive can be unpacked into the folder C: \ OpenCV2.3.1 and then add the addresses to the PATH variable of this user.
To do this, go to Start-Control Panel-All Controls-System (or select the Properties item when right-clicking on the Computer icon), select Advanced System Settings. In the system properties window, go to the Advanced tab and click on the Environment Variables ... button, a dialog box opens where you will need to Create or Change the PATH variable with the values of the addresses given above (I have version 2.2 preinstalled, so it’s in the screenshot):
Install Cmake, the latest version of which is under Windows 2.8.6 and run Cmake-gui.
First, a window with empty fields will appear.
It is necessary to indicate the unpacked folder with OpenCV on the C drive in the Where the source code field:
Then, in the Where to build the binaries field, specify the build folder enclosed in OpenCV:
Click the Confige button and specify the generator there, otherwise the Visual Studio compiler, in my VS2010 it is Visual Studio 10 Win64. After that, a list of available parameters for compilation will be indicated. Because we haven’t generated a makefile yet, the main compilation area will be red.
After that, you need to select the files that will be compiled, tick them, and click the Generate button.
We are waiting for the generation of a universal make-file to complete and close cmake. As a result, we should add cmake manifests in the C: \ OpenCV2.3.1 \ build \ folder, as well as the OpenCV.sln project.
Run this project (it will automatically open in Microsoft Visual Studio) using the F7 key. Using this project, OpenCV methods are linked, as well as examples of their use, available in samples. If in Cmake many parameters were checked with daws, then compilation will have to wait. The window of the working compiler is shown below:
For those who already had an earlier version of OpenCV or had additional parameters specified during the installation, Visual Studio will find all the unaccounted for and will display a message about adding update data and new files. If you want to "upgrade", then click Reload all:
The time has come, after all the dances with a tambourine, to make your own project template . To do this, you need:
I selected and created a project with a CLI (with command line support, Command Line Interface).
To do this, go to General Properties. The action plan is as follows: right-click on the project name in the Solution Explorer / select the Project (Solution) Explorer and select Properties:
Then, in the project properties window, select the Configuration properties section and its subsection VC ++ Directories / VC ++ Directories (Directories ) Find in the list of parameters Include Directories / Included Directories (Directories):
After that, click on the edit field, an arrow will appear in the drop-down list. Click on it. Select the Edit item:
In the window that opens, click on the icon for adding a new directory and select the directories to include the following: After adding new directories, click the OK button:
Then, in the window that is already familiar to us, write the paths to the OpenCV libraries in the Library Directories section (my screenshot shows the paths to library files of version 2.2): You can go to the Linker section, select the subsection in it Input and enter in the item Additional Dependencies / Additional dependencies of the path to additional libraries that you want to use in your projects. For example, etc. If necessary, add them to this item: After adding, registering all the paths, click OK.
Since we work mostly with images, as well as with finding objects, we add the following header files to the project template:
For more comfortable work and for using certain methods, libraries must be connected. In almost any project, they turn out to be necessary:
Press F5 and try compiling the project. It should work out. Thus, we now have a single template for further development.
Starting from version 2.3, the developed project will be able to start successfully on another machine if you transfer it along with the required dll files, which are conveniently located in folders called “bin”, the path to which lies through the build \ x86 or build \ x64 folders.
Also, there are files that are used for advanced features (for example, the dll ffmpeg codec), they are folded into one place for convenience (hanging out in the 3dparty folder).
Together with an executable file (.exe), your work should start even where special programs are not installed.
In the near future, the program will have to acquire its own user interface, more interactive and intuitive.
For the project to work correctly, the following minimum system configurations are required :
• processor: at least Intel Core 2 Duo E6550 2.33 GHz (2.33 GHz)
• RAM, RAM: SDRAM (English Synchronous Dynamic Random Access Memory - synchronous dynamic memory with random access) : at least 2Gb (2GB) DDR3 (double-data-rate three) from any manufacturer
• 150 Mb (MB) of free hard disk space The
recommended system configuration is as follows:
• processor: Intel Core i3-370M 2.4 GHz (2.4 GHz)
• RAM, RAM: SDRAM 4Gb (4GB) DDR3
• 300 Mb (MB) of free hard disk space
At the end of development, both the module using the developed algorithm and its batch implementation will be compiled. Accordingly, two types of possible software characteristics are distinguished :
Minimum software characteristics (launch of version 0.1 release):
• Windows XP SP3 / Vista / Windows 7 (32-bit),
- image viewer from any company.
Recommended software parameters (launch of package version 0.1):
• Windows 7 (64-bit),
- image viewer from any company
- any development environment that supports working with C ++ (Microsoft Visual Studio 2010),
- pre-installed standard OpenCV 2.2 library ,
- the universal compiler of the Cmake 2.8.4 project,
- MySQL Database 6.0 database management system,
- GitBash console panel of a single free GitHub repository (GitVersion 1.7.4),
- PSPad Editor, for working with xml format
• Linux,
- Linux kernel - 2.2.14 or higher with the following libraries:
- glibc 2.3.2 or higher,
- XFree86-3.3.6 or higher,
- gtk + 2.0 or higher,
- fontconfig (also known as xft),
- libstdc ++ 5,
- gcc for Linux compiler,
- unpacked and ready to work MySQL for Linux,
- further, when working with the program, Linux will automatically find all the necessary service packs and software installer packages.
The source code of this module must be open (in accordance with the GNU General Public License v3), because I want it that way. It can be copied from the release branch of the project from a single repository. The project will most likely be located at: github.com/kalian/EmotionRecognitionTool-v.-1.0 (the check-out of the project is free).
With this result, I will finish the first part of the story of developing my own emotion recognition module.
Thank you all again for your attention! SkyNoName was with you , wait for the continuation, I hope it will be before the New Year.
The following manuals and resources, forums helped me in preparing this article.
For those who have just started developing applications using OpenCV, I recommend:
1.© «RoboCraft» - this resource was created with the aim of informational reflection of the RoboCraft team's Kaliningrad team's work in the field of robotics. There are many examples of digital image processing, and more.
2. Blog of the programmer Andrei from Tomsk with the nickname Troyashka, who has long been familiar with OpenCV. His blog has tons of useful information.
3. The forum of the site about machine vision Compvision.ru , in the bowels of which there is a lot of information for beginners, I sat on it at the beginning of my acquaintance with OpenCV
Let's start with module design
At the “input” of the automatic emotion recognition apparatus there should be a color or black-and-white image or video stream loaded into the module, and at the “output” there should be an array of rectangles described around faces and facial features, a textual conclusion about the presence of a particular emotion on the found faces .
The module consists of three main stages:
1. Preliminary preparation of the image for more convenient recognition of faces and facial features;
2. Recognition of faces and facial features by the developed modified Viola-Jones algorithm ;
3. Confirmation of the detected emotion, displaying relevant information on the screen.
Choosing a development environment and tools
The planned program module, I had previously called «EmotionRecognitionTool v 0.1 by SkyNoName», which implements the algorithm Viola-Jones with a modification thereof, for demonstration purposes will be written in the IDE (Integrated Development Environment, the IDE) the Microsoft the Visual Studio © 2010 language C or C ++ using the latest OpenCV (Open Source Computer Vision library) computer vision library ( but I had version 2.2 installed earlier), as well as videoInput version 0.1995 from theo watson user . I will try to develop a module under Windows 7 Ultimate and Ubuntu 10.10 operating systems usingcross-platform build system of the Cmake module .
In fact, the choice of funds is not limited in any way. You can make your own applications, for example, in PHP using GD , using the cross-platform toolkit Qt and other programming languages. But I chose what is at hand at the moment ...
First, it's worth downloading, paying (;)) and installing Microsoft Visual Studio from the official application download page .
Then download the OpenCV library . By the way, a new library release has recently been released with many additions and changes, namely version 2.3.1, but more on that later. For minimal development, you can choose either 2.3or 2.2 on the official download page .
Also, we need the Cmake build system, distributed with the Creative commons license (cc by-nd 3.0) . Download .
When working with a video stream, we need a free-distributed VideoInput module , it can be taken from Git (you can also use the code from ready-made solutions fdlib and stasm ) for the interaction of input and output from cameras to cameras.
Installation procedure
There should be no problems installing Visual Studio. However, as with the installation of OpenCV, the truth is there is one point - you need to add some paths to the compiled dll files in the environment variable.
For version 2.2 for win32, this is done during installation: the checkbox is checked on the item Add OpenCV to the system PATH for current user.
For the version “2.3.1 superpack”, the archive can be unpacked into the folder C: \ OpenCV2.3.1 and then add the addresses to the PATH variable of this user.
C:\OpenCV2.3.1\build\bin;
C:\OpenCV2.3.1\build\x64\mingw\bin;
C:\OpenCV2.3.1\build\x64\vc9\bin;
C:\OpenCV2.3.1\build\x64\vc10\bin;
C:\OpenCV2.3.1\build\x86\mingw\bin;
C:\OpenCV2.3.1\build\x86\vc9\bin;
C:\OpenCV2.3.1\build\x86\vc10\bin;
To do this, go to Start-Control Panel-All Controls-System (or select the Properties item when right-clicking on the Computer icon), select Advanced System Settings. In the system properties window, go to the Advanced tab and click on the Environment Variables ... button, a dialog box opens where you will need to Create or Change the PATH variable with the values of the addresses given above (I have version 2.2 preinstalled, so it’s in the screenshot):
Install Cmake, the latest version of which is under Windows 2.8.6 and run Cmake-gui.
First, a window with empty fields will appear.
It is necessary to indicate the unpacked folder with OpenCV on the C drive in the Where the source code field:
C:/OpenCV2.3.1Then, in the Where to build the binaries field, specify the build folder enclosed in OpenCV:
C:/OpenCV2.3.1/buildClick the Confige button and specify the generator there, otherwise the Visual Studio compiler, in my VS2010 it is Visual Studio 10 Win64. After that, a list of available parameters for compilation will be indicated. Because we haven’t generated a makefile yet, the main compilation area will be red.
After that, you need to select the files that will be compiled, tick them, and click the Generate button.
We are waiting for the generation of a universal make-file to complete and close cmake. As a result, we should add cmake manifests in the C: \ OpenCV2.3.1 \ build \ folder, as well as the OpenCV.sln project.
Run this project (it will automatically open in Microsoft Visual Studio) using the F7 key. Using this project, OpenCV methods are linked, as well as examples of their use, available in samples. If in Cmake many parameters were checked with daws, then compilation will have to wait. The window of the working compiler is shown below:
For those who already had an earlier version of OpenCV or had additional parameters specified during the installation, Visual Studio will find all the unaccounted for and will display a message about adding update data and new files. If you want to "upgrade", then click Reload all:
Project Template with OpenCV
The time has come, after all the dances with a tambourine, to make your own project template . To do this, you need:
1. Create a new project in Microsoft Visual Studio.
I selected and created a project with a CLI (with command line support, Command Line Interface).
2. In the properties of the created project, "bind" the necessary dll-, lib-files and directories to it.
To do this, go to General Properties. The action plan is as follows: right-click on the project name in the Solution Explorer / select the Project (Solution) Explorer and select Properties:
Then, in the project properties window, select the Configuration properties section and its subsection VC ++ Directories / VC ++ Directories (Directories ) Find in the list of parameters Include Directories / Included Directories (Directories):
After that, click on the edit field, an arrow will appear in the drop-down list. Click on it. Select the Edit item:
In the window that opens, click on the icon for adding a new directory and select the directories to include the following: After adding new directories, click the OK button:
C:\OpenCV2.3.1\include\
C:\OpenCV2.3.1\include\opencv
C:\OpenCV2.3.1\include\opencv2
C:\OpenCV2.3.1\include\opencv\3rdparty\include
C:\OpenCV2.3.1\include\opencv\modulesThen, in the window that is already familiar to us, write the paths to the OpenCV libraries in the Library Directories section (my screenshot shows the paths to library files of version 2.2): You can go to the Linker section, select the subsection in it Input and enter in the item Additional Dependencies / Additional dependencies of the path to additional libraries that you want to use in your projects. For example, etc. If necessary, add them to this item: After adding, registering all the paths, click OK.
C:\OpenCV2.3.1\build\x64\vc9\lib
C:\OpenCV2.3.1\build\x64\vc10\lib
C:\OpenCV2.3.1\build\x86\vc9\lib
C:\OpenCV2.3.1\build\x86\vc10\libopencv_calib3d231.lib
opencv_features2d231d.lib
opencv_haartraining_engined.lib
opencv_video231d.lib
3. Register the main headers and the necessary plug-in libraries.
Since we work mostly with images, as well as with finding objects, we add the following header files to the project template:
//Указываем заголовочники
#include
#include
#include
#include
#include
#include For more comfortable work and for using certain methods, libraries must be connected. In almost any project, they turn out to be necessary:
//нужные подключаемые библиотеки
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include Press F5 and try compiling the project. It should work out. Thus, we now have a single template for further development.
Starting from version 2.3, the developed project will be able to start successfully on another machine if you transfer it along with the required dll files, which are conveniently located in folders called “bin”, the path to which lies through the build \ x86 or build \ x64 folders.
Also, there are files that are used for advanced features (for example, the dll ffmpeg codec), they are folded into one place for convenience (hanging out in the 3dparty folder).
Together with an executable file (.exe), your work should start even where special programs are not installed.
Emotion Recognition Project Development Requirements
In the near future, the program will have to acquire its own user interface, more interactive and intuitive.
For the project to work correctly, the following minimum system configurations are required :
• processor: at least Intel Core 2 Duo E6550 2.33 GHz (2.33 GHz)
• RAM, RAM: SDRAM (English Synchronous Dynamic Random Access Memory - synchronous dynamic memory with random access) : at least 2Gb (2GB) DDR3 (double-data-rate three) from any manufacturer
• 150 Mb (MB) of free hard disk space The
recommended system configuration is as follows:
• processor: Intel Core i3-370M 2.4 GHz (2.4 GHz)
• RAM, RAM: SDRAM 4Gb (4GB) DDR3
• 300 Mb (MB) of free hard disk space
Software Features
At the end of development, both the module using the developed algorithm and its batch implementation will be compiled. Accordingly, two types of possible software characteristics are distinguished :
Minimum software characteristics (launch of version 0.1 release):
• Windows XP SP3 / Vista / Windows 7 (32-bit),
- image viewer from any company.
Recommended software parameters (launch of package version 0.1):
• Windows 7 (64-bit),
- image viewer from any company
- any development environment that supports working with C ++ (Microsoft Visual Studio 2010),
- pre-installed standard OpenCV 2.2 library ,
- the universal compiler of the Cmake 2.8.4 project,
- MySQL Database 6.0 database management system,
- GitBash console panel of a single free GitHub repository (GitVersion 1.7.4),
- PSPad Editor, for working with xml format
• Linux,
- Linux kernel - 2.2.14 or higher with the following libraries:
- glibc 2.3.2 or higher,
- XFree86-3.3.6 or higher,
- gtk + 2.0 or higher,
- fontconfig (also known as xft),
- libstdc ++ 5,
- gcc for Linux compiler,
- unpacked and ready to work MySQL for Linux,
- further, when working with the program, Linux will automatically find all the necessary service packs and software installer packages.
The source code of this module must be open (in accordance with the GNU General Public License v3), because I want it that way. It can be copied from the release branch of the project from a single repository. The project will most likely be located at: github.com/kalian/EmotionRecognitionTool-v.-1.0 (the check-out of the project is free).
With this result, I will finish the first part of the story of developing my own emotion recognition module.
Thank you all again for your attention! SkyNoName was with you , wait for the continuation, I hope it will be before the New Year.
The following manuals and resources, forums helped me in preparing this article.
For those who have just started developing applications using OpenCV, I recommend:
1.© «RoboCraft» - this resource was created with the aim of informational reflection of the RoboCraft team's Kaliningrad team's work in the field of robotics. There are many examples of digital image processing, and more.
2. Blog of the programmer Andrei from Tomsk with the nickname Troyashka, who has long been familiar with OpenCV. His blog has tons of useful information.
3. The forum of the site about machine vision Compvision.ru , in the bowels of which there is a lot of information for beginners, I sat on it at the beginning of my acquaintance with OpenCV