Webcam, Node.js and OpenCV: making a face recognition system

Original author: Shahid (UnixRoot) Shaikh
  • Transfer
Computer vision is, in a nutshell, a set of technologies based on the principles of human vision, which allow a computer to see and understand what it sees. At first glance, it seems to be simple, but in reality it is far from the case. If you want to recognize the importance of computer vision and learn about its uses, watch this video . As the saying goes: “it’s better to see once”, in this case, to see how Amazon uses this technology to create new-generation shopping centers. Amazing, right? If you want to get involved in computer vision technologies, I suggest talking about how to create an interactive face recognition system using a conventional webcam, Node.js and OpenCV.









About the study of computer vision


Perhaps it is impossible to give a full description of computer vision technologies in one material or even in a series of articles. Therefore, we will now consider only the most important things that will allow us to begin development.

Of course, you can expand your knowledge in this area yourself, for example, by enrolling in appropriate courses, say, on EDX .

There is an organization called OpenCV working in the field of computer vision. She is developing the open-source library of the same name. This library supports C, C ++ and Java. There is no official module for Node.js, however, there is a modulenode-opencvcreated by enthusiasts. We will use this module in the following sections of the material. Namely, we consider the following questions below:

  • Install OpenCV.
  • Work with the OpenCV module for Node.js.
  • Development of an interactive face recognition system.

Install OpenCV


First you need to install the OpenCV library. If you use Ubuntu, the following commands will help you. I tested them on Ubuntu 16.04.

So, let's start by installing the dependencies required for OpenCV.

sudo apt-get install build-essential
sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev

Now install the OpenCV library itself.

sudo apt-get install libopencv-dev

You can install it in another way, using the official installation guide and the commands provided in it.

The library can also be installed for Windows.

If you are working on a Mac, you can either build OpenCV from source or use it brew.

brew tap homebrew/science
brew install opencv

Node-opencv module


As already mentioned, OpenCV does not officially offer a driver for Node.js. However, the npm registry has a module node-opencvthat Peter Braden and other programmers are working on. This module is still under development; it does not yet support all OpenCV APIs.

It should be noted that at the moment the latest build with GitHub is not integrated with the npm module node-opencv, therefore, when installing the module from npm, you may run into problems. It is recommended to install the module directly from GitHub using the following command:

npm install peterbraden/node-opencv

Now let's get down to programming.

Development of an interactive face recognition system


Before we recognize faces in the video stream of a webcam, we will figure out how to perform the same operation for a regular photo.

How to recognize faces in photos? OpenCV provides various classifiers that can be used to recognize faces, eyes, cars, and many other objects. These classifiers, however, are quite simple, they are not trained using machine learning technologies, therefore, when recognizing faces, we can rely on an accuracy of about 80%.

Here are photos of my friends from one of the usual office parties.



We will write a program that will recognize the faces in this picture. Here is the code (file face-detector.js) that reads the original image from the disk and displays a new image with the faces marked on it.

var cv = require('opencv');
cv.readImage("./friends.jpg", function(err, im){
  if (err) throw err;
  if (im.width() < 1 || im.height() < 1) throw new Error('Image has no size');
  im.detectObject(cv.FACE_CASCADE,{}, function(err, faces){
    if (err) throw err;
    for (var i = 0; i < faces.length; i++){
      var face = faces[i];
      im.ellipse(face.x + face.width / 2, face.y + face.height / 2, face.width / 2, face.height / 2);
    }
    im.save('./friends-faces.jpg');
    console.log('Image saved as friends-faces.png');
  });
});

This is how the program recognized the faces in the picture.



I know, it didn’t work out very accurately, but a start has been made, and that’s good. Now we will do the same using the video stream from the webcam. First you need to connect to the camera and display the video. Here is the code ( camera.js) that does this.

var cv = require('opencv');
try {
  var camera = new cv.VideoCapture(0);
  var window = new cv.NamedWindow('Video', 0)
  setInterval(function() {
    camera.read(function(err, im) {
      if (err) throw err;
      console.log(im.size())
      if (im.size()[0] > 0 && im.size()[1] > 0){
                if (err) throw err;
                window.show(im);
      }
      window.blockingWaitKey(0, 50);
    });
  }, 20); 
} catch (e){
  console.log("Couldn't start camera:", e)
}

Run this code with the following command.

node camera.js

If everything works as it should, you will see a window in which what the camcorder shoots will be displayed. In your case, the picture will be similar to the one shown below, only your face and not mine will get into the frame.



I do not have a webcam, but honestly, in life I look much better. Let's analyze our program.

var camera = new cv.VideoCapture(0);
var window = new cv.NamedWindow('Video', 0)

Here we try to access the video stream from the camera and create a new window for showing the video.

Now, in order to show the video and not the static picture, you need to display in the window a fresh image received from the camera. We do this by using the function setInterval()and updating the contents of the window every 20 milliseconds.

  setInterval(function() {
    camera.read(function(err, im) {
      if (err) throw err;
      console.log(im.size())
      if (im.size()[0] > 0 && im.size()[1] > 0){
                if (err) throw err;
                window.show(im);
      }
      window.blockingWaitKey(0, 50);
    });
  }, 20);

The first image is 0x0 pixels, we do not need to display it. The expression takes care of this if.

So, we know how to recognize faces in photographs and how to access the video stream from a webcam. Now we will combine all this and our interactive face recognition system will be ready to work. Here is the code (file camera.js) that recognizes faces in the video stream.

var cv = require('opencv');
try {
  var camera = new cv.VideoCapture(0);
  var window = new cv.NamedWindow('Video', 0)
  // face detection properties
  var rectColor = [0, 255, 0]; 
   var rectThickness = 2;
  setInterval(function() {
    camera.read(function(err, im) {
      if (err) throw err;
      console.log(im.size())
      if (im.size()[0] > 0 && im.size()[1] > 0){
                im.detectObject(cv.FACE_CASCADE, {}, function(err, faces) {
                if (err) throw err;
                for (var i = 0; i < faces.length; i++) {
                        face = faces[i];
                        im.rectangle([face.x, face.y], [face.width, face.height], rectColor, rectThickness);
                 }
                window.show(im);
                });
      }
      window.blockingWaitKey(0, 50);
    });
  }, 20); 
} catch (e){
  console.log("Couldn't start camera:", e)
}

Run it.

node camera.js

That's what eventually happened

Again, the recognition accuracy here is low, according to rough estimates - about 80%.

Summary


OpenCV is an amazing open source project. It can be used free of charge to develop applications that can be used, for example, in areas such as motion recognition and face recognition. In addition, standard classifiers can be trained to increase recognition accuracy. I hope that OpenCV will officially support Node.js, which will allow to use more features of this wonderful library in this environment. If you want to delve into machine vision technology, then, in addition to various training courses, it will be useful to read this book and take a look at the materials from the official OpenCV website.

Dear readers! Do you use machine vision technology in your projects?

Also popular now: