Artyom Kukharenko, founder of NTechLab - on face recognition, the potential of neural networks and business

Everyone heard about NTechLab at the moment when the FindFace ID algorithm demonstrator became available on the Web - it shocked people that it’s so simple, with a few clicks, you can determine the network coordinates of almost any person from a photo: a passerby, a passenger opposite and so on.

Although Artyom Kukharenko’s path to facial recognition began long before that, and the most significant point, in fact, was the victory in the MegaFace University of Washington competition - yesterday the company announced its cloud-based SaaS product FindFace.pro , which already has a great future.

We talked with Artyom about algorithms and business, as well as the prospects for smart object recognition technologies.

Artem Kukharenko received higher education at Moscow State University. M.V. Lomonosov, where he graduated from the Faculty of Computational Mathematics and Cybernetics. As a student, he worked at the Laboratory for Computer Graphics and Multimedia, Moscow State University.

2013 - began to collaborate with the e-Lab laboratory at Purdue American University.

2014 - Artem Kukharenko became an employee of the Moscow Research Center Samsung. At the same time, he appeared as the author of a number of scientific articles on the subject of neural networks.

What you do is not like most startups. What was the trigger to start doing this project?

There are generally many factors. I had a face recognition diploma at Moscow State University. At that time, we were just starting to study neural networks and mainly used the so-called hand-crufted features - algorithms, for example, SVM on top of LBP or HoG descriptors that arose even before neural networks.

Then I worked in different companies. They then already began to actively use neural networks. We realized that using neural networks, we can improve the quality of work in various fields.

In my free time, I developed an application that recognizes dog breeds and showed it to friends. They introduced me to investors. Together, we came up with several options for applying the technology and understood how to make money on it. As a result, they came to face recognition.

Who are your investors?

These are private individuals. Now one of them has become a co-founder. Others are also close to this. Now they devote 25-50% of their time to the project.

Then, probably, it would be more correct to call them partners?

Yes. In addition, a cool team of development engineers was selected. Initially, we were engaged in the development of the three. Just then, we managed to come up with and implement a face recognition algorithm that won the The MegaFace Benchmark contest. It was conducted by the University of Washington. It was necessary to recognize faces in a database of 1 million people. Previously, such competitions were held on small bases, and in MegaFace everything was close to real conditions.

Naturally involved more than 100 teams from around the world. And we also sent our algorithm - just to understand what stage of development we are at. And suddenly they took first place. From this moment, there was a lot of interest from customers, investors, and the press. And we began to actively develop b2b products.

FindFace appeared a little later?

Yes. Initially, we did a search engine to search large arrays of photographs (up to 1 billion). Perhaps that is why we took first place. Search accuracy is the first important criterion. And the second is search speed. If the algorithm searches for half an hour or even a minute, this is not particularly interesting to anyone.

How much have you ever thought about privacy per se? Everything has long been on the Internet. There are no people who are professionally engaged in espionage. Giving our photos to Facebook, Google, VK, we must understand that at some point they will search for these photos. What is your opinion on the negative consequences of implementing recognition algorithms?

When we created the algorithm, it was simply interesting for us engineers to make it effective. And only then we began to think about how to use it correctly and release it in the form of products for a wide audience. But we decided that when powerful technologies appear, people should know about them. If we gave the technology only to the special services that would use it, and no one would know about it, this is one thing. And we put it in open use - this is different. People know about it and draw conclusions about what to post on the network and what not.

Public opinion adjusts itself, knowing the existence of this thing.

Nevertheless, we received many letters of thanks from the Ministry of Internal Affairs, the police, etc.

Watching your profile on Facebook, I noticed that after the appearance of FindFace, your photos changed, the general background - such a "finest hour" happened. After MegaFace came a lot of orders, customers. Does the company begin to grow at a serious pace or are there structural barriers to the commercialization of these solutions? What are your business plans?

Everything is growing very rapidly in our country in all directions. The company has about 25 people. Until the new year there will be about 30. You need to quickly recruit a team, “stick together”. We seem to be doing a good job of this. Our company is divided into three logical components - a business unit, a product team (makes the final product) and a laboratory (improves algorithms, search quality, etc.).

Now we have more than 500 applications from customers. At some point, we were faced with the fact that there are a lot of orders, and as such there is no product for the business.

In what form can this exist on the market? SaaS, Black box serverside app, mobile app?

FindFace is just a demonstration of our algorithm. We do not plan to earn money on it. Now we are launching the FindFace Cloud API (SaaS) cloud service, it allows you to upload a photo and search on it.

On October 18, NTechLab introduced a business solution - FindFace.Pro.

The product allows you to integrate face recognition technologies with third-party applications through the "cloud", the service description says . With it, you can detect faces in pictures, identify and form a base for future use.

The product is aimed at shops, casinos, companies that organize concerts and other public events, and other enterprises. At the conclusion of the agreement, NTechLab credits the client $ 100, to which he can make 18 thousand requests for face recognition or create a gallery for 10 thousand faces within three months.

The service is in demand in many industries - from social networks and dating services to retail and security systems.

This is applicable in retail: for example, there is a store and a base of regular customers. In addition, the technology is used in the banking sector - confirmation of payments by selfie, authorization.

There are all kinds of applications in the field of entertainment: for example, the search for similar people. In dating services, this can be used to moderate photos, since the use of other people's photos is an urgent problem for all social networks.

You have been to CyberCrime - this is also one example of use.

Yes. There was also a project with Alfa Future People - there are many photographers. They had a chat bot that showed users photos from all the events that this or that person attended.

Our second product is the SDK. The library is written in C ++, it provides mechanisms for detecting faces, building a feature vector, base, and searching in this database. This is interesting for various integrators who will build SDKs under license in their products.

It turns out that we have SaaS, SDK, as well as custom solutions for large projects.

Not so long ago, Facebook launched a duplicate video verification service. Each video is checked in order to identify whether the copyright holder has posted it or not. Do they use technology similar to yours?

As I understand it, they are looking for full compliance. And we find from one photograph not the same, but a set of similar ones. This may work in the fight against fake accounts. We are now just chatting with several large social networks. They want to embed our algorithm. They just have a problem with searching through large databases of photographs. In the meantime, they have an “army” of moderators who do this manually.

Your target consumer is the state. They need these products the most. But no policy has happened to you yet? Will there be a problem if you sell your products not only in Russia but also in the USA, for example?

Not yet. We have a purely market history. There are no preferences. Moreover, we are building a global company. We made SaaS just for this. It is easier to sell all over the world.

What will happen to NTechLab by 2020? How many employees will be? Will your office only be in Moscow, or will you open headquarters somewhere in the states?

We will have an office in the states until the end of 2016. We have a plan - by 2020 to embed our algorithm in all cameras.

To all commercially available videos and cameras?

Yes. But perhaps we will still be engaged in other tasks: for example, the use of deep learning in medicine.

Is that what Watson is doing now? Diagnostics is mostly yes?

Yes. Medicine is the first thing that comes to mind. But next year we have enough current tasks, recognition of emotions, gender, etc.

How do you plan to implement emotion recognition? Do the same thing: sad person / funny person?

Yes. We already have it. We are testing this feature now. Perhaps soon she will appear in SaaS. It is clear that there will not be the same as in the series Lie to me. There will be no magic, but at the level of a person who does not try to hide emotions, recognition will occur.



I know that face recognition technologies before the advent of neural networks were based on fixing, determining the position of some points on a person’s face. How is this implemented with you?

We are not trying to find any points. The neural network receives the whole face at the entrance and the correct recognition answers: "This is Vasya, this is Petya, this is Masha." In the learning process, a feature vector is formed on the last layer, which contains all the information about the face. On the intermediate layers, important features according to the neural network are calculated.

If you set the structure correctly, on the last layer we get a vector of signs of 80 numbers.

That is, 80 parameters that determine the uniqueness of a person?

Yes, 80 numbers is less than half a kilobyte. Therefore, we are effectively able to do a search on large databases and economically use memory to store attributes.

What capacities are needed for this?

When the network is trained, it can work efficiently on a regular computer. For example, FindFace runs on five Amazon EC2 servers - and not on top-end machines. The system holds 50 queries per second when searching through a database of 250 million photos.

And if we need to train her?

We train her on the GPU, because it is best suited for training neural networks. One training on 20 million photos takes an average month in our case. But here we still have servers on which about 20 experiments are simultaneously running.

That is, one full GPU supercomputer is not needed, is any cluster enough?

Yes.

How much more difficult is working with video than with photos?

Video from a photo differs only in the number of frames. Therefore, when working with video, the requirements for the speed of the algorithm increase. From the point of view of training, nothing changes.

Will the SaaS that you launch allow you to work with the video?

No, but the SDK works with video. Sending to the cloud the entire video stream does not look very reasonable. There will be a module with a face detector. Therefore, for SaaS, you need to select frames and individually send them to the cloud.

How expensive is the development of such software solutions? How high is the price of the product for the consumer?

It all depends on the project. SaaS per thousand requests costs from $ 1 to $ 5.

So this is a mass product?

Of course. That is, for some event or store, it will cost about several hundred dollars a month. If there are large individual projects, the price will be different.



More about technology. SaaS and SDK are based on the same algorithm. The SDK is written in C ++. It’s clear why: it works quickly and reliably. What other languages, technologies do you use?

For training, we use standard libraries - Caffe, Theano, Torch7. We write in Python, C ++ and the Cuda platform. Using Cuda on the GPU, learning is much faster. We mainly need Python for experimentation, so that we can write something, try it. The cloud is also written on it, because it is more convenient.

What are your requirements for programmers at R&D?

What matters here is not languages ​​and technologies, but knowledge of algorithms, in general, requirements for mathematical training. Of course, we need some knowledge in the field of machine learning and C-like languages.

If we talk about the product team, then, of course, C ++ is needed there.


Photo: Alexander Utkin

Question about the database. Where is all this stored?

If we are talking about the search index, this is our development. It allows you to search in large databases in less than a second. This is part of the product.
If we talk about storing information about users, the database does not matter much here. We are using Mongo. But there is nothing related to the search.

Many people began to offer money after you became famous?

A lot, but we still have not taken money from anyone. In the near future we will close the round.

You went to the USA not so long ago. Was this your first trip?

Yes.

Was there a direct interest in you? Who did you talk to? Do I understand correctly that you combined leisure and business there?

We drove in the spring. Recently, my partners traveled to the United States without me. The interest is huge: we talked with both investors and clients. We have good relations with some of them, we work, we sent them an SDK.

Did you manage to communicate with the developers? Someone to invite to work?

Not. But we have all the development now in Russia, and there are many advantages, in particular, economic ones. We will open offices in the Valley, and after some time in Europe.