“Keep your finger on the pulse and look around” - an interview about AI with Intento co-founder Grigory Sapunov
The other day, we decided to talk with our main teacher at the Deep Learning program , Grigory Sapunov, and discuss with him topical issues related to the field of artificial intelligence (AI). Gregory a few years ago was the head of Yandex.News development. He is currently CTO and co-founder of Intento. He has been engaged in data analysis, artificial intelligence and machine learning for 15 years, since 2011 he has been engaged in Deep Learning, participated in the projects RoadAR (neural network recognition of objects on the road), Icon8 (neural network filters), etc.
- For a long time I wanted to substantively communicate with you on the topic of artificial intelligence. You’ve been in this topic for a long time, you are well versed and follow it, you are advising the nth number of startups ... What did you initially attract from this topic?
- I have always been partial to cross-disciplinary topics, especially at the intersection of such interesting areas for me as programming, biology, mathematics, brain sciences, psychology, linguistics, philosophy. Artificial intelligence is just at the intersection of all these areas. There are many difficult challenges here, there is a constant feeling that you are on the approaches to the unknown, and for all the already rather long history of this direction, all the same, in fact, you regularly find yourself in places where few people have visited before. This is insanely interesting.
“You started to get carried away with it, when it still wasn't fashionable.” What do you associate with the boom that we are currently experiencing with the development of AI? And is he really there? Maybe this is another hype with high expectations?
- As usual, there is a constructive story and a non-constructive one.
The constructive part is that in recent years there has been a breakthrough in working with neural networks. The paradox is that many of the ideas, algorithms and methods related to this breakthrough have been developed for a long time and are not a special novelty. About five years ago, the growth of computing capabilities and the emergence of large datasets successfully converged on one point. And then it turned out that the old algorithms worked pretty well in general, but before that there simply weren’t enough resources to properly train them. Since then, a lot of conceptually new things have appeared, and in several areas (for example, speech recognition and computer vision), deep learning has successfully passed and replaced traditional methods. Most AI books over five years old are already dangerous to read because they are out of date. For example, they constantly exploit an example task, which a computer is not able to solve, but a three-year-old child can solve - to distinguish a cat from a dog. This is a long time ago (several years ago) not true. The computer perfectly recognizes images, and does it with a quality higher than a person. Or more recently (a year or two ago) it was believed that the game of go would remain impregnable for another twenty years. In the spring of 2016, it became clear that this was no longer the case. Few expected. That is, in this place the boom is completely justified.
Moreover, there is a feeling that at the moment the fantasy is not keeping pace with technology. The current developments are very versatile, can be used for very different tasks, and nobody has just thought of a lot of such applications. Or thought of it, but have not yet devoted the necessary amount of time to this. I have my shortlist of such tasks, so I’m waiting for him to get to it, or someone will do it :)
Hype with high expectations, unfortunately, is also present. And by the example of the history of AI, we already know that everything repeats and what it leads to. Even at the dawn of the emergence of the field of AI in 1956, one of the founders of the region (John McCarthy) stated that serious progress could be achieved in one summer, and a little later in 1965, no less cool person (Herbert Simon) stated that within 20 years, machines can do everything that a person can do. And these high expectations already led to “winters of artificial intelligence”, when everyone was disappointed by the lack of promised results and covered up the financing of AI projects. There is a risk of another winter and now. Because along with the successes there are still a lot of unresolved issues that remain outside the brackets in the enthusiastic press publications. But as usual, I want to believe
- Many now fear that AI will destroy a lot of professions. What tasks can AI solve at a high level now? And who should ultimately fear AI competition?
- Various international research groups and media already regularly issue lists of professions ranked by the risk of being replaced by computers. Some of these lists look weird, some are pretty believable. You yourself can easily find several of these on the Internet.
AI can already replace or force people out in a heap of those places where people were needed only for recognition of visual or voice images. Or where people are forced to shovel huge amounts of information (this is a more native environment for computers).
I think the tragedy and comic situation will be that in the first place AI will wash a lot of people from the same area and more broadly from IT. At the moment, there are a large number of data scientists engaged in the use of ready-made recipes and enumeration of parameters for models. This stupid non-creative work should and will be automated, and different companies are already doing this (see, for example, FBLearner Flow ). In programming, there is also a huge number of tedious and boring activities that have long been time to automate (see the wonderful words of Sassman, one of the authors of SICP:“Programming today is more like science: you take a part of the library and“ poke ”it - look at what it does. Then you ask yourself, “Can I customize this so that it does what I need?”. The “analysis through synthesis” approach used in SICP, when you build a large system from simple, small parts, has become irrelevant. Today we program “by typ”, habrahabr.ru/post/282986 ). By the way, in the current Intento project, we aim to automate one of these programming areas. Follow the announcements :)
- And if you dream up for 5-10 years? The question may be urgent for those who are just entering university now.
- First of all, mediocrity will go away everywhere. Mediocre teachers, mediocre programmers, mediocre translators, mediocre lawyers. If you learn something, become the best. Well, do not shy away from new technologies. Strength is not in replacing a person with AI, but in complementing his AI. Those who will constantly improve and will be able to successfully supplement themselves with information technologies of the new century will never be the losers.
Some professions will certainly go into oblivion, but even if you are now studying for one of these professions, keep your finger on the pulse and look around. Some professions will leave, but how many new ones will appear that we still have no idea about? When horses disappeared from everyday life, a whole sea of new professions arose around automobiles and transport. One could still guess about these new professions by analogy, but when computers appeared, professions arose, many of which were impossible to even think of in advance. If you are always open to the new, then without any problems you will find a place for yourself in this new world.
- Now, saying “artificial intelligence”, we almost immediately mean or build the association “deep learning”. How justified is this at all? What else can be wired inside the AI, except for deep neural networks?
- Deep learning (DL) supplanted all other AI methods in the media field and almost became synonymous with AI. But this, of course, is not true. There are a huge number of methods outside of DL. There are, for example, evolutionary calculations and swarm intelligence methods that can find solutions to very complex optimization problems (often NP-complex). There are symbolic methods for representing knowledge and inference, which are very strong in the field of automatic reasoning and can, for example, explain how a particular conclusion is obtained. DL currently practically does not know how to do this (a recent recent publication by DeepMind about Differentiable neural computers opens the way for DL in this direction). There is an interesting area called Probabilistic Programming. Open any sane book about AI, there are a huge number of different methods in the table of contents.
I can advise my relatively recent presentation about the current state of AI, which I did for students as part of the GoTo School project. It is far from complete, but there are many examples of success other than DL approaches.
- Why do you think deep learning is good at solving many tasks that a person performs? Could something be invented as an alternative?
- This is actually a difficult question: why does DL work, and why does it work so efficiently. A parallel question, in fact, is why the human brain is capable of solving the same problems well. Scientists are trying to explain this using physics .
Apparently, there is some kind of commonality of principles, although artificial neural networks are quite far from biological.
A complementary interesting question is what tasks a person performs poorly and a machine is capable of performing well (possibly some kind of new architecture). In addition to the obvious quick calculation and storage of large amounts of data. In this sense, I really like the words of Hamming :“Just as there are odors that dogs can smell and we cannot, as well as sounds that dogs can hear and we cannot, so too there are wavelengths of light we cannot see and flavors we cannot taste. Why then, given our brains wired the way they are, does the remark "Perhaps there are thoughts we cannot think," surprise you? Evolution, so far, may possibly have blocked us from being able to think in some directions; there could be unthinkable thoughts. ” This is where AI could potentially revolutionize us.
It should be remembered that there are many other tasks for which DL is not suitable, but other methods are suitable. See for example the symbolic methods that I talked about earlier.
- In your opinion, is the market now experiencing a shortage of specialists who are capable of training deep neural networks? What trend do you expect?
- The demand for specialists is much larger than they are available. I see this by the number of calls, the dynamics of open vacancies, as well as by the presence of untouched fields, where DL must obviously make a revolution, as he did in the processing of sound and images.
Of the trends, I still expect a quick washout of “surface” data scientists. If you decide to enter this area, do not stop after the first successes, dig further. This is an area in which we must constantly study. If you are uncomfortable, DL / ML / AI is not for you. However, what are you doing in IT then?
- I understand that this is one of the reasons why you should go to oureducational program for deep learning. Tell me a little more about her - how do you see her? What is its use?
- The area is huge, you can’t become a specialist in two days or three months. I see my goal in giving the participants a basic understanding of the DL area, the terminology and intuition behind the basic methods, as well as a framework for orientation in this area, which they can subsequently meaningfully fill out based on their interests.
Another goal is to remove fear of a new area and offer easy entry into it, teach quick practical things and show how you can build working solutions from practically ready-made components, or assemble your own using powerful accessible libraries.
Although the course is very saturated, and the amount of material is huge, I did not want to stretch this course for months, but to make it compact so that even very busy people could afford it. Time is a very important resource that I strive to optimize.
- What are the requirements for a person who is going to go to her?
- Interest in the field, the desire to understand, the ability to allocate time for this, as well as the basic ability to program in python, basic knowledge of linear algebra (vectors, matrices) and matanalysis (it is useful to be friends with derivatives if you want to better understand the essence of the processes, but learn how to work with DL is possible without this, now all modern frameworks can independently make differentiation for all types of layers and for a long time it is no longer necessary to program this manually). Acquaintance with the field of machine learning is also necessary (we will not retell the basics of machine learning, for example, we will not re-examine in detail what logistic regression is, but we will restrict ourselves to a small reminder of important points).
“Everything sounds very interesting to me.” The last question is probably a classic for this topic. Do we face the uprising of cars? The community somehow adheres to different points of view. Some believe that cars can become smarter than us, but they will not threaten us. Others, including well-known technology companies, are worried and want to somehow prevent possible threats.
- I’m more concerned not with the uprising of cars, but with human stupidity, greed, shortsightedness, lack of maturity, etc., which leads to permanent wars and conflicts. By the standards of evolution, too little time has passed since the advent of civilization; we are still cavemen. I hope computers can slightly compensate for these weaknesses. I believe in augmented intelligence.
A machine smarter than a person will appear sooner or later. She reads this text with all the comments :) Such a machine will certainly change our world and I can’t predict how it will behave or what it will lead to. These are the risks. But here we are not indifferent players, something depends on us. We need to prepare for this moment.
But even earlier, there are a bunch of other risks from the fact that we are strongly tied to new technologies, without having to be aware of them properly . We create complex systems (which can be very far from AI), about which we do not understand how they work . We are poorly thinking through the risks of deployed technologies. Finally, we simply use new technologies as tools for our dirty deeds.
But apart from the risks, I would consider potential gains. For example, in biology, where huge amounts of data are accumulated that cannot be grasped by the clever head of a single super expert, or even a group of experts. The potential for using this data is huge - it’s a better understanding of all aspects of a living cell’s work, a cure for many diseases and a radical extension of human life, this is a deeper understanding of the principles of the brain. In any other science, there are also many problems and unsolved problems that AI could help solve, from physics to history. It would be possible to radically change education, increasing its effectiveness by orders of magnitude and transferring it to a whole new level. AI could help solve one of the worst “invisible” tragedies - the untapped potential of every person.
- For a long time I wanted to substantively communicate with you on the topic of artificial intelligence. You’ve been in this topic for a long time, you are well versed and follow it, you are advising the nth number of startups ... What did you initially attract from this topic?
- I have always been partial to cross-disciplinary topics, especially at the intersection of such interesting areas for me as programming, biology, mathematics, brain sciences, psychology, linguistics, philosophy. Artificial intelligence is just at the intersection of all these areas. There are many difficult challenges here, there is a constant feeling that you are on the approaches to the unknown, and for all the already rather long history of this direction, all the same, in fact, you regularly find yourself in places where few people have visited before. This is insanely interesting.
“You started to get carried away with it, when it still wasn't fashionable.” What do you associate with the boom that we are currently experiencing with the development of AI? And is he really there? Maybe this is another hype with high expectations?
- As usual, there is a constructive story and a non-constructive one.
The constructive part is that in recent years there has been a breakthrough in working with neural networks. The paradox is that many of the ideas, algorithms and methods related to this breakthrough have been developed for a long time and are not a special novelty. About five years ago, the growth of computing capabilities and the emergence of large datasets successfully converged on one point. And then it turned out that the old algorithms worked pretty well in general, but before that there simply weren’t enough resources to properly train them. Since then, a lot of conceptually new things have appeared, and in several areas (for example, speech recognition and computer vision), deep learning has successfully passed and replaced traditional methods. Most AI books over five years old are already dangerous to read because they are out of date. For example, they constantly exploit an example task, which a computer is not able to solve, but a three-year-old child can solve - to distinguish a cat from a dog. This is a long time ago (several years ago) not true. The computer perfectly recognizes images, and does it with a quality higher than a person. Or more recently (a year or two ago) it was believed that the game of go would remain impregnable for another twenty years. In the spring of 2016, it became clear that this was no longer the case. Few expected. That is, in this place the boom is completely justified.
Moreover, there is a feeling that at the moment the fantasy is not keeping pace with technology. The current developments are very versatile, can be used for very different tasks, and nobody has just thought of a lot of such applications. Or thought of it, but have not yet devoted the necessary amount of time to this. I have my shortlist of such tasks, so I’m waiting for him to get to it, or someone will do it :)
Hype with high expectations, unfortunately, is also present. And by the example of the history of AI, we already know that everything repeats and what it leads to. Even at the dawn of the emergence of the field of AI in 1956, one of the founders of the region (John McCarthy) stated that serious progress could be achieved in one summer, and a little later in 1965, no less cool person (Herbert Simon) stated that within 20 years, machines can do everything that a person can do. And these high expectations already led to “winters of artificial intelligence”, when everyone was disappointed by the lack of promised results and covered up the financing of AI projects. There is a risk of another winter and now. Because along with the successes there are still a lot of unresolved issues that remain outside the brackets in the enthusiastic press publications. But as usual, I want to believe
- Many now fear that AI will destroy a lot of professions. What tasks can AI solve at a high level now? And who should ultimately fear AI competition?
- Various international research groups and media already regularly issue lists of professions ranked by the risk of being replaced by computers. Some of these lists look weird, some are pretty believable. You yourself can easily find several of these on the Internet.
AI can already replace or force people out in a heap of those places where people were needed only for recognition of visual or voice images. Or where people are forced to shovel huge amounts of information (this is a more native environment for computers).
I think the tragedy and comic situation will be that in the first place AI will wash a lot of people from the same area and more broadly from IT. At the moment, there are a large number of data scientists engaged in the use of ready-made recipes and enumeration of parameters for models. This stupid non-creative work should and will be automated, and different companies are already doing this (see, for example, FBLearner Flow ). In programming, there is also a huge number of tedious and boring activities that have long been time to automate (see the wonderful words of Sassman, one of the authors of SICP:“Programming today is more like science: you take a part of the library and“ poke ”it - look at what it does. Then you ask yourself, “Can I customize this so that it does what I need?”. The “analysis through synthesis” approach used in SICP, when you build a large system from simple, small parts, has become irrelevant. Today we program “by typ”, habrahabr.ru/post/282986 ). By the way, in the current Intento project, we aim to automate one of these programming areas. Follow the announcements :)
- And if you dream up for 5-10 years? The question may be urgent for those who are just entering university now.
- First of all, mediocrity will go away everywhere. Mediocre teachers, mediocre programmers, mediocre translators, mediocre lawyers. If you learn something, become the best. Well, do not shy away from new technologies. Strength is not in replacing a person with AI, but in complementing his AI. Those who will constantly improve and will be able to successfully supplement themselves with information technologies of the new century will never be the losers.
Some professions will certainly go into oblivion, but even if you are now studying for one of these professions, keep your finger on the pulse and look around. Some professions will leave, but how many new ones will appear that we still have no idea about? When horses disappeared from everyday life, a whole sea of new professions arose around automobiles and transport. One could still guess about these new professions by analogy, but when computers appeared, professions arose, many of which were impossible to even think of in advance. If you are always open to the new, then without any problems you will find a place for yourself in this new world.
- Now, saying “artificial intelligence”, we almost immediately mean or build the association “deep learning”. How justified is this at all? What else can be wired inside the AI, except for deep neural networks?
- Deep learning (DL) supplanted all other AI methods in the media field and almost became synonymous with AI. But this, of course, is not true. There are a huge number of methods outside of DL. There are, for example, evolutionary calculations and swarm intelligence methods that can find solutions to very complex optimization problems (often NP-complex). There are symbolic methods for representing knowledge and inference, which are very strong in the field of automatic reasoning and can, for example, explain how a particular conclusion is obtained. DL currently practically does not know how to do this (a recent recent publication by DeepMind about Differentiable neural computers opens the way for DL in this direction). There is an interesting area called Probabilistic Programming. Open any sane book about AI, there are a huge number of different methods in the table of contents.
I can advise my relatively recent presentation about the current state of AI, which I did for students as part of the GoTo School project. It is far from complete, but there are many examples of success other than DL approaches.
- Why do you think deep learning is good at solving many tasks that a person performs? Could something be invented as an alternative?
- This is actually a difficult question: why does DL work, and why does it work so efficiently. A parallel question, in fact, is why the human brain is capable of solving the same problems well. Scientists are trying to explain this using physics .
Apparently, there is some kind of commonality of principles, although artificial neural networks are quite far from biological.
A complementary interesting question is what tasks a person performs poorly and a machine is capable of performing well (possibly some kind of new architecture). In addition to the obvious quick calculation and storage of large amounts of data. In this sense, I really like the words of Hamming :“Just as there are odors that dogs can smell and we cannot, as well as sounds that dogs can hear and we cannot, so too there are wavelengths of light we cannot see and flavors we cannot taste. Why then, given our brains wired the way they are, does the remark "Perhaps there are thoughts we cannot think," surprise you? Evolution, so far, may possibly have blocked us from being able to think in some directions; there could be unthinkable thoughts. ” This is where AI could potentially revolutionize us.
It should be remembered that there are many other tasks for which DL is not suitable, but other methods are suitable. See for example the symbolic methods that I talked about earlier.
- In your opinion, is the market now experiencing a shortage of specialists who are capable of training deep neural networks? What trend do you expect?
- The demand for specialists is much larger than they are available. I see this by the number of calls, the dynamics of open vacancies, as well as by the presence of untouched fields, where DL must obviously make a revolution, as he did in the processing of sound and images.
Of the trends, I still expect a quick washout of “surface” data scientists. If you decide to enter this area, do not stop after the first successes, dig further. This is an area in which we must constantly study. If you are uncomfortable, DL / ML / AI is not for you. However, what are you doing in IT then?
- I understand that this is one of the reasons why you should go to oureducational program for deep learning. Tell me a little more about her - how do you see her? What is its use?
- The area is huge, you can’t become a specialist in two days or three months. I see my goal in giving the participants a basic understanding of the DL area, the terminology and intuition behind the basic methods, as well as a framework for orientation in this area, which they can subsequently meaningfully fill out based on their interests.
Another goal is to remove fear of a new area and offer easy entry into it, teach quick practical things and show how you can build working solutions from practically ready-made components, or assemble your own using powerful accessible libraries.
Although the course is very saturated, and the amount of material is huge, I did not want to stretch this course for months, but to make it compact so that even very busy people could afford it. Time is a very important resource that I strive to optimize.
- What are the requirements for a person who is going to go to her?
- Interest in the field, the desire to understand, the ability to allocate time for this, as well as the basic ability to program in python, basic knowledge of linear algebra (vectors, matrices) and matanalysis (it is useful to be friends with derivatives if you want to better understand the essence of the processes, but learn how to work with DL is possible without this, now all modern frameworks can independently make differentiation for all types of layers and for a long time it is no longer necessary to program this manually). Acquaintance with the field of machine learning is also necessary (we will not retell the basics of machine learning, for example, we will not re-examine in detail what logistic regression is, but we will restrict ourselves to a small reminder of important points).
“Everything sounds very interesting to me.” The last question is probably a classic for this topic. Do we face the uprising of cars? The community somehow adheres to different points of view. Some believe that cars can become smarter than us, but they will not threaten us. Others, including well-known technology companies, are worried and want to somehow prevent possible threats.
- I’m more concerned not with the uprising of cars, but with human stupidity, greed, shortsightedness, lack of maturity, etc., which leads to permanent wars and conflicts. By the standards of evolution, too little time has passed since the advent of civilization; we are still cavemen. I hope computers can slightly compensate for these weaknesses. I believe in augmented intelligence.
A machine smarter than a person will appear sooner or later. She reads this text with all the comments :) Such a machine will certainly change our world and I can’t predict how it will behave or what it will lead to. These are the risks. But here we are not indifferent players, something depends on us. We need to prepare for this moment.
But even earlier, there are a bunch of other risks from the fact that we are strongly tied to new technologies, without having to be aware of them properly . We create complex systems (which can be very far from AI), about which we do not understand how they work . We are poorly thinking through the risks of deployed technologies. Finally, we simply use new technologies as tools for our dirty deeds.
But apart from the risks, I would consider potential gains. For example, in biology, where huge amounts of data are accumulated that cannot be grasped by the clever head of a single super expert, or even a group of experts. The potential for using this data is huge - it’s a better understanding of all aspects of a living cell’s work, a cure for many diseases and a radical extension of human life, this is a deeper understanding of the principles of the brain. In any other science, there are also many problems and unsolved problems that AI could help solve, from physics to history. It would be possible to radically change education, increasing its effectiveness by orders of magnitude and transferring it to a whole new level. AI could help solve one of the worst “invisible” tragedies - the untapped potential of every person.