August 3, 2016 at 20:08Deep Learning - what to do, whom to beat
Nowhere, probably, is there such an urgent need for a synergy of knowledge in various fields of science - such as in the field of machine learning and Deep Learning. It’s enough to open the hood of TensorFlow and be horrified - a huge amount of python code working with tensors inside ... C ++, mixed with numpy , for laying out in production it takes a little bit of “pluses”, a bit of bazel (it’s so exciting, I ’ve dreamed about it all my life! ) And the other extreme - the guys from Deeplearning4j cursed python to hell and rotate the tensors on good old java. But the farthest away from all seemed to be students from the University of New York - people, and not only students , and for a long time and seriously burn onLuajit + nginx ( Amen in Catholic ). The situation is complicated by the recent demarche Google DeepMind against "grandfather torch»: all projects translate into their internal engine, born of DistBelief .
Sheer chaos and mess.
But let's start from the end. Is it worth the time to learn Deep Learning, how much business needs it and what is it all about? Recall that the "classical" machine learning works the same, and not bad, but, BUT ... a careful selection and magical-unscientific manipulation of attributes with the involvement of experts in the subject area is required - and this is long, expensive and fragile. Everyone wants to give the car data, and so that she herself will learn everything (here’s an archive with books for you — learn to write new tomorrow; here are photos for you — learn to draw). And do not get distracted, beauty, work. And it began to turn out.
Well-known good working examples that do not need language experts are machine translation . Well, it's fun - we take the corpus with translations, teach the neuron and voila - it translates not worse, but even better than Moses.
Another example where DeepLearning blossomed in its glory - machine vision . Also fun - several layers of different types of neurons (convolutional, pooling) and a miracle - the network classifies the image faster and sometimes more accurately than a person. And again - it learns this only on data, features are selected automatically during training. Perfectly!
Another promising example is recurring neurons for chatbots and for language modeling .
Nevertheless, Amazon will release a product that internally represents ETL + the old, good, “free” logistic regression! "Madness" and courage.
Life is so arranged that a mathematically well-savvy colleagues usually written on the "not very convenient to exploitation under high load language" . Moreover, these colleagues are very smart , know a lot, multiply matrices in the mind and distinguish logistic regression from discriminant analysis. And experienced experienced experienced engineers of highload, big data and enterprise, DevOps angels - program well in large volumes and then it doesn’t go out, they translate the code into assembler in their minds - but it’s unlikely that they will even be able to master and explain the principle of LDA operation on fingers referring to the Beta distribution . And do not show these seasoned fighters “programming” in the “ipython notebook” - they will be locked in the server room and forced to rewrite the code on ANSI C overnight ;-)
It is not clear yet what to do with all of this - Google hires scientists from all over the world, merges them with programmersand gets distortions from a mixture of python and C ++ . Programmers hire scientists and write all the mathematics in "Orthodox" java, and to understand the code, you need to have experience in system programming for 10 years . But there is still a feeling that in the coming years an intensive exchange of knowledge will begin and the first group of colleagues will begin to comprehend the secrets of writing “good code”, and the second - to learn how to multiply matrices in memory :-)
In the meantime, we learn to negotiate, there is no way out.
What to do to understand and succeed - to constantly learn and acquire related knowledge! Good algorithms, high-quality models that learn from data without the help of experts - the business needs it more than ever. Deeplearning is developing very intensively, this is an active area of scientific research and penetrates into our lives more and more deeply.
Fortunately, when training models on the GPU (and many frameworks already support this feature ), you can speed up every 10 times. There is not enough speed - we start on the cluster. TensorFlow can do this from the box, Deeplearning4j - scales on Spark .
Yes, you need to get used to the scientific style of presentation- but over time it becomes clear that there are not so many options for Deep Learning architectures and the tasks they solve, and the mechanics of each class of solutions are gradually becoming clearer.
If it’s hard for you to understand the three-story matrices that describe the physics of neurons - put off WorldOfTanks, spend one weekend on an introductory course in linear algebra and everything will open over time. Scientists create models, and we engineers only use them; Well, sometimes we combine.
And you never need to rush to extremes. Here is the final tasty and interesting article on how Google implemented its recommendation system for Google Play - it’s clear that we started with the classic, though “wide” classifier, and ended with Deep & Wideand with good quality indicators of the model :-)
Good luck, inspiration, energy and new, interesting tasks and beautiful, effective solutions!
Sheer chaos and mess.
Experts - "in the furnace"
But let's start from the end. Is it worth the time to learn Deep Learning, how much business needs it and what is it all about? Recall that the "classical" machine learning works the same, and not bad, but, BUT ... a careful selection and magical-unscientific manipulation of attributes with the involvement of experts in the subject area is required - and this is long, expensive and fragile. Everyone wants to give the car data, and so that she herself will learn everything (here’s an archive with books for you — learn to write new tomorrow; here are photos for you — learn to draw). And do not get distracted, beauty, work. And it began to turn out.
Well-known good working examples that do not need language experts are machine translation . Well, it's fun - we take the corpus with translations, teach the neuron and voila - it translates not worse, but even better than Moses.
Another example where DeepLearning blossomed in its glory - machine vision . Also fun - several layers of different types of neurons (convolutional, pooling) and a miracle - the network classifies the image faster and sometimes more accurately than a person. And again - it learns this only on data, features are selected automatically during training. Perfectly!
Another promising example is recurring neurons for chatbots and for language modeling .
Nevertheless, Amazon will release a product that internally represents ETL + the old, good, “free” logistic regression! "Madness" and courage.
Theory and practice
Life is so arranged that a mathematically well-savvy colleagues usually written on the "not very convenient to exploitation under high load language" . Moreover, these colleagues are very smart , know a lot, multiply matrices in the mind and distinguish logistic regression from discriminant analysis. And experienced experienced experienced engineers of highload, big data and enterprise, DevOps angels - program well in large volumes and then it doesn’t go out, they translate the code into assembler in their minds - but it’s unlikely that they will even be able to master and explain the principle of LDA operation on fingers referring to the Beta distribution . And do not show these seasoned fighters “programming” in the “ipython notebook” - they will be locked in the server room and forced to rewrite the code on ANSI C overnight ;-)
It is not clear yet what to do with all of this - Google hires scientists from all over the world, merges them with programmers
In the meantime, we learn to negotiate, there is no way out.
What to do?
What to do to understand and succeed - to constantly learn and acquire related knowledge! Good algorithms, high-quality models that learn from data without the help of experts - the business needs it more than ever. Deeplearning is developing very intensively, this is an active area of scientific research and penetrates into our lives more and more deeply.
Fortunately, when training models on the GPU (and many frameworks already support this feature ), you can speed up every 10 times. There is not enough speed - we start on the cluster. TensorFlow can do this from the box, Deeplearning4j - scales on Spark .
Yes, you need to get used to the scientific style of presentation- but over time it becomes clear that there are not so many options for Deep Learning architectures and the tasks they solve, and the mechanics of each class of solutions are gradually becoming clearer.
If it’s hard for you to understand the three-story matrices that describe the physics of neurons - put off WorldOfTanks, spend one weekend on an introductory course in linear algebra and everything will open over time. Scientists create models, and we engineers only use them; Well, sometimes we combine.
And you never need to rush to extremes. Here is the final tasty and interesting article on how Google implemented its recommendation system for Google Play - it’s clear that we started with the classic, though “wide” classifier, and ended with Deep & Wideand with good quality indicators of the model :-)
Good luck, inspiration, energy and new, interesting tasks and beautiful, effective solutions!