June 27, 2019 at 17:39Why Swift Can Become a Big Event in Deep Learning
Hello, Habr! I present to you the translation of the article “Why Swift May Be the Next Big Thing in Deep Learning” by Max Pechyonkin, which interested me , in which the author discusses the language in which deep training will be carried out in the coming years.
If you are doing in-depth training, then most likely you should start learning Swift
If you are programming, then most likely when you hear Swift, you think about developing applications on iOS and MacOS. If you are in-depth training, then you should have heard of Swift for Tensorflow (S4TF). Then you may wonder, “Why did Google create a version of TensorFlow for Swift? After all, there are already versions for Python and C ++, why another language? ”In this post, I will try to answer this question and state the reasons why you should closely monitor S4TF, as well as the Swift language itself. I will not try to make a detailed analysis, I will only try to describe the big picture with a lot of links, if you are interested, you can dig deeper.
Swift was created by Chris Luttner when he worked at Apple. Now Chris works for Google Brain (one of the best artificial intelligence research teams in the world). The very fact that the creator of the Swift language is now working in the laboratory, which is engaged in in-depth training, speaks about the seriousness of this project.
Some time ago, people in Google realized that even if Python and a wonderful language, it still has a lot of restrictions that are hard to overcome. A new language was needed for TensorFlow, and after much deliberation, Swift was chosen as a candidate. I will not go into details here, but here a document that describes Python flaws, as well as what other languages were considered and how it all came together in Swift.
Swift for TensorFlow is not just TF for yet another language. This is essentially another branch (in the git sense) of the Swift language itself. This means that S4TF is not a library; it is a language with its own rules, with its own built-in functions that support all the functionality required by TensorFlow. For example, S4TF has a very powerful system of automatic differentiation : it is one of the fundamental things in deep learning, it is needed for calculating gradients. Compare this with Python, in which automatic differentiation is not the main component of the language. Some features originally developed as part of S4TF were later integrated into the Swift language itself.
When I first found out that Swift is as fast as C, I was amazed. I knew that C was highly optimized and allowed to achieve very high speed, but this was due to micro memory management, and therefore C had problems with memory security. In addition, C is not so easy to learn.
Swift is now running as fast as C in numerical calculations, it has no problems with memory security, and is much easier to learn. Swift's LLVM compiler is very powerful and has very efficient optimization, it will provide very fast code speed.
Since Swift for machine learning is at a very early stage of its development, so many libraries for MO have not been created for it. You should not worry too much about this since Swift has amazing Python compatibility with Python. You simply import any Python library into Swift, and it works. Also, you can import the C and C ++ libraries in Swift (for C ++, you need to make sure that the header files are written in pure C, not C ++ functions).
To summarize, if you need specific functionality, but it is not yet implemented in Swift, you can import the appropriate Python, C or C ++ packages.
Impressive!
If you ever used TensorFlow, most likely you did it with the Python package. Under the hood, the Python version of the TensorFlow library uses C code at a very low level. Therefore, when you call a function in TensorFlow, at some level you use some C code. This means that there is a limit to how deep you can check the source code. For example, if you want to see how convolutions are implemented, you won’t see the Python code, because they are implemented in C.
In Swift, it’s different. Chris Luttner calls Swift “ syntactic sugar for LLVM”[assembly language]. " This means that, in fact, Swift works almost directly with hardware, and between there are no other lines of code written in C. It also means that Swift is very fast, as I described above. This all leads to the fact that you, as a developer, can check the code at both very high and very low levels, without the need to use C.
Swift is just one part of the in-depth learning innovations that go on at Google. There is another component that is also very strongly connected: MLIR , which means Multi-level intermediate representation. MLIR will be a unifying compiler infrastructure that will allow you to write code in Swift (or any other language) and compile it on any supporting hardware. Now there are many compilers for different target equipment, but MLIR is capable of changing this, it will allow not only reusing the code, but also writing your own low-level compiler components. It will also allow researchers to use machine learning to optimize low-level algorithms.
While MLIR acts as a compiler for machine learning, we also see the possibility of using machine learning methods in compilers. This is especially important since the number of engineers developing numerical libraries is not growing as fast as the variety of models or hardware for machine learning.
Imagine using deep learning to optimize low-level algorithms for partitioning memory into data ( Halide is trying to solve a similar problem ). And this is just the beginning, we are waiting for many more other creative applications using machine learning in compilers.
If you are in the field of deep learning, then you probably should start learning Swift. This will bring many advantages over Python. Google is seriously investing in making Swift the main component of their TensorFlow ML infrastructure, and most likely Swift will become an in-depth learning language. If you are already starting to learn and use Swift, this will give you a pioneer advantage.
If you are doing in-depth training, then most likely you should start learning Swift
Introduction
If you are programming, then most likely when you hear Swift, you think about developing applications on iOS and MacOS. If you are in-depth training, then you should have heard of Swift for Tensorflow (S4TF). Then you may wonder, “Why did Google create a version of TensorFlow for Swift? After all, there are already versions for Python and C ++, why another language? ”In this post, I will try to answer this question and state the reasons why you should closely monitor S4TF, as well as the Swift language itself. I will not try to make a detailed analysis, I will only try to describe the big picture with a lot of links, if you are interested, you can dig deeper.
Swift has very strong support
Swift was created by Chris Luttner when he worked at Apple. Now Chris works for Google Brain (one of the best artificial intelligence research teams in the world). The very fact that the creator of the Swift language is now working in the laboratory, which is engaged in in-depth training, speaks about the seriousness of this project.
Some time ago, people in Google realized that even if Python and a wonderful language, it still has a lot of restrictions that are hard to overcome. A new language was needed for TensorFlow, and after much deliberation, Swift was chosen as a candidate. I will not go into details here, but here a document that describes Python flaws, as well as what other languages were considered and how it all came together in Swift.
Swift for TensorFlow is much more than just a library
Swift for TensorFlow is not just TF for yet another language. This is essentially another branch (in the git sense) of the Swift language itself. This means that S4TF is not a library; it is a language with its own rules, with its own built-in functions that support all the functionality required by TensorFlow. For example, S4TF has a very powerful system of automatic differentiation : it is one of the fundamental things in deep learning, it is needed for calculating gradients. Compare this with Python, in which automatic differentiation is not the main component of the language. Some features originally developed as part of S4TF were later integrated into the Swift language itself.
Swift fast
When I first found out that Swift is as fast as C, I was amazed. I knew that C was highly optimized and allowed to achieve very high speed, but this was due to micro memory management, and therefore C had problems with memory security. In addition, C is not so easy to learn.
Swift is now running as fast as C in numerical calculations, it has no problems with memory security, and is much easier to learn. Swift's LLVM compiler is very powerful and has very efficient optimization, it will provide very fast code speed.
You can use Python, C and C ++ in Swift
Since Swift for machine learning is at a very early stage of its development, so many libraries for MO have not been created for it. You should not worry too much about this since Swift has amazing Python compatibility with Python. You simply import any Python library into Swift, and it works. Also, you can import the C and C ++ libraries in Swift (for C ++, you need to make sure that the header files are written in pure C, not C ++ functions).
To summarize, if you need specific functionality, but it is not yet implemented in Swift, you can import the appropriate Python, C or C ++ packages.
Impressive!
Swift can go very low
If you ever used TensorFlow, most likely you did it with the Python package. Under the hood, the Python version of the TensorFlow library uses C code at a very low level. Therefore, when you call a function in TensorFlow, at some level you use some C code. This means that there is a limit to how deep you can check the source code. For example, if you want to see how convolutions are implemented, you won’t see the Python code, because they are implemented in C.
In Swift, it’s different. Chris Luttner calls Swift “ syntactic sugar for LLVM”[assembly language]. " This means that, in fact, Swift works almost directly with hardware, and between there are no other lines of code written in C. It also means that Swift is very fast, as I described above. This all leads to the fact that you, as a developer, can check the code at both very high and very low levels, without the need to use C.
What's next?
Swift is just one part of the in-depth learning innovations that go on at Google. There is another component that is also very strongly connected: MLIR , which means Multi-level intermediate representation. MLIR will be a unifying compiler infrastructure that will allow you to write code in Swift (or any other language) and compile it on any supporting hardware. Now there are many compilers for different target equipment, but MLIR is capable of changing this, it will allow not only reusing the code, but also writing your own low-level compiler components. It will also allow researchers to use machine learning to optimize low-level algorithms.
While MLIR acts as a compiler for machine learning, we also see the possibility of using machine learning methods in compilers. This is especially important since the number of engineers developing numerical libraries is not growing as fast as the variety of models or hardware for machine learning.
Imagine using deep learning to optimize low-level algorithms for partitioning memory into data ( Halide is trying to solve a similar problem ). And this is just the beginning, we are waiting for many more other creative applications using machine learning in compilers.
Summary
If you are in the field of deep learning, then you probably should start learning Swift. This will bring many advantages over Python. Google is seriously investing in making Swift the main component of their TensorFlow ML infrastructure, and most likely Swift will become an in-depth learning language. If you are already starting to learn and use Swift, this will give you a pioneer advantage.