December 20, 2016 at 10:27A selection of machine learning frameworks
In recent years, machine learning has become a mainstream of unprecedented power. This trend is fueled not only by the cheapness of cloud environments, but also by the availability of the most powerful video cards used for such calculations - there are also a lot of frameworks for machine learning. Almost all of them are open source, but more importantly, these frameworks are designed in such a way as to abstract from the most difficult parts of machine learning, making these technologies more accessible to a wide class of developers. Under the cut is a selection of machine learning frameworks, both recently created and redesigned in the past year. If you are doing well with English, then the original article is available here .
Apache Spark MLlib
Apache Spark is best known for its involvement in the Hadoop family. But this in-memory framework has emerged outside of Hadoop, and still continues to earn a reputation outside of this ecosystem. Spark has grown into a familiar machine learning tool thanks to a growing library of algorithms that can be quickly applied to in-memory data.
Spark is not frozen in its development, its algorithms are constantly expanding and revising. In release 1.5many new algorithms were added, existing ones were improved, and support for MLlib, the main platform for solving mathematical and statistical problems, was strengthened in Python. In Spark 1.6, among other things, thanks to continuous pipelines, the ability to pause and continue Spark ML tasks has been added.
Apache singa
The “ deep learning ” frameworks are used to solve the difficult tasks of machine learning, such as processing natural languages and image recognition. Recently , the Singa open source framework was introduced into the Apache incubator , designed to facilitate the training of deep learning models on large data volumes.
Singa provides a simple software model for training networks based on a cluster of machines, and also supports many standard types of training tasks: convolutional neural networks , restricted Boltzmann machines, and recurrent neural networks. Models can be trained synchronously (one after the other) and asynchronously (jointly), depending on what is best suited for this problem. Singa also makes it easy to set up a cluster using Apache Zookeeper .
Caffe
Caffe is a deep learning framework. It is made "with the expectation of expressiveness, speed and modularity." Initially, the framework was created for machine vision projects, but since then it has evolved and is now used for other tasks, including speech recognition and working with multimedia.
The main advantage of Caffe is speed. The whole framework is written in C ++, supports CUDA, and if necessary is able to switch the processing flow between the processor and the video card. The package includes a set of free and open source reference models for standard classification tasks. Also, many models were created by the Caffe user community .
Microsoft Azure ML Studio
Given the enormous amount of data and processing power needed for machine learning, clouds are an ideal medium for ML applications. Microsoft has equipped Azure with its own machine learning service, for which you can only pay for the fact of use - Azure ML Studio. Monthly and hourly versions are available, as well as free (tier). In particular, the HowOldRobot project was created using this system .
Azure ML Studio allows you to create and train models, turn them into APIs for providing other services. Up to 10 GB of storage space can be allocated per user account, although you can also connect your own Azure storage. A wide range of algorithms created by Microsoft and third parties are available. To try the service, you don’t even need to create an account, just log in anonymously and you can drive Azure ML Studio for eight hours.
Amazon machine learning
Amazon has its own standard approach to providing cloud services: first, an interested audience is provided with basic functionality, this audience sculpts something from it, and the company finds out what people really need.
The same goes for Amazon Machine Learning.. The service connects to data stored in Amazon S3, Redshift or RDS, it can perform binary classification, multiclass categorization, as well as regression on the specified data to create a model. However, this service is tied to Amazon. Not only does it use the data stored in company-owned storages, it’s also impossible to import or export models, and data samples for training can not be more than 100 GB. But still, this is a good tool to start with, illustrating that machine learning is turning from luxury into a practical tool.
Microsoft Distributed Machine Learning Toolkit
The more computers you can use to solve the problem of machine learning, the better. But combining a large fleet of vehicles and creating ML applications that run efficiently on them can be a daunting task. The DMTK (Distributed Machine Learning Toolkit) framework is designed to solve the problem of distributing various ML operations across a cluster of systems.
DMTK is considered to be a framework, and not a full-blown boxed solution, so a small number of algorithms come with it. But the DMTK architecture allows you to expand it, as well as squeeze everything possible out of clusters with limited resources. For example, each cluster node has its own cache, which reduces the amount of data exchange with the central node, which provides on-demand parameters for tasks.
Google TensorFlow
Like Microsoft DMTK, Google TensorFlow is a machine learning framework designed to distribute computing across a cluster. Along with Google Kubernetes, this framework was developed to solve Google’s internal problems, but in the end the company released it in the open swimming as an open source product.
TensorFlow implements data flow graphs when pieces of data (“tensors”) can be processed by a series of algorithms described by the graph. Moving data around the system is called “streams”. Graphs can be collected using C ++ or Python, and processed by a processor or graphics card. Google has long-term plans for developing TensorFlow through third-party developers.
Microsoft Computational Network Toolkit
Hot on the heels of DMTK, Microsoft has released yet another machine learning toolkit - CNTK .
CNTK is similar to Google TensorFlow, it allows you to create neural networks through directed graphs . Microsoft compares this framework with products such as Caffe, Theano, and Torch. Its main advantage is speed, especially when it comes to the parallel use of several processors and video cards. Microsoft claims that using CNTK in conjunction with Azure-based GPU clusters can speed up the training for speech recognition by Cortana's virtual assistant.
Initially, CNTK was developed as part of a speech recognition research program and was offered as an open source project, but the company has since re-released it on GitHub under a much more liberal license.
Veles (Samsung)
Veles is a distributed platform for creating a deep learning application. Like TensorFlow and DMTK, it is written in C ++, although Python is used to automate and coordinate nodes. Before being fed to a cluster of data samples, they can be analyzed and automatically normalized. The REST API allows you to immediately use trained models in work projects (if you have powerful enough equipment).
Using Python in Veles goes beyond sticking code. For example, IPython (now Jupyter), a tool for visualizing and analyzing data, can output data from a Veles cluster. Samsung hopes open source status will help drive further product development, as will porting to Windows and Mac OS X.
Brainstorm
The Brainstorm project was developed by graduate students from the Swiss Institute IDSIA ( Institute Dalle Molle for Artificial Intelligence ). It was created "in order to make deep learning neural networks faster, more flexible and more interesting." Already there is support for various recurrent neural networks, for example, LSTM.
Brainstorm uses Python to implement two "handlers" - the data management API: one for processor computing using the Numpy library, and the second for using video cards using CUDA. Most of the work is done in Python scripts, so do not expect a luxurious front-end interface, unless you screw something on your own. But the authors have far-reaching plans for "learning from earlier open source projects" and using "new design elements that are compatible with various platforms and computing backends."
mlpack 2
Many machine learning projects use mlpack , a C ++ library created in 2011 and designed to “scale, speed up and simplify use”. You can implement mlpack to perform whipped up operations such as a “black box” using the cache of files executed via the command line, and for more complex tasks, using the C ++ API.
A lot of work was done in mlpack 2.0 on refactoring and implementing new algorithms, processing, speeding up and getting rid of inefficient old algorithms. For example, for native C ++ 11 random number generation functions, the Boost library generator was excluded.
One of the long-standing drawbacks of mlpack is the lack of binders for any other languages except C ++. Therefore, programmers who write in these other languages cannot use mlpack until someone rolls the appropriate wrapper. MATLAB support has been added , but such projects benefit most when they are directly useful in the main environments where machine learning is used.
Marvin
Another relatively fresh product. Marvin is a neural network framework created by the Princeton Vision Group. It is based on just a few files written in C ++, and a CUDA framework. Despite the minimal code, Marvin comes with a good number of pre-trained models that can be used with proper quoting and implemented using pull requests, like the code for the project itself.
Neon
Nervana creates a hardware and software platform for deep learning. And as an open source project, it offers the Neon framework . With the help of plug-ins, it can perform heavy computing on processors, video cards or equipment created by Nervana.
Neon is written in Python, with several pieces in C ++ and assembler. So if you are doing scientific work in Python, or using some other framework that has Python binders, you can immediately use Neon.
In conclusion, I want to say that, of course, these are far from all popular frameworks. Surely there are a dozen of your favorite tools in your bins. Feel free to share your findings in the comments to this article.
