ML.NET 0.7 (Machine Learning .NET)
- Transfer
We are pleased to announce the release of ML.NET 0.7 - the latest version of the cross-platform and open source machine learning system for .NET developers ( ML.NET 0.1 was released on // Build 2018 ). This release aims to enhance the functionality of the platform. More under the cut!
Recommender systems allow you to create personalized recommendations for products in the catalog, songs, films, etc. We have improved support for building recommendation systems in ML.NET by adding Matrix factorization (MF). This is a popular approach to recommendations, when you can get data on how users rate certain items in a directory. For example, you might know how users rate certain movies. Thanks to this knowledge, we can recommend other films that they probably also want to watch.
We added MF to ML.NET because it is often much faster than Field-Aware Factorization Machines (which we added to ML.NET 0.3), and it can maintain ratings that are numerical (for example, 1-5 stars) instead binary values (like or dislike). Although we’ve added MF, you can still use FFM if you want to use other information outside of the rating that the user assigns to the element (for example, movie genre, movie release date, user profile). A more detailed discussion of the differences can be found here .
An example of using MF can be found here.. This example is general, but you can imagine that the rows of the matrix correspond to the users, the columns of the matrix correspond to the films, and the values of the matrices correspond to the ratings. This matrix is rather scarce, since users evaluated only a small subset of the catalog.
MF ML.NET uses LIBMF .
Anomaly detection identifies unusual values or events. It is used in such scenarios as fraud detection (detection of suspicious credit card transactions) and server monitoring (detection of unusual activity).
ML.NET 0.7 allows you to detect two types of abnormal behavior:
These anomalies can be detected on two types of data using different ML.NET components:
Sample code using anomaly detection in ML.NET can be found here .
ML.NET offers many data transformations (for example, word processing, images, categorical functions, etc.). However, for some use cases, specific conversions are required. Now we have added support for custom transformations so you can easily incorporate custom solutions.
CustomMappingEstimator allows you to create your own data processing methods and enter them into the ML.NET pipeline. Here is what it will look like in the pipeline:
Below is a definition of what this custom mapping will do. In this example, we convert the text label (“spam” or “ham”) to a logical label (true or false).
A more complete example of CustomMappingEstimator can be found here .
In this ML.NET release, you can now use machine learning models on x86 / 32-bit devices (Windows only). ML.NET was previously limited to x64 devices (Windows, Linux and Mac). Please note that some components based on external connections (for example, TensorFlow) are not available on x86-Windows.
NimbusML provides experimental Python bindings for ML.NET. We have seen feedback from the external community and internal development teams regarding the use of several programming languages. We wanted as many people as possible to use ML.NET.
ML.NET not only allows data-spinners to use machine learning models in Python (with components that can also be used in scikit-learn pipelines), but also allows you to save models that can be easily used in .NET applications via ML.NET (more see here ).
ML.NET 0.6 has a new set of APIs for ML.NET, which provides increased flexibility. These APIs are still evolving in version 0.7 and later, and we would like to receive your feedback in order to make the system even better.
Want to participate? Leave feedback on ML.NET GitHub !
If you have not done so already, download ML.NET here . And explore other useful resources:
The guys from the AI-Community are now holding an online championship in Data Science with a prize fund of 600,000 rubles. Join us, the case may interest you.
Support system recommendations with Matrix Factorization
Recommender systems allow you to create personalized recommendations for products in the catalog, songs, films, etc. We have improved support for building recommendation systems in ML.NET by adding Matrix factorization (MF). This is a popular approach to recommendations, when you can get data on how users rate certain items in a directory. For example, you might know how users rate certain movies. Thanks to this knowledge, we can recommend other films that they probably also want to watch.
We added MF to ML.NET because it is often much faster than Field-Aware Factorization Machines (which we added to ML.NET 0.3), and it can maintain ratings that are numerical (for example, 1-5 stars) instead binary values (like or dislike). Although we’ve added MF, you can still use FFM if you want to use other information outside of the rating that the user assigns to the element (for example, movie genre, movie release date, user profile). A more detailed discussion of the differences can be found here .
An example of using MF can be found here.. This example is general, but you can imagine that the rows of the matrix correspond to the users, the columns of the matrix correspond to the films, and the values of the matrices correspond to the ratings. This matrix is rather scarce, since users evaluated only a small subset of the catalog.
MF ML.NET uses LIBMF .
Anomaly Detection Scenarios — Unusual Event Detection
Anomaly detection identifies unusual values or events. It is used in such scenarios as fraud detection (detection of suspicious credit card transactions) and server monitoring (detection of unusual activity).
ML.NET 0.7 allows you to detect two types of abnormal behavior:
- Spike Detection: Spikes are attributed to sudden temporal spikes in input values. These could be outliers due to malfunctions, cyber attacks, viral web content, etc.
- Change point detection: change-points mark the beginning of constant deviations in data behavior.
These anomalies can be detected on two types of data using different ML.NET components:
- IidSpikeDetector and IidChangePointDetector are used for data taken from one stationary distribution (each data point does not depend on the previous one).
- SsaSpikeDetector and SsaChangePointDetector are used for data that has components of seasonality / trend (possibly ordered by time, for example, product sales).
Sample code using anomaly detection in ML.NET can be found here .
Improved customizability of ML.NET pipelines
ML.NET offers many data transformations (for example, word processing, images, categorical functions, etc.). However, for some use cases, specific conversions are required. Now we have added support for custom transformations so you can easily incorporate custom solutions.
CustomMappingEstimator allows you to create your own data processing methods and enter them into the ML.NET pipeline. Here is what it will look like in the pipeline:
var estimator = mlContext.Transforms.CustomMapping<MyInput, MyOutput>(MyLambda.MyAction, "MyLambda")
.Append(...)
.Append(...)Below is a definition of what this custom mapping will do. In this example, we convert the text label (“spam” or “ham”) to a logical label (true or false).
publicclassMyInput
{
publicstring Label { get; set; }
}
publicclassMyOutput
{
publicbool Label { get; set; }
}
publicclassMyLambda
{
[Export("MyLambda")]
public ITransformer MyTransformer => ML.Transforms.CustomMappingTransformer<MyInput, MyOutput>(MyAction, "MyLambda");
[Import]
public MLContext ML { get; set; }
publicstaticvoidMyAction(MyInput input, MyOutput output)
{
output.Label= input.Label == "spam" ? true : false;
}
}A more complete example of CustomMappingEstimator can be found here .
X86 support in addition to x64
In this ML.NET release, you can now use machine learning models on x86 / 32-bit devices (Windows only). ML.NET was previously limited to x64 devices (Windows, Linux and Mac). Please note that some components based on external connections (for example, TensorFlow) are not available on x86-Windows.
NimbusML - experimental Python binding for ML.NET
NimbusML provides experimental Python bindings for ML.NET. We have seen feedback from the external community and internal development teams regarding the use of several programming languages. We wanted as many people as possible to use ML.NET.
ML.NET not only allows data-spinners to use machine learning models in Python (with components that can also be used in scikit-learn pipelines), but also allows you to save models that can be easily used in .NET applications via ML.NET (more see here ).
If you missed it: feedback on the new API
ML.NET 0.6 has a new set of APIs for ML.NET, which provides increased flexibility. These APIs are still evolving in version 0.7 and later, and we would like to receive your feedback in order to make the system even better.
Want to participate? Leave feedback on ML.NET GitHub !
Additional resources
- Here are the most important ML.NET concepts for understanding the new API.
- Here you can find a guide that shows how to use these APIs for various existing and new scripts.
- A link to the ML.NET API with all API documentation is provided here .
Get started!
If you have not done so already, download ML.NET here . And explore other useful resources:
- Tutorials and Guides on Microsoft Docs
- Code examples
Minute advertising
The guys from the AI-Community are now holding an online championship in Data Science with a prize fund of 600,000 rubles. Join us, the case may interest you.