Using Apache Ignite at Home

In continuation of the topic “in an accessible language about Ignite / GridGain ”, begun in a previous post ( Why Apache Ignite is needed ), let's look at examples of using the product “for mere mortals”.

Terabytes of data, clusters for hundreds of machines, big data , high load, machine learning , microservices and other scary words - all this is available Ignite. But this does not mean that it is not suitable for less ambitious purposes.

Today we will look at how Ignite can easily store any of your objects, share them over the network and provide .NET and Java interaction.

We will talk more about the .NET API, but some things are applicable in Java.

Embedded Database

Ignite does not require installation and configuration, it can be launched right in your process, and since version 2.1 it can store data on disk . Add to this the fast and compact built-in serialization and get the easiest way to save any of your objects to disk.

var cfg = new IgniteConfiguration
{
    // Включить хранение данных на диске
    PersistentStoreConfiguration = new PersistentStoreConfiguration()
};
var ignite = Ignition.Start(cfg);
ignite.SetActive(true);  // Явная активация необходима при включённом persistencevar cache = ignite.GetOrCreateCache<int, MyClass>("foo");
cache[cache.GetSize() + 1] = new MyClass();

That's all the code! We start several times and make sure that the data is not lost.

In this case MyClass, no requirements are imposed! It can be anything, of any nesting, including delegates, anonymous types and methods , expression trees , dynamic objects, and so on. Can your serializer do this? I don’t think so. The only exception is that you cannot do it IntPtr, but this is for your own good. Serializing pointers is a bad idea.

Moreover, serialized objects are not a “black box”; IBinaryObjectit allows you to selectively receive and modify individual fields of objects.

using (var ignite = Ignition.Start())
{
    var cache = ignite.CreateCache<int, object>("c");
    // Сериализуем Type, обратно получаем тот же инстанс!
    cache[1] = typeof(int);
    Console.WriteLine(ReferenceEquals(typeof(int), cache[1]));  // true// Сериализуем делегат.var greeting = "Hi!";
    cache[2] = (Action) (() => { Console.WriteLine(greeting); });
    ((Action) cache[2])();  // Дезериализуем, выполняем: Hi!// Теперь отредактируем сериализованный делегат!// Переключаемся в режим Binary - работа в сериализованном виде.
    ICache<int, IBinaryObject> binCache = cache.WithKeepBinary<int, IBinaryObject>();
    // Читаем делегат в виде BinaryObject.
    IBinaryObject binDelegate = binCache[2];
    // Получим поле target0 - это класс, содержащий захваченные переменные.
    IBinaryObject target = binDelegate.GetField<IBinaryObject>("target0");
    // Меняем значение захваченной переменной greeting.
    target = target.ToBuilder().SetField("greeting", "Woot!").Build();
    // Собираем всё обратно и кладём в кэш.
    binCache[2] = binDelegate.ToBuilder().SetField("target0", target).Build();
    // Берём из кэша в обычном, десериализованном режиме, и запускаем.
    ((Action)cache[2])();  // Woot!// Анонимный тип.
    cache[3] = new { Foo = "foo", Bar = 42 };
    // Поля выглядят интересно.
    Console.WriteLine(binCache[3]);  // ...[<Bar>i__Field=42, <Foo>i__Field=foo]// Динамический объект.dynamic dynObj = new ExpandoObject();
    dynObj.Baz = "baz";
    dynObj.Qux = 1.28;
    cache[4] = dynObj;
    Console.WriteLine(binCache[4]); // _keys=[Baz, Qux], _dataArray=[baz, 1.28, ]
}

Of course, you can work with all this data both in key-value mode ( ICache.Put, ICache.Get) and through SQL , LINQ , full-text search .

I note that, in addition to the built-in mode, Ignite can be operated through ODBC and JDBC .

Interprocess communication

The Google Chrome browser uses a separate process for each tab. Implementing this approach with Ignite is very simple: you can easily and transparently exchange data through the cache, synchronize code execution through distributed data structures, and exchange messages through Messaging .

Processes can be equitable. For example, the navigation history can be stored in the Replicatedcache, so that each tab, when clicking on the link, updates the history, can read it, and the fall of one of the processes does not affect the data safety. In code, it all looks just like working with a collection.

var history = ignite.GetCache<Guid, HistoryItem>("history");
history.Put(Guid.NewGuid(), new HistoryItem(url, DateTime.UtcNow));
// Вывести последние 10 посещённых страниц
history.AsCacheQueryable()
          .Select(x => x.Value)
          .OrderByDescending(x => x.Date)
          .Take(10);

Ignite provides thread-safe data access across the entire cluster.

Cross platform collaboration

Ignite has full APIs for Java, .NET, C ++. You can run on Windows, Linux, Mac. For example, part of your application can be written in Java and run on Linux, the other part in .NET and under Windows.

The serialization protocol is universal, objects recorded on one platform can be read on another. The data structures mentioned above are also cross-platform.

Moreover, it is possible to transparently invoke services written in Java from the .NET side :

publicclassMyJavaServiceimplementsService{
  public String sayHello(String x){
    return"Hello, " + x;
  }
}
ignite.services().deployClusterSingleton("myJavaSvc", new MyJavaService());

interfaceIJavaService// имя не имеет значения
{
  stringsayHello(string str);  // имя идентично, сигнатура должна быть совместима
}
var prx = ignite.GetServices().GetServiceProxy<IJavaService>("myJavaSvc");
string result = prx.sayHello("Vasya");  

As elsewhere, instead of strings, you can pass any objects.

A detailed guide to building a cross-platform Ignite cluster, using an example of a simple chat, is here (in English).

Conclusion

Ignite makes working with any data quick and enjoyable: serialize, save to memory or to disk, transfer it over the network, ensure consistency and thread safety. Even the smallest projects can benefit from the product without worrying about a possible increase in workload in the future.