IBM continues to work with Apache Spark: corporation launches Spark-as-a-service

At the IBM Insight 2015 conference , several interesting announcements were made at once. The main thing is the continuation of the development of the idea of ​​supporting the Apache Spark project. IBM launches IBM Analytics on Apache Spark, with Bluemix serving as the cloud platform. Recall that in June, IBM announced its intention to invest in the project more than $ 300 million over several years. In addition, it was previously announced that Apache Spark for Linux will be supported by z Systems.

Such support will be provided as part of the mainframe analytics project. Thanks to this, data mining experts will be able to use Apache Spark on the powerful z Systems mainframes.

Apache Spark will not only work as a service on the Bluemix platform, the system will also integrate with other cloud and analytic solutions, including the Cloudant NoSQL solution and the SashDB cloud storage platform. Developers using Bluemix will be able to integrate their projects with analytical solutions and DBMS from IBM.

Together with Spark, IBM also offers what is called Insight Cloud Services. This solution allows you to get "external data about people, events, companies, business projects from sources like Twitter and The Weather Company." IBM customers will be able to supplement and expand existing information using Insight Cloud Services, and then conduct a full analysis of the collected data complex using Apache Spark.

Because Spark supports machine learning, natural language recognition, and image processing technology, as well as offering a host of other features, IBM sees Spark as a complete data environment. For example, using the IBM Datacap service, which is part of Insight Cloud Services, a client can automatically classify and recognize the contents of a document, including its format and structure, text and numerical information.

The company believes its tool is very reliable, so more than fifteen IBM's own commercial and analytical products have been transferred to Spark. Thanks to this, for example, it was possible to reduce the number of lines of code in DataWorks from 40 to 5 million.

In the near future, IBM will expand its support for Apache Spark beyond analytics in all areas of its own business.