OS for Data Center Mesosphere: What is it and for whom

Geektimes already wrote that Mesosphere attracted investments (several rounds of financing brought investments in the company up to $ 50 million) to create the "first operating system for data centers" [ about why data centers need operating systems can be read in this translation from our blog ].

OS for data centers ( DCOS, data center operating system) is a new type of operating systems that operate on a scale of the entire data center and control not individual machines, but operate at a higher level and allow administrators to work with all resources of the data center, as with a single virtual entity. This will simplify management and give administrators the opportunity to put into operation, or, conversely, to decommission the server and software much faster than is happening now.

Virtual machines help make better use of the resources of individual computers, but the Mesosphere team wants to take this concept to a new level and allow users to apply the same principle to the resources of the entire data center. As today's applications are increasingly focused on working with multiple servers, it becomes unimportant how exactly the resources of an individual machine are distributed, and the current approach simply cannot provide all the arising needs.

By applying the concept of virtual machines to the entire data center, it is possible to form a single pool of resources that specialists can manipulate - this can be extremely useful for system administrators. Moreover, the ability to virtualize the entire potential of the data center drastically reduces the time to deploy server clusters from days and weeks to several hours or even minutes, depending on what size the cluster needs to be formed.



The virtual resource pool is designed to work with thousands of machines. In fact, Mesosphere talked about how their standard consumer could have a data center containing about 50 thousand nodes, and that, according to their assumptions, this standard number could be increased to 500 thousand by the end of next year.

Administrators can work with the OS for data centers through the command line or use the graphical interface to quickly select and drag resources from the library to the command line panel. This gives administrators the ability to very quickly create multiple instances of any software in the library. The default library includes popular open source products such as Apache Spark, Apache Cassandra, Apache Hadoop and Google Kubernetes, to which users can add their own internal software packages.

The data center OS is compatible with some versions of Linux, including Redhat, CentOS, Ubuntu and CoreOS, as well as with the infrastructure of public cloud services from companies such as Amazon, Google and Microsoft. Finally, it can be used in conjunction with services from a private cloud infrastructure, including virtual machines, bare metal and OpenStack.

Administrators can also set triggers using specialized tools so that when certain conditions are met, the system automatically starts a predefined set of commands.



For example, administrators can create a set of commands based on an understanding of the cost of external services, so that it only starts when the required cost threshold is exceeded. Moreover, administrators can take advantage of the API to create applications immediately for a single pool of data center resources available to the OS from Mesosphere.

Who needs an OS for data centers?

The editors of Readwrite asked this question to Brad Silverberg, the man behind the super-successful launch of Windows 95, who received the title “Mr. Windows ”and recently invested in Mesosphere.

According to the expert, one of the main purposes of cloud computing was the creation of a real operating system for data centers. However, the existing model of such a system does not work, since it is difficult to scale effectively. It is too complicated and costly. The thing is that there are several technological trends that are increasing the scale of this problem, in particular the explosive growth in the number of mobile devices that need access to data center resources and mass migration of applications to the cloud.

In addition, the emergence of radically new patterns of behavior, both people (primarily using smartphones) and machines (the Internet of things), is currently being observed. The number of smartphones on the planet is about to exceed two billion, and soon this number will double.

Meanwhile, more and more machines are getting IP addresses and are starting to interact with data centers. Applications are turning into large distributed collections of microservices that run from thousands of servers, and the amount of data continues to grow.

At the top of the iceberg are the expectations of the user who wants to get an adaptive application with the most recent content. But the usual ways to ensure that all these needs are met are very difficult.

It was previously believed that virtualization could help create scalable solutions, but Silverberg believes that this did not happen.

This is just a transitional technology, not a fundamental paradigm shift. Think about all this in terms of system complexity. Instead of splitting one physical machine into several virtual machines (which you still need to manage), is it better to combine all the machines in your data center into one large abstract computer with one operating system? And you know what? This approach is already working.

The concept has already been tested on large-scale projects on Google with their Borg / Omega architecture and on Twitter supporting Apache Mesos.

OS for data centers removes all resource restrictions. It creates a single pool of resources, within which it is possible to create applications and dynamically scale them in real time in accordance with the needs of changing the workload.

This system uses the metaphor of a single computer, which is quite simple as a model for programming, and extends it to hundreds of thousands or even millions of machines. This approach allows you to level all the difficulties associated with scaling processes.

[The full version of the interview can be read here ]

Is everything really good

Despite the positive attitude towards the OS for data centers on the part of venture capitalists and technology experts, not all users currently share their optimism. In comments to the mentioned topic on GT habrayuzer (? Giktaymsyuzer) kovyl so he described his experience with the software mesosphere:

I apologize for the harshness. Just for me, Mesosphere is like a red rag for a bull. Does anyone present here have a positive experience using IT in production? Can you share it? They tried to use it at home. We spent about six months (!!!) and a bunch of nerves in the hope of squeezing at least something out of it. The impression was that it was not functional in principle.

According to the developers, the OS for data centers from Mesosphere will be available next year (although access to an earlier version of the system already exists) - perhaps investments and support from the expert community will help the team create a more balanced product.