How FB, Apple, and Google break tradition in the computer business

Andrey Chernyshev (Delta Solutions)

Today I wanted to talk about interesting things, as it seems. The fact that probably more than fifty percent of us do not know well. We all know what OpenSource technologies are in software and, probably, a surprise for many of us are OpenSource technologies in hardware , in hardware. Today I’m trying not to spend a lot of time on this, but to tell as informative as possible what revolution is happening and how it can even affect us.

The trendsetters in this whole story are our great grandees. Ours, not in the sense of “Russian,” but in the sense of “known to all.” They are moving in this direction not of their own free will. We will discuss this now.



As you know, the biggest costs for large web projects are the data center component, the component that costs them the most in capital costs. And in operating costs, too. And new movements in the part of server infrastructure are pushing the entire industry of the data center-building and, in general, the entire infrastructure industry to change. And these changes are by and large revolutionary. Let's not hide a sin, it really is.

And the most important thing is what they struggle with. Those of you who have faced infrastructure problems within your projects are all afraid and, in general, all are struggling with a certain vendor-lock, with a certain lock on a particular equipment manufacturer. This is connected with the price component and not only with the price component.


The essence of all this revolution-evolution is to try to simplify the infrastructure. And the grandees are fighting to simplify every piece of data center infrastructure. But easier - it’s clear that it’s cheaper in the end, well, and more reliable.



This notorious vendor not only keeps everyone by the throat. In Internet projects, the dependence is somewhat less than in the enterprise, but it also exists. And, most importantly, why we are fighting with him is the cost of infrastructure.

Of course, the grandees set a goal and asked the question: how to significantly, not by 3-5-10-15 percent, but significantly affect the cost of infrastructure.



They organized in a certain consortium called the Open Compute Project, which today, a little later, I want to say a few words. In our reality, in which we now live, there is some hysteria about import substitution, which is moving not only in the minds of our officials, but, in principle, some kind of patriotism also arises in our IT bodies. Because in software we learned to do a lot with our own hands and, in principle, we are proud of it. And now we are moving slowly to the fact that in iron we will begin to do something with our own hands. Import substitution, probably, on these open standards - those standards that are written by industry grandees.



What to do? The most important thing is what to do? Try to save on infrastructure just a little bit? Also one way out. But, as you know, all revolutionary breakthroughs are not driven by a 1-2 percent improvement, but by some significant improvement, or a rethinking of what is now. That is, we are struggling with certain habits and with certain foundations. And this creates just revolutionary products.



The principles that allow to compete with the vendor-lock, to compete with the price and, in general, compete, most importantly, for the effectiveness of those resources that are cheap in Russia, namely: electricity to a greater extent and, as previously thought, working force. These principles allow us to look a little differently at the design of all the information systems that we come up with, do, and develop. And some postulates that lie in those. tasks for new open platforms, they are, well, known to everyone:

• The first is hyperconvergence. That is, we stop separating storage and computing from each other. For us, software developers, this is a universal one, and Amazon has taught us this, a certain resource.
• Further, an important thing. The main functionality is exactly what we need as software developers, because we can write the rest ourselves.
• That is, the minimum implementation of the functionality inside the hardware. We will write the rest ourselves.
• A good path, a good understanding of where this is going is also very important. And we ourselves want to influence where the hardware evolution will move, i.e. how to make our systems, our platforms that we are developing, to make absolutely independent of the trends in the global hardware development market. We want, on the contrary, hardware development to develop under our architectural solutions.
• Well, of course, everything is completely open there, and we want to convince not only that Linux and other open source operating systems, but, in principle, would like to look at the BIOS. But this has a greater effect on some specials. applications.

The main tasks themselves are formed from ideological directions voiced earlier. Among other things, ultra-low power consumption is important as a result of a compacting infrastructure. After all, our electricity is also becoming more expensive, and this is a very large part of infrastructure costs.

What was born?



Two big ones flashed ... We always love to say that there is Microsoft and something else. Unfortunately, such a super-big grandee that could have encountered it did not happen in the Linux world. Share, we can say, this large landscape of Canonical and RedHat among themselves. So, the big consumers of infrastructure solutions, which, in principle, come up with and decide now for us where to move the industry in this direction, are all of us beloved Facebook and, beloved, probably Microsoft.

And two giants within the framework of one project, as it happened by chance, implement two new interesting concepts. All of them take off under the OCP (Open Compute Project) mushroom and what is commonly called Facebook Pure OCP (now I’ll tell you a little about it), and Microsoft is moving its own initiative - Open Cloud Server.



The views are very different. In short, Facebook has always built the infrastructure for itself, including data centers. And the idea is to give Facebook internal developers some components, some bricks (bricks) from which any solution of any architecture can be assembled. Absolutely any. This gave rise to a certain unification within the components, and also opened up the possibility - primarily due to the pricing structure - opened up the possibility for third party manufacturers to make compatible components with this infrastructure, with OCP.



The components are very simple: these are storage modules, these are computing nodes, these are microservers, these are network solutions and power management systems.

Within the allotted time, there is no way to talk about the details, about technical implementation, if anyone is interested, it will be possible to talk about it separately, but the most serious breakthrough they made is a breakthrough in the efficiency of using electricity inside the rack (server rack). If it will be interesting, I will tell about it further.

The most important thing is that these are all bricks, all these components, they are absolutely independent from each other. Served separately, upgraded separately. Very comfortably.



What does Microsoft have? Microsoft has moved its Open Computing Server as part of its Azure rollout project worldwide. Somewhere where they did not fully believe in the markets, they used a standard 19-inch infrastructure, or rented existing data centers with existing racks and, as a result, the standard has a certain legacy trace, like an acquired trace, this is the use standard 19-inch racks for server and storage'ovoy economy.

There are disadvantages and advantages to this approach. To a greater extent, the density is very similar to Facebook, but not so. There are problems with heat dissipation - it is struggling slowly. There are problems with electricity, but they catch up with the use of special power supplies. At the same time, from the point of view of the cost-model, they didn’t get so bad.

In our opinion, it is interesting to consider this thread, this fork of a real Open Computing project, if you build your projects on Windows and plan to develop the infrastructure on Windows.



There are only three components: compute components, storage components and mixed components. And this is only one difference from Facebook. The network infrastructure in Microsoft's concept is not part of the project itself.



Two interpretations, two "Big" are fighting. Is this a technical war or a political one? This is actually a political war.

Some of us continue to work for a large company, some of us work for ourselves, but sometimes we catch ourselves thinking that we want for our leadership, for our company to be some indispensable, let's say, an important link. This very seriously affects the things that we do.

By the way, we posted publicly available videos of the last five years of the conference of developers of high-loaded systems HighLoad ++ . Watch, study, share and subscribe to the YouTube channel .

Perhaps this is not true in the first instance, but the people who move the two forks are mainly driven by this action. That is, they develop their individual forks and concepts solely in order to have a field for activity. Because in terms of goals, they are alone.



Is it a war or not a war?

War! First of all, the war with Intel. She is very serious. No matter how much we love him, but, unfortunately, he now has no competitors in the processor market. I will make a reservation: there are practically no competitors in the processor market.

And most importantly: it is very important that the lack of competition always, you know, stops a little progress.

The same ones, if we talk about Microsoft, they are “out of the race”, but the same Facebook, flirting with ARM some time ago, gave birth to a new project on the sidelines of Intel. And this project is now known to everyone, it is no longer NDA'ny (NDA = Non Disclosure Agreement - non-disclosure agreement). This Broadwell-D is the new processor that the marketing name is Xeon-D. This is a single of chip processor, which is specially designed for Facebook's needs from the beginning.

All the time I confuse: “hip-hop” or “hard rock” Virtual Machine in PHP uses J, but the point is that all Facebook’s frontends work in PHP, and, in principle, these are quite heavy frontends. And the processors of small, low-nuclear, with low-power cores, were always not enough to twist this story. And twist it on dual-processor Xeon'ah quite expensive in terms of cost and efficiency. And so, in competition with ARM, a new processor was born that Facebook, thanks to this political war and the struggle of standards, gave birth, in general, to the whole world. And now, I know, I’ve talked here with several guys right now, even in projects on the frontlines, Xeon-D is already used to the full height, they have grown from Atom’s pants for a long time.

These are the changes that in fact really influenced our lives. And we see this, as Open Source Hardware Computing itself affects the whole of our lives.

What happens next? What are the benefits in the end? New products are born - what do we get from this? Can we use this somehow? Of course we can.

New stories with Cavium, with new processors designed, designed mostly for storage applications, which we are starting to use gradually. This is the same SIF, for example, or distributed file systems. They are demanding on the cores. And, in general, these are processors, and this silicone, and this iron, which is being developed, including printed circuit boards, servers, and so on, are being developed thanks to these guys who are actually fighting among themselves.

What is the signal of all this? The signal is as follows: a large customer is now able to influence the product line from which our infrastructure is built. For the entire product line. If earlier, be it Hewlett-Packard, IBM, they offered the market a product formed on the basis of some kind of their own opinions that will be sold in one or another SMB, or in one or another Internet project, or in one or another large enterprise customer. Now the situation, a couple of years ago, began to unfold in a completely different area. Large customers determine what the infrastructure should look like, and this is done not without the help of colleagues involved, in fact, with software infrastructure.

Do we need to keep track of this? I think it is necessary.

How do we know all about this? In our activities, we are developing a real-time platform for telecommunications operators. Recently, when the ruble was divided in half, or the dollar was multiplied by two — whoever they like to call — customers want to consider everything in rubles. They want to consider primarily in the context of the entire project: hardware plus software, everything is in rubles. The ruble has fallen in price - this means that in the integrated project, part of the infrastructure has doubled. That is, money for software and work, in general, was left. And that made us look at this story. Since life has forced me to write software, which, in principle, is not very demanding on the infrastructure in terms of redundancy (redundancy, duplication in order to increase reliability). Perhaps this is more the same as what you do, the same story.

Looking at the “titan struggle” in detail, we saw that the new standards make it possible not only to develop your own software, but also hardware.

If there is a minute - look, they are interested in the consortium’s website, they have laid out new licenses, with the help of which, after reading them carefully - there are pitfalls - you can develop your own hardware within the existing constructs. I'm not talking about any Baikal processors, what our state is moving, with the support of Elbrus, or something else. It's about the constructs needed for your applications.

Not so long ago, when discussing the issue of optimizing their mail, the guys from MAIL.RU and I clashed and talked on a very interesting topic - processors are becoming more powerful, cores are more powerful, and cold storage, as it was cold, is now becoming even colder. Because from the Nativity of Christ we keep all our photos there. And thus, the requirement for the number of spindles that need to be added to the same or slightly increased computational resource of the processor, it increases. And while the industry is not really running this way, since there are not many of these custom applications.

Nothing complicated: Inventor, Solid Works, a couple of good engineers who can work with metal, some motherboards that are developed by industry giants and have normal reliability parameters - and here you have a custom device in your form factor.



Big ones, what are they doing?

Formally, everyone joined this consortium in order to join, to be somewhere nearby. But HP is moving, for example, with FoxConn, where FoxConn produces hardware, HP is trying to sell them under its own brand.

Cisco uses the same Bloadcom Trident 2 chipset in Nexus switches, which is used in Bare-Metall switches in Facebook, Amazon, Apple iCloud with Open Network Linux. And, in principle, this is probably the dream of any developer - try load balancing on the switch, not on the servers, and do it with ASICs. Here you are! At the same time, Cisco began selling its switches without software. Do you think they have become cheaper from this? Of course not.

Hitachi, Ericsson, Nokia and others continue to buy ready-made OCP-components, but believe in this initiative, as this increases competition.

But in the end, why did Facebook start this business? Price! Remove the marketing component, remove the brand component from the cost of the final hardware. Because Intel is the same everywhere, Broadcom chipsets are the same everywhere, Mellanox chipsets are the same everywhere, five companies in the world make memory, they produce memory chips.

There are big manufacturers, now, the main manufacturer for Facebook - QCT - the whole range produces compatible equipment and, in principle, I can’t say that for all occasions, but the covering part, including these tricky MAIL.RU'shnyh puzzles, I think that not only they have such puzzles there. In general, the idea is to find a good supplier, if you have not yet matured for your own decisions.



We are ready to discuss if someone is interested in this whole OCP story, we are ready to discuss it in detail, tell what benefits this can give your specific project, how to deal with infrastructure costs and how to bargain with your Cloud Service provider about the cost even the infrastructure rented from him. Come, we can discuss it separately.

That's all.



By the way, sorry, these machines with Xeon-D processors, called the Yosemite project, they arrived to us not so long ago. Facebook had an embargo on selling them to the side, and this has been overcome. You can come, touch the stand, see - a rather funny interesting thing for the frontend.

Thanks!

QUESTIONS AND ANSWERS

Question: You mainly talked about the confrontation between Microsoft and Facebook, and in the title of the Facebook report, yes, but there are Apple and Google. Somehow about them could? Already Apple, it would seem, he is proprietary-proprietary. And what role does Google have?

Answer: Actually, in March of this year (2016) a terrible thing happened. Google was a supporter of non-participation in all these wonderful stories, but in March they joined the OCP project, they are already there, documents are signed. Now a battle is going on, they are struggling with Facebook’s nutrition engineers. That is, they are satisfied and constructive, happy with the whole approach.

If you delve a little bit into the technique, literally for two minutes. The whole history of OCP suggests that the blade baskets where the infrastructure is inserted (not only servers, but also storage, and network components), the blade basket is a wreck. Those. wreck is a blade basket and centralized food is on wreck. Well, in the Facebook version it’s 12 volts, and Google’s own projects used 48 volts, explaining that a smaller cross-section of copper conductors is necessary in order to transmit the same current. And now in the new consortium, I participate there, I see correspondence - in the new standard of the rack they add the ability to use 48 volts as power.

They also act as the initiator to add a special bus to the rack so that it is possible to manage the components among themselves. Because now in the concept, each component is a separate island.

Google is there, it’s not gone anywhere. From this we indirectly conclude that Microsoft's story is probably “yes”, but slippers are put in another basket.

Question: Well, and Apple?

Answer: The guys from Apple are one of the serious participants in this consortium and the last, largest data center that they have built for Apple Music, it’s completely OCP’s, it’s Facebook, it’s completely OCP’s. They are engaged in management, participate in the management part, and they are more likely to be a consumer rather than a contributor.

They are the most serious supporter of dragging ARM processors into the fight, since their story with the Power PC, which they had before, with migration, respectively, to Intel and Intel’s behavior that they now have. We understand that everyone is conquering a market with a cheap price, and then we must restore the margin. They use this argument and get it on the table in order to get good financial conditions from the guys. Because really Intel has no competition.

There is OpenPOWER, which is moving slowly and, in principle, Rack Space is one of the largest contributors, the only one who uses OpenPOWER under KVMs. They invest a lot in it, but IBM is cunning. The OpenPOWER that is available now, it is only with DDR3-memory, it is significantly worse than DDR4, we know. There are performance issues, but Rack Space believes in them.

Question: But this long-term philosophy, will it not cut the branch on which Apple sits? Because we are talking about commoditization, and Apple is selling the same hardware twice as much for some brand uniqueness.

Answer: Now we are not talking about Open Source Hardware on the desktop, not about Open Source Hardware in our pocket, but we are talking about Open Source Hardware inside the server farm.

Question:I am talking about long-term philosophy. If the market unfolds somewhere, then all this will begin to take and questions about the MacBooks, too, probably ask.

Answer: This is a different market - MacBooks. If we talk about server management, of course. There is another piece of ideological history, one of Intel vice presidents said an important thing at this private event. They predict that in 2017 60% of server processors will be sold to Cloud Service providers.

Can you imagine what it is? This means, and now only 30% is being sold, this means that according to their forecasts, and they, in principle, are smart people, according to their forecasts, the market for server processors will fall sharply for everyone else except Cloud Service providers. This is the answer, there the war will be crazy just for this market.

This report is a transcript of one of the best speeches at a professional conference of developers of highly loaded systems Highload ++ , and specifically - the section "Hardware, infrastructure".

The company Delta Soultions , which is headed by Andrey, is a partner of the festival " Russian Internet Technology " and will bring new pieces of iron for the exhibition that can be felt and studied.