How to calculate how much hardware is needed for your OpenStack-based cloud?
Author: Kirill Ishanov
Often, customers who ask us to deploy a cloud based on OpenStack also attract us to performance planning, i.e. hardware configurations for OpenStack on which the cloud will run. Recently, we posted a video on the Internet in which this topic is described in more detail.
The simplest scenario assumes that the client has several physical servers that it is planned to use specifically for OpenStack, but the client does not actually consider whether the server configuration is optimal for the cloud and the intended usage model. Therefore, as a rule, we try to analyze the specific use case more deeply and give recommendations regarding the hardware that the client will need to succeed.
Having done this kind of work over and over the past 3 years, we noticed that this can be a daunting task for a number of reasons:
-Different usage scenarios require a different approach. If you are designing a data center for hosting functioning web applications or a Dev & QA cloud or a cloud for supporting internal information technologies, your choice will be different. In most cases, it’s not very practical to reduce the usage scenario to “I'll just take all of the above.”
-There is a desire to optimize the list of equipment in terms of either cost or performance; in fact, optimization is required somewhere in between.
- There are, without exaggeration, a large number of variables (processor, RAM, network, data storage), and you can easily invest excessive funds in one element and insufficient in another.
-Websites of various equipment suppliers do not completely solve the problem. Yes, there are servers that are “well suited for virtualization” or “optimal for solving critical tasks,” but such characteristics are usually difficult to quantify or compare.
-There are good solutions on the market that have been tested in practice and approved for VMWare-based clouds, but there is practically no information on OpenStack-based clouds.
During the time during which we were engaged in such planning of hardware, we have accumulated sufficient experience and empirically derived several formulas in order to “fairly well” evaluate which hardware can be used for a specific workload.
Today, it is not enough to formulate these heuristic algorithms: a simple overview of various server configurations from various vendors shows that there are a huge number of options. For example, on a single Dell PowerEdge R720 server, more than 7,000 feature combinations are possible. This can easily lead you to the paradox of choice .
Since the final goal is to make some practical decisions regarding your cloud, we decided to try to limit the complexity of obtaining all possible server configurations. Applying the rules and formulas obtained using the “magic” of automation, we created a simplified online calculator of hardware specifications to achieve a “reasonably good” result based on the input data you provided for several key parameters. Taking into account the variety of ways of dividing into parts of the cloud based on OpenStack, today we are implementing a pilot version and finalizing it based on feedback from users, customers and vendors.
In the first version, the Hardware Specification Calculator for OpenStack stores user input in two parameters: (1) the total number of virtual machines that you plan to use for the entire cloud, and (2) the average size of each of these virtual machines. Then we apply a few simplifying restrictions with the following assumptions:
-You do not plan to exceed the original amount of memory.
-If you plan to "optimize performance", you do not intend to use more than two VCPU processors on one physical core.
-If you plan to "optimize costs", the number of VCPU processors will not exceed six per physical core.
- For each server model, the cheapest configuration is offered using no more than 60% RAM (this will ensure the correct functioning of more virtual machines and host OS).
The latest version of the hardware specification calculator can be found here . The video just demonstrates how the calculator works:
We learned a lot by collecting all of this data and obtaining some useful results from it. And, as mentioned above, the first release did not take into account several important points. In the coming weeks, we plan to expand the available features and include in them:
-Data on HP and Cisco servers plus the missing models (for example, DELL C-series servers) for existing vendors.
- Matters related to network architecture (network adapters, switches).
- Data storage options (for both short-term and block data storage).
- Hardware specification calculator for Swift clusters.
-Optimization for various use cases and load levels.
In one of the following posts, we will also provide more detailed information about the formulas we use, as well as open the veil of how filtering and evaluation of possible server configurations occurs.
In addition, we are clearly aware that the creation of this kind of calculator requires expert knowledge in various fields, so if you find deviations in the results obtained using the calculator or you have additional ideas that you can share, please contact us at Email bom-calculator [at] mirantis.com.
Try the calculator, provide us feedback, and stay tuned. And, if you haven't done it yet, be sure to watch a video on the Internet about hardware planning methods for OpenStack as a whole.
Original article in English
Often, customers who ask us to deploy a cloud based on OpenStack also attract us to performance planning, i.e. hardware configurations for OpenStack on which the cloud will run. Recently, we posted a video on the Internet in which this topic is described in more detail.
The simplest scenario assumes that the client has several physical servers that it is planned to use specifically for OpenStack, but the client does not actually consider whether the server configuration is optimal for the cloud and the intended usage model. Therefore, as a rule, we try to analyze the specific use case more deeply and give recommendations regarding the hardware that the client will need to succeed.
Having done this kind of work over and over the past 3 years, we noticed that this can be a daunting task for a number of reasons:
-Different usage scenarios require a different approach. If you are designing a data center for hosting functioning web applications or a Dev & QA cloud or a cloud for supporting internal information technologies, your choice will be different. In most cases, it’s not very practical to reduce the usage scenario to “I'll just take all of the above.”
-There is a desire to optimize the list of equipment in terms of either cost or performance; in fact, optimization is required somewhere in between.
- There are, without exaggeration, a large number of variables (processor, RAM, network, data storage), and you can easily invest excessive funds in one element and insufficient in another.
-Websites of various equipment suppliers do not completely solve the problem. Yes, there are servers that are “well suited for virtualization” or “optimal for solving critical tasks,” but such characteristics are usually difficult to quantify or compare.
-There are good solutions on the market that have been tested in practice and approved for VMWare-based clouds, but there is practically no information on OpenStack-based clouds.
A pragmatic compromise?
During the time during which we were engaged in such planning of hardware, we have accumulated sufficient experience and empirically derived several formulas in order to “fairly well” evaluate which hardware can be used for a specific workload.
Today, it is not enough to formulate these heuristic algorithms: a simple overview of various server configurations from various vendors shows that there are a huge number of options. For example, on a single Dell PowerEdge R720 server, more than 7,000 feature combinations are possible. This can easily lead you to the paradox of choice .
Since the final goal is to make some practical decisions regarding your cloud, we decided to try to limit the complexity of obtaining all possible server configurations. Applying the rules and formulas obtained using the “magic” of automation, we created a simplified online calculator of hardware specifications to achieve a “reasonably good” result based on the input data you provided for several key parameters. Taking into account the variety of ways of dividing into parts of the cloud based on OpenStack, today we are implementing a pilot version and finalizing it based on feedback from users, customers and vendors.
How the hardware spec calculator for OpenStack deployment works
In the first version, the Hardware Specification Calculator for OpenStack stores user input in two parameters: (1) the total number of virtual machines that you plan to use for the entire cloud, and (2) the average size of each of these virtual machines. Then we apply a few simplifying restrictions with the following assumptions:
-You do not plan to exceed the original amount of memory.
-If you plan to "optimize performance", you do not intend to use more than two VCPU processors on one physical core.
-If you plan to "optimize costs", the number of VCPU processors will not exceed six per physical core.
- For each server model, the cheapest configuration is offered using no more than 60% RAM (this will ensure the correct functioning of more virtual machines and host OS).
The latest version of the hardware specification calculator can be found here . The video just demonstrates how the calculator works:
We learned a lot by collecting all of this data and obtaining some useful results from it. And, as mentioned above, the first release did not take into account several important points. In the coming weeks, we plan to expand the available features and include in them:
-Data on HP and Cisco servers plus the missing models (for example, DELL C-series servers) for existing vendors.
- Matters related to network architecture (network adapters, switches).
- Data storage options (for both short-term and block data storage).
- Hardware specification calculator for Swift clusters.
-Optimization for various use cases and load levels.
In one of the following posts, we will also provide more detailed information about the formulas we use, as well as open the veil of how filtering and evaluation of possible server configurations occurs.
In addition, we are clearly aware that the creation of this kind of calculator requires expert knowledge in various fields, so if you find deviations in the results obtained using the calculator or you have additional ideas that you can share, please contact us at Email bom-calculator [at] mirantis.com.
Try the calculator, provide us feedback, and stay tuned. And, if you haven't done it yet, be sure to watch a video on the Internet about hardware planning methods for OpenStack as a whole.
Original article in English