Measurements in OpenStack using Ceilometer
Author: Ruslan Kiyanchuk
The goal of the OpenStack project is to create an open source cloud computing platform that will meet the needs of public and private clouds due to its simple deployment and scalability. Since OpenStack provides infrastructure as a service (IaaS) to end customers, it is important to be able to measure its performance and use for billing, performance measurement, scalability, and statistics. Several projects are available
to measure the OpenStack infrastructure :
• Zabbix is an enterprise-wide distributed open source solution for monitoring network and applications that can be configured for use with OpenStack.
• Synaps is an AWS CloudWatch-compatible cloud monitoring system designed to collect metrics, provide statistics, track and notify about alarm events in the system.
• HP Healthmon is committed to providing “Cloud Resource Monitor,” an extensible monitoring service for the OpenStack infrastructure.
• StackTach is a debugging and monitoring utility in OpenStack that can work with several data centers, including multi-cell deployments.
• Ceilometer project- This is an infrastructure for collecting metrics in the OpenStack cloud, created to avoid the development of numerous solutions with the same functions. The main goal of the project is monitoring and measurement, but the framework’s capabilities can be expanded for other needs.
Among these projects, the most promising and actively developing is the Ceilometer , which is well suited for measuring OpenStack infrastructure. The project emerged from the incubation stage and became part of OpenStack. In meteorology, a ceilometer is a device that uses a laser or other light source to determine the height of the lower boundary of clouds. Thus, the Ceilometer project is the basis for monitoring and measuring the OpenStack cloud. Also, the project can be expanded for other needs.
The main targets of the Ceilometer project are the following [1] :
• Efficient collection of metrics with optimal consumption of processor and network resources.
• Data collection by tracking notifications from services or by surveying infrastructure.
• Customize the type of data collected to meet different operational requirements.
• Access and add measurement data using the REST API.
• Expanding the infrastructure for collecting custom metrics with additional plugins.
• Receive signed and irrefutable messages with measurement results.
To meet these requirements , the following architecture is implemented in the Grizzly release :
The API server provides access to metrics in the database through the REST API.
The central agent polls resource utilization data that is not associated with virtual machines or compute nodes. Only one instance of a central agent can be run in an infrastructure.
A compute agent polls measurement data and statistics from compute nodes (mainly the hypervisor). Computing agents must be running on each compute node that needs to be monitored.
The collector tracks message queues (for notifications sent by the infrastructure, and for the results of measurements from agents). Notifications are processed, converted into measurement data, then signed and returned to the message bus in the relevant topic. The collector can run on one or more management servers.
A data store is a database that can handle simultaneous writing (from one or more collectors) and reading data (from an API server). The collector, central agent and API can work on any node.
These services communicate using the standard OpenStack messaging bus. Only the collector and API server have access to the data store. Supported SQL databases compatible with SQLAlchemy, as well as MongoDB and HBase; however, Ceilometer developers recommend MongoDB for efficiently handling concurrent read / write operations. In addition, only Ceilometer configuration with MongoDB has been rigorously tested and deployed in commercial environments. It is recommended to use a dedicated node for the Ceilometer database, since the infrastructure can create a large number of records in the database [2]. According to developers, the measurement of infrastructure at the commercial level involves up to 386 records per second and 33,360,480 events per day, which will require up to 239 GB for storing statistics for the month.
With the development of the OpenStack project to a commercial level, there is a need for new functionality for its successful use as a cloud provider: billing system, usage statistics, automatic scaling, benchmarking, tools for diagnosing and fixing errors. After starting several projects to satisfy the data requirements, it became obvious that most of their implementations have the common functionality of monitoring and measuring infrastructure.
To avoid fragmentation and duplication of functionality, it is planned to integrate related projects in order to provide other services with a single monitoring and measurement interface.
The Healthmon project was decided to integrate into the Ceilometer for the same purpose (infrastructure measurement), although the data model and measurement mechanisms were different. [4] . For the OpenStack Havana release, the Healthmon and Ceilometer integration project was created and approved . Also, Synaps and StackTach projectshave unique functionality that integrates into the Ceilometer as additional features. The main reason for saving the Ceilometer project and integrating other solutions into it is not so much in the large list of features, but in modularity and a clear definition of the functional task. Most other similar projects have implemented limited measurement functionality and some additional narrowly targeted functions. In turn, Ceilometer provides a full-featured measurement service and an API for accessing the obtained data, on the basis of which you can build any other functionality, whether it be billing, autoscaling or performance monitoring.
Cloud Application Management Project, OpenStack Heat, also plans to modify its server part (back-end) of processing measurements and notifications to work with the Ceilometer API, which will allow to implement autoscaling [3] . The integration process includes the development of an API for notifications and the ability to send measurement samples via REST requests to the Ceilometer, as well as Heat processing for the possibility of modular connection of measurement logic.
Integration will expand the Ceilometer interface with additional functions and plug-ins, which will lead to the following changes in the architecture [5] :
Most of the work on integration and implementation of additional functions is planned for the release of OpenStack Havana. The main implementation plan includes covering most of the measurement and monitoring functionalities, as well as providing the ability to build other services (command line interface, graphical interface, visualization, execution of the alarm function, etc.) around the Ceilometer API.
The Ceilometer project implements three types of measurements:
• Cumulative : increase over time (for example, the lifetime of a virtual machine)
• Gauge (indicator): discrete events (for example, floating IP addresses or image downloads) and changing values (such as disk I / O)
• Delta (delta): change over time (for example, network bandwidth)
Each meter collects data from one or more samples (collected from the message queue or agents), which are represented by counters. Each counter has the following fields:
- counter_name
Counter identifier string. By agreement, the delimiter '.' used to move from the least specific word to the most specific (e.g. disk.ephemeral.size).
- counter_type
One of the types of counters described above (cumulative, indicator, delta).
- counter_volume The
number of measured data (CPU cycles, number of bytes transferred over the network, virtual machine deployment time, etc.). This field is not relevant for indicator type counters; in this case, it should be assigned the default value (usually: 1).
- counter_unit
Description of the unit of measure for the counter. SI units of measurement and their approved abbreviations are used for designation. The amount of information should be expressed in bits ('b') or bytes ('B'). When a measurement is not a quantity of data, the description should always contain accurate information about what is being measured (virtual machines, images, floating IP addresses, etc.).
- resource_id
Identifier of the measured resource (virtual machine UUID, network, image, etc.).
- project_id
Identifier of the project to which the resource belongs.
- user_id
Identifier of the user who owns the resource.
- resource_metadata
Some additional metadata from the content of the measurement message.
A complete list of measurements currently available can be found in the OpenStack Ceilometer documentation [6] .
Due to the active development of Ceilometer and its integration with other projects, many additional features are planned for the release of OpenStack Havana. Ceilometer functionality (implemented or planned for the next release) is described below [7] .
The implementation of this function allows sending measurement data using the Ceilometer REST API v2 interface. The list of sent counters must be defined in JSON format and sent as a POST request to the url http: //: 8777 / v2 / meters /(the name of the counter corresponds to the identifier of the meter). For example:
[
{
"counter_name": "instance",
"counter_type": "gauge",
"counter_unit": "instance",
"counter_volume": "1",
"resource_id": "bd9431c1-8d69-4ad3-803a-8d4a6b89fd36 ",
" Project_id ":" 35b17138-b364-4e6a-a131-8f3099c5be68 ",
" user_id ":" efd87807-12d2-4b38-9c70-5f5c2ac427ff ",
" resource_metadata ": {
" name1 ":" value1 ",
" name2 " : “Value2”
}
}
]
This allows third-party programs to easily send measurement data to the Ceilometer.
Notifications allow you to monitor the status of the counter and report after it reaches a threshold value. This functionality will allow you to build many features based on Ceilometer, such as auto-scaling, diagnostics and correction of errors and many other actions in the infrastructure. An appropriate implementation plan for the notification interface has been approved with high priority and is scheduled for release by Havana.
The Ceilometer API will be supplemented to provide additional functionality that is necessary for billing engines, for example:
• Maximum resource utilization lasting more than 1 hour;
• Statistics of resource use over time;
• Providing additional statistics (standard deviation, median, variance, distribution, etc.).
An appropriate plan has been approved and planned for the release of Havana-2.
The Ceilometer project will be complemented by calculating network bandwidth using Quantum. Quantum Bandwidth Measurement Plan Approved Medium Priority for Havana-3 Release.
Ceilometer will monitor physical devices in the OpenStack infrastructure, including physical servers running Glance, Cinder, Quantum, Swift, Nova compute nodes and controllers, as well as network devices used in the OpenStack environment (switches, firewalls). The physical device monitoring plan has been approved for Havana-2 release and its implementation is already at the stage of code verification.
The Ceilometer project involves simple expansion and customization with an individual fit for each installation. The system of plug-ins based on the entry points to the configuration tools provides the ability to add new monitors to the collector or subagents for polling. The plug-in system based on setuptools entry points provides the ability to add new monitors to the collector or polling agents.
Two types of plug-ins are provided: questionnaires (pollesters) and listeners (listeners). Listeners process notifications generated and queued by the OpenStack components to create corresponding counter objects. Questionnaires are used to selectively poll the infrastructure by metrics for which notifications are not placed in the message queue by OpenStack components. All plugins are configured in the setup.cfg file in the [entry_points] section. For example, to enable custom plugins located in the ceilometer / plugins directory and defined as the MyCustomListener and MyCustomPollster classes, you need to configure the setup.cfg file as follows:
[entry_points]
ceilometer.collector =
custom_listener = ceilometer.plugins: MyCustomListener
...
ceilometer.poll.central =
custom_pollster = ceilometer.plugins: MyCustomPollster
...
The purpose of questionnaire plugins is to get the necessary measurement results from the cloud infrastructure and create counters from them. Plugins for the central agent are defined in the ceilometer.poll.central namespace of the setup.cfg entry points, and for computing agents, in the ceilometer.poll.compute namespace. Listener plugins are loaded from the ceilometer.collector section.
At the heart of the system is a collector that monitors the message bus for data provided by questionnaires, as well as notifications from other OpenStack components, such as Nova, Glance, Quantum and Swift.
The class of a typical listener plugin should have several methods for receiving certain notifications from the message queue and creating counters from them. The get_event_types () method should return a list of strings containing event types in which the plugin is interested. These events are sent to the plugin each time a notification is received. The notification_to_metadata () method is responsible for handling notifications and creating metadata that will be included in the measurement messages, which will be accessible through the Ceilometer API. The process_notification () method determines the logic of creating a counter based on data from received notifications. This method may also return an empty list if no suitable measurement data were found in the current notification. Counters are created by the constructor ceilometer.counter.Counter (), which takes the values of the required fields of the counter (see Measurements in the Ceilometer). The meters provided by Ceilometer are also implemented as plugins by default and can be used as supporting information for creating additional plugins.
Ceilometer is a promising project designed to provide extensive capabilities for measuring and monitoring the cloud infrastructure, implementing the functionality that is needed for commercial use of OpenStack. Although there are already cases of commercial use of the Ceilometer (CloudWatch, AT&T, Dreamhost [5] ), by October 2013 a lot of changes and additional functions will be added to the project. Thus, the Ceilometer project should become much more adapted for commercial use with the release of Havana, which is planned to implement major changes and new functionality.
Original article in English
The goal of the OpenStack project is to create an open source cloud computing platform that will meet the needs of public and private clouds due to its simple deployment and scalability. Since OpenStack provides infrastructure as a service (IaaS) to end customers, it is important to be able to measure its performance and use for billing, performance measurement, scalability, and statistics. Several projects are available
to measure the OpenStack infrastructure :
• Zabbix is an enterprise-wide distributed open source solution for monitoring network and applications that can be configured for use with OpenStack.
• Synaps is an AWS CloudWatch-compatible cloud monitoring system designed to collect metrics, provide statistics, track and notify about alarm events in the system.
• HP Healthmon is committed to providing “Cloud Resource Monitor,” an extensible monitoring service for the OpenStack infrastructure.
• StackTach is a debugging and monitoring utility in OpenStack that can work with several data centers, including multi-cell deployments.
• Ceilometer project- This is an infrastructure for collecting metrics in the OpenStack cloud, created to avoid the development of numerous solutions with the same functions. The main goal of the project is monitoring and measurement, but the framework’s capabilities can be expanded for other needs.
Among these projects, the most promising and actively developing is the Ceilometer , which is well suited for measuring OpenStack infrastructure. The project emerged from the incubation stage and became part of OpenStack. In meteorology, a ceilometer is a device that uses a laser or other light source to determine the height of the lower boundary of clouds. Thus, the Ceilometer project is the basis for monitoring and measuring the OpenStack cloud. Also, the project can be expanded for other needs.
OpenStack Ceilometer Project Architecture
The main targets of the Ceilometer project are the following [1] :
• Efficient collection of metrics with optimal consumption of processor and network resources.
• Data collection by tracking notifications from services or by surveying infrastructure.
• Customize the type of data collected to meet different operational requirements.
• Access and add measurement data using the REST API.
• Expanding the infrastructure for collecting custom metrics with additional plugins.
• Receive signed and irrefutable messages with measurement results.
To meet these requirements , the following architecture is implemented in the Grizzly release :
The API server provides access to metrics in the database through the REST API.
The central agent polls resource utilization data that is not associated with virtual machines or compute nodes. Only one instance of a central agent can be run in an infrastructure.
A compute agent polls measurement data and statistics from compute nodes (mainly the hypervisor). Computing agents must be running on each compute node that needs to be monitored.
The collector tracks message queues (for notifications sent by the infrastructure, and for the results of measurements from agents). Notifications are processed, converted into measurement data, then signed and returned to the message bus in the relevant topic. The collector can run on one or more management servers.
A data store is a database that can handle simultaneous writing (from one or more collectors) and reading data (from an API server). The collector, central agent and API can work on any node.
These services communicate using the standard OpenStack messaging bus. Only the collector and API server have access to the data store. Supported SQL databases compatible with SQLAlchemy, as well as MongoDB and HBase; however, Ceilometer developers recommend MongoDB for efficiently handling concurrent read / write operations. In addition, only Ceilometer configuration with MongoDB has been rigorously tested and deployed in commercial environments. It is recommended to use a dedicated node for the Ceilometer database, since the infrastructure can create a large number of records in the database [2]. According to developers, the measurement of infrastructure at the commercial level involves up to 386 records per second and 33,360,480 events per day, which will require up to 239 GB for storing statistics for the month.
Integration of related projects in Ceilometer
With the development of the OpenStack project to a commercial level, there is a need for new functionality for its successful use as a cloud provider: billing system, usage statistics, automatic scaling, benchmarking, tools for diagnosing and fixing errors. After starting several projects to satisfy the data requirements, it became obvious that most of their implementations have the common functionality of monitoring and measuring infrastructure.
To avoid fragmentation and duplication of functionality, it is planned to integrate related projects in order to provide other services with a single monitoring and measurement interface.
The Healthmon project was decided to integrate into the Ceilometer for the same purpose (infrastructure measurement), although the data model and measurement mechanisms were different. [4] . For the OpenStack Havana release, the Healthmon and Ceilometer integration project was created and approved . Also, Synaps and StackTach projectshave unique functionality that integrates into the Ceilometer as additional features. The main reason for saving the Ceilometer project and integrating other solutions into it is not so much in the large list of features, but in modularity and a clear definition of the functional task. Most other similar projects have implemented limited measurement functionality and some additional narrowly targeted functions. In turn, Ceilometer provides a full-featured measurement service and an API for accessing the obtained data, on the basis of which you can build any other functionality, whether it be billing, autoscaling or performance monitoring.
Cloud Application Management Project, OpenStack Heat, also plans to modify its server part (back-end) of processing measurements and notifications to work with the Ceilometer API, which will allow to implement autoscaling [3] . The integration process includes the development of an API for notifications and the ability to send measurement samples via REST requests to the Ceilometer, as well as Heat processing for the possibility of modular connection of measurement logic.
Integration will expand the Ceilometer interface with additional functions and plug-ins, which will lead to the following changes in the architecture [5] :
Most of the work on integration and implementation of additional functions is planned for the release of OpenStack Havana. The main implementation plan includes covering most of the measurement and monitoring functionalities, as well as providing the ability to build other services (command line interface, graphical interface, visualization, execution of the alarm function, etc.) around the Ceilometer API.
Measurements in Ceilometer
The Ceilometer project implements three types of measurements:
• Cumulative : increase over time (for example, the lifetime of a virtual machine)
• Gauge (indicator): discrete events (for example, floating IP addresses or image downloads) and changing values (such as disk I / O)
• Delta (delta): change over time (for example, network bandwidth)
Each meter collects data from one or more samples (collected from the message queue or agents), which are represented by counters. Each counter has the following fields:
- counter_name
Counter identifier string. By agreement, the delimiter '.' used to move from the least specific word to the most specific (e.g. disk.ephemeral.size).
- counter_type
One of the types of counters described above (cumulative, indicator, delta).
- counter_volume The
number of measured data (CPU cycles, number of bytes transferred over the network, virtual machine deployment time, etc.). This field is not relevant for indicator type counters; in this case, it should be assigned the default value (usually: 1).
- counter_unit
Description of the unit of measure for the counter. SI units of measurement and their approved abbreviations are used for designation. The amount of information should be expressed in bits ('b') or bytes ('B'). When a measurement is not a quantity of data, the description should always contain accurate information about what is being measured (virtual machines, images, floating IP addresses, etc.).
- resource_id
Identifier of the measured resource (virtual machine UUID, network, image, etc.).
- project_id
Identifier of the project to which the resource belongs.
- user_id
Identifier of the user who owns the resource.
- resource_metadata
Some additional metadata from the content of the measurement message.
A complete list of measurements currently available can be found in the OpenStack Ceilometer documentation [6] .
Ceilometer Functionality
Due to the active development of Ceilometer and its integration with other projects, many additional features are planned for the release of OpenStack Havana. Ceilometer functionality (implemented or planned for the next release) is described below [7] .
Sending measurement samples using the REST API
The implementation of this function allows sending measurement data using the Ceilometer REST API v2 interface. The list of sent counters must be defined in JSON format and sent as a POST request to the url http: //
[
{
"counter_name": "instance",
"counter_type": "gauge",
"counter_unit": "instance",
"counter_volume": "1",
"resource_id": "bd9431c1-8d69-4ad3-803a-8d4a6b89fd36 ",
" Project_id ":" 35b17138-b364-4e6a-a131-8f3099c5be68 ",
" user_id ":" efd87807-12d2-4b38-9c70-5f5c2ac427ff ",
" resource_metadata ": {
" name1 ":" value1 ",
" name2 " : “Value2”
}
}
]
This allows third-party programs to easily send measurement data to the Ceilometer.
Notification Interface in Ceilometer
Notifications allow you to monitor the status of the counter and report after it reaches a threshold value. This functionality will allow you to build many features based on Ceilometer, such as auto-scaling, diagnostics and correction of errors and many other actions in the infrastructure. An appropriate implementation plan for the notification interface has been approved with high priority and is scheduled for release by Havana.
Ceilometer Interface Extension
The Ceilometer API will be supplemented to provide additional functionality that is necessary for billing engines, for example:
• Maximum resource utilization lasting more than 1 hour;
• Statistics of resource use over time;
• Providing additional statistics (standard deviation, median, variance, distribution, etc.).
An appropriate plan has been approved and planned for the release of Havana-2.
Quantum Bandwidth Measurement
The Ceilometer project will be complemented by calculating network bandwidth using Quantum. Quantum Bandwidth Measurement Plan Approved Medium Priority for Havana-3 Release.
Physical device monitoring
Ceilometer will monitor physical devices in the OpenStack infrastructure, including physical servers running Glance, Cinder, Quantum, Swift, Nova compute nodes and controllers, as well as network devices used in the OpenStack environment (switches, firewalls). The physical device monitoring plan has been approved for Havana-2 release and its implementation is already at the stage of code verification.
Extending Ceilometer Features
The Ceilometer project involves simple expansion and customization with an individual fit for each installation. The system of plug-ins based on the entry points to the configuration tools provides the ability to add new monitors to the collector or subagents for polling. The plug-in system based on setuptools entry points provides the ability to add new monitors to the collector or polling agents.
Two types of plug-ins are provided: questionnaires (pollesters) and listeners (listeners). Listeners process notifications generated and queued by the OpenStack components to create corresponding counter objects. Questionnaires are used to selectively poll the infrastructure by metrics for which notifications are not placed in the message queue by OpenStack components. All plugins are configured in the setup.cfg file in the [entry_points] section. For example, to enable custom plugins located in the ceilometer / plugins directory and defined as the MyCustomListener and MyCustomPollster classes, you need to configure the setup.cfg file as follows:
[entry_points]
ceilometer.collector =
custom_listener = ceilometer.plugins: MyCustomListener
...
ceilometer.poll.central =
custom_pollster = ceilometer.plugins: MyCustomPollster
...
The purpose of questionnaire plugins is to get the necessary measurement results from the cloud infrastructure and create counters from them. Plugins for the central agent are defined in the ceilometer.poll.central namespace of the setup.cfg entry points, and for computing agents, in the ceilometer.poll.compute namespace. Listener plugins are loaded from the ceilometer.collector section.
At the heart of the system is a collector that monitors the message bus for data provided by questionnaires, as well as notifications from other OpenStack components, such as Nova, Glance, Quantum and Swift.
The class of a typical listener plugin should have several methods for receiving certain notifications from the message queue and creating counters from them. The get_event_types () method should return a list of strings containing event types in which the plugin is interested. These events are sent to the plugin each time a notification is received. The notification_to_metadata () method is responsible for handling notifications and creating metadata that will be included in the measurement messages, which will be accessible through the Ceilometer API. The process_notification () method determines the logic of creating a counter based on data from received notifications. This method may also return an empty list if no suitable measurement data were found in the current notification. Counters are created by the constructor ceilometer.counter.Counter (), which takes the values of the required fields of the counter (see Measurements in the Ceilometer). The meters provided by Ceilometer are also implemented as plugins by default and can be used as supporting information for creating additional plugins.
Conclusion
Ceilometer is a promising project designed to provide extensive capabilities for measuring and monitoring the cloud infrastructure, implementing the functionality that is needed for commercial use of OpenStack. Although there are already cases of commercial use of the Ceilometer (CloudWatch, AT&T, Dreamhost [5] ), by October 2013 a lot of changes and additional functions will be added to the project. Thus, the Ceilometer project should become much more adapted for commercial use with the release of Havana, which is planned to implement major changes and new functionality.
Original article in English