Interview with Julien Danjou, OpenStack Ceilometer Project Manager
- Transfer
We present the fifth of a series of interviews with technical managers of the OpenStack project on the Mirantis blog. Our goal is to educate the wider community of technical experts and help people understand how they can contribute to and benefit from the OpenStack project. Naturally, the following is the viewpoint of the interviewee, not Mirantis.
Below we present an interview with Julien Danjou, Technical Project Manager, OpenStack Ceilometer.
Mirantis: Tell us about yourself.
Julien Danju: I am a freelancer, freelance developer, involved in many FOSS projects, and also a Debian developer for more than ten years.
Question: What is your history of relationships with OpenStack? Why are you participating in the project?
Answer: I started working on the OpenStack project with eNovance, which hired me at the end of 2011 to create the first European cloud platform based on OpenStack. At that moment, I had no idea what OpenStack was, but working on the open source IaaS platform in Python was very exciting, so I joined. Since that time, the whole OpenStack ecosystem has been created and new technical challenges arise; enough reasons to take part!
Question: What is the difference between the OpenStack project and other open source projects in which you participated?
Answer: I would name pragmatism and a good selection of technologies among the differences, which are probably the main signs of a successful open source project.
In addition, many of the open source projects in which I have participated or are participating now have a sufficient number of users and developers who work on the project in their free time as a hobby. In OpenStack, almost every user or developer participates in a project on behalf of their company. This makes the project much more dynamic, it grows faster than the rest, and gives us the opportunity to achieve ambitious goals in almost an instant.
Question: What is your responsibility as a technical project manager for Ceilometer?
A: I observe the project and guarantee the movement in a certain direction, which we took during the last gathering of developers in Portland. I also act as a kind of gluing between the project participants, since in any case I control every area of the project. The technical project manager is almost full-time, taking into account the planned volume of projects under this release.
Question: Can you explain the role of Ceilometer in the framework of the OpenStack project? Why is the Ceilometer so important?
Answer: Most people who build their IaaS platform want to bill for the use of platform resources. Based on this first use case, we defined the role of the Ceilometer in OpenStack as the place to measure the OpenStack platform. Then, the scope extended to a wider collection of meters that implement many different options, from billing to triggering alarm messages.
The goal of the Ceilometer project is to calculate everything that happens on the OpenStack platform in order to invoice for it or otherwise conduct an analysis.
Question: What is unique in the project, what is the breakthrough of the Ceilometer project?
Answer: The ability to have a single point of collection and interrogation of measurement results in the cloud, for operators and users, with the ability to generate events and actions based on these measurements. This is what will open up the possibility of creating various applications never seen before.
We are not limited to OpenStack either. Almost every Ceilometer area is expandable with a plug-in, so you can build your own measurement system, publish measurements to external systems, or measure your PaaS platform directly with Ceilometer.
Question: What has the Ceilometer community achieved so far?
Answer: We built the project from scratch, embedding it in each component of OpenStack without explicit intervention. After the results of our work were accumulated, we entered the incubation process and then left it as an integrated project. And all this in just one year. After only 6 months of development, OpenStack operators used the Ceilometer to measure their platforms and bill or raise funds for their use, while generating gigabytes of measurement data per day.
Question: What features will Ceilometer provide in the release of OpenStack Havana?
Answer: I think we are one of the projects with the largest planned implementation volume. At Havana, our primary goal is alarm reporting functionality, which allows users and operators to trigger events based on evaluation of measurement results. In addition, this will be the cornerstone for the Heat project, which will provide auto-scaling based on this Ceilometer functionality and related data.
We also plan and implement new measurement capabilities, such as measuring network bandwidth in Quantum and scheduling events in Nova, as well as improving our public API to allow finer queries and more detailed slices of statistics.
Question: Which providers are the most invested and provide plugins? Who would you like to see as Ceilometer developers?
Answer: Now we do not have external developers who distribute plugins for Ceilometer. What's good is that all of our plugins are open source software distributed with Ceilometer. All of them are part of the base code created by the developers of Ceilometer. Nevertheless, we will be glad to see the contribution to systems on other platforms, not only on OpenStack, for example Ceph.
Question: Does the Ceilometer project still have “childhood illnesses” - something that needs to be changed?
Answer: No, and in fact, this was one of the criteria for getting out of the incubation process. You cannot exit the project with the OpenStack platform if you know that you have to redo the project from scratch. So, we have diseases, but these are adult mistakes.
Question: What are typical usage recommendations?
Answer: The recommended method for Ceilometer is to deploy a project with fine-tuning that would meet your needs in terms of audit, billing and statistics. Ceilometer generates a huge amount of measurement results, so you need to make sure that you generate the amount of data that matches your needs.
Question: What do you want people to know about the project (hidden features / functionality)?
Answer: I think that the data collected by the Ceilometer tool has great potential. People usually see the data as useful for billing, but the value in it is much more - from capacity planning to determining usage trends for your cloud platform. You can make deployment more convenient by directing this data to the right tools.
Question: Are there any general misconceptions regarding the Ceilometer?
Answer: We had some misconceptions at the beginning of the project, when people expected Ceilometer to print PDF bills or monitor their platform and call the system administrator in case of failure. Now it’s clear enough that we are a storehouse of OpenStack measurement results and you can use it to build any application on top of it, including billing or active monitoring. But we do not aim to provide this functionality.
Question: What are the necessary prerequisites?
Answer: There are no necessary hardware requirements. For software, Ceilometer uses the same technologies (Python) as the other components of the OpenStack platform. In terms of technology knowledge, you definitely need to understand the global OpenStack architecture., since you need to connect a Ceilometer to each component!
Question: What are the requirements for running Ceilometer?
Answer: A simple installation requires the work of an OpenStack engineer for a couple of days. The more you use and load, the more you'll need to improve your Ceilometer deployment. That is why some of our developers actually started working on the project.
Question: What advice would you give to people / corporations who are faced with the same tasks that you had at the beginning of your journey? .. when you have no idea about OpenStack, but you need to jump into the project ...
Answer: The opportunity to discuss OpenStack with other people or companies definitely helps. I would advise making friends with those who can help you, as they have already studied OpenStack. So much is happening in this ecosystem now that it’s sometimes difficult to keep track of events and get up-to-date information or documentation. Therefore, it is necessary to maintain communication.
Then, I suppose, it depends on whether you are used to working on open source projects. If you have personally participated in the free software development communities, you can simply join OpenStack, as its functioning and its values coincide with the values of any open source project. Therefore, the same rules apply: read, find out, listen, ask (smart) questions, and then start contributing.
If you represent a company, you definitely need to convey to people the dynamics and work of open source projects for effective work. If your developers cannot contribute to OpenStack by participating in a discussion of the features that they would like to implement or embed, you will end up as a private provider with locally developed software: you will not end up investing in OpenStack.
Question: Who would you like to see as Ceilometer developers?
Answer: The Ceilometer project has many different aspects, from data collection to the REST API, which returns data. Therefore, depending on what part someone wants to invest in, it can be a person who implements Ceilometer support in his “product” (for example, PaaS platform) or is engaged in data analysis and helps in creating the REST API.
Question: What features need to be improved or tested?
Answer: At this stage, we definitely need more data consumers. We already accumulate and store a lot of data, but we don’t have enough feedback - how people would like to consume data, what requests they would like to fulfill, therefore our API is sometimes not quite convenient.
As for the tests, there is a big gap in terms of storage drivers. The very first and best supported is MongoDB, and the SQLAlchemy and HBase storage drivers will require more testing and possibly fixes.
Question: How exactly can people get started?
Answer: I think that as with most OpenStack projects, the easiest way to get started and play with Ceilometer is to use the devstack installation. Ceilometer is fully integrated with devstack, so it’s easy to enable and deploy along with Nova, Glance, etc. After that, you can experiment with the Ceilometer tool - use or contribute!
Question: Thank you very much, Julien.
Answer: Not at all!
Below we present an interview with Julien Danjou, Technical Project Manager, OpenStack Ceilometer.
Mirantis: Tell us about yourself.
Julien Danju: I am a freelancer, freelance developer, involved in many FOSS projects, and also a Debian developer for more than ten years.
Question: What is your history of relationships with OpenStack? Why are you participating in the project?
Answer: I started working on the OpenStack project with eNovance, which hired me at the end of 2011 to create the first European cloud platform based on OpenStack. At that moment, I had no idea what OpenStack was, but working on the open source IaaS platform in Python was very exciting, so I joined. Since that time, the whole OpenStack ecosystem has been created and new technical challenges arise; enough reasons to take part!
Question: What is the difference between the OpenStack project and other open source projects in which you participated?
Answer: I would name pragmatism and a good selection of technologies among the differences, which are probably the main signs of a successful open source project.
In addition, many of the open source projects in which I have participated or are participating now have a sufficient number of users and developers who work on the project in their free time as a hobby. In OpenStack, almost every user or developer participates in a project on behalf of their company. This makes the project much more dynamic, it grows faster than the rest, and gives us the opportunity to achieve ambitious goals in almost an instant.
Question: What is your responsibility as a technical project manager for Ceilometer?
A: I observe the project and guarantee the movement in a certain direction, which we took during the last gathering of developers in Portland. I also act as a kind of gluing between the project participants, since in any case I control every area of the project. The technical project manager is almost full-time, taking into account the planned volume of projects under this release.
Question: Can you explain the role of Ceilometer in the framework of the OpenStack project? Why is the Ceilometer so important?
Answer: Most people who build their IaaS platform want to bill for the use of platform resources. Based on this first use case, we defined the role of the Ceilometer in OpenStack as the place to measure the OpenStack platform. Then, the scope extended to a wider collection of meters that implement many different options, from billing to triggering alarm messages.
The goal of the Ceilometer project is to calculate everything that happens on the OpenStack platform in order to invoice for it or otherwise conduct an analysis.
Question: What is unique in the project, what is the breakthrough of the Ceilometer project?
Answer: The ability to have a single point of collection and interrogation of measurement results in the cloud, for operators and users, with the ability to generate events and actions based on these measurements. This is what will open up the possibility of creating various applications never seen before.
We are not limited to OpenStack either. Almost every Ceilometer area is expandable with a plug-in, so you can build your own measurement system, publish measurements to external systems, or measure your PaaS platform directly with Ceilometer.
Question: What has the Ceilometer community achieved so far?
Answer: We built the project from scratch, embedding it in each component of OpenStack without explicit intervention. After the results of our work were accumulated, we entered the incubation process and then left it as an integrated project. And all this in just one year. After only 6 months of development, OpenStack operators used the Ceilometer to measure their platforms and bill or raise funds for their use, while generating gigabytes of measurement data per day.
Question: What features will Ceilometer provide in the release of OpenStack Havana?
Answer: I think we are one of the projects with the largest planned implementation volume. At Havana, our primary goal is alarm reporting functionality, which allows users and operators to trigger events based on evaluation of measurement results. In addition, this will be the cornerstone for the Heat project, which will provide auto-scaling based on this Ceilometer functionality and related data.
We also plan and implement new measurement capabilities, such as measuring network bandwidth in Quantum and scheduling events in Nova, as well as improving our public API to allow finer queries and more detailed slices of statistics.
Question: Which providers are the most invested and provide plugins? Who would you like to see as Ceilometer developers?
Answer: Now we do not have external developers who distribute plugins for Ceilometer. What's good is that all of our plugins are open source software distributed with Ceilometer. All of them are part of the base code created by the developers of Ceilometer. Nevertheless, we will be glad to see the contribution to systems on other platforms, not only on OpenStack, for example Ceph.
Question: Does the Ceilometer project still have “childhood illnesses” - something that needs to be changed?
Answer: No, and in fact, this was one of the criteria for getting out of the incubation process. You cannot exit the project with the OpenStack platform if you know that you have to redo the project from scratch. So, we have diseases, but these are adult mistakes.
Question: What are typical usage recommendations?
Answer: The recommended method for Ceilometer is to deploy a project with fine-tuning that would meet your needs in terms of audit, billing and statistics. Ceilometer generates a huge amount of measurement results, so you need to make sure that you generate the amount of data that matches your needs.
Question: What do you want people to know about the project (hidden features / functionality)?
Answer: I think that the data collected by the Ceilometer tool has great potential. People usually see the data as useful for billing, but the value in it is much more - from capacity planning to determining usage trends for your cloud platform. You can make deployment more convenient by directing this data to the right tools.
Question: Are there any general misconceptions regarding the Ceilometer?
Answer: We had some misconceptions at the beginning of the project, when people expected Ceilometer to print PDF bills or monitor their platform and call the system administrator in case of failure. Now it’s clear enough that we are a storehouse of OpenStack measurement results and you can use it to build any application on top of it, including billing or active monitoring. But we do not aim to provide this functionality.
Question: What are the necessary prerequisites?
Answer: There are no necessary hardware requirements. For software, Ceilometer uses the same technologies (Python) as the other components of the OpenStack platform. In terms of technology knowledge, you definitely need to understand the global OpenStack architecture., since you need to connect a Ceilometer to each component!
Question: What are the requirements for running Ceilometer?
Answer: A simple installation requires the work of an OpenStack engineer for a couple of days. The more you use and load, the more you'll need to improve your Ceilometer deployment. That is why some of our developers actually started working on the project.
Question: What advice would you give to people / corporations who are faced with the same tasks that you had at the beginning of your journey? .. when you have no idea about OpenStack, but you need to jump into the project ...
Answer: The opportunity to discuss OpenStack with other people or companies definitely helps. I would advise making friends with those who can help you, as they have already studied OpenStack. So much is happening in this ecosystem now that it’s sometimes difficult to keep track of events and get up-to-date information or documentation. Therefore, it is necessary to maintain communication.
Then, I suppose, it depends on whether you are used to working on open source projects. If you have personally participated in the free software development communities, you can simply join OpenStack, as its functioning and its values coincide with the values of any open source project. Therefore, the same rules apply: read, find out, listen, ask (smart) questions, and then start contributing.
If you represent a company, you definitely need to convey to people the dynamics and work of open source projects for effective work. If your developers cannot contribute to OpenStack by participating in a discussion of the features that they would like to implement or embed, you will end up as a private provider with locally developed software: you will not end up investing in OpenStack.
Question: Who would you like to see as Ceilometer developers?
Answer: The Ceilometer project has many different aspects, from data collection to the REST API, which returns data. Therefore, depending on what part someone wants to invest in, it can be a person who implements Ceilometer support in his “product” (for example, PaaS platform) or is engaged in data analysis and helps in creating the REST API.
Question: What features need to be improved or tested?
Answer: At this stage, we definitely need more data consumers. We already accumulate and store a lot of data, but we don’t have enough feedback - how people would like to consume data, what requests they would like to fulfill, therefore our API is sometimes not quite convenient.
As for the tests, there is a big gap in terms of storage drivers. The very first and best supported is MongoDB, and the SQLAlchemy and HBase storage drivers will require more testing and possibly fixes.
Question: How exactly can people get started?
Answer: I think that as with most OpenStack projects, the easiest way to get started and play with Ceilometer is to use the devstack installation. Ceilometer is fully integrated with devstack, so it’s easy to enable and deploy along with Nova, Glance, etc. After that, you can experiment with the Ceilometer tool - use or contribute!
Question: Thank you very much, Julien.
Answer: Not at all!