April 20, 2017 at 12:37What famous companies use Docker in production and for what?
In the yard in 2017, and in various articles and, especially, comments on them, one has to meet with the opinion that Docker is the lot of inexperienced enthusiasts, exotic one-day companies and any experiments that are as far removed from real production as possible. And while my colleagues are preparing new technical materials, I decided to show using publicly available examples and facts that this is more a myth and a stereotype than the world in which IT lives today.
The starting point for this article is statistics from the Datadog monitoring service , which first appeared in 2015 and was last updated just the other day, in April 2017. Datadog is not some kind of traditional analytical company, but a monitoring system known for its friendliness to cloudy environments (its agent code is written in Python and published as Open Source) .
Having at its disposal a huge database on the structure of infrastructures in companies from around the world, Datadog collected statistics on adaptation of Docker, which she described as "no hype, just facts." The collected statistics represent 10 thousand companies and 185 million containers used for any purpose.(of course, this is not necessarily production, but at least a part of the infrastructure that makes sense to monitor) . Here are the facts:
1. If at the beginning of March 2016, about 13.6% of Datadog customers used Docker, then in a year (i.e. by March 2017) this figure increased by ~ 38%, amounting to 18.8% .
2. Docker is running on 15% of all hosts represented in monitoring. For comparison: two years ago there were only 3% of such hosts, and this is a much more significant growth.
3. The leaders of the adaptation of Docker are large companies. The more hosts the company has in monitoring, the more likely Docker is. About 60% of organizations with 500+ hosts are named Docker users. The latest research update also recorded a growing interest in Docker from medium-sized organizations (from 100 to 500 hosts): the number of Docker users among them was already close to that among large companies.
4. About 40% of Docker users use container tools like Kubernetes, Mesos, Amazon ECS, and Google Container Engine. A year ago, there were half as many.
5. Docker users increase the number of running containers by about 5 times during the period from the first to the tenth month of its use.
6. Most often in Docker they launchfollowing services: nginx, Redis, Elasticsearch, Registry (18% of companies using Docker), PostgreSQL, MySQL, etcd, Fluentd, MongoDB, RabbitMQ.
7. On average , 7 Docker containers are launched on a single host . 9 months ago, the figure was lower - 5 containers each.
8. The average lifespan of a Docker container is 2.5 days; for comparison, a virtual machine has 23 days. A significant dependence was recorded on the availability of orchestration instruments - in companies without them, containers live on average 5.5 days each.
Similar statistics for Docker were collected at Sysdig ( published April 12, 2017), a service that specializes in monitoring Docker and has good support for related technologies. True, much less was analyzed there: only 45,000 containers. Of the data interesting in their context, Kubernetes is very popular (43% of all users, and products like OpenShift and Tectonic are included in this category), Quay (22% of users against the background of 15% of the Docker Registry).
(The most popular services launched in Docker, according to Sysdig)
Finally, we turned to our Russian colleagues from okmeter.io , whose monitoring we use in our projects. According to the statistics, the users of this service also have a stable and significant increase in the popularity of Docker. If in January 2016 only 9% of companies used Docker, and it was installed on 3% of hosts, in April 2017 the percentage of companies almost doubled, amounting to 17% , and hosts grew by more than 7 times, rising to 23% . Thus, the number of users is very close to the largest sample from Datadog, and the prevalence of Docker on hosts on okmeter.io clients was one and a half times higher.
From general to particulars. The site of Docker Inc itself, of course, has a section with customers and their success stories. You can find truly famous brands in it. I will not deal with the full listing, and instead I will give examples of some of them with the years of implementation / updates.
BBC News daily publishes 80,000 news stories in English alone, serviced by a system running on 150 servers with JVM and PHP applications. The infrastructure had ten different environments for Continuous Integration, launching a huge number of tasks used by 500+ developers. The tasks were carried out for a very long time (half an hour for setting in the schedule and another half hour for execution) and sequentially. Having switched to containers in 2015, we managed to get rid of half an hour for planning, because task launch has become parallel, and task execution time has been reduced. In addition, the new environment has become more open, scalable and reliable, which has allowed it to become a best practice, adapted in other BBC projects. (Details of the project are available in a 2-pagePDF file .)
One of the largest online stores has run into virtual machine limitations that were used to optimize application development processes. Having switched to Docker for continuous integration, the companies organized a process that ensures roll-out of containerized applications to developers' laptops, test environments, and production. In 2014, the eBay PaaS Team published details about the CI model used with Mesos in its technology blog . In 2015, eBay representatives talked about using Docker containers with Kubernetes (and OpenStack).
Travel company Expedia (more than 18 thousand employees in 60 countries) in ~ 2012 decided to switch to cloud infrastructure, so that its engineers began their journey to microservice architecture. To create and deploy such applications, suitable and reliable tools were needed. The result was that in 2016 more than 200 containerized applications were launched in production using Docker (hosted in AWS). Additionally, developers have their own data center, in which developers also have deployed. Among the tools used are the Docker Engine, Docker Trusted Registry (more than 1000 repositories) and Docker Compose. YouTube Speech AvailableExpedia specialists with details about their use of Docker in production. By the way, this year, the London division of Expedia posted on GitHub its educational course on Docker for developers, consisting of 8 lessons.
Groupon's e-commerce service was one of the earliest users of Docker: back in 2014, the company's engineers introduced containers for continuous integration needs. In the same year, at the DockerCon conference, they presented their Open Source project DotCi - a plug-in for Jenkins that simplifies job management (with built-in Docker support). Even then, it became a critical tool for managing the build and release processes, as Groupon CTO told the press.
The well-known electronic payment service PayPal (processes 200+ payments every second) uses a private OpenStack cloud, which runs more than 100 thousand virtual machines (all web applications and services work in them). The task was to modernize the infrastructure in order to make it more responsive, safe and cost-effective. As a tool, the path was chosen to refactor existing applications (in Java and C ++) with the goal of containerizing them and conveniently deploying them both to their cloud and to the public (which may be more cost-effective than supporting their own infrastructure). The result was the introduction ofin 2014, the Docker technology stack in production, including the Docker Commercially Supported Engine (CS Engine), Docker Trusted Registry, and Docker Compose. Containers coexist with virtual machines and significantly reduce the total number of VMs, which led to infrastructure optimization. In addition, the rollout speed of new applications has been significantly reduced due to the lightness of containers (compared to virtualization). In 2015, PayPal engineers went further, speaking at the DockerCon SF conference with a report ( video , presentation ) about their infrastructure in several data centers, CI with Jenkins and working with Docker images. And in 2016, LinuxCon NA was toldthat the PayPal CI system using Git, Docker, Mesos, Jenkins, Aurora, and the Travis CI API resulted in 10x savings.
(Docker orchestration at PayPal)
The popular news publication The Washington Post had difficulties with a growing number of applications written in different languages (Java, Scala, Node, Python, Go). Having decided to build a continuous deployment system for their AWS infrastructure, engineers chose the Docker Engine and Docker Compose. The current environment has more than 2500 Docker containers that fully serve the entire production, including a website for online readers (news, mobile application, mobile version of the site), their Arc Publishing publishing platform and news analytics system.
Among the other major users of Docker, listed on the website of the developer, you can find General Electric Appliances, New Relic, New York Times, Spotify, Uber, Yelp and Yandex (talking about PaaS Cocaine ). By the way, about Russia . Using Docker in Russian companies / Russian engineers is also a reality. Here are some examples:
By the way, if you also use Docker in production - please tell us about it in the comments . I am sure that for many it will be very interesting!
(View Docker containers through the baDocker utility created in Badoo)
The popularity and prevalence of software products, especially those that form the infrastructure, is determined not only by their customers, but also by how industry giants support them. And here only the lazy did not declare about Docker. In particular (there are, of course, much more examples, so only the main thing) :
(Microsoft’s active friendship with Docker)
Particular attention is paid to the growing ecosystem of projects created by the industry: Kubernetes from Google, Tectonic and Quay from CoreOS, RancherOS, Apache Mesos and DC / OS, Contiv from Cisco ... Finally, I will mention wide cooperation with non-profit organizations and projects such as Cloud Foundry , Open Containers Initiative , Cloud Native Computing Foundation .
Docker really “fired” and, not being some kind of supernova and super complicated technology, came at the right time to take its place in the world of DevOps and CI / CD. We ourselves have been using it in production for several years, we are telling about our experience and will continue to do so. Subscribe to our hub if you are interested :)
The scope and trends of dockerization
The starting point for this article is statistics from the Datadog monitoring service , which first appeared in 2015 and was last updated just the other day, in April 2017. Datadog is not some kind of traditional analytical company, but a monitoring system known for its friendliness to cloudy environments (its agent code is written in Python and published as Open Source) .
Having at its disposal a huge database on the structure of infrastructures in companies from around the world, Datadog collected statistics on adaptation of Docker, which she described as "no hype, just facts." The collected statistics represent 10 thousand companies and 185 million containers used for any purpose.(of course, this is not necessarily production, but at least a part of the infrastructure that makes sense to monitor) . Here are the facts:
1. If at the beginning of March 2016, about 13.6% of Datadog customers used Docker, then in a year (i.e. by March 2017) this figure increased by ~ 38%, amounting to 18.8% .
2. Docker is running on 15% of all hosts represented in monitoring. For comparison: two years ago there were only 3% of such hosts, and this is a much more significant growth.
3. The leaders of the adaptation of Docker are large companies. The more hosts the company has in monitoring, the more likely Docker is. About 60% of organizations with 500+ hosts are named Docker users. The latest research update also recorded a growing interest in Docker from medium-sized organizations (from 100 to 500 hosts): the number of Docker users among them was already close to that among large companies.
4. About 40% of Docker users use container tools like Kubernetes, Mesos, Amazon ECS, and Google Container Engine. A year ago, there were half as many.
5. Docker users increase the number of running containers by about 5 times during the period from the first to the tenth month of its use.
6. Most often in Docker they launchfollowing services: nginx, Redis, Elasticsearch, Registry (18% of companies using Docker), PostgreSQL, MySQL, etcd, Fluentd, MongoDB, RabbitMQ.
7. On average , 7 Docker containers are launched on a single host . 9 months ago, the figure was lower - 5 containers each.
8. The average lifespan of a Docker container is 2.5 days; for comparison, a virtual machine has 23 days. A significant dependence was recorded on the availability of orchestration instruments - in companies without them, containers live on average 5.5 days each.
Sysdig Statistics
Similar statistics for Docker were collected at Sysdig ( published April 12, 2017), a service that specializes in monitoring Docker and has good support for related technologies. True, much less was analyzed there: only 45,000 containers. Of the data interesting in their context, Kubernetes is very popular (43% of all users, and products like OpenShift and Tectonic are included in this category), Quay (22% of users against the background of 15% of the Docker Registry).
(The most popular services launched in Docker, according to Sysdig)
Statistics from Russia
Finally, we turned to our Russian colleagues from okmeter.io , whose monitoring we use in our projects. According to the statistics, the users of this service also have a stable and significant increase in the popularity of Docker. If in January 2016 only 9% of companies used Docker, and it was installed on 3% of hosts, in April 2017 the percentage of companies almost doubled, amounting to 17% , and hosts grew by more than 7 times, rising to 23% . Thus, the number of users is very close to the largest sample from Datadog, and the prevalence of Docker on hosts on okmeter.io clients was one and a half times higher.
Success Stories from Docker Inc
From general to particulars. The site of Docker Inc itself, of course, has a section with customers and their success stories. You can find truly famous brands in it. I will not deal with the full listing, and instead I will give examples of some of them with the years of implementation / updates.
Bbc news
BBC News daily publishes 80,000 news stories in English alone, serviced by a system running on 150 servers with JVM and PHP applications. The infrastructure had ten different environments for Continuous Integration, launching a huge number of tasks used by 500+ developers. The tasks were carried out for a very long time (half an hour for setting in the schedule and another half hour for execution) and sequentially. Having switched to containers in 2015, we managed to get rid of half an hour for planning, because task launch has become parallel, and task execution time has been reduced. In addition, the new environment has become more open, scalable and reliable, which has allowed it to become a best practice, adapted in other BBC projects. (Details of the project are available in a 2-pagePDF file .)
eBay
One of the largest online stores has run into virtual machine limitations that were used to optimize application development processes. Having switched to Docker for continuous integration, the companies organized a process that ensures roll-out of containerized applications to developers' laptops, test environments, and production. In 2014, the eBay PaaS Team published details about the CI model used with Mesos in its technology blog . In 2015, eBay representatives talked about using Docker containers with Kubernetes (and OpenStack).
Expedia
Travel company Expedia (more than 18 thousand employees in 60 countries) in ~ 2012 decided to switch to cloud infrastructure, so that its engineers began their journey to microservice architecture. To create and deploy such applications, suitable and reliable tools were needed. The result was that in 2016 more than 200 containerized applications were launched in production using Docker (hosted in AWS). Additionally, developers have their own data center, in which developers also have deployed. Among the tools used are the Docker Engine, Docker Trusted Registry (more than 1000 repositories) and Docker Compose. YouTube Speech AvailableExpedia specialists with details about their use of Docker in production. By the way, this year, the London division of Expedia posted on GitHub its educational course on Docker for developers, consisting of 8 lessons.
Groupon
Groupon's e-commerce service was one of the earliest users of Docker: back in 2014, the company's engineers introduced containers for continuous integration needs. In the same year, at the DockerCon conference, they presented their Open Source project DotCi - a plug-in for Jenkins that simplifies job management (with built-in Docker support). Even then, it became a critical tool for managing the build and release processes, as Groupon CTO told the press.
Paypal
The well-known electronic payment service PayPal (processes 200+ payments every second) uses a private OpenStack cloud, which runs more than 100 thousand virtual machines (all web applications and services work in them). The task was to modernize the infrastructure in order to make it more responsive, safe and cost-effective. As a tool, the path was chosen to refactor existing applications (in Java and C ++) with the goal of containerizing them and conveniently deploying them both to their cloud and to the public (which may be more cost-effective than supporting their own infrastructure). The result was the introduction ofin 2014, the Docker technology stack in production, including the Docker Commercially Supported Engine (CS Engine), Docker Trusted Registry, and Docker Compose. Containers coexist with virtual machines and significantly reduce the total number of VMs, which led to infrastructure optimization. In addition, the rollout speed of new applications has been significantly reduced due to the lightness of containers (compared to virtualization). In 2015, PayPal engineers went further, speaking at the DockerCon SF conference with a report ( video , presentation ) about their infrastructure in several data centers, CI with Jenkins and working with Docker images. And in 2016, LinuxCon NA was toldthat the PayPal CI system using Git, Docker, Mesos, Jenkins, Aurora, and the Travis CI API resulted in 10x savings.
(Docker orchestration at PayPal)
The washington post
The popular news publication The Washington Post had difficulties with a growing number of applications written in different languages (Java, Scala, Node, Python, Go). Having decided to build a continuous deployment system for their AWS infrastructure, engineers chose the Docker Engine and Docker Compose. The current environment has more than 2500 Docker containers that fully serve the entire production, including a website for online readers (news, mobile application, mobile version of the site), their Arc Publishing publishing platform and news analytics system.
Other examples
Among the other major users of Docker, listed on the website of the developer, you can find General Electric Appliances, New Relic, New York Times, Spotify, Uber, Yelp and Yandex (talking about PaaS Cocaine ). By the way, about Russia . Using Docker in Russian companies / Russian engineers is also a reality. Here are some examples:
- Badoo's view of the year of using Docker in production;
- report on the development and deployment of services using Docker and Kubernetes in Avito ;
- report from Aviasales.ru about migration of the search engine to Docker;
- and for a change - the impression of using Docker at the bank (from the architect of Alfa Laboratory, Alfa-Bank 's special-purpose department ).
By the way, if you also use Docker in production - please tell us about it in the comments . I am sure that for many it will be very interesting!
(View Docker containers through the baDocker utility created in Badoo)
IT adoption
The popularity and prevalence of software products, especially those that form the infrastructure, is determined not only by their customers, but also by how industry giants support them. And here only the lazy did not declare about Docker. In particular (there are, of course, much more examples, so only the main thing) :
- Docker is integrated / supported / maintained by Amazon Web Services, Google Cloud Platform, IBM Bluemix, Jelastic, Microsoft Azure, OpenStack, Oracle Container Cloud Service, VMware vSphere Integrated Containers ...
- It is supported in configuration management systems: Ansible, CFEngine, Chef, Puppet, Salt.
- Red Hat is not just “supporting” Docker, but is developing its OpenShift platform (based on Kubernetes), Canonical is officially collaborating , and a special subscription is offered for SUSE .
- Microsoft is not limited to Azure, and recently recently bought Deis.
(Microsoft’s active friendship with Docker)
Particular attention is paid to the growing ecosystem of projects created by the industry: Kubernetes from Google, Tectonic and Quay from CoreOS, RancherOS, Apache Mesos and DC / OS, Contiv from Cisco ... Finally, I will mention wide cooperation with non-profit organizations and projects such as Cloud Foundry , Open Containers Initiative , Cloud Native Computing Foundation .
Total
Docker really “fired” and, not being some kind of supernova and super complicated technology, came at the right time to take its place in the world of DevOps and CI / CD. We ourselves have been using it in production for several years, we are telling about our experience and will continue to do so. Subscribe to our hub if you are interested :)
