Four releases 1.0 from CNCF and major announcements about Kubernetes with KubeCon 2017 / Flant Blog

These days (December 6-8), the local version of the KubeCon + CloudNativeCon 2017 conferences is being held in the USA, in the focus of attention of which are numerous projects of the CNCF non-profit organization led by Kubernetes. The event attracted more than 4100 visitors, 77% of whom use K8s, and in 75% of cases we are talking about production. The event turned out to be not only rich in announcements from various start-ups and industry giants, but also became an occasion for summing up the results of the Open Source community from the cloud native world - it’s enough to notice that in December, four CNCF projects immediately launched a significant release 1.0. So what's new?

Series of releases

CoreDNS 1.0.0

On December 1st, CoreDNS 1.0.0 was released . We have already talked more about this project in a separate article . In short, CoreDNS is a DNS server written in Go and similar to Caddy (moreover, it originated as its fork) in its key architectural pattern - the use of a set of handlers chained together. As a backend (storage) for data used in DNS records, CoreDNS supports etcd, Kubernetes and a zone file in RFC 1035 format. CoreDNS development plans for April 2017 The main area of ​​developers' work in preparing CoreDNS 1.0.0 was improvement in Kubernetes support plugin. And this is logical, if we recall that the authors of the project seek




replace kube-dns . They reinforce their intent with evidence that CoreDNS functionality is wider, performance is better, and memory consumption is less. An example of testing a Kubernetes cluster with 5,000 services is given, in which CoreDNS was able to process 18,000 requests per second using 73 MB of RAM, against the figure of 7000 qps for kube-dns with a consumption of 97 MB of RAM.

Among the features in which CoreDNS is superior to kube-dns are filtering records by namespace and label selector; mode pods verifiedfor checking hearths before responding to a request pod.cluster.local; endpoint_pod_namesto use hearth names when the host name is not set; autopathto autocomplete the search path on the server side.

CoreDNS support has already been added to various Kubernetes tools (alpha feature for 1.9): kubeadm, kops, minikube and kubespray.

containerd 1.0.0

About containerd , we also had a separate article . The history of this project is such that, initially being part of Docker, containerd survived its separation from the common code base and turned into an independent project under the control of CNCF at the same time as its competitor implementing the same functions (i.e., the executable environment for containers), - rkt from CoreOS

The further fate of containerd is its integration into Kubernetes via CRI ("Container Runtime Interface") through a connecting layer called cri-containerd :



We wrote about this and the next main competitor of the project on a new "ground" - CRI-O here. By the way, at the end of November, significant progress in the development of rkt was also announced : CNCF announced the first release of rktlet - the implementation of rkt on top of the same CRI.

But returning to the topic: on December 5, containerd 1.0.0 was released. And he brought a lot of innovations to the capabilities of this product, and in particular:

• A new model of the runtime environment based on tasks
• client-defined push / pull for images and the ability to use a third-party image distribution system,
• Snapshot container storage system
• arbitrary storage support for object metadata,
• Advanced gRPC API with namespaces.

Fluentd 1.0

Fluentd - a data collector written in Ruby designed to unify the logging layer between log sources ( data sources : syslog, web server logs, etc.) and their storage systems ( data outputs : various DBMSs, queue systems, AWS, etc. .). Its widespread adoption by “thousands of companies” allows CNCF to call fluentd “the industry standard for logging.” (By the way, we ourselves use it now in our logging system for Kubernetes - loghouse .)



In the release 1.0 of December 6, appeared:

• multiprocessor workers for better performance with SMP,
• storage of exact time (with an accuracy of less than a second) for all records,
• Windows support
• new API for plugins (and there are already over 700 of them),
• compression of data stored on disk using built-in buffers,
• native TLS support and the new Fluentd Forward Protocol v1 with support for authentication via keys and authorization by login / password.

Jaeger 1.0

Jaeger joined the ranks of CNCF projects quite recently (in September). This is a distributed tracing system written in Go, created by Uber and compatible with OpenTracing (this is also a CNCF project). The mission of Jaeger is convenient monitoring of complex microservice architecture, designed to help identify the causes of problems (taking into account all the dependencies) and help optimize performance.



Released on December 6, Jaeger 1.0 brought this project:

• support for new backends for data storage (ElasticSearch was added to Cassandra, and work is underway on MySQL + in the community experimenting with ScyllaDB and InfluxDB),
• numerous improvements in the web user interface,
• full integration of all components with Prometheus as the default metric system,
• improvements in integration with Kubernetes ( templates for launching in K8s and a chart for Helm ),
• earlier version of the client in C ++,
• significant progress in backward compatibility with Zipkin.

New CNCF members

Of course, not without numerous replenishment in the ranks of CNCF participating companies.

Together with the announcement of the start of KubeCon + CloudNativeCon, the fund presented 31 new members at once, among which are Datadog (this SaaS monitoring solution had wonderful statistics about Docker), Grafana Labs (as you might guess, authors of Grafana), HPE (Hewlett Packard Enterprise ), InfluxData (the authors of the DBMS for working with time series are InfluxDB), NGINX, Pinterest (see details below), SAP Concur (they wrote about them in this success story ).

And in some news, new members of the “highest categories” at CNCF were announced:

• On December 4, JFrog received Gold Member status, offering an infrastructure for managing software development; the press release notes that the company “uses technologies like Kubernetes to help 4000+ customers build and release software quickly, reliably and safely”;
• On December 6, the CNCF ranks were replenished with a platinum member (there are only 21 of them now) in the form of Alibaba Cloud - the cloud-responsible division of the large Chinese group of companies Alibaba Group (its chief architect joined the CNCF managing board); we wrote about their experience using Docker in production here ;
• December 6, another Chinese company - Baidu - became a gold member of CNCF; their main application Kubernetes is in the field of machine learning and deep learning (in particular, last year the code of the PaddlePaddle framework was opened , which supports launching in the K8s cluster);
• On December 8, Salesforce, which “adapted many CNCF technologies,” became the CNCF ’s gold member ; in particular, they talk about the use of Kubernetes as the basis for internal CI / CD processes, as well as the active use of gRPC in the service mesh team.

Other events

Other news from prominent IT market participants were announced at KubeCon 2017. Among them:

• Brendan Burns, one of the founders of the Kubernetes project and a current Microsoft employee, introduced Metaparticle - “a standard library for cloud-based (cloud native) applications on Kubernetes”, whose goal is to simplify the development of distributed systems by providing “simple and powerful building blocks based on containers and Kubernetes "; only JavaScript / Node.js, Java and .NET are declared as supported programming languages, but they promise to expand the list soon;

Example implementation of master election in JavaScript in Metaparticle

• Oracle has released two Open Source products related to Kubernetes: an installer for its serverless platform Fn project and a set of utilities for distributed clusters (Global Multi-Cluster Management);
• Microsoft brought even more news, because:
  
  1. opened the code of Virtual Kubelet - an alternative implementation of the Kubernetes component of the same name, specially designed for connecting K8s to third-party APIs (Azure Container Instances, Hyper.sh, AWS are given as examples);
     
     
  2. opened the Open Service Broker for Azure (OSBA) code created based on the Open Service Broker API to simplify working with Azure data services through the Kubernetes API: “For example, using OSBA and Helm you can now easily install a Wordpress instance with Azure Database for MySQL instead of starting the DBMS in the container ”; The project has alpha version status;
  3. introduced Kashti - a dashboard and visualization tool for the Brigade pipeline ( YouTube demo );
• another announcement from Microsoft, already with the participation of Heptio, says that the company is working on a backup system for Kubernetes clusters (as part of its recently introduced Azure Kubernetes Service) using the Heptio Ark Open Source Utility ;
• the product manager from the already mentioned Pinterest online service spoke about the migration of their production infrastructure to containers, which began in 2016 and is nearing completion, as well as plans for its further transfer to Kubernetes (by the middle of next year);
• AWS Cloud Strategy Vice President made a presentation announcing the company's plan to use CNI as the network basis for containers and promised that all changes made by AWS to Open Source code will go upstream (for of this, the company is already “working with various Open Source projects, communities and foundations”).

Finally, the upcoming release of Kubernetes 1.9 is also tied to the dates of KubeCon 2017: literally beta2 was released that night, completion of work on all documentation is expected within the next 24 hours, and the release itself is scheduled for next Monday (December 11).

PS CNCF collected statistics about visitors to KubeCon + CloudNativeCon North America 2017 and how they use containers and various fund projects in their infrastructure. Here, for example, are the main difficulties that respondents face and how they have changed over the past year:

PPS

Read also in our blog:

• “ The New Stack Statistics on Kubernetes Implementation Difficulties ”;
• « Kubernetes 1.8: overview of the major new features ";
• “The Linux Foundation has introduced a free introductory online course on Kubernetes ”;
• " CNCF has offered a free cloud to Open Source projects for DevOps / microservices ";
• “ Infrastructure with Kubernetes as an affordable service .”