shurup March 21, 2019 at 09:18

Linux Foundation launches new DevOps project with Jenkins and Spinnaker

Last week, The Linux Foundation, during its Open Source Leadership Summit event, announced the creation of a new foundation for Open Source projects. Another independent institute for the development of open [and demanded in the industry] technologies is designed to combine the tools for DevOps engineers, and more precisely - to organize and implement continuous delivery processes, CI / CD pipelines. The organization was called: The Continuous Delivery Foundation (CDF).

To better understand why such funds are created at all with the parent organization Linux Foundation, just look at a more well-known example - CNCF (Cloud Native Computing Foundation). This fund appeared in 2015 and has since accepted many Open Source projects that truly define the modern landscape of cloud IT infrastructure: Kubernetes, containerd, Prometheus, etc.

The organization itself acts as an independent platform on the basis of which these projects are managed and developed in the interests of different market participants. For this, CNCF created technical and marketing committees, adopted certain standards and rules (if details are interesting, then we recommend reading, for example, CNCF TOC Principles )... And, as we see in “living” examples, the scheme works: projects located at CNCF are becoming more mature, gaining popularity in the industry, both among end users and among developers involved in their development.

Following this success (after all, many CNCF cloud projects have already entered the daily life of DevOps engineers) , the general trends in IT and their manifestations in the Open Source world, The Linux Foundation decided to “take” (or rather, say “advance” ») A new niche:

“The Continuous Delivery Foundation (CDF) will become a vendor-independent home for critical Open Source projects designed for continuous delivery and specifications that accelerate pipeline processes. CDF will facilitate the interaction of leading developers, end users and vendors from the industry, promote CI / CD and DevOps methodologies, identify and document best practices, create guides and training materials that will enable the software development team from anywhere in the world to implement the best CI / CD practices ” .

Idea

The core values and principles that guide the CDF are currently formulated in such a way that the organization:

... believes in the power of continuous delivery and that it provides new opportunities for developers and teams, allows the release of high-quality software more often;
... believes in open source solutions whose combined use spans the entire software delivery cycle;
... cultivates and supports an ecosystem of Open Source projects that are independent of vendors through collaboration and interoperability;
... promotes and promotes, among specialists involved in continuous delivery in practice, collaboration, dissemination of their practices and their improvement.

Participants and projects

But beautiful words are the lot of marketers, which does not always coincide with what is actually happening. And in this sense, the first impression of the organization can be made by those of the companies that formed it and which projects became its “first-born”.

The main members of the CDF are 8 companies , namely: Capital One, one of the top 10 banks in the USA, and industry representatives represented by CircleCI, CloudBees, Google, Huawei, IBM, JFrog and Netflix, which are much more familiar to IT engineers. Some of them have already talked about such a significant event on their blogs, but more on that below.

The CDF participants also include the end users of his projects - CNCF has a similar category where you can find eBay, Pinterest, Twitter, Wikimedia and many others. In the case of the new fund, there are only 15 such participants so far, but among them there are already interesting and well-known names: Autodesk, GitLab, Puppet, Rancher, Red Hat, SAP and Sysdig, which joined the day before yesterday .

Now, perhaps, about the main thing - about the projects on which CDF was entrusted. There were four of those at the time of the creation of the organization:

Jenkins and Jenkins X

Jenkins is hardly a CI / CD system, written in Java, that has existed for many years (think only: the first release - in the form of Hudson - took place 14 years ago!) , For which it has gained an uncountable army of plugins.

Today, CloudBees can be considered the main commercial entity behind Jenkins,whose technical director is the original author of the project (Kohsuke Kawaguchi) and who became one of the founders of the foundation.

Jenkins X - this project also owes much to CloudBees (as you might guess, its main developers are on the staff of the same company), however, unlike Jenkins itself, the solution is completely new - it is only a year old.

Jenkins X offers a turnkey CI / CD solution for modern cloud applications deployed within Kubernetes clusters. To do this, JX offers pipeline automation, a built-in implementation of GitOps, release preview environments, and other features. The architecture of Jenkins X is presented as follows:

Product Stack - Jenkins, Knative Build, Prow, Skaffold and Helm. We already wrote in more detail about the project on a habr

Spinnaker

Spinnaker is Netflix’s continuous delivery platform, which was released in 2015. Currently, Google is also actively involved in its development: through their joint efforts, the product is developing as a solution for large organizations whose DevOps teams serve many development teams.

The key concepts in Spinnaker for describing services are applications, clusters and server groups, and load balancers and firewalls are available for the outside world: You can read

more about the basic Spinnaker device read in the project documentation .

The platform allows you to work with many cloud environments including Kubernetes, OpenStack and various cloud providers (AWS EC2, GCE, GKE, GAE, Azure, Oracle Cloud Infrastructure), as well as integrate with various products and services:

with CI systems (Jenkins, Travis CI) in pipelines;
with Datadog, Prometheus, Stackdriver and SignalFx - for monitoring events;
with Slack, HipChat and Twilio - for notifications;
with Packer, Chef, and Puppet for virtual machines.

Here's what Netflix wrote about incorporating Spinnaker into the new fund:

“Spinnaker’s success has largely come about thanks to the wonderful community of companies and people who use it and contribute to development. Transferring Spinnaker to CDF will strengthen this community. This step will contribute to the emergence of changes and investments from other companies that observed what was happening from the outside. Open doors for new companies will bring new innovations to Spinnaker that everyone will benefit from. ”

And in a Google publication on the occasion of the creation of the Continuous Delivery Foundation, it is specifically noted that “Spinnaker is a multi-component system that is conceptually consistent with Tekton.” So we come to the last project included in the new fund.

Tekton

Tekton is a framework presented in the form of common components for creating and standardizing CI / CD systems, implying the operation of pipelines in various environments including ordinary virtual machines, serverless and Kubernetes.

These components themselves are “Kubernetes-style” resources (implemented in K8s itself as CRD), which act as building blocks for defining pipelines. A brief illustration of their use in the K8s cluster is presented here .

The Tekton-supported product stack will seem familiar: Jenkins, Jenkins X, Skaffold, and Knative. Google Cloud believes that Tekton is solving "the problem of collaboration between the Open Source community and leading vendors to upgrade the infrastructure for CI / CD."

...

By analogy with CNCF, a technical committee (Technical Oversight Committee, TOC) has been created in CDF, whose responsibilities include the consideration of issues (and decision-making) on the inclusion of new projects in the fund. There is not so much other information about the organization on the CDF website yet, but this is normal and only a matter of time.

We conclude with a quote from the announcement of JFrog :

“Now, having become one of the companies of the newly created Continuous Delivery Foundation, we will take our commitment [to create a technology that is universal in its support of other CI / CD solutions] to a new level. Following the ideas of collaboration and openness, this new organization will manage future continuous delivery standards that will accelerate the software release cycle. With the adoption of Jenkins, Jenkins X, Spinnaker and other technologies under the wing of this fund, we see a bright future for CI / CD! ”