Harbor - Registry for Docker Containers with Out-of-Box Security

    On July 31, CNCF announced the adoption of a new Open Source project, characterized as a cloud native registry, Harbor, in its sandbox (that is, at the earliest stage of support) . On his website, they explain to us that the product was created to manage images of Docker containers in a secure environment.



    It would seem that there is already a Docker Registry (or, say, Quay from CoreOS), but it is obvious that new solutions do not appear and do not ripen to use in production just like that - especially, Open Source solutions ... and even more so, falling into CNCF. This review article aims to shed light on the causes of the emergence of Harbor, its key features and features.

    Harbor's first focus: network and enterprise


    The history of the project begins in 2016, in March of which the first public release took place - 0.1.0. Its creation was backed by VMware engineers who described the project as an “enterprise-class registry server”, which, based on the Docker development, “expands the capabilities of the Docker Registry by adding more functions to it, which are usually required in an enterprise”.

    At that time, the emphasis was more on the possibility of using this registry within the company, in particular, potentially improving productivity by storing images in the corporate network: “[Harbor] is very useful for users of containers who do not have a good internet connection. For example, Harbor improves the performance of Chinese developers by eliminating the difficulty of downloading images from the public Internet. ”(from README to Harbor 0.1.0 ) .

    Note : Harbor's orientation on China, confirmed by the presence of appropriate localization from the first releases, was not accidental: the creation of the project as such was initiated by the Chinese division of the company (VMware China R & D).

    What has become commonplace for the cloud native ecosystem, Harbor was written from the very beginning in Go and licensed under the Apache License 2.0. If we talk about the functionality of the project, then in the first release, the authors laid down some of the actual and to this day features, such as:

    • role-based access control ( RBAC , which allows to organize users and repositories in the form of “projects” and issue different rights to images within these projects), as well as support for LDAP and AD for user authentication;
    • user web interface to view the repository, search through them, project management;
    • audit of all operations;
    • REST API to manage.


    Project management in the Harbor web interface

    Harbor Evolution: Safety


    In 2017, VMware found a logical use for its new project - integration with other solutions of the company for containers. In particular, vSphere Integrated Containers (VIC), which, while not being a full-fledged PaaS (Platform as a Service) or CaaS (Containers as a Service), were actively developing , offered some kind of foundation for working with containers. Today, for example, vSphere Integrated Containers Engine , which is an executable container environment for vSphere , has grown . This is how VIC solution engineer VMware (Nate Reid) described at that time:

    “The VIC takes the skeleton of a single Docker host and scales it to multiple ESXi hosts. His proposed architecture for orchestration is similar to Swarm, Kubernetes (K8s), and Mesos. However, there are some nuances, and there are no tasks to replace all these products. VIC provides a container for the network, allows you to get their names, offers the HTML5 control panel (Admiral; for more details about it, see our Docker GUI Review - approx. Translation ), corporate level registry (Harbor), many similar Docker commands, integration with vSphere tools. [..]

    If you need orchestration support with Kubernetes and / or VMware CI / CD capabilities from VMware, you should look at VMware PKS and / or Pivotal Cloud Foundry. These are CaaS and PaaS class solutions. ”

    At the same time, the issue of security of Docker images is becoming increasingly important . In early 2018, Pivotal’s brotherly engineers for VMware cite a study that even the latest versions of images posted on the Docker Hub (both from the community and official ones) contain numerous vulnerabilities (an average of 70 per image).

    It was then that Harbor appeared with its new safety-oriented purpose and already on the mentioned “soil” of CaaS - in the Pivotal Container Service (PKS) :

    “This is [the presence of vulnerabilities and other security issues in Docker images] and there is a reason that we have included so many useful additions that make PKS safe and secure! [..]

    Since Kubernetes by itself does not deal with such issues [secure image management], we worked with friends from VMware to turn Harbor into PKS. ”

    What is so safe added to Harbor (besides those already implemented in the RBAC project and audit)? Two main directions are indicated:

    1. Vulnerability scan . To do this, Harbor implements CVE acquisition using known databases (NIST NVD, Ubuntu CVE Tracker, Red Hat Security Data, etc.) and automatically scanning each container image for their presence when performing push and pull operations. If a vulnerability is found, operations are canceled depending on the security settings, and the image itself is flagged, which will be visible to the registry administrator. To realize this opportunity, Harbor integrated with Clair from CoreOS.
    2. Signature images . Here we also use the groundwork of another project - Notary (we mentioned it in this article ) - which makes a signature when pushing images, and then validates such signatures with each pull.

    The general scheme of applying Harbor to PKS is as follows:



    Harbor today


    So, by offering a registry of container images for use on-premises and providing security in various aspects of working with them, today Harbor has evolved to the following architecture, apparently, combining functions from other Open Source projects:



    In addition to the already mentioned Docker Registry, Clair and Notary, implementing the key capabilities of Harbor, in this scheme, you can also see the presence of two DBMS:

    1. PostgreSQL ( previously here was MySQL / MariaDB), which is used to store metadata about projects, users and their roles, images;
    2. Redis - for storing sessions .


    Databases in the Harbor Architecture

    Some details about the overall internal structure of Harbor can also be found in this wiki page , which is referenced by the official documentation of the project (however, it is suspected that some of the details on the architecture could be out of date in some places) . You can also find links to Harbor installation instructions and its deployment to Kubernetes . The latter, however, has been declared obsolete (based on older versions, does not support Clair and Notary), and instead it is proposed to use the Helm-chart .

    Current version Harbor - v1.5.2released in late July. Requirements for a Linux-based machine on which a fresh release of the registry is installed - Docker version 17.03.0-ce (or higher) and Docker Compose 1.10.0+, as well as Python and OpenSSL.

    A very interesting innovation for the future version of Harbor (already released v1.6.0-rc1 ) is support for the Helm charts : “Harbor, starting from version 1.6.0, will become a composite cloud native registry that will support both image management and chart management” . Other development plans include OAuth 2.0 support for user authentication, the ability to deploy on multiple sites for fault tolerance and load balancing, collecting repository statistics and a utility for migration to Harbor.

    Instead of conclusion


    Harbor is an example of a successful project from the modern cloud world, which managed to find its niche and establish itself, simultaneously integrating the capabilities of other Open Source tools. The evidence of his success is not only the inclusion of CNCF in the list of projects, but also 5000+ stars on GitHub , and a rather active developer community.

    PS


    Read also in our blog:


    Also popular now: