Mission feasible: how to develop DevOps in a company with many projects

    A lot of articles have already been written about the benefits of implementing DevOps on Habré and other IT resources, and it is not in doubt. This is understandable: a creative mess with sometimes unobvious areas of responsibility of the “usual” development, where different people are responsible for the code, brunches, test benches, assembly and deployment and do not really want to climb into the clearing of colleagues, is opposed to a high level of organization .

    In companies with embedded DevOps, developers have more support and can work more efficiently. But it’s not enough just to want to work “correctly”; you still need to somehow come to this. But here everything is not so simple.





    You can’t just take and develop DevOps (c)


    More precisely, it can, but easily, this process can only go through in small companies where one or two projects are developed, or when all teams initially have a high level of development culture. But what about internal startups, where the idea often overtakes the physical and technical capabilities of the company?

    When we at Positive Technologies decided to develop DevOps, we ran into dozens of teams that worked on projects, both public and non-public. The teams were very different in size, using different release models and a technological stack. All this did not allow to create a solution on the principle of one size fits all.

    Moreover, resources are always limited and they were not enough to introduce DevOps approaches within each team and project. Some new approach was needed. And this is what we came up with.

    First steps


    We needed a set of elementary DevOps practices that we could quickly adapt to the tasks of any project. To do this, it was necessary to develop templates for assembly, deployment and test configurations, to ensure scaling of projects for many git-branches of individual components, as well as a large number of components and their versions.

    We managed to solve these problems by transferring the Continuous Integration infrastructure to using a bundle of several tools:

    • TeamCity - organization system Continuous Integration;
    • GitLab - a storage system for the source code of component products;
    • Artifactory is a storage system for assembled binary versions of components and products, caching proxies of external repositories for various package managers (nuget, npm, pip, rails).

    In the course of numerous experiments on real projects, the TeamCity project hierarchy and a standard interface for all projects in this system were developed.

    The processes were elaborated in detail:

    • assembly and deployment to test stands;
    • launch functional and other tests;
    • assignment of quality labels to packages (test, stable, ready for distribution to customers);
    • Publishing release builds to update servers
    • delivery of assemblies and updates to production and many others.



    By clicking on the picture will open in full size

    What is the result


    All this is good, but there were questions about the implementation of DevOps in all projects of the company.

    But what if we combine several distinguished specialists with various specialized knowledge into one group that can solve DevOps problems of the whole company?

    So the idea of ​​creating a "fire brigade" came up. Now, instead of inventing automation from scratch for each project independently, employees working on it can order the DevOps service. 

    As a result, we were able to significantly reduce the cost of training and starting work on the new methodology, as well as reduce the number of possible errors.



    The topic of implementing DevOs is extremely interesting, various approaches are possible here. And it is extremely important to meet and share experiences. Usually such events are called “mitaps”.

    We are going to essentially do something similar, but not quite in the usual format: we want to collect the “devs” and developers in a relaxed atmosphere in order to tell the story of building our own DevOps in fast track mode and listen to you, those for whom we did it : developers, automation engineers and testers.

    Waiting for you at Op! DevOps! Friday, October 7th from 3 p.m. to 7 p.m. at Blacksmith Irish Pub

    More about the event program

    No.

    Topic

    Presenter

    Brief Abstracts

    The first block of reports
    1Continuous Integration System Model at Positive Technologies [ slides , video ]Timur Gilmullin1) The initial standard schemes offered by DevOps for all projects of the company: Build - Deploy - Testing - Promote.
    2) Implementation of the scheme on the examples of our projects in TeamCtiy.
    3) What have we come to. General Continuous Integration: Build - Deploy - Testing - Promote - Publishing - Delivery - Install & Update
    2SupplyLab - publication, delivery, deployment, licensing [ slides , videos ]Alexander Pazdnikov1) Organization of an open system for managing the full cycle of delivery, deployment and licensing to the Customer.
    2) Designing a publishing, delivery, deployment and licensing system - SupplyLab.
    3General concept of the server environment deployment system based on SaltStack [ slides , video ]Dmitry Miroshnichenko1) Designing an update system.
    2) About SaltStack.
    4Toolkit for creating product distributions [ slides , videos ]Vladimir Selin1) What is a big product distribution?
    2) Problem: a small number of people have knowledge about the product installation process.
    3) Templates + DSL - the solution to all problems!
    The second block of reports
    5Organization of workflow in the TFS tracker [ slides , video ]Alexey Soloviev1) TFS as a tracker: a brief overview of the features.
    2) Structure of a typical Workflow: basic elements.
    3) The difficulty of customizing WorkFlow in TFS.
    6vSphereTools - a tool for automating work with vSphere [ slides , video ]Timur Gilmullin1) VIX API against pysphere.
    2) vSphereTools is a set of scripts from DevOps to support working with vSphere and virtual machines.
    3) Description of the tool, its advantages and disadvantages, possible improvements.
    7TeamCity and Character Server Integration [ slides , video ]Alexey Soloviev1) What is a debug symbol server, its purpose.
    2) Debugging information (debugging symbols) - information that the compiler generates on the basis of source codes. Contains information about the source file names, variables, procedures, functions.
    3) Debug information server - a server whose main purpose is to store debug information, its indexing and providing access.
    8Tools for conducting competitive analysis of software products [ slides , videos ]Vladimir Sofin1) What is the competitive analysis (CA) of software products?
    2) Methodology and stages of spacecraft.
    3) Difficulties in the implementation of the various stages of the spacecraft.
    4) Tools for spacecraft automation.
    The third block of reports
    9Neuro-fuzzy classification of poorly formalized data [ slides , video ]Timur Gilmullin1) Problems of automation of classification of weakly formalized (fuzzy) data.
    2) Fuzzy sets and fuzzy measuring scales.
    3) Neural network modeling for data classification.
    4) The FuzzyClassificator tool and its implementation in the Company.
    5) Automation of data classification based on TeamCity.
    10From simple to complex: we automate manual test plans [ slides , videos ]Sergey Timchenko1) We look around - the usual process of auto-testing.
    2) Remove unnecessary - realistic target process.
    3) DataDrivenTesting - creating specials. tools for specific scenarios.
    4) RobotFramework - what to do if there are too many simple scripts.
    elevenZabbix monitoring system in the development and testing processes [ slides , video ]Alexey Burov1) A system for monitoring the resources of various departments.
    2) Templates and roles of servers, differentiation of access and areas of responsibility.
    3) ptzabbixtools - monitoring configuration on target servers.
    4) An example of embedding a monitoring system in development / testing processes.
    12Automation of load testing in conjunction JMeter + TeamCity + Grafana [ slides , video ]Ivan Ostanin and Sergey Tikhonov1) Description of the old process of littering test data: how it was before, what is good, what is bad.
    2) Influxdb, as a repository of time-series data.
    3) Zabbix - monitoring of load stands: Windows and Linux agents, active data collection, autodiscovery virtual machines in esx.
    4) Grafana, as a way to turn graphics and dashboards into candy.
    5) Automation of user load via web-UI using Jmeter, real-time statistics display, CI Teamcity.
    The fourth block of reports
    thirteenCrossPM package manager: simplifying complex dependencies [ slides , video ]Alexander Kovalev1) Difficulties in unraveling cross and nested dependencies.
    2) CrossPM package manager. Its capabilities and examples of use.
    3) Integration of CrossPM and Artifactory package storage systems.
    14Practical recommendations for using the TestRail system [ slides , video ]Dmitry Ryltsov and Alexey Vasiliev1) Purpose of using TestRail.
    2) The essence of the TestRail system.
    3) Features of the project.
    4) Our decision.
    5) TestRail Integration & Customization.
    fifteenTeamPass - management of access control for service passwords in a team [ slides , video ]Dmitry Miroshnichenko1) What did not work in keepass?
    2) Company specifics.
    3) Solution.
    16DevOpsHQ community [ slides , video ]Alexander Pazdnikov1) About the project DevOpsHQ - a community of automation engineers.
    2) The pursued goals.
    3) Suggested tools.
    4) Suggested techniques.
    5) Ready-made solutions.



    The event is free, but the number of seats is limited, so you need to register to participate. To do this, you need to register .

    Also popular now: