One bot from all worries
Until the convention "On the Protection of the Rights of an Inhuman Person" has been adopted, you need to use this and give the working routine to bots. It makes sense to start right now, or after 5 years, the uprising of cars will begin, mass lawsuits to insult the feelings of bots with boring tasks will flood the courts for regulating human-machine relations. So hurry up.
There is such work that you do not want to do, but you need to. This article will not have large inserts with code and instructions on how to create your bot from scratch. I’d better tell you how the release process works in our Dodo Pizza, how we automate and speed it up, giving some of the boring routine to our bot written in C #. I will pay attention to the functionality and logic of the bot. It will be cool if this material inspires you to write your assistant who will make life easier for you and your colleagues.
A couple of years ago we released one release once a week. In May 2018, having a maximum rate of 147 hours for release, we set a goal to release every day. Now our minimum: four hours to release, but this does not happen often. We want to fix the record and be able to reduce the whole process to the press of a single button.
Now in Dodo IS seven teams take turns releasing a release. One person from the team becomes “lucky” - relizmen. Relizmen as an airplane pilot: he has a bunch of levers and devices that he needs to skillfully wield in order to roll out next updates. We thought and decided: "It's time to do an autopilot, and we’d better spend our time on something more exciting than filling out boring tables with statistics and tracking auto test runs."
So, in order to become a relisman, it’s necessary that the turn comes to your team. So that everything was clear and no one was confused, we stuck stickers with the names of teams on the wall. The release team receives an honorary crown, which we move with handles each time.
After receiving theblack markRelisman's crowns should remember where the checklist of release steps lies. Further: remembered -> found -> created a copy from the template and, finally, opened the checklist in the Kaiten system. Kaiten is an electronic board on which in the form of cards we place and monitor the status of tasks in development. For new developers, this procedure as a whole was very unobvious. And how do you know where to look for this check sheet and where to start when there are no clues?
Having gathered our will into a fist, we decided that it was enough to endure it. It's time to give some of the boring routine to the bot. Who if not us? When, if not now?
The decision is made, it's time to start designing our autopilot! Having found the Kaiten API here: https://faq.kaiten.io/docs/api , with just one request, we created a card for the new relizmen.
Now this card needs to be handed to the team that will release the next.
Perfectly! Now we have a clue what to do next. We open the checklist, look at it: “Create a channel for release in Slack, invite all the teams there whose changes are in the release and find out if they will need manual testing.” It remains to teach this to our bot.
Open the Slack API documentation here https://api.slack.com and look for a method for creating a channel there.
As you can see, in Slack, as in other tools, there is nothing complicated. It all comes down to sending one POST request with two required parameters (these are the ones opposite which it says required). In the request, we need to pass the name of the created channel (name parameter) and the authorization token (token parameter). Pay attention to the line "Works with: Token type - user, required scope (s) - channels: write".
Slack has two types of tokens: user - issued to users and bot - issued to applications. When issuing a token, we determine what rights to vest its owner. To create channels, we need a user token (token type - user), which has rights to write channels (channels: write).
The second successful solution was to connect the database in which we store the status of the execution of the actions of the commands. Now, having started a new release using the “/ startregress” command, we are not afraid that something will fall, and when you call it again, the commands will be executed from the beginning a second time. We don’t need to create a new channel in Slack every time, do a pull request, etc. We created a channel in Slack - we recorded the status of successful execution in the database and we will not return to this action.
So, now by clicking one button we create a release channel and invite everyone there whose tasks will be released.
Next in line were integration with Github and TeamCity. We work on Gitflow. When we want to release, we take the DEV branch, the working = green = on which the tests pass, and from it we create the release branch.
To do this, our bot goes to TeamCity, looks there to launch a test run for the DEV branch. If the tests are green, like grass at home, go ahead on GitHub, create a release branch and a pull request!
When creating a pull request, we need to add a description of the changes that we are rolling out. Then Kaiten comes to our aid. Our bot creates a column with the name of the release, takes tasks from the DEV column and moves them to the release. So we know and see what will be ground with us. After moving, the bot copies the names of the cards and adds them to the pull request description with reference to the cards themselves. Now we can see for each release what tasks it went out and, by opening the card via the link, find out all the details on these tasks.
It is almost possible to release, it remains only to thoroughly test our changes. To do this, the release branch is deployed to an environment close to production (we call it stage), and is tested after the deployment. The deployment and tests are assembled in one pipeline in TeamCity, and our task is to launch it and wait, cross-fingers, that the tests will work fine. The bot launches a pipeline during the start of the regression.
Let me remind you that we started with the fact that all this was done manually. Gritting his fists, Relizmen switched on links and tools: TeamCity, Kaiten, Slack, Github and that's not all! And now we have a whole set of actions from which the first team for the bot is already formed. The releaseman dials “/ startregress” and, leaning back in his chair, watches as our bot:
We analyze the entire release process, write down how much time each stage of the release takes. This gives the development and business an understanding of what time is wasted on and what inhibits us in delivering new features to users. We are sitting for two days and can’t run the tests ?! So something is wrong with our tests, you need to give them more time and attention. Previously, performing the actions of our checklist, we visited Google Sheets at least 5 times. Each time you enter one date there and set the time.
Ok Google, we’ll automate you too! To easily and naturally work with tables, we connect the nuget package Google.Apis.Sheets.v4 to the project of our bot. And then everything happens according to the scheme similar to other services: we send a request in which we say what value to insert into which cell. This query looks like this:
Having set up integration with Google, we prepared the second command of our bot “/ update” and this is what it does:
The next step is the releaseman rolls the release. Now the calculation is done manually. Having laid out one country, we watch how the release behaves in battle. After making sure that everything is good according to the logs and monitoring tools of Kibana and Grafana, we post the next country. This process is not so easy to automate. But here there is something to improve, albeit not with the help of a bot. For example, before, each time we asked the infrastructure team if it was possible to release. Because when we were going to do this, other work could take place on our servers and our release would have come in inappropriately.
We have put together a meeting to optimize the release process. One of the solutions was that now the releaseman simply looks at the status in one of the Slack channels, where the infrastructure posts permission to take off. This is more convenient than constantly asking the same thing and in 90% of cases getting the answer “You can.”
For some reason, such seemingly elementary things did not immediately come to mind. Special thanks should be said to new developers in the company. Having come to us, sooner or later they became relismen. For the guys who worked with us for a long time, the process did not seem to be something complicated, rather something familiar. New team members turned our attention to growth points and took an active part in organizing work on improvements.
In the meantime, we have come to the last stage. Our release liner has landed, only one “/ releasecomplete” team separates us from the end. What is she doing:
To summarize, I would like you to ask yourself the question: do you have boring routine processes? Are you still doing them with your hands? What prevents you from ending this once and for all? Gather a meeting, review processes, throw away everything you don’t need and it’s just become a ritual. By automating everything you need, you will begin to get the joy of what hurt before, and save up to the heap by speeding up the release or any other processes.
Conservative routine and method of work, slavish adherence to an established pattern, turned into a mechanical habit. 6 letters.
There is such work that you do not want to do, but you need to. This article will not have large inserts with code and instructions on how to create your bot from scratch. I’d better tell you how the release process works in our Dodo Pizza, how we automate and speed it up, giving some of the boring routine to our bot written in C #. I will pay attention to the functionality and logic of the bot. It will be cool if this material inspires you to write your assistant who will make life easier for you and your colleagues.
A couple of years ago we released one release once a week. In May 2018, having a maximum rate of 147 hours for release, we set a goal to release every day. Now our minimum: four hours to release, but this does not happen often. We want to fix the record and be able to reduce the whole process to the press of a single button.
Release cycle
Now in Dodo IS seven teams take turns releasing a release. One person from the team becomes “lucky” - relizmen. Relizmen as an airplane pilot: he has a bunch of levers and devices that he needs to skillfully wield in order to roll out next updates. We thought and decided: "It's time to do an autopilot, and we’d better spend our time on something more exciting than filling out boring tables with statistics and tracking auto test runs."
So, in order to become a relisman, it’s necessary that the turn comes to your team. So that everything was clear and no one was confused, we stuck stickers with the names of teams on the wall. The release team receives an honorary crown, which we move with handles each time.
After receiving the
Having gathered our will into a fist, we decided that it was enough to endure it. It's time to give some of the boring routine to the bot. Who if not us? When, if not now?
What we managed to automate
The decision is made, it's time to start designing our autopilot! Having found the Kaiten API here: https://faq.kaiten.io/docs/api , with just one request, we created a card for the new relizmen.
// Делаем POST запрос на создание карточкиvar createCardRequest = new RestRequest("https://<domain>.kaiten.io/api/latest/cards/", Method.POST);
// Добавляем информацию для авторизацииAddBasicAuth(createCardRequest);
// В теле запроса укажем на какой доске со-дать карточку и как ее назвать
createCardRequest.AddJsonBody(
new
{
board_id = _kaitenSettings.ReleasesBoardId,
column_id = _kaitenSettings.ReleasesColumnId,
lane_id = _kaitenSettings.ReleasesLaneId,
title = $"release-{nextReleaseNumber}"
}
);Now this card needs to be handed to the team that will release the next.
The logic here is:
- The previous relizmen completes the release by typing a command for the bot in Slack.
- The bot takes the Relisman ID in Slack and looks for which development team our lucky guy is on. To do this, the bot runs through the Slack user groups and looks at which of them our releisen consists of.
- Having found the group, the bot looks at which team will release next and sends an alert to the general chat. In the message, the bot carefully gives a link to a checklist already created so that you don’t go anywhere for it.
Perfectly! Now we have a clue what to do next. We open the checklist, look at it: “Create a channel for release in Slack, invite all the teams there whose changes are in the release and find out if they will need manual testing.” It remains to teach this to our bot.
Open the Slack API documentation here https://api.slack.com and look for a method for creating a channel there.
As you can see, in Slack, as in other tools, there is nothing complicated. It all comes down to sending one POST request with two required parameters (these are the ones opposite which it says required). In the request, we need to pass the name of the created channel (name parameter) and the authorization token (token parameter). Pay attention to the line "Works with: Token type - user, required scope (s) - channels: write".
Slack has two types of tokens: user - issued to users and bot - issued to applications. When issuing a token, we determine what rights to vest its owner. To create channels, we need a user token (token type - user), which has rights to write channels (channels: write).
I want to note one nuance of our sending messages to Slack. Initially, we did not think about what we would do if something went wrong. We recruit a team in Slack, and it performs all the tasks that we put into it. And what will happen if on one of the tasks the team falls? In our case, the answer was: "nothing." And that's bad. The solution for us was to write to the release chat what action is currently being performed, and if the command did not complete, report an error to the chat and log the error.
The second successful solution was to connect the database in which we store the status of the execution of the actions of the commands. Now, having started a new release using the “/ startregress” command, we are not afraid that something will fall, and when you call it again, the commands will be executed from the beginning a second time. We don’t need to create a new channel in Slack every time, do a pull request, etc. We created a channel in Slack - we recorded the status of successful execution in the database and we will not return to this action.
So, now by clicking one button we create a release channel and invite everyone there whose tasks will be released.
Next in line were integration with Github and TeamCity. We work on Gitflow. When we want to release, we take the DEV branch, the working = green = on which the tests pass, and from it we create the release branch.
To do this, our bot goes to TeamCity, looks there to launch a test run for the DEV branch. If the tests are green, like grass at home, go ahead on GitHub, create a release branch and a pull request!
When creating a pull request, we need to add a description of the changes that we are rolling out. Then Kaiten comes to our aid. Our bot creates a column with the name of the release, takes tasks from the DEV column and moves them to the release. So we know and see what will be ground with us. After moving, the bot copies the names of the cards and adds them to the pull request description with reference to the cards themselves. Now we can see for each release what tasks it went out and, by opening the card via the link, find out all the details on these tasks.
It is almost possible to release, it remains only to thoroughly test our changes. To do this, the release branch is deployed to an environment close to production (we call it stage), and is tested after the deployment. The deployment and tests are assembled in one pipeline in TeamCity, and our task is to launch it and wait, cross-fingers, that the tests will work fine. The bot launches a pipeline during the start of the regression.
Let me remind you that we started with the fact that all this was done manually. Gritting his fists, Relizmen switched on links and tools: TeamCity, Kaiten, Slack, Github and that's not all! And now we have a whole set of actions from which the first team for the bot is already formed. The releaseman dials “/ startregress” and, leaning back in his chair, watches as our bot:
- creates a channel in the messenger
- calls there everyone you need
- checks if a pull request can be created
- creates a pull request and fills its description
- creates a release column on an electronic board and moves task cards there
- launches a pipeline that will release the release branch to the environment and run tests there
We analyze the entire release process, write down how much time each stage of the release takes. This gives the development and business an understanding of what time is wasted on and what inhibits us in delivering new features to users. We are sitting for two days and can’t run the tests ?! So something is wrong with our tests, you need to give them more time and attention. Previously, performing the actions of our checklist, we visited Google Sheets at least 5 times. Each time you enter one date there and set the time.
Ok Google, we’ll automate you too! To easily and naturally work with tables, we connect the nuget package Google.Apis.Sheets.v4 to the project of our bot. And then everything happens according to the scheme similar to other services: we send a request in which we say what value to insert into which cell. This query looks like this:
public void InsertEntry(string cellAddress, string value)
{
// Задаем из настроек лист в который будем вставлять значение - _googleSettings.SheetName и адрес ячейки в которую вставляем значение - cellAddressvar range = $"{_googleSettings.SheetName}!{cellAddress}";
var valueRange = new ValueRange();
// Добавляем значпение которе нужно вставить в ячейку - valuevar objectsList = new List<object> {$"{value}"};
valueRange.Values = new List<IList<object>> {objectsList};
// Используя метод библиотеки Google.Apis.Sheets.v4 отправляем запрос на вставку значения в ячейку таблицы и идентификатором SpreadsheetIdvar insertEntryRequest =
_service.Spreadsheets.Values.Update(valueRange, _googleSettings.SpreadsheetId, range);
insertEntryRequest.ValueInputOption =
SpreadsheetsResource.ValuesResource.UpdateRequest.ValueInputOptionEnum.USERENTERED;
insertEntryRequest.Execute();
}Having set up integration with Google, we prepared the second command of our bot “/ update” and this is what it does:
- adds an empty string to insert values into it
- goes to GitHub, looks when they created the release branch and adds the date of its creation to the tablet
- from TeamCity takes data on the first start of the pipeline and information about when the pipeline successfully finished
The next step is the releaseman rolls the release. Now the calculation is done manually. Having laid out one country, we watch how the release behaves in battle. After making sure that everything is good according to the logs and monitoring tools of Kibana and Grafana, we post the next country. This process is not so easy to automate. But here there is something to improve, albeit not with the help of a bot. For example, before, each time we asked the infrastructure team if it was possible to release. Because when we were going to do this, other work could take place on our servers and our release would have come in inappropriately.
We have put together a meeting to optimize the release process. One of the solutions was that now the releaseman simply looks at the status in one of the Slack channels, where the infrastructure posts permission to take off. This is more convenient than constantly asking the same thing and in 90% of cases getting the answer “You can.”
For some reason, such seemingly elementary things did not immediately come to mind. Special thanks should be said to new developers in the company. Having come to us, sooner or later they became relismen. For the guys who worked with us for a long time, the process did not seem to be something complicated, rather something familiar. New team members turned our attention to growth points and took an active part in organizing work on improvements.
In the meantime, we have come to the last stage. Our release liner has landed, only one “/ releasecomplete” team separates us from the end. What is she doing:
- mergit pull request with release to master branch
- removes the release branch
- creates release description on github
- archives the release channel in Slack
- moves cards in Kaiten from the “release -...” column to the “Done” column and deletes the release column
- passes the baton, inviting to release the next command
To summarize, I would like you to ask yourself the question: do you have boring routine processes? Are you still doing them with your hands? What prevents you from ending this once and for all? Gather a meeting, review processes, throw away everything you don’t need and it’s just become a ritual. By automating everything you need, you will begin to get the joy of what hurt before, and save up to the heap by speeding up the release or any other processes.