Console to the masses. Go to the bright side. Automation of routine tasks

Introduction

Cars will always be faster, no matter how productive we are or how fast we recruit teams. The harsh truth of life. On the other hand, if we perform the same action many times, then why not make cars suffer. Write a script on bash(your favorite programming language) and call this script every time, rather than typing monotonous commands that take so much time, effort and energy. And while the script will do its job, we can dream about how spaceships plow the expanses of our universe.

In a previous article, we covered the basics of programming on bash. Today we will put the acquired knowledge into practice.

Automation plan

1. Fast diff
2. Fast diff+ Jira API
3. Cleaning _dist
4. Up a large number of repositories
5. Cloning a large number of repositories
6. Useful aliases

This list includes tasks that I perform daily several times a day, or even an hour. In general, automation of routine processes is a creative and purely personal process.
You can automate everything you can come up with. I hope that at the end of the article you will have your own plan for automation and you will make cars suffer. Make yourself a cup of aromatic coffee and sit back. An exciting journey into the world of automation means is waiting for you bash.

Quick diff

In our work on the project we use Git . Creating diffis a fairly common task. To create a diffspecific branch, you need to run the following command:

git diff origin/master origin/ > "${HOME}/diff/diff-.diff"

- name of the branch for which you want to create diff

Disadvantages of this approach

1. Each time the team needs to be typed manually
2. High probability of error during the recruitment process
3. Difficult to remember

These shortcomings can be easily solved with bash. Ideally, everything should work like this:

1. Typed a team
2. They gave her the name of the branch
3. Received diff

Final view of the team

gdd 

Automate

Now instead of a long line of commands, just type on the keyboard . If you forgot to specify the name of the branch, this script will kindly inform us of this../fast_diff.sh

Final touch

Stop, you say. But what about the final look of the team. Indeed, in the current form, this script is not very convenient, we are tied to the directory in which it is located.

Let us consider in more detail how to make a separate command for the executable file, rather than writing a relative / absolute path to it each time.

Each user ~has a subdirectory in his home directory ( ) bin. If there is none, then you can create it. It can store executable files. The convenience is that such files are accessible by name and you do not need to specify a relative / absolute path to them. In this directory I put the file gddthat is responsible for creating diff:

#!/bin/bash
"${HOME}/htdocs/rybka/tools/fast_diff.sh" "$@"

A few important points:

1. The file is not accepted to specify the extension.
2. An attribute must be explicitly specified for the file .x (chmod +x )
   
3. If the directory is binnot in a variable $PATH, you need to add it explicitly PATH="${PATH}:${HOME}/bin".

To make this file available, restart the terminal. Now, to create diff, just run the following command:

gdd 

It is not very convenient to create a separate file in the directory binfor each command all the time . We can optimize this using symbolic links:

ln -s "${HOME}/htdocs/rybka/tools/fast_diff.sh" gdd

Fast diff + Jira API

If you use the Jira task manager or any other task manager that provides the API in your work , you can go further. For example, you can use the Jira API to attach diffto a specific task. For this we need cURL as well .

Decision algorithm

1. We call a script
2. We transfer idtasks
3. If the idtask is not transferred, display a message to the user
4. If everything is correct, we generate diffand attach it to the task

Final view of the team

gdd_jira 

Automate

As you may have noticed, the branch name does not need to be specified. We get it with the help of simple manipulations with git commands:

branch=$(git rev-parse --abbrev-ref HEAD)

Cleaning _dist

To get started, let's figure out what the directory is responsible for _dist. This place, which lists the files CSS, JavaScript, all kinds of patterns ( Jade / Pug , Handlebars , et al.), And other files after running the build system ( Grunt , Gulp , et al.). This directory does not have to be named _dist. Variations are possible.


For one of the projects we use Grunt. Quite often, our team encounters a problem that Grunt does not always notice changes in some files (the problem is mainly with Less files). To fix this situation, you need to clear the directory_distfor one of the topics or for all topics at once. Yes, this problem can be solved with the help of Grunt. You can even delete this directory manually all the time. But it will not be as effective and convenient as in the case of bash. The number of these directories ( _dist) is not one or two, or even ten or twenty. A lot of them. The main requirement for the script is not to apply unnecessary wrappers and / or dependencies unnecessarily.

Consider the option without application bash. We use all the power of the shell to solve this problem:

find  -type d -name "_dist" | xargs rm -rfv

- the path to the directory where all topics are located.

The disadvantages of this approach are approximately the same as I cited in the case of the creation task diff. Plus, in this case, there is no way to indicate for which specific topic you need to delete the directory _dist.

Decision algorithm

1. We call a script
2. If the topic name has not been transferred, delete the directory _distin all topics
3. If the topic name was transferred, we delete the directory _distin a specific topic

Final view of the team

clean_dist []

Automate

Up a large number of repositories

Imagine that you are working with a large project. This project has a directory reserved for third-party repositories that you do not develop, but are required to keep them up to date. Of course, if there are two or three of these repositories, then this is not such a big problem. Although here I would argue. And if you have such repositories 10-15, and this figure is constantly growing. As a result, you do not have time to follow them or spend a disproportionate amount of time on support. Why not automate this task.

Decision algorithm

1. Go to the directory with the repository
2. Check that the repository is on a branch master
3. If the repository is not on a branch master, dogit checkout
4. Make git pull

An important point . Even if the repository is on a branch master, we cannot be sure that it is up to date. Based on this, we do it git pullanyway.

Final view of the team

up_repo

Automate

Cloning a large number of repositories

This task is closely related to the previous point of automation. In order for the end user to be able to use the previous team in practice, it is necessary to provide a set of third-party repositories that will be located in the directory bash/core/vendors, and of which the user, by and large, does not need to know anything. By analogy with npm modules, this set of repositories should not come with the main repository. All the user needs to do is execute the command and wait until the repositories are cloned.

Decision algorithm

1. The list of repositories is given as an array
2. We start a loop on this array
3. We pay special attention if one vendor has more than one repository
4. We carry out additional checks
5. We carry out git clone

Final view of the team

clone_repo

Automate

Useful aliases

I have a few questions for readers. You must answer honestly to yourself. How often do you use this command?

git branch

What about this team?

git status

But this team?

git push origin 

How about this team?

ps aux | grep 

That's right, this list can be continued endlessly and everyone has his own. And then, unexpectedly, an insight comes to us:



That's right. For teams that you use often, create aliases. Here is just a small list of the aliases that I use:


To check which aliases you have set, just run the command aliaswithout parameters.

Where to put aliases

In most cases, for such purposes, they use a file .bashrcthat is located in the user's home directory. There is also a file called .gitconfig, in which you can add aliases to work with git.

Do not change aliases late at night

Aliases are a powerful tool. Only here as with passwords. Do not change aliases late at night. One fine night, I changed one of the aliases and forgot. The next day I spent half a day figuring out why nothing was working.

Instead of a conclusion

As soon as I started to understand the basics of programming on bash, the first thought that visited me was: “Stop, this is needed more for system administrators ...”. But at the same time, I realized that I needed this knowledge to at least somehow save myself from the daily routine tasks. Now I can say with confidence that this knowledge is needed not only by system administrators. They will be useful to anyone who at least somehow interacts with a remote server or works on OS *nixsimilar systems. For users who work on Windows OS, this knowledge is also useful ( Bash on Ubuntu on Windows , Windows and Ubuntu Interoperability) In the simplest case, a script is nothing more than a simple list of system commands written to a file. Such a file can facilitate your workdays and eliminate the need to perform routine tasks manually.

Useful links on some of the features bashthat were used in the examples:

1. Input / output redirection
2. Functions
3. Arrays
4. Double parentheses
5. Chaining teams (conveyor)
6. Completion and completion code
7. Aliases
8. How to correctly add paths to the $ PATH variable

That's all. Thanks for attention. Who read to the end, special thanks.