Linux: redirection

If you have already mastered the basics of the terminal, perhaps you are ready to combine the learned commands. Sometimes executing shell commands one by one is quite enough to solve a certain task, but in some cases, entering command after command is too tedious and irrational. In such a situation, we need some special characters, such as angle brackets. For the shell, the Linux command interpreter, these extra characters are not a waste of screen space. They are powerful teams that can link together various pieces of information, share what was previously solid, and do much more. One of the simplest, and at the same time, powerful and widely used features of the shell is the redirection of standard input / output streams.

Three standard input / output streams

In order to understand what we are going to talk about here, it is important to know where the data that can be redirected comes from and where it goes. On Linux, there are three standard input / output data streams.

The first is the standard input stream. In the system, this is flow # 0 (since in computers the score usually starts from zero). Thread numbers are also called descriptors. This stream represents some information transmitted to the terminal, in particular, instructions transmitted to the shell for execution. Typically, data falls into this stream during user input from the keyboard.

The second stream is the standard output stream, it is assigned the number 1. This is the data stream that the shell displays after performing some action. Usually, these data fall into the same window of the terminal where the command was entered that caused their appearance.

And finally, the third stream is a standard error stream, it has a handle 2. This stream is similar to the standard output stream, since usually what gets into it appears on the terminal screen. However, it, in essence, differs from the standard output, as a result, these flows, if desired, can be controlled separately. This is useful, for example, in the following situation. There is a team that processes a large amount of data, performing a complex and error-prone operation. It is necessary that the useful data that this command generates does not mix with error messages. This is achieved through separate redirection of output streams and errors.

As you probably already guessed, I / O redirection means working with the above flows and redirecting data to where the programmer needs. This is done using symbols >and <in various combinations, the use of which depends on where, in the end, the redirected data should go.

Redirecting standard output

Suppose you want to create a file in which the current date and time will be written. The matter simplifies the fact that there is a team, well-named date, which returns what we need. Typically, commands output data to standard output. In order for this data to appear in the file, you need to add a character >after the command, in front of the name of the target file. Before and after >you need to put a space.

When using redirection, any file specified after >will be overwritten. If there is nothing valuable in the file and its contents can be lost, it is permissible to use an existing file in our design. Usually it is better to use in this case the name of a file that does not yet exist. This file will be created after the command. Let's call himdate.txt. A file extension after a period usually does not play a special role, but extensions help to maintain order. So here is our team:

$ date > date.txt

This is not to say that this team itself is incredibly useful, however, based on it, we can already do something more interesting. Let's say you want to find out how the routes of your traffic going through the Internet to a certain endpoint are changing, recording the corresponding data daily. In solving this problem, a team will help traceroute, which provides details about the traffic route between our computer and the endpoint specified when the command is called in the form of a URL. The data includes information about all routers through which traffic passes.

Since we already have a file with the date, it will be quite justified to simply attach data received from to this file traceroute. In order to do this, you must use two characters>set one by one. As a result, a new command that redirects the output to a file but does not overwrite it, but adds new data after the old, will look like this:

$ traceroute google.com >> date.txt

Now we just need to change the file name to something more meaningful, using the command mv, which, as the first argument, is transferred to the original file name, and as a second, a new one:

$ mv date.txt trace1.txt

Redirecting a standard input stream

Using the sign <instead, >we can redirect standard input, replacing it with the contents of the file.

Suppose there are two files: list1.txtand list2.txt, each of which contains an unsorted list of strings. Each list has elements that are unique to it, but some of the elements in the list are the same. We can find the lines that are in both the first and second lists by applying the command comm, but before using it, the lists must be sorted.

There is a command sortthat returns the sorted list to the terminal without saving the sorted data to the file from which it was taken. You can send a sorted version of each list to a new file using the command >, and then use the commandcomm. However, this approach will require at least two commands, although the same can be done on one line without creating unnecessary files.

So, we can use the command <to redirect the sorted version of each file to the command comm. Here's what we got:

$ comm <(sort list1.txt) <(sort list2.txt)

Parentheses here have the same meaning as in mathematics. The shell first processes the commands in brackets, and then everything else. In our example, we first sort the lines from the files, and then what happened is passed to the command comm, which then displays the result of the comparison of the lists.

Redirecting the standard error stream

And finally, let's talk about redirecting the standard error stream. This may be necessary, for example, to create error log files or to combine error messages and data returned by some data command in one file.

For example, what if you need to search the entire system for information on wireless interfaces that are available to users who do not have superuser rights? In order to do this, you can use a powerful team find.

Usually when a regular user runs a commandfindthroughout the system, it outputs to the terminal both useful data and errors. At the same time, there are usually more of the latter than the former, which complicates finding what is needed in the output of the command. Solving this problem is quite simple: just redirect the standard error stream to a file using the 2> command (recall, 2 is a standard error stream handle). As a result, only what the command sends to standard output will appear on the screen:

$ find / -name wireless 2> denied.txt

What if I need to save the results of the command to a separate file without mixing this data with error information? Since the streams can be redirected independently of each other, we can add the command to redirect the standard output stream to the file at the end of our construction:

$ find / -name wireless 2> denied.txt > found.txt

Please note that the first angle bracket comes with the number - 2>, and the second without it. This is because the standard output has a handle of 1, and the command >involves redirecting the standard output if the handle number is not specified.

And finally, if you want everything that the command prints to fall into one file, you can redirect both streams to the same place using the command &>:

$ find / -name wireless &> results.txt

Summary

Here we examined only the basics of the thread redirection mechanism in the Linux command line interpreter, however even the little that you learned today gives you almost unlimited possibilities. And by the way, like everything else that concerns work in the terminal, mastering the flow redirection requires practice. Therefore, we recommend that you start your own experiments with >and <.

Dear readers! Do you know interesting examples of the use of thread redirection in Linux that will help beginners to get better with this trick of working in the terminal?