Common Bash programming errors

Original author: Greg Wooledge
  • Transfer
The quality of the scripts used to automate and optimize the system is the key to its stability and longevity, and also saves the time and nerves of the administrator of this system. Despite the seeming primitiveness of bash as a programming language, it is full of pitfalls and tricky trends that can significantly spoil the mood for both the developer and the administrator.

Most of the manuals available are dedicated to how to write. I’ll tell you how to NOT write :-)

This text is a free translation of the Bash pitfalls wiki page as of December 13, 2008. Due to the wiki nature of the source, this translation may differ from the original. Since the volume of the text is too large to be published in its entirety, it will be published in parts.


1. for i in `ls * .mp3`


One of the most common errors in bash scripts is loops like this:

for i in `ls * .mp3`; do # False!
    some command $ i # False!
done


This will not work if there are spaces in the name of one of the files, as the result of the substitution of the team ls *.mp3is divided into words. Suppose we have a file in the current directory 01 - Don't Eat the Yellow Snow.mp3. The cycle forwill be held on every word of the name of the file and $iwill take the values: "01", "-", "Don't", "Eat", "the", "Yellow", "Snow.mp3".

Quoting the entire command also does not work:

for i in "` ls * .mp3` "; do # False!
    ...


The entire output is now considered as one word, and instead of going through each of the files in the list, the cycle will be executed only once, and at the same time it iwill take on the value, which is a concatenation of all file names through a space.

In fact, use is lscompletely redundant: it is an external command that is simply not needed in this case. How is it right then? And like this:

for i in * .mp3; do # Much better, but ...
    some command "$ i" # ... see catch # 2
done


Let the bash substitute the file names. Such a substitution will not lead to the division of the string into words. Each file name that matches the pattern *.mp3will be considered as one word, and a cycle will go through each file name once.

For more information, see paragraph 20 of the Bash FAQ .

An attentive reader should have noticed the quotation marks in the second line of the above example. This smoothly brings us to the trick number 2.

2. cp $ file $ target


What is wrong with this team? It seems to be nothing special, unless you absolutely know for sure that the variables $fileand $targetdo not contain spaces or wildcards.

But if you don’t know what kind of files you come across, or you are paranoid, or simply trying to follow a good bash programming style, then you will enclose the names of your variables in quotation marks so as not to put them into words.

cp "$ file" "$ target"


Without double quotes, the script will execute the command cp 01 - Don't Eat the Yellow Snow.mp3 /mnt/usb, and you will get a lot of type errors cp: cannot stat `01': No such file or directory. If the variable values $fileeither $targetcontain the *,?, [..] or (..) characters used in wildmats, then if there are files matching the template, the variable values ​​will be converted to the names of these files . Double quotes solve this problem, unless "$file"it starts with a hyphen -, in which case it cpthinks that you are trying to give it another command line option.

One workaround is to insert a double hyphen ( --) between the command cpand its arguments. A double hyphen tells cpyou to stop searching for options:

cp - "$ file" "$ target"


However, you may come across one of the systems in which such a trick does not work. Or the command you are trying to execute does not support the option --. In this case, read on.

Another way is to make sure that file names always begin with the directory name (including ./for the current one). For instance:

for i in ./*.mp3; do
    cp "$ i" / target
    ...


Even if we have a file whose name begins with "-", the template substitution mechanism ensures that the variable will contain something ./-foo.mp3that is absolutely safe to use with cp.

3. [$ foo = "bar"]



In this example, quotation marks are not correctly aligned: in bash, there is no need to enclose a string literal in quotation marks; but you should definitely quote the variable if you are not sure that it does not contain spaces or wildcards.

This code is erroneous for two reasons:

1. If the variable used in the condition [does not exist or is empty, the line

[$ foo = "bar"]


will be perceived as

[= "bar"]


which will cause a "unary operator expected" error. (The operator "=" is binary, not unary, so the command [will be shocked by this syntax)
2. If the variable contains a space inside itself, it will be divided into different words before it is processed by the command [:
[multiple words here = "bar"]

Even if you personally think that this is normal, this syntax is erroneous.

It will be correct like this:

["$ foo" = bar] # is close!


But this option will not work if $ foo starts with -.

In bash, a keyword can be used to solve this problem [[, which includes and greatly extends the old command test(also known as [)

[[$ foo = bar]] # right!


Inside [[and it is ]]no longer necessary to quote the names of variables, since variables are no longer broken into words and even empty variables are processed correctly. On the other hand, even if you put them in quotes once again, this will not hurt anything.

You may have seen code like this:

[x "$ foo" = xbar] # right too!


A hack x"$foo"is required in code that should work in non-supporting shells [[, because if it $foostarts with -, the command [will be disoriented.

If one of the parts of the expression is a constant, you can do this:
[bar = "$ foo"] # so right too!


The team [does not care that the expression to the right of the = sign starts with -. She just uses this expression as a string. Only the left side requires such close attention.

4. cd `dirname" $ ​​f "`



So far we are basically talking about the same thing. In the same way as with the expansion of the values ​​of variables, the result of the command substitution is split into words and the expansion of file names (pathname expansion). Therefore, we must enclose the command in quotation marks:

cd "` dirname "$ f" `"


What is not entirely obvious here is the sequence of quotation marks. A C programmer might suggest that the first and second quotation marks are grouped, as well as the third and fourth. However, in this case this is not so. Bash treats double quotes inside a command as the first pair, and outer quotes as the second.

In other words, the parser treats back quotes ( `) as a level of nesting, and the quotes inside it are separated from external ones.

The same effect can be achieved using the preferred syntax $():

cd "$ (dirname" $ ​​f ")"


The quotes inside $()are grouped.

5. ["$ foo" = bar && "$ bar" = foo]



Cannot be used &&inside the "old" team testor its equivalent [. The bash parser sees &&outside the brackets and breaks your command into two, before and after &&. Better use one of the options:

[bar = "$ foo" -a foo = "$ bar"] # Right!
[bar = "$ foo"] && [foo = "$ bar"] # Also right!
[[$ foo = bar && $ bar = foo]] # That's right too!


Please note that we swapped the constant and the variable inside [- for the reasons discussed in the previous paragraph.

The same applies to ||. Use [[either -oor two commands [.

6. [[$ foo> 7]]


If the operator is >used internally [[ ]], it is considered as an operator for comparing strings, not numbers. In some cases, this may or may not work (and this will happen just when you least expect it). If it >is inside [ ], it’s even worse: in this case, it is redirecting the output from the file descriptor with the specified number. An empty file with a name will appear in the current directory 7, and the command testwill succeed unless the variable $foois empty.

Therefore, the operators> and < cannot be used to compare numbers inside [ .. ]or [[ .. ]].

If you want to compare two numbers, use (( )):

((foo> 7)) # That's right!


If you are writing for Bourne Shell (sh) and not for bash, the correct way is this:

[$ foo -gt 7] # That's right too!


Note that a command test ... -gt ...will throw an error if at least one of its arguments is not an integer. Therefore, it no longer matters whether the quotation marks are correctly placed: if the variable is empty, or contains spaces, or its value is not an integer, an error will occur in any case. Just check the value of the variable carefully before using it in a command test.

Double square brackets also support this syntax:

[[$ foo -gt 7]] # That's right too!


7. count = 0; grep foo bar | while read line; do ((count ++)); done; echo "number of lines: $ count"


At first glance, this code looks fine. But in fact, the variable $countwill remain unchanged after exiting the loop, much to the surprise of the bash developer. Why it happens?

Each command in the pipeline is executed in a separate subshell, and changes to the variable inside the subshell do not affect the value of this variable in the parent shell instance (i.e., in the script that called this code).

In this case, the loop foris part of the pipeline and runs in a separate subshell with its copy of the variable $count, initialized by the value of the variable $countfrom the parent shell: “0”. When the loop ends, the copy used in the loop is $countdiscarded and the command echoshows the unchanged initial value $count("0").

There are several ways around this.

You can execute the loop in your subshell (a bit crooked, but it’s simpler and more understandable and works in sh):

# POSIX compatible
count = 0
cat / etc / passwd | (
    while read line; do
        count = $ ((count + 1))
    done
    echo "total number of lines: $ count"
)


To completely avoid creating a subshell, use redirection (in Bourne shell (sh), a subshell is also created for redirection, so be careful, this trick will work only in bash):

# only for bash!
count = 0
while read line; do
    count = $ (($ count + 1))
done </ etc / passwd
echo "total number of lines: $ count"


The previous method works only for files, but what if I need to process the output of a command line by line? Use process substitution:
while read LINE; do
    echo "-> $ LINE"
done <<(grep PATH / etc / profile)


A couple more interesting ways to solve the problem with sub-shells are discussed in the Bash FAQ # 24 .

8. if [grep foo myfile]



Many people are embarrassed by the practice of putting square brackets after ifand beginners often get the false impression that they [are part of the conditional syntax, just like the brackets in the conditional constructs of C.

However, such an opinion is a mistake! An opening square bracket ( [) is not part of the syntax, but a command that is the equivalent of a command test, except that the closing argument must be the last argument to this command ].

Syntax if:

if COMMANDS
then
    COMMANDS
elif COMMANDS # optional
then
    COMMANDS
else # optional
    COMMANDS
fi


As you can see, there are no [or ors in the if syntax [[.

Once again, [this is a command that takes arguments and returns a return code; like all normal commands, it can display error messages, but, as a rule, does not produce anything in STDOUT.

ifexecutes the first set of commands, and depending on the return code of the last command from this set, determines whether the block of commands from the "then" section will be executed or the script will continue to execute.

If you need to make a decision depending on the output of the command grep, you don’t need to enclose it in parentheses, brackets or braces, backticks, or any other syntax element. Just write grepas a command after if:

if grep foo myfile> / dev / null; then
    ...
fi


Note that we discard the standard output grep: we do not need the search result, we just want to know if a line is present in the file. If it grepfinds a string, it returns 0, and the condition is satisfied; otherwise (there is no line in the file) it grepreturns a value other than 0. In GNU grep, redirection >/dev/nullcan be replaced with an option -qthat says grep' y that you do not need to output anything.

9. if [bar = "$ foo"]



As explained in the previous paragraph, [this is a team. As with any other command, bash assumes that the command is followed by a space, then the first argument, then again a space, etc. Therefore, you can’t write everything without spaces! That's right like this:

if [bar = "$ foo"]


bar, =, "$foo"(After the substitution, but without division into words) and ]are the arguments of the team [, so between each pair of arguments necessary space must be present to shell could determine where any argument begins and ends.

10. if [[a = b] && [c = d]]



Again the same mistake. [- a command, not a syntax element between a ifcondition and, moreover, not a means of grouping. You cannot take C syntax and convert it to bash syntax by simply replacing parentheses with square ones.

If you want to implement a difficult condition, here is the correct way:

if [a = b] && [c = d]


Notice that here we have two commands after if, combined by the && operator. This code is equivalent to a command like this:

if test a = b && test c = d


If the first test command returns a value false(any nonzero number), the condition body is skipped. If it returns true, the second condition is true; if it returns true, then the condition body is satisfied.

To be continued.

The first publication of this translation took place on the pages of my blog .

Also popular now: