The book “Shell scripts. Linux, OS X, and Unix. 2nd edition
Shell scripts help system administrators and programmers automate routine tasks ever since the first computers appeared. Since the release of the first edition of this book in 2004, much has changed, but the bash shell has only strengthened its leadership position. Therefore, the ability to use all its capabilities is becoming an urgent need for system administrators, engineers and enthusiasts. This book describes common problems you might encounter, such as building software or coordinating other programs. And the solutions are given in such a way that they can easily be taken as a basis and extrapolated to other similar tasks.The purpose of this book is to demonstrate practical techniques for programming bash scripts and introduce the most common utilities in short and compact examples without going into too much detail. Experiment with these scenarios - break, fix and adapt them to your needs in order to understand how they work. Only in this way can you solve the most complex tasks.
What disappeared in the second edition
This book describes the typical difficulties that you may encounter when trying to write a portable automated solution, for example, to build software or coordinate the actions of other programs, and how to overcome them. The solutions in the book are presented so that you can take them as a basis and extrapolate to other similar tasks. For example, in chapter 1, we write a portable version of the echo program as a small wrapper script. Many system administrators may find this particular scenario useful, but the basic idea is to create a wrapper script that guarantees consistent behavior across platforms. Later in the book, we will analyze some interesting features of bash scripts and typical utilities available on Unix systems that give us the widest possibilities.
This book is for you if ...
Bash remains the main tool for anyone working with servers or workstations running on Unix-like operating systems, including web developers (many of whom are developing in OS X and deploying their applications on Linux servers) , analysts, mobile app developers, and programmers. In addition, there are more and more enthusiasts launching Linux on their open architecture microcomputers, such as the Raspberry Pi, to automate household appliances. Shell scripts are great for all of these cases.
The scripts presented in the book will certainly be useful to those who wish to expand their already considerable bash experience by studying practical examples, and those who use the terminal or shell scripts only occasionally. If you belong to the second camp, you will probably need to refresh your knowledge or supplement it by reading the introduction to the advanced features of bash.
This book is not a textbook! Our goal is to demonstrate the practical techniques for programming bash scripts and introduce common utilities to (in most) short and compact examples, but we do not describe them line by line. We explain only the most basic parts, and experienced script writers will be able to understand for themselves how the rest of the code works by reading it. We hope that you, dear reader, will experiment with these scenarios - break them, fix them and adapt to your needs - in order to understand how they work. Our main goal is to show how to solve typical tasks, such as network management or file synchronization, which any technical specialist faces.
Book structure
This second edition includes supplemented original 12 chapters and 3 new chapters. Each chapter demonstrates new features or options for using shell scripts, and together they cover the full breadth of scripting capabilities for easier work on Unix. Most of the scripts presented in the book will work on both Linux and OS X. In other cases, we will write about this directly.
Chapter 0: A Brief Introduction to Shell Scripts
This is a completely new chapter that appeared in the second edition, which will serve novice Unix users with a brief introduction to the syntax of the bash shell language and the features of its use. This chapter will quickly and without lyric digression tell you everything you need to successfully read Chapter 1: from simply defining shell scripts to creating and running straightforward examples.
Chapter 1: The Missing Library
Programming languages widely used in Unix environments, such as C, Perl, and Python, have extensive libraries of various functions and utilities for checking number formats, calculating time intervals between dates, and solving many other problems. But, working with the shell, we have to deal with almost everything on our own, so this chapter discusses tools and techniques that will make shell scripts more friendly. Everything that you learn in the first chapter will help you read the scripts that you will meet in this book and write your own. We have included various input verification functions here, a simple and powerful interface to bc, a tool for quickly adding commas to improve the readability of large numbers, a trick for Unix variants in which the echo command does not support the useful -n flag,
Chapters 2 and 3: Enhancing Custom Commands and Creating Utilities
These two chapters introduce new commands that complement and expand the standard Unix toolkit. In the end, continuous development and improvement is one of the hallmarks of Unix. We are also involved in this process and in chapters 2 and 3 we propose scenarios that implement: a friendly interactive calculator, a file deletion tool that does not erase them from disk, two reminder and event tracking systems, an improved version of the locate command, a date command with support for several time zones and the new version of the ls command, which adds additional data to the directory contents lists.
Chapter 4: Tuning Unix
It may sound like heresy, but some aspects of Unix look unfinished even after decades of development. If you happen to use different versions of Unix, for example, switching from freely distributed Linux distributions to commercial versions of Unix, such as OS X, Solaris, or Red Hat, you will encounter missing flags and commands, inconsistent behavior of some commands and other similar problems. Therefore, this chapter will introduce revised versions and interfaces to Unix commands that make them a bit more friendly or more consistent with other Unix variants. Among other things, it describes how to add long GNU-style flags to non-GNU commands. Here you will find a couple of smart scripts that simplify working with different file compression utilities.
Chapters 5 and 6: System Administration: User Management and System Maintenance
If you are interested in our book, it is likely that you have administrator privileges and you are responsible for administering one or more Unix systems, even if it is only a personal computer with Ubuntu or BSD. These two chapters contain several scenarios that will help you with administration, including utilities for analyzing disk space usage, a disk quota system that automatically notifies users by e-mail of exceeding the allocated disk space, an improved implementation of the killall command, and a verification script crontab, a log file rotation tool and a couple of backup utilities.
Chapter 7: Internet Users
This chapter includes a package of really interesting shell scripts that showcase some great and easy ways to use Unix command-line resources on the Internet. Including: a tool for extracting URL addresses from any web page, a tool for obtaining weather forecasts, a tool for searching databases of videos and a tool for detecting changes on a website that automatically reports them by e-mail.
Chapter 8: Webmaster Tools
If you are a webmaster and maintain a website running on your own Unix system or on a remote server somewhere on the network, in this chapter you will find very interesting tools for building web pages on the fly, creating web albums with photos and even logging web search results.
Chapters 9 and 10: Web Server Administration and Internet Server Administration
These two chapters describe solutions to problems often faced by server administrators with Internet access. Here you will find two scenarios that analyze different aspects of web server traffic logging, tools for detecting invalid internal or external links available on the website, and a convenient password management tool on the Apache web server that simplifies support for .htaccess files. In addition, techniques for mirroring directories and entire websites are explored.
Chapter 11: Scripts for OS X
OS X, with its commercially successful and attractive graphical interface, has been a huge step forward in turning Unix into a friendly operating system. More importantly, OS X is a full-fledged Unix operating system, hidden behind a pretty interface, which means that you can write many useful and instructive scripts for it. This is what this chapter is about. In addition to a tool for automating screen capturing, this chapter provides scripts to help you explore the structure of your iTunes music library, change the titles of your Terminal windows, and enhance the open command.
Chapter 12: Scripts for Games and Fun
What kind of programming book is this if it doesn't even have a couple of toys? Chapter 12 brings together many of the ideas and techniques presented earlier, and describes the creation of six fun and fairly complex games. Although the chapter is written to entertain you, the code for each game is very instructive. Particularly noteworthy is the game "Hangman", which demonstrates some tricks and unusual techniques for programming scripts.
Chapter 13: Work in the Cloud
Since the release of the first edition of this book, the Internet has occupied more and more space in our daily lives. Especially important for us is the topic of synchronizing devices and files with cloud services such as iCloud, Dropbox and Google Drive. The chapter demonstrates shell scripts that make full use of these services and ensure timely synchronization and copying of files and directories. In addition, here you will find a couple of scenarios that use the features of OS X for working with photos and voicing text.
Chapter 14: ImageMagick and image processing
Command line applications can process not only text data, but also graphics. This chapter focuses on the identification and processing of images from the command line using a set of tools for working with graphics, including the open source ImageMagick software. The scripts in this chapter implement typical image operations, from defining their types to cropping and adding watermarks, plus a few more use cases.
Chapter 15: Days and Dates
The final chapter demonstrates tricks that simplify date and time operations: how many days the two dates share, what day of the week the day is, or how many days are left before it. We will solve these problems with easy-to-use shell scripts.
Appendix A: Installing Bash on Windows 10
While we were working on the second edition, Microsoft significantly changed its attitude towards open source software and in 2016 even released a full-fledged bash system for Windows 10. Despite the fact that the examples from the book were not tested in this version of bash, many ideas and solutions will not be difficult transfer to it. In the application, we will describe the installation of bash on Windows 10 so that you can try your hand at creating scripts on a Windows computer!
Appendix B: Additional Scenarios
Any good scout knows that there should always be a backup plan! Working on this book, we created backup scripts in case we needed to replace any of the main ones. As a result, we did not need backup scenarios, but on our part it would be ugly to keep them secret from you, our friends. This application includes three additional scenarios: for mass renaming of files, for mass execution of commands, and for calculating the phases of the moon, which we could not hide after we showed you 101 scenarios.
About the authors
Dave Taylor has been working in the computer industry since 1980. He participated in the creation of BSD 4.4 UNIX, its programs are included in all major UNIX distributions. Outstanding speaker and author of thousands of articles for magazines and newspapers. He has written over 20 books, including Learning Unix for OS X (O'Reilly Media), Solaris 9 for Dummies (Wiley Publishing), and Sams Teach Yourself Unix in 24 Hours (Sams Publishing). Popular columnist for the Linux Journal and founder of the askdavetaylor.com website, where he provides technical support to users and uploads reviews of new gadgets.
Brandon Perry began writing C # applications with the release of the .NET open source implementation, Mono. In his free time he likes to write modules for the Metasploit framework, examine binary files and test all sorts of things.
About the scientific reviewer
Jordi Gutiérrez Hermoso is a computer programmer, mathematician and freelance hacker. Since 2002, it exclusively uses Debian GNU / Linux not only at home, but also at work. Jordi is involved in the development of GNU Octave, a free computing environment that is largely compatible with Matlab, as well as Mercurial, a distributed version control system. He is fond of pure and applied mathematics, ice skating, swimming and knitting. Recently, he thinks a lot about the problems of greenhouse gas emissions and participates in actions to preserve rhinos.
»More information on the book can be found on the publisher’s website
» Table of Contents
» Excerpt
For Savory Agents 25% off coupon - Shell Scripts