Start thinking

Good day dear% username%!
I would like to congratulate all administrators on the holiday and in honor of this it came up to me to write a post. By the nature of their activity (* nix admin), I am approached by friends with various requests for help on the servers. Typically, requests in the spirit - we began to slow down the site, or something hung from us, etc. Very often, problems arise due to the actions of programmers who do not always understand what they are doing, or do not understand the consequences of what they do. Having looked at all this, I decided to share with you some cases and instructions.

Initially, I thought of calling the post “stop administering” and collecting typical mistakes of programmers of admins in it, but the idea went a bit differently, so the title turned out like this. I want to apologize in advance for the confusion of the post, I just rolled something to write, and as the thought went, I wrote it.

Case 1

- “Anton, we began to slow down the site. Take a look? ”
The wording is fairly standard, so to get any understanding of what’s going on, I’m climbing ssh to the server.
Looking at top, I see a php process eating up a bunch of resources. What is this? No crime at all - remove_old_thumbs.php. As you might guess, it removes old thumbnails of images. And everything would be fine, but he does it very actively.
iotop says that the script is very actively tormenting the hard drive. It’s understandable, woolly folders and deleting thousands of files are resource-intensive, especially on virtual machines. Decision?
ionice -c3 php remove_old_thumbs.php
Since the procedure for removing old thumbnails is not critical, and does not require high priority, you can run it with a low I / O priority. We start - the process went more quietly, the site stopped braking, the miniatures are slowly being deleted - everyone is happy.

Case 2

- “Anton, we started making miniatures here, but something is slowing everything down”
Well, let's see. Here are those on. Thumbnail links look like this. thumb.php? image = images / 12311.jpg. Well, here you have to dig deeper into the code.

- Pictures for thumbnails are in one folder, and there are a million of them. There is no need to force fs.
- The script does not save the generated thumbnails. For each appeal, he generates a new miniature - not comme il faut gentlemen!
- Generating thumbnails for all pictures will take a long time.

We analyze the case. For starters, it would be nice to put all the pictures in folders. It is proposed to arrange by date, take the date from the publication. There are no special problems with this, everything is transparent and clear. And it turned out somehow like this: images / 2013/08 / 12311.jpg.
To kill the last 2 points, it was proposed to make the thumbnail addresses such thumbs / 2013/08 / 12311.jpg, and in nginx set up a rule that checks for the presence of a file at the specified url and, if not, redirect the request to thump.php. In turn, thumb.php generates a thumbnail, shows it to the client and puts it on disk at the desired address. Thus, we unloaded cpu and fs, and the site crashed.

Case 3

- “Anton, we began to slow down the site. Look? ”
There are 10 remove_old_thumbs.php processes hanging in the processes. Guys, well, let's do a script for neglect in scripts running about the crown? At least by creating a lock-file in / tmp, and when the script is completed using register_shutdown_function, call the function that deletes this file. Quick and easy.

Case 4

I will not dwell on this in great detail, although the topic is very relevant, but each case is strictly individual. I won’t reproduce an example for memory, but in short:
“SELECT * FROM posts” - we pulled out the entire database of posts, with all the descriptions and how much garbage, and we are actively working with this data array, although only the id and poster_url are needed from the entire array.
“SELECT description FROM posts WHERE post_time> '2013-01-01' AND post_moderated = '1' ORDER BY name DESC” - Imagine this query with a bunch of joins and very massive. And why does he suddenly slow down? And all because there is no index on the name field, but new programmers have not heard about EXPLAIN.

Well. These cases are pulled by the mistakes of programmers. What can be reproached admins?

To begin with, with this:
ssh : // root : om7ooS3righoob4Xe7ri @ hostname

In 90% of cases, remote access to servers is performed from the root user and almost immediately, these servers start brute-forcing. This is not right.
What can be done? First, create a user, which then go to ssh. And then, prohibit authorization from root (PermitRootLogin no - prohibits authorization from root) in sshd_config. To get root rights under the user, you can type “su -” or configure sudo. Not difficult, then?
What more can be done? You can restrict access to ssh from unknown IPs and you do not have to write a bunch of rules for iptables, just tweak a couple of files.

hosts.deny
sshd: all

hosts.allow
sshd: 123.231.132.213

And that’s it! We forbade anyone except 123.231.132.213 to go to the server via ssh.

Ok, what else?
[root @ localhost ~] # ps auwx | grep php
root 795 0.0 0.5 321152 9948? Ss Jul 22 0:10 php-fpm: master process (/etc/php-fpm.conf)
root 884 0.0 0.3 321696 6724? S Jul 22 0:00 php-fpm: pool www
root 885 0.0 0.3 321696 6644? S Jul 22 0:00 php-fpm: pool www
root 886 0.0 0.3 321696 6720? S Jul 22 0:00 php-fpm: pool www
root 887 0.0 0.3 321728 6720? S Jul 22 0:00 php-fpm: pool www
root 888 0.0 0.3 321728 6728? S Jul 22 0:00 php-fpm: pool www

Why do php scripts run from root? What for?! It's like leaving your car keys in the car itself. The solution in each case will be different, in this case, in the php-fpm.conf config, the user changes to the desired one.

Those who did not start running scripts from the root sin for others - chmod 777 . I will not describe the consequences and methods of treatment, I will say only one thing. Set file permissions by considering your actions. And never expose them like that
“chmod 777 -R. “. By fate, you can break the OS by running the command in the wrong folder.

Many administrators sin by compiling software and putting it into the system. I will not venture to say that this is a huge evil, but still it is evil. Familiar ./configure && make && make install? Guys, do not be too lazy to find a package. If there is no package, try using the same checkinstall.

In fact, there are a lot of different cases. You can disassemble them for a long time and hard, but I would like to contact the programmers. Guys, if you maintain a site with serious traffic, do optimizations and create some kind of harsh functionality, consult experienced colleagues (especially admins) for advice. They will help you with advice and save you from scary and not very mistakes.