January 30, 2011 at 14:21How to quickly check a Linux server for hacking
About two years ago, I rented from a German hoster a not very powerful server based on Centos 5.2. It has several web projects that bring some profit, and therefore, I try to keep an eye on it whenever possible.
Centos has a standard Logwatch log analyzer that runs daily on the crown, analyzes the contents of / var / log, makes a summary report and sends it via email. One fine day, I found a record in this report:
At that moment, she was very embarrassed to me, since on the previous day I did not log in to the server and even more so did not install anything. The first thing that came to mind - the server was compromised. I considered myself a confident Linux user, but I was confused. Fortunately, at that moment icq was my former colleague, the best system administrator I know, and just a very good person.
He helped to quickly check the system. As a result, I formed a short HowTo on how to quickly check my server for hacking. I am sure that it will be useful to many Brave readers. It is assumed that the user is familiar with the Linux / Unix console.
So, first of all, change the root password:
Next, we scan the host for suspicious open ports. This can be done, for example, from another Unix machine using the nmap utility:
111 and 620 ports looked suspiciously in this list, therefore we look further at who listens to them:
Everything turned out to be in order, since these were nfs components. Next, we check the UDP connections:
Here, too, everything turned out to be in order. Most likely, they could not get to the server through the console, since when logging in to the server, Centos wrote that the last login was from my IP. Therefore, the next item you need to check the /root/.ssh folder, they could put the key for the ssh client there in some way.
Only the key file turned out to be here, which I renamed immediately after the provider transferred the host. Next, check the / etc / passwd file. It should not have users with uid = 0, except root:
And here, too, everything was okay. The final chord was checking the host for installed rootkits. You can use the free rkhunter utility for this . Download the archive with the latest version, unpack it and run the installer. Next, we update it and run the check:
Rkhunter will first check important system files, then look for rootkits. After there is a check for various vulnerabilities. At the end, the program checks the versions of popular products such as Apache, OpenSSH, etc. for the latest versions.
Rkhunter displays the results of its work to the console, and also forms a consolidated log /var/log/rkhunter.log. You can run this anti-rootkit every day for the crown and receive a scan report by email.
Fortunately, rkhunter did not detect any malware on my server, it allowed me to calm down and think about what strange installs occurred on the server. And then I remembered that immediately after receiving the server, I installed and configured a VPN server on this server. Apparently, some kind of error occurred in Logwatch and he added two years ago data to the report.
Of course, if the attackers do everything correctly, then Logwatch will not detect any traces and the server owner will not suspect anything for a long time. However, the steps described in this HowTo, if carried out regularly, will help protect your servers from being compromised.
Centos has a standard Logwatch log analyzer that runs daily on the crown, analyzes the contents of / var / log, makes a summary report and sends it via email. One fine day, I found a record in this report:
--------------------- yum Begin ------------------------
Packages Installed:
lzo2 - 2.02-3.el5.rf.i386
dnstracer - 1.8-1.2.el5.rf.i386
openvpn - 2.0.9-1.el5.rf.i386
---------------------- yum End -------------------------
At that moment, she was very embarrassed to me, since on the previous day I did not log in to the server and even more so did not install anything. The first thing that came to mind - the server was compromised. I considered myself a confident Linux user, but I was confused. Fortunately, at that moment icq was my former colleague, the best system administrator I know, and just a very good person.
He helped to quickly check the system. As a result, I formed a short HowTo on how to quickly check my server for hacking. I am sure that it will be useful to many Brave readers. It is assumed that the user is familiar with the Linux / Unix console.
So, first of all, change the root password:
[root @ myhost etc] # passwd Changing password for user root. New UNIX password: ...
Next, we scan the host for suspicious open ports. This can be done, for example, from another Unix machine using the nmap utility:
[root @ myhost ~] # nmap -P0 123.123.123.123 Starting Nmap 4.11 (http://www.insecure.org/nmap/) at 2011-01-23 11:47 MSK Interesting ports on myhost.myprovider.net (123.123.123.123): Not shown: 1679 filtered ports PORT STATE SERVICE 21 / tcp open ftp 22 / tcp open ssh 25 / tcp open smtp 53 / tcp open domain 80 / tcp open http 106 / tcp open pop3pw 110 / tcp open pop3 111 / tcp open rpcbind 135 / tcp filtered msrpc 136 / tcp filtered profile 137 / tcp filtered netbios-ns 138 / tcp filtered netbios-dgm 139 / tcp filtered netbios-ssn 143 / tcp open imap 443 / tcp open https 445 / tcp filtered microsoft-ds 465 / tcp open smtps 620 / tcp open unknown 993 / tcp open imaps 995 / tcp open pop3s 3306 / tcp open mysql 8443 / tcp open https-alt
111 and 620 ports looked suspiciously in this list, therefore we look further at who listens to them:
[root @ myhost ~] # netstat -anp | grep LISTEN | grep 620 tcp 0 0 0.0.0.0:620 0.0.0.0:* LISTEN 1710 / rpc.statd
[root @ myhost ~] # netstat -anp | grep LISTEN | grep 111 tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1685 / portmap
Everything turned out to be in order, since these were nfs components. Next, we check the UDP connections:
[root @ myhost ~] # netstat –anp | grep udp udp 0 0 0.0.0.0:32773 0.0.0.0:* 2345 / named udp 0 0 127.0.0.1:32780 127.0.0.1:32780 ESTABLISHED 2539 / postmaster udp 0 0 0.0.0.0:32783 0.0.0.0:* 2790 / avahi-daemon: udp 0 0 123.123.123.123:53 0.0.0.0:* 2345 / named udp 0 0 123.123.123.123:53 0.0.0.0:* 2345 / named udp 0 0 127.0.0.1:53 0.0.0.0:* 2345 / named udp 0 0 0.0.0.0:614 0.0.0.0:* 1710 / rpc.statd udp 0 0 0.0.0.0/10353 0.0.0.0:* 2790 / avahi-daemon: udp 0 0 0.0.0.0:617 0.0.0.0:* 1710 / rpc.statd udp 0 0 0.0.0.0:111 0.0.0.0:* 1685 / portmap udp 0 0 0.0.0.0:631 0.0.0.0:* 2069 / cupsd udp 0 0 ::: 32774 ::: * 2345 / named udp 0 0 ::: 32784 ::: * 2790 / avahi-daemon: udp 0 0 ::: 5353 ::: * 2790 / avahi-daemon:
Here, too, everything turned out to be in order. Most likely, they could not get to the server through the console, since when logging in to the server, Centos wrote that the last login was from my IP. Therefore, the next item you need to check the /root/.ssh folder, they could put the key for the ssh client there in some way.
[root @ myhost ~] # dir /root/.ssh authorized_keys_
Only the key file turned out to be here, which I renamed immediately after the provider transferred the host. Next, check the / etc / passwd file. It should not have users with uid = 0, except root:
[root @ myhost ~] # cat / etc / passwd | more root: x: 0: 0: root: / root: / bin / bash bin: x: 1: 1: bin: / bin: / sbin / nologin daemon: x: 2: 2: daemon: / sbin: / sbin / nologin adm: x: 3: 4: adm: / var / adm: / sbin / nologin lp: x: 4: 7: lp: / var / spool / lpd: / sbin / nologin sync: x: 5: 0: sync: / sbin: / bin / sync ...
And here, too, everything was okay. The final chord was checking the host for installed rootkits. You can use the free rkhunter utility for this . Download the archive with the latest version, unpack it and run the installer. Next, we update it and run the check:
[root @ myhost ~] # ./installer.sh --install --layout / usr / local [root @ myhost ~] # / usr / local / bin / rkhunter –-update [root @ myhost ~] # / usr / local / bin / rkhunter –-check
Rkhunter will first check important system files, then look for rootkits. After there is a check for various vulnerabilities. At the end, the program checks the versions of popular products such as Apache, OpenSSH, etc. for the latest versions.
Rkhunter displays the results of its work to the console, and also forms a consolidated log /var/log/rkhunter.log. You can run this anti-rootkit every day for the crown and receive a scan report by email.
Fortunately, rkhunter did not detect any malware on my server, it allowed me to calm down and think about what strange installs occurred on the server. And then I remembered that immediately after receiving the server, I installed and configured a VPN server on this server. Apparently, some kind of error occurred in Logwatch and he added two years ago data to the report.
Of course, if the attackers do everything correctly, then Logwatch will not detect any traces and the server owner will not suspect anything for a long time. However, the steps described in this HowTo, if carried out regularly, will help protect your servers from being compromised.