General overview of standard system monitoring tools

    Level of difficulty: Beginner

    An introductory overview of GNU / Linux standard tools should help novice Linux users control the OS. Used applications are usually included in the distribution of well-known distributions. A useful side effect, I believe, is the ability to orient young system and network administrators in conditions of [suspicion] of compromise.

    Local surveillance

    Magazines.

    Logging system performance is very important. Log files are the first source of information about the processes that take place in the system, and, of course, the first goal of an attacker, noticing the signs of penetration. Cybercriminals always try to hide as much as possible any evidence of their activities, so a good approach is to save or duplicate magazines on a separate resource or by email. Naturally, the electronic box where the system operation reports come in must be serviced by a separate node. In this case, as a rule, they can be trusted.
    FHS offers a directory in which log files should be located. Here is a short list of standard log locations:
    • / var / log / acpid - the service that is responsible for power (acpi for intel);
    • /var/log/boot.log - system boot log;
    • / var / log / cron - event planning;
    • / var / log / messages - messages of the system (kernel);
    • /var/log/daemon.log - service messages;
    • / var / log / debug - debugging system messages;
    • /var/log/Xorg.*.log - graphic subsystem (Xorg X-server);
    • / var / log / cups / * - cups printing system (quite popular and widespread);
    • / var / log / samba / * (default) - logs network drives.

    Journaling of http-, ftp-server, MTA / MDA, network services is already a narrower and professional topic. For obvious reasons, I hope the reasons are not considered in this article.
    The output of the dmesg command and the entries in messages , debug and daemon.log should be studied first when (system-wide) problems occur. The last two, especially if the problems are related to services.
    I will make a reservation that the indicated paths to the registration files can of course differ in different systems. In such situations, you should refer to the distribution documentation.
    I gave only a list, for example. You can find out the location of the log file from the man pages or configuration files. Usually this should not be a problem. For example, where to put records about the operation of shared network folders, the log file directive defines the configuration file (/etc/samba/smb.conf, as a rule) of the corresponding Samba daemon. In the settings of most network, mainly services, you can also specify the format and, for some, the level of detail of logging entries.

    File changes.

    A very good tool for detecting file modification is the standard find program . The wise use of the mtime parameter will help to establish changes in the file system, for example, the command
    find /etc -mtime -1

    will show files that have been changed in the last day. A very good search for files by different times is described in the Unix find tutorial .

    Processes.

    Analysis of running processes should be carried out using the top program . Output Example:
    top - 15:21:36 up 4 days, 19:04, 2 users, load average: 0.06, 0.01, 0.00
    Tasks: 92 total, 1 running, 91 sleeping, 0 stopped, 0 zombie
    Cpu(s): 0.0%us, 0.7%sy, 0.0%ni, 97.4%id, 1.7%wa, 0.3%hi, 0.0%si, 0.0%st
    Mem: 256088k total, 249192k used, 6896k free, 5320k buffers
    Swap: 265064k total, 304k used, 264760k free, 176876k cached

    PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
    27362 decoy 15 0 17400 11m 3616 S 0.3 4.7 3:48.28 rtorrent
    1 root 18 0 2912 1848 524 S 0.0 0.7 0:01.83 init
    ...

    The first column of the table contains the process number. It can be used, for example, for removal by the kill command . Next (from left to right) follows the name of the user who initiated the launch, then the priority of the task, some overhead information, in particular, the application's use of system resources, the time during which the process runs, and the start command.
    In addition to the real-time monitor top, there is also a powerful ps process generator . For example, you can find out if a given program, say, Midnight Commander, is currently running or not, with the command
    ps aux | grep mc | grep -v grep

    It is strongly recommended that you install htop, which is more pleasant and visual for the user (just see once) the analogue of top .

    Network status

    Nmap scanner.

    A tremendously powerful tool for which almost separate books on use are written. Here is an example:
    $ nmap -sT -v -v -v localhost
    Starting nmap 3.55 ( www.insecure.org/nmap ) at 2005-02-04 22:01 EET
    Machine 127.0.0.1 MIGHT actually be listening on probe port 80
    Host localhost.localdomain (127.0.0.1) appears to be up ... good.
    Initiating Connect() Scan against localhost.localdomain (127.0.0.1) at 22:17
    Adding open port 111/tcp
    Adding open port 25/tcp
    Adding open port 113/tcp
    Adding open port 22/tcp
    Adding open port 80/tcp
    Adding open port 631/tcp
    The Connect() Scan took 1 second to scan 1660 ports.
    Interesting ports on localhost.localdomain (127.0.0.1):
    (The 1654 ports scanned but not shown below are in state: closed)
    PORT STATE SERVICE
    22/tcp open ssh
    25/tcp open smtp
    80/tcp open http
    111/tcp open rpcbind
    113/tcp open auth
    631/tcp open ipp

    Nmap run completed -- 1 IP address (1 host up) scanned in 0.503 seconds

    Therefore, a web server is running on my machine. Another example (we check the node for RPC DCOM):
    # nmap -sS -p 135 xxx.xxx.xxx.xxx

    Starting nmap 3.55 ( www.insecure.org/nmap ) at 2005-02-04 22:06 EET
    Interesting ports on *.* (xxx.xxx.xxx.xxx):
    PORT STATE SERVICE
    135/tcp filtered msrpc

    Nmap run completed -- 1 IP address (1 host up) scanned in 45.798 seconds

    Note : some types of scans require root privileges.
    The nmap scanner can check not only individual nodes, but also entire ranges. As a rule, it is not included in the “clean” installation of the system, but it is easy to install and highly recommended for audit purposes.
    Sometimes, with suspicious anomalies or based on the results of nmap , it is very useful to start the tcpdump traffic analyzer (sniffer) , filter out the “unsuspecting” network packets and see what remains. In fact, tcpdump is not a sniffer, it takes pictures, but in such situations it will be extremely useful.
    Another great tool to use isnetstat . It is used to list active network connections.
    $ netstat -l
    Active Internet connections (only servers)
    Proto Recv-Q Send-Q Local Address Foreign Address State
    tcp 0 0 *:32769 *:* LISTEN
    tcp 0 0 *:62978 *:* LISTEN
    tcp 0 0 *:sunrpc *:* LISTEN
    tcp 0 0 *:auth *:* LISTEN
    tcp 0 0 localhost.localdoma:ipp *:* LISTEN
    tcp 0 0 localhost.localdom:5335 *:* LISTEN
    tcp 0 0 *:29305 *:* LISTEN
    tcp 0 0 localhost.localdom:smtp *:* LISTEN

    In conjunction with Linux tools such as lsof and fuser , we learn which programs use which ports.

    Conclusion

    I tried to review the system monitors for the "smallest". All described tools can be found in almost any Linux build. We are not talking about powerful and heavyweight third-party IDS software systems and FS shells.
    In conclusion, I would like to say that the competent use of the most standard set of tools allows for a basic audit. Remember: the security issue is not fiction paranoid! Many people think about their own protection after the break-in. “The best way to make sure that no precautions are needed is to take them in time,” Murphy said.

    A brief overview last ...

    Process Analysis:
    • top
    • ps

    Network Analysis:
    • nmap
    • tcpdump
    • netstat
    • fuser
    • lsof

    File System Analysis:
    • find

    Also popular now: