Monitoring server resources running * nix using RRDtool

Good day, dear% user%! Today I will tell you how to increase real-time monitoring of hardware and systems using a set of utilities - RRDtool. Our goal is to collect and graphically display in real time data on the operation of the system: CPU load, memory status, load of network interfaces, as well as processor temperature for various periods of time.

A bit of theory. RRDtool - a set of utilities for working with ring databases. Such databases are specially designed to store time-varying sequences of data (network traffic, CPU load) - just what we need. In addition, we will use the collectd daemon - this is a small daemon that collects statistics on system resources over time periods - i.e. and creates the necessary ring databases on the basis of which with the help of RRDtool we can build graphs. Before installing, let's say that you have already installed and configured a web server, you will need it to view the graphs in the browser. Let's proceed with the installation and configuration.

All manipulations were carried out on freshly installed Debian 7 wheezy: To draw graphs, you need to install RRDtool:

root@kd-ast:/etc/collectd# uname -a
Linux kd-ast 3.2.0-4-686-pae #1 SMP Debian 3.2.46-1 i686 GNU/Linux



apt-get install rrdtool

To collect statistics on the use of system resources
instead of self-written scripts, use the “Collectd” daemon:

apt-get install collectd

Run: After loading, we will see in /var/lib/collectd/router.local, where “router.local” is the name of your server. Now you can move on to the script itself, which will generate graphs:

cd /usr/sbin/
./collectd onestart

#!/bin/sh
### CPU
/usr/bin/rrdtool graph /var/www/monitor/cpu0.png \ #путь до RRDtoll и путь куда класть график 
 -e now \
 -s 'end - 6 hours' \
 -S 60 \
 --title "CPU USAGE: AMD Athlon(tm) II X3 455 Processor" \ #CPU
 --vertical-label "Percents" \
 --imgformat PNG \
 --slope-mode   \
 --lower-limit 0 \
 --upper-limit 100 \
 --rigid \
 -E \
 -i \
 --color SHADEA#FFFFFF \
 --color SHADEB#FFFFFF \
 --color BACK#CCCCCC \
 -w 600 \
 -h 150 \
 --interlaced \
 --font DEFAULT:8:/usr/share/fonts/truetype/ttf-dejavu/DejaVuSansMono.ttf \
 DEF:a=/var/lib/collectd/rrd/kd-ast/cpu-0/cpu-idle.rrd:value:MAX \ #пути к кольцевым БД
 DEF:b=/var/lib/collectd/rrd/kd-ast/cpu-0/cpu-system.rrd:value:MAX \
 DEF:c=/var/lib/collectd/rrd/kd-ast/cpu-0/cpu-user.rrd:value:MAX \
LINE2:b#2cc320: \
 AREA:b#54eb48:System \
LINE2:c#e7ad4a: \
 AREA:c#ebd648:User
#LINE1:a#CCCCCC:Idle \
###eth0
/usr/bin/rrdtool graph /var/www/monitor/network0.png \
 -e now \
 -s 'end - 6 hours' \
 -S 60 \
 --title 'Traffic on ext_if: eth0 (local) (100Mb/s)' \ ## название интерфейса
 --vertical-label 'Mbyte\s' \
 --imgformat PNG \
 --slope-mode   \
 --lower-limit 0 \
 --upper-limit 20000000 \ # пропускная способность - max
 --rigid \
 -E \
 -i \
 --color SHADEA#FFFFFF \
 --color SHADEB#FFFFFF \
 --color BACK#CCCCCC \
 -w 600 \
 -h 150 \
 --interlaced \
 --font DEFAULT:8:/usr/share/fonts/truetype/ttf-dejavu/DejaVuSansMono.ttf \
 DEF:a=/var/lib/collectd/rrd/kd-ast/interface-eth0/if_octets.rrd:tx:MAX \
 DEF:b=/var/lib/collectd/rrd/kd-ast/interface-eth0/if_octets.rrd:rx:MAX \
 DEF:c=/var/lib/collectd/rrd/kd-ast/interface-eth0/if_errors.rrd:tx:MAX \
 AREA:a#4169E1:Tx \
LINE2:b#2cc320: \
AREA:b#54eb48:Rx \
LINE1:c#FF0000:Errors
### RAM
/usr/bin/rrdtool graph /var/www/monitor/memory.png \
 -e now \
 -s 'end - 6 hours' \
 -S 60 \
 --title 'MEMORY USAGE: 2Gb' \ # RAM
 --vertical-label 'Mbyte' \
 --imgformat PNG \
 --slope-mode   \
 --lower-limit 0 \
 --upper-limit 2000000000 \ # верхний лимит
 --rigid \
 -E \
 -i \
 --color SHADEA#FFFFFF \
 --color SHADEB#FFFFFF \
 --color BACK#CCCCCC \
 -w 600 \
 -h 150 \
 --interlaced \
 --font DEFAULT:8:/usr/share/fonts/truetype/ttf-dejavu/DejaVuSansMono.ttf \
 DEF:a=/var/lib/collectd/rrd/kd-ast/memory/memory-buffered.rrd:value:MAX \ # пути до кольцевых БД
 DEF:b=/var/lib/collectd/rrd/kd-ast/memory/memory-cached.rrd:value:MAX \
 DEF:c=/var/lib/collectd/rrd/kd-ast/memory/memory-free.rrd:value:MAX \
 DEF:d=/var/lib/collectd/rrd/kd-ast/memory/memory-used.rrd:value:MAX \
 LINE1:a#6959CD:buffered \
AREA:b#00FF00:cache \
LINE2:c#006400: \
AREA:c#00CD66:free \
AREA:d#FF1493:used \

Add it to the crowns, for execution every minute:

nano /etc/crontab

*/1 * * * * root /home/sysbes/Scripts/RRDtool/mon.sh > /dev/null 2>&1

Create index.php with the following contents:

Визуализация системных ресурсов: kd-ast

We can watch the resulting graphs by opening in a browser: ваш_сервер/monitor

To determine the temperature, we will use lm-sensors - a set of drivers and a utility that allows you to monitor the temperature, voltage, and fan speed in your system. It should be remembered that the set of sensors is individual for your system, so some features may not be available. You can also use mbmon.

apt-get install lm-sensors

To identify sensors:

sensors-detect

The script will ask you a few questions - do not be alarmed.

We are trying to determine:

sensors

If the temperature is incorrect or not detected, it may be necessary to load the kernel module, it helped me:

modprobe it87

and again sensors

If the temperature is correct, add the it87 module to startup:

nano /etc/modules

it87

sensors

Next, as you understand, you need to write a script that will take values,
draw through rrdtool, and in a day, week, month, year ... Then we add it
to cron and it will perform its manipulations.

The script was as follows:

#!/bin/bash
RRDTOOL=/usr/bin/rrdtool #путь до RRDtool
DATABASE=/var/lib/collectd/rrd/kd-ast/space-temperature.rrd # путь до кольцевой ДБ
PERIOD=60
SENSOR=` /usr/bin/sensors | grep temp3 | awk '{ print $2-0}'` # команда запроса температуры
IMAGE_PATH=/var/www/monitor # куда складывать графики
TITLE_TEXT='CPU TEMP'
UNDER_TEXT='TEMP'
TEMP_MIN=20
TEMP_MAX=90
if ! [ -f $DATABASE ]
        then
                $RRDTOOL create $DATABASE -s $PERIOD DS:temperature:GAUGE:600:10:80 \
                RRA:AVERAGE:0.5:1:576 \
                RRA:AVERAGE:0.5:6:672 \
                RRA:AVERAGE:0.5:24:732 \
                RRA:AVERAGE:0.5:144:1460
        fi
$RRDTOOL update $DATABASE N:$SENSOR
function DRAW_GRAPHIC
{
        NOW_HOUR=`date +%H`
        NOW_MIN=`date +%M`
        NOW_SEC=`date +%S`
        case $2 in
        day)
                 TIME_TEXT="Last 24 hours"
                 ;;
        week)
                 TIME_TEXT="Last week"
                 ;;
        month)
                 TIME_TEXT="Last mounth"
                 ;;
        year)
                 TIME_TEXT="Last year"
                 ;;
        esac
        $RRDTOOL graph $IMAGE_PATH/$1 \
        -s -1$2 \
        -e now \
        -a PNG \
        -v 'C' \
        -t "$TITLE_TEXT [$HOSTNAME] - $TIME_TEXT" \
        -l $TEMP_MIN \
        -u $TEMP_MAX \
        -r \
        -E \
        -i \
        -R light \
        --zoom 1.0 \
        -w 600 \
        -h 150 \
        DEF:temperature=$DATABASE:temperature:AVERAGE \
        LINE2:temperature#33cc33:"$UNDER_TEXT" \
        GPRINT:temperature:MIN:'MIN\:%2.lf' \
        GPRINT:temperature:MAX:'MAX\:%2.lf' \
        GPRINT:temperature:AVERAGE:'AVG\:%4.1lf' \
        GPRINT:temperature:LAST:'NOW\:%2.lf \n' \
        COMMENT:"TIME \: $NOW_HOUR\:$NOW_MIN\:$NOW_SEC \n"
}
DRAW_GRAPHIC 'temp_d.png' 'day'
DRAW_GRAPHIC 'temp_w.png' 'week'
DRAW_GRAPHIC 'temp_m.png' 'month'
DRAW_GRAPHIC 'temp_y.png' 'year'

Add the script to crontab, to execute every minute:

*/1 * * * * root /home/sysbes/temp.sh > /dev/null 2>&1

An approximate view of the statistics table is as follows:

  Температура процессора @ маршрутизатор БЛА-БЛА  

As a result, we get:

Real Time Load kd-ast:



CPU TEMP kd-ast:

Upon request, ваш_сервер/monitor you can view statistics in a browser.