Taming py-rrdtool

    As one ancient wisdom says: three things can be forever observed: how the fire burns, how the water flows and how the rrdtool draws graphs. This post is about the last.
    So what is rrdtool? rddtool is a great utility that allows you to manage data stored in rrd (round-robin database) form. This format is noteworthy in that it was originally intended to store periodic information, and so that the size of the database itself turns out to be fixed (rather small) regardless of the amount of data. At the same time, it remains possible to receive reports for long periods of time with arbitrary accuracy. This system found the greatest distribution in monitoring systems, where, as they say, it is better to see a picture once than to read the logs 7 times. At the end of the article you can get something similar to This will require: python , rrdtool , py-rrdtool .

    example





    After installing the necessary components, we get ready-made python-binders for rrdtool, with which you can continue to fantasize. Of course, the number of features of rrdtool can scare off at first, but I will try to consider a few basic steps + I will give a bit of native code that you can directly run and enjoy.

    They say that real geeks draw graphs of everything that is possible, up to the daily number of walks of their dog. For example, I consider two types of stored-parsed information:

    1. information automatically calculated by approximating the derivative of the input values ​​(for example, the speed on the network interface according to the number of bytes transmitted)
    2. simple information that is entered and stored in the form of separate, independent values ​​(hdd temperature, number of tape subscribers, etc.)

    Work with rrd, as with any other database, begins with determining the structure of the stored data.
    From python, it looks something like this: first of all, we import the necessary module and specify the database file name:

    from rrdtool import *
    fname = 'database.rrd'
    rrd = RoundRobinDatabase (fname)
    then, in fact, we create the database, setting the structure.


    rrd.create (
       # the database will contain two independent data sources (`rrdtool.DataSource`),
       # named 'in' and 'out', of type DeriveDST - derivative. The limit
       # values ​​of these indicators are immediately set :
       DataSource (" in ", type = DeriveDST, heartbeat = 600, min = 0, max = 12500000),
       DataSource ("out", type = DeriveDST, heartbeat = 600, min = 0, max = 12500000),
       # describe what reports we want to store in the database.
       # last 48 hours, every 5 min
       RoundRobinArchive (cf = AverageCF, xff = 0.5, steps = 1, rows = 576),
       # last 2 weeks, average for 30
       RoundRobinArchive (cf = AverageCF, xff = 0.5, steps = 6, rows = 672),
       # last 2 months, average for every 2 hours
       RoundRobinArchive (cf = AverageCF, xff = 0.5, steps = 24, rows = 732),
       # last 2 years, 12-hour
       average of RoundRobinArchive (cf = AverageCF, xff = 0.5, steps = 144, rows = 1460),
       # step 300 s - the data stored in the database will be attached to the “grid”, in five-minute increments
       step = 300
       )


    Example 2 will differ only in rows with a DataSource:

    DataSource ("value", type = GaugeDST, heartbeat = 600, min = _min, max = _max)

    Now we have a database in which we need to enter data from two sources with a frequency of 5 minutes - inand out.

    To do this, use a method rrd.update()that takes a value (of the rrd.Val class) as an argument and an optional template needed to set the data in a different order from the default.

    For example:


    values ​​= (323132312, 128539593)
    template = ("in", "out")
    rrd.update (Val (* values), template = template)
    It is logical to arrange this action as a separate script and run the crown with a five-minute periodicity for this an example.

    The next step is to actually create the picture itself with information for the period period(day, week, month, year) from the database in the file fname


    graph = RoundRobinGraph ('eth0-% s.png'% (period))
      

    graph.graph (
      
    Def ("in" , fname, data_source = "in", cf = AverageCF),
      

    Def ("out", fname, data_source = "out", cf = AverageCF),
      

    Cdef ("out_neg", "out, -1, *"),
      
    AREA ("In", rrggbb = "32CD32", legend = "Incoming"),
      

    LINE1 ("in", rrggbb = "336600"),
      
    GPRINT ("in", cf = MaxCF, format = r "Max \:% 5.1 lf% S "),
      

    GPRINT (" in ", cf = AverageCF, format = r" Avg \:% 5.1lf% S "),
      

    GPRINT (" in ", cf = LastCF, format = r" Current \:% 5.1 lf% Sbytes / sec \ n "),
      

    AREA (" out_neg ", rrggbb =" 4169E1 ", legend =" Outgoing "),
      

    LINE1 (" out_neg ", rrggbb =" 0033CC "),
      
    GPRINT ("out", cf = MaxCF, format = r "Max \:% 5.1lf% S"),
      

    GPRINT ("out", cf = AverageCF, format = r "Avg \:% 5.1lf% S"),
      

    GPRINT ("out", cf = LastCF, format = r "Current \:% 5.1lf% Sbytes / sec \ n"),
      

    HRULE ("0", rrggbb = "000000"),
      

    vertical_label = "bytes / sec",
      

    lower_limit = 0,
      
    start = "- 1% s"% (period,),
      

    title = title,
      
    width = 600,
      
    height = 80,
      

    lazy = None,
      
    )
      

    In a nutshell - here you specify data from the database that you need to display and details design, such as: lines, filled areas, their colors, etc.

    As a result of this piece, we can get something that resembles,

    my_example

    or for example 2, a graph of the number of subscribers to the feed from google reader:

    subscribers_example

    I tried to arrange these actions in a library that is more convenient for use (although it seems to be worthless for expansion) , which includes several ready-made counter-drafts (for traffic, hdd temperature, number of clients on the wifi interface, number of subscribers) and suitable for extension, the simplest class GaugeRRD, with which you can independently create your own classes for working with py-rrdtool.

    Use cases can be found at http://undefined.org.ua/static/rrdtool/src .
    My example of scripting, designed in the “analytical center” lies at http://xa4a.org.ua/rrdtool .

    In conclusion, I want to note that the purpose of the article was to show the possibility of using a tool such as rrdtool to analyze non-trivial data, or in your python applications. If you need a system monitoring tool that is ready to work out of the box, there are already many ready-made projects, for example: munin , mrtg , cacti and others.


    The original article is in a personal blog at: http://undefined.org.ua/blog/2008/08/05/py-rrdtool/

    Also popular now: