About bulk adding monitoring objects to Zabbix

    Somehow, a colleague and I faced a simple, essentially, task - to monitor under Zabbix means about five hundred identical Linux-based terminals scattered across the country. The terminals belonged to the same network - 10.0.0.0/8. It would seem that the task is completely trivial. In fact - bungle the template, start auto-detection and automatically add all the found hosts to the group and roll this template onto them. It’s easier just to brew tea from a bag. Rolling up our sleeves, we got down to business ...

    Birth problem


    In general, nothing happened in our country (in full accordance with the laws of the genre). We made a template, started autodiscover and went culturally, because it was on Friday. Returning on Monday, we found that over the weekend our Zabbix “auto-detected” only one host. Saddened by this, we began to reflect on the reasons. And then a simple thought dawned on us - what is, in fact, a Class A subnet?

    The calculator helpfully suggested that the class A subnet contains almost 17 million hosts. Further, our electronic accountant clarified that if Zabbix spends 3 seconds on disassembling one address (lags in the network, polling obviously not our customers, etc.), then it will take him about a year and a half to bypass the class A subnet. For one round, I emphasize. This process can be carried out in parallel - say, check addresses in batches of 20 pieces. But even in this case, it will take about a month to complete a round of class A subnet. At the same time, it is obvious that one bypass is not enough - our terminal may be turned off or inaccessible at the very moment when Zabbix is ​​trying to interrogate it, which means it will be necessary to walk through the network three or four more times. In general, we suddenly realized that the standard Zabbix autodiscover tools are working as usual,

    What to do? © Chernyshevsky


    Add hosts to Zabbix by hand? This meant recognizing shameful surrender in the face of a simple, essentially, problem.

    The Internet has told us that we are not the first to encounter such a task. On the official Zabbix forum, a similar question was asked. And they received a traditional, alas, answer for such forums - an offer of the most complex and inefficient way. Since there was no time to learn the Zabbix API, we decided to try our luck with the database again.

    Decision algorithm


    The Zabbix database has a table that lists all the monitored hosts with their parameters - the hosts table. Moreover, each host in this table corresponds to a unique identifier - an integer. At the same time, there are not so many parameters sufficient to set the host to control - its identifier in the database, its IP and its name in Zabbix.

    The approach is simple - first we find out which index on the database the last host added has:
    select hostid from zabbix.hosts order by hostid desc limit 1;
    Hereinafter, the queries are for MySQL, which, in principle, does not affect the essence. We increase the obtained value by one - the index on the database for the new host is ready. We know the host name and its IP address, respectively, we can add the host to Zabbix:
    insert into zabbix.hosts (hostid,host,useip,ip) values ($hostid,$hostname,1,$IP);
    In this case:
    • $ hostid - the new database index we received earlier;
    • $ hostname - host name;
    • $ IP is its IP address.

    The above query is made under the assumption that we control the hosts by their address, and not by their DNS name. However, for this second option, the query will not fundamentally change - only a new field will be added.

    So, the host has been added, but this is not enough - each host should belong to some group. And this can also be done directly in the database. Suppose for certainty that we want to drive our terminals into the Linux servers group.

    All groups in the database are stored in the groups table (trivial, isn't it?). We will need the index of our group, which we can find out with a simple query:
    select groupid from zabbix.groups where name=‘Linux servers’;
    To account for the belonging of hosts to groups in the database there is a special table - hosts_groups. This table stores simple correspondences - a group with an index such and such corresponds to a host with an index. For each host, exactly as many matches are entered as how many groups this host contains. And each correspondence, as usual, is assigned a unique index - an integer. We find out the index of the last match:
    select hostgroupid from zabbix.hosts_groups order by hostgroupid desc limit 1;
    Increase the obtained value by one - here you have the index of the new match for our newly added host. We know the host index, the group index was found out - what prevents us from adding the host to the group?
    insert into zabbix.hosts_groups (hostgroupid,hostid,groupid) values ($hostgroupid,$hostid,$groupid);
    Wherein:
    • $ hostgroupid - index of the new mapping;
    • $ hostid - the index of our new host (we used it above to add the host itself);
    • $ groupid - the index of the group we need (we found out above).

    After all these manipulations, we can look into the Zabbix web interface and find our host in the Linux servers group. Looking at its parameters, we see that this host does not have sensors, triggers and graphics associated with it - there is a host, Zabbix knows about it but does not control it, since not a single data element is set. We apply the pre-created template to this host and it’s all about the hat! Zabbix independently mounts sensors and triggers on our host and starts monitoring.

    As a result, our task is solved by a combined approach:
    1. by manipulating the database we add all our terminals to Zabbix;
    2. using the host mass update function, we attach the appropriate template to the added terminals.


    PS Around the article, under the name of the host is meant the very name by which this host appears in Zabbix. And it has nothing to do with the DNS host name (although no one bothers to match them). If you use a monitoring agent, then the host name is the value of the hostname parameter in its configuration file.

    Instead of an afterword


    The failure of previous generations in manipulating the database, it is obvious, is that matching the template to the host in the database is by no means the same as applying this template from the Zabbix interface. Obviously, using the template creates mappings between the host and sensors, the host and triggers, etc., and the structure of these tables is already becoming too complicated.

    I do not provide a ready-made script here. Firstly, because I don’t have it - the practical part was done by my colleague in Perl by parsing a text file with the names and IP terminals and creating an SQL script from them. I would do everything in Ruby, most likely. Another colleague of mine said that a regular shell script is enough with ears here. However, the algorithm is simple and can be implemented as you wish.

    And finally ... When we added our terminals, we did not take into account one point - for Zabbix the number of monitored sensors suddenly increased from 80 to 4000. The poor little animal was stunned by this turn of events and fainted. Therefore, I recommend that you turn off Zabbix hosts before such a massive addition of hosts, if only for reasons of humanity. Well, of course, after adding and applying the template, Zabbix will take some time to realize and digest a bunch of controlled objects that have fallen on it, so be patient. However, this time is by an order of magnitude less than the time required to bypass a Class A subnet.

    Also popular now: