Zabbix: monitoring disk storage DELL MD36XX

Published on July 10, 2018

Zabbix: monitoring disk storage DELL MD36XX

    Hello!


    We have opened a new set for the updated course "Linux Administrator" : all the same new teachers, a dynamic program and interesting discussions. Well, at the same time we share an interesting note from the real case of one of the teachers - Alexey Tsykunova .


    Go.



    Task


    You must configure the monitoring of the load on the disk storage DELL MD36XX.
    There is a problem - the shelves are not able to give data on snmp. By the way, similar problems are also found in IBM, HP and other vendors.


    Environment


    The vendors themselves provide client software. The so-called StorageManager's. Some of them are only under Windows, but recently appeared under Linux. In my case, the software was installed under Linux and incorporates the SMcli console utility. With the help of SMcli, you can remove statistics to a file:


    SMcli -n StorageName -S -quick -c "save storageArray performanceStats file=\\"/var/log/md36xx.stat\\";"

    And it looks like this


    "Performance Monitor Statistics for Storage Array: StorageName - Date/Time: 1/17/18 9:37:04 PM - Polling interval in seconds: 5"
    "Objects","Total IOs","Read %","Primary Read Cache Hit %","Primary Write Cache Hit %","SSD Read Cache Hit %","Current MBs/sec","Maximum MBs/sec","Current IOs/sec","Ma
    ximum IOs/sec","Minimum IOs/sec","Average IOs/sec","Minimum MBs/sec","Average MBs/sec","Current IO Latency","Maximum IO Latency","Minimum IO Latency","Average IO Late
    ncy"
    "Capture Iteration: 1","","","","","","","","","","","","","","","","",""
    "Date/Time: 1/17/18 9:37:05 PM","","","","","","","","","","","","","","","","",""
    "Storage Array StorageName ","6396.0","52.0","64.6","100.0","0.0","31.2","31.2","1279.0","1279.0","1279.0","1279.0","31.2","31.2","-","-","-","-"
    "RAID Controller Module 0","4043.0","47.3","55.3","100.0","0.0","27.3","27.3","808.0","808.0","808.0","808.0","27.3","27.3","-","-","-","-"
    "RAID Controller Module 1","2353.0","60.0","77.3","100.0","0.0","3.9","3.9","470.0","470.0","470.0","470.0","3.9","3.9","-","-","-","-"
    "Disk Pool Disk_Pool_1","5181.0","59.6","69.3","100.0","0.0","29.1","29.1","1036.0","1036.0","1036.0","1036.0","29.1","29.1","-","-","-","-"
    "Disk Pool Disk_Pool_vps","1215.0","19.4","3.8","100.0","0.0","2.1","2.1","243.0","243.0","243.0","243.0","2.1","2.1","-","-","-","-"
    "Virtual Disk VPS1","645.0","1.6","30.0","100.0","0.0","0.9","0.9","129.0","129.0","129.0","129.0","0.9","0.9","1.7","1.7","1.7","1.7"
    ...

    Solutions


    This information can be parsed and sent to zabbix, and there are several options for sending


    Zabbix agent and UserParameter, in which you can paint each of the parameters of interest, for example:


    UserParameter = dell.md.discovery, /path/script_discovery
    UserParameter = dell.md.totalio[*], /path/script2 $1
    UserParameter = dell.md.currmb[*], /path/script3 $1

    Zabbix_sender, which allows you to send all the parameters at once with a single script call. This is the option I chose.


    Settings in zabbix


    In zabbix, you need to configure a new host and create a discover rule in it



    Add filter to it:



    And add item prototypes


    item_proto.png


    Here is more detailed how exactly the prototype is written:



    Link to Zabbix


    Data can be sent via zabbix_sender -i <key_value_file>


    File format:


    HOST key value

    And as a value, you can pass json for discovery in the format:


    { 'data' :[
                   {'{#MDDEV}': 'drive1'},
                   {'{#MDDEV}': 'drive1'},
                 ]
    }

    Where {#MDDEV} is a macro through which we find the names of our devices / disks from which we are going to take statistics.


    Total algorithm of actions is as follows:


    - remove statistics through SMcli , save to a file;
    - Parsing the resulting file;
    - Generation file for zabbix_sende r.


    All this can be implemented with a bash script . But the easiest thing for me was to implement one Python script, which would then be called from the crown once a minute. Difficulties arose only with the module of sending in zabbix. It was installed both under 2.7 and under 3.6, but it was launched only under 3.6.


    Script can be taken in GitHub


    As always, we are waiting for questions and comments here or in an open lesson .