April 2, 2019 at 09:38Zabbix 4.2 released
Our team is very happy to share the news that the release of the free open source monitoring system Zabbix 4.2 !
Is version 4.2 the answer to the main question of life, the universe and monitoring in general? Let's get a look!
Recall that Zabbix is a universal system for monitoring the performance and availability of servers, engineering and network equipment, applications, databases, virtualization systems, containers, IT services, and web services.
Zabbix implements a full cycle from collecting data, processing and converting them, analyzing received data, and ending with storing this data, visualizing and sending alerts using escalation rules. The system also provides flexible options for expanding methods for collecting data and alerts, as well as automation capabilities through the API. A single web interface provides centralized management of monitoring configurations and the distribution of access rights to various user groups. The project code is freely distributed under the GPLv2 license .
Zabbix 4.2 is a new non-LTS version with a shortened official support period. Users who are guided by a long cycle of operation of software products are recommended to use LTS versions, such as 3.0 and 4.0.
So, let's talk about the latest updates and major improvements of version 4.2:
In addition to the existing official packages, we also offer new builds for:
Zabbix can collect data in various ways (push / pull) from different data sources. These are JMX, SNMP, WMI, HTTP / HTTPS, RestAPI, XML Soap, SSH, Telnet, agents and scripts, and other sources. Now meet Prometheus support!
Strictly speaking, data collection from Prometheus exporters was possible earlier thanks to the HTTP / HTTPS data element type and regular expressions.
However, the new version allows working with Prometheus as efficiently as possible due to the built-in support for PromQL query language. And the use of dependent metrics allows you to collect and process data most efficiently: once you apply for data, and then lay them out according to the necessary metrics. Get the value of a specific metric
It is important to note that now low-level discovery can use the collected data to automatically create metrics. In this case, Zabbix converts the received data to the JSON format, which is very convenient to work with. We find metrics using a filter in the PromQL query language. At the moment, there are more than 300 integrations and recipes for monitoring third-party services and applications using Zabbix. Support for Prometheus will add a whole range of applications that have official or community-supported Prometheus exporters. This is a monitoring of popular services, containers and cloud resources.
Do we want to detect problems as quickly as possible? Of course, no doubt! Most often, this approach leads to the fact that we need to interrogate devices and collect data too often, which leads to a greater load on the monitoring system. How to avoid this?
We implemented the throttling mechanism in the preprocessing rules. Trotting, in fact, allows us to skip the same values.
Suppose we monitor the status of a critical application. Every second, we check whether our application is functioning or not. In this case, Zabbix receives a continuous stream of data from 1 (works) and 0 (does not work). For example: 111111111111000111111111111111 ...
When everything is in order with our application, then Zabbix receives a stream from only one unit. Do I need to process them? In general, no, because we are only interested in changing the state of the application, we do not want to collect and store so much data. So, trotting allows you to skip a value if it is identical to the previous one. As a result, we get only data on the state change, for example, 01010101 ... There is enough information to detect problems!
Zabbix simply ignores the missing values, they are not written to the history and do not affect the triggers in any way. From the point of view of Zabbix, missing values do not exist. Ignore duplicate values
Wow! Now we can very often poll devices, while instantly detecting problems without storing unnecessary information in the database.
But what about the graphics? They will be empty due to lack of data! And how do you know if Zabbix collects data if most of this data is skipped?
We thought about that! Zabbix offers another type of throttling, throttling with heartbeat. Once a minute we check if the metric is alive
In this case, Zabbix, despite the repeated flow of data, will save at least one value in the specified time interval. If data is collected once per second, and the interval is set to one minute, then Zabbix will turn a one-second stream of ones into a one-minute stream. It is easy to see that this leads to a 60-fold compression of the received data.
Now we are sure that the data is being collected, the nodata () trigger function works and everything is in order with the charts!
None of us wants to collect erroneous or inaccurate data. For example, we know that a temperature sensor should return data between 0 ° C and 100 ° C and any other value should be considered erroneous and / or ignored.
Now it is possible with the help of built-in preprocessing rules for data validation for matching or not matching regular expressions, range of values, JSONPath and XMLPath.
Now we can control the response to the error. If the temperature is out of range, we can simply ignore this value, set the default value (for example, 0 ° C), or define our own error message, for example, “Sensor is damaged” or “Replace the battery.” The temperature should be from 0 to 100, ignore the rest
A good example of using validation is the ability to check the input for the presence of an error message and set this error for the entire metric. This is a very useful feature when retrieving data from external APIs.
If the built-in preprocessing rules are not enough for us, then now we offer complete freedom using arbitrary JavaScript scripts! Just one line of code to convert degrees Fahrenheit to degrees Celsius This opens up unlimited possibilities for processing incoming data. The practical benefit of this functionality is that now we do not need external scripts that we used for any data operations. Now all this can be done using JavaScript. Now data transformation, aggregation, filters, arithmetic and logical operations and much much more are possible! We extract useful information from the output of Apache mod_status!
Now we do not need to wonder how our complex preprocessing scripts work. There was a convenient check of the correctness of the preprocessing directly from the interface!
Prior to Zabbix 4.2, only Zabbix server was engaged in preprocessing, which limited the possibility of using proxies for load balancing.
Starting with version Zabbix 4.2, we get incredibly efficient load scaling due to support of pro-processing on the proxy side. Now proxies are doing it! In combination with throttling, this approach allows for high-frequency large-scale monitoring and millions of checks per second, without loading the central Zabbix server. Proxies process huge amounts of data, while only a small fraction of them reaches the Zabbix server due to throttling, one or two orders of magnitude less.
Recall that low-level detection (LLD) is a very powerful mechanism for automatically detecting any kind of monitoring resources (file systems, processes, applications, services, etc.) and automatically creating data elements, triggers, network nodes on their basis and other objects. This is incredibly time-saving, simplifies configuration, and allows you to use one template for hosts that have different monitoring resources.
Low-level discovery required specially crafted JSON input. That's it, it won’t be anymore!
Zabbix 4.2 allows low-level discovery (LLD) to use arbitrary JSON formatted data. Why is it important? This allows, without resorting to scripts, to communicate, for example, with external APIs and use the information received to automatically create network nodes, data elements and triggers.
Together with JavaScript support, this creates fantastic opportunities for creating templates for working with various data sources, such as, for example, cloud APIs, application APIs, data in XML, CSV formats and so on and so on. Associating JSON with process information with LLD The possibilities are really endless!
What is TimescaleDB? This is a regular PostgreSQL plus extension module from the TimescaleDB team. TimescaleDB promises better performance with more efficient algorithms and data structures.
In addition, another advantage of TimescaleDB is the automatic partitioning of tables with history. TimescaleDB is speed and ease of maintenance! Although, I should note that our team has not yet made a serious comparison of performance with regular PostgreSQL.
At the moment, TimescaleDB is a fairly young and fast-growing product. Use with caution!
If earlier tags could only be controlled at the trigger level, now tag management is much more flexible. Zabbix supports tags for templates and hosts!
All detected problems receive tags not only of the trigger, but also of the host, as well as the templates of this host. Define the tags for the host
Zabbix 4.2 allows you to filter hosts by name using regular expressions. This makes it possible to create different discovery scripts for different groups of hosts. It is especially convenient if we use complex device naming rules.
Another improvement is the naming of hosts. Now you can manage device names during network discovery and get the device name from the metric value.
This is a very necessary functionality, especially for network discovery using SNMP and Zabbix agent. Automatically assign the local hostname to the visible name
Now, right from the Web interface, you can send yourself a test message and check if the notification method works. This functionality is especially useful for checking scripts combining Zabbix with various warning systems, task systems, and other external programs and APIs.
Now you can remotely monitor internal Zabbix server and proxy metrics (performance and health metrics of Zabbix components).
What is it for? Functionality allows you to monitor internal metrics of servers and proxies from the outside, allows you to quickly detect and notify about problems even if the components themselves are overloaded or, for example, a large amount of unsent data is sent to the proxy.
Now we are not limited to simple text and can form beautiful e-mail messages, thanks to the support of the HTML format. It's time to learn HTML + CSS! Messages are easier to read even with minimal use of HTML
There is support for a whole set of new macros in custom URLs for better integration of maps with external systems. This allows one or two clicks on the host icon to open, for example, a ticket in the task system. One click to open a ticket in Jira
Why do you need this - you ask. This allows the use of basic metric data for both detection and direct data collection. For example, in the case of collecting data from the Prometheus exporter, Zabbix will make one HTTP request and immediately use the received information for all dependent data elements: metric values and low-level discovery rules.
Support for animated GIF images on maps has appeared for more visible visualization of problems. Problem devices become more visible
In Web monitoring, the ability to select data from the received HTTP header has been added.
This allows you to create multi-step scripts for web monitoring or monitoring third-party APIs using the authorization token obtained in one of the steps. Extract AuthID from HTTP header
Zabbix Sender now sends data to all IP addresses from the ServerActive parameter of the agent configuration file.
The trigger configuration page has got an advanced filter for quick and convenient selection of triggers according to specified criteria. Choose triggers related to the K8S service
It's simple, now Zabbix shows the exact time when you mouse over the graph.
To migrate from earlier versions, you only need to install new binary files (server and proxy) and a new interface. Zabbix will automatically update the database. Installation of new agents is not required.
We hold free webinars for those who want to learn more about Zabbix 4.2 and be able to ask questions to the Zabbix team. Sign up!
Do not forget about the popular Telegram channel of the Zabbix community, where you can always get advice and answers to your questions in Russian from more experienced colleagues, and, if you're lucky, from the Zabbix developers themselves. For beginners, we recommend a group for beginners .
- Release notes
- Upgrade notes
- Original article
Is version 4.2 the answer to the main question of life, the universe and monitoring in general? Let's get a look!
Recall that Zabbix is a universal system for monitoring the performance and availability of servers, engineering and network equipment, applications, databases, virtualization systems, containers, IT services, and web services.
Zabbix implements a full cycle from collecting data, processing and converting them, analyzing received data, and ending with storing this data, visualizing and sending alerts using escalation rules. The system also provides flexible options for expanding methods for collecting data and alerts, as well as automation capabilities through the API. A single web interface provides centralized management of monitoring configurations and the distribution of access rights to various user groups. The project code is freely distributed under the GPLv2 license .
Zabbix 4.2 is a new non-LTS version with a shortened official support period. Users who are guided by a long cycle of operation of software products are recommended to use LTS versions, such as 3.0 and 4.0.
So, let's talk about the latest updates and major improvements of version 4.2:
More official platforms
In addition to the existing official packages, we also offer new builds for:
- RaspberryPi, Mac OS / X, SUSE Enterprise Linux Server 12
- MSI for Windows Agent
- Docker images
Built-in Prometheus Support for Application Monitoring
Zabbix can collect data in various ways (push / pull) from different data sources. These are JMX, SNMP, WMI, HTTP / HTTPS, RestAPI, XML Soap, SSH, Telnet, agents and scripts, and other sources. Now meet Prometheus support!
Strictly speaking, data collection from Prometheus exporters was possible earlier thanks to the HTTP / HTTPS data element type and regular expressions.
However, the new version allows working with Prometheus as efficiently as possible due to the built-in support for PromQL query language. And the use of dependent metrics allows you to collect and process data most efficiently: once you apply for data, and then lay them out according to the necessary metrics. Get the value of a specific metric
It is important to note that now low-level discovery can use the collected data to automatically create metrics. In this case, Zabbix converts the received data to the JSON format, which is very convenient to work with. We find metrics using a filter in the PromQL query language. At the moment, there are more than 300 integrations and recipes for monitoring third-party services and applications using Zabbix. Support for Prometheus will add a whole range of applications that have official or community-supported Prometheus exporters. This is a monitoring of popular services, containers and cloud resources.
Efficient high frequency monitoring
Do we want to detect problems as quickly as possible? Of course, no doubt! Most often, this approach leads to the fact that we need to interrogate devices and collect data too often, which leads to a greater load on the monitoring system. How to avoid this?
We implemented the throttling mechanism in the preprocessing rules. Trotting, in fact, allows us to skip the same values.
Suppose we monitor the status of a critical application. Every second, we check whether our application is functioning or not. In this case, Zabbix receives a continuous stream of data from 1 (works) and 0 (does not work). For example: 111111111111000111111111111111 ...
When everything is in order with our application, then Zabbix receives a stream from only one unit. Do I need to process them? In general, no, because we are only interested in changing the state of the application, we do not want to collect and store so much data. So, trotting allows you to skip a value if it is identical to the previous one. As a result, we get only data on the state change, for example, 01010101 ... There is enough information to detect problems!
Zabbix simply ignores the missing values, they are not written to the history and do not affect the triggers in any way. From the point of view of Zabbix, missing values do not exist. Ignore duplicate values
Wow! Now we can very often poll devices, while instantly detecting problems without storing unnecessary information in the database.
But what about the graphics? They will be empty due to lack of data! And how do you know if Zabbix collects data if most of this data is skipped?
We thought about that! Zabbix offers another type of throttling, throttling with heartbeat. Once a minute we check if the metric is alive
In this case, Zabbix, despite the repeated flow of data, will save at least one value in the specified time interval. If data is collected once per second, and the interval is set to one minute, then Zabbix will turn a one-second stream of ones into a one-minute stream. It is easy to see that this leads to a 60-fold compression of the received data.
Now we are sure that the data is being collected, the nodata () trigger function works and everything is in order with the charts!
Validation of collected data and error handling
None of us wants to collect erroneous or inaccurate data. For example, we know that a temperature sensor should return data between 0 ° C and 100 ° C and any other value should be considered erroneous and / or ignored.
Now it is possible with the help of built-in preprocessing rules for data validation for matching or not matching regular expressions, range of values, JSONPath and XMLPath.
Now we can control the response to the error. If the temperature is out of range, we can simply ignore this value, set the default value (for example, 0 ° C), or define our own error message, for example, “Sensor is damaged” or “Replace the battery.” The temperature should be from 0 to 100, ignore the rest
A good example of using validation is the ability to check the input for the presence of an error message and set this error for the entire metric. This is a very useful feature when retrieving data from external APIs.
Any data transformation using JavaScript
If the built-in preprocessing rules are not enough for us, then now we offer complete freedom using arbitrary JavaScript scripts! Just one line of code to convert degrees Fahrenheit to degrees Celsius This opens up unlimited possibilities for processing incoming data. The practical benefit of this functionality is that now we do not need external scripts that we used for any data operations. Now all this can be done using JavaScript. Now data transformation, aggregation, filters, arithmetic and logical operations and much much more are possible! We extract useful information from the output of Apache mod_status!
Testing preprocessing
Now we do not need to wonder how our complex preprocessing scripts work. There was a convenient check of the correctness of the preprocessing directly from the interface!
We process millions of metrics per second!
Prior to Zabbix 4.2, only Zabbix server was engaged in preprocessing, which limited the possibility of using proxies for load balancing.
Starting with version Zabbix 4.2, we get incredibly efficient load scaling due to support of pro-processing on the proxy side. Now proxies are doing it! In combination with throttling, this approach allows for high-frequency large-scale monitoring and millions of checks per second, without loading the central Zabbix server. Proxies process huge amounts of data, while only a small fraction of them reaches the Zabbix server due to throttling, one or two orders of magnitude less.
Simpler low-level detection
Recall that low-level detection (LLD) is a very powerful mechanism for automatically detecting any kind of monitoring resources (file systems, processes, applications, services, etc.) and automatically creating data elements, triggers, network nodes on their basis and other objects. This is incredibly time-saving, simplifies configuration, and allows you to use one template for hosts that have different monitoring resources.
Low-level discovery required specially crafted JSON input. That's it, it won’t be anymore!
Zabbix 4.2 allows low-level discovery (LLD) to use arbitrary JSON formatted data. Why is it important? This allows, without resorting to scripts, to communicate, for example, with external APIs and use the information received to automatically create network nodes, data elements and triggers.
Together with JavaScript support, this creates fantastic opportunities for creating templates for working with various data sources, such as, for example, cloud APIs, application APIs, data in XML, CSV formats and so on and so on. Associating JSON with process information with LLD The possibilities are really endless!
TimescaleDB Support
What is TimescaleDB? This is a regular PostgreSQL plus extension module from the TimescaleDB team. TimescaleDB promises better performance with more efficient algorithms and data structures.
In addition, another advantage of TimescaleDB is the automatic partitioning of tables with history. TimescaleDB is speed and ease of maintenance! Although, I should note that our team has not yet made a serious comparison of performance with regular PostgreSQL.
At the moment, TimescaleDB is a fairly young and fast-growing product. Use with caution!
Easy tag management
If earlier tags could only be controlled at the trigger level, now tag management is much more flexible. Zabbix supports tags for templates and hosts!
All detected problems receive tags not only of the trigger, but also of the host, as well as the templates of this host. Define the tags for the host
More flexible auto-registration
Zabbix 4.2 allows you to filter hosts by name using regular expressions. This makes it possible to create different discovery scripts for different groups of hosts. It is especially convenient if we use complex device naming rules.
More flexible network discovery
Another improvement is the naming of hosts. Now you can manage device names during network discovery and get the device name from the metric value.
This is a very necessary functionality, especially for network discovery using SNMP and Zabbix agent. Automatically assign the local hostname to the visible name
Checking the operability of notification methods
Now, right from the Web interface, you can send yourself a test message and check if the notification method works. This functionality is especially useful for checking scripts combining Zabbix with various warning systems, task systems, and other external programs and APIs.
Remote monitoring of Zabbix infrastructure components
Now you can remotely monitor internal Zabbix server and proxy metrics (performance and health metrics of Zabbix components).
What is it for? Functionality allows you to monitor internal metrics of servers and proxies from the outside, allows you to quickly detect and notify about problems even if the components themselves are overloaded or, for example, a large amount of unsent data is sent to the proxy.
HTML format support for email messages
Now we are not limited to simple text and can form beautiful e-mail messages, thanks to the support of the HTML format. It's time to learn HTML + CSS! Messages are easier to read even with minimal use of HTML
Access to external systems from network interface cards
There is support for a whole set of new macros in custom URLs for better integration of maps with external systems. This allows one or two clicks on the host icon to open, for example, a ticket in the task system. One click to open a ticket in Jira
The discovery rule may be a dependent data item.
Why do you need this - you ask. This allows the use of basic metric data for both detection and direct data collection. For example, in the case of collecting data from the Prometheus exporter, Zabbix will make one HTTP request and immediately use the received information for all dependent data elements: metric values and low-level discovery rules.
A new way to visualize problems on maps
Support for animated GIF images on maps has appeared for more visible visualization of problems. Problem devices become more visible
Retrieving data from HTTP headers in Web monitoring
In Web monitoring, the ability to select data from the received HTTP header has been added.
This allows you to create multi-step scripts for web monitoring or monitoring third-party APIs using the authorization token obtained in one of the steps. Extract AuthID from HTTP header
Zabbix Sender uses all IP addresses
Zabbix Sender now sends data to all IP addresses from the ServerActive parameter of the agent configuration file.
Convenient new filter in trigger configuration
The trigger configuration page has got an advanced filter for quick and convenient selection of triggers according to specified criteria. Choose triggers related to the K8S service
We show the exact time
It's simple, now Zabbix shows the exact time when you mouse over the graph.
Other innovations
- Implemented a more predictable algorithm for changing the order of widgets in the dashboard (panel)
- The ability to mass change the parameters of the prototypes of data elements
- IPv6 support for DNS checks: "net.dns" and "new.dns.record"
- Added skip parameter for vmware.eventlog checks
- The preprocessing step execution error includes the step number
How to upgrade?
To migrate from earlier versions, you only need to install new binary files (server and proxy) and a new interface. Zabbix will automatically update the database. Installation of new agents is not required.
We hold free webinars for those who want to learn more about Zabbix 4.2 and be able to ask questions to the Zabbix team. Sign up!
Do not forget about the popular Telegram channel of the Zabbix community, where you can always get advice and answers to your questions in Russian from more experienced colleagues, and, if you're lucky, from the Zabbix developers themselves. For beginners, we recommend a group for beginners .
useful links
- Release notes
- Upgrade notes
- Original article