CsvLogWriter plugin for JMeter
Introduction
Surely everyone who used standard listeners in JMeter encountered the following limitations:
- standard listeners do not allow to receive detailed information on all subqueries prescribed in the test plan;
- standard listeners output information in XML format, which complicates further analysis of logs using Excel and Pandas.
To circumvent these restrictions, it was decided to re-create the log files using the new CsvLogWriter plugin.
Tasks
It was necessary to develop a plugin for JMeter, which would have the following features:
- display the full text of the error in string format;
- capturing data on child subqueries.
In standard listeners for JMeter, fixing the full error content is possible in XML format, which is inconvenient for analysis. There was a need to save the error text in a string format and then write it, for example, in a CSV format to enable graphing in Excel and Pandas. Typically, the resulting log files do not display data for child subqueries. Which is extremely inconvenient when using the complex structure of the test plan. An example of the structure is shown in Figure 1.
Figure 1. The structure of the test plan
When using standard listeners, we will not be able to get the data of subqueries (embedded resources), CSVLogWriter gives us such an opportunity.
Description of the plugin functionality pflb @ CsvLogWriter
In the course of work, the pflb @ CsvLogWriter plugin for JMeter was written. The key features of this plugin include the fact that it can capture the results of the work of child subqueries and record the full text of the error, when it occurs, in plain text, and not in XML format.
Description of the log fields
The data recorded in the log file is presented in table 1.
Table 1. Log format
Log format
The data recorded in the log file is presented in table 1.
Table 1. Log format
| No. | Name | A type | Description | Sample values for the log column | unit of measurement |
|---|---|---|---|---|---|
| 1 | timeStamp | long | Request start or end time | ms | |
| 2 | elapsed | long | Request Processing Duration: endTime - startTime - idleTime | 49, 434 | ms |
| 3 | label | String | JMeter Component Name | ||
| 4 | responseCode | String | Request Response Code | “200”, “Non HTTP response code: java.net.UnknownHostException” | |
| 5 | responseMessage | String | Decoding of the response code | “OK”, “Internal Server Error” | |
| 6 | threadName | String | Stream name | "Thread Group 1-1" | |
| 7 | dataType | String | The data type in the answer, in practice, takes two values - “bin” or “text” | "Bin", "text" | |
| 8 | success | boolean | Request Success Status | true or false | |
| 9 | failureMessage | String | Error message if the Assertion component added to the Sampler is triggered . In CsvLogWriter, the field records messages from all Assertion components, the execution of which generated an error. Only the first message is recorded in the base logger. | ||
| 10 | bytes | int | The size of the response. The value and calculation algorithm depend on the sampler settings . The value may be affected by responseData.length, headersSize, bodySize. | Byte | |
| eleven | grpThreads | int | The number of active threads in the current group | ||
| 12 | allThreads | int | The number of active virtual users of all groups | ||
| thirteen | URL | String | Link | ||
| 14 | Filename | String | The name of the file in which the responses are recorded. The field is populated when using Save Response to File Listener. This Listener is rarely used, usually the column value is an empty string. | ||
| fifteen | Latency | int | Time to get the first server response. This time delay includes the time spent connecting to the server, delays due to the establishment of a secure connection, and JMeter internal delays in receiving the first bytes of the server response. If for some reason Sampler’s work has been suspended and then resumed, then the Latency value will be adjusted for the duration of the Sampler’s suspension. | ms | |
| 16 | Encoding | String | Encoding. The response encoding is returned; if no response encoding is specified, the default encoding value is returned. The default encoding value is set in "sampleresult.default.encoding". For HTTP Request, the default value is “ISO-8859-1”. | "ISO-8859-1", "UTF-8" | |
| 17 | Samplecount | int | The number of samples. For HTTP Request, the value is 1. For JMS Sampler, which subscribes to events and reads several messages at a time, the value is equal to the number of polling cycles or the number of messages read. The value is always greater than or equal to one. | 12 | PCS |
| 18 | ErrorCount | int | Number of mistakes. For HTTP Request, the value is 0 if success and equal to 1 if the request is not successful. For other samplers that process multiple messages at a time, the value may be greater than 1. | 0, 1 | PCS |
| 19 | Hostname | String | Machine name | ||
| 20 | Idletime | int | Downtime | ms | |
| 21 | Connect | int | Connection setup time | ms | |
| 22 | headersSize | int | Header Size | Byte | |
| 23 | bodySize | int | Body size | Byte | |
| 24 | contentType | String | Content Type from Response Header | ||
| 25 | endTime | long | Request End Time | ms | |
| 26 | isMonitor | boolean | Sign whether Use as Monitor is checked | true false | |
| 27 | threadName_label | String | Name of thread and JMeter component | ||
| 28 | parent_threadName_label | String | Name of thread and parent JMeter component | ||
| 29th | startTime | long | Request Start Time | ms | |
| thirty | stopTest | boolean | Whether the test is stopped is the Stop button. Also in the settings of the Thread Group there is an option “Stop Test” on error. If the stopTest column is true, then this is exactly what happened. The script terminated on the current request. | true false | |
| 31 | stopTestNow | boolean | A sign of whether the test is stopped abruptly is the Shutdown button. Also in the settings of the Thread Group there is an option “Stop Test Now” in case of an error. If the stopTestNow column is true, then this is exactly what happened. The script terminated on the current request. | true false | |
| 32 | stopThread | boolean | Whether the current thread is stopped. In the settings of the Thread Group there is an option “Stop Thread” on error. If the stopThread column is true, then this is exactly the situation. The script terminated on the current request. | true false | |
| 33 | startNextThreadLoop | boolean | Does the replay start. In the settings of the Thread Group there is an option “Start Next Thread Loop” on error. If the column "startNextThreadLoop" is true, then this is exactly what happened. The script terminated on the current request. | true false | |
| 34 | isTransactionSampleEvent | boolean | Indicates that the current event is a transaction (TransactionController). That is, it is only a grouping element. | true false | |
| 35 | transactionLevel | int | Request nesting level. If the Transaction Controller hierarchy is used or subqueries have subqueries, then this column will have a nesting level. | 0 for the root query or root Transaction Controller. 1 for a subquery whose parent is the root. | |
| 36 | responseDataAsString | String | The full content of the error in the format of the string if it occurs, if success == false, then the complete response body will be contained in responseData | ||
| 37 | requestHeaders | String | Request Headers | ||
| 38 | responseData | String | The full content of the error if it occurs, if success == false, then the responseData will contain the full response body, encoded in base64 | ||
| 39 | responseHeaders | String | Response headers | ||
| 40 | <Variable Name> | String | JMeter Variables |
The log structure extends the basic CSV log format. The basic set of parameters can be found at jmeter.apache.org/usermanual/listeners.html#csvlogformat.
The resulting CSV log can be downloaded with any basic JMeter Listener that supports CSV log loading.
Also, additional columns appeared in the log that may be useful in analysis (p. 22-35 of the Log Format table).
The bodies and headers of responses and requests are recorded only in case of errors (p. 36-39 of the Log format table).
Columns for variables are also logged : jmeter.apache.org/usermanual/listeners.html#sample_variables - a description of the ability to log the values of variables.
Interface
To start the plugin, fill in the Filename field. The Filename field contains the path to the file in which the results of the work will be recorded. You can register the directory manually, or select a file using the Browse button. If the specified file exists, a new file is created. The name of the new log file is formed by adding a postfix with the number of the log file to the original name.
There are also flags on the form. Using the flags, you can manipulate the data recorded in the log. The Additional parameters checkbox is responsible for fixing additional parameters (22-35 lines in the Log format table), Response data is responsible for fixing the error text (36-39 lines in the Log format table), User variables is responsible for fixing user variables.
The plugin interface is shown in Figure 2.
Figure 2. CSVLogWriter plugin interface
Comparison of CSVLogWriter plugin and Simple Data Writer
In this section, we will try to compare the CsvLogWriter plugin and the standard Simple Data Writer listener.
The composition of the logged data
The Simple Data Writer Listener gives the user the ability to customize a list of committed data. Figure 3 shows the output data settings window.
Figure 3. The settings window for the fixed data of the Simple Data Writer
listener The CsvLogWriter plugin always displays a basic list of parameters (similar to Simple Data Writer) and allows you to configure the list of additional data using 3 check boxes on the form:
- Additional parameters - additional parameters (22-35 lines in the Log format table);
- Response data - error texts (36-39 lines in the Log format table);
- User variables - user variables (for output you need to run JMeter with the -Jsample_variables switch).
A detailed list of recorded data is hardcoded in the code. But, as can be seen in Figure 3, Simple Data Writer cannot output error text in string format. The full text of the error is recorded only in XML format. For this reason, in the case when we need a text of errors, we have to keep 2 logs - 1 in CSV format (if further processing of the log is necessary with graphing in Excel or Pandas) and 1 in XML format.
Subquery Logging
Also, the Simple Data Writer listener does not record the results of the work of child subqueries; accordingly, such a log file cannot be called exhaustive. For a clear comparison of the volume of output data, run a test corresponding to the test plan in Figure 1 and look at the log files. The SimpleDataWriter log file is shown in Figure 4.
Figure 4. Simple Data Writer log file
As shown in Figure 4, SimpleDataWriter does not display information about embedded resources. In turn, the CsvLogWriter plugin displays more information through the processing of child subqueries. The contents of the CsvLogWriter plugin log file are shown in Figure 5.
Figure 5. CsvLogWriter log file
Performance comparison
We also analyzed the performance of the SimpleOccured method, which is responsible for event processing. Profiling was done in Java VisualVM. No subqueries were used in the test plan. For SimpleDataWriter, testing was started with writing to 1 CSV file and writing to 2 files - CSV and XML. The number of virtual users was 10, the number of repetitions was 100. The results of the comparison are shown in table 2.
Table 2. Comparison of the performance of SimpleDataWriter and the plugin CsvLogWriter
| CsvLogWriter | SimpleDataWriter (CSV + XML) | SimpleDataWriter (CSV) |
| Duration (ms), number of calls | Duration (ms), number of calls | Duration (ms), number of calls |
| 215, 2000 | 23076, 4000 | 101, 2000 |
Conclusions:
- CsvLogWriter is 10 times faster than SimpleDataWriter (XML with maximum granularity);
- CsvLogWriter is 2 times slower than SimpleDataWriter (CSV with maximum detail);
- SimpleDataWriter (XML with maximum granularity) is 20 times slower than SimpleDataWriter (CSV with maximum granularity).
Link to the plugin
github.com/pflb/Jmeter.Plugin.CsvLogWriter