JMeter: Forget BeanShell Sampler
Using standard test plan elements in Jmeter, you can do a lot, but not all. To expand the functionality and implement more complex logic, it is customary to use BeanShell Sampler - it has somehow historically developed around the world. And all over the world they periodically suffer from this, but continue to eat a cactus.
In principle, in order to achieve the best results in terms of the performance of the load station itself, it is most effective to implement your Java Request or even write your own sampler. But personally, I do not like this solution because of:
So I always tried to avoid such a decision (and so far avoided!). I personally like it when the code is in the test plan itself, and you can track the time it takes to execute this code using JMeter. In this sense, the BeanShell sampler approach itself is very convenient, only its implementation is bad. I’ll tell you why.
Once we developed a highly loaded test plan using code inserts in BeanShell Sampler. Everything went fine at the development stage, but when we started trial runs of load tests, we encountered very unpleasant behavior. On a certain number of threads, everything worked fine. But with an increase in the number of threads, some inadequacy began. The test greatly lacked the target load intensity, and reports showed that the processing time of individual BeanShell samples reached tens of seconds! Moreover, the BeanShell samplers were quite few - one or two per thread group, and not to say that something complicated happened there. Studying the performance parameters of the loading station itself did not reveal any problems: CPU load was 20-30 percent, there was enough memory for the JMeter process, and the garbage collector cleaned it in a timely manner. It is clear that the problem is in JMeter's program code itself or in the implementation of the BeanShell interpreter. Playing with a daw Reset bsh.Interpreter gives nothing; not only that: in one place they write that it is better to install it so that there is no memory overrun, in another - which is better to remove for performance reasons.
Messages about such problems are occasionally found on the JMeter forums and come in the Apache JMeter User mailing list. Colleagues also complained about the behavior of some tests, but tended to attribute the problem to the tool itself.
JMeter has a very similar sampler called the JSR223 Sampler. Not even just a sampler, but a whole family: Sampler, Timer, Pre- and PostProcessor, Assertion, Timer and Listener. The documentation for it begins with very encouraging words that this sampler will achieve significant performance improvements. But the attentive reader is immediately upset: to achieve this effect, you should choose a scripting engine that supports compilation. Right there, nearby, it is indicated that the Java engine is not.
Regarding Java, I will say even more: it is implemented by the same engine as BeanShell. This can be easily verified by causing an error in the executable code. In the exception stack in the log, you will see that the bsh interpreter is called both there and there. Therefore, there will be absolutely no difference between JSR223 / java and BeanShell Sampler. Nothing is said about the rest of the engines, but they are also all interpreted. Thus, in the standard JMeter package there are no engines on which one could get profit from compilation.
The only compiled scripting engine mentioned in the documentation is Groovy . There are other engines that support JSR223. I tried Scala, was horrified at how slowly this bunch worked and left this topic until better times.(Note: the point here is probably not Scala, but the implementation of the JSR 223 standard and the implementation of the Compilable interface .)
To enable Groovy support, you need to download the latest version of binaries from the project site or here . From the archive we need only one file: embeddable \ groovy-all- {version} .jar. Unpack it into the lib folder of the Zhimetr. After restarting the program, Groovy will appear in the list of available JSR223 languages:
After we remade all BeanShell samplers of our test plan on JSR223 + Groovy, a miracle happened: everything began to work as it should (well, or at least as we programmed), without brakes, and the CPU load became even lower. The response time of the JSR223 samplers became orders of magnitude lower and the test reached the required load.
If we return to what we started with - various ways of implementing additional program logic - then the solution with Groovy should be enough for almost all cases, except for those where you really need to squeeze percentages. Groovy scripts are compiled into regular Java bytecode and executed in the context of each thread as if it were native Java code (but remember that it has its own compiler, and there is still some overhead for calling the engine). The guys from Blazemeter compared the speed of various implementation options and came to the conclusion that the Groovy code is only slightly inferior in speed to the code in pure Java.
I also did a little experiment. I wrote a small fragment that performed some artificial calculations in integer arithmetic:
Dependence on input data (Parameters) and logging were added just in case, to prevent any tricky compilers and interpreters from optimizing the code, excluding its execution at all or caching the result. Moreover, Parameters was also unique. On my laptop with Core i7 with 100 threads of 1000 iterations, each result was as follows:
The Groovy gap is so significant that it is even hard to believe, nevertheless, judging by the log, everything worked correctly.
Groovy's big plus is that in 95% of cases, arbitrary Java code is valid Groovy code. Even BeanShell syntax is further away from the current Java standard (for example, in BeanShell you have to be perverted when calling functions with an arbitrary number of arguments). If you are not interested in learning all its possibilities right now, then you don’t need to. On the other hand, if you master it , you will surely be able to increase your effectiveness.
If you used the global namespace bsh.shared in BeanShell, then there is a small ambush: Groovy has nothing of the kind. Fortunately, this problem is easy to solve on your own. For this, 10 lines of code are written:
In fact, this is a singleton, which always (to each thread) will return the same object. Further it is collected in the jar and placed in the lib meter folder. Since the class is declared in the global namespace (yes, I am reprehensible for this), in Groovy code, without any import, you can use SharedHashMap to put something there:
When you need to pick up, then similarly:
* Groovy does not have to declare variable types, as well as semicolons.
Suppose you already have a test plan that already has many BeanShell samplers, and you found this article because you are having a problem. You want to switch to Groovy. Connecting Groovy is described above and will take you no more than five minutes.
First of all, create a JSR223 Sampler and transfer the code from BeanShell into it. You can significantly simplify your life if you can unify the code and select it in a separate file by specifying it in the File Name field. Then you just need to paste the JSR223 samplers in the right places with Copy / Paste. If not, copy the code from BeanShell in each case.
It is important to note here that JMeter will compile the code entered in the sampler itself only if the compilation key is specified (the Compilation Cache Key field). It should be just a line that is unique within the test plan. For scripts connected via files, you do not need to enter a compilation key, as it uses the full path to the file.
There is one subtlety to Groovy syntax. First, there are two types of strings:
See here for more details . Groovy strings have the ability to use expressions like $ {expression}, which are automatically expanded inside the strings to the value expression. This is a pretty convenient point, but it surprisingly matches the syntax with reference to JMeter variables. So if you write in Groovy
and at the same time the usual JMeter variable with the name currId is defined in the current thread, then its value will be substituted directly into the script. In addition, it is substituted once, because after that, the code will be compiled, and the result will be cached. Therefore, you must ensure that the variable names used in such expressions do not overlap with JMeter variables. And if you really need to pass the value to the JSR223 sampler, then you need to use the Parameters field for this.
When using an external file as source code, JMeter variable substitution does not occur (they occur only in the fields), but you can use Parameters.
If you do not plan to use the capabilities of Groovy strings, then it is advisable to use Java strings (i.e. in single quotes). In addition, it will be better for performance, although a penny, of course.
The behavior of the BeanShell sampler illustrates a typical interpreter problem: the low speed of the interpreted code. If you have only a few lines in BeanShell, you probably won't notice the problems, but you will definitely notice if there is a lot of code or if there are loops. Exactly the same problem was observed in the LoadRunner interpreter.
If you haven’t had any problems running the test using BeanShell so far, I would recommend playing it safe and not creating them for yourself in the future. Instead, it is best to use JSR223 + Groovy immediately, thus reducing the likelihood of performance problems with load stations.
Important points to take out of the article.
How without him?
In principle, in order to achieve the best results in terms of the performance of the load station itself, it is most effective to implement your Java Request or even write your own sampler. But personally, I do not like this solution because of:
- b of lshih effort to develop solutions;
- non-transparent support for the solution (the code lies elsewhere; it needs to be recompiled using additional actions);
- possible loss of bonuses that JMeter gives (for example, if you have implemented a request to the DBMS in your code and then an HTTP request, you will see them in the statistics in total, and not separately).
So I always tried to avoid such a decision (and so far avoided!). I personally like it when the code is in the test plan itself, and you can track the time it takes to execute this code using JMeter. In this sense, the BeanShell sampler approach itself is very convenient, only its implementation is bad. I’ll tell you why.
What is bad BeanShell Sampler
Once we developed a highly loaded test plan using code inserts in BeanShell Sampler. Everything went fine at the development stage, but when we started trial runs of load tests, we encountered very unpleasant behavior. On a certain number of threads, everything worked fine. But with an increase in the number of threads, some inadequacy began. The test greatly lacked the target load intensity, and reports showed that the processing time of individual BeanShell samples reached tens of seconds! Moreover, the BeanShell samplers were quite few - one or two per thread group, and not to say that something complicated happened there. Studying the performance parameters of the loading station itself did not reveal any problems: CPU load was 20-30 percent, there was enough memory for the JMeter process, and the garbage collector cleaned it in a timely manner. It is clear that the problem is in JMeter's program code itself or in the implementation of the BeanShell interpreter. Playing with a daw Reset bsh.Interpreter gives nothing; not only that: in one place they write that it is better to install it so that there is no memory overrun, in another - which is better to remove for performance reasons.
Messages about such problems are occasionally found on the JMeter forums and come in the Apache JMeter User mailing list. Colleagues also complained about the behavior of some tests, but tended to attribute the problem to the tool itself.
What to do
JMeter has a very similar sampler called the JSR223 Sampler. Not even just a sampler, but a whole family: Sampler, Timer, Pre- and PostProcessor, Assertion, Timer and Listener. The documentation for it begins with very encouraging words that this sampler will achieve significant performance improvements. But the attentive reader is immediately upset: to achieve this effect, you should choose a scripting engine that supports compilation. Right there, nearby, it is indicated that the Java engine is not.
Regarding Java, I will say even more: it is implemented by the same engine as BeanShell. This can be easily verified by causing an error in the executable code. In the exception stack in the log, you will see that the bsh interpreter is called both there and there. Therefore, there will be absolutely no difference between JSR223 / java and BeanShell Sampler. Nothing is said about the rest of the engines, but they are also all interpreted. Thus, in the standard JMeter package there are no engines on which one could get profit from compilation.
The only compiled scripting engine mentioned in the documentation is Groovy . There are other engines that support JSR223. I tried Scala, was horrified at how slowly this bunch worked and left this topic until better times.(Note: the point here is probably not Scala, but the implementation of the JSR 223 standard and the implementation of the Compilable interface .)
To enable Groovy support, you need to download the latest version of binaries from the project site or here . From the archive we need only one file: embeddable \ groovy-all- {version} .jar. Unpack it into the lib folder of the Zhimetr. After restarting the program, Groovy will appear in the list of available JSR223 languages:
After we remade all BeanShell samplers of our test plan on JSR223 + Groovy, a miracle happened: everything began to work as it should (well, or at least as we programmed), without brakes, and the CPU load became even lower. The response time of the JSR223 samplers became orders of magnitude lower and the test reached the required load.
Groovy Performance
If we return to what we started with - various ways of implementing additional program logic - then the solution with Groovy should be enough for almost all cases, except for those where you really need to squeeze percentages. Groovy scripts are compiled into regular Java bytecode and executed in the context of each thread as if it were native Java code (but remember that it has its own compiler, and there is still some overhead for calling the engine). The guys from Blazemeter compared the speed of various implementation options and came to the conclusion that the Groovy code is only slightly inferior in speed to the code in pure Java.
I also did a little experiment. I wrote a small fragment that performed some artificial calculations in integer arithmetic:
int i;
int s = Integer.parseInt(Parameters);
for (i = 1; i < 10000; i++)
{
s += i * (i % 2 * 2 - 1);
}
log.warn(s.toString());
Dependence on input data (Parameters) and logging were added just in case, to prevent any tricky compilers and interpreters from optimizing the code, excluding its execution at all or caching the result. Moreover, Parameters was also unique. On my laptop with Core i7 with 100 threads of 1000 iterations, each result was as follows:
| Implementation | Throughput |
|---|---|
| Beanshell sampler | ~ 20 / sec |
| JSR223 + (java | beanshell | bsh) | ~ 20 / sec |
| JSR223 + Groovy | ~ 13800 / sec |
The Groovy gap is so significant that it is even hard to believe, nevertheless, judging by the log, everything worked correctly.
Java as a subset of Groovy
Groovy's big plus is that in 95% of cases, arbitrary Java code is valid Groovy code. Even BeanShell syntax is further away from the current Java standard (for example, in BeanShell you have to be perverted when calling functions with an arbitrary number of arguments). If you are not interested in learning all its possibilities right now, then you don’t need to. On the other hand, if you master it , you will surely be able to increase your effectiveness.
bsh.shared
If you used the global namespace bsh.shared in BeanShell, then there is a small ambush: Groovy has nothing of the kind. Fortunately, this problem is easy to solve on your own. For this, 10 lines of code are written:
import java.util.concurrent.ConcurrentHashMap;
public class SharedHashMap
{
private static final ConcurrentHashMap instance = new ConcurrentHashMap();
public static ConcurrentHashMap GetInstance()
{
return instance;
}
}
In fact, this is a singleton, which always (to each thread) will return the same object. Further it is collected in the jar and placed in the lib meter folder. Since the class is declared in the global namespace (yes, I am reprehensible for this), in Groovy code, without any import, you can use SharedHashMap to put something there:
// Получаем ссылку на глобальный hash map.
sharedHashMap = SharedHashMap.GetInstance()
// Кладём туда что-нибудь.
sharedHashMap.put('Counter', new java.util.concurrent.atomic.AtomicInteger())
When you need to pick up, then similarly:
sharedHashMap = SharedHashMap.GetInstance() // *
counter = sharedHashMap.get('Counter')
counter.incrementAndGet()
//..
* Groovy does not have to declare variable types, as well as semicolons.
Migrating with BeanShell Sampler
Suppose you already have a test plan that already has many BeanShell samplers, and you found this article because you are having a problem. You want to switch to Groovy. Connecting Groovy is described above and will take you no more than five minutes.
First of all, create a JSR223 Sampler and transfer the code from BeanShell into it. You can significantly simplify your life if you can unify the code and select it in a separate file by specifying it in the File Name field. Then you just need to paste the JSR223 samplers in the right places with Copy / Paste. If not, copy the code from BeanShell in each case.
Caching key
It is important to note here that JMeter will compile the code entered in the sampler itself only if the compilation key is specified (the Compilation Cache Key field). It should be just a line that is unique within the test plan. For scripts connected via files, you do not need to enter a compilation key, as it uses the full path to the file.
Java and Groovy Strings
There is one subtlety to Groovy syntax. First, there are two types of strings:
- double quotation marks - Groovy strings
- in single quotes - Java strings
See here for more details . Groovy strings have the ability to use expressions like $ {expression}, which are automatically expanded inside the strings to the value expression. This is a pretty convenient point, but it surprisingly matches the syntax with reference to JMeter variables. So if you write in Groovy
currId = 123
log.info("Current ID: ${currId}")
and at the same time the usual JMeter variable with the name currId is defined in the current thread, then its value will be substituted directly into the script. In addition, it is substituted once, because after that, the code will be compiled, and the result will be cached. Therefore, you must ensure that the variable names used in such expressions do not overlap with JMeter variables. And if you really need to pass the value to the JSR223 sampler, then you need to use the Parameters field for this.
When using an external file as source code, JMeter variable substitution does not occur (they occur only in the fields), but you can use Parameters.
If you do not plan to use the capabilities of Groovy strings, then it is advisable to use Java strings (i.e. in single quotes). In addition, it will be better for performance, although a penny, of course.
Conclusion
The behavior of the BeanShell sampler illustrates a typical interpreter problem: the low speed of the interpreted code. If you have only a few lines in BeanShell, you probably won't notice the problems, but you will definitely notice if there is a lot of code or if there are loops. Exactly the same problem was observed in the LoadRunner interpreter.
If you haven’t had any problems running the test using BeanShell so far, I would recommend playing it safe and not creating them for yourself in the future. Instead, it is best to use JSR223 + Groovy immediately, thus reducing the likelihood of performance problems with load stations.
Important points to take out of the article.
- You should not use BeanShell Sampler, instead we use JSR223 + Groovy.
- JSR223 + java = the same BeanShell, so see section 1.
- If it is possible to unify the code of several JSR223, then we use an external file. Besides the fact that eliminating code duplication in itself is a good programming style, there is no need to worry about Compilation Cache Key.
- If we use the script built into the sampler, do not forget about Compilation Cache Key.
- If you need an analog of bsh.shared, use the solution above.