November 13, 2014 at 11:32JMeter as a relatively convenient and practical tool for testing API
The article will discuss testing in a short time using the JMeter tool, as well as how to successfully complete the work in case of a forced replacement of specialists on the project.
As a tester, I love it when everything is in order, but life is full of dirty hacks. I like to automate by tying Selenium to Python, but when I encounter the problem of limited resources, I rush for the tool that allows you to do "everything the same, but faster." In this post I will tell you that JMeter is a great tool for both load and functional testing.
My task was to test the API for a mobile application. Her features were as follows:
Hands habitually reached for Python, but something switched in my head. “Dear friend,” I told myself, “if you are the only person in the team who will write in Python, then who will understand what you wrote and continue to work if you are absent?”
It was necessary to explicitly look for another tool. And then I looked at the beloved JMeter from a completely different perspective: JMeter, of course, leaves much to be desired externally, but instead of a thousand small scripts, we will have an understandable test structure. Therefore, I can ask my colleagues to run random tests instead of me from any machine. Even programmers appreciated that it was much more convenient than “running off with your hands” for debugging.
Out of the box there is everything (well, almost) that you need:
The only thing that was a bit embarrassing was that Jenkins sees JMeter solely as a load testing tool, so the history of bugs found was in the Performance report tab.
Briefly about the testing process:
Instead of a thousand words. This is how a small piece of the finished test looks like:
One of many tests. Groups are broken up in Simple Controller. Http request, Regular Expression Extractor, Response Assertion, a bit of User Defined Variables, Random Variable and that's about it.
Almost any test begins with creating the necessary users. Here the only "hard-coded place" is the admin authorization to create a new item in the database.
What is funny, a method was created to create a user, but the removal occurred only manually through the admin panel. I decided not to complicate the logic of each individual case, adding separate tests on Selenium to clean users, and did not delete the created users inside the test itself. For this I received thanks from the programmers: when they started writing the paginator for the admin pages, there were already thousands of test users in the database.
To check the result, after each request there is a long series of assertions. She checks the accuracy of the data: http-response codes, messages.
I disabled all graphs and reports except for simple tree view for debugging. All bugs can be viewed directly in Jenkins, and for load testing it was more convenient to upload directly to LoadSophia with their proprietary plug-in and pick it up there.
Here's how nice the programmer looks like a message about why the next build failed:
And in more detail. Waited for 400 code - returned 404:
The result of this Black Book and obscurantism :
Of the 1,200 hours laid down for development, in total, 50 hours were spent on the writing and actual maintenance of autotests. Bug fixes - 70 hours. And about half of them were related to a misunderstanding of the customer and developers. The result is very good. An alternative to manually checking everything would take almost more time than it would take for programming. Programmers immediately saw how something goes wrong, in the whole system with each new assembly.
In my absence of problems with launching and changing autotests, there were no problems both among programmers and testers. Sometimes tests were changed right in the notepad, because * .jmx is also XML.
To conduct load testing, I simply turned off unnecessary tests for a while, calculating where the weaknesses of the system and which tests are slow. There were no problems using the new JMeter buns. Everything is documented and everything is clear.
Of the difficulties encountered:
PS A slight lyrical digression. In a recent articleThey talked about a way to reduce the work of the tester through decomposition and, more importantly, about the problems of the testing market. One of the market's answers to the “vicious circle of testers” described at the beginning of the article is the standardization of testing skills. With it, the employer will become more confident that he will find in the market the necessary specialist who can solve the problem. The tester will know that with the knowledge he will not get lost in the market. In this case, making functional testing through JMeter is not an academic decision at all. But it reduced time and a pound of nerves with a similar result. I had a friend who knew only the most elementary basis of JMeter and Selenium, but was taken with open arms into one very respected company. I haven’t been asked at any interview in my life, Do I know of some rarely used library. They asked mainly again about JMeter, Selenium (and what I write on for these solutions). The “most important, know well several of the most important tools and know how to apply” approach makes life easier for both the tester, the employer, and the market as a whole.
As a tester, I love it when everything is in order, but life is full of dirty hacks. I like to automate by tying Selenium to Python, but when I encounter the problem of limited resources, I rush for the tool that allows you to do "everything the same, but faster." In this post I will tell you that JMeter is a great tool for both load and functional testing.
My task was to test the API for a mobile application. Her features were as follows:
- Timing is tightened as never before;
- There is a specification of all methods, but it changes literally on the fly;
- The project is confusing. If I die in the mountains or get sick, introducing a new person into the project will be very expensive and difficult;
- We use Jenkins as the CI system. Tests should stand up easily without any rarely used, raw, and overly complex plugin;
- Mandatory stress testing, but little time for it;
- The mobile application for this API itself is written by a completely different team from another city. The only way to test is to look at the responses to queries.
Hands habitually reached for Python, but something switched in my head. “Dear friend,” I told myself, “if you are the only person in the team who will write in Python, then who will understand what you wrote and continue to work if you are absent?”
It was necessary to explicitly look for another tool. And then I looked at the beloved JMeter from a completely different perspective: JMeter, of course, leaves much to be desired externally, but instead of a thousand small scripts, we will have an understandable test structure. Therefore, I can ask my colleagues to run random tests instead of me from any machine. Even programmers appreciated that it was much more convenient than “running off with your hands” for debugging.
Out of the box there is everything (well, almost) that you need:
- Work with variables, regular expressions, JSON parsing, work with cookies and a huge bag of goodies;
- A simple and easy-to-use plugin for Jenkins, which allows you to connect everything;
- For load testing, I do not have to write new scripts, you can only slightly modify existing ones.
The only thing that was a bit embarrassing was that Jenkins sees JMeter solely as a load testing tool, so the history of bugs found was in the Performance report tab.
Briefly about the testing process:
- Standard Thread group on 1 Virtual user;
- Each group of tests is enclosed in Simple Controller (it has only one distinctive property: all requests go one after another. This is what you need, just like in a test case);
- Each individual test case is also enclosed in a separate Simple Controller. Quite simple test cases, for example, checking whether the 401 code will return when trying to get in without authorization, are made as a separate test case.
Instead of a thousand words. This is how a small piece of the finished test looks like:
One of many tests. Groups are broken up in Simple Controller. Http request, Regular Expression Extractor, Response Assertion, a bit of User Defined Variables, Random Variable and that's about it.
Almost any test begins with creating the necessary users. Here the only "hard-coded place" is the admin authorization to create a new item in the database.
What is funny, a method was created to create a user, but the removal occurred only manually through the admin panel. I decided not to complicate the logic of each individual case, adding separate tests on Selenium to clean users, and did not delete the created users inside the test itself. For this I received thanks from the programmers: when they started writing the paginator for the admin pages, there were already thousands of test users in the database.
To check the result, after each request there is a long series of assertions. She checks the accuracy of the data: http-response codes, messages.
I disabled all graphs and reports except for simple tree view for debugging. All bugs can be viewed directly in Jenkins, and for load testing it was more convenient to upload directly to LoadSophia with their proprietary plug-in and pick it up there.
Here's how nice the programmer looks like a message about why the next build failed:
And in more detail. Waited for 400 code - returned 404:
The result of this Black Book and obscurantism :
Of the 1,200 hours laid down for development, in total, 50 hours were spent on the writing and actual maintenance of autotests. Bug fixes - 70 hours. And about half of them were related to a misunderstanding of the customer and developers. The result is very good. An alternative to manually checking everything would take almost more time than it would take for programming. Programmers immediately saw how something goes wrong, in the whole system with each new assembly.
In my absence of problems with launching and changing autotests, there were no problems both among programmers and testers. Sometimes tests were changed right in the notepad, because * .jmx is also XML.
To conduct load testing, I simply turned off unnecessary tests for a while, calculating where the weaknesses of the system and which tests are slow. There were no problems using the new JMeter buns. Everything is documented and everything is clear.
Of the difficulties encountered:
- The plugin for JMeter that implements oAuth authentication is very outdated and does not work with oAuth 2.0. A small part of the functionality associated with authorization through social networks had to be tested separately.
- Tie Selenium to JMeter is also a dubious pleasure. UI Admins was also tested separately.
- For people who have not experienced the JMeter interface, first acquaintance ends with
retinal detachment amild disorder.
PS A slight lyrical digression. In a recent articleThey talked about a way to reduce the work of the tester through decomposition and, more importantly, about the problems of the testing market. One of the market's answers to the “vicious circle of testers” described at the beginning of the article is the standardization of testing skills. With it, the employer will become more confident that he will find in the market the necessary specialist who can solve the problem. The tester will know that with the knowledge he will not get lost in the market. In this case, making functional testing through JMeter is not an academic decision at all. But it reduced time and a pound of nerves with a similar result. I had a friend who knew only the most elementary basis of JMeter and Selenium, but was taken with open arms into one very respected company. I haven’t been asked at any interview in my life, Do I know of some rarely used library. They asked mainly again about JMeter, Selenium (and what I write on for these solutions). The “most important, know well several of the most important tools and know how to apply” approach makes life easier for both the tester, the employer, and the market as a whole.