Automation System Integration Testing

Hello, Khabravchans!

I want to share with you my personal experience in system integration testing. Our team is developing an integration layer through which all systems in the bank are connected. We have many tasks, not enough time, and the issue of integration testing has always been postponed.

How is integration testing done? The shortest answer is no way, although we have more than a hundred systems that communicate through the Oracle Service Bus integration bus(OSB). This product has an OSB Console tool that allows you to send a test request and displays the received response. After the developer implements a new service on the bus, the service is manually checked through the OSB Console. If the verification is successful, then the service is declared working and changes only if developers of external systems start complaining about it.

Support for the OSB we were using was coming to an end, and there was a need to upgrade to a new version. Although the migration itself did not cause big problems, the question arose, but how to check the operability of the migrated solution? And here our team once again thought about introducing automatic testing.

How we imagined it


The picture was simple and everyone liked it right away.

In fact, we just need to automate what we do with our hands. So let's create a test that will store pairs of messages (request, response). After starting, our test will send a request, receive a response and compare it with the answer stored in it. If the answers match, then the test was successful.

Service virtualization (mocks, stubs)


The question arose, but what to use in our environment as back-end systems? Obviously, a real back-end system is not suitable for several reasons:
  • For tests, we need the system to work on a specific data set. Suppose we check how a service that returns an account balance works. For such a test, an account with a given number must exist, and the service must return a certain value. Supporting test data on a live system can be a time consuming process.
  • The system may not be available at all in a test environment. Or at the time of integration, the service may still be in development.
  • To check for exceptions, the system must return an error.

It became clear that we needed a service simulator, and the logical decision was to see the finished products.

It turned out that there is nowhere to watch, because there are only 5 such products on the market:

The first three products could not be seen, simply because they cannot be downloaded and viewed. It was necessary to fill out long forms, leave telephones on which salespeople would contact us, in general, all this could last for months. HP Service Virtualization, in principle, also falls into this group, but it turned out that this product was already purchased from us. However, after a week of torment, it turned out that it would not work. This product does not have an open API, and creating thousands of services through a visual editor was unrealistic. An HP spokesman said that services can only be clicked on manually, and they did not think about automation.

Great hopes were placed on the Soap UI, which can run mock services as a web application. But, it turned out that SOAP UI is really a “UI”. Firstly, it is not thread-safe, and secondly, it uses a lot of memory and is unstable.

In the process of research, it turned out that our bank already has four (!) Self-written simulators of web services. One of them turned out to be very personal, was taken as a basis and added as needed. So in the test environment, we got a simulator - a web-application that returns certain HTTP responses for certain HTTP requests.

Each virtual service is a maven project. In the project configuration (service-descriptor.xml, response-selector.xml) it is written how, based on the HTTP request, to determine which service is called, what operation is needed, and by what rule to return an HTTP response.



After changing the source code, the project is automatically built on Jenkins and, using the maven wagon, is deployed to a running simulator application.

Now we are writing a test


The next step is to write a test that will actually be a front-end system simulator. That is, we need to write a web-service client.

Our test is a maven project. Inside the project are pairs of files (request, response), which actually are the source. The build of the project is that our self-written maven plugin calls the web service, passes a test request to it, receives a response and compares it with the test answer.



Tests run automatically every night at Jenkins.

Test data creation


Since the main task was to test the migration, test requests and responses were exported from the audit database, which works in production. According to these data, the corresponding maven projects were generated.

In the case of developing new services, there is no data from living systems yet. For such cases, we wrote our own Eclipse-based tool. In a few clicks, we can create a new project, fill it with test data, put it in the version control system and make a new project on Jenkins.

What happened


Of course, we were not able to fully cover the integration layer, which was created over ten years, with tests.
  • Not all back-end systems work through web services; adapters for other protocols are needed.
  • There are requirements to test not one service, but a whole business process. In this case, you need to store consistent datasets for multiple services.
  • Writing a simulator that supports everything that the back-end can do is quite a lot of work. We did not manage to make support for REST services, asynchronous messages, and also various authentication methods.

Most importantly, the implemented tests detected errors that would have occurred during the migration. And also found some services that, in principle, did not work and were not used. So our experience is definitely positive!

Future plans


We liked it and we will continue. In the near future, we plan to add simulators for JMS, support for business processes and come up with how to conduct performance tests.

Do you test integration? Share your experience!

Also popular now: