April 21, 2017 at 15:11About MegaFon Map - Technical Details, Part 2
We continue to talk about the technical features of the project for the issuance and maintenance of MegaFon bank cards. In previous posts, we talked about the card as a financial product, about its capabilities and about the device software that provides the system. In this post, we will touch upon issues related to the integration of IT systems of Bank Round, MegaFon's project partner, with the operator’s IT systems. Neoflex, a system integrator with more than twelve years of experience in the IT market , has become the technological partner in creating an integration solution combining the bank’s IT systems and MegaFon .
The project took place in several stages:
We understood that it is important not to make a mistake when designing the design of data flows. This may sound corny, but for this project this aspect was critical, since the integration affected the IT landscapes of two different industries. Both telecom and banking have their own rules and historical features that needed to be taken into account and “seamlessly” combined into a single data exchange process. We preferred to take a break at the start of the project and, together with the architects of the bank and the operator, develop an integration design, identify the necessary services, analyze business processes, make changes to them, then transfer them to the SOA concept. The design really managed to be worked out qualitatively, of course there were changes, but they were minor and did not subsequently affect the architecture of the integration solution.
One of the most difficult aspects of the project at the planning stage was the presence of a large number of suppliers and contractors. The implementation of such a project with a classic waterfall would lead us to increase the timeline, which was unacceptable. Due to the fact that the work was performed in a dynamic mode, fixing the scope at the initial stage was almost impossible. We could not work with classic scrum either, since many control points depended on suppliers of other systems. We applied a spiral approach: together with the bank we analyzed the priorities of other suppliers, the general objectives of the project and identified blocks of functionality. This model determined the composition of iterations for the development of the project for us. As a result, when half the deadline for the implementation of the entire project has passed, we started analytics on services for IVR and the client’s personal account, at the same time, an integration solution supporting the card issuing process was already on the test environment. This made it possible at an early stage to determine what changes and improvements to make to the process, and the ABS and front card issuing teams could conduct end-to-end testing of the new functionality.
The key task facing the integration solution concerned the automation of the data flow as part of the processes of issuing, personifying and activating the card, performing various operations on it, both payment and service. Given the specifics of the problem, high non-functional requirements were imposed on the solution. Everything was important: performance and message processing time, availability and change management, scalability and fault tolerance.
Oracle Service Bus was chosen as the main platform for integration. This is an industrial integration platform from one of the world leaders in the IT industry, which is designed to build an infrastructure for access to business services and their virtualization. Like a number of other industrial platforms, OSB has an advanced solution development and debugging tool, a large amount of documentation, it is also possible to note the ability to quickly and conveniently create complex data transformations, support out of the box a large number of different transport protocols, efficiently manage a large number of requests, convenient management changes. Neoflex company already had successful experience in fulfilling projects programs on the OSB platform, so we were confident in the solution and understood
On the MegaFon side, the project affected an e-commerce gateway for debiting and replenishing a personal account, a system for managing services connected to customers' phones, IVR, as well as a subscriber’s personal account. Also, on the operator’s side, the “Sales and Service System” is actively used, through which requests for processing and issuing cards are sent. On the bank side, the solution integrates with the processing system, ABS, and customer validation system. With all systems except billing, we integrated directly using industry standard technologies - SOAP and REST. Interaction with the billing system was configured using a gateway from InPlat, which provides a single entry point for all MegaFon partners and provides access to e-commerce services.
Part of the project’s functionality included not only the need to create transport streams, but also complex business logic with support for asynchronous transactional computing. To implement such tasks as part of the architecture, we decided to bring this functionality into a separate scalable solution based on java components, which was based on both the principles of java enterprise and MSA (micro service architecture). The chosen approach allows for easy and inexpensive scaling of the load solution with the ability to quickly make the necessary changes to the functionality.
Based on our experience, we understood that a critical part of such a solution is a system for monitoring the state of the infrastructure and components of the solution and an audit system for processed messages. The bank uses Zabbix as a means of collecting and processing data on the state of infrastructure, and we decided to integrate into it, having experience in its use in complex projects. When designing and creating the system, we laid the monitoring points, there were dozens of them in total, they allow you to control the amount of resources used (processing threads, database connections, connections to external systems) and analyze message processing indicators, for example, the number of unsuccessful requests for Recently, the number of processed requests and so on. The developed audit system of processed messages allows you to view full information about all requests using filters and search through a convenient user interface. At the same time, various messages within one business process are combined into one chain. Naturally, the information about requests available in monitoring is pre-processed to satisfy the bank's security policies and PCI DSS requirements, but the bank has enough funds to resolve any controversial situation as quickly as possible. This is very important for integration, as many systems and even several organizations are involved in the process. Naturally, the information about requests available in monitoring is pre-processed to satisfy the bank's security policies and PCI DSS requirements, but the bank has enough funds to resolve any controversial situation as quickly as possible. This is very important for integration, as many systems and even several organizations are involved in the process. Naturally, the information about requests available in monitoring is pre-processed to satisfy the bank's security policies and PCI DSS requirements, but the bank has enough funds to resolve any controversial situation as quickly as possible. This is very important for integration, as many systems and even several organizations are involved in the process.
Naturally, the development of the system does not end on the development itself. Currently, we support the developed system and implement changes as part of the development of the solution. I would especially like to note that the time spent on thorough study and development of monitoring tools more than pays for itself now during the maintenance of the decision. It is convenient to search for messages using various data, to trace the entire message chain within a business operation, and to see exchanges with end systems. This allows you to quickly understand which system the problem is, because according to the experience of developing integration solutions, the lion's share of calls are not related to integration errors, to integrated systems. And the most laborious is localization.
The project took place in several stages:
- The Neoflex team studied business processes designed on the side of MegaFon.
- Together with the bank’s IT architects, we proposed a solution that allowed us to put these processes at the heart of the SOA architecture, taking into account the stability and sustainability of the system.
- Then the requirements were formulated for a future integration solution, which was also developed by Neoflex.
- The next stage is development and testing; it was done iteratively, breaking the functionality into logical blocks.
We understood that it is important not to make a mistake when designing the design of data flows. This may sound corny, but for this project this aspect was critical, since the integration affected the IT landscapes of two different industries. Both telecom and banking have their own rules and historical features that needed to be taken into account and “seamlessly” combined into a single data exchange process. We preferred to take a break at the start of the project and, together with the architects of the bank and the operator, develop an integration design, identify the necessary services, analyze business processes, make changes to them, then transfer them to the SOA concept. The design really managed to be worked out qualitatively, of course there were changes, but they were minor and did not subsequently affect the architecture of the integration solution.
One of the most difficult aspects of the project at the planning stage was the presence of a large number of suppliers and contractors. The implementation of such a project with a classic waterfall would lead us to increase the timeline, which was unacceptable. Due to the fact that the work was performed in a dynamic mode, fixing the scope at the initial stage was almost impossible. We could not work with classic scrum either, since many control points depended on suppliers of other systems. We applied a spiral approach: together with the bank we analyzed the priorities of other suppliers, the general objectives of the project and identified blocks of functionality. This model determined the composition of iterations for the development of the project for us. As a result, when half the deadline for the implementation of the entire project has passed, we started analytics on services for IVR and the client’s personal account, at the same time, an integration solution supporting the card issuing process was already on the test environment. This made it possible at an early stage to determine what changes and improvements to make to the process, and the ABS and front card issuing teams could conduct end-to-end testing of the new functionality.
Briefly about the main thing
The key task facing the integration solution concerned the automation of the data flow as part of the processes of issuing, personifying and activating the card, performing various operations on it, both payment and service. Given the specifics of the problem, high non-functional requirements were imposed on the solution. Everything was important: performance and message processing time, availability and change management, scalability and fault tolerance.
Oracle Service Bus was chosen as the main platform for integration. This is an industrial integration platform from one of the world leaders in the IT industry, which is designed to build an infrastructure for access to business services and their virtualization. Like a number of other industrial platforms, OSB has an advanced solution development and debugging tool, a large amount of documentation, it is also possible to note the ability to quickly and conveniently create complex data transformations, support out of the box a large number of different transport protocols, efficiently manage a large number of requests, convenient management changes. Neoflex company already had successful experience in fulfilling projects programs on the OSB platform, so we were confident in the solution and understood
Empirically
On the MegaFon side, the project affected an e-commerce gateway for debiting and replenishing a personal account, a system for managing services connected to customers' phones, IVR, as well as a subscriber’s personal account. Also, on the operator’s side, the “Sales and Service System” is actively used, through which requests for processing and issuing cards are sent. On the bank side, the solution integrates with the processing system, ABS, and customer validation system. With all systems except billing, we integrated directly using industry standard technologies - SOAP and REST. Interaction with the billing system was configured using a gateway from InPlat, which provides a single entry point for all MegaFon partners and provides access to e-commerce services.
Part of the project’s functionality included not only the need to create transport streams, but also complex business logic with support for asynchronous transactional computing. To implement such tasks as part of the architecture, we decided to bring this functionality into a separate scalable solution based on java components, which was based on both the principles of java enterprise and MSA (micro service architecture). The chosen approach allows for easy and inexpensive scaling of the load solution with the ability to quickly make the necessary changes to the functionality.
Based on our experience, we understood that a critical part of such a solution is a system for monitoring the state of the infrastructure and components of the solution and an audit system for processed messages. The bank uses Zabbix as a means of collecting and processing data on the state of infrastructure, and we decided to integrate into it, having experience in its use in complex projects. When designing and creating the system, we laid the monitoring points, there were dozens of them in total, they allow you to control the amount of resources used (processing threads, database connections, connections to external systems) and analyze message processing indicators, for example, the number of unsuccessful requests for Recently, the number of processed requests and so on. The developed audit system of processed messages allows you to view full information about all requests using filters and search through a convenient user interface. At the same time, various messages within one business process are combined into one chain. Naturally, the information about requests available in monitoring is pre-processed to satisfy the bank's security policies and PCI DSS requirements, but the bank has enough funds to resolve any controversial situation as quickly as possible. This is very important for integration, as many systems and even several organizations are involved in the process. Naturally, the information about requests available in monitoring is pre-processed to satisfy the bank's security policies and PCI DSS requirements, but the bank has enough funds to resolve any controversial situation as quickly as possible. This is very important for integration, as many systems and even several organizations are involved in the process. Naturally, the information about requests available in monitoring is pre-processed to satisfy the bank's security policies and PCI DSS requirements, but the bank has enough funds to resolve any controversial situation as quickly as possible. This is very important for integration, as many systems and even several organizations are involved in the process.
Naturally, the development of the system does not end on the development itself. Currently, we support the developed system and implement changes as part of the development of the solution. I would especially like to note that the time spent on thorough study and development of monitoring tools more than pays for itself now during the maintenance of the decision. It is convenient to search for messages using various data, to trace the entire message chain within a business operation, and to see exchanges with end systems. This allows you to quickly understand which system the problem is, because according to the experience of developing integration solutions, the lion's share of calls are not related to integration errors, to integrated systems. And the most laborious is localization.