IBM System i (aka AS / 400) - How we did green screen application auto-tests
Hello! My name is Anton Vorobyov, I am responsible at Alfa Bank for the development of applications for a centralized banking system.
In this post, I will tell you about what green screen applications are, why they are needed and how we did autotests for them by writing our own solution for this, which allowed us to accelerate autotests by 11 times. The AS / 400 platform (Application System / 400) was born in 1988. The first OS for this platform is OS / 400, later renamed to i5 / OS and even later to IBM i. Not so long ago, she celebrated her thirtieth birthday.
Plunging into the world of development under the IBM i operating system, you understand that this is actually not “legacy” in the classical sense of the word. This is a different, completely different environment, which is little similar to the usual Windows or Unix systems. The main task of this OS is to be as productive as possible on the equipment with which it works, and not to be convenient for the user.
IMHO, this OS can make you crazy about how ineffective the usual approaches to writing C ++ code are ineffective on it (up to tens of times CPU loss), that some antipatterns demonstrated in textbooks are the best-practice of effective code, and the source with the date of writing for 1978 not only assemble without problems, but also work as designed! All this makes us take a fresh look at modern approaches to software development.
The issue of improving the quality of software under development excites the minds of each development team. One of our credit teams, whose task is to develop and develop the Back part of the module for the Misys Equation automated banking system, has not bypassed this moment either. The peculiarity of this ABS is that:
IBM i (logo) We are
developing applications for this ABS (ABS options) in accordance with the Misys ITP Integrator's Technical Package standard package, which states that the option should consist of an interactive program for terminal interaction with the end user and implement the API according to installed interface for background execution.
Such interactive programs developed under the IBM i operating system are historically called green screen applications and are the only UI with which the user of this ABS interacts.
The simple answer is an application that looks like this:
Or like this :
Historically, the only interactive applications running on the low and mid-range systems of the AS / 400 family and other IBM mainframes that allowed you to request any user input are green screen applications. Installation, administration, configuration and development on the IBM i operating system (and its predecessors i5 / OS and AS400) were carried out (and are still being carried out somewhere) exclusively using green screen applications.
The green screen application image has two sizes - 24x80 and 27x132 characters and 16 possible colors. Within this scale, most of the work of developers and users of this operating system is performed.
Such screen sizes are the result of the evolution of “workstations” that were connected to the AS400 progenitors from the low-end and mid-range segments of the business computers IBM System / 32, System / 34, System / 36 and System / 38. These workstations were called terminals and consisted of a screen in a metal case with a keyboard and additional equipment in the form of a light pen. Initially, only two screen colors were supported - green and bright green, which is why the established phrase “green screen application” (green screen application in English literature) went. In the 1970s, the number of supported colors increased to 16.
5251 Display Station Model 11
The most common terminal options were 5251 Display Station Model 1 (960 characters on the screen) and Model 11 (1920 characters on the screen) with dimensions Width / Depth / Height equal to 530/400/400 mm, respectively, and weighing 34 kg. The screen resolution of Model 1 was 12x80, Model 11 - 24x80. The terminal was connected directly to the host system.
The terminals 5251 Display Station Model 2 (960 characters on the screen) and Model 12 (1920 characters on the screen) with large dimensions and weight of 45 kg were also quite common. They are distinguished from Model 1 and Model 11 by the possibility of forwarding the upstream connection through themselves to the host machine from cheaper clients in the form of Model 1 (or 11) terminals with desktop printers or a separate floor printer. Thus, models 2 and 12 acted as a hub, proxying the connection to the host from devices requiring a direct connection to the host machine, and cost significantly more.
The terminals of the 5252 Dual Display Station series will also seem unusual to the modern layman.
Promotional image from the IBM System / 38 Equipment and Programs brochure (5252 Dual Display Station)
The price of one terminal kit with a printer connected to it could reach several thousand US dollars.
The terminals were connected via a twinaxial cable to the host machine with a bus topology in half-duplex mode with a transmission speed of up to 1 Mbps. The maximum number of terminals supported by twinaxial is up to 6 terminals, and the terminal most remote from the host should be located at a distance of no more than 1500 meters.
The number of each terminal is set during its installation by three switches, so a unique address is determined within the bus. In the presence of an existing coaxial network, it is possible to use adapters from a twinaxial cable to a coaxial cable and an appropriate set of cable terminations for crimping. With this scheme, it was possible to connect only two devices on the bus with a maximum segment length of up to 30 meters. The total number of connected devices varied from a dozen to several dozen, depending on the model.
With the development of Desktop-systems and access networks, bulky terminals were replaced by workstations, where various expansion cards of third-party companies were used as means of access to the host machine, supporting direct connection via twinaxial. After IBM developed Token Ring technology in 1984, software solutions for accessing the machine, including through this interface, appeared.
5250 ISA adapter (unknown manufacturer)
Blackbox 5250 Adapter Cards (PC470C, PC471C, PC472C, PC473C, PC478C)
Emulators for MS-DOS and MS Windows appear both from IBM and from third-party manufacturers, including OpenSource implementations (for example, tn5250j.sourceforge.net). In the mid-90s, the TCP / IP stack came into the mid world -range and low-end business machines. To support access to host machines under the new protocol, IBM is developing 5250 series terminal emulators.
In order to create a protocol for exchanging with the host system, IBM is developing
extensions to the Telnet protocol (RFC 854, RFC 855, RFC 856, RFC 860, RFC 885, RFC 1091 , RFC 1205, RFC 1572, RFC 2877), collectively referred to as Telnet5250 (TN5250), which describes the process of receiving and transmitting 5250 data streams (5250 data streams) over the standard Telnet protocol.
IBM Client Access / 400 for Windows 3.1 Installer
A feature of the IBM 5250 terminals (and, accordingly, the TN5250 protocol) is its block orientation, in contrast to the usual * nix terminals, which are symbol-oriented . This means that the 5250 data flows by which the host communicates with the terminal are transmitted by data blocks, and a separate symbol in it without the context of the transmitted block does not make sense.
For example, the host machine transmits to the terminal a data block containing the displayed static information on the screen along with the attributes and coordinates of the input fields and an indication of the offset in this block, where to write the result of user input to the fields. After that, the host machine expects messages from the terminal and does not participate in the user input process.
The login screen for the IBM i host RZKH.de (pub400.com) The
next task of the terminal emulator is to interpret the data block from the machine and form the user an input screen where they are given the opportunity to enter any information into the allowed fields. Also, the tasks of the terminal emulator include a reaction to user actions. The F1-F24 keys (F13-F24 are simulated via SHIFT + Fx), Enter, Home, End, PageUp, PageDn, and some other special keys that are absent on modern keyboards are considered host keys. This means that by pressing this key, a stream buffer with information from the input fields and the cursor position on the screen, previously filled with the terminal emulator, will be sent to the host for processing.
WIreshark 5250 Data Stream login attempt dump at pub400.com
The host receives the buffer, parses it, and the input result is transmitted to the program that requested the user’s reaction for further data validation and the application continues to work, while the application receives the code of the pressed host key.
We thought about automating manual testing of green screen applications when we were faced with the need to test hundreds of screens of a developed module, where up to eighty different business checks (validations) could occur on one screen.
The team’s particular pain was the almost complete absence for 2017 of green screen auto-testing tools, except for the proprietary UIPath solution . Even today there are not many similar solutions, the author is aware of Automate from HelpSystems and the JMeter extension for BlazeMeter (I will be glad to know about other similar products).
The standard emulator of the TN5250 terminal installed at workplaces in the bank is IBM Personal Communications for Windows 6.0 (PCOMM 6.0). Colleagues found that this product has regular means of automating its management in the form of a diverse API, namely:
The first three interfaces are the oldest and have been supported since DOS and 16-bit versions of Windows. Work on the EHLLAPI interface is implemented by calling a single function according to the following prototype:
where the first parameter is a pointer to the numerical number of the function being executed, the other two are its arguments context-sensitive to the function being called, and the last is the result of the function. That is, in order to request the connection status 'A' (sessions in the emulator are numbered with a Latin letter of the range from 'A' to 'Z'), you must run the following code (taken from IBM documentation):
The number of functions available for calling in this way is about 60.
The WinHLLAPI interface slightly expands this functionality with several additional functions that allow registering callback functions for asynchronous calls in order to notify about events of establishing a connection with the host, disconnecting from the host, changing data on the terminal screen, etc. .
interface Host Access Client Library (HACL) seemed more friendly to work because, in contrast to call "function one name", was granted an option of object-oriented hierarchy of classes s, allowing fully simulate any user action.
HACL Emulator Class Library Class Hierarchy (C ++)
There are HACL implementations for C ++, Java, LotusScript and a COM automation server for Windows (convenient for Visual Basic and .NET).
Due to the enormous complexity of the 5250 data flow protocol and the extremely scarce information about its internal device with links to closed paid literature from IBM, it became obvious that developing your own emulator is extremely non-trivial and time-consuming. In this regard, the idea came up to use the middleware layer, which will allow you to control the terminal emulator within the minimum required functionality, in particular “enter a value in the field”, “compare part of the screen with the standard” or “press the host key F22”.
Colleagues who previously used HACL interfaces claimed (and a search on StackOverflow confirmed) that the COM object had stability problems and could hang after a certain number of commands were executed. Only restarting the automation server process helped. A quick analysis of the Java version showed that Wrapper is used over the C ++ interface through JNI. Therefore, the choice fell on the C ++ interface. The corresponding header files and .lib files were available in the installation directory of Personal Communications For Windows itself.
The first prototype was based on Qt5, where it was possible to execute JavaScript code through QtScript. In the environment of the executable script, an object was registered with a small number of methods that allowed executing commands in the terminal emulator as if they were being executed by a person (entering a field, pressing host keys, waiting for a line to appear on the screen). We demonstrated a live “demo”, where we scripted a user case for launching a green screen application from ABS Equation with a test of the reaction of the application to incorrect input to the fields. The demonstration showed that the prototype was successful and that we can move on.
Along with the demonstration of the first prototype, colleagues from another department put together a bunch of Ruby + Cucumber + Quick3270 + Ruby module ( cheeze / te3270 ). The proposed option uses a Ruby module that interacts with the DN32 Computing Quick3270 terminal emulator through its specialized COM objects (incompatible with HACL interfaces). It was a complete solution for BDD-style green screen auto-testing applications, with a few pre-described steps. However, in the proposed solution, we were alarmed by the following:
Quick3270 terminal emulator
Based on the prototype, we decided to try to implement our own automation server in order to connect Cucumber to Personal Communications for Windows and design the steps so that the downtime between the actions on the emulator screen is minimal.
Lyrical digression. Despite the fact that there are a huge number of technical problems around the supposedly "legacy" IBM, which, it would seem, should have already been solved for systems of the medium and enterprise business level, the relevance of adapting and transferring existing technical solutions is very high simply because of their absence on the platform. Often the absence is connected with the very features of this OS, which is fundamentally different from modern * nix, Windows or MacOS X, which requires significant optimization of software for this stack.
As our own solution, we created an automation server as a development of the previously demonstrated prototype. This server executes commands to automate interaction from consumers through an RPC server (Qt5 WebSocket). It interacts with Personal Communications for Windows, which is part of the corporate Windows operating system image, and allows you to:
Starting an Automation Server
However, with all the advantages of the HACL API, it has one drawback - it does not know how to work with the DB2 for i DBMS built into the OS and does not allow executing commands that are vital for building a mock environment where a test script would be executed. If the DB2 client for Ruby exists from IBM, then the client for the remote command and distributed program call server is only for Java in the form of the JTOpen library: The Open Source Version of the IBM Toolbox for Java (also known as jt400). We “peeped” the solution to this problem at IBM itself by analyzing the behavior of its products with similar functionality (in particular, Personal Communications for Windows Data Transfer, iSeries to PC / PC to iSeries Transfer, etc.). It turned out that these products, by their implementation, run IBM JRE 6 or 8, depending on the version of the application, and use the jt400 library.
For the automation server, we decided to do the same. The JNI launches the IBM JVM, which ships with Personal Communications for Windows. Using special wrapper methods, commands from the RPC server coming from outside are executed by proxying them into calls to the necessary jt400 functionality. Since the latter also contains a JDBC driver for DB2, it was decided to use it to access the DBMS on IBM i.
It is important to note that you cannot use Oracle JVM when using HACL. If you run a terminal emulator session, then trying to create an instance of the virtual machine will crash. Similarly, if you run the Oracle JVM in the address space of a process interacting with the HACL, the latter hangs without explanation.
Over time, the solution was implemented on an increasing number of jobs. It worked faster than the solution with Quick3270. Popularity grew, as did the number of autotests. However, during operation, additional difficulties arose:
By analyzing the trace logs of numerous autotest launches, finding bottlenecks in the performance of frequently used steps, the total regression execution time was reduced to 4-5 hours. But it was clear that using the middleware layer in the form of an automation RPC server in conjunction with the HACL interface, which also has “floating” errors that accumulate over the duration of the entire system, will not help in improving the performance of the solution.
On the other hand, as an alternative to IBM Personal Communications for Windows, the vendor provides a cross-platform solution called IBM i Access - Client Solutions.
IBM i Access - Client Solutions
Analysis of its internal structure on Saturday and Sunday mornings over cups of coffee showed that its code base is based on another product from IBM called IBM Host on-Demand (IBM HOD). This is a full-fledged solution for accessing IBM i, developed in Java 6, which not only has the full implementation of various communication protocols used in IBM machines (TN3270, TN5250, VTxxx, etc.), but also high-level java-swing UI components, used to build your own terminal emulators in the form of a constructor, which can be assembled from scant documentationIBM A more detailed study of the IBM HOD showed that the UI components are based on the Java implementation of the HACL interface, whose documentation is open. Their behavior coincides with only slight differences from the C ++ HACL documentation.
IBM Host On-Demand (logo)
Next, we created a Java library for internal use, which implements the same interface as the C ++ RPC automation server, but internally uses IBM HOD. To reduce the overhead during the execution of the autotest steps, we migrated from Ruby Cucumber to cucumber-jvm with the reimplementation of all steps similar to the Ruby options. In the presence of a software interface similar to the RPC server, this was not a big deal, especially considering that we tried to restrain the uncontrolled growth in the number of steps themselves and we had this value in the region of 30 units.
As a result, we achieved the operability of all autotests without changing them, and the speed of work became so high that we had to introduce an artificial delay between the steps so that when developing an autotest, you could observe its work, otherwise the UI did not have time to draw the screen to the end.
Already existing 180 autotests with more than 16,000 steps with a set delay of 60 ms between steps began to run about 30 minutes against 5 hours 30 minutes, which corresponds to an eleven-fold increase in the performance of the regression stand.
The results exceeded all expectations. We are close to the physical limits of the TN5250 protocol.
To date, the decision has been published to the entire bank, and colleagues from other cities have joined in the improvement. Of the recent changes, colleagues are integrating the solution with Jenkins, in the test version, testing the launch on a Linux server with Xvfb was completed and the stage of pilot operation of running autotests on it began.
Thank you for reading to the end!
All success!
PS In December 2018, the next IBMi Developer Conference was held, at which a report was made on the topic of this article.
Until now, we have held the Conference annually only for Bank employees. From 2019, we will invite participants from other companies. It is very interesting to expand the circle of professional and personal communication, share emotions, knowledge and experience.
In this post, I will tell you about what green screen applications are, why they are needed and how we did autotests for them by writing our own solution for this, which allowed us to accelerate autotests by 11 times. The AS / 400 platform (Application System / 400) was born in 1988. The first OS for this platform is OS / 400, later renamed to i5 / OS and even later to IBM i. Not so long ago, she celebrated her thirtieth birthday.
Plunging into the world of development under the IBM i operating system, you understand that this is actually not “legacy” in the classical sense of the word. This is a different, completely different environment, which is little similar to the usual Windows or Unix systems. The main task of this OS is to be as productive as possible on the equipment with which it works, and not to be convenient for the user.
IMHO, this OS can make you crazy about how ineffective the usual approaches to writing C ++ code are ineffective on it (up to tens of times CPU loss), that some antipatterns demonstrated in textbooks are the best-practice of effective code, and the source with the date of writing for 1978 not only assemble without problems, but also work as designed! All this makes us take a fresh look at modern approaches to software development.
Introduction
The issue of improving the quality of software under development excites the minds of each development team. One of our credit teams, whose task is to develop and develop the Back part of the module for the Misys Equation automated banking system, has not bypassed this moment either. The peculiarity of this ABS is that:
- the first versions of the ABS worked under the predecessor AS / 400 - the IBM System / 38 platform (appeared in 1978) under the CPF OS - “Control Program Facility”;
- It has been developed since the 70s of the twentieth century, and you may come across code written before your birth (a lot of old code);
- the features of working with ABS are due to close integration with IBM i, and because of the colossal backward compatibility of the latter, it seems that you are working as an archaeologist in the excavations of the Great Pyramid.
IBM i (logo) We are
developing applications for this ABS (ABS options) in accordance with the Misys ITP Integrator's Technical Package standard package, which states that the option should consist of an interactive program for terminal interaction with the end user and implement the API according to installed interface for background execution.
Such interactive programs developed under the IBM i operating system are historically called green screen applications and are the only UI with which the user of this ABS interacts.
What is a green screen application?
The simple answer is an application that looks like this:
Or like this :
Why green screen apps?
Historically, the only interactive applications running on the low and mid-range systems of the AS / 400 family and other IBM mainframes that allowed you to request any user input are green screen applications. Installation, administration, configuration and development on the IBM i operating system (and its predecessors i5 / OS and AS400) were carried out (and are still being carried out somewhere) exclusively using green screen applications.
The green screen application image has two sizes - 24x80 and 27x132 characters and 16 possible colors. Within this scale, most of the work of developers and users of this operating system is performed.
Such screen sizes are the result of the evolution of “workstations” that were connected to the AS400 progenitors from the low-end and mid-range segments of the business computers IBM System / 32, System / 34, System / 36 and System / 38. These workstations were called terminals and consisted of a screen in a metal case with a keyboard and additional equipment in the form of a light pen. Initially, only two screen colors were supported - green and bright green, which is why the established phrase “green screen application” (green screen application in English literature) went. In the 1970s, the number of supported colors increased to 16.
5251 Display Station Model 11
The most common terminal options were 5251 Display Station Model 1 (960 characters on the screen) and Model 11 (1920 characters on the screen) with dimensions Width / Depth / Height equal to 530/400/400 mm, respectively, and weighing 34 kg. The screen resolution of Model 1 was 12x80, Model 11 - 24x80. The terminal was connected directly to the host system.
The terminals 5251 Display Station Model 2 (960 characters on the screen) and Model 12 (1920 characters on the screen) with large dimensions and weight of 45 kg were also quite common. They are distinguished from Model 1 and Model 11 by the possibility of forwarding the upstream connection through themselves to the host machine from cheaper clients in the form of Model 1 (or 11) terminals with desktop printers or a separate floor printer. Thus, models 2 and 12 acted as a hub, proxying the connection to the host from devices requiring a direct connection to the host machine, and cost significantly more.
The terminals of the 5252 Dual Display Station series will also seem unusual to the modern layman.
Promotional image from the IBM System / 38 Equipment and Programs brochure (5252 Dual Display Station)
The price of one terminal kit with a printer connected to it could reach several thousand US dollars.
The terminals were connected via a twinaxial cable to the host machine with a bus topology in half-duplex mode with a transmission speed of up to 1 Mbps. The maximum number of terminals supported by twinaxial is up to 6 terminals, and the terminal most remote from the host should be located at a distance of no more than 1500 meters.
The number of each terminal is set during its installation by three switches, so a unique address is determined within the bus. In the presence of an existing coaxial network, it is possible to use adapters from a twinaxial cable to a coaxial cable and an appropriate set of cable terminations for crimping. With this scheme, it was possible to connect only two devices on the bus with a maximum segment length of up to 30 meters. The total number of connected devices varied from a dozen to several dozen, depending on the model.
With the development of Desktop-systems and access networks, bulky terminals were replaced by workstations, where various expansion cards of third-party companies were used as means of access to the host machine, supporting direct connection via twinaxial. After IBM developed Token Ring technology in 1984, software solutions for accessing the machine, including through this interface, appeared.
5250 ISA adapter (unknown manufacturer)
Blackbox 5250 Adapter Cards (PC470C, PC471C, PC472C, PC473C, PC478C)
Emulators for MS-DOS and MS Windows appear both from IBM and from third-party manufacturers, including OpenSource implementations (for example, tn5250j.sourceforge.net). In the mid-90s, the TCP / IP stack came into the mid world -range and low-end business machines. To support access to host machines under the new protocol, IBM is developing 5250 series terminal emulators.
In order to create a protocol for exchanging with the host system, IBM is developing
extensions to the Telnet protocol (RFC 854, RFC 855, RFC 856, RFC 860, RFC 885, RFC 1091 , RFC 1205, RFC 1572, RFC 2877), collectively referred to as Telnet5250 (TN5250), which describes the process of receiving and transmitting 5250 data streams (5250 data streams) over the standard Telnet protocol.
IBM Client Access / 400 for Windows 3.1 Installer
What is special about the IBM 5250?
A feature of the IBM 5250 terminals (and, accordingly, the TN5250 protocol) is its block orientation, in contrast to the usual * nix terminals, which are symbol-oriented . This means that the 5250 data flows by which the host communicates with the terminal are transmitted by data blocks, and a separate symbol in it without the context of the transmitted block does not make sense.
For example, the host machine transmits to the terminal a data block containing the displayed static information on the screen along with the attributes and coordinates of the input fields and an indication of the offset in this block, where to write the result of user input to the fields. After that, the host machine expects messages from the terminal and does not participate in the user input process.
The login screen for the IBM i host RZKH.de (pub400.com) The
next task of the terminal emulator is to interpret the data block from the machine and form the user an input screen where they are given the opportunity to enter any information into the allowed fields. Also, the tasks of the terminal emulator include a reaction to user actions. The F1-F24 keys (F13-F24 are simulated via SHIFT + Fx), Enter, Home, End, PageUp, PageDn, and some other special keys that are absent on modern keyboards are considered host keys. This means that by pressing this key, a stream buffer with information from the input fields and the cursor position on the screen, previously filled with the terminal emulator, will be sent to the host for processing.
WIreshark 5250 Data Stream login attempt dump at pub400.com
The host receives the buffer, parses it, and the input result is transmitted to the program that requested the user’s reaction for further data validation and the application continues to work, while the application receives the code of the pressed host key.
Why is autotesting here at all
We thought about automating manual testing of green screen applications when we were faced with the need to test hundreds of screens of a developed module, where up to eighty different business checks (validations) could occur on one screen.
The team’s particular pain was the almost complete absence for 2017 of green screen auto-testing tools, except for the proprietary UIPath solution . Even today there are not many similar solutions, the author is aware of Automate from HelpSystems and the JMeter extension for BlazeMeter (I will be glad to know about other similar products).
The first research on the problem
The standard emulator of the TN5250 terminal installed at workplaces in the bank is IBM Personal Communications for Windows 6.0 (PCOMM 6.0). Colleagues found that this product has regular means of automating its management in the form of a diverse API, namely:
- High-level Language Application Program Interface (HLLAPI);
- Enhanced HLLAPI;
- Windows HLLAPI
- Host Access Client Library (HACL).
The first three interfaces are the oldest and have been supported since DOS and 16-bit versions of Windows. Work on the EHLLAPI interface is implemented by calling a single function according to the following prototype:
long hllapi (LPWORD, LPSTR, LPWORD, LPWORD);where the first parameter is a pointer to the numerical number of the function being executed, the other two are its arguments context-sensitive to the function being called, and the last is the result of the function. That is, in order to request the connection status 'A' (sessions in the emulator are numbered with a Latin letter of the range from 'A' to 'Z'), you must run the following code (taken from IBM documentation):
#include "hapi_c.h"
struct HLDQuerySessionStatus QueryData;
int Func, Len, Rc;
long Rc;
memset(QueryData, 0, sizeof(QueryData)); // Init buffer
QueryData.qsst_shortname = 'A'; // Session to query
Func = HA_QUERY_SESSION_STATUS; // Function number
Len = sizeof(QueryData); // Len of buffer
Rc = 0; // Unused on input
hllapi(&Func, (char *)&QueryData, &Len, &Rc); // Call EHLLAPI
if (Rc != 0) { // Check return code
// ...Error handling
}
The number of functions available for calling in this way is about 60.
The WinHLLAPI interface slightly expands this functionality with several additional functions that allow registering callback functions for asynchronous calls in order to notify about events of establishing a connection with the host, disconnecting from the host, changing data on the terminal screen, etc. .
interface Host Access Client Library (HACL) seemed more friendly to work because, in contrast to call "function one name", was granted an option of object-oriented hierarchy of classes s, allowing fully simulate any user action.
HACL Emulator Class Library Class Hierarchy (C ++)
There are HACL implementations for C ++, Java, LotusScript and a COM automation server for Windows (convenient for Visual Basic and .NET).
First prototype
Due to the enormous complexity of the 5250 data flow protocol and the extremely scarce information about its internal device with links to closed paid literature from IBM, it became obvious that developing your own emulator is extremely non-trivial and time-consuming. In this regard, the idea came up to use the middleware layer, which will allow you to control the terminal emulator within the minimum required functionality, in particular “enter a value in the field”, “compare part of the screen with the standard” or “press the host key F22”.
Colleagues who previously used HACL interfaces claimed (and a search on StackOverflow confirmed) that the COM object had stability problems and could hang after a certain number of commands were executed. Only restarting the automation server process helped. A quick analysis of the Java version showed that Wrapper is used over the C ++ interface through JNI. Therefore, the choice fell on the C ++ interface. The corresponding header files and .lib files were available in the installation directory of Personal Communications For Windows itself.
The first prototype was based on Qt5, where it was possible to execute JavaScript code through QtScript. In the environment of the executable script, an object was registered with a small number of methods that allowed executing commands in the terminal emulator as if they were being executed by a person (entering a field, pressing host keys, waiting for a line to appear on the screen). We demonstrated a live “demo”, where we scripted a user case for launching a green screen application from ABS Equation with a test of the reaction of the application to incorrect input to the fields. The demonstration showed that the prototype was successful and that we can move on.
The appearance of a neighbor
Along with the demonstration of the first prototype, colleagues from another department put together a bunch of Ruby + Cucumber + Quick3270 + Ruby module ( cheeze / te3270 ). The proposed option uses a Ruby module that interacts with the DN32 Computing Quick3270 terminal emulator through its specialized COM objects (incompatible with HACL interfaces). It was a complete solution for BDD-style green screen auto-testing applications, with a few pre-described steps. However, in the proposed solution, we were alarmed by the following:
- We used a third-party paid emulator not from IBM (all emulators work a little differently, but we need to check the work on the standard ones used in the bank, the price of the error is incredibly high);
- The implementations of Cucumber's steps for Quick3270 used a large number of sleeps to wait for a response from the machine;
- Very poor performance of Quick3270 via the automation interface (working with HACL in the prototype through the C ++ interface looked much more dynamic).
Quick3270 terminal emulator
Based on the prototype, we decided to try to implement our own automation server in order to connect Cucumber to Personal Communications for Windows and design the steps so that the downtime between the actions on the emulator screen is minimal.
Lyrical digression. Despite the fact that there are a huge number of technical problems around the supposedly "legacy" IBM, which, it would seem, should have already been solved for systems of the medium and enterprise business level, the relevance of adapting and transferring existing technical solutions is very high simply because of their absence on the platform. Often the absence is connected with the very features of this OS, which is fundamentally different from modern * nix, Windows or MacOS X, which requires significant optimization of software for this stack.
Own decision
As our own solution, we created an automation server as a development of the previously demonstrated prototype. This server executes commands to automate interaction from consumers through an RPC server (Qt5 WebSocket). It interacts with Personal Communications for Windows, which is part of the corporate Windows operating system image, and allows you to:
- start / stop terminal emulator sessions;
- Perform Screen Scraping Green Screen;
- search input fields on the screen;
- control the cursor and simulate keystrokes (including host);
- and etc.
Starting an Automation Server
However, with all the advantages of the HACL API, it has one drawback - it does not know how to work with the DB2 for i DBMS built into the OS and does not allow executing commands that are vital for building a mock environment where a test script would be executed. If the DB2 client for Ruby exists from IBM, then the client for the remote command and distributed program call server is only for Java in the form of the JTOpen library: The Open Source Version of the IBM Toolbox for Java (also known as jt400). We “peeped” the solution to this problem at IBM itself by analyzing the behavior of its products with similar functionality (in particular, Personal Communications for Windows Data Transfer, iSeries to PC / PC to iSeries Transfer, etc.). It turned out that these products, by their implementation, run IBM JRE 6 or 8, depending on the version of the application, and use the jt400 library.
For the automation server, we decided to do the same. The JNI launches the IBM JVM, which ships with Personal Communications for Windows. Using special wrapper methods, commands from the RPC server coming from outside are executed by proxying them into calls to the necessary jt400 functionality. Since the latter also contains a JDBC driver for DB2, it was decided to use it to access the DBMS on IBM i.
It is important to note that you cannot use Oracle JVM when using HACL. If you run a terminal emulator session, then trying to create an instance of the virtual machine will crash. Similarly, if you run the Oracle JVM in the address space of a process interacting with the HACL, the latter hangs without explanation.
Over time, the solution was implemented on an increasing number of jobs. It worked faster than the solution with Quick3270. Popularity grew, as did the number of autotests. However, during operation, additional difficulties arose:
- Occasional terminal freezes;
- Inability to work on a regression stand due to the terminal emulator refusing to start if the user’s desktop under which the emulator starts is blocked or its RDP session is blocked;
- Windows-only;
- A complex procedure for installing, configuring, and updating tools (via an msi package);
- Our regression cycle for 130 autotests (about 4000 steps) began to take 7-8 hours.
Something needs to be done ...
By analyzing the trace logs of numerous autotest launches, finding bottlenecks in the performance of frequently used steps, the total regression execution time was reduced to 4-5 hours. But it was clear that using the middleware layer in the form of an automation RPC server in conjunction with the HACL interface, which also has “floating” errors that accumulate over the duration of the entire system, will not help in improving the performance of the solution.
On the other hand, as an alternative to IBM Personal Communications for Windows, the vendor provides a cross-platform solution called IBM i Access - Client Solutions.
IBM i Access - Client Solutions
Analysis of its internal structure on Saturday and Sunday mornings over cups of coffee showed that its code base is based on another product from IBM called IBM Host on-Demand (IBM HOD). This is a full-fledged solution for accessing IBM i, developed in Java 6, which not only has the full implementation of various communication protocols used in IBM machines (TN3270, TN5250, VTxxx, etc.), but also high-level java-swing UI components, used to build your own terminal emulators in the form of a constructor, which can be assembled from scant documentationIBM A more detailed study of the IBM HOD showed that the UI components are based on the Java implementation of the HACL interface, whose documentation is open. Their behavior coincides with only slight differences from the C ++ HACL documentation.
IBM Host On-Demand (logo)
Next, we created a Java library for internal use, which implements the same interface as the C ++ RPC automation server, but internally uses IBM HOD. To reduce the overhead during the execution of the autotest steps, we migrated from Ruby Cucumber to cucumber-jvm with the reimplementation of all steps similar to the Ruby options. In the presence of a software interface similar to the RPC server, this was not a big deal, especially considering that we tried to restrain the uncontrolled growth in the number of steps themselves and we had this value in the region of 30 units.
What is the result
As a result, we achieved the operability of all autotests without changing them, and the speed of work became so high that we had to introduce an artificial delay between the steps so that when developing an autotest, you could observe its work, otherwise the UI did not have time to draw the screen to the end.
Already existing 180 autotests with more than 16,000 steps with a set delay of 60 ms between steps began to run about 30 minutes against 5 hours 30 minutes, which corresponds to an eleven-fold increase in the performance of the regression stand.
The results exceeded all expectations. We are close to the physical limits of the TN5250 protocol.
To date, the decision has been published to the entire bank, and colleagues from other cities have joined in the improvement. Of the recent changes, colleagues are integrating the solution with Jenkins, in the test version, testing the launch on a Linux server with Xvfb was completed and the stage of pilot operation of running autotests on it began.
Thank you for reading to the end!
All success!
PS In December 2018, the next IBMi Developer Conference was held, at which a report was made on the topic of this article.
Until now, we have held the Conference annually only for Bank employees. From 2019, we will invite participants from other companies. It is very interesting to expand the circle of professional and personal communication, share emotions, knowledge and experience.