July 29, 2013 at 13:17PBX Management - The Devil in Details
About a year ago, I had to face a new task for myself - automation of automatic telephone exchange management . At first glance, there is nothing complicated in it: you need to connect to the PBX control port via TCP or Serial-interface, send a command and analyze the response. As it turned out during the work, this simplicity turned out to be misleading.
In my article I want to talk about the difficulties that I had to face in the process of work. It is unlikely that this article will be interesting to a wide circle of readers, but perhaps telephony specialists will find something useful in it. As an illustrative material, I will use examples of Alcatel S12 and M200 PBX commands currently supported by the project.
To begin with, you should decide on the terminology.
A task is a set of actions performed atomically. From the point of view of the equipment management subsystem, the task consists of a set of services . Each service controls the subscriber’s access to a particular service (for example, it is possible to single out access services to long-distance and international calls).
When performing a task, individual services can be turned on or off. In addition, for a number of services, the values of configuration parameters can be set (for example, such as the alarm time or PIN code value). Boolean values that determine the state of connection or disconnection of a service are also considered as parameters.
From the point of view of equipment, the process of connecting or disconnecting a service is reduced to the execution of one or more commands . The process of executing commands on equipment will be called activation .
Specifications control the work of a specialized ORM designed to integrate an application with the Order Management subsystem that controls the execution of tasks. I will not delve into this topic, since it is not related to the specifics of interaction with the telephone exchange. To understand the following statement, it is enough to know that this layer provides read and write access to named variables containing values loaded from the database.
In addition to accessing scalar values, access to tuple collections is provided. So, for example, by opening a collection represented by the name 'tk' (Tech. Card), you can access records containing technical card data, such as 'tk.phone' - the subscriber's phone or 'tk.ats_type' - the type of PBX. In turn, these entries may contain links to other collections. The nesting of collections is not limited.
From all of the above, it is probably not very clear why the term specification is used? The fact is that when developing this layer, SID concepts such as Specification and Policy were actively used , which, from my point of view, greatly simplified the development. In general, I strongly recommend that you familiarize yourself with the TM Forum materials for anyone who has the opportunity.
This sub-system I described earlier . Initially, the script performed two tasks: it described the order in which commands were executed within the framework of the order and hid platform-dependent features. The script accessed the variables provided by the specifications and formed the commands, depending on the type of PBX. Here's how it looked:
From the above snippet, you can see that by analyzing the tk.attach.service variable we determine which command should be executed. Then, depending on the platform, by substituting the phone number from the tk.phone variable in the appropriate command template, a command is formed for execution on the equipment.
Since there were quite a lot of different commands, the script turned out to be large (more than 1000 lines) and obscure. As a result, when developing the second version, it was decided to abandon the formation of individual teams in the script (moving it to a lower level), passing service codes instead of ready-made commands. This decision had a very beneficial effect on the size of the script. Here's what the whole second version script looks like:
I will explain the logic of its implementation in the following sections, for now I will only say that now the script only controls the sequence of connecting and disconnecting services within the framework of the task. We can still execute different code, depending on the type of equipment used, but now it is used only if the dependence on the platform is important when performing the task as a whole, rather than individual commands.
The adapter is a plug ( plugin ), which implements the logic of a particular type of equipment. In the first version, the adapter received ready-made commands from the script and its role was to transfer these commands to the PBX using a TCP or Serial connection and process the received response. In the current (second) version, the logic of forming a set of commands, when connecting or disconnecting a particular service, is transferred to the adapter. An explanation of the reasons why this was done is the subject of this article.
We will call synchronization the process of obtaining the current settings of subscribers from equipment. The problem of inconsistent data on the status of subscribers' settings on the PBX with respect to the database is fundamental for this class of tasks. This problem is relevant not only for telephony. In the previous activation project related to the management of cisco equipment, he was no less acute. We can say that this article talks about how, using synchronization, to make the activation process more “smart”.
Obviously, completely different systems of commands are used to control various types of automatic telephone exchanges, for example, to disable outgoing communication in the automatic telephone exchange M-200, the following command is used:
and in Alcatel S12:
But such differences in command syntax are far from the only complexity. We will deal with this issue in more detail.
The first thing we encounter is that the possibilities for activating various services provided by different exchanges are also different. Of course, there is a certain general set of services supported by most types of automatic telephone exchanges, such as outgoing call management, setting an alarm, managing auto redial, etc.
At the same time, a significant part of the teams that are interesting from a business point of view may not be implemented on one or another type of PBX (and the larger the set of types of PBXs supported, the more such commands). This means that the service, which is part of the task, can by no means always be performed on the exchange where activation is performed. An attempt to activate such a service may be considered a mistake, or it may simply be ignored. It is important that a mechanism for displaying the services used as part of tasks for services implemented on supported types of automatic telephone exchanges should be implemented.
One job service can be mapped to several different services supported by the PBX. For example, the M-200 supports the following commands to enable a PIN code:
The first involves the use of a PIN code for long-distance and international calls, and the second for all outgoing calls. In our case, one of the requirements of the Customer was that both of these services, at the job level, should be displayed in one common PINCODE service. What kind of service will be performed is determined by the value of additional variables. For Alcatel S12, only one PIN code enable command is implemented:
In addition to the fact that there is no separation of this service into two types, you can see that this command not only includes the use of a PIN code, but also sets its value. Of course, the M-200 allows you to execute, for example, the following command:
But, in the process of introducing the first version of the activation module into commercial operation, it turned out that these commands are more convenient to activate separately. In addition, it turned out that in order for the PIN code to work, you need to activate another command. Thus, for the M-200, the PIN installation command should lead to the activation of three commands in the specified order:
Moreover, the enb_pincode command, which controls the possibility of using a PIN code, can be executed separately (thus adding another option to display the 'PINCODE' service). In this case, the PIN code can be set by the subscriber. This leads to a rather non-trivial problem, which I will discuss below in the section “Problems related to synchronization”.
Automatic telephone exchanges of various types can differ not only in the set of supported services, but also in how these services are implemented. So for the M-200 access to intraband, intercity and international calls can be controlled separately:
For Alcatel S12, similar commands look like this:
You may notice that these commands control the state of only one setting (when you enable long distance communication, the restrictions are simply removed). This leads to the following problem:
This means that if we need to enable intra-area and long-distance communication on the Alcatel S12, we only need to activate the long-distance communication enable command. If the intra-zone communication enable command is subsequently activated, long-distance communication for the subscriber will be disconnected! Simultaneous activation of intrazonal and international communications, with long-distance disconnected, is impossible (unlike M-200).
This is a good example of functionality implemented at the activation script level. Let's see how this can be implemented:
We are saved by variables. Having activated the first command that controls outgoing communication from the tk.attach collection, we set the is_dou_activated flag, which prohibits activation of subsequent commands. Moreover, this code only works for Alcatel S12.
The only thing we need for this approach to work is that the teams come in the correct order: first, the inclusion of international communications (if any), then long-distance and intra-regional. Fortunately, specifications allow you to sort
collections of samples by any criterion. You only need to add priority fields to the service table when connecting and disconnecting.
As I said above, the ability to obtain current settings from the equipment is one of the main requirements for performing at least some correct activation. Let's see why this is so?
The very first additional requirement that we received from the Customer when introducing the first version of the product was that the settings previously activated on the equipment should not be re-activated. This is really important for two reasons:
All this means that before activating any command, we must receive the current settings from the PBX that affect the execution of this command. If the settings are already in the required state, we can not execute the activated command and move on (if the activated command cannot be executed due to conflicting settings, we can also not execute the command, but immediately raise an exception). This approach works great for the M-200 because:
For Alcatel S12, everything is not so rosy. To read the settings, you will have to execute various commands and all of them will be executed slowly. Thus, storage of the state of settings in the database is indispensable. When activating, we can get the settings from the database and, only if they are not there, generate the necessary commands to get the missing values from the equipment.
Of course, this is not an ideal solution, because when manually executing commands, bypassing the activation subsystem, the data in the database will be out of sync, but this is the lesser evil that we can afford. In addition, upon completion of the activation of the command, the actual settings will become known to us (since the team successfully changed them) and we will be able to update the out of sync data in the database.
There is also a problem when using the M-200. In some cases, the tune server used to interact with the PBX, which I will discuss below, starts to work incorrectly. He answers Ok with all settune commands, although the data on the exchange does not change! Calling gettune, in this case, leads to an error.
This means that for the M-200, we must call gettune not only before, but after activation, and, in the event of an error, we must initiate a second task. As soon as the tune-server starts working correctly, we will be able to activate those teams whose activation did not actually take place at the previous task. What was successfully activated will not be reactivated.
As I said above, on the M-200 PBX, the inclusion of the PINCODE service (depending on the values of the specification variables) can lead to the activation of one of three sequences of commands:
The enb_pincode setting is enabled in all three cases. It is easy to imagine a possible error that occurs with the naive implementation of activating these commands:
As a result of this sequence of actions, the enb_pincode setting is turned off and the PIN code set in the second step stops working! What can be done to prevent this from happening?
The solution, in general, is obvious. We must record all activated services in the database and conduct additional checks when activating shutdown commands. In case the disconnected setting is still used by some other non-disconnected service, the disconnected command should be ignored.
Yes, the subscriber will be able to use the enb_pincode service, which he has actually refused, but the set PIN codes will work without any complaints from the subscriber. There are not many similar dependencies in the M200 command system, but all of them are associated with important services (such as setting an alarm or enabling call forwarding). Accounting for these dependencies is important from the point of view of correct activation.
Not every task can be completed successfully. The activation of the command may result in an error due to incorrect data (the subscriber’s number is not served by the given PBX, incorrect time when setting the alarm, etc.), or in case of a conflict of activated services (this happens quite often on Alcatel S12). Such errors will be called unrecoverable.
Since the task must be performed atomically, if an irreparable error occurs, we will have to “roll back” all the commands that were successfully activated earlier, as part of the same task. The first thing that comes to mind is to remember all the commands that it was possible to execute and, if an error occurs, to execute the reverse commands. The implementation of this approach can be seen in the old version of the script:
Here, immediately after the successful disconnection of the 'PINCODE' service, the reverse commands are saved to activate the service in the rollback command stack. In addition to being too verbose, this approach has a number of other disadvantages.
Thus, the only place from which we can get the correct settings, at the time of activation, is equipment or a database, provided that the data in it is synchronized with the state of the exchange.
In order to close this question, it remains to mention one more point. The fact is that the activation task can perform activation of commands on several exchanges (for example, when a subscriber is transferred from one exchange to another).
The settings for access to the PBX are stored in the tk collection and it may turn out that after successfully executing the commands on one PBX, we received an error while executing the command on another. In the open record of the tk collection, at this point, there are no more parameters for access to the PBX on which the rollback should be performed. The easiest way to deal with this problem is to initiate a restart to activate the task after initiating the rollback. In this case, all collections will be re-opened and the “ROLLBACK” service will be performed for all exchanges participating in the activation:
In addition to the unrecoverable errors discussed in the previous section, errors are possible, after which the activation can continue. For example, if the TCP connection to the PBX was broken during the activation process, you do not need to roll back. In this case, it is enough to repeat the activation, skipping the execution of the commands activated earlier. Here's what the hierarchy of exceptions thrown during the activation process looks like:
Exceptions implying the need to resume activation are inherited from RetryRequiredException. RollbackRequiredException additionally triggers an activation rollback. Unplanned errors raise a RuntimeAeException, terminating the activation service with the appropriate diagnostics. Resumption of activation due to loss of connection with the PBX (TcpConnectionLostException) can occur in two fundamentally different cases:
In the first case, no intervention by the administrator is required. Since the PBX, in principle, is available by repeating the activation several times (with a delay of several minutes), we will complete the task sooner or later.
In the second case, an error, for example, can be caused by the fact that access parameters to the PBX are set incorrectly. In this case, several connection attempts should be made, after which an error message should be generated. This logic is implemented by the script:
At the very beginning of the script, we increase the counter of attempts (if the variable is not defined by the specification, it is automatically created with a zero value), and after the successful execution of any command, we reset it to 0. Exceeding the counter value of the maximum number of connection attempts is monitored by the specification and throws an exception, with appropriate diagnostics.
An important point is the need to validate parameter values that are substituted into commands. As I said above, passing an incorrect value to a parameter can lead to the inability to automatically execute the following commands (for example, rollback commands):
Of course, you can analyze the patterns of such errors and automatically generate additional commands that put the exchange in the waiting state of the next command, but this will significantly complicate the algorithm for interacting with the exchange. It is much easier to avoid such mistakes. This is pretty easy to do. For example, to validate the value of the number of times the alarm goes off, the zero value of which led to an error in this example, it is enough to use the following regular expression:
Of course, it is best to perform this check back in Order Management, preventing the creation of jobs with incorrect parameter values. But, regardless of whether such a check is implemented or not, it makes sense to check the parameter values immediately before sending the command to the exchange. This will help to avoid unnecessary mistakes and will simplify the logic of interaction with the PBX.
Features of access to the PBX should also be considered when designing. A common point for both Alcatel S12 and M-200 (using tune) is that a management session is not created for each connection to the PBX. In fact, every time we connect to the same “session” (this is a bit of a simplification, but, for the activation subsystem, it looks just like that). There are two important consequences of this fact:
You should also consider a number of features Alcatel S12. So, in its output there may be unprintable characters (0x05, 0x17), and when entering commands the characters of line feed and carriage return are not used. The command line sent to the Alcatel S12 must end with the character '.' or ';' (the password transmitted during authorization must also end with the symbol ';'). The most significant bit in each byte transmitted to the PBX can be used for parity (this is configured on the PBX). These settings do not apply to text transmitted from the PBX.
Currently, a Telnet-like TCP connection is used to connect to both the Alcatel S12 and the M-200, but this solution is not the only one possible. Other options for connecting to these exchanges are briefly discussed below.
As I said earlier, at the time of the start of the project, I did not have any experience with the Alcatel S12. The fact that with this type of automatic telephone exchange (in any case, in the configuration used by the Customer) will have to work on a Serial connection, from the very beginning, was considered as one of the most significant risks of the project. This may seem strange, but at the same time I was right and wrong.
Of course, when using libraries such as RXTX or jSSC (which I learned from an article on Habré), connecting to the Serial port from Java SE (used in the project) by itself is not a problem, but there are two points to consider:
The solution was found in the process of communication with the Customer, who had more experience in interacting with the PBX. Of course, the telephone operators also wanted to manage all the PBXs from one place and they had been successfully doing this for a long time, using the MOXA communication equipment , which allowed them to “flip through” the Serial connection via Intranet. Unfortunately, there were problems trying to connect to the virtual port created by MOXA on the client side .
We spent quite a bit of time trying to solve this problem until we found out that some MOXA models allow you to use a TCP connection. This approach was used. In our work, the Serial-port monitor and sniffer helped us a lot .
There were fewer problems connecting to the M-200, as it initially supported a TCP connection. It was originally planned to use direct access to the "subscriber card". I will not clutter up the article with dumps of TCP sessions, I will only say that the settings of subscribers in the M-200 are stored in a bitmap. The PBX control protocol allows (using TCP) to read this entire bitmap and write it back, after making the necessary changes.
We managed to decipher some of the fields of the “subscriber card” (we could not find the documentation), but in the end, it was decided to use the tune server as a more documented method for managing the PBX. In addition, working in the input mode of text commands was more convenient.
In this article I tried to describe (and somehow classify) the difficulties that arose in a real activation project. Despite the pronounced focus on traditional telephony equipment, the material can be useful when working on automation of control of any equipment (for example, cisco switches). Hope this article is helpful to someone.
In my article I want to talk about the difficulties that I had to face in the process of work. It is unlikely that this article will be interesting to a wide circle of readers, but perhaps telephony specialists will find something useful in it. As an illustrative material, I will use examples of Alcatel S12 and M200 PBX commands currently supported by the project.
1. Glossary
To begin with, you should decide on the terminology.
1.1 Assignments
A task is a set of actions performed atomically. From the point of view of the equipment management subsystem, the task consists of a set of services . Each service controls the subscriber’s access to a particular service (for example, it is possible to single out access services to long-distance and international calls).
When performing a task, individual services can be turned on or off. In addition, for a number of services, the values of configuration parameters can be set (for example, such as the alarm time or PIN code value). Boolean values that determine the state of connection or disconnection of a service are also considered as parameters.
From the point of view of equipment, the process of connecting or disconnecting a service is reduced to the execution of one or more commands . The process of executing commands on equipment will be called activation .
1.2 Specifications
Specifications control the work of a specialized ORM designed to integrate an application with the Order Management subsystem that controls the execution of tasks. I will not delve into this topic, since it is not related to the specifics of interaction with the telephone exchange. To understand the following statement, it is enough to know that this layer provides read and write access to named variables containing values loaded from the database.
In addition to accessing scalar values, access to tuple collections is provided. So, for example, by opening a collection represented by the name 'tk' (Tech. Card), you can access records containing technical card data, such as 'tk.phone' - the subscriber's phone or 'tk.ats_type' - the type of PBX. In turn, these entries may contain links to other collections. The nesting of collections is not limited.
From all of the above, it is probably not very clear why the term specification is used? The fact is that when developing this layer, SID concepts such as Specification and Policy were actively used , which, from my point of view, greatly simplified the development. In general, I strongly recommend that you familiarize yourself with the TM Forum materials for anyone who has the opportunity.
1.3 Scripts
This sub-system I described earlier . Initially, the script performed two tasks: it described the order in which commands were executed within the framework of the order and hid platform-dependent features. The script accessed the variables provided by the specifications and formed the commands, depending on the type of PBX. Here's how it looked:
Fragment of the script of the first version
...
[002082] platform:M-200; if (tk.attach.service = 'SET_HOLD') {
[020820] < text:settune %s enb_flash=on; var_list:tk.phone;
[001041] > is_error:1; regexp:(.+); var_list:error;
[020821] is_rollback:1; {
[001020] < device_id:tk.ats_id;
[020822] < text:settune %s enb_flash=off; var_list:tk.phone;
[001041] > is_error:1; regexp:(.+); var_list:error;
}
}
[003082] platform:S-12; if (tk.attach.service = 'SET_HOLD') {
[030820] < text:MODIFY-SUBSCR:DN=K'%s,RECALL=ADD&ECTRF.; var_list:tk.phone;
[001041] > is_error:1; regexp:(.+); var_list:error;
[030821] is_rollback:1; {
[001020] < device_id:tk.ats_id;
[030822] < text:MODIFY-SUBSCR:DN=K'%s,RECALL=REMOVE&ECTRF.; var_list:tk.phone;
[001041] > is_error:1; regexp:(.+); var_list:error;
}
}
...
From the above snippet, you can see that by analyzing the tk.attach.service variable we determine which command should be executed. Then, depending on the platform, by substituting the phone number from the tk.phone variable in the appropriate command template, a command is formed for execution on the equipment.
Since there were quite a lot of different commands, the script turned out to be large (more than 1000 lines) and obscure. As a result, when developing the second version, it was decided to abandon the formation of individual teams in the script (moving it to a lower level), passing service codes instead of ready-made commands. This decision had a very beneficial effect on the size of the script. Here's what the whole second version script looks like:
The script of the second version
[002000] {
[200001] var_list:retry_cnt = retry_cnt + 1, state = 1;
[200002] if (subtype = -1) {
[200003] foreach (devices) {
[200004] < device_id:devices.ats_id; device_ip:devices.ats_ip; device_password:devices.ats_password; device_port:devices.ats_tcp_port; device_protocol:devices.ats_type;
[200005] < device_id:devices.ats_id;
[200006] < text:ROLLBACK;
[200007] var_list:retry_cnt = 0;
}
}
[200008] if (subtype = 1) {
[200009] foreach (tk) {
[200010] if (tk.phone = '') {
[200011] var_list:tk.phone = tk.phone_old;
}
[200012] < device_id:tk.ats_id; device_protocol:tk.ats_type; device_ip:tk.ats_ip; device_port:tk.ats_tcp_port; device_password:tk.ats_password;
[200013] var_list:action = 0, is_dou_activated = 0;
[200014] target:tk.ats_type; foreach (tk.detach) {
[200015] < device_id:tk.ats_id;
[200016] var_list:activate_command = 1;
[200017] platform:S-12; if (tk.detach.service = 'C_INTRAAREAL' | tk.detach.service = 'C_INTERCITY' | tk.detach.service = 'C_INTERNATIONAL') {
[200018] if (is_dou_activated = 1) {
[200019] var_list:activate_command = 0;
}
[200020] var_list:is_dou_activated = 1;
}
[200025] if (activate_command = 1) {
[200026] < text:%s; var_list:tk.detach.service;
[200027] var_list:retry_cnt = 0;
}
}
[200028] var_list:action = 1, is_dou_activated = 0, is_cat_activated = 0;
[200029] target:tk.ats_type; foreach (tk.attach) {
[200030] < device_id:tk.ats_id;
[200031] var_list:activate_command = 1;
[200032] platform:S-12; if (tk.attach.service = 'C_INTRAAREAL' | tk.attach.service = 'C_INTERCITY' | tk.attach.service = 'C_INTERNATIONAL') {
[200033] if (is_dou_activated = 1) {
[200034] var_list:activate_command = 0;
}
[200035] var_list:is_dou_activated = 1;
}
[200021] if (tk.attach.service = 'CATEG_AON_0' | tk.attach.service = 'CATEG_AON_1' | tk.attach.service = 'CATEG_AON_2' | tk.attach.service = 'CATEG_AON_3' | tk.attach.service = 'CATEG_AON_4' | tk.attach.service = 'CATEG_AON_6' | tk.attach.service = 'CATEG_AON_7' | & tk.attach.service = 'CATEG_AON_8' | tk.attach.service = 'CATEG_AON_9') {
[200022] if (is_cat_activated = 1) {
[200023] var_list:activate_command = 0;
}
[200024] var_list:is_cat_activated = 1;
}
[200036] if (activate_command = 1) {
[200037] < text:%s; var_list:tk.attach.service;
[200038] var_list:retry_cnt = 0;
}
}
}
}
}
I will explain the logic of its implementation in the following sections, for now I will only say that now the script only controls the sequence of connecting and disconnecting services within the framework of the task. We can still execute different code, depending on the type of equipment used, but now it is used only if the dependence on the platform is important when performing the task as a whole, rather than individual commands.
1.4 adapter
The adapter is a plug ( plugin ), which implements the logic of a particular type of equipment. In the first version, the adapter received ready-made commands from the script and its role was to transfer these commands to the PBX using a TCP or Serial connection and process the received response. In the current (second) version, the logic of forming a set of commands, when connecting or disconnecting a particular service, is transferred to the adapter. An explanation of the reasons why this was done is the subject of this article.
1.5 Sync
We will call synchronization the process of obtaining the current settings of subscribers from equipment. The problem of inconsistent data on the status of subscribers' settings on the PBX with respect to the database is fundamental for this class of tasks. This problem is relevant not only for telephony. In the previous activation project related to the management of cisco equipment, he was no less acute. We can say that this article talks about how, using synchronization, to make the activation process more “smart”.
2. Job Level
Obviously, completely different systems of commands are used to control various types of automatic telephone exchanges, for example, to disable outgoing communication in the automatic telephone exchange M-200, the following command is used:
settune XXXXXXX cmn_outcome=off
and in Alcatel S12:
MODIFY-SUBSCR:DN=K'XXXXXXX,OCB=REMOVE&TOTALBAR.
But such differences in command syntax are far from the only complexity. We will deal with this issue in more detail.
2.1 Mapping services to teams
The first thing we encounter is that the possibilities for activating various services provided by different exchanges are also different. Of course, there is a certain general set of services supported by most types of automatic telephone exchanges, such as outgoing call management, setting an alarm, managing auto redial, etc.
At the same time, a significant part of the teams that are interesting from a business point of view may not be implemented on one or another type of PBX (and the larger the set of types of PBXs supported, the more such commands). This means that the service, which is part of the task, can by no means always be performed on the exchange where activation is performed. An attempt to activate such a service may be considered a mistake, or it may simply be ignored. It is important that a mechanism for displaying the services used as part of tasks for services implemented on supported types of automatic telephone exchanges should be implemented.
One job service can be mapped to several different services supported by the PBX. For example, the M-200 supports the following commands to enable a PIN code:
settune XXXXXXX dvo_pincode=on
settune XXXXXXX dvo_pincodetwo=on
The first involves the use of a PIN code for long-distance and international calls, and the second for all outgoing calls. In our case, one of the requirements of the Customer was that both of these services, at the job level, should be displayed in one common PINCODE service. What kind of service will be performed is determined by the value of additional variables. For Alcatel S12, only one PIN code enable command is implemented:
MODIFY-SUBSCR:DN=K'XXXXXXX,PASSWORD=ADD&"YYYY",SUBCTRL=ADD&OCBVAR.
In addition to the fact that there is no separation of this service into two types, you can see that this command not only includes the use of a PIN code, but also sets its value. Of course, the M-200 allows you to execute, for example, the following command:
settune XXXXXXX dvo_pincode=on pincode=YYYY
But, in the process of introducing the first version of the activation module into commercial operation, it turned out that these commands are more convenient to activate separately. In addition, it turned out that in order for the PIN code to work, you need to activate another command. Thus, for the M-200, the PIN installation command should lead to the activation of three commands in the specified order:
settune XXXXXXX enb_pincode=on
settune XXXXXXX dvo_pincode=on
settune XXXXXXX pincode=YYYY
Moreover, the enb_pincode command, which controls the possibility of using a PIN code, can be executed separately (thus adding another option to display the 'PINCODE' service). In this case, the PIN code can be set by the subscriber. This leads to a rather non-trivial problem, which I will discuss below in the section “Problems related to synchronization”.
2.2 Outgoing call restriction
Automatic telephone exchanges of various types can differ not only in the set of supported services, but also in how these services are implemented. So for the M-200 access to intraband, intercity and international calls can be controlled separately:
settune XXXXXXX cmn_82xxx=on
settune XXXXXXX cmn_8xxx=on
settune XXXXXXX cmn_10xxx=on
For Alcatel S12, similar commands look like this:
MODIFY-SUBSCR:DN=K'XXXXXXX,OCB=MODIFY&PERM&5.
MODIFY-SUBSCR:DN=K'XXXXXXX,OCB=MODIFY&PERM&4.
MODIFY-SUBSCR:DN=K'XXXXXXX,OCB=REMOVE.
You may notice that these commands control the state of only one setting (when you enable long distance communication, the restrictions are simply removed). This leads to the following problem:
- For the M-200, the intrazonal, intercity and international inclusion services must be activated independently (and assigned to the corresponding services of the assignment one-to-one)
- When activated on Alcatel S12, only one of the included services should be activated, the least restrictive access
This means that if we need to enable intra-area and long-distance communication on the Alcatel S12, we only need to activate the long-distance communication enable command. If the intra-zone communication enable command is subsequently activated, long-distance communication for the subscriber will be disconnected! Simultaneous activation of intrazonal and international communications, with long-distance disconnected, is impossible (unlike M-200).
This is a good example of functionality implemented at the activation script level. Let's see how this can be implemented:
[200028] var_list:action = 1, is_dou_activated = 0, is_cat_activated = 0;
[200029] target:tk.ats_type; foreach (tk.attach) {
[200031] var_list:activate_command = 1;
[200032] platform:S-12; if (tk.attach.service = 'C_INTRAAREAL' |
tk.attach.service = 'C_INTERCITY' |
tk.attach.service = 'C_INTERNATIONAL') {
[200033] if (is_dou_activated = 1) {
[200034] var_list:activate_command = 0;
}
[200035] var_list:is_dou_activated = 1;
}
[200036] if (activate_command = 1) {
[200037] < text:%s; var_list:tk.attach.service;
[200038] var_list:retry_cnt = 0;
}
}
We are saved by variables. Having activated the first command that controls outgoing communication from the tk.attach collection, we set the is_dou_activated flag, which prohibits activation of subsequent commands. Moreover, this code only works for Alcatel S12.
The only thing we need for this approach to work is that the teams come in the correct order: first, the inclusion of international communications (if any), then long-distance and intra-regional. Fortunately, specifications allow you to sort
collections of samples by any criterion. You only need to add priority fields to the service table when connecting and disconnecting.
3. Problems related to synchronization
As I said above, the ability to obtain current settings from the equipment is one of the main requirements for performing at least some correct activation. Let's see why this is so?
3.1 Intelligent Activation
The very first additional requirement that we received from the Customer when introducing the first version of the product was that the settings previously activated on the equipment should not be re-activated. This is really important for two reasons:
- Activating commands on equipment is not a quick process. The average execution time of one command on Alcatel S12 is ~ 10 seconds (the M-200 works much faster (~ 1 sec), since it returns a much smaller amount of data). Add to this the fact that the connection with the PBX is very unstable and we can lose it during these very 10 seconds. If at each repeated execution of the task (and we must execute all the commands making up the task to ensure atomicity of its execution) we will reactivate all the commands from the very beginning, then in the worst case, we will activate these commands again and again and never get to the end of the assignment
- In some cases (on Alcatel S12), reactivating a command can lead to an error. In this case, we cannot see the status of the settings in the database, because, perhaps, activation for this subscriber is carried out for the first time. In this case, the only right decision is to get the status of the settings directly from the equipment, before the activation command
All this means that before activating any command, we must receive the current settings from the PBX that affect the execution of this command. If the settings are already in the required state, we can not execute the activated command and move on (if the activated command cannot be executed due to conflicting settings, we can also not execute the command, but immediately raise an exception). This approach works great for the M-200 because:
- All subscriber settings are returned in response to just one gettune command
- Activation on the M-200 is relatively quick and the connection is stable. We can afford to execute one additional command at the beginning of each activation
For Alcatel S12, everything is not so rosy. To read the settings, you will have to execute various commands and all of them will be executed slowly. Thus, storage of the state of settings in the database is indispensable. When activating, we can get the settings from the database and, only if they are not there, generate the necessary commands to get the missing values from the equipment.
Of course, this is not an ideal solution, because when manually executing commands, bypassing the activation subsystem, the data in the database will be out of sync, but this is the lesser evil that we can afford. In addition, upon completion of the activation of the command, the actual settings will become known to us (since the team successfully changed them) and we will be able to update the out of sync data in the database.
There is also a problem when using the M-200. In some cases, the tune server used to interact with the PBX, which I will discuss below, starts to work incorrectly. He answers Ok with all settune commands, although the data on the exchange does not change! Calling gettune, in this case, leads to an error.
This means that for the M-200, we must call gettune not only before, but after activation, and, in the event of an error, we must initiate a second task. As soon as the tune-server starts working correctly, we will be able to activate those teams whose activation did not actually take place at the previous task. What was successfully activated will not be reactivated.
3.2 Dependent Services
As I said above, on the M-200 PBX, the inclusion of the PINCODE service (depending on the values of the specification variables) can lead to the activation of one of three sequences of commands:
settune XXXXXXX enb_pincode=on
settune XXXXXXX enb_pincode=on
settune XXXXXXX dvo_pincode=on
settune XXXXXXX pincode=YYYY
settune XXXXXXX enb_pincode=on
settune XXXXXXX dvo_pincodetwo=on
settune XXXXXXX pincode=YYYY
The enb_pincode setting is enabled in all three cases. It is easy to imagine a possible error that occurs with the naive implementation of activating these commands:
- The service is activated, including the enb_pincode setting, which allows the subscriber to set the PIN code independently
- A service is activated that sets the PIN code for outgoing communication (within which enb_pincode is activated again)
- The subscriber refuses the service enb_pincode
As a result of this sequence of actions, the enb_pincode setting is turned off and the PIN code set in the second step stops working! What can be done to prevent this from happening?
The solution, in general, is obvious. We must record all activated services in the database and conduct additional checks when activating shutdown commands. In case the disconnected setting is still used by some other non-disconnected service, the disconnected command should be ignored.
Yes, the subscriber will be able to use the enb_pincode service, which he has actually refused, but the set PIN codes will work without any complaints from the subscriber. There are not many similar dependencies in the M200 command system, but all of them are associated with important services (such as setting an alarm or enabling call forwarding). Accounting for these dependencies is important from the point of view of correct activation.
3.3 Rollback Implementation
Not every task can be completed successfully. The activation of the command may result in an error due to incorrect data (the subscriber’s number is not served by the given PBX, incorrect time when setting the alarm, etc.), or in case of a conflict of activated services (this happens quite often on Alcatel S12). Such errors will be called unrecoverable.
Since the task must be performed atomically, if an irreparable error occurs, we will have to “roll back” all the commands that were successfully activated earlier, as part of the same task. The first thing that comes to mind is to remember all the commands that it was possible to execute and, if an error occurs, to execute the reverse commands. The implementation of this approach can be seen in the old version of the script:
...
[002391] platform:M-200; if (tk.detach.service = 'PINCODE' &
tk.detach.act_mode = 0 &
tk.detach.act_level = 2) {
[023910] < text:settune %s dvo_pincode=off; var_list:tk.phone;
[001041] > is_error:1; regexp:(.+); var_list:error;
[001610] < text:settune %s pincode=; var_list:tk.phone;
[001041] > is_error:1; regexp:(.+); var_list:error;
[022010] < text:settune %s enb_pincode=off; var_list:tk.phone;
[001041] > is_error:1; regexp:(.+); var_list:error;
[023911] is_rollback:1; if (tk.detach.value != '') {
[001020] < device_id:tk.ats_id;
[023912] < text:settune %s dvo_pincode=on; var_list:tk.phone;
[001041] > is_error:1; regexp:(.+); var_list:error;
[001603] < text:settune %s pincode=%s; var_list:tk.phone, tk.detach.value;
[001041] > is_error:1; regexp:(.+); var_list:error;
}
}
...
Here, immediately after the successful disconnection of the 'PINCODE' service, the reverse commands are saved to activate the service in the rollback command stack. In addition to being too verbose, this approach has a number of other disadvantages.
- The correct implementation of this approach is very laborious. The fact is that to execute inverse commands, the values of the variables are required at the time of activation of the direct command, but some of these variables are in the collections viewed by the foreach operator. This means that when forming the reverse command, it is required to save “snapshots” of the state of these collections (and the subcollections embedded in them) and to use them instead of the original specifications when rolling back. In addition, commands such as resetting the PIN code require a PIN code value for the reverse command so that it can be restored, and we have to add this value to the specification, although the direct command does not use it
- Running rollback commands after an activation error occurs can lead to an error on the Alcatel S12. For example, if one of the parameters is incorrectly set, the PBX requests the correct value and expects to receive it, rather than a rollback command. Receiving a command in this context leads to an error, as a result of which, the rollback command is not executed. In the section “Validation of parameters” I will analyze this question in more detail.
- This approach does not work! In fact, we have nowhere to get the correct parameter values not only for the PIN code installation command, but also for the enb_pincode and dvo_pincode enable commands. If we disabled them during the task, this does not mean that before the task was activated, they were no longer turned off. In this case, the inclusion of these settings during the rollback process will be an error
Thus, the only place from which we can get the correct settings, at the time of activation, is equipment or a database, provided that the data in it is synchronized with the state of the exchange.
In order to close this question, it remains to mention one more point. The fact is that the activation task can perform activation of commands on several exchanges (for example, when a subscriber is transferred from one exchange to another).
The settings for access to the PBX are stored in the tk collection and it may turn out that after successfully executing the commands on one PBX, we received an error while executing the command on another. In the open record of the tk collection, at this point, there are no more parameters for access to the PBX on which the rollback should be performed. The easiest way to deal with this problem is to initiate a restart to activate the task after initiating the rollback. In this case, all collections will be re-opened and the “ROLLBACK” service will be performed for all exchanges participating in the activation:
...
[200002] if (subtype = -1) {
[200003] foreach (tk) {
[200015] < device_id:tk.ats_id;
[200006] < text:ROLLBACK;
...
}
}
...
3.4 Renewable Activation
In addition to the unrecoverable errors discussed in the previous section, errors are possible, after which the activation can continue. For example, if the TCP connection to the PBX was broken during the activation process, you do not need to roll back. In this case, it is enough to repeat the activation, skipping the execution of the commands activated earlier. Here's what the hierarchy of exceptions thrown during the activation process looks like:
Exceptions implying the need to resume activation are inherited from RetryRequiredException. RollbackRequiredException additionally triggers an activation rollback. Unplanned errors raise a RuntimeAeException, terminating the activation service with the appropriate diagnostics. Resumption of activation due to loss of connection with the PBX (TcpConnectionLostException) can occur in two fundamentally different cases:
- Lost PBX connection during activation
- The inability to connect to the PBX
In the first case, no intervention by the administrator is required. Since the PBX, in principle, is available by repeating the activation several times (with a delay of several minutes), we will complete the task sooner or later.
In the second case, an error, for example, can be caused by the fact that access parameters to the PBX are set incorrectly. In this case, several connection attempts should be made, after which an error message should be generated. This logic is implemented by the script:
[002000] {
[200001] var_list:retry_cnt = retry_cnt + 1, state = 1;
...
[200008] if (subtype = 1) {
[200009] foreach (tk) {
...
[200014] target:tk.ats_type; foreach (tk.detach) {
[200015] < device_id:tk.ats_id;
...
[200025] if (activate_command = 1) {
[200026] < text:%s; var_list:tk.detach.service;
[200027] var_list:retry_cnt = 0;
}
}
...
}
At the very beginning of the script, we increase the counter of attempts (if the variable is not defined by the specification, it is automatically created with a zero value), and after the successful execution of any command, we reset it to 0. Exceeding the counter value of the maximum number of connection attempts is monitored by the specification and throws an exception, with appropriate diagnostics.
4. Validation of parameters
An important point is the need to validate parameter values that are substituted into commands. As I said above, passing an incorrect value to a parameter can lead to the inability to automatically execute the following commands (for example, rollback commands):
10.0.5.130:4001> MODIFY-SUBSCR:DN=K'8553377684,ALMCALL=ACTIVATE&06&00&00.
10.0.5.130:4001> SEQ=1306.130403 9002
10.0.5.130:4001> COM=4294
10.0.5.130:4001> JOB SUBMITTED
10.0.5.130:4001>
10.0.5.130:4001>
10.0.5.130:4001> ARGUMENT SEMANTIC ERROR : ALMCALL ARG 0004
10.0.5.130:4001> ERROR CODE = 0008
10.0.5.130:4001> <
10.0.5.130:4001< MODIFY-SUBSCR:DN=K'8553377684,ALMCALL=ACTIVATE&06&00&00.
10.0.5.130:4001> MODIFY-SUBSCR:DN=K'8553377684,SUBCTRL=REMOVE&CWTG.
10.0.5.130:4001> PARAMETER NOT EXISTENT ; MODIFY-S
10.0.5.130:4001> <
Of course, you can analyze the patterns of such errors and automatically generate additional commands that put the exchange in the waiting state of the next command, but this will significantly complicate the algorithm for interacting with the exchange. It is much easier to avoid such mistakes. This is pretty easy to do. For example, to validate the value of the number of times the alarm goes off, the zero value of which led to an error in this example, it is enough to use the following regular expression:
^([1-9])$
Of course, it is best to perform this check back in Order Management, preventing the creation of jobs with incorrect parameter values. But, regardless of whether such a check is implemented or not, it makes sense to check the parameter values immediately before sending the command to the exchange. This will help to avoid unnecessary mistakes and will simplify the logic of interaction with the PBX.
5. Access to the telephone exchange
Features of access to the PBX should also be considered when designing. A common point for both Alcatel S12 and M-200 (using tune) is that a management session is not created for each connection to the PBX. In fact, every time we connect to the same “session” (this is a bit of a simplification, but, for the activation subsystem, it looks just like that). There are two important consequences of this fact:
- Подключаясь к АТС мы можем получить часть вывода оставшегося от предыдущего подключения. Возможны очень неприятные сценарии. Например, при подключении к Alcatel S12 может потребоваться авторизация. Мы определяем необходимость выполнения авторизации по наличию в выводе паттерна 'PASSWORD' и готовимся передать пароль. Если в этот момент теряется соединение с АТС, она продолжает ожидать ввода пароля, но, при очередном подключении к ней, не выводит ничего. Соответственно, нужно быть готовым к тому, что при вводе очередной команды, будет получено сообщение 'INVALID PASSWORD'. В этом случае, придется передать пароль и, по завершении авторизации, повторить команду, вызвавшую сообщение об ошибке
- Второй момент очевиден. Необходимо следить за тем, чтобы не возникало одновременных попыток подключения к одной и той-же АТС. При этом, на различных АТС, активация, разумеется, может выполняться параллельно (в целях улучшения производительности). В совокупности с тем фактом, что в одном активационном задании может быть задействовано несколько АТС, это несколько усложняет архитектуру сервера активации
You should also consider a number of features Alcatel S12. So, in its output there may be unprintable characters (0x05, 0x17), and when entering commands the characters of line feed and carriage return are not used. The command line sent to the Alcatel S12 must end with the character '.' or ';' (the password transmitted during authorization must also end with the symbol ';'). The most significant bit in each byte transmitted to the PBX can be used for parity (this is configured on the PBX). These settings do not apply to text transmitted from the PBX.
Currently, a Telnet-like TCP connection is used to connect to both the Alcatel S12 and the M-200, but this solution is not the only one possible. Other options for connecting to these exchanges are briefly discussed below.
5.1 Serial vs TCP
As I said earlier, at the time of the start of the project, I did not have any experience with the Alcatel S12. The fact that with this type of automatic telephone exchange (in any case, in the configuration used by the Customer) will have to work on a Serial connection, from the very beginning, was considered as one of the most significant risks of the project. This may seem strange, but at the same time I was right and wrong.
Of course, when using libraries such as RXTX or jSSC (which I learned from an article on Habré), connecting to the Serial port from Java SE (used in the project) by itself is not a problem, but there are two points to consider:
- В настоящее время, уже довольно сложно найти материнскую плату с честным Serial-портом. В то же время, различные USB-Serial конвертеры (как правило китайские) проявили себя не лучшим образом. Мы сожгли парочку при попытке подключения к АТС, перед тем как поняли, что ими лучше не пользоваться. Кроме того, часто такой виртуальный Serial-порт вел себя не совсем так как «железный»
- То что управляющий компьютер должен находиться на расстоянии не превышающем максимальную длину RS-232 кабеля от АТС, является серьезным ограничением. Разумеется, нам хотелось управлять всеми АТС с одного сервера (особенно учитывая тот факт, что активационное задание могло обращаться к нескольким различным АТС), а АТС находились в разных городах!
The solution was found in the process of communication with the Customer, who had more experience in interacting with the PBX. Of course, the telephone operators also wanted to manage all the PBXs from one place and they had been successfully doing this for a long time, using the MOXA communication equipment , which allowed them to “flip through” the Serial connection via Intranet. Unfortunately, there were problems trying to connect to the virtual port created by MOXA on the client side .
We spent quite a bit of time trying to solve this problem until we found out that some MOXA models allow you to use a TCP connection. This approach was used. In our work, the Serial-port monitor and sniffer helped us a lot .
5.2 Tune vs TCP
There were fewer problems connecting to the M-200, as it initially supported a TCP connection. It was originally planned to use direct access to the "subscriber card". I will not clutter up the article with dumps of TCP sessions, I will only say that the settings of subscribers in the M-200 are stored in a bitmap. The PBX control protocol allows (using TCP) to read this entire bitmap and write it back, after making the necessary changes.
We managed to decipher some of the fields of the “subscriber card” (we could not find the documentation), but in the end, it was decided to use the tune server as a more documented method for managing the PBX. In addition, working in the input mode of text commands was more convenient.
Conclusion
In this article I tried to describe (and somehow classify) the difficulties that arose in a real activation project. Despite the pronounced focus on traditional telephony equipment, the material can be useful when working on automation of control of any equipment (for example, cisco switches). Hope this article is helpful to someone.
