Future OpenSIPS Design

Foreword

OpenSIPS is a signaling SIP switch. If you want to handle a lot of SIP calls, then you probably won't get past OpenSIPS.
The system is really “mature”, tested in battle and, over time, overgrown with many useful (and not so) modules.

At the same time, it is obvious that the architecture laid down in 2001 does not meet modern requirements.
Therefore, OpenSIPS developers said that version 2.0 will be "from scratch."

The following is a translation of the OpenSIPS 2.0 Design document . I wonder what the habrasociety thinks about it.

Essential comments I will try to convey to the developers.

Why do we need a new architecture

The current OpenSIPS architecture (prior to version 2.0) is based on concepts that are more than 7 years old. At that time, the requirements were simple (simple stateless SIP proxy, only UDP) and decisions were made in accordance with these requirements. But with all the additions, both in SIP and functionality (such as TCP / TLS, script manipulation, dialog support, integration with external systems, etc.), the existing architecture can no longer satisfy the requirements and real use cases.

Attention! Inside is a large and structured text with pictures.


Problems to be solved by the new architecture:

• I / O locks (transport, DB, applications)
• Scaling with hardware resources (I / O and synchronization of parallel processes are a bottleneck in the current architecture)
• When designing a configuration, you have to deal with low-level functions (TM, dialogs, NAT) instead of focusing on the logic of service provision.
• Horizontal scaling (both core and routing logic)
• The routing mechanism (in the form of a specialized programming language) has a very limited set of capabilities not related to SIP (integration with other applications, complex routing script logic, working with arrays and lists, support for complex operations and data types) and requires additional skills from the administrator.
• The routing mechanism is too closely connected with the SIP stack, this leads to the fact that it cannot be changed on the go without restarting OpenSIPS.
• It is not possible to do distributed routing between multiple systems (to improve scalability).

Overview of New Architecture

The new architecture was developed taking into account the problems and successful solutions in the current architecture, and also takes into account the development directions of SIP and OpenSIPS.
At the top level, OpenSIPS will consist of two completely independent parts:

• SIP-core (SIP-core) - provides support for low-level operations with SIP
• Routing Engine - responsible for the high-level routing logic

It is assumed that several routing servers will be able to connect to one Core to implement different functionality, or simply to increase the capacity of the system as a whole.

SIP core

The kernel is a low-level component that provides functions related to SIP. They can be executed automatically and do not require complex configuration. Core will be responsible for the transport layer, packet analysis, transactions and dialogs, for automatic support for NAT access, SIP registrations and online statuses (presentations), functions that are clearly defined in the RFC and do not require any intervention from the high-level Subsystem routing.

The core is divided into several horizontal levels, each of which performs certain functions:

• L ₀ - Transport layer. Implements support for UDP, TCP, TLS, SCTP and hides all the specifics for transport
• L ₁ - SIP analyzer level (SIP parser layer). Responsible for parsing SIP messages
• L ₂ - Transaction layer - implements support for SIP transactions. Each incoming SIP message is mapped to a SIP transaction.
• L ₃ - Dialogue Level - implements support for SIP conversations. SIP message can be mapped to SIP conversation
• L ₄ - Level of passage through NAT - automatic processor for passing through NAT (support at the level of SIP signaling and transport)
• L ₅ - Level of support for online statuses (present) - implements the SIP Presence Agent. Automatic processing of messages related to online statuses.
• L ₆ and higher ... other levels (such as user location) may be added as necessary.
• L _p - the last level - this is the level that provides interaction with the Routing Engine (via API or socket)

Core structure

SIP message goes to the lower level L ₀ . Each level processes messages and either transfers it to the next level, or returns it to a level lower. If a return occurs, this indicates that the message processing is finished at this level (for example, this is how automatic replies to KeepAlive messages or resending packets work). When the message reaches the layer that provides interaction with the routing server, the message is transmitted (with all the changes that were made at each level) to the routing server, which continues to further process the message. Then the message returns back along the path that goes from top to bottom from level to level, until it reaches the transport level (L ₀ ) and is sent to the network.

The kernel has two message flows:

• Incoming stream (from bottom to top) - the incoming message is transmitted upward, subject to changes at each level, until it reaches a place where a decision can be made about what to do with the message (this can be either in one of the levels of the Kernel or in the Routing Subsystem )
• Outgoing flow (from top to bottom) - when the decision on what to do with the message is made, the message is returned back through all the layers that it went through until it was sent to the network.

In both streams, each level can perform different actions (for example, the SIP analyzer level parses messages from the incoming stream and collects them back, placing them in the outgoing stream).

SIP messages are read from network to network at level L ₀(transport level), and then they are passed on to higher levels. Each layer performs the appropriate actions: for example, the SIP analyzer level will add the data in a parsed format to the message, the transaction level will add the transaction identifier, the dialogue level will add the dialogue identifier, etc. After completing its task, the level can: (a) transfer the message to a higher level for processing (if the current level cannot complete the processing) or (b) decide what to do with the message and transfer the message to the outgoing stream for sending (in this case , all higher levels will be skipped).

Such an algorithm provides the most efficient message processing at each level. There is no need to transmit all messages through all levels to the Routing Subsystem, if the message can be processed automatically using the lower levels. For example, responses to keepalive messages can be automatically processed by the Kernel at the L4 level (pass through NAT). Another example is when the Routing Subsystem is only interested in receiving and processing initial requests. In this case, subsequent requests will be automatically processed and routed using the dialog level without the need to transfer them further down the chain.

In addition to tiers, the Core also implements several database backends. They are used to save between restarts of internal data that each of the levels of the Kernel stores in memory (dialogs, online statuses, registrations, information for passing through NAT).

Inside the Core, levels can be divided by importance:

• Base core - includes mandatory levels (L ₀ -L ₃ and L _n ). They provide the features that are always required.
• Additional levels (L ₄ - L _n-1 ) that provide optional functions. They can connect or disconnect during operation.

The kernel has its own configuration file, which contains parameters for the main levels and backends of the databases: on which network interfaces to listen, transaction parameters, dialog parameters, etc.

The kernel is implemented as an asynchronous reactor that supports non-blocking I / O. To effectively use resources on machines with multi-core processors, the kernel uses several threads (according to the number of CPU cores).

Routing Engine

The routing subsystem provides routing, for which the configuration script is responsible for the current version of OpenSIPS and provides the functionality of most of the modules that currently exist.

The routing subsystem runs on top of the Kernel as a separate component. As mentioned earlier, several Routing Subsystems can be connected to one Core - either to implement various services, or to increase the throughput of the system by distributing the load on several physical machines.

The new design allows using two approaches for connecting the Kernel to the Routing Subsystem:

• Routing logic in the form of a library that is closely related to the Kernel, and implements several additional levels and modules on top of the Kernel. They are interconnected within one executable file.
• Routing logic as an external application that communicates with the Kernel using a socket. This application can be written in a high-level language such as Python or Java.

Why use two approaches? These two options are not mutually exclusive, but complement each other - the internal Routing Server is more efficient in terms of performance and manageability (due to close integration with the Kernel directly at the C level), while the external Routing Subsystem will be a much more universal and easy to implement solution (being high-level language application) that is much easier to control.

Internal Routing Module

A routing module is a set of additional levels, presented in the form of a library, that will be associated with the Kernel, forming a single application. The routing module includes:

• Script interpreter - for implementing logic
• Additional modules - to support ready-made functionality in the script

Internal Routing Subsystem

The script interpreter is based on the existing specialized language with the ability to directly insert blocks in a high-level language (for example, embedded Perl). Depending on the degree of impact on performance, you can completely replace a specialized script with a high-level language (for example, write the entire routing module logic in Perl from which the kernel functions or C modules are called).

In the new architecture, it remains possible to use the existing native script, because its interpreter is very fast, and it is perfect for configurations with simple logic and copes well with high loads.

Embedding scripts in high-level languages ​​also looks useful, since this solution greatly simplifies the implementation of scripts where complex routing logic is required, which can be implemented using the capabilities of a high-level language. In addition to reducing the complexity of writing complex scripts, this feature eliminates the need to learn a specialized built-in scripting language. However, in this case, some performance degradation will occur, as high-level language interpreters are heavier and slower. On the other hand, the use of high-level languages ​​will achieve horizontal scaling. Imagine an OpenSIPS cluster (on multiple servers),

Modules of the Routing Subsystem that provide predefined functions (dialplan, LCR, QoS, B2BUA etc.) can be used from a script, regardless of its format.

It is possible to reload the script during operation.

External Routing Subsystem

As well as the built-in, external Routing Subsystem (or application) is implemented as a set of levels through which messages pass in two directions (bottom to top and top to bottom). These layers end with a layer that implements routing logic for a particular SIP service. The last layer is the equivalent of the script used in the old architecture.

Some of the levels are active - this means that they change messages during transmission and, possibly, decide to stop relaying messages up, end processing and return the message down. This is done either by analyzing the contents of the message, or based on instructions from the previous routing level. Some other levels are passive. This means that they only provide classes and functions that implement certain capabilities (for example, LCR or call control), but they are not used for end-to-end messaging between levels up. They are called explicitly from the routing logic and modify the message.

Routing Subsystem as an Application

The application is registered in a specific Core (or on several Core, if necessary) and reports on its capabilities to the Core. Features include messages that this Routing Subsystem and restrictions are interested in. This allows the Kernel to filter which messages and how much to transmit to this application. In addition, the application may request the Kernel to dynamically enable / disable some of the optional levels of the Kernel when the Kernel decides that a message should be routed to this application. This allows you to create flexible configurations in which a particular application can, for example, tell the Kernel that for all messages that are transmitted to a specific application, disable the level of passage through NAT. As a result, when the Kernel transmits a message to this application, it will skip the pass level through NAT. This means that the application wants to deal with passing through NAT on its own. It also means that the application will receive keepalive messages. Other applications, if they have not disabled this level, will not see keepalive messages and will not solve the problem of passing through NAT, because the level of passage through NAT will be used. This allows for a very flexible configuration, where some of the optional Core levels can be turned on or off at run time without the need to change the Core configuration or restart the Core. because the level through NAT will be used. This allows for a very flexible configuration, where some of the optional Core levels can be turned on or off at run time without the need to change the Core configuration or restart the Core. because the level through NAT will be used. This allows for a very flexible configuration, where some of the optional Core levels can be turned on or off at run time without the need to change the Core configuration or restart the Core.

Multiple single core applications

An entire application can be written in a high-level language such as Python. An implementation in OpenSIPS will provide all the necessary functions that enable the user to write high-level routing logic. This corresponds to the last level in the application diagram. It will be like an existing script, only it will be written in the same language (Python in this case) as the rest of the application. Of course, the application can be implemented in other languages, such as Java, Ruby or Perl.

Each of us can help the OpenSIPS project.

Thanks for attention.