Direct work with the MTS SMS service: the history of one integration

Many people believe that SMS is one of the reasons that Twitter shot so well in the West. In the United States and several European countries, tweets can be either published or received via SMS, while the price of an outgoing tweet is equal to the price of an ordinary SMS, and incoming messages are free. For users without smartphones (of which most), this significantly reduces the threshold for entry.

In this article I will share the experience of direct integration with a major Russian mobile operator (note: it is directly, not through gateways), as well as at the introductory level I will discuss about SMS technologies and SMPP protocol - without boring tables and specifications, in the style of short detective story.

Plot plot

Once we ( Rutvit ) made an offer to the largest telecom operator - MTS. We understood: in order to make the price of a tweet indistinguishable from the price of a regular SMS, we need direct integration with OPSOS (by the way, this word stands for “Cellular Operator”). MTS is a huge company focused on cooperation with large partners, so I thought that our proposal would not be considered soon, and I didn’t particularly count on success. What was our joy when MTS not only quickly answered, but also as a result of constructive negotiations decided to launch a joint project with Ruthwit.

We all know that large companies are often slow and slow, but this does not seem to apply to MTS. The working group that we contacted during the integration process responded quickly to our questions, and we moved forward. This was especially valuable in that the direct integration with the mobile operator turned out to be an order of magnitude more complicated than connecting some popular SMS gateway: I’ll also talk about technical details near the end of the article.

And so we first launched the alpha version of the SMS service on Rutvit (read the joint release with MTS press releaseon the PRIME-TASS website), and today we are replacing the alpha icon in this service with a beta, which we report in this article. The service, among other things, allows you to send tweets via SMS at the usual cost (according to the subscriber’s tariff plan) not only to Root Tweet, but also to large Twitter through the export mechanism.

So, you can send tweets via SMS for exactly the same money for which you send any other SMS. For the same purpose, we decided to use not a short number, but a more familiar, long one: +7 916 140-0-140 . People often do not trust short numbers, because not sure how much money will be withdrawn from them for SMS: with a long number there is no such problem as if.

How to start tweeting via SMS?

To start writing tweets via SMS, enter your phone number on the Rootwith website:



Then send the word “yes” (the register is not important) to the Rootwith number +7 916 140-0-140 . As soon as the robot sees this message, it will attach your number to your account, and all subsequent SMS to +7 916 140-0-140 will be perceived as tweets on your behalf.

And you can write SMS at cost to Twitter, FriendFeed, Facebook, LiveJournal, Buzz

If you are a fan of Twitter, configure the export of messages Rutvit -> Twitter (or FriendFeed, Facebook, LiveJournal, Buzz) and write on Twitter (FriendFeed, Facebook, LiveJournal, Buzz) SMS at cost using Rutvit as a gateway:



(For fairness, it is worth noting that you can send SMS via foreign gateways to Twitter, but in Russia it turns out to be much more expensive: after all, an SMS to the British number + 44xxxx provided by Twitter is more expensive than an SMS to the Russian Rutwith number +7 916 140-0-140.)

Now a fly in the ointment, present in beta. One of our goals is to make sure that the user can not only send SMS to Rutvit at the price of a regular SMS, but also receive tweets via SMS on a mobile phone for free. Some microblogging services simply buy a GSM modem (or a smartphone with Linux) and receive / send SMS through it, however this method works only on small amounts of traffic: it does not scale well. Nevertheless, we want to save users of social networks from additional costs when activating and using this service. But only technical integration and the conclusion of a contract directly with the operator will allow this to be achieved. In the meantime, negotiations are ongoing, the beta version does not implement the ability to receive SMS on the phone: you can only send tweets to the system. In this matter we are betting on the future.

Direct Integration Technical Details

We have never worked with SMS and SMPP before . And, when we signed an agreement with MTS, we were faced with the choice:

• either make the SMPP client manually (using modules for Perl, for example);
• Or use a ready-made SMPP server on an industrial scale (the most popular of them is Kannel ).

As a result, it was decided to stop on the second option, and we did not regret it, because during the integration process we stepped on a lot of underwater rakes, many of which were successfully resolved by Kannel.

A little bit about SMPP, SMS messaging protocol

Minutes of the SMPP (stands for «Short Message Peer to Peer») itself is very complicated. It is very low level (specifications occupy 170 pages). The difficulty is mainly due to historical reasons. A group of protocols for voice transmission was developed in 1975. In those days, there were no SMS (there weren’t even LCD displays for cell phones, as were the mobile phones themselves). And so, in 1985, they decided to “push” support for text messages into standard voice transmission protocol packages. But there was little useful space in these packages, so the messages turned out to be short, only 160 characters - Short Messages.

The useful amount of data that can be transmitted in an SMPP packet is 140 bytes. This is not a typo. How did they fit 160 characters? You probably already guessed: 1 ASCII character can be encoded with 7 bits. Thus, 160 * 7 = 1120 bits, or 140 bytes, are obtained.

If we lived in the USA, where ASCII encoding “should be enough for everyone”, then we would quietly write tweets of 160 characters. However, when sending Russian letters by SMS, they are packed in the UTF-16 encoding, i.e. 2 bytes per 1 Russian letter. Therefore, 1 message can fit no more than 140/2 = 70 characters of the Russian alphabet. If the text is longer, it is packed in 2 or more SMS messages, which are then “glued” when displayed on the phone.

But that is not all. In 2 SMS messages it is possible to deliver not 70 * 2 = 140 Russian letters at all, but only 134. An additional place is required for encoding service information, in particular, information about the number of SMS in the chain and the number of the current SMS in it.

Kannel companions

The Kannel server we used actually has a wider scope. It is intended for content provider, when the request is SMS, and the answer is some SMS content: melodies, pictures, etc. We use only the part that organizes the reception and broadcast of SMS: Kannel receives an SMPP message, unpacks / decrypts it and sends us a phone number and text with an HTTP request. In response, we do not send any content.

True, there were some surprises. MTS has a very meticulous testing scheme: we not only have to correctly receive messages, but also fit into all kinds of timeouts when queuing requests, support throttling, and send delivery confirmation messages in time with explosive load growth. At the test bench, we encountered a problem, for the solution of which we had to patch Kannel a little (it is written in C). Briefly, it can be described as follows: suppose Kannel received 2 SMPP packets, and the third came up against the bandwidth limit set by MTS, and therefore got stuck in the queue. If now the restriction has been removed, but the 4th message has immediately arrived, then in the standard version, Kannel will put this 4th message in front of all previously stuck ones. Therefore, the 3rd message will again be at the end of the queue and will move further and further as new SMS arrives. It turns out that for MTS load tests this behavior matters - like many other aspects that had to be taken into account during direct integration.

We did not have experience with SMPP before, but Kannel allowed us to get out. Fortunately for us, Kannel, even in its basic configuration, processes a lot of what OPSOS needed. But, honestly, now that we’ve already "eaten the dog" at SMPP, we would do everything using Perl and modules with CPAN, not Kannel. But this would be possible only thanks to the experience that we got with Kannel. Without this experience, programming direct integration with OPSOS on Perl is like teaching a person to swim by throwing him into the water from a cliff: maybe he will learn, or maybe he will drown. MTS is especially meticulous when testing error handling, withstanding a variety of timeouts. For example, the time for sending * _resp confirmations to SMPP messages is clearly regulated: delays should fit into the time intervals specified by the MTS.