Telegram bot API limits and working with them on Go

Quite often, articles about writing a bot for Telegram appear on Habré, which, in its own way, if you uniqueness of the idea, are the most common tutorial on the topic "how to receive a message from Telegram, process it and send a response to the user." However, in none of the articles I read (of course, I can’t pretend to have read all of them, but nonetheless) I did not find mention of limits on sending messages to users and how to work with them. Who interested, I ask under the cat.

Some time ago, I sat down to develop a text-based multi-player strategy based on the Telegram bot API, and a month later I launched the first release with meager, but playable functionality. According to the chain of acquaintances, the game quickly gained a small actively playing audience and continued to gain in the following days thanks to the in-game referral program. And everything seems to be fine, until the daily online has exceeded the mark of two hundred users. This is where the problems started. Increasingly, users turned to me asking why the bot did not respond for several minutes. The players were most discomforted, especially during wars, when the user tried to quickly restore the army for a counterattack, and the game treacherously hung and did not respond to any actions. Moreover, Telegram could be banned as sending all messages,

We already had experience with the bot API, however, at a smaller audience and with a lower sending intensity. It was also known about the limits, but really came across them only when working with groups. Everything is much tougher there than when working with personal chats. To learn more about the limits, just refer to the FAQ on the official Telegram website.


From the above, we have that it is impossible to send messages to a specific user more than once per second and no more than 30 messages per second when mass mailing to different users. But some errors are allowed. Therefore, we need to send a message to the user every 1/30 seconds, checking whether we have already sent him a message within the current second, otherwise send a message to the next user if he passed the same test.

Since the development was initially conducted in the Go language, where there are channels and coroutines (they are goroutines), the idea of ​​sending pending messages came to mind immediately. First, we add the processed response to the channel, and in a separate stream we rake this channel every 1/30 second that is allowed to us. But the idea with one channel for all messages did not work. Having got the message from the channel and making sure that we cannot send messages to this user yet, we need to get this message somewhere. Sending again to the same channel is not good, because we will break the chronological order of the user's messages, and also strongly delay the delivery of this message with a large number of active players. We can’t check the message without taking it out of the channel and move on to the next, as far as I know.

Then the idea of ​​introducing the channel to the user appears. From this moment in more detail.

// Мап каналов для сообщений, где ключом является id пользователя
var deferredMessages = make(map[int]chan deferredMessage)
// Здесь будем хранить время последней отправки сообщения для каждого пользователя
var lastMessageTimes = make(map[int]int64)
// chatId – id пользователя, которому шлем сообщения
// method, params, photo – заранее подготовленные параметры для запроса согласно bot API Telegram
// callback будем вызывать для обработки ошибок при обращении к API
type deferredMessage struct {
	chatId		int
	method 		string
	params 		map[string]string
	photo 		string
	callback 	func (SendError)
}
// Метод для отправки отложенного сообщения
func MakeRequestDeferred(chatId int, method string, params map[string]string, photo string, callback func (SendError)) {
	dm := deferredMessage{
		chatId: 	chatId,
		method: 	method,
		params: 	params,
		photo: 		photo,
		callback: 	callback,
	}
	if _, ok := deferredMessages[chatId]; !ok {
		deferredMessages[chatId] = make(chan deferredMessage, 1000)
	}
	deferredMessages[chatId] <- dm
}
// error.go, где ChatId – id пользователя
type SendError struct {
	ChatId 	int
	Msg	string
}
// Имплементация интерфейса error
func (e *SendError) Error() string {
	return e.Msg
}

Now, on the go, I’d like to use the select case design to process the resulting set of channels, but the problem is that it describes a fixed set of channels for each case, and in our case the set of channels is dynamic, as users are added during the game, creating new channels for their messages. Otherwise, you can not do without locks. Then, turning to Google, as usual, in the vastness of StackOverflow there was an excellent solution. And it consists in using the Select function from the reflect package .

In short, this function allows us to extract from a pre-formed array of SelectCases, each of which contains a channel, a message ready for sending. The principle is the same as in the select case, but with an indefinite number of channels. That is what we need.

func (c *Client) sendDeferredMessages() {
        // Создаем тикер с периодичностью 1/30 секунд
	timer := time.NewTicker(time.Second / 30)
	for range timer.C {
                // Формируем массив SelectCase'ов из каналов, пользователи которых готовы получить следующее сообщение
		cases := []reflect.SelectCase{}
		for userId, ch := range deferredMessages {
			if userCanReceiveMessage(userId) && len(ch) > 0 {
                                // Формирование case
				cs := reflect.SelectCase{Dir: reflect.SelectRecv, Chan: reflect.ValueOf(ch)}
				cases = append(cases, cs)
			}
		}
		if len(cases) > 0 {
                        // Достаем одно сообщение из всех каналов
			_, value, ok := reflect.Select(cases)
			if ok {
				dm := value.Interface().(deferredMessage)
                                // Выполняем запрос к API
				_, err := c.makeRequest(dm.method, dm.params, dm.photo)
				if err != nil {
					dm.callback(SendError{ChatId: dm.chatId, Msg: err.Error()})
				}
                                // Записываем пользователю время последней отправки сообщения.
				lastMessageTimes[dm.chatId] = time.Now().UnixNano()
			}
		}
	}
}
// Проверка может ли уже пользователь получить следующее сообщение
func userCanReceiveMessage(userId int) bool {
	t, ok := lastMessageTimes[userId]
	return !ok || t + int64(time.Second) <= time.Now().UnixNano()
}

Now in order.

• First, we create a timer that will “tick” every 1/30 second we need, and start the for loop on it.
  
  
• After that, we begin to form the array of SelectCases we need, sorting through our map channels, and adding to the array only those non-empty channels whose users can already receive messages, that is, one second has passed since the last sending.
  
  
• We create a reflect.SelectCase structure for each channel in which we need to fill in two fields: Dir - direction (sending to the channel or extraction from the channel), in our case we set the flag reflect.SelectRecv (extraction) and Chan - the channel itself.
  
  
• Having finished forming the array of SelectCases, we give it to reflect.Select () and we get the channel id in the array of SelectCases, the value extracted from the channel and the flag of the successful operation. If all is well, make an API request and get a response. Having received the error, call callback and pass the error there. Do not forget to write to the user the date of the last message sent

So, everything seems to be simple. Now Telegram will not pick on our bot due to the frequent sending of messages to the user, and players will be comfortable playing. Of course, it is clear that with a huge number of users, messages will be sent to the player more slowly and slowly, but this will be done evenly, creating less inconvenience than with single locks for several minutes, if you do not follow the limits.

By the way, we recall the errors specified in the FAQ. In my implementation, I send users two messages per second instead of one and not once in 1/30 second, but once in 1/40, which is much more often than recommended. But so far no problems have arisen.

The source code of the client can be viewed on gitlab.

Well, if someone was interested in what it was about, then in Telegram @ BastionSiegeBot