RabbitMQ tutorial 1 - Hello World

RabbitMQ allows various programs to communicate using the AMQP protocol. RabbitMQ is an excellent solution for building SOA (service-oriented architecture) and the distribution of pending resource-intensive tasks.

Under the cut is the translation of the first of six lessons from the official site. Examples are in python, but knowing it is not necessary at all. Similar programs can be reproduced on almost any popular YP. [comments of the translator look like this, i.e. me]

Introduction

RabbitMQ is a message broker. Its main purpose is to receive and give messages. It can be imagined as a post office: when you drop a letter in a box, you can be sure that sooner or later the postman will deliver it to the addressee [apparently, the author has never dealt with the Russian Post] . In this analogy, RabbitMQ is both a mailbox, and a post office, and a postman.

The biggest difference between RabbitMQ and the post office is that it does not deal with paper envelopes - RabbitMQ receives, stores and sends binary data - messages.

In RabbitMQ, as well as messaging in general, the following terminology is used:

• Producer (supplier) - a program that sends messages. In the diagrams it will be represented by a circle with the letter "P":
  
  
• Queue - The name of the mailbox. It exists inside RabbitMQ. Although messages go through RabbitMQ and applications, they are only stored in queues. The queue has no restrictions on the number of messages; it can accept an arbitrarily large number of messages — it can be considered an infinite buffer. Any number of suppliers can send messages in one queue, and any number of subscribers can receive messages from one queue. In schemes, the queue will be indicated by the stack and signed by the name:
  
  
  
• Consumer (Subscriber) - a program that accepts messages. Typically, the subscriber is in a waiting state for messages. In the diagrams it will be represented by a circle with the letter “C”:
  
  
  

The supplier, subscriber and broker are not required to be on the same physical machine, usually they are on different.

Hello World!

The first example will not be particularly difficult - let's just send a message, accept it and display it. To do this, we need two programs: one will send messages, the other will receive and display them on the screen.
The general scheme is as follows: The



provider sends messages to the queue with the name "hello", and the subscriber receives messages from this queue.

RabbitMQ Library

RabbitMQ uses the AMQP protocol. To use RabbitMQ, you need a library that supports this protocol. Such libraries can be found for almost every programming language. Python is no exception; there are several libraries for it:

• py-amqplib
• txAMQP
• pika

In the examples, the pika library will be used. It can be installed using the pip package manager:

$ sudo pip install pika==0.9.5

If pip or git-core are missing, then you must first install them:

• For Ubuntu:
  
  

  $ sudo apt-get install python-pip git-core
  

  
  
• For Debian:
  
  

  $ sudo apt-get install python-setuptools git-core
  $ sudo easy_install pip
  

  
  
• For Windows (to install easy_install, you must run MS Windows Installer for setuptools):
  
  

  > easy_install pip
  > pip install pika==0.9.5
  

  
  

Sending messages

Our first send.py program will simply send one message to the queue.

#!/usr/bin/env python
import pika
connection = pika.BlockingConnection(pika.ConnectionParameters(
               'localhost'))
channel = connection.channel()

We are connected to a message broker located on the local host. To connect to a broker located on another machine, it is enough to replace “localhost” with the IP address of this machine.

Before sending a message, we must make sure that the queue receiving the message exists. If you send a message to a nonexistent queue, RabbitMQ will ignore it. Let's create a queue to which a message will be sent, call it “hello”:

channel.queue_declare(queue='hello')

Now everything is ready to send a message. Our first message will contain the string “Hello World!” And will be sent to the queue with the name “hello”.

In general, in RabbitMQ messages are not sent directly to the queue, they must go through exchange ( exchange point). But now we will not focus on this point of exchange will be discussed in the third lesson. Now it’s enough to know that the default exchange point can be determined by specifying an empty string. This is a special point of exchange - it allows you to determine which queue the message is sent to. The queue name must be specified in the routing_key parameter :

channel.basic_publish(exchange='',
                      routing_key='hello',
                      body='Hello World!')
print " [x] Sent 'Hello World!'"

Before exiting the program, make sure that the buffer has been cleared and the message has reached RabbitMQ. You can be sure of this if you use secure closing the connection with the broker.

connection.close()

Receiving Messages

Our second receive.py program will receive messages from the queue and display them on the screen.

As in the first program, you must first connect to RabbitMQ. To do this, use the same code as before. The next step, as before, is to make sure that the queue exists. The queue_declare command will not create a new queue if it already exists, so no matter how many times this command is called, only one queue will be created anyway.

channel.queue_declare(queue='hello')

You may wonder why we announce the queue again, because it has already been announced in the first program. This is necessary to ensure that the queue exists, so it will be if send.py is run first . But we do not know which program will be launched earlier. In such cases, it is better to queue in both programs.

Queue Monitoring

If you want to see what queues exist in RabbitMQ at the moment, you can do this with the rabbitmqctl command (superuser rights will be required):

$ sudo rabbitmqctl list_queues
Listing queues ...
hello    0
...done.

(for Windows - without sudo)

[our company uses a more convenient monitoring script:]

watch 'sudo /usr/sbin/rabbitmqctl list_queues name messages_unacknowledged messages_ready messages durable auto_delete consumers | grep -v "\.\.\." | sort | column -t;'

[the script displays and updates every 2 seconds a table with a list of queues: the name of the queue; number of messages in processing; number of messages ready for processing; total number of messages; the stability of the queue to restart the service whether it is a temporary queue; number of subscribers]

Receiving messages from the queue is a more complicated process than sending. Receiving is done using a subscription using the callback function. Upon receipt of each message, the Pika library calls this callback function. In our example, it will display the message text.

def callback(ch, method, properties, body):
    print " [x] Received %r" % (body,)

Next, we need to indicate that the callback function will receive messages from the queue with the name "hello":

channel.basic_consume(callback,
                      queue='hello',
                      no_ack=True)

Here you need to be sure that the queue we want to subscribe to has been announced. We did this earlier with the queue_declare command .

The no_ack parameter will be discussed later [in the second lesson ] .
And finally, start an endless process that waits for messages from the queue and calls the callback function when necessary.

print ' [*] Waiting for messages. To exit press CTRL+C'
channel.start_consuming()

Well, now all together

Full send.py code :

#!/usr/bin/env python
import pika
connection = pika.BlockingConnection(pika.ConnectionParameters(
        host='localhost'))
channel = connection.channel()
channel.queue_declare(queue='hello')
channel.basic_publish(exchange='',
                      routing_key='hello',
                      body='Hello World!')
print " [x] Sent 'Hello World!'"
connection.close()

Full receive.py code :

#!/usr/bin/env python
import pika
connection = pika.BlockingConnection(pika.ConnectionParameters(
        host='localhost'))
channel = connection.channel()
channel.queue_declare(queue='hello')
print ' [*] Waiting for messages. To exit press CTRL+C'
def callback(ch, method, properties, body):
    print " [x] Received %r" % (body,)
channel.basic_consume(callback,
                      queue='hello',
                      no_ack=True)
channel.start_consuming()

Now we can try to run our programs in the terminal. First, send a message using the send.py program :

$ python send.py
[x] Sent 'Hello World!'

This program will end after each message has been sent. Now the message needs to be received:

$ python receive.py
[*] Waiting for messages. To exit press CTRL+C
[x] Received 'Hello World!'

Excellent! We sent our first message via RabbitMQ. As you may have noticed, the execution of the receive.py program did not end. She will expect the following messages, and you can stop it by pressing the Ctrl + the C .
Try running send.py again in a new terminal window.

We learned how to send and receive messages through named queues. In the next lesson, we will create a simple [resource-intensive] task queue .

UPD: a library working with RabbitMQ, for your favorite PL you can find on the official website here .