AMQP in Russian

    Today, there is little information about the AMQP (Advanced Message Queuing Protocol) and its application, especially in Russian. In general, this is a wonderful, widely supported open protocol for transferring messages between system components with low latency and high speed. At the same time, the semantics of messaging is adjusted to the needs of a particular project. Such solutions existed before, but this is the first standard for which there are a large number of free implementations.

    The main idea is that individual subsystems (or independent applications) can exchange messages randomly through an AMQP broker that provides routing, possibly guaranteeing delivery, distribution of data streams, and subscribing to the necessary types of messages. Classical examples are usually financial applications related, for example, to the delivery to consumers of real-time information about the exchange rates of securities, RPC interaction of two subsystems that are not connected to each other (interaction via the general AMQP protocol) is also possible further and the like.

    Today, the topic of real-time information delivery is extremely relevant (just recall Twitter, Google Wave). And here, messaging systems can serve as an internal data exchange mechanism that ensures the delivery of data (data changes) to customers.

    I do not set as my goal today to talk about how to write applications for AMQP. I just want to talk a little bit about the fact that this is not at all scary, not very difficult, and really works, although the standard is still in development, new versions of the protocol, brokers, etc. But this is already quite production-quality. I’ll only tell you basic tips to help you “enter" the protocol.

    For starters, a small collection of links (mostly in English): what messaging is all about and why AMQP is (Messaging in general and AMQP design ); Comparison of various messaging implementations, in particular based on AMQP ( Message Queue Comparison ); AMQP client library for Twisted Framework (Python) with Thrift support ( Thrift, AMQP in Twisted ); Red Hat's guide on what messaging is and how to work with AMQP describes their “boxed” product based on AMQP, but is also suitable for any AMQP brokers ( AMQP Programming Tutorial for C ++, Java, and Python ); a lot of documentation, descriptions of architectural solutions on the ZeroMQ website , which is not quite an AMQP broker, but the general architecture, implementation details are of particular interest; Duncan McGregor review article on txAMQP and AMQP in general (A Simfonia on Messaging with txAMQP , II , III ).

    Next, you need to choose the AMQP broker that you will use. When choosing, you need to consider the server’s own characteristics: speed, reliability, ease of installation and support, but also carefully look at the version of the AMQP protocol that is supported by the broker — it must match the version of the client AMQP library. Of the brokers, I would recommend RabbitMQ , written in Erlang, and Qpid , versions in C ++ (AMQP 0-10) and Java (0-8, 0-9).

    The AMQP protocol itself is arranged quite interestingly: at the very lowest level, the format for encoding data in binary form for transmission over a TCP connection is defined, above is the format for transmitting RPC requests between the server and client. The semantics of working with messages, creating queues, etc. described in the XML specification, which essentially defines the RPC interface of the server and client (examples of such XML files for versions 0-8 and 0-10) This XML is the latest and final protocol specification. Moreover, protocol versions 0-8 and 0-10 differ so much that it is hardly possible to support them at the same time in one program. More interestingly, sometimes such spec files for different AMQP brokers formally supporting the same protocol version differ so much that they are not interchangeable. But these are rather small technical problems.

    AMQP is based on three concepts:
    1. Message (message) - a unit of transmitted data, its main part (content) is not interpreted by the server, structured headers can be attached to the message.
    2. Exchange point - messages are sent to it. An exchange point distributes messages into one or more queues. At the same time, messages are not stored at the exchange point. Exchange points are of three types: fanout - the message is transmitted to all queues attached to it; direct - the message is sent to the queue with a name that matches the routing key (routing key is specified when sending the message); topic is a cross between fanout and exchange, a message is sent in a queue for which the mask for the routing key matches, for example, app.notification.sms.*all messages sent with keys starting with will be delivered to the queue app.notification.sms.
    3. Queue - messages are stored here until it is picked up by the client. The client always picks up messages from one or more queues.

    I was attracted to AMQP by an effective binary protocol, the protocol's focus on minimum delays and configuration flexibility. I used it to send messages to all cluster servers (fanout exchange) and organize analogues of “mailboxes” to which messages addressed to specific clients (direct exchange) are delivered. To receive messages there is no need to interrogate the server, it is enough to subscribe to messages from the queue, and the server will send them the moment they appear.

    As the client library, I chose the txAMQP library for the Twisted Framework (Python). In general, everything works, but somewhere small “tweaks” and “twists” are required, which I plan to publish on launchpad. There are a lot of interesting and promising things in and around AMQP. For example, the RabbitMQ broker can scale and work in a single cluster. It seems to me that this is a very useful and promising technology.

    Also popular now: