Comparison of Azure Queues and Service Bus Queues
- Transfer
- Tutorial
Microsoft Azure supports two types of queue mechanisms: Azure Queues and Service Bus Queues .
Azure Queues , is part of the Azure storage infrastructure , providing a simple REST-oriented Get / Put / Peek interface for reliable, guaranteed message delivery within a service or between several services.
Service Bus Queues is part of the Azure messaging infrastructure that deals with queues, publishing / subscribing, web services and integration patterns (more about this in the Overview of Service Bus Messaging Patterns ).
Services work in parallel, Azure Queues appeared earlier as a separate queue mechanism built on top of Azure storage services.
Service Bus Queues are built on top of the “message broker” infrastructure designed to connect applications or application components that may require support for various protocols, work in different domains or network environments.
This article compares these two queuing systems, focusing on the differences in their behavior and implementation. The article also gives tips on choosing the type of queue that is best suited to your application.
Both Azure Queues and Service Bus Queues are queuing services on the Microsoft Azure platform. They have a slightly different set of features, so you can use one of them, or the other, or both together - depending on what your application needs. The following are recommendations for selection.
So you should choose Azure Queues if:
You should choose Service Bus Queues if:
The tables in the following sections are divided into logical groups, each of which compares Azure Queues and Service Bus Queues for some parameters.
Prices are subject to change. Check out the Pricing Overview page .
The decision to choose between Azure Queues and Service Bus Queues depends on many factors, the needs of your application and its architecture. If you already use other Azure services, such as tables or blobs, you will just start using Azure Queues, especially if you only need basic scenarios for using queues. On the other hand, Service Bus Queues offers a number of additional features, such as sessions, transactions, duplicate detection, automatic processing of "dead" messages, which may be necessary for your application.
Azure Queues , is part of the Azure storage infrastructure , providing a simple REST-oriented Get / Put / Peek interface for reliable, guaranteed message delivery within a service or between several services.
Service Bus Queues is part of the Azure messaging infrastructure that deals with queues, publishing / subscribing, web services and integration patterns (more about this in the Overview of Service Bus Messaging Patterns ).
Services work in parallel, Azure Queues appeared earlier as a separate queue mechanism built on top of Azure storage services.
Service Bus Queues are built on top of the “message broker” infrastructure designed to connect applications or application components that may require support for various protocols, work in different domains or network environments.
This article compares these two queuing systems, focusing on the differences in their behavior and implementation. The article also gives tips on choosing the type of queue that is best suited to your application.
Technology Considerations
Both Azure Queues and Service Bus Queues are queuing services on the Microsoft Azure platform. They have a slightly different set of features, so you can use one of them, or the other, or both together - depending on what your application needs. The following are recommendations for selection.
So you should choose Azure Queues if:
- Your application needs to store more than 80 GB of data in one queue, with each message lasting no more than 7 days.
- Your application needs to monitor the status of message processing in the queue. For example, you want to be aware of the fall of the worker who was processing the message, and when this event occurs, transfer further processing to another worker.
- You need logs of all transactions in your queue.
You should choose Service Bus Queues if:
- You need to be able to receive messages without constant “polling”. Service Bus Queues supports the TCP protocol, which allows you to quickly receive information about new messages.
- You need to guarantee the preservation of the message processing order (FIFO).
- You have experience with Service Bus for Windows Server and want to use it when working with queues in Azure.
- You want to automatically detect duplicate messages.
- You want to process messages in your application in parallel in the so-called “streams” (messages are associated with a stream by SessionId ). In this model, each node of the application receiving the messages is responsible for processing one thread. When a stream is submitted for processing to a node, the latter can work with the stream using transactions.
- Your application requires transactionality when sending or receiving a group of messages.
- Message Lifetime (TTL) may exceed 7 days.
- Your application needs to process messages larger than 64 KB (but not more than 256 KB).
- You need to provide a role-based security model, implement different levels of access rights for both message providers and message consumers.
- The total size of all messages in the queue does not exceed 80 GB.
- You want to use AMQP 1.0 for messaging. More information here - Service Bus AMQP Overview .
- You assume the theoretical possibility of moving from the “one supplier, one recipient” model to a model where messages can be generated by several providers and redirected to more than one recipient (possibly by some filtering rules).
- You need a “At-Most-Once” type delivery guarantee and you do not want to build a special infrastructure for this.
- You need the ability to batch send and receive messages.
- You need full integration with the Windows Communication Foundation (WCF) stack and the .NET Framework.
Comparing Azure Queues and Service Bus Queues
The tables in the following sections are divided into logical groups, each of which compares Azure Queues and Service Bus Queues for some parameters.
Fundamental opportunities
| Comparison criterion | Azure queues | Service bus queues |
|---|---|---|
| Conservation Guarantee | Not | Yes - FIFO (through using sessions) |
| Delivery guarantee | “At least once” | “At least once” “At most once” |
| Transaction support | Not | Yes (through the use of local transactions) |
| Production method | Not blocked (terminates immediately if there are no new messages) | Blockable with or without timeout (see “ Comet technique ”) Non-blocking (through use of the .NET API) |
| "Pushing" messages by the server to the client | Not | Yes OnMessage and OnMessage sessions .NET API |
| Production method | Peek & lease | Peek & Lock Receive & Delete |
| Exclusive Message Access | Based on Lease | Lock Based |
| Lease / Lock timeout duration | 30 seconds (default) 7 days (maximum) (can be updated or reset using the UpdateMessage API) | 60 seconds (default) (can be updated or reset using the RenewLock API) |
| Granularity Lease / Lock | At the message level (each message can have its own timeout, for each message it can be changed using the UpdateMessage API) | At the queue level (each queue has its own timeout, which applies to all messages in this queue, the timeout can be updated using the RenewLock API) |
| Batch receipt | Yes (upon receipt, it is clearly indicated how many messages are required, maximum 32) | Yes (implicitly through enabling preload settings or explicitly through transactions) |
| Batch sending | Not | Yes (via transactions or batch sending on the client side) |
Additional Information
- If you already use Azure Storage Blobs and Tables and start using queues, you still have guaranteed availability of services within 99.9%. If you use Blobs and Tables together with Service Bus Queues - reliability will be less.
- Guaranteed Message Delivery Order (FIFO) in Service Bus Queues requires the use of transactions. If the application crashes while the message is in Peek & Lock mode , after restarting the application, it will receive the same message as soon as it expires (TTL).
- Azure Queues is designed to support standard queuing scenarios, such as separating application components for increased scalability and fault tolerance for individual components.
- Service Bus Queues supports the At-Least-Once delivery guarantee model. In addition, the “Once Upon a Time” model can be implemented using transactions and atomic receipt of messages with updating session state. Azure Workflow uses this technique to provide its Maximum Once Upon A Time Guarantee.
- Azure Queues provides a unified and consistent access interface similar to the table and blob access interface, which can be convenient if you need to use all of these services.
- Service Bus Queues provides local transaction support in the context of each queue.
- The “Receive and Delete” mode supported by Service Bus Queues allows you to reduce the number of operations for receiving messages in exchange for reducing the guarantee of their receipt.
- Azure Queues allows you to handle the size of the message processing timeout. A worker can quickly process “short” messages, and if he suddenly crashes, messages will be quickly transferred to another worker. At the same time, if the worker just needs more time to process the message, he can increase its timeout, making it clear that the message is still being processed.
- Azure Queues allow you to set a timeout for the “invisibility” of a message when it is published. The message will be available to the consumer only after this timeout. Service Bus Queues also allows you to set such a timeout, but only at the queue metadata level.
- The maximum timeout for a blocked message receiving operation in Service Bus Queues is 24 days. However, when using the REST interface, this timeout is 55 seconds.
- Client-side batch operations provided by Service Bus Queues allow multiple messages to be combined into a single send operation. Batch sending is only available for asynchronous send operations.
- Support for up to 200 TB given in Azure Queues queues (and even more in case of account virtualization) and an unlimited number of queues makes this service an ideal platform for SaaS providers.
- Azure Queues provides a flexible and powerful access delegation mechanism.
Additional features
| Comparison criterion | Azure queues | Service bus queues |
|---|---|---|
| Scheduled Delivery | Yes | Yes |
| Automatic marking of dead messages | Not | Yes |
| Increase message lifetime in a queue | Yes (timeout update separately for each message in the queue) | Yes (separate API function) |
| Support for incomplete messages | Yes | Yes |
| Message Update | Yes | Yes |
| Server-side transaction log | Yes | Not |
| Storage Metrics | Yes Minute metrics: in real time for availability, TPS, number of API calls, number of errors, etc., see About Storage Analytics Metrics | Yes ( GetQueues Method ) |
| State management | Not | Yes Microsoft.ServiceBus.Messaging.EntityStatus.Active , Microsoft.ServiceBus.Messaging.EntityStatus.Disabled , Microsoft.ServiceBus.Messaging.EntityStatus.SendDisabled , Microsoft.ServiceBus.Messaging.EntityStatus.ReceiveDisabled |
| Auto Message Forwarding | Not | Yes |
| Clear the entire queue | Yes | Not |
| Message Groups | Not | Yes (via message sessions) |
| Application Status for a Message Group | Not | Yes |
| Duplicate Detection | Not | Yes (configurable on the sender side) |
| Integration with WCF | Not | Yes (WCF Bindings Provided) |
| Integration with WF | External (requires building your own WF activities) | Built-in (provided by WF activities) |
| Message View Groups | Not | Yes |
| Fetching messages of specific messages by session ID | Not | Yes |
Additional Information
- Both queuing technologies allow you to delay message delivery for a specific time.
- Automatic message redirection allows you to collect messages from thousands of queues in one queue. You can use this mechanism to provide security, control processes, and isolate message providers from each other.
- Azure Queues supports the ability to modify the contents of a message. You can use this to update the progress of the task corresponding to this message, so that, if necessary, show this information to the user or continue processing from the desired stage, and not first. In Service Bus Queues, you can implement a similar scenario using sessions. The session allows you to save and get the state of the application (using the SetState and GetState methods ).
- The dead letter processing mechanism, which is supported by Service Bus Queues, allows you to isolate messages that cannot be successfully processed by the recipient or that have reached the end of their life time (TTL). Such messages will be moved to a special queue called $ DeadLetterQueue.
- A partially similar mechanism can be implemented in Azure Queues on the application side - by checking the DequeueCount property, you can redirect certain messages to another queue.
- Azure Queues allows you to get a detailed log of all transactions for each queue, as well as some aggregated metrics. Both features are very useful for debugging and understanding how your applications use the queue. They also help optimize performance and minimize costs when using queues.
- The concept of “message sessions” supported by Service Bus Queues allows each message to belong to a logical group that can be associated with a single recipient. You can use this functionality by filling in the SessionID field for each message. The recipient can then request messages from the queue with only a specific SessionID .
- The duplicate detection mechanism supported by Service Bus Queues automatically removes messages with the same MessageID from the queue .
Capacity and quotas
| Comparison criterion | Azure queues | Service bus queues |
|---|---|---|
| Maximum Queue Size | 200 TB (limited by the total amount of storage for one account) | From 1 GB to 80 GB (determined during the queue creation phase) |
| Maximum message size | 64 KB (48 KB when using Base64 encoding) Azure supports large messages by combining queues and blobs - so you can actually store up to 200 GB of data in one message | 256 KB (including the header and body of the message, the maximum size of the header is 64 KB) |
| Maximum message lifetime | 7 days | Not limited |
| The maximum number of queues | Not limited | 10000 (per namespace, may be increased) |
| Maximum number of customers | Not limited | Unlimited (100 connections for TCP protocol clients) |
Additional Information
- For Azure Queues, if the message content contains characters that cannot be stored in XML, it must be encoded in Base64 . Please note that in this case the size of the payload in the message should not exceed 48 KB.
- For Service Bus Queues, each message consists of a header and a body. The total message size should not exceed 256 KB.
- When clients use Service Bus Qeues over TCP, the total number of active connections to one queue should not exceed 100. Both senders and recipients are taken into account. If this quota is reached, subsequent requests will be rejected. This limit does not apply to clients who use the queue through the REST API.
- Service Bus Queues has queue size limits. They are set when the queue is created and can range from 1 to 80 GB. If the limit is reached, the next message will not be accepted into the queue. More info - Azure Service Bus Quotas .
- If you need more than 10,000 Service Bus Queues in one namespace, you can contact Azure technical support and ask for an increase in the limit for you. But it can be simpler and faster to create additional namespace through the Microsoft Azure Management Portal.
Management and access
| Comparison criterion | Azure queues | Service bus queues |
|---|---|---|
| Control protocol | REST over HTTP / HTTPS | REST over HTTPS |
| Protocol of operations | REST over HTTP / HTTPS | REST over HTTPS AMQP 1.0 Standard (TCP with TLS) |
| .NET API | Yes (.NET Storage Client API) | Yes (.NET API for message brokers) |
| Native c ++ | Yes | Not |
| Java API | Yes | Yes |
| PHP API | Yes | Yes |
| Node.js API | Yes | Yes |
| Custom metadata support | Yes | Not |
| Queue Naming Rules | Up to 63 characters (letters must be lowercase) | Up to 260 characters (queue names are case-insensitive) |
| Getting the number of messages in the queue | Yes (approximate, excluding messages for which TTL has expired and they are in the process of deletion) | Yes (exact) |
| The function of viewing messages in the queue without selecting them | Yes | Yes |
Additional Information
- Azure queues provides the ability to add an arbitrary amount of metadata to the queue in the form of key / value pairs.
- Both queue technologies allow viewing messages in the queue without changing / deleting them, which can be useful for implementing the queue viewing tools.
- Service Bus supports the .NET API over TCP connections, which works more efficiently than REST requests or the AMQP 1.0 protocol.
- The queue name in Azure Queue can be from 3 to 63 characters long, it can include letters in different cases, numbers and hyphens. More info - Naming Queues and Metadata .
- The queue name in the Service Bus Queue can be up to 260 characters and can include, in addition to letters and numbers, the characters '.', '-' and '_'.
Performance
| Comparison criterion | Azure queues | Service bus queues |
|---|---|---|
| Maximum throughput | Up to 2000 messages per second (for messages of 1 KB) | Up to 2000 messages per second (for messages of 1 KB) |
| Average delay | 10 ms (with TCP Nagle disabled) | 20-25 ms |
| Overload behavior | HTTP code 503 (request not paid) | HTTP code 503 (request not paid) |
Additional Information
- A single Azure Queue queue can process up to 2000 transactions per second. Transactions are a Put , Get, and Delete operations . Sending one message to the queue ( Put ) is considered a single transaction, but receiving a message is often a two-step operation - the message must first be received ( Get ) and then deleted from the queue ( Delete ), which results in two transactions. Do not forget that in order to reduce the number of transactions, you can receive up to 32 messages in one request, although you still have to delete them from the queue one at a time. To increase throughput, you can create more queues (the total number is not limited).
- When your application tries to process more than 2000 messages per second, you will receive a “HTTP 503 Server Busy” response. In this case, you should reduce the frequency of calls to the queue.
- The average latency in Azure Queues is 10 ms when processing messages up to 10 KB when working from an Azure service virtual machine located in the same region as the queue
- Both queue services reject queue requests if the maximum bandwidth is exceeded. Both services do not consider such requests to be paid.
- Service Bus Queues performance tests showed that a single queue can process up to 2000 messages per second at a message size of 1 KB. For more throughput, use multiple queues. More Information - Best Practices for Performance Improvements Using Service Bus Brokered Messaging .
- When maximum throughput is achieved using Service Bus Queues, you will get a ServerBusyException (when using the .NET API) or “HTTP 503 Server Busy” (when using the REST API).
Authentication and Authorization
| Comparison criterion | Azure queues | Service bus queues |
|---|---|---|
| Authentication | By symmetric key | By symmetric key |
| Access control model | Delegated SAS Token Access | RBAC over ACS |
| Association of Identity Providers | Not | Yes |
Дополнительная информация
- Каждый запрос к очереди должен пройти аутентификацию. Очереди с публичным анонимным доступом не поддерживаются. Используя SAS, вы можете построить различные сценарии работы вашего приложения.
- Схема аутентификации в Azure Queues состоит в использовании симметричного ключа, который представляет собой SHA-256 хэш-сумму, закодированную в Base64. Больше информации об аутентификации в Azure Queues — Authenticating Access to Your Storage Account, для Service Bus Queues — Shared Access Signature Authentication with Service Bus.
- Microsoft Azure Active Directory Access Control (также известный как Access Control Service или ACS) поддерживаемый Service Bus, предоставляет три различных роли: Admin, Sender и Receiver, который в данный момент не поддерживаются Azure Queues.
- Поскольку Service Bus поддерживает интеграцию с ACS, вы можете настроить федерацию с Active Directory (через использование ADFS) и общедоступными провайдерами идентификации, такими как Live ID, Google, Facebook или Yahoo.
Цены
| Критерий сравнения | Azure Queues | Service Bus Queues |
|---|---|---|
| Стоимость транзакции | $0.0005 (за 10,000 транзакций) | $0.01 (за 10,000 сообщений) |
| Оплачиваемые операции | Все | Только Send и Receive (остальные — бесплатны) |
| Холостые транзакции | Оплачиваются (запрос сообщения при пустой очереди стоит столько же, сколь запрос при наличии сообщений) | Оплачиваются (запрос сообщения из пустой очереди оплачивается как получение одного сообщения) |
| Стоимость хранения данных | $0.07 (за ГБ в месяц) | $0.00 |
| Стоимость внешнего трафика | $0.12 — $0.19 (зависит от дата-центра) | $0.12 — $0.19 (зависит от дата-центра) |
Дополнительная информация
- Передача данных оплачивается исходя из общего количества данных, исходящих из дата-центра Azure в Интернет за оплачиваемый период.
- Передача данных между сервисами Azure внутри одного дата-центра не оплачивается.
- На данный момент входящий трафик не оплачивается.
- The cost of ACS transactions using Service Bus Queues is very low. Service Bus Queues generates one ACS token per queue and uses it until it expires (about 20 minutes). Thus, the number of operations is not directly proportional to the number of ACS transactions.
- In situations where the message should be received very quickly, it is more cost-effective to use the Service Bus Queues and the mechanism of notifications about the arrival of new messages.
Comment
Prices are subject to change. Check out the Pricing Overview page .
Conclusion
The decision to choose between Azure Queues and Service Bus Queues depends on many factors, the needs of your application and its architecture. If you already use other Azure services, such as tables or blobs, you will just start using Azure Queues, especially if you only need basic scenarios for using queues. On the other hand, Service Bus Queues offers a number of additional features, such as sessions, transactions, duplicate detection, automatic processing of "dead" messages, which may be necessary for your application.