Asynchronous messaging and queues

Asynchronous messaging provides freedom in communication: the communicating parties do not need to be available at the same time but trust that the messages will be delivered safely and securely. So, why do we need asynchronous communication? If we consider a world without asynchronous communication, there would be little notion of multitasking and we would spend most of our time waiting for things to happen.

Say you want to know if your friend would be meeting you at a café. Ordinarily, you would just send them a text and wait for their response. Without this asynchronous communication however, you would be at the café indefinitely waiting for your friend to arrive.

In this article, we’ll take a closer look at how asynchronous messaging works in IBM MQ, and the importance of the queue in this type of messaging.

What is asynchronous communication?

Asynchronous communication is a framework which can be applied to a range of use cases. It’s basically any kind of communication where you don’t expect an immediate response. We communicate asynchronously all the time without even realising it, whether that be through emails, texts, or even letters in the post.

In this framework, a system will first kick-off a process (typically asking something of another system). The system is then free to do anything it wants; it does not have to stay idle until the process is completed. The system responsible for completing this process can do so whenever it is able to.

The queue plays a key role in facilitating this, ensuring essential performance guarantees such as exactly-once delivery. In enterprise messaging, this involves a producing application putting a message onto a queue. This message will remain on the queue until it is picked up and successfully processed by a consuming application, at which point it will be removed from the queue. The consuming application can then process this message as required, without the producing application having to wait around for the outcome.

While alternative technologies like HTTP and REST offer a simple way to connect simple applications directly to facilitate responsive messaging, that is great for interactive sessions. However, this responsive quality is lost as you begin to scale up and increase load because greater developer intervention is required to ensure security and proper error handling. Also, since the applications are connected directly, the system becomes reliant on all down-stream services being available, which leads to brittle solutions.

Using a queue decouples applications and allows you to delegate security and error handling to the broker. It also means that the system can scale independently, leading to a more flexible and scalable solution.

Message queues were created with asynchronous communication in mind, hiding the complexity of the messaging broker from the producing and consuming applications to facilitate a ‘smart broker, simple application’ model. Let’s now take a closer look at how the queue uniquely facilitates asynchronous communication.

How message queues uniquely facilitate asynchronous communication

Let’s set the scene with an example. Imagine we have an online shop that suddenly receives an influx in orders, causing a backlog of orders which need to be fulfilled. We still want the order submission process to remain available, even if the order fulfilment process can’t keep up yet. So the order submission process will continue to receive orders, which will be stored securely in a queue. These orders will then be consumed (and removed) from the queue by the order fulfilment process on a first-in-first-out (FIFO) basis.

Here, the queue acts as a shock absorber, allowing orders to continue to flow in without overwhelming the fulfilment process. This scenario also demonstrates the key characteristics of a queue which make it ideal for asynchronous communication:

• Decoupling: the producing application could send messages to the queue without the consuming application being available at the same time.
• FIFO: orders are consumed in the order they are received.
• Resilience: if the order submission or fulfilment process were to crash in response to the sudden heavy traffic, the messages would remain safely on the queue to be processed once the systems came back online.
• Message removal: once the consumer picks up and processes a message, it is safely removed from the queue, so there is no risk of processing a message twice – providing the exactly-once delivery guarantee that is crucial in enterprise messaging scenarios.

Queues provide a secure, scalable, and resilient environment for asynchronous communication. While it is possible to carry out asynchronous communication over alternative technologies such as consumer share groups, they do not provide the same assurances or the same level of abstraction as the queue. With consumer share groups, messages are not removed once “consumed”.

In an asynchronous setting, this poses the risk of messages being delivered more than once. Also, unlike queues, consumer share groups are not optimised for ordering messages, so require more configuration on the application side to make them suitable for the asynchronous, which leads to a more ‘smart application, simple broker’ model.

Summary and next steps

In this article, we explored asynchronous messaging and its importance in an enterprise context. We also looked at the role that message queues play in this context and the advantages of queues for asynchronous messaging, over other technologies.

Now, head to the next article in the series, Message and queue fundamentals for an in depth look at the components of a queue you can leverage to enhance your messaging solutions!