Skip to main content
Back to path

Welcome to a Grasp lesson

Create a free account to keep studying and create your own study path.

Lesson 74

Message Brokers in Distributed Systems

Recommended resources

Hello! Welcome to the next lesson in our journey through asynchronous and event-driven architectures.

In our last lesson, we saw how to use Spring's internal event system (ApplicationEventPublisher and @EventListener) to create decoupled components within a single application. We learned that by default, this communication is synchronous, but can be made asynchronous with @Async. This is a powerful pattern for building modular monoliths.

However, as we move toward your goal of building microservices, we face a new challenge: how do separate, independently running services communicate with each other? Spring's internal events are confined to their own application context and cannot cross network boundaries.

Today, we'll explore the standard solution to this problem. This lesson will explain the role of message brokers (e.g., RabbitMQ) in distributed systems. By the end, you will understand what message brokers are, the critical problems they solve, and the fundamental communication patterns they enable, setting the stage for building robust, scalable microservice applications.

1. The Challenge: Communication Between Services

Imagine our e-commerce application is split into two separate microservices:

  1. UserService: Handles user registration and profiles.
  2. NotificationService: Responsible for sending emails and SMS messages.

When a new user registers via the UserService, we need to trigger the NotificationService to send a welcome email.

A simple approach would be for the UserService to make a direct, synchronous REST API call to the NotificationService.

A simple but brittle approach: direct, synchronous API calls between services.

While this works, it introduces several significant problems that are common in distributed systems:

  • Tight Coupling: The UserService must know the network address (e.g., http://notification-service:8081) of the NotificationService. If the address changes, or if we want to add a second service that also needs to react to user registrations, we have to modify the UserService code.
  • Poor Resilience: What happens if the NotificationService is temporarily down or responding slowly? The synchronous API call in UserService will block or fail, potentially causing the entire user registration process to fail. The health of one service is directly tied to the health of another.
  • Scalability Bottlenecks: If we have a sudden surge of registrations, the UserService will bombard the NotificationService with requests. The NotificationService might become overwhelmed and crash, taking down the notification functionality for the entire system.

This synchronous, tightly-coupled approach is fragile. We need a way to decouple our services in both space (not needing to know addresses) and time (not needing to be available simultaneously).

2. The Solution: The Message Broker

This is where a message broker comes in. A message broker is a piece of middleware that acts as a central post office for your services. Instead of talking directly to each other, services send messages to the broker, and the broker ensures they are delivered to the correct recipients.

What Is a Message Broker?

To get a formal definition, let's start with an excellent article from IBM that clearly explains the concept.

Please read the first section titled 'What is a message broker?'. Focus on how it acts as an intermediary, enables asynchronous communication, and uses message queues for storage and reliable delivery.

As the article explains, the broker allows a service (a producer or publisher) to send a message without knowing anything about the receiving service (the consumer or subscriber).

Let's re-imagine our architecture with a message broker:

A decoupled architecture using a message broker. Services communicate indirectly, improving resilience and scalability.

Here's how this solves our previous problems:

  • Decoupling: The UserService only needs to know the address of the broker. It sends a "UserRegistered" message and its job is done. It has no idea who, if anyone, is listening.
  • Asynchronous Communication & Resilience: The UserService sends the message and can immediately continue its work. The call is non-blocking. If the NotificationService is down, the broker holds onto the message in a durable queue and delivers it once the service comes back online. No data is lost.
  • Load Leveling & Scalability: During a traffic spike, the UserService can rapidly publish thousands of messages to the broker. The broker queues them up, and the NotificationService can consume and process them at its own pace, preventing it from being overwhelmed.

This image nicely summarizes the key benefits of using a message broker.

Why Use Message Brokers?
This visual from Serverless Land illustrates the key advantages of using a message broker: reducing coupling, enabling parallel processing, managing consumer load, and preventing data loss.

3. Core Messaging Patterns

Message brokers typically support two primary models for distributing messages. Choosing the right one is essential for designing your system correctly.

What Is a Message Broker?

The IBM article also clearly explains these two fundamental messaging models.

Now, read the section 'Message broker models'. Pay close attention to the difference between the one-to-one nature of Point-to-Point and the one-to-many nature of Publish/Subscribe, and the examples given for each (e.g., financial transactions vs. flight updates).

Let's summarize these two patterns.

Message Brokers: Models, Pros, Cons, and Examples
This diagram illustrates the two primary messaging models. On the left, Point-to-Point shows a message going to a single consumer. On the right, Publish/Subscriber shows a message being broadcast to multiple consumers.

1. Point-to-Point (P2P) Messaging

  • What it is: A message is sent by a producer to a queue. One and only one consumer receives and processes that message.
  • Analogy: A task queue or a to-do list. Each task should only be done once.
  • When to use it: When work needs to be done exactly once. For example, processing a payment, fulfilling an order, or generating a unique report. If you have multiple consumers on the same queue, they will compete for messages, which is a great way to parallelize work.

2. Publish/Subscribe (Pub/Sub) Messaging

  • What it is: A message is published by a producer to a topic or exchange. Every consumer subscribed to that topic receives a copy of the message.
  • Analogy: A newsletter or a broadcast notification. The same information is sent to all interested parties.
  • When to use it: When an event occurs that multiple parts of the system may care about. For our "UserRegistered" event, you might have:
    • NotificationService to send a welcome email.
    • AnalyticsService to track new sign-ups.
    • CRMService to create a customer profile.
      All three services would subscribe to the "UserRegistered" topic and receive the same message independently.
Test your understanding!

For each of the following scenarios in an e-commerce application, would you choose a Point-to-Point (Queue) or Publish/Subscribe (Topic) model? Explain your reasoning.

  1. A user finalizes an order, and an invoice PDF needs to be generated.
  2. An order has been shipped, and this status update needs to be reflected in multiple systems: the customer's order history page, the shipping department's dashboard, and the inventory management system.
  3. A request is made to resize a large product image into several different thumbnail sizes (small, medium, large).
Show answer

  1. Point-to-Point (Queue). Generating an invoice is a specific task that should be performed exactly once for each order. A dedicated "InvoiceGeneration" service would consume a message from an invoice-requests queue.

  2. Publish/Subscribe (Topic). The "OrderShipped" event is a piece of information that multiple, distinct parts of the system need to react to. Publishing this event to an order-events topic allows the OrderHistoryService, ShippingDashboardService, and InventoryService to all receive the update independently without knowing about each other.

  3. Point-to-Point (Queue). Although multiple outputs are created (different thumbnail sizes), this is best modeled as a single, discrete task: "process this image". You would send one message to an image-processing queue. A consumer would pick up the message and perform all the resizing operations. Using Pub/Sub here would be incorrect, as you don't want three different services to try and resize the same image.

4. Message Brokers vs. REST APIs in Microservices

Your journey in this course is focused on building REST APIs, so it's vital to understand when to use a direct API call versus a message broker. It's not an "either/or" choice; they are both essential tools for building distributed systems.

What Is a Message Broker?

So, when should you use a message broker instead of a direct REST API call? This is a crucial design decision in microservices. The IBM article offers a great comparison.

Read the section 'Message brokers vs. APIs'. Focus on the contrast between the synchronous, request/response nature of REST/HTTP and the asynchronous, resilient nature of broker-based communication.

In summary:

  • Use synchronous REST APIs when the client needs an immediate response. For example, a GET /users/{id} request must return the user data right away for the frontend to display. This is a query.
  • Use a message broker for commands or events that can be processed in the background and do not require an immediate response. For example, "send a welcome email" or "process this video upload". This is an asynchronous workflow.

RabbitMQ: A Concrete Example

RabbitMQ is one of the most widely used open-source message brokers. It is mature, reliable, and implements the patterns we've discussed.

Microservices - why use RabbitMQ?

Let's see how RabbitMQ, a specific message broker we will use later, fits into this picture.

Read the sections 'What is Message Queueing?' and 'Handling communication with brokers'. Note the key terms: producer, consumer, and how RabbitMQ introduces a component called an exchange to route messages to queues.

In RabbitMQ, a producer doesn't publish directly to a queue. It sends a message to an exchange, which is responsible for routing the message to one or more queues based on a set of rules. This provides a very flexible and powerful routing mechanism that we will explore in detail later.

Conclusion

Today, you've taken a significant step from in-process communication to understanding the backbone of distributed systems. We've seen how direct, synchronous communication between services can lead to fragile systems and how message brokers solve these issues.

Key Takeaways:

  • Message brokers act as intermediaries (like a post office) to facilitate asynchronous communication between services.
  • They decouple services, improve resilience against failures, and help manage load by queuing messages.
  • The two main messaging patterns are Point-to-Point (for tasks that must be done once) and Publish/Subscribe (for broadcasting events to multiple interested parties).
  • RabbitMQ is a popular and powerful message broker that implements these patterns using exchanges and queues.
  • You should use message brokers for asynchronous workflows, and REST APIs for synchronous request-response interactions.

What's Next?

We've covered the "what" and "why." In our next lesson, we will move to the "how." We'll get hands-on and set up RabbitMQ for local development using Docker. This will prepare our environment for the subsequent lessons where we'll integrate it into our Spring Boot application and build our first truly distributed, event-driven feature.