Skip to main content
Back to path

Welcome to a Grasp lesson

Create a free account to keep studying and create your own study path.

Lesson 64

Choreographing Sagas with Message Brokers

Recommended resources

Hello! Welcome back to our module on Microservices Architectural Principles.

In our last lesson, we compared distributed transaction patterns and established why the Saga pattern is often preferred over two-phase commit (2PC) in microservices. We contrasted the decentralized, event-driven Choreography style with the centralized, command-driven Orchestration style.

Today, we transition from theory to practice. Your learning outcome is to implement a choreography-based Saga using a message broker to manage a distributed transaction. We will build a concrete example using Spring Boot and Apache Kafka, implementing both the "happy path" and the critical "failure path" with compensating transactions. This is a very common and practical problem you'll be expected to solve in senior-level interviews.

The Architectural Blueprint

Let's establish our system's architecture. We will implement a simplified e-commerce order flow involving two services:

  1. Order Service: Initiates the transaction by creating an order.
  2. Payment Service: Processes the payment for that order.

These services are independent, each with its own database, and will communicate asynchronously using Kafka as a message broker.

Microservices Choreography Saga with Kafka
This diagram shows the high-level architecture. Each service (e.g., Order, Payment) is a self-contained Spring Boot application with its own database, communicating via Kafka topics. This loose coupling is a hallmark of choreography.

The flow of events will look like this:

Choreography-based Saga Implementation for Order Processing
This diagram illustrates the event sequence in a choreography. An initial command triggers the `OrderService`, which publishes an `OrderCreatedEvent`. The next service in the chain (`StockService` in the diagram, `PaymentService` in our case) consumes this event and performs its action, publishing its own event. Crucially, it also shows the compensation flow (e.g., `PaymentRejectedEvent`) that triggers rollbacks.

1. Setting the Stage: Events and Project Structure

Before writing the services, we must define the "contract" they will use to communicate: the events. In a real-world project, these event DTOs (Data Transfer Objects) are often placed in a shared library or module that all participating services can depend on.

This ensures that the OrderCreatedEvent published by the Order Service has the exact same structure as the one consumed by the Payment Service.

Microservices Architecture Patterns | SAGA Choreography Implementation | Live Coding | JavaTechie

Let's begin by watching a segment from the 'SAGA Choreography Implementation' video by Java Techie. This part focuses on setting up a common module for DTOs and event classes. This is a crucial first step for establishing a clear contract between our microservices.

Watch from the beginning to 10:29. Pay attention to how OrderEvent and PaymentEvent are defined, along with their associated DTOs and status enums (OrderCreated, PaymentCompleted, PaymentFailed).

As shown in the video, the key components of the contract are:

  • Request/Response DTOs: OrderRequestDto, PaymentRequestDto, etc.
  • Status Enums: OrderStatus (ORDER_CREATED, ORDER_COMPLETED, ORDER_CANCELLED) and PaymentStatus (PAYMENT_COMPLETED, PAYMENT_FAILED).
  • Event Classes: OrderEvent and PaymentEvent, which wrap the DTOs and status information.

With this shared contract, we can now build our services.

2. The Saga Initiator: The Order Service

The saga begins in the Order Service when a customer places an order. Its responsibilities are:

  1. Receive a request to create an order.
  2. Execute a local transaction: Persist the order to its own database with an initial status of ORDER_CREATED.
  3. Publish an OrderCreated event to a Kafka topic.

For communication, we'll use Spring Cloud Stream. It provides a powerful, declarative way to connect to message brokers like Kafka. We'll use its functional programming model, which is a modern and common approach.

Microservices Architecture Patterns | SAGA Choreography Implementation | Live Coding | JavaTechie

Now, let's implement the Order Service. Watch this segment to see how to create the order, save it, and publish the OrderCreated event.

Watch from 10:29 to 29:41. Focus on these key implementation details: OrderService.createOrder(): Notice how it saves the order entity and then calls a publisher to send the event. This entire method is wrapped in @Transactional to ensure the database save and the event preparation are atomic. OrderPublisherConfig: Understand how a Supplier<Flux<OrderEvent>> bean is used. In Spring Cloud Stream, a Supplier is a source of messages—a producer. application.yaml: See how the orderSupplier function is bound to the order-event Kafka topic under spring.cloud.stream.bindings. This is how Spring knows where to send the events produced by our Supplier.

To summarize the publishing mechanism:

  • We use a reactive Sinks.many() object, which acts as a processor that can programmatically receive and publish events.
  • The OrderService emits an event to the sink.
  • A Supplier bean in our configuration exposes this sink as a reactive stream (Flux).
  • Spring Cloud Stream automatically binds this Supplier to the configured output topic in application.yaml, sending the event to Kafka.

This reactive, functional approach is highly decoupled. The service logic simply emits an event to a sink, unaware of the underlying messaging system (Kafka, RabbitMQ, etc.).

3. The Saga Participant: The Payment Service

The Payment Service is the next link in our distributed transaction chain. Its responsibilities are:

  1. Listen for OrderCreated events from the order-event Kafka topic.
  2. Execute its local transaction: Check the user's balance. If sufficient, deduct the amount.
  3. Publish the outcome: A PaymentCompleted event or a PaymentFailed event to a payment-event topic.

This service acts as both a consumer and a producer.

Microservices Architecture Patterns | SAGA Choreography Implementation | Live Coding | JavaTechie

Let's build the Payment Service. This section covers consuming the order event, processing the payment, and publishing the result.

Watch from 29:41 to 55:13. Pay close attention to: PaymentConsumerConfig: Here, a Function<Flux<OrderEvent>, Flux<PaymentEvent>> is used. A Function in Spring Cloud Stream is a processor that consumes messages from an input, processes them, and produces messages to an output. This is perfect for our use case. PaymentService Logic: Observe the core business logic. It checks the user's balance. If the balance is sufficient, it updates the database and prepares a PAYMENT_COMPLETED event. If not, it prepares a PAYMENT_FAILED event. @Transactional: The methods that update the user's balance and transaction history are marked as transactional to ensure consistency within the service's own database.

The Payment Service perfectly demonstrates the decentralized nature of choreography. It subscribes to an event it cares about (OrderCreated), performs its own independent work, and then publishes its result as another event (PaymentEvent) for any other interested service to consume.

4. Closing the Loop: Compensation in the Order Service

Our saga is not yet complete. The Order Service initiated the process, but it is still unaware of the payment outcome. The final step is to implement the compensating transaction.

The Order Service must now also act as a consumer:

  1. Listen for events on the payment-event topic.
  2. If a PaymentCompleted event is received, update the corresponding order's status to ORDER_COMPLETED.
  3. If a PaymentFailed event is received, execute the compensating transaction: update the order's status to ORDER_CANCELLED.

This step ensures that the system achieves eventual consistency.

Microservices Architecture Patterns | SAGA Choreography Implementation | Live Coding | JavaTechie

Finally, let's implement the compensation logic back in the Order Service.

Watch from 55:13 to 01:11:36. Focus on: EventConsumerConfig: A new configuration is added to the Order Service with a Consumer<PaymentEvent> bean. A Consumer in Spring Cloud Stream is a sink for messages—it only consumes. OrderStatuUpdateHandler: This class contains the logic to find the original order and update its status based on the PaymentStatus from the incoming PaymentEvent. application.yaml Update: The Order Service's configuration is updated to define both the producer (orderSupplier-out-0) and the new consumer (paymentEventConsumer-in-0).

You have now implemented a full, distributed transaction with a success path and a compensating failure path. When the Order Service receives a PaymentFailed event, it doesn't just change the order status; it is effectively "rolling back" the initial CREATE operation by marking the order as CANCELLED.

Test your understanding! (Interview Question)

An interviewer asks: "In the Saga you just described, what happens if the Payment Service processes a payment successfully but then crashes before it can publish the PaymentCompleted event? How would you handle this to prevent the system from getting stuck?"

Show answer

This is a great question that tests your understanding of the challenges in event-driven systems. A strong answer would be:

"This is a classic problem in distributed systems where an operation completes but the notification fails. The Order Service would be stuck waiting for a PaymentEvent that never comes. There are two common patterns to solve this:

  1. Transactional Outbox Pattern: This is the most robust solution. Instead of the Payment Service publishing the event directly, it writes the event to an "outbox" table within its own database, as part of the same local transaction that updates the user's balance. A separate, asynchronous process (or a tool like Debezium using Change Data Capture) then reads from this outbox table and reliably publishes the events to Kafka. This guarantees that an event is published if and only if the local transaction was successful. If the service crashes after committing the transaction but before publishing, the outbox process will pick it up and send it once the service restarts.

  2. Polling Publisher / Reconciliation Job: A simpler, though less real-time, approach is to have a background job. The Order Service could have a job that periodically scans for orders that have been in the CREATED state for too long (e.g., more than 5 minutes). For each expired order, it could query the Payment Service directly via a synchronous API call to get the payment status, or it could re-publish a 'CheckOrderStatus' event. This reconciles the state but introduces latency and synchronous coupling for the reconciliation path."

Mentioning the Transactional Outbox pattern in an interview demonstrates a deep understanding of building reliable, event-driven microservices.

Conclusion

In this lesson, you have successfully moved from Saga theory to a practical, code-level implementation. You now have a tangible project that demonstrates one of the most important patterns in microservice architecture.

Key Takeaways:

  • Choreography is event-driven: Services communicate by producing and consuming events from a message broker, without direct knowledge of each other.
  • Spring Cloud Stream simplifies messaging: Using Supplier, Function, and Consumer beans with simple YAML configuration provides a powerful abstraction over the message broker.
  • Sagas ensure eventual consistency: The happy path involves a sequence of local transactions, while the failure path uses compensating transactions (e.g., setting the order status to CANCELLED) to roll back the business process.
  • Reliability is key: Patterns like Transactional Outbox are critical for handling failures during event publication to build a truly resilient system.

Preview of the Next Module:
We've just built a system that maintains data consistency even when parts of the process fail. But what happens if a service doesn't just fail a single operation, but becomes completely unresponsive? This can cause a chain reaction, or cascading failure, that takes down the entire system.

In our next module, Microservices: Resilience & Observability, we will start by tackling this exact problem. Our first lesson will be on how to implement the Circuit Breaker pattern using Resilience4j to isolate failures and prevent them from spreading.