Saga Pattern: Part 2 - Choreography

Introduction Link to heading

In the previous post, we explored the Saga Pattern using the Orchestrator approach, where a central service is responsible for managing the sequence of operations and handling failures across multiple microservices. While this approach provides clear control and visibility, it can also introduce a single point of failure and tight coupling between services.

In this follow-up, we will look at the other main way to implement the Saga Pattern: Choreography. Instead of relying on a central orchestrator, the Choreography approach lets each service participate in the saga by listening for and emitting events. Each service knows how to react to certain events and, if necessary, trigger the next step in the process. This results in a more decentralized and loosely coupled system, where the overall workflow emerges from the collaboration of independent services.

Choreography is especially popular in event-driven architectures, where services communicate through a message broker or event bus. It’s a great fit for systems that need to scale, evolve independently, or avoid bottlenecks. However, it also comes with its own challenges, such as increased complexity in tracing the flow of a transaction and handling compensating actions.

Folder Structure Link to heading

saga-pattern/
├── docker-compose.yml
├── order-service/
│   ├── app.py
│   ├── pubsub.py
│   └── Dockerfile
├── inventory-service/
│   ├── app.py
│   ├── pubsub.py
│   └── Dockerfile
├── payment-service/
│   ├── app.py
│   ├── pubsub.py
│   └── Dockerfile

# docker-compose.yml
services:
  redis:
    image: redis:latest
    ports:
      - "6379:6379"

  order:
    build: ./order-service
    ports:
      - "5000:5000"
    depends_on:
      - redis

  inventory:
    build: ./inventory-service
    depends_on:
      - redis

  payment:
    build: ./payment-service
    depends_on:
      - redis

# pubsub.py: Same for all services
import json
import threading

import redis

r = redis.Redis(host="redis", port=6379, decode_responses=True)

def publish(channel, data):
    r.publish(channel, json.dumps(data))

def subscribe(channel, handler):
    def listen():
        pubsub = r.pubsub()
        pubsub.subscribe(channel)
        for msg in pubsub.listen():
            if msg["type"] == "message":
                handler(json.loads(msg["data"]))
    thread = threading.Thread(target=listen)
    thread.start()

# order-service/app.py
from flask import Flask, request, jsonify

from pubsub import publish, subscribe

app = Flask(__name__)
orders = {}

@app.route("/create_order", methods=["POST"])
def create_order():
    data = request.json
    order_id = data["order_id"]
    product_id = data["product_id"]
    amount = data["amount"]
    orders[order_id] = {"status": "created", "product_id": product_id, "amount": amount}
    print(f"[ORDER] Created order {order_id}")
    publish("OrderCreated", data)
    return jsonify({"status": "OrderCreated"})

def handle_payment_completed(data):
    order_id = data["order_id"]
    print(f"[ORDER] Payment completed for order {order_id}")
    orders[order_id]["status"] = "completed"

def handle_payment_failed(data):
    order_id = data["order_id"]
    print(f"[ORDER] Payment failed for order {order_id}")
    orders[order_id]["status"] = "cancelled"

subscribe("PaymentCompleted", handle_payment_completed)
subscribe("PaymentFailed", handle_payment_failed)

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000)

# inventory-service/app.py
from pubsub import publish, subscribe

inventory = {"item-1": 10}

def handle_order_created(data):
    order_id = data["order_id"]
    product_id = data["product_id"]
    if inventory.get(product_id, 0) > 0:
        inventory[product_id] -= 1
        print(f"[INVENTORY] Reserved item {product_id} for order {order_id}. Instock inventory: {inventory[product_id]}")
        publish("InventoryReserved", data)
    else:
        print(f"[INVENTORY] Out of stock for order {order_id}. Instock inventory: {inventory[product_id]}")
        publish("InventoryFailed", {"order_id": order_id})

def handle_payment_failed(data):
    order_id = data["order_id"]
    product_id = data["product_id"]
    inventory[product_id] += 1
    print(f"[INVENTORY] Rolled back reservation for order {order_id}. Instock inventory: {inventory[product_id]}")

subscribe("OrderCreated", handle_order_created)
subscribe("PaymentFailed", handle_payment_failed)

if __name__ == "__main__":
    print("Inventory service running...")
    while True:
        pass

# payment-service/app.py
from pubsub import publish, subscribe

def handle_inventory_reserved(data):
    order_id = data["order_id"]
    product_id = data["product_id"]
    print(f"[PAYMENT] Processing payment for order {order_id}")
    # Simulate failure
    if order_id.endswith("fail"):
        print(f"[PAYMENT] Payment failed for order {order_id}")
        publish("PaymentFailed", {"order_id": order_id, "product_id": product_id})
    else:
        print(f"[PAYMENT] Payment completed for order {order_id}")
        publish("PaymentCompleted", {"order_id": order_id, "product_id": product_id})

subscribe("InventoryReserved", handle_inventory_reserved)

if __name__ == "__main__":
    print("Payment service running...")
    while True:
        pass

# Dockerfile: Same for all services
FROM python:3.11-slim

ENV PYTHONUNBUFFERED=1

WORKDIR /app

COPY app.py pubsub.py ./

RUN pip install --no-cache-dir Flask==3.1.1 redis==6.2.0

CMD ["python", "app.py"]

Test the Choreography implementation Link to heading

Build and run all services

docker-compose up --build

Test create a successful order

curl -X POST http://localhost:5000/create_order \
  -H "Content-Type: application/json" \
  -d '{"order_id": "123", "product_id": "item-1"}'

Check the docker-composelogs, you will see:

Order is created
Inventory is reserved
Payment is processed
Order is completed

The logs will look like this:

order-1      | [ORDER] Created order 123
order-1      | 192.168.65.1 - - [17/Jan/2024 02:40:45] "POST /create_order HTTP/1.1" 200 -
inventory-1  | [INVENTORY] Reserved item item-1 for order 123. Remaining: 9
payment-1    | [PAYMENT] Processing payment for order 123
payment-1    | [PAYMENT] Payment completed for order 123
order-1      | [ORDER] Payment completed for order 123

Test create a failed order

curl -X POST http://localhost:5000/create_order \
  -H "Content-Type: application/json" \
  -d '{"order_id": "123fail", "product_id": "item-1"}'

Check the docker-compose logs, you will see:

Order is created
Inventory is reserved
Payment fails
Inventory is rolled back
Order is failed

The logs will look like this:

order-1      | [ORDER] Created order 123fail
order-1      | 192.168.65.1 - - [17/Jan/2024 02:49:54] "POST /create_order HTTP/1.1" 200 -
inventory-1  | [INVENTORY] Reserved item item-1 for order 123fail. Instock inventory: 8
payment-1    | [PAYMENT] Processing payment for order 123fail
payment-1    | [PAYMENT] Payment failed for order 123fail
order-1      | [ORDER] Payment failed for order 123fail
inventory-1  | [INVENTORY] Rolled back reservation for order 123fail. Instock inventory: 9

When to Use Choreography Pattern Link to heading

When your workflow is simple and event-driven.
When you want to avoid a central point of failure.
When services are loosely coupled and can evolve independently.

Drawbacks Link to heading

Harder to trace the flow of a transaction (distributed tracing is important!).
More difficult to coordinate complex workflows or compensations.
Risk of cyclic or lost events if not designed carefully.