Pub/Sub and events

  • Lesson 22 of 40
  • 10 min
  • Beginner
  • 10 steps

When a service calls everyone who depends on it, one by one, it becomes slow and fragile. With events it only announces what happened, and the others react at their own pace.

Purchase time
85 ms
Calls per purchase
0
Failed purchases
0%
Subscribers
5
buyersOrdersOrdersEvent bustopic: ordersEvent busEmailEmailInventoryInventoryAnalyticsAnalyticsLoyaltyLoyaltyFraudcheckFraud checkOrders DBwaits for 4 answersPurchase timesave the order80 ms4 calls × 180 ms720 msuntil it answers800 msexample figuresdownpurchase failedOrderCreatedOrderCreatedQueue or topicqueue1 consumershares out worktopicevery groupspreads a factgroup1 copyits instances share itnew subscriber0 changesdown · 4 events waitingpurchase OKThe fine printwho reacts?5 repositoriesconsistencyeventualdeliverymay duplicateorderper partition onlyidempotent subscribers + tracingone transactionrelaykeeps 30 daysrewound 30 days
  • request
  • response
  • event
  • failed request
  • waiting in queue

Step 1 of 10

Problem

Every purchase calls four services

Confirming an order triggers four more jobs: send the confirmation email, set aside the stock, record the sale in analytics and add loyalty points.

The order service saves the order (80 ms), then calls each service in turn and waits for its answer (about 180 ms each). The buyer only sees «thanks» after 800 ms. Every figure in this lesson is an example.

Step 2 of 10

Problem

Email goes down and nobody can buy

The email provider has an outage. The order service's call fails and, because the purchase waits on every call, the whole purchase fails.

An email that could have gone out ten minutes later has just stopped every sale.

What changed

  • Email service down
  • Every purchase fails

Step 3 of 10

Solution

Announce an event instead of calling

Now the order service saves the order, publishes a single event, OrderCreated, to the orders topic of an event bus and answers the buyer right away: 80 ms + 20 ms to publish = 100 ms.

Email, inventory, analytics and loyalty are subscribed to that topic. The bus hands each of them its own copy of the event, and each one processes it when it can.

What changed

  • Event bus with the «orders» topic
  • 4 calls → 1 published event
  • Purchase: 800 → 100 ms

Step 4 of 10

Concept

Queue or topic?

In a queue, each message is handled by a single consumer: it shares out work. In a topic, each event reaches every subscriber: it spreads the word that something happened.

In practice they are combined through consumer groups: each subscribed service forms a group that gets one copy of every event, and inside the group that service's instances share the work, as with a queue.

Step 5 of 10

Solution

A new feature just subscribes

The risk team wants to screen every order for fraud. They build a fraud check service that subscribes to OrderCreated, and that is all.

The order service did not change, was not redeployed and does not even know the fraud check exists. That is loose coupling.

What changed

  • Fraud check subscribes
  • Order service: unchanged

Step 6 of 10

Concept

A subscriber goes down and catches up

Email goes down again, but this time the purchase does not even notice: the bus keeps the email group's events until it returns. In the example, 4 events are waiting.

When email recovers, it works through the backlog and every confirmation goes out: a little late, but none lost.

What changed

  • Email service down
  • Its events wait on the bus
  • Purchases keep working

Step 7 of 10

Trade-off

What events cost

Decoupling has a price. Answering «what happens when an order is created?» is no longer reading one function: you have to look through five repositories.

Subscribers act a little later (eventual consistency). Delivery is usually at least once, so an event can arrive twice. Order is only guaranteed inside a partition: if you use the order id as the key, the events of one order arrive in order, but there is no global order.

Debugging needs traces that follow an event from service to service.

Step 8 of 10

Problem

Saved, but never announced

The order service saves the order and then publishes the event. One day it crashes right between the two steps: the order exists, but the event never went out. No email, no stock set aside, no points.

The fix is the transactional outbox: the event is written to an outbox table in the same transaction as the order. A separate process, the relay, reads that table and publishes whatever it finds to the bus. The purchase no longer waits for the bus: 80 ms + 5 ms for the extra row = 85 ms.

What changed

  • Orders database with an outbox table
  • The relay publishes from the outbox
  • Purchase: 100 → 85 ms

Step 9 of 10

Concept

Processing the past again

Analytics finds a bug that miscounted revenue for the last 30 days. In a classic queue, messages are deleted once consumed: they are gone.

An event log keeps events for a configurable time (here, 30 days). Each consumer group keeps its own position in that log. Analytics fixes its code, moves its position 30 days back and processes everything again; the other groups never notice.

What changed

  • The bus keeps 30 days of events
  • Analytics rewinds and reprocesses

Step 10 of 10

Summary

Summary

Chained calls make the caller slow and fragile, and tie every team to it.

Publish an event to a topic: each subscriber gets its own copy.

A queue shares out work, a topic spreads news; consumer groups combine both.

New features just subscribe, and a subscriber that went down catches up later.

The price: eventual consistency, duplicates, order only per partition and more tracing.

Use an outbox so data and its event never disagree; an event log lets you reprocess.

You will learn

  • Why direct calls couple services
  • Publishers, topics and subscribers
  • Queues, topics and consumer groups
  • What an event-driven design costs
  • The outbox pattern and replaying events