Playbook

CQRS (Command Query Responsibility Segregation) in Distributed Systems: Events, Brokers, and Projections (CQRS In Distributed Systems Events Brokers And Projections)

How does CQRS work in distributed systems? A practical guide to domain events, message brokers, projections, Outbox, idempotency, and eventual consistency.

Distributed CQRS event flow from command to projections and read models

The problem we are solving first

An order has been created. Inventory must know, notifications must send an email, analytics must prepare a report, and search must index the new order.

Order Service
  → HTTP call for Inventory
  → HTTP call for Notifications
  → HTTP call for Analytics
  → HTTP call for Search

Should the Order Service call each of them over HTTP? Every new capability expands the call chain, failure surface, and pressure to deploy together. No. The Order Service should share the business fact that happened; how it is used belongs to each consumer.

The first two chapters separated how a request moves through the command and query sides. In a distributed system, a new question appears: after an order is accepted, how do inventory, notifications, analytics, and search learn about that change safely?

POST /orders → Command Handler → OrderPlaced event → Outbox → Relay → Message Broker
  ├─ Inventory projection
  ├─ Notification consumer
  ├─ Analytics projection
  └─ Search read model

An event is an immutable business fact that happened; OrderPlaced is not an intention. A broker transports it from producers to consumers. A consumer processes it for its own responsibility. A projection builds a read model for a particular question.

Concepts at first use

📦 Domain event
An immutable business fact, named in the past tense.

📦 Broker
The communication layer that durably distributes a message to consumers.

📦 Projection
A read model built from events for a specific screen or query.

📦 Idempotency
The property of not changing the result when the same event is handled twice.

A command says PlaceOrder; it asks the system to act. If it succeeds, it may publish OrderPlaced. A command can be rejected, while an event is a fact you can no longer change.

Broker delivery is often at least once. Consumers therefore need to retain event identity or make their operation naturally idempotent.

if processedEvents.contains(event.id): return
applyProjection(event)
markProcessed(event.id)

This is normally an indexed O(1) average lookup or O(log n) B-tree lookup. The operational cost of a duplicate order is much higher.

Why the design looks this way

PlaceOrder  → an intent.
OrderPlaced → a fact that happened.

We make this distinction because a command can fail, while an event is a completed fact other systems can safely react to.

  • Outbox is needed because the broker and database do not share one ACID transaction.
  • A projection is needed because an admin panel, mobile app, dashboard, and analytics need the same order in different shapes without loading the aggregate.
  • Idempotency is needed because a broker can redeliver the same message for reliable delivery.
  • A saga is needed because inventory must not remain reserved forever when payment fails in e-commerce.
  • Most applications that use CQRS do not use Event Sourcing; they are independent decisions.

What changes when a service boundary is crossed?

Inside one application, a transaction, a database write, and side effects can often be coordinated locally. Once the system is distributed, inventory, email, and reporting own their own life cycles. A synchronous HTTP call can look easy at first, but every new call adds latency, failure propagation, and pressure to deploy together.

An event-driven flow moves that dependency to a data contract. The order service owns the schema of OrderPlaced; the inventory service consumes only the part it needs. The producer does not know the consumer's database or availability. Independent deployability without independent observability only moves risk elsewhere.

Safely leaving the transaction boundary

The naive flow is:

Save order → Commit → Publish OrderPlaced

If commit succeeds and publishing fails, the order exists while other services do not know about it. Publishing first and then failing the commit creates a ghost event. The Outbox pattern writes both facts in one local transaction:

Transaction
  → write Order
  → write OrderPlaced to Outbox
  → commit

Relay
  → deliver Outbox record to broker
  → mark it delivered

Outbox does not create a distributed transaction. It guarantees the critical fact: if the order is in the database, an event exists to be published. The relay may retry, which is why consumer idempotency remains mandatory.

CDC can capture changes from a database log. It is powerful for propagating existing data changes, but it gives the application less control over which changes carry domain meaning. Outbox makes that choice explicit.

Question CDC Outbox
Choose events in domain language Limited Explicit
Bind to the application transaction Indirect Direct
Consumer idempotency needed Yes Yes

Projection: not a copy, a purposeful view

A read model is not an incomplete copy of the write model. ProductSearchRow can serve a product list; OrderFulfilmentSummary can serve an operations screen. The same event can feed different projections for different teams.

OrderPlaced → OrderSummaryProjection → { orderId, customerName, total, status }
OrderPlaced → InventoryProjection → { sku, reservedQuantity, availability }

Projections should be rebuildable. When code changes or a defect is repaired, replay the event stream in a controlled way and validate the new read model. That requires ordering, checkpoints, versioning, and measured replay speed. Is it acceptable for an order to appear in a list a few seconds after it is placed? The answer is a product decision, not merely a technical one.

A broker is not a brand choice

A broker provides guarantees around ordering, durability, consumer groups, replay, and failure queues. If order matters for one key, design a partition key. If consumers can fall behind, monitor lag and define retry and dead-letter behavior. If schemas change, keep versions backward compatible: add fields deliberately and do not remove old fields casually.

Event Sourcing and CQRS are not the same thing

CQRS separates read and write responsibilities. Event Sourcing derives aggregate state from an append-only event stream rather than from a current row. They can be combined, but CQRS does not require Event Sourcing and Event Sourcing does not require Kafka. With Event Sourcing, snapshots, stream versions, and replay cost need their own design.

Saga: the compensation decision in a distributed workflow

An order can cross payment, inventory, and shipment; those steps do not fit in one ACID transaction. A saga defines a compensating step for each local transaction.

Order placed → reserve inventory → capture payment → create shipment
payment fails → release inventory → mark order failed

Choreography lets services trigger each other with events; local coupling is low, but the complete flow is harder to follow. Orchestration makes the flow visible through a central process manager while adding coordination responsibility.

Operational checklist

  1. Is every event named in the past tense and in the language of the business?
  2. Does Outbox close the gap between a database write and event publication?
  3. Is each consumer safe against duplicates, reordering, and late events?
  4. Can you observe projection checkpoints, lag, and the rebuild procedure?
  5. Are owner, version strategy, and backward compatibility explicit for every event contract?
  6. Has the product accepted the eventual-consistency window visible to users?

Without those answers, a system may look event-driven while behaving failure-driven.

Commonly confused distinctions

❌ Event = Command
✓ A command is intent; an event is a fact that happened.

❌ Broker = Event Store
✓ A broker transports events; an Event Store can be the durable source of domain history.

❌ Projection = cache
✓ A cache is temporary for speed; a projection is an intentional read model for a business query.

❌ At-least-once = error
✓ Redelivery is normal; consumers must be idempotent.

A real end-to-end flow

POST /orders
  → Controller → Mediator.Send()
  → Validation + Authorization → Transaction Behavior
  → PlaceOrderHandler → Order aggregate → Order + Outbox commit
  → Relay → Broker → Projection consumer → Read database
  → GET /orders → Query Handler → OrderSummaryDto → Frontend

When should I use this model?

Start with logical CQRS: name commands in the language of the business, shape queries for the screen, and make the transaction boundary explicit. Add a broker and separate projections only when independent consumers, asymmetric read load, or replayable integration needs are demonstrated.

Every new consumer can look like an O(1) code change, but its operational cost is not constant: it needs a contract, dashboard, alert, retry policy, owner, and test scenarios.

Reflection

The promise of distributed CQRS is not more messages; it is more visible responsibilities. Events carry history, the broker carries flow, and projections carry the result a user sees.

An event flow becomes an architectural decision only when it remains safe when messages arrive twice, arrive late, and are replayed.

The next chapter examines what changes when the system breaks in production: consistency, duplicates, corrupted projections, and recovery strategies.

What should remain with you?

If you remember only five things:

  1. A command requests behavior.
  2. An event is a fact that happened.
  3. A broker carries events to the right consumers.
  4. A projection reads the same fact in the shape each screen needs.
  5. Outbox prevents event loss because the broker and database do not share one ACID transaction.

Every separation in this chapter has a reason: idempotency is needed because a broker can redeliver; a projection is needed because querying an aggregate for every screen is expensive and the wrong abstraction.

FAQ

Frequently asked questions

What is Domain event?

An immutable business fact, named in the past tense.

What is Broker?

The communication layer that durably distributes a message to consumers.

Is it true that "Event = Command"?

A command is intent; an event is a fact that happened.

Engineering Principles Learned

  • In a distributed system, an event is not a command but an immutable business fact.
  • At-least-once delivery requires an idempotent consumer; duplicates are a design input, not an exception.
  • A projection must be rebuildable, measurable, and have its latency defined in product language.

Continue reading

Continue reading

Related articles

Related articles

Related articles

Paylaş