Dual Writes vs Outbox Pattern in Microservices

⏱ 20 min read

Distributed systems don’t usually fail with a bang. They fail with a shrug.

A customer places an order. The payment clears. The inventory reservation never appears. Support opens three screens, engineering opens five dashboards, and everyone asks the same tired question: “How can the database say one thing and Kafka say another?” event-driven architecture patterns

That question sits at the heart of one of the most common integration mistakes in modern microservices: dual writes. Teams split a monolith, adopt domain-driven design, add Kafka, and feel they’re becoming event-driven. Then they write to a service database and publish an event in the same request path as if the network were a local function call. It works in lower environments. It passes happy-path tests. It survives demos. And then production reminds everyone that two independent systems do not magically become atomic because a developer placed the lines next to each other.

This is why the outbox pattern matters. Not because it is fashionable architecture, but because it acknowledges a hard truth: persistence and message publication are different resources, with different failure characteristics, different recovery models, and different semantics. Good architecture starts when we stop pretending otherwise.

This article compares dual writes and the outbox pattern in microservices, especially in Kafka-based enterprise platforms. We’ll look at the real forces at play, the domain semantics that should shape the design, migration strategy for brownfield estates, operational tradeoffs, and where this pattern is simply the wrong tool. microservices architecture diagrams

Context

Microservices changed the shape of enterprise integration, but not the laws of distributed computing. We decomposed systems by business capability, gave each service ownership of its own data, and replaced direct database sharing with APIs and events. That was the right direction. It aligns with domain-driven design: each bounded context should own its model, its invariants, and the language it speaks.

But decomposition creates a new kind of tension. Business actions are no longer confined to one process. An OrderPlaced decision in the ordering context often needs to trigger work in payment, inventory, shipping, risk, customer notifications, and analytics. Those downstream capabilities increasingly expect events, often delivered through Kafka or another durable log.

So a simple flow emerges:

1. update the local database
2. publish an event

The code is trivial. The architecture is not.

In a monolith, a single ACID transaction made persistence and side effects feel synchronized. In a distributed estate, they are not. Database commits and broker publishes live in separate worlds. If you treat them as one operation, you are building a race condition into your business.

This is not merely a technical concern. It’s a domain concern. An event is not “some message we send after saving.” It represents a business fact that other bounded contexts may act upon. If the fact exists in one place but not the other, your enterprise has split reality.

That is the architectural issue.

Problem

Dual writes happen when a service updates its database and sends a message to a broker as two separate operations, usually inside one application method.

A typical implementation looks innocent:

• save order to database
• publish OrderCreated to Kafka
• return success

Or sometimes the reverse:

• publish event
• save database row

Both are flawed.

If the database commit succeeds and the event publish fails, the service state changes but the rest of the platform never hears about it. Inventory is not reserved. Fraud checks are not triggered. Downstream projections drift from reality.

If the event publish succeeds and the database transaction later rolls back, downstream systems act on a business fact that never really happened. That is worse. Now you have ghost events.

Developers often try to patch this with retries, local transactions, or clever exception handling. Those help, but they do not make two independent systems atomic. The nasty cases remain:

• process crashes after DB commit but before publish
• network timeout during publish where delivery outcome is unknown
• broker acknowledges, but producer cannot persist its own state
• DB rollback after event already left the building
• retry logic causing duplicates
• reprocessing causing out-of-order effects

In other words, dual writes create inconsistency windows and ambiguous outcomes. The problem is not programmer carelessness. The problem is architectural shape.

Forces

There are several forces pulling teams toward dual writes, and they are understandable.

Simplicity pressure

Dual writes are easy to explain, easy to code, and easy to ship. In early microservice programs, that matters. Teams are rewarded for delivery, not for elegantly handling a process crash between line 42 and line 43.

Domain pressure

Business workflows cross bounded contexts. An order without an event is incomplete from an enterprise perspective. So teams feel pressure to “just send the event right away.”

Latency pressure

Some domains need near-real-time propagation. Fraud scoring, inventory reservation, customer notifications, and SLA tracking all push for fast event publication. Architects then worry that introducing an outbox adds delay.

Usually that delay is measured in milliseconds or a few seconds. Usually that is acceptable. Usually the business never asked for “immediate,” only for “reliably soon.”

Autonomy pressure

Microservices should own their own data and avoid distributed transactions. That rules out many heavyweight coordination mechanisms. The architecture needs local autonomy without cross-system atomicity.

Compliance and audit pressure

In regulated environments, you need evidence of what happened, when, and in what sequence. Ambiguous outcomes are not just operational annoyances; they become audit findings.

Operational pressure

At scale, failures are not rare. They are ordinary. Broker partitions rebalance. Producers timeout. Databases fail over. Pods die. The design has to survive the weekday, not the perfect day.

These forces explain why this topic comes up in almost every serious event-driven modernization program.

Solution

The outbox pattern is the practical answer.

Instead of trying to update the business table and publish the event as two independent actions, the service writes both the business change and an outbox record in the same local database transaction. A separate publisher then reads the outbox and sends events to Kafka. Once published successfully, the outbox row is marked processed or removed.

This is the key move: make the only atomic step the one your service can actually control locally.

If the transaction commits, both the business state and the intent to publish exist durably. If the transaction fails, neither exists. That removes the worst class of split-brain outcomes.

A simplified sequence looks like this:

1. start DB transaction
2. update aggregate state
3. append outbox event row
4. commit DB transaction
5. background process publishes outbox row to Kafka
6. mark outbox row as published

This is not magic. It does not create global exactly-once semantics across your enterprise. But it does establish a reliable handoff boundary. And that is enough to build sane systems.

Here’s the comparison at a glance.

The outbox pattern works because it accepts eventual consistency while making inconsistency recoverable. That distinction matters. In enterprises, temporary delay is usually tolerable; unrecoverable ambiguity is not.

Architecture

There are a few architectural variants of the outbox pattern, and the right choice depends on your platform maturity.

1. Application-managed outbox

The service writes an outbox table and a scheduled publisher process reads pending rows, publishes to Kafka, and updates status.

This is straightforward and gives teams explicit control. It’s easy to reason about, easy to test, and often good enough for many domains.

The downside is polling overhead, duplicate handling complexity, and bespoke implementation across teams if no platform standard exists.

2. Change Data Capture with Debezium

A more industrial approach is to write outbox rows only, then use change data capture tooling such as Debezium to stream outbox inserts from the database transaction log into Kafka.

This reduces custom polling code and often scales better organizationally. It also creates a cleaner separation between transactional service logic and integration publication.

But it introduces platform dependencies, operational complexity, and requires careful governance over schemas, topic contracts, and connector operations. CDC is not “free reliability.” It just moves the machinery into the platform layer. EA governance checklist

3. Embedded transaction log publishing

Some databases and frameworks offer direct support for transaction log tailing or event emission. These can be elegant, but they often tie you tightly to a stack. In large enterprises, stack coupling becomes tomorrow’s migration problem.

Here is the basic outbox flow.

Domain semantics matter

A common mistake is to treat the outbox as a technical sidecar full of generic payloads. That misses the point.

Events should reflect domain semantics, not CRUD noise.

OrderPlaced means something.

OrderUpdated often means nothing.

In domain-driven design terms, an outbox event should represent a meaningful domain event emitted by an aggregate or application service at a business boundary. It should be understandable to downstream bounded contexts without leaking internal persistence details.

That means the design needs discipline around:

• event naming
• aggregate identity
• versioning
• idempotency keys
• business timestamps
• correlation and causation IDs
• payload shape and schema evolution

The outbox is not just a reliability mechanism. It is an integration contract boundary.

Ordering and partitioning

If events for a given aggregate must be processed in order, Kafka partitioning strategy matters. Usually the aggregate ID should be the partition key. Otherwise, you can publish valid events and still break business behavior through reordering.

For example, InventoryReserved arriving before OrderPlaced might be harmless in one system and catastrophic in another. Architecture lives in those details.

Idempotency is still required

The outbox pattern prevents lost publication intent, but it does not prevent duplicates. Publishers can retry. CDC connectors can replay. Consumers can crash after processing and before offset commit.

So consumers must be idempotent where the business requires it. If shipping creates a parcel twice for the same order because it trusted infrastructure for exactly-once semantics, that is not a messaging problem. That is a service design problem.

Memorable line: The outbox gives you durable intent, not absolution from distributed systems.

Migration Strategy

Brownfield enterprises rarely get to stop the world and rebuild event publication the right way. They have existing services, inconsistent messaging styles, and a long tail of critical flows running on luck and retries.

So the migration needs to be progressive. This is where the strangler pattern earns its keep.

Start by identifying the business flows where split writes are expensive: order capture, payment settlement, policy issuance, claim registration, customer onboarding. Not every service deserves immediate treatment. Focus on the workflows where inconsistency creates financial loss, customer harm, or manual reconciliation.

A sensible migration path looks like this:

Phase 1: Observe the dual-write hotspots

Instrument current services to detect mismatches between database state and published events. Build reports for:

• records created with no corresponding event
• events with no corresponding record
• delayed publication beyond SLA
• duplicate events
• consumer reconciliation exceptions

Don’t migrate blind. Measure the failure shape first.

Phase 2: Introduce outbox alongside existing service logic

Modify the service so that instead of publishing directly to Kafka, it writes domain events into an outbox table within the same transaction as the aggregate state change.

Keep the old publication path temporarily if necessary for low-risk shadowing, but be careful: running both in parallel can create duplicate events unless carefully isolated.

Phase 3: Add a publisher or CDC connector

Stand up a dedicated outbox publisher. In lower maturity environments, a polling publisher is often the fastest route. In platform-heavy estates, Debezium-based CDC can become the standard.

Phase 4: Reconcile and compare

Before cutting over fully, reconcile old direct-publish output against outbox-published output. This matters in enterprise settings because migration defects are often semantic, not technical. The event might publish, but with the wrong business meaning, wrong key, or wrong timing.

Phase 5: Strangle direct publishing

Once confidence is established, remove direct broker publishing from the service request path. The application should only perform local transaction work and rely on outbox-based publication.

Phase 6: Standardize platform contracts

Create enterprise guidance for outbox schema, event envelope, retention, retry policy, dead-letter handling, and observability. Without this, every team invents its own pattern and operations inherit a zoo.

Here is a progressive strangler migration.

Reconciliation is not optional

Enterprises often underestimate reconciliation. They assume the outbox will eliminate mismatches entirely. It won’t. It will reduce a dangerous class of inconsistencies, but downstream consumers can still fail, retry, duplicate, or lag.

You still need reconciliation processes for critical domains.

For example:

• compare orders marked placed with inventory reservations received
• compare payments settled with accounting journal entries emitted
• compare policy issuance with customer notification delivery

Reconciliation is the institutional acknowledgment that distributed systems are messy. Mature architecture does not deny this. It operationalizes it.

Enterprise Example

Consider a global retailer modernizing its order management platform.

The original monolith handled order capture, payment authorization, inventory allocation, shipping orchestration, and customer email in one database. During the microservices split, the retailer created separate services for Order, Payment, Inventory, Shipping, and Notification, with Kafka as the event backbone.

The first implementation used dual writes inside the Order Service:

• insert order row in PostgreSQL
• publish OrderPlaced to Kafka

It worked well enough in testing. In production, under peak traffic and broker turbulence during partition rebalances, the service began producing a small but painful class of failures: orders committed in the database without corresponding OrderPlaced events. Support saw paid orders stuck in “created” state with no reservation and no fulfillment. A nightly batch tried to detect them. Customers noticed first.

The retailer considered distributed transactions and quickly backed away. Too much coupling, too much fragility, and Kafka was not going to become a participant in some grand enterprise XA fantasy.

So they moved to an outbox pattern.

The Order aggregate commit now wrote two things atomically:

• the order state
• an outbox row containing OrderPlaced, aggregate ID, order version, correlation ID, and event payload

Debezium streamed the outbox table into Kafka topics. Inventory consumed OrderPlaced and reserved stock. Payment subscribed for settlement orchestration. Notification handled customer emails independently.

The interesting part wasn’t the mechanics. It was the semantic cleanup.

The team discovered they had been publishing events that mirrored persistence operations rather than domain facts. OrderStatusUpdated was replaced by specific business events such as:

• OrderPlaced
• OrderCancelled
• OrderReadyForFulfillment

That change reduced downstream confusion dramatically. Inventory did not need every mutation; it needed the business moments that mattered to inventory.

The migration was done gradually. High-value channels moved first: web commerce, then mobile, then partner orders. For six weeks the platform ran reconciliation jobs comparing:

• committed orders in the Order DB
• emitted outbox events in Kafka
• inventory reservations
• payment authorizations

They found edge cases around duplicate publication after publisher retries, which they solved with idempotent consumer logic keyed on event ID and aggregate version. They also discovered stale topic consumers in one region lagging enough to create visible customer delay, which led to better lag alerting and topic partition redesign.

The result was not theoretical perfection. It was something much more valuable: operationally trustworthy behavior under real-world failure.

That is what enterprise architecture is for.

Operational Considerations

The outbox pattern shifts complexity from application code to system operations. That is usually a good trade, but only if you acknowledge it.

Outbox table growth

If you never purge or archive published rows, the outbox becomes a landfill. Indexes bloat. Queries slow down. Vacuum jobs sulk. You need retention policies.

Common options:

• soft mark as published, then archive or delete after retention window
• partition the outbox table by date
• move published records to audit storage
• use CDC plus compact retention strategy elsewhere

Publisher behavior

A polling publisher needs careful tuning:

• batch size
• polling interval
• locking strategy
• parallelism
• backoff and retry policy

Too aggressive and you hammer the database. Too timid and business latency suffers.

Monitoring

At minimum, monitor:

• oldest unpublished outbox record age
• outbox row count growth
• publish success/failure rate
• retry counts
• Kafka topic lag
• DLQ volume
• reconciliation mismatch counts

If you cannot answer “how long from local commit to broker publish?” you are not operating an outbox pattern. You are hoping one exists.

Schema evolution

Event payloads evolve. Consumers lag. Contracts live longer than anyone expects. Use explicit schema governance, versioning, and compatibility checks. This is especially true in Kafka ecosystems with many consuming teams. ArchiMate for governance

Security and privacy

Outbox payloads may contain customer or regulated data. Since the outbox persists integration events in the service database, ensure encryption, masking, access controls, and retention align with policy. Architects often focus on reliability and forget that they have created a second store of business data.

Multi-region realities

In multi-region deployments, think carefully about where the outbox lives, where the broker lives, and what failover means. Active-active patterns complicate ordering, deduplication, and reconciliation. There is no free lunch here.

Tradeoffs

The outbox pattern is better than dual writes for many business-critical microservice integrations. But “better” is not “free.”

Advantages of dual writes

• very simple implementation
• minimal moving parts
• lower perceived latency in trivial cases
• acceptable for low-value, non-critical notifications

Disadvantages of dual writes

• non-atomic across DB and broker
• difficult recovery from partial success
• ambiguous failure outcomes
• higher reconciliation burden
• hidden business inconsistency

Advantages of outbox

• atomic persistence of business state plus publication intent
• recoverable publication pipeline
• cleaner audit trail
• better fit for event-driven microservices
• easier platform standardization

Disadvantages of outbox

• additional storage and operational machinery
• eventual consistency rather than immediate publication
• duplicate handling still required
• publisher or CDC complexity
• possible increased end-to-end latency

The critical tradeoff is this: dual writes optimize for coding convenience; outbox optimizes for survivability.

For serious business workflows, survivability usually wins.

Failure Modes

No pattern is complete until we discuss how it fails.

Outbox row written, publisher down

Business state commits successfully, but events remain unpublished until the publisher recovers. This is usually acceptable if lag is monitored and the business tolerates delay.

Event published, row not marked published

The publisher crashes after sending but before updating outbox status. On restart, it republishes the same event. This is why consumers must be idempotent.

Polling contention

Multiple publisher instances may compete for rows and cause duplicates or lock contention if the selection strategy is weak. Use robust claiming semantics.

CDC connector lag or outage

With Debezium or log-based capture, transaction log streaming may lag or fail. Publication is delayed, and consumers see stale state. Operations must treat CDC as production-critical infrastructure, not a side project.

Out-of-order event emission

If the publisher reads rows in the wrong order, or partitioning is inconsistent, downstream services may process domain events out of sequence. Aggregate versioning and partition key discipline help.

Poison events

A malformed payload or incompatible schema can cause repeated publish or consumer failure. Dead-letter strategy and schema validation are required.

Reconciliation backlog ignored

The most dangerous failure mode is organizational: teams assume the pattern solved consistency permanently and stop reconciling critical flows. Then a new downstream failure creates business drift that nobody notices until finance does.

Architecture fails socially before it fails technically.

When Not To Use

The outbox pattern is not mandatory everywhere. Good architects know when to leave well enough alone.

Do not use it when:

The event is non-critical and lossy by design

If you are emitting best-effort telemetry, UI hints, or disposable analytics signals, dual writes or even fire-and-forget publication may be acceptable. Don’t drag every low-value signal through heavyweight reliability machinery.

The service is not truly event-driven

If no downstream bounded context depends on the event for business behavior, an outbox may be unnecessary. Not every integration is a domain event.

The database is not under service ownership

If the service doesn’t own its persistence model cleanly, bolting on an outbox can become ugly and politically brittle. Fix ownership boundaries first.

A simpler synchronous API is the right integration

Sometimes another service needs a direct answer now, not eventual notification later. For command-style interactions with tight consistency requirements, a synchronous call may be more honest than pretending everything should be an event.

The platform cannot operate the pattern reliably

A badly operated outbox is better than dual writes only on paper. If the enterprise cannot support CDC, publisher monitoring, retention, and schema governance, start with a simpler and well-managed polling approach or defer the pattern to the critical flows only.

This is one place where architectural maturity matters more than architectural purity.

Related Patterns

The outbox pattern rarely lives alone.

Inbox pattern

Consumers record processed message IDs to enforce idempotency and prevent duplicate handling. Outbox on the producer side, inbox on the consumer side, is a powerful combination.

Saga pattern

For long-running business transactions across bounded contexts, events from the outbox often drive saga orchestration or choreography. The outbox makes saga triggers more reliable; it does not replace saga logic.

Event sourcing

People often confuse event sourcing with outbox. They are not the same. Event sourcing makes the event log the source of truth for domain state. The outbox is a reliable publication mechanism from a conventional transactional model. They can coexist, but one does not imply the other.

Change Data Capture

CDC is often the implementation mechanism for the outbox, not a domain pattern by itself. Dumping table changes directly into Kafka is not automatically good event design. Domain semantics still matter.

Strangler fig pattern

For migration from monolith or fragile dual-write services, the strangler pattern provides the gradual path. Modernization succeeds in slices, not in declarations.

Summary

Dual writes are appealing because they look simple. But in microservices, especially Kafka-based enterprise platforms, they create a dangerous illusion of atomicity. A database commit and a broker publish are separate acts with separate failure modes. Treating them as one operation is how systems drift into inconsistency.

The outbox pattern is the more mature answer. It writes business state and publication intent in one local transaction, then publishes asynchronously through a reliable handoff. That does not erase the distributed nature of the system. It does something better: it makes inconsistency manageable, observable, and recoverable.

The deeper lesson is not just technical. It is about domain-driven design. Events should represent business facts, not database twitching. Boundaries should reflect bounded contexts. Migration should be progressive, strangling risky paths rather than betting on rewrites. Reconciliation should exist because enterprises live in the real world, where failures are routine and certainty is expensive.

If the business flow matters, if downstream services act on the event, and if inconsistency causes customer pain or financial risk, choose the outbox pattern over dual writes.

Not because it is elegant.

Because it tells the truth about the system you actually have.

Frequently Asked Questions

What is a service mesh?

A service mesh is an infrastructure layer managing service-to-service communication. It provides mutual TLS, load balancing, circuit breaking, retries, and observability without each service implementing these capabilities. Istio and Linkerd are common implementations.

How do you document microservices architecture for governance?

Use ArchiMate Application Cooperation diagrams for the service landscape, UML Component diagrams for internal structure, UML Sequence diagrams for key flows, and UML Deployment diagrams for Kubernetes topology. All views can coexist in Sparx EA with full traceability.

What is the difference between choreography and orchestration in microservices?

Choreography has services react to events independently — no central coordinator. Orchestration uses a central workflow engine that calls services in sequence. Choreography scales better but is harder to debug; orchestration is easier to reason about but creates a central coupling point.