⏱ 21 min read
There’s a moment in nearly every microservices program when the team discovers an uncomfortable truth: the hardest dependency to break is not a library, an API, or a deployment pipeline. It’s the database. microservices architecture diagrams
People say they want independent services. What they often build is distributed code wrapped around a shared relational model. The services have separate repositories, separate containers, separate owners, and then—quietly, fatally—the same tables. A “customer” table here, an “orders” table there, and before long every service is peering through the same database keyhole. The architecture diagram says microservices. The runtime says coordinated monolith.
This is where data duplication stops being a smell and starts becoming a tool.
In enterprise architecture, duplication is usually treated like a sin to be confessed and eliminated. We are trained to normalize, centralize, and maintain one source of truth. That instinct is useful inside a bounded context. It becomes dangerous across bounded contexts. When different domains need to move at different speeds, serve different use cases, and uphold different invariants, forcing them to share one canonical schema creates coupling of the worst kind: hidden, brittle, semantic coupling.
Sometimes the right move is to copy the data.
Not casually. Not sloppily. Not as an excuse for design laziness. But deliberately, with domain semantics in mind, so that each service owns the data shape it needs and can evolve independently. Replicated data, done properly, is not redundancy for its own sake. It is a decoupling mechanism. It trades storage efficiency for autonomy. In modern distributed systems, that is often a bargain worth making.
This article digs into that bargain. We’ll look at why data duplication appears in healthy microservice architectures, how it fits with domain-driven design, where Kafka and event streams help, how progressive strangler migration works in practice, and where the whole idea falls apart. Because it does fall apart if you use it in the wrong place. event-driven architecture patterns
Context
Microservices are usually introduced for organizational reasons before they are introduced for technical ones. A company wants teams to deliver faster, reduce release coordination, isolate failures, or modernize an aging estate. The business language is agility. The architectural consequence is decomposition.
But decomposition is easy to fake.
A service is not independent because it has an HTTP endpoint. It is independent when another team can change its internals, deploy it, scale it, and reshape its data without forcing everybody else into a synchronized dance. In practice, that means data ownership matters more than service boundaries on a slide deck.
Domain-driven design gives us the language for this. Different bounded contexts may use the same words while meaning different things. “Customer” in billing is not “customer” in marketing. “Order” in fulfillment is not “order” in finance. The mistake is to see these as one enterprise entity with many consumers. The reality is that each context holds its own model, optimized for its own decisions and workflows.
That’s why the old dream of a single enterprise data model tends to fail in living organizations. It promises consistency and delivers negotiation overhead. Every schema change becomes political. Every new field becomes a committee discussion. Every team works around constraints introduced by another team’s needs. It is architecture by hostage situation.
Replicated data breaks that cycle. A service can ingest a subset of facts from another context and store them locally in its own schema, under its own lifecycle, indexed for its own queries. It no longer asks another service for permission every time it wants to think.
That local copy is not “the source of truth” in the universal sense. It is the source of truth for that service’s decisions.
And that distinction matters.
Problem
Shared databases and chatty runtime dependencies create a slow-motion failure pattern in microservices.
A service handling order placement needs customer credit status. Another needs shipping preferences. A third needs product availability. If every request reaches across the network to ask another service for basic reference information, latency rises, failure domains spread, and throughput falls. The architecture becomes a web of synchronous dependencies where one degraded service causes a traffic jam across the estate.
Teams then compensate in predictable ways. They add caches. They add retries. They add fallback logic. They add circuit breakers. These are all useful mechanisms, but they are often treating the symptom rather than the disease. The disease is that one service cannot do its job without consulting another service’s operational data in real time.
Worse, if services share the same database directly, they bypass each other’s APIs and business rules. That looks efficient until schema evolution becomes impossible. One team renames a column and breaks three others. One team adds a transaction assumption and deadlocks another. Nobody can tell where the contract really is because the contract is implicit in SQL scattered across the organization.
This is the architectural tax of centralized operational data in a distributed system. Every local optimization becomes a global coordination problem.
The microservice dream dies not with an outage, but with a meeting invite titled “cross-team database change review.”
Forces
This problem is pulled in several directions at once. Good architecture lives in those tensions, not outside them.
Autonomy versus consistency
A local replica gives a service autonomy. It can answer queries without network hops and survive upstream outages. But replicas are, by definition, behind reality at least some of the time. The question is not whether staleness exists. The question is whether the business can tolerate it.
A recommendation engine can live with a customer profile that is a few seconds old. Fraud detection often cannot. Shipment planning may tolerate eventual consistency for address enrichment, but not for hazardous goods classification.
Domain semantics versus enterprise standardization
Centralized data models feel clean. They also erase context. If every service must use the same “Customer” shape, then nobody gets the customer model they actually need. Domain-driven design pushes the other way: let each bounded context define its own language and structures. Replication supports this by allowing services to translate incoming facts into local meaning.
Read efficiency versus write complexity
Local copies make reads fast and resilient. But now writes have to propagate. This introduces event publishing, subscriptions, replay, reconciliation, and observability concerns. Reading gets simpler. Data movement gets harder.
Delivery speed versus operational burden
Teams move faster when they can evolve schemas independently. Yet every replicated dataset becomes a pipeline to monitor, backfill, and repair. You are buying decoupling with operational machinery.
Availability versus correctness guarantees
A service with local data can keep operating during upstream incidents. But if decisions require perfectly current state, availability gained through replication may come at the cost of business correctness. There is no free lunch here. There is only choosing which pain you want on purpose.
Solution
The core pattern is straightforward: each microservice owns its operational data store and keeps local copies of data from other bounded contexts that it needs to perform its responsibilities. Changes are propagated through events, logs, or controlled synchronization mechanisms. The receiving service stores the data in its own schema, not a cloned copy of the source database.
That last point is where many implementations go right or wrong.
Data duplication as decoupling is not database mirroring. It is semantic replication.
The upstream service publishes business facts: CustomerRegistered, CustomerCreditStatusChanged, ProductPriceUpdated, OrderShipped. The downstream service consumes those facts and updates its own model accordingly. It may keep only a subset of attributes. It may denormalize aggressively. It may merge several upstream event streams into one read-optimized structure. It may rename fields entirely to reflect local language.
That is healthy. In fact, it is the point.
A billing service may store customer payment risk, invoicing preferences, and tax identifiers. A support service may store contact preferences, loyalty tier, and the latest open orders. Both duplicate customer-related data. Neither is wrong. They are not trying to become the customer system of record. They are creating a local truth sufficient for their jobs.
Kafka often fits well here because it acts as the event backbone between services. A service emits domain events when significant state changes occur. Downstream consumers process those events asynchronously and update local stores. Kafka’s durable log also helps with replay and bootstrap scenarios, where a new service needs to rebuild its local projection from historical events.
Still, Kafka is not magic. If the publisher emits low-level table-change events instead of meaningful domain events, consumers become coupled to source internals. That’s just a shared database with extra steps. The architecture needs business events, stable contracts, and clear ownership.
Here is the basic shape:
This pattern is especially effective when the duplicated data is reference-like from the consumer’s perspective. Not static in the absolute sense, but stable enough that eventual propagation is acceptable for most decisions. Think catalog data, customer profile facets, account status, pricing snapshots, fulfillment capabilities, organizational hierarchies.
The service no longer performs synchronous lookups for every request. It carries what it needs, like a ship carrying provisions rather than dragging a pipe back to the harbor.
Architecture
A robust architecture for data duplication in microservices usually has five moving parts.
1. A clear system of record per bounded context
Each bounded context must have one service that owns authoritative writes for a given business concept in that context. Ownership is the anchor. Without it, duplicates become contested copies and reconciliation turns into politics.
This does not mean one universal source of truth for the whole enterprise. It means one owner for specific facts in a context. Customer legal identity may belong to customer management. Credit exposure may belong to billing. Marketing segmentation may belong to marketing.
That separation is pure domain-driven design. Different truths for different purposes.
2. Domain event publication
When business-significant changes occur, the owning service publishes events. The event should describe what happened in domain terms, not how a table changed.
Good:
- CustomerMovedAddress
- CreditLimitAdjusted
- ProductDiscontinued
Bad:
- customer_row_updated
- column_x_changed
The first gives semantic stability. The second leaks implementation detail.
3. Local projections or replicas
Consumers translate events into local tables, documents, or views. These local models should serve the service’s use cases. They are usually denormalized and query-oriented. This is where read optimization belongs.
4. Replay and backfill capability
A service must be able to rebuild its local replica from an event log, snapshot export, or source-owned feed. This is not optional. Without replay, a broken consumer forces dangerous manual repair.
5. Reconciliation mechanisms
Eventually consistent systems drift. Messages are delayed, consumers fail, schemas evolve, or historical data is corrected. Reconciliation detects and repairs divergence. This may involve periodic comparison jobs, checksums, snapshot reloads, or business-level audit reports.
A more detailed view looks like this:
There is a subtle but important architectural choice around how events are produced. In many enterprises, the outbox pattern is the practical answer. The source service writes business state and an event record into the same local transaction, then a relay publishes the event to Kafka. This avoids the classic dual-write problem where the database commit succeeds but the event publish fails—or vice versa.
Without that safeguard, replicated data starts with cracks in the foundation.
Migration Strategy
Most enterprises do not begin with clean bounded contexts and tidy event streams. They begin with a large application, one large schema, and many people who are already tired. The migration path matters more than the target picture.
The right migration strategy here is usually a progressive strangler. You do not rip out the central data model in one brave act. You gradually carve out capabilities, establish ownership, replicate what is needed, and tighten the seams over time.
A typical progression works like this.
Step 1: Identify a bounded context with painful read dependencies
Start where one emerging service repeatedly queries a shared database or makes synchronous calls for reference data. Order management consuming customer and product information is a classic candidate.
Step 2: Establish data ownership
Decide which service owns which facts. This sounds obvious; in enterprises it is often the hardest part. Teams discover that ownership was never explicit. Make it explicit.
Step 3: Publish events from the existing system
At first, this may come from the monolith. CDC tooling can help as a bridge, but only as a bridge. Use change data capture carefully to seed and bootstrap migration, while moving toward domain events as soon as feasible. Raw CDC is useful for plumbing, but weak on semantics.
Step 4: Build local replicas in the new service
Store only what the service needs. Resist the temptation to clone whole tables “just in case.” Fat replicas become shadow monoliths.
Step 5: Redirect reads
Once confidence is high, switch the service from shared database reads or remote lookups to its local store.
Step 6: Move writes to the owning service
This is the real cut. Reads are easy; writes define power. Over time, remove unauthorized writes from the old system and route changes through the new owner.
Step 7: Reconcile and retire old dependencies
Run comparison reports, fix edge cases, monitor lag, and then remove legacy joins and direct table access.
The migration often looks like this:
The strangler pattern is not merely technical sequencing. It is risk management. You keep old and new paths alive long enough to compare outcomes. You preserve delivery momentum while reducing coupling one dependency at a time.
A practical note: if you use CDC from the monolith during migration, be disciplined about where you stop. CDC can become a permanent crutch that exports internal table changes forever. That leaves downstream consumers coupled to the monolith schema, which defeats the point. Use CDC to get moving. Replace it with domain events to get free.
Enterprise Example
Consider a large insurance company modernizing its policy administration platform.
The estate started as you’d expect: one central Oracle database, a policy administration monolith, and a tangle of reporting and channel applications reading directly from shared tables. The company wanted independent services for policy quoting, billing, claims intake, customer service, and partner distribution. The first wave of service extraction looked successful on paper. Each new service had its own deployment pipeline and API.
Then the cracks appeared.
The claims intake service needed policy coverage details at claim time. The billing service needed customer correspondence preferences and policy status. The partner portal needed product eligibility and agent hierarchy. Rather than replicate data, teams made synchronous calls back to the monolith and, in some cases, were granted read access to legacy tables “temporarily.”
Temporary lasted two years.
Incidents told the real story. When the policy platform slowed during renewal peaks, claims and billing degraded too. A schema change for policy endorsements broke a partner report. Support teams spent nights tracing why a request took eight seconds only to discover three nested service calls and two database hops back into the monolith.
The company changed strategy.
They defined clearer bounded contexts:
- Policy service owned policy terms and coverage facts.
- Billing service owned invoices, balances, and payment status.
- Customer service owned party contact details and consent preferences.
- Claims service owned claim lifecycle and adjudication state.
Kafka became the event backbone. The policy service emitted events such as PolicyBound, CoverageChanged, PolicyCancelled. Customer service emitted AddressChanged and ConsentUpdated. Billing emitted PaymentFailed and AccountDelinquent.
Claims did not call policy synchronously for every intake screen. Instead, it maintained a local policy summary projection containing the subset of policy data relevant to claims initiation: active status, covered assets, deductibles, limits, and effective dates. Billing held its own customer contact projection for invoice delivery and collections workflows. The partner portal built denormalized views from multiple event streams.
This introduced duplication, undeniably. It also removed a huge amount of runtime coupling.
The operational result was more impressive than the slideware result. Claims intake remained available during policy-system maintenance windows. Billing no longer suffered because customer service had a temporary outage. Teams evolved schemas independently. The policy team could redesign internal persistence without forcing the claims team to rewrite queries against shared tables.
But there were costs. Reconciliation became a serious discipline. Historical policy corrections sometimes required replaying event streams or issuing compensating events. Teams had to learn that “customer address” did not mean the same thing everywhere: mailing address for billing, insured location for policy, contact address for service. Duplicating data forced them to confront semantics they had previously blurred.
That is one of the hidden virtues of this pattern. It reveals where the enterprise vocabulary was lying to itself.
Operational Considerations
Decoupling through duplication is won or lost in operations.
Event lag and observability
You need to know how stale local replicas are. Monitor consumer lag, processing latency, dead-letter volumes, replay duration, and projection health. A service depending on replicated data should expose freshness indicators so operators and even end users can understand whether they are looking at current-enough information.
Idempotency
Consumers must be able to process duplicate events safely. Kafka can deliver more than once depending on the setup and failure conditions. Projection updates should be idempotent or use version checks.
Ordering
Some business flows depend on event order. If a customer is created, updated, and then deactivated, consumers must not apply those transitions arbitrarily. Partitioning strategy, aggregate identifiers, and version sequencing matter. If you ignore ordering, your replicas become fiction.
Schema evolution
Events evolve. Consumers lag. Backward compatibility is not a nice-to-have. Use additive changes where possible, version carefully, and avoid coupling consumers to every field. Stable contracts keep autonomy intact.
Bootstrap and replay
New consumers need an initial state. You can replay from Kafka if retention and event completeness support it. Otherwise use snapshots plus event catch-up. Plan this from day one. Bootstrap is not an edge case; it is how your architecture handles growth.
Reconciliation
No matter how elegant the event flow, discrepancies will happen. Source data may be corrected manually. Consumers may miss events during outages. Business rules may change. Reconciliation should be designed as a first-class process:
- periodic compare jobs between source and replica
- checksums by aggregate or time window
- repair workflows with replay or snapshot refresh
- business audit reports for material mismatches
A system with duplicated data but no reconciliation strategy is not decoupled. It is just drifting.
Security and privacy
Duplicating data multiplies the surface area for sensitive information. Be ruthless about minimization. Replicate only what is required. Apply retention, masking, encryption, and access control at every replica. A copied customer profile is still regulated customer data.
Tradeoffs
This pattern solves real problems by introducing different ones.
The upside is powerful:
- fewer synchronous dependencies
- faster local reads
- better resilience to upstream outages
- autonomous schema evolution
- domain-specific models
- easier team independence
The downside is equally real:
- eventual consistency
- more moving parts
- reconciliation overhead
- duplicated storage
- harder debugging across asynchronous flows
- increased governance burden for sensitive data
The key tradeoff is this: you are moving complexity from request time to data movement time.
That is usually a good trade in large enterprises because request paths are where customers feel pain and outages spread fastest. But it is still a trade. You are not eliminating complexity. You are choosing where it lives.
A memorable rule of thumb: if two services need the same data for different decisions, duplicate it. If they need the same data for the same decision, stop and reconsider your boundaries.
Failure Modes
Architectures fail in patterns. This one is no exception.
1. Semantic cloning
A team copies the source schema wholesale into its local database. Now the replica is tightly coupled to source structure, not domain facts. Every source schema change ripples out. You have recreated the shared database with Kafka as decoration.
2. No clear ownership
Multiple services publish conflicting updates about the same fact. Consumers cannot tell which stream to trust. Reconciliation becomes impossible because there is no authoritative owner.
3. Dual-write inconsistency
The source service updates its database and publishes to Kafka in separate, unreliable steps. One succeeds, the other fails. Downstream replicas diverge. This is a classic preventable wound; use an outbox or equivalent transactional mechanism.
4. Infinite eventual consistency
Teams wave away all correctness concerns with “it’s eventually consistent.” That phrase has excused a lot of bad architecture. Some business rules have real freshness requirements. If stale data changes money, safety, legal commitments, or fraud exposure, “eventually” may be too late.
5. Missing replay strategy
A consumer fails, a topic retention window expires, and the local replica cannot be rebuilt. The service is now dependent on manual repair or direct database access to recover. That is not resilience.
6. Over-replication
Teams duplicate everything because storage is cheap. Soon there are dozens of stale copies of sensitive data, unclear deletion obligations, and spiraling operational complexity. Cheap storage is not cheap architecture.
When Not To Use
This pattern is useful, not universal.
Do not use replicated data as decoupling when the business operation requires strict, synchronous consistency across contexts. If a workflow must validate the current state at decision time and stale data is unacceptable, then either keep that logic within one bounded context or use synchronous coordination intentionally. High-value trading, real-time fraud authorization, hard inventory reservation, and some regulatory controls fall into this category.
Do not use it when your domain boundaries are still chaos. Replication amplifies ownership assumptions. If you have not clarified semantics and stewardship, copies will simply multiply confusion.
Do not use it for tiny systems with one team and one deployment unit. A modular monolith is often the saner answer. You can preserve domain boundaries without paying the operational tax of asynchronous replication.
Do not use it as a substitute for API design. Some teams replicate data because they cannot agree on service contracts. That usually produces stale local workarounds instead of healthy integration. API architecture lessons
And do not use it where data minimization rules make duplication disproportionate to the benefit. In some privacy-heavy environments, copying personal or financial data broadly across services is a governance nightmare. EA governance checklist
Related Patterns
Data duplication as decoupling sits alongside several adjacent patterns.
CQRS often appears here because local replicas are commonly read models optimized for a service’s queries.
Event sourcing can support replay and projection building, though you do not need full event sourcing to use replicated data effectively.
Outbox pattern is the practical backbone for reliable event publication from a transactional service.
Change Data Capture is often useful in migration, especially in strangler scenarios, but should not become a permanent substitute for domain events unless your use case truly is low-level replication.
Materialized view is a helpful analogy for local projections, though in microservices these views are owned per service, not globally.
Anti-corruption layer matters when consuming legacy or external events. A downstream service should translate foreign semantics into its own model rather than absorb them raw.
Together, these patterns form a style of architecture where each service is computationally and semantically self-sufficient enough to do its job.
Summary
The central insight is simple and unfashionable: in microservices, duplication is often cheaper than dependency.
Not all duplication. Not blind copying. Not the lazy spread of enterprise data into every corner of the platform. But deliberate, bounded, semantically aware replication that lets a service own its decisions without reaching across the network or into another team’s schema every few milliseconds.
Domain-driven design explains why this works. Bounded contexts are allowed to model the world differently because they serve different purposes. Migration strategy explains how to get there without setting the enterprise on fire. Kafka and event streams provide a practical transport for propagating facts. Reconciliation keeps reality and replicas from drifting too far apart. Operational discipline turns a smart diagram into a reliable system.
The tradeoff is clear. You give up the fantasy of one perfectly centralized operational model. In return, you gain resilience, autonomy, local performance, and room for teams to evolve.
That is a trade many enterprises should make.
The database, after all, is often the last monolith standing. If you want services that are truly decoupled, you eventually have to let them carry their own copy of the world.
Frequently Asked Questions
What is a service mesh?
A service mesh is an infrastructure layer managing service-to-service communication. It provides mutual TLS, load balancing, circuit breaking, retries, and observability without each service implementing these capabilities. Istio and Linkerd are common implementations.
How do you document microservices architecture for governance?
Use ArchiMate Application Cooperation diagrams for the service landscape, UML Component diagrams for internal structure, UML Sequence diagrams for key flows, and UML Deployment diagrams for Kubernetes topology. All views can coexist in Sparx EA with full traceability.
What is the difference between choreography and orchestration in microservices?
Choreography has services react to events independently — no central coordinator. Orchestration uses a central workflow engine that calls services in sequence. Choreography scales better but is harder to debug; orchestration is easier to reason about but creates a central coupling point.