⏱ 18 min read
There is a particular kind of disappointment that shows up a year after a microservices program starts. microservices architecture diagrams
The teams have split the monolith. They have Kubernetes clusters. They have Kafka topics with serious-looking names. They have service boundaries, API gateways, platform engineering decks, and a migration roadmap that once felt heroic. event-driven architecture patterns
And yet the weekly executive meeting still begins with the same question: why do the numbers not match?
Finance says revenue is one thing. Sales operations says another. The customer service dashboard has a third version. Data engineering has built a beautiful warehouse, but it behaves like an archaeological dig: every metric is an artifact, every report a debate about provenance. The organization has modern services, but the data platform still acts like a feudal kingdom with too many maps.
That is the hard truth: microservices solve operational decomposition far better than they solve analytical truth. They help teams ship. They help domains move at different speeds. They are often excellent for transaction processing. But if you split services without rethinking the semantics of data, the warehouse becomes the place where all your modeling sins come to be negotiated.
The industry told a flattering story for a while. Break the monolith, let every team own its schema, publish events, and let downstream consumers compose what they need. The story sounds elegant. In enterprise reality, it usually lands with a thud. Data platforms do not merely need data movement. They need meaning, lineage, reconciliation, and a durable answer to a deeply annoying question: what does “customer,” “order,” “booked revenue,” or “active account” actually mean?
This is where architecture becomes less about topology and more about language. Domain-driven design matters here, not as ceremony, but as survival.
Context
Most large enterprises did not arrive at microservices because they wanted a better data platform. They arrived there because the old application landscape had become intolerably coupled.
A monolith had too many release dependencies. Shared databases had turned into organizational tar pits. Every change crossed team lines. Scaling one hotspot meant scaling the whole thing. The answer, reasonably enough, was service decomposition: define bounded contexts, assign ownership, and let teams move independently.
That part often makes sense.
The mistake is assuming that the same decomposition automatically gives you a clean analytical architecture. It does not. In fact, it often creates the opposite. The service split reduces code coupling while increasing semantic fragmentation. Every service becomes an island of local truth. The warehouse then inherits the burden of stitching those islands into a continent.
You can see the pattern everywhere:
- Customer data split across CRM service, billing service, identity service, and support service
- Order lifecycle split across checkout, fulfillment, invoicing, returns, and subscriptions
- Revenue facts split across sales contracts, product usage, invoices, credits, and finance adjustments
- Event streams flowing through Kafka with immaculate schemas and deeply inconsistent business meaning
At this point, the enterprise discovers a painful asymmetry. Operational services can tolerate local definitions if their workflows are narrow enough. Data platforms cannot. They exist precisely to answer questions that cross those boundaries.
So the warehouse becomes the unofficial integration layer. Not because that was the strategy, but because no one else volunteered.
Problem
The problem is not simply that data is distributed. Distributed data is manageable. The problem is that the semantics are distributed and unstable.
A service boundary is a useful software design decision. It is not a universal truth boundary.
Take something as ordinary as “customer.” In one service, customer means an authenticated identity. In another, it means a billed legal entity. In another, it means a support contact. In marketing, it means a reachable profile. In analytics, someone eventually writes a “customer dimension” and quietly chooses one of these, then spends six months explaining exceptions.
That is warehouse coupling in disguise. Teams think they have independent services because they no longer share a schema. But the business still requires joined meaning. So the coupling moves downstream into ETL, semantic layers, materialized views, metric definitions, and endless reconciliation logic.
This is the hidden architecture:
On paper, the services are decoupled. In practice, the warehouse is tightly coupled to all of them because it must reconcile contradictory representations of the same business concepts.
This creates several common pathologies.
First, every analytics initiative becomes a semantic integration project. The data team is no longer modeling facts and dimensions. They are adjudicating business language.
Second, event-driven architecture gets overcredited. Kafka helps move data, buffer load, and preserve ordered streams in limited contexts. It does not resolve whether an “order completed” event means money recognized, goods shipped, payment settled, or simply a button was clicked and later reversed. A fast pipe does not make a shared meaning.
Third, ownership becomes muddled. Service teams own their transactional stores. Data teams own the downstream representations. But no one fully owns the cross-domain business truth. That gap becomes expensive.
Forces
There are real forces pushing organizations into this trap, and they are not irrational.
Team autonomy
Microservices exist for a reason. Independent teams, separate deployability, localized failure, and bounded context ownership are all good things. If every report requirement is allowed to punch through service boundaries and demand shared canonical schemas in the operational plane, autonomy disappears.
Analytical integration
The business needs cross-domain views. The CFO does not care that invoice, usage, and contract are different bounded contexts. The CFO wants recognized revenue by segment by quarter, and wants it before the board meeting.
Domain semantics
Bounded contexts deliberately allow different meanings in different places. That is healthy in software design. It becomes dangerous when analytics consumers assume those meanings are globally interchangeable.
Latency expectations
Operational systems often need immediacy. Analytical systems often tolerate delay, but only to a point. Near-real-time dashboards, fraud detection, operational reporting, and customer 360 applications all blur the line between analytical and operational workloads.
Compliance and lineage
Regulated industries need explainability. If metrics are composed from ten services and six event streams, you need lineage, auditability, and reconciliation. “It came off Kafka” is not a control framework.
Legacy gravity
The old warehouse, mainframe, ERP, and MDM systems do not disappear when microservices arrive. They remain authoritative for some domains, deeply inconvenient for others, and politically untouchable in at least one important revenue process.
These forces mean the answer cannot be “just centralize everything” or “just decentralize everything.” Both are slogans. Enterprises need a more deliberate design.
Solution
The right move is not to pretend the warehouse can disappear, nor to force one giant canonical model across all services. The move is subtler:
Design the data platform around domain semantics, with explicit reconciliation between bounded contexts, and treat the warehouse as a semantic integration product rather than a passive sink.
That sentence carries weight, so let’s unpack it.
First, use domain-driven design thinking seriously. Boundaries should reflect real business capabilities and language, not just code packaging. Order Management, Billing, Pricing, Identity, Product Catalog, and Fulfillment may each be valid bounded contexts. They should publish data as they understand it, with clear contracts and events.
Second, accept that cross-domain analytics is a separate concern with its own model. Do not smuggle it in as “raw tables in the warehouse” and hope BI will sort it out. Build an explicit semantic integration layer that maps, reconciles, and versions shared concepts like customer, order lifecycle, contract value, and revenue state.
Third, define where truth is local and where it must be reconciled. A billing invoice total may be authoritative in Billing. Shipment status may be authoritative in Fulfillment. “Net revenue recognized” is not authoritative in either alone. It is a reconciled business fact requiring rules across contexts.
Fourth, use Kafka and event streams as part of the architecture, not the whole architecture. Events are useful for propagation, decoupling, and replay. But analytical truth often depends on late events, corrections, idempotency, backfills, and semantic interpretation. The platform must handle all of that.
The shape looks more like this:
This architecture does not erase bounded contexts. It respects them. But it also acknowledges that enterprise analytics needs a designed place where meanings are aligned, exceptions are explicit, and reconciliation rules are first-class architecture.
That is the missing move in many microservices programs.
Architecture
A sound architecture for this problem usually has five layers.
1. Operational systems by bounded context
Each service owns its transactional model and lifecycle. This is where domain logic lives. This is where invariants belong. If Order Management has to reserve stock, create an order aggregate, and track state transitions, do it there. Do not export that complexity into SQL transformations as a substitute for application design.
This is classic DDD discipline. Keep the model rich where behavior lives.
2. Event and change propagation
Use Kafka, CDC, outbox patterns, or a combination. Here the goal is reliable propagation, not semantic perfection. Capture facts with enough metadata to support lineage, ordering, and replay.
A good event contract says:
- what happened
- in what bounded context
- with what business key
- at what effective time
- at what processing time
- with what version and causation metadata
A bad event contract says:
status_changed=true
You would be amazed how much enterprise confusion begins with lazy event naming.
3. Raw and history-preserving ingestion
Land data immutably where possible. Preserve source payloads, keys, timestamps, versions, and source lineage. This gives you replay, debugging, auditability, and the ability to repair downstream transformations.
If your warehouse only keeps current-state convenience tables, you have built a reporting appliance, not a trustworthy platform.
4. Domain-aligned analytical products
Before global integration, build analytical models aligned to bounded contexts: orders, invoices, shipments, identity profiles, subscriptions, claims, policies, trades, whatever your enterprise domains require.
This reduces the classic anti-pattern where every reporting need directly hits raw events. Domain-aligned data products are where local semantics are normalized and made analytically useful without pretending they are universal.
5. Reconciliation and semantic integration
This is the heart of the matter. Cross-domain facts must be assembled deliberately.
For example:
- link order placed to payment authorized to invoice issued to shipment delivered to return processed
- decide which state transitions matter for booked revenue, billed revenue, and recognized revenue
- map identity-level customer to legal-account customer and household customer depending on use case
- handle late-arriving corrections, duplicates, and compensating events
This is not plumbing. This is business architecture encoded in data form.
A useful mental model is this: bounded contexts create valid local truths; the semantic integration layer creates valid enterprise truths.
Service split vs warehouse coupling
The common misconception is that if services are split, coupling is reduced everywhere. In reality, coupling is often displaced.
The warehouse becomes a second monolith unless you architect it as a set of explicit semantic products with ownership and contracts.
Migration Strategy
This is not a “big redesign” problem. Big redesigns are where architecture goes to die under PowerPoint.
Use a progressive strangler migration.
Start by identifying one painful cross-domain capability where semantic confusion is costly. Revenue reporting is a classic candidate. Customer 360 is another, though it is often more political and less crisply bounded. Build the new semantic path beside the old warehouse logic, compare outputs, reconcile differences, and gradually shift consumers.
A practical sequence looks like this:
Step 1: Map bounded contexts and business concepts
Do event storming or a simpler workshop if your organization hates sticky notes. Identify core concepts, local meanings, authoritative systems, and the most disputed metrics. Write down where the same term means different things.
If you skip this, you are just automating ambiguity.
Step 2: Establish source-aligned ingestion
Introduce CDC or outbox/event publishing from key services. Preserve raw history. Add lineage metadata. You are building the foundation for reproducibility.
Step 3: Build one domain data product at a time
Model orders, invoices, subscriptions, claims, or whatever matters. Keep them faithful to source semantics. Do not prematurely force one canonical “golden record” for everything.
Step 4: Add reconciliation for one enterprise question
For example: “What is booked annual contract value?” or “What is net sales after returns?” Build explicit business rules. Version them. Compare them to current reports. Measure divergence.
Step 5: Run dual reporting
For a period, produce both old and new outputs. Reconciliation is not a one-time task. It is a discipline. Expect mismatches. Use them to discover hidden assumptions, source defects, and timing gaps.
Step 6: Migrate consumers incrementally
Move dashboards, finance packs, downstream ML features, and APIs to the new semantic layer one use case at a time. Deprecate old warehouse marts slowly and visibly.
Step 7: Formalize ownership
Assign domain teams responsibility for source data quality and event contracts. Assign a semantic platform team or federated data product owners responsibility for cross-domain models and reconciled metrics. Without ownership, architecture is theater.
This is the strangler fig pattern applied to data semantics rather than only application endpoints. The old warehouse remains standing while new trusted semantic products grow around the most important enterprise questions. Over time, the brittle center is enclosed and retired.
Enterprise Example
Consider a global subscription software company moving from a large CRM/ERP-centric estate to microservices.
They split into services for:
- Customer Identity
- Subscription Management
- Usage Metering
- Billing
- Collections
- Support
Kafka became the integration backbone. Every team published events. The platform team celebrated. Then quarter-end happened.
Sales reported bookings from Subscription Management. Finance reported invoices from Billing. Revenue accounting applied recognition schedules based on contract obligations and usage realization. Customer success measured “active customers” from Identity and Support engagement. The CEO saw four metrics that all sounded commercially important and none that agreed.
The company had not failed at microservices. It had failed at semantic architecture.
The rescue did not involve collapsing back to a monolith. It involved creating a data platform with:
- immutable ingestion from Kafka and ERP extracts
- domain data products for subscriptions, invoices, usage, and collections
- a reconciled commercial model distinguishing booking, billing, cash, and recognition
- a semantic customer model that explicitly separated identity, account, and legal entity
- a metric layer with versioned definitions and business sign-off
The biggest breakthrough was linguistic, not technical. They stopped saying “revenue” as if it were one thing. They started saying “booked revenue,” “billed revenue,” “collected cash,” and “recognized revenue.” Once the language got honest, the architecture followed.
That is DDD for the data platform in practice. Ubiquitous language is not just for code. It is for dashboards, board packs, and audit trails.
Operational Considerations
A design like this lives or dies operationally.
Reconciliation is permanent
Many teams talk about reconciliation as a migration task. That is too optimistic. In real enterprises, source defects, late events, manual adjustments, backdated corrections, and reference-data changes never stop. Reconciliation must be continuous, observable, and explainable.
Data quality must be measured at boundaries
Check completeness, freshness, uniqueness, referential integrity, and semantic validity as data enters the platform and as it moves into reconciled products. If invoices arrive without contract identifiers, no amount of dashboarding will save you later.
Time matters more than people think
You need multiple time axes:
- event time
- effective business time
- ingestion time
- processing time
Without them, restatements and backfills become guesswork. This matters especially with Kafka streams, CDC, and financial reporting periods.
Idempotency matters
Duplicate events happen. Replays happen. Downstream jobs rerun. If your semantic integration layer is not idempotent, quarter-close becomes a horror film.
Metadata is not optional
Lineage, schema evolution, ownership, data product contracts, and metric definitions should be visible and governed. The warehouse is full of haunted tables created by people who have since joined other companies.
Security and privacy become cross-domain concerns
A customer identifier that is harmless in one context may become sensitive when linked across several. Microservices do not reduce privacy obligations; they often increase re-identification risk in the data platform.
Tradeoffs
There is no free lunch here.
The upside of this approach is clarity. It preserves team autonomy in operational systems while creating a deliberate place for enterprise truth. It improves explainability, lineage, and trust. It also handles legacy coexistence better than naive “everything is an event” strategies.
But there are costs.
It introduces another layer to design and govern. Reconciliation logic can become sophisticated, especially in finance, healthcare, telecom, and supply chain domains. Some teams will complain that the semantic layer is “just another central team.” If done badly, they will be right.
There is also latency tradeoff. A fully reconciled fact may not be as immediate as a local event. That is acceptable for many business questions and unacceptable for some. You may need both fast local views and slower trusted enterprise views.
And there is a modeling tradeoff. If you over-centralize semantics, you suffocate bounded contexts. If you under-centralize, you create analytical chaos. The art is in deciding which concepts deserve enterprise reconciliation and which should remain explicitly local.
That is architecture. Not purity. Judgment.
Failure Modes
A few failure modes appear with depressing regularity.
The canonical model fantasy
Someone proposes a single enterprise-wide canonical schema for customer, order, product, and revenue. It looks tidy in a diagram and awful in production. Canonical models often erase real domain differences and create endless committee-driven compromises.
The raw zone religion
Another group says, “just land everything raw and let consumers self-serve.” This scales confusion better than it scales insight. Self-service without semantic curation is an elegant way to multiply inconsistent dashboards.
Event absolutism
Teams assume that because every change is emitted to Kafka, the platform has solved integration. It has not. Events preserve occurrences, not necessarily settled meaning.
Central team bottleneck
If one central data team must understand every domain and build every reconciliation, they become the new monolith. Federated ownership with strong standards usually works better.
Metric drift
Definitions change informally over time. Dashboards continue to use old logic. Trust evaporates quietly, then all at once.
No path for corrections
Real businesses restate, reverse, backdate, and amend. If the architecture assumes append-only happy paths without compensations and restatements, it will eventually lie.
When Not To Use
Do not build this level of architecture if your world does not need it.
If you are a small product company with one operational system and modest analytical needs, a simple warehouse with carefully managed transformations is probably enough. You do not need a semantic federation manifesto.
If your domain is operationally simple and analytics does not cross many bounded contexts, service-local reporting plus a conventional dimensional model may be entirely adequate.
If your organization lacks stable ownership and cannot even agree on source system accountability, adding semantic integration layers will just produce more diagrams than outcomes.
And if your microservices are not real bounded contexts but merely a distributed monolith carved by technical layer or team politics, fix that first. A bad service decomposition creates bad data products.
Related Patterns
Several related patterns are useful here.
Data Mesh, when interpreted sensibly, aligns well with domain-oriented data products. But it needs stronger semantic integration than some advocates admit.
CQRS helps when read models need to differ from write models, especially for operational analytics. It does not remove cross-domain reconciliation.
Event sourcing can provide rich histories, but it is not automatically a reporting strategy. You still need enterprise semantics.
MDM still has a role, especially for identities, parties, products, and reference data. But MDM is not a substitute for cross-domain metric reconciliation.
Outbox pattern is often the safest way to publish business events from transactional services without dual-write hazards.
Strangler fig migration remains the right mental model for replacing warehouse logic incrementally.
Summary
Microservices did not fix your data platform because they were never designed to.
They fix a different problem: deployability, team autonomy, operational scale, and bounded change. Those are worthy gains. Keep them. But do not confuse service boundaries with enterprise truth.
The data platform sits where local truths collide. That is why semantics matter more than topology. A warehouse, lakehouse, or streaming platform without explicit domain language and reconciliation is just a faster place to be inconsistent.
The better approach is to combine DDD and data architecture with a little more honesty: enterprise architecture with ArchiMate
- bounded contexts own local truth
- event streams propagate change
- domain-aligned data products preserve meaning
- a semantic integration layer reconciles cross-domain concepts
- metrics are versioned business products, not SQL accidents
- migration happens progressively, through strangler-style coexistence and reconciliation
In the end, the architecture question is not whether to choose microservices or the warehouse. That is too shallow.
The real question is this: where in your enterprise do meanings get resolved, and who is accountable when they do not?
If you cannot answer that, your platform is still coupled. You have simply moved the coupling to a quieter room.
Frequently Asked Questions
What is a data mesh?
A data mesh is a decentralised data architecture where domain teams own and serve their data as products. Instead of a central data team, each domain is responsible for data quality, contracts, and discoverability.
What is a data product in architecture terms?
A data product is a self-contained, discoverable, trustworthy dataset exposed by a domain team. It has defined ownership, SLAs, documentation, and versioning — treated like a software product rather than an ETL output.
How does data mesh relate to enterprise architecture?
Data mesh aligns data ownership with business domain boundaries — the same boundaries used in domain-driven design and ArchiMate capability maps. Enterprise architects play a key role in defining the federated governance model that prevents data mesh from becoming data chaos.