⏱ 20 min read
Distributed systems don’t fail because we lacked boxes and arrows. They fail because we treated architecture like a catalog of components instead of a system of constraints.
That distinction matters.
A monolith can survive years of bad modeling because everything is close together. One database. One transaction boundary. One deployable unit. The mistakes are still there, but they are padded by proximity. In distributed systems, proximity disappears. Every ambiguity in the domain becomes a timeout. Every hidden assumption becomes a broken contract. Every convenient shortcut grows teeth.
So when people ask for the “right” architecture for microservices, Kafka, event-driven systems, or enterprise integration, I usually give an answer they don’t love at first: architecture is not primarily about choosing technologies. It is about deciding which things the system must not be allowed to do. event-driven architecture patterns
That is the job. A good architecture is a constraint system.
It constrains which services may own which facts. It constrains where transactions can happen. It constrains how models evolve, how teams interact, how failures are absorbed, and how truth is reconstructed after things inevitably drift apart. In a distributed estate, these constraints are not bureaucratic overhead. They are the steel frame of the building. Remove them and the glass still shines, right up until the first strong wind.
This article explores architecture as a constraint system in distributed systems: what problem it solves, how domain-driven design sharpens the boundaries, how Kafka and microservices fit into the picture, how migration works with a progressive strangler approach, where reconciliation belongs, and when this style of architecture is simply the wrong answer. microservices architecture diagrams
Context
Enterprise architecture has spent too long pretending that freedom is the same as agility. It isn’t. Unlimited freedom in a distributed system is just ungoverned coupling.
You see it in organizations that “modernized” by creating dozens of services while keeping the old habits: shared databases, duplicated business logic, synchronous chains for every request, and event streams carrying facts no one can quite define. They didn’t build a distributed architecture. They distributed their confusion.
A healthier view starts with a simpler idea: every system is shaped by forces that pull in different directions. Speed versus consistency. Team autonomy versus central governance. Local optimization versus global coherence. Rich domain semantics versus generic integration. Compliance versus developer convenience. You cannot erase these tensions. You can only make them explicit and design around them. EA governance checklist
That is why I like the phrase constraint system. It makes architecture sound less like artistic expression and more like engineering. Bridges stand because constraints are respected. Distributed systems survive for the same reason.
Domain-driven design is particularly useful here, not because bounded contexts are fashionable, but because they provide a language for deciding where constraints should sit. In a distributed enterprise, the most expensive failure is not a crashed pod. It is semantic drift. One service says “customer” and means legal entity. Another means account holder. A third means billing party. Data still flows. Reports still render. But the business starts arguing with its own software.
Once semantics are unstable, every integration becomes political.
Problem
The core problem is deceptively ordinary: how do we preserve business meaning while splitting behavior across independently deployable parts?
Most distributed architecture discussions start too low in the stack. They begin with API gateways, message brokers, service meshes, or streaming platforms. Those things matter, but they are not the root problem. The root problem is that distribution breaks the comforting illusion of one system with one truth moving in one transaction.
In a distributed environment, truth is fragmented. Ownership is fragmented. Time is fragmented. Different parts of the system see different versions of reality at different moments. Some of that is intentional. Some of it is accidental. Architecture has to decide which is which.
A common failure pattern goes like this:
- teams break a monolith into services by technical layer or UI function
- multiple services need the same business data
- they either share the same database or clone each other’s tables
- updates happen through a mixture of direct calls and events
- reporting starts disagreeing with operational screens
- reconciliation becomes a permanent department rather than a controlled mechanism
The architecture looked modular on a slide. In practice, it created a constraint vacuum.
A better question is not “how many services should we have?” It is “what constraints must exist so that business meaning survives distribution?”
That leads to a different set of design concerns:
- Which bounded context owns which decisions?
- Which data is authoritative, and where?
- What must be strongly consistent, and what can be eventually consistent?
- Which events are domain events versus integration events?
- How are invariants enforced when there is no global transaction?
- How does the system detect and repair divergence?
These are architectural questions in the old-fashioned sense. They shape the structure of the whole.
Forces
Constraint systems exist because the forces are real.
Team autonomy vs enterprise coherence
Microservices promise independent teams moving quickly. Enterprises need coherence across channels, products, regulation, and reporting. Let every team invent its own model and you get semantic entropy. Centralize everything and you get a queue disguised as governance. ArchiMate for governance
The answer is not total freedom or total standardization. It is selective constraint: shared meaning where the business demands it, local autonomy where it does not.
Consistency vs availability
If a payment is captured, inventory reserved, and a shipment created across three services, what counts as success? In a monolith this could be one transaction. In distributed systems it usually cannot. So architecture must declare where immediate consistency is mandatory and where eventual consistency is acceptable.
This is not merely a technical tradeoff. It is a domain decision. “Customer can see order immediately” is a product statement. “Ledger entries must balance exactly once” is an accounting statement. Different parts of the same enterprise live under different consistency rules.
Reuse vs ownership
Shared services and common platforms sound efficient. They often produce a different result: no one truly owns the business outcome. In a constraint-driven architecture, ownership must be crisp. Reuse is allowed, but authority is not ambiguous.
Rich domain semantics vs generic integration
Enterprises love canonical models. They also suffer from them. A canonical model often becomes a lowest-common-denominator language that strips away domain meaning in the name of interoperability. The result is not harmony. It is widespread vagueness.
Sometimes translation between bounded contexts is healthier than forced uniformity.
Flow efficiency vs failure isolation
Long synchronous call chains make a system feel immediate until one service degrades and drags half the estate with it. Event-driven designs improve decoupling but introduce lag, duplicate handling, and the need for reconciliation. There is no free lunch. The architecture should place constraints so the failure blast radius is understandable.
Solution
The solution is to treat the architecture as a network of explicit constraints grounded in domain semantics.
That sounds abstract, so let’s make it concrete.
At the center are bounded contexts from domain-driven design. Each bounded context owns a coherent slice of the business model, its language, and its invariants. It is not just a code boundary. It is a semantic boundary. Inside it, terms have precise meaning. Across it, translation may be necessary.
From there, the architecture introduces several key constraints:
- Single authoritative owner for each core fact
Customer credit status, policy premium, shipment state, product price—pick one bounded context as the source of truth. Replicas may exist. Authority may not be shared casually.
- No shared operational database across autonomous services
Shared databases are not integration. They are distributed denial. They erase ownership and make change management political.
- Explicit interaction style by domain need
Use synchronous APIs where immediate decision feedback is required. Use events where state changes must be propagated without temporal coupling. Use commands when one context needs another to perform an action. Do not blur them.
- Business invariants anchored where they belong
Some invariants are local and enforceable within one context. Others span contexts and require sagas, process managers, or compensating actions. The architecture must identify them upfront.
- Reconciliation as a first-class capability
In distributed systems, drift is not a possibility. It is a certainty. Reconciliation is how mature systems acknowledge reality without surrendering control.
- Progressive migration over replacement fantasy
Large enterprises rarely get to start clean. A strangler approach lets new bounded contexts emerge around old systems while constraints are tightened over time.
Here is the heart of it visually.
The point of this diagram is not that Kafka is mandatory. It is that event flow, authority, and reconciliation are deliberate structural choices, not incidental plumbing.
Architecture
A constraint-system architecture in distributed systems usually has five layers of concern.
1. Domain boundaries
Start with bounded contexts, not services. Services are delivery units; bounded contexts are meaning units. One bounded context may be implemented by one service, several services, or a modular monolith at first. That is a practical decision. The semantic boundary comes first.
For example, in retail:
- Catalog owns product description and discoverability
- Pricing owns sell price and discount rules
- Ordering owns order lifecycle
- Fulfillment owns pick-pack-ship
- Billing owns invoices and payment application
- Customer may own profile and preferences, but not necessarily credit or loyalty
The same noun may appear in all of them, but with different meanings. That is normal. A “product” in Catalog is not the same thing as a “product” in Fulfillment if one cares about marketing attributes and the other cares about physical handling constraints.
Trying to erase these differences with a universal enterprise object model is usually an expensive mistake.
2. Interaction constraints
Not every communication style is equal.
Use synchronous request/response when a caller genuinely needs an immediate answer to complete a user interaction or make a business decision. But keep synchronous dependency chains short. A service that cannot answer without calling five others does not own much.
Use Kafka or similar event infrastructure for domain notifications and asynchronous propagation. Events are especially useful where multiple downstream consumers need to react independently, and where temporal decoupling matters.
But be disciplined with event types:
- Domain events: meaningful inside or near a domain, such as
OrderPlaced - Integration events: stable contracts published for external consumers, often more curated
- Technical events: tracing or CDC artifacts; useful, but not a substitute for domain language
A common anti-pattern is publishing database change events and pretending the architecture is event-driven. It may be eventful. It is not necessarily well-modeled.
3. State ownership and data flow
Every distributed system eventually becomes a data distribution problem. The architecture should state clearly:
- who owns the write model
- who may cache or replicate data
- how stale data may become
- how consumers recover when events are missed or delayed
This is where Kafka often earns its keep. Not because it is fashionable, but because an ordered durable log gives you a backbone for propagation, replay, and downstream rebuilding. It supports architectures where state changes are shared as a stream rather than fetched repeatedly by tightly coupled calls.
Still, Kafka is not a magical semantic machine. It preserves records, not meaning. If events are badly designed, Kafka will preserve confusion with great reliability.
4. Process coordination
Cross-context business processes need explicit coordination. Sometimes choreography through events is enough. Sometimes an orchestrated saga is safer.
If the process has many branches, compensations, regulatory rules, or timeout handling, orchestration often wins because someone can explain the flow without reading six codebases and a topic list.
This kind of flow is where constraints become visible. Payment authorization may be synchronous. Shipment planning may be asynchronous. Reconciliation tracks the whole process because partial failure is expected, not exceptional.
5. Reconciliation and audit
In serious enterprises, reconciliation is architecture, not housekeeping.
If Ordering says an order is accepted but Billing has no corresponding receivable, something is wrong. If Inventory reserved stock but Shipping never created a shipment, something is wrong. If two systems disagree on premium collected, position held, or claim paid, something is definitely wrong.
The naive approach is to hope retries solve everything. Mature systems do more:
- maintain idempotent handlers
- track process states and deadlines
- retain immutable event history where practical
- provide replay and repair mechanisms
- run periodic reconciliation jobs against authoritative sources
- expose operational dashboards for semantic exceptions, not just CPU and memory
A distributed system without reconciliation is like double-entry bookkeeping with one side removed. It may look fast. It is not trustworthy.
Migration Strategy
Most enterprises do not get to invent a clean architecture on an empty whiteboard. They inherit a tangle. So the migration strategy matters as much as the target state.
The strangler pattern remains one of the few migration ideas that has aged well because it respects reality. You do not replace the estate in one heroic program. You progressively surround and displace the legacy system, one bounded context at a time.
But here is the twist: strangling by UI route alone is not enough. You need semantic strangling.
Step 1: identify bounded contexts and authority
Map where business decisions really happen today, not where org charts claim they happen. Often the legacy ERP, mainframe, or monolith is authoritative for more things than anyone admits. Fine. Start by making that explicit.
Step 2: carve out a high-value, high-cohesion context
Choose an area where domain boundaries are reasonably stable and business value is visible. Pricing, customer preferences, product catalog, or order orchestration are common choices. Avoid starting with the most entangled financial core unless your organization is unusually disciplined.
Step 3: introduce anti-corruption layers
An anti-corruption layer translates between the new bounded context model and the legacy model. This is one of DDD’s most practical ideas. It prevents the old model from leaking into the new one and contaminating language, contracts, and assumptions.
Step 4: publish events and replicate selectively
Use CDC, integration events, or APIs to propagate needed state out of the legacy platform, but be careful: migration events are often transitional. Don’t confuse temporary extraction mechanics with permanent domain contracts.
Step 5: move decision authority, not just reads
A read replica outside the monolith is not a migrated capability. The real milestone is moving a business decision boundary. For example, not just exposing prices from legacy, but having the new Pricing context own how prices are determined and published.
Step 6: reconcile relentlessly during coexistence
During migration, dual-running and reconciliation are essential. Coexistence is where hidden assumptions emerge. If old and new produce different results, treat that as learning, not embarrassment.
Here is a typical progressive strangler path.
The migration logic is straightforward: first intercept interactions, then translate, then shift ownership, then retire old paths. The enterprise trick is patience. Most migration failures are not technical. They are caused by pretending coexistence will be brief and underfunding the middle.
Enterprise Example
Consider a large insurer modernizing policy administration and claims across regions.
The company had a core policy platform, a billing engine, a claims system, and several customer-facing portals. Over the years, “customer,” “policy,” and “coverage” had taken on different meanings in each platform. Agents could quote products online, but policy issuance still depended on nightly batch updates. Claims handlers often saw stale billing status. Finance maintained separate reconciliation teams because policy transactions and cash application did not align cleanly.
The first modernization plan was the usual dream: replace core systems with a full microservices platform. Mercifully, it failed in the planning phase.
The second plan was better. The insurer treated architecture as a constraint system.
They defined bounded contexts:
- Quotation owned rating inputs and quote calculation
- Policy Administration owned policy state and endorsements
- Billing owned invoices, receivables, and payment allocation
- Claims owned loss events and claim adjudication
- Party Management owned party identity and contactability, but not every “customer” meaning
They used Kafka as an integration backbone, but only after clarifying event semantics. PolicyIssued meant the legal policy was active in Policy Administration. It did not mean the first invoice was generated or payment was collected. Separate events represented those facts.
They introduced an anti-corruption layer around the legacy policy system and progressively strangled quotation first. That was a smart move. Quotation had high business value and fewer downstream accounting invariants than billing. Once quote ownership moved, they introduced a new policy orchestration service that coordinated issuance requests while the legacy system still held the authoritative policy record.
Reconciliation became a formal capability. Every policy issuance had to reconcile across quotation outcome, policy activation, invoice creation, and broker notification. Exceptions landed in an operational workbench, not a pile of database scripts. That one move changed the culture. Teams stopped arguing about whose system was “probably right” and started managing explicit semantic exceptions.
The result was not a pure microservices paradise. Some domains remained in a modular core for years. Good. Architecture is not scored on ideological purity. It is scored on whether the business can change safely.
Operational Considerations
Constraint-driven architecture lives or dies in operations.
Observability must include business state
Technical telemetry is necessary but insufficient. Latency, throughput, and error rates won’t tell you that orders are accepted without invoices, or claims are paid without reserve updates. You need business observability: metrics, traces, and dashboards tied to domain milestones.
Idempotency is mandatory
Kafka consumers, message handlers, retry loops, and saga participants must tolerate duplicates. In theory everyone agrees. In practice many systems still behave like every message arrives once and in order forever. They do not.
Dead letter queues are not a strategy
They are a parking lot. Useful, but not a destination. You need clear policies for replay, repair, escalation, and root cause analysis. Otherwise the dead letter queue becomes institutionalized denial.
Schema evolution needs discipline
Events and APIs evolve. Consumers lag. Producers change. Without versioning rules, compatibility checks, and contract testing, the event backbone becomes a field of hidden mines.
Data retention and replay have legal consequences
Kafka retention settings, event storage, and replay capabilities are not merely technical operations. In regulated industries they intersect with auditability, privacy, and data minimization. Rebuild capability is useful. Unbounded retention of personal data may not be.
Team topology matters
Conway’s Law never retired. If your teams are split by technology layer while your architecture is split by domain, the design will erode. Constraint systems need organizational reinforcement.
Tradeoffs
This style of architecture is powerful, but let’s not romanticize it.
The biggest benefit is clarity. Authority is explicit. Boundaries are meaningful. Failure handling becomes part of the design rather than an afterthought. Teams can move independently where constraints permit it.
The biggest cost is cognitive load.
You are choosing to model the business more carefully and to live with distributed consistency honestly. That means more design effort, more operational sophistication, and more thought about semantics than many organizations are used to.
Event-driven architectures reduce temporal coupling but make flow comprehension harder. Kafka provides durable propagation but introduces partitioning, ordering limits, replay concerns, and operational overhead. Bounded contexts protect semantics but force translation work. Reconciliation improves trust but adds process and tooling complexity.
There is also the subtle tradeoff of local purity versus enterprise usability. If every bounded context exposes only its own perfect language with no regard for broader enterprise workflows, you create a federation of elegant islands. Users do not care how semantically pristine your islands are. They care whether the end-to-end process works.
Good architecture holds both truths at once: protect domain integrity, and make the whole estate operable.
Failure Modes
Constraint systems fail in recognizable ways.
Constraint theater
The documentation says every domain has clear ownership. In reality, teams still update each other’s data stores through “temporary” scripts and side channels. The architecture exists on paper only.
Bounded contexts cut by org chart
If contexts are defined by department boundaries rather than business semantics, the resulting services usually reflect politics more than domain logic. They become awkward to evolve because the seams are wrong.
Kafka as a dumping ground
Teams publish every internal state change “just in case.” Topic sprawl follows. Consumers depend on accidental details. Event contracts become impossible to evolve.
Reconciliation bolted on late
Without process state tracking and auditable identifiers from the start, reconciliation becomes forensic archaeology. Painful, expensive, and slow.
Over-fragmentation
Breaking the system into too many services too early creates a tax no one can pay. If a context is not genuinely independent in ownership, cadence, and semantics, keep it together longer.
Synchronous dependency chains
A service that needs three downstream calls before it can answer a user request may appear well-factored. Under load, it behaves like a distributed monolith.
When Not To Use
There are cases where this architecture style is the wrong answer.
Do not use a heavy constraint-system approach for a small product with one team and a stable domain if a modular monolith will do. Distribution is expensive. If your problem is not scale of organization, autonomy, regulatory separation, or heterogeneous domain evolution, the costs may outweigh the gains.
Do not reach for Kafka because you want to feel modern. If you have simple integration needs, low event volume, and little replay or fan-out value, straightforward APIs and a well-designed database may be better.
Do not force bounded contexts prematurely in a domain you barely understand. Sometimes the right move is to keep the system more cohesive until domain knowledge matures. DDD is not about slicing early for sport. It is about finding meaningful boundaries.
And do not pretend eventual consistency is acceptable where the business cannot tolerate it. Ledgers, balances, regulatory filings, and certain safety-critical operations demand stronger controls. Some domains simply deserve fewer moving parts.
A memorable rule of thumb: if your organization cannot run reliable reconciliation, contract evolution, and operational incident response, it is not ready for broad event-driven microservices no matter how persuasive the platform demo was.
Related Patterns
Several patterns fit naturally alongside this approach:
- Bounded Context for semantic boundaries
- Anti-Corruption Layer for legacy and external translation
- Strangler Fig for progressive migration
- Saga / Process Manager for long-running cross-context processes
- Outbox Pattern for reliable event publication from transactional updates
- CQRS when read and write concerns genuinely diverge
- Event Sourcing in selected domains where historical reconstruction is valuable, though certainly not everywhere
- Data Mesh-style ownership in analytical ecosystems, with care to preserve operational authority distinctions
- Modular Monolith as a valid stepping stone or even end state for some bounded contexts
These patterns are not a shopping list. They are tools for enforcing particular constraints.
Summary
Architecture in distributed systems is not the art of drawing more boxes. It is the discipline of placing the right constraints in the right places.
That means starting with domain semantics, not infrastructure. It means assigning clear authority for business facts. It means choosing interaction styles deliberately. It means accepting that consistency has gradients, not slogans. It means building reconciliation into the architecture because drift is inevitable. And it means migrating progressively, with anti-corruption and strangler tactics, instead of betting the company on one grand rewrite.
Kafka, microservices, APIs, orchestration, and event streams all have a role. None of them rescue a system whose meanings are vague and whose constraints are undefined.
The best enterprise architectures are not the most flexible in the abstract. They are the ones that constrain chaos without strangling change.
That is the paradox worth keeping: good architecture limits what the system may do, so the business can trust what it does do.
Frequently Asked Questions
What is a service mesh?
A service mesh is an infrastructure layer managing service-to-service communication. It provides mutual TLS, load balancing, circuit breaking, retries, and observability without each service implementing these capabilities. Istio and Linkerd are common implementations.
How do you document microservices architecture for governance?
Use ArchiMate Application Cooperation diagrams for the service landscape, UML Component diagrams for internal structure, UML Sequence diagrams for key flows, and UML Deployment diagrams for Kubernetes topology. All views can coexist in Sparx EA with full traceability.
What is the difference between choreography and orchestration in microservices?
Choreography has services react to events independently — no central coordinator. Orchestration uses a central workflow engine that calls services in sequence. Choreography scales better but is harder to debug; orchestration is easier to reason about but creates a central coupling point.