Posted on May 27, 2020 by Rajesh Rajagopalan

In the first part of the blog, we looked at Domain Driven Design (DDD) from a practitioner’s point of view.

• We considered how DDD helps break down complex systems into individual Bounded Contexts
• And then we discussed how Bounded Contexts are model as self-contained microservices
• Finally, we saw how microservices can communicate with each other using Domain Events and a messaging middleware

Using Domain Events as a means of communication helps our microservices remain self-contained and loosely coupled. However, many of our clients have long-running business processes that span multiple microservices.

Choreographed Microservices

Let’s consider the example of Billing System again. We have various microservices – Vendor Management, Billing, Approvals and Payment. Each microservice communicates significant business events that occur in their respective Bounded Contexts via Domain Events.

• The microservice’s responsibility ends once the Domain Event is emitted
• After emitting the Domain Event, microservice does not wait for any response, or worry about what happens next
• And one or more microservices subscribe to these Domain Events, and take action when they receive them
  • For example, from our earlier solution, Approvals microservice will check for “New Bill Added” Domain Event and trigger an approval workflow
  • And Payment microservice will look for “Bill Approved” Domain Event and initiate payment when it receives one

This is a good example of “Choregraphed Microservices.”

Now consider there is a new business requirement: any bill less than $1000 is automatically paid, and does not require any approval. To implement this requirement, we need to make changes to:

• Approvals Microservice, so it does not take those bills through approval workflow
• Payment Microservice, so it now starts to check for “New Bill Added” event, and initiate payment when it receives one that is less than $1000

While this is a simple example, imagine a complex system that has several hundreds or thousands of microservices. Implementing requirements such as these will result in modifications to multiple microservices. Above all, when a business process owner is looking at this, it becomes hard to understand the overall business process flow, and more difficult to make changes.

Microservice Death-star

As many have described such systems in the past – when you build such a complex system with thousands of interconnected Microservices, you end up with a Microservice death-star.

“The danger is that it’s very easy to make nicely decoupled systems with event notification, without realizing that you’re losing sight of that larger-scale flow, and thus set yourself up for trouble in future years.” – Martin Fowler

Orchestrating Microservices

We can overcome simple problems like the one described earlier using better designs. For example, have the Billing Microservice emit “New Bill under $1000” or “New Bill over $1000” event. And then, Payment Microservice can listen to “New Bill under $1000” event to initiate the payment. Now, the Approvals Microservice can listen to “New Bill over $1000” to initiate the approval workflow. However, such design approaches will be harder to implement and manage at scale. Add to this a requirement for what happens when the payment fails, who should retry etc.. These are some of the key challenges when it comes to designing distributed systems using Choreographed Microservices .

An alternative to above approach is “Orchestrating Microservices.” Having a microservice whose responsibility is orchestrating business process flow across various microservices helps address several of our requirements and concerns detailed above.

Orchestrating Microservices help with the following:

• Coordinate communication between various Microservices in the domain
• Localize changes to business process rules. Avoid unintended impact on multiple Microservices
• Provide end-to-end business process view
• Help maintain the single responsibility design principle

The above example can be modified to have a “Vendor Payment” microservice that coordinates between other microservices.

There are a number of options available to design and implement an Orchestrating Microservice such as the one above. Based on your use-case and the complexity of requirement, you can chose from a simple state machine to a workflow engine.

Common Pitfalls and Anti-Patterns

We need to address cautious against common pitfalls and anti-patterns while implementing Orchestrating Microservices:

• Avoid turning your system into a distributed monolith
• Concentrating business logic more than needed within the Orchestrating Microservice will result in anemic domain models
• Don’t build custom state machines / workflow engines unless there is a strong need

Key takeaways, and a checklist to determine the right solution option

• Start with choreography
• BPM Acid Test: BPM is suited only for applications with an essential sense of state or process—that is, applications that are process-oriented
  • Long-running
  • Persisted state
  • Bursty, sleeps most of the time
  • Orchestration of system and human communications
• Ownership of the workflow stays within the Bounded Context
• Polyglot state machine / workflow engine based on your requirement
  • Choose from simple state machines to more sophisticated BPM Workflow engines based on the bounded context
• Decentralize state management
• Use Workflow Engine just to coordinate / orchestrate, and don’t build implementation logic into it
  • Consider lightweight and embeddable state machines / workflow engines as first choice
• Stay compliant with latest BPMN 2.0, DMN, CMMN standards
• Choose state management / workflow solution that satisfy your organization’s non-functional requirements of
  • Observability
  • Testability
  • DevOps Integration
  • Performance and Scalability

Find the right balance

From my experience, leveraging the right blend of Choreography and Orchestration helps develop solutions that can be scaled, while providing end-to-end business process visibility. As Martin Fowler explained, you don’t want to be building a highly decoupled and autonomous microservices system that does not provide you the right business process visibility, making it harder to adapt to future requirements.

In Part 3 of the blog, will explore DDD Aggregates and how to avoid anemic Domain Models.

References, and additional reading:

1. Microservices by Martin Fowler
   • https://martinfowler.com/articles/microservices.html
2. Complex event flows in distributed systems by Bernd Ruecker
   • https://www.slideshare.net/BerndRuecker/complex-event-flows-in-distributed-systems
   • https://blog.bernd-ruecker.com/bizdevops-the-true-value-proposition-of-workflow-engines-f342509ba8bb
3. Essential Business Process Modeling
   • https://learning.oreilly.com/library/view/essential-business-process/0596008430/
4. Microservices Death-star