How we tried DDD, CQRS and Event Sourcing and what conclusions we made

For about three years now I have been using the principles of Spec By Example , Domain Driven Design and CQRS in my work . During this time, experience has been gained in the practical application of these practices on the .NET platform. In the article I want to share our experience and conclusions, which may be useful to teams wishing to use these approaches in development.

DDD conclusions

1. Very expensive
2. Works well in established business processes
3. Sometimes it's the only way to do what you need.
4. Poorly scaled
5. Difficult to implement in heavily loaded applications
6. Runs poorly in startups
7. Not suitable for reporting
8. Requires special attention with ORM
9. Entity is best avoided because everyone understands it in their own way.
10. With LINQ, the standard implementation of Specification "does not work"

Very expensive

All development managers using DDD, with whom I discussed the topic, noted the “high cost” of this methodology, primarily because of the lack of answers in the Evans book to practical questions “How can I make FooBar without violating the principles of DDD?”.

The most common question in Google’s CQRS group, according to Greg Young: “The boss is asking me to build an annual report. When I pick up all the roots of aggregation in RAM, everything starts to slow down. What should I do?". There is an obvious answer to this question: "you need to write an SQL query." However, writing a manual SQL query is clearly against DDD rules.

Evans himself agreed with Young that the book should be written in a different order. The key concepts are Bounded Context and Ubiquitous Language, not Entity and ValueObject .

Reports do not need a domain model. A report is just a data table. Data Driven is much better suited for reporting than Domain Driven. At first glance, at this point, you need to say DDD sucks. However, it is not. Just using DDD to build reports is not a true Bounded Context.

Bounded context

1. Fowler in English
2. My material is in Russian

The most important thesis of DDD is that you should not try to develop one large domain model for the entire application. This is too complicated and nobody needs. It is possible to create one domain model for the entire application only if at the level of company management it was decided that all departments use the same terminology and understand all business processes in the same way.

Entity everyone understands in their own way

We made sure that it is very difficult to agree with all team members on terminology. The term Entity became a stumbling block : we tried to use the IEntity interface, however, they quickly realized that Id could use ValueObjects to transmit commands. Using IEntityconfused people for such objects, and we abandoned IEntity in favor of IHasId .

DDD requires special attention with ORM

There are quite a few discussions on NHacknate vs Entity Framework for DDD on Stack Overflow. NHibernate generally does better, but there are still many problems. The standard approach when using ORM is to use parameterless constructors and set values ​​through property setters. This is an encapsulation break. There are certain problems with collections and Lazy Load. In addition, the team must decide where the "domain" ends and the "infrastructure" begins and how to provide Persistence Ignorance.

With LINQ, the standard implementation of Specification "does not work"

Evans is a man from the Java world. In addition, the book was written a long time ago.

public abstract class Specification
{
    public abstract bool IsSatisfiedBy(T entity)
}; 

This interface allows you to work with collections in memory, but is in no way suitable for building SQL queries. In modern C #, this option is more suitable :

public abstract class Specification
{
    public bool IsSatisfiedBy(T item)
    {
        return SatisfyingElementsFrom(new[] { item }.AsQueryable()).Any();
    }
    public abstract IQueryable SatisfyingElementsFrom(IQueryable candidates);
}

Application area

Domain modeling is not an easy task. DDD involves delegating some analytics tasks to developers. This is justified in cases where the cost of error is high. It doesn’t matter how fast you wrote the code and how fast your system works if it doesn’t work correctly and you lose money. In fact, the opposite is true - if you are developing software for HFT and do not fully understand how it should work, it is better that your software slows down or does not work at all. So at least you won’t lose money on super-fast, but not correct trading :)

In unstable businesses (especially startups), there is often no understanding of the subject model. Everything can change daily. In these conditions, it is useless to require participants in the business process to use a single terminology.

Cqrs

The conclusion is obvious: DDD is not a “silver bullet”, but a pity :) However, you can get a significant gain due to the “targeted application” of DDD in certain Bounded Contexts .

In 1980, Bertrand Meyer formulated the very simple term CQS. In the early 2000s, Greg Young expanded and popularized this concept. So CQRS appeared ... and CQRS in many respects repeated the fate of DDD, in the sense that it was repeatedly misinterpreted.

Despite the fact that there are plenty of CQRS materials on the Internet, everyone “prepares” it in different ways. Many teams use the principles of CQRS, although they do not call it that. The system may not have abstractions of Command and Query . They can be replaced by IOperation or even Funcand action.

There is a simple explanation for this. The first CQRS results produce something like the image below:



Dino Esposito calls this implementation DELUXE . The point here is that CQRS interests Greg Young mainly in the context of Event Sourcing . Actually, for Event Sourcing you need to use CQRS, but not vice versa.



Thus, using CQRS we can solve the problem of brake reports by dividing the application stacks into Read and Write and not using the Domain Model in the Read stack. The read stack may use a different database and / or other more optimal data access API.

Dividing an application into commands, handlers, and requests has another advantage: better predictability. In the case of DDD, to know where to look for a particular business logic, you need to understand the subject area. In the case of CQRS, the programmer always knows that writing occurs in command handlers, and Query is used to access data. In addition, there are several not obvious, at first glance, advantages. We will consider them below.

CQRS key findings

1. Event Sourcing requires CQRS, but not vice versa
2. Cheap
3. Fits everywhere
4. Scalable
5. Does not require 2 data stores. This is one of the possible implementations, but not obligatory
6. The command handler can return a value. If you do not agree, argue with Greg Young and Dino Esposito, and not with me
7. If the handler returns a value, it scales worse, however there is async / await, but you need to understand how they work

The main interfaces in CQRS may look like this :

    [PublicAPI]
    public interface IQuery
    {
        TOutput Ask();
    }
    [PublicAPI]
    public interface IQuery
    {
        TOutput Ask([NotNull] TSpecification spec);
    }
    [PublicAPI]
    public interface IAsyncQuery
        : IQuery>
    {
    }
    [PublicAPI]
    public interface IAsyncQuery
        : IQuery>
    {
    }
    [PublicAPI]
    public interface ICommandHandler
    {
         void Handle(TInput input);
    }
    [PublicAPI]
    public interface ICommandHandler
    {
        TOutput Handle(TInput input);
    }
    [PublicAPI]
    public interface IAsyncCommandHandler
        : ICommandHandler
    {
    }
    [PublicAPI]
    public interface IAsyncCommandHandler
        : ICommandHandler>
    {
    }

We have agreed that:

1. Query always only receives data, but does not change the state of the system. To change the system, use the commands
2. Query can return necessary projections on a straight line, bypassing the domain model

In this case, in the absence of commands, all Query always return the same results on the same input . Such an organization greatly simplifies debugging, because in Query there is no state that could change the return result.

If necessary, Audit Log or a full-fledged Event Sourcing can be connected to all command handlers through the base class.

It's not hard to notice that the main CQRS interfaces can be brought to Funcand action. Add stateless and immutable, and you will get pure functions (hi functional programming;) Strictly speaking, this is certainly not the case, because most Query will work with the file system, database or network. You also probably want to cache the results of Query, however, you can get the benefits of linearizing data-flow and composability.

CQRS over HTTP

CQRS principles are very well suited to implement over HTTP. The HTTP specification clearly says GET requests should return data from the server. POST, PUT, PATCH - change state. A good form in web programming is considered a redirect to GET after performing a POST operation, for example, a form submit.

so

1. GET is Query
2. POST / PUT / PATCH / DELETE is Command

Base classes for commonly used operations

Reports are not the only frequent task of reading data. A more general definition of typical read operations is:

1. Filtration
2. Pagination (pagination)
3. Creation of projections (representation of aggregates in the form necessary on the client side)

We actively use AutoMapper to build projections. One of the distinguishing features of this mapper is Queryable-Extensions : the ability to build Expression for conversion to SQL, instead of mapping in RAM. These projections are not always accurate and efficient, but they are ideal for rapid prototyping.

For paged output from any table to the database and filtering support, you can use only one IQuery implementation .

    public class ProjectionQuery
        : IQuery>
        , IQuery
        where TSource : class, IHasId
        where TDest : class
    {
        protected readonly ILinqProvider LinqProvider;
        protected readonly IProjector Projector;
        public ProjectionQuery([NotNull] ILinqProvider linqProvier, [NotNull] IProjector projector)
        {
            if (linqProvier == null) throw new ArgumentNullException(nameof(linqProvier));
            if (projector == null) throw new ArgumentNullException(nameof(projector));
            LinqProvider = linqProvier;
            Projector = projector;
        }
        protected virtual IQueryable GetQueryable(TSpecification spec)
        => LinqProvider
            .GetQueryable()
            .ApplyIfPossible(spec)
            .Project(Projector)
            .ApplyIfPossible(spec);
        public virtual IEnumerable Ask(TSpecification specification)
            => GetQueryable(specification).ToArray();
        int IQuery.Ask(TSpecification specification)
            => GetQueryable(specification).Count();
    }
    public class PagedQuery : ProjectionQuery,
        IQuery> 
        where TEntity : class, IHasId
        where TDto : class, IHasId
        where TSpec : IPaging
    {
        public PagedQuery(ILinqProvider linqProvier, IProjector projector)
            : base(linqProvier, projector)
        {
        }
        public override IEnumerable Ask(TSpec spec)
            => GetQueryable(spec).Paginate(spec).ToArray();
        IPagedEnumerable IQuery>.Ask(TSpec spec)
            => GetQueryable(spec).ToPagedEnumerable(spec);
        public IQuery> AsPaged()
            => this as IQuery>;
    }

The ApplyIfPossible method will check whether filtering is performed at the aggregate or projection level (it may be necessary this way and that). The Project method creates a projection using AutoMapper.

AutoFilter and Dynamic Linq can help if you work with a lot of similar forms.

    public static class AutoFilterExtensions
    {
        public static IQueryable ApplyDictionary(this IQueryable query
           , IDictionary filters)
        {
            foreach (var kv in filters)
            {
                query = query.Where(kv.Value is string
                    ? $"{kv.Key}.StartsWith(@0)"
                    : $"{kv.Key}=@0", kv.Value);
            }
            return query;
        }
        public static IDictionary GetFilters(this object o) => o.GetType()
            .GetTypeInfo()
            .GetProperties(BindingFlags.Public)
            .Where(x => x.CanRead)
            .ToDictionary(k => k.Name, v => v.GetValue(o));
    }
    public class AutoFilter : ILinqSpecification
        where T: class
    {
        public IDictionary Filter { get; } 
        public AutoFilter()
        {
            Filter = new Dictionary();
        }
        public AutoFilter([NotNull] IDictionary filter)
        {
            if (filter == null) throw new ArgumentNullException(nameof(filter));
            Filter = filter;
        }
        public IQueryable Apply(IQueryable query)
            => query.ApplyDictionary(Filter);
    }

To build aggregates from create / edit commands, you can use the generic TypeConverter .

In order to simplify the registration in the container, you can use the agreement .

Conclusion

We actively use CQRS without Event Sourcing at work and so far the impressions are very good.

1. It’s easier to test the code because the classes are small and guaranteed to be responsible for only one thing.
2. For the same reason, making changes to the system is simplified.
3. Communication has been simplified, disputes about where this or that code should be located have disappeared. The code of different team members has become uniform
4. DDD is used for initial system simulation and aggregate creation. Aggregates may not be instantiated at all, in case all methods on the corresponding table are rigidly optimized (implemented bypassing ORM)
5. Event Sourcing in full banana - implementation has never been required, Audit Log is implemented quite often.