Introduction to CQRS + Event Sourcing: Part 1. Basics
The first time I heard about CQRS when I got a new job. The company in which I work to this day was immediately told that CQRS, Event Sourcing, and MongoDB are used as a database on the project I will work on. From all this, I only heard about MongoDB. Trying to delve into CQRS, I did not immediately understand all the intricacies of this approach, but for some reason I liked the idea of dividing the data interaction model into two - read and write. Perhaps because it somehow resonated with the “separation of duties” programming paradigm, perhaps because it was very in the spirit of DDD.
Generally, many talk about CQRS as a design pattern. In my opinion, it influences the overall architecture of the application too much, to be called simply a “design pattern”, so I prefer to call it a principle or approach. Using CQRS penetrates almost every corner of the application.
I want to clarify right away that I only worked with a bunch of CQRS + Event Sourcing, and I never tried just CQRS, since it seems to me that without Event Sourcing it loses a lot of benefits. I will use our corporate Paralect.Domain as the CQRS framework. He is somehow better than others than worse. In any case, I advise you to familiarize yourself with the rest. I will only mention a few .NET frameworks here. The most popular are NCQRS , Lokad CQRS ,SimpleCQRS . You can also look at the Jonathan Oliver Event Store with support for a huge number of different databases.
What is CQRS?
CQRS stands for Command Query Responsibility Segregation. This is the design pattern that I first heard about from Greg Young. It is based on a simple concept that you can use different models to update and read information. However, this simple concept leads to serious consequences in the design of information systems. (c) Martin Fowler
Not to say that an exhaustive definition, but now I will try to explain what exactly Fowler had in mind.
To date, such a situation has developed that practically everyone works with the data model as with a CRUD repository. CQRS offers an alternative approach, but affects not only the data model. If you use CQRS, then this greatly affects the architecture of your application.
This is how I depicted the CQRS flowchart
The first thing that catches your eye is that you already have two data models, one for reading (Queries), one for writing (Commands). And usually that means you also have two databases. And since we use CQRS + Event Sourcing, the write-base (write-model) is the Event Store, something like a log of all user actions (in fact, not all, but only those that are important from the point of view of the business model and affect the construction of the read-base). And the read-base is generally a denormalized repository of the data that you need to display to the user. Why did I say that the read base is denormalized? Of course, you can use any data structure as a read-model, but I think that when using CQRS + Event Sourcing you should not bother much with normalizing the read-base, as it can be completely rebuilt at any time.
The write-base is generally one collection of events. That is, there is also no point in using a relational database.
The idea behind Event Sourcing is to record every event that changes the state of an application into a database. Thus, it turns out that we do not store the state of our entities, but all the events that relate to them. However, we are used to manipulating precisely the state, it is stored in our database and we can always see it.
In the case of Event Sourcing, we also operate on the state of the entity. But unlike the usual model, we do not store this state, but play it every time we call it.
If you look at the code that raises the unit from the database, you may not notice any difference with the traditional approach.
In fact, the repository does not take the ready state of the UserAR aggregate from the database (AR = Aggregate Root), it selects from the database all the events that are associated with this user, and then plays them in order passing to the On () method of the aggregate.
For example, the UserAR aggregate class must have the following method in order to restore its ID in the user state
From the entire state of the unit, I only need the _id of the user, so I could restore the state of the password, name, etc. However, these fields can be modified by other events, not only User_CreatedEvent, respectively, I will need to process them all. Since all events are played back in order, I am sure that I always work with the latest current state of the aggregate, unless of course I wrote On () handlers for all events that change this state.
Let's look at the example of creating a user how CQRS + Event Sourcing works.
The first thing I will do is form and send a team. For simplicity, let the user creation command have only the most necessary set of fields and look as follows.
Let the class name of the team not confuse you, it does not correspond to the generally accepted guidelines, but it allows you to immediately understand which unit it belongs to and what action is performed.
Then I need a handler for this command. It is necessary for the command handler to pass the ID of the required aggregate, by this ID it will receive the aggregate from the repository. The repository builds the aggregate object as follows: it takes from the database all the events that belong to this aggregate, creates a new empty aggregate object, and in order plays the received events on the aggregate object.
But since we have creation teams, there is nothing to raise from the database, which means we create the aggregate ourselves and pass the command metadata to it.
Let's see what the unit designer looks like.
Also, an aggregate must have a constructor without parameters, because when the repository reproduces the state of the aggregate, it must first create an instance with the path and then pass the events to the projection methods (the On method (User_CreatedEvent created) is one of the projection methods).
I’ll clarify a little about the projection. A projection is a reproduction of the state of an implement based on events from the Event Store that are relevant to that implement. In the user example, these are all events for this particular user. And the aggregate, all the same events that are saved through the Apply method, can be processed during the playback of its state. In our framework, for this it is enough to write the On method (/ * EventType arg * /), where EventType is the type of event that you want to process.
The Apply method of the aggregate triggers the sending of events to all handlers. In fact, events will be dispatched only when the aggregate is saved to the repository, and Apply simply adds them to the aggregate's internal list.
Here is the event handler (!) For creating a user that writes to the read base of the user itself.
An event can have multiple handlers. This architecture helps maintain data integrity if your data is heavily denormalized. Suppose I need to often show the total number of users. But I have too many of them and the count operation on all is very expensive for my database. Then I can write another event handler that will already be related to statistics and each time I add a user I will increase the total user counter by 1. And I’ll be sure that no one will create a user without updating the counter. If I hadn’t used CQRS, but if I had the usual ORM, I would have to follow in every place where it is added and deleted to use so that the counter is updated.
And the use of Event Sourcing gives me additional benefits. Even if I made a mistake in some EventHandler or didn’t process the event wherever I need it, I can easily fix it simply by redirecting the read database with the correct business logic.
With the creation is understandable. How does aggregate change and team validation occur? Consider the password change example.
I will give only the command handler and the ChangePassword () aggregate method, since in other places the difference is generally not big.
I want to note that it is not advisable that an invalid event be passed to the Apply () method. Of course you can process it later in the event handler, but it’s better not to save it at all if it is not important to you, as this will only clog the Event Store.
In the case of a password change, it makes no sense to save this event, unless of course you collect statistics on failed password changes. And even in this case, you should carefully think about whether this event is footy for you in the write model or whether it makes sense to write it to some tempo storage. If you assume that the business logic of event validation may change, then save it.
That's actually all that I wanted to talk about in this article. Of course, it does not reveal all aspects and possibilities of CQRS + Event Sourcing, I plan to talk about this in future articles. The problems that arise when using this approach have also remained behind the scenes. And we'll talk about that too.
If you have any questions, ask them in the comments. I will be happy to answer them. Also, if there are any suggestions for the following articles - I really want to hear them.
A fully working example on ASP.NET MVC is here .
There is no database there, everything is stored in memory. If desired, it is enough just to screw it. Also out of the box is a ready-made implementation of the Event Store on MongoDB for storing events.
To screw it in the Global.asax file, it is enough to replace InMemoryTransitionRepository with MongoTransitionRepository.
As a read model, I have a static collection, so every time I restart, the data is destroyed.
I have several ideas about articles on this topic. Suggest more. Say what is most interesting.
Generally, many talk about CQRS as a design pattern. In my opinion, it influences the overall architecture of the application too much, to be called simply a “design pattern”, so I prefer to call it a principle or approach. Using CQRS penetrates almost every corner of the application.
I want to clarify right away that I only worked with a bunch of CQRS + Event Sourcing, and I never tried just CQRS, since it seems to me that without Event Sourcing it loses a lot of benefits. I will use our corporate Paralect.Domain as the CQRS framework. He is somehow better than others than worse. In any case, I advise you to familiarize yourself with the rest. I will only mention a few .NET frameworks here. The most popular are NCQRS , Lokad CQRS ,SimpleCQRS . You can also look at the Jonathan Oliver Event Store with support for a huge number of different databases.
Let's start with CQRS
What is CQRS?
CQRS stands for Command Query Responsibility Segregation. This is the design pattern that I first heard about from Greg Young. It is based on a simple concept that you can use different models to update and read information. However, this simple concept leads to serious consequences in the design of information systems. (c) Martin Fowler
Not to say that an exhaustive definition, but now I will try to explain what exactly Fowler had in mind.
To date, such a situation has developed that practically everyone works with the data model as with a CRUD repository. CQRS offers an alternative approach, but affects not only the data model. If you use CQRS, then this greatly affects the architecture of your application.
This is how I depicted the CQRS flowchart
The first thing that catches your eye is that you already have two data models, one for reading (Queries), one for writing (Commands). And usually that means you also have two databases. And since we use CQRS + Event Sourcing, the write-base (write-model) is the Event Store, something like a log of all user actions (in fact, not all, but only those that are important from the point of view of the business model and affect the construction of the read-base). And the read-base is generally a denormalized repository of the data that you need to display to the user. Why did I say that the read base is denormalized? Of course, you can use any data structure as a read-model, but I think that when using CQRS + Event Sourcing you should not bother much with normalizing the read-base, as it can be completely rebuilt at any time.
The write-base is generally one collection of events. That is, there is also no point in using a relational database.
Event sourcing
The idea behind Event Sourcing is to record every event that changes the state of an application into a database. Thus, it turns out that we do not store the state of our entities, but all the events that relate to them. However, we are used to manipulating precisely the state, it is stored in our database and we can always see it.
In the case of Event Sourcing, we also operate on the state of the entity. But unlike the usual model, we do not store this state, but play it every time we call it.
If you look at the code that raises the unit from the database, you may not notice any difference with the traditional approach.
var user = Repository.Get<UserAR>(userId);
In fact, the repository does not take the ready state of the UserAR aggregate from the database (AR = Aggregate Root), it selects from the database all the events that are associated with this user, and then plays them in order passing to the On () method of the aggregate.
For example, the UserAR aggregate class must have the following method in order to restore its ID in the user state
protectedvoidOn(User_CreatedEvent created)
{
_id = created.UserId;
}
From the entire state of the unit, I only need the _id of the user, so I could restore the state of the password, name, etc. However, these fields can be modified by other events, not only User_CreatedEvent, respectively, I will need to process them all. Since all events are played back in order, I am sure that I always work with the latest current state of the aggregate, unless of course I wrote On () handlers for all events that change this state.
Let's look at the example of creating a user how CQRS + Event Sourcing works.
Team creation and sending
The first thing I will do is form and send a team. For simplicity, let the user creation command have only the most necessary set of fields and look as follows.
publicclassUser_CreateCommand: Command
{
publicstring UserId { get; set; }
publicstring Password { get; set; }
publicstring Email { get; set; }
}
Let the class name of the team not confuse you, it does not correspond to the generally accepted guidelines, but it allows you to immediately understand which unit it belongs to and what action is performed.
var command = new User_CreateCommand
{
UserId = “1”,
Password = “password”,
Email = “test@test.com”,
};
command.Metadata.UserId = command.UserId;
_commandService.Send(command);
Then I need a handler for this command. It is necessary for the command handler to pass the ID of the required aggregate, by this ID it will receive the aggregate from the repository. The repository builds the aggregate object as follows: it takes from the database all the events that belong to this aggregate, creates a new empty aggregate object, and in order plays the received events on the aggregate object.
But since we have creation teams, there is nothing to raise from the database, which means we create the aggregate ourselves and pass the command metadata to it.
publicclassUser_CreateCommandHandler: CommandHandler<User_CreateCommand>
{
publicoverridevoidHandle(User_CreateCommand message)
{
var ar = new UserAR(message.UserId, message.Email, message.Password, message.Metadata);
Repository.Save(ar);
}
}
Let's see what the unit designer looks like.
publicUserAR(string userId, string email, string password, ICommandMetadata metadata): this()
{
_id = userId;
SetCommandMetadata(metadata);
Apply(new User_CreatedEvent
{
UserId = userId,
Password = password,
Email = email
});
}
Also, an aggregate must have a constructor without parameters, because when the repository reproduces the state of the aggregate, it must first create an instance with the path and then pass the events to the projection methods (the On method (User_CreatedEvent created) is one of the projection methods).
I’ll clarify a little about the projection. A projection is a reproduction of the state of an implement based on events from the Event Store that are relevant to that implement. In the user example, these are all events for this particular user. And the aggregate, all the same events that are saved through the Apply method, can be processed during the playback of its state. In our framework, for this it is enough to write the On method (/ * EventType arg * /), where EventType is the type of event that you want to process.
The Apply method of the aggregate triggers the sending of events to all handlers. In fact, events will be dispatched only when the aggregate is saved to the repository, and Apply simply adds them to the aggregate's internal list.
Here is the event handler (!) For creating a user that writes to the read base of the user itself.
publicvoidHandle(User_CreatedEvent message)
{
var doc = new UserDocument
{
Id = message.UserId,
Email = message.Email,
Password = message.Password
};
_users.Save(doc);
}
An event can have multiple handlers. This architecture helps maintain data integrity if your data is heavily denormalized. Suppose I need to often show the total number of users. But I have too many of them and the count operation on all is very expensive for my database. Then I can write another event handler that will already be related to statistics and each time I add a user I will increase the total user counter by 1. And I’ll be sure that no one will create a user without updating the counter. If I hadn’t used CQRS, but if I had the usual ORM, I would have to follow in every place where it is added and deleted to use so that the counter is updated.
And the use of Event Sourcing gives me additional benefits. Even if I made a mistake in some EventHandler or didn’t process the event wherever I need it, I can easily fix it simply by redirecting the read database with the correct business logic.
With the creation is understandable. How does aggregate change and team validation occur? Consider the password change example.
I will give only the command handler and the ChangePassword () aggregate method, since in other places the difference is generally not big.
Command handler
publicclassUser_ChangePasswordCommandHandler: IMessageHandler<User_ChangePasswordCommand>
{
// Конструктор опущенpublicvoidHandle(User_ChangePasswordCommand message)
{
// берем из репозитория агрегатvar user = _repository.GetById<UserAR>(message.UserId);
// выставляем метаданные
user.SetCommandMetadata(message.Metadata);
// меняем пароль
user.ChangePassword(message.OldPassword, message.NewPassword);
// сохраняем агрегат
_repository.Save(user);
}
}
Aggregate root
publicclassUserAR : BaseAR
{
//...publicvoidChangePassword(string oldPassword, string newPassword)
{
// Если пароль не совпадает со старым, кидаем ошибкуif (_password != oldPassword)
{
thrownew AuthenticationException();
}
// Если все ОК - принимаем ивент
Apply(new User_Password_ChangedEvent
{
UserId = _id,
NewPassword = newPassword,
OldPassword = oldPassword
});
}
// Метод проекции для восстановления состояния пароляprotectedvoidOn(User_Password_ChangedEvent passwordChanged)
{
_password = passwordChanged.NewPassword;
}
// Так же добавляем восстановления состояния пароля на событие создания пользователяprotectedvoidOn(User_CreatedEvent created)
{
_id = created.UserId;
_password = created.Password;
}
}
}
I want to note that it is not advisable that an invalid event be passed to the Apply () method. Of course you can process it later in the event handler, but it’s better not to save it at all if it is not important to you, as this will only clog the Event Store.
In the case of a password change, it makes no sense to save this event, unless of course you collect statistics on failed password changes. And even in this case, you should carefully think about whether this event is footy for you in the write model or whether it makes sense to write it to some tempo storage. If you assume that the business logic of event validation may change, then save it.
That's actually all that I wanted to talk about in this article. Of course, it does not reveal all aspects and possibilities of CQRS + Event Sourcing, I plan to talk about this in future articles. The problems that arise when using this approach have also remained behind the scenes. And we'll talk about that too.
If you have any questions, ask them in the comments. I will be happy to answer them. Also, if there are any suggestions for the following articles - I really want to hear them.
Sources
A fully working example on ASP.NET MVC is here .
There is no database there, everything is stored in memory. If desired, it is enough just to screw it. Also out of the box is a ready-made implementation of the Event Store on MongoDB for storing events.
To screw it in the Global.asax file, it is enough to replace InMemoryTransitionRepository with MongoTransitionRepository.
As a read model, I have a static collection, so every time I restart, the data is destroyed.
What's Next?
I have several ideas about articles on this topic. Suggest more. Say what is most interesting.
- What are Snapshots, why are they needed, details and implementation options.
- Event Store.
- Database regeneration. Opportunities. Performance problems. Parallelization. Patches.
- Designed by Aggregate Roots.
- Application on real projects. One project is outsourced, the second is my startup.
- Features of integration of third-party services.