SObjectizer: From Simple to Complex. Part II
The first article talked about what SObjectizer is. In the second article, we started talking about how agents might look, why, how, and where they evolve. Today we continue this story, further complicating the implementation of the demonstration agents. At the same time, we check the reliability of asynchronous messaging.
Last time, we settled on the fact that the operation of reading the contents of a file with an email should be left to a separate IO agent. Let's do it and see what happens.
First, we need a set of messages that the IO agent and email_analyzer will exchange among themselves:
Secondly, we need to determine where exactly the email_analyzer agent will send the request message load_email_request. We could have gone the usual way: when registering an IO agent, save its direct_mbox, then pass this mbox as a parameter to the analyzer_manager agent constructor, then pass a parameter to the email_analyzer agent constructor for each agent ... Basically, if we would need to have several different IO- agents, that’s how it should be done. But in our problem, one IO agent is quite enough. Which allows us to demonstrate named mboxes.
A named mbox is created by calling so_5 :: environment_t :: create_mbox (name). If you call create_mbox several times with the same name, then the same mbox will always be returned, created during the first call to create_mbox with this name.
The IO agent creates a named mbox and subscribes to it. Email_analyzer agents receive the same mbox when they need to send a load_email_request message. Thus, we get rid of the need to "drag" the mbox IO agent through analyzer_manager.
Now that we have decided on the interface between the IO agent and email_manager, we can make a new version of the email_analyzer agent :
Email_analyzer agents now delegate IO operations to another agent who knows how to do this efficiently. Accordingly, the email_analyzer agents on their work threads will either deal with the assignment of tasks to the IO agent, or with the processing of email_analyzer replies. This gives us the opportunity to change our view of how many email_analyzer agents we can create and how many work threads they need.
When each email_analyzer agent itself performed a synchronous IO operation, we needed to have as many work threads in the pool as we wanted to allow parallel IO operations. At the same time, there was no point in creating much more email_analyzer agents than the number of work threads in the pool. If there are 16 threads in the pool, and we allow 32 agents to exist at the same time, then this will lead to the fact that half of these agents will simply wait for any of the working threads to be released for them.
Now, after the IO operations are moved to a different work context, you can, firstly, reduce the number of work threads in the pool. Email_analyzer agents in their events will perform mainly CPU-loading operations. Therefore, it makes no sense to create more work flows than there are available processing cores. So, if we have a 4-core processor, then we need not 16 threads in the pool, but not more than 4.
Secondly, if IO operations take longer than processing the contents of email, then we get the opportunity to create more email_analyzer agents than threads in the pool. Simply, most of these agents will wait for the result of their IO operation. Although, if the download time of an email is comparable or less than the time of analyzing its contents, this item will lose its relevance and we will be able to create only 1-2-3 email_analyzer agents more than the number of threads in the pool. All these settings are easily made in one place - in the analyzer_manager agent. It is enough to change just a couple of constants in its code and see how the changes affect the performance of our solution. However, performance tuning is a separate big topic and it’s premature to delve into it now ...
So, we have the next version of the email_analyzer agent, which fixes the problems of previous versions. Can we consider it acceptable?
Not.
The problem is that the resulting implementation cannot be considered reliable.
This implementation is designed for an optimistic scenario in which the messages we send are never lost and always reach the addressee, and when they reach the addressee, they are always processed. After which we always get the answer we need.
The harsh truth of life, however, is that when a system is built on an asynchronous message exchange between individual actors / agents, this same asynchronous exchange cannot be considered an absolutely reliable thing. Messages may be lost. And this is normal .
Loss of messages can occur for various reasons. For example, the receiving agent has not yet managed to subscribe to the message. Or there is no recipient agent at all at the moment. Either it is, but it has worked overload protection mechanism (more on this in a subsequent article). Either there is an agent and the message even reached him, but the agent is in a state in which this message is not processed. Either there is an agent, the message reached him, he even began to process it, but during the processing some kind of application error occurred and the failed agent did not send anything in response.
In general, communication between agents through asynchronous messages is like host interaction through the UDP protocol. In most cases, datagrams reach the recipients. But sometimes they get lost along the way or even during processing.
The above means that load_email_request may not reach the IO agent. Or, the response messages load_email_successed / load_email_failed may not reach the email_analyzer agent. And what will happen in this case?
We will get an email_analyzer agent that is present in the system but does nothing. Does not work. Not going to die. And does not start any other agent email_analyzer. If we are not lucky, then we may encounter a situation where all the email_analyzer agents created by the analyzer_manager will turn into half-corpses that do nothing. After that, analyzer_manager will simply accumulate requests in its turn, and then throw them out after the timeout expires. But no useful work will be performed.
How to get out of this situation?
For example, due to the control of timeouts. We can either introduce control of the execution time of the IO operation by the email_analyzer agent (i.e., if there is no answer for too long, then assume that the IO operation failed). Or, you can enter control over the execution time of the entire email analysis operation in the analyzer_manager agent. Or do both.
For simplicity, we restrict ourselves to counting the timeout of the IO operation in the email_analyzer agent :
This option email_analyzer can already be considered quite acceptable. In his code, refactoring suggests taking a couple of operations (send and so_deregister_agent_coop_normally) into a separate helper method. But this was not done on purpose, so that the code for each subsequent version of the email_analyzer agent would be minimally different from the code for the previous version.
And just if you compare the two versions of the email_analyzer agent shown above, you will notice one feature that is very appreciated by programmers who have long used SObjectizer in their daily work: the simplicity and clarity of the agent extension procedure. Did the agent need to respond to another event? So you need to add another subscription and another event handler. And since subscriptions are usually made in the same places, it’s immediately clear where to go and what to edit.
SObjectizer does not impose any restrictions on where and how the agent writes its events, but following a simple agreement - subscriptions are done in so_define_agent (), or, in very simple cases, for the final classes of agents, in the constructor - which greatly simplifies life . You look at the code of someone else’s agent or even the code of your agent, but written several years ago, and you immediately know what you need to look to understand the behavior of the agent. It’s convenient, although to understand this convenience, you probably need to write and debug more than one real agent, or even two ...
However, let us return to the topic of agent reliability, which was raised above and because of which the next, sixth, appeared email_analyzer agent version:the mechanism of asynchronous messaging between agents is not reliable and you need to somehow live with it .
Here we need to make an important remark: it’s wrong to say that the message delivery mechanism in SObjectizer is really “leaky” and allows itself to lose any messages when it wants to.
Messages in SObjectizer are simply not lost, every loss has its own reason. If the agent sends a message to itself and the send function succeeds, the message will reach the agent. Unless the developer himself takes explicitly some action requiring the SObjectizer to throw this message in a specific case (for example, the developer does not subscribe the agent to the message in any of the states or uses limit_then_drop to protect it from overload).
So, if the developer himself does not allow the SObjectizer to throw out certain messages in certain situations, then the message that the agent sent to himself should reach the agent. Therefore, in the code shown above, we completely calmly sent pending messages to ourselves without fear that these messages would be lost somewhere along the way.
However, when a message is sent to another agent, the situation changes somewhat. There are times when we are confident in the success of delivery. For example, if we ourselves implemented a recipient agent, and even included it in the same cooperation in which the sender agent lives.
But if the recipient agent is not written by us, it is created and destroyed as part of another's cooperation, if we do not control its behavior, if we do not know how the agent is protected from overloads, how it behaves in a given situation, then we have confidence the same as when sending a datagram via UDP protocol: if everything is fine, then most likely the datagram will reach the sender, and then we will get a response. If everything is fine. But if not?
We came to an interesting point: the development of software on actors / agents due to the relative unreliability of asynchronous messaging can look more time-consuming than using approaches based on the synchronous interaction of objects in the program.
Sometimes it is. But in the end, in our opinion, the software is more reliable, because in the code, one has to handle a lot of contingencies related to both the loss of messages and variations in the delivery and processing times of messages.
Suppose email_analyzers access io_agent via a synchronous request rather than an asynchronous message, and io_agent informs about failures during the IO operation by throwing exceptions. For a long time, everything will work fine: email_analyzer synchronously requests io_agent and receives either the contents of the email or an exception in response. But at one point, a hidden bug appears somewhere inside io_agent, and the synchronous call just hangs. No answer, no exception, just freezing. Accordingly, one email_analyzer first hangs, then another, then another, etc. As a result, the entire application is suspended.
Whereas with asynchronous messaging, the io_agent agent can hang somewhere in its giblets. But this will not affect email_analyzer agents, which can easily track the expiration of a request timeout and send a negative result. That is, even in case of failures in one part of the application, other parts of the application will be able to continue their work, let this work consist in generating a stream of negative responses. Indeed, the very fact of this flow can become an important symptom and tell the observer that something went wrong in the application.
By the way, on the topic of monitoring the work of an application written on agents.
Over the years of working with SObjectizer, we have developed the belief that the ability to see what is happening inside an application built on actors / agents is very important. In principle, this has been shown even in this article. If you take the fifth version of email_analyzer without timeout control and try to run it, you can see how query processing slows down until it stops at all. But how exactly to understand what is the matter?
Information about how many email_analyzer agents are currently created and what each of them is doing could give a good clue. This requires the ability to monitor what is happening inside the application. Just what Erlang and its platform are so valued for: there you can connect to a working Erlang VM, see a list of Erlang processes, their parameters, etc. But in Erlang, this is possible due to the fact that the Erlang application is running Erlang VM.
In the case of the native C ++ application, the situation is more complicated. Tools have been added to SObjectizer for monitoring what is happening inside the SObjectizer Environment (although these tools still provide only the most basic functionality). So, with their help, during the operation of our demo application, you can get the following information:
This exhaust of monitoring information allows us to understand that there is a dispatcher with a thread pool called “analyzers”, in which 4 workflows work. It is on this dispatcher that the email_analyzer agents work in the example. 16 agents are attached to the dispatcher, each of which constitutes a separate cooperation. And these agents have no applications. That is, there are agents, but there is no work for them. And this is an occasion to figure out why this happened.
Obviously, the low-level information available to the SObjectizer Environment is far from always useful to the application programmer. Say, in the example under discussion, the counter of the number of email_analyzer agents and the size of the length of the list of requests in the analyzer_manager agent could give a developer much more benefit. But this is applied data, SObjectizer has no idea about it. Therefore, when developing an application on agents, the programmer needs to make sure that outside the application information is available that is most useful for assessing the health and viability of the application. Although this is already a big topic for another discussion.
Perhaps this is the next article you can finish. In the next article, we will try to show what can be done if we try to parallelize email analysis operations even more. Say, if you perform parallel analysis of the headers, body of the letter and attachments. And then what will the email_analyzer agent code turn into.
The source codes for the examples shown in the article can be found in this repository .
Last time, we settled on the fact that the operation of reading the contents of a file with an email should be left to a separate IO agent. Let's do it and see what happens.
First, we need a set of messages that the IO agent and email_analyzer will exchange among themselves:
// Запрос на загрузку содержимого файла.
struct load_email_request
{
// Имя файла для загрузки.
string email_file_;
// Куда нужно прислать результат.
mbox_t reply_to_;
};
// Успешный результат загрузки файла.
struct load_email_succeed
{
// Содержимое файла.
string content_;
};
// Неудачный результат загрузки файла.
struct load_email_failed
{
// Описание причины неудачи.
string what_;
};
Secondly, we need to determine where exactly the email_analyzer agent will send the request message load_email_request. We could have gone the usual way: when registering an IO agent, save its direct_mbox, then pass this mbox as a parameter to the analyzer_manager agent constructor, then pass a parameter to the email_analyzer agent constructor for each agent ... Basically, if we would need to have several different IO- agents, that’s how it should be done. But in our problem, one IO agent is quite enough. Which allows us to demonstrate named mboxes.
A named mbox is created by calling so_5 :: environment_t :: create_mbox (name). If you call create_mbox several times with the same name, then the same mbox will always be returned, created during the first call to create_mbox with this name.
The IO agent creates a named mbox and subscribes to it. Email_analyzer agents receive the same mbox when they need to send a load_email_request message. Thus, we get rid of the need to "drag" the mbox IO agent through analyzer_manager.
Now that we have decided on the interface between the IO agent and email_manager, we can make a new version of the email_analyzer agent :
// Пятая версия. С передачей IO-операции специальному IO-агенту.
class email_analyzer : public agent_t {
public :
email_analyzer( context_t ctx,
string email_file,
mbox_t reply_to )
: agent_t(ctx), email_file_(move(email_file)), reply_to_(move(reply_to))
{}
// Агент усложнился, у него появилось несколько обработчиков событий.
// Поэтому подписки агента лучше определять в специально предназначенном
// для этого виртуальном методе.
virtual void so_define_agent() override {
// Нам нужно получить два сообщения от IO-агента. Каждое
// из эти сообщений будет обрабатываться своим событием.
so_subscribe_self()
.event( &email_analyzer::on_load_succeed )
.event( &email_analyzer::on_load_failed );
}
virtual void so_evt_start() override {
// При старте сразу же отправляем запрос IO-агенту для загрузки
// содержимого email файла.
send< load_email_request >(
// mbox IO-агента будет получен по имени.
so_environment().create_mbox( "io_agent" ),
email_file_,
// Ответ должен прийти на наш собственный mbox.
so_direct_mbox() );
}
private :
const string email_file_;
const mbox_t reply_to_;
void on_load_succeed( const load_email_succeed & msg ) {
try {
// Стадии обработки обозначаем лишь схематично.
auto parsed_data = parse_email( msg.content_ );
auto status = check_headers( parsed_data->headers() );
if( check_status::safe == status )
status = check_body( parsed_data->body() );
if( check_status::safe == status )
status = check_attachments( parsed_data->attachments() );
send< check_result >( reply_to_, email_file_, status );
}
catch( const exception & ) {
// В случае какой-либо ошибки отсылаем статус о невозможности
// проверки файла с email-ом по техническим причинам.
send< check_result >(
reply_to_, email_file_, check_status::check_failure );
}
// Больше мы не нужны, поэтому дерегистрируем кооперацию,
// в которой находимся.
so_deregister_agent_coop_normally();
}
void on_load_failed( const load_email_failed & ) {
// Загрузить файл не удалось. Возвращаем инициатору запроса
// отрицательный результат и завершаем свою работу.
send< check_result >(
reply_to_, email_file_, check_status::check_failure );
so_deregister_agent_coop_normally();
}
};Email_analyzer agents now delegate IO operations to another agent who knows how to do this efficiently. Accordingly, the email_analyzer agents on their work threads will either deal with the assignment of tasks to the IO agent, or with the processing of email_analyzer replies. This gives us the opportunity to change our view of how many email_analyzer agents we can create and how many work threads they need.
When each email_analyzer agent itself performed a synchronous IO operation, we needed to have as many work threads in the pool as we wanted to allow parallel IO operations. At the same time, there was no point in creating much more email_analyzer agents than the number of work threads in the pool. If there are 16 threads in the pool, and we allow 32 agents to exist at the same time, then this will lead to the fact that half of these agents will simply wait for any of the working threads to be released for them.
Now, after the IO operations are moved to a different work context, you can, firstly, reduce the number of work threads in the pool. Email_analyzer agents in their events will perform mainly CPU-loading operations. Therefore, it makes no sense to create more work flows than there are available processing cores. So, if we have a 4-core processor, then we need not 16 threads in the pool, but not more than 4.
Secondly, if IO operations take longer than processing the contents of email, then we get the opportunity to create more email_analyzer agents than threads in the pool. Simply, most of these agents will wait for the result of their IO operation. Although, if the download time of an email is comparable or less than the time of analyzing its contents, this item will lose its relevance and we will be able to create only 1-2-3 email_analyzer agents more than the number of threads in the pool. All these settings are easily made in one place - in the analyzer_manager agent. It is enough to change just a couple of constants in its code and see how the changes affect the performance of our solution. However, performance tuning is a separate big topic and it’s premature to delve into it now ...
So, we have the next version of the email_analyzer agent, which fixes the problems of previous versions. Can we consider it acceptable?
Not.
The problem is that the resulting implementation cannot be considered reliable.
This implementation is designed for an optimistic scenario in which the messages we send are never lost and always reach the addressee, and when they reach the addressee, they are always processed. After which we always get the answer we need.
The harsh truth of life, however, is that when a system is built on an asynchronous message exchange between individual actors / agents, this same asynchronous exchange cannot be considered an absolutely reliable thing. Messages may be lost. And this is normal .
Loss of messages can occur for various reasons. For example, the receiving agent has not yet managed to subscribe to the message. Or there is no recipient agent at all at the moment. Either it is, but it has worked overload protection mechanism (more on this in a subsequent article). Either there is an agent and the message even reached him, but the agent is in a state in which this message is not processed. Either there is an agent, the message reached him, he even began to process it, but during the processing some kind of application error occurred and the failed agent did not send anything in response.
In general, communication between agents through asynchronous messages is like host interaction through the UDP protocol. In most cases, datagrams reach the recipients. But sometimes they get lost along the way or even during processing.
The above means that load_email_request may not reach the IO agent. Or, the response messages load_email_successed / load_email_failed may not reach the email_analyzer agent. And what will happen in this case?
We will get an email_analyzer agent that is present in the system but does nothing. Does not work. Not going to die. And does not start any other agent email_analyzer. If we are not lucky, then we may encounter a situation where all the email_analyzer agents created by the analyzer_manager will turn into half-corpses that do nothing. After that, analyzer_manager will simply accumulate requests in its turn, and then throw them out after the timeout expires. But no useful work will be performed.
How to get out of this situation?
For example, due to the control of timeouts. We can either introduce control of the execution time of the IO operation by the email_analyzer agent (i.e., if there is no answer for too long, then assume that the IO operation failed). Or, you can enter control over the execution time of the entire email analysis operation in the analyzer_manager agent. Or do both.
For simplicity, we restrict ourselves to counting the timeout of the IO operation in the email_analyzer agent :
// Шестая версия. С контролем тайм-аута для ответа IO-агента.
class email_analyzer : public agent_t {
// Этот сигнал потребуется для того, чтобы отслеживать отсутствие
// ответа от IO-агента в течении разумного времени.
struct io_agent_response_timeout : public signal_t {};
public :
email_analyzer( context_t ctx,
string email_file,
mbox_t reply_to )
: agent_t(ctx), email_file_(move(email_file)), reply_to_(move(reply_to))
{}
virtual void so_define_agent() override {
so_subscribe_self()
.event( &email_analyzer::on_load_succeed )
.event( &email_analyzer::on_load_failed )
// Добавляем еще обработку тайм-аута на ответ IO-агента.
.event< io_agent_response_timeout >( &email_analyzer::on_io_timeout );
}
virtual void so_evt_start() override {
// При старте сразу же отправляем запрос IO-агенту для загрузки
// содержимого email файла.
send< load_email_request >(
so_environment().create_mbox( "io_agent" ),
email_file_,
so_direct_mbox() );
// И сразу же начинам отсчет тайм-аута для ответа от IO-агента.
send_delayed< io_agent_response_timeout >( *this, 1500ms );
}
private :
const string email_file_;
const mbox_t reply_to_;
void on_load_succeed( const load_email_succeed & msg ) {
try {
auto parsed_data = parse_email( msg.content_ );
auto status = check_headers( parsed_data->headers() );
if( check_status::safe == status )
status = check_body( parsed_data->body() );
if( check_status::safe == status )
status = check_attachments( parsed_data->attachments() );
send< check_result >( reply_to_, email_file_, status );
}
catch( const exception & ) {
send< check_result >(
reply_to_, email_file_, check_status::check_failure );
}
so_deregister_agent_coop_normally();
}
void on_load_failed( const load_email_failed & ) {
send< check_result >(
reply_to_, email_file_, check_status::check_failure );
so_deregister_agent_coop_normally();
}
void on_io_timeout() {
// Ведем себя точно так же, как и при ошибке ввода-вывода.
send< check_result >(
reply_to_, email_file_, check_status::check_failure );
so_deregister_agent_coop_normally();
}
};This option email_analyzer can already be considered quite acceptable. In his code, refactoring suggests taking a couple of operations (send and so_deregister_agent_coop_normally) into a separate helper method. But this was not done on purpose, so that the code for each subsequent version of the email_analyzer agent would be minimally different from the code for the previous version.
And just if you compare the two versions of the email_analyzer agent shown above, you will notice one feature that is very appreciated by programmers who have long used SObjectizer in their daily work: the simplicity and clarity of the agent extension procedure. Did the agent need to respond to another event? So you need to add another subscription and another event handler. And since subscriptions are usually made in the same places, it’s immediately clear where to go and what to edit.
SObjectizer does not impose any restrictions on where and how the agent writes its events, but following a simple agreement - subscriptions are done in so_define_agent (), or, in very simple cases, for the final classes of agents, in the constructor - which greatly simplifies life . You look at the code of someone else’s agent or even the code of your agent, but written several years ago, and you immediately know what you need to look to understand the behavior of the agent. It’s convenient, although to understand this convenience, you probably need to write and debug more than one real agent, or even two ...
However, let us return to the topic of agent reliability, which was raised above and because of which the next, sixth, appeared email_analyzer agent version:the mechanism of asynchronous messaging between agents is not reliable and you need to somehow live with it .
Here we need to make an important remark: it’s wrong to say that the message delivery mechanism in SObjectizer is really “leaky” and allows itself to lose any messages when it wants to.
Messages in SObjectizer are simply not lost, every loss has its own reason. If the agent sends a message to itself and the send function succeeds, the message will reach the agent. Unless the developer himself takes explicitly some action requiring the SObjectizer to throw this message in a specific case (for example, the developer does not subscribe the agent to the message in any of the states or uses limit_then_drop to protect it from overload).
So, if the developer himself does not allow the SObjectizer to throw out certain messages in certain situations, then the message that the agent sent to himself should reach the agent. Therefore, in the code shown above, we completely calmly sent pending messages to ourselves without fear that these messages would be lost somewhere along the way.
However, when a message is sent to another agent, the situation changes somewhat. There are times when we are confident in the success of delivery. For example, if we ourselves implemented a recipient agent, and even included it in the same cooperation in which the sender agent lives.
But if the recipient agent is not written by us, it is created and destroyed as part of another's cooperation, if we do not control its behavior, if we do not know how the agent is protected from overloads, how it behaves in a given situation, then we have confidence the same as when sending a datagram via UDP protocol: if everything is fine, then most likely the datagram will reach the sender, and then we will get a response. If everything is fine. But if not?
We came to an interesting point: the development of software on actors / agents due to the relative unreliability of asynchronous messaging can look more time-consuming than using approaches based on the synchronous interaction of objects in the program.
Sometimes it is. But in the end, in our opinion, the software is more reliable, because in the code, one has to handle a lot of contingencies related to both the loss of messages and variations in the delivery and processing times of messages.
Suppose email_analyzers access io_agent via a synchronous request rather than an asynchronous message, and io_agent informs about failures during the IO operation by throwing exceptions. For a long time, everything will work fine: email_analyzer synchronously requests io_agent and receives either the contents of the email or an exception in response. But at one point, a hidden bug appears somewhere inside io_agent, and the synchronous call just hangs. No answer, no exception, just freezing. Accordingly, one email_analyzer first hangs, then another, then another, etc. As a result, the entire application is suspended.
Whereas with asynchronous messaging, the io_agent agent can hang somewhere in its giblets. But this will not affect email_analyzer agents, which can easily track the expiration of a request timeout and send a negative result. That is, even in case of failures in one part of the application, other parts of the application will be able to continue their work, let this work consist in generating a stream of negative responses. Indeed, the very fact of this flow can become an important symptom and tell the observer that something went wrong in the application.
By the way, on the topic of monitoring the work of an application written on agents.
Over the years of working with SObjectizer, we have developed the belief that the ability to see what is happening inside an application built on actors / agents is very important. In principle, this has been shown even in this article. If you take the fifth version of email_analyzer without timeout control and try to run it, you can see how query processing slows down until it stops at all. But how exactly to understand what is the matter?
Information about how many email_analyzer agents are currently created and what each of them is doing could give a good clue. This requires the ability to monitor what is happening inside the application. Just what Erlang and its platform are so valued for: there you can connect to a working Erlang VM, see a list of Erlang processes, their parameters, etc. But in Erlang, this is possible due to the fact that the Erlang application is running Erlang VM.
In the case of the native C ++ application, the situation is more complicated. Tools have been added to SObjectizer for monitoring what is happening inside the SObjectizer Environment (although these tools still provide only the most basic functionality). So, with their help, during the operation of our demo application, you can get the following information:
mbox_repository / named_mbox.count -> 1 coop_repository / coop.reg.count -> 20 coop_repository / coop.dereg.count -> 0 coop_repository / agent.count -> 20 coop_repository / coop.final.dereg.count -> 0 timer_thread / single_shot.count -> 0 timer_thread / periodic.count -> 1 disp / ot / DEFAULT / agent.count -> 3 disp / ot / DEFAULT / wt-0 / demands.count -> 8 disp / tp / analyzers / threads.count -> 4 disp / tp / analyzers / agent.count -> 16 disp / tp / analyzers / cq / __ so5_au ... 109 __ / agent.count -> 1 disp / tp / analyzers / cq / __ so5_au ... 109 __ / demands.count -> 0 disp / tp / analyzers / cq / __ so5_au ... 124 __ / agent.count -> 1 disp / tp / analyzers / cq / __ so5_au ... 124 __ / demands.count -> 0 ... disp / tp / analyzers / cq / __ so5_au ..._ 94 __ / agent.count -> 1 disp / tp / analyzers / cq / __ so5_au ..._ 94 __ / demands.count -> 0 disp / ot / req_initiator / agent.count -> 1 disp / ot / req_initiator / wt-0 / demands.count -> 0
This exhaust of monitoring information allows us to understand that there is a dispatcher with a thread pool called “analyzers”, in which 4 workflows work. It is on this dispatcher that the email_analyzer agents work in the example. 16 agents are attached to the dispatcher, each of which constitutes a separate cooperation. And these agents have no applications. That is, there are agents, but there is no work for them. And this is an occasion to figure out why this happened.
Obviously, the low-level information available to the SObjectizer Environment is far from always useful to the application programmer. Say, in the example under discussion, the counter of the number of email_analyzer agents and the size of the length of the list of requests in the analyzer_manager agent could give a developer much more benefit. But this is applied data, SObjectizer has no idea about it. Therefore, when developing an application on agents, the programmer needs to make sure that outside the application information is available that is most useful for assessing the health and viability of the application. Although this is already a big topic for another discussion.
Perhaps this is the next article you can finish. In the next article, we will try to show what can be done if we try to parallelize email analysis operations even more. Say, if you perform parallel analysis of the headers, body of the letter and attachments. And then what will the email_analyzer agent code turn into.
The source codes for the examples shown in the article can be found in this repository .
