SPA architecture for CRM systems: part 1
Introduction
Recently, I was faced with a project to finalize the once written CRM. The purpose of the refinement was to increase the system’s speed when interacting with the user and add a bit of new functionality, as well as defeat memory leaks discovered by previous developers and not yet defeated in JavaScript, on which the entire user interface was implemented.
Having started to engage in the project, delving into the bowels of a huge number of libraries and frameworks used and not very friendly interacting with each other, having conducted a number of experiments, we came to an unexpected conclusion that everything was to blame ... SPA architecture.
A few words about SPA architecture
Speaking about the modern Web, one can increasingly hear about the technology of Single Page Application (SPA), although to be precise, SPA is the collective name of a set of technologies that allow you to implement a WEB application executed by a WEB browser as a single WEB page, like , for example, implemented the Gmail service from Google. From the point of view of the user, this technology attracts primarily by the quick response to actions in the user interface, since it does not require a complete or even partial reload of the WEB page from the server, and all visual elements are constructed directly in the browser using JavaScript by manipulating the DOM- document structure.
Thus, WEB applications become very similar to ordinary applications for workstations downloading information from the Internet, only the runtime environment for them is not the operating system, but the browser, which as a result is forced to bear the entire load associated with the execution of third-party code namely, memory management, ensuring a safe environment, providing functionality for working with system functions and hardware environment, etc.
The central place of SPA-architecture is the View - what the user sees and interacts with. The result of the presentation is the most common HTML displayed by the browser. Unlike “transient” “WEB 2.0” applications that actively work with the DOM structure of the document, for example, using jQuery or underscore, the SPA application uses the DOM only for recording changes, but not for reading, that is, not for storing data . To store data, another component of SPA architecture is now used - Model.
A model is a collection of data, functions for manipulating data and events. All model data is fully stored in memory. In order for the data in the model and the data displayed by the view to maintain integrity, the view subscribes to model events, thus tracking changes in the data in the model. In turn, the model also responds to presentation notifications and provides an inextricable communication between the WEB application and the server, performing requests to receive or send data (in particular, using the REST methodology).
But back to the view. Performance is the most important and most complex part of modern SPA. Usually, representations are built around the so-called templates - blanks that are converted to HTML. Also, the view updates the HTML received when the model changes and vice versa - notifies the model of user actions with the view, for example, a mouse click, keyboard input or device rotation, as a result of which the model can perform data manipulations and then again notify the view of the data change for for the view to update or generate new HTML.
The work of the classic WEB-application (or WEB-site) is completely built on top of data caching: on the server, on the proxy server and on the client. If the data and application state are updated very often, the advantage of using caching is almost leveled. Theoretically, a single-page application should use the cache less, since the data is loaded once during the page life cycle, but in practice this is not always the case, and we will talk about this below.
Features of CRM that do not fit into the SPA implementation
Let's see how we can use SPA technology to implement a simple customer relationship management system, or CRM. First of all, we are interested in the operational and analytical levels of information processing.
Take for example a very simplified set of sections of simple CRM:
• Desktop (Dashboard) - a summary of all the system data that makes sense for a particular user.
• Events - view the joint actions of users of marketing, sales and production departments.
• Clients - management of the client base, contacts and companies.
• Projects and transactions - customer relationship management.
• Tasks - implementation workflow management.
• Reports - viewing and managing analytical reports on the accumulated information.
• Profile - manage your user profile.
In each section, the user works with samples of frequently modified data. One of the critical requirements for CRM at the operational level is high performance when interacting with the user when performing operations such as frequent access to the information registry, the active use of complex filtering and sorting of a large amount of results. Also at the analytical level, you need to quickly receive reports, statistics, analysis and forecasting for various metrics and indicators.
In a one-page application, we operate with “screens” instead of pages. Each screen is part of a subsystem with its own set of templates, modules, controllers, route and model. When the user navigates to another section of the application, the data, templates and modules required for the operation of this section are loaded, it is important that the user is on the same page, and the current state while loading a new screen is saved.
The general model of the entire system is complex, consisting of independent models for each screen. Switching between screens clears the current layer of the view and generates a new layer, which is often implemented in such a way that the previous layer simply becomes invisible and a new one is displayed instead, that is, the process of switching between screens is arranged in such a way that ready-made views after initial loading in hereinafter simply hidden and displayed.
This is very different from the classical way of working with a WEB application, where clicking on links causes a full page load, which automatically causes the release of resources and updating data, and the user can expect that the data is always up to date. In the context of rapidly changing data in CRM, in order to confirm the user’s expectations during the SPA implementation, in the process of switching between screens, it also becomes necessary to reload all the data from the server, which negates the advantage of storing all the data on the client, and, consequently, the application itself SPA architecture.
Problems encountered when using SPA in CRM
SPA technology forces the developer to write code very carefully, since any error or oversight can lead to memory leaks, and as a result, to the “brakes” of the entire system.
Why is memory flowing?
A huge amount of data accumulated on the client during long work, very quickly become irrelevant. But meanwhile, this data is stored in the model, and the views are stored in the DOM structure of the document, if you do not purge purposefully. In order to keep the model and the DOM “clean”, it is necessary to engage in periodic “garbage collection”, since you should not rely on the built-in JIT garbage collector in this case, because object references, as a rule, remain achievable, therefore, previously created and objects no longer needed are still in memory. It is also necessary to consider that in JavaScript there is always a risk of losing all links, but the data will not be cleared.
Why is everything slowing down?
The slowdown of the page is mainly caused by memory leaks and a complex model with a large number of event handlers. When switching from screen to screen, when each new form is opened, data is loaded into the browser process, as well as executable code (in the case of AMD), which is often implemented using eval () constructs. Modular frameworks for building SPA also have their own infrastructure with their own costs. The most common reason is the developer's mistakes and shortcomings, which are very easy to make and extremely difficult to track. Profiling and debugging complex SPAs is expensive: although debugging tools are already quite advanced, the complexity of debugging grows exponentially with the increasing complexity of the application. As a result, the problem is solved only by modular debugging and testing,
How are these issues reflected in CRM development?
When developing CRM, a large number of screens and forms, different logic depending on the type of user, their rights and permissions, very outdated data, the need for periodic updating of the state of the entire system are the main factors that complicate the development of CRM on SPA technology. In addition, during operation, as the data volume grows, the model grows, and, consequently, the time for processing the data increases, as a result, the system begins to slow down even where it behaved acceptable in tests.
Multiple Screen Problem
Users of tabbed browsers on the same page often open separate pages by links in separate browser tabs, combining them with clicks on links within the same tab. I would like to have such an opportunity when working with a WEB application: for example, to open the project page in a separate tab and keep it on the pulse. In the case of SPA, this is also possible, but in this case, development overheads increase sharply - where there was a saving in the case of loading pages, now there is an overrun, since in each tab the application will download all the code it needs to work; the meaning of SPA as a one-page application is lost, because it is obvious to the Internet user that working in a browser, the link should open a new page, and not behave like a normal application.
Thus, in the development of CRM systems, the SPA architecture is, in our opinion, less preferable than the classical architecture of a multi-page application with the active use of AJAX to update data.
The problem statement turned out to be quite extensive, so we leave the story of its solution for a separate material. Thanks to everyone who read, stay tuned.
Recently, I was faced with a project to finalize the once written CRM. The purpose of the refinement was to increase the system’s speed when interacting with the user and add a bit of new functionality, as well as defeat memory leaks discovered by previous developers and not yet defeated in JavaScript, on which the entire user interface was implemented.
Having started to engage in the project, delving into the bowels of a huge number of libraries and frameworks used and not very friendly interacting with each other, having conducted a number of experiments, we came to an unexpected conclusion that everything was to blame ... SPA architecture.
A few words about SPA architecture
Speaking about the modern Web, one can increasingly hear about the technology of Single Page Application (SPA), although to be precise, SPA is the collective name of a set of technologies that allow you to implement a WEB application executed by a WEB browser as a single WEB page, like , for example, implemented the Gmail service from Google. From the point of view of the user, this technology attracts primarily by the quick response to actions in the user interface, since it does not require a complete or even partial reload of the WEB page from the server, and all visual elements are constructed directly in the browser using JavaScript by manipulating the DOM- document structure.
Thus, WEB applications become very similar to ordinary applications for workstations downloading information from the Internet, only the runtime environment for them is not the operating system, but the browser, which as a result is forced to bear the entire load associated with the execution of third-party code namely, memory management, ensuring a safe environment, providing functionality for working with system functions and hardware environment, etc.
The central place of SPA-architecture is the View - what the user sees and interacts with. The result of the presentation is the most common HTML displayed by the browser. Unlike “transient” “WEB 2.0” applications that actively work with the DOM structure of the document, for example, using jQuery or underscore, the SPA application uses the DOM only for recording changes, but not for reading, that is, not for storing data . To store data, another component of SPA architecture is now used - Model.
A model is a collection of data, functions for manipulating data and events. All model data is fully stored in memory. In order for the data in the model and the data displayed by the view to maintain integrity, the view subscribes to model events, thus tracking changes in the data in the model. In turn, the model also responds to presentation notifications and provides an inextricable communication between the WEB application and the server, performing requests to receive or send data (in particular, using the REST methodology).
But back to the view. Performance is the most important and most complex part of modern SPA. Usually, representations are built around the so-called templates - blanks that are converted to HTML. Also, the view updates the HTML received when the model changes and vice versa - notifies the model of user actions with the view, for example, a mouse click, keyboard input or device rotation, as a result of which the model can perform data manipulations and then again notify the view of the data change for for the view to update or generate new HTML.
The work of the classic WEB-application (or WEB-site) is completely built on top of data caching: on the server, on the proxy server and on the client. If the data and application state are updated very often, the advantage of using caching is almost leveled. Theoretically, a single-page application should use the cache less, since the data is loaded once during the page life cycle, but in practice this is not always the case, and we will talk about this below.
Features of CRM that do not fit into the SPA implementation
Let's see how we can use SPA technology to implement a simple customer relationship management system, or CRM. First of all, we are interested in the operational and analytical levels of information processing.
Take for example a very simplified set of sections of simple CRM:
• Desktop (Dashboard) - a summary of all the system data that makes sense for a particular user.
• Events - view the joint actions of users of marketing, sales and production departments.
• Clients - management of the client base, contacts and companies.
• Projects and transactions - customer relationship management.
• Tasks - implementation workflow management.
• Reports - viewing and managing analytical reports on the accumulated information.
• Profile - manage your user profile.
In each section, the user works with samples of frequently modified data. One of the critical requirements for CRM at the operational level is high performance when interacting with the user when performing operations such as frequent access to the information registry, the active use of complex filtering and sorting of a large amount of results. Also at the analytical level, you need to quickly receive reports, statistics, analysis and forecasting for various metrics and indicators.
In a one-page application, we operate with “screens” instead of pages. Each screen is part of a subsystem with its own set of templates, modules, controllers, route and model. When the user navigates to another section of the application, the data, templates and modules required for the operation of this section are loaded, it is important that the user is on the same page, and the current state while loading a new screen is saved.
The general model of the entire system is complex, consisting of independent models for each screen. Switching between screens clears the current layer of the view and generates a new layer, which is often implemented in such a way that the previous layer simply becomes invisible and a new one is displayed instead, that is, the process of switching between screens is arranged in such a way that ready-made views after initial loading in hereinafter simply hidden and displayed.
This is very different from the classical way of working with a WEB application, where clicking on links causes a full page load, which automatically causes the release of resources and updating data, and the user can expect that the data is always up to date. In the context of rapidly changing data in CRM, in order to confirm the user’s expectations during the SPA implementation, in the process of switching between screens, it also becomes necessary to reload all the data from the server, which negates the advantage of storing all the data on the client, and, consequently, the application itself SPA architecture.
Problems encountered when using SPA in CRM
SPA technology forces the developer to write code very carefully, since any error or oversight can lead to memory leaks, and as a result, to the “brakes” of the entire system.
Why is memory flowing?
A huge amount of data accumulated on the client during long work, very quickly become irrelevant. But meanwhile, this data is stored in the model, and the views are stored in the DOM structure of the document, if you do not purge purposefully. In order to keep the model and the DOM “clean”, it is necessary to engage in periodic “garbage collection”, since you should not rely on the built-in JIT garbage collector in this case, because object references, as a rule, remain achievable, therefore, previously created and objects no longer needed are still in memory. It is also necessary to consider that in JavaScript there is always a risk of losing all links, but the data will not be cleared.
Why is everything slowing down?
The slowdown of the page is mainly caused by memory leaks and a complex model with a large number of event handlers. When switching from screen to screen, when each new form is opened, data is loaded into the browser process, as well as executable code (in the case of AMD), which is often implemented using eval () constructs. Modular frameworks for building SPA also have their own infrastructure with their own costs. The most common reason is the developer's mistakes and shortcomings, which are very easy to make and extremely difficult to track. Profiling and debugging complex SPAs is expensive: although debugging tools are already quite advanced, the complexity of debugging grows exponentially with the increasing complexity of the application. As a result, the problem is solved only by modular debugging and testing,
How are these issues reflected in CRM development?
When developing CRM, a large number of screens and forms, different logic depending on the type of user, their rights and permissions, very outdated data, the need for periodic updating of the state of the entire system are the main factors that complicate the development of CRM on SPA technology. In addition, during operation, as the data volume grows, the model grows, and, consequently, the time for processing the data increases, as a result, the system begins to slow down even where it behaved acceptable in tests.
Multiple Screen Problem
Users of tabbed browsers on the same page often open separate pages by links in separate browser tabs, combining them with clicks on links within the same tab. I would like to have such an opportunity when working with a WEB application: for example, to open the project page in a separate tab and keep it on the pulse. In the case of SPA, this is also possible, but in this case, development overheads increase sharply - where there was a saving in the case of loading pages, now there is an overrun, since in each tab the application will download all the code it needs to work; the meaning of SPA as a one-page application is lost, because it is obvious to the Internet user that working in a browser, the link should open a new page, and not behave like a normal application.
Thus, in the development of CRM systems, the SPA architecture is, in our opinion, less preferable than the classical architecture of a multi-page application with the active use of AJAX to update data.
The problem statement turned out to be quite extensive, so we leave the story of its solution for a separate material. Thanks to everyone who read, stay tuned.