February 8, 2013 at 10:13Application Caching Strategy
When it comes to caching, a paradoxical situation arises. On the one hand, everyone understands the importance and the need for caching in the application architecture. On the other hand, few can clearly explain what and how to cache.
Usually, people immediately start offering ready-made cache implementations, like memcached or HTTP cache, but this is just the answer to the question of where to cache.
Caching is one of many topics, along with security and logging that everyone knows and talks about, but few can do it right.
The cache brings data closer to where it is used. In the modern world, consisting of 98% of the Internet, data usually lies very far from the user. There are caches all the way from the repository to the user, which serve only one purpose - so that the user receives his data as quickly as possible.
If we take a closer look, we see that valuable time is wasted on processing data in the supplier and transferring data from the supplier to the client, we do not take into account the time of processing data on the client.
With high loads, caching is a must. It allows you to serve more customers with the same resources, because data providers are more relaxed. But even with low loads, caching positively affects the responsiveness of the application.
One of the main misconceptions about caching is that many people think that cache can simply be turned on.
At the dawn of my programming career, I once just turned on caching once, literally an hour later I had to turn it off. Then I ran into the main problem with caching - data obsolescence . After changing the data, the user did not see the result for 15 minutes.
It is very important to understand what and how you are going to cache, so as not to violate the logic of the application. And the first question you need to answer is how outdated data can be given to the client. Naturally, you can make your own cache for each client, this will simplify the solution of the issue of data relevance, but will bring many other problems.
There are three main types of caching for work mechanics:
Probably you can come up with other types of caches, but I have not met.
The cache size is always limited. Often it is less than the amount of data that can be put in this cache. Therefore, items placed in the cache will be crowded out sooner or later. Modern caching frameworks allow very flexible management of obsolescence, taking into account priorities, time of obsolescence, data volumes, etc.
If the same data falls into different caches, then the cache coherence problem arises. For example, the same data is used to form different pages and pages are cached. Pages formed later will contain updated data, and pages cached earlier will contain outdated data. Thus, the consistency of behavior will be violated.
A simple way to maintain coherence is to force the cache to become obsolete (reset) when data changes. Therefore, increasing the memory for the cache so that it is less obsolete is not always a good idea.
The main parameter that characterizes the cache system is the percentage of requests that hit the cache . This setting is pretty easy to measure to see how efficient your caching system is.
Frequent cache flushes, caching of rarely requested data, insufficient cache size - all this leads to a waste of RAM (usually) memory, without increasing work efficiency.
Sometimes the data changes so often and unpredictably that caching will have no effect, the percentage of hits will be close to zero. But usually, data is read much more often than it is written, so caches are efficient.
This is the simplest type of caching, but it must be used carefully, as it returns obsolete data. You can flush the lazy cache with each entry to keep the data up-to-date, but then the implementation costs will be comparable to more complex types of caching.
This type of caching can be used for data that almost never changes. Another use case is to make a lazy cache with a short aging time for stable operation during bursts of load.
This type of caching allows you to give the answer the fastest.
This is the most useful type of caching, because it gives up fresh data and allows you to implement a multi-level cache.
This type of caching is built into the HTTP protocol. The server gives the change label, and the client caches your result and in the next request passes this label. The server can give an answer that the state has not changed and you can use the object cached on the client. The server, in turn, having received the label can ask the repository whether there were changes or not.
This type of caching does not eliminate the overhead of communication between systems. Therefore, it is often supplemented by other types of caching in order to speed up the work.
If there is a distributed caching system (memcached, Windows Sever App Fabric, Azure Cache), then you can use the write-through cache. The hand-to-hand implementation of synchronizing caches between nodes is in itself a separate large project, so you should not deal with it as part of application development.
You should not try to cache everything in a synchronized cache, otherwise most of the application code will rebuild the cache.
Also, do not forget that distributed caching systems also require communication between systems, which can affect performance.
Choose the correct granularity of the cached data. For example, caching data for each user is most likely to be inefficient with a large number of users. If you cache data for all users at once, then there will be problems with data obsolescence and cache coherency.
Cache data as late as possible, immediately before uploading it to an external system. Caching data received from the outside is necessary only in case of performance problems at this stage. External storages, such as DBMS and file systems, implement caching themselves, so usually it makes no sense to cache query results.
You do not need to fence your bikes for caching in applications, usually there are already ready-made tools and you need to be able to use them.
I hope the article was interesting and useful for you. Comment, rate, I will be glad to any suggestions.
Usually, people immediately start offering ready-made cache implementations, like memcached or HTTP cache, but this is just the answer to the question of where to cache.
Caching is one of many topics, along with security and logging that everyone knows and talks about, but few can do it right.
Why cache needed
The cache brings data closer to where it is used. In the modern world, consisting of 98% of the Internet, data usually lies very far from the user. There are caches all the way from the repository to the user, which serve only one purpose - so that the user receives his data as quickly as possible.
If we take a closer look, we see that valuable time is wasted on processing data in the supplier and transferring data from the supplier to the client, we do not take into account the time of processing data on the client.
With high loads, caching is a must. It allows you to serve more customers with the same resources, because data providers are more relaxed. But even with low loads, caching positively affects the responsiveness of the application.
Cache can't just be turned on
One of the main misconceptions about caching is that many people think that cache can simply be turned on.
At the dawn of my programming career, I once just turned on caching once, literally an hour later I had to turn it off. Then I ran into the main problem with caching - data obsolescence . After changing the data, the user did not see the result for 15 minutes.
It is very important to understand what and how you are going to cache, so as not to violate the logic of the application. And the first question you need to answer is how outdated data can be given to the client. Naturally, you can make your own cache for each client, this will simplify the solution of the issue of data relevance, but will bring many other problems.
Types of caching
There are three main types of caching for work mechanics:
- Lazy cache, it's a lazy cache , it's a dumb cache - the easiest type of caching to implement, often built into frameworks. The cache simply saves the data and gives it until it is out of date.
- Synchronized cache, synchronized cache - the client, together with the data, gets the label of the last change and may ask the provider if the data has changed, so as not to request again. This type of caching allows you to always have fresh data, but it is very difficult to implement.
- Write-through cache, or cache write-through - any change in the data is performed directly in the store and in the cache. This type of cache may never expire, but there are problems with the so-called “ coherence ”.
Probably you can come up with other types of caches, but I have not met.
Obsolescence and cache coherency
The cache size is always limited. Often it is less than the amount of data that can be put in this cache. Therefore, items placed in the cache will be crowded out sooner or later. Modern caching frameworks allow very flexible management of obsolescence, taking into account priorities, time of obsolescence, data volumes, etc.
If the same data falls into different caches, then the cache coherence problem arises. For example, the same data is used to form different pages and pages are cached. Pages formed later will contain updated data, and pages cached earlier will contain outdated data. Thus, the consistency of behavior will be violated.
A simple way to maintain coherence is to force the cache to become obsolete (reset) when data changes. Therefore, increasing the memory for the cache so that it is less obsolete is not always a good idea.
Cache performance
The main parameter that characterizes the cache system is the percentage of requests that hit the cache . This setting is pretty easy to measure to see how efficient your caching system is.
Frequent cache flushes, caching of rarely requested data, insufficient cache size - all this leads to a waste of RAM (usually) memory, without increasing work efficiency.
Sometimes the data changes so often and unpredictably that caching will have no effect, the percentage of hits will be close to zero. But usually, data is read much more often than it is written, so caches are efficient.
Using different types of caching
Lazy cache
This is the simplest type of caching, but it must be used carefully, as it returns obsolete data. You can flush the lazy cache with each entry to keep the data up-to-date, but then the implementation costs will be comparable to more complex types of caching.
This type of caching can be used for data that almost never changes. Another use case is to make a lazy cache with a short aging time for stable operation during bursts of load.
This type of caching allows you to give the answer the fastest.
Synchronized cache
This is the most useful type of caching, because it gives up fresh data and allows you to implement a multi-level cache.
This type of caching is built into the HTTP protocol. The server gives the change label, and the client caches your result and in the next request passes this label. The server can give an answer that the state has not changed and you can use the object cached on the client. The server, in turn, having received the label can ask the repository whether there were changes or not.
This type of caching does not eliminate the overhead of communication between systems. Therefore, it is often supplemented by other types of caching in order to speed up the work.
Write-through cache
If there is a distributed caching system (memcached, Windows Sever App Fabric, Azure Cache), then you can use the write-through cache. The hand-to-hand implementation of synchronizing caches between nodes is in itself a separate large project, so you should not deal with it as part of application development.
You should not try to cache everything in a synchronized cache, otherwise most of the application code will rebuild the cache.
Also, do not forget that distributed caching systems also require communication between systems, which can affect performance.
What else to consider in your caching strategy
Choose the correct granularity of the cached data. For example, caching data for each user is most likely to be inefficient with a large number of users. If you cache data for all users at once, then there will be problems with data obsolescence and cache coherency.
Cache data as late as possible, immediately before uploading it to an external system. Caching data received from the outside is necessary only in case of performance problems at this stage. External storages, such as DBMS and file systems, implement caching themselves, so usually it makes no sense to cache query results.
You do not need to fence your bikes for caching in applications, usually there are already ready-made tools and you need to be able to use them.
Conclusion
I hope the article was interesting and useful for you. Comment, rate, I will be glad to any suggestions.