Scaling out small Java Web applications (interviewing questions)
- Tutorial
This topic was raised in the course of several (3+) interviews that I went over the past month and a half - in different variations, but about one thing. It would seem, well-known things - but having collected all the answers and explanations that I gave (and found something later in Google), I decided not to keep them in my Google drive, but to write a short review.
It was about small and typical Enterprise / Web Java applications, of which there are many (well, such, for 10-100 thousand customers, a million visits, etc.). Let it be a generalized dialogue in the form of questions and answers.
Q: Suppose you have an application (the most common - JSP, Spring, Hibernate for example) deployed on a tomato (Apache Tomcat) and you once notice that the server with the tomato is 80% loaded on average. What to do?
A: Let's put several tomatoes on separate servers in parallel. They will still use the same database on the same server.
Q: But how will the user access your multiple servers?
A: We use a load-balancer, for example, apache (Apache httpd) with mod_proxy can stand in front of the tomkats - it will distribute (proxy) requests coming to it between all our tomkats.
Q: But it may turn out that the user logs on to one tomcat, and the next load-balancer request sends to another, where the user is not logged in!
A: We are talking about how to organize a session. For example, we do sticky sessions (for example, when the load-balancer adds a cookie to the request indicating which tomkat it proxies this request - and sends all subsequent requests with this cookie to the same server. Thus, each individual user will work with only one . server
: and if this particular server falls?
A:.. Session lost user is therefore better to use the session storage in the cache Tomkat "out of the box" is able to store them in a memcached instance So we we finish line in the configuration and on a separate server for. OSCAL the memcached - now all tomkaty hold session on it and if you hit with another request to another server, he did not even notice - the session will still work.
Q: What other benefits does the session cache have?
A: For example, you can deploy the new version of the application to only one of several tomkats, so say 25% of users see the new login page and have time to express us wishes if they do not like it, i.e. they involuntarily work as beta testers :)
Q: But if the application versions use the database in different ways?
A: We can design base changes to maintain backward compatibility between two neighboring versions. It is not hard. Add columns, for example, you need to along with the new version, but delete unnecessary only at the next release.
Q: Well, now our base is becoming a bottleneck. What will we do with increasing load on it?
A: First of all, it is useful to create a cache between the base and the tomkats. Even earlier, we probably use the cache at the ORM level (for example, the second cache level in Hibernate). The general point is that during the session, the user uses a limited set of data, so it is convenient to cache them.
Q: But still, let's say even the cache does not save us. How can I reduce the load on the base?
A: We have several ways. For example, you can allocate a part of the database (some particularly pumping table) to another database on a separate server, maybe even in NoSQL storage or some special cache. Of course, it is better to make this separation during designing :)
Q: And what are other ways? What are the solutions at the database level?
A: You can use sharding - in this case, the tables are divided into several servers and access to the necessary occurs, for example, regarding the id-shnik. In some cases, you can immediately divide, for example, transactions, transactions, electronic documents, etc. concerning one user since usually the user does not work with other people's documents - which means that all of his data can be conveniently stored on one server.
Q: What is the disadvantage of this approach?
A: It will subsequently become more difficult to work with such tables - join with a table lying on several servers will obviously be less efficient - indexing, queries by criteria, etc., become more complicated. In general, the design itself is significantly complicated.
Q: Well, do you know any other options?
A: The easiest way is to configure replication, for example, so that the database contains copies on several servers, one of which is used for writing, and the rest for reading. These latter quickly synchronize their content with updates. It turns out that the total number of queries to the database is now distributed across several machines. Of course, this is useful when reading more than writing.
Q: What further scaling paths could you offer?
A: For example, message queues. Let's say the user saves a new transaction - but we do not write it to the database ourselves. Instead, we send a message to the queue (say RabbitMQ) that such and such data should be saved. This message will be issued to one of several servers processing and saving to the database. Increasing the number of such servers (when using a distributed / replicated database or cache) is generally very easy. However, the architecture itself at this level already requires more attention and reflection - maybe even this is the moment when the application should be rewritten as a whole :)
Q: Okay, this is clear, let's talk about something else ... (and here they can start about garbage collectors, or ask to write a binary search in an array - check for lice - but it doesn’t matter anymore)
Having shared my “observations” on interviews, I will of course be glad of additions, corrections, etc. which may be useful to me and other colleagues :)
It was about small and typical Enterprise / Web Java applications, of which there are many (well, such, for 10-100 thousand customers, a million visits, etc.). Let it be a generalized dialogue in the form of questions and answers.
Q: Suppose you have an application (the most common - JSP, Spring, Hibernate for example) deployed on a tomato (Apache Tomcat) and you once notice that the server with the tomato is 80% loaded on average. What to do?
A: Let's put several tomatoes on separate servers in parallel. They will still use the same database on the same server.
Q: But how will the user access your multiple servers?
A: We use a load-balancer, for example, apache (Apache httpd) with mod_proxy can stand in front of the tomkats - it will distribute (proxy) requests coming to it between all our tomkats.
Q: But it may turn out that the user logs on to one tomcat, and the next load-balancer request sends to another, where the user is not logged in!
A: We are talking about how to organize a session. For example, we do sticky sessions (for example, when the load-balancer adds a cookie to the request indicating which tomkat it proxies this request - and sends all subsequent requests with this cookie to the same server. Thus, each individual user will work with only one . server
: and if this particular server falls?
A:.. Session lost user is therefore better to use the session storage in the cache Tomkat "out of the box" is able to store them in a memcached instance So we we finish line in the configuration and on a separate server for. OSCAL the memcached - now all tomkaty hold session on it and if you hit with another request to another server, he did not even notice - the session will still work.
Q: What other benefits does the session cache have?
A: For example, you can deploy the new version of the application to only one of several tomkats, so say 25% of users see the new login page and have time to express us wishes if they do not like it, i.e. they involuntarily work as beta testers :)
Q: But if the application versions use the database in different ways?
A: We can design base changes to maintain backward compatibility between two neighboring versions. It is not hard. Add columns, for example, you need to along with the new version, but delete unnecessary only at the next release.
Q: Well, now our base is becoming a bottleneck. What will we do with increasing load on it?
A: First of all, it is useful to create a cache between the base and the tomkats. Even earlier, we probably use the cache at the ORM level (for example, the second cache level in Hibernate). The general point is that during the session, the user uses a limited set of data, so it is convenient to cache them.
Q: But still, let's say even the cache does not save us. How can I reduce the load on the base?
A: We have several ways. For example, you can allocate a part of the database (some particularly pumping table) to another database on a separate server, maybe even in NoSQL storage or some special cache. Of course, it is better to make this separation during designing :)
Q: And what are other ways? What are the solutions at the database level?
A: You can use sharding - in this case, the tables are divided into several servers and access to the necessary occurs, for example, regarding the id-shnik. In some cases, you can immediately divide, for example, transactions, transactions, electronic documents, etc. concerning one user since usually the user does not work with other people's documents - which means that all of his data can be conveniently stored on one server.
Q: What is the disadvantage of this approach?
A: It will subsequently become more difficult to work with such tables - join with a table lying on several servers will obviously be less efficient - indexing, queries by criteria, etc., become more complicated. In general, the design itself is significantly complicated.
Q: Well, do you know any other options?
A: The easiest way is to configure replication, for example, so that the database contains copies on several servers, one of which is used for writing, and the rest for reading. These latter quickly synchronize their content with updates. It turns out that the total number of queries to the database is now distributed across several machines. Of course, this is useful when reading more than writing.
Q: What further scaling paths could you offer?
A: For example, message queues. Let's say the user saves a new transaction - but we do not write it to the database ourselves. Instead, we send a message to the queue (say RabbitMQ) that such and such data should be saved. This message will be issued to one of several servers processing and saving to the database. Increasing the number of such servers (when using a distributed / replicated database or cache) is generally very easy. However, the architecture itself at this level already requires more attention and reflection - maybe even this is the moment when the application should be rewritten as a whole :)
Q: Okay, this is clear, let's talk about something else ... (and here they can start about garbage collectors, or ask to write a binary search in an array - check for lice - but it doesn’t matter anymore)
Having shared my “observations” on interviews, I will of course be glad of additions, corrections, etc. which may be useful to me and other colleagues :)