April 14, 2011 at 09:02What types of OutOfMemoryError are there or what parts does the java process memory consist of
If you caught OutOfMemoryError, it doesn’t mean at all that your application creates many objects that cannot be cleaned by the garbage collector and fill all the memory that you selected using the -Xmx parameter. At a minimum, I can come up with two other cases where you can see this error. The fact is that the java process memory is not limited to the -Xmx area, where your application programmatically creates objects.
The memory area occupied by the java process consists of several parts. The type of OutOfMemoryError depends on which of them ran out of space.
There is not enough space on the heap, namely, in the memory area into which objects created programmatically in your application are placed. The size is specified by the -Xms and -Xmx options. If you try to create an object, and there is no space left on the heap, you get this error. Usually the problem lies in the memory leak, of which there are a great many, and the Internet is simply replete with articles on this topic.
This error occurs when there is not enough space in the Permanent area, the size of which is specified by the -XX: PermSize and -XX: MaxPermSize parameters. What lies there and how to deal with the OutOfMemoryError that arises there, I have already described in the most detailed way here .
This error can occur during overflow of the first and second areas. It is connected with the fact that there is little memory left and the GC is constantly working, trying to free up some space. This error can be disabled using the -XX: -UseGCOverheadLimit parameter, but, of course, you must not disable it, but either solve the memory leak problem, or allocate more volume, or change the GC settings.
I first encountered this error several years ago when I was doing load testing and trying to figure out the maximum number of users who can work with our web application. I used a special tool that allowed users to log in and emulate their standard actions. On a certain number of clients, I started getting OutOfMemoryError. Not really reading the message body and thinking that I do not have enough memory to create a user session and other necessary objects, I increased the size of the application heap (-Xmx). Imagine my surprise when after that the number of users simultaneously working with the system only decreased. Let's take a closer look at how this happened.
In fact, it is very simple to reproduce on windows on a 32-bit machine, since there the process allocates no more than 2 GB.
Suppose you have an application with a large number of concurrent users that starts with the parameters -Xmx1024M -XX: MaxPermSize = 256M -Xss512K. If 2G is available to the whole process, then another 768M will remain free. It is in this remaining memory that stacks of threads are created. Thus, approximately you can create no more than 768 * (1024/512) = 1536 (I managed to create 1316 with these parameters) threads (see the picture at the beginning of the article), after which you will get OutOfMemoryError. If you increase -Xmx, then the number of threads you can create will decrease accordingly. The option with decreasing -Xss, to be able to create more threads, is not always an option, since you probably have threads in the system that require quite large stacks. For example, an initialization thread or some background tasks. But still there is a solution. It turns out that when creating a thread programmatically, you can specify the stack size:Thread (ThreadGroup group, Runnable target, String name, long stackSize) . Thus, you can set -Xss to be quite small, and perform actions requiring large stacks in separate threads created using the constructor mentioned above.
In more detail, what lies in the stream stack, and where this memory goes, can be read here .
Of course, this problem may seem a little far-fetched to you, since most servers now spin on a 64-bit architecture, but still I think this example is very useful, since it helps to figure out what parts the java process memory consists of.
The memory area occupied by the java process consists of several parts. The type of OutOfMemoryError depends on which of them ran out of space.
1. java.lang.OutOfMemoryError: Java heap space
There is not enough space on the heap, namely, in the memory area into which objects created programmatically in your application are placed. The size is specified by the -Xms and -Xmx options. If you try to create an object, and there is no space left on the heap, you get this error. Usually the problem lies in the memory leak, of which there are a great many, and the Internet is simply replete with articles on this topic.
2.java.lang.OutOfMemoryError: PermGen space
This error occurs when there is not enough space in the Permanent area, the size of which is specified by the -XX: PermSize and -XX: MaxPermSize parameters. What lies there and how to deal with the OutOfMemoryError that arises there, I have already described in the most detailed way here .
3.java.lang.OutOfMemoryError: GC overhead limit exceeded
This error can occur during overflow of the first and second areas. It is connected with the fact that there is little memory left and the GC is constantly working, trying to free up some space. This error can be disabled using the -XX: -UseGCOverheadLimit parameter, but, of course, you must not disable it, but either solve the memory leak problem, or allocate more volume, or change the GC settings.
4.java.lang.OutOfMemoryError: unable to create new native thread
I first encountered this error several years ago when I was doing load testing and trying to figure out the maximum number of users who can work with our web application. I used a special tool that allowed users to log in and emulate their standard actions. On a certain number of clients, I started getting OutOfMemoryError. Not really reading the message body and thinking that I do not have enough memory to create a user session and other necessary objects, I increased the size of the application heap (-Xmx). Imagine my surprise when after that the number of users simultaneously working with the system only decreased. Let's take a closer look at how this happened.
In fact, it is very simple to reproduce on windows on a 32-bit machine, since there the process allocates no more than 2 GB.
Suppose you have an application with a large number of concurrent users that starts with the parameters -Xmx1024M -XX: MaxPermSize = 256M -Xss512K. If 2G is available to the whole process, then another 768M will remain free. It is in this remaining memory that stacks of threads are created. Thus, approximately you can create no more than 768 * (1024/512) = 1536 (I managed to create 1316 with these parameters) threads (see the picture at the beginning of the article), after which you will get OutOfMemoryError. If you increase -Xmx, then the number of threads you can create will decrease accordingly. The option with decreasing -Xss, to be able to create more threads, is not always an option, since you probably have threads in the system that require quite large stacks. For example, an initialization thread or some background tasks. But still there is a solution. It turns out that when creating a thread programmatically, you can specify the stack size:Thread (ThreadGroup group, Runnable target, String name, long stackSize) . Thus, you can set -Xss to be quite small, and perform actions requiring large stacks in separate threads created using the constructor mentioned above.
In more detail, what lies in the stream stack, and where this memory goes, can be read here .
Of course, this problem may seem a little far-fetched to you, since most servers now spin on a 64-bit architecture, but still I think this example is very useful, since it helps to figure out what parts the java process memory consists of.