If you keep on allocating & keeping references to object, you will fill up any amount of memory you have.
One option is to do a transparent file close & open when they switch tabs (you only keep a pointer to the file, and when the user switches tab, you close & clean all the objects… it’ll make the file change slower… but…), and maybe keep only 3 or 4 files on memory.
Other thing you should do is, when the user opens a file, load it, and intercept any OutOfMemoryError, then (as it is not possible to open the file) close that file, clean its objects and warn the user that he should close unused files.
Your idea of dynamically extending virtual memory doesn’t solve the issue, for the machine is limited on resources, so you should be carefull & handle memory issues (or at least, be carefull with them).
A couple of hints i’ve seen with memory leaks is:
—> Keep on mind that if you put something into a collection and afterwards forget about it, you still have a strong reference to it, so nullify the collection, clean it or do something with it… if not you will find a memory leak difficult to find.
—> Maybe, using collections with weak references (weakhashmap…) can help with memory issues, but you must be carefull with it, for you might find that the object you look for has been collected.
—> Another idea i’ve found is to develope a persistent collection that stored on database objects least used and transparently loaded. This would probably be the best approach…
All the applications that you’re trying to execute require memory. It doesn’t matter if the application was developed using assembly language. Or if you used a low-level programming language like C or a language compiled to a bytecode like Java. Running the application requires memory for the code itself, the variables, and the data that the code processes. Depending on your usage, the memory requirements will vary. Some programs will require very little memory – for example, a simple app designed to process small text files; others will require gigabytes of memory because of the amount of data they process in memory.
In Java, at least initially, you can forget about the memory. You create objects, use them and leave them alone. Eventually, the Java garbage collector (GC) will free the memory occupied by unused objects and release the used memory. However, there is always a limited amount of data you can keep in memory at the same time – the size of the heap.
The heap is the place where the Java Virtual Machine keeps the data needed by the application. The heap is not unlimited – you control it during application start and you can’t keep more objects in memory than it allows. If the heap is full and you create that one more object you may receive an OutOfMemory error.
In this blog post, I’ll tell you what Java OutOfMemory errors are, what causes them and how to deal with them.
A java.lang.OutOfMemoryError means that something is wrong in the application – to be precise there was an issue with a part of application memory. To be more specific than that, we need to dive into the reasons that the Java Virtual Machine can go out of memory, and each has a different cause.
Java Heap Space
Java Heap space is one of the most common errors when it comes to memory handling in the Java Virtual Machine world. This error means that you tried to keep too much data on the heap of the JVM process and there is not enough memory to create new objects, and that the garbage collector can’t collect enough garbage. This can happen for various reasons – for example, you may try to load files that are too large into the application memory or you keep the references to the objects even though you don’t need the data.
Basically, the java.lang.OutOfMemoryError Java heap space tells that the heap of your application is not large enough or you are doing something wrong, or you have an old, good Java memory leak.
Here’s an example that illustrates the Java heap space problem:
public class JavaHeapSpace {
public static void main(String[] args) throws Exception {
String[] array = new String[100000 * 100000];
}
}
The code tries to create an array of String objects that is quite large. And that’s it. With the default settings for the memory size, you should see the following result when executing the above code:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at memory.JavaHeapSpace.main(JavaHeapSpace.java:5)
And the result is simple – there is just not enough memory on the heap to assign the array, and thus the JVM throws an error informing us about that.
How to fix it: In some cases, to mitigate the problem, it is enough to increase the maximum heap size by adding the -Xmx to your JVM application startup settings and setting it to a larger value. For example, to increase the maximum heap size to 8GB, you would add the -Xmx8g parameter to your JVM application start parameters. However, if you have a memory leak you will eventually see the error appearing again. This means that you need to go through the application code and look for places where potential memory issues can happen. Tools like Java profilers and the good, old heap dump will help you with that.
GC Overhead Limit
The GC Overhead Limit is exactly what its name suggests – a problem with the Java Virtual Machine garbage collector not being able to reclaim memory. You will see the java.lang.OutOfMemoryError: GC overhead limit exceeded if the Java Virtual Machine spends more than 98% of its time in the garbage collection, for 5 consecutive garbage collections and can reclaim less than 2% of the heap.
When using older Java versions that were using the older implementations of the garbage collection (like Java 8), you can easily simulate the GC Overhead exception by running a code similar to the following:
public class GCOverhead {
public static void main(String[] args) throws Exception {
Map<Long, Long> map = new HashMap<>();
for (long i = 0l; i < Long.MAX_VALUE; i++) {
map.put(i, i);
}
}
}
When run with a small heap, like 25MB you would get an exception like this:
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.base/java.lang.Long.valueOf(Long.java:1211)
at memory.GCOverhead.main(GCOverhead.java:10)
That means that the heap is almost full and the garbage collector spent at least 5 consecutive garbage collections removing less than 2% of the assigned objects.
How to fix it: The possible solution to such an error is increasing the heap by adding the -Xmx to your JVM application startup settings and setting it to a larger value than you are currently using.
Array Size Limits
One of the errors that you may encounter is the java.lang.OutOfMemoryError: Requested array size exceeds VM limit, which points out that the size of the array that you’re trying to keep in memory is larger than the Integer.MAX_INT or that you’re trying to have an array larger than your heap size.
How to fix it: If your array is larger than your heap size, you can try increasing the heap size. If you are trying to put more than the 2^31-1 entries into a single array, you will need to modify your code to avoid such situations.
Number of Thread Issues
The operating system has limits when it comes to the number of threads you can run inside a single application. When you see a java.lang.OutOfMemoryError: unable to create native thread error being thrown by the Java Virtual Machine running your code, you can be sure that you hit the limit or your operating system runs out of resources to create a new thread. Basically, a new thread was not created on the operating system level and the error happened in the Java Native Interface or in the native method itself.
To illustrate the issue with the creation of threads let’s create a code that continuously creates threads and puts them to sleep. For example like this:
public class ThreadsLimits {
public static void main(String[] args) throws Exception {
while (true) {
new Thread(
new Runnable() {
@Override
public void run() {
try {
Thread.sleep(1000 * 60 * 60 * 24);
} catch (Exception ex) {}
}
}
).start();
}
}
}
Right after you run the above code, you can expect an exception thrown:
[0.420s][warning][os,thread] Failed to start thread - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached. Exception in thread "main" java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached at java.base/java.lang.Thread.start0(Native Method) at java.base/java.lang.Thread.start(Thread.java:802) at memory.ThreadsLimits.main(ThreadsLimits.java:15)
We can clearly see that our Java code exhausted the operating system limits and couldn’t create more threads.
To diagnose the issue, we suggest referring to the appropriate section of the Java documentation. For example, Java 17 documentation includes a section called Troubleshooting Tools Based on the Operating System, which mentions tools that can help you find the problem.
PermGen Issues
The PermGen or Permanent Generation is a special place in the Java heap that the Java Virtual Machine uses to keep track of all the loaded classes metadata, static methods, references to static objects, and primitive variables. The PermGen was removed with the release of Java 8, so at this point, you’ll probably never hit the issue with it.
The problem with PermGen was its limited default size – 64MB in 32-bit Java Virtual Machine version and up to 82MB in the 64-bit version of the JVM. This was problematic because if your application contained a lot of classes, static methods, and references to static objects, you could easily get into issues with too small PermGen space.
How to fix it: If you ever encounter the java.lang.OutOfMemoryError: PermGen space error you can start by increasing the size of the PermGen space by including the -XX:PermSize and -XX:MaxPermSize JVM parameters.
Metaspace Issues
With the removal of the PermGen space, the classes metadata now lives in the native space. The space that keeps the classes metadata is now called Metaspace and is part of the Java Virtual Machine heap. However, the region is still limited and can be exhausted if you have a lot of classes.
How to fix it: The problems with the Metaspace region are signaled by the Java Virtual Machine when a java.lang.OutOfMemoryError: Metaspace error is thrown. To mitigate the issue, you can increase the size of the Metaspace by adding the -XX:MaxMetaspaceSize flag to startup parameters of your Java application. For example, to set the Metaspace region size to 128M, you would add the following parameter: -XX:MaxMetaspaceSize=128m.
Out of swap
Your operating system uses the swap space as the secondary memory to handle the memory management scheme’s paging process. When the native memory–both the RAM and the swap–is close to exhaustion, the Java Virtual Machine may not have enough space to create new objects. This may happen for various reasons – your system may be overloaded, other applications may be heavy memory users and are exhausting the resources. In this case the JVM will throw the java.lang.OutOfMemoryError: Out of swap space error, which means that the reason is a problem on the operating system side.
How to fix it: The exact exception stack is usually helpful for mitigatin the error, as it will include the amount of memory that the JVM tried to allocate and the code which did that. When this error occurs, you can expect your Java Virtual Machine to create a file with a detailed description of what happened. You may also want to check your operating system swap settings and increase it if that is too low. At the same time, you need to verify if there are other heavy memory consumers running on the same machine as your application.
How to Catch java.lang.OutOfMemoryError Exceptions
Java has the option to catch exceptions and handle them gracefully. For example, you can catch the FileNotFoundException that can be thrown when trying to work with a file that doesn’t exist. The same can be done with the OutOfMemoryError – you can catch it, but it doesn’t make much sense, at least in most cases. As the developers, we usually can’t do much about the lack of memory in our application. But maybe your specific use-case is such that you would like to do that.
To catch the OutOfMemoryError you just need to surround the code that you expect to cause memory issues with the try-catch block, just like this:
public class JavaHeapSpace {
public static void main(String[] args) throws Exception {
try {
String[] array = new String[100000 * 100000];
} catch (OutOfMemoryError oom) {
System.out.println("OutOfMemory Error appeared");
}
}
}
The execution of the above code, instead of resulting in the OutOfMemoryError will result in printing the following:
OutOfMemory Error appeared
In such a case, you can try recovering from that error, but that is highly use-case dependent. The best solution is to analyze the places where you’re trying to catch the OutOfMemoryError. Definitely avoid catching the mentioned error in the main method where you just start the whole execution. If you don’t know everything about exception handling in Java read our blog post to learn more about how to deal with OutOfMemoryError and other types of Java errors.
Monitor and Analyze Java OutOfMemoryError with Sematext
To ensure a healthy environment for your business process you need to be sure you will not miss any of the errors that can be caused by memory issues when running your Java applications. This means that you need to pay close attention to the logs produced by your Java applications and set up alerting on the relevant events – the OutOfMemoryError ones. You can achieve all of this by using Sematext Logs – an intelligent and easy to use logs centralization solution allowing you to get all the needed information in one place, create alerts and be proactive when dealing with memory issues.
You can read more about Sematext Logs and how it compares with similar solutions in our blog posts about the best log management software, log analysis tools, and cloud logging services available today. Or, if you’d like, check out the short video below to get more familiar with Sematext Logs and how they can help you.
Conclusion
Each memory-related error in Java is different and the approach that we need to take to fix it is different. The first and the most important thing is understanding. To know what needs to be fixed, we need to understand what kind of error happened, when it happened, and finally, why it happened. This information is crucial to take proper reaction and fix the underlying issue that is the root cause of the error.
That is where log management tools, like the Sematext Logs come into play. Having a place where you can see all your exceptions and analyze them is priceless. Sematext Logs is a part of Sematext Cloud, an all-in-one observability solution with Java monitoring integration and JVM Garbage Collector logging capabilities. All of that combined gives you a single platform that allows you to correlate all the necessary metrics. This provides you with a full view of the problem and helps you get to its root cause fast and efficiently. There’s a 14-day free trial available for you to try its features, so give it a try!
Overview
An out of memory error in Java formally known as java.lang.OutOfMemoryError is a runtime error that occurs when the Java Virtual Machine (JVM) cannot allocate an object in the Java heap memory. In this article, we will be discussing several reasons behind “out of memory” errors in Java and how you can avoid them.
The JVM manages the memory by setting aside a specific size of the heap memory to store the newly allocated objects. All the referenced objects remain active in the heap and keep that memory occupied until their reference is closed. When an object is no longer referenced, it becomes eligible to be removed from the heap by the Garbage collector to free up the occupied heap memory. In certain cases, the Java Garbage Collector (GC) is unable to free up the space required for a new object and the available heap memory is insufficient to support the loading of a Java class, this is when an “out of memory” error occurs in Java.
What causes the out of memory error in Java?
An “out of memory” error in Java is not that common and is a direct indication that something is wrong in the application. For instance, the application code could be referencing large objects for too long that is not required or trying to process large amounts of data at a time. It is even possible that the error could have nothing to do with objects on the heap and the reason behind it like because of third-party libraries used within an application or due to an application server that does not clean up after deployment.
Following are some of the main causes behind the unavailability of heap memory that cause the out of memory error in Java.
· Java heap space error
It is the most common out of memory error in Java where the heap memory fills up while unable to remove any objects.
See the code snippet below where java.lang.OutOfMemoryError is thrown due to insufficient Java heap memory available:
public class OutOfMemoryError01 {
public static void main(String[] args) {
Integer[] arr = new Integer[1000 * 1000 * 1000];
}
}
Output:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at OutOfMemoryErrorExample.main(OutOfMemoryErrorExample.java:8)
In the above code, an array of integers with a very large size is attempted to be initialized. As the Java heap is insufficient to allocate such a huge array, it will eventually throw a java.lang.OutOfMemoryError: Java heap space error. Initially, it might seem fine but over time, it will result in consuming a lot of Java heap space and when it fills all of the available memory in the heap, Garbage Collection will not be able to clean it as the code would still be in execution and the no memory can be freed.
Another reason for a Java heap space error is the excessive use of finalizers. If a class has a finalize() method, the GC will not clean up any objects of that class, instead, they all will be queued up for finalization at a later stage. If a finalizer thread cannot keep up with the finalization queue because of excessive usage of finalizers, the Java heap will eventually fill up resulting in an “out of memory” error in Java.
Prevention:
Developers need to use the finalize methods only when required and they must monitor all the objects for which finalization would be pending.
· GC Overhead limit exceeded:
This error indicates that the garbage collector is constantly running due to which the program will also be running very slowly. In a scenario where for minimum consecutive 5 garbage collection cycles, if a Java process utilizes almost 98% of its time for garbage collection and could recover less than 2% of the heap memory then a Java Out of Memory Error will be thrown.
This error typically occurs because the newly generated data could barely fit into the Java heap memory having very little free space for new object allocations.
Prevention:
Java developers have the option to set the heap size by themselves. To prevent this error, you must Increase the heap size using the -Xmx attribute when launching the JVM.
· PermGen space error:
JVM separates the memory into different sections. One of the sections is Permanent Generation (PermGen) space. It is used to load the definitions of new classes that are generated at the runtime. The size of all these sections, including the PermGen area, is set at the time of the JVM launch. If you do not set the sizes of every area yourself, platform-specific defaults sizes will be then set. If the Permanent Generation’s area is ever exhausted, it will throw the java.lang.OutOfMemoryError: PermGen space error.
Prevention:
The solution to this out of Memory Error in Java is fairly simple. The application just needs more memory to load all the classes to the PermGen area so just like the solution for GC overhead limit exceeding error, you have to increase the size of the PermGen region at the time of Java launch. To do so, you have to change the application launch configuration and increase or if not used, add the –XX:MaxPermSize parameter to your code.
· Out of MetaSpace error:
All the Java class metadata is allocated in native memory (MetaSpace). The amount of MetaSpace memory to be used for class metadata is set by the parameter MaxMetaSpaceSize. When this amount exceeds, a java.lang.OutOfMemoryError exception with a detail MetaSpace is thrown.
Prevention:
If you have set the MaxMetaSpaceSize on the command line, increasing its size manually can solve the problem. Alternatively, MetaSpace is allocated from the same address spaces as the Java heap memory so by reducing the size of the Java heap, you can automatically make space available for MetaSpace. It should only be done when you have excess free space in the Java heap memory or else you can end up with some other Java out of memory error.
· Out of swap space error:
This error is often occurred due to certain operating system issues, like when the operating system has insufficient swap space or a different process running on the system is consuming a lot of memory resources.
Prevention:
There is no way to prevent this error as it has nothing to do with heap memory or objects allocation. When this error is thrown, the JVM invokes the error handling mechanism for fatal errors. it generates an error log file, which contains all the useful information related to the running threads, processes, and the system at the time of the crash. this log information can be very useful to minimize any loss of data.
How to Catch java.lang.OutOfMemoryError?
As the java.lang.OutOfMemoryError is part of the Throwable class, it can be caught and handled in the application code which is highly recommended. The handling process should include the clean up the resources, logging the last data to later identify the reason behind the failure, and lastly, exit the program properly.
See this code example below:
public class OutOfMemoryError02 {
public void createArr (int size) {
try {
Integer[] myArr = new Integer[size];
} catch (OutOfMemoryError ex) {
//creating the Log
System.err.println("Array size is too large");
System.err.println("Maximum JVM memory: " +
Runtime.getRuntime().maxMemory());
}
}
public static void main(String[] args) {
OutOfMemoryError02 oomee = new OutOfMemoryError02();
ex.createArr (1000 * 1000 * 1000);
}
}
In the above code, as the line of code that might cause an out of Memory Error is known, it is handled using a try-catch block. In case, if the error occurs, the reason for the error will be logged that is the large size of the array and the maximum size of the JVM, which will be later helpful for the caller of the method to take the action accordingly.
In case of an out of memory error, this code will exit with the following message:
Array size is too large Maximum JVM memory: 9835679212
It is also a good option to handle an out of Memory Error in Java when the application needs to stay in a constant state in case of the error. This allows the application to keep running normally if any new objects are not required to be allocated.
See Also: CompletableFuture In Java With Examples
Conclusion
In this article, we have extensively covered everything related to the “out of memory” error in Java. In most cases, you can now easily prevent the error or at least will be able to retrieve the required information after the crashing of the program to identify the reason behind it. Managing errors and exceptions in your code is always challenging but being able to understand and avoid these errors can help you in making your applications stable and robust.
In Java, all objects are stored in a heap. They are allocated using a new operator. The OutOfMemoryError Exception in Java looks like this:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
Usually, this error is thrown when the Java Virtual Machine cannot allocate an object because it is out of memory. No more memory could be made available by the garbage collector.
OutOfMemoryError usually means that you’re doing something wrong, either holding onto objects too long or trying to process too much data at a time. Sometimes, it indicates a problem that’s out of your control, such as a third-party library that caches strings or an application server that doesn’t clean up after deploys. And sometimes, it has nothing to do with objects on the heap.
The java.lang.OutOfMemoryError exception can also be thrown by native library code when a native allocation cannot be satisfied (for example, if swap space is low). Let us understand various cases when the OutOfMemory error might occur.
Symptom or Root cause?
To find the cause, the text of the exception includes a detailed message at the end. Let us examine all the errors.
Error 1 – Java heap space:
This error arises due to the applications that make excessive use of finalizers. If a class has a finalize method, objects of that type do not have their space reclaimed at garbage collection time. Instead, after garbage collection, the objects are queued for finalization, which occurs later.
Implementation:
- finalizers are executed by a daemon thread that services the finalization queue.
- If the finalizer thread cannot keep up with the finalization queue, the Java heap could fill up, and this type of OutOfMemoryError exception would be thrown.
- The problem can also be as simple as a configuration issue, where the specified heap size (or the default size, if it is not specified) is insufficient for the application.
Java
import java.util.*;
public class Heap {
static List<String> list = new ArrayList<String>();
public static void main(String args[]) throws Exception
{
Integer[] array = new Integer[10000 * 10000];
}
}
Output:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at Heap.main(Heap.java:11)
When you execute the above code above you might expect it to run forever without any problems. As a result, over time, with the leaking code constantly used, the “cached” results end up consuming a lot of Java heap space, and when the leaked memory fills all of the available memory in the heap region and Garbage Collection is not able to clean it, the java.lang.OutOfMemoryError:Java heap space is thrown.
Prevention: Check how to monitor objects for which finalization is pending in Monitor the Objects Pending Finalization.
Error 2 – GC Overhead limit exceeded:
This error indicates that the garbage collector is running all the time and Java program is making very slow progress. After a garbage collection, if the Java process is spending more than approximately 98% of its time doing garbage collection and if it is recovering less than 2% of the heap and has been doing so far the last 5 (compile-time constant) consecutive garbage collections, then a java.lang.OutOfMemoryError is thrown.
This exception is typically thrown because the amount of live data barely fits into the Java heap having little free space for new allocations.
Java
import java.util.*;
public class Wrapper {
public static void main(String args[]) throws Exception
{
Map m = new HashMap();
m = System.getProperties();
Random r = new Random();
while (true) {
m.put(r.nextInt(), "randomValue");
}
}
}
If you run this program with java -Xmx100m -XX:+UseParallelGC Wrapper, then the output will be something like this :
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.lang.Integer.valueOf(Integer.java:832)
at Wrapper.main(error.java:9)
Prevention: Increase the heap size and turn off it with the command line flag -XX:-UseGCOverheadLimit.
Error 3 – Permgen space is thrown:
Java memory is separated into different regions. The size of all those regions, including the permgen area, is set during the JVM launch. If you do not set the sizes yourself, platform-specific defaults will be used.
The java.lang.OutOfMemoryError: PermGen space error indicates that the Permanent Generation’s area in memory is exhausted.
Java
import javassist.ClassPool;
public class Permgen {
static ClassPool classPool = ClassPool.getDefault();
public static void main(String args[]) throws Exception
{
for (int i = 0; i < 1000000000; i++) {
Class c = classPool.makeClass("com.saket.demo.Permgen" + i).toClass();
System.out.println(c.getName());
}
}
}
In the above sample code, code iterates over a loop and generates classes at run time. Class generation complexity is being taken care of by the Javassist library.
Running the above code will keep generating new classes and loading their definitions into Permgen space until the space is fully utilized and the java.lang.OutOfMemoryError: Permgen space is thrown.
Prevention : When the OutOfMemoryError due to PermGen exhaustion is caused during the application launch, the solution is simple. The application just needs more room to load all the classes to the PermGen area, so we need to increase its size. To do so, alter your application launch configuration and add (or increase if present) the -XX:MaxPermSize parameter similar to the following example:
java -XX:MaxPermSize=512m com.saket.demo.Permgen
Error 4 – Metaspace:
Java class metadata is allocated in native memory. Suppose metaspace for class metadata is exhausted, a java.lang.OutOfMemoryError exception with a detail MetaSpace is thrown.
The amount of metaspace used for class metadata is limited by the parameter MaxMetaSpaceSize, which is specified on the command line. When the amount of native memory needed for a class metadata exceeds MaxMetaSpaceSize, a java.lang.OutOfMemoryError exception with a detail MetaSpace is thrown.
Java
import java.util.*;
public class Metaspace {
static javassist.ClassPool cp
= javassist.ClassPool.getDefault();
public static void main(String args[]) throws Exception
{
for (int i = 0; i < 100000; i++) {
Class c = cp.makeClass(
"com.saket.demo.Metaspace" + i)
.toClass();
}
}
}
This code will keep generating new classes and loading their definitions to Metaspace until the space is fully utilized and the java.lang.OutOfMemoryError: Metaspace is thrown. When launched with -XX:MaxMetaspaceSize=64m then on Mac OS X my Java 1.8.0_05 dies at around 70, 000 classes loaded.
Prevention: If MaxMetaSpaceSize, has been set on the command line, increase its value. MetaSpace is allocated from the same address spaces as the Java heap. Reducing the size of the Java heap will make more space available for MetaSpace. This is only a correct trade-off if there is an excess of free space in the Java heap.
Error 5 – Requested array size exceeds VM limit:
This error indicates that the application attempted to allocate an array that is larger than the heap size. For example, if an application attempts to allocate an array of 1024 MB but the maximum heap size is 512 MB then OutOfMemoryError will be thrown with “Requested array size exceeds VM limit”.
Java
import java.util.*;
public class GFG {
static List<String> list = new ArrayList<String>();
public static void main(String args[]) throws Exception
{
Integer[] array = new Integer[10000 * 10000];
}
}
Output:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at GFG.main(GFG.java:12)
The java.lang.OutOfMemoryError: Requested array size exceeds VM limit can appear as a result of either of the following situations:
- Your arrays grow too big and end up having a size between the platform limit and the Integer.MAX_INT
- You deliberately try to allocate arrays larger than 2^31-1 elements to experiment with the limits.
Error 6 – Request size bytes for a reason. Out of swap space?:
This apparent exception occurred when an allocation from the native heap failed and the native heap might be close to exhaustion. The error indicates the size (in bytes) of the request that failed and the reason for the memory request. Usually, the reason is the name of the source module reporting the allocation failure, although sometimes it is the actual reason.
The java.lang.OutOfMemoryError: Out of swap space error is often caused by operating-system-level issues, such as:
- The operating system is configured with insufficient swap space.
- Another process on the system is consuming all memory resources.
Prevention: When this error message is thrown, the VM invokes the fatal error handling mechanism (that is, it generates a deadly error log file, which contains helpful information about the thread, process, and system at the time of the crash). In the case of native heap exhaustion, the heap memory and memory map information in the log can be useful.
Error 7 – reason stack_trace_with_native_method:
Whenever this error message(reason stack_trace_with_native_method) is thrown then a stack trace is printed in which the top frame is a native method, then this is an indication that a native method has encountered an allocation failure. The difference between this and the previous message is that the allocation failure was detected in a Java Native Interface (JNI) or native method rather than the JVM code.
Java
import java.util.*;
public class GFG {
public static void main(String args[]) throws Exception
{
while (true) {
new Thread(new Runnable() {
public void run()
{
try {
Thread.sleep(1000000000);
}
catch (InterruptedException e) {
}
}
}).start();
}
}
}
The exact native thread limit is platform-dependent. For example, tests Mac OS X reveals that: 64-bit Mac OS X 10.9, Java 1.7.0_45 – JVM dies after #2031 threads have been created
Prevention: Use native utilities of the OS to diagnose the issue further. For more information about tools available for various operating systems, see Native Operating System tools.
This article is contributed by Saket Kumar. If you like GeeksforGeeks and would like to contribute, you can also write an article using write.geeksforgeeks.org or mail your article to review-team@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks. Please write comments if you find anything incorrect, or you want to share more information about the topic discussed above.
Founder & Expert Button Pusher of Teaspoon Consulting
Running out of memory is one of the most common issues affecting
Java (and other JVM language) applications in production. This
article explains how to recognise low memory issues, and uses a
small program to demonstrate some of the tools you can use to find
what’s using all your memory.
Overview
Memory issues are an unfortunate part of the Java landscape. If
you’re running programs on the Java Virtual Machine (JVM) and haven’t
seen an error like the one shown above, count yourself lucky. For
everyone else, these sort of issues are all-too-common—often dealt
with by increasing heap sizes, or trying random permutations of JVM
switches until they go away.
We tend to see these sort of issues with the JVM because it runs in a
fixed-size heap, set when the JVM first starts. As the person running
an application, it’s up to you to estimate the peak amount of memory
your application will require, then to size the JVM heap accordingly.
This is tough to get right and often requires some level of
experimentation.
Complicating this, low memory issues are not always obvious. If
you’re lucky, your application crashes outright when it runs out of
memory. But just as often it continues limping along, with the
garbage collector constantly trying to free some space. This often
causes memory issues to be misdiagnosed as application performance
issues: top reports high CPU usage from the application’s JVM, but
it’s actually the garbage collector thread doing all the work.
This article focuses on two topics: how to know when you’re running
out of memory, and how to track down the spot in your application
that’s using it all.
Identifying low memory
If you’re lucky, you know your application is running out of memory
because it has told you. If you see this in your log files:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
Then you’re running out of memory. A popular option at this point is
to try increasing your JVM heap size with a JVM switch like:
-Xmx1g
and then hope your problem goes away. This might actually be the
right thing to do! If your application’s working set peaks at around
1GB for normal usage, then there’s no problem here: set your JVM
options and take the rest of the day off.
For those still here, we need to do some more digging. An
excellent starting point is to run your application with the
-verbose:gc JVM switch. This tells the garbage collector to log
diagnostic information after each garbage collection run. Once you
enable this, you’ll see lines like the following in your console (or
log file):
[GC 52082K->44062K(116544K), 0.0040520 secs]
[GC 58654K->45190K(116544K), 0.0042610 secs]
[GC 59782K->47526K(116544K), 0.0015520 secs]
[GC 71590K->52294K(116544K), 0.0010260 secs]
[GC 66886K->93618K(116544K), 0.0074450 secs]
[Full GC 93618K->42708K(116544K), 0.0126130 secs]
[GC 56599K->49312K(116544K), 0.0040300 secs]
[GC 63904K->50976K(116544K), 0.0069590 secs]
[GC 65568K->53248K(116544K), 0.0014260 secs]
[GC 77056K->57888K(116544K), 0.0019720 secs]
[Full GC 99310K->47201K(116544K), 0.0139520 secs]
[GC 61793K->54476K(116544K), 0.0044440 secs]
[GC 69068K->56180K(116544K), 0.0054920 secs]
[GC 70772K->58452K(116544K), 0.0019430 secs]
[GC 82260K->63060K(116544K), 0.0008430 secs]
[Full GC 99119K->50845K(116544K), 0.0137490 secs]
[GC 65437K->58441K(116544K), 0.0050000 secs]
[GC 73033K->60161K(116544K), 0.0054100 secs]
Each line records the details of a garbage collection run. The lines
marked with GC represent garbage collections of the JVM’s young
generation, while lines with Full GC represent garbage collections
of the JVM’s old generation.
We won’t describe the complete workings of the JVM’s memory management
here, so here’s the short version: the young generation and old
generation are different areas of memory where objects are stored, and
the JVM garbage collects each area using a different method. The
young generation garbage collector is optimised for lots of objects
that live fast and die young; the old generation garbage collector is
optimised for long-lived objects that are more likely to stick around
than not. Objects are allocated from the young generation when first
created and, if they live long enough, eventually wind up in the old
generation where they live out their days listening to the radio and
complaining about the young generation. Good.
So we tend to see more GC lines than Full GC lines, but they’re
both telling us the same stuff. Picking one line at random:
[GC 52082K->44062K(116544K), 0.0040520 secs]
This tells us that during this garbage collection run against the young generation:
-
The heap started out with 52,082KB in use.
-
Once the garbage collection run had finished, it had reduced the
heap down to 44,062KB (freeing about 8MB). -
The total committed size of the JVM heap is 116,544KB.
-
It took about 4 milliseconds to complete the garbage
collection run.
What low memory problems look like
When an application is in trouble, you’ll start to see a relatively
high number of Full GC messages. That’s a sign that objects are
sticking around and filling up the old generation area:
[GC 85682K->73458K(104448K), 0.0010870 secs]
[GC 88050K->75730K(116160K), 0.0017100 secs]
[Full GC 100370K->80306K(116160K), 0.0110770 secs]
[Full GC 95730K->81808K(116160K), 0.0256440 secs]
[Full GC 97232K->76495K(116160K), 0.0116220 secs]
[Full GC 90013K->77613K(116160K), 0.0114290 secs]
[Full GC 93037K->82221K(116160K), 0.0060700 secs]
[Full GC 91372K->82222K(116160K), 0.0081970 secs]
[Full GC 97646K->84525K(116160K), 0.0097500 secs]
[Full GC 98993K->87661K(116160K), 0.0202760 secs]
People often misinterpret garbage collection logs by focusing on the
number on the left of the arrow (the heap utilisation pre-collection).
Keep in mind that the normal mode of operation for the JVM is to
accumulate garbage until some threshold is reached, then to run the
garbage collector to clear it.
If we take some garbage collection logs and plot, over time, the
numbers on either side of the arrows (showing heap utilisation before
and after garbage collection), a healthy application looks like this:
Here we see a sawtooth pattern, where the points at the top represent
heap utilisation prior to garbage collection, and the points at the
bottom represent heap utilisation immediately after. It’s the points
at the bottom that matter here: we see that the garbage collector is
successfully freeing memory and reverting heap utilisation back to a
fairly consistent baseline.
If we plot the garbage collection log lines shown above, things aren’t
so rosy:
This application seems to be consuming more memory over time. There’s
a lot of variability in how much memory the garbage collector frees
during each run, and a long-term trend of freeing less memory with
each collection.
To summarise, you might have low memory problems if:
-
You’re seeing frequent
Full GCmessages in your logs; and -
The post-garbage-collection utilisation number is growing over time.
If that’s the case, it’s time to start tracking down your memory leak.
Finding memory leaks
In this section we’ll look at some of the tools you can use for
finding memory leaks in your code. In the spirit of contrived
examples, here’s a contrived example! This example has two parts:
-
GibberishGeneratorgenerates very long lines of gibberish words
of varying lengths. -
MemoryLeakexamines these lines of gibberish to find the first
«good» word matching some criteria and keeps the list of good
words it finds in a list.
Here’s the code:
import java.util.ArrayList;
import java.util.List;
import java.util.Iterator;
import java.util.Random;
public class MemoryLeak
{
// Generates long lines of gibberish words.
static class GibberishGenerator implements Iterator<String>
{
private int MAX_WORD_LENGTH = 20;
private int WORDS_PER_LINE = 250000;
private String alphabet = ("abcdefghijklmnopqrstuvwxyz" +
"ABCDEFGHIJKLMNOPQRSTUVWXYZ");
public boolean hasNext() {
return true;
}
public String next() {
StringBuffer result = new StringBuffer();
for (int i = 0; i < WORDS_PER_LINE; i++) {
if (i > 0) { result.append(" "); }
result.append(generateWord(MAX_WORD_LENGTH));
}
return result.toString();
}
public void remove() {
// not implemented
}
private String generateWord(int maxLength) {
int length = (int)(Math.random() * (maxLength - 1)) + 1;
StringBuffer result = new StringBuffer(length);
Random r = new Random();
for (int i = 0; i < length; i++) {
result.append(alphabet.charAt(r.nextInt(alphabet.length())));
}
return result.toString();
}
}
// A "good" word has as many vowels as consonants and is more than two
// letters long.
private static String vowels = "aeiouAEIOU";
private static boolean isGoodWord(String word) {
int vowelCount = 0;
int consonantCount = 0;
for (int i = 0; i < word.length(); i++) {
if (vowels.indexOf(word.charAt(i)) >= 0) {
vowelCount++;
} else {
consonantCount++;
}
}
return (vowelCount > 2 && vowelCount == consonantCount);
}
public static void main(String[] args) {
GibberishGenerator g = new GibberishGenerator();
List<String> goodWords = new ArrayList<String>();
for (int i = 0; i < 1000; i++) {
String line = g.next();
for (String word : line.split(" ")) {
if (isGoodWord(word)) {
goodWords.add(word);
System.out.println("Found a good word: " + word);
break;
}
}
}
}
}
This code has a memory leak (gasp!). Let’s give it a run with a
modest JVM heap and see what happens:
$ java -verbose:gc -Xmx64m MemoryLeak
...
Found a good word: EDeGOG
[Full GC 44032K->32382K(51136K), 0.0088630 secs]
[GC 39742K->33566K(58368K), 0.0027320 secs]
[GC 44382K->36990K(58304K), 0.0041990 secs]
[Full GC 45982K->44686K(58304K), 0.0224270 secs]
Found a good word: eZjovI
[Full GC 51071K->38076K(58304K), 0.0059460 secs]
[GC 45436K->39228K(58304K), 0.0007690 secs]
[GC 46588K->40412K(53440K), 0.0009190 secs]
[Full GC 52380K->42684K(53440K), 0.0085100 secs]
[Full GC 50044K->42684K(53440K), 0.0086750 secs]
[Full GC 50044K->42313K(53440K), 0.0092640 secs]
[Full GC 47351K->42313K(53440K), 0.0077370 secs]
[GC 42313K->42313K(57984K), 0.0008850 secs]
[Full GC 42313K->42297K(57984K), 0.0151990 secs]
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2882)
at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390)
at java.lang.StringBuffer.append(StringBuffer.java:224)
at MemoryLeak$GibberishGenerator.next(MemoryLeak.java:24)
at MemoryLeak.main(MemoryLeak.java:74)
It finds a few words and then runs out of memory. Let’s look at some
tools for finding out more.
Using jmap
The jmap tool comes as standard with the Java SDK, so there’s a good
chance you have it already. Using jmap, you can connect to a
running JVM and walk its memory pools, printing a summary of the
number of instances of each class and how many bytes of heap each is
using.
To start, get the process ID of the JVM you’re debugging:
$ ps -ef | grep MemoryLeak
mst 14979 14511 99 13:53 pts/29 00:00:03 java -verbose:gc -Xmx64m MemoryLeak
Then read the warning below and instruct jmap to attach to that
JVM and walk its heap:
$ jmap -F -histo 14979
Attaching to process ID 14916, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 20.12-b01
Iterating over heap. This may take a while...
Object Histogram:
num #instances #bytes Class description
—————————————————————————————————————
1: 952 37806240 char[]
2: 4997 684344 * MethodKlass
3: 4997 617648 * ConstMethodKlass
4: 344 381264 * ConstantPoolKlass
5: 7790 346488 * SymbolKlass
Warning: The target JVM will be suspended while
jmapdoes its
thing, so don’t do this on an application that people are actually
using!
Looking at the top few lines of jmap‘s output, there is already a
clue: at the moment we took our sample, there were 952 character
arrays occupying some 37MB of our 64MB JVM heap. Seeing big character
arrays at the top of the list is often a good sign that there are some
big strings being stored somewhere. Interesting…
A handy trick for running
jmap(or any command!) at the exact
moment your JVM throws an OutOfMemoryError is to use the
-XX:OnOutOfMemoryErrorJVM switch. You can invoke it like this:java -Xmx64m -XX:OnOutOfMemoryError='jmap -histo -F %p' MemoryLeakThe JVM’s process ID will automatically be substituted for the
%p
placeholder.
Working with heap dumps
Sometimes seeing the jmap histogram is enough to make you realise
where the memory leak is—if the topmost entry is one of your own
classes (like MyBrilliantCacheEntry) then just curse yourself and
move on. But, unfortunately, the largest user of space is very often
either char[] or byte[], so all you have is the not-that-helpful
knowledge that your memory is being used by… data?
At this point, a heap dump can be helpful. A heap dump gives you a
static snapshot of your application’s memory and, using tools like
jvisualvm or the Eclipse Memory Analyzer, you can inspect your heap
dump and look at what’s in there.
Getting a heap dump using jmap
The jmap tool can connect to a running JVM and produce a heap dump.
Here’s how that works:
$ jmap -F -dump:format=b,file=heap.bin 14979
Attaching to process ID 16610, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 20.12-b01
Dumping heap to heap.bin ...
Heap dump file created
This produces a file called heap.bin that can be read by several
analysis tools. We’ll see how that works in the following sections.
Getting a heap dump on OutOfMemoryError
If you can run your JVM with special switches, you can ask it to
produce a heap dump whenever an OutOfMemoryError is thrown. Just
run your JVM with:
java -Xmx64m
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/tmp/heap.bin
MemoryLeak
As before, this will produce a heap dump file called heap.bin
Producing a heap dump from a core file
Usually one of the previous two ways is sufficient to get a heap dump,
but here’s a bit of trivia. If you connect to your JVM process with
gdb:
$ gdb -p 14979
GNU gdb (GDB) 7.4.1-debian
...
(gdb)
you can use it to dump a standard Unix core file:
(gdb) gcore
Saved corefile core.14979
and then use jmap to create a heap dump from the resulting core
file:
$ jmap -dump:format=b,file=heap.bin `which java` core.14979
Attaching to core core.14979 from executable /usr/local/bin/java, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 20.12-b01
Dumping heap to heap.bin ...
Finding object size using Printezis bits and skipping over...
Heap dump file created
This also works under Solaris for core files produced with the gcore
command.
Exploring heap dumps
Whichever method you chose, you should now have a file called
heap.bin containing a snapshot of your application’s memory from
around the time it was having problems.
There are several tools that you can use to analyse heap dumps. If
you’re feeling old-school, there’s jhat, which comes with the Java
SDK. Run it against your dump file like this:
$ jhat heap.bin
Reading from heap.bin...
Dump file created Tue Oct 29 14:19:49 EST 2013
Snapshot read, resolving...
Resolving 29974 objects...
Chasing references, expect 5 dots.....
Eliminating duplicate references.....
Snapshot resolved.
Started HTTP server on port 7000
Server is ready.
and then browse to http://localhost:7000/ to explore the heap dump.
For a long time, jhat was my go-to for this sort of thing. But
these days, the interface provided by jvisualvm is my weapon of
choice. The Java SDK ships with jvisualvm as standard, so you
should be able to start it up by running jvisualvm (or just
visualvm on some systems). You should see something like:
We’ll load our application’s heap dump:
- Click the
Filemenu - Click
Load - Select «Heap Dumps» for «Files Of Type»
- Navigate to your
heap.binfile
You’ll see your heap dump open in the main jvisualvm pane:
Clicking the Classes tab, we see pretty much what jmap showed us
before: lots of char[] instances using all of our memory.
From this screen, we can drill down and look at the instances
themselves. Double click on the entry for char[] and you’ll be
transferred to the Instances tab.
At the top of the list on the left, we see several instances, each
about 5MB. That’s a lot of character data! Looking at the pane on
the right gives the game away:
qUqCMAzhjybtUitV vgiUVYuTcROHFygG gjoYlXqOhKdQkvOqot gEQlgdcINvhqjxJ ...
Welsh?! No… gibberish.
An epiphany!
What jvisualvm is showing us is seven large strings, each around
5MB, and each containing lots of gibberish words. That’s the clue we
needed: something in our program is holding the input lines of
gibberish in memory, stopping the garbage collector from freeing them.
Now that you know the cause, you might like to return to the original
code and see if you can spot the problem. Spoiler below…
Spoiler
Here’s the culprit:
String line = g.next();
for (String word : line.split(" ")) {
if (isGoodWord(word)) {
goodWords.add(word);
We take each (5MB) line of gibberish, split it into words, check each
word, and keep the first one we find. What’s wrong with that?
It’s actually a curiosity of Java: strings are immutable, so there’s
no harm if two strings share the same character array behind the
scenes. In fact, Java uses this to good effect to make taking
substrings very fast. If we have two strings like:
String s1 = "hello world";
String s2 = "hello world".substring(0, 5);
Then s2 can be created without the need for an additional character
array at all: it just stores a reference to the character array of
s1, along with the appropriate offset and length information to
capture the fact that it’s a «slice» of the original string.
In the case of our code, split creates a substring of the original
5MB input line, but that substring contains a reference to the
character array from the original input line. Even after the original
string is long gone and garbage collected, its character array lives
on in the substring! It’s sort of beautiful, really.
We can fix it by explicitly cloning the substring:
String line = g.next();
for (String word : line.split(" ")) {
if (isGoodWord(word)) {
goodWords.add(new String(word));
and now our program runs without blowing up, and this concludes our
emotional journey.
Update: As of Java 1.7 update 6, string behaviour has changed so
that taking a substring doesn’t hold on to the original byte array.
That’s a bit of a blow to my contrived example, but the overall
approach to finding the issue still holds.
Update 2: Since around 2017 I’ve used
YourKit for this sort of stuff. It’s a
great general-purpose profiler and its tools for memory analysis are
hard to beat. It costs money (like you) and if you do this sort of
thing often it’s well worth it.
Links
Some of the tools mentioned in this article:
-
The
jmaptool -
The
jhattool -
The
jvisualvmtool -
The Eclipse Memory Analyzer
-
YourKit
Need a hand? Find me at mark@teaspoon-consulting.com.
- Upto my knowledge, Heap space is occupied by instance variables only. If this is correct, then why this error occurred after running fine for sometime as space for instance variables are alloted at the time of object creation.
That means you are creating more objects in your application over a period of time continuously. New objects will be stored in heap memory and that’s the reason for growth in heap memory.
Heap not only contains instance variables. It will store all non-primitive data types ( Objects). These objects life time may be short (method block) or long (till the object is referenced in your application)
- Is there any way to increase the heap space?
Yes. Have a look at this oracle article for more details.
There are two parameters for setting the heap size:
-Xms:, which sets the initial and minimum heap size
-Xmx:, which sets the maximum heap size
- What changes should I made to my program so that It will grab less heap space?
It depends on your application.
-
Set the maximum heap memory as per your application requirement
-
Don’t cause memory leaks in your application
-
If you find memory leaks in your application, find the root cause with help of profiling tools like MAT, Visual VM , jconsole etc. Once you find the root cause, fix the leaks.
Important notes from oracle article
Cause: The detail message Java heap space indicates object could not be allocated in the Java heap. This error does not necessarily imply a memory leak.
Possible reasons:
- Improper configuration ( not allocating sufficiant memory)
- Application is unintentionally holding references to objects and this prevents the objects from being garbage collected
- Applications that make excessive use of finalizers. If a class has a finalize method, then objects of that type do not have their space reclaimed at garbage collection time. If the finalizer thread cannot keep up, with the finalization queue, then the Java heap could fill up and this type of OutOfMemoryError exception would be thrown.
On a different note, use better Garbage collection algorithms ( CMS or G1GC)
Have a look at this question for understanding G1GC