Меню

Ошибка system outofmemoryexception что это такое

I know this is an old question but since none of the answers mentioned the large object heap, this might be of use to others who find this question …

Any memory allocation in .NET that is over 85,000 bytes comes from the large object heap (LOH) not the normal small object heap. Why does this matter? Because the large object heap is not compacted. Which means that the large object heap gets fragmented and in my experience this inevitably leads to out of memory errors.

In the original question the list has 50,000 items in it. Internally a list uses an array, and assuming 32 bit that requires 50,000 x 4bytes = 200,000 bytes (or double that if 64 bit). So that memory allocation is coming from the large object heap.

So what can you do about it?

If you are using a .net version prior to 4.5.1 then about all you can do about it is to be aware of the problem and try to avoid it. So, in this instance, instead of having a list of vehicles you could have a list of lists of vehicles, provided no list ever had more than about 18,000 elements in it. That can lead to some ugly code, but it is viable work around.

If you are using .net 4.5.1 or later then the behaviour of the garbage collector has changed subtly. If you add the following line where you are about to make large memory allocations:

System.Runtime.GCSettings.LargeObjectHeapCompactionMode = System.Runtime.GCLargeObjectHeapCompactionMode.CompactOnce;

it will force the garbage collector to compact the large object heap — next time only.

It might not be the best solution but the following has worked for me:

int tries = 0;
while (tries++ < 2)
{
  try 
  {
    . . some large allocation . .
    return;
  }
  catch (System.OutOfMemoryException)
  {
    System.Runtime.GCSettings.LargeObjectHeapCompactionMode = System.Runtime.GCLargeObjectHeapCompactionMode.CompactOnce;
    GC.Collect();
  }
}

of course this only helps if you have the physical (or virtual) memory available.

Источник

You’ve just created a Console app in the latest Visual Studio, and wrote some C# code that allocates some non-negligible quantity of memory, say 6 GB. The machine you’re developing has a decent amount of RAM – 16GB – and it’s running 64-bit Windows 10.

You hit F5, but are struck to find the debugger breaking into your code almost immediately and showing:

Figure 1 – Visual Studio breaking into an exception

What’s going on here ? You’re not running some other memory-consuming app. 6 GB surely should have been in reach for your code. The question that this post will set out to answer thus becomes simply: “Why do I get a System.OutOfMemoryException when running my recently created C# app on an idle, 64-bit Windows machine with lots of RAM ?“.

TL;DR (small scroll bar => therefore I deduct a lot of text => I’ve got no time for that, and need the answer now): The default build settings for Visual Studio limit your app’s virtual address space to 4 GB. Go into your project’s Properties, go to Build, and choose Platform target as x64. Build your solution again and you’re done.

Not so fast ! Tell me more about what goes on under the hood: Buckle up, since we’re going for a ride. First we’ll look at a simple example of code that consumes a lot of memory fast, then uncover interesting facts about our problem, hit a “Wait, what ?” moment, learn the fundamentals of virtual memory, find the root cause of our problem then finish with a series of Q&A.

The Sample Code

Let’s replicate the issue you’ve encountered first – the out-of-memory thing. We’ll pick a simple method of allocating lots of memory – creating several large int arrays. Let’s make each array contain 10 million int values. As for how many of these arrays should be: our target for now is to replicate the initial scenario that started this blog post – that is consuming 6 GB of memory – so we should choose the number of arrays accordingly.

What we need to know is how much an int takes in memory. As it turns out, an int will always take 4 bytes of memory. Thus, an array of 10 million int elements would take 40 million bytes of memory. This will actually be the same on either a 32-bit platform or a 64-bit one. If we divide the 6 GB (6.442.450.944 bytes) to 40 million bytes, we’ll get roughly 162. This should be in theory the number of 40 mil arrays required to fill 6 GB of memory.

Now that the numbers are clear, let’s write the code:

using System;

namespace LeakMemory
{
    class Program
    {
        static void Main(string[] args)
        {
            const int BlockSIZE = 10000000;  // 10 million
            const int NoOfBlocks = 162;
            int[][] intArray = new int[NoOfBlocks][];

            Console.WriteLine("Press a key to start");
            Console.ReadLine();

            try
            {
                for (int k = 0; k < NoOfBlocks; k++)
                {
                    // Generate a large array of ints. This will end up on the heap
                    intArray[k] = new int[BlockSIZE];
                    Console.WriteLine("Allocated (but not touched) for array {0}: {1}", k, BlockSIZE);
                    // Sleep for 100 ms
                    System.Threading.Thread.Sleep(100);
                }
            }
            catch (Exception e)
            {
                Console.WriteLine(e.Message);
            }

            Console.WriteLine("done");
            Console.ReadLine();

            // Prevent the GC from destroying the objects created, by
            //  keeping a reference to them
            Console.WriteLine(intArray.Length);
        }
    }
}

Aside from allocating the arrays themselves, most of the code is fluff, and deals with writing output messages, waiting for a key to be pressed to get to the next section or delaying allocating the subsequent array. However, this all will come in handy when we’ll analyze the memory usage in detail. We’re also catching any exception that might come up, and write it on the screen directly.

Something ain’t right

Let’s hit F5 and see how the sample code performs:

Figure 2 – The sample code in action

Not only that it doesn’t complete successfully, but the code doesn’t even make it till the 100th 10-million int array. The exception thrown is our familiar System.OutOfMemoryException. Visual Studio’s built-in profiling dashboard (top right) shows the memory used by our process going close to 3 GB – just as the exception hits.

Can I Get Some Light Over Here ?

Ok, we need to understand what goes on. Luckily, Visual Studio has a built-in memory profiler we can use right away. This will run the code once more, and allow us to take a snapshot after the exception is thrown, so that we understand where the usage goes:

Figure 3 – Memory profiling using Visual Studio

Oddly enough, this time the code can successfully allocate 67 arrays (the code fails just after displaying error for the 0-based array no 66). When we first ran the code, it could only allocate 66 arrays.

Drilling down into the objects for which memory is allocated, we see the same number of arrays successfully allocated (67) as in the console output. Each array takes roughly 40 million bytes, as expected. But why only allocate barely close to 3 GB – to be more precise 2.6 GB, as the profiler shows above -, when it was supposed to go up to 6 GB ?

Anyone Got A Bigger Light ?

Clearly we need a tool that can shed more light on the problem, and allow us to see the memory usage in better detail. Enter VMMap, which is “a process virtual and physical memory analysis utility. It shows a breakdown of a process’s committed virtual memory types as well as the amount of physical memory (working set) assigned by the operating system to those types“. The “committed” and “virtual memory” parts might sound scary for now, but nonetheless, the tool seems to tell where the memory goes from the operating system’s point of view, which should show significantly more than Visual Studio’s integrated memory profiler. Let’s give it a spin:

Figure 4 – VMMap showing detailed memory usage

The Timeline… button allows going to a particular point in time from the process’ life to see the exact memory usage then. The resulting window – Timeline (Committed) – shows (1) a gradual upward trend, then (2) an approximately constant usage, followed by (3) a sudden drop to zero. You also briefly see the time of the selection being changed to a point where the memory usage line is pretty flat (within (2) described above, which happens after the exception was thrown, but before all the lines start dropping as part of (3)). Ignore the yellow/green/purple colors mixed in the chart for a second, and also note that when the time of the selection is changed, all the values in the main window change as well.

Back in the main window, it’s a lot like a christmas tree, with multiple colors and lots of columns with funny names, but let’s leave that aside for a moment, and only focus on the Size column in the top left. Actually, let’s take a closer look at only 2 values there, the first one – Total – which represents the total memory consumed, and last one – Free – representing the total free memory. Here they are highlighted:

Figure 5 – VMMap’s Size column

Hmm, the total size in the figure represents about 3.7 GB. That’s significantly larger than the 2.6 GB value we’ve got from Visual Studio’s Memory Profiler.

But look at the value for free space – that’s almost 300 MB of memory. This should have been more than enough for allocating 7 more of our 10 million int arrays with no problem.

How about we sum the 2 values – the total size and the free space ? The number is exactly 4 GB. Intriguing. This seems to suggest that the total memory our process gets is exactly 4 GB.

Wait, What ?

VMMap has a concise, to-the-point help. If you lookup what Total WS means, it says “The amount of physical memory assigned to the type or region“. So the value at the intersection of the Total WS column and the Total row will tell us exactly how much physical memory the process is taking at its peak usage, right after the out-of-memory exception is thrown:

FIgure 6 – VMMap’s Total Working Set

The value is… 12,356 KB. In other words about 12 MB. So VMMap is telling us that our program, which was supposed to allocate 6 GB of memory (but fails somewhere midway by throwing an exception) is only capable of allocating 12 MB of RAM ? But that’s not even the size of one array of 10 million int, and we know for sure we’ve allocated not one, but 67 of them successfully ! What kind of sorcery is this ?

A Trip Down Memory Lane

Before moving on, you need to know that there’s a very special book, called “Windows Internals“, that analyses in depth how Windows works under the hood. It’s been around since 1992, back when Windows NT roamed the Earth. The current 7th edition handles Windows 10 and Windows Server 2016. The chapter describing memory management alone is 182 pages long. Extensive references to the contents found there will be made next.

Back to our issue at hand, we have to start with some basic facts about how programs access memory. Specifically, in our small C# example, the resulting process is never allocating chunks of physical memory directly. Windows itself doesn’t hand out to the process any address for physical RAM. Instead, the concept of virtual memory is used.

Let’s see how “Windows Internals” defines this notion:

Windows implements a virtual memory system based on a flat (linear) address space that provides each process with the illusion of having its own large, private address space. Virtual memory provides a logical view of memory that might not correspond to its physical layout.

Windows Internals 7th Edition – Chapter 1 “Concepts and tools”

Let’s visualize this:

Figure 7 – Virtual Memory assigned to a process

So our process gets handed out a range of “fake”, virtual addresses. Windows works together with the CPU to translate – or map – these virtual addresses to the place where they actually point – either the physical RAM or the disk.

In figure 7, the green chunks are in use by the process, and point to a “backing” medium (RAM or disk), while the orange chunks are free.

Note something of interest: contiguous virtual memory chunks can point to non-contiguous chunks in physical memory. These chunks are called pages, and they are usually 4 KB in size.

Guilty As Charged

Let’s keep in mind the goal we’ve set out in the beginning of this post – we want to find out why we’ve got an out-of-memory exception. Remember that we know from VMMap that the total virtual memory size allocated to our process is 4 GB.

In other words, our process gets handed by Windows 4 GB of memory space, cut into pages, each 4 KB long. Initially all those pages will be “orange” – free, with no data written to them. Once we start allocating our int arrays, some of the pages will start turning “green”.

Note that there’s a sort of dual reality going on. From the process’ point of view, it’s writing and allocating the int arrays in either the “orange” boxes or “green” boxes that haven’t yet filled up; it knows nothing about where such a box is really stored in the back. The reality however, which Windows knows too well, is that there’s no data stored in either the “green” or “orange” boxes in figure 7, only simple mappings that lead to the data itself – stored in RAM or on the disk.

Since there’s really no compression at play here, there won’t really be a way to fit those 6 GB of data into just 4 GB. Eventually we’ll exhaust even the last available free page. You can’t just place 6 eggs into an egg carton that can only accommodate 4. We just have to accept that the exception raised is a natural thing, given the circumstances.

So The World Is A Small Place ?

“Are you saying that every process out there only gets access to 4 GB of memory ?(!)” I rightfully hear you asking.

Let’s take a look at the default configuration options used by Visual Studio for a C# console app:

Figure 8 – Visual Studio’s default build settings

Note the highlighted values. To simplify for the sake of our discussion, this combo (Any CPU as Platform target plus Prefer 32-bit) will get us 2 things:

  1. Visual Studio will compile the source code to an .exe file that will be run as a 32-bit process when started, regardless if the underlying system is 32-bit or 64-bit Windows.
  2. The Large Address Aware flag will be set in the resulting .exe file, which essentially tells Windows that it can successfully handle more than 2 GB of virtual address space.

These 2 points combine on a 64-bit Windows so that the process is granted via the Wow64 mechanism its maximum allocable space given its 32-bit constraint – that is of 2^32 bytes, or exactly 4 GB.

If the code is compiled specifically for 64-bit systems – eg by simply unticking the Prefer 32-bit option back in figure 8, suddenly the process – when run on a 64-bit machine – will get access to 128 TB of virtual address space.

An important point to remember: the values presented above for a 64-bit system, namely 4 GB (for a 32-bit process that is large address aware) and 128 TB (for a 64-bit process) respectively are the maximum addressable virtual address space ranges currently for a Windows 10 box. A system can have only 2 GB of physical memory, yet it doesn’t change the fact that it will be able to address 4 GB of address space; how that address space is distributed when actually needed – eg say 700 MB in physical RAM, while the rest on disk – is up to the underlying operating system. Conversely however, having 6 GB (or 7/10/20/50 GB) won’t help a 32-bit large address aware process get more than 4 GB of virtual address space.

So 1 mystery down, 2 more to go…

Bits and Pieces

Remember those 300+ MB of free space in Figure 5 back when the out-of-memory exception was thrown ? Why is the exception raised when there’s still some space remaining ?

Let’s look first at how .NET actually reserves memory for an array. As this older Microsoft article puts it: “The contents of an array are stored in contiguous memory“.

But where in memory are these arrays actually placed ? Every object ends up in one of 2 places – the stack or the heap. We just need to figure out which. Luckily, “C# in Depth” (Third Edition) by Jon Skeet has the answer, all within a couple of pages:

Array types are reference types, even if the element type is a value type (so int[] is still a reference type, even though int is a value type)

C# in Depth (Third Edition), Jon Skeet

[…]an instance of a reference type is always created on the heap.

C# in Depth (Third Edition), Jon Skeet

The thing is that there are 2 types of heaps that a process can allocate: unmanaged and managed. Which kind is used by ours ? “Writing High-Performance .NET Code” (2nd Edition) by Ben Watson has the answer:

The CLR allocates all managed .NET objects on the managed heap, also called the GC heap, because the objects on it are subject to garbage collection.

“Writing High-Performance .NET Code” (2nd Edition), Ben Watson

If the words “managed heap” look familiar, it’s because VMMap has a dedicated category just for it in the memory types it’s showing.

Now let’s look at what happens in the last seconds of our process’ lifetime, shortly before the exception is thrown. We’ll use the “Address Space Fragmentation” window, which displays the various types of memory in use and their distribution within the process’ address space. Ignore the colors in the “Address Space Fragmentation” window to the right for now, but keep an eye out for the free space. We’ll also do one more thing: sort the free space blocks in descending order.

Figure 9 – Address space fragmentation in action

We can see the free space gradually filling up. The allocations are all contiguous, just like the theory quoted before said they would be. So we don’t see, for example, the free space around the “violet” data being filled, since there’s no large enough “gap” to accommodate it. Yet in the end we’re still left with 2 free blocks, each in excess of 40 mil bytes, which should accept at least 2 more int arrays. You can see each of them highlighted, and their space clearly indicated on the fragmentation window towards the end of the animation.

The thing is that so far we’ve made an assumption – that each array will occupy the space required to hold the actual data, that is 4 bytes (/int) x 10 million (int objects) = 40 mil bytes. But let’s see how each block actually looks like in the virtual address space. We’ll go a to a point in time midway – when we know sufficient data has been already allocated – and only filter for the “Managed Heap” category, and sort the blocks by size in descending order:

Figure 10 – Tracking blocks across the fragmentation view

It turns out that each block is 49,152 KB in size – or 48 MB -, and is composed of 2 sub-blocks: one of 39,068 KB and another of 10,084 KB. The first value – 39,068 KB – is really close to our expected 40.000.000 bytes, with only 5,632 bytes to spare, which suggests this is were our int elements are stored. The second value seems to indicate some sort of overhead. Note several such sub-blocks being highlighted in the fragmentation view. Note that for each 48 MB block, both of the sub-blocks contained are contiguous.

What this means is that there has to be a free space “gap” big enough to accomodate 49,152 KB in order to successfully allocate another array of int elements. But if you look back at Figure 9, you’ll see that we’ve just run out of luck – the largest free space block only has 41,408 KB. The system no longer has contiguous free memory space to use for one more subsequent allocation, and – despite having several hundred MB of free space made up from small “pieces” – throws an out-of-memory exception.

So it wasn’t the fact that we’ve exhausted our 4 GB virtual memory limit that threw the out-of-memory exception, but the inability to find a large enough block of free space.

This leaves one with one more question to answer.

Your Reservation Is Now Confirmed

Remember the ludicrously low number of actual used physical memory of 12,356 KB back in Figure 6 ? How come it’s so low ?

We briefly touched on this issue in the last paragraph of So the World Is A Small Place ? by saying that some of the virtual address space can be backed up by physical memory, or can be paged out to disk.

There are 4 kinds of memory pages:

Pages in a process virtual address space are either free, reserved, committed, or shareable.

Windows Internals 7th Edition – Chapter 5 “Memory Management”

When we’re allocating each int array, what’s happening under the hood (through .NET and the underlying operating system) is that memory for that array is committed. Committing in this context involves the OS performing the following:

  1. Setting aside virtual address space within our process large enough for it to be able to address the area being allocated
  2. Providing a guarantee for the memory requested

For (1) this is relatively straightforward – the virtual address space is marked accordingly in structures called process VADs (virtual address descriptors). For (2), the OS needs to ensure that the memory requested is readily available to the process whenever it will need it in the future.

Note that neither of the two conditions demands providing the details of all the memory locations upfront. Giving out a guarantee that – say 12,000 memory pages – will be readily available when requested is very different than finding a particular spot for each of those 12,000 individual pages in a backing medium – be it physical RAM or one of the paging files on disk. The latter is a lot of work.

And the OS takes the easy way out – it just guarantees that the memory will be available when needed. It will do this by ensuring the commit limit – which is the sum of the size of RAM plus the current size of the paging files – is enough to honor all the requests for virtual address space that the OS has agreed to so far.

So if 3 processes commit memory – the first one 400 MB, the second 200 MB and the third 300 MB – the system must ensure that somewhere either in RAM or in the paging files there is enough space to hold at least 900 MB, that can be used to store the data if those processes might be accessing the data in the future.

The OS is literally being lazy. And this is actually the name of the trick employed: lazy-evaluation technique. More from “Windows Internals“:

For example, a page of private committed memory does not actually occupy either a physical page of RAM or the equivalent page file space until it’s been referenced at least once.

Why is this so ? Because:

When a thread commits a large region of virtual memory […], the memory manager could immediately construct the page tables required to access the entire range of allocated memory. But what if some of that range is never accessed? Creating page tables for the entire range would be a wasted effort.

And if you think back to our code, it’s simply allocating int arrays, it doesn’t write to any of the elements. We never asked to store any values in the arrays, so the OS was lazy enough to not go about building the structures – called PTEs (Page Table Entries) that would have linked the virtual address space within our process to physical pages that were to be stored in RAM.

But what does the term working set stand for back in Figure 6 ?

A subset of virtual pages resident in physical memory is called a working set.

Yet we never got to the point where we demanded the actual virtual pages, therefore the system never built the PTE structures that would have linked those virtual pages to physical ones in the RAM, which resulted in our process having a close-to-nothing working set, as can be clearly seen in Figure 6.

Is It All Lies ?

But what if we were to actually “touch” the data that we’re allocating ? According to what we’ve seen above, this would have to trigger the creation of virtual pages mapped to RAM. Writing a value to every int element in the arrays we’re spawning should do the trick.

However there’s one shortcut we can take. Remember that an int element takes 4 bytes, and that a page is 4 KB in size – or 4096 bytes. We also know that the array will be allocated as contiguous memory. Therefore, we don’t really need to touch every single element of the array, but only every 1024th element. This is just enough to demand for a page to be created and brought within the working set. So let’s slightly modify the for block that’s allocating the arrays in our code:

            for (int k = 0; k < NoOfBlocks; k++)
            {
                // Generate a large array of ints. This will end up on the heap
                intArray[k] = new int[BlockSIZE];
                //Console.WriteLine("Allocated (but not touched) for array {0}: {1} bytes", k, BlockSIZE);
                for(int i=0;i<BlockSIZE;i+=1024)
                {
                    intArray[k][i] = 0;
                }
                Console.WriteLine("Allocated (and touched) for array {0}: {1} bytes", k, BlockSIZE);
                // Sleep for 100 ms
                System.Threading.Thread.Sleep(100);
            }

Let’s see the result after running this code:

Figure 11 – Touching the committed memory brings it in the WS

The values are almost identical this time, meaning pages were created and our data currently sits in the physical RAM.

Q & A

Q: You mentioned back in one of sections that the pages are usually 4 KB in size. What’s the instance they have a different size, and what are those sizes ?
A: There are small (4 KB), large (2 MB) and – as of Windows 10 version 1607 x64 – huge pages (1 GB). For more details look in the “Large and small pages” section close to the beginning of chapter 5 in “Windows Internals, Part 1” (7th Edition).

Q: Why use this virtual memory concept in the first place ? It just seems to insert an unneeded level of indirection. Why not just write to RAM physical addresses directly ?
A: Microsoft itself lists 3 arguments going for the notion of virtual memory here. It also has some nice diagrams, and it’s concise for what it’s communicating across.

Q: You mentioned that on a 64-bit Windows, 64-bit compiler generated code will result in a process that can address up to 128 TB of virtual address space. However if I compute 2^64 I get a lot more than 128 TB. How come ?
A: A quote from Windows Internals:

Sixty-four bits of address space is 2 to the 64th power, or 16 EB (where 1 EB equals 1,024 PB, or 1,048,576 TB), but current 64-bit hardware limits this to smaller values.

Q: But AWE could be used from Pentium Pro times to allocate 64 GB of RAM.
A: Remember that the virtual address space is limited to 4 GB for a large-address aware, 32-bit process running on 64-bit Windows. A *lot* of physical memory could be mapped using the (by comparison, relatively small) virtual address space. In effect, the virtual address space is used as a “window” into the large physical memory.

Q: What if allocating larger int blocks, from 10 mil to say 12 mil elements each. Would the overhead be increased proportionally ?
A: No. There are certain block sizes that seem to be used by the Large Object Heap. When allocating 12 mil elements, the overall size of the block is still 49,152 KB, with a “band” of only 2,272 KB of reserved memory. When allocating 13 mil elements, the overall size of the block goes up to 65,536 KB, with 14,748 KB of reserved space for each:

Q: What’s causing the overhead seen in the question above, as well as within the article ?
A: At this time (4/21/2019) I don’t have the answer. I do believe the low-fragmentation heap, which .NET is using under the hood for its heap implementation, holds the secret to this.

Q: Does the contiguous data found within each virtual page map to correspondingly contiguous data within the physical memory pages ? Or to rephrase, are various consecutive virtual space addresses within the same virtual page pointing to spread-out locations within a physical page, or even multiple physical pages ?
A: They are always contiguous. Refer to “Windows Internals, Part 1” (7th Edition) to chapter 5, where it’s detailed how in the process of address translation the CPU copies the last 12 bits in every virtual address to reference the offset in a physical page. This means the order is the same within both the virtual page as well as the physical one. Note how RamMap shows the correspondence of physical-to-virtual addresses on a 4 KB boundary, or exactly the size of a regular page.

Q: In all the animation and figures I’m seeing a yellow chunk of space, alongside the green one for “Managed Heap”. This yellow one is labeled “Private Data”, and it’s quite large in size. What’s up with that ?
A: There’s a bug in the current version of VMMap, whereby the 32-bit version – needed to analyze the 32-bit executable for the int allocator code – incorrectly classifies virtual addresses pointing to .NET allocated data above 2 GB as private data, instead of managed heap. You’ll also see that the working set for all int arrays classified as such appears to be nothing – when in reality this is not the case. I’m currently (4/21/2019) in contact with Mark Russinovich (the author of VMMap) to see how this can be fixed. The bug however doesn’t exist in the 64-bit version of VMMap, and all the allocations will correctly show up as ‘Managed Heap’.

Q: I’d like to understand more about the PTE structures. Where can I find more information ?
A: Look inside chapter 5 (“Memory Management“) within “Windows Internals, Part 1” (7th Edition). There’s an “Address Translation” section that goes into all the details, complete with diagrams.

Q: Your article is hard to follow and I can’t really understand much. Can you recommend some sources that do a better job than you at explaining these concepts ?
A: Eric Lippert has a very good blog post here. There’s also a very nice presentation by Mark Russinovich here which handles a lot of topics about memory (including a 2nd presentation, also 1+ hours long). Though both sources are quite dated, being several years old, the concepts are very much current.

Q: Where can I find more info about the Platform Target setting in Visual Studio ?
A: The previous post on this very blog describes that in detail. You can start reading from this section.

Q: I’ve tried duplicating your VMMap experiment, but sometimes I’m seeing that the largest free block available is in excess of 100 KB. This is more than double the size of an int array, which should take around 49 KB (39KB + 10KB reserve), so there should’ve been space for at least one subsequent allocation. What’s going on ?
A: I don’t have a thorough answer for this right now (4/21/2019). I’ve noticed this myself. My only suspicion is that something extra goes on behind the scenes, aside the simple allocation for the int array, such as the .NET allocation mechanism going after some extra blocks of memory.

Q: I heard an int takes a double amount of space on a 64-bit system. You’re stating in this article that it’s 4 bytes on either 32-bit/64-bit. You’re wrong !
A: Don’t confuse an IntPtr – whose size is 4 bytes on a 32-bit platform and 8 bytes on a 64-bit one – which represents a pointer, to an int value. The pointer contains that int variable’s address, but what’s found at that address is the int value itself.

Источник

When the .NET Framework was first released, many developers believed the introduction of the garbage collector meant never having to worry about memory management ever again. In fact, while the garbage collector is efficient in managing memory in a managed application, it’s still possible for an application’s design to cause memory problems.

One of the more common issues we see regarding memory involves System.OutOfMemoryExceptions. After years of helping developers troubleshoot OutOfMemoryExceptions, we’ve accumulated a short list of the more common causes of these exceptions. Before I go over that list, it’s important to first understand the cause of an OutOfMemoryException from a 30,000 foot view.

What Is an OutOfMemoryException?

A 32-bit operating system can address 4GB of virtual address space, regardless of the amount of physical memory that is installed in the box. Out of that, 2GB is reserved for the operating system (Kernel-mode memory) and 2GB is allocated to user-mode processes. The 2GB allocated for Kernel-mode memory is shared among all processes, but each process gets its own 2GB of user-mode address space. (This all assumes that you are not running with the /3gb switch enabled.)

When an application needs to use memory, it reserves a chunk of the virtual address space and then commits memory from that chunk. This is exactly what the .NET Framework’s garbage collector (GC) does when it needs memory to grow the managed heaps. When the GC needs a new segment for the small object heap (where objects smaller than 85K reside), it makes an allocation of 64MB. When it needs a new segment for the large object heap, it makes an allocation of 32MB. These large allocations must be satisfied from contiguous blocks of the 2GB of address space that the process has to work with. If the operating system is unable to satisfy the GC’s request for a contiguous block of memory, a System.OutOfMemoryException (OOM) occurs.

There are two reasons why you might see an OOM condition.

  1. Your process is using a lot of memory (typically over 800MB.)
  2. The virtual address space is fragmented, reducing the likelihood that a large, contiguous allocation will succeed.

It’s also possible to see an OOM condition due to a combination of 1 and 2.

Let’s examine some of the common causes for each of these two reasons.

Common Causes of High Memory

When your worker process approaches 800MB in private bytes, your chances of seeing an OOM condition begin to increase simply because the chances of finding a large, contiguous piece of memory within the 2GB address space begin to decrease significantly. Therefore, you want to avoid these high memory conditions.

Let’s go over some of the more common causes of high memory that we see in developer support at Microsoft.

Large DataTables

DataTables are common in most ASP.NET applications. DataTables are made up of DataRows, DataColumns, and all of the data contained within each cell. Large DataTables can cause high memory due to the large number of objects that they create.

The most common cause of large DataTables is unfiltered data from a back-end data source. For example, if your site queries a database table containing hundreds of thousands of records and your design makes it possible to return all of those records, you’ll end up with a huge amount of memory consumed by the result set. The problem can be greatly exacerbated in a multi-user environment such as an ASP.NET application.

The easiest way to alleviate problems like this is to implement filtering so that the number of records you return is limited. If you are using a DataTable to populate a user-interface element such as a GridView control, use paging so that only a few records are returned at a time.

Storing Large Amounts of Data in Session or Application State

One of the primary considerations during application design is performance. Developers can come up with some ingenious ways to improve application performance, but sometimes at the expense of memory. For example, we’ve seen customers who stored entire database tables in Application state in order to avoid having to query SQL Server for the data! That might seem like a good idea at first glance, but the end result is an application that uses an extraordinary amount of memory.

If you need to store a lot of state data, consider whether using ASP.NET’s cache might be a better choice. Cache has the benefit of being scavenged when memory pressure increases so that you don’t end up in trouble as easily.

Running in Debug Mode

When you’re developing and debugging an application, you will typically run with the debug attribute in the web.config file set to true and your DLLs compiled in debug mode. However, before you deploy your application to test or to production, you should compile your components in release mode and set the debug attribute to false.

ASP.NET works differently on many levels when running in debug mode. In fact, when you are running in debug mode, the GC will allow your objects to remain alive longer (until the end of the scope) so you will always see higher memory usage when running in debug mode.

Another often unrealized side-effect of running in debug mode is that client scripts served via the webresource.axd and scriptresource.axd handlers will not be cached. That means that each client request will have to download any scripts (such as ASP.NET AJAX scripts) instead of taking advantage of client-side caching. This can lead to a substantial performance hit.

Running in debug mode can also cause problems with fragmentation. I’ll go into more detail on that later in this post. I’ll also show you how you can tell if an ASP.NET assembly was compiled with debug enabled.

Throwing a Lot of Exceptions

Exceptions are expensive when it comes to memory. When an exception is thrown, not only does the GC allocate memory for the exception itself, the message of the exception (a string), and the stack trace, but also memory needed to store any inner exceptions and the corresponding objects associated with that exception. If your application is throwing a lot of exceptions, you can end up with a high memory situation quite easily.

The easiest way to determine how many exceptions your application is throwing is to monitor the # of Exceps Thrown / sec counter in the .NET CLR Exceptions Performance Monitor object. If you are seeing a lot of exceptions being thrown, you need to find out what those exceptions are and stop them from occurring.

Regular Expression Matching of Very Large Strings

Regular expressions (often referred to as regex) represent a powerful way to parse and manipulate a string by matching a particular pattern within that string.  However, if your string is very large (megabytes in size) and your regex has a large number of matches, you can end up in a high memory situation.

The RegExpInterpreter class uses an Int32 array to keep track of any matches for a regex and the positions of those matches. When the RegExpInterpreter needs to grow the Int32 array, it does so by doubling its size. If your use of regex creates a very large number of matches, you’ll likely see a substantial amount of memory used by these Int32 arrays.

What do I mean by “large number of matches”? Suppose you are running a regex against the HTML from a page that is several megabytes in size. (You might think that this isn’t a feasible scenario, but we have seen a customer do this with HTML code that was over 5MB!) Suppose also that the regex you are using against this HTML is as follows.

<body(.|n)*</body>

This regex does the following:

  • “<body” matches the literal characters “<body”.
  • The parenthesis tells the regex engine to match the regex within them and store the match as a back-reference.
  • The dot (.) will match any single character that is not a line break.
  • The “n” will match any character that is a line break.
  • The “*” repeats the regex in parenthesis between zero and an unlimited number of times. It also indicates a greedy match, meaning that it will match as many times as possible within the string.
  • “</body>” matches the literal characters “</body>”.

In other words, if you use this regex against the HTML code from a page, it will match the entire body of the page. It will also store that body as a back-reference. The result is a very large Int32 array.

Incidentally, this problem isn’t specific to our implementation of regex. This same type of problem will be encountered with any regex engine that is NFA-based. The solution to this problem is to rethink the architecture so as to avoid such large strings and large matches.

Common Causes of Fragmentation

Fragmentation is problematic because it can cause allocations of contiguous memory to fail. Assume that you have only 100MB of free address space for a process (you’re almost certain to have much more than that in real life) and one 4KB DLL loaded into the middle of that address space as shown in Figure 1. In this scenario, an allocation that requires 64MB of contiguous free space will fail with an OOM exception.

Figure 1 – Fragmented Address Space

The following are common causes of fragmentation.

Running in Debug Mode

One of the features in ASP.NET that is designed to avoid fragmentation is a feature called batch compilation. When batch compilation is enabled, ASP.NET will dynamically compile each folder of your application into a single DLL when the application is JITted. If batch compilation is not enabled, each page and user control is compiled into a separate DLL that is then loaded into the address space for the process. Each of these DLLs is very small, but because they are loaded into a non-specific address in memory, they tend to get peppered all over the address space. The result is a radical decrease in the amount of contiguous free memory, and that leads to a much greater probability of running into an OOM condition.

When you deploy your application, you need to make sure that you set the debug attribute in the web.config file to false as follows.

<compilation debug=»false» />

If you’d like to ensure that debug is disabled on your production server regardless of the setting in the web.config file, you can use the <deployment> element introduced in ASP.NET 2.0. This element should be set in the machine.config file as follows.

<configuration>
    <system.web>
        <deployment retail=»true» />
    </system.web>
</configuration>

Adding this setting to your machine.config file will override the debug attribute in any web.config file on the server.

When debugging is enabled, ASP.NET will add a Debuggable attribute to the assembly. You can use .NET Reflector or ildasm.exe to examine an ASP.NET assembly and determine if it was compiled with the Debuggable attribute. If it was, debugging is enabled for the application.

Figure 2 shows two ASP.NET assemblies from the Temporary ASP.NET Files folder opened in .NET Reflector. The top assembly is selected and you can see that the Debuggable attribute is highlighted in red. (In order to see the manifest information in the right pane, right-click the assembly and select Disassemble from the menu.) The application running this assembly is running in debug mode.

Figure 2 — .NET Reflector showing an assembly compiled with debug enabled.

Figure 3 shows .NET Reflector with the second assembly selected. Notice that this assembly doesn’t have a Debuggable attribute. Therefore, the application running this assembly is not running in debug mode.

Figure 3 — .NET Reflector showing an assembly compiled without debug enabled.

Generating Dynamic Assemblies

Another common cause of fragmentation is the creation of dynamic assemblies. Dynamic assemblies fragment the address space of the process for the same reason that running in debug mode does.

Instead of going into the details here on how this happens, I’ll point you to my colleague Tom Christian’s blog post on dynamic assemblies. Tom goes into detail on what can create dynamic assemblies and how to work around those issues.

Resources

The following resources are helpful when tracking down memory problems in your application.

Gathering Information for Troubleshooting

Tom Christian’s blog post on gathering information for troubleshooting an OOM condition will help you if you need to open a support incident with us. Read more from Tom in this post.

Post-mortem Debugging of Memory Issues

Tess Ferrandez is famous for her excellent blog on debugging ASP.NET applications. She’s accumulated quite a collection of excellent posts on memory issues that includes everything from common memory problems to case studies that include debugging walkthroughs with Windbg. You can find Tess’s 21 most popular blog posts in this post.

Using DebugDiag to Troubleshoot Managed Memory

Tess has also recently published a blog post that includes a DebugDiag script that she wrote for the purpose of troubleshooting managed memory problems. The great thing about using DebugDiag with this script is that you can simply point it to a dump file of your worker process and it will automatically tell you a wealth of information that can help you track down memory usage.

You can find out how to use Tess’s script and download a copy of it here.

Understanding GC

If you ever wanted to know how the .NET garbage collector works, Tess can help! She wrote a great blog post that includes links to other great GC resources, and you can read it here.

“I Am a Happy Janitor!”

Maoni, a developer on the Common Language Runtime team, wrote a blog post that explains how the garbage collector works using the colorful analogy of a janitor. Read Maoni’s enlightening post here.

Using GC Efficiently

Maoni also wrote an excellent series on using GC efficiently in order to prevent memory issues. You can read the series here.

Conclusion

I hope that this information will help you to identify memory problems in your ASP.NET application that can lead to OutOfMemoryExceptions. However, if you have exhausted these ideas and are still plagued with memory problems, contact us and open a support ticket. We’ll be happy to help you troubleshoot!


Источник

Sep 18, 2017

C# - Out of Memory Exception

c#,

c# outofmemoryexception,

exception,

outofmemory,

system.outofmemoryexception c#,

systemoutofmemory

September 18, 2017

C# System.OutOfMemoryException

While programming for my web app the other day, my application threw an System.OutOfMemoryException when initializing a list of objects. To me this seemed pretty much impossible because my machine has 4GB RAM. Specifically the error happened when I tried to add an existing collection to a new list. For example:

List newShapes = new List(selectedShapes);

My original thought was that this assignment shouldn’t allocate much memory since the shapes from my list should already exist inside memory. The Shape class is a complex object and I was trying to assign about 25,000 items to the new list at once. We should also note that the total size of all Shapes in the application come from a database that is only about 250MB in size. At first I was stumped and had no idea what may be causing the System.OutOfMemoryException. 

What happened

  1. If you are running a 32 bit Windows, you won’t have all the 4GB accessible, only 2GB per object array. This is a limitation of the .NET framework. (This happened to be my issue, as my shape array was greater than 2GB after debugging..)
  2. Don’t forget that the underlying implementation of List is an array. If your memory is heavily fragmented, there may not be enough contiguous space to allocate your List, even though in total you have plenty of free memory.

Solutions

  1. If you are running on 64 bit Windows and you have enough free memory, ensure that the platform target is set correctly in the project properties > build properties.
    System Out Of Memory Fix
  2. If you are using .NET 4.5 or greater you can add this line to the runtime section of your App.Config:

<runtime> 
   <gcallowverylargeobjects enabled="true"></gcallowverylargeobjects> 
</runtime>

Tips when debugging a C# System.OutOfMemoryException

  • Remember that data stored in the database compared to in memory is very different. Use this C# snippet to observe how much your memory is changing as you load the object(s) into memory.

 

GC.GetTotalMemory()

    Common causes of C# System.OutOfMemoryException

    • Initializing an array which is not large enough, ensure the array size is correct! This is by far the most common cause of this exception.

    • Reading very large data sets into memory, as demonstrated in the example above.

    • Having too many active sessions or storing too much data in session state. 

    • Applying a complex regex (regular expression) to a large file or string.

    • Downloading, streaming or uploading large files.

    • You are calling the StringBuilder.Insert method and are repeatedly concatenating strings.

    • You are not implementing the Implementing a Dispose method when dealing with unmanaged resources, and your app is leaking.

    • You are storing very large sets of data in memory. An example could be querying large database data sets via LINQ in memory.

    Источник

    Welcome to the second part of the series about Debugging common .NET exceptions. The series is my attempt to demystify common exceptions as well as to provide actual help fixing each exception.

    Debugging System.OutOfMemoryException using .NET tools

    In this post, I take a look at one of the more tricky exceptions to fix: System.OutOfMemoryException. As the name suggests, the exception is thrown when a .NET application runs out of memory. There are a lot of blog posts out there, trying to explain why this exception occurs, but most of them are simply rewrites of the documentation on MSDN: System.OutOfMemoryException Class. In the MSDN article, two different causes of the OutOfMemoryException are presented:

    1. Attempting to expand a StringBuilder object beyond the length defined by its StringBuilder.MaxCapacity property. This type of error typically has this message attached: «Insufficient memory to continue the execution of the program.»
    2. The common language runtime (CLR) cannot allocate enough contiguous memory.

    In my past 13 years as a .NET developer, I haven’t experienced the first problem, why I won’t bother spending too much time on it. In short, doing something like this will cause a System.OutOfMemoryException:

    StringBuilder sb = new StringBuilder(1, 1);
sb.Insert(0, "x", 2);

    Why? Well, we define a new StringBuilder with a max capacity of one character and then try to insert two characters.

    With that out of the way, let’s talk about why you are probably experiencing the exception: because the CLR cannot allocate the memory that your program is requesting. To translate this into something that your Mom would understand, your application is using more resources than available.

    .NET programs often use a lot of memory. The memory management in .NET is based on garbage collection, which means that you don’t need to tell the framework when to clean up. When .NET detects that an object is no longer needed, it is marked for deletion and deleted next time the garbage collector is running. This also means that an OutOfMemoryException doesn’t always equal a problem. 32-bit processes have 2 GB of virtual memory available, and 64-bit processes have up to 8 TB. Always make sure to compile your app to 64-bit if running on a 64-bit OS (it probably does that already). If you’re interested in more details about this subject, I recommend this article from Eric Lippert, a former Microsoft employee working on the C# compiler: “Out Of Memory” Does Not Refer to Physical Memory. It’s important to distinguish between heavy memory usage and a memory leak. The first scenario can be acceptable, while the second always requires debugging.

    To start debugging the OutOfMemoryException, I recommend you to look at your application either through the Task Manager or using perfmon.msc. Both tools can track the current memory consumption, but to get a better overview over time, perfmon is the best. When launched, right-click the graph area and click Add Counters… Expand the .NET CLR Memory node and click # Total committed Bytes. Finally, select the process you want to monitor in the Instances of selected object list and click the OK button.

    For the rest of this post, I will use and modify a sample program, adding strings to a list:

    class Program
{
    static void Main(string[] args)
    {
        try
        {
            var list = new List<string>();
            int counter = 0;
            while (true)
            {
                list.Add(Guid.NewGuid().ToString());
                counter++;
                if (counter%10000000 == 0)
                {
                    list.Clear();
                }

            }
        }
        catch (OutOfMemoryException e)
        {
            Environment.FailFast(String.Format($"Out of Memory: {e.Message}"));
        }
    }
}

    In its current state, the program keeps adding strings to a list and every 10,000,000 times clear the list. When looking at the current memory usage in Perfmon, you’ll see the current picture:

    Current memory usage in Perfmon

    Garbage collection at its finest. Here, I’ve removed the call to list.Clear():

    class Program
{
    static void Main(string[] args)
    {
        try
        {
            var list = new List<string>();
            while (true)
            {
                list.Add(Guid.NewGuid().ToString());
            }
        }
        catch (OutOfMemoryException e)
        {
            Environment.FailFast(String.Format($"Out of Memory: {e.Message}"));
        }
    }
}

    We now get a completely other picture:

    Allocating memory in Perfmon

    The program keeps allocating memory, until a System.OutOfMemoryException is thrown.

    The example illustrates how you can utilize Perfmon to monitor the state of your application. Like the chefs on TV, I cheated and made up an example for this post. In your case, you probably have no clue to what causes the extensive use of memory. Memory profilers to the rescue!

    Unlike Task Manager and Perfmon, memory profilers are tools to help you find the root cause of a memory problem or memory leak. There a lot of useful tools out there like JetBrains dotMemory and ANTS Memory Profiler. For this post, I’ll use .NET Memory Profiler, which I have used heavily in the past. BTW, as an elmah.io customer, you will get a 20% discount on .NET Memory Profiler.

    .NET Memory Profiler integrates nicely into Visual Studio, why profiling your application is available by clicking the new Profiler > Start Memory Profiler menu item. Running our sample from previously, we see a picture similar to that of Perfmon:

    Memory allocation in Visual Studio

    The picture looks pretty much like before. The process allocates more and more memory (the orange and red lines), and the process throws an exception. In the bottom, all objects allocated from the profiling sessions are shown and ordered allocations. Looking at the top rows is a good indicator of what is causing a leak.

    In the simple example, it’s obvious that the strings added to the list if the problem. But most programs are more complex than just adding random strings to a list. This is where the snapshot feature available in .NET Memory Profiler (and other tools as well) shows its benefits. Snapshots are like restore points in Windows, a complete picture of the current memory usage. By clicking the Collect snapshot button while the process is running, you get a diff:

    Diff memory usage in Visual Studio

    Looking at the Live Instances > New column, it’s clear that someone is creating a lot of strings.

    I don’t want this to be an ad for .NET Memory Profiler, so check out their documentation for the full picture of how to profile memory in your .NET programs. Also, make sure to check out the alternative products mentioned above. All of them have free trials, so try them out and pick your favorite.

    I hope that this post has provided you with «a very particular set of skills» (sorry) to help you debug memory issues. Unfortunately, locating memory leaks can be extremely hard and requires some training and experience.

    Also make sure to read the other posts in this series: Debugging common .NET exception.

    elmah.io: Error logging and Uptime Monitoring for your web apps

    This blog post is brought to you by elmah.io. elmah.io is error logging, uptime monitoring, deployment tracking, and service heartbeats for your .NET and JavaScript applications. Stop relying on your users to notify you when something is wrong or dig through hundreds of megabytes of log files spread across servers. With elmah.io, we store all of your log messages, notify you through popular channels like email, Slack, and Microsoft Teams, and help you fix errors fast.

    elmah.io app banner

    See how we can help you monitor your website for crashes
    Monitor your website

    Источник

    C# OutOfMemoryException

    Introduction to C# OutOfMemoryException

    OutOfMemoryException in C# is an exception that is thrown by the .NET framework execution engine when the program does not have enough memory to continue its execution. As its name suggests this exception will occur in our program when the CLR i.e. Common Language Runtime will not be able to allocate enough memory which will be required to perform certain operations of our program.

    This exception does not always mean that we do not have enough space available in memory but sometimes it means that we do not have enough contiguous memory which is required by our program for allocation.

    Syntax

    The syntax to catch an OutOfMemoryException in C# is as follows:

    try
{
//user code which can throw OutOfMemoryException
}
catch(OutOfMemoryException exception)
{
//statements to handle the exception
}

    The syntax to throw an OutOfMemoryException in C# is as follows:

    throw new OutOfMemoryException();

    In the above statement ‘throw’ is the keyword which is used to throw exceptions in C#.

    How OutOfMemoryException works in C#?

    In C#, we get OutOfMemoryException when our program does not have enough space to continue its execution. There could be many reasons for getting this exception. We also encounter this exception when in total we have enough space for our program to execute but this space is not contiguous for the allocations required to be done by our program. The two major reasons for this exception are as follows:

    Trying to increase the length of a StringBuilder object beyond the length which is specified by the MaxCapacity property of StringBuilder.

    We will get the exception saying “Insufficient memory to continue the execution of the program.”

    • When we are making an assignment or calling a method that requires memory allocation and at the same time the CLR is not able to provide enough contiguous memory for our allocation then we will get OutOfMemoryException.

    The other reasons which can become the cause of this exception include:

    • Running the application on a 32-bit system which has only 2GB of virtual memory due to which CLR finds it difficult to provide contiguous memory for the allocations required by the application.
    • After working with unmanaged resources like file handlers, database connections, pointers, etc. if we do not dispose of these resources then it leads to a memory leak which in result degrades the performance of our application and can lead to OutOfMemoryException.
    • Working with large data sets requires a huge amount of memory and if CLR does not have enough contiguous space available for it then it results in OutOfMemoryException.
    • As strings are immutable, the operations performed on string create a new string in the memory. So if we are working with large strings and if we are performing concatenation operation repeatedly on the string then it can lead to multiple memory allocations which in result will degrade the performance of our application and can become a cause of OutOfMemoryException.
    • If we have pinned multiple objects in the memory for a very long period then it becomes difficult for the garbage collector to provide contiguous memory allocation to these objects.

    Examples

    Here are the following examples mention below

    Example #1

    Example showing OutOfMemoryException thrown by the program when we try to expand the StringBuilder object beyond its maximum capacity.

    Code:

    using System;
using System.Text;
public class Program
{
public static void Main()
{
StringBuilder stringBuilder = new StringBuilder(17, 17);
stringBuilder.Append("Welcome to the ");
try
{
stringBuilder.Insert(0, "world of C# programming", 1);
Console.WriteLine(stringBuilder.ToString());
Console.ReadLine();
}
catch (OutOfMemoryException exception)
{
Console.WriteLine(exception.Message);
Console.ReadLine();
}
}
}

    Output:

    C# OutOfMemoryException output 1

    Example #2

    Example showing a program that encounters OutOfMemoryException while trying to add the element in the list where the number of elements to be added is more than the capacity of the list.

    Code:

    using System;
using System.Text;
using System.Collections.Generic;
namespace ConsoleApp4
{
public class Program
{
public static void Main()
{
try
{
string[] strArray = GetArray();
Console.WriteLine(strArray);
Console.ReadLine();
}
catch (OutOfMemoryException exception)
{
Console.WriteLine(exception);
Console.ReadLine();
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
Console.ReadLine();
}
}
public static string[] GetArray()
{
List<string> strList = new List<string>();
for (int i = 0; i <= int.MaxValue; i++)
{
strList.Add("Hello");
}
return strList.ToArray();
}
}
}

    Output:

    C# OutOfMemoryException output 2

    How to avoid OutOfMemoryException in C#?

    We can avoid OutOfMemoryException in C# by keeping in mind the following points:

    • To avoid this exception while working with StringBuilder, we can call the constructor StringBuilder.StringBuilder(Int32, Int32) and can set the MaxCapacity property to a value that will be large enough to serve the accommodation required when we expand the corresponding StringBuilder object.
    • To avoid this exception while working on a 32-bit system, we can recompile our application from 32 bit to 64-bit system in visual studio by below steps:
    1. Menu Bar -> Build -> Configuration Manager
    2. Click on the list of Active Solution Platform and select the 64-bit platform and then click on the Close button.

    If the 64-bit platform is not available in the list then:

    • Click on New option from the list
    • On the New Solution Platform Window, click on the ‘Type or select the new platform’ list and then select the ‘x64’ option.
    • Click on the OK button.
    • To avoid getting this exception while working with unmanaged resources, we should always call Dispose() method after completing our work with the unmanaged resource which is no longer required.
    • To avoid this exception while working with large data sets, we should try to filter the data and then should pass only the required data for processing.
    • To avoid this exception while working with large strings or while performing large string concatenation, we can make use of StringBuilder instead of string because StringBuilder is mutable and does not create a new instance of the string when we perform any operation on it.

    Conclusion

    The OutOfMemoryException is a runtime exception that tells the programmer that there is no enough memory or there is a lack of contiguous memory for the allocations required by the C# program.

    To avoid this exception the user should always take necessary precautions and should handle this exception.

    Recommended Articles

    This is a guide to C# OutOfMemoryException. Here we discuss how OutOfMemoryException works in C# along with the programming examples. You may also have a look at the following articles to learn more –

    1. DataReader C#
    2. C# StreamReader
    3. C# nameof
    4. C# StringReader
    Источник

    Думаю, многие разработчики в своей практике сталкивались с такой неприятной исключительной ситуацией, как OutOfMemoryException. Чем она неприятна? Тем, что она свидетельствует об одной из двух вещей: либо ваше приложение скушало всю доступную память и сделало это ожидаемо, потому что вы его так запрограммировали, либо оно сделало это вследствие утечки памяти (memory leak). И первый, и второй случай чреваты серьезными проблемами. В первом случае нам нужно рефакторить код и, возможно, даже архитектуру с целью избежания этой ситуации в дальнейшем, и не факт, что при этом не придется придумывать какие-нибудь обходные методы. Во втором случае мы сталкиваемся с тем, что мы, скорее всего, понятия не имеем, где происходит утечка и как с этим бороться, то есть у нас налицо предстоящий достаточно сложный дебаг. И самое неприятное во всей этой ситуации то, что мы не знаем, с каким же все-таки вариантом мы имеет дело. А если не знаем, значит, самое время взять скальпель стетоскоп и прослушать пациента.

    Для начала нужно разобраться с тем, как приложения (в нашем случае, веб-приложения) работают с памятью в Windows. Я думаю, многие замечали, что очень часто проблемы с OutOfMemoryException начинаются еще когда приложение только-только вышло за рубеж 1 Гб. Но ведь мы работаем в 32-битной системе, значит приложение может адресовать до 4 Гб памяти, не так ли? Не совсем. Дело в том, что из этих 4 Гб половину забирает себе Windows, а остальное делится между исполняющей средой .NET Framework и нашим приложением. Вот и получается, что памяти у нашего приложения не так уж и много. Заинтересовавшихся отсылаю за деталями к хорошей статье некоего Jesper’а, отличной ресторанной аналогии Tess и статье Johan Straarup.

    Итак, попробуем разобраться, что же происходит внутри нашего приложения

    Первый шаг очень простой: необходимо собрать как можно больше информации. Для начала общаемся с QA, которые нашли баг и пытаемся выяснить условия, при которых воспроизводится баг. OutOfMemoryException – исключение подлое, в большинстве случаев оно не воспроизводится в лоб, а просто вылетает в какой-то момент жизни приложения, когда память переполнится. Полученная информация нам поможет и для воспроизведения бага, и для собственного понимания того, что творится внутри процесса. В частности, у нас вот проблема в основном проявлялась при большом количестве пользователей, что неудивительно, так как сессии довольно большие, но все равно странно, так как непонятно, чем же занимается в таком случае сборщик мусора? Ну, о причинах этого потом – в примере.

    Используем встроенные средства мониторинга ОС

    На втором шаге мы пробуем минимальными усилиями разобраться с тем, что у нас происходит с памятью в процессе и сколько памяти у нас выделяется под сессию. Сделать это очень просто при помощи стандартных счетчиков производительности (performance counters) Windows. Нам пригодятся следующие как минимум счетчики, но вы можете воспользоваться и расширенным списком (помните, что включать их нужно для вашего процесса aspnet_wp/w3wp):

    .NET CLR Memory/# Bytes in all heaps
    .NET CLR Memory/# Total committed Bytes
    .NET CLR Memory/# Total reserved Bytes
    .NET CLR Memory/Gen 0 heap size
    .NET CLR Memory/Gen 1 heap size
    .NET CLR Memory/Gen 2 heap size
    .NET CLR Memory/Large Object Heap Size

    Выглядеть это будет примерно так:

    Собственно, здесь мы хотим посмотреть размеры куч всех трех поколений и LOH, а также общую занятую память и committed/reserved bytes. Reserved bytes показывает, сколько наше приложение зарезервировало себе памяти, committed – сколько оно реально из этого количества уже использует. Отличное описание этих и других полезных счетчиков и их назначение можно найти здесь. Итак, запускаем счетчики и Windows Task Manager (с ним будем отслеживать Mem usage и VM usage для процесса aspnet_wp/w3wp), заполняем себе табличку в Excel – и поехали.

    Примечание:

    Настоятельно рекомендую перед работой с Task Manager прочитать пост Tess. В частности, там можно прочитать вот такую интересную вещь:

    «If you want to see this in action, you can create a winforms application and allocate a bunch of objects and see the working set go up, and then if you minimize the app, the working set drops. This doesn’t by any means mean that you have just released all this memory. It just means that you are looking at a counter that is totally irrelevant for determining how much stuff you store in memory 🙂 Yet… this is the counter that people most often look at to determine memory usage…

    I know that by now you are probably thinking «yeah right», you haven’t even heard of this counter before, why would I say that this is the counter most people look at??? The answer is, because most people use task manager to look at memory usage of a process, and specifically look at the Memory Usage column. Surprise surprise:) what this actually shows you is the working set of the process…

    If you want to see private bytes which is a far more interesting counter, you should look at the column in task manager that is labeled Virtual Memory Size (yeah, that’s really intuitive:)), or better yet, look in performance monitor at processprivate bytes and processvirtual bytes, there is no reason not to if your intent is to investigate high memory usage or a memory leak. «

    Так что будьте осторожны 😉

    Логинимся в приложение одним пользователем, делаем замеры, логинимся следующим, делаем замеры. По пути можно попробовать просмотреть разную функциональность системы, чтобы увидеть, где именно мы получаем большое увеличение памяти. По размерам куч видим, как постепенно увеличивается количество памяти, и что у нас чистится GC, а что – нет. По Mem usage и VM usage определяем общее количество используемой памяти и пытаемся путем несложных арифметических операций определить, сколько же все-таки у нас занимает один пользователь в сессии. В нашем случае уже на 10 пользователях приходит понимание, что общее количество памяти растет непомерно и освобождаться почему-то не хочет. Ну, это логично – созданные сессии еще активны и сборщик мусора их собрать не может. Ставим опыт: отпускаем приложение на уровне 700 Мб и уходим на полчаса пить чай. Через полчаса заходим на сайт новым пользователем и делаем очередной замер. Нет, почти ничего не изменилось, используемая память продолжает увеличиваться, хотя должна была существенно уменьшится. Размер одной сессии в среднем – 30-40 Мб, что слишком много. Итак, либо у нас по какой-то причине не очищаются сессии, либо утечка памяти в другом месте. Информации по-прежнему мало. Пора браться за более серьезные инструменты анализа.

    Знакомимся с тяжелой артиллерией

    В роли тяжелой артиллерии может выступать практически любой performance tool, который умеет нормально мониторить память. Тулов много, но толковую информацию предоставляют единицы. Лично я все не смотрел, но вот мой любимый dotTrace, который не раз помогал мне находить performance-проблемы в коде, здесь полностью облажался. Ничего толкового из него я добиться не смог, может, просто руки кривые 🙂 Поэтому, если у вас нет особых вариантов, то советую сразу же взяться за WinDbg. WinDbg – это универсальный дебаггер, которая позволяет отлаживать любые win-приложения. А самое классное в нем то, что он позволяет не только подключаться к приложению и отлаживать в онлайн-режиме, но умеет еще и анализировать т.н. user dump’ы, то есть снимки внутреннего состояния приложения. Сделать эти снимки можно при помощи тулы User Mode Process Dumper, а WinDbg можно найти в пакете Debugging Tools for Windows. Итак, выполним подготовительные действия по шагам:

    1) Устанавливаем оба пакета: Dumper на сервер приложения, Tools на девелоперскую.
    2) Загружаем наше приложение так, чтобы оно начало валяться и дрыгать ногами. Необязательно добиваться OutOfMemoryException, можно просто подождать, пока приложение скушает около гигабайта памяти.
    3) Делаем дамп памяти, запустив userdump.exe из соответствующего каталога как минимум с одним параметром – идентификатором процесса ASP.NET. Узнать pid можно несколькими способами, самый простой – через колонку PID в Task Manager. Строка запуска будет выглядеть где-то так:
    userdump.exe 1234
    где 1234 – идентификатор процесса.
    Также можно дополнительно указать, куда складывать dump-файл. Если вы не укажете куда, не забудьте глянуть, что по этому поводу вам напишет userdump. Дамп занимает около минуты, в зависимости от размера процесса.
    4) Открываем WinDbg и через File -> Open Crash Dump загружаем дамп в приложение

    После всех указанных действий вы увидите картинку наподобие следующей:

    Анализируем дамп процесса

    Первым делом нам необходимо подключить к WinDbg библиотеку команд для отлаживания .NET-приложений. Называется эта библиотека sos.dll и для .NET 2.0 она входит в поставку фреймворка, поэтому лежит обычно по следующему пути:

    C:<windows_home>Microsoft.NETFrameworkv2.0.50727sos.dll.

    Для .NET 1.1 библиотеку можно найти по немного другому адресу:

    %DEBUGGING_TOOLS_HOME%clr10

    Теперь подключаем sos.dll к WinDbg при помощи команды:

    .load C:<windows_home>Microsoft.NET Frameworkv2.0.50727sos.dll

    Теперь наш дебаггер пополнился кучей дополнительных команд. Чтобы посмотреть их все, наберите в консоли команду «!help». Вот список команд версии sos.dll для .NET 2.0:

    Как видите, команд здесь довольно много. Я не буду останавливаться на всех них. Подробное описание процесса инсталляции и команд можно найти в секции Debugging School блога Johan Straarup:

    Инсталляция: How to install Windbg and get your first memory dump
    Список комманд: Getting started with windbg — part I, Getting started with windbg — part II, Using WinDbg — Advanced commands
    Решение различных проблем: Using WinDbg — Hunting Exceptions, Walkthrough — Troubleshooting a native memory leak

    Теперь, когда мы знаем минимально необходимый набор команд, я покажу приблизительный алгоритм поиска источника проблемы.

    Начнем мы с анализа памяти. Проверим, что у нас вообще творится в куче:

    0:000> !eeheap -gc
    Number of GC Heaps: 8
    ——————————
    Heap 0 (000dacf8)
    generation 0 starts at 0x33187580
    generation 1 starts at 0x33180038
    generation 2 starts at 0x02960038
    ephemeral segment allocation context: none
    segment begin allocated size
    1b63e3f0 6c10260c 6c105d38 0x0000372c(14124)
    1b5fd5b0 7b463c40 7b47a744 0x00016b04(92932)
    1b5ce420 6c267294 6c26d410 0x0000617c(24956)
    000eb0d8 7a733370 7a754b98 0x00021828(137256)
    000e8750 790d8620 790f7d8c 0x0001f76c(128876)
    02960000 02960038 048419dc 0x01ee19a4(32381348)
    33180000 33180038 332cb58c 0x0014b554(1357140)
    Large object heap starts at 0x12960038
    segment begin allocated size
    12960000 12960038 1394a698 0x00fea660(16688736)
    29fa0000 29fa0038 2a82b8c0 0x0088b888(8960136)
    Heap Size 0x3904120(59785504)
    ——————————
    Heap 1 (000dbcd0)
    …
    Heap Size 0x4195f68(68771688)
    ——————————
    Heap 2 (000dd1e8)
    …
    Heap Size 0x3d09fb4(64004020)
    ——————————
    Heap 3 (000de6d0)
    …
    Heap Size 0x38a3fbc(59391932)
    ——————————
    Heap 4 (000dfdf0)
    …
    Heap Size 0x390c360(59818848)
    ——————————
    Heap 5 (000e10f0)
    …
    Heap Size 0x36b2fc8(57356232)
    ——————————
    Heap 6 (000e2600)
    …
    Heap Size 0x3f69ef8(66494200)
    ——————————
    Heap 7 (000e3b10)
    …
    Heap Size 0x39411f0(60035568)
    ——————————
    GC Heap Size 0x1d8b2408(495657992)

    Я сознательно опустил некоторые не очень важные подробности. Главное, что мы отсюда видим – что у нас 8 куч (по количеству ядер, виртуальных, и не только) и общий размер этих куч составляет около полугигабайта, что есть немало. Значит, кто-то там живет. Теперь выведем статистику по хранящимся в куче объектам:

    0:000> !dumpheap -stat
    ——————————
    Heap 0
    total 359790 objects
    ——————————
    Heap 1
    total 325369 objects
    ——————————
    Heap 2
    total 384784 objects
    ——————————
    Heap 3
    total 424903 objects
    ——————————
    Heap 4
    total 379269 objects
    ——————————
    Heap 5
    total 346442 objects
    ——————————
    Heap 6
    total 410719 objects
    ——————————
    Heap 7
    total 367846 objects
    ——————————
    total 2999122 objects
    Statistics:
    MT Count TotalSize Class Name
    7ae7ab58 1 12 System.Drawing.ColorConverter
    7a776af8 1 12 System.Diagnostics.CorrelationManager
    7a75b4dc 1 12 System.CodeDom.Compiler.CodeDomConfigurationHandler
    7a75b1ac 1 12 System.Net.DefaultCertPolicy
    7a75a2e4 1 12 System.ComponentModel.TimeSpanConverter
    7a759dc4 1 12 System.ComponentModel.SingleConverter
    7a759c2c 1 12 System.ComponentModel.Int32Converter
    7a759b5c 1 12 System.ComponentModel.StringConverter
    7a759a70 1 12 System.ComponentModel.DoubleConverter
    7a759824 1 12 System.ComponentModel.BooleanConverter
    …
    6639d878 1 12 System.Web.Caching.Cache
    …
    663b0cdc 6 288 System.Web.SessionState.InProcSessionState
    …
    6641194c 39326 629216 System.Web.UI.StateBag
    1f49c0c4 11174 670440 System.Data.Objects.DataClasses.EntityReference`1[[Custom.Domain.Model.Identity, CUSTOM.Domain]]
    79104368 29236 701664 System.Collections.ArrayList
    1f4e6b2c 6899 814452 System.Collections.Generic.Dictionary`2+Entry[[System.String, mscorlib],[System.Data.Mapping.ObjectMemberMapping, System.Data.Entity]][]
    7a7580d0 42592 851840 System.Collections.Specialized.HybridDictionary
    79109778 16469 922264 System.Reflection.RuntimeMethodInfo
    7a75820c 33900 949200 System.Collections.Specialized.ListDictionary
    1f47b648 24872 994880 System.Collections.Generic.HashSet`1[[System.Data.Mapping.ViewGeneration.Structures.CellConstant, System.Data.Entity]]
    1ed28b24 26572 1169168 Custom.Domain.Model.ModelChangeNotification
    663c1de8 76432 1222912 System.Web.UI.StateItem
    1f47d600 24872 1273584 System.Collections.Generic.HashSet`1+Slot[[System.Data.Mapping.ViewGeneration.Structures.CellConstant, System.Data.Entity]][]
    7912d9bc 6241 1379616 System.Collections.Hashtable+bucket[]
    1f670640 59056 1417344 System.Collections.Generic.List`1[[System.Data.Objects.DataClasses.RelatedEnd, System.Data.Entity]]
    7a7582d8 75305 1506100 System.Collections.Specialized.ListDictionary+DictionaryNode
    1fb88ec4 27196 1631760 System.Data.Objects.DataClasses.EntityReference`1[[Custom.Domain.Model.ChangeSet, CUSTOM.Domain]]
    79102290 141087 1693044 System.Int32
    1f67044c 60893 1705004 System.Data.Objects.DataClasses.RelationshipManager
    1f671b90 69 1826948 System.Collections.Generic.Dictionary`2+Entry[[System.Data.EntityKey, System.Data.Entity],[System.Data.Objects.ObjectStateEntry, System.Data.Entity]][]
    1f6720fc 116747 2334940 System.Data.Objects.RelationshipWrapper
    1f497584 49230 2953800 System.Data.Objects.DataClasses.EntityReference`1[[Custom.Domain.Model.User, CUSTOM.Domain]]
    7912dae8 3769 3262224 System.Byte[]
    1f670984 66309 3448068 System.Data.Objects.EntityEntry
    1f672fec 64 3699100 System.Collections.Generic.Dictionary`2+Entry[[System.Data.Objects.RelationshipWrapper, System.Data.Entity],[System.Data.Objects.ObjectStateEntry, System.Data.Entity]][]
    1f6743a4 164482 3947568 System.Data.Objects.DataClasses.RelationshipNavigation
    1f65151c 118239 4729560 System.Data.EntityKey
    1f67223c 116740 6070480 System.Data.Objects.RelationshipEntry
    7912d7c0 55276 6703116 System.Int32[]
    7912d8f8 224375 12356408 System.Object[]
    000d9080 1518 31164860Free
    790fd8c4 318231 362482876 System.String

    Здесь мы тоже видим довольно много интересного. Во-первых, для будущих тестов нам нужны MT (что-то типа идентификатора типа объекта) объектов System.Web.Caching.Cache и System.Web.SessionState.InProcSessionState, которые соответствуют глобальному кешу приложения и его сессиям, которые наравне с другими объектами кеша хранятся в нем. Еще мы видим, что у нас очень много объектов хранится в массивах строк, но толку нам пока от этого немного. Вот если бы там были какие-то более специфические объекты, например, DataTable, мы могли бы туда сходить и посмотреть более серьезно. А так глянем сначала, сколько у нас занимается кеш в целом и сессии в частности. Для этого сначала нужно найти адреса этих объектов:

    0:000> !dumpheap -mt 6639d878
    ——————————
    Heap 0
    Address MT Size
    total 0 objects
    ——————————
    Heap 1
    Address MT Size
    total 0 objects
    ——————————
    Heap 2
    Address MT Size
    total 0 objects
    ——————————
    Heap 3
    Address MT Size
    total 0 objects
    ——————————
    Heap 4
    Address MT Size
    total 0 objects
    ——————————
    Heap 5
    Address MT Size
    total 0 objects
    ——————————
    Heap 6
    Address MT Size
    0e961294 6639d878 12
    total 1 objects
    ——————————
    Heap 7
    Address MT Size
    total 0 objects
    ——————————
    total 1 objects
    Statistics:
    MT Count TotalSize Class Name
    6639d878 1 12 System.Web.Caching.Cache
    Total 1 objects
    0:000> !objsize 0e961294
    sizeof(0e961294) = 336106272( 0x14089320) bytes (System.Web.Caching.Cache)
    0:000> !dumpheap -mt 663b0cdc
    ——————————
    Heap 0
    Address MT Size
    0355a598 663b0cdc 48
    03969aa0 663b0cdc 48
    039711e4 663b0cdc 48
    total 3 objects
    ——————————
    Heap 1
    Address MT Size
    total 0 objects
    ——————————
    Heap 2
    Address MT Size
    07f62150 663b0cdc 48
    total 1 objects
    ——————————
    Heap 3
    Address MT Size
    08f53ec8 663b0cdc 48
    total 1 objects
    ——————————
    Heap 4
    Address MT Size
    total 0 objects
    ——————————
    Heap 5
    Address MT Size
    total 0 objects
    ——————————
    Heap 6
    Address MT Size
    total 0 objects
    ——————————
    Heap 7
    Address MT Size
    10d01948 663b0cdc 48
    total 1 objects
    ——————————
    total 6 objects
    Statistics:
    MT Count TotalSize Class Name
    663b0cdc 6 288 System.Web.SessionState.InProcSessionState
    Total 6 objects

    Итак, кеш весит 336 Мб, а сессий у нас 6 штук. Берем первую попавшуюся и смотрим ее размер:

    0:000> !objsize 0355a598
    sizeof(0355a598) = 335808888( 0x14040978) bytes (System.Web.SessionState.InProcSessionState)

    Смотрим следующую – размер тот же, причем до байта. Еще одну – аналогично. Явно что-то не так в датском королевстве. Суммарный размер 6 сессий будет явно превышать размер объекта кеша, чего не может быть в принципе, так как объект Cache содержит в себе сессии. Однако, здесь есть особенность, которая может объяснить эту нестыковку – дело в том, что sos при подсчете размера объекта включает в него и размер всех объектов, на которые данный объект ссылается. Значит, у нас где-то сессии друг на друга ссылаются. Вот только где и как? Кроме того, мы можем сделать еще один важный вывод: чистая информация, которую мы вносили в кеш руками, занимает очень мало памяти, что очень хорошо. Если бы разница была больше, то нужно было бы идти анализировать еще и закешированные данные.

    Посмотрим, что у нас хранится в сессии:

    0:000> !do 10d01948
    Name: System.Web.SessionState.InProcSessionState
    MethodTable: 663b0cdc
    EEClass: 663b0c6c
    Size: 48(0x30) bytes
    (C:WINDOWSassemblyGAC_32System.Web2.0.0.0__b03f5f7f11d50a3aSystem.Web.dll)
    Fields:
    MT Field Offset Type VT Attr Value Name
    663ea4e4 4001edc 4 …ateItemCollection 0 instance 0704f73c _sessionItems
    663a9944 4001edd 8 …ObjectsCollection0 instance 00000000 _staticObjects
    79102290 4001ede c System.Int32 1 instance 25 _timeout
    7910be50 4001edf 18 System.Boolean 1 instance 0 _locked
    7910c878 4001ee0 1c System.DateTime 1 instance 10d01964 _utcLockDate
    79102290 4001ee1 10 System.Int32 1 instance 36 _lockCookie
    663a7220 4001ee2 24 …ReadWriteSpinLock 1 instance 10d0196c _spinLock
    79102290 4001ee3 14 System.Int32 1 instance 0 _flags
    0:000> !do 0704f73c
    Name: System.Web.SessionState.SessionStateItemCollection
    MethodTable: 664107dc
    EEClass: 66410764
    Size: 60(0x3c) bytes
    (C:WINDOWSassemblyGAC_32System.Web2.0.0.0__b03f5f7f11d50a3aSystem.Web.dll)
    Fields:
    MT Field Offset Type VT Attr Value Name
    7910be50 4001172 24 System.Boolean 1 instance 0 _readOnly
    79104368 4001173 4 …ections.ArrayList 0 instance 0704f784 _entriesArray
    79116ef8 4001174 8 …IEqualityComparer0 instance 02961a28 _keyComparer
    79101fe4 4001175 c …ections.Hashtable 0 instance 0704f79c _entriesTable
    7a75a878 4001176 10 …e+NameObjectEntry 0 instance 00000000 _nullKeyEntry
    7a772bc4 4001177 14 …se+KeysCollection 0 instance 00000000 _keys
    79111df0 4001178 18 …SerializationInfo 0 instance 00000000 _serializationInfo
    79102290 4001179 20 System.Int32 1 instance 6 _version
    790fd0f0 400117a 1c System.Object 0 instance 00000000 _syncRoot
    791168f8 400117b 538 …em.StringComparer 0 shared static defaultComparer
    >> Domain:Value 000c8390:NotInit … <<
    7910be50 4001f2b 25 System.Boolean 1 instance 1 _dirty
    66421eb8 4001f2c 28 …n+KeyedCollection 0 instance 00000000 _serializedItems
    79101924 4001f2d 2c System.IO.Stream 0 instance 00000000 _stream
    79102290 4001f2e 34 System.Int32 1 instance 0 _iLastOffset
    790fd0f0 4001f2f 30 System.Object 0 instance 0704f778 _serializedItemsLock
    79101fe4 4001f2a c5c …ections.Hashtable 0 shared static s_immutableTypes
    >> Domain:Value 000c8390:NotInit ..    . <<
    0:000> !do 0704f784
    Name: System.Collections.ArrayList
    MethodTable: 79104368
    EEClass: 791042bc
    Size: 24(0x18) bytes
    (C:WINDOWSassemblyGAC_32mscorlib2.0.0.0__b77a5c561934e089mscorlib.dll)
    Fields:
    MT Field Offset Type VT Attr Value Name
    7912d8f8 40008ea 4 System.Object[] 0 instance 10d0189c _items
    79102290 40008eb c System.Int32 1 instance 2 _size
    79102290 40008ec 10 System.Int32 1 instance 2 _version
    790fd0f0 40008ed 8 System.Object 0 instance 00000000 _syncRoot
    7912d8f8 40008ee 1c0 System.Object[] 0shared static emptyArray
    >> Domain:Value 000c8390:08961f48 …<<
    0:000> !da 10d0189c
    Name: System.Object[]
    MethodTable: 7912d8f8
    EEClass: 7912de6c
    Size: 32(0x20) bytes
    Array: Rank 1, Number of elements 4, Type CLASS
    Element Methodtable: 790fd0f0
    [0] 10d0188c
    [1] 034f5ccc
    [2] null
    [3] null
    0:000> !do 10d0188c
    Name: System.Collections.Specialized.NameObjectCollectionBase+NameObjectEntry
    MethodTable: 7a75a878
    EEClass: 7a7c51c4
    Size: 16(0x10) bytes
    (C:WINDOWSassemblyGAC_MSILSystem2.0.0.0__b77a5c561934e089System.dll)
    Fields:
    MT Field Offset Type VT Attr Value Name
    790fd8c4 400117c 4 System.String 0 instance 0698fdd8 Key
    790fd0f0 400117d 8 System.Object 0 instance 030cc5d8 Value
    0:000> !do 030cc5d8
    Name: Custom.Namespace.SomeClass
    MethodTable: 1b109548
    EEClass: 1d351918
    Size: 116(0x74) bytes
    (C:WINDOWSMicrosoft.NETFrameworkv2.0.50727Temporary ASP.NET FilessomeApp-3.7.23.30196191ae903c3217assemblydl35f395f4004dd5b1_3ffec801Custom.UI.Web.DLL)
    Fields:
    MT Field Offset Type VT Attr Value Name
    1b10e18c 400006e 4 …del.DomainContext 0 instance 030cc6f0 model
    1b10d484 400006f 8 …OrganizationScope0 instance 0ec75578 scopeModel
    1fe48b6c 4000070 c …el.TotalViewModel 0 instance 00000000 totalViewModel
    1b3dac14 4000071 10 …odel.Organization 0 instance 08f552bc organizationModel
    1fb80314 4000072 14 …on.Model.Variable 0 instance 00000000 variableModel
    1fbe7afc 4000073 18 …Model.Requirement 0 instance 00000000 requirementModel
    1b3dbcc4 4000074 1c …n.Model.Objective 0 instance 00000000 objectiveModel
    1fbe938c 4000075 20 …uirementAttribute 0 instance 0f3843ac requirementAttributeModel
    7910ff9c 4000076 24 System.Void 0 instance 00000000 requirementAttributeClassModel
    1b3dba6c 4000077 28 …ion.Model.Project 0 instance 00000000 projectModel
    1b3db714 4000078 2c …cation.Model.Task 0 instance 00000000 taskModel
    7910ff9c 4000079 30 System.Void 0 instance 00000000 profileModel
    1b3db48c 400007a 34 …on.Model.Resource 0 instance 00000000 resourceModel
    7910ff9c 400007b 38 System.Void 0 instance 00000000 commentModel
    1ed25b14 400007c 3c …on.Model.Identity 0 instance 00000000 identityModel
    30f67d1c 400007d 40 ….Model.DataModule 0 instance 00000000 dataModuleModel
    1ed25e14 400007e 44 …cation.Model.User 0 instance 0ec74f7c userModel
    1fb81614 400007f 48 …cation.Model.Role 0 instance 0ec75068 roleModel
    1fb8146c 4000080 4c …Web.Model.CCFUser 0 instance 0ec74ffc currentUser
    1b3db174 4000081 50 …tion.Model.Folder 0 instance 0f36b6d4 folderModel
    1fc814c4 4000082 54 …on.Model.Document 0 instance 0d0d93cc documentModel
    1b10ed0c 4000083 58 …on.Model.Workflow 0 instance 030cc66c workflowModel
    1fc83ebc 4000084 5c …odel.Notification 0 instance 031958ec notificationModel
    7910ff9c 4000085 60 System.Void 0 instance 00000000 reportModel
    1b10f6fc 4000086 64 …n.Model.ChangeSet 0 instance 050cc0a0 changeSetModel
    7910ff9c 4000087 68 System.Void 0 instance 00000000 scheduledTasksManagerModel
    1b3dc034 4000088 6c …esWorkflowMonitor 0 instance 00000000 changesWorkflowMonitor
    1fb8c638 4000089 34 …hing.RequestCache 0 static 109e1158 requestCache
    1fe44870 400008a 38 ….ApplicationCache 0 static 0aee10c4 applicationCache
    1ed24714 400008b 3c …g.StatisticsCache 0 static 0c9a4e7c statisticsCache

    Как мы видим, в сессии хранится лишь некий экземпляр класса Custom.Namespace.SomeClass, который содержит перечисленный набор полей. В принципе, это известно давно тем, кто приложение программировал 🙂 Колонка Value показывает значения ссылок, нолики, как несложно догадаться – это null. Теперь посмотрим размеры объекта model:

    0:000> !objsize 030cc6f0
    sizeof(030cc6f0) = 335808160( 0x140406a0) bytes (Custom.Namespace.SomeAnotherClass)
    0:000> !do 030cc6f0
    Name: Custom.Namespace.SomeAnotherClass
    MethodTable: 1b10e18c
    EEClass: 1d35c7d8
    Size: 48(0x30) bytes
    (C:WINDOWSMicrosoft.NETFrameworkv2.0.50727Temporary ASP.NET FilessomeApp-3.7.23.30196191ae903c3217assemblydl3a41ba0b020a4b0_3ffec801Custom.Application.DLL)
    Fields:
    MT Field Offset Type VT Attr Value Name
    1dad2c38 4000119 4 …Views.DomainLogic 0 instance 050cbf34 context
    7910be50 400011a 28 System.Boolean 1 instance 0 isChanged
    00000000 400011b 8 0 instance 030cc720 workflowModelChanges
    00000000 400011c c 0 instance 030cc738 notificationsChanges
    1fb804fc 400011d 10 …olledBackDelegate 0 instance 0f3843e4 ContextRolledBack
    1fb805ec 400011e 14 …RefreshedDelegate 0 instance 0f384424 ContextRefreshed
    1b10f4d4 400011f 18 …extSavingDelegate 0 instance 050cc040 ContextSaving
    1b10f8e4 4000120 1c …textSavedDelegate 0 instance 0d0d9cbc ContextSaved
    1fb806dc 4000121 20 …aveFailedDelegate 0 instance 0f384464 ContextSaveFailed
    1b10f5c4 4000122 24 …CancelledDelegate 0 instance 050cc060 ContextSaveCancelled
    7910be50 4000123 29 System.Boolean 1 instance 0k__BackingField
    0:000> !objsize 050cbf34
    sizeof(050cbf34) = 63643700 ( 0x3cb2034) bytes (Custom.Namespace.OneMoreAnotherClass)

    Итак, главный объект SomeAnotherClass весит около 335 Мб, в то время как вложенный в него OneMoreAnotherClass – всего 63 Мб. Возможно, другие объекты содержат остальные данные? Проверяем – нет, не так. Хотя в нашем случае даже проверять не нужно было – мы и так знаем свой код 🙂 Проверяем другие сессии аналогичным образом – там та же картина, что вполне ожидаемо. Итак, к чему мы пришли? Наш Custom.Namespace.SomeClass и вложенный в него Custom.Namespace.SomeAnotherClass содержится в каждой сессии, а то, что objsize показывает для него размер всех сессий, указывает, что на этом уровне у нас возможны проблемы.

    А теперь посмотрим, кто на кого ссылается:

    0:000> !gcroot 030cc6f0
    Note: Roots found on stacks may be false positives. Run «!help gcroot» for more info.
    Scan Thread 17 OSTHread c64
    Scan Thread 27 OSTHread 1c24
    Scan Thread 28 OSTHread 824
    Scan Thread 29 OSTHread 1d68
    Scan Thread 15 OSTHread 1ee4
    Scan Thread 31 OSTHread ec4
    Scan Thread 32 OSTHread 1ea4
    Scan Thread 33 OSTHread 1e9c
    Scan Thread 34 OSTHread 3ec
    Scan Thread 35 OSTHread 1e10
    Scan Thread 36 OSTHread 6ec
    Scan Thread 37 OSTHread c90
    Scan Thread 38 OSTHread b6c
    ESP:1f17ecbc:Root:069b83b8(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.Storage.ConfigurationChangeFileWatcher)->
    069b8514(System.Threading.Thread)->
    08963fe0(System.Runtime.Remoting.Contexts.Context)->
    08963e40(System.AppDomain)->
    0a935d24(System.ResolveEventHandler)->
    0a0e9918(System.Object[])->
    0e9a9c34(System.ResolveEventHandler)->
    0e9a9250(System.Web.Compilation.BuildManager)->
    0e9a9e38(System.Web.Compilation.MemoryBuildResultCache)->
    0e96137c(System.Web.Caching.CacheMultiple)->
    0e961394(System.Object[])->
    0e9679dc(System.Web.Caching.CacheSingle)->
    0e967aa0(System.Web.Caching.CacheExpires)->
    0e967ac0(System.Object[])->
    0e9685d4(System.Web.Caching.ExpiresBucket)->
    118ede14(System.Web.Caching.ExpiresPage[])->
    118ede98(System.Web.Caching.ExpiresEntry[])->
    03971214(System.Web.Caching.CacheEntry)->
    039711e4(System.Web.SessionState.InProcSessionState)->
    0b5fa220(System.Web.SessionState.SessionStateItemCollection)->
    0b5fa280(System.Collections.Hashtable)->
    0b5fa2b8(System.Collections.Hashtable+bucket[])->
    03971170(System.Collections.Specialized.NameObjectCollectionBase+NameObjectEntry)->
    0d7b54d4(Custom.Namespace.SomeClass)->
    0d7b5568(Custom.Application.Model.Workflow)->
    0d7b55ac(Custom.Application.Model.Workflow+WorkflowRemindDelegate)->
    0698fec8(Custom.ApproveActionWorkflow.WorkflowManager)->
    11bdadd4(Custom.ApproveActionWorkflow.WorkflowManager+WorkflowNotificationsSentDelegate)->
    03971088(System.Object[])->
    050cc020(Custom.ApproveActionWorkflow.WorkflowManager+WorkflowNotificationsSentDelegate)->
    030cc6f0(Custom.Namespace.SomeAnotherClass)
    DOMAIN(001020B0):HANDLE(Strong):29310e0:Root:0e968b38(System.Threading._TimerCallback)->
    0e968af0(System.Threading.TimerCallback)->
    0e964010(System.Web.Caching.CacheExpires)->
    0e964030(System.Object[])->
    0e9646f8(System.Web.Caching.ExpiresBucket)->
    0517489c(System.Web.Caching.ExpiresPage[])->
    05174920(System.Web.Caching.ExpiresEntry[])->
    10d01978(System.Web.Caching.CacheEntry)->
    10d01948(System.Web.SessionState.InProcSessionState)->
    0704f73c(System.Web.SessionState.SessionStateItemCollection)->
    0704f79c(System.Collections.Hashtable)->
    0704f7d4(System.Collections.Hashtable+bucket[])->
    10d0188c(System.Collections.Specialized.NameObjectCollectionBase+NameObjectEntry)->
    030cc5d8(Custom.Namespace.SomeClass)->
    030cc6f0(Custom.Namespace.SomeAnotherClass)

    А вот и корень зла. Если посмотреть этот trace, то можно увидеть, что поначалу у нас все замечательно, кеш, сессия, а потом начинаются странности. Откуда-то появляются ссылки через делегаты, а класс SomeAnotherClass вообще фигурирует дважды, причем по разным адресам. Однако подсказка тоже есть – Custom.ApproveActionWorkflow.WorkflowManager+WorkflowNotificationsSentDelegate. Немного подумав, вспоминаем, что у нас есть замечательный синглтон-класс WorkflowManager, который предназначен для взаимодействия между пользовательскими потоками и потоком WorkflowRuntime, и оповещает все сессии при помощи генерации события о том, что им пора бы обновить некоторые специфические данные из базы. И он, конечно же, подписывает все создаваемые сессии на свое событие. А потом держит эту ссылку бесконечно, таким образом не давая сборщику мусора собрать сессии. Ну все, разобрались. Теперь дело за малым: зарефакторить код, чтобы сессии сами опрашивали класс об изменениях и избавиться от OutOfMemoryException 🙂 Дело сделано.

    Выводы

    Выводы из данной ситуации очень простые:

    1) Делегаты, как оказывается – звери совсем не безобидные, за ними тоже нужен глаз да глаз
    2) Старайтесь внимательно реализовывать межсессионное взаимодействие, чтобы не допускать не только ссылок на объекты, но и на функции
    3) Не забывайте про особенности работы Task Manager и то, что он реально показывает. Не нужно на него сильно полагаться
    4) Если же у вас все-таки вылетает OutOfMemoryException, не отчаивайтесь. Вооружайтесь дампом и WinDbg – и вперед. Люди и не такие баги отлаживают 🙂
    5) WinDbg подойдет и для отладки других сложных ситуаций. Вот, например, рассказ Юры Скалецкого и некоего JKealey. Там есть что почитать.

    Ну, и напоследок я настоятельно вам советую следующие статьи из блога Tess и Johan’а по поводу отладки утечек памяти:

    Debugging Script: Dumping out ASP.NET Session Contents
    ASP.NET Memory Leak Case Study: Sessions Sessions Sessions…
    ASP.NET Memory — How much are you caching? + an example of .foreach
    ASP.NET Case Study: Tracing your way to Out Of Memory Exceptions
    Why adding more memory won’t resolve OutOfMemoryExceptions
    I am getting OutOfMemoryExceptions. How can I troubleshoot this?
    Are you getting OutOfMemoryExceptions when uploading large files?

    Да и вообще, эти блоге стоит добавить в ваш персональный RSS feeder. Можно вместе с блогом Марка Руссиновича (ага, это тот, который написал кучу полезных системных тулов, известных под названием Sysinternals). Пригодятся.

    Успехов в отладке!

    Источник

    IIS8

    This post was most recently updated on October 4th, 2022.

    2 min read.

    This post describes one of the more no-brainerish ways of fixing a ‘System.OutOfMemoryException’ exception being thrown in your ASP.Net MVC application using C# and Entity Framework.

    Problem

    While developing a web project, for example, an ASP.NET MVC web application that is using EF, sometimes when handling a lot of data or complex entities on your dev machine, you encounter this error:

    OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.]
   System.Text.StringBuilder.ToString() +35
   System.IO.StreamReader.ReadToEnd() +123
   System.Web.Optimization.BundleFile.ApplyTransforms() +74
   System.Web.Optimization.DefaultBundleBuilder.BuildBundleContent(Bundle bundle, BundleContext context, IEnumerable`1 files) +472
   System.Web.Optimization.Bundle.GenerateBundleResponse(BundleContext context) +127
   System.Web.Optimization.Bundle.GetBundleResponse(BundleContext context) +45
   System.Web.Optimization.BundleResolver.GetBundleContents(String virtualPath) +166
   System.Web.Optimization.AssetManager.DeterminePathsToRender(IEnumerable`1 assets) +205
   System.Web.Optimization.AssetManager.RenderExplicit(String tagFormat, String[] paths) +35
   System.Web.Optimization.Scripts.RenderFormat(String tagFormat, String[] paths) +107
   System.Web.Optimization.Scripts.Render(String[] paths) +21

    This of course blocks you from debugging or even running at least this part of your code locally. What to do?

    Reason

    By default, Visual Studio uses a 32-bit version of IIS Express for your deployments. A lot of times, this is good and intended, and not a problem. In some rare cases, this might mean, however, that your IIS processes are running out of memory.

    I’m going to argue, that most of the time you shouldn’t end up having this issue if your code is sane and smart. Most of the time, if you get this error, you’re building infinite loops or forcing the Entity Framework to load millions and millions of rows in memory. Fix that first.

    However, with EF it’s somewhat easy to build apps that use a lot of memory, but work as they should. Perhaps your architecture does make sense, and using a lot of memory is ok?

    In this case, you might want to accommodate this memory requirement by enabling a 64-bit version of IIS locally. That’s probably what you’re using in production anyway.

    This is luckily a pretty easy change.

    How to solve System.OutOfMemoryException in Visual Studio?

    The first thing is to make sure you’re not hogging all the available memory for no reason – no neverending loops or infinite recursion! After that, consider the following.

    Time needed: 5 minutes.

    How to enable 64-bit IIS Express from Visual Studio settings

    1. Navigate to the following settings page:

      Visual Studio > Tools > Options > Projects and Solutions > Web Projects

    2. Enable the following setting

      “Use the 64-bit version of IIS Express for websites and projects” (see screenshot 1 below)
      How to enable the 64-bit version of IIS Express in Visual Studio 2017
      Screenshot 1: How to enable the 64-bit version of IIS Express in Visual Studio 2017

    3. Also, don’t forget to verify you’re building a 64-bit version of your app!

      See screenshot 2 below:
      64-bit build option for the solution in Visual Studio
      Screenshot 2: 64-bit build option for the solution in Visual Studio

    And you should be good to go!

    • Author
    • Recent Posts

    mm

    Antti Koskela is a proud digital native nomadic millennial full stack developer (is that enough funny buzzwords? That’s definitely enough funny buzzwords!), who works as Solutions Architect for Precio Fishbone, building delightful Digital Workplaces.

    He’s been a developer from 2004 (starting with PHP and Java), and he’s been working on .NET projects, Azure, Office 365, SharePoint and a lot of other stuff. He’s also Microsoft MVP for Office Development.

    This is his personal professional (e.g. professional, but definitely personal) blog.

    mm

    Источник

    0 0 голоса
    Рейтинг статьи
    Подписаться
    Уведомить о
    guest

    0 комментариев
    Старые
    Новые Популярные
    Межтекстовые Отзывы
    Посмотреть все комментарии

    А вот еще интересные материалы:

  • Яшка сломя голову остановился исправьте ошибки
  • Ясность цели позволяет целеустремленно добиваться намеченного исправьте ошибки
  • Ясность цели позволяет целеустремленно добиваться намеченного где ошибка
  • Ошибка system io filenotfoundexception
  • Ошибка system head no handled

    • ➤