HOWTO: Force GC to reduce reserved memory
Jochen Kalmbach
is it possible to force the Garbage Collector to reduce the
"# of Total reserved bytes" to almost
"# of Total commited bytes" ???
And also to free this memory, so the "# of Private bytes"
will decrise ?
Greetings
Jochen
NETMaster
Windows 2000/XP does reduce the working set.
(watch in task manager)
I don't know any methods for GC specific...
But the Process class has
.MinWorkingSet;
.MaxWorkingSet;
properties, wrapping the Win32::SetProcessWorkingSetSize API
with a limited functionality.
I didn't manage to set Max < effective Min, nor -1 ('swap process')
Note, all this 'tuning' should be done (if at all) only
while your app is idle (background).
I did some experimental use:
class MemoryTune
{
// http://msdn.microsoft.com/library/en-us/dllproc/prothred_1w6s.asp
// http://msdn.microsoft.com/library/en-us/dllproc/prothred_5dnp.asp
[DllImport("kernel32.dll")]
public static extern bool GetProcessWorkingSetSize( IntPtr proc, out int min, out int max );
// http://msdn.microsoft.com/library/en-us/dllproc/prothred_07l1.asp
[DllImport("kernel32.dll")]
public static extern bool SetProcessWorkingSetSize( IntPtr proc, int min, int max );
[STAThread]
static void Main(string[] args)
{
Process p = Process.GetCurrentProcess();
// ------------------------------ Process class
int min = (int) p.MinWorkingSet;
int max = (int) p.MaxWorkingSet;
Console.WriteLine( "Min = {0}, Max = {1}", min, max );
Console.WriteLine( "hit <enter> to set using Process.MXXWorkingSet" );
Console.ReadLine();
// http://msdn.microsoft.com/library/en-us/cpref/html/frlrfSystemDiagnosticsProcessClassMaxWorkingSetTopic.asp
// p.MinWorkingSet = (IntPtr) unchecked( (int) -1 );
p.MaxWorkingSet = (IntPtr) (min + 100000);
Console.WriteLine( "hit <enter> to restore" );
Console.ReadLine();
// p.MinWorkingSet = min;
p.MaxWorkingSet = (IntPtr) max;
// ------------------------------ PInvoke:
int smin = 0, smax = 0;
bool f = GetProcessWorkingSetSize( p.Handle, out smin, out smax );
Console.WriteLine( "Min = {0}, Max = {1}", smin, smax );
Console.WriteLine( "hit <enter> to set using SetProcessWorkingSetSize" );
Console.ReadLine();
f = SetProcessWorkingSetSize( p.Handle, -1, -1 );
Console.WriteLine( "hit <enter> to restore" );
Console.ReadLine();
f = SetProcessWorkingSetSize( p.Handle, smin, smax );
// ------------------------------ done:
Console.WriteLine( "hit <enter> to quit" );
Console.ReadLine();
Console.WriteLine( "end." );
}
NETMaster
http://www.cetus-links.org/oo_csharp.html
"Jochen Kalmbach" <nospam-Joch...@holzma.de> wrote in message news:Xns91E56E78983B2Jo...@127.0.0.1...
Jochen Kalmbach
news:#QB7Apv2BHA.2840@tkmsftngp05:
> First note, if you minimize any app window,
> Windows 2000/XP does reduce the working set.
> (watch in task manager)
Thanks for this answer, but I know that I can reduce the working set...
But this has nothing to do with the "# of Private Bytes (also called
"PageFile usage")"...
I have a situation (simpel C# program) with the following values:
GC commited bytes: about 0,7 MByte
GC reserved bytes: about 35,0 MByte
Page file usage" : about 42,0 MByte
The question is:
How can I reduce the "GC reserved bytes"
(and also the "Page file usage") ?????
Is this possible or not ????
Any help is welcome...
Greetings
Jochen
NETMaster
where do you read this numbers?
PerfMon? TaskMan? PView? ...?
C# app console / winforms ?
"Jochen Kalmbach" <nospam-Joch...@holzma.de> wrote in message news:Xns91E585B52CC64Jo...@127.0.0.1...
> Thanks for this answer, but I know that I can reduce the working set...
> But this has nothing to do with the "# of Private Bytes (also called
> "PageFile usage")"...
Jochen Kalmbach
news:eByEVew2BHA.1872@tkmsftngp03:
>> GC commited bytes: about 0,7 MByte
>> GC reserved bytes: about 35,0 MByte
>> Page file usage" : about 42,0 MByte
>
> This seems quite huge...
>
> where do you read this numbers?
> PerfMon? TaskMan? PView? ...?
Page file usage: Either with Perfmon (Process, Private bytes) or with
System.Diagnostics.Process.GetCurrentProcess().PagedMemorySize
GC commited / reserved bytes: Perfmon:
.NET CLR memory | # Total commited Bytes
.NET CLR memory | # Total reserved Bytes
> C# app console / winforms ?
Simple C# app console...
--
Greetings
Jochen
Here is the code:
using System;
using System.Collections;
using System.Runtime.InteropServices;
namespace CS_Perf3_Test
{
class TestObject
{
public TestObject(int iValue)
{
Value = iValue;
}
public int Value;
}
class Class1
{
DateTime m_StartTime;
DateTime m_EndTime;
void StartTime()
{
m_StartTime = DateTime.Now;
}
void EndTime(string Comment)
{
m_EndTime = DateTime.Now;
TimeSpan diff = m_EndTime - m_StartTime;
System.Diagnostics.Process p =
System.Diagnostics.Process.GetCurrentProcess();
Console.WriteLine("{0}: Diff: {1} ms, Memory: {2}",
Comment, diff.TotalMilliseconds, p.PagedMemorySize);
}
void Test(int ObjCount, int IterCount, int SleepTimeBetween)
{
for(int k=0; k<IterCount; k++)
{
if (SleepTimeBetween > 0)
System.Threading.Thread.Sleep
(SleepTimeBetween);
StartTime();
ArrayList ar = new ArrayList(ObjCount);
for(int i = 0; i< ObjCount; i++)
{
ar.Add(new TestObject(i));
if ((i % 100000) == 0)
{
System.Diagnostics.Process p =
System.Diagnostics.Process.GetCurrentProcess();
Console.WriteLine(i.ToString() + ",
" + p.PagedMemorySize.ToString());
}
}
ar.Clear();
GC.Collect();
GC.WaitForPendingFinalizers();
EndTime("");
}
}
static void Main(string[] args)
{
System.Threading.Thread.CurrentThread.Priority =
System.Threading.ThreadPriority.Highest;
Class1 c = new Class1();
c.Test(100, 50, 700);
c.Test(7000000, 1, 0);
c.Test(100, 50, 700);
Console.ReadLine();
}
}
}
Willy Denoyette [MVP]
The # Committed bytes for the GC are accounted for Pagefile usage, and they are part of the WS, so they can be returned to the OS
when the WS trimmer runs.
Question, what did you do in your "simple program" to get at such a size for the reserved bytes?
Willy.
"Jochen Kalmbach" <nospam-Joch...@holzma.de> wrote in message news:Xns91E585B52CC64Jo...@127.0.0.1...
Jochen Kalmbach
news:uYNDgww2BHA.2084@tkmsftngp04:
> You can't reduce the # reserved bytes for the GC.
Where did you have this information from ????
> The # Committed bytes for the GC are accounted for Pagefile usage, and
> they are part of the WS, so they can be returned to the OS when the
> WS trimmer runs.
No, if you trim the WorkingSet, no memory is returned to the OS. It is only
swaped out to the paging file...
See: SetProcessWorkingSetSize(GetCurrentProcess(), -1, -1);
The "Pagefile usage" is not affected by this WS trimming...
> Question, what did you do in your "simple program" to get at such a
> size for the reserved bytes?
I just posted the code in this thread...
Greetings
Jochen
NETMaster
7'000'000
objects? what do you expect?
"Jochen Kalmbach" <nospam-Joch...@holzma.de> wrote in message news:Xns91E59230FEF6Jo...@127.0.0.1...
> "NETMaster" <spam.ne...@swissonline.ch> wrote in
> news:eByEVew2BHA.1872@tkmsftngp03:
>
> >> GC commited bytes: about 0,7 MByte
> >> GC reserved bytes: about 35,0 MByte
> >> Page file usage" : about 42,0 MByte
> >
> > This seems quite huge...
> >
> > where do you read this numbers?
> > PerfMon? TaskMan? PView? ...?
>
> Page file usage: Either with Perfmon (Process, Private bytes) or with
> System.Diagnostics.Process.GetCurrentProcess().PagedMemorySize
>
> GC commited / reserved bytes: Perfmon:
> .NET CLR memory | # Total commited Bytes
> .NET CLR memory | # Total reserved Bytes
> c.Test(100, 50, 700);
Jochen Kalmbach
news:#5VR31w2BHA.2284@tkmsftngp02:
> didn't check all code, but with
> 7'000'000
> objects? what do you expect?
I expect that the memory should decrese after all objects are freed
(and of course also collected...)
Why does the GC not frees the memory to the OS ???
Greetings
Jochen
Jochen Kalmbach
news:ubnd9Fx2BHA.2744@tkmsftngp04:
> If I watch the 'private bytes' graph in PerfMon, I see
> the memory rises while adding the 7'000'000 objects
> from ~5 Meg to ~60 Meg,
> then falls with GC.Collect/WaitForPendingFinalizers back
> to around ~30 Meg.
>
> so the GC does free memory!
> Why not all?
> - may be heap fragmentation?
> - optimization strategy, assuming memory consumption will rise again
> later?
>
>
> may be Jeffrey Richter writes about this topic:
> http://msdn.microsoft.com/msdnmag/issues/1100/GCI/GCI.asp
> http://msdn.microsoft.com/msdnmag/issues/1200/GCI2/GCI2.asp
>
> or Emmanuel Schanzer
> http://msdn.microsoft.com/library/en-us/dndotnet/html/dotnetperftechs
> .asp
>
Thanks for the links...
I read the articles already...
It would be nice if it were clear, when the GC would release the reserved
memory...
Even if I start other applications which consuming many memory, so the
page-file is heavily in action, the GC of this process keeps his
memory... This is bad...
Greetings
Jochen
NETMaster
the memory rises while adding the 7'000'000 objects
from ~5 Meg to ~60 Meg,
then falls with GC.Collect/WaitForPendingFinalizers back
to around ~30 Meg.
so the GC does free memory!
Why not all?
- may be heap fragmentation?
- optimization strategy, assuming memory consumption will rise again later?
may be Jeffrey Richter writes about this topic:
http://msdn.microsoft.com/msdnmag/issues/1100/GCI/GCI.asp
http://msdn.microsoft.com/msdnmag/issues/1200/GCI2/GCI2.asp
or Emmanuel Schanzer
http://msdn.microsoft.com/library/en-us/dndotnet/html/dotnetperftechs.asp
"Jochen Kalmbach" <nospam-Joch...@holzma.de> wrote in message news:Xns91E5982EEA926Jo...@127.0.0.1...
Willy Denoyette [MVP]
Willy.
"Jochen Kalmbach" <nospam-Joch...@holzma.de> wrote in message news:Xns91E5962C6E6D7Jo...@127.0.0.1...
> "Willy Denoyette [MVP]" <willy.d...@pandora.be> wrote in
> news:uYNDgww2BHA.2084@tkmsftngp04:
>
> > You can't reduce the # reserved bytes for the GC.
>
> Where did you have this information from ????
>
**** I mean, only the CLR can reduce this, by returning the memory allocated for the managed heaps to the OS.
>
>
> > The # Committed bytes for the GC are accounted for Pagefile usage, and
> > they are part of the WS, so they can be returned to the OS when the
> > WS trimmer runs.
>
> No, if you trim the WorkingSet, no memory is returned to the OS. It is only
> swaped out to the paging file...
> See: SetProcessWorkingSetSize(GetCurrentProcess(), -1, -1);
> The "Pagefile usage" is not affected by this WS trimming...
>
**** This is not what I see.
During the second c.Test call (7000000 objects) the WS climbs up to ~121MB and the Pagefile bytes to ~125 MB.
When this one finishes, the WS gets trimmend and drops to ~41MB and the pagefile bytes to ~45 MB.
>
> > Question, what did you do in your "simple program" to get at such a
> > size for the reserved bytes?
>
> I just posted the code in this thread...
>
**** Ok, here you hit a known bug in V1.
In your sample you allocate 7000000 objects, each object takes 12 bytes that makes 94 MB in total spread over GC heaps G0, G1 and
G2.
But you also store the reference in an Arraylist (one single object), that list holds a pointer per object so takes 4 * 7Mo = 28MB,
these are reserved from the Large object heap, and a bug in V1 prevents this to schrink.
To test this just run the second call 3times successively, and watch the GC Large Heap , process private bytes and process pagefile
bytes counters.
You will see the Large Heap increase and drop for each call, but the private bytes and obviously page file bytes increase without
returning.
When reducing the number of objects to lets say 1000000 everything returns to normal.
>
> Greetings
> Jochen
Jochen Kalmbach
> **** I mean, only the CLR can reduce this, by returning the memory
> allocated for the managed heaps to the OS.
I known that only the CLR can return this memory to the OS. May question
is: Is there any method to tell the GC to free the memory... ???
>> See: SetProcessWorkingSetSize(GetCurrentProcess(), -1, -1);
>> The "Pagefile usage" is not affected by this WS trimming...
>>
> **** This is not what I see.
> During the second c.Test call (7000000 objects) the WS climbs up to
> ~121MB and the Pagefile bytes to ~125 MB. When this one finishes, the
> WS gets trimmend and drops to ~41MB and the pagefile bytes to ~45 MB.
No, the working set gets reduced because of the freeing of "pagefile
bytes" from the process, not because there is someobe doing a WS
trimming!
The working set can be reduced to 0 bytes, but this does not mean that
the process does not need any memory...
> **** Ok, here you hit a known bug in V1.
Is this a confirmed bug ? Where is the KB-Article ???
Greetings
Jochen
leo lin[MS]
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnmag00/htm
l/GCI2.asp
You can use GC.collect two time to manaul release unreference managed
resource.
leolin
Willy Denoyette [MVP]
Willy.
"Jochen Kalmbach" <nospam-Joch...@holzma.de> wrote in message news:Xns91E64BB678201Jo...@127.0.0.1...
> "Willy Denoyette [MVP]" <willy.d...@pandora.be> wrote in
> news:O2Y8vU12BHA.2332@tkmsftngp03:
> >
> I known that only the CLR can return this memory to the OS. May question
> is: Is there any method to tell the GC to free the memory... ???
>
As far as I know, no.
>
>
> >> See: SetProcessWorkingSetSize(GetCurrentProcess(), -1, -1);
> >> The "Pagefile usage" is not affected by this WS trimming...
> >>
> > **** This is not what I see.
> > During the second c.Test call (7000000 objects) the WS climbs up to
> > ~121MB and the Pagefile bytes to ~125 MB. When this one finishes, the
> > WS gets trimmend and drops to ~41MB and the pagefile bytes to ~45 MB.
>
> No, the working set gets reduced because of the freeing of "pagefile
> bytes" from the process, not because there is someobe doing a WS
> trimming!
>
****
Bad wording, the WS and the Pagefile size drops because the part of the heap allocated for the Large heap (consisting of several
chunks) is given back to the OS.
> The working set can be reduced to 0 bytes, but this does not mean that
> the process does not need any memory...
>
That's correct.
>
>
> > **** Ok, here you hit a known bug in V1.
>
> Is this a confirmed bug ? Where is the KB-Article ???
****
No KB article as far as I know, but it is confirmed
<quote from: Serge Lidin <sli...@MICROSOFT.COM>
and Brian Harry <bha...@MICROSOFT.COM>
(Brian owns the CLR)
see: DevelopMentor lists at http://discuss.develop.com.
Subject : Whoa - GC incorrectly handles large freed objects
Sent: Friday, February 02 >
This is a known issue. Here is the reply I've got from Patrick Dussud who is our GC guru:
"We know of an issue with the large object heap (gmheap). It cannot fully reclaim the space allocated to large arrays (I don't
remember the threshold 8MB or 16MB) thus leading to excessive memory consumption due to fragmentation"
How bad is the problem? We need to know it in order to prioritize our fixes.
Thanks,
Serge
We will make sure it is on our triage list but I can't promise what
release it will be in. We will get it fixed as quickly as we can.
Brian
<end quote>
>
> Greetings
> Jochen
Jochen Kalmbach
>> Is this a confirmed bug ? Where is the KB-Article ???
>
> ****
> No KB article as far as I know, but it is confirmed
> <quote from: Serge Lidin <sli...@MICROSOFT.COM>
> and Brian Harry <bha...@MICROSOFT.COM>
Tanhx for the answer...
--
Greetings
Jochen