Dynamic array BUG when using stringlists....
Toni
Just wondering if somebody else run into this problm.
For me it seems that this is a BUG in Delphi 6 +Delphi 7.
I testest the same code in Delphi 2007, and here it works.
This is just a simple "test" code, but it seems that if I use
dynamic array + tstringlist, Delphi does not free up
correct the memory of the dynamic array.
If you run this code under Delphi 6 or 7, the app will crasch with "Out-of
memory".
The "Setlength(DB_CACHE,0);" seems not to free up the memory.....
Is this is know bug ?
(As I say in Delphi 2007, this works. But the problem is that we have a lot
of projects still runing D7)
procedure TForm1.BitBtn1Click(Sender: TObject);
Var
DB_CACHE : Array of Longint;
ULIST :Array [1..100] of TStringlist;
TT :Word;
begin
For TT:=1 to 100 do
Begin
Setlength(DB_CACHE,4096*2000);
UList[TT]:=TStringlist.Create;
Setlength(DB_CACHE,0);
End;
For TT:=1 to 100 do
ULIST[TT].Free;
end;
Regards
Toni
Finn Tolderlund
<Toni> skrev i meddelelsen news:4879...@newsgroups.borland.com...
> For me it seems that this is a BUG in Delphi 6 +Delphi 7.
No, it's because of an extremely inefficient use of memory on your part.
> I testest the same code in Delphi 2007, and here it works.
That's just pure luck because Delphi 2007 uses a new memory manager that
apparantly is more capable of handling that kind of weird memory allocation
and deallocation.
> This is just a simple "test" code, but it seems that if I use
> dynamic array + tstringlist, Delphi does not free up
> correct the memory of the dynamic array.
>
> If you run this code under Delphi 6 or 7, the app will crasch with "Out-of
> memory".
> The "Setlength(DB_CACHE,0);" seems not to free up the memory.....
>
> Is this is know bug ?
Well, sort of, constantly allocating memory will always be a problem.
> (As I say in Delphi 2007, this works. But the problem is that we have a
> lot of projects still runing D7)
>
> For TT:=1 to 100 do
> Begin
> Setlength(DB_CACHE,4096*2000);
> UList[TT]:=TStringlist.Create;
> Setlength(DB_CACHE,0);
You are constantly allocating and freeing memory. You are doing in every
loop cycle.
That is extremely inefficient.
In this case you can rewrite it to only allocate and deallocate memory for
DB_CACHE once.
Like this:
> begin
> Setlength(DB_CACHE,4096*2000);
> For TT:=1 to 100 do
> Begin
> UList[TT]:=TStringlist.Create;
> End;
> Setlength(DB_CACHE,0);
> For TT:=1 to 100 do
> ULIST[TT].Free;
> end;
Without actually trying it in Delphi 7, I'll bet it will work without
crashing.
And what's more, it will most likely run a lot faster too.
Thinking about what you are doing is always a good idea.
--
Finn Tolderlund
Pieter Zijlstra
> Just wondering if somebody else run into this problm.
No, not really.
> I testest the same code in Delphi 2007, and here it works.
Delphi 2006 and onwards are using a new memory manager which handles
memory fragmentation better. You can use the same in D6 and D7,
download FastMM4 from SourceForge.
> For me it seems that this is a BUG in Delphi 6 +Delphi 7.
The bug is more in how you allocate memory, but Finn has already made
some comments regarding that.
--
Pieter
Toni
As I say, this is just a "sample code".
I know that i can to in another way and that this way is ineficient.....but
on the original code
what we do is "a little more complex".....this was just a "sample"!
But basicly i run into this problem and spend many hours until i found it..
So not clear why Delphi do not "free" the memory allocated by the dynamic
array?
Even DB_CACHE is defined local , if you call this 100 times, it run also out
of mem....
An Pieter say to use FastMM....I try to check the mem with "memCheck" and it
show NO memory leak!
So its not really clear, waht exactly hapen....but the memory seems not to
get "real" free.
If I use Microsoft "Process explorer" to track the memory usage.....
Setlength(DB_CACHE1,$FFFFFFF); // ---> 1 GB allocated
UList:=TStringlist.Create; --> Still 1 GB
Setlength(DB_CACHE1,0); // Still 1GB !!!!
Setlength(DB_CACHE2,$FFFFFFF); // ---> CRASH
If you do the same "without" tsring list...
Setlength(DB_CACHE1,$FFFFFFF); // ---> 1 GB allocated to DB_CACHE1
Setlength(DB_CACHE1,0); // ---> The 1 GB is free!!!
Setlength(DB_CACHE2,$FFFFFFF); // ---> 1 GB allocated to DB_CACHE2...no
problem
Here the full code... (this is just a "dummy" code and do nothing, but just
wondering, why this DOSE NOT WORK`!)
Non working CODE.....
procedure TForm1.BitBtn2Click(Sender: TObject);
Var DB_CACHE1 : Array of Longint;
DB_CACHE2 : Array of Longint;
ULIST :TStringlist;
begin
Setlength(DB_CACHE1,$FFFFFFF);
UList:=TStringlist.Create;
Setlength(DB_CACHE1,0);
Setlength(DB_CACHE2,$FFFFFFF); --> Crash
end;
working CODE.....
procedure TForm1.BitBtn2Click(Sender: TObject);
Var DB_CACHE1 : Array of Longint;
DB_CACHE2 : Array of Longint;
ULIST :TStringlist;
begin
Setlength(DB_CACHE1,$FFFFFFF);
Setlength(DB_CACHE1,0);
Setlength(DB_CACHE2,$FFFFFFF); --> OK
end;
Maybe i'm really "crazy", but what of hell has a "TStringlist" to do with
the memory allocation of a dynamic array?
Regards
Toni
"Finn Tolderlund" <n...@spam.com> schrieb im Newsbeitrag
news:487a...@newsgroups.borland.com...
Q Correll
| Begin
| Setlength(DB_CACHE,4096*2000);
| UList[TT]:=TStringlist.Create;
B_CACHE := nil; // Try using this line
| Setlength(DB_CACHE,0); // Instead of this line
|
| End;
--
Q
07/13/2008 21:23:32
XanaNews Version 1.17.5.7 [Q's Salutation mod]
Krisztian Pinter
> Setlength(DB_CACHE1,$FFFFFFF); // ---> 1 GB allocated
this line is a bug in itself. you can't safely allocate a memory
block of this size. your total address space is 2G big. this
should hold some minor system areas, your entire program, all
the dlls, all globals, stack, etc. that theoretically leaves
you around 1.x gigs of space, but this might not be continuous.
asking for 1G chunk is very risky. never do that. allocating gigs
of mem requires more advanced technics. you need to be extra
careful to avoid fragmentation. it is like changing your dress
inside a phone booth. a general purpose memory manager is never
enough good for that.
another point is that the delphi memory manager is provided to
hold small objects. the opsys gives you pages in 64K chunks. delphi
memory manager helps you organize your data in those big pages.
if you want to work with large memory blocks, you can work directly
with windows, and bypass delphi's memory manager entirely. not
that it does not work for large blocks, but its advantages fade.
Toni
But why this line is a bug?
As you say..i can alocate 2 GB, and i allocate only 1 GB, so from my point
of view this is not a bug.
But what i found has nothing to do with the "size" of the memory
block.......
You can try this sample with just "256MB" memory blocks...or even with "8MB"
blocks...
It always happen the same....
Try following...(here it CRASCH!! --> Out of memory)
procedure TForm1.BitBtn3Click(Sender: TObject);
Const MB256 = 256*1001024;
Const MB8 = 8*1001024;
Var
DB_CACHE1 : Array of Byte;
DB_CACHE2 : Array of Byte;
ULIST :Array [1..400] of TStringlist;
TT :Word;
Begin
For TT:=1 to 400 do
Begin
Setlength(DB_CACHE1,MB8);
UList[TT]:=TStringlist.Create;
DB_CACHE1:=NIL;
Setlength(DB_CACHE2,MB8);
DB_CACHE2:=NIL;
End;
end;
Now swap the first 2 lines in the loop....and now, surprise...it works!
procedure TForm1.BitBtn3Click(Sender: TObject);
Const MB256 = 256*1001024;
Const MB8 = 8*1001024;
Var
DB_CACHE1 : Array of Byte;
DB_CACHE2 : Array of Byte;
ULIST :Array [1..400] of TStringlist;
TT :Word;
Begin
For TT:=1 to 400 do
Begin
UList[TT]:=TStringlist.Create;
Setlength(DB_CACHE1,MB8);
DB_CACHE1:=NIL;
Setlength(DB_CACHE2,MB8);
DB_CACHE2:=NIL;
End;
end;
Is that not strange?
So whats the difference if I first set a dynamic array, and then a
stringlist, or first a stringlist and then a dynamic array?
I know that there are "other" way to do it, but should not Delphi handel
this things?
Regards
Toni
"Krisztian Pinter" <pinter.k...@chello.hu> schrieb im Newsbeitrag
news:op.ud9wb3tqobk3tt@karwst_pk.karatnet.hu...
Krisztian Pinter
> But why this line is a bug?
> As you say..i can alocate 2 GB, and i allocate only 1 GB, so from my
> point
> of view this is not a bug.
there is a strict distinction between memory and address space. if you
allocate
blocks that are comparable to the available address space, you risk it will
fail even if there is enough memory.
your case is different. 256MB is a block size i would consider safe.
smaller is
better, but it can be a good choice. but here comes the next problem: heap
fragmentation. there is NO memory manager that is totally fragmentation
free.
the most dangerous patterns are the loops that allocate and release large
amout of memory in few blocks, but keep a small amout allocated. exactly
your case.
we always say: no premature optimization. but another saying could be: when
time comes, start optimizing. playing with large memory blocks are beyond
the scope of the standard memory manager of delphi.
> Is that not strange?
it is. i tried to understand some fragmentation cases, model the
allocation and
deallocation on paper. never succeeded. too many details, too many hidden
allocations. finally we changed the allocation pattern, or changed the
memory
manager, and the problem disappeared.
Ian Boyd
> some comments regarding that.
Just so i'm clear...
Toni assumed the problem was that memory that he was freeing wasn't really
getting freed, and so was a bug. But that's not what's happening; the memory
is getting freed, correct?
The problem is that the memory manager is running out of contigious memory
blocks to allocate from?
And it's not so much a bug, as an ugly detail that most people can normally
ignore. And while there are things he could do to try to limit experiencing
this problem, or work around it, it really is a fundamental problem?
Toni
> Toni assumed the problem was that memory that he was freeing wasn't really
> getting freed, and so was a bug. But that's not what's happening; the
> memory is getting freed, correct?
Thats a good question.
If i watch the memory usage with Microsoft Process explorer, following
happen....
Setlength(DB_CACHE1,$FFFFFF); "Private bytes" and "Virtual size" both
grow. (In this case both 16MB)
UList[TT]:=TStringlist.Create; "No change at this does
basicly just use a few bytes...
Setlength(DB_CACHE1,0); The "Private bytes", decrease,
but "virtual size, still have the same size as when i allocated!
No if you continue with,
Setlength(DB_CACHE1,$FFFFFF); "Private bytes" grow "Virtual size"
grow.the pld size + new size....(So virtualsize is now 32MB)
Setlength(DB_CACHE1,0); The "Private bytes", decrease,
but "virtual size, still have the 32MB!
If you now continue, once "virtual bytes" reach 2GB ---> it goes ot of
memroy.
As i describe in the other post, I don't understand why it only happen if I
also use this line...(after SETLENGTH)
UList[TT]:=TStringlist.Create
>
> The problem is that the memory manager is running out of contigious memory
> blocks to allocate from?
>
Why continues blocks?
I allocated with
Setlength(DB_CACHE1,$FFFFFF);
and do a
Setlength(DB_CACHE1,0); // or NIL
Basicly Delphi should free this memory....
> And it's not so much a bug, as an ugly detail that most people can
> normally ignore. And while there are things he could do to try to limit
> experiencing this problem, or work around it, it really is a fundamental
> problem?
>
An ugly detail?
And don't worry about it?
The full code of our app is >100'000 lines of code, and the problem was that
the app, crash only after a few day...!
So I spend many hours to find this "ugly" thing...
But still don't understand why it happens.
I thing yes its a fundemantal problem.....because Delphi seems to do
something wrong with dynamic array + stringlist...
Krisztian Pinter
<ian.borla...@avatopia.com> wrote:
> ignore. And while there are things he could do to try to limit
> experiencing
> this problem, or work around it, it really is a fundamental problem?
what are you referring to with "it"?
the fact that memory will be fragmented sometimes is a fundamental problem
of
all memory management. it is impossible to write a memory manager that is
free
of fragmentation in every circumstances.
tha fact that the OP didn't know about fragmentation is also a fundamental
problem,
although this one can be cured easily: he has to be informed.
Pieter Zijlstra
> > The bug is more in how you allocate memory, but Finn has already
> > made some comments regarding that.
>
> Just so i'm clear...
>
> Toni assumed the problem was that memory that he was freeing wasn't
> really getting freed, and so was a bug. But that's not what's
> happening; the memory is getting freed, correct?
Yes
> The problem is that the memory manager is running out of contigious
> memory blocks to allocate from?
Yes
> And it's not so much a bug, as an ugly detail that most people can
> normally ignore.
Yes, like precission of FP numbers, bankers rounding, etc, etc ;-)
> And while there are things he could do to try to limit experiencing
> this problem, or work around it, it really is a fundamental problem?
Although the new memory manager (FastMM4) handles this a lot better,
there are still limitations to what it can handle.
--
Pieter
Toni
Whats wrong with this code?
(It's only a "dummy" code, but let assume we really need to do some
calculation....)
If I click on the "Button" to start the calculations, the app crash, with
"out of memory...."
I set and free DB_CACHE correctly, so why it does goes out of memory?
Const MB128 = 128*1001024;
Function TForm1.Calculate:TStringList;
Var
DB_CACHE1 : Array of Byte;
Begin
Try
Setlength(DB_CACHE1,MB128);
Result:=TStringlist.Create;
//Do some calculation, with DB_CACHE + the string list....
// Then return the results trough the stringlist
Finally
DB_CACHE1:=NIL;
End;
end;
procedure TForm1.BitBtn3Click(Sender: TObject);
Var
ULIST :Array [1..200] of TStringlist;
TT :Word;
Begin
For TT:=1 to 200 do
Begin
ULIST[TT]:=Calculate;
End;
//Continue working with ULIST
end;
Toni
So maybe really a BUG in Delphi 6+7.........
"Pieter Zijlstra" <p.zylstr...@hccnet.nl> schrieb im Newsbeitrag
news:487b76a2$1...@newsgroups.borland.com...
Krisztian Pinter
> Whats wrong with this code?
that it is prone to memory corruption. if you ignorant to the technology
you use, all sort of havoc can happen.
> I set and free DB_CACHE correctly, so why it does goes out of memory?
it does not. memory is correctly freed. just the free blocks are
fragmented,
so you have less and less chance to get a large block.
if you don't understand this, you are not trained well enough to implement
such programs. bad luck.
Krisztian Pinter
> Just by the way, with FastMM it works.
> So maybe really a BUG in Delphi 6+7.........
it is not. fastmm is designed with that goal in mind. it reduces
fragmentation in most circumstances. what you need to know is:
- it does not eliminate fragmentation entirely. it is more stable,
but there is no silver bullet.
- every implementation has tradeoffs. improvement always comes at
a cost.
Toni
Why it works on this way?
procedure TForm1.BitBtn3Click(Sender: TObject);
Var
ULIST :Array [1..2000] of TStringlist;
TT :Word;
Begin
For TT:=1 to 2000 do
Begin
ULIST[TT]:=Calculate;
ULIST[TT].Free; // If I free the Stringlist
here....no Problem.
End;
end;
The only difference is that i "free" the stringlist!
And you can run with 2000, even 20'000 loops...no problem......
Why on this way, the "memory corruption/fragmentation" you mention has gone?
Even if you do a crazy thing like.......
Setlength(DB_CACHE1,random(MB512)); // Reserve a random memory block
0..512MB
Here the fragmentation should be very "high", as each block has a differente
size...but the code still works!
interesting...not?
I tested with 2000+20000 loops -> no problem....app keep runing, an never
goes out of memory!
I'm maybe not trained well, possible...........maybe someone can explain the
differnce of the memory usage,
if I "free the stringlist" (app works) and if I do NOT free the
stringlist....(app crash)
Krisztian Pinter
> If it really a "memory corruption/fragmentation" problem.......
> Why it works on this way?
actually, what is your point here? you are really asking, or you
doubt that it is a fragmentation? to me, your questions sound like
you feel offended because your nice program doesn't work.
to answer the question directly: i don't know. i told you that
tracking the memory allocation and deallocation is a pain.
but it is irrelevant. relevant is: some memory usages lead to
fragmentation. you have to either find another MM, or alter your
algorithm.
Toni
Maybe somebody can explain why this code work
ULIST[1]:=TStringlist.Create;
ULIST[2]:=TStringlist.Create;
GetMem(DB_CACHEP,MB512*2); //1GB allocated, OK
FreeMem(DB_CACHEP); // 1GB is free, "Private
bytes"+Virtual bytes are free.
GetMem(DB_CACHEP,MB512*2); --> No problem...again 1 GB allocated...
And this not... (it does basicly the same..or not?)
GetMem(DB_CACHEP,MB512*2); //1GB allocated, OK
ULIST[1]:=TStringlist.Create;
ULIST[2]:=TStringlist.Create;
FreeMem(DB_CACHEP); // 1GB is free, but only "Private
bytes". Virtual size still 1GB
GetMem(DB_CACHEP,MB512*2); --> here it crash...
As I say, I know how to fix it, and my app is running...but just want to
know the nature of the problem.
Ian Boyd
>> experiencing
>> this problem, or work around it, it really is a fundamental problem?
>
> what are you referring to with "it"?
The error - the attempt to allocate memory failing.
Ian Boyd
makes me an ignorant end-user.
i allocate 1GB of contigious memory. Then i allocate 1 byte of memory.
Next i free the 1GB of memory.
Now i try to allocate another 1GB of memory. But for some reason the 1GB i
just freed isn't 'good enough' to be used, so the memory manager has to find
another 1GB chunk somewhere else. What was wrong with that 1GB i just gave
back to you? Is the 1GB fragment not big enough to hold 1GB?
Krisztian Pinter
<ian.borla...@avatopia.com> wrote:
>>> it really is a fundamental problem?
>>
>> what are you referring to with "it"?
>
> The error - the attempt to allocate memory failing.
i still don't understand what do you mean. if you ask for a block
of memory, and there is no continuous block big enough, you get a
memory overflow. that's not new, nor it is surprising.
the fact that some allocation pattern leads to fragmented memory
is also not new, nor surprising.
actually, we know that every implementation can be fooled.
avoiding such patterns is not easy. there are no simple rules.
we know some patterns that are risky. after some time, some kind
of instinct will ring a bell in your head. but one can never be
sure. if you play with gigabytes, or even hundred megabytes,
be prepared for difficulties.
Toni
In the real app, there is a loop like..
1. Load a large image, and store the bytes in a dynamic array. (The size
of the image is not always the same)
2. Get some attributes from the array, and store in a stringlist. (For
each image one stringlist)
3. free the memory as no longer needed.
4. Repeat step 1-3 until all images are loaded.
Each image has different sizes, so i allocate each time the correct size.
So i',m not looking for a solution, (I know how to do it so it works),
but I'm just wondering if somebody can explain "why" this happen,
if basicliy both (sample) codes do the same........
"Ian Boyd" <ian.borla...@avatopia.com> schrieb im Newsbeitrag
news:487cb76c$1...@newsgroups.borland.com...
Krisztian Pinter
<ian.borla...@avatopia.com> wrote:
> i really don't understand why its happening either - which i guess also
> makes me an ignorant end-user.
but you don't have to. you want to? okay, here is what you have to do:
trace every allocation and deallocation. for a string list, it consists
of allocation of the class, allocation of the internal array, allocation
of each individual strings (or not, depending on how you get them, see
reference counting), reallocation of the internal array as the list
grows. you also need to consider any local string variables, and any
hidden local string variables that will hold temporary results of
string expressions.
are you familiar with the internals of delphi classes? ansi strings?
CPU window and machine code? delphi's built-in MM?
go ahead, good luck with it. but if you don't have time for this,
simply accept that vigorous allocation and deallocation of large
chunck of memory can lead to unexpected results.
Ian Boyd
> of memory, and there is no continuous block big enough, you get a
> memory overflow. that's not new, nor it is surprising.
>
> the fact that some allocation pattern leads to fragmented memory
> is also not new, nor surprising.
But you can understand everyone's confusion, right?
i) allocate 1 GB of memory
ii) allocate 1 byte of memory
iii) release the 1GB of memory
iv) allocate 1 GB of memory
You're telling me that due to fragmentation problems there isn't 1GB of
contigious memory available. But what about that 1GB block i just gave back
to you? Isn't that 1GB of contigious memory good enough?
Now you are free to answer,
"Yes, you would think that would work. But memory management is much
more complicated than that - far to complicated to explain here. It will
suffice to say that even though you think there is 1GB of contigious memory
available, there really isn't. That's just how it is, and you have to accept
it, and change the way you do things in order to work around the limitation
of the way things are."
Which is fine, as long as you can agree that it's not obvious, intuitiave,
or even explainable to anyone without experience of writing a memory
manager - which is a majority of all developers on earth. So perhaps you can
get down off your horse and cut Toni some slack.
Toni
> CPU window and machine code? delphi's built-in MM?
>
> go ahead, good luck with it. but if you don't have time for this,
> simply accept that vigorous allocation and deallocation of large
> chunck of memory can lead to unexpected results.
I'm not a "low-lvl" developer, and I don't need to know al this things...
If I have so much know-how over this things,
i can write my own MM, and even my own compiler!
So for what then I need Delphi?
Is Delphi not a "RAD-developer-suite" and should do this things for me?
What you are saying is that (hundert)tousends of developer,
are just writing codes , and maybe Delphi does it right maybe not.......
From your point of view, its not possible to write "100%" stabel code,
without learning first "years" of internas....of Delphi, Windows, maschine
codes.etc.etc...
As Ian say....
If a "free" 1GB, and the next line I allocate it again, why the memory is
corrupted?
There is "nothing" else beetween........
Setlength (DC_CACHE,0)
Setlength (DC_CACHE,MB512)
What has gone with the "freed" 512MB?
And you always mention "large" blocks, fragmentation...etc.etc......
Do follwing with "small" 8MB blocks....--> It crash!
For TT:=1 to 1000 do
Begin
GetMem(DB_CACHEP,MB8);
ULIST[TT]:=TStringlist.Create;
//Work with the mem....
FreeMem(DB_CACHEP);
End;
And now try this...
For TT:=1 to 1000 do
ULIST[TT]:=TStringlist.Create;
For TT:=1 to 1000 do
Begin
GetMem(DB_CACHEP,MB8);
//Work with the mem....
FreeMem(DB_CACHEP);
End;
Whats the difference?
I Allocated + free 1000x a 8MB block, in both the same......
I'm quite sure this is a BUG of Delphis MM........
Just surpising that nobody else runs into this problem
Pieter Zijlstra
> Just by the way, with FastMM it works.
> So maybe really a BUG in Delphi 6+7.........
I can understand it is bugging you, but I wouldn't call it a bug...
more like a design choice made about a decade ago. And it worked well
for a long time (and still does in most cases). You just ran in to a
limitation of that design.
Newer PCs have plenty of RAM these days and developers are (ab)using
more and more of it, sometimes a little to careless IMO. Disclaimer, I
don't know your actual situation, so I can't tell in which category you
belong ;-)
About 4 years ago there was a Fastcode challenge for memory managers, a
dozen memory managers entered that challenge and FastMM4 came out as
winner but there are others, each with their pro and cons (and some
just dead slow :).
FastMM4 was specifically written for 1) speed (roughly twice as fast as
the orginal Borland memory manager) and 2) to prevent fragmentation.
Borland liked it and assimilated it in D2006 and beyond :)
http://www.fastcode.dk/fastcodeproject/fastcodeproject/34.htm
(also google the b.p.d.l.basm group for memory manager challenge)
--
Pieter
Pieter Zijlstra
> An Pieter say to use FastMM....I try to check the mem with "memCheck"
> and it show NO memory leak!
FastMM4 does have a lot of options (see the "inc" file). One of them is
called FullDebugMode. Define that in your project options and make sure
your app can see the included debug dll. Easy way to test for memory
leaks.
--
Pieter
Krisztian Pinter
<ian.borla...@avatopia.com> wrote:
>> the fact that some allocation pattern leads to fragmented memory
>> is also not new, nor surprising.
>
> But you can understand everyone's confusion, right?
at first, yes. i guess fragmentation is not something people find out
themselves. one needs to hear about it. but once you heard about it,
you have to know and incorporate this knowledge.
> i) allocate 1 GB of memory
> ii) allocate 1 byte of memory
> iii) release the 1GB of memory
> iv) allocate 1 GB of memory
most likely it is not what happens. most likely there is a lot of hidden
stuff going on in the background you don't know about. for example if
there is a hidden allocation between step 3 and 4, you get two small
blocks occupied almost 1G apart. getting another 1G leads to allocate
another 1G chunk from the opsys.
bottom line is as i already said: don't play with gigabytes.
or i could say: allocating 1G is an error, period.
> Which is fine, as long as you can agree that it's not obvious,
> intuitiave,
not obvious. you need to be told about it, then you run curiously and read
all resources you can find on internet. you understand it on the level you
want. then from now on, you design your allocation schemas with that in
mind.
> or even explainable to anyone without experience of writing a memory
> manager
no, you can understand it. just needs some effort. instead of complaining
here.
i remember i told you that i'm not the person who in depth looked into how
and why fragmentation occur in delphi MM in some circumstances. i
understand
the issue on general level, and i can avoid it. but i can't tell you the
exact
allocation and deallocation sequence, and how it is handled by the MM. MM
is
not documented, to begin with.
Krisztian Pinter
> So for what then I need Delphi?
> Is Delphi not a "RAD-developer-suite" and should do this things for me?
there is only one "thing" on this earth that can do everything for you:
a programmer. no tool is 100% bulletproof. knowing a tool's limitations
is essential in order to use it correctly.
> If a "free" 1GB, and the next line I allocate it again, why the memory
> is corrupted?
not corrupted. in this case, for a reason unknown to me, the mm chosen not
to reuse this
part of memory. it seems delphi does not like to allocate memory in a hole
where it
precisely fits. it can be a subtle bug, or can be a side effect of some
allocation
stategy. after all, it is not usual that someone often allocates and frees
the same space
repeatedly. most of the time, we allocate and deallocate memory in a quite
random order.
either way, your memory is not wasted, it is sitting there for you to use.
just for some
reason it is not reused.
is it a poor implementation? maybe. but don't even know that. for all we
know, it can be
optimized for other, more common uses.
rbwinston
the project and tried again. This time there was no problem. Borland
has acknowledged that the memory manager that was used in D7 and
earlier could lead to memory corruption. That is part of the reason
they now use FastMM4 by default in more recent versions of Delphi.
<Toni> wrote:
>HI,
>Just wondering if somebody else run into this problm.
>
>For me it seems that this is a BUG in Delphi 6 +Delphi 7.
>I testest the same code in Delphi 2007, and here it works.
>
>This is just a simple "test" code, but it seems that if I use
>dynamic array + tstringlist, Delphi does not free up
>correct the memory of the dynamic array.
>
>If you run this code under Delphi 6 or 7, the app will crasch with "Out-of
>memory".
>The "Setlength(DB_CACHE,0);" seems not to free up the memory.....
>
>Is this is know bug ?
>(As I say in Delphi 2007, this works. But the problem is that we have a lot
>of projects still runing D7)
>
>procedure TForm1.BitBtn1Click(Sender: TObject);
>Var
> DB_CACHE : Array of Longint;
> ULIST :Array [1..100] of TStringlist;
> TT :Word;
>begin
> For TT:=1 to 100 do
> Begin
> Setlength(DB_CACHE,4096*2000);
> UList[TT]:=TStringlist.Create;
> Setlength(DB_CACHE,0);
>
> End;
> For TT:=1 to 100 do
> ULIST[TT].Free;
>end;
>
>
>Regards
>Toni
>
John Herbster
Krisztian wrote:
>> are you familiar with the internals of delphi classes?
>> ANSI strings?
>> CPU window and machine code?
>> Delphi's built-in MM?
>
>> Go ahead, good luck with it. but if you don't have
>> time for this, simply accept that vigorous allocation
>> and deallocation of large chunk of memory can lead
>> to unexpected results.
> I'm not a "low-lvl" developer, and I don't need to know
> all these things. If I have so much know-how over this
> things, i can write my own MM, and even my own compiler!
Toni,
There is such a big leap between just *understanding* the
fundamental concept of memory fragmentation and *writing*
one's own MM and compiler, that I think that your argument
is specious, in other words worthless.
Memory fragmentation is a fundamental concept of present
day computing. You may find this explanation useful:
http://en.wikipedia.org/wiki/Fragmentation_(computer)
There are ways around it; but they can be complicated and
slow down execution.
> What you are saying is that (hundreds) thousands of
> developers are just writing codes, and maybe Delphi
> does it right maybe not.
Delphi does its best to guess what the programmers are
trying to do. If programmers ignorant of the internals
avoid certain practices, they can have much better chances
of success.
> From your point of view, it's not possible to write "100%"
> stable code, without learning first "years" of internals
> of Delphi, Windows, machine codes, etc.
I would agree with that point of view.
> As Ian says:
> If [I] "free" 1GB, and [in] the next line I allocate it
> again, why the memory is corrupted?
I doubt that memory is corrupted. It is more likely that
some other allocation took a few bytes at one end or the
other of the freed 1GB, so that all the freed 1GB is no
longer available as one contiguous block.
Toni, the good news is that if you can understand what 1GB
is then you are on the verge of understanding memory
fragmentation.
HTH, JohnH