making an object pool
David Novo
Hello,
Sorry for the cross post. I originally put this in
borland.public.delphi.language.delphi.win23 but was suggested to move
it here.
I have been experimenting with making an object pool that I can use
with almost any object in my system. I have come up with the scheme
below. It is relatively simple, and it seems to work. I am wondering if
anyone has any comments as to how I can improve, or if there are any
subtle errors I am creating.... I will show the code first, and then
make some comments at the end
TObjectPool=class(TObjectList)
protected
procedure Notify(Ptr: Pointer; Action: TListNotification); override;
public
function RequestObjectFromPool(const aClass: TClass): TObject;
procedure ReturnObjectToPool(aObject: TObject);
end;
procedure TObjectPool.Notify(Ptr: Pointer; Action: TListNotification);
begin
{
TObjectList.Notify frees the object if ownsObjects. That will create an
endless loop if the pool itself is trying to free the object. We have
to tell the object not to return itself to the pool. Therefore, we
cannot call inherited if OwnsObjects=True and the Action=lnDeleted
}
if OwnsObjects then
begin
if Action = lnDeleted then
begin
TPoolableObject(Ptr).ReturnToPoolOnFree:=False;;
TPoolableObject(Ptr).Free;
end
else
inherited Notify(Ptr, Action);
end
else
inherited Notify(Ptr,Action);
end;
function TObjectPool.RequestObjectFromPool(const aClass: TClass):
TObject;
var
cntr: Integer;
curObj: TObject;
begin
Result := nil;
for cntr := Count-1 downto 0 do
begin
curObj:=TObject(List[cntr]);
if curObj.classType = aClass then
begin
{
doing this for optimization. Extract only takes a pointer, and will
search the entire list again. But I already know the index.
do my own low-level "extract"
}
list[cntr]:=nil;
Delete(cntr);
result:=curObj;
break;
end;
end;
end;
procedure TObjectPool.ReturnObjectToPool(aObject: TObject);
begin
Add(aObject);
end;
I have also made a TPoolableObject
TPoolableObject=class(TObject)
private
FReturnToPoolOnFree: Boolean;
public
procedure AfterConstruction; override;
procedure FreeInstance; override;
class function NewInstance: TObject; override;
property ReturnToPoolOnFree: Boolean read FReturnToPoolOnFree write
FReturnToPoolOnFree;
end;
procedure TPoolableObject.AfterConstruction;
begin
inherited;
ReturnToPoolOnFree:=True;
end;
procedure TPoolableObject.FreeInstance;
begin
if ReturnToPoolOnFree then
objectPool.ReturnObjectToPool(self)
else
inherited FreeInstance;
end;
class function TPoolableObject.NewInstance: TObject;
begin
Result:=objectPool.RequestObjectFromPool(self);
if Result=nil then
Result := inherited NewInstance;
end;
Some comments I noticed:
1. The pool is quite stupid right now and just stores things in the
list. Optimization on the number/types of elements as well as faster
lookup tables to find the different class types more quickly can be
added.
2. There is a requirement to add the .ReturnToPoolOnFree in order to
prevent a circular reference. I would love to get rid of this, so that
in order to make an object poolable I only have to override NewInstance
and FreeInstance. Does anyone have any suggestions?
3. When returning to the pool, or when being retrieved from the pool,
the object should be "reset" to default values, as if it was just
created. I don't think that there is a general way to do this, and
probably the TPoolableObject would have to have a method implement
.ResetToDefaultValues
4. Would it be a good idea to suggest to Codegear to add some basic
pooling hooks to TObject to more easily implement your own pooling
mechanism, the same way you can plug in your own memory manager?
Aside from these points, it seems to work quite well. If you make a lot
of transient object during the course of your program, an objectPool
can certainly be more efficient than actually creating and destroying
the objects over and over.
--
Gerard
> 1. The pool is quite stupid right now and just stores things in the
> list. Optimization on the number/types of elements as well as faster
> lookup tables to find the different class types more quickly can be
> added.
>
One first and easy step would be to split the pool into a per class
basis, the pool being internally a list of pools.
> Aside from these points, it seems to work quite well. If you make a lot
> of transient object during the course of your program, an objectPool
> can certainly be more efficient than actually creating and destroying
> the objects over and over.
>
This is just curiosity, did you measure the difference?
I guess there is not much gain if you count the time to get objects from
the pool and putting them back + the time to "clean" the objects for
further reuse?
Regards,
Gerard.
Gerard
>> Aside from these points, it seems to work quite well. If you make a lot
>> of transient object during the course of your program, an objectPool
>> can certainly be more efficient than actually creating and destroying
>> the objects over and over.
>>
> This is just curiosity, did you measure the difference?
> I guess there is not much gain if you count the time to get objects from
> the pool and putting them back + the time to "clean" the objects for
> further reuse?
Answering myself:
I did a little test with the opf I am currently working on.
Creating 100.000 objects: average 3,734 seconds.
Creating 100.000 objects and adding them to a TObjectList: average 3,825
seconds.
Getting 100.000 objects from the object list : average 6,28 seconds.
The code to retrieve the objects from the list:
while l.Count>0 do
begin
c := TClient(l[0]);
l.Delete(0);
end;
Getting 100.000 objects from the object list, second method : average
1,25 seconds.
The code to retrieve the objects from the list, second method:
while l.Count>0 do
begin
c := TCompte(l[l.Count-1]);
l.Delete(l.Count-1);
end;
These are big fat objects and object creation involves
- Acces the object metadata and create and the attributes from the metadata
- Set several state flags both for the object and the attributes
Conclusion 1: The Pool manager has to be very well thought to be really
efficient
Conclusion 2: In my case, if I add the cost of reinitiale the object
property values (even if this could be done by setting the IsNull flag
to true for each attribute) I think it's not worth all the hassle.
Of course, you mileage may vary. ;-)
Regards,
Gerard.
David Novo
> I did a little test with the opf I am currently working on.
>
> Creating 100.000 objects: average 3,734 seconds.
> Creating 100.000 objects and adding them to a TObjectList: average
> 3,825 seconds.
The whole point of the pool is that you may need 100,000 objects during
the life of the program, but maybe only 50 at one time. So you will not
be adding 100,000 to the pool, maybe only 50 will be needed from the
pool. From your timing above, adding 100,00 objects to a list is very
fast (3.825-3.734). So if you only create 50 items and add them back
repeatedly, it will be very fast.
>
> Getting 100.000 objects from the object list : average 6,28 seconds.
> The code to retrieve the objects from the list:
>
> while l.Count>0 do
> begin
> c := TClient(l[0]);
> l.Delete(0);
> end;
>
> Getting 100.000 objects from the object list, second method :
> average 1,25 seconds. The code to retrieve the objects from the
> list, second method:
>
> while l.Count>0 do
> begin
> c := TCompte(l[l.Count-1]);
> l.Delete(l.Count-1);
> end;
This is a more accurate comparison, as the first retrieval above is
very inneficient. So we have 3.7 sec for creating all the objects vs
1.25 secs for retreiving 100,000) from a list. Presuming we only have
to have a small number alive at once, that is a huge percentage
savings. During the lifetime of my typical application, millions of
objects are created and destroyed.
> Conclusion 1: The Pool manager has to be very well thought to be
> really efficient Conclusion 2: In my case, if I add the cost of
> reinitiale the object property values (even if this could be done by
> setting the IsNull flag to true for each attribute) I think it's not
> worth all the hassle.
Setting up the default state has to be done whether the object is
created from scratch or just "re-initialized". That initialization time
is essentially a constant, so does not impact the calculation. If you
Container object contains aggregated-objects, it is certainly faster to
"clear" the aggregated object than free and re-create it.
My business objects already all have a .SetDefaultValues, that
nitializes the objects to a default state. This is already called from
the AfterConstruction of business object. So after creation, all
objects get set to their default state, not via the .Create method, but
via the SetDefaultValues. So it there is already a way to restore
everything to the default state.
David Novo
I would love to do this, but I want to put the pooling in my ancestor
business object. I have hundreds of concrete objects and don't want to
add pooling manually to every class. I could create a "pool of pools"
and then easily find a particular pool by hashing the classname,
although sorting the pools by .ClassType may be faster. I think
.ClassType is an ordinal.
Gerard wrote:
> One first and easy step would be to split the pool into a per class
> basis, the pool being internally a list of pools.
--
Gerard
In my case, I use that aproach for the object cache.
It's a list of lists, indexed by class name.
So far, it's implemented as a sorted TStringlist, with each node having
a pointer to a class cache.
Gerard
>
>
>
>> I did a little test with the opf I am currently working on.
>>
>> Creating 100.000 objects: average 3,734 seconds.
>> Creating 100.000 objects and adding them to a TObjectList: average
>> 3,825 seconds.
>
> The whole point of the pool is that you may need 100,000 objects during
> the life of the program, but maybe only 50 at one time. So you will not
> be adding 100,000 to the pool, maybe only 50 will be needed from the
> pool. From your timing above, adding 100,00 objects to a list is very
> fast (3.825-3.734). So if you only create 50 items and add them back
> repeatedly, it will be very fast.
Well, I tested with 100.000 because 50 was too small to be significant<g>
>
>> Getting 100.000 objects from the object list : average 6,28 seconds.
>> The code to retrieve the objects from the list:
>>
>> while l.Count>0 do
>> begin
>> c := TClient(l[0]);
>> l.Delete(0);
>> end;
>>
>
>> Getting 100.000 objects from the object list, second method :
>> average 1,25 seconds. The code to retrieve the objects from the
>> list, second method:
>>
>> while l.Count>0 do
>> begin
>> c := TCompte(l[l.Count-1]);
>> l.Delete(l.Count-1);
>> end;
>
> This is a more accurate comparison, as the first retrieval above is
> very inneficient. So we have 3.7 sec for creating all the objects vs
> 1.25 secs for retreiving 100,000) from a list. Presuming we only have
> to have a small number alive at once, that is a huge percentage
> savings. During the lifetime of my typical application, millions of
> objects are created and destroyed.
To be fair we should compare the cost to create 100.000 objects vs
create + add to the list + retrieve 100.000 objects.
In that case it's 3.7 for object creation vs 3.8 + 1.2 for object
pooling, so object pooling would be slower at for one object, and faster
for recycled objects.
>
>> Conclusion 1: The Pool manager has to be very well thought to be
>> really efficient Conclusion 2: In my case, if I add the cost of
>> reinitiale the object property values (even if this could be done by
>> setting the IsNull flag to true for each attribute) I think it's not
>> worth all the hassle.
>
> Setting up the default state has to be done whether the object is
> created from scratch or just "re-initialized". That initialization time
> is essentially a constant, so does not impact the calculation.
True
If you
> Container object contains aggregated-objects, it is certainly faster to
> "clear" the aggregated object than free and re-create it.
>
True
David Novo
That brings up a good point. At the extreme, if you have to create the
same number of objects that you eventually use (i.e. all objects are
alive at the same time), pooling does not make sense. At the other
extreme, you may only need 1 object alive at once and need to just
retreive it 100,000 times pooling will be much faster. I guess it
depends on the useage in your application which is the case. The
correct comparison depends on how your app works.
That would be another use of the object pool, to keep track of
statistics of how many, and which types of objects. Max allocated etc,
could all be kept track of by the pool.
Jens Gruschel
> .ClassType is an ordinal.
Right, ClassType basically is a pointer to the VMT (more or less). It
equals Self in class methods like NewInstance which you are using anyway.
--
Jens Gruschel
http://www.pegtop.net
Jens Gruschel
> However I'm not sure whether that's a good idea, because I think
> reference counting for strings and things like that would not work
> properly any more.
Argh, that'd delete the class type, too, which is stored in the first 4
bytes Self points to.
Jens Gruschel
> with almost any object in my system.
Very interesting. Of course I'm not sure about it, but my first thoughts
were:
1) This could prevent memory fragmentation (well, but a good memory
manager should also do so)
2) I'd guess it's quite good for some objects but not very suitable for
other ones. For example it makes no sense for objects mainly consisting
of strings, dynamic arrays or other members dynamically created, except
maybe a new instance can make use of such memory allocated. But a pool
would be perfect for simple things with mainly ordinal members (like
game objects holding coordinates, current speed, life points etc.).
Object creation BTW can be speed up in case no null initialization is
required (which is essential for strings and similar types I think)...
http://www.cerebral-bicycle.co.uk/viewdoc.asp?doc=326
> 1. The pool is quite stupid right now and just stores things in the
> list. Optimization on the number/types of elements as well as faster
> lookup tables to find the different class types more quickly can be
> added.
Probably deriving the pool from TObjectList is the most simple approach,
but is it the most flexible? I think I'd derive it from TObject,
implementing all the list stuff myself, being able to use hash tables or
other optimization internally. That would also solve following one,
wouldn't it?
> 2. There is a requirement to add the .ReturnToPoolOnFree in order to
> prevent a circular reference. I would love to get rid of this, so that
> in order to make an object poolable I only have to override NewInstance
> and FreeInstance. Does anyone have any suggestions?
The problem remaining is: when you destroy the pool all objects it holds
should be freed, too. But since all poolable objects know their pool
anyway this could also be done with a flag belonging to the pool, not
the objects, right?
> 3. When returning to the pool, or when being retrieved from the pool,
> the object should be "reset" to default values, as if it was just
> created. I don't think that there is a general way to do this, and
> probably the TPoolableObject would have to have a method implement
> .ResetToDefaultValues
Not tested: FillChar(Pointer(Self)^, InstanceSize, 0);
However I'm not sure whether that's a good idea, because I think
reference counting for strings and things like that would not work
properly any more.
> 4. Would it be a good idea to suggest to Codegear to add some basic
> pooling hooks to TObject to more easily implement your own pooling
> mechanism, the same way you can plug in your own memory manager?
Not sure. On one hand I like the idea. On the other hand I don't want my
existing applications to become slower in case such a modification of
TObject would mean more overhead.
David Novo
> The problem remaining is: when you destroy the pool all objects it
> holds should be freed, too. But since all poolable objects know their
> pool anyway this could also be done with a flag belonging to the
> pool, not the objects, right?
Brilliant idea. That should work perfectly!
>
> Not sure. On one hand I like the idea. On the other hand I don't want
> my existing applications to become slower in case such a modification
> of TObject would mean more overhead.
The checking will simply consist of something simple like
If ObjPool=nil then
// do regular stuff
else
// do pool stuff
An extra check for nil will not add much time, compared to what is
already going on when freeing an object
--
Marc Rohloff [TeamB]
> The checking will simply consist of something simple like
>
> If ObjPool=nil then
> // do regular stuff
> else
> // do pool stuff
>
> An extra check for nil will not add much time, compared to what is
> already going on when freeing an object
Assuming you have some sort of abstract base class or interface, then
you could also just create a 'null' pool which doesn't do any pooling
but just creates a new object each time.
http://en.wikipedia.org/wiki/Null_Object_pattern
--
Marc Rohloff [TeamB]
marc -at- marc rohloff -dot- com
Jens Gruschel
> you could also just create a 'null' pool which doesn't do any pooling
> but just creates a new object each time.
> http://en.wikipedia.org/wiki/Null_Object_pattern
I'd call it "ObjectLifetimeManager" ;-) One could provide one "normal"
one that creates and destroys objects as usual. Another one could manage
a pool. Even another one could create borg objects (see
http://log.pegtop.net/2006/05/borg-in-c-delphi.html)...