[boost] [pool] Objects of varying sizes

[Robert Dailey]

Is it possible to create a pool for polymorphic types? For example:

class A;

class B : public A;
class C : public A;

I want to create a pool of type A, but the allocation can be for B or C at
any time. Will this work?

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Sylvain Bougerel]

> Is it possible to create a pool for polymorphic types? For example:
>
> class A;
>
> class B : public A;
> class C : public A;
>

Well, if class B and C have additional members (if they are not empty
classes) you will need to use a memory pool that is capable of allocating
varying amount of contiguous memory. I don't see why this is not possible.

Have you looked at Boost.Pool? It uses a free list and allocate varying
amounts of chuncks of memory, based on your request. If you choose an
appropriate size for the small chunk, then you may be able to dynamically
allocate without wasting too much space.

Cheers,
Sylvain

[Robert Dailey]

On Fri, May 4, 2012 at 6:44 PM, Sylvain Bougerel <
sylvain.bou...@gmail.com> wrote:

> > Is it possible to create a pool for polymorphic types? For example:
> >
> > class A;
> >
> > class B : public A;
> > class C : public A;
> >
>
> Well, if class B and C have additional members (if they are not empty
> classes) you will need to use a memory pool that is capable of allocating
> varying amount of contiguous memory. I don't see why this is not possible.
>
> Have you looked at Boost.Pool? It uses a free list and allocate varying
> amounts of chuncks of memory, based on your request. If you choose an
> appropriate size for the small chunk, then you may be able to dynamically
> allocate without wasting too much space.

My post was actually in regards to Boost.Pool. I don't see how to use
Boost.Pool for objects of varying sizes. I was looking at the object pool,
which accepts a type, but it wouldn't know that I require varying sizes
just by looking at 1 type. Each subclass would ideally have more data
members, so the sizes would not be the same depending on which concrete
type I'm allocating.

Could you provide an example if it's not too much trouble? Thanks!

[Robert Dailey]

Any advice for me on this?

[Klaim - Joël Lamotte]

Hi,

don't look at the object_pool, look at pool. It provide only allocation and
destruction methods, given a size at runtime.
That way you can wrap it with a template function that give the size of the
object you want to create to the pool for it to allocate memory for you.

Joel Lamotte

[Robert Dailey]

On Mon, Jun 25, 2012 at 6:48 PM, Klaim - Joël Lamotte <mjk...@gmail.com>wrote:

> Hi,
>
> don't look at the object_pool, look at pool. It provide only allocation and
> destruction methods, given a size at runtime.
> That way you can wrap it with a template function that give the size of the
> object you want to create to the pool for it to allocate memory for you.

Thanks for the response. I am reviewing the boost.pool reference here:
http://www.boost.org/doc/libs/1_49_0/libs/pool/doc/html/boost/pool.html

Based on that, I do not see a way to allocate blocks of varying sizes. The
documentation makes it sound like I have to set size_type to the largest
object that will be allocated, which I have no reasonable way of knowing (I
have hundreds of objects in my inheritance hierarchy, each with varying
size, and no way to calculate the largest in a maintainable way).

The malloc() method does not take any parameters.

Am I misunderstanding you? Could you please point me in the right
direction? Thanks again for your help.

[Nikolay Mladenov]

1) size_type is just a type, and it should be able to hold the eventual
size of the biggest requested object

if size_type == boost::uint32_t you can request to allocate sizes of up
to 2^32 - 1

size_type == size_t is probably ok for general use.

2) the first parameter in the pools constructor is the *chunk size*, which
is the smallest amount of memory you can allocate
and defines the granularity of your pool.

3) ordered_malloc(n) allows you to allocate a continuous array of n chunks.
I think you have to free those with ordered_free.


HTH

[Robert Dailey]

On Tue, Jun 26, 2012 at 11:42 AM, Nikolay Mladenov <
nikolay....@gmail.com> wrote:

> 1) size_type is just a type, and it should be able to hold the eventual
> size of the biggest requested object
>
> if size_type == boost::uint32_t you can request to allocate sizes of up
> to 2^32 - 1
>
> size_type == size_t is probably ok for general use.
>
> 2) the first parameter in the pools constructor is the *chunk size*, which
> is the smallest amount of memory you can allocate
> and defines the granularity of your pool.
>
> 3) ordered_malloc(n) allows you to allocate a continuous array of n chunks.
> I think you have to free those with ordered_free.

Ah I see now, thank you for this explanation!

[Cory Nelson]

On Fri, May 4, 2012 at 2:14 PM, Robert Dailey <rcdaile...@gmail.com>wrote:

> Is it possible to create a pool for polymorphic types? For example:
>
> class A;
>
> class B : public A;
> class C : public A;
>
> I want to create a pool of type A, but the allocation can be for B or C at
> any time. Will this work?
>

For performance reasons, you'll only be able to allocate objects of a fixed
size. You might try a pool of variant<B,C> instead.

For a variable-sized pool, as an implementation detail most runtime
environments do implement a variable-sized pool behind the scenes for
malloc/new.

--
Cory Nelson
http://int64.org

[Robert Dailey]

On Tue, Jun 26, 2012 at 12:48 PM, Cory Nelson <phr...@gmail.com> wrote:

> On Fri, May 4, 2012 at 2:14 PM, Robert Dailey <rcdaile...@gmail.com
> >wrote:
>
> > Is it possible to create a pool for polymorphic types? For example:
> >
> > class A;
> >
> > class B : public A;
> > class C : public A;
> >
> > I want to create a pool of type A, but the allocation can be for B or C
> at
> > any time. Will this work?
> >
>
> For performance reasons, you'll only be able to allocate objects of a fixed
> size. You might try a pool of variant<B,C> instead.
>
> For a variable-sized pool, as an implementation detail most runtime
> environments do implement a variable-sized pool behind the scenes for
> malloc/new.

So are you saying it isn't beneficial or practical to use boost::pool to
allocate chunks of varying sizes, and that the Windows Memory Manager is
better suited?

[Alex Perry]

Robert Dailey on 26 June 2012 19:03 wrote :

>So are you saying it isn't beneficial or practical to use boost::pool to allocate chunks of varying sizes, and that the Windows Memory Manager is better suited?

For me the major benefit of an arena allocator like boost:pool is to avoid memory fragmentation which can happen with the general purpose (variable sized) heap allocations (are provided by compiler ie gcc or VC etc AFAIK rather than using Windows Memory Manager directly).

Ie you know that all these objects are probably sharing some lifetime - avoid too much fragmentation by allocating them from some easily reused larger block by some mechanism such as boost pool.

If you have different sized objects then calculate the smallest "sensible" size for these allocations and set the pool up with this granularity and then allocate as many of these "grains" as you need for each object.

You still get the separation from the general purpose heap (and also a fast destruct of the entire pool without calling individual destructors if required which can be a win in some scenarios_

HTH

Alex

[Rowan James]

On 27 Jun 2012, at 02:19, Robert Dailey wrote:
> On Mon, Jun 25, 2012 at 6:48 PM, Klaim - Joël Lamotte <mjk...@gmail.com>wrote:
>
>> Hi,
>>
>> don't look at the object_pool, look at pool. It provide only allocation and
>> destruction methods, given a size at runtime.
>> That way you can wrap it with a template function that give the size of the
>> object you want to create to the pool for it to allocate memory for you.
>
>
> Thanks for the response. I am reviewing the boost.pool reference here:
> http://www.boost.org/doc/libs/1_49_0/libs/pool/doc/html/boost/pool.html
>
> Based on that, I do not see a way to allocate blocks of varying sizes. The
> documentation makes it sound like I have to set size_type to the largest
> object that will be allocated, which I have no reasonable way of knowing (I
> have hundreds of objects in my inheritance hierarchy, each with varying
> size, and no way to calculate the largest in a maintainable way).
>
> The malloc() method does not take any parameters.
>
> Am I misunderstanding you? Could you please point me in the right
> direction? Thanks again for your help.
>

Sorry for the late response, I've created a couple of code snippets that seem to do something like what I understand you're after.

My first reaction on hearing your question was to create a distinct boost::object_pool instance for each class type. Whenever you construct an object of a given type, just use the pool for that object's construct method:

https://gist.github.com/3004098

This strategy is straightforward, scalable, and can be integrated fairly readily with varying amounts of intrusiveness into the class hierarchy - but will require at least some level of coordination from the class authors or an exhaustive list of classes at some other point. Adding them to a namespace can be quite simple, and done in each class' .hpp file such that whenever you can construct an Example, you can call

MyPools::Example.construct();

--

Secondly, and more directly along these lines of discussion; the pool_allocator (i.e. std::allocator) implementation provides:
static pointer allocate(size_type n);

… which could be used with placement new to wrap your construction however you liked without being intrusive into the constructed class.

https://gist.github.com/3004102

--

Of course, I'm not handling the cleanup stage of this; which I expect is a very important consideration. I hope this helps :-)

Regards,
Rowan James