[Boost-users] multi_index reserve but allocate
Olivier Tristan via Boost-users
Looks like that even if I call reserve on a multi_index, it stills
allocate memory while I am still below the reserve size.
Is this the intended behavior ?
If so, what is the best way to avoid allocation in a fixed sized multi_index
Thanks !
--
Olivier Tristan
Research & Development
www.uvi.net
_______________________________________________
Boost-users mailing list
Boost...@lists.boost.org
https://lists.boost.org/mailman/listinfo.cgi/boost-users
Joaquin M López Muñoz via Boost-users
> Hi,
Hi Olivier,
> Looks like that even if I call reserve on a multi_index, it stills
> allocate memory while I am still below
> the reserve size.
>
> Is this the intended behavior ?
Yes it is. multi_index_container is node based, and, like std node-based
containers (std::list, std::set,
std::unordered_set, etc.), will allocate memory for its nodes as needed.
Hashed and random-access indices provide reserve() member functions
(with non-equivalent
semantics) that control the size of internal contiguous memory chunks
maintained by these indices
additionally to the nodes. So, reserve() (when available) does not (nor
is designed to) guarantee
that no additional memory will be allocated on further insertions.
> If so, what is the best way to avoid allocation in a fixed sized
> multi_index
You can write your own allocator that allocates the memory beforehand
and then serves it
piece by piece on allocate(). I suspect, however, that this is not going
to be faster than using
the default std::allocator.
> Thanks !
Best,
Joaquín M López Muñoz
Olivier Tristan via Boost-users
My goal is not to be fast but to avoid allocation as I am in a high
priority thread and I want to avoid possible allocation lock.
I tried the allocator solution following a thread on stackoverflow
(where you answered as well)
using something like that
boost::pool_allocator<ActiveNote,
boost::default_user_allocator_new_delete,
boost::details::pool::default_mutex, 256U, 0U>>;
My issue with this solution is that the pool is actually shared across
all object using a boost::pool_allocator that requires a mutex as there
are multiple thread in my app.
One solution could be to tag the allocator but this seems to be not
possible anymore unless I am mistaken
Is there a way otherwise to get a pool for each instances of my
multi_index ?
Thanks !
--
Olivier Tristan
Research & Development
www.uvi.net
_______________________________________________
Joaquin M López Muñoz via Boost-users
> Hi Joaquin,
Hi Olivier,
Please don't top-post:
https://www.boost.org/community/policy.html#quoting
> My goal is not to be fast but to avoid allocation as I am in a high
> priority thread and I want
> to avoid possible allocation lock.
>
> I tried the allocator [...] using something like that
>
> boost::pool_allocator<
> ActiveNote, boost::default_user_allocator_new_delete,
> boost::details::pool::default_mutex, 256U, 0U>>;
>
> My issue with this solution is that the pool is actually shared across
> all object
> using a boost::pool_allocator that requires a mutex as there are
> multiple thread in my app.
Maybe you can give this allocator a try:
https://probablydance.com/2014/11/09/plalloc-a-simple-stateful-allocator-for-node-based-containers/
It works with Boost.MultiIndex and does not use any mutex at all
(because it's stateful and
as such instances are owned by the containers using them).
Olivier Tristan via Boost-users
Le 20/03/2019 à 11:18, Joaquin M López Muñoz via Boost-users a écrit :
> Maybe you can give this allocator a try:
>
> https://probablydance.com/2014/11/09/plalloc-a-simple-stateful-allocator-for-node-based-containers/
>
>
> It works with Boost.MultiIndex and does not use any mutex at all
> (because it's stateful and
> as such instances are owned by the containers using them).
>
allocator that actually compiled (last issue was related to swap)
This is a great example to start from.
In the meantime, I've found as well info regarding
boost::container::node_allocator but the memory is shared across
instances and do not allow preallocation.
Didn't found any other out of the box solution in boost though
so I've ended up modifying the example.
Still I need to do a reserve on multi_index to force the creation of all
the allocator
Thanks a lot !
For those interested, here is what I end up with
template <typename T>
struct PoolAlloc
{
typedef T value_type;
PoolAlloc(size_t reserveSize)
: reservedSize(reserveSize)
{
reserve(reserveSize);
}
template <typename U>
PoolAlloc(const PoolAlloc<U>& other)
: reservedSize(other.reservedSize)
{
reserve(reservedSize);
}
PoolAlloc(const PoolAlloc&) = delete;
PoolAlloc& operator=(const PoolAlloc&) = delete;
PoolAlloc(PoolAlloc&&) = default;
PoolAlloc& operator=(PoolAlloc&&) = default;
typedef std::true_type propagate_on_container_copy_assignment;
typedef std::true_type propagate_on_container_move_assignment;
typedef std::true_type propagate_on_container_swap;
void reserve(size_t to_allocate)
{
available.reserve(to_allocate);
std::unique_ptr<value_holder[]> allocated(new
value_holder[to_allocate]);
value_holder* first_new = allocated.get();
memory.emplace_back(std::move(allocated));
for (size_t i = 0; i < to_allocate; ++i)
{
available.push_back(std::addressof(first_new[i].value));
}
}
bool operator==(const PoolAlloc& other) const
{
return this == &other;
}
bool operator!=(const PoolAlloc& other) const
{
return !(*this == other);
}
T* allocate(size_t num_to_allocate)
{
if (num_to_allocate != 1)
{
return static_cast<T*>(::operator new(sizeof(T) *
num_to_allocate));
}
else
{
if (available.empty())
{
// first allocate 8, then double whenever
// we run out of memory
size_t to_allocate = 8 << memory.size();
reserve(to_allocate);
}
T* result = available.back();
available.pop_back();
return result;
}
}
void deallocate(T* ptr, size_t num_to_free)
{
if (num_to_free == 1)
{
available.push_back(ptr);
}
else
{
::operator delete(ptr);
}
}
// boilerplate that shouldn't be needed, except
// libstdc++ doesn't use allocator_traits yet
template <typename U>
struct rebind
{
typedef PoolAlloc<U> other;
};
typedef T* pointer;
typedef const T* const_pointer;
typedef T& reference;
typedef const T& const_reference;
template <typename U, typename... Args>
void construct(U* object, Args&&... args)
{
new (object) U(std::forward<Args>(args)...);
}
template <typename U, typename... Args>
void construct(const U* object, Args&&... args) = delete;
template <typename U>
void destroy(U* object)
{
object->~U();
}
union value_holder
{
value_holder() {}
~value_holder() {}
T value;
};
size_t reservedSize;
std::vector<std::unique_ptr<value_holder[]>> memory;
std::vector<T*> available;
};
--
Olivier Tristan
Research & Development
www.uvi.net
_______________________________________________