[Boost-users] Boost::shared::memory consistency
Josmon Paul
I have a doubt regarding boost shared memory. If i am reading data from my shared memory and in between if some other program deletes or overwrites the same shared memory then how it will affect the reading. Will it throw segmentation fault ? I am expecting a segmentation fault but when i wrote a small program to read the memory and in the same time i deleted the same memory through another program using the function (shared_memory_object::remove ()) the behavior is like, it gave the content which was already present in the old shared memory and exited without segmentation fault. I would be verymuch grateful if someone tells the exact behavior in this situation
Josmon Paul
Mailto: josmo...@tcs.com
Website: http://www.tcs.com
____________________________________________
Experience certainty. IT Services
Business Solutions
Outsourcing
____________________________________________
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you
Simonson, Lucanus J
...
>I would be verymuch grateful if someone tells the exact behavior in this situation
The exact behavior is "undefined".
Regards,
Luke
_______________________________________________
Boost-users mailing list
Boost...@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Ion Gaztañaga
> Hi All
> I have a doubt regarding boost shared memory. If i am reading data from
> my shared memory and in between if some other program deletes or
> overwrites the same shared memory then how it will affect the reading.
> Will it throw segmentation fault ? I am expecting a segmentation fault
> but when i wrote a small program to read the memory and in the same time
> i deleted the same memory through another program using the function
> (shared_memory_object::remove ()) the behavior is like, it gave the
> content which was already present in the old shared memory and exited
> without segmentation fault. I would be verymuch grateful if someone
> tells the exact behavior in this situation
POSIX semantics: just like a UNIX unlink call when anotherprocess is
reading a file. The already attached program continues using the old
file. the file is destroyed when the last attached program closes the
file/shm.
Ion
Josmon Paul
I was able to construct the vector of strings using basic_string. But my problem is not exactly that . I want to store objects in vector list and the member of this object are string type. When i accessing this object from shared memory, i need to get this string also. So is there any solution for this other than adding strings directly to the vector list.
Josmon Paul
____________________________________________
-----boost-use...@lists.boost.org wrote: -----
To: Boost User List <boost...@lists.boost.org>
From: Ion Gaztañaga
Sent by: boost-use...@lists.boost.org
Date: 12/30/2011 04:25AM
Subject: Re: [Boost-users] Boost::shared::memory consistency
Ion Gaztañaga
> Hi
> I was able to construct the vector of strings using basic_string. But my
> problem is not exactly that . I want to store objects in vector list and
> the member of this object are string type. When i accessing this object
> from shared memory, i need to get this string also. So is there any
> solution for this other than adding strings directly to the vector list.
Sorry, but I could not understand your question. Can you add more details?
Josmon Paul
I have a class whose private members are string type of data. Each time i will create a new instance of my class and to each instance i will set some string type of data to it . After this i have to push this object it to the vector.
Now i have a function called write_data which will take this vector list as input and will iterate through each vector element and push to shared memory vector . Now from this shared memory i have to iterate these objects and i have to get the string data present in these instance. Right now i am declaring this string type of data as boost::interprocess::string and will set this data to the instance of the class. But it will create segmentation fault when i am accessing this string type of data from the object when the size of string is more than 10. So is there any method to increase the buffer size of boost::interprocess::string .?
Regards
Josmon
________________________________
-----boost-use...@lists.boost.org wrote: -----
To: Boost User List <boost...@lists.boost.org>
From: Ion Gaztañaga
Sent by: boost-use...@lists.boost.org
Date: 01/02/2012 05:23PM
Subject: Re: [Boost-users] Boost::shared::memory consistency
Ion Gaztañaga
> Hi Ion,
> I have a class whose private members are string type of data. Each time
> i will create a new instance of my class and to each instance i will set
> some string type of data to it . After this i have to push this object
> it to the vector.
> Now i have a function called write_data which will take this vector list
> as input and will iterate through each vector element and push to shared
> memory vector . Now from this shared memory i have to iterate these
> objects and i have to get the string data present in these instance.
> Right now i am declaring this string type of data as
> boost::interprocess::string and will set this data to the instance of
> the class. But it will create segmentation fault when i am accessing
> this string type of data from the object when the size of string is more
> than 10. So is there any method to increase the buffer size of
> boost::interprocess::string .?
boost::interprocess::string is a normal string (indeed, from Boost 1.48
is a typedef for boost::container::string). To put it shared memory you
will need to use a shared memory allocator: you must use
boost::interprocess::basic_string<char, std::char_traits<char>,
boost::interprocess::allocator<char> >.
To help you with more details, I would need a compilable cpp example.
Anthony Foiani
Josmon --
I'd hoped to be able to give you a more complete example, but I ran
out of time.
I have a working program at the bottom of this message; here's a bit
of text I was trying to keep in sync with the code.
------------------------------------------------------------------------
I think that the important observation is that a string which works in
a snared memory container does not have the same C++ type as a
"std::string". The interfaces are similar (probably identical), but
they have different template arguments; specifically, strings used in
shared memory must have an allocator that pulls from that shared
memory.
I learned about this when I was trying to use strings as keys in a
shared map. I'll try to provide a quick example below, but there are
relevant (if somewhat terse) examples in the Boost.Interprocess
documentation:
(or here, if that wraps: http://preview.tinyurl.com/7cmb76n )
Another link:
http://en.highscore.de/cpp/boost/interprocesscommunication.html#interprocesscommunication_managed_shared_memory
(or: http://preview.tinyurl.com/6tpg9dm )
Josmon Paul <josmo...@tcs.com> writes:
> I have a class whose private members are string type of data. Each
> time i will create a new instance of my class and to each instance i
> will set some string type of data to it . After this i have to push
> this object it to the vector.
So we have a simple local memory object:
typedef std::string local_string_t;
struct local_info_t
{
local_info_t( const local_string_t & str ) : m_str( str ) {}
local_string_t m_str;
};
And a corresponding vector of them:
typedef std::vector< local_info_t > local_info_vector_t;
Which you fill by doing something like:
local_info_vector_t local_vec;
local_vec.push_back( local_info_t( "foo" ) );
local_vec.push_back( local_info_t( "bar" ) );
local_vec.push_back( local_info_t( "baz" ) );
Good so far?
> Now i have a function called write_data which will take this vector
> list as input and will iterate through each vector element and push
> to shared memory vector.
Ok, so now we have our shared structures (with some bits left
undeclared until later):
struct shared_info_t
{
// shared_info_t constructor, version 1
shared_info_t( const local_info_t & loc ) : m_str( loc.m_str ) {}
shared_string_t m_str; // mystery type 1
};
shared_info_vector_t shared_vec; // mystery type 2
So what are are the actual C++ types of "shared_string_t" and
"shared_info_vector_t"?
Both std::vectors and std::strings need to allocate "bookkeeping"
space (capacity, length) as well as actual data-holding space. In the
case of shared memory, we need to make sure that the bookkeeping space
is in the shared memory segment as well as the contents.
This is where Ion's previous comments about allocators is relevant.
(I think they were to you, but I'm not sure -- they were within the
last week or so.)
I'm not an expert, so I'll just try to transcribe what I did to get
something similar working. (I needed a shared map, not a shared
vector, but the overall outline is very similar.)
First, we need something that will manage the shared memory itself:
namespace bi = boost::interprocess;
typedef bi::managed_shared_memory
managed_shared_memory_t;
Next, we need an allocator that can give us memory for holding
characters:
typedef bi::allocator<
char,
shared_manager_t::segment_manager
> shared_char_allocator_t;
This is enough for us to define strings which can be safely stored in
shared memory:
typedef bi::basic_string<
char,
std::char_traits< char >,
shared_char_allocator_t
> shared_string_t;
The interprocess vectors need to know how to allocate strings in
shared memory, so we need another allocator (one for shared_strings,
instead of just characters in shared memory):
typedef bi::allocator<
shared_info_t,
shared_manager_t::segment_manager
> shared_info_allocator_t;
typedef bi::vector<
shared_info_t,
shared_info_allocator_t
> shared_info_vector_t;
Whew.
Finally, it takes a bit of magic to make sure that the shared bits are
actually placed in shared storage. (This is all well-described at the
documentation link given above.) Within a segment manager, individual
objects are referred to by name, and they must be either found or
constructed.
So the call site looks like so:
// create the shared memory area
managed_shared_memory_t managed_shared_memory(
create_only, // throw if already exists
"my_shared_memory", // unique name for shared memory block
10000 // size in bytes
);
// get a pointer to the manager (just to save typing)
managed_shared_memory_t::segment_manager * shared_manager_p =
managed_shared_memory.get_segment_manager();
// now create allocators
shared_char_allocator_t shared_char_allocator( shared_manager_p );
shared_info_allocator_t shared_info_allocator( shared_manager_p );
// get a pointer to the shared vector
shared_info_vector_t * shared_vec_p =
shared_manager_p->find_or_construct< shared_info_vector_t >
( "my_shared_vector" ) // name of vector in shared memory
( shared_info_allocator ); // vector constructor args
// finally, copy from local to shared storage
BOOST_FOREACH( const local_info_t & i, local_vec )
shared_vec->push_back( shared_info_t( i ) );
> Now from this shared memory i have to iterate these objects and i
> have to get the string data present in these instance. Right now i
> am declaring this string type of data as boost::interprocess::string
> and will set this data to the instance of the class.
To get those values in another process, you first need to get a
pointer to the shared vector. If you're forking the same process that
created it, then that's easy; if this is an unrelated process, you'll
need to do something like this:
// find the previously-created shared memory area
managed_shared_memory_t managed_shared_memory(
open_only, // throw if it doesn't already exist
"my_shared_memory" // same name as used in creation
);
// find the already-created shared vector by name
shared_info_vector_t * shared_vec_p =
shared_manager.find< shared_info_vector_t >
( "my_shared_vector" );
// and now iterate over the shared vector
BOOST_FOREACH( const shared_info_t & i, *shared_vec_p )
cout << i.m_str << endl;
> But it will create segmentation fault when i am accessing this
> string type of data from the object when the size of string is more
> than 10. So is there any method to increase the buffer size of
> boost::interprocess::string .?
You have the correct cause, but increasing the internal buffer isn't a
good answer (and, honestly, I don't know if it's possible at all).
The segmentation fault is occurring because the string living in
shared memory tries to allocate an outside-of-object buffer. Without
the correct allocator, that buffer will be in memory local to the
writing process; however, the pointer value is written to shared
memory. When another process tries to use that pointer, it's (at
best) pointing to non-mapped memory and giving you yours seg-fault.
(At worst, it's pointing at valuable information that might get
overwritten or cause nearly-endless loops, etc.)
The more correct answer is to make sure that all the allocations are
done from shared memory.
------------------------------------------------------------------------
And the program follows my signature. I'm sure there are things that
could be fixed, but this was intended to be a quick demo...
Best regards,
Anthony Foiani
------------------------------------------------------------------------
// linux compile line:
// g++ -o shared-vec shared-vec.cpp -lpthread -lrt
#include <iostream>
#include <string>
#include <vector>
#include <boost/foreach.hpp>
#include <boost/interprocess/containers/string.hpp>
#include <boost/interprocess/containers/vector.hpp>
#include <boost/interprocess/managed_shared_memory.hpp>
// =====================================================================
// local types
typedef std::string local_string_t;
struct local_info_t
{
local_info_t( const local_string_t & str ) : m_str( str ) {}
local_string_t m_str;
};
typedef std::vector< local_info_t > local_info_vector_t;
// =====================================================================
// shared types
namespace bi = boost::interprocess;
typedef bi::managed_shared_memory
managed_shared_memory_t;
typedef bi::allocator<
char,
managed_shared_memory_t::segment_manager
> shared_char_allocator_t;
typedef bi::basic_string<
char,
std::char_traits< char >,
shared_char_allocator_t
> shared_string_t;
inline local_string_t
local_from_shared_string( const shared_string_t & ss )
{
return local_string_t( ss.begin(), ss.end() );
}
struct shared_info_t // version 2
{
shared_info_t( const local_info_t & loc,
shared_char_allocator_t & char_alloc )
: m_str( loc.m_str.begin(), loc.m_str.end(), char_alloc ) {}
shared_string_t m_str;
shared_info_t & operator=( const shared_info_t & rhs )
{
m_str = rhs.m_str;
}
};
typedef bi::allocator<
shared_info_t,
managed_shared_memory_t::segment_manager
> shared_info_allocator_t;
typedef bi::vector<
shared_info_t,
shared_info_allocator_t
> shared_info_vector_t;
// =====================================================================
int main( int argc, char * argv [] )
{
const std::string cmd( argc > 1 ? argv[1] : "show" );
// get some shared memory
managed_shared_memory_t managed_shared_memory(
bi::open_or_create, "my_shared_memory", 10000 /* bytes */
);
// get a pointer to the manager (just to save typing)
managed_shared_memory_t::segment_manager * shared_manager_p =
managed_shared_memory.get_segment_manager();
// construct our allocators
shared_char_allocator_t shared_char_allocator( shared_manager_p );
shared_info_allocator_t shared_info_allocator( shared_manager_p );
// get a pointer to the shared vector
shared_info_vector_t * shared_vec_p =
shared_manager_p->find_or_construct< shared_info_vector_t >
( "my_shared_vector" ) // name of vector in shared memory
( shared_info_allocator ); // vector constructor args
if ( cmd == "add" )
{
// create a vector of info objects in process-local storage
local_info_vector_t local_vec;
if ( argc > 2 )
{
for ( int i = 2; i < argc; ++i )
local_vec.push_back( local_info_t( argv[i] ) );
}
else
{
local_vec.push_back( local_info_t( "foo" ) );
local_vec.push_back( local_info_t( "bar" ) );
local_vec.push_back( local_info_t( "baz" ) );
}
// copy from local storage to shared storage
BOOST_FOREACH( const local_info_t & i, local_vec )
shared_vec_p->push_back( shared_info_t( i, shared_char_allocator ) );
}
else if ( cmd == "show" )
{
// use data from shared storage directly
for ( int i = 0; i < shared_vec_p->size(); ++i )
std::cout << i << ". " << shared_vec_p->at( i ).m_str << std::endl;
}
else if ( cmd == "slurp" )
{
// copy from shared to local
local_info_vector_t local_vec;
BOOST_FOREACH( const shared_info_t & i, *shared_vec_p )
local_vec.push_back( local_info_t( local_from_shared_string( i.m_str ) ) );
// and display it out of local memory
for ( int i = 0; i < local_vec.size(); ++i )
std::cout << i << ". " << local_vec.at( i ).m_str << std::endl;
}
else if ( cmd == "clear" )
{
shared_vec_p->clear();
}
else if ( cmd == "destroy" )
{
shared_manager_p->destroy_ptr( shared_vec_p );
bi::shared_memory_object::remove( "my_shared_memory" );
}
else
{
using std::clog;
using std::endl;
clog << "usage: " << argv[0] << " COMMAND" << endl
<< "commands:" << endl
<< " add - insert values into shared vector" << endl
<< " show - list values in shared vector" << endl
<< " slurp - copy from shared to local, then show" << endl
<< " clear - empty shared vector" << endl
<< " destroy - remove shared memory" << endl;
if ( cmd != "help" && cmd != "?" )
{
clog << argv[0] << ": unknown command: '" << cmd << "'" << endl;
return 1;
}
else
{
return 0;
}
}
return 0;
Josmon Paul
Thanks for your reply. I will check into this and will let you know immediately
Regards
Josmon Paul
____________________________________________
-----boost-use...@lists.boost.org wrote: -----
To: boost...@lists.boost.org
From: Anthony Foiani
Sent by: boost-use...@lists.boost.org
Date: 01/06/2012 09:04AM
Subject: Re: [Boost-users] Boost::interprocess::string