Does someone know about such a library? In case there is nothing like
this
available and I have to write my own, does someone have a good idea
about
how the interface to the lib could look like?
regards
Torsten
> I'm looking for a portable library with functions for atomical
> increment/reads decrement/reads for c/c++ to implement reference
> counting smartpointers in c++.
>
> Does someone know about such a library?
You can find something like in the task portions of ZThreads. It uses
atomic reads & write for reference counting on some systems, and mutexes
for the others. That would probably give you a good starting point for
creating a full smart pointer.
- Eric
http://www.code-foo.com/
I would recommend boost at www.boost.org
Contains boost::shared_ptr which is an reference counted portable smart
pointer with atomic counts.
/Michel
Naked "atomic counts" (w/o *memory synchronization* protocol
ala POSIX mutexes) don't work for *MUTABLE* objects, unless
you acquire the right lock in the destructor as well, which
is rather silly.
http://groups.yahoo.com/group/Boost-Users/message/278
And, perhaps:
http://lists.boost.org/MailArchives/boost/msg30364.php
"....
> > What should we do?
>
> Well, I guess...
>
> Bombard-to-death ;-) TOG/Austin Group reflector[1] (MS
> IS the 'silver' TOG member[2] as well ;-)) with zillions
> of messages requesting new opaque type pthread_refcount_t
> (or something like that) plus 'count'-calls/macros...
> with perhaps even more 'optimized' [for immutable objects
> only] counterparts -- operations at least; not sure w.r.t
> the separate extra opaque type -- could be the same one,
> perhaps.
...."
regards,
alexander.
thanks, that gave me indeed a good starting point.
regards
Torsten
Jeff
"Torsten Robitzki" <Tor...@Robitzki.de> wrote in message
news:3DA7208F...@Robitzki.de...
Alexander Terekhov wrote:
>
> "Michel André" wrote:
> >
> > > > I'm looking for a portable library with functions for atomical
> > > > increment/reads decrement/reads for c/c++ to implement reference
> > > > counting smartpointers in c++.
> > > >
> > > > Does someone know about such a library?
> >
> > I would recommend boost at www.boost.org
> > Contains boost::shared_ptr which is an reference counted portable smart
> > pointer with atomic counts.
>
> Naked "atomic counts" (w/o *memory synchronization* protocol
> ala POSIX mutexes) don't work for *MUTABLE* objects, unless
> you acquire the right lock in the destructor as well, which
> is rather silly.
>
> http://groups.yahoo.com/group/Boost-Users/message/278
>
> And, perhaps:
>
> http://lists.boost.org/MailArchives/boost/msg30364.php
>
Actually, I did a "smart pointer" patent while at IBM which takes care of a lot
of the problems with "smart pointers" though you still need a write lock for
the destructor but you are after all changing the object in a rather radical way.
IBM let the patent expire. IANAL, but I believe this would put it in the public
domain. Some disclaimers. Patent attorneys didn't have a lot software experience
back then. It was extremely difficult to explain software to them and it produced
a lot of badly worded and difficult to understand patents. The water tank analogy
was supposed to explain the idea to the patent attorney and wasn't supposed to be
part of the patent application, the attorney insisted in putting it in. Also I
wasn't around for the final part of the patent application, so the claims were really
screwed up rather badly. As far as I can tell, the patent office may have though
I was trying to patent linked lists, they cited the algorithm book by Sedgewick.
There also wasn't a lot of software experience in the PTO at the time either. Anyway,
the munged claims might explain why Microsoft got 2 patents on a subcase of the stuff
I invented.
Patent number is 5,295,262. http://www.uspto.gov/ for gory details.
Basically, the idea is keep the "reference count" with the pointer. You atomically
increment the counter at the same time you fetch the reference value. Compare and
swap would be a good candidate if it will handle a count and pointer. This takes
care the nasty race condition usually associated with reference counting.
Reference counts can only be incremented. To drop a reference, you need to reference
another object via a "smart pointer" contained in the current object. The other object
could be null essentially making that "smart pointer" just a counter.
To remove a reference to an object, you atomically swap the pointer to it and transfer
the old reference count to a counter in the object.
To deallocate an object, you make it unreachable (remove all references to it) and
wait for the references to the object to become equal to the references from the object.
The counts can wrap a long as the number of active references is less than, I think, half
the max count value.
A simple case is just a single node object. To access the object, you increment the
"to" smart pointer to the object. To drop access, you increment the "from" smart
pointer, just a null pointer so it can devolve to just a count. To deallocate the
object, you remove or change the smart pointer to it, transfer the "to" reference
count to the object (there are different ways you could implement this), and wait
for the "from" references to become equal to the "to" references. This appears to
be what Microsoft seems to have patented. I only took a quick look at at their
patents though, but I don't think they have a valid claim naturally.
Joe Seigh
Joe Seigh
Yes, of course.
regards,
alexander.
Apparantly defining BOOST_NO_THREADING (or something like that) kills off
the reference counting, although I wonder what else I broke :)
(well, at least it teaches you to never upgrade anything until you really
need it)
--
Arnold Hendriks <a.hen...@b-lex.com>
B-Lex Information Technologies, http://www.b-lex.com/
Arnold Hendriks wrote:
>
> Joseph Seigh <jsei...@xemaps.com> wrote:
> > Hmm... I have to get into C++ sometime. Would there be any interest
> > in smart pointers that would actually work?
> Define 'work'. I personally got pretty annoyed with boost 'suddenly' making
> all their shared pointers thread-safe - I already dealt with the proper
> locking myself, only at those places where necessary, and the extra
> overhead caused by boost's shared_ptrs locking everywhere caused
> quite a noticable speed degradation.
>
"work" in the sense of not having the problem that you have when keeping the
reference count in object, how do you ensure that the object stays itself
long enough to increment the count safely. There are ways of dealing with
it of course but usually it involves some sort of global lock which obviates
some of the benefits of threading.
I'm going to do it anyway as an exercise as it will touch on a bunch of issues
in programming in C++.
Joe Seigh
(untested code)
shared_ptr.h
---------
#include "refcounter.h"
template <typename T> class SharedPtr
{
T* ptr;
RefCounter* rc;
public:
explicit SharedPtr(T* p) : ptr(p), rc(new RefCounter) {}
SharedPtr(const SharedPtr& rhs) : ptr(rhs.ptr), rc(rhs.rc->AddRef()) {}
~SharedPtr() { rc->Release(ptr); }
...
}
-----------
refcounter.h:
#if defined(__ALPHA)
#include "refcounter_alpha.h"
#elif defined(__X86)
#include "refcounter_x86.h"
#elif defined(_POSIX_THREADS)
#include "refcounter_pthread.h"
#endif
--------
refcounter_alpha.h:
class RefCounter
{
unsigned long count;
public:
RefCounter() : count(1) {}
~RefCounter() {}
RefCounter* AddRef()
{
__ATOMIC_INCREMENT_LONG(&count);
return this;
}
template <typename T> void Release(T* ptr)
{
if (__ATOMIC_DECREMENT_LONG(&count) == 1) {
delete(ptr);
delete(this);
}
}
};
"Alexander Terekhov" <tere...@web.de> wrote in message
news:3DA7F14F...@web.de...
>
> "Michel André" wrote:
> >
> > > > I'm looking for a portable library with functions for atomical
> > > > increment/reads decrement/reads for c/c++ to implement reference
> > > > counting smartpointers in c++.
> > > >
> > > > Does someone know about such a library?
> >
> > I would recommend boost at www.boost.org
> > Contains boost::shared_ptr which is an reference counted portable smart
> > pointer with atomic counts.
>
> Naked "atomic counts" (w/o *memory synchronization* protocol
> ala POSIX mutexes) don't work for *MUTABLE* objects, unless
> you acquire the right lock in the destructor as well, which
> is rather silly.
>
hys
--
Hillel Y. Sims
FactSet Research Systems
hsims AT factset.com
- fix (severe lack of) exception-safety in ctor:
>
> (untested code)
> shared_ptr.h
> ---------
> #include "refcounter.h"
#include "scopeguard.h"
>
> template <typename T> class SharedPtr
> {
> T* ptr;
> RefCounter* rc;
static void CleanupPtrOnExc(T* ptr) { delete ptr; }
>
> public:
explicit SharedPtr(T* p) : ptr(p)
{
ScopeGuard sg = MakeScopeGuard(CleanupPtrOnExc, ptr);
rc = new RefCounter;
sg.Dismiss();
}
> SharedPtr(const SharedPtr& rhs) : ptr(rhs.ptr), rc(rhs.rc->AddRef()) {}
> ~SharedPtr() { rc->Release(ptr); }
> ...
> }
>
> -----------
> refcounter.h:
> #if defined(__ALPHA)
> #include "refcounter_alpha.h"
> #elif defined(__X86)
> #include "refcounter_x86.h"
> #elif defined(_POSIX_THREADS)
> #include "refcounter_pthread.h"
> #endif
>
> --------
> refcounter_alpha.h:
>
#include <builtins.h>
;-)
[...]
> #include "scopeguard.h"
^^^^^^^^^^^^^^^^^^^^^^^
Nah, #include <memory> ;-)
> > template <typename T> class SharedPtr
> > {
> > T* ptr;
> > RefCounter* rc;
I'd rather have one single shared "Rep" object ("rep"-ptr here).
> static void CleanupPtrOnExc(T* ptr) { delete ptr; }
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Nah, *NOT* needed! ;-)
> >
> > public:
> explicit SharedPtr(T* p) : ptr(p)
> {
> ScopeGuard sg = MakeScopeGuard(CleanupPtrOnExc, ptr);
> rc = new RefCounter;
> sg.Dismiss();
> }
Nah, why too fancy. ;-) ;-)
explicit SharedPtr(T* ptr_)
std::auto_ptr<T> auto_ptr_(ptr_); // ``simple and terse'' res.manager
rep = new Rep(auto_ptr_); // pass ownership to rep's auto_ptr
}
[...]
> > RefCounter* AddRef()
> > {
> > __ATOMIC_INCREMENT_LONG(&count);
> > return this;
> > }
I'd make it void, but it's okay.
[...]
> > template <typename T> void Release(T* ptr)
> > {
> > if (__ATOMIC_DECREMENT_LONG(&count) == 1) {
> > delete(ptr);
> > delete(this);
> > }
> > }
AFAICS, this IS severely broken in the case of MUTABLE
objects, but it would work quite well for >>IMMUTABLE<<
objects ("== 1" aside ;-) ). You could probably fix it:
if (__ATOMIC_DECREMENT_LONG(&count) == 0) {
__INJECT_ACQUIRE_MEMORY_BARRIER(); // acquire semantics
delete(this); // invokes auto_ptr's d-tor
}
else
__INJECT_RELEASE_MEMORY_BARRIER(); // release semantics
regards,
alexander.
Because, depending on what you think, i.e. what rules you are
following, it's possible to make something even as simplistic as
the stuff referenced here work. But the problem is, nobody has
even stated what those rules are.
The big bugaboo with reference counting in a threaded environment
is how you deal with thread local references which probably aren't
even part of the count (depending on what rules you are using).
Handling thread local references is problematic even with formal
garbage collection, which is why you don't see a lot of concurrant
garbage collection schemes out there. I'm guessing it's why Java
has that weird 2 level access/using scheme, to allow it to determine
what references a thread has.
And I haven't even mentioned the issues of using reference counts
as a read lock, which IMO really isn't what reference counts are for.
Joe Seigh
To me, they simply wrap/"manage" other objects. Here are
just a few links:
< heck, the 1st one below has even "class monitor"! ;-) >
http://www.research.att.com/~bs/wrapper.pdf
(Wrapping C++ Member Function Calls)
http://www.cuj.com/experts/2008/sutter.htm
(The New C++: Smart(er) Pointers)
http://www.informit.com/content/index.asp?session_id={DCA56184-2BA5-426F-A30D-7621C59D04A1}&product_id={7CBDD5B1-129D-427A-9C36-9C506D3DFABA}&t={94AE5B48-1D7D-462A-A4A6-83CE19EC0705}&p={7CBDD5B1-129D-427A-9C36-9C506D3DFABA}
(MC++D, Chapter 7: Smart Pointers )
> I gather that part of it is to get a poor man's version
> of garbage collection, issues with circular references aside.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
< Forward Inline >
-------- Original Message --------
Message-ID: <3DAE893E...@web.de>
Newsgroups: comp.lang.c++
Subject: Re: Resource cleanup in C++
Arnold the Aardvark wrote:
[...]
> >>> A) RAII/ref.counting is way too expensive [think of threading].
>
> Doesn't the GC just have to undertake all the expensive work instead?
Sort of. ;-)
http://www.hpl.hp.com/personal/Hans_Boehm/gc/example.html
(Expensive Explicit Deallocation: An Example)
Note that even with GC one would still need memory barriers
to insure proper visibility [Java's revised volatiles would
work really good here], but GC makes it possible to get rid
of rather expensive locking/ref.counting.
regards,
alexander.
www.informit *SUCKS* [*awful* links, I mean].
http://www.moderncppdesign.com/book/main.html
(Click on "Chapter 7: Smart Pointers" here)
regards,
alexander.
double-dereference per access?
SharedPtr<int> p(new int(69));
*p = 420; // ==> *(*(p.rep).ptr)
>
> [...]
> > > RefCounter* AddRef()
> > > {
> > > __ATOMIC_INCREMENT_LONG(&count);
> > > return this;
> > > }
>
> I'd make it void, but it's okay.
>
syntatic convenience:
> > > SharedPtr(const SharedPtr& rhs) : ptr(rhs.ptr),
rc(rhs.rc->AddRef()) {}
I> [...]
> > > template <typename T> void Release(T* ptr)
> > > {
> > > if (__ATOMIC_DECREMENT_LONG(&count) == 1) {
> > > delete(ptr);
> > > delete(this);
> > > }
> > > }
>
> AFAICS, this IS severely broken in the case of MUTABLE
> objects, but it would work quite well for >>IMMUTABLE<<
> objects ("== 1" aside ;-) ). You could probably fix it:
__ATOMIC_DECREMENT_LONG(&count) returns _previous_ value of count, so check
for ==1 is correct (I forgot to mention this logic in the original post)
>
if (__ATOMIC_DECREMENT_LONG(&count) == 1) {
//> __INJECT_ACQUIRE_MEMORY_BARRIER(); // acquire semantics
__MB();
> delete(this); // invokes auto_ptr's d-tor
> }
> else
//> __INJECT_RELEASE_MEMORY_BARRIER(); // release semantics
__MB();
>
Hmm.. so when thread 2 sees value of count drop to 0 (==1 ;-), without the
memory barriers its view of data pointed to by mutable T* ptr may be out of
synch when it tries to delete? Actually, won't you really need the memory
barrier then even for immutable objects, as there's no guarantee that thread
2 will ever see the initialized data in the correct order otherwise?
(and just to really stir things up, does "unsigned long count" need to be
volatile in this particular case?)
Don't like it? Well,
explicit SharedPtr(T* ptr_) : ptr(ptr_) { // dumb pointer: ``speed vs. size''
std::auto_ptr<T> auto_ptr_(ptr_); // ``simple and terse'' res.manager
rep = new Rep(auto_ptr_); // pass ownership to rep's auto_ptr
}
[...]
> Hmm.. so when thread 2 sees value of count drop to 0 (==1 ;-), without the
> memory barriers its view of data pointed to by mutable T* ptr may be out of
> synch when it tries to delete?
Yep. Trivial/NOOP d-tors would work but non-trivial d-tors would >>fail<<
miserably, unless you manage to somehow acquire the proper lock in a sort
of pre-destructor too [that's rather silly and *deadlock-prone* (recursive
locking aside) to begin with, IMO].
> Actually, won't you really need the memory
> barrier then even for immutable objects, as there's no guarantee that thread
> 2 will ever see the initialized data in the correct order otherwise?
Nope.
http://groups.google.com/groups?threadm=3BC8463D.6E23002A%40web.de
(note that unref() for mutable objects needs to inject "release"
membar too... since threads could unref() mutable objects INSIDE
synchronized regions for mutation/updates)
> (and just to really stir things up, does "unsigned long count" need to be
> volatile in this particular case?)
I don't think so. What for?
regards,
alexander.
But one can pass a copy of a smart pointer to an other thread and
don't have to worry about who have to deleting the object pointed to
when all references are gone.
I think a smart pointers should behave in a threaded context exactly
like normal pointers, except that the reference counter have to be
thread save if it should be allowed to make copies of pointers pointing
to the same object in different threads.
regards
Torsten
I can think of at least 3 different and usefull kinds of reference
counting
smart pointers. All differ from what information is in the body and what
in
the handle (the smart pointer). All have there pro and cons
1) Hillel's suggestion
The body is the referenced object itself. The handle consists of a
reference to the body and a reference to the counter.
pros:
- Works with polymorphical objects
- Most funktions will work without seeing the full declaration of T
cons:
- 3 dynamic allocations needed to create a body plus the handle
- more pointers in the handle thus more dereferencing and copying
needed
2) Alexander's suggestion
The body consists of a pointer to the object of type T and the counter.
The handle only have one pointer (to the body).
pros:
- only 2 dynamic allocations needed
- less dereferencing and copying needed
cons:
- Will not work with polymorphical objects (without casts in the
implementation)
2) What I'm using atm
The body consists of a full copy of the object of type T and the
counter.
The handle have a pointer to the body.
pros:
- only 1 dynamic allocation needed
- space effective for very small objects
cons:
- Will not work with polymorphical objects
Torsten
counter
Ok, that's better, but:
explicit SharedPtr(T* ptr_) : ptr(ptr_) {
std::auto_ptr<T> auto_ptr_(ptr_);
rep = new Rep<T>(auto_ptr_); // Rep<T> for auto_ptr<T>!
}
I think this could generate _slightly_ more 'code bloat' vs Release(ptr)
version, since entire Rep class is now templated instead of just Release()
function... ;-) ;-)
> [...]
> > Hmm.. so when thread 2 sees value of count drop to 0 (==1 ;-), without
the
> > memory barriers its view of data pointed to by mutable T* ptr may be out
of
> > synch when it tries to delete?
>
> Yep. Trivial/NOOP d-tors would work but non-trivial d-tors would >>fail<<
> miserably, unless you manage to somehow acquire the proper lock in a sort
> of pre-destructor too [that's rather silly and *deadlock-prone* (recursive
> locking aside) to begin with, IMO].
>
> > Actually, won't you really need the memory
> > barrier then even for immutable objects, as there's no guarantee that
thread
> > 2 will ever see the initialized data in the correct order otherwise?
>
> Nope.
>
> http://groups.google.com/groups?threadm=3BC8463D.6E23002A%40web.de
So your hypothesis is basically that it is impossible to (correctly) share
the object initially across threads without invoking some pthread_[create |
mutex_lock/unlock]() function(s) that will already guarantee the 'read'
memory synchronization for us (and since the object is immutable we never
need to worry about it again). That makes sense, thanks. (This is especially
useful for COW strings too.)
> (note that unref() for mutable objects needs to inject "release"
> membar too... since threads could unref() mutable objects INSIDE
> synchronized regions for mutation/updates)
:-O!! Fascinating... though it seems it would almost take malicious intent
to release the ref inside the synchronized region (?) Something like:
SharedPtr<int>* pp = new SharedPtr<int>(otherptr);
...
pthread_mutex_lock(&mx);
*(*pp) = 69;
delete pp;
pthread_mutex_unlock(&mx);
I guess? (Why would anyone do this??)
>
> > (and just to really stir things up, does "unsigned long count" need to
be
> > volatile in this particular case?)
>
> I don't think so. What for?
>
I guess not too, just "double-checking". volatile is actually invoked
indirectly, anyhow:
<BUILTINS.H>
------------
extern "C" {
int __ATOMIC_INCREMENT_LONG
(volatile void *__addr);
int __ATOMIC_DECREMENT_LONG
(volatile void *__addr);
}
thanks,
"Hillel Y. Sims" <use...@phatbasset.com> wrote in message
news:G3Mr9.4814$eW3....@news4.srv.hcvlny.cv.net...
>
> "Alexander Terekhov" <tere...@web.de> wrote in message
> news:3DAEE376...@web.de...
> >
> >
> > http://groups.google.com/groups?threadm=3BC8463D.6E23002A%40web.de
>
> So your hypothesis is basically that it is impossible to (correctly) share
> the object initially across threads without invoking some pthread_[create
|
> mutex_lock/unlock]() function(s) that will already guarantee the 'read'
> memory synchronization for us (and since the object is immutable we never
> need to worry about it again). That makes sense, thanks. (This is
especially
> useful for COW strings too.)
>
Hmm, http://groups.google.com/groups?th=572afcb9963e2cdb#link2 :
From: Kaz Kylheku (k...@ashi.footprints.net)
Subject: Re: Lockless/non-blocking ref.counting?
Newsgroups: comp.programming.threads
Date: 2001-10-13 09:13:59 PST
<quote>
I don't think you even need that. You already have a pointer to the
object which you safely obtained at some point. The integrity of the
data structures used by your allocator routines is in the hands of those
routines, which use their own memory barriers as necessary.
If you have the correct pointer, you should be able to trust your
allocator to free the object without having to ensure memory visibility
yourself.
</quote>
???
Well, since allocator just ought to protect its internal
shared structure(s) with some synch, "free" itself would
be perfectly OK... no matter what you do with respect to
"atomic"/non-blocking counting. The concern is visibility
for proper "cleanup" -- object *destruction* [non-trivial
d-tor(s)]!
http://groups.yahoo.com/group/Boost-Users/message/267
Given:
- a mutable shared object OBJ;
- two threads THREAD1 and THREAD2 each holding
a private smart_ptr object pointing to that OBJ.
----
t1: THREAD1 updates OBJ (thread-safe via some synchronization)
and a few cycles later (after "unlock") destroys smart_ptr;
t2: THREAD2 destroys smart_ptr WITHOUT doing any synchronization
with respect to shared mutable object OBJ; OBJ destructors
are called driven by smart_ptr interface...
Hmm... unless you imply that smart_ptr users
should treat smart_ptrs as being thread-unsafe
wrt object cleanup (destructors) and should
somehow always lock/unlock (synchronize)
mutable shared ref.counted objects in their
destructors, the bottom line is:
Atomicity of --/++ is just one of two issues with
respect to correctness of ref. counting in the MT
environment. Visibility of eventual updates for
proper destruction (object clean-up) is another
issue. Currently only lock/unlock could fulfill
both requirements in a portable way.
regards,
alexander.
Okay. Can we settle on this:
explicit SharedPtr(T* ptr_) : ptr(ptr_) {
std::auto_ptr<T> auto_ptr_(ptr_);
rc = new RefCounter;
auto_ptr_.release();
}
<?> ;-) ;-)
[...]
> > (note that unref() for mutable objects needs to inject "release"
> > membar too... since threads could unref() mutable objects INSIDE
> > synchronized regions for mutation/updates)
>
> :-O!! Fascinating... though it seems it would almost take malicious intent
> to release the ref inside the synchronized region (?) Something like:
> SharedPtr<int>* pp = new SharedPtr<int>(otherptr);
*int* has >>trivial<< destructor in C++! ;-)
< ISO/IEC 14882:1998(E), 12.4 Destructors, Pg 193 >
"....
[Note: the notation for explicit call of a destructor can
be used for any scalar type name (5.2.4). Allowing this
makes it possible to write code without having to know if
a destructor exists for a given type. For example,
typedef int I;
I* p;
// ...
p->I::~I();
容nd note]
...."
So, I guess, there should be NO problems with respect to its
visibility and cleanup. Well, ``highly unlikely.'' ;-)
> ...
> pthread_mutex_lock(&mx);
> *(*pp) = 69;
> delete pp;
> pthread_mutex_unlock(&mx);
>
> I guess? (Why would anyone do this??)
Well, why not? Let's modify the scenario from my previous msg:
Given:
- a mutable shared object OBJ;
- two threads THREAD1 and THREAD2 each holding
a private smart_ptr object pointing to that OBJ.
----
t1: THREAD1 updates OBJ (thread-safe via some synchronization)
and a few cycles later (BEFORE "unlock") destroys smart_ptr...
and continues doing some other shared objects updates that are
synchronized via the same lock;
t2: THREAD2 destroys smart_ptr WITHOUT doing any synchronization
with respect to shared mutable object OBJ; OBJ destructors
are called driven by smart_ptr interface...
regards,
alexander.
Ok, now to deal with C++ and find a compare and swap double routine somewhere.
Joe Seigh
Joe Seigh