boost::shared_ptr revisited again
Rafal Dabrowa
===================================== MODERATOR'S COMMENT:
If your message is rejected, the notice is sent to you by the moderation software, not directly from a human. If your address is munged, you won't see it.
===================================== END OF MODERATOR'S COMMENT
I have posted my news about one week ago, it still doesn't appear on the
newsgroup.
I'm not sure, what is reason for that -- I haven't received any answer,
maybe because
of my e-mail address was spammers-protected (although my post contained
signature with
way of decoding my address). I guess that my motivation of my proposal
was not enough strong. So, I want to afford to rewrite my post and
describe my
motivation more detailed.
---------------
Boost library contains smart_ptr classes family, which likely becomes a part
of C++ standard library in future. I want to take a closer look at
shared_ptr
class. This class is very useful in wide range of applications. It
simplifies a
truly polimorphic programming. Containers of polymorphic objects
contain usually pointers to these objects. Replace a pointer by
shared_ptr takes off head pointers deletion. Shared pointers simplify
also maintenance of a data shared among objects. Shared pointers
are similar (in some aspects) to object references known from
Java. Although full implementation of Java object references is not
possible in C++, but at least a quite safe surrogate of them can we
have using shared_ptr objects.
Up to be widely used, shared_ptr class should be easy in
use and as safe as possible.
Although quite good, the shared_ptr class has some disadvantages.
First of all, with its "take ownership of pointer" property, it provides
rather moderate level of abstraction. It does not ensure that
the pointer will be always correct and gives programmer the
opportunity to introduce difficult bugs in program.
It would be better to allocate the pointer by the class itself.
This would ensure that the pointer is always allocated correctly.
For example, the class may provide a method named "New".
The method would release previous pointer and allocate new one.
Of course, sometimes allocation of an object of a derived class is needed.
Also, sometimes a programmer needs to pass constructor parameters.
So, method "New" would be rather a template, like this:
template <class X, class A>
shared_ptr<T>& New(A arg);
This method would create an object of type X using its
constructor with one argument (arg).
Similar overloaded methods might create objects with two, three
arguments and so forth.
Use of "New" method rather than pass allocated pointers has
one more advantage: it makes harder to write code which
may cause memory leaks. Current implementation causes
memory leaks in cases when exception occurs between an object
allocation and pass it to shared_ptr object.
Some minor problems: I see that shared_ptr class does not have
any obvious method for check, whether it is set to 0. It would be
nice to have something like:
bool is_nil() const;
Another method would allow to set the pointer value to 0, e.g.:
void set_to_nil();
This would work like reset() without arguments, but, in my
opinion, the function name describes better what the function does.
One more method might simplify implementation of a
"copy on write" technique, i.e. creation of a separate copy
of object when it is about to change. For example, a method:
void own_copy();
It might work as follows: if the object is shared among more than
one pointer, the function would create own copy of the object.
If the pointer is the only owner of the object, function would do
nothing.
Rafal Dabrowa
---
[ comp.std.c++ is moderated. To submit articles, try just posting with ]
[ your news-reader. If that fails, use mailto:std...@ncar.ucar.edu ]
[ --- Please see the FAQ before posting. --- ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html ]
Hendrik Belitz
> Up to be widely used, shared_ptr class should be easy in
> use and as safe as possible.
> Although quite good, the shared_ptr class has some disadvantages.
> First of all, with its "take ownership of pointer" property, it provides
> rather moderate level of abstraction. It does not ensure that
> the pointer will be always correct and gives programmer the
> opportunity to introduce difficult bugs in program.
The runtime assertions associated with the shared_ptr implementation assist
you in this, so I don't see the problem here.
> It would be better to allocate the pointer by the class itself.
> This would ensure that the pointer is always allocated correctly.
But it won't prevent using uninitialised pointers. This also means that you
need to use a new mechanism which differs from standard pointer allocation.
> Some minor problems: I see that shared_ptr class does not have
> any obvious method for check, whether it is set to 0.
Oh, actually it has:
>From boost::shared_ptr documentation:
>operator unspecified-bool-type () const; // never throws
>
> Returns: an unspecified value that, when used in boolean contexts, is
>equivalent to get() != 0.
>
> Throws: nothing.
>
> Notes: This conversion operator allows shared_ptr objects to be used in
>boolean contexts, like if (p && p->valid()) {}. The actual target type is
>typically a pointer to a member function, avoiding many of the implicit
>conversion pitfalls.
> void set_to_nil();
>
> This would work like reset() without arguments, but, in my
> opinion, the function name describes better what the function does.
I think reset() is decribing very good what's actually happening. Why should
someone blow up the classes interface with redundant member functions?
> One more method might simplify implementation of a
> "copy on write" technique, i.e. creation of a separate copy
> of object when it is about to change. For example, a method:
>
> void own_copy();
>
> It might work as follows: if the object is shared among more than
> one pointer, the function would create own copy of the object.
> If the pointer is the only owner of the object, function would do
> nothing.
That's a good point but has nothing to do with shared_ptr whose solely
purpose is to share a "single" instance and delete this after deletion of
the last shared_ptr pointing to it.
--
- Abort, Retry, Fthagn? -
John Nagle
> Rafal Dabrowa wrote:
>
>
>> Up to be widely used, shared_ptr class should be easy in
>>use and as safe as possible.
>> Although quite good, the shared_ptr class has some disadvantages.
>>First of all, with its "take ownership of pointer" property, it provides
>>rather moderate level of abstraction. It does not ensure that
>>the pointer will be always correct and gives programmer the
>>opportunity to introduce difficult bugs in program.
Well, yes. After a decade of failure, it's clear that
reference counted pointers cannot be implemented safely in
C++ via the template mechanism. The sordid history of
auto_ptr, which, after three major revisions, still has
holes, demonstrates this. If you can get a raw pointer
out of a shared_ptr, it's breakable.
I once wrote a paper on this
(http://www.animats.com/papers/languages/). The basic
problem is that, to create safe smart pointers, you
have to disallow some things that are allowed now.
Opposition to this is so strong that it's hopeless.
C++ will thus remain unsafe for the remainder of its life.
John Nagle
Pete Becker
>
> Well, yes. After a decade of failure, it's clear that
> reference counted pointers cannot be implemented safely in
> C++ via the template mechanism. The sordid history of
> auto_ptr, which, after three major revisions, still has
> holes, demonstrates this.
std::auto_ptr is not a reference counted pointer.
--
Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)
John Nagle
> John Nagle wrote:
>
>>
>> Well, yes. After a decade of failure, it's clear that
>> reference counted pointers cannot be implemented safely in
>> C++ via the template mechanism. The sordid history of
>> auto_ptr, which, after three major revisions, still has
>> holes, demonstrates this.
>
>
> std::auto_ptr is not a reference counted pointer.
I didn't say that it was. But it has most of the
same problems.
The basic problem is "how do you prevent any reference
to an object from outliving the object"? Unless you have
some reliable means of preventing that, the safety problems
associated with "smart pointers" and auto_ptr remain.
Bear in mind that unsafe memory allocation is the
main reason that C# and Java were created. C++ remains
the only major language with abstraction but without safety.
A high percentage of software bugs come from this fact.
I've at least proposed a solution that could add safety
within the general framework of C++. I'm not seeing much
of that from elsewhere.
John Nagle
Larry Evans
[snip]
>
> Well, yes. After a decade of failure, it's clear that
> reference counted pointers cannot be implemented safely in
> C++ via the template mechanism. The sordid history of
> auto_ptr, which, after three major revisions, still has
> holes, demonstrates this. If you can get a raw pointer
> out of a shared_ptr, it's breakable.
>
> I once wrote a paper on this
> (http://www.animats.com/papers/languages/). The basic
> problem is that, to create safe smart pointers, you
> have to disallow some things that are allowed now.
> Opposition to this is so strong that it's hopeless.
What if the only "import" types (arg types to CTOR or operator=
which don't involve other smart_ptr's) consisted of
something like this:
http://lists.boost.org/MailArchives/boost/msg42996.php
and the only "export" methods (anything allowing
access to the pointee) was operator->(void). IOW, there wouldn't
be any method returning a raw pointer. Wouldn't that make a safe
smart_ptr?
Is maybe the strong_new you mention here:
http://www.animats.com/papers/languages/strictpointers.html
what msg42996.php talks about? There's also something similar,
i.e. a safe "import" type, auto_overhead, at:
Pete Becker
> Pete Becker wrote:
>
>> John Nagle wrote:
>>
>>>
>>> Well, yes. After a decade of failure, it's clear that
>>> reference counted pointers cannot be implemented safely in
>>> C++ via the template mechanism. The sordid history of
>>> auto_ptr, which, after three major revisions, still has
>>> holes, demonstrates this.
>>
>>
>>
>> std::auto_ptr is not a reference counted pointer.
>
>
> I didn't say that it was.
I didn't say that you did. Readers of this thread can judge for
themselves what the implication of your words is.
> But it has most of the
> same problems.
>
> The basic problem is "how do you prevent any reference
> to an object from outliving the object"? Unless you have
> some reliable means of preventing that, the safety problems
> associated with "smart pointers" and auto_ptr remain.
Well, yes, whatever problems arise from having access to raw pointers
are present if smart pointers give you access to raw pointers. It's
certainly fashionable these days to label this "unsafe", but that
doesn't make it hazardous.
One of the wonderful things about C++, though, is its extensibility. If
auto_ptr and shared_ptr don't meet your needs, write your own smart
pointer, use it, and get others to use it. Then propose it for the
standard. If opposition is too strong, then maybe your assumptions are
wrong.
--
Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)
---
John Nagle
> On 06/01/2005 11:52 AM, John Nagle wrote:
> [snip]
> i.e. a safe "import" type, auto_overhead, at:
>
> http://cvs.sourceforge.net/viewcvs.py/boost-sandbox/boost-sandbox/boost/managed_ptr/auto_overhead.hpp?rev=1.2&view=auto
That says:
"In fact, I have been arguing for years that our smart pointers should
never have had a public interface which adopts unmanaged resources on
construction. Instead, we should write:"
std::auto_ptr<T> = std::auto_ptr_new<T>(arg1, arg2, arg3);
Voila, a managed T straight out of the box."
That's a good first step.
But eventually you reach the limits of the template mechanism.
For smart pointers to work right, you need be able to write
struct Point {
double x, y;
Point(double xin, yin)
: x(xin), y(yin) {}
};
typedef std::auto_ptr<Point> ManagedPoint;
...
ManagedPoint p = std::auto_ptr_new<Point>(10,200);
// (or, preferably)
ManagedPoint q = ManagedPoint::new(10,200);
// But then you need to be able to write
p.x = 1.0; // can't do that
Whatever happened to "operator." and "operator->"?
There was much discussion of allowing those overloads
as far back as 1997, but they don't seem to have
reached many implementations.
John Nagle
"Daniel Krügler (ne Spangenberg)"
John Nagle schrieb:
> But eventually you reach the limits of the template mechanism.
> For smart pointers to work right, you need be able to write
>
> struct Point {
> double x, y;
> Point(double xin, yin)
> : x(xin), y(yin) {}
> };
>
> typedef std::auto_ptr<Point> ManagedPoint;
> ...
> ManagedPoint p = std::auto_ptr_new<Point>(10,200);
> // (or, preferably)
> ManagedPoint q = ManagedPoint::new(10,200);
> // But then you need to be able to write
> p.x = 1.0; // can't do that
>
> Whatever happened to "operator." and "operator->"?
> There was much discussion of allowing those overloads
> as far back as 1997, but they don't seem to have
> reached many implementations.
There ist still a quite new proposal for operator., see
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2004/n1671.pdf
Greetings from Bremen,
Daniel Krügler
John Nagle
> Hello John Nagle,
>
> John Nagle schrieb:
>> Whatever happened to "operator." and "operator->"?
>> There was much discussion of allowing those overloads
>> as far back as 1997, but they don't seem to have
>> reached many implementations.
>
>
> There ist still a quite new proposal for operator., see
>
> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2004/n1671.pdf
It's discouraging to see the same discussion from 1991.
The 2004 paper makes it clear that "operator." can be
made to work. The key concept is that if you write
&foo.bar
when "foo" is an instance of some smart pointer class
and "operator." is overloaded, you must also overload
unary "operator&" and have it return a "smart reference".
What puzzles me is that "operator->" was added, but
"operator." was not. I could see having neither, or
both, but not one of the two. If you can overload "operator->",
logically you should be able to overload "operator.".
What was the rationale there?
John Nagle
David Abrahams
> Larry Evans wrote:
>> On 06/01/2005 11:52 AM, John Nagle wrote:
>> [snip]
> There's also something similar,
>> i.e. a safe "import" type, auto_overhead, at:
>> http://cvs.sourceforge.net/viewcvs.py/boost-sandbox/boost-sandbox/boost/managed_ptr/auto_overhead.hpp?rev=1.2&view=auto
>
> That says:
>
> "In fact, I have been arguing for years that our smart pointers should
> never have had a public interface which adopts unmanaged resources on
> construction. Instead, we should write:"
>
> std::auto_ptr<T> = std::auto_ptr_new<T>(arg1, arg2, arg3);
>
> Voila, a managed T straight out of the box."
I think you must've confused your links. You're quoting a post from
me: http://lists.boost.org/MailArchives/boost/msg42996.php
> That's a good first step.
>
> But eventually you reach the limits of the template mechanism.
The limitations you run into are
a. the forwarding problem
b. the lack of varargs templates
Until we get rvalue references, you can do pretty darned well for
reasonable numbers of arguments (say, 6) by using the preprocessor to
generate all the overloads taking const and mutable references.
> For smart pointers to work right, you need be able to write
>
> struct Point {
> double x, y;
> Point(double xin, yin)
> : x(xin), y(yin) {}
> };
>
> typedef std::auto_ptr<Point> ManagedPoint;
> ...
> ManagedPoint p = std::auto_ptr_new<Point>(10,200);
> // (or, preferably)
> ManagedPoint q = ManagedPoint::new(10,200);
> // But then you need to be able to write
> p.x = 1.0; // can't do that
>
> Whatever happened to "operator." and "operator->"?
We already have operator-> What's wrong with
p->x = 1.0
?
> There was much discussion of allowing those overloads
> as far back as 1997, but they don't seem to have
> reached many implementations.
I don't know of any implementations without operator->
--
Dave Abrahams
Boost Consulting
www.boost-consulting.com
Randy
John Nagle wrote:
[snip]
>
> What puzzles me is that "operator->" was added, but
> "operator." was not. I could see having neither, or
> both, but not one of the two. If you can overload "operator->",
> logically you should be able to overload "operator.".
> What was the rationale there?
>
> John Nagle
>
See D&E 11.5.2 for a thorough discussion. I'll be very interested to
see how far this goes given the issues Stroustrup discusses there.
It's been quite a long discussion so far.
Randy.
Rafal Dabrowa
> The runtime assertions associated with the shared_ptr implementation assist
> you in this, so I don't see the problem here.
I can't imagine how you may validate the pointer obtained from user.
>> It would be better to allocate the pointer by the class itself.
>>This would ensure that the pointer is always allocated correctly.
>
> But it won't prevent using uninitialised pointers. This also means that you
> need to use a new mechanism which differs from standard pointer allocation.
I don't understand you. It is impossible to have language
mechanisms which would prevent from all possible bugs, and
my proposal does not aim to do this.
I want only prevent from some errors. Of course, you can
forget about setting of proper pointer value like forget about
initializing an integer variable. But, please note: if
shared_ptr class would care about object allocation,
it would keep either a null pointer or a pointer which refers to
a valid object. In case of an run-time error, programmer has much
less to check.
Of course, you may do the following
delete shared_ptr_obj->get()
but it is not easy to do that accidentally, especially when using
smart pointers would eliminate need of any explicit deletion.
>>One more method might simplify implementation of a
>>"copy on write" technique, i.e. creation of a separate copy
>>of object when it is about to change. For example, a method:
>>
>> void own_copy();
>>
>>It might work as follows: if the object is shared among more than
>>one pointer, the function would create own copy of the object.
>>If the pointer is the only owner of the object, function would do
>>nothing.
>
>
> That's a good point but has nothing to do with shared_ptr whose solely
> purpose is to share a "single" instance and delete this after deletion of
> the last shared_ptr pointing to it.
A class, which would support "copy on write" techinque would
be almost a duplicate of shared_ptr (or shared_array) class.
The own_copy() method would be the only difference. Why introduce
plenty of classes which would perform the same things ?
Rafal Dabrowa
John Nagle
> Hendrik Belitz wrote:
> I don't understand you. It is impossible to have language
> mechanisms which would prevent from all possible bugs, and
> my proposal does not aim to do this.
There are languages that eliminate all bad-pointer bugs.
Java, for one. All the scripting languages. C# (almost).
C++ (and C) are the last mainstream languages that don't solve
that problem. Every day, millions of computers crash or
are taken over by viruses, trojans, and worms for that
single reason.
There are many unsafe shared pointer classes available.
They're not used much. Bad reference counting can be worse
than no reference counting, because the bugs are really hard to
find. Automatic memory management for C++ has to be airtight
to be competitive with the newer languages.
Thus, a proposal that "almost" fixes the problem is not
useful.
On the other hand, neither is denying that there is a problem.
John Nagle
Pete Becker
>
> There are languages that eliminate all bad-pointer bugs.
> Java, for one.
Nope. Java traps them, and throws a NullPointerException. The price of
doing that, of course, is runtime pointer checks.
--
Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)
---
Peter Dimov
> Rafal Dabrowa wrote:
>
> > Hendrik Belitz wrote:
>
> > I don't understand you. It is impossible to have language
> > mechanisms which would prevent from all possible bugs, and
> > my proposal does not aim to do this.
>
> There are languages that eliminate all bad-pointer bugs.
> Java, for one. All the scripting languages. C# (almost).
> C++ (and C) are the last mainstream languages that don't solve
> that problem. Every day, millions of computers crash or
> are taken over by viruses, trojans, and worms for that
> single reason.
Crash, yes. Taken over, no. There do exist dangling pointer exploits,
but they are incredibly rare; buffer overflows are the primary attack
vector. Limiting the security risks of buffer overflows is trivial and
requires no change to C or C++.
All languages have security issues. PHP web applications crash and are
taken over daily. Java applications crash. The important thing is
whether an application can adversely affect the outer layer. This is a
property of the outer layer and not of the programming language in
which the inner layer is written.
Dave Harris
> There are languages that eliminate all bad-pointer bugs.
> Java, for one.
The underlying problem is profound. An object has a class invariant, and
some objects need to own resources for that invariant to be true.
Eventually the object must give up the resources, return them to the OS or
whatever, and then the object becomes unusable. To use an object via a
pointer after that has happened is a bad-pointer bug.
In C++ it's destructors and in Java-like languages it's "dispose", but the
problem is fundamental. It can't be eliminated at the language level
without draconian measures. What you can do is replace undefined behaviour
with well-defined behaviour, which is more or less what Java does. It
helps, but it doesn't solve the deep problems of object lifetime.
-- Dave Harris, Nottingham, UK.
Peter Dimov
> A class, which would support "copy on write" techinque would
> be almost a duplicate of shared_ptr (or shared_array) class.
> The own_copy() method would be the only difference. Why introduce
> plenty of classes which would perform the same things ?
The main problem is that now shared_ptr would need to know how to make
a copy. If you have
shared_ptr<Base> pb( new Derived );
and you tell pb to make itself unique (and its use_count is 2 or more),
it would need to use one of new Base( *pb ), new Derived(
*<original-stored-pointer> ), or perhaps pb->clone().
The first alternative will slice, the second will prevent you from
using shared_ptr with noncopyable types because a copy function would
have to be instantiated at creation time, and the third has the obvious
problem that not all classes would conform to a specific clone()
interface.
Then there are the weak pointers. :-)
While it is true that a copy-on-write pointer would be almost a
duplicate of shared_ptr implementation-wise, I think that it deserves
its own type and semantics (requires CopyConstructible on creation,
supports no weak pointers.)
Joe Gottman
"Peter Dimov" <pdi...@gmail.com> wrote in message
news:1117888774.2...@g14g2000cwa.googlegroups.com...
> Rafal Dabrowa wrote:
>
>> A class, which would support "copy on write" techinque would
>> be almost a duplicate of shared_ptr (or shared_array) class.
>> The own_copy() method would be the only difference. Why introduce
>> plenty of classes which would perform the same things ?
>
> The main problem is that now shared_ptr would need to know how to make
> a copy. If you have
>
> shared_ptr<Base> pb( new Derived );
>
> and you tell pb to make itself unique (and its use_count is 2 or more),
> it would need to use one of new Base( *pb ), new Derived(
> *<original-stored-pointer> ), or perhaps pb->clone().
>
> The first alternative will slice, the second will prevent you from
> using shared_ptr with noncopyable types because a copy function would
> have to be instantiated at creation time, and the third has the obvious
> problem that not all classes would conform to a specific clone()
> interface.
The clone() option is much safer for shared_ptr than for most other
classes. The boost implementation of shared_ptr internally stores a pointer
to an abstract base class, sp_counted_base, that has a pure virtual
dispose() function. This is how it knows the correct way to delete the
stored pointer when use_count() goes to zero. There's no reason why
sp_counted_base can't also have a pure virtual clone() function that knows
how to clone itself. The derived classes that inherit from sp_counted_base
are templated on the actual type of the pointer that they hold, so a clone()
method should know what type it has to construct.
Joe Gottman
Peter Dimov
As I already said in the post you quoted, when:
shared_ptr<Base> pb( new Derived );
is used, the instantiation of that virtual clone() in the
implementation of sp_counted_base will impose a new CopyConstructible
requirement on Derived.
Pete Becker
>
> As I already said in the post you quoted, when:
>
> shared_ptr<Base> pb( new Derived );
>
> is used, the instantiation of that virtual clone() in the
> implementation of sp_counted_base will impose a new CopyConstructible
> requirement on Derived.
>
It also won't guarantee a valid copy:
shared_ptr<Base> bp((Base*)new Derived);
Now the virtual clone() slices the original object.
--
Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)
---
Dave Harris
> As I already said in the post you quoted, when:
>
> shared_ptr<Base> pb( new Derived );
>
> is used, the instantiation of that virtual clone() in the
> implementation of sp_counted_base will impose a new CopyConstructible
> requirement on Derived.
I think the intent is that that constructor not be supported at all; that
there be no way to assign a raw pointer to the smart pointer. So
compatibility with shared_ptr is impossible anyway. I'll call it safe_ptr
instead. We'd have something like:
template <typename T>
safe_ptr<T> safe_new() {
safe_ptr<T> p;
p.make();
return p;
}
safe_ptr<Base> pb = safe_new<Derived>();
safe_ptr<Base> pb2 = pb.clone();
Although safe_ptr could impose a CopyConstructible requirement on Derived,
I expect it is also possible to have safe_ptr<T> detect whether the copy
constructor is available, and, if it isn't, provide a default
implementation of clone() which throws an exception. Callers have to cope
with out-of-memory exceptions anyway.
This is replacing a compile-time error with a run-time error for the sake
of simplifying the static type system. I think it could be the right
choice if it led to a single, safe, general, widely-compatible smart
pointer that could be used by default. Of course some people will prefer
policy-based designs :-)
-- Dave Harris, Nottingham, UK.
---
Andrei Alexandrescu (See Website For Email)
> John Nagle wrote:
>
>>
>> There are languages that eliminate all bad-pointer bugs.
>> Java, for one.
>
> Nope. Java traps them, and throws a NullPointerException. The price of
> doing that, of course, is runtime pointer checks.
Not always. Some implementations implement the access violation trap and
translate it to a Java exception. For non-null pointers, that incurs no
runtime cost.
Andrei
David Abrahams
> I expect it is also possible to have safe_ptr<T> detect whether the copy
> constructor is available,
If you figure that puzzle out, please be sure to post it here! AFAIK
there is no way to detect whether a type is copiable, primarily
because every type gets a copy ctor -- it just may turn out to be
private -- and access can't be detected at compile-time.
--
Dave Abrahams
Boost Consulting
www.boost-consulting.com
---
John Nagle
> na...@animats.com (John Nagle) wrote (abridged):
>
>> There are languages that eliminate all bad-pointer bugs.
>>Java, for one.
>
>
> The underlying problem is profound. An object has a class invariant, and
> some objects need to own resources for that invariant to be true.
> Eventually the object must give up the resources, return them to the OS or
> whatever, and then the object becomes unusable. To use an object via a
> pointer after that has happened is a bad-pointer bug.
>
> In C++ it's destructors and in Java-like languages it's "dispose", but the
> problem is fundamental. It can't be eliminated at the language level
> without draconian measures. What you can do is replace undefined behaviour
> with well-defined behaviour, which is more or less what Java does. It
> helps, but it doesn't solve the deep problems of object lifetime.
It takes language-level machinery to solve the problem properly,
yes. My main point in all this is that trying to address the problem
via the template mechanism has been a dismal failure. Fundamentally,
adding an abstraction with subtle safety problems introduces
defects in software.
Ignoring it has led to a decline in C++ market share and
serious reliability problems. (Java enthusiasts would say, and have
written, "C++ has failed" and "C++ is fundamentally unsafe.")
The head-in-the-sand option isn't working, people.
C++ must change or die. It's not good enough any more.
John Nagle
Pete Becker
>
> (Java enthusiasts would say, and have
> written, "C++ has failed" and "C++ is fundamentally unsafe.")
>
The first assertion is simply false, the second is based on undefined
terms, hence, meaningless. Design decisions should never be based on
marketing hype.
--
Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)
---
Pete Becker
> Pete Becker wrote:
>
>>
>> Nope. Java traps them, and throws a NullPointerException. The price of
>> doing that, of course, is runtime pointer checks.
>
>
> Not always. Some implementations implement the access violation trap and
> translate it to a Java exception. For non-null pointers, that incurs no
> runtime cost.
>
Always. The implementation you describe is a runtime check.
--
Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)
---
ka...@gabi-soft.fr
I can remember reading an article by Lisp enthusiasts, way back
when, which said that C has failed.
My impression is that Java has failed. It has failed to fulfil
its promesses, that is certain. But in a way, that was obvious
from the start -- promess the impossible, and you will fail.
C++ has some serious problems (IMHO, at least). For better or
for worse, however, the language is flexible enough to work
around them. Java also has some serious problems. Maybe less
that C++ -- I'll not say anything about that. But the language
is rigid enough that it is often impossible to work around the
problem. It's very hard work to write robust softare in C++.
It's impossible in Java.
> The head-in-the-sand option isn't working, people. C++
> must change or die. It's not good enough any more.
IMHO, C++ would certainly be improved by standard support for
optional garbage collection. But the language isn't going to
die, with or without it. And from what I can see, Java has
given up attacking C++, and switched its target to Perl
(e.g. CGI scripts) and PHP. If there was a language war, C++
won it.
--
James Kanze GABI Software
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
ka...@gabi-soft.fr
> John Nagle wrote:
> > There are languages that eliminate all bad-pointer bugs.
> > Java, for one.
> Nope. Java traps them, and throws a NullPointerException. The
> price of doing that, of course, is runtime pointer checks.
Most C++ implementations also trap cases of dereferencing a null
pointer; at least, mine do. That, of course, doesn't suffice to
eliminate all bad-pointer bugs.
The problem is, of course, that there are many types of bad
pointer bugs. Pointer arithmetic is a frequent source of buffer
overflow, and Java does eliminate that one. Another frequent
error that I've seen is using an invalid pointer: after delete
in C++, or after having called dispose() (or whatever the
convention of the day calls the function) in Java. As far as I
can see, neither C++ nor Java formally protect you against this
one, although a C++ implementation could scribble over the
memory in delete, which would notably reduce the chances of the
error going unnoticed for any significant amount of time.
--
James Kanze GABI Software
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
Llewelly
[snip]
> Well, yes. After a decade of failure, it's clear that
> reference counted pointers cannot be implemented safely in
> C++ via the template mechanism. The sordid history of
> auto_ptr, which, after three major revisions, still has
> holes, demonstrates this.
auto_ptr's problem isn't smart pointerness, or templates. auto_ptr's
problem is destructive copying. No widely used smart pointer which
does not support transfer of ownership has had as many problems as
auto_ptr.
> If you can get a raw pointer
> out of a shared_ptr, it's breakable.
If you aren't willing to trust your users to not deliberately break
your library component, you have no business writing C++ library
components, of any kind. What few saftey features C++ has are
designed to protect against accident - not against deliberate
fraud.
Perhaps more importantly, people do not try to get raw pointers from
smart pointers because they are irrational; they do so because
they use one of the tens of thousands of existing C or C++ apis
that use raw pointers. And api implementors won't stop providing
interfaces in terms of raw pointers - raw pointers are what allow
the user to choose the right smart pointer for the job.
A smart pointer from which one could not get raw pointers would be
crippled to the point of uselessness for the vast majority of
real-world applications.
> I once wrote a paper on this
> (http://www.animats.com/papers/languages/). The basic
> problem is that, to create safe smart pointers, you
> have to disallow some things that are allowed now.
> Opposition to this is so strong that it's hopeless.
>
> C++ will thus remain unsafe for the remainder of its life.
[snip]
If people always chose safety over utility, no-one would ever drive an
automobile.
ka...@gabi-soft.fr
> Andrei Alexandrescu (See Website For Email) wrote:
> > Pete Becker wrote:
> >> Nope. Java traps them, and throws a
> >> NullPointerException. The price of doing that, of course,
> >> is runtime pointer checks.
> > Not always. Some implementations implement the access
> > violation trap and translate it to a Java exception. For
> > non-null pointers, that incurs no runtime cost.
> Always. The implementation you describe is a runtime check.
But it's one that is done by the hardware, systematically;
virtual memory can't be made to work without it.
Note that the C++ implementations that I use all do define
dereferencing a null pointer, even if the standard doesn't.
They define it to cause send the process a SIGSEGV. (Given my
druthers... a fatal signal seems a better response to this type
of error than a C++ exception. But the point is: the logic to
detect the error is the same, and the cost of checking for it is
present in C++ just as much as in Java, provided the JVM uses
the trick Andrei described.)
--
James Kanze GABI Software
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
Rafal Dabrowa
> Thus, a proposal that "almost" fixes the problem is not
> useful.
Okay, so what is unsafe in current shared_ptr implementation ?
-- Possibility of access a null pointer ?
As I know, Java has null pointers too. They are named
"nil reference". Isn't this a problem in Java, but in C++ is ?
-- Possibility of destroy referred object ?
This problem may be eliminated. shared_ptr unnecessarily
exposes non-const raw pointer ("get" method).
-- Possibility of inherit a bad pointer ?
This is a problem in current shared_ptr implementation.
But, it will be not a problem when the class will
own the referred object entirely, from "new" till "delete".
-- Pointer arithmetic ?
shared_ptr class does not have any arithmetic.
shared_array class may check bounds in operator[], but
this is a choice of programmers, which prefer performance
at the expense of safety.
What ever ? The only problem, which cannot be eliminated
in C++ is - possibility of forming a cycle of objects
referring to each other using shared_ptr. This
causes memory leaks in case when the cycle is not broken
before de-access it. Java's garbage collector finds
these objects and removes from heap. We can't build
garbage collector for C++. But this is another level
of problems. Even with this flaw, shared pointers
may serve well. Much better than auto_ptr which,
in my opinion, should never become a part of standard.
Maybe it should be not a shared_ptr correction but
a new class, because shared_ptr improvements would
break backward compatibility.
Rafal
Pete Becker
> Pete Becker wrote:
>
>>Andrei Alexandrescu (See Website For Email) wrote:
>
>
>>>Pete Becker wrote:
>
>
>>>>Nope. Java traps them, and throws a
>>>>NullPointerException. The price of doing that, of course,
>>>>is runtime pointer checks.
>
>
>>>Not always. Some implementations implement the access
>>>violation trap and translate it to a Java exception. For
>>>non-null pointers, that incurs no runtime cost.
>
>
>>Always. The implementation you describe is a runtime check.
>
>
> But it's one that is done by the hardware, systematically;
Yes, it is true that this check can be done in hardware. This has turned
into one of those strange discussions where messages start out with
"but..." or "not always..." and then say the same thing as they purport
to be disagreeing with. Except, however, that the original context has
disappeared. The words that I replied to were:
> There are languages that eliminate all bad-pointer bugs.
> Java, for one.
--
Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)
---
John Nagle
> John Nagle wrote:
>
>> Thus, a proposal that "almost" fixes the problem is not
>> useful.
>
>
> Okay, so what is unsafe in current shared_ptr implementation ?
> This problem may be eliminated. shared_ptr unnecessarily
> exposes non-const raw pointer ("get" method).
That's the problem you can't resolve with template
based smart pointers. See previous discussion.
When thinking about how to address memory allocation,
it's worth looking at Perl, which has less trouble with
allocation than most languages. (Yes, Perl is interpreted,
but that's not the relevant issue here.)
Perl has reference-counted allocation, strong and
weak pointers, destructors, and no cycle detection.
As a combination, that works well in practice, and
better than most of the other points in that space.
It's a direction that C++ could go without too much pain.
I realize that this isn't going to happen, and that
C++ software will continue to crash until the language
is phased out. But it's useful to remind everyone that
unreliable and unsafe software is a direct consequence
of the decision to keep C++ an unsafe language.
This is not an unfixable problem. It is a problem that
the Committee has chosen not to fix.
John Nagle
David Abrahams
> I realize that this isn't going to happen, and that
> C++ software will continue to crash until the language
> is phased out. But it's useful to remind everyone that
> unreliable and unsafe software is a direct consequence
> of the decision to keep C++ an unsafe language.
> This is not an unfixable problem. It is a problem that
> the Committee has chosen not to fix.
By the same token, we have "chosen" not to fix nuclear
proliferation. Nobody has come to us with a proposal for fixing
either problem. Coincidence?
--
Dave Abrahams
Boost Consulting
www.boost-consulting.com
---
David Abrahams
> ka...@gabi-soft.fr wrote:
>> Pete Becker wrote:
>>
>>>Andrei Alexandrescu (See Website For Email) wrote:
>>
>>>>Pete Becker wrote:
>>
>>>>>Nope. Java traps them, and throws a
>>>>>NullPointerException. The price of doing that, of course,
>>>>>is runtime pointer checks.
>>
>>>>Not always. Some implementations implement the access
>>>>violation trap and translate it to a Java exception. For
>>>>non-null pointers, that incurs no runtime cost.
>>
>>>Always. The implementation you describe is a runtime check.
>> But it's one that is done by the hardware, systematically;
>
> Yes, it is true that this check can be done in hardware. This has turned
> into one of those strange discussions where messages start out with
> "but..." or "not always..." and then say the same thing as they purport
> to be disagreeing with. Except, however, that the original context has
> disappeared. The words that I replied to were:
>
>> There are languages that eliminate all bad-pointer bugs.
>> Java, for one.
Hopefully not adding to the confusion, but I think "not always" was in
response to your statement that "The price of doing that... is runtime
pointer checks," specifically the implication that the runtime pointer
checks have a non-zero cost.
Unless you count factors that are fixed once you have chosen
particular hardware, the checks that implement access violation traps
don't usually cost anything.
Not flogging a particular position here; just trying to grease the
communication pathways.
--
Dave Abrahams
Boost Consulting
www.boost-consulting.com
---
Andrei Alexandrescu (See Website For Email)
> Andrei Alexandrescu (See Website For Email) wrote:
>
>> Pete Becker wrote:
>>
>>> Nope. Java traps them, and throws a NullPointerException. The price
>>> of doing that, of course, is runtime pointer checks.
>>
>> Not always. Some implementations implement the access violation trap
>> and translate it to a Java exception. For non-null pointers, that
>> incurs no runtime cost.
>>
>
> Always. The implementation you describe is a runtime check.
Not always.
I wonder how one could refer to something I describe, which I myself
haven't described. :o)
Anyway... see
http://www.prolangs.rutgers.edu/refs/docs/kawahito-asplos00.pdf and the
bibliography references 2 (the Jalapeno VM), 13 (IBM's JIT), and 15
(LaTTe).
Andrei
Andrei Alexandrescu (See Website For Email)
> ka...@gabi-soft.fr wrote:
>
>> Pete Becker wrote:
>>
>>> Andrei Alexandrescu (See Website For Email) wrote:
>>
>>>> Pete Becker wrote:
>>
>>>>> Nope. Java traps them, and throws a
>>>>> NullPointerException. The price of doing that, of course,
>>>>> is runtime pointer checks.
>>
>>>> Not always. Some implementations implement the access
>>>> violation trap and translate it to a Java exception. For
>>>> non-null pointers, that incurs no runtime cost.
>>
>>> Always. The implementation you describe is a runtime check.
>>
>> But it's one that is done by the hardware, systematically;
>
> Yes, it is true that this check can be done in hardware. This has turned
> into one of those strange discussions where messages start out with
> "but..." or "not always..." and then say the same thing as they purport
> to be disagreeing with. Except, however, that the original context has
> disappeared. The words that I replied to were:
>
>> There are languages that eliminate all bad-pointer bugs.
>> Java, for one.
But Java did eliminate all dangling pointer bugs.
To clarify my response to the "runtime check": it is runtime, but if it
is done in hardware, it has zero time cost. My understanding of the
context of the discussion was that we were discussing that there is a
price to the "runtime check". The hardware trap mechanism costs only
silicon on the chip, and zero time.
Andrei
John Nagle
> Pete Becker wrote:
>
>> ka...@gabi-soft.fr wrote:
>>
>>> Pete Becker wrote:
> But Java did eliminate all dangling pointer bugs.
>
> To clarify my response to the "runtime check": it is runtime, but if it
> is done in hardware, it has zero time cost. My understanding of the
> context of the discussion was that we were discussing that there is a
> price to the "runtime check". The hardware trap mechanism costs only
> silicon on the chip, and zero time.
True.
It's also worth noting that, with compiler support, many
run-time checks can be hoisted out of loops. This is
particularly effective for subscript checks. The technology
to do this was developed in the Pascal era, but has to
some extent been forgotten. For one Pascal compiler,
it was observed that 95% of subscript checks could be
optimized out.
Similarly, reference count updates can sometimes be
optimized out or hoisted out of loops.
In both cases, though, the compiler needs to know what
to look for. The compiler also has to be allowed to detect
errors "early". That is, once an error becomes
inevitable, the implementation can report the error.
This allows hoisting subscript checks outside of loops.
The notion that checking is inherently expensive comes
from poor implementations. Berkeley Pascal for BSD comes
to mind; it made a subroutine call for every subscript check.
John Nagle
Animats
Jean-Marc Bourguet
> >> There are languages that eliminate all bad-pointer bugs.
> >> Java, for one.
>
> But Java did eliminate all dangling pointer bugs.
I don't see how a GC could possibly eliminate all dangling pointer
bugs. In my experience, most of these are accessing objects which
have been freed at the correct time but to which some pointers have
been incorrectly kepts. In these cases, a GC would simply keep the
objects alive, but the bugs would still be present (the consequences
of the bugs change, sometimes for the worse -- silent wrong results
instead of a crash -- sometimes for the best -- obvious wrong result
instead of a security break). GC doesn't even elimitate all memory
leaks in you use the term "memory leaks" in a broad enough sense like
keeping objects around which will no more be accessed.
> To clarify my response to the "runtime check": it is runtime, but if
> it is done in hardware, it has zero time cost.
More precisely, once it has been decided for other raisons to activate
the features doing such checks, using them to trap null pointers has
zero time cost. Completely desactivating virtual memory could have a
gain.
> My understanding of the context of the discussion was that we were
> discussing that there is a price to the "runtime check". The
> hardware trap mechanism costs only silicon on the chip, and zero
> time.
Simply increasing die size can have effects on the minimal cycle time
achievable (increase of net length and capacitance, decrease of yield).
And obviously depending on other features present and architectural
choices (some things can be done in parallel, like cache checks -- but
having a cache organized for virtual adress prevent sharing between
process*), adding virtual memory would probably also have effects on
latence and/or cycle time: there is simply more things to do for a
memory access.
(*) And security driven minds can consider this as a good effect as it
reduce/supress one cover channel.
--
Jean-Marc
Pete Becker
> Pete Becker wrote:
>
>> Andrei Alexandrescu (See Website For Email) wrote:
>>
>>> Pete Becker wrote:
>>>
>>>> Nope. Java traps them, and throws a NullPointerException. The price
>>>> of doing that, of course, is runtime pointer checks.
>>>
>>>
>>> Not always. Some implementations implement the access violation trap
>>> and translate it to a Java exception. For non-null pointers, that
>>> incurs no runtime cost.
>>>
>>
>> Always. The implementation you describe is a runtime check.
>
>
> Not always.
>
> I wonder how one could refer to something I describe, which I myself
> haven't described. :o)
Sigh. "Some implementations implement the access violation trap and
translate it to a Java exception" sure sounds like a description to me.
>
> Anyway... see
> http://www.prolangs.rutgers.edu/refs/docs/kawahito-asplos00.pdf and the
> bibliography references 2 (the Jalapeno VM), 13 (IBM's JIT), and 15
> (LaTTe).
>
Not interested. I made a simple statement, and I'm not going to play
word games about whether you can misinterpret it to mean something other
than what it said.
In any case, this thread has descended far below the threshold of
meaningful discussion.
--
Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)
---
ka...@gabi-soft.fr
> John Nagle wrote:
> > Thus, a proposal that "almost" fixes the problem is not
> > useful.
> Okay, so what is unsafe in current shared_ptr implementation ?
[...]
> -- Possibility of destroy referred object ?
> This problem may be eliminated. shared_ptr unnecessarily
> exposes non-const raw pointer ("get" method).
But of course, "delete p.operator->()" is still possible:-).
There's also the possibility of shared_ptr itself deleting the
object prematurely, because someone has created the shared_ptr
twice from the same raw pointer, or from this. (The
boost::shared_ptr has a mechanism which allows creating a
shared_ptr from this, but you have to explicitly use it.)
[...]
> What ever ? The only problem, which cannot be eliminated in
> C++ is - possibility of forming a cycle of objects referring
> to each other using shared_ptr.
The real question isn't usually: can it be eliminated, but is
the cost of eliminating it worth the gains. I imagine that you
could eliminate things like "delete p.operator->()" by means of
some sort of proxy class, but frankly, I can't imagine it being
worth the bother. For the rest, I find that the current
implementation represents a pretty good compromize. For a smart
pointer. If memory management is the goal, it still falls far
short of garbage collection, but it has other features which
make it useful even in the presense of garbage collection.
--
James Kanze GABI Software
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
Pete Becker
>
>>> There are languages that eliminate all bad-pointer bugs.
>>> Java, for one.
>
>
> But Java did eliminate all dangling pointer bugs.
>
That's not what was asserted. The claim, once again, is that it
eliminates "all bad-pointer bugs", not just "dangling pointer bugs."
--
Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)
---
ka...@gabi-soft.fr
> "Andrei Alexandrescu (See Website For Email)" writes:
> > >> There are languages that eliminate all bad-pointer
> > >> bugs. Java, for one.
> > But Java did eliminate all dangling pointer bugs.
> I don't see how a GC could possibly eliminate all dangling
> pointer bugs. In my experience, most of these are accessing
> objects which have been freed at the correct time but to which
> some pointers have been incorrectly kepts. In these cases, a
> GC would simply keep the objects alive,
Just a nit concerning vocabulary (but precision is important
here): garbage collection wouldn't keep the object alive.
Garbage collection knows nothing about objects, or their
lifetimes. All garbage collection would do is ensure that the
memory wasn't used for something else (which might be the case
if you had deleted the object).
This allows for better and more controlled detection of the
error, *IF* you code it that way. If you don't, it actually
reduced the probability of the error being seen.
> but the bugs would still be present (the consequences of the
> bugs change, sometimes for the worse -- silent wrong results
> instead of a crash -- sometimes for the best -- obvious wrong
> result instead of a security break).
Obviously, in well written code, such objects would have a flag,
which is positionned in the "destructor" (dispose(), or whatever
it is called), and checked in every function call, generating an
error of some sort on any attempt to access the dead object.
(Just as obviously, by that definition, well written code is the
exception, rather than the rule.)
> GC doesn't even elimitate all memory leaks in you use the term
> "memory leaks" in a broad enough sense like keeping objects
> around which will no more be accessed.
Not even Sun claimed that. At least, not after they had several
releases to correct memory leaks in Swing (and they weren't all
in the JNI part).
--
James Kanze GABI Software
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
John Nagle
> Rafal Dabrowa wrote:
>
>>John Nagle wrote:
>>What ever ? The only problem, which cannot be eliminated in
>>C++ is - possibility of forming a cycle of objects referring
>>to each other using shared_ptr.
>
>
> The real question isn't usually: can it be eliminated, but is
> the cost of eliminating it worth the gains.
Everybody seems to worry about cycles in reference counted
systems. But it's striking to note that Perl, which is
widely used in long-running server applications, does not
protect against cycles. Perl only added "weak pointers"
a few years ago, yet even before that, memory leaks in
Perl programs were reasonably rare.
Generally, the result of a cycle is that some sizable
data structure with back pointers, like a tree, is not
reclaimed. This is seldom a random event, and
it's not catastrophic. It's usually a repeatable bug.
Looking for orphaned memory during debugging is usually
sufficient. Perl deals with this problem by
providing "Test::Memory::Cycle" to search for
orphaned structures.
There are formal methods for finding cycles by global
analysis, but they seem to be overkill for this problem.
Provided that you have "weak pointers" to help deal with
the problem, cycle elimination does not seem to be a
major issue in practice. Dangling pointers cause far
worse problems.
John Nagle
Animats
Pete Becker
>
> There's also the possibility of shared_ptr itself deleting the
> object prematurely, because someone has created the shared_ptr
> twice from the same raw pointer, or from this. (The
> boost::shared_ptr has a mechanism which allows creating a
> shared_ptr from this, but you have to explicitly use it.)
>
That mechanism is also present in TR1's shared_ptr.
--
Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)
---
Peter Dimov
> To clarify my response to the "runtime check": it is runtime, but if it
> is done in hardware, it has zero time cost. My understanding of the
> context of the discussion was that we were discussing that there is a
> price to the "runtime check". The hardware trap mechanism costs only
> silicon on the chip, and zero time.
The main cost of a NULL pointer synchronous exception is that the
compiler can't reorder visible side effects across a pointer
dereference. I don't know how significant is this in practice.
Dave Harris
> C++ software will continue to crash until the language
> is phased out. But it's useful to remind everyone that
> unreliable and unsafe software is a direct consequence
> of the decision to keep C++ an unsafe language.
> This is not an unfixable problem. It is a problem that
> the Committee has chosen not to fix.
It seems to me that most of the problems are manifestations of undefined
behaviour, and that vendors are free to detect the situation and provide a
more reasonable behaviour. That they do not do so suggests there isn't
demand. We shouldn't necessarily blame the standards committee for that.
-- Dave Harris, Nottingham, UK.
Peter Dimov
> Rafal Dabrowa wrote:
> > This problem may be eliminated. shared_ptr unnecessarily
> > exposes non-const raw pointer ("get" method).
>
> But of course, "delete p.operator->()" is still possible:-).
The usual shared_ptr technique of preventing that is making the
destructor protected/private.
http://www.boost.org/libs/smart_ptr/sp_techniques.html#abstract
http://www.boost.org/libs/smart_ptr/sp_techniques.html#preventing_delete
Andrei Alexandrescu (See Website For Email)
> "Andrei Alexandrescu (See Website For Email)" writes:
>
>>>> There are languages that eliminate all bad-pointer bugs.
>>>>Java, for one.
>>
>>But Java did eliminate all dangling pointer bugs.
>
> I don't see how a GC could possibly eliminate all dangling pointer
> bugs. In my experience, most of these are accessing objects which
> have been freed at the correct time but to which some pointers have
> been incorrectly kepts.
Oh, this is a little confusion. By "dangling pointer" I understand
"pointer to invalid memory", not "pointer to useless object".
So yes, GC doesn't (can't) eliminate memory leaks because it simply
can't predict the future and as such it doesn't know when the last
dynamic use of a pointer is. But GC rigurously does eliminate invalid
(what I call "dangling") pointers.
> More precisely, once it has been decided for other raisons to activate
> the features doing such checks, using them to trap null pointers has
> zero time cost. Completely desactivating virtual memory could have a
> gain.
This takes the discussion in another dimension. And I honestly doubt it,
but it doesn't matter. At any rate, desktop OSs of today don't even
offer that because their very fundaments rely on hardware-enforced
memory protection.
> Simply increasing die size can have effects on the minimal cycle time
> achievable (increase of net length and capacitance, decrease of yield).
Yes, but once the choice is made, chipping away from the die ain't going
to improve things in the least :o).
So I'm not sure where you're taking this argument: are you addressing it
to hardware designers? Because for all practical purposes, we have
hardware address checking. Whatever it costed.
> And obviously depending on other features present and architectural
> choices (some things can be done in parallel, like cache checks -- but
> having a cache organized for virtual adress prevent sharing between
> process*), adding virtual memory would probably also have effects on
> latence and/or cycle time: there is simply more things to do for a
> memory access.
Same comment. Even discussing whether or not your conjecture is correct
would take the discussion farther astray. You wrote your email from a
system that checks every single address. I am doing the same. What gives?
But I can't stop myself :o). I've TAd a class on processor architecture
(the textbook was Hennesy/Patterson's "Computer Organization and
Design"). At no point in the book was address translation considered a
bottleneck or a determinant factor in clock speed or latency. It's
simple hardware; any section of the ALU has a longer critical path.
> (*) And security driven minds can consider this as a good effect as it
> reduce/supress one cover channel.
Have no idea about that. :o)
Andrei
ka...@gabi-soft.fr
> Jean-Marc Bourguet wrote:
> > "Andrei Alexandrescu (See Website For Email)" writes:
> >>>> There are languages that eliminate all bad-pointer bugs.
> >>>>Java, for one.
> >>But Java did eliminate all dangling pointer bugs.
> > I don't see how a GC could possibly eliminate all dangling
> > pointer bugs. In my experience, most of these are accessing
> > objects which have been freed at the correct time but to
> > which some pointers have been incorrectly kepts.
> Oh, this is a little confusion. By "dangling pointer" I
> understand "pointer to invalid memory", not "pointer to
> useless object".
Wouldn't "pointer to an unusable object" be a better definition?
The object might be unusable because the memory underlying it
has been freed, but if the memory is in an invalid state because
the programmer as called dispose(), or whatever his equivalent
of a destructor is, then it is just as unusable.
About the only difference is that with garbage collection, the
object can reliably set flag, which it verifies at the start of
each function, and signal the error in a reliable and defined
manner. But unless the object actually does this, I don't
really see too much difference between the two.
--
James Kanze GABI Software
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
Sebastian Kaliszewski
> What ever ? The only problem, which cannot be eliminated
> in C++ is - possibility of forming a cycle of objects
> referring to each other using shared_ptr. This
> causes memory leaks in case when the cycle is not broken
> before de-access it. Java's garbage collector finds
> these objects and removes from heap. We can't build
> garbage collector for C++.
We can. See http://smieciuch.sf.net -- this is smartpointer based garbage
collector for C++, this is effect of collective wisdom of at least few
people glued together by me to form usable library with reasonable
performance. This is not bug free (there is currently next bufix release in
the works) but it clearly shows the approach is sound.
rgds
--
Sebastian Kaliszewski
Jean-Marc Bourguet
> Jean-Marc Bourguet wrote:
> > "Andrei Alexandrescu (See Website For Email)" writes:
> >
> >>>> There are languages that eliminate all bad-pointer
> >>>>bugs. Java, for one.
> >>
> >>But Java did eliminate all dangling pointer bugs.
> >
> > I don't see how a GC could possibly eliminate all
> > dangling pointer bugs. In my experience, most of these
> > are accessing objects which have been freed at the
> > correct time but to which some pointers have been
> > incorrectly kepts.
>
> Oh, this is a little confusion. By "dangling pointer" I
> understand "pointer to invalid memory", not "pointer to
> useless object".
>
> So yes, GC doesn't (can't) eliminate memory leaks because
> it simply can't predict the future and as such it doesn't
> know when the last dynamic use of a pointer is. But GC
> rigurously does eliminate invalid (what I call "dangling")
> pointers.
I agree with your definition of dangling pointer. I object
to your use of "eliminate" when more often than not the bug
would still be there, just have another symptom. A GC
transform most deferences of a dangling pointer to an access
to the wrong object. In this case, the bug would still be
there if you used a GC and, if you have some other workmates
than James Kanze, more difficult to find. (purify is quite
good at finding the problems when you suspect a memory
access problem; I know of no tools beeing able to sort out
the logic and decide that you are accessing the wrong
object).
> > More precisely, once it has been decided for other
> > raisons to activate the features doing such checks,
> > using them to trap null pointers has zero time cost.
> > Completely desactivating virtual memory could have a
> > gain.
>
> This takes the discussion in another dimension. And I
> honestly doubt it, but it doesn't matter.
Some obvious cases where it cost time to have activated
virtual memory:
- you need an off chip MMU between the processor and
the memory
- the processor core is designed to work with or without
an MMU
- you use more memory than can be mapped in the TLB.
> At any rate, desktop OSs of today don't even offer that
> because their very fundaments rely on hardware-enforced
> memory protection.
Did I wrote otherwise? I could have precised that it is
usually decided to activate those features.
> > Simply increasing die size can have effects on the
> > minimal cycle time achievable (increase of net length
> > and capacitance, decrease of yield).
>
> Yes, but once the choice is made, chipping away from the
> die ain't going to improve things in the least :o).
> So I'm not sure where you're taking this argument: are you
> addressing it to hardware designers?
Chip designers know this better than me, at least the one I
know. I was adressing this to someone who wrote:
> > > The hardware trap mechanism costs only silicon on the
> > > chip, and zero time.
> Because for all practical purposes, we have hardware
> address checking. Whatever it costed.
Yes. I just pointed that it costed more than silicon area.
> > And obviously depending on other features present and
> > architectural choices (some things can be done in
> > parallel, like cache checks -- but having a cache
> > organized for virtual adress prevent sharing between
> > process*), adding virtual memory would probably also
> > have effects on latence and/or cycle time: there is
> > simply more things to do for a memory access.
>
> Same comment. Even discussing whether or not your
> conjecture is correct would take the discussion farther
> astray. You wrote your email from a system that checks
> every single address. I am doing the same. What gives?
>
> But I can't stop myself :o). I've TAd a class on processor
> architecture (the textbook was Hennesy/Patterson's
> "Computer Organization and Design").
How does it compare with "Computer Architecture: a
Quantitative Approach." by the same?
> At no point in the book was address translation considered
> a bottleneck or a determinant factor in clock speed or
> latency.
It is obviously false for latency in the case of a TLB
miss. Handling the translation is then on the critical path
and take a significant part of the delay.
Even in the absence of TLB miss, it is not so negligeable
that cache adressed by virtual memory have never been
proposed (I'm too lazy to try and find if they are used
currently). Just so that searching the cache can be done in
the same time as translating the address.
> It's simple hardware;
Well, when the page is mapped and in the TLB, its easy (read
the associative cache -- still not a so fast operation) and
append the less significant bits. If the page is not in the
TLB, then things becomes more complicated. So much that
some have decided to simply trap at leave the mess to the
OS.
> any section of the ALU has a longer critical path.
I'm no more familliar with memory timing, but when I was
playing with CPU design, introducing a MMU either increased
the cycle time (if you tried to do the translation at the
same time as address computation) or lenghtened the pipeline
(translation was done on its own cycle). With the
increasing pipeline depth in the last years (even addressing
L1 cache takes severy cycle), I doubt this has changed
(address translation need a cache access).
> > (*) And security driven minds can consider this as a
> > good effect as it reduce/supress one cover channel.
>
> Have no idea about that. :o)
Some have claimed beeing able to extract informations from
different timing patterns in shared cache situation. Don't
ask me the details, I haven't tried to understand how they
extract sensible information from that.
--
Jean-Marc
Andrei Alexandrescu (See Website For Email)
> I agree with your definition of dangling pointer. I object
> to your use of "eliminate" when more often than not the bug
> would still be there, just have another symptom. A GC
> transform most deferences of a dangling pointer to an access
> to the wrong object.
By wrong object do you mean an object of some arbitrary type that
happens to be in memory at the address pointed-to by the pointer? Or what?
This is not a rhethorical question; depending on the answer to it, I'll
either argue "but that's wrong" or "oh ok". :o)
I agree (as in "true but as agreed by both of us irrelevant to the
discussion at hand") with the hardware considerations. You obviously
know what you're talking about. One possibly helpful tidbit of information:
>>But I can't stop myself :o). I've TAd a class on processor
>>architecture (the textbook was Hennesy/Patterson's
>>"Computer Organization and Design").
>
>
> How does it compare with "Computer Architecture: a
> Quantitative Approach." by the same?
The book I mentioned is pretty much the 3rd edition of the one you
mention. I don't like it much. It's thinner but puts much useful
material on the attached CD, and this segregation can be annoying at times.
Andrei
Jean-Marc Bourguet
SeeWebsit...@moderncppdesign.com ("Andrei Alexandrescu (See
Website For Email)") writes:
> Jean-Marc Bourguet wrote:
> > I agree with your definition of dangling pointer. I object
> > to your use of "eliminate" when more often than not the bug
> > would still be there, just have another symptom. A GC
> > transform most deferences of a dangling pointer to an access
> > to the wrong object.
>
> By wrong object do you mean an object of some arbitrary type that happens
> to be in memory at the address pointed-to by the pointer? Or what?
By wrong object I mean an object of the correct type but which is not
the one to be accessed as logically it's usefullness would have ended
at the time where the delete would have been done without a GC.
A+
--
Jean-Marc
Andrei Alexandrescu (See Website For Email)
> Andrei Alexandrescu (See Website For Email) wrote:
> Wouldn't "pointer to an unusable object" be a better definition?
> The object might be unusable because the memory underlying it
> has been freed, but if the memory is in an invalid state because
> the programmer as called dispose(), or whatever his equivalent
> of a destructor is, then it is just as unusable.
>
> About the only difference is that with garbage collection, the
> object can reliably set flag, which it verifies at the start of
> each function, and signal the error in a reliable and defined
> manner. But unless the object actually does this, I don't
> really see too much difference between the two.
Um... memory safety? That no access can obliterate something it's not
supposed to? :o)
Can't believe you said that James :o)).
Andrei Alexandrescu
> SeeWebsit...@moderncppdesign.com ("Andrei Alexandrescu (See
> Website For Email)") writes:
>
>
>>Jean-Marc Bourguet wrote:
>>
>>>to your use of "eliminate" when more often than not the bug
>>>would still be there, just have another symptom. A GC
>>>transform most deferences of a dangling pointer to an access
>>>to the wrong object.
>>
>>to be in memory at the address pointed-to by the pointer? Or what?
>
>
> By wrong object I mean an object of the correct type but which is not
> the one to be accessed as logically it's usefullness would have ended
> at the time where the delete would have been done without a GC.
Cool, then we're on the same page. However, GC makes the behavior
defined by not allowing memory to be read that's not of the type that
was once written. And that's a big deal.
The programmer can choose to implement dispose() or
idunneedthisdudeanymore() or whatnot to make the state of the object a
violation of its invariant (e.g., a date with all fields -1). I
personally think a nicer practice is to put the object in a "valid, but
uninteresting state". But in a memory-safe language like Java, nobody
can ever, ever read anything that wasn't previously written according to
the same type.
Andrei
ka...@gabi-soft.fr
I'm not sure what I have to do with it. It doesn't take any
particularly great level of competence to add a isValid member
to the data, and an assert(isValid) as the first line in all
member functions.
Or do you mean that I care more about quality than some of your
collegues, or that I am more paranoid. In that case, that's
what coding guidelines (and code review which enforces them) is
for.
> (purify is quite good at finding the problems when you suspect
> a memory access problem; I know of no tools beeing able to
> sort out the logic and decide that you are accessing the wrong
> object).
Purify works, but given the resources it uses, it isn't always
that simple to activate. (Log in to a special account on a very
big machine, set up my environment there, etc.) The procedure I
just described above is worth implementing in C++ as well, since
it will indicate immediately the where and whyfore of the error,
a lot easier than Purify. For that matter, just replacing the
default deallocation function with one which writes junk all
over the memory can do wonders.
(Don't get me wrong. I would not consider releasing code which
has not been tested under Purify, or something similar. But I
don't like the idea of having to turn to such heavy tools to
find such a simple bug.)
Personally, I don't see this as an argument either for or
against garbage collection. If an object must be explicitly
invalidated (disposed of, destructed, etc.), then you must do so
explicitly. Garbage collection or not. And of course, whether
the operator is called delete, and frees the underlying raw
memory as well, or is called dispose(), and the underlying raw
memory is only collected later, changes absolutely nothing in
your source code. (Not all objects are like this, and of
course, garbage collection reduces the amount of code you have
to write for other types of objects.)
> > > More precisely, once it has been decided for other raisons
> > > to activate the features doing such checks, using them to
> > > trap null pointers has zero time cost. Completely
> > > desactivating virtual memory could have a gain.
> > This takes the discussion in another dimension. And I
> > honestly doubt it, but it doesn't matter.
> Some obvious cases where it cost time to have activated
> virtual memory:
> - you need an off chip MMU between the processor and
> the memory
> - the processor core is designed to work with or without
> an MMU
> - you use more memory than can be mapped in the TLB.
> > At any rate, desktop OSs of today don't even offer that
> > because their very fundaments rely on hardware-enforced
> > memory protection.
> Did I wrote otherwise? I could have precised that it is
> usually decided to activate those features.
It's usually not decided. Those features are simply activated
by default, and nobody thinks about it.
With regards to the discussion concerning garbage collection,
the only relevance I can see is in the fact that garbage
collection usually results in a process using more memory. This
can increase page faults (which can slow the process down
considerably), and in some cases, can mean that the process
can't even run on the system. Most of the time, on a typical
general purpose processor, running Windows, one of the
mainstream Unix, or Linux, this doesn't matter. Typically
doesn't mean always, and it is important to be able to not use
garbage collection when it does matter. Just as in some
applications, it is important to be able to lock pages (or the
entire process) in memory, even though typically, virtual memory
works well.
--
James Kanze GABI Software
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
Larry Evans
[snip]
> The programmer can choose to implement dispose() or
> idunneedthisdudeanymore() or whatnot to make the state of the object a
> violation of its invariant (e.g., a date with all fields -1). I
> personally think a nicer practice is to put the object in a "valid, but
> uninteresting state". But in a memory-safe language like Java, nobody
http://polaris.cs.uiuc.edu/polaris/polaris_developer/node42.html
what happens is that the original referent type has a virtual function,
is_valid which indicates it's valid. Each deref of the referent checks
this to see if it is a valid referent. After all strong smart pointers
to the referent are gone; yet, there are still weak smart pointers to
the referent, the referent is transformed, via placement new, into a
"zombie" whose is_valid indicates it's no longer valid. Of course this
requires each refcounted object to be derived from an abstract base
class with pure virtual is_valid, but it does save debug time.
John Nagle
> Jean-Marc Bourguet wrote:
>
>>>Jean-Marc Bourguet wrote:
>>>
>>>>"Andrei Alexandrescu (See Website For Email)" writes:
> With regards to the discussion concerning garbage collection,
> the only relevance I can see is in the fact that garbage
> collection usually results in a process using more memory.
Not much more; you need some flags. If you use
a Boehm-type collector which allocates objects within
pages of uniformly sized blocks, there's a padding cost.
Garbage collection and virtual memory tend not to
play well together. Naive garbage collectors thrash
virtual memory and flush caches as they read through
all of memory. There are "generation scavenging"
garbage collections that work well with virtual
memory, because most GC cycles only do a scan of
recently allocated memory, but they require the
ability to move objects during garbage collection.
That's hard to do in C++.
Garbage collection works much better when
the language is designed for it. The language
should not rely heavily on destructors, objects
should be moveable during GC, it should be possible to
mark pages as "flag as dirty if written" at the
paging level, and the language implementation
needs to talk to the virtual memory system.
All of these things have been done, and quite well.
See any modern LISP implementation. But they
don't map well to C++.
It took the Java people about five years to
get their garbage collectors beyond the simple
mark and sweep level. Garbage collection is
neither easy nor a panacea.
John Nagle
Animats
Andrei Alexandrescu
> I'm not sure what I have to do with it. It doesn't take any
> particularly great level of competence to add a isValid member
> to the data, and an assert(isValid) as the first line in all
> member functions.
Yes indeed. And in a GC system, you can be sure that that isValid bit is
your object's, not some random object that happens to be in memory at
the location of the defunct object we think is there.
> Personally, I don't see this as an argument either for or
> against garbage collection. If an object must be explicitly
> invalidated (disposed of, destructed, etc.), then you must do so
> explicitly. Garbage collection or not. And of course, whether
> the operator is called delete, and frees the underlying raw
> memory as well, or is called dispose(), and the underlying raw
> memory is only collected later, changes absolutely nothing in
> your source code. (Not all objects are like this, and of
> course, garbage collection reduces the amount of code you have
> to write for other types of objects.)
Allow me to try to change your opinion. I will do so by actually quoting
--- typing by hand! that ought to be appreciated! :o) --- from Benjamin
Pierce's classic "Types and Programming Languages", page 158:
-----------------
A last issue that we should mentione before we move on formalizing
references is storage deallocation. We have not provided any primitives
for freeing reference cells when they are no longer needed. Instead,
like many modern languages (including ML and Java) we rely on the
run-time system to perform garbage collection, collecting and reusing
cells that can no longer be reached by the program. This is not just a
question of taste in language design; it is extremely difficult to
achieve type safety in the presence of an explicit deallocation
operation. The reason for this is the familiar dangling reference
problem: we allocate a cell holding a number, save a reference to it in
some data structure, use it for a while,
-----------------
.here comes the interesting part, James, read on...
-----------------
then deallocate it and allocate a new cell holding a boolean, possibly
reusing the same storage. Now we can have two names for the same storage
cell --- one for with type Ref Nat and the other with type Ref Bool.
-----------------
> With regards to the discussion concerning garbage collection,
> the only relevance I can see is in the fact that garbage
> collection usually results in a process using more memory.
As shown above, there is some more relevance in that explicit
deallocation can readily lead to violations of type safety, while GC
never has that problem.
Disposing of other resources than memory deterministically is definitely
a good thing to do, but you see, memory has a special status in that it
is the flatbed of everything, and the cradle of type safety. Try to
write to a closed file, and you'll get an error response --- big-o
deal-o; but mess with memory, and you'll violate every single tenet of
your program.
Some of the people who hate GC don't realize, or (worse) don't care for
the benefit above, consider GC the appanage (2nd time I got to use this
word today :o)) of lazy programmers and go on to focus on the costs
alone. Yes, there are costs. But the benefits must be understood.
Peter Dimov
[...]
> Disposing of other resources than memory deterministically is definitely
> a good thing to do, but you see, memory has a special status in that it
> is the flatbed of everything, and the cradle of type safety. Try to
> write to a closed file, and you'll get an error response --- big-o
> deal-o; but mess with memory, and you'll violate every single tenet of
> your program.
>
> Some of the people who hate GC don't realize, or (worse) don't care for
> the benefit above, consider GC the appanage (2nd time I got to use this
> word today :o)) of lazy programmers and go on to focus on the costs
> alone. Yes, there are costs. But the benefits must be understood.
Everything you say is true in general. But in C++ you can introduce a
type error with a single static_cast (and in C, even without a cast).
To achieve type safety nirvana, you need to make the logical next step
and either eliminate type casts, or make them checked and make
everything derive from a polymorphic base instead of void. The
resulting language will no longer be C++.
("appanage" == "birthright", no?)
Jean-Marc Bourguet
===================================== MODERATOR'S COMMENT:
This thread is drifting further from our core topic; please
ensure
that followups have some bearing on C++ standardisation.
===================================== END OF MODERATOR'S COMMENT
James Kanze wrote:
> Jean-Marc Bourguet wrote:
> > > Jean-Marc Bourguet wrote:
> > > > "Andrei Alexandrescu (See Website For Email)" writes:
> I'm not sure what I have to do with it.
It was more an homage to your notion of minimal quality
which sometimes looks like an unachievable graal than
anything else.
> It doesn't take any particularly great level of competence
> to add a isValid member to the data, and an
> assert(isValid) as the first line in all member functions.
It does take quite a particular great level of commitment to
add such member in shared ownership situation and trace the
place where the last owning reference goes out of existence
to update the member when you are using a garbage collector.
> Or do you mean that I care more about quality than some of
> your collegues, or that I am more paranoid.
Replace "some" by "most" and you'll probably still be
overestimating the reality (it obviously how much your
public statements overexagerate your practice). But then
I'm of the opinion that as an organisation we are far from
the best possible. But then at least I think we are
progressing. But that's another story.
> > (purify is quite good at finding the problems when you suspect
> > a memory access problem; I know of no tools beeing able to
> > sort out the logic and decide that you are accessing the wrong
> > object).
>
> Purify works, but given the resources it uses, it isn't
> always that simple to activate. (Log in to a special
> account on a very big machine, set up my environment
> there, etc.)
At least that's not a problem in our organisation. We seem
to do some things right :-)
> The procedure I just described above is worth implementing
> in C++ as well, since it will indicate immediately the
> where and whyfore of the error, a lot easier than Purify.
Well, it depend also how easy it is to identify then access
and modify the cultprit objects. Sometimes purify show that
the problem is in a library you don't even have the sources.
> For that matter, just replacing the default deallocation
> function with one which writes junk all over the memory
> can do wonders.
Most often than not, the memory has been reallocated. But I
agree that a good debugging allocator could help but the
improvement over purify is probably not important enough
that I could defend the evaluation or the writing of one.
> (Don't get me wrong. I would not consider releasing code
> which has not been tested under Purify, or something
> similar. But I don't like the idea of having to turn to
> such heavy tools to find such a simple bug.)
It's a tool we have and we use regularly in preventive way.
That's far better than a tool we don't have and don't know
how to use effectively even if in theory that tool would be
better for the case at hand.
> Personally, I don't see this as an argument either for or
> against garbage collection.
It wasn't an argument for or against GC (BTW, globally I'm
more for GC than against) . I just warned that a GC
doesn't elimate the bugs whose visible effect is an access
to a dangling pointer, it just change their effect. (It can
also be easier to prevent, with the use of weak pointers).
> > Did I wrote otherwise? I could have precised that it is
> > usually decided to activate those features.
>
> It's usually not decided. Those features are simply activated
> by default, and nobody thinks about it.
Well, usually the decision has been made by the OS designers
to activate virtual memory management or not (AFAIK the
latest is still the case for some embedded systems).
> With regards to the discussion concerning garbage collection,
> the only relevance I can see is in the fact that garbage
> collection usually results in a process using more memory.
This part as indeed little to do with GC. Andrei and I both
get carried away in something we both knew be a sidetrack to
the GC issue.
> This can increase page faults (which can slow the process
> down considerably), and in some cases, can mean that the
> process can't even run on the system.
Another thing is that some GC algorithm make use of memory
protection to reduce sweeping on the memory.
Yours,
--
Jean-Marc
John Nagle
> ===================================== MODERATOR'S COMMENT:
> This thread is drifting further from our core topic; please
> ensure
> that followups have some bearing on C++ standardisation.
>
>
> ===================================== END OF MODERATOR'S COMMENT
Good point.
What I'm trying to do here is to come up with a minimal set
of changes to the C++ language that will make reference-counted
allocation work reliably. Attempts to do it within the
existing language consistently run into problems.
One of the big issues is forwarding. The usual forwarding
mechanism in C++ is inheritance. However, the obvious solution,
template<class T> class refcounted_object: public T {
// ...
};
is illegal. Is there a compelling reason to prohibit that?
John Nagle
Animats
Dave Harris
(abridged):
> Some of the people who hate GC don't realize, or (worse) don't care for
> the benefit above, consider GC the appanage (2nd time I got to use this
> word today :o)) of lazy programmers and go on to focus on the costs
> alone. Yes, there are costs. But the benefits must be understood.
Except those benefits cannot be gained by bolting GC onto C++
retrospectively. So they are all but irrelevant in this newsgroup.
The problem with destructors is not just that they allow the memory to be
reused by an object of different type. They also mess with the object's
vtable and, in general, break its class invariant. You still have a broken
object even if the memory isn't reclaimed. So GC will not have all the
benefits in C++ that it has in Java or Lisp.
-- Dave Harris, Nottingham, UK.
---
Andrei Alexandrescu (See Website For Email)
> SeeWebsit...@moderncppdesign.com (Andrei Alexandrescu) wrote
> (abridged):
>
>>Some of the people who hate GC don't realize, or (worse) don't care for
>>the benefit above, consider GC the appanage (2nd time I got to use this
>>word today :o)) of lazy programmers and go on to focus on the costs
>>alone. Yes, there are costs. But the benefits must be understood.
>
>
> Except those benefits cannot be gained by bolting GC onto C++
> retrospectively. So they are all but irrelevant in this newsgroup.
Sigh. It's not black and white. "The Jedi don't see things in black and
white." :o) Type safety can be broken in several ways in C++, dangling
pointers being one of them. Reducing or eliminating one of those ways
might be beneficial.
> The problem with destructors is not just that they allow the memory to be
> reused by an object of different type.
(Well technically it's not the destructors that do that; it's either the
going out of scope or the operator delete.)
They also mess with the object's
> vtable and, in general, break its class invariant. You still have a broken
> object even if the memory isn't reclaimed. So GC will not have all the
> benefits in C++ that it has in Java or Lisp.
That depends on how the language redefines destructor behavior.
Again, saying that it can't be done is easy. Looking into ways in which
it can be done to good effect means progress.
Andrei
Dave Harris
For Email)") wrote (abridged):
> > They also mess with the object's vtable and, in general, break
> > its class invariant. You still have a broken object even if the
> > memory isn't reclaimed. So GC will not have all the benefits in
> > C++ that it has in Java or Lisp.
>
> That depends on how the language redefines destructor behavior.
Go on.
I have thought about this a little bit, but I have no idea what you have
in mind here. Given the C++ model of inheritance, where base class and
derived class have their own class invariants, the current behaviour for
destructors seems right to me. Once the derived class has been "disposed"
so that it no longer owns its resources and its class invariant is no
longer true, virtual functions have to be redirected to the base class. I
don't see how you can improve on this, let alone do so without breaking
old code which relies upon the status quo.
-- Dave Harris, Nottingham, UK.
---
ka...@gabi-soft.fr
> > Andrei Alexandrescu (See Website For Email) wrote: Wouldn't
> > "pointer to an unusable object" be a better definition? The
> > object might be unusable because the memory underlying it
> > has been freed, but if the memory is in an invalid state
> > because the programmer as called dispose(), or whatever his
> > equivalent of a destructor is, then it is just as unusable.
> > About the only difference is that with garbage collection,
> > the object can reliably set flag, which it verifies at the
> > start of each function, and signal the error in a reliable
> > and defined manner. But unless the object actually does
> > this, I don't really see too much difference between the
> > two.
> Um... memory safety? That no access can obliterate something
> it's not supposed to? :o)
> Can't believe you said that James :o)).
We are, I believe, talking about garbage collection in C++.
Where I suppose that pointers will continue to (mis)behave like
C/C++ pointers have always done. So there's no point in talking
too much about memory safety. An uninitialized pointer, or
writing beyond the end of an allocated zone, will still be
possible, and it will corrupt the garbage collected arena in
exactly the same way it currently corrupts the malloc/free
arena. Adding garbage collection is a no-op with regards to
the problem of memory safety.
In the scenario being discussed, it is a question of accessing a
deleted pointer. Or in the case of garbage collection, a
pointer to an object which has logically ceased to exist, even
if the memory where it resided is still there. Globally, I
think that garbage collection is a no-op there, too. If it has
any effect here, it is negative; there exist tools for C++ which
detect accesses to freed memory, and it is almost trivial to
write a custom global allocator/deallocator which will ensure
that the error will be immediate and recognizable, with a very
high degree of probability. But the difference is extremely
small; it's also pretty trivial to add the necessary code to
immediately detect the problem when garbage collection is used.
(Of course, with garbage collection, the code has to be added to
all of the classes in this situation. But typically, there
aren't that many.)
Look. I'm in favor of garbage collection. Very much so. But I
can also understand that there exist cases where it isn't
appropriate, and I think that C++ should support those cases as
well. And I don't think that the cause of garbage collection is
advanced by claiming more for it that it can possibly deliver.
It's a useful tool. It is not a silver bullet. And making
claims that it will eliminate all memory leaks, or all dangling
pointers, or worse, all pointer bugs, only tends to discredit
garbage collection in the eyes of those technical persons who
are still evaluating the issues. Garbage collection is just a
tool. It's a valuable tool, which can reduce the amount of work
needed to write correct programs, at least in a number of
cases. But if you're writing junk now, garbage collection isn't
going to miraculously turn it into correct code.
--
James Kanze GABI Software
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
ka...@gabi-soft.fr
I'm aware of this. In practice, however, I've not found it to
be a problem. You arrange for the allocator to not reuse memory
too quickly, and you scribble junk over it on deallocation, and
it's very rare for the error to pass. Purify detects almost
all, if not all, such errors as well.
The solution with garbage collection is slightly more robust,
*IF* all objects with limited lifetime do take the described
precautions. The solution in the current environment works
almost as well, only missing on a few rare occasions, and
requires no code at the user class level -- it's all in the
allocator/deallocator. And when I worked in Java, I noticed
that while I (and one or two other collegues) were pretty
insistant about using such validation variables, not all
programmers are as careful -- you'll be hard put to find any in
the Java standard library, for example.
And of course, even with garbage collection, in C++, you don't
really know that the assert( isValid == true ) reflects
something the class wrote. Any one could have written it, via
an uninitialized pointer, or whatever.
The result is that, while I definitely favor garbage collection,
I really don't expect it to help much here.
> > With regards to the discussion concerning garbage
> > collection, the only relevance I can see is in the fact that
> > garbage collection usually results in a process using more
> > memory.
> As shown above, there is some more relevance in that explicit
> deallocation can readily lead to violations of type safety,
> while GC never has that problem.
That's because you modify the definition of type safety to
conform to what you can control. In C++, an object of type X
ceases to exist when its destructor is called. The raw memory
where that object was situated is *NOT* an object of type X, and
accessing it through a pointer to an X is undefined behavior.
Whether the memory has been freed or not.
Note that in the discussion at the top of the posting,
concerning the fact that my isValid boolean will not be
overwritten elsewhere, we are not talking about any static type
safety. We are talking about the possibilities of dynamically
checking a violation of type safety -- when my assertion of
isValid fails, it means that client code has accessed a dangling
pointer.
Given the total C++ environment, I don't think the difference
that garbage collection might make here is significant.
> Disposing of other resources than memory deterministically is
> definitely a good thing to do, but you see, memory has a
> special status in that it is the flatbed of everything, and
> the cradle of type safety. Try to write to a closed file, and
> you'll get an error response --- big-o deal-o; but mess with
> memory, and you'll violate every single tenet of your program.
I'm not sure you've chosen a good analogy:-). Close a file,
then open another one, and Unix, at least, will reuse the file
descriptor. You can write to the closed file. Except, of
course, it is some other file which will be modified.
Just like memory:-).
> Some of the people who hate GC don't realize, or (worse) don't
> care for the benefit above, consider GC the appanage (2nd time
> I got to use this word today :o)) of lazy programmers and go
> on to focus on the costs alone. Yes, there are costs. But the
> benefits must be understood.
Certainly. Garbage collection reduced the total amount of code
I need to write in a correct application. That's a win, often a
big win. And the cost in many applications is very low. But if
I'm writing incorrect code now, it's not going to miraculously
become correct just because there is garbage collection.
--
James Kanze GABI Software
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
ka...@gabi-soft.fr
> James Kanze wrote:
> > Jean-Marc Bourguet wrote:
> > > > Jean-Marc Bourguet wrote:
> > > > > "Andrei Alexandrescu (See Website For Email)" writes:
> > It doesn't take any particularly great level of competence
> > to add a isValid member to the data, and an assert(isValid)
> > as the first line in all member functions.
> It does take quite a particular great level of commitment to
> add such member in shared ownership situation and trace the
> place where the last owning reference goes out of existence to
> update the member when you are using a garbage collector.
I thought that the question concerned objects with a definite
lifetime, and the problem of accessing them after that lifetime
had ended. What results in C++ today in using a dangling
pointer. I thought that you were saying that in C++ without
garbage collection, tools like Purify permit finding the problem
quite rapidly. And that earlier, you had characterized this as
a dangling pointer problem which garbage collection doesn't
prevent.
The way I see it, in order to write correct programs: if the
type requires a "destructor" to be called at a deterministic
moment, garbage collection, at least as proposed for C++,
doesn't change anything -- it's totally neutral, because you
still have to call delete manually (supposing the object isn't
on the stack). If the type doesn't require this, then garbage
collection is a win, because it is less code that I have to
write.
Of course, not all programs are correct at the first
compilation. The ability to detect and correct errors easily is
an important attribute as well. And Purify (and other such
tools) do help, by detecting cases where we have accessed an
object after it has been destructed. My only point was that in
cases where an object does have a real destructor, but we still
want to garbage collect it, this can be easily made to work.
(And that I've done so in Java, in the past.) It's something
that developers should keep in mind, but I don't think it is
really relevant vis-à-vis adding garbage collection to C++.
[...]
> > Personally, I don't see this as an argument either for or
> > against garbage collection.
> It wasn't an argument for or against GC (BTW, globally I'm
> more for GC than against) . I just warned that a GC doesn't
> elimate the bugs whose visible effect is an access to a
> dangling pointer, it just change their effect. (It can also
> be easier to prevent, with the use of weak pointers).
IMHO, about the only real errors garbage collection may prevent
are certain categories of memory leaks. I'm not arguing for it
as a means of preventing errors. I'm arguing for it on the
grounds that it means less work for the developer. That, and
only that -- writing correct programs still means reasoning
about object lifetimes, and such. With garbage collection,
however, in certain specific cases (which are rather frequent in
practice), the job stops there. If your analysis shows that the
garbage collector does the correct job, you don't have to write
the code to do the equivalent functions.
--
James Kanze GABI Software
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
John Nagle
> "Andrei Alexandrescu See Website For Email wrote:
>
>>ka...@gabi-soft.fr wrote:
> We are, I believe, talking about garbage collection in C++.
> Where I suppose that pointers will continue to (mis)behave like
> C/C++ pointers have always done. So there's no point in talking
> too much about memory safety.
That's just not good enough any more. With C and C++
the last languages without memory safety, it's time for
the C/C++ community to grow up and accept they have a
problem. A big problem. Denial is not helping.
It's not an unfixable problem. We have
"std::string", and need to push harder for its near universal
use, C strings being the biggest single source of security
problems in programming. Iterator checking is well defined,
and there are implementations. Reference counting needs
language support, but that's not impossible. Smart pointers,
references, and iterators need to replace raw pointers.
APIs need to be modified accordingly. Raw pointers
need to be deprecated. Programmers have to be retrained.
Yet this is less traumatic than what C++ already been
through. Remember how awful the early versions were? Remember
those horrible macros used to fake a generics system?
All those unchecked downcasts? The collection libraries that
preceded the STL? The widespread use of "friend"?
It took about five years to rewrite everything after each
big change. Yet it happened.
If it doesn't happen, C++ will end up on the trash
heap of language history, like Pascal.
John Nagle
David Abrahams
> It's a useful tool. It is not a silver bullet. And making
> claims that it will eliminate all memory leaks, or all dangling
> pointers, or worse, all pointer bugs, only tends to discredit
> garbage collection in the eyes of those technical persons who
> are still evaluating the issues. Garbage collection is just a
> tool. It's a valuable tool, which can reduce the amount of work
> needed to write correct programs, at least in a number of
> cases. But if you're writing junk now, garbage collection isn't
> going to miraculously turn it into correct code.
This leads me to think the value of GC is really going to be
proportional to how much easier it makes reasoning about a program's
correctness.
Will it improve things in that area at all? If an object needing
deterministic destruction can always be hidden at the end of a chain
of GC objects, it seems like not much of an improvement.
I also wonder how many applications need any deterministic destruction.
--
Dave Abrahams
Boost Consulting
www.boost-consulting.com
Andrei Alexandrescu (See Website For Email)
> This leads me to think the value of GC is really going to be
> proportional to how much easier it makes reasoning about a program's
> correctness.
I think it does. There's a number of clear "infection points" to the
type system: dangling pointers, null pointers, unions, varargs, casts...
all detailed in a dedicated section of C++ Coding Standards. If you have
one less infection point, one could argue that makes easier to reason
about a program's correctness, especially since dynamic memory
allocation and pointers are so often used.
> Will it improve things in that area at all? If an object needing
> deterministic destruction can always be hidden at the end of a chain
> of GC objects, it seems like not much of an improvement.
Good question. Some systems distinguish statically between non-GC,
region-based, and GC objects, at the cost of some expressiveness. Those
systems would disallow a non-GC object to be held as a member of a GC
object.
> I also wonder how many applications need any deterministic destruction.
Ionno. I, for one, like the idea especially when it comes about other
resources than memory. Besides, deterministic destruction is so useful
for useful related C++ idioms such as scope guards.
John Nagle
> ka...@gabi-soft.fr writes:
> I also wonder how many applications need any deterministic destruction.
It's a language design issue. In LISP, you don't have deterministic
destruction, but you have constructs like
(with-open-file
(fileid filename)
(code))
which opens "filename" at scope entry, and closes it at
scope exit.
Java has the try/finally mechanism, which allows
execution at scope exit.
C++ has neither.
In C++, the usual method for getting things done
at scope exit is to construct something, then use a
destructor.
It's perfectly possible to have a language that
doesn't need deterministic destruction, but C++ isn't it.
If you try, you get a mess like Microsoft Managed C++.
For a review of the destructor problem in Managed C++,
see
and read "Deterministic Finalization I..IV". Also
see this MSDN article on Managed C++ destructors:
http://msdn.microsoft.com/msdnmag/issues/04/12/CQA/
(Note that the example in that item has a memory
leak and a patch has been issued for that article.)
Please review the Microsoft Managed C++ experience
before proposing garbage collection with destructors.
Thank you.
John Nagle
Animats
ka...@gabi-soft.fr
> ka...@gabi-soft.fr writes:
> > It's a useful tool. It is not a silver bullet. And making
> > claims that it will eliminate all memory leaks, or all
> > dangling pointers, or worse, all pointer bugs, only tends to
> > discredit garbage collection in the eyes of those technical
> > persons who are still evaluating the issues. Garbage
> > collection is just a tool. It's a valuable tool, which can
> > reduce the amount of work needed to write correct programs,
> > at least in a number of cases. But if you're writing junk
> > now, garbage collection isn't going to miraculously turn it
> > into correct code.
> This leads me to think the value of GC is really going to be
> proportional to how much easier it makes reasoning about a
> program's correctness.
I'm not sure. I think much of the value is simply less code to
write.
> Will it improve things in that area at all? If an object
> needing deterministic destruction can always be hidden at the
> end of a chain of GC objects, it seems like not much of an
> improvement.
There are different things you have to reason about.
> I also wonder how many applications need any deterministic
> destruction.
Most. You certainly don't want to deprive us of things like
scoped_lock. A more valid question, in this context, is how
many need deterministic destruction of non-local objects?
In my own application, there are three types of objects which
need deterministic destruction:
-- Local resources, like scoped_lock. But all of the instances
of these objects are local variables, so they aren't
affected by garbage collection.
-- Transacion management. But this is really an extended case
of local resources, and here, too, all of the instances of
the variable are local.
-- Explicitly destructed objects: these are dynamically
allocated objects, but they are destructed explicitly as the
result of a command, either external, or internally
generated by a special thread (purge of out of date
objects). In this case, garbage collection is neutral: it
doesn't hurt, but it doesn't really help any, either.
Garbage collection doesn't really affect any of them.
--
James Kanze GABI Software
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
ka...@gabi-soft.fr
> ka...@gabi-soft.fr wrote:
> > "Andrei Alexandrescu See Website For Email wrote:
> >>ka...@gabi-soft.fr wrote:
> > We are, I believe, talking about garbage collection in C++.
> > Where I suppose that pointers will continue to (mis)behave
> > like C/C++ pointers have always done. So there's no point
> > in talking too much about memory safety.
> That's just not good enough any more. With C and C++ the
> last languages without memory safety, it's time for the C/C++
> community to grow up and accept they have a problem. A big
> problem. Denial is not helping.
Well, I sort of agree. But my point was simply that the
contribution of garbage collection to solving this problem is
limited. As long as you have uninitialized pointers, pointers
to local objects, and pointer arithmetic, you're going to have
problems.
I might add that regardless of what other languages are doing,
and despite the high cost in safety, I would not like to do
without pointers to local objects. And with regards to "modern
languages": how does Ada handle this? (Ada is the only modern
language I know of which tries to be type safe, and also has
local objects. Can you take the address of a local object? Are
there restrictions which will prevent it from dangling?)
--
James Kanze GABI Software
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
Jean-Marc Bourguet
> I might add that regardless of what other languages are doing, and
> despite the high cost in safety, I would not like to do without
> pointers to local objects. And with regards to "modern languages":
> how does Ada handle this? (Ada is the only modern language I know
> of which tries to be type safe, and also has local objects. Can you
> take the address of a local object? Are there restrictions which
> will prevent it from dangling?)
You can create a pointer to a local object (with the access attribute,
the object has to be marked aliased BTW) but it is of a type that has
the same livetime as the object (so you can't store it in a variable
which is longer lived without doing "unchecked" things).
But Ada has also a unchecked_access attribute does the same thing as
the access attribute but without some static checking (the scope for
sure, perhaps the aliased mark), leaving to the programmer the
responsability of checking.
Ada has also an address attribute which return an address which is
more or less the equivalent of a void*, obviously without any
accessability check. Then you can convert the address to a pointer,
using a generic function.
You can also create dangling pointer with the generic procedure
unchecked_deallocation (which is the equivalent of delete).
BTW, using generic function in Ada is more heavy way than in C++ (you
need to explicitly instantiate it, giving it a name).
Yours,
--
Jean-Marc
Andrei Alexandrescu (See Website For Email)
> Well, I sort of agree. But my point was simply that the
> contribution of garbage collection to solving this problem is
> limited.
Limited but not nonexisting.
> As long as you have uninitialized pointers, pointers
> to local objects, and pointer arithmetic, you're going to have
> problems.
True. They ought to be restricted to reduce dangers.
> I might add that regardless of what other languages are doing,
> and despite the high cost in safety, I would not like to do
> without pointers to local objects. And with regards to "modern
> languages": how does Ada handle this? (Ada is the only modern
> language I know of which tries to be type safe, and also has
> local objects. Can you take the address of a local object? Are
> there restrictions which will prevent it from dangling?)
That can be resolved statically, and has been done. A pointer can have a
"scoped" attribute that restricts its use. A scoped pointer can
initialize another scoped pointer, but can't be duplicated in any other
way. Scoped pointers can be also initialized from non-scoped pointers.
With these rules, it's easy to prove that no scoped pointer will ever
dangle.
John Nagle
> ka...@gabi-soft.fr wrote:
>> I might add that regardless of what other languages are doing,
>> and despite the high cost in safety, I would not like to do
>> without pointers to local objects. And with regards to "modern
>> languages": how does Ada handle this? (Ada is the only modern
>> language I know of which tries to be type safe, and also has
>> local objects. Can you take the address of a local object? Are
>> there restrictions which will prevent it from dangling?)
>
>
> That can be resolved statically, and has been done. A pointer can have a
> "scoped" attribute that restricts its use. A scoped pointer can
> initialize another scoped pointer, but can't be duplicated in any other
> way. Scoped pointers can be also initialized from non-scoped pointers.
> With these rules, it's easy to prove that no scoped pointer will ever
> dangle.
I'd proposed something like that in my "strict C++" proposal.
(http://www.animats.com/papers/languages/)
The general idea is that if you want to work on a reference-counted
object for a while, you use a stack-based ref object to generate a
scoped reference to it. Creation of the stack-based ref
object increments the reference count, and when the ref object
goes out of scope, the reference count is decremented, of course.
Accessing the variable is through the ref object, which
outputs a scoped reference. That scoped reference can be
used for access, and passed to suitable functions, but can't be
copied to anything of larger scope. There's no reference
counting penalty for using the scoped reference.
It's a good idea, but generates the usual objections about
keywords and backwards compatibility. Most references could
be scoped references, but making that the default breaks code.
I tend to favor a "strict mode": optional, more
restrictive, and not entirely backwards compatible.
Some syntax establishes that a translation unit is considered
to be in strict mode. If all translation units are in strict
mode, the program is memory-safe.
With a strict mode, the normal case for references would
be "scoped".
John Nagle
Animats
David Abrahams
> David Abrahams wrote:
>> ka...@gabi-soft.fr writes:
>
>> > It's a useful tool. It is not a silver bullet. And making
>> > claims that it will eliminate all memory leaks, or all
>> > dangling pointers, or worse, all pointer bugs, only tends to
>> > discredit garbage collection in the eyes of those technical
>> > persons who are still evaluating the issues. Garbage
>> > collection is just a tool. It's a valuable tool, which can
>> > reduce the amount of work needed to write correct programs,
>> > at least in a number of cases. But if you're writing junk
>> > now, garbage collection isn't going to miraculously turn it
>> > into correct code.
>
>> This leads me to think the value of GC is really going to be
>> proportional to how much easier it makes reasoning about a
>> program's correctness.
>
> I'm not sure. I think much of the value is simply less code to
> write.
How often do you write destructors or memory deallocation? I can't
remember the last time I did any of that explicitly.
>> Will it improve things in that area at all? If an object
>> needing deterministic destruction can always be hidden at the
>> end of a chain of GC objects, it seems like not much of an
>> improvement.
>
> There are different things you have to reason about.
>
>> I also wonder how many applications need any deterministic
>> destruction.
>
> Most. You certainly don't want to deprive us of things like
> scoped_lock.
scoped_lock doesn't require deterministic destruction... except in
C++. In principle, it just requires an on_block_exit construct. Of
course, where C++ is concerned, we don't want to break backward
compatibility with things like scoped_lock. I guess using objects is
more natural than on_block_exit for many purposes, since the latter
construct doesn't give you a nice place to attach other functions like
unlock().
> A more valid question, in this context, is how
> many need deterministic destruction of non-local objects?
When I think about that question it gets kinda murky. Even if those
non-local objects have only memory resources, the choice to destroy
them eagerly or lazily has performance implications that I presume can
vary widely depending on the application.
> In my own application, there are three types of objects which
> need deterministic destruction:
>
> -- Local resources, like scoped_lock. But all of the instances
> of these objects are local variables, so they aren't
> affected by garbage collection.
>
> -- Transacion management. But this is really an extended case
> of local resources, and here, too, all of the instances of
> the variable are local.
>
> -- Explicitly destructed objects: these are dynamically
> allocated objects, but they are destructed explicitly as the
> result of a command, either external, or internally
> generated by a special thread (purge of out of date
> objects). In this case, garbage collection is neutral: it
> doesn't hurt, but it doesn't really help any, either.
>
> Garbage collection doesn't really affect any of them.
Definitely not, if you don't have GC ;-)
I can certainly imagine that _your mental model_ of C++ GC doesn't
affect them at all.
--
Dave Abrahams
Boost Consulting
www.boost-consulting.com
---
ka...@gabi-soft.fr
> ka...@gabi-soft.fr writes:
> > David Abrahams wrote:
> >> ka...@gabi-soft.fr writes:
> >> > It's a useful tool. It is not a silver bullet. And
> >> > making claims that it will eliminate all memory leaks, or
> >> > all dangling pointers, or worse, all pointer bugs, only
> >> > tends to discredit garbage collection in the eyes of
> >> > those technical persons who are still evaluating the
> >> > issues. Garbage collection is just a tool. It's a
> >> > valuable tool, which can reduce the amount of work needed
> >> > to write correct programs, at least in a number of cases.
> >> > But if you're writing junk now, garbage collection isn't
> >> > going to miraculously turn it into correct code.
> >> This leads me to think the value of GC is really going to
> >> be proportional to how much easier it makes reasoning about
> >> a program's correctness.
> > I'm not sure. I think much of the value is simply less code
> > to write.
> How often do you write destructors or memory deallocation? I
> can't remember the last time I did any of that explicitly.
Really? Destructors are pretty common: the compiler generated
default is inline, public and non-virtual. It's pretty rare
that I can accept all three conditions.
But less code can mean a lot of things. In my mind:
T* p ;
is less code than:
boost::shared_ptr< T > p ; // Or do I need a weak pointer here?
--
James Kanze GABI Software
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
martin...@yahoo.co.uk
ka...@gabi-soft.fr wrote:
> David Abrahams wrote:
> > How often do you write destructors or memory deallocation? I
> > can't remember the last time I did any of that explicitly.
>
> Really? Destructors are pretty common: the compiler generated
> default is inline, public and non-virtual. It's pretty rare
> that I can accept all three conditions.
Public:
That might be an issue (although most of my destructors /are/ public).
Inline:
What is your aversion to inline here? Normally, I understand that
inline functions increase the coupling between the implementation and
the use of a class. However in the case of a compiler generated
destructor I don't see how that applies; all the things that will
change the destructor (changes to the members and bases of the class),
will require clients to recompile anyway.
Non-virtual:
Hmm. The compiler generated destructor is only non virtual if none of
the base classes have a non virtual destructor. That gets very nearly
all of my classes right.
I would have thought the compiler generated destructor is usually
pretty close to what is required.
David Abrahams
> David Abrahams wrote:
>> How often do you write destructors or memory deallocation? I
>> can't remember the last time I did any of that explicitly.
>
> Really? Destructors are pretty common: the compiler generated
> default is inline, public and non-virtual. It's pretty rare
> that I can accept all three conditions.
>
> But less code can mean a lot of things. In my mind:
> T* p ;
> is less code than:
> boost::shared_ptr< T > p ; // Or do I need a weak pointer here?
Good points, all.
--
Dave Abrahams
Boost Consulting
www.boost-consulting.com
---
ka...@gabi-soft.fr
> ka...@gabi-soft.fr wrote:
> > David Abrahams wrote:
> > > How often do you write destructors or memory deallocation?
> > > I can't remember the last time I did any of that
> > > explicitly.
> > Really? Destructors are pretty common: the compiler
> > generated default is inline, public and non-virtual. It's
> > pretty rare that I can accept all three conditions.
> Public:
> That might be an issue (although most of my destructors /are/
> public).
For non-value classes, the only public destructors are the
virtual ones.
> Inline:
> What is your aversion to inline here? Normally, I understand
> that inline functions increase the coupling between the
> implementation and the use of a class. However in the case of
> a compiler generated destructor I don't see how that applies;
> all the things that will change the destructor (changes to the
> members and bases of the class), will require clients to
> recompile anyway.
General source code coupling. Suppose I add a trace statement
to the destructor (something that happens more often than one
might think). If the destructor is already declared, and not
inline, I recompile the module and relink. If the destructor is
inline, all of my client code recompiles.
Note too that if your class uses templates (e.g. smart
pointers), and the templates have a non-trivial destructor, the
class they are intantiated over must be defined whenever the
template destructor is invoked. If the destructor is inline,
this means in every compilation unit which uses the destructor,
which pretty much means that you must include the complete
definition of the type in your header file. If the destructor
isn't inline, often, a "class X ;" declaration is sufficient.
(I think that the Boost shared_ptr has a work-around for this,
but most resource handling classes don't.)
> Non-virtual:
> Hmm. The compiler generated destructor is only non virtual if
> none of the base classes have a non virtual destructor. That
> gets very nearly all of my classes right.
That's probably true.
Curiously enough, one of the rare cases where I do use an inline
destructor is for the virtual destructor of an interface -- all
of the other functions are pure virtual, and it seems too much
work to create and compile a separate source file just for the
empty destructor.
> I would have thought the compiler generated destructor is
> usually pretty close to what is required.
Sometimes. Not as often as I'd like. Let's face it: there's
not much difference between using an std::auto_ptr<> as a class
member, and putting a delete in the destructor. Except that if
you use the std::auto_ptr, the definition of what is pointed to
must be present in your header; if you use delete in your
destructor, a "class X ;" is typically sufficient.
--
James Kanze GABI Software
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
Steven T. Hatton
>
>
> John Nagle wrote:
>
> [snip]
>
>>
>> What puzzles me is that "operator->" was added, but
>> "operator." was not. I could see having neither, or
>> both, but not one of the two. If you can overload "operator->",
>> logically you should be able to overload "operator.".
>> What was the rationale there?
>>
>> John Nagle
>>
>
> See D&E 11.5.2 for a thorough discussion. I'll be very interested to
> see how far this goes given the issues Stroustrup discusses there.
> It's been quite a long discussion so far.
>
> Randy.
I don't have the book on hand, but if there is a compelling argument against
it, I would like to know what it is. Would you care to elaborate a bit on
what Stroustrup wrote about this?
--
STH
http://www.kdevelop.org
http://www.suse.com
http://www.mozilla.org
David Abrahams
>> I would have thought the compiler generated destructor is
>> usually pretty close to what is required.
>
> Sometimes. Not as often as I'd like. Let's face it: there's
> not much difference between using an std::auto_ptr<> as a class
> member, and putting a delete in the destructor. Except that if
> you use the std::auto_ptr, the definition of what is pointed to
> must be present in your header; if you use delete in your
> destructor, a "class X ;" is typically sufficient.
I don't think so.
Either one will delete an incomplete type, and either one will
generate undefined behavior when that type has a nontrivial
destructor.
--
Dave Abrahams
Boost Consulting
www.boost-consulting.com
---
John Nagle
> Randy wrote:
>
>
>>
>>John Nagle wrote:
>>
>> [snip]
>>
>>
>>> What puzzles me is that "operator->" was added, but
>>>"operator." was not. I could see having neither, or
>>>both, but not one of the two. If you can overload "operator->",
>>>logically you should be able to overload "operator.".
>>>What was the rationale there?
>>>
>>>John Nagle
>>>
>>
>>See D&E 11.5.2 for a thorough discussion. I'll be very interested to
>>see how far this goes given the issues Stroustrup discusses there.
>>It's been quite a long discussion so far.
>>
>>Randy.
>
>
> I don't have the book on hand, but if there is a compelling argument against
> it, I would like to know what it is. Would you care to elaborate a bit on
> what Stroustrup wrote about this?
The main problem Strostrup discusses is that if you encapsulate
some type that already has operator overloads in some class
that performs indirection, you have to repeat those operator
overloads in the encapsulating class. There's some discussion of
implicit invocation of "operator.()".
But his writing on that dates from around 1990, before
templates really worked. It's worth looking hard at that again,
since it's such an obstacle to smart pointer implementation.
Is there implicit invocation of "operator->()"?
After all, within a function member, a reference
to data member "foo", written as
foo
is equivalent to
this->foo
Is "operator->()" ever invoked in that situation?
John Nagle
Bob Bell
If you mean a user-written operator->(), I don't see why it would;
"this" is a raw pointer, and users can't write an operator->() for raw
pointer types.
Bob
Steven T. Hatton
> Steven T. Hatton wrote:
>> Randy wrote:
>>
>>
>>>
>>>John Nagle wrote:
>>>
>>> [snip]
>>>
>>>
>>>> What puzzles me is that "operator->" was added, but
>>>>"operator." was not. I could see having neither, or
>>>>both, but not one of the two. If you can overload "operator->",
>>>>logically you should be able to overload "operator.".
>>>>What was the rationale there?
>>>>
>>>>John Nagle
>>>>
>>>
>>>See D&E 11.5.2 for a thorough discussion. I'll be very interested to
>>>see how far this goes given the issues Stroustrup discusses there.
>>>It's been quite a long discussion so far.
>>>
>>>Randy.
>>
>>
>> I don't have the book on hand, but if there is a compelling argument
>> against
>> it, I would like to know what it is. Would you care to elaborate a bit
>> on what Stroustrup wrote about this?
>
> The main problem Strostrup discusses is that if you encapsulate
> some type that already has operator overloads in some class
> that performs indirection, you have to repeat those operator
> overloads in the encapsulating class.
One approach to that situation might be the "don't do that" rule. But such
trivial injunctions rarely deter determined C++ programmers.
> There's some discussion of
> implicit invocation of "operator.()".
>
> But his writing on that dates from around 1990, before
> templates really worked. It's worth looking hard at that again,
> since it's such an obstacle to smart pointer implementation.
One question I've come to ponder is whether such a solution really needs to
be absolute. I saw your comments in this regard, and can appreciate your
sentiments, but can we not create something sufficiently predictable,
understandable, and above all usable that isn't 100% airtight?
Specifically, I believe the desire for overloading the remaining member
access operators in C++ come form the desire to prevent "inadvertent"
access to the "raw" data held by a smart handle (pointer or reference).
But if you provide a reasonable usable and comprehendable design that
provides all that raw access would provide, is the informed user going to
take advantage of the fact that you really did hand him or her a genuine
reference rather than an interface that behaves like a reference?
I use a lot of programs written in C++, often hot out of the development
repository, and I don't get too very many crashes. Rarely do I get a crash
or other major problem from a release version - if ever I run something
that ancient.
My problem with some of the cautionary measures I've seen implemented in C++
is that they put Rube Goldberg to shame. Real solutions are tight and
compact. They make sense and they don't have a lot of moving parts.
> Is there implicit invocation of "operator->()"?
> After all, within a function member, a reference
> to data member "foo", written as
>
> foo
>
> is equivalent to
>
> this->foo
>
> Is "operator->()" ever invoked in that situation?
>
> John Nagle
Funny you should ask that:
Message-ID:<PsadnTRNjPP...@speakeasy.net>
> I find writing things such as (*ptr)(arg), and (*ptr)[i], to be at least
> awkward. I've often thought the following would be useful as an
> alternative form of ptr->operator(arg), ptr->operator[i]:
>
> ptr->(arg)
> ptr->[i]
>
> Some people may find such an expression unintelligible. If it could be
made
> to work, I believe it would be quickly learned, and probably used by a
good
> number of programmers. Does anybody know of a purely technical reason
that
> functionality couldn't be implemented if it were added to the standard?
I've taken an interest in this proposal:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2004/n1671.pdf
---
martin...@yahoo.co.uk
David Abrahams wrote:
> ka...@gabi-soft.fr writes:
>
> >> I would have thought the compiler generated destructor is
> >> usually pretty close to what is required.
> >
> > Sometimes. Not as often as I'd like. Let's face it: there's
> > not much difference between using an std::auto_ptr<> as a class
> > member, and putting a delete in the destructor. Except that if
> > you use the std::auto_ptr, the definition of what is pointed to
> > must be present in your header; if you use delete in your
> > destructor, a "class X ;" is typically sufficient.
>
> I don't think so.
>
> Either one will delete an incomplete type, and either one will
> generate undefined behavior when that type has a nontrivial
> destructor.
If you use a delete in the destructor, 'class X;' is sufficient in the
header. The .CPP file will need to ' #include "x.h" '. In other words
your class is coupled to the X header.
With std::auto_ptr, you need to ' #include "x.h" ' in your header,
which means that all the /users/ of your class will be coupled to the X
header.
John Nagle
> One question I've come to ponder is whether such a solution really needs to
> be absolute. I saw your comments in this regard, and can appreciate your
> sentiments, but can we not create something sufficiently predictable,
> understandable, and above all usable that isn't 100% airtight?
> Specifically, I believe the desire for overloading the remaining member
> access operators in C++ come form the desire to prevent "inadvertent"
> access to the "raw" data held by a smart handle (pointer or reference).
> But if you provide a reasonable usable and comprehendable design that
> provides all that raw access would provide, is the informed user going to
> take advantage of the fact that you really did hand him or her a genuine
> reference rather than an interface that behaves like a reference?
We have that now. And such shared pointers aren't used much,
even though unreliable implementations are widely available.
To compete with the more modern languages with memory safety,
Java and C#, C++ will have to do better than that.
Inaccurate reference counting systems result in non-local defects,
where the cause of the problem is often far from the place where
the problem is detected. Those are difficult defects to find,
especially in large systems.
Memory safety defects are the single largest cause of program
crashes and security holes. Every remaining major language other
than C and C++ has language level support to deal with memory
safety. It is no longer acceptable to ignore
the problem.
John Nagle
Animats
Steven T. Hatton
> Steven T. Hatton wrote:
>
>> One question I've come to ponder is whether such a solution really needs
>> to
>> be absolute. I saw your comments in this regard, and can appreciate your
>> sentiments, but can we not create something sufficiently predictable,
>> understandable, and above all usable that isn't 100% airtight?
>> Specifically, I believe the desire for overloading the remaining member
>> access operators in C++ come form the desire to prevent "inadvertent"
>> access to the "raw" data held by a smart handle (pointer or reference).
>> But if you provide a reasonable usable and comprehendable design that
>> provides all that raw access would provide, is the informed user going to
>> take advantage of the fact that you really did hand him or her a genuine
>> reference rather than an interface that behaves like a reference?
>
> We have that now. And such shared pointers aren't used much,
> even though unreliable implementations are widely available.
> To compete with the more modern languages with memory safety,
> Java and C#, C++ will have to do better than that.
>
> Inaccurate reference counting systems result in non-local defects,
> where the cause of the problem is often far from the place where
> the problem is detected. Those are difficult defects to find,
> especially in large systems.
OK, I think I see your point. It's really not the frequency of the possible
problems that concerns you. It's the magnitude of those few which may
arise.
> Memory safety defects are the single largest cause of program
> crashes and security holes. Every remaining major language other
> than C and C++ has language level support to deal with memory
> safety. It is no longer acceptable to ignore
> the problem.
Well, to be pedantic, it is not only the language, but the execution
environment that provides the protection. The Java Virtual Machine is
written in C and C++ using features such as direct assembly code
invocation, etc., which are twice removed from these safer languages. It
is with C and C++ that Sun Microsystems protects the Java programmer form
himself and the world form him.
Nonetheless, the point is well taken. C++ could and should provide a far
better means of managing memory under typical circumstances. The problems
I've described regarding reference counting are real problems for me. They
became a stumbling block when I tried to write my own collection of
functors for 3D simulations. I was aware of, and comfortable with using,
OSG's Referenced class, but I wanted to write a self-contained library, and
wanted a bit more flexibility and completeness than OSG's solution
provides.
That put me into thrash mode. I looked at several people's approaches to
this problem, and none of them provided a comprehensive approach at the
level of abstraction that seems appropriate.
I believe there exists a set of general problems that can be abstracted and
defined categorically. Clearly many of the more common ones have been
addressed with the Boost Smart Pointers. I really like the looks of
unique_ptr<>, though I haven't used it.
With the exception of boost::intrusive_ptr<> these seem to assume that the
problem really isn't a significant need to share objects between handles,
that sharing objects is just part of the solution to the 'real problem'. I
suspect the assumption is valid in many cases, and thus the offerings are
sufficient for that problem domain.
What this proposal seems to offer is a way of creating handles that behave
almost exactly like the object they handle. I will admit that I don't have
the solid background that may of the people on the Standards Committee
have, but from my limited experience with C++, I am struggling to see any
significant cost in accepting the proposal in comparison to the apparent
benefit.
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2004/n1671.pdf
To my untrained eye, this looks like the closest thing to a silver bulled
I've ever seen.
---
John Nagle
> John Nagle wrote:
>
>
>>Steven T. Hatton wrote:
>> Inaccurate reference counting systems result in non-local defects,
>>where the cause of the problem is often far from the place where
>>the problem is detected. Those are difficult defects to find,
>>especially in large systems.
>
>
> OK, I think I see your point. It's really not the frequency of the possible
> problems that concerns you. It's the magnitude of those few which may
> arise.
And their non-local effects. The key issue is that memory safety
problems in one part of a program generate faults in other, unrelated
parts of the program. That's why memory safety is such an issue,
and one which becomes more significant with program size.
>> Memory safety defects are the single largest cause of program
>>crashes and security holes. Every remaining major language other
>>than C and C++ has language level support to deal with memory
>>safety. It is no longer acceptable to ignore
>>the problem.
>
>
> Well, to be pedantic, it is not only the language, but the execution
> environment that provides the protection.
No. It's purely a language issue. It's possible to have unsafe
languages on a virtual machine, and safe languages on a real machine.
Hard-code compilers for Java exist. See "http://gcc.gnu.org/java".
> That put me into thrash mode.
?
> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2004/n1671.pdf
That's reasonable, but needs to be firmed up a bit. Are their
serious objections to that proposal?
Calling functions of the proxy class via "operator&()"(see 4.2.2)
is ugly, but probably acceptable.
John Nagle
Animats