optional_ptr - A smart pointer which optionally owns the resource

[Matthew Fioravante]

For all resources, we have owners who manage the lifetime of the resource and viewers who view the resource. To be correct, all of the viewers have to stop using the pointer before the last owner goes out of scope and destroys the resource.

 

For both unique_ptr and shared_ptr, the set of owners and viewers has to be decided explicitly ahead of time. This can very limiting because I've run into cases where I have types which 99% of the time are viewers (because the resource is often shared) but in some rare contexts need to be an owner because the object is being used standalone with no external entity to manage the ownership.

 

The heavy handed solution to use shared_ptr everywhere. Now everyone is an owner. Using shared_ptr adds runtime overhead to manage the reference count. The biggest problem with shared_ptr is that once you use it in one place, you have to use it everywhere. Everything now must be dynamically allocated individually. You can't allocate 20 objects in an array and view one of them. You can't create objects on the stack or as a data member. Forcing individual dynamic allocation is a performance killer. We use C++ because we need to be fast and a big part of being fast beings avoiding allocations or failing that, batching them into fewer allocations of large contiguous cache friendly chunks.

 

The solution I'm toying with now is something called optional_ptr. Maybe its not such a great name because it could be confused with std::optional but lets leave bikeshedding aside for now.

 

Essentially, optional_ptr is a move only type that works like a variant<T*,unique_ptr<T>>. You can construct it with a raw pointer for non-owning semantics or construct it with a unique_ptr to get owning semantics.

 

The naive implementation of optional_ptr is to use T* and a bool, but for any type with alignof(T) > 1, you can actually store the ownership bit in lowest bit of the pointer itself since that bit will always be 0 due to alignment. The result is that OptionalPtr has minimal runtime overhead over unique_ptr/T* and for most types zero space overhead as well.

 

Using optional_ptr means an entity has the flexibility to be either an owner or a viewer. Move only ensures unique ownership just like unique_ptr.

 

Here is a sketch of what it might look like:

 

template <typename T>
class optional_ptr {
public:
  optional_ptr() = default;
  optional_ptr(T* p) : _p(p), _own(false) {}
  optional_ptr(unique_ptr<T> p) : _p(p.release()), _own(true) {}
  ~optional_ptr() { if(_own) delete p; }

  optional_ptr(const optional_ptr<T>&) = delete;
  optional_ptr<T>& operator=(const optional_ptr<T>&) = delete;

  optional_ptr(optional_ptr<T>&&) noexcept;
  optional_ptr<T>& operator=(optional_ptr<T>&&) noexcept;
 
  bool owns() const { return _own; }
  T* get() const { return _p; }
  T* release() noexcept { auto* p = _p; _own = false; _p = nullptr; return p; }
  void reset(T* p);
  void reset(unique_ptr<T> p);

  //etc...
private:
  T* _p = nullptr;
  bool _own = false;
};

How have you dealt with the problem of having a viewer which sometimes needs to be an owner? Do you see this as a valid problem that needs addressing or an anti-pattern?

 

 

 

[Andrey Semashev]

On Thu, Sep 24, 2015 at 7:35 PM, Matthew Fioravante
<fmatth...@gmail.com> wrote:
> For all resources, we have owners who manage the lifetime of the resource
> and viewers who view the resource. To be correct, all of the viewers have to
> stop using the pointer before the last owner goes out of scope and destroys
> the resource.

If anything, this sounds fragile and dangerous.

But, in case you can easily guarantee that the owner outlives the
viewers, did you consider using unique_ptr with a custom deleter?

[Nicol Bolas]

On Thursday, September 24, 2015 at 12:35:28 PM UTC-4, Matthew Fioravante wrote:

For all resources, we have owners who manage the lifetime of the resource and viewers who view the resource. To be correct, all of the viewers have to stop using the pointer before the last owner goes out of scope and destroys the resource.

 

For both unique_ptr and shared_ptr, the set of owners and viewers has to be decided explicitly ahead of time. This can very limiting because I've run into cases where I have types which 99% of the time are viewers (because the resource is often shared) but in some rare contexts need to be an owner because the object is being used standalone with no external entity to manage the ownership.

... This seems to be a problem with how you're thinking of types and ownerships.

Types do not define ownership. Or at least, not in the way you seem to mean. What defines ownership is code, at compile time.

If I have some function:

void do_something(unique_ptr<T> p);

That function is explicitly claiming ownership of the object it is given. The fact that it claims ownership is an integral part of what it means to `do_something`.

If I have this function:

void do_something2(T *p);

It is (by convention) not claiming ownership of `p`.

If you want to call `do_something2`, and you have a `unique_ptr<T>`, simply call `get` to fetch the pointer.

If you have a T* and you want to call `do_something`... you can't. You have no right to do so. Because you do not own that object, and you therefore cannot transfer something you don't have.

That's just a good interface.

I'm just having a hard time imagining what a hypothetical `do_something3` function would actually be doing if it does not statically know if it's claiming ownership or not.

If the `do_something3` is a free function, one for which the parameter is not stored beyond its scope, then the concept of ownership is pointless. Because again, you just pass it a T*. If the caller has a `unique_ptr`, and is finished with that `unique_ptr` immediately after calling `do_something2`, they can simply destroy it right after the call.

So this is only meaningful if you have an object which may or may not own some other object it is given. Can you give a more concrete example of an object that cannot statically know if expects to be given ownership of an object?

The heavy handed solution to use shared_ptr everywhere. Now everyone is an owner. Using shared_ptr adds runtime overhead to manage the reference count. The biggest problem with shared_ptr is that once you use it in one place, you have to use it everywhere. Everything now must be dynamically allocated individually. You can't allocate 20 objects in an array and view one of them. You can't create objects on the stack or as a data member.

You can in fact do all of those things with `shared_ptr`. Simply provide the initial shared_ptr<T> with a deleter that doesn't delete it.

Sure, you still have the reference count overhead which now accomplishes nothing. But other than that, it's a perfectly functional solution.

Essentially, optional_ptr is a move only type that works like a variant<T*,unique_ptr<T>>.

So why not just use a variant<T*, unique_ptr<T>>? Just create a simple visitor to get a T* out of it.

Because `unique_ptr` is move only, so too will the `variant` containing it. It does everything you want, just not quite with the interface you'd prefer.

Not only that, it gives you the flexibility to ask the question: do I actually have ownership of this object? Your proposed interface doesn't provide taht facility. Which means that your class can only talk to code that either itself doesn't know or that takes a T*. It can't call code that takes a unique_ptr<T>.

Also, remember that `unique_ptr` has options as well. It could take an array, and it has a deleter type. With the `variant` case, users can specify options as needed. In your proposed syntax, they can only use the conceptual equivalent of unique_ptr<T>.

Given that this variant solves your problem adequately, and that your use-case is fairly niche, I'd say to take the 80% solution and move on.

[Jeffrey Yasskin]

We have an optional ownership pattern at Google, involving storing an enum next to the pointer. It's fairly bad, but it gets the job done. I suspect an optional-ownership class isn't the most important thing the committee could be spending time on, but it's not crazy.

Using unique_ptr deleters for this is clever, which is as much criticism as it is praise. Having a deleter that doesn't delete makes uses of unique_ptr harder to read, and to some extent is an indication that move-only types are too hard to write.

Jeffrey

--

---
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposal...@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.
Visit this group at http://groups.google.com/a/isocpp.org/group/std-proposals/.

[Matthew Fioravante]

On Thursday, September 24, 2015 at 12:54:55 PM UTC-4, Andrey Semashev wrote:

On Thu, Sep 24, 2015 at 7:35 PM, Matthew Fioravante

<fmatth...@gmail.com> wrote:
> For all resources, we have owners who manage the lifetime of the resource
> and viewers who view the resource. To be correct, all of the viewers have to
> stop using the pointer before the last owner goes out of scope and destroys
> the resource.

If anything, this sounds fragile and dangerous.

 

This was my concern as well and what I anticipate to be the biggest argument against this idiom. However I'm not yet entirely convinced that its any less dangerous to the situation where you have a unique_ptr owned by A and viewed by B as a T*. Its up to the programmer to ensure that B's lifetime is within A's lifetime.

 

If someone can identify a situation where optional_ptr is much more dangerous than unique_ptr/T* that would be really helpful.

 

 

But, in case you can easily guarantee that the owner outlives the
viewers, did you consider using unique_ptr with a custom deleter?

 

 I'm not sure how a custom deleter would help. If the deleter does nothing, then you just have T*, if the deleter conditionally deletes the object based on a boolean state, you have my optional_ptr. A reusable common pattern should have a name, not be a backwards hack on a unique_ptr deleter.

 

 

On Thursday, September 24, 2015 at 1:36:37 PM UTC-4, Nicol Bolas wrote:

On Thursday, September 24, 2015 at 12:35:28 PM UTC-4, Matthew Fioravante wrote:

For all resources, we have owners who manage the lifetime of the resource and viewers who view the resource. To be correct, all of the viewers have to stop using the pointer before the last owner goes out of scope and destroys the resource.

 

For both unique_ptr and shared_ptr, the set of owners and viewers has to be decided explicitly ahead of time. This can very limiting because I've run into cases where I have types which 99% of the time are viewers (because the resource is often shared) but in some rare contexts need to be an owner because the object is being used standalone with no external entity to manage the ownership.

... This seems to be a problem with how you're thinking of types and ownerships.

Types do not define ownership. Or at least, not in the way you seem to mean. What defines ownership is code, at compile time.

 

Not sure I agree, unless I'm misunderstanding you somehow. If a type stores an object by unique_ptr or shared_ptr, that type defines ownership of the resource. If the type stores by T*, he defines non-ownership. 

 

 

If I have some function:

void do_something(unique_ptr<T> p);

That function is explicitly claiming ownership of the object it is given. The fact that it claims ownership is an integral part of what it means to `do_something`.

If I have this function:

void do_something2(T *p);

It is (by convention) not claiming ownership of `p`.

If you want to call `do_something2`, and you have a `unique_ptr<T>`, simply call `get` to fetch the pointer.

If you have a T* and you want to call `do_something`... you can't. You have no right to do so. Because you do not own that object, and you therefore cannot transfer something you don't have.

That's just a good interface.

I'm just having a hard time imagining what a hypothetical `do_something3` function would actually be doing if it does not statically know if it's claiming ownership or not.

 

If a function needs to take ownership of a resource, it gets a unique_ptr. If it needs to use/view the resource someone else owns, it gets T*. If it needs to view/use the resource and also allow optionally taking ownership, use an optional_ptr. What choice was taken is clearly documented at the call site because you have to construct optional_ptr by moving in a unique_ptr if you want it to be an owner.

 

I'm using unique_ptr in my code a lot and I'm often having to create extra structures just to store the unique_ptr's when the actual consumer in this context only accepts a T*.

 

 

If the `do_something3` is a free function, one for which the parameter is not stored beyond its scope, then the concept of ownership is pointless.

 

Of course, for a function that is not a sink you should never pass any kind of smart pointer. That seems like a general guideline outside of the scope of this discussion.

 

Because again, you just pass it a T*. If the caller has a `unique_ptr`, and is finished with that `unique_ptr` immediately after calling `do_something2`, they can simply destroy it right after the call.

So this is only meaningful if you have an object which may or may not own some other object it is given. Can you give a more concrete example of an object that cannot statically know if expects to be given ownership of an object?

 

Sure, one real example I have is a pipeline. lets say we have pipeline that manages 3 data streams A, B, and C with a configurable number of stages. Each stage can apply some transformation to A, B, or C or just pass it along. When a stage needs to modify a stream, he gets a view of the previous stream, allocates and takes ownership of a new one, and then applies his transformations to convert A -> A`. If he does not need to modify a stream, he just passes along a view of it.

 

This pattern can be accomplished using shared_ptr but again its unecessary. It can also be accomplished with unique_ptr but then each stage has to make a copy of each stream and copy the data even if he does not modify it. Every pipeline stage that has a non-owning pointer is guaranteed that the pointer is owned by a previous stage. When the stages are destroyed in reverse order, all resources are released correctly with epsilon overhead.

 

 

The heavy handed solution to use shared_ptr everywhere. Now everyone is an owner. Using shared_ptr adds runtime overhead to manage the reference count. The biggest problem with shared_ptr is that once you use it in one place, you have to use it everywhere. Everything now must be dynamically allocated individually. You can't allocate 20 objects in an array and view one of them. You can't create objects on the stack or as a data member.

You can in fact do all of those things with `shared_ptr`. Simply provide the initial shared_ptr<T> with a deleter that doesn't delete it.

Sure, you still have the reference count overhead which now accomplishes nothing. But other than that, it's a perfectly functional solution.

 

Aside from the wasted reference count overhead this seems very brittle. Its very easy to construct a shared_ptr and forget to pass in that noop deleter. 

 

 

Essentially, optional_ptr is a move only type that works like a variant<T*,unique_ptr<T>>.

So why not just use a variant<T*, unique_ptr<T>>? Just create a simple visitor to get a T* out of it.

Because `unique_ptr` is move only, so too will the `variant` containing it. It does everything you want, just not quite with the interface you'd prefer.

 

Again for a reusable pattern, I'd rather the thing had a name. Also variant can't take advantage of the optimization I mentioned of storing the bit in the pointer. Having a smart pointer type that stores { T*, bool } is very bad because sizeof() will be equal to 2 * T* due to padding. If you have a lot of these you will waste of a lot of valuable cache line space.

 

Not only that, it gives you the flexibility to ask the question: do I actually have ownership of this object? Your proposed interface doesn't provide taht facility.

 

Sure you can, the owns() method tells you whether or not you have it, but I'm not sure thats actually very useful in practice.

 

Which means that your class can only talk to code that either itself doesn't know or that takes a T*.

 

This is the intended use case. Once I've given up static proof that the object owns the resource I can't get it back. Once you reduce from unique_ptr to optional_ptr can only stay at the optional_ptr level or reduce further to viewing via T*.

 

 

It can't call code that takes a unique_ptr<T>.

 

Its not designed to ever call code that takes a unique_ptr. You can't blindly pass a T* someone else owns to a unique_ptr. You also can't pass something that dynamically may or may not own to something that staticly always owns.

 

If you really had to, you could check the owns() flag and pass it mechanically using release() but I would consider this use a huge anti-pattern hack.

 

Also, remember that `unique_ptr` has options as well. It could take an array, and it has a deleter type. With the `variant` case, users can specify options as needed. In your proposed syntax, they can only use the conceptual equivalent of unique_ptr<T>.

 

optional_ptr can have deleters and support arrays too. I've left all that out for now because its not central to the first question of whether this is actually a good pattern and second whether it has a place in the standard library.

[Matthew Fioravante]

On Thursday, September 24, 2015 at 1:53:47 PM UTC-4, Jeffrey Yasskin wrote:

We have an optional ownership pattern at Google, involving storing an enum next to the pointer. It's fairly bad, but it gets the job done.

 

Hi Jeffrey, could you expand on some specific use cases? Why did you feel the need to invent optional ownership? Why were the built in unique and shared ownership patterns not adequate?

 

 

I suspect an optional-ownership class isn't the most important thing the committee could be spending time on, but it's not crazy.

Using unique_ptr deleters for this is clever, which is as much criticism as it is praise. Having a deleter that doesn't delete makes uses of unique_ptr harder to read, and to some extent is an indication that move-only types are too hard to write.

 

optional_ptr has to be a move only type because it might own even though copying is perfectly valid if it does not own. It feels a bit odd but in practice it hasn't been a problem for me. If you want to copy that means what you really want is to copy the pointer and have a view, in which case you can call .get() when can then be passed on to T* or another optional_ptr which will not be an owner.

[Jeffrey Yasskin]

On Thu, Sep 24, 2015 at 11:32 AM, Matthew Fioravante <fmatth...@gmail.com> wrote:

On Thursday, September 24, 2015 at 1:53:47 PM UTC-4, Jeffrey Yasskin wrote:

We have an optional ownership pattern at Google, involving storing an enum next to the pointer. It's fairly bad, but it gets the job done.

 

Hi Jeffrey, could you expand on some specific use cases? Why did you feel the need to invent optional ownership? Why were the built in unique and shared ownership patterns not adequate?

Many uses are in wrappers around other data, for example, a gunzip view on a string or a filtered view of a table, where the library fundamentally only needs a view on the data, but in many cases the user doesn't want to deal with stashing ownership of the data somewhere else. Uses are about evenly split between ones that want the library to take ownership and ones that don't.

 

I suspect an optional-ownership class isn't the most important thing the committee could be spending time on, but it's not crazy.

Using unique_ptr deleters for this is clever, which is as much criticism as it is praise. Having a deleter that doesn't delete makes uses of unique_ptr harder to read, and to some extent is an indication that move-only types are too hard to write.

 

optional_ptr has to be a move only type because it might own even though copying is perfectly valid if it does not own. It feels a bit odd but in practice it hasn't been a problem for me. If you want to copy that means what you really want is to copy the pointer and have a view, in which case you can call .get() when can then be passed on to T* or another optional_ptr which will not be an owner.

 

I wasn't questioning that. :) In general, we should be willing to allocate new names, instead of re-using unique_ptr for every move-only type.

[Andrzej Krzemieński]

W dniu czwartek, 24 września 2015 18:54:55 UTC+2 użytkownik Andrey Semashev napisał:

On Thu, Sep 24, 2015 at 7:35 PM, Matthew Fioravante
<fmatth...@gmail.com> wrote:
> For all resources, we have owners who manage the lifetime of the resource
> and viewers who view the resource. To be correct, all of the viewers have to
> stop using the pointer before the last owner goes out of scope and destroys
> the resource.

If anything, this sounds fragile and dangerous.

I found myself a couple of times in need of such functionality. I called my version `optional_owner`. For sure, it is inferior to unique_ptr (for exclusive ownership) or a raw reference (for no ownership), provided you can use either. But there are some rare cases of resorce-managing schemes where unique_ptr will not do. In these cases people resort to shared_ptr, and it works, but incorrectly suggest shared ownership, which does not in fact take place.

My use case was this: I have an OO-like interface (base class with virtual functions), with 4 implementations, one of which is a singleton. In 3 other cases I have a factory function that allocates the implementation on the heap and returns a unique_ptr. In the 4th (singleton) case I return a reference to the single object. My colleagues always solve these problems by turning everything into shared_ptr (as though it was an ultimate garbage collection solution to every management problem). I would prefer they used optional_owner instead. Not everywhere: only in specific, rare, complicated memory management situations.

Regards,
&rzej  

[Matthew Fioravante]

Nicol, I can provide another general use case, again related to building pipelines.

 

Lets imagine we have data streams. In my example, each data stream will an object represented by a letter.

 

For example, we have can a stream A, and another stream B(A) which takes A, performs some transformations, and then produces new stream B. B accesses A by storing a pointer to A.

 

A common case is one producer and many consumers.

 

A

B(A)

C(A)

D(A)

 

In this case, A can be owned externally (via unique_ptr or directly on the stack/data member) and B, C, and D get a view to it.

 

However we can also have this scenario.

 

D(C(B(A))).

 

Here, maybe I'm only interested in the output of D, not the intermediate C, B, and A streams. A, B, and C may even be completely hidden via a factory function that creates D. In this case, the most natural thing is to just let them tag along with D and let D own them. Since D is the only consumer, once D goes away C, B, and A can all be destroyed. I don't have to manage any external object to maintain the unique_ptr's that own A, B, and C.

 

Of course this can all be solved by shared_ptr, but for reasons mentioned earlier that has a lot of drawbacks. In my mind shared_ptr is only really needed when you cannot build a heirarchical scoped architecture to manage the lifetime invariants for you. I try to avoid it unless absolutely necessary.

 

Essentially using unique_ptr/T* means that you always have to have an external manager object to take ownership of the lifetimes. If you want to construct an producer and consumer in isolation you also need a third object to hold the unique_ptr object. With optional_ptr, the consumer can just take the producer if needed.

 

 

On Thursday, September 24, 2015 at 2:46:51 PM UTC-4, Jeffrey Yasskin wrote:

On Thu, Sep 24, 2015 at 11:32 AM, Matthew Fioravante <fmatth...@gmail.com> wrote:

On Thursday, September 24, 2015 at 1:53:47 PM UTC-4, Jeffrey Yasskin wrote:

We have an optional ownership pattern at Google, involving storing an enum next to the pointer. It's fairly bad, but it gets the job done.

Hi Jeffrey, could you expand on some specific use cases? Why did you feel the need to invent optional ownership? Why were the built in unique and shared ownership patterns not adequate?

Many uses are in wrappers around other data, for example, a gunzip view on a string or a filtered view of a table, where the library fundamentally only needs a view on the data, but in many cases the user doesn't want to deal with stashing ownership of the data somewhere else. Uses are about evenly split between ones that want the library to take ownership and ones that don't.

 

That sounds pretty similar to my use case, another pipeline.

 

 

 

 

 

 

[Nicol Bolas]

On Thursday, September 24, 2015 at 2:21:31 PM UTC-4, Matthew Fioravante wrote:

On Thursday, September 24, 2015 at 12:54:55 PM UTC-4, Andrey Semashev wrote:

On Thu, Sep 24, 2015 at 7:35 PM, Matthew Fioravante
<fmatth...@gmail.com> wrote:
> For all resources, we have owners who manage the lifetime of the resource
> and viewers who view the resource. To be correct, all of the viewers have to
> stop using the pointer before the last owner goes out of scope and destroys
> the resource.

If anything, this sounds fragile and dangerous.

 

This was my concern as well and what I anticipate to be the biggest argument against this idiom. However I'm not yet entirely convinced that its any less dangerous to the situation where you have a unique_ptr owned by A and viewed by B as a T*. Its up to the programmer to ensure that B's lifetime is within A's lifetime.

 

If someone can identify a situation where optional_ptr is much more dangerous than unique_ptr/T* that would be really helpful.

 

 

But, in case you can easily guarantee that the owner outlives the
viewers, did you consider using unique_ptr with a custom deleter?

 

 I'm not sure how a custom deleter would help. If the deleter does nothing, then you just have T*, if the deleter conditionally deletes the object based on a boolean state, you have my optional_ptr. A reusable common pattern should have a name, not be a backwards hack on a unique_ptr deleter.

 

 

On Thursday, September 24, 2015 at 1:36:37 PM UTC-4, Nicol Bolas wrote:

On Thursday, September 24, 2015 at 12:35:28 PM UTC-4, Matthew Fioravante wrote:

For all resources, we have owners who manage the lifetime of the resource and viewers who view the resource. To be correct, all of the viewers have to stop using the pointer before the last owner goes out of scope and destroys the resource.

 

For both unique_ptr and shared_ptr, the set of owners and viewers has to be decided explicitly ahead of time. This can very limiting because I've run into cases where I have types which 99% of the time are viewers (because the resource is often shared) but in some rare contexts need to be an owner because the object is being used standalone with no external entity to manage the ownership.

... This seems to be a problem with how you're thinking of types and ownerships.

Types do not define ownership. Or at least, not in the way you seem to mean. What defines ownership is code, at compile time.

 

Not sure I agree, unless I'm misunderstanding you somehow. If a type stores an object by unique_ptr or shared_ptr, that type defines ownership of the resource. If the type stores by T*, he defines non-ownership. 

What I mean is that code chooses the ownership method. You pick which type to use based on what ownership that code specifically needs.

Or more to the point, you don't use `unique_ptr` because it provides unique ownership. You use `unique_ptr` because a particular piece of code needs to uniquely own an object that's heap-allocated. The code defines the need for ownership; the type is simply how you implement the ownership model.

Code is static, while your maybe_owns pointer is dynamic. It's not clear to me what kind of code requires an ownership model that is dynamic.

 

 

 

 

Because again, you just pass it a T*. If the caller has a `unique_ptr`, and is finished with that `unique_ptr` immediately after calling `do_something2`, they can simply destroy it right after the call.

So this is only meaningful if you have an object which may or may not own some other object it is given. Can you give a more concrete example of an object that cannot statically know if expects to be given ownership of an object?

 

Sure, one real example I have is a pipeline. lets say we have pipeline that manages 3 data streams A, B, and C with a configurable number of stages. Each stage can apply some transformation to A, B, or C or just pass it along. When a stage needs to modify a stream, he gets a view of the previous stream, allocates and takes ownership of a new one, and then applies his transformations to convert A -> A`. If he does not need to modify a stream, he just passes along a view of it.

Allow me to restate this, so that we can make sure I'm correctly understanding what you've described in C++ terms.

You have these stage constructs. They're pass 3 objects and return three objects to the next stage in the sequence. They may modify or not modify those objects as they see fit as part of their processing.

Your implementation of this concept is that the 3 objects, the data streams, are immutable. Stages don't actually modify them; they create a modified copy of them. And therefore, a stage may have heap-allocated one of streams, or it may simply pass the object from the previous stage through.

It's not clear to me why `unique_ptr` doesn't work here. Maybe I'm missing something here.

To me, each stage is effectively a function of this form:

void stage(unique_ptr<A> streamA, unique_ptr<B> streamB, unique_ptr<C> streamC);

By taking `unique_ptr` as a value, that ensures that each stage owns the object for the duration of that function call. When one stage calls the next, it passes them pointers. The previous stage therefore transfers ownership to the next.

If a stage needs to create a modified copy, they simply do this:

unique_ptr<A> oldValue = std::move(streamA);
streamA = make_unique(...);

Or they create the new stream's `unique_ptr` as a local variable and explicitly move it into the next stream.

Each stage owns the objects it gets, but only for the duration of its function call. I don't see why stages need to pass optional ownership.

This pattern can be accomplished using shared_ptr but again its unecessary. It can also be accomplished with unique_ptr but then each stage has to make a copy of each stream and copy the data even if he does not modify it.

How does using `unique_ptr` require copying of the data? It's just passing ownership of an object through to the next stage. Once a stream is passed to the next stage, the previous stage has no right to access it anymore.

So where is this need to copy each stream coming from? Why does a stage need to hold onto the objects it was given, instead of transfering ownership to the next stage?

If a stage needs a modified copy, it can preserve the old object as shown above. But if it doesn't, then it should just pass ownership to the next stage. So rather than having each stage store a non-owning pointer that's owned by a previous stage, each stage stores an owning pointer that is owned only so long as that stage is processing, or if it's an old copy left behind.

So I remain confused as to when you would need this functionality.
 

[Andrey Semashev]

On 24.09.2015 21:21, Matthew Fioravante wrote:
>
> On Thursday, September 24, 2015 at 12:54:55 PM UTC-4, Andrey Semashev wrote:
>
> On Thu, Sep 24, 2015 at 7:35 PM, Matthew Fioravante
> <_fmatth...@gmail.com_ <javascript:>> wrote:
> > For all resources, we have owners who manage the lifetime of the
> resource
> > and viewers who view the resource. To be correct, all of the
> viewers have to
> > stop using the pointer before the last owner goes out of scope
> and destroys
> > the resource.
>
> If anything, this sounds fragile and dangerous.
>
> This was my concern as well and what I anticipate to be the biggest
> argument against this idiom. However I'm not yet entirely convinced that
> its any less dangerous to the situation where you have a
> unique_ptr owned by A and viewed by B as a T*. Its up to the programmer
> to ensure that B's lifetime is within A's lifetime.
> If someone can identify a situation where optional_ptr is much more
> dangerous than unique_ptr/T* that would be really helpful.

unique_ptr/T* and optional_ptr are significantly different. You can use
unique_ptr and raw pointers to the same object because you know who owns
the object and is responsible for its deletion - the unique_ptr. With
optional_ptr there is no distinction (on the interface level) and you
are much more likely to destroy the owning optional_ptr leaving other
optional_ptrs dangling.

> But, in case you can easily guarantee that the owner outlives the
> viewers, did you consider using unique_ptr with a custom deleter?
>
> I'm not sure how a custom deleter would help. If the deleter does
> nothing, then you just have T*, if the deleter conditionally deletes the
> object based on a boolean state, you have my optional_ptr. A reusable
> common pattern should have a name, not be a backwards hack on a
> unique_ptr deleter.

Using unique_ptr with a no-op deleter (a.k.a. null_deleter) is not much
different from using a raw pointer. I'm assuming you want
optional_ptr<T> type to be independent from whether it actually deletes
the object or not. So yes, I was suggesting something like this:

struct optional_deleter
{
optional_deleter() noexcept : m_f(false) {}
explicit optional_deleter(bool f) noexcept : m_f(f) {}

optional_deleter(optional_deleter&& that) noexcept : m_f(that.m_f)
{
that.m_f = false;
}

optional_deleter& operator= (optional_deleter&& that) noexcept
{
m_f = that.m_f;
that.m_f = false;
return *this;
}

template< typename T >
void operator() (T* p) const noexcept
{
if (m_f)
delete p;
}

private:
bool m_f;
};

template< typename T >
using optional_ptr = unique_ptr< T, optional_deleter >;

[Matthew Fioravante]

On Thursday, September 24, 2015 at 3:38:56 PM UTC-4, Nicol Bolas wrote:

So I remain confused as to when you would need this functionality.

 

Hi Nicol, sorry if my example was confusing. The A, B, and C objects are streams, not individual peices of data. Its not a chain of function calls but instead an object graph which we initialize once and then stream data through.

 

First we setup our pipeline stages (think of it like a graph) and then we start streaming data through it one element at a time. The one time setup phase is where need to manage lifetimes and where optional_ptr comes into play.

 

Here is one example:

 

      | 0  | 1  | 2  | 3  |
A0 -> | A0 | A0 | A1 | A1 | ->
B0 -> | B0 | B0 | B0 | B1 | ->
C0 -> | C1 | C2 | C3 | C4 | ->

 

In this example, stage 0 and 1 only work on stream C, stage 2 on A and C, and stage 3 B and C.

This means that 0 owns C1, 1 owns C2,  2 owns A1 and C3, and finally 3 owns B1 and C4.

 

A0, B0, C0 are raw pointers coming from outside and are owned by someone else, possibly another pipeline. The point is we don't care because we just pass pointers to streams in and get pointers to streams back out.

 

The code would look something like this (pardon the ugliness and terseness but hopefully it makes the point):

using s = stream;

struct stage {
stage(s* ain, s* bin, s* cin, optional_ptr<s> aout, optional_ptr<s> bout, optional_ptr<s> cout)
 : inputs{ain, bin, cin}, outputs{std::move(aout), std::move(bout), std::move(cout)} {}

s* inputs[3];
optional_ptr<s> outputs[3];
virtual void process_next() = 0;
};

struct stage0 : public stage {
  stage0(s* a, s* b, s* c)
   : stage(a, b, c, a, b, std::make_unique<stream>()) {}
  void process_next() {
    outputs[2].push_back(transform(inputs[2].back()));
  }
};

[Matthew Woehlke]

On 2015-09-24 15:38, Nicol Bolas wrote:
> It's not clear to me what kind of code requires an ownership model
> that is dynamic.

Generic algorithms. The pipeline example, for one. That is, something
that will operate on an object and not leak memory, whether it gets a
view of the object, or ownership of the object. (It's implied that the
pipeline stages are not static, or at least that the ownership model is
not baked into the API.)

> Allow me to restate this, so that we can make sure I'm correctly
> understanding what you've described in C++ terms.
>
> You have these stage constructs. They're pass 3 objects and return three
> objects to the next stage in the sequence. They may modify or not modify
> those objects as they see fit as part of their processing.
>
> Your implementation of this concept is that the 3 objects, the data

> streams, are *immutable*. Stages don't actually modify them; they create a

> modified copy of them. And therefore, a stage may have heap-allocated one
> of streams, or it may simply pass the object from the previous stage
> through.
>
> It's not clear to me why `unique_ptr` doesn't work here. Maybe I'm missing
> something here.

A stage may retain its input for later use. Or the caller of the
pipeline may retain the input for whatever reason, and want to avoid
needing to make a copy. (Maybe the original input came from a source
that can't be coerced into a proper unique_ptr<T>, e.g. because we
loaded the input as U, and can get a view T of the input, but can't take
ownership of the T. This can often happen if T is an array of primitive
type and U is some library-specific container type.)

> How does using `unique_ptr` require copying of the data? It's just passing
> ownership of an object through to the next stage.

...which precludes retaining ownership. Let's say we are talking about a
stepped pipeline (fairly common, e.g. a video processing pipeline).
Let's say that a particular stage's output is dependent on its previous
output. Obviously, then, in the stage's internal state, it needs to
retain a copy of its previous output (at least in some architectures).

With unique_ptr, this means it has to make a copy of the output to pass
to the next stage, since it is required to relinquish ownership. With
optional_ptr, it can just return a non-owning pointer.

--
Matthew

[Arthur O'Dwyer]

Doesn't this prove absolutely why optional_ptr is a bad idea? Let's say I have a pipeline stage

optional_ptr<T> A(optional_ptr<T> input)

{

    static optional_ptr<T> saved = nullptr;

    compute_something_with(input, saved);

    saved = std::move(input);  // the original input is now owned by me

    return saved.get();  // return a non-owning handle to the original input

}

then if I chain together a couple of calls —

    optional_ptr<T> t1 = ...;

    optional_ptr<T> t2 = A(std::move(t1));

    optional_ptr<T> t3 = A(std::move(t2));

t1 was an owning handle, but now it's moved-from.

A took ownership of the t1-pointed object, saved ownership in `saved`, and passed back a non-owning handle as t2.

Then the second A takes t2, saves the non-owning handle in `saved` (remember, it can't tell the difference between the owning pointer and a non-owning handle, by design), and returns another non-owning handle.

However, when the second A saved the non-owning handle in `saved`, it clobbered `saved`'s OLD value, which is to say, the owning pointer to the original object! So that object's lifetime is now over, but we have these non-owning handles still pointing to it. In other words, things are dangling all over the place.

The smart pointer classes are designed specifically to eliminate this whole class of mistakes.

There's no sense in trying to wrap a might-be-owning-might-not-be-owning pointer in something that looks like a smart pointer class, because it fundamentally is NOT a smart pointer; it's just a plain old might-be-owning-might-not-be-owning-might-be-dangling raw pointer. IF you want one of those, C++ already has them; they're spelled (T*).

–Arthur

[Matthew Fioravante]

I'm not sure where you guys are getting these insane designs. That code doesn't even make sense. optional_ptr is not meant to be passed around willy nilly. Its meant to be used as a sink. In general you may be passing an optional_ptr to a function for final storage, but rarely if ever returning it.

In my design, each stage receives a non-owning view to the input and owns his output. The external world only deals in T*. A T* is passed in for the input, and a T* is exposed as the output. The clients of the interface are completely unburdened with memory management concerns.

Lets try this one more time. We start with unique_ptr and a single stream:

class stage {
public:
 stage(T* input) : _input(input), _output(std::make_unique<T>()) {}
 T* input() { return _input; }
 T* output() { return _output.get(); }

 virtual void process_next() = 0;

private:
 T* _input;
 unique_ptr<T> _output;
};

Now suppose I want to have a stage which just passes through the input without doing anything. Using unique_ptr, that means I need to allocate a new output stream and copy the data.

class noop_stage : public stage {
 virtual void process_next() {
 _output()->push_back(input()->back());
 }
};

This of course is pointless and wastes memory and cpu time. So instead lets use optional pointer and now the noop_stage can just passthrough his input and not do any computation at all.

class stage {
protected:
 stage(T* input, optional_ptr<T> output) : _input(input), _output(std::move(output)) {}
public:
 T* input() { return _input; }
 T* output() { return _output.get(); }

 virtual void process_next() = 0;

private:
 T* _input;
 optional_ptr<T> _output;
};

class noop_stage : public stage {
public:
 noop_stage(T* input) : stage(input, input) {}
 virtual void process_next() { /* do nothing! */ }
};

class real_stage : public stage {
public:
 real_stage(T* input) : stage(input, std::make_unique<T>()) {}
 virtual void process_next() { output()->push_back(transform(input()->back())); }
};

With optional_ptr, we don't need to pollute our interface with shared_ptr. Clients are none the wiser and can safely deal with T* pointers whose lifetimes are managed. This of course assumes the client manages the lifetimes of the pipeline stages correctly, but of course in any sane system that part will also be handled by a higher level pipeline class which correctly manages the stages and continues to shield clients from the possibility of memory management bugs.

class pipeline {
public:
 pipeline(T* input) : _input(input);

 T* input() { return _input; }
 T* output() { return _stages.empty() ? input() : _stages.back().output(); }

 template <typename S, typename... Args>
 void append_stage<S>(Args&&... args) {
  _stages.push_back(std::make_unique<S>(output(), std::forward<Args>(args)...);
 }

 void process_next() {
   for(auto& s: _stages) {
     s->process_next();
   }
 }

private:
 T* _input;
 std::vector<std::unique_ptr<stage>> _stages;
};

Of course for a single stream design like this, a noop_stage has little value outside of debugging and it doesn't make much sense to alter the design to accomodate optimizing it. For a pipeline with multiple inputs and outputs, where its common for many of them to not be transformed by every stage, being able to passthrough the inputs as outputs is critical to performance. I'll leave extending this code snippet for multiple inputs and outputs as an exercise for the reader.

[Matthew Woehlke]

On 2015-09-24 23:52, Arthur O'Dwyer wrote:
> Doesn't this prove absolutely why optional_ptr is a bad idea? Let's say I
> have a pipeline stage
>
> optional_ptr<T> A(optional_ptr<T> input)
> {
> static optional_ptr<T> saved = nullptr;
> compute_something_with(input, saved);
> saved = std::move(input); // the original input is now owned by me

This is wrong; 'saved' is now owned by you IFF 'input' was owned by you.

> return saved.get(); // return a non-owning handle to the original input
> }

Like many constructs, it's possible to misuse this. Holding a reference
to something you don't own is going to be dangerous without careful
design. A better design is:

if (input.is_owned())
saved = std::move(input);
else
saved = clone(input);

Getting a design like this correct and safe is not trivial, but absent a
construct like optional_ptr, you MUST make copies all the time (or use
shared_ptr and pay the cost both in doubled pointer size and atomic
reference counting).

> then if I chain together a couple of calls —
>
> optional_ptr<T> t1 = ...;
> optional_ptr<T> t2 = A(std::move(t1));
> optional_ptr<T> t3 = A(std::move(t2));
>
> t1 was an owning handle, but now it's moved-from.

...and is now null, like unique_ptr.

> A took ownership of the t1-pointed object, saved ownership in `saved`, and
> passed back a non-owning handle as t2.
> Then the second A takes t2, saves the non-owning handle in `saved`
> (remember, it can't tell the difference between the owning pointer and a
> non-owning handle, by design), and returns another non-owning handle.

Wait... what? Who said it *can't* tell the difference? That's *NOT* the
objective. The objective is to be able to use it without *caring* about
the difference (in cases where that makes sense).

> However, when the second A saved the non-owning handle in `saved`, it
> clobbered `saved`'s OLD value, which is to say, the owning pointer to the
> original object!

Not necessarily; at least in this case, self-assignment could trivially
prevent that happening. That said, this happens because your design is
flawed; if you intend to retain a non-owning reference, it is on your
head to ensure that the previous stage will not invalidate that
reference while you still need it.

> The smart pointer classes are designed specifically to eliminate this whole
> class of mistakes.
> There's no sense in trying to wrap a might-be-owning-might-not-be-owning
> pointer in something that looks like a smart pointer class, because it
> fundamentally is NOT a smart pointer; it's just a plain old
> might-be-owning-might-not-be-owning-might-be-dangling raw pointer. IF you
> want one of those, C++ already has them; they're spelled (T*).

A T* would be similarly unhelpful in your example; the memory would just
be leaked outright. However, it's not possible to fix your example using
T*. It *is* possible to use optional_ptr *correctly* in your example.

--
Matthew

[Tony V E]

On Thu, Sep 24, 2015 at 12:35 PM, Matthew Fioravante <fmatth...@gmail.com> wrote:

For all resources, we have owners who manage the lifetime of the resource and viewers who view the resource. To be correct, all of the viewers have to stop using the pointer before the last owner goes out of scope and destroys the resource.

 

For both unique_ptr and shared_ptr, the set of owners and viewers has to be decided explicitly ahead of time. This can very limiting because I've run into cases where I have types which 99% of the time are viewers (because the resource is often shared) but in some rare contexts need to be an owner because the object is being used standalone with no external entity to manage the ownership.

 

The heavy handed solution to use shared_ptr everywhere. Now everyone is an owner. Using shared_ptr adds runtime overhead to manage the reference count. The biggest problem with shared_ptr is that once you use it in one place, you have to use it everywhere. Everything now must be dynamically allocated individually. You can't allocate 20 objects in an array and view one of them. You can't create objects on the stack or as a data member. Forcing individual dynamic allocation is a performance killer. We use C++ because we need to be fast and a big part of being fast beings avoiding allocations or failing that, batching them into fewer allocations of large contiguous cache friendly chunks.

 

The solution I'm toying with now is something called optional_ptr. Maybe its not such a great name because it could be confused with std::optional but lets leave bikeshedding aside for now.

If you are just toying with it now, come back in a year or so, and tell us your experience.  We prefer standardizing things with lots of experience behind it.  (And yes, I understand that you are somewhat asking if others have similar experience.)

And I know you don't want to bikeshed, but please just don't call it optional_ptr.  Maybe casual_ptr? (laidback_? hippie_?) ie "sure whatever man, if you don't want it, I can delete it for you, but if you want to keep it, that's OK too, I'll just use it for a moment, chill dude..."

 

--

[Bengt Gustafsson]

I see this pattern quite often. Usually you want to keep the option open to send a pointer to a by value member to conserve allocations. Often in framework code when you want to keep options open for framework users.

[Ross Smith]

On 2015-09-30 07:38, Bengt Gustafsson wrote:
> I see this pattern quite often. Usually you want to keep the option open to send a pointer to a by value member to conserve allocations. Often in framework code when you want to keep options open for framework users.
>

One place I frequently need an optionally-owning pointer is in simple
command line utilities, where the user can optionally supply a file name
to read from (or write to), with a filename of "-" traditionally meaning
"read from standard input" (or "write to standard output"). So a lot of
programs intended to be run from the command line start with code like
this (error checking skipped for simplicity):

string filename = argv[1];
shared_ptr<istream> in;
if (filename == "-")
in.reset(&cin, [] (void*) {});
else
in = make_shared<ifstream>(filename);
string line;
getline(*in, line);

Another situation where I used an optionally-owning pointer recently is
in an image manipulation class, where the image class object might own
the image buffer itself, or might be operating on an image owned by some
other entity.

Ross Smith

[Christopher Horvath]

I also run into this pattern fairly frequently. An example is in an OpenGL renderer, where a user may want to share vertex buffers between multiple meshes, but the default case is for the meshes to have unique vertex buffers.  Right now, in different implementations I have one solution where everything is managed in one central place, which avoids having to use shared_ptr by making everything's reference essentially a weak_ptr.  However, this is a brittle design and requires frequent changes to multiple code locations.  A much better solution would be to have this optional_ptr (or whatever it ends up being called, I agree that optional_ptr is a bad name), instead of what I have now:

// This is a sketch, please don't over-criticize!

class Mesh {

private:

    std::unique_ptr<VertexBuffer> m_unique_positions;

    std::shared_ptr<VertexBuffer> m_shared_positions;

public:

    // This assumes the invariant that one of the two pointers is non-null, enforced elsewhere

    VertexBuffer& getVertexBuffer() { return m_unique_positions ? *m_unique_positions : *m_shared_positions; }

};

C

[Bengt Gustafsson]

How about generalizing this idea to a erased_ptr<T> which can be constructed from a unique_ptr&&, a shared_ptr, a weak_ptr or a plain ptr + bool.

Internally this could be mainly a variant over the different pointer types, but with an operator-> which works as for any smart pointer. The pointer + owned == true constructor could maybe be equivalent to sending in a unique_ptr&&

More thinking is definitely needed but intuitively it seemed to increase the usefulness to include also shared ownership in the mix. 

[Matthew Fioravante]

On Wednesday, September 30, 2015 at 10:50:12 AM UTC-4, Bengt Gustafsson wrote:

How about generalizing this idea to a erased_ptr<T> which can be constructed from a unique_ptr&&, a shared_ptr, a weak_ptr or a plain ptr + bool.

 

While the erased_ptr construct seems like it might be more generic and therefore better, I'm reluctant to go down this path because I don't have any use case which would actually need to use an erased_ptr thats sometimes unique, sometimes a view, and sometimes shared. Do you have a use case for this?

 

Also I'm not sure I like the (ptr, bool) constructor. The original version I wrote had this but I took it out. There is zero cost to creating a temporary unique_ptr and moving it in, and by doing so you very clearly document that you want the optional_ptr to own the resource, much moreso than ptr, true. Also optional_ptr will be designed to be 100% compatible with unique_ptr with regards to deleters, so if you have an owned resource you want to pass in, make the appropriate unique_ptr with deleter and then move it in.

 

Compare the following callsites:

 

void f(optional_ptr<T> ptr);

T* p = new T();
f({p, true});
f(std::unique_ptr<T>(p));

The second stands out much more than the first and doesn't cost anything after the optimizer inlines it away.

 

Finally my second concern with erased_ptr is performance. optional_ptr can be implemented with zero space overhead for all alignof(T) > 1 types and near zero overhead for retreiving (& with a mask) and deleting the pointer (check conditional first and then delete).

 

I think at this point we have enough legitimate use cases from people to show that this tool is actually a good one when used responsibly and will benefit many users. Lets open the discussion up to bikeshedding the name.

 

Pretty much everyone hates optional_ptr, including myself so does anyone have better ideas?

 

Here are some ideas:

semi_unique_ptr (this one is my favorite so far)

optionally_unique_ptr

optional_ptr

dynamic_unique_ptr

maybe_unique_ptr

variant_ptr

unique_or_view_ptr

 

 

I like the idea of having unique in the name, because this thing works exactly like a unique_ptr when it has ownership.

 

[Matthew Woehlke]

On 2015-09-29 18:43, Ross Smith wrote:
> Another situation where I used an optionally-owning pointer recently is
> in an image manipulation class, where the image class object might own
> the image buffer itself, or might be operating on an image owned by some
> other entity.

Ack. It would be really useful if QImage worked like that... having to
either make a copy or separately track ownership of the data is a real pain.

I imagine there are various cases of data wrapping conversion where it's
useful to be able to optionally transfer ownership. (That said, a
problem I've run into there is when the original owner has some esoteric
mechanism for releasing the memory, e.g. because it is irrevocably owned
by some other wrapper class. But I think unique_ptr with a custom
deleter containing a bound value can deal with this.)

--
Matthew

[Tony V E]

On Wed, Sep 30, 2015 at 11:11 AM, Matthew Fioravante <fmatth...@gmail.com> wrote:

On Wednesday, September 30, 2015 at 10:50:12 AM UTC-4, Bengt Gustafsson wrote:

How about generalizing this idea to a erased_ptr<T> which can be constructed from a unique_ptr&&, a shared_ptr, a weak_ptr or a plain ptr + bool.

 

While the erased_ptr construct seems like it might be more generic and therefore better, I'm reluctant to go down this path because I don't have any use case which would actually need to use an erased_ptr thats sometimes unique, sometimes a view, and sometimes shared. Do you have a use case for this?

 

Also I'm not sure I like the (ptr, bool) constructor. The original version I wrote had this but I took it out. There is zero cost to creating a temporary unique_ptr and moving it in, and by doing so you very clearly document that you want the optional_ptr to own the resource, much moreso than ptr, true. Also optional_ptr will be designed to be 100% compatible with unique_ptr with regards to deleters, so if you have an owned resource you want to pass in, make the appropriate unique_ptr with deleter and then move it in.

 

Compare the following callsites:

 

void f(optional_ptr<T> ptr);

T* p = new T();
f({p, true});
f(std::unique_ptr<T>(p));

The second stands out much more than the first and doesn't cost anything after the optimizer inlines it away.

 

Finally my second concern with erased_ptr is performance. optional_ptr can be implemented with zero space overhead for all alignof(T) > 1 types and near zero overhead for retreiving (& with a mask) and deleting the pointer (check conditional first and then delete).

 

I think at this point we have enough legitimate use cases from people to show that this tool is actually a good one when used responsibly and will benefit many users. Lets open the discussion up to bikeshedding the name.

 

Pretty much everyone hates optional_ptr, including myself so does anyone have better ideas?

 

Here are some ideas:

semi_unique_ptr (this one is my favorite so far)

semi-unique implies that it is (always) half unique (oxymoron) instead of sometimes unique.  Is it like Schrodinger's cat, in two states at once? No. (schrodinger_ptr?)

 

optionally_unique_ptr

correct idea, but long and close to optional
 

optional_ptr

don't go there
 

dynamic_unique_ptr

suggests thoughts about dynamic_cast?
 

maybe_unique_ptr

correct idea
 

variant_ptr

don't go there
 

unique_or_view_ptr

correct but long
 

For naming, I think there are two choices:

- correct & concise

- co-opt a term (& concise)

Correct is always what you strive for, but concise is very important for our brains as well.

If you can't find a 'nice' correct name, go for co-opting a term.  Like 'dynamic' in dynamic_cast really isn't quite the usual English meaning of dynamic, but it was close, and now it has new meaning in the context of programming.  Or "lazy" for types that delay computation. (Instead of calling them "delayed computation" types.)

It is actually often _good_ to co-opt a term where the user isn't _quite_ sure what you meant - this forces them to READ THE DOCS, and then they will not make the wrong assumptions, and easily remember how it works later (assuming the co-opted term forms a reasonable association with your class).

P.S. concise doesn't necessarily mean short. It means (in this case) efficient in concepts.  Growing up the added room on the back of our house was called the "back room".  If it had to be called "the added room at the back of the house", it would be harder to talk about, and probably even harder to _think_ about. Let alone type. Ideally a name is a single concept, or, at most, 2 together (that form a new concept).  Once you go beyond that, your brain is working to make sure it has understood how the concepts (ie maybe, unique, ptr) fit together, and is second guessing itself.  (If everyone has become comfortable with unique_ptr as a single concept, then you have 2 concepts: maybe + unique_ptr.)  Again consider "lazy" (single _connotation_) vs "delayed_computation" (two meanings stuck together).  In the end, they mean the same thing (in the context of programming), but brains seem to be easier at substituting in the final concept when seeing/thinking/hearing "lazy" vs seeing/thinking/hearing "delayed computation".

tl;dr. Sorry.

Tony

 

 

I like the idea of having unique in the name, because this thing works exactly like a unique_ptr when it has ownership.

 

--

[j4...@dropbox.com]

On Thursday, September 24, 2015 at 9:35:28 AM UTC-7, Matthew Fioravante wrote:

The heavy handed solution to use shared_ptr everywhere. Now everyone is an owner. Using shared_ptr adds runtime overhead to manage the reference count. The biggest problem with shared_ptr is that once you use it in one place, you have to use it everywhere. Everything now must be dynamically allocated individually. You can't allocate 20 objects in an array and view one of them. You can't create objects on the stack or as a data member. Forcing individual dynamic allocation is a performance killer. We use C++ because we need to be fast and a big part of being fast beings avoiding allocations or failing that, batching them into fewer allocations of large contiguous cache friendly chunks.

You can, with the shared_ptr aliasing constructor (which I just discovered yesterday):

    std::shared_ptr<std::vector<Thing>> owner = ...;

    std::shared_ptr<Thing> viewer(owner, &(*owner)[5]);

    // viewer shares ownership of the vector but points to a specific element

Or, less safe but potentially faster:

    std::vector<Thing> owner = ...;

    std::shared_ptr<Thing> viewer(std::shared_ptr<Thing>(), &owner[5]);

    // viewer owns no object (so no atomic inc/dec will happen) but points to owner[5]

[Arthur O'Dwyer]

Nit on the latter suggestion:

shared_ptr(shared_ptr<Y>&&, T*) does not exist, so the above makes an unnecessary copy.

This is more like a defect in the Standard than a problem with the above idiom, though.

–Arthur

P.S. on a pet peeve topic: The Standard wording is super subtle w.r.t. the differences between

- an empty object (i.e. one with no control block)

- an expired object (i.e. a weak_ptr with a control block whose strong ref count is zero)

- an object whose stored pointer is null

In particular, it appears from my current reading of N4296 that after

    std::shared_ptr<int> sptr1 = nullptr;

    std::shared_ptr<int> sptr2 = (int *)nullptr;

sptr1 must be an empty object whose stored pointer is null, but

sptr2 must be a non-empty object whose stored pointer is null.

[j4...@dropbox.com]

On Wednesday, September 30, 2015 at 4:31:57 PM UTC-7, Arthur O'Dwyer wrote:

Nit on the latter suggestion:

shared_ptr(shared_ptr<Y>&&, T*) does not exist, so the above makes an unnecessary copy.

This is more like a defect in the Standard than a problem with the above idiom, though.

I'd like a shared_ptr(std::nullptr_t, T*) constructor too. (Would require some SFINAE in the (nullptr_t, Deleter) constructor to avoid ambiguity.) Anyone wanna write a proposal? :)

[David Krauss]

On 2015–09–30, at 10:50 PM, Bengt Gustafsson <bengt.gu...@beamways.com> wrote:

How about generalizing this idea to a erased_ptr<T> which can be constructed from a unique_ptr&&, a shared_ptr, a weak_ptr or a plain ptr + bool.

Internally this could be mainly a variant over the different pointer types, but with an operator-> which works as for any smart pointer. The pointer + owned == true constructor could maybe be equivalent to sending in a unique_ptr&&

More thinking is definitely needed but intuitively it seemed to increase the usefulness to include also shared ownership in the mix. 

This might as well be std::unique_ptr< T, std::function<void(T&)> >. All implementations of std::function that I’ve seen are able to embed a shared_ptr without going to the heap. 

It would also be useful to have this as an idiomatic type, for example when extracting a node from a std::list or a std::set. I.e., std::unique_ptr< T, unspecified > std::set<T>::extract( iterator ) where unspecified converts to std::function<void(T&)>. The same goes for the currently missing function std::allocate_unique.

However,

1. It’s a bad idea to break the invariant that unique_ptr is owning, so that it’s safe to move from its referent. It does need a Boolean flag in addition to the deleter.

2. weak_ptr requires validation before use, which would add overhead to all the other cases. It doesn’t even implement operator*. Perhaps that support doesn’t belong.

[David Krauss]

On 2015–10–01, at 11:31 AM, David Krauss <pot...@gmail.com> wrote:

std::unique_ptr< T, std::function<void(T&)> >

Oops, deleters take a pointer, not a reference. Same difference though.

[Matthew Fioravante]

I've found this optional_ptr to be very useful in practice. In most of my use cases I just want to optionally store the resource into its final destination and never touch the resource management logic again until the parent container destructor destroys it for me. If you start passing and returning optional_ptr's all over the place the logic would get really nonsensical pretty fast as seen in peoples examples so don't do that.

There are enough real use cases and positive feedback to argue strongly for this simple utility and overall idiom.