Idea: New Language Feature for Safer, Easier Optional Access

[David Peterson]

Hi Everyone, this is my first time floating an idea so apologies if I get the format wrong.  Constructive criticism is very welcome!

I've had a look at the new std::optional<T> and as exciting as it is to finally have such a useful type in the standard, I worry that the feature may be too cumbersome and possibly too easy to use incorrectly.  Here's an example of its current usage as provided by the C++ 17 standard:

// C++ 17 New Optional Usage //

if (auto customer = get_customer(customer_id)) {

    process(*customer);

}

else {

    show_customer_not_found();

}

This isn't too bad, but I think it could be better.  The first issue I have is with `customer` being an actual std::optional instance.  This creates a level of indirection that seems awkward to use.  One must also remember to dereference the `customer` optional when attempting to gain access to the value as can be seen during the `process` call.  It would be nice to remove this indirection and gain direct access to the underlying type rather than through the optional itself.  After some thinking, I realized that the range-based for loop had the solution I was looking for.  Essentially, an optional is a list containing either no items or exactly one item.  In such a case, one could think of iterating over an optional as if it were a simple container:

// Example of an optional as if it were a container of one or zero items //

for (auto& customer : get_customer(customer_id)) {

    process(customer);

}

Obviously, this syntax is a bit awkward.  It just doesn't make sense to consider an optional as a container.  Furthermore, there would likely be a high degree of unnecessary overhead in interpreting the optional as a container.  Iterators would also have to be made to support the begin/end range.  To correct these shortcomings, my proposal is to extend the range-based for loop syntax for single instance item which may or may not exist (optionals and pointers, primarily) to the `if` statement.  Here is an example:

// New way with `if` auto-dereferencing language feature //

if (auto& customer : get_customer(customer_id)) {

    process(customer); // customer is a Customer& and not an optional.

                       // We can safely use customer as we are guaranteed

                       // to have a valid reference and that reference

                       // cannot extend beyond the `if` scope

}

else {

    show_customer_not_found();

}

As you can see, this feature is readable and addresses the indirection problem from earlier.  Furthermore, it can be optimized easier for optional types and can theoretically work with any type, including user types, so long as they provide a conversion to boolean operator and a dereference operator.  An example of how this could be interpreted follows:

// New feature translates into... //

auto _optional_customer = get_customer(customer_id);

if (_optional_customer) {

    auto& customer = *_optional_customer;

    process(customer);

}

else {

    show_customer_not_found();

}

I'm sure language features are not the easiest thing to add, and I'm no compiler developer, but I'm anxious to hear any feedback you may have in regards to short comings, enhancements and overall complexity to implement.  Thank you for considering this feature. - David Peterson

[Ville Voutilainen]

This seems to be essentially suggesting what was proposed in
http://open-std.org/JTC1/SC22/WG21/docs/papers/2014/n4127.html,
and that paper failed to gain consensus.

[Zhihao Yuan]

On Sat, Apr 22, 2017 at 2:23 PM, David Peterson
<djpete...@hotmail.com> wrote:
> // Example of an optional as if it were a container of one or zero items //
> for (auto& customer : get_customer(customer_id)) {
> process(customer);
> }
>
> Obviously, this syntax is a bit awkward. It just doesn't make sense to
> consider an optional as a container. Furthermore, there would likely be a
> high degree of unnecessary overhead in interpreting the optional as a
> container. Iterators would also have to be made to support the begin/end
> range.

Not quite, given the relaxed range-for, `end` can be
as simple as a sentinel with a different type, such as
nullopt or some equivalents. But, after I tried to
implement these in various ways, I found that, indeed,
both gcc and clang have difficulties optimizing this
construct into non-loops. However, I expect these
problems to be overcame by special casing some
particulate patterns in front-ends.

--
Zhihao Yuan, ID lichray
The best way to predict the future is to invent it.
_______________________________________________

[David Peterson]

Thanks for the feedback.  I'm new to this so didn't realize someone beat me to it.  Though I'm more of a hobbyist with C++ in general, I've worked with optionals in the past in other languages in a much more professional context.  I've found proper implementation and ease of use of optionals can seriously increase the stability of a project, especially when newer members are introduced to the development team.  I would consider this my vote towards the aforementioned proposal.

@Zhihao Yuan, it's good to know that some optimization can occur inside my ranged for-loop example.  I still find its usage awkward for optionals, and I too have built an implementation around this.  It also doesn't address the `else` clause which I feel is very much needed.

I did find out with some serious macro abuse that a very close copy of this functionality can exist.  I wouldn't recommend this in production code, of course:

// Badness ensues...

#define if_has(a, b) if(b) for (a : {*b})

.

.

.

if_has(auto& customer, get_customer(customer_id)) {

    process(customer);

}

else {

    show_customer_not_found();

}

Since there doesn't seem to be much of a draw for this kind of feature natively, I'll just patiently wait in the hopes that more support comes to pass.  Thanks again for your review!

[Casey Carter]

Transforming the optional into a contiguous range seems to optimize well (https://godbolt.org/g/RUfWZK):

template<class O>

class optional_range {

    O o_;

public:

    optional_range() = default;

    template<class Arg>

    optional_range(Arg&& a) : o_{std::forward<Arg>(a)} {}

    auto data() {

        return o_ ? std::addressof(*o_) : nullptr;

    }

    auto data() const {

        return o_ ? std::addressof(*o_) : nullptr;

    }

    bool empty() const { return !o_; }

    std::size_t size() const { return !empty(); }

    auto begin() { return data(); }

    auto end() { return data() + size(); }

    auto begin() const { return data(); }

    auto end() const { return data() + size(); }

};

template<class T>

optional_range(std::optional<T>&&) -> optional_range<std::optional<T>>;

template<class T>

optional_range(std::optional<T>&) -> optional_range<std::optional<T>&>;

template<class T>

optional_range(std::optional<T> const&) -> optional_range<std::optional<T> const&>;

template<class T>

optional_range(std::optional<T> const&&) -> optional_range<std::optional<T>>;

This approach could be easily standardized as an explicit conversion from optional<T> to span<T>.

[Tony V E]

"Your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should."

- Dr. Ian Malcolm (Jurassic Park)

We could make optional into a range, but is it really a good idea? I doubt it.

We could also make any object a range of size 1, for example.

--

Be seeing you,

Tony

[Zhihao Yuan]

On Sat, Apr 22, 2017 at 7:50 PM, Casey Carter <cart...@gmail.com> wrote:
> Transforming the optional into a contiguous range seems to optimize well
> (https://godbolt.org/g/RUfWZK):
>

I haven't tried this form. Better than I thought.

>
> This approach could be easily standardized as an explicit conversion from
> optional<T> to span<T>.

Then this has to be a member function
on optional, and <optional> has to include
<span>*. What I have in mind is a free
function `if_present`, like `erase_if`,
customizable for each type/class template,
returning ranges of unspecified types.
I don't yet know why these ranges has
to be span<T> and/or contiguous, even
though contiguous range is so far the
most efficient way to implement them,
but IMO this part is still QoI.

[Nevin Liber]

On Sat, Apr 22, 2017 at 10:51 PM, Tony V E <tvan...@gmail.com> wrote:

We could make optional into a range, but is it really a good idea? I doubt it.

We could also make any object a range of size 1, for example.

Those are two different things.

It is a perfectly valid model that optional is a container of at most one element.
--

 Nevin ":-)" Liber  <mailto:ne...@eviloverlord.com>  +1-847-691-1404

[inkwizyt...@gmail.com]

Your macro is "incorrect" because it evaluate `b` two times, but its fix by adding `if (auto&& some_local_value = b)`.


Another way to archive that you want could be by extending structur binding to support syntax:

if(auto [x] = func())
{
    //A
}
else
{
    //B
}

This will be equivalent to:

if(auto __temp = func())
{
    auto&& x = get<0>(__temp);
    //A
}
else
{
    //B
}

Only side effect would be that `std::optional` and pointers (for consistency) would have need to implements `get<0>` that could be confusing.
Alterative you could add another extension that for `auto [x] = z;`  would use `auto x = *z;` if `get<0>` is not defined for `z` but this could be again too much things in one basket.

[Vicente J. Botet Escriba]

Le 23/04/2017 à 08:53, Nevin Liber a écrit :

On Sat, Apr 22, 2017 at 10:51 PM, Tony V E <tvan...@gmail.com> wrote:

We could make optional into a range, but is it really a good idea? I doubt it.

We could also make any object a range of size 1, for example.

I will say any object that it is not a range to be considered as a range of size 1 ;-)

Those are two different things.

It is a perfectly valid model that optional is a container of at most one element.

I don't know if making optional a range is a good thing or not, but iterating over the possible value of a nullable type seems right to me.

Following http://open-std.org/JTC1/SC22/WG21/docs/papers/2014/n4127.html, we could have a nullble-based for loop and translate

    for (T x : e1) s2

to

    for (;auto && __p = e1; ) { T x = *__p; s2 }

when decltype(e1) is nullable.


Vicente

[Dan Raviv]

How about:

if ([auto& customer] = get_customer(customer_id)) {

    process(customer);

}

else {

    show_customer_not_found();

}

This extends structured bindings to allow "trying" to bind a result. Of course, to prevent using invalid references in case of a failed binding, this should only compile in the context of an if statement, and the binded reference should only be visible within the if part, not the else part.

Thanks,

Dan

[David Peterson]

Your macro is "incorrect" because it evaluate `b` two times, but its fix by adding `if (auto&& some_local_value = b)`.

That's true.  I threw that together, so I didn't really test it for that case.  Thanks for the catch. I was trying to avoid creating a temporary in an attempt to avoid name conflicts during nesting.

[Tristan Brindle]

On Sunday, April 23, 2017 at 4:51:57 AM UTC+1, Tony V E wrote:

We could also make any object a range of size 1, for example.

Correct me if I'm wrong, but isn't this precisely what Range-V3's view::single does?

A can appreciate not wanting to clutter optional's interface with begin() and end() member functions etc, but providing a wrapper view as in Casey's example seems to me like it would be useful.

Tristan

[Vicente J. Botet Escriba]

Le 26/04/2017 à 02:21, Tristan Brindle a écrit :

On Sunday, April 23, 2017 at 4:51:57 AM UTC+1, Tony V E wrote:

We could also make any object a range of size 1, for example.

Correct me if I'm wrong, but isn't this precisely what Range-V3's view::single does?

I don't think so. You will need a view::nullable.

A can appreciate not wanting to clutter optional's interface with begin() and end() member functions etc,

These functions can be non-members :)

but providing a wrapper view as in Casey's example seems to me like it would be useful.

Sure, this could be useful in some cases.
 
I believe the question of this thread is if we want to iterate over the possible value directly even if optional is not a model of Range without adding any noise.
There are other ways to iterate over an object than using a Range e.g. Nullables and Functors could allow iteration over its elements.

If we had several forms of XXX-based for loops, the implementation could take the concept that is more constrained as it his implementation should/could be more efficient than the less constrained one.

Vicente

[Vicente J. Botet Escriba]

Le 22/04/2017 à 23:23, David Peterson a écrit :

Hi Everyone, this is my first time floating an idea so apologies if I get the format wrong.  Constructive criticism is very welcome!

I've had a look at the new std::optional<T> and as exciting as it is to finally have such a useful type in the standard, I worry that the feature may be too cumbersome and possibly too easy to use incorrectly.  Here's an example of its current usage as provided by the C++ 17 standard:

// C++ 17 New Optional Usage //

if (auto customer = get_customer(customer_id)) {

    process(*customer);

}

else {

    show_customer_not_found();

}

This isn't too bad, but I think it could be better.  The first issue I have is with `customer` being an actual std::optional instance.  This creates a level of indirection that seems awkward to use.  One must also remember to dereference the `customer` optional when attempting to gain access to the value as can be seen during the `process` call.  It would be nice to remove this indirection and gain direct access to the underlying type rather than through the optional itself. 

<snip>

  To correct these shortcomings, my proposal is to extend the range-based for loop syntax for single instance item which may or may not exist (optionals and pointers, primarily) to the `if` statement.  Here is an example:

// New way with `if` auto-dereferencing language feature //

if (auto& customer : get_customer(customer_id)) {

    process(customer); // customer is a Customer& and not an optional.

                       // We can safely use customer as we are guaranteed

                       // to have a valid reference and that reference

                       // cannot extend beyond the `if` scope

}

else {

    show_customer_not_found();

}

As you can see, this feature is readable and addresses the indirection problem from earlier.  Furthermore, it can be optimized easier for optional types and can theoretically work with any type, including user types, so long as they provide a conversion to boolean operator and a dereference operator.  An example of how this could be interpreted follows:

// New feature translates into... //

auto _optional_customer = get_customer(customer_id);

if (_optional_customer) {

    auto& customer = *_optional_customer;

    process(customer);

}

else {

    show_customer_not_found();

}

I'm sure language features are not the easiest thing to add, and I'm no compiler developer, but I'm anxious to hear any feedback you may have in regards to short comings, enhancements and overall complexity to implement.  Thank you for considering this feature. - David Peterson

The language based variant proposal [1] (I hope will be part of C++20) will allow to do pattern matching.
I believe this will be more adapted and will take in account std::expected as well.

We could have something like

inspect  (get_customer(customer_id)) {

    just customer =>    process(customer);

    nothing =>  show_customer_not_found();

}

once std::optional would be adapted  to the patter matching constraints.

Vicente

[1] http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0095r1.html

[David Peterson]

The language based variant proposal [1] (I hope will be part of C++20) will allow to do pattern matching.I believe this will be more adapted and will take in account std::expected as well.

 

This looks similar to Rust's enumeration and pattern matching.  In all honesty, I think I would prefer this pattern matching system to a built in optional.  This seems extremely flexible and could be used for run-time validation as well (such as a variant which is either a SuccessObject or a FailureObject after a process completes).  I hope that templating is included when (if?) the proposal is ratified as this would easily solve the issue with optionals in a very safe, easy to use way.

[Dan Raviv]

+1 for this.

But even if we don't get language based variants, is there any good reason why optional<T> couldn't be implemented as a variant<nullopt_t, T> ? Which would solve the original issue brought up here by just allowing std::visit to be used.

Cheers,

Dan

[Matt Calabrese]

On Thu, Apr 27, 2017 at 3:13 PM, Dan Raviv <dan....@gmail.com> wrote:

+1 for this.

But even if we don't get language based variants, is there any good reason why optional<T> couldn't be implemented as a variant<nullopt_t, T> ? Which would solve the original issue brought up here by just allowing std::visit to be used.

I wish, but they have subtly different semantics, which makes things annoying. Ideally those differences wouldn't exist, but here we are.

[Nicol Bolas]

To expand on what Matt is no doubt eluding to, the basic problem with such a thing (besides the interface) is that `optional` can provide an exception guarantee with no overhead, while `variant` cannot in the general case. Lots of ink and discussion time has been spilled over this issue, which finally led to the current "not bad enough to keep arguing over" compromise `std::variant` version.

But the compromise puts us in a place where `optional<T>` provides a different exception guarantee than `variant<nullopt, T>`. If assignment or emplacement of `optional<T>` fails, then we know that it has no value. Whereas `variant<nullopt, T>` gets (possibly) put into the "valueless_by_exception"-state.

[Dan Raviv]

Ah yes, the lovely "valueless_by_exception" state :)

Thanks for the explanation!

I guess the original issue brought up in the thread could still be solved relatively nicely just by adding a method to std::optional<T> similar to what std::visit does for variants. This (templated on Ret) method could just accept two std::function parameters: one std::function<Ret(T)> and one std::function<Ret()>, where the right function would be called according to whether the optional has a value or not.

Cheers,

Dan

[Vicente J. Botet Escriba]

I proposed a match function [1] that should works with multiple variants, optionals, expected or any sum type. It is inline with the languages based inspect statement and BTW, the proposal had a inspect function also. At the time there were too much work around variant and we needed a something consensual.

I'm reworking this proposal to define once for all a SumType concept and function that work with them, in particular match [2].
I'm not sure the proposal will be ready for Toronto. The way I'm customizing all this concepts needs a paper that I'm sure will not be consensual :( on an alternative way to customize in the standard. I did a presentation in code:dive about this subject last fall [3]. Any feedback in a separated thread is welcome.

Vicente

[1] http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/p0050r0.pdf
[2] https://github.com/viboes/std-make/tree/master/include/experimental/fundamental/v3/sum_type
[3] https://cdn2-ecros.pl/event/codedive/files/presentations/2016/TraitsAsCustomizationPoints.pdf and https://www.youtube.com/watch?v=Dsm12yd637E