Generalizing range based for loops even further

[rmn...@gmail.com]

C++17 has a new feature that would improve the functionality of range based for loops to work with begin and end iterators that are not of the same type as proposed by Eric Niebler here http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0184r0.html.  This is a great feature, as it generalizes the range based for loop to fit scenarios that would otherwise not be possible, essentially making it more adaptable.  

But something like the following is still not possible using the range based for loop, since the dereferenced iterators must be the same type, and therefore cannot contain any compile time information

    for (auto ele : std::make_tuple(1, 2)) {
        cout << ele << endl;
    }

Iterating through a tuple requires use of minimal amounts of metaprogramming, it is easy to write a small helper that would automate this for us.  But then again, it requires extra work on the programmers part. A language solution seems to be more intuitive to me - generalize the range based for loop to the stage where it can accept a range of different iterator types. 

You might be thinking that such a change cannot be made without sacrificing runtime efficiency in the range based for loop as it expects the same type for iterators it dereferences.  What I propose is to not use the operator++() method (the pre-increment member function) to increment iterators when they contain a trait that signals that they are heterogenous iterators.  Perhaps this could be enabled with the help of a preferred_increment iterator tag, which if absent will cause the range based for loop to default to prefix operator++(), but if present will make the range based for loop go with the preferred increment approach (which can either be a post-increment and initialization or a reference pre-increment).  With this small change the tuple class can be modified to support iteration.  The iterator for a tuple might look something like the following

template <typename... Args>
class tuple {
public:
     
    // The iterator class is templated with an integer that denotes which
    // member of the tuple it must fetch from, the tuple type is included
    // for forwarding reasons
    template <int index, typename TupleType>
    class Iterator {
    public:
        // this tells the range based for loop to not use the operator++()
        // member function to increment the iterator, but rather to use the
        // operator++(int) postfix increment method
        using preferred_increment = std::post_increment_tag;

        // The constructor initializes the iterator with a reference to the
        // tuple
        Iterator(tuple<Args...>& tup_in) : tup{tup_in} {}

        // The dereference operator forwards the tuple and returns the forwarded
        // result obtained on a call to std::get<>
        decltype(auto) operator*() {
            return std::get<index>(std::forward<TupleType>(this->tup));
        }
       
        // The iterator only defines a postfix increment, as is needed for
        // heterogenosity
        auto operator++(int) {
            return Iterator<index + 1, TupleType>(this->tup);
        }

        // return a false for iterators that are not of the same type
        template <int index_other, typename TupleType>
        constexpr bool operator!=(const Iterator<index_other, TupleType&) {
            return index_other != index;
        }
    };

    // The begin() member function returns an iterator type that will return
    // lvalue references on dereferencing
    auto begin() & {
        return Iterator<0, tuple<Args...>&>{*this};
    }
   
    // The begin() member function returns an iterator type that will return
    // rvalue references on dereferencing
    auto begin() && {
        // or Iterator<0, tuple<Args...>>{*this};
        return Iterator<0, tuple<Args...>&&>{*this};
    }

    // The end iterator returns an iterator with the sizeof(Args)... as the
    // index, so any iterator type that reaches it will compare the same
    // The tuple type really doesn't matter here
    auto end() {
        return Iterator<sizeof(Args)..., tuple<Args...>>{*this};
    }
};

And that would allow the range based for loop to increment the iterator with the heterogenous postfix iterator increment like so

for (auto element : std::make_tuple(1, "some string")) {
    cout << element << endl;
}

Of course making an iterator class for a tuple seems to outweigh the benefits, but making such a modification to the range based for loop to generalize it further seems to be a good idea in my mind as it opens up doors for a whole bunch of other things.  Like iteration through a range computed at compile time.  The canonical example here could be through the fibonacci range

for (auto val : std::fibonacci{}) {
    cout << val << endl;
}

or iterate through a function's arguments

template <typename... Args>
void foo(Args&&... args_in) {
    auto args = std::forward_as_tuple(std::forward<Args>(args_in)...);
    for (auto&& arg : args) {
        // ...
    }
}

even something like so will be possible

// compile time iteration through a range of integers
for (auto index : std::make_integer_sequence<int,10>{}) {
    cout << std::get<static_cast<int>(index)>(some_tuple) << endl;
}

Apologies for the mistakes in the code, I didn't actually compile any of it, it is just a vision of what I hope might come to pass if people like this idea. 

[Avi Kivity]

On 05/23/2017 07:10 AM, rmn...@gmail.com wrote:

C++17 has a new feature that would improve the functionality of range based for loops to work with begin and end iterators that are not of the same type as proposed by Eric Niebler here http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0184r0.html.  This is a great feature, as it generalizes the range based for loop to fit scenarios that would otherwise not be possible, essentially making it more adaptable.  

But something like the following is still not possible using the range based for loop, since the dereferenced iterators must be the same type, and therefore cannot contain any compile time information

    for (auto ele : std::make_tuple(1, 2)) {
        cout << ele << endl;
    }

Perhaps:


    for constexpr (auto element : std::tuple(1, "a")) {
        std::cout << element << std::endl;
    }

The rules can then be relaxed that operator++ can return new types; the compiler will essentially unroll the loop if that's the case.

--
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposal...@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.
To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/05e0675e-4c5d-4ad4-a1e4-4cf10ef5bf96%40isocpp.org.

[Anthony Hall]

You may already be aware of the boost::hana library, but if not: part of its design intent is to provide facilities and algorithms for traversal of heterogeneously-typed containers much like what's being proposed.  Of course, as a C++14 library-based solution, it can't always offer the kind of syntactic sugar that's been proposed here with `for constexpr` or these expansions of operator++ overloads.

[Nicol Bolas]

On Tuesday, May 23, 2017 at 12:10:40 AM UTC-4, rmn...@gmail.com wrote:

C++17 has a new feature that would improve the functionality of range based for loops to work with begin and end iterators that are not of the same type as proposed by Eric Niebler here http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0184r0.html.  This is a great feature, as it generalizes the range based for loop to fit scenarios that would otherwise not be possible, essentially making it more adaptable.  

But something like the following is still not possible using the range based for loop, since the dereferenced iterators must be the same type, and therefore cannot contain any compile time information

    for (auto ele : std::make_tuple(1, 2)) {
        cout << ele << endl;
    }

Iterating through a tuple requires use of minimal amounts of metaprogramming, it is easy to write a small helper that would automate this for us.

No, it's not. It's not even possible unless every element of the tuple is of the same type (or is implicitly convertible to the same type). `auto ele` cannot change its type; it must be of the begin iterator's value_type. Which is a static property that cannot change based on where the iterator is.

I much prefer we just have this. It will allow plenty of ways to "iterate" over elements of a tuple.

But then again, it requires extra work on the programmers part. A language solution seems to be more intuitive to me - generalize the range based for loop to the stage where it can accept a range of different iterator types.

No.

[rmn...@gmail.com]

@AviKivity

Firstly rethinking the design decision to have an iterator representing a compile time range, it seems wrong and might not even work.  Rather a design based on std::get<> and std::tuple_size<> seems better and easier

Regarding the for constexpr I feel like that would defeat the main purpose of such a change to the range based for loop - code reusability for compile time and and runtime ranges.  What I had in mind was the ability to do something like this

template <typename Range>
void do_something_with_range(Range&& range) {
    for (auto&& element : range) {
        // do something with range
    }
}

Now this function can be invoked as 

// with the new rules for type deduction in constructors
do_something_with_range(std::tuple{1, "something", 3.14159265});
do_something_with_range(std::vector{1, 2, 3, 4});

Now one function can be used in cases where the user wants to pass a compile time range as well as a runtime range.  More importantly, you can reuse code! 

template <typename... Args>
void do_something_with_range(Args&&... args) {
    auto t = std::forward_as_tuple(std::forward<Args>(args)...);
    do_something_with_range_impl(t);
}

template <typename Vector>
void do_something_with_range(Vector&& vec) {
    do_something_with_range_impl(std::forward<Vector>(vec));
}

The change would not be breaking since no iterator class includes a tag of the type that is proposed, and most standard libraries do not include a tag by that name, so the change would look something like the following

Until C++17

{
    auto && __range = range_expression ;
    for (auto __begin = begin_expr, __end = end_expr;
            __begin != __end; ++__begin) {
        range_declaration = *__begin;
        loop_statement
    }
}

Since C++17

{
    auto && __range = range_expression ;
    auto __begin = begin_expr ;
    auto __end = end_expr ;
    for ( ; __begin != __end; ++__begin) {
        range_declaration = *__begin;
        loop_statement
    }
}

Since C++20 (Thinking about it now, I feel like a better choice is to allow iteration over a range that has std::get<> and std::tuple_size<> defined for it)

auto && __range = range_expression ;
constexpr if (is_std_get_defined_for(__range)) {
    // unrolled loop from 0 to
    // std::tuple_size_v<std::decay_t<decltype(__range)>> follows
    {
        range_declaration = std::get<0>(__range);
        loop_statement
    }
    {
        range_declaration = std::get<0>(__range);
        loop_statement
    }
    ...
} constexpr else {
    auto __begin = begin_expr ;
    auto __end = end_expr ;
    for ( ; __begin != __end; ++__begin) {
        range_declaration = *__begin;
        loop_statement
    }
}

[rmn...@gmail.com]

@AnthonyHall

Yep!  I was, I was myself using something just like it in my projects.  I just felt like the range based for loop still has some more room for improvement and changes.  And this change would hopefully make it much more usable in generic code.  

I just posted an update regarding what I think the best design decision here it, I decided to drop the iterator idea entirely for something simpler!  

[rmn...@gmail.com]

@Nicol Bolas

Thanks for the reply!  While there are other solutions, why not improve the range based for loop even more?  There was no strict need for sentinels, but that was added as well.  There were other ways to accomplish the same thing without them but adding them brings more generalizability to the construct.  That's why I suggested the change.  

Regarding the second thing you said about ele not changing its type, it can change its type based on what I proposed in the update I just posted,  since it's initialized every time the loop is run.  

[Barry Revzin]

On Monday, May 22, 2017 at 11:10:40 PM UTC-5, rmn...@gmail.com wrote:

C++17 has a new feature that would improve the functionality of range based for loops to work with begin and end iterators that are not of the same type as proposed by Eric Niebler here http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0184r0.html.  This is a great feature, as it generalizes the range based for loop to fit scenarios that would otherwise not be possible, essentially making it more adaptable.  

But something like the following is still not possible using the range based for loop, since the dereferenced iterators must be the same type, and therefore cannot contain any compile time information

    for (auto ele : std::make_tuple(1, 2)) {
        cout << ele << endl;
    }

Iterating through a tuple requires use of minimal amounts of metaprogramming, it is easy to write a small helper that would automate this for us.  But then again, it requires extra work on the programmers part. A language solution seems to be more intuitive to me - generalize the range based for loop to the stage where it can accept a range of different iterator types. 

See http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0589r0.pdf

From Botond's trip report:

Concerns with this specific proposal included performance considerations (for example, it makes it very easy to write innocent-looking code that triggers a lot of template instantiations and is expensive to compile), and the fact that it didn’t generalize to some use cases (such as cases where you want the type of a result to depend on logic in your “loop”). Further exploration of the topic was encouraged.

[rmn...@gmail.com]

@Barry Revzin

It seems like this is the second time I have made a post on this group that has just recently been explored elsewhere that I didn't know about..

Couldn't the problem of innocent-looking code causing a lot of instantiations be simply resolved by limiting the number of iterations that are allowed in such a setting?  The same way infinite compile time recursion is prevented without the use of special flags in g++?

Also about the loop not generalizing to cases where you want the type of a result to depend on the logic in a loop, why can you not simply use a constexpr if in the loop and eliminate the problem?

I feel like the idea is still good because there are a lot of cases where you just need to loop over the elements of a compile time range, and perform computations on them.  An example could be the compile time overload of std::future::when_all as opposed to the runtime overload accepting two iterators.

[Nicol Bolas]

On Tuesday, May 23, 2017 at 11:09:17 AM UTC-4, rmn...@gmail.com wrote:

@Nicol Bolas

Thanks for the reply!  While there are other solutions, why not improve the range based for loop even more?  There was no strict need for sentinels, but that was added as well.

Yes, there are strict needs for sentinels. That's why the Range TS added them; they're more efficient than having to create end iterators in many cases.
 

There were other ways to accomplish the same thing without them but adding them brings more generalizability to the construct.  That's why I suggested the change.

But it's not "generalizability"; it's a completely different construct, which requires a completely different implementation that has completely different behavior.

This is not some minor tweaking of range-based `for`. It's a fundamentally different thing in every single way; it just happens to be wearing the same skin as range-based `for`.