Improve fundamentals of parameter packs

[Daniel Frey]

I’d like to propose three basic functionalities wrt parameter packs.

1) Allow expansion of only a single element via pattern...[I].

2) Allow in-situ creation of unexpanded non-type parameter packs via Type... N.

3) Allow aliases via using Is = std::size_t... 5.

I think that those are the fundamentals missing for efficient implementation of variadic functions dealing with parameter packs.

I attached a first draft of a formal proposal. Any feedback welcome.

Daniel

[Arthur O'Dwyer]

On Sunday, February 28, 2016 at 1:32:56 PM UTC-8, Daniel Frey wrote:

I’d like to propose three basic functionalities wrt parameter packs.

1) Allow expansion of only a single element via pattern...[I].

2) Allow in-situ creation of unexpanded non-type parameter packs via Type... N.

3) Allow aliases via using Is = std::size_t... 5.

 

Example 2 under "Pack Aliases" is:

template<typename... Ts> using Ds = decay_t<Ts>;

Presumably you're thinking of using Ds like this:

template<typename... X> void foo(Ds<X>... ds);

being a synonym for

template<typename... X> void foo(std::decay_t<X>... ds);

But that's not a "pack alias"; that's just a plain old alias template! You can get that effect today by writing

template<typename T> using Ds = decay_t<T>;

template<typename... X> void foo(Ds<X>... ds);

The first example is better, because it gives us something we don't already have; namely a way to make a synonym for a naked pack. But I think you run into trouble there, because once you have the ability to create a pack from a non-pack, you can do things like this:

template<int J> using Is = size_t... J;  // new "pack alias" syntax

template<int K> void foo() { return f(Is<K>...); }  // return f(0,1,2,3...K-1)

template<int... L> void bar() { return f(g(Is<L>...)...; }  // return f(g(0,1,2,3...L1-1), g(0,1,2,3...L2-1), ...) ???

template<int... M> void baz() { return f(Is<M>... ...);  // return f(0,1,2,3...L1-1, 0,1,2,3...L2-1,...) ???

template<int N> void baz() { return f(Is<Is<N>>... ...);  // return ???

I ran into similar problems with my tuple-exploding tilde-notation (let x~ be a pack consisting of all the elements of tuple x). Either there are fundamental semantic problems that mean you can't do this, period; or else the syntax is so confusing that it seems like there are fundamental problems when there aren't. :)

No specific comment on the other proposals, except to say that they don't add any new expressiveness to the language, just core-language syntactic sugar for constructs we can already express efficiently through the standard library (tuple_element_t, for example), and therefore I don't think they stand much chance. Also, they use up valuable syntactic real estate, so I don't think they should be adopted.

my $.02,

–Arthur

[Daniel Frey]

> On 29.02.2016, at 06:36, Arthur O'Dwyer <arthur....@gmail.com> wrote:
>
> On Sunday, February 28, 2016 at 1:32:56 PM UTC-8, Daniel Frey wrote:
> I’d like to propose three basic functionalities wrt parameter packs.
>
> 1) Allow expansion of only a single element via pattern...[I].
> 2) Allow in-situ creation of unexpanded non-type parameter packs via Type... N.
> 3) Allow aliases via using Is = std::size_t... 5.
>
> Example 2 under "Pack Aliases" is:

[...]

> The first example is better, because it gives us something we don't already have; namely a way to make a synonym for a naked pack. But I think you run into trouble there, because once you have the ability to create a pack from a non-pack, you can do things like this:

Good catch. Maybe the 3rd proposal (pack aliases) is too much. I’ll reconsider that one, so let’s focus on the first two for now.

> No specific comment on the other proposals, except to say that they don't add any new expressiveness to the language, just core-language syntactic sugar for constructs we can already express efficiently through the standard library (tuple_element_t, for example), and therefore I don't think they stand much chance. Also, they use up valuable syntactic real estate, so I don't think they should be adopted.

FWIW, three compilers (VC++, GCC, Clang) are implementing a compiler intrinsic called __make_integer_seq which could be replaced by 2) and we gain the following advantages:

- More expressiveness as an unexpanded parameter pack is generated, not just an instance of a class (std::integer_sequence)
- Portable, standardized syntax

The fact that those three compilers are already implementing __make_integer_seq also shows that the general benefits are significant enough to warrant such a step. I arrived at the above proposal (especially 1) and 2)) while trying to avoid recursive implementations of variadic functions when creating a better tuple. If you want to study some background, this is the core of tuple_element_t:

https://github.com/taocpp/sequences/blob/master/include/tao/seq/type_by_index.hpp

when you don’t have 1) available. It’s ugly, hard to understand, hard to teach as well as time and memory consuming. Further material to study if you are interested:

https://github.com/taocpp/sequences
https://github.com/taocpp/tuple

Even without 1) and 2), the non-recursive implementations allowed some significant improvements against libstd++ and libc++, some numbers for one example (generating a std::tuple with 265 elements via std::tuple_cat, MID=maximum instantiation depth):

Linux, GCC5 + libstdc++, std::tuple: 19.6s, MID 3719
Linux, GCC5 + libstdc++, tao::tuple: 1.2s, MID 26
MacOS, Apple LLVM 7.0 (Clang ~3.7), std::tuple: 70.0s, MID 514
MacOS, Apple LLVM 7.0 (Clang ~3.7), tao::tuple: 1.7s, MID 15

This means that the amount of instantiations, and thereby the memory used by the compiler when working with variadic templates can have a great impact on compile times. C++ has a reputation for being slow to compile, improvements in this area are more than just syntactic sugar IMHO. While working on the above, I realized that the need for 1) and 2) comes up over and over again. I believe that adding them to the language can lead to real-world benefits as variadic templates are used more and more and often they are used behind the scenes. Even if the user doesn’t see it directly, but I think that the average compile time of most projects will benefit.

Daniel

[Arthur O'Dwyer]

On Tue, Mar 1, 2016 at 1:27 PM, Daniel Frey <d.f...@gmx.de> wrote:
> On 29.02.2016, at 06:36, Arthur O'Dwyer <arthur....@gmail.com> wrote:
> > On Sunday, February 28, 2016 at 1:32:56 PM UTC-8, Daniel Frey wrote:
> > > I’d like to propose three basic functionalities wrt parameter packs.
> > >
> > > 1) Allow expansion of only a single element via pattern...[I].
> > > 2) Allow in-situ creation of unexpanded non-type parameter packs via Type... N.
> > > 3) Allow aliases via using Is = std::size_t... 5.
>

> > No specific comment on the other proposals (1 and 2), except to say that they don't add

> > any new expressiveness to the language, just core-language syntactic sugar for constructs
> > we can already express efficiently through the standard library (tuple_element_t, for
> > example), and therefore I don't think they stand much chance. Also, they use up
> > valuable syntactic real estate, so I don't think they should be adopted.
>
> FWIW, three compilers (VC++, GCC, Clang) are implementing a compiler intrinsic
> called __make_integer_seq which could be replaced by 2) and we gain the following
> advantages:
>
> - More expressiveness as an unexpanded parameter pack is generated, not just an instance of a class (std::integer_sequence)
> - Portable, standardized syntax

In the part of my post that you snipped, I indicated that "generating an unexpanded parameter pack" is a liability, not an advantage. You seemed to agree ("Good catch ... I'll reconsider that one ...").  Maybe I should say is currently a liability. It would be awesome to have first-class parameter packs, but I think there are deep problems someone needs to solve before we can get them.

Portable, standardized syntax for std::make_integer_sequence already exists; it's std::make_integer_sequence!

You have correctly noticed that current implementations of make_integer_sequence are pretty slow, and that recently there's been some behind-the-scenes work in VC++, GCC, and Clang to speed up make_integer_sequence via the use of compiler builtins. However, as a user of C++1z, I don't need to know about these builtins; I just use the same syntax I've used since C++11 (std::make_integer_sequence), and it's magically faster, and that's awesome.

My understanding is that, as an implementor of the C++ standard library, I probably would need to know about these compiler builtins. But the whole point of the standard library is to hide the nitty-gritty of builtins behind nice standardized facades. And we already have a facade for generating integer sequences; it's std::make_integer_sequence. We already have a facade for extracting one element from a tuple; it's std::tuple_element_t. We don't need more facades, more syntax. What we need is more behind-the-scenes magic to make the existing syntax faster... and we're already getting that, too, in all three major compilers, right?

> If you want to study some background, this is the core of tuple_element_t:
>
> https://github.com/taocpp/sequences/blob/master/include/tao/seq/type_by_index.hpp
>
> when you don’t have 1) available. It’s ugly, hard to understand, hard to teach as well as time and memory consuming. 

I guess I don't understand... isn't that just a reimplementation of std::tuple_element_t with a weird name?

In practice I'd never implement std::tuple_element_t yourself; I'd use the library implementation, which will be implemented in terms of the magically fast compiler builtins.

What does that code look like if you take all the identifiers and rename them according to standard library names, e.g. std::get<> instead of get_nth<>, std::tuple_element<> instead of type_by_index<>, etc.?  And then at that point, is there any actual code left, or are all those lines of code just reimplementing things that already exist in the standard library?

> Further material to study if you are interested:
>
> https://github.com/taocpp/sequences
> https://github.com/taocpp/tuple
>
> Even without 1) and 2), the non-recursive implementations allowed some significant improvements against libstd++ and libc++, some numbers for one example (generating a std::tuple with 265 elements via std::tuple_cat, MID=maximum instantiation depth):
>
> Linux, GCC5 + libstdc++, std::tuple: 19.6s, MID 3719
> Linux, GCC5 + libstdc++, tao::tuple:  1.2s, MID 26
> MacOS, Apple LLVM 7.0 (Clang ~3.7), std::tuple: 70.0s, MID 514
> MacOS, Apple LLVM 7.0 (Clang ~3.7), tao::tuple:  1.7s, MID 15

Well, that seems like a massive improvement! Either tao::tuple is missing something that std::tuple provides (i.e. it's trading functionality for speed), or else it's just massively better, in which case I would fully expect all the major vendors (MSVC++, libstdc++, libc++) to adopt it relatively quickly.

Did tao::tuple use the above-proposed core language changes to achieve that speed, or does it achieve speed based only on the existing core language plus vendor magic like __make_integer_seq (which is equally available to the writers of libc++)?  If the latter, then tao::tuple is just a better implementation of std::tuple, and I hope it gets picked up quickly.

–Arthur

[Daniel Frey]

> On 02.03.2016, at 02:50, Arthur O'Dwyer <arthur....@gmail.com> wrote:
>
> On Tue, Mar 1, 2016 at 1:27 PM, Daniel Frey <d.f...@gmx.de> wrote:
> > On 29.02.2016, at 06:36, Arthur O'Dwyer <arthur....@gmail.com> wrote:
> > > On Sunday, February 28, 2016 at 1:32:56 PM UTC-8, Daniel Frey wrote:
> > > > I’d like to propose three basic functionalities wrt parameter packs.
> > > >
> > > > 1) Allow expansion of only a single element via pattern...[I].
> > > > 2) Allow in-situ creation of unexpanded non-type parameter packs via Type... N.
> > > > 3) Allow aliases via using Is = std::size_t... 5.
> >
> > > No specific comment on the other proposals (1 and 2), except to say that they don't add
> > > any new expressiveness to the language, just core-language syntactic sugar for constructs
> > > we can already express efficiently through the standard library (tuple_element_t, for
> > > example), and therefore I don't think they stand much chance. Also, they use up
> > > valuable syntactic real estate, so I don't think they should be adopted.
> >
> > FWIW, three compilers (VC++, GCC, Clang) are implementing a compiler intrinsic
> > called __make_integer_seq which could be replaced by 2) and we gain the following
> > advantages:
> >
> > - More expressiveness as an unexpanded parameter pack is generated, not just an instance of a class (std::integer_sequence)
> > - Portable, standardized syntax
>
> In the part of my post that you snipped, I indicated that "generating an unexpanded parameter pack" is a liability, not an advantage. You seemed to agree ("Good catch ... I'll reconsider that one ..."). Maybe I should say is currently a liability. It would be awesome to have first-class parameter packs, but I think there are deep problems someone needs to solve before we can get them.

When I said “Good catch” I meant that you spotted a problem that I haven’t realized before. It’s an interesting problem and, of course, it makes me reconsider given that you, IIUC, raised it in the context of my proposed language addition 3). That said, I have now thought about it some more and I think that the problem is completely independent of what I’m suggesting. This problem should be solved in its own proposal. My proposal addresses a different issue as I’ll try to explain below.

> Portable, standardized syntax for std::make_integer_sequence already exists; it's std::make_integer_sequence!
> You have correctly noticed that current implementations of make_integer_sequence are pretty slow, and that recently there's been some behind-the-scenes work in VC++, GCC, and Clang to speed up make_integer_sequence via the use of compiler builtins. However, as a user of C++1z, I don't need to know about these builtins; I just use the same syntax I've used since C++11 (std::make_integer_sequence), and it's magically faster, and that's awesome.
>
> My understanding is that, as an implementor of the C++ standard library, I probably would need to know about these compiler builtins. But the whole point of the standard library is to hide the nitty-gritty of builtins behind nice standardized facades. And we already have a facade for generating integer sequences; it's std::make_integer_sequence. We already have a facade for extracting one element from a tuple; it's std::tuple_element_t. We don't need more facades, more syntax. What we need is more behind-the-scenes magic to make the existing syntax faster... and we're already getting that, too, in all three major compilers, right?

I disagree, and I’ll try to explain why: The whole point of my proposal is that having at least 1) and 2) and maybe also 3) is the bread and butter needed for efficient handling of variadic templates. With std::make_index_sequence, you end up instantiating a template - even if you do so with a compiler intrinsic. The result is a type, not a parameter pack. If you need the latter (and you very often do need the latter), you need to write another method, forward the type to it and extract the parameter pack by deduction. You thereby instantiate another function/method.

The number of instantiations created just because of the lack of what I’m proposing is not to be underestimated. I learned this lesson when working on tao::tuple. It started when I listened to a talk in the C++ user group in Aachen where the shown solutions for working with variadic templates where almost all recursive. I already knew from my workplace that recursion leads to tons of instantiations, so I offered to give an alternative talk to show how to do things without recursion. When I started to write the slides, I realized that I should also back up the claimed importance with some numbers, so I created taocpp/sequence and taocpp/tuple. I was more than surprised to find out that the benefits for tuple, especially tuple_cat where much larger than I anticipated. What I also learned is that you can not simply pin the inefficiency to a single method, e.g., make_integer_sequence or tuple_element_t.

While working on toacpp, I also realized that there are two things missing to get rid of another bunch of instantiations and also of some overly complicated code. This is what lead to my proposal, points 1) and 2). Point 3) is mostly the logical extension.

> > If you want to study some background, this is the core of tuple_element_t:
> >
> > https://github.com/taocpp/sequences/blob/master/include/tao/seq/type_by_index.hpp
> >
> > when you don’t have 1) available. It’s ugly, hard to understand, hard to teach as well as time and memory consuming.
>
> I guess I don't understand... isn't that just a reimplementation of std::tuple_element_t with a weird name?
> In practice I'd never implement std::tuple_element_t yourself; I'd use the library implementation, which will be implemented in terms of the magically fast compiler builtins.
> What does that code look like if you take all the identifiers and rename them according to standard library names, e.g. std::get<> instead of get_nth<>, std::tuple_element<> instead of type_by_index<>, etc.? And then at that point, is there any actual code left, or are all those lines of code just reimplementing things that already exist in the standard library?

type_by_index is used to implement tuple_element_t, among others. But the point is that the interface, std::tuple_element_t, is a bad interface as you are again creating instantiations. Even without a full instantiation, the compiler still creates some entity for the type.

> > Linux, GCC5 + libstdc++, std::tuple: 19.6s, MID 3719
> > Linux, GCC5 + libstdc++, tao::tuple: 1.2s, MID 26
> > MacOS, Apple LLVM 7.0 (Clang ~3.7), std::tuple: 70.0s, MID 514
> > MacOS, Apple LLVM 7.0 (Clang ~3.7), tao::tuple: 1.7s, MID 15
>
> Well, that seems like a massive improvement! Either tao::tuple is missing something that std::tuple provides (i.e. it's trading functionality for speed), or else it's just massively better, in which case I would fully expect all the major vendors (MSVC++, libstdc++, libc++) to adopt it relatively quickly.
> Did tao::tuple use the above-proposed core language changes to achieve that speed, or does it achieve speed based only on the existing core language plus vendor magic like __make_integer_seq (which is equally available to the writers of libc++)? If the latter, then tao::tuple is just a better implementation of std::tuple, and I hope it gets picked up quickly.

tao::tuple does not trade functionality for speed. It gains its (compilation) speed by avoiding to instantiate tons of helpers. One source of superfluous instantiations was recursion, and while implementing the alternative I realized that there is a second source of superfluous instantiations which could only be removed when we don’t rely on make_integer_sequence or tuple_element_t anymore. Removing the first part (recursion) already shows how big the impact is. Compiler intrinsics that are used behind the scenes might provide some improvements, but this is only half the way. We can do more, we can do better.

Besides the speed up and reduced memory usage for the compiler, there is another benefit for the user: Code becomes much easier to write. Consider the example I gave in the paper:

template<class F, class Tuple, size_t... I>
decltype(auto) apply_impl(F&& f, Tuple&& t, index_sequence<I...>) {
return forward<F>(f)(get<I>(forward<Tuple>(t))...);
}

template<class F, class Tuple>
decltype(auto) apply(F&& f, Tuple&& t) {
using Indices = make_index_sequence<tuple_size<decay_t<Tuple>>::value>;
return apply_impl(forward<F>(f), forward<Tuple>(t), Indices());
}

This is already using the existing interface for make_index_sequence. Try explaining it to a student. Heck, try explaining to *me* why C++ does need to create a type first just to then extract “size_t... I” from it. ;) Why can’t we do this directly? With the proposed extension, it becomes:

template<class F, class Tuple>
decltype(auto) apply(F&& f, Tuple&& t) {
constexpr auto Size = tuple_size<decay_t<Tuple>>::value;
return forward<F>(f)(get<size_t... Size>(forward<Tuple>(t))...);
}

Still complicated, but much easier and without additional instantiations of apply_impl.

For me, that is the difference between something that is possible in C++ and something that is supported by C++. Keep in mind that the std::tuple’s were written by some of the (at least IMHO) best C++ developers around, they are way above average. Yet, it seems that the importance of instantiating large numbers of helpers was not fully seen/realized. I hope that this will change in the future and I think that my proposal would give another important options to avoid those instantiations.

For the implementation of tao::tuple being picked up by the different STLs: I’d like that and I can only offer it to them - which I already did. FWIW, I also offered to work on it myself for open-source STLs (libstdc++, libc++) or to relicense where/when necessary (MS?). There wasn’t much response, but my offer still stands.

Best regards,
Daniel

[Daniel Frey]

> On 02.03.2016, at 02:50, Arthur O'Dwyer <arthur....@gmail.com> wrote:
>
> Portable, standardized syntax for std::make_integer_sequence already exists; it's std::make_integer_sequence!

Just one more remark about this: The improvements on libc++ for std::tuple vs. tao::tuple are *not* because of make_integer_sequence. In fact, tao::make_integer_sequence is *slower* than the (incredibly smart) implementation of std::make_integer_sequence found in libc++. The improvements are caused by avoiding instantiations in lots of places (some/most of them using make_integer_sequence). I believe there is no one point where you can fix it by “just” improving make_integer_sequence and/or tuple_element_t, hence the language needs to support it directly.

[Arthur O'Dwyer]

On Wed, Mar 2, 2016 at 11:34 AM, Daniel Frey <d.f...@gmx.de> wrote:
> On 02.03.2016, at 02:50, Arthur O'Dwyer <arthur....@gmail.com> wrote:
> > On Tue, Mar 1, 2016 at 1:27 PM, Daniel Frey <d.f...@gmx.de> wrote:
> > > On 29.02.2016, at 06:36, Arthur O'Dwyer <arthur....@gmail.com> wrote:
> > > > On Sunday, February 28, 2016 at 1:32:56 PM UTC-8, Daniel Frey wrote:
> > > > > I’d like to propose three basic functionalities wrt parameter packs.
> > > > >
> > > > > 1) Allow expansion of only a single element via pattern...[I].
> > > > > 2) Allow in-situ creation of unexpanded non-type parameter packs via Type... N.
> > > > > 3) Allow aliases via using Is = std::size_t... 5.

[...]

> > >
> > > - More expressiveness as an unexpanded parameter pack is generated, not just

> > > an instance of a class (std::integer_sequence)
> > > - Portable, standardized syntax
> >
> > In the part of my post that you snipped, I indicated that "generating an unexpanded
> > parameter pack" is a liability, not an advantage. You seemed to agree ("Good catch
> > ... I'll reconsider that one ...").  Maybe I should say is currently a liability. It would be
> > awesome to have first-class parameter packs, but I think there are deep problems
> > someone needs to solve before we can get them.
>
> When I said “Good catch” I meant that you spotted a problem that I haven’t realized
> before. It’s an interesting problem and, of course, it makes me reconsider given that
> you, IIUC, raised it in the context of my proposed language addition 3).

It also applies to addition (2), because that also turns a non-pack expression into a (first-class) parameter pack.
I do think addition (1) is immune from this particular problem, but I believe addition (1) is 100% equivalent to syntactic sugar for std::get<I>(std::forward_as_tuple(pattern...)).

Section 5.2 of the original "Variadic Templates" proposal, N1603, deals with the idea of pattern.[I] being syntactic sugar for std::get<I>(std::forward_as_tuple(pattern...)), and apparently the original authors didn't think it would be useful. Obviously this may have changed in the 12 years since 2004.
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2004/n1603.pdf

What if C++1z were to add a single "unwrapped" version of std::get, something like this?

template<size_t Idx, typename... T>

inline __attribute__((__visibility__("hidden"), always_inline))

decltype(auto) getp(T&&... t) {

    return std::get<Idx>(std::forward_as_tuple(t...));

}

That is, getp would be to std::get as std::experimental::invoke is to std::experimental::apply.

The library vendor would be free to optimize the internal implementation of getp so that it could be constant-time instead of logarithmic (or worse) in the value of Idx.

Would this solve your problem (1) without a core language change?  That is, if you could take your existing hypothetical code using params...[I] and rewrite it to use std::getp<I>(params...), would that be acceptable?

(I can confirm that just because a call to std::getp<I> appears in the source code for a TU does not mean that the compiler is forced to generate out-of-line code for std::getp<I>, because empirically Clang does not generate it. The above gives perfect codegen at -O3. I can definitely believe that it doesn't give perfect performance in terms of compile time, but I think that would be just a QoI issue, solvable by the vendor in whatever way they found most appropriate.)

[...]

> Heck, try explaining to *me* why C++ does need to create a type first just to then extract “size_t... I” from it. ;)

> Why can’t we do this directly? With the proposed extension, it becomes:

>
> template<class F, class Tuple>
> decltype(auto) apply(F&& f, Tuple&& t) {
>   constexpr auto Size = tuple_size<decay_t<Tuple>>::value;
>   return forward<F>(f)(get<size_t... Size>(forward<Tuple>(t))...);
> }

Because the above version involves creating a pack from a non-pack (in the "expression" size_t... Size). Once you allow that, you have to explain what would happen with

template<size_t I>

auto foo() { return bar<size_t... I ...>(); }

template<size_t... Is>

auto bar() { return baz<size_t... Is ... ...>(); }

and so on. Either you have to devote a lot of thought and wording to explaining the semantics of naked parameter-packs, or you have to make sure that every pack is always wrapped up in a type where it can't escape and wreak havoc.

–Arthur

[Matthew Woehlke]

On 2016-02-28 16:32, Daniel Frey wrote:
> I’d like to propose three basic functionalities wrt parameter packs.
>
> 1) Allow expansion of only a single element via pattern...[I].

I want to put in a strong vote for `[I]pack` instead. This is consistent
with my proposals for `[I]tuple` (which in fact I extended to `[I]pack`
already).

> 2) Allow in-situ creation of unexpanded non-type parameter packs via Type... N.

I think you need to talk to Mike Spertus regarding this one. He has a
proposal for parameter pack literals that is very similar and possibly
should incorporate this. At least, it would be best if the two of you
are aware of each other's work.

> 3) Allow aliases via using Is = std::size_t... 5.

Mike *might* already be covering this case, also.

--
Matthew

[Daniel Frey]

> On 06.03.2016, at 19:55, Arthur O'Dwyer <arthur....@gmail.com> wrote:

FWIW, I think I now have a much better understanding about the problem you raised about (2) and (3), I’ll address that in a separate reply when I had enough time to think about it. In this reply I’ll only address everything wrt proposal (1), the single expansion via pattern...[I]

> I do think addition (1) is immune from this particular problem, but I believe addition (1) is 100% equivalent to syntactic sugar for std::get<I>(std::forward_as_tuple(pattern...)).

Not quite, as it doesn’t work for type-packs, only for value-packs. For value-packs, since C++14 both std::get as well as std::forward_as_tuple are constexpr so that is also covered. It’s still a lot to write compared to pattern...[I] and it feels very indirect. It also internally will most likely have to go through the type-pack case.

Also, my proposal should not be limited to provide pack expansion only in the existing places where you’d have a parameter list. Since I’m only expanding a single pattern, it might just be a single call to a function or a single value. For example:

int i = Ns...[I];

which would simply pick the I-th element from the Ns. A single selection always returns a single type, a single value, or a single statement. It could be used in much more contexts as normal parameter pack expansion:

print(Ns*2+1)...[I];

which would be a call to print(N*2+1); where N is the I-th element of the parameter pack Ns.

Anyways, for types, you’d need std::tuple_element_t<std::tuple<pattern...>>. Certainly possible, but still inefficient. Internally, it could benefit from something like:

template< std::size_t I, typename... Ts >
using type_by_index_t = typename type_by_index< I, Ts... >::type;

(an implementation of type_by_index is available at https://github.com/taocpp/sequences/blob/master/include/tao/seq/type_by_index.hpp)

In fact, I used type_by_index to implement tao::tuple’s “tuple_element_t" as well as “get<I>". A compiler vendor might provide some compiler intrinsic with much better efficiency, but I still think that the additional steps required for values will justify (1). For values, the interface I was using is:

template< std::size_t I, typename T, T... Ns >
struct select
: std::integral_constant< T, values< T, Ns... >::data[ I ] >
{};

template< std::size_t I, typename T, T... Ns >
struct select< I, integer_sequence< T, Ns... > >
: select< I, T, Ns... >
{};

with

template< typename T, T... Ns >
struct values
{
static constexpr T data[] = { Ns... };
};

While this is not too bad from an implementation’s point of view, it is still very indirect and you’d either have to have an index_sequence S already to call

select<I, S>

or you need to provide the type:

select<I, T, Vs...> // Vs are values of integral type T

> Section 5.2 of the original "Variadic Templates" proposal, N1603, deals with the idea of pattern.[I] being syntactic sugar for std::get<I>(std::forward_as_tuple(pattern...)), and apparently the original authors didn't think it would be useful. Obviously this may have changed in the 12 years since 2004.

IMHO my work on tao::tuple vs. std::tuple shows the potential. tao::tuple only uses better implementations using existing language features, but it also made me aware of the additional places where inefficiencies are only avoidable by adding some primitives wrt variadic templates to the language.

> What if C++1z were to add a single "unwrapped" version of std::get, something like this?
>
> template<size_t Idx, typename... T>
> inline __attribute__((__visibility__("hidden"), always_inline))
> decltype(auto) getp(T&&... t) {
> return std::get<Idx>(std::forward_as_tuple(t...));
> }
>
> That is, getp would be to std::get as std::experimental::invoke is to std::experimental::apply.
> The library vendor would be free to optimize the internal implementation of getp so that it could be constant-time instead of logarithmic (or worse) in the value of Idx.
> Would this solve your problem (1) without a core language change? That is, if you could take your existing hypothetical code using params...[I] and rewrite it to use std::getp<I>(params...), would that be acceptable?
> (I can confirm that just because a call to std::getp<I> appears in the source code for a TU does not mean that the compiler is forced to generate out-of-line code for std::getp<I>, because empirically Clang does not generate it. The above gives perfect codegen at -O3. I can definitely believe that it doesn't give perfect performance in terms of compile time, but I think that would be just a QoI issue, solvable by the vendor in whatever way they found most appropriate.)

Make “getp” constexpr, but yes, that’s another version of the above select, where your usage is somehow easier: “std::getp<I>(vs...)”. This could work, together with “std::type_by_index_t<I, Ts...>”. Still both interfaces are quite different and we certainly need better names :)

FWIW, there was http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3761.html which used std::type_at and std::value_at.

But for the user (1) is certainly more convenient plus is would allow additional use cases like the print-example I gave above. To quote from N4492:

> Bad (committee) habits to avoid, to avoid giving critics more ammunition:
> • Make something a library because that's easier to get accepted in the committee than a language feature (even if there is good argument that what is provided is fundamental)

I’d like to explore which primitives are missing wrt variadic templates and parameter packs. (1) seems like such a basic thing and it is useful in many places, that I’d really like to see it as part of the language. We’ll see about (2) and (3)… :)

@MatthewWoehlke: Thanks for the hint. I am aware of N4235 from Daveed Vandevoorde (also making a good case for (1) and (2)), haven’t contacted Mike Spertus yet. I might do so when I make some more progress with (2) and (3) and the other issue raised by Arthur or he might join the discussion here if he wants to. The syntax for (1) seems to be a concern, so far pattern...[I], pattern.[I] and your [I]pattern are the options. But it’s pointless to discuss when it’s not clear yet if the functionality is actually wanted in the language, let’s wait that out first.

-- Daniel

[Arthur O'Dwyer]

On Wed, Mar 9, 2016 at 2:15 PM, Daniel Frey <d.f...@gmx.de> wrote:

On 06.03.2016, at 19:55, Arthur O'Dwyer <arthur....@gmail.com> wrote:
> I do think addition (1) is immune from this particular problem, but I believe addition (1) is 100% equivalent to syntactic sugar for std::get<I>(std::forward_as_tuple(pattern...)).

Not quite, as it doesn’t work for type-packs, only for value-packs. For value-packs, since C++14 both std::get as well as std::forward_as_tuple are constexpr so that is also covered. It’s still a lot to write compared to pattern...[I] and it feels very indirect. It also internally will most likely have to go through the type-pack case.

My thinking was that you could get the type via decltype(std::get<I>(std::forward_as_tuple(pattern...))),

or if you had a type-pack already, decltype(std::get<I>(std::forward_as_tuple(std::declval<pattern>()...))).

I admit this has problems with type-packs containing void, though, so there's room for improvement in the library. (See below.)

You wrote: 

    print(Ns*2+1)...[I];

Is the above equivalent to

    print((Ns...[I])*2+1);

? (Maybe the former has some SFINAE implications, I guess, but basically yes?)  I'm not really seeing why I'd want to use your example instead of the latter. Can you come up with another example that shows why you might want to put the ...[I] after the whole pattern, instead of right after the pack?

I added the "redundant" inner parentheses because I think otherwise we run into grammatical ambiguities again, due to the fact that ... syntactically doesn't behave like a postfix operator but more like a comma. (You'll find posts in here from two months ago where I claimed ... does act basically like a postfix operator. My views have evolved. :))

 

> What if C++1z were to add a single "unwrapped" version of std::get, something like this?
>
> template<size_t Idx, typename... T>
> inline __attribute__((__visibility__("hidden"), always_inline))
> decltype(auto) getp(T&&... t) {
>     return std::get<Idx>(std::forward_as_tuple(t...));
> }
>
> That is, getp would be to std::get as std::experimental::invoke is to std::experimental::apply.
> The library vendor would be free to optimize the internal implementation of getp so that it could be constant-time instead of logarithmic (or worse) in the value of Idx.
> Would this solve your problem (1) without a core language change?  That is, if you could take your existing hypothetical code using params...[I] and rewrite it to use std::getp<I>(params...), would that be acceptable?
> (I can confirm that just because a call to std::getp<I> appears in the source code for a TU does not mean that the compiler is forced to generate out-of-line code for std::getp<I>, because empirically Clang does not generate it. The above gives perfect codegen at -O3. I can definitely believe that it doesn't give perfect performance in terms of compile time, but I think that would be just a QoI issue, solvable by the vendor in whatever way they found most appropriate.)

Make “getp” constexpr, but yes, that’s another version of the above select, where your usage is somehow easier: “std::getp<I>(vs...)”. This could work, together with “std::type_by_index_t<I, Ts...>”. Still both interfaces are quite different and we certainly need better names :)

FWIW, there was http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3761.html which used std::type_at and std::value_at.

I wasn't aware of that paper.  I'd be in favor of that!  If I understand correctly, Sean's std::value_at<I>(Vs...) is what I just called std::getp<I>(Vs...) and what you call Vs...[I]. Sean's std::type_at<I, Ts...>::type is what you call Ts...[I], and what I'd have to call some monstrosity like decltype(std::getp<I>(std::declval<Ts>()...)) and still wouldn't handle void correctly.

Notice that in C++14land, N3761's std::type_at<I, Ts...>::type would normally be used as std::type_at_t<I, Ts...>.

–Arthur

[barry....@gmail.com]

I am a fan of simply allowing pattern...[I], but the rest of the proposal strikes me as maybe wanting to solve the wrong problem? It does make writing something like apply easier, but it'd be even easier if I could just do:

    template <class F, class Tuple>

    decltype(auto) apply(F&& f, Tuple&& tuple) {

        return forward<F>(f)(forward<Tuple>(tuple)...);

    }

I'm not sure what rules you'd have to create in order to allow for the direct unpacking of tuple, but I think that's what we're going to want to do to improve pack fundamentals. Unpacking the tuple just makes sense. All the other std::get<> and index_sequence stuff is a workaround for not being able to do that. 

My $0.01,

Barry

[Daniel Frey]

> On 10.03.2016, at 01:05, Arthur O'Dwyer <arthur....@gmail.com> wrote:
>
> On Wed, Mar 9, 2016 at 2:15 PM, Daniel Frey <d.f...@gmx.de> wrote:
> On 06.03.2016, at 19:55, Arthur O'Dwyer <arthur....@gmail.com> wrote:
> > I do think addition (1) is immune from this particular problem, but I believe addition (1) is 100% equivalent to syntactic sugar for std::get<I>(std::forward_as_tuple(pattern...)).
>
> Not quite, as it doesn’t work for type-packs, only for value-packs. For value-packs, since C++14 both std::get as well as std::forward_as_tuple are constexpr so that is also covered. It’s still a lot to write compared to pattern...[I] and it feels very indirect. It also internally will most likely have to go through the type-pack case.
>
> My thinking was that you could get the type via decltype(std::get<I>(std::forward_as_tuple(pattern...))),
> or if you had a type-pack already, decltype(std::get<I>(std::forward_as_tuple(std::declval<pattern>()...))).
> I admit this has problems with type-packs containing void, though, so there's room for improvement in the library. (See below.)

To make void work, you’d end up with something like:

unwrap_t<decltype(std::get<I>(std::forward_as_tuple(wrap<pattern>()...)))>

Not really what I’d like to write just to get the I-th type. Could be hidden behind some other interface (std::type_at_t<I,Ts...>, but I feel like using tuple to implement this is the wrong thing to do. tuple needs all those fundamental operations to be implemented, not the other way round.

> You wrote:
> print(Ns*2+1)...[I];
>
> Is the above equivalent to
> print((Ns...[I])*2+1);

The above example, yes, it’s equivalent. But an expression (pattern) to expand may also contain multiple parameter packs (of the same length) or the same parameter pack multiple times. Example:

print(Ns * Ns + Ms)...[I];

or if you don’t allow statements as patterns:

print((Ns * Ns + Ms)...[I]);

which is much easier than the alternative:

print(Ns...[I] * Ns...[I] + Ms...[I]);

Also, as pattern..,.[I] is an expression it may contain an unexpanded parameter pack in place of I and be used like this:

print_variadic((Ns * Ns + Ms)...[Is]...);

which, given Is = {4, 1, 3} is equivalent to:

print_variadic(Ns*Ns+Ms...[4], Ns*Ns+Ms...[1], Ns*Ns+Ms...[3]);

> ? (Maybe the former has some SFINAE implications, I guess, but basically yes?) I'm not really seeing why I'd want to use your example instead of the latter. Can you come up with another example that shows why you might want to put the ...[I] after the whole pattern, instead of right after the pack?

Of course the above could, in theory, be rewritten with only immediate application to a parameter pack, not a more complex pattern. Not sure if you agree that the above example is convincing, but to me it seems that it’s worth it.

> I added the "redundant" inner parentheses because I think otherwise we run into grammatical ambiguities again, due to the fact that ... syntactically doesn't behave like a postfix operator but more like a comma. (You'll find posts in here from two months ago where I claimed ... does act basically like a postfix operator. My views have evolved. :))

I’ll have to think about those details, but thank you for pointing me to all those subtleties. While this part is only about (1), I also now had an idea how to combine (2) and (3) into a better proposal which should address most of the points you raised. I’ll write an answer to that at the weekend if I have time.

> > That is, getp would be to std::get as std::experimental::invoke is to std::experimental::apply.
> > The library vendor would be free to optimize the internal implementation of getp so that it could be constant-time instead of logarithmic (or worse) in the value of Idx.
> > Would this solve your problem (1) without a core language change? That is, if you could take your existing hypothetical code using params...[I] and rewrite it to use std::getp<I>(params...), would that be acceptable?
> > (I can confirm that just because a call to std::getp<I> appears in the source code for a TU does not mean that the compiler is forced to generate out-of-line code for std::getp<I>, because empirically Clang does not generate it. The above gives perfect codegen at -O3. I can definitely believe that it doesn't give perfect performance in terms of compile time, but I think that would be just a QoI issue, solvable by the vendor in whatever way they found most appropriate.)

I’m concerned about either the time it takes to compile this or the complexity the compiler vendor has to put in to optimize this case. I think it would be actually easier to get better performance and an easier implementation using a core language change. But mostly it’s the compile time I’m looking at. C++ has a bad reputation for being slow to compile, I think that variadic templates and how parameter packs are handled have a great influence here, encouraging and often requiring an inefficient programming style. Plus it’s currently often quite hard and indirect to express yourself, leading to more bugs. With SFINAE overloads being enabled/disabled when calling some function/method, with noexcept checking parameters for properties, etc. we are using more and more of those “little helpers” behind the scenes, often without noticing them. But as little as each single helper or invocation is, it piles up and can make a huge difference.

> Make “getp” constexpr, but yes, that’s another version of the above select, where your usage is somehow easier: “std::getp<I>(vs...)”. This could work, together with “std::type_by_index_t<I, Ts...>”. Still both interfaces are quite different and we certainly need better names :)
>
> FWIW, there was http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3761.html which used std::type_at and std::value_at.
>
> I wasn't aware of that paper. I'd be in favor of that! If I understand correctly, Sean's std::value_at<I>(Vs...) is what I just called std::getp<I>(Vs...) and what you call Vs...[I]. Sean's std::type_at<I, Ts...>::type is what you call Ts...[I], and what I'd have to call some monstrosity like decltype(std::getp<I>(std::declval<Ts>()...)) and still wouldn't handle void correctly.
>
> Notice that in C++14land, N3761's std::type_at<I, Ts...>::type would normally be used as std::type_at_t<I, Ts...>.

Yes, it’s as good as it gets for a library-based interface. You’d have std::type_at_t<I, Ts...> and std::value_at<I>(Vs...). Still there are a few points in favor of ...[I]:

* It’s shorter to write, more direct and consistent (both cases, type and value, have the same syntax).
* It’s most likely more efficient wrt compiler resources (time and memory), especially when using a more complex pattern. No need for the compiler to expand all the patterns and throw away all-but-one. And throwing away does not mean that all memory of the compiler will be freed.
* It could be used to expand a pattern into a single statement, but that is an option I need to think about more deeply.

— Daniel

[Arthur O'Dwyer]

On Fri, Mar 11, 2016 at 3:28 PM, Daniel Frey <d.f...@gmx.de> wrote:
> On 10.03.2016, at 01:05, Arthur O'Dwyer <arthur....@gmail.com> wrote:
> >
> > My thinking was that you could get the type via decltype(std::get<I>(std::forward_as_tuple(pattern...))),
> > or if you had a type-pack already, decltype(std::get<I>(std::forward_as_tuple(std::declval<pattern>()...))).
> > I admit this has problems with type-packs containing void, though, so there's room for improvement

> > in the library. (See below.)
>
> To make void work, you’d end up with something like:
>
>   unwrap_t<decltype(std::get<I>(std::forward_as_tuple(wrap<pattern>()...)))>
>
> Not really what I’d like to write just to get the I-th type.

Agreed.

> Could be hidden behind some other interface (std::type_at_t<I,Ts...>, but I feel like using

> tuple to implement this is the wrong thing to do. tuple needs all those fundamental operations

> to be implemented, not the other way round.

Once you hide it behind an interface (e.g. std::type_at), you don't have to implement it using std::tuple anymore; you could implement it using __builtin_nth_element<> or whatever. And once you hide it behind an interface and standardize that interface, the vendor can implement it using whatever, even including a core language extension (possibly a non-standard but conforming extension) if a certain compiler happens to provide it, but falling back on builtins otherwise, and even falling back on template meta-programming if the compiler is too old for the builtins.

>
> > You wrote:
> >     print(Ns*2+1)...[I];
> >
> > Is the above equivalent to
> >     print((Ns...[I])*2+1);
>
> The above example, yes, it’s equivalent. But an expression (pattern) to expand may also contain multiple parameter packs (of the same length) or the same parameter pack multiple times. Example:
>
>   print(Ns * Ns + Ms)...[I];

Ah, right. I'd forgotten about that application.

But then you really need to specify what the above would mean for SFINAE. Let's say (Ns) ...[1] would cause a substitution failure; does (Ns * Ns) ...[0] still compile? What about (Ns * Ns) ...[2]?

> Also, as pattern..,.[I] is an expression it may contain an unexpanded parameter pack in place of I and be used like this:
>   print_variadic((Ns * Ns + Ms)...[Is]...);
> which, given Is = {4, 1, 3} is equivalent to:
>   print_variadic(Ns*Ns+Ms...[4], Ns*Ns+Ms...[1], Ns*Ns+Ms...[3]);

FYI, despite the "nested" ...s, I have not found anything troublesome about that example yet. It might be okay. ;)

–Arthur

[Daniel Frey]

> On 12.03.2016, at 00:50, Arthur O'Dwyer <arthur....@gmail.com> wrote:
>
> On Fri, Mar 11, 2016 at 3:28 PM, Daniel Frey <d.f...@gmx.de> wrote:
> > On 10.03.2016, at 01:05, Arthur O'Dwyer <arthur....@gmail.com> wrote:
> > >
> > > My thinking was that you could get the type via decltype(std::get<I>(std::forward_as_tuple(pattern...))),
> > > or if you had a type-pack already, decltype(std::get<I>(std::forward_as_tuple(std::declval<pattern>()...))).
> > > I admit this has problems with type-packs containing void, though, so there's room for improvement
> > > in the library. (See below.)
> >
> > To make void work, you’d end up with something like:
> >
> > unwrap_t<decltype(std::get<I>(std::forward_as_tuple(wrap<pattern>()...)))>
> >
> > Not really what I’d like to write just to get the I-th type.
>
> Agreed.
>
> > Could be hidden behind some other interface (std::type_at_t<I,Ts...>, but I feel like using
> > tuple to implement this is the wrong thing to do. tuple needs all those fundamental operations
> > to be implemented, not the other way round.
>
> Once you hide it behind an interface (e.g. std::type_at), you don't have to implement it using std::tuple anymore; you could implement it using __builtin_nth_element<> or whatever. And once you hide it behind an interface and standardize that interface, the vendor can implement it using whatever, even including a core language extension (possibly a non-standard but conforming extension) if a certain compiler happens to provide it, but falling back on builtins otherwise, and even falling back on template meta-programming if the compiler is too old for the builtins.

Yes, but the compiler would still expand all expressions (pattern), then pass them to std::type_at_t (or std::value_at) and only then the compiler intrinsic will be able to optimize - too late IMHO. Unless the compiler would treat std::type_at_t and std::value_at as the intrinsic, giving it a special treatment.

> > > You wrote:
> > > print(Ns*2+1)...[I];
> > >
> > > Is the above equivalent to
> > > print((Ns...[I])*2+1);
> >
> > The above example, yes, it’s equivalent. But an expression (pattern) to expand may also contain multiple parameter packs (of the same length) or the same parameter pack multiple times. Example:
> >
> > print(Ns * Ns + Ms)...[I];
>
> Ah, right. I'd forgotten about that application.
> But then you really need to specify what the above would mean for SFINAE. Let's say (Ns) ...[1] would cause a substitution failure; does (Ns * Ns) ...[0] still compile? What about (Ns * Ns) ...[2]?

I think that this is actually another benefit of having ...[I] over a library-based interface. For the library interface, all expansions must be valid - the pack selection would only require the selected expansion to be valid (and participate in SFINAE). Let’s say I identified all indices from a pack that could be printed. Those are indices 0, 1, 3, 4 and 7. Having Is = {0,1,3,4,7} would allow for

print<Ns*Ns+Ms>()...[Is]...;

without triggering an invalid instantiation of print (I’m using print<N>() because now there could be some SFINAE going on allowing only specific values for N).

An open question for the above would be the order of evaluation. Sequential, i.e., sequence points between each expression? Or unspecified? Hm. Sequential when ending with semicolon, unspecified when ending in a sequence of comma and semicolon? Eeek :(

In theory there is also the question of multiple parameter packs in the pattern of different length. As long as the expansion does not exceed the smallest parameter pack, would that be OK? Technically it shouldn’t be too difficult, but I’d be concerned that it might lead to more bugs and it will probably not be needed very often. For now, I think the length of the parameter packs should be forced to be equal.

> > Also, as pattern..,.[I] is an expression it may contain an unexpanded parameter pack in place of I and be used like this:
> > print_variadic((Ns * Ns + Ms)...[Is]...);
> > which, given Is = {4, 1, 3} is equivalent to:
> > print_variadic(Ns*Ns+Ms...[4], Ns*Ns+Ms...[1], Ns*Ns+Ms...[3]);
>
> FYI, despite the "nested" ...s, I have not found anything troublesome about that example yet. It might be okay. ;)

That’s a very good sign, thanks :)

— Daniel

[Daniel Frey]

> On 06.03.2016, at 19:55, Arthur O'Dwyer <arthur....@gmail.com> wrote:
>
> > > In the part of my post that you snipped, I indicated that "generating an unexpanded
> > > parameter pack" is a liability, not an advantage. You seemed to agree ("Good catch
> > > ... I'll reconsider that one ..."). Maybe I should say is currently a liability. It would be
> > > awesome to have first-class parameter packs, but I think there are deep problems
> > > someone needs to solve before we can get them.
> >
> > When I said “Good catch” I meant that you spotted a problem that I haven’t realized
> > before. It’s an interesting problem and, of course, it makes me reconsider given that
> > you, IIUC, raised it in the context of my proposed language addition 3).
>
> It also applies to addition (2), because that also turns a non-pack expression into a (first-class) parameter pack.

I think I now have a solution for the problem you spotted with (2) and (3), the new proposal (attached) solves this by not allowing in-situ creation or in-situ access to member packs. In short, I only propose two new features now:

(1) Pack selection:

pattern...[I]

(2) Pack declaration:

typename... Ts; // empty pack
typename... Ts = int, void, double&&;
using typename... Ts = my_class<int>::types; // access pack member, this can not be templated
std::size_t... Vs; // empty pack
std::size_t... Vs = 3, 0, 1, 42;
using std::size_t... Vs = my_class<int>::values;
std::size_t... Vs = ... 4; // yields 0, 1, 2, 3

I'll improve the proposal with better examples, rationale, etc. later on, but I'd like to get some feedback if you (and others) agree that the above avoids the problems that the previous proposal had and in general if the syntax and the feature are the right fundamentals to write easier and better code.

-- Daniel

[inkwizyt...@gmail.com]

 I think syntax of `using` should look like:

using Ts = typename... X<int>::pack; //like `using T = typename X<int>::type;`
using Vs = std::size_t... Y<int>::values_pack;
template<typename T, typename X>
using Xs = T... X::pack; // using "Name" = "Category" "Value";

I don't think that you need `...4` because if we allow templated `using` packs then you can get this from:

template<int I>
using seq_p = std::size_t... std::make_index_sequence<I>::pack;

using Vs = seq_p<I>; //like `using T = aligned_storage_t<16>;`

[Daniel Frey]

Or maybe just

using... Ts = X<int>::pack;

After all, ::pack defines what it is already. I wonder if the proposal's (2) should be split into (2) containing syntax 1.1, 1.2, 2.1, 2.2 and 2.4 and a separate (3) for syntax 1.3 and 2.3 or even better the above "using...".

> template<typename T, typename X>
> using Xs = T... X::pack; // using "Name" = "Category" "Value";

This would be a major problem, exactly what I tried to avoid. The usage could otherwise be an expression like

template<typename... Ts> void bar() { return f(g(Xs<Ts>{}...)...; } // return ???

and you have to define what is expanded in which order. That's the problem Arthur spotted and which you either solve (which seems hard) or avoid.

> I don't think that you need `...4` because if we allow templated `using` packs then you can get this from:

Still it is very indirect and very inefficient. Plus your example is using the templated using for parameter packs which I can not allow.

The proposal is mostly meant to speed up compiler efficiency, hence most uses of make_integer_sequence should be replaced with the new syntax 2.4 and make_integer_sequence itself should be implemented with it as shown in the proposal.

-- Daniel

[Daniel Frey]

> On 13.03.2016, at 15:54, Daniel Frey <d.f...@gmx.de> wrote:
>
>> template<typename T, typename X>
>> using Xs = T... X::pack; // using "Name" = "Category" "Value";
>
> This would be a major problem, exactly what I tried to avoid. The usage could otherwise be an expression like
>
> template<typename... Ts> void bar() { return f(g(Xs<Ts>{}...)...; } // return ???

That should be

template<typename T, typename... Ts>
void bar()
{
return f(g(Xs<T, Ts>{}...)...);
}

but you get the idea... :)

[inkwizyt...@gmail.com]

 
`Xs` is pack and can be expand only once. First `...` apply to `Xs` not to `Ts`. This mean `Xs<T, Ts>` is invalid because `Ts` is not expand.
I think it should work like:

return f(g(Xs<T, Ts...>{}...));
//or
return f(g(Xs<T, Ts...>{})...);

As we see this is only one expansion. To have second one, I think we would need pack of pack to handle it:

template<typename T>
using id_t = T;
template<typename... T>
using id_p = typename... T; //Identity pack `id_p<T...> === T`
template<typename... T>
using apply_p = typename... id_t<T>; //Transform `apply_p<T...> === id_t<T>`

template<typename T, typename... Ts>

using Xpp = typename... ... apply_p<T, Ts>; //Transform `Xpp<T, Ts...> === apply_p<T, Ts>` this is pack of pack

return f(g(Xpp<Tsingle, Ts...>{}...)...); //if Ts=[T1, T2] then this is equal `return f(g(id_t<Tsingle>{}, id_t<Tsingle>{}), g(id_t<T1>{}, id_t<T2>{}));`

This is because first `...` expand to `g(apply_p<Tsingle, T1>{}, apply_p<Tsingle, T2>{})` where this both are parameter packs.

[Daniel Frey]

> On 13.03.2016, at 20:32, inkwizyt...@gmail.com wrote:
>> On Sunday, March 13, 2016 at 3:59:58 PM UTC+1, Daniel Frey wrote:
>> > On 13.03.2016, at 15:54, Daniel Frey <d.f...@gmx.de> wrote:
>> >
>> >> template<typename T, typename X>
>> >> using Xs = T... X::pack; // using "Name" = "Category" "Value";
>> >
>> > This would be a major problem, exactly what I tried to avoid. The usage could otherwise be an expression like
>> >
>> > template<typename... Ts> void bar() { return f(g(Xs<Ts>{}...)...; } // return ???
>>
>> That should be
>>
>> template<typename T, typename... Ts>
>> void bar()
>> {
>> return f(g(Xs<T, Ts>{}...)...);
>> }
>>
>> but you get the idea... :)
>
> `Xs` is pack and can be expand only once. First `...` apply to `Xs` not to `Ts`. This mean `Xs<T, Ts>` is invalid because `Ts` is not expand.

Ts as also a pack. This is not defined by todays C++, hence you are expressing an idea you have, not a current reality. Other options exist of what the above could mean in the future. No option is the right one, it is a choice.

> I think it should work like:
>
> return f(g(Xs<T, Ts...>{}...));

This would be unambiguous. Let's say Xs = {A,B,C}, Ts = {int,long}:

return f(g(Xs<T, Ts...>{}...));

return f(g(Xs<T, int, long>{}...));
return f(g(A<T, int, long>{}, B<T, int, long>{}, C<T, int, long>{}));

> return f(g(Xs<T, Ts...>{})...);

return f(g(Xs<T, Ts...>{}...));

return f(g(Xs<T, int, long>{}...));
return f(g(A<T, int, long>{}, g(B<T, int, long>{}), g(C<T, int, long>{})));

OK, how do you expand to:

return f(g(A<T, int>{}, A<T, long>{}), g(B<T, int>{}, B<T, long>{}), g(C<T, int>{}, C<T, long>{})); // (e1)

or

return f(g(A<T, int>{}, B<T, int>{}, C<T, int>{}), g(A<T, long>{}, B<T, long>{}, C<T, long>{})); // (e2)

or

return f(g(A<T, int>{}, A<T, long>{}, B<T, int>{}, B<T, long>{}, C<T, int>{}, C<T, long>{})); // (e3)

or

return f(g(A<T, int>{}, B<T, int>{}, C<T, int>{}, A<T, long>{}, B<T, long>{}, C<T, long>{})); // (e4)

> As we see this is only one expansion. To have second one, I think we would need pack of pack to handle it:

And this is one of the things I'd like to avoid: Creating a pack of packs (and by induction a pack of packs of packs, etc.). It means creating new entities that don't exist today in a compiler's backend. The complexity to implement a proposal which does something like that is much higher and the risk of introducing new bugs makes it less likely that a proposal will be accepted.

Anyways, if you *really* want to do something like this I think it makes much more sense to have "partial expansion" something like:

return f(g(Xs<T, Ts>{}...[Ts])...); // generates (e1)
^^^^^^^
possible syntax when the brackets don't contain an integral constant as in my proposal (1), but instead a pack's name (or a set of names) that occur in the expression. Each step requires all packs mentioned to have the same length, but separate steps may have different lengths. Only those packs are expanded in each step which are listed, the result is another pack of expressions which contain unexpanded packs which are then expanded by the second ellipsis. The above therefore yields (e1). Complete list:

return f(g(Xs<T, Ts>{}...[Ts])...); // generates (e1)
return f(g(Xs<T, Ts>{}...[Xs])...); // generates (e2)
return f(g(Xs<T, Ts>{}...[Ts]...)); // generates (e3)
return f(g(Xs<T, Ts>{}...[Xs]...)); // generates (e4)

But this is probably material for a second proposal and I haven't really thought it through - it's just an idea. Unless everyone thinks it's a brilliant idea and should be part of my proposal right now :-P

-- Daniel

[Daniel Frey]

> On 10.03.2016, at 01:42, barry....@gmail.com wrote:
>
> I am a fan of simply allowing pattern...[I], but the rest of the proposal strikes me as maybe wanting to solve the wrong problem? It does make writing something like apply easier, but it'd be even easier if I could just do:
>
> template <class F, class Tuple>
> decltype(auto) apply(F&& f, Tuple&& tuple) {
> return forward<F>(f)(forward<Tuple>(tuple)...);
> }
>
> I'm not sure what rules you'd have to create in order to allow for the direct unpacking of tuple, but I think that's what we're going to want to do to improve pack fundamentals. Unpacking the tuple just makes sense. All the other std::get<> and index_sequence stuff is a workaround for not being able to do that.

It is an interesting idea, but I don't feel it is fundamental for handling packs. Instead it as applying the ellipsis to an expression which contain *no* packs. I think we are currently wrapping too many things into std::tuple when most of the time we'd like to handle packs directly. Unpacking a tuple is therefore, in my mind at least, trying to solve to wrong problem. It might still be worth as an additional feature to look into (maybe via an "operator..."), but it'll create a lot of headaches when an expression which is followed by an ellipsis contains both packs and object with an "operator...".

[inkwizyt...@gmail.com]

On Sunday, March 13, 2016 at 10:27:21 PM UTC+1, Daniel Frey wrote:

> On 13.03.2016, at 20:32, inkwizyt...@gmail.com wrote:
>> On Sunday, March 13, 2016 at 3:59:58 PM UTC+1, Daniel Frey wrote:
>> > On 13.03.2016, at 15:54, Daniel Frey <d.f...@gmx.de> wrote:
>> >
>> >> template<typename T, typename X>
>> >> using Xs = T... X::pack; // using "Name" = "Category" "Value";
>> >
>> > This would be a major problem, exactly what I tried to avoid. The usage could otherwise be an expression like
>> >
>> >  template<typename... Ts> void bar() { return f(g(Xs<Ts>{}...)...; }  // return ???
>>
>> That should be
>>
>>   template<typename T, typename... Ts>
>>   void bar()
>>   {
>>     return f(g(Xs<T, Ts>{}...)...);
>>   }
>>
>> but you get the idea... :)
>
> `Xs` is pack and can be expand only once. First `...` apply to `Xs` not to `Ts`. This mean `Xs<T, Ts>` is invalid because `Ts` is not expand.

Ts as also a pack. This is not defined by todays C++, hence you are expressing an idea you have, not a current reality. Other options exist of what the above could mean in the future. No option is the right one, it is a choice.

Right, this should be "Part 2" of this proposition.
 

> I think it should work like:
>
> return f(g(Xs<T, Ts...>{}...));

This would be unambiguous. Let's say Xs = {A,B,C}, Ts = {int,long}:

return f(g(Xs<T, Ts...>{}...));
return f(g(Xs<T, int, long>{}...));
return f(g(A<T, int, long>{}, B<T, int, long>{}, C<T, int, long>{}));

> return f(g(Xs<T, Ts...>{})...);

return f(g(Xs<T, Ts...>{}...));
return f(g(Xs<T, int, long>{}...));
return f(g(A<T, int, long>{}, g(B<T, int, long>{}), g(C<T, int, long>{})));

OK, how do you expand to:

return f(g(A<T, int>{}, A<T, long>{}), g(B<T, int>{}, B<T, long>{}), g(C<T, int>{}, C<T, long>{})); // (e1)

or

return f(g(A<T, int>{}, B<T, int>{}, C<T, int>{}), g(A<T, long>{}, B<T, long>{}, C<T, long>{})); // (e2)

or

return f(g(A<T, int>{}, A<T, long>{}, B<T, int>{}, B<T, long>{}, C<T, int>{}, C<T, long>{})); // (e3)

or

return f(g(A<T, int>{}, B<T, int>{}, C<T, int>{}, A<T, long>{}, B<T, long>{}, C<T, long>{})); // (e4)

You can't because you want have pack of packs (one is for `g` another is for `f` in case e1).
 

> As we see this is only one expansion. To have second one, I think we would need pack of pack to handle it:

And this is one of the things I'd like to avoid: Creating a pack of packs (and by induction a pack of packs of packs, etc.). It means creating new entities that don't exist today in a compiler's backend. The complexity to implement a proposal which does something like that is much higher and the risk of introducing new bugs makes it less likely that a proposal will be accepted.

Anyways, if you *really* want to do something like this I think it makes much more sense to have "partial expansion" something like:

return f(g(Xs<T, Ts>{}...[Ts])...); // generates (e1)
                      ^^^^^^^
possible syntax when the brackets don't contain an integral constant as in my proposal (1), but instead a pack's name (or a set of names) that occur in the expression. Each step requires all packs mentioned to have the same length, but separate steps may have different lengths. Only those packs are expanded in each step which are listed, the result is another pack of expressions which contain unexpanded packs which are then expanded by the second ellipsis. The above therefore yields (e1). Complete list:

return f(g(Xs<T, Ts>{}...[Ts])...); // generates (e1)
return f(g(Xs<T, Ts>{}...[Xs])...); // generates (e2)
return f(g(Xs<T, Ts>{}...[Ts]...)); // generates (e3)
return f(g(Xs<T, Ts>{}...[Xs]...)); // generates (e4)

But this is probably material for a second proposal and I haven't really thought it through - it's just an idea. Unless everyone thinks it's a brilliant idea and should be part of my proposal right now :-P

-- Daniel

Partial expand is interesting, this will easy creating packs of packs of packs of... :) Because this is new functionality (similar to my version but implicit) will have probably similar implementation complexity.
For now I think you should stick to current functionality of template packs.

From another mail:
 

It is an interesting idea, but I don't feel it is fundamental for handling packs. Instead it as applying the ellipsis to an expression which contain *no* packs. I think we are currently wrapping too many things into std::tuple when most of the time we'd like to handle packs directly. Unpacking a tuple is therefore, in my mind at least, trying to solve to wrong problem. It might still be worth as an additional feature to look into (maybe via an "operator..."), but it'll create a lot of headaches when an expression which is followed by an ellipsis contains both packs and object with an "operator...".

With templated packs, expanding tuples would be easy:

template<typename Tuple>
using Expand = typename... Tuple::pack; //of if tuple don't have we can crate helper template that will extract parameter pack from tuple

return f(forward<Expand<Tuple>>(tuple)...); //`Expand<Tuple>` is pack that can be expanded by `...`

[Arthur O'Dwyer]

On Sun, Mar 13, 2016 at 5:44 AM, Daniel Frey <d.f...@gmx.de> wrote:
>
> > On 06.03.2016, at 19:55, Arthur O'Dwyer <arthur....@gmail.com> wrote:
> >
> > > > In the part of my post that you snipped, I indicated that "generating an unexpanded
> > > > parameter pack" is a liability, not an advantage. You seemed to agree ("Good catch
> > > > ... I'll reconsider that one ...").  Maybe I should say is currently a liability. It would be
> > > > awesome to have first-class parameter packs, but I think there are deep problems
> > > > someone needs to solve before we can get them.
> > >
> > > When I said “Good catch” I meant that you spotted a problem that I haven’t realized
> > > before. It’s an interesting problem and, of course, it makes me reconsider given that
> > > you, IIUC, raised it in the context of my proposed language addition 3).
> >
> > It also applies to addition (2), because that also turns a non-pack expression into a (first-class) parameter pack.
>
> I think I now have a solution for the problem you spotted with (2) and (3), the new proposal (attached) solves this by not allowing in-situ creation or in-situ access to member packs. In short, I only propose two new features now:
>
> (1) Pack selection:
>
>   pattern...[I]

(0a) In the intro paragraph, "put in practice" should be "but in practice".

(0b) Same sentence: IMHO the paper would be much stronger if you gave a couple of examples of compilers "generating inferior code for several cases", in a footnote or appendix, going all the way down to assembly output and/or the output of "time clang++ -O2 test.cc", whichever is the more appropriate.

(0c) You say "Those primitives, together with the existing language, combine nicely...", but in fact you never show an example that uses both new features in combination. Either you're overstating the dependency between these two new features, or you're missing a motivating example. My personal opinion (so far) is that you've made a good case for feature #1, but I still mistrust feature #2, and I think it's feasible to standardize #1 without #2. How coupled are they, really?
That said, my newbie impression of the Committee's workings is that it makes sense to stuff both of your proposals into a single paper, as you're doing, so that you can get them both considered at once; and if the Committee says "we like #1 but not #2," then that's fine, just rip #2 out of the next revision of the paper. :)

(1a) You use the term "integral-constant" without definition. I assume you mean "constant-expression" where the constant-expression is constrained to be of integral type, i.e. an "integral constant expression" per N4567 5.20 [expr.const]/3.

"An integral constant expression is an expression of integral or unscoped enumeration type, implicitly converted to a prvalue, where the converted expression is a core constant expression. [ Note: Such expressions may be used as array bounds (8.3.4, 5.3.4), as bit-field lengths (9.6), as enumerator initializers if the underlying type is not fixed (7.2), and as alignments (7.6.2). — end note ]"

This is fine, but I think it pays to be very explicit (by using super-precise standard terminology) about what you want to happen in weird corner cases such as pattern ...[my_literal_type_with_constexpr_implicit_conversion_to_size_t()].

> (2) Pack declaration:
>
>    Syntax 1.1: typename... Ts; // empty pack
>    Syntax 1.2: typename... Ts = int, void, double&&;
>    Syntax 1.3: using typename... Ts = my_class<int>::types; // access pack member, this can not be templated
>    Syntax 2.1: std::size_t... Vs; // empty pack
>    Syntax 2.2: std::size_t... Vs = 3, 0, 1, 42;
>    Syntax 2.3: using std::size_t... Vs = my_class<int>::values;
>    Syntax 2.4: std::size_t... Vs = ... 4; // yields 0, 1, 2, 3


(2a) If I understand correctly, Syntax 1.1 makes Ts a pack of size zero, i.e., is_same_v<tuple<Ts...>, tuple<>>, is that right? That strikes me as a bit confusing, compared to picking a syntax that admits some explicitly "empty" syntactic element, such as typename... Ts<>;.

(2b) I don't understand Syntax 1.3 (described in your paper as using typename... Ts = class :: pack;). What kind of entity is pack in this context? Aren't you quietly introducing a new kind of C++ entity, i.e. what I've been calling a "naked pack" or "first-class pack"? I think I can smuggle my usual double-... into this feature.

template<int I> struct A {
    using typename... ints = int, int, int;  // this is Syntax 1.2, right?
};

template<int I> void f1() {
    using typename... Ts = A<I>::ints;  // this is Syntax 1.3, right?
    std::tuple<Ts...> t;
}

template<int I> void f2() {
    std::tuple<A<I>::ints ...> t;  // same as f1, just inlined
}

template<int... Is> void f3() {
    std::tuple<A<Is>::ints ... ...> t;  // boom?
}

(2c) I think you've successfully locked down the syntax for Pack Declarations to the point where there's no grammatical ambiguity, but it's too arbitrary for my personal taste. How come I can write

    int Alpha = 0, Beta = 1;

but I can't write

    int... Alphas = 0, 1, Betas = 2, 3;
    // or should it really be:
    int ...Alphas = 0, 1, ...Betas = 2, 3;

I mean, obviously that would introduce grammatical ambiguity; but by locking out the ambiguity, you've also lost some symmetry with the existing language — lost too much for my taste.

One counterargument to that point would be: C++ has lots of syntactic "freedoms" like that for purely historical reasons. Modern style would never declare two different variables on the same line. Declarations like typedef int i, *p, a[10]; wouldn't pass code review in 2016. Therefore, modern additions to the language quite properly lock down the syntax wherever they can. For example, when new-style type aliases were introduced in C++11, we got

    using i = int;
    using p = int *;
    using a = int[10];

but we pointedly did not get

    using i = int, p = int *, a = int[10];  // try it; it doesn't compile

So this new declaration syntax quite properly doesn't replicate the historical mistakes of C.

This counterargument would put the burden back on naysayers like me to come up with an example of improperly-locked-out grammar that doesn't look so much like a "historical mistake of C." :)

my $.02,
Arthur

[Matthew Woehlke]

On 2016-03-13 08:44, Daniel Frey wrote:
> (1) Pack selection:
>
> pattern...[I]

Doesn't this already have a defined meaning (`pattern<0>[I],
pattern<1>[I], ...`)?

Please explain why `[I]pack` is objectionable.

Too many people have conflicting ideas in this area. We should be
working *together* towards a single, unified vision, not operating in a
vacuum.

--
Matthew

[Barry Revzin]

No, that would be pattern[I]...

Putting the index first means that indexing into a parameter pack would have backwards syntax from indexing into anything else in the language. That's objectionable.

--
You received this message because you are subscribed to a topic in the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this topic, visit https://groups.google.com/a/isocpp.org/d/topic/std-proposals/ajLcDl8GbpA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to std-proposal...@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.
To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/nc7082%24p34%241%40ger.gmane.org.

[Matthew Woehlke]

(Please trim-post: http://www.palmyanoff.com/trimpost.htm)

On 2016-03-14 14:38, Barry Revzin wrote:

> On Mon, Mar 14, 2016, 1:35 PM Matthew Woehlke wrote:
>> On 2016-03-13 08:44, Daniel Frey wrote:
>>> (1) Pack selection:
>>>
>>> pattern...[I]
>>
>> Doesn't this already have a defined meaning (`pattern<0>[I],
>> pattern<1>[I], ...`)?
>>
>> Please explain why `[I]pack` is objectionable.
>>
>> Too many people have conflicting ideas in this area. We should be
>> working *together* towards a single, unified vision, not operating in a
>> vacuum.
>

> No, that would be pattern[I]...
>
> Putting the index first means that indexing into a parameter pack would
> have backwards syntax from indexing into anything else in the language.
> That's objectionable.

It also *works with non-packs* (and is less typing). *I* object to
adding two new syntaxes where one would do (see last comment in my
previous message).

Demonstrate a rational manner to incorporate slicing, tuple-unpacking,
and operation on non-packs with the proposed syntax and I will withdraw
the objection. Prefix indexing supports all of these features.

--
Matthew

[Daniel Frey]

> On 14.03.2016, at 19:52, Matthew Woehlke <mwoehlk...@gmail.com> wrote:
>
> On 2016-03-14 14:38, Barry Revzin wrote:
>> On Mon, Mar 14, 2016, 1:35 PM Matthew Woehlke wrote:
>>> On 2016-03-13 08:44, Daniel Frey wrote:
>>>> (1) Pack selection:
>>>>
>>>> pattern...[I]
>>>
>>> Doesn't this already have a defined meaning (`pattern<0>[I],
>>> pattern<1>[I], ...`)?
>>>
>>> Please explain why `[I]pack` is objectionable.
>>>
>>> Too many people have conflicting ideas in this area. We should be
>>> working *together* towards a single, unified vision, not operating in a
>>> vacuum.
>>
>> No, that would be pattern[I]...

Correct, and currently an ellipsis can never be followed by an opening square bracket so that syntax was available.

>> Putting the index first means that indexing into a parameter pack would
>> have backwards syntax from indexing into anything else in the language.
>> That's objectionable.

Another strong point, thanks Barry.

To me, [I]pack is also extremely unintuitive.

A technical problem I can see is, that it also raises problems with

// Is = 0, 2, 1, 3
// Ps = tuple<int, double, float, unsigned>, tuple<...>, tuple<...>, tuple<...>

[Is]Ps{}... // ??? now what?

If your prefix syntax may also index into non-packs, what does the above do? Expand Is first, then Ps? Ps first, then Is? Both in parallel?

> It also *works with non-packs* (and is less typing). *I* object to
> adding two new syntaxes where one would do (see last comment in my
> previous message).

What you see as an advantage, I consider a problem. The syntax doesn't help me to read my code. I chose the ellipsis to be part of all the things I propose to make sure you can always immediately spot places where a parameter pack is involved. I might just think that different things are important, different features should be provided or avoided and I have a different taste of syntax. I think that what I propose would be an important feature to improve compile-times, I think your additional features might be implementable as a library - but I haven't looked in detail. I am doing this in my (limited) spare time, so I can not look at everyone's favorite proposal. I looked at some, talked to some people and I am open to reasonable discussions and suggestions, but so far you haven't written anything that I found convincing.

> Demonstrate a rational manner to incorporate slicing, tuple-unpacking,
> and operation on non-packs with the proposed syntax and I will withdraw
> the objection. Prefix indexing supports all of these features.

No. I don't have to. You want those features, not me.

[Daniel Frey]

> On 14.03.2016, at 01:03, inkwizyt...@gmail.com wrote:
>
>> Ts as also a pack. This is not defined by todays C++, hence you are expressing an idea you have, not a current reality. Other options exist of what the above could mean in the future. No option is the right one, it is a choice.
>
> Right, this should be "Part 2" of this proposition.

> Partial expand is interesting, this will easy creating packs of packs of packs of... :) Because this is new functionality (similar to my version but implicit) will have probably similar implementation complexity.

That is probably true. Creating packs of packs [of packs...] would be a lot of work, especially if "pack" also stands for more complex expressions.

> For now I think you should stick to current functionality of template packs.

I agree :) The "Part 2" is too much for now and I don't see a conflict with my current proposal, so it could still be added later as it's own proposal.

> From another mail:
>
>> It is an interesting idea, but I don't feel it is fundamental for handling packs. Instead it as applying the ellipsis to an expression which contain *no* packs. I think we are currently wrapping too many things into std::tuple when most of the time we'd like to handle packs directly. Unpacking a tuple is therefore, in my mind at least, trying to solve to wrong problem. It might still be worth as an additional feature to look into (maybe via an "operator..."), but it'll create a lot of headaches when an expression which is followed by an ellipsis contains both packs and object with an "operator...".
>>
> With templated packs, expanding tuples would be easy:
>
> template<typename Tuple>
> using Expand = typename... Tuple::pack; //of if tuple don't have we can crate helper template that will extract parameter pack from tuple

Note that this is not part of what I'm proposing. The using syntax in my proposal for packs does explicitly not allow to be templated, otherwise the same problem with nested packs and the expansion will come up again.

-- Daniel

[Daniel Frey]

> On 14.03.2016, at 02:24, Arthur O'Dwyer <arthur....@gmail.com> wrote:
>
> On Sun, Mar 13, 2016 at 5:44 AM, Daniel Frey <d.f...@gmx.de> wrote:
> >
> > I think I now have a solution for the problem you spotted with (2) and (3), the new proposal (attached) solves this by not allowing in-situ creation or in-situ access to member packs. In short, I only propose two new features now:
> >
> > (1) Pack selection:
> >
> > pattern...[I]
>
> (0a) In the intro paragraph, "put in practice" should be "but in practice".

Fixed, thanks.

> (0b) Same sentence: IMHO the paper would be much stronger if you gave a couple of examples of compilers "generating inferior code for several cases", in a footnote or appendix, going all the way down to assembly output and/or the output of "time clang++ -O2 test.cc", whichever is the more appropriate.

I'll try to come up with better examples, providing hard numbers might be a problem since I can not provide the code with my proposal implemented.

> (0c) You say "Those primitives, together with the existing language, combine nicely...", but in fact you never show an example that uses both new features in combination. Either you're

I meant that they combine overall. In some places you need (1), in others (2). Applied in all the places where possible, the code becomes shorter, more direct and creates much fewer instances. I know I can only give you my word right now, but this is my experience from implementing tao::tuple, some code in the company as well as lots of answer I gave on StackOverflow. The same patterns come up time and again and it always feels indirect and inefficient to use the traditional work-arounds. tao::tuple showed me how much can be gained by avoiding unnecessary instantiations and I think my proposal has a lot of potential to further improve the situation.

For the run-time part: I've seen people that were writing Lua-wrappers in C++ to generate very inefficient code because the compiler was creating too many instantiations. There was nothing in theory that would prevent the compiler from producing optimal code, but the reality is that compilers have their limits. My experience also tells me that with more direct code, it becomes *much* more likely that the compiler will generate optimal code.

Finally, (2) is useful whenever you use make_integer_sequence just to pass it on into some second function/method where it is immediately used to deduce a parameter pack. In my database framework (not yet public), I also modify the indices, then pass them on again just to deduce the results. The scheme is so common that I would really like to express myself more directly with (2).

> (1a) You use the term "integral-constant" without definition. I assume you mean "constant-expression" where the constant-expression is constrained to be of integral type, i.e. an "integral constant expression" per N4567 5.20 [expr.const]/3.

You are probably right, I was mostly just using it as a descriptive placeholder. But when I do so and create the impression that this is taken from the standard, then it better be right (or I explain it). I will have to put some serious time into the proposals wording, rationale, ... - but my time is currently also quite limited.

> This is fine, but I think it pays to be very explicit (by using super-precise standard terminology) about what you want to happen in weird corner cases such as pattern ...[my_literal_type_with_constexpr_implicit_conversion_to_size_t()].

Intersting case, I can't tell you whether I'd like to support that right now :)

> > (2) Pack declaration:
> >
> > Syntax 1.1: typename... Ts; // empty pack
> > Syntax 1.2: typename... Ts = int, void, double&&;
> > Syntax 1.3: using typename... Ts = my_class<int>::types; // access pack member, this can not be templated
> > Syntax 2.1: std::size_t... Vs; // empty pack
> > Syntax 2.2: std::size_t... Vs = 3, 0, 1, 42;
> > Syntax 2.3: using std::size_t... Vs = my_class<int>::values;
> > Syntax 2.4: std::size_t... Vs = ... 4; // yields 0, 1, 2, 3
>
> (2a) If I understand correctly, Syntax 1.1 makes Ts a pack of size zero, i.e., is_same_v<tuple<Ts...>, tuple<>>, is that right? That strikes me as a bit confusing, compared to picking a syntax that admits some explicitly "empty" syntactic element, such as typename... Ts<>;.

Yes, I also though about that. It's just that I couldn't come up with something more convincing.

int... Ts = {}; // Ambiguous. Empty pack or a pack with one element initialized with {}? Even if we define it, users might confuse it.
int... Ts = <>;
int... Ts = ();
int... Ts = [];
int... Ts = default;

Nothing really convinces me. OTOH, I also entertained the idea of:

int... Is<>; // empty
int... Is<0, 1, 2, 3>; // 4 elements
int... Is<...4>; // generate 4 elements as above
int... Js<2*Is+1...>; // transform 4 elements

(likewise for types if applicable)

> (2b) I don't understand Syntax 1.3 (described in your paper as using typename... Ts = class :: pack;). What kind of entity is pack in this context? Aren't you quietly introducing a new kind of C++ entity, i.e. what I've been calling a "naked pack" or "first-class pack"? I think I can smuggle my usual double-... into this feature.

It is a pack define by the above syntax in the scope of some class/class template. But I do not propose that the syntax allows to be templated and I do not propose that any other kind of access is allowed. class::pack may only be accessed via the proposed syntax. In my newest proposal version, I changed 1.3 and 2.3 into a new point (3) separate from (2) and simplified it to:

using... Ps = class::pack;

as the type of the pack is already known, no need to repeat it. Any reference to class::pack anywhere else is an error. Hence:

> template<int I> void f1() {
> using typename... Ts = A<I>::ints; // this is Syntax 1.3, right?
> std::tuple<Ts...> t;
> }

Yes, OK.

> template<int I> void f2() {
> std::tuple<A<I>::ints ...> t; // same as f1, just inlined
> }

Already an error as described above.

> template<int... Is> void f3() {
> std::tuple<A<Is>::ints ... ...> t; // boom?
> }

I saw it coming and the above restriction makes sure this is avoided :)

> (2c) I think you've successfully locked down the syntax for Pack Declarations to the point where there's no grammatical ambiguity, but it's too arbitrary for my personal taste. How come I can write
>
> int Alpha = 0, Beta = 1;
>
> but I can't write
>
> int... Alphas = 0, 1, Betas = 2, 3;
> // or should it really be:
> int ...Alphas = 0, 1, ...Betas = 2, 3;
>
> I mean, obviously that would introduce grammatical ambiguity; but by locking out the ambiguity, you've also lost some symmetry with the existing language — lost too much for my taste.

I don't think I personally would like to support

int... Alphas = 0, 1, ...Betas = 2, 3;

but it is certainly possible.

> One counterargument to that point would be: C++ has lots of syntactic "freedoms" like that for purely historical reasons. Modern style would never declare two different variables on the same line. Declarations like typedef int i, *p, a[10]; wouldn't pass code review in 2016. Therefore, modern additions to the language quite properly lock down the syntax wherever they can. For example, when new-style type aliases were introduced in C++11, we got
>
> using i = int;
> using p = int *;
> using a = int[10];
>
> but we pointedly did not get
>
> using i = int, p = int *, a = int[10]; // try it; it doesn't compile
>
> So this new declaration syntax quite properly doesn't replicate the historical mistakes of C.

I agree that we should learn from past mistakes and only allow a single pack declaration per statement.

> This counterargument would put the burden back on naysayers like me to come up with an example of improperly-locked-out grammar that doesn't look so much like a "historical mistake of C." :)

:)

I don't want to exclude reasonable syntax or extensions for no good reason, but especially you have made me quite cautious - and for very good reasons. It is quite easy to screw up expressions and how they are expanded when you don't keep a close control over what is allowed and in which context. I hope my current proposal does not leak any of those problems into other parts of the language while still providing enough expressiveness to cover all the low-level cases that are currently only implementable with costly work-arounds.

-- Daniel

[Matthew Woehlke]

On 2016-03-14 16:37, Daniel Frey wrote:

> On 14.03.2016, at 19:52, Matthew Woehlke wrote:
>> On Mon, Mar 14, 2016, 1:35 PM Matthew Woehlke wrote:
>>> On 2016-03-13 08:44, Daniel Frey wrote:
>>>> (1) Pack selection:
>>>>
>>>> pattern...[I]
>>>
>>> Doesn't this already have a defined meaning (`pattern<0>[I],
>>> pattern<1>[I], ...`)?
>>>
>>> Please explain why `[I]pack` is objectionable.
>

> To me, [I]pack is also extremely unintuitive.
>
> A technical problem I can see is, that it also raises problems with
>
> // Is = 0, 2, 1, 3
> // Ps = tuple<int, double, float, unsigned>, tuple<...>, tuple<...>, tuple<...>
>
> [Is]Ps{}... // ??? now what?
>
> If your prefix syntax may also index into non-packs, what does the
> above do? Expand Is first, then Ps? Ps first, then Is? Both in parallel?

I'm not convinced that your syntax solves this (or, alternatively, that
mine can't solve it equally well). (I'm *also* not convinced that using
packs for indices is a good idea. That seems like a great way to create
illegible code. Daveed's pack generators would probably do this in a
much more readable manner.)

>> It also *works with non-packs* (and is less typing). *I* object to
>> adding two new syntaxes where one would do (see last comment in my
>> previous message).
>
> What you see as an advantage, I consider a problem. The syntax
> doesn't help me to read my code. I chose the ellipsis to be part of
> all the things I propose to make sure you can always immediately spot
> places where a parameter pack is involved.

I rather have the opposite reaction... if you're picking out a single
element, why does it *matter* if the input is a parameter pack or not?
In fact, I would go so far as to call this a misfeature. That would be
like saying we should not allow vector::operator[] because the input is
a vector rather than a C-style array.

> I might just think that different things are important, different
> features should be provided or avoided and I have a different taste
> of syntax.

I've heard a lot of people complaining about teachability... IMHO,
consistency is important.

> I think that what I propose would be an important feature to improve
> compile-times

Don't get me wrong; I fully agree that we *want* indexed access to
parameter packs. I only disagree with inventing Yet Another syntactic
construct to do so (that is incompatible with other goals; see later
comments).

> I think your additional features might be
> implementable as a library - but I haven't looked in detail.

I believe that generalized unpacking is not possible without *some* core
language change. There are several possibilities that don't provide the
feature directly, but would make a library solution possible. I still
feel strongly that the aforementioned operations are sufficiently common
to warrant a language feature, *especially* since prefix indexing can
consistently handle a large number of use cases.

Put differently: we need *some* core language change; why not pick the
one that gives us the most utility?

> I am doing this in my (limited) spare time, so I can not look at
> everyone's favorite proposal.

I understand that. That's why we have this forum :-).

> so far you haven't written anything that I found convincing.

We *probably* want generalized unpacking. We *probably* want indexed
access to parameter packs (your proposal notwithstanding, Daveed's
proposal and past proposals make me think that we do). We *may* want (as
a language feature) slicing of tuple-likes and/or parameter packs (we
almost certainly *do* want it, but once could argue if a library
solution is sufficient).

That being the case, let's try this... which do you prefer? (Let `tuple`
be tuple-like and `pack` be a parameter pack)

Option 1:

auto x = std::get<N>(tuple); // can 'N' be any constexpr?
foo(std::unpack(tuple));
foo(std::pp::<2,3>(std::unpack(tuple))); // unpack may be unneeded?
bar(pack...[N]);
bar(std::pp::mid<2,3>(<pack>)...);

Option 2:

auto x = [N]tuple; // definitely takes constexpr for 'N'
foo([:]tuple...);
foo([2:3]tuple...);
bar([N]pack);
bar([2:3]pack);

Note the consistency, and the simplicity of performing common
operations. (Also note that the second invocation of `foo` in Option 1
is less efficient compared to Option 2.)

This is what I mean by "most utility". Creating new syntax is hard and
results in more things for users to learn (see previous comment re:
teachability). This is *one* syntax that provides *many* features and
works the same in many contexts. Accordingly, it is easier to learn and
easier to specify.

>> Demonstrate a rational manner to incorporate slicing, tuple-unpacking,
>> and operation on non-packs with the proposed syntax and I will withdraw
>> the objection. Prefix indexing supports all of these features.
>
> No. I don't have to. You want those features, not me.

You lost a qualifier there... you "have to" *if* you want me to withdraw
the objection. Otherwise, I continue to object strongly to your proposed
syntax. Certainly, you have the right to ignore that objection. If you
do so, however, it is my intent to present a hostile proposal for prefix
notation.

I'd *prefer* to champion your proposal :-), but I can't conscionably do
over such an objection.

p.s. I wrote a paper that addresses this topic (among others) that I
hope/plan to submit for the next mailing. Here is a relevant excerpt:
"We also urge the committee to consider these issues and how such
features relate (or can be made to relate) to tuple-like objects in
order to maximize consistency of operations on both object types, and we
urge authors working on such proposals to do likewise."

--
Matthew

[Matthew Woehlke]

On 2016-03-13 08:44, Daniel Frey wrote:

> [updated proposal]

I took a closer look at the other parts of the proposal. I don't find
your examples compelling. Compilers are already implementing constexpr
generation of integer sequences as an intrinsic, and I think that
Daveed's pack generators proposal is far more compelling for such use
cases. For `std::apply`, an even better implementation is:

template <typename Func, typename Tuple>
std::apply(Func&& f, Tuple&& t)
{
return forward<F>(f)([:]forward<Tuple>(t)...);
}

...i.e. generalized unpacking (which also relegates use of std::apply to
esoteric uses only).

I suspect you'll need a much stronger rationale to get any traction in EWG.

--
Matthew

[Matthew Woehlke]

On 2016-03-14 17:26, Daniel Frey wrote:
> On 14.03.2016, at 02:24, Arthur O'Dwyer wrote:

>> On Sun, Mar 13, 2016 at 5:44 AM, Daniel Frey wrote:
>>> (2) Pack declaration:
>>>
>>> Syntax 1.1: typename... Ts; // empty pack
>>> Syntax 1.2: typename... Ts = int, void, double&&;
>>> Syntax 1.3: using typename... Ts = my_class<int>::types; // access pack member, this can not be templated
>>> Syntax 2.1: std::size_t... Vs; // empty pack
>>> Syntax 2.2: std::size_t... Vs = 3, 0, 1, 42;
>>> Syntax 2.3: using std::size_t... Vs = my_class<int>::values;
>>> Syntax 2.4: std::size_t... Vs = ... 4; // yields 0, 1, 2, 3
>>
>> (2a) If I understand correctly, Syntax 1.1 makes Ts a pack of size zero, i.e., is_same_v<tuple<Ts...>, tuple<>>, is that right? That strikes me as a bit confusing, compared to picking a syntax that admits some explicitly "empty" syntactic element, such as typename... Ts<>;.
>
> Yes, I also though about that. It's just that I couldn't come up with something more convincing.

This looks suspiciously like a proposal Mike Spertus is working on...
IIRC he uses <>'s to delineate parameter pack literals. Also, IIRC his
are not limited to integral types. That seems like an unnecessary
limitation.

(Sigh. And this is why I'm writing a paper encouraging people to *talk
to each other*. And I don't mean just Daniel. There is an unfortunate
disconnect between people posting on std-proposals and people actually
going to meetings. Fortunately, there are a few people¹ that do both,
but I think we'd have less reinventing of the wheel if more of the
people driving the language development paid attention to std-proposals,
and if more of the people throwing up any old thing here did their
homework as to what similar ideas have gone before, especially as
published papers. It would also help if more people didn't skip the
"float your idea past std-proposals" step. I get the feeling this tends
to be the more seasoned committee participants who feel they "don't need
the feedback", which is fair, but on the other hand I think they are to
some extent doing the community a disservice by making it harder for
this group to know what is happening.)

(¹ Ville, I'm looking at you, appreciatively :-).)

--
Matthew

[Daniel Frey]

> On 14.03.2016, at 22:50, Matthew Woehlke <mwoehlk...@gmail.com> wrote:
>
> On 2016-03-14 16:37, Daniel Frey wrote:
>> On 14.03.2016, at 19:52, Matthew Woehlke wrote:
>>> On Mon, Mar 14, 2016, 1:35 PM Matthew Woehlke wrote:
>>>> On 2016-03-13 08:44, Daniel Frey wrote:
>>>>> (1) Pack selection:
>>>>>
>>>>> pattern...[I]
>>>>

>>>> Please explain why `[I]pack` is objectionable.
>>
>> To me, [I]pack is also extremely unintuitive.
>>
>> A technical problem I can see is, that it also raises problems with
>>
>> // Is = 0, 2, 1, 3
>> // Ps = tuple<int, double, float, unsigned>, tuple<...>, tuple<...>, tuple<...>
>>
>> [Is]Ps{}... // ??? now what?
>>
>> If your prefix syntax may also index into non-packs, what does the
>> above do? Expand Is first, then Ps? Ps first, then Is? Both in parallel?
>
> I'm not convinced that your syntax solves this (or, alternatively, that
> mine can't solve it equally well). (I'm *also* not convinced that using

Please explain why my proposal would not solve this.

> packs for indices is a good idea. That seems like a great way to create
> illegible code. Daveed's pack generators would probably do this in a
> much more readable manner.)
>
>>> It also *works with non-packs* (and is less typing). *I* object to
>>> adding two new syntaxes where one would do (see last comment in my
>>> previous message).
>>
>> What you see as an advantage, I consider a problem. The syntax
>> doesn't help me to read my code. I chose the ellipsis to be part of
>> all the things I propose to make sure you can always immediately spot
>> places where a parameter pack is involved.
>
> I rather have the opposite reaction... if you're picking out a single
> element, why does it *matter* if the input is a parameter pack or not?
> In fact, I would go so far as to call this a misfeature. That would be
> like saying we should not allow vector::operator[] because the input is
> a vector rather than a C-style array.

Let me provide more rationale for the syntax I've chosen and why I don't like the syntax you proposed. My main concern is that I don't want any new syntax, because we already have it: postfix indexing. This is exactly what you should use to access elements from a tuple t: t[I]. The problem there is not the syntax, it's the semantics. Currently, the [I] can not be used to call std::get<I>(t). This is what should be made possible. Also, you may think about improving indexing to support slicing, etc. - but I don't see that prefix indexing is required or even desirable.

Now to explain my choice of syntax: Given that we already have a syntax to pick an element from something (indexing with [I]), I simply apply the existing syntax to a pack expansion to only select one element of the expansion (including not even expanding what is not needed). To you have a pattern (with pack(s)), expand it with an ellipsis and select an element with [I] - et voilà: pattern...[I]

Another problem with prefix indexing: The square brackets at the begin of an expression nowadays starts a lambda, with your proposal I need to first read the rest of the expression to figure out if I look at a lambda or prefix indexing. And have you considered all cases to be unambiguous? Think about a lambda having [], then some parameters with () and a body with {}. But the compile might see {} as just a block scope following a prefix indexed expression, as () may be part of an expression.

>> I might just think that different things are important, different
>> features should be provided or avoided and I have a different taste
>> of syntax.
>
> I've heard a lot of people complaining about teachability... IMHO,
> consistency is important.

I think that my above rationale will also help to teach pattern...[I], I fail to see why it should not be at least as teachable as your proposed syntax.

>> I think your additional features might be
>> implementable as a library - but I haven't looked in detail.
>
> I believe that generalized unpacking is not possible without *some* core
> language change. There are several possibilities that don't provide the
> feature directly, but would make a library solution possible. I still
> feel strongly that the aforementioned operations are sufficiently common
> to warrant a language feature, *especially* since prefix indexing can
> consistently handle a large number of use cases.
>
> Put differently: we need *some* core language change; why not pick the
> one that gives us the most utility?

Because I'd like to improve the language by using existing syntax and provide semantics first. Only the use-cases that can really not be handled like this might warrant a new syntax. It seems we agree on the basic idea, but I think I am not convinced that your syntax is needed for most of the use-cases you'd like to support.

> We *probably* want generalized unpacking. We *probably* want indexed
> access to parameter packs (your proposal notwithstanding, Daveed's
> proposal and past proposals make me think that we do). We *may* want (as
> a language feature) slicing of tuple-likes and/or parameter packs (we
> almost certainly *do* want it, but once could argue if a library
> solution is sufficient).

Some amount of slicing can be build on top of my proposal without further language changes. Example:

template<size_t Begin, size_t End, typename... Ts>
class slice
{
// add static_asserts as needed
size_t... Is = ...End-Begin;
public:
size_t... values = Is+Begin...;
};


template<size_t Begin, size_t End, typename... Ts>
class slice_types
{
using... Is = slice<Begin, End>::values;
public:
typename... types = Ts...[Is]...;
};

And now you can use:

typename... Ts = int, double, float, void, unsigned;
using... Us = slice_types<1, 3, Ts...>::types; // yields a pack with: double, float

(the actual functions and their signature should be improved, but just to show an example of how it would be possible)

> That being the case, let's try this... which do you prefer? (Let `tuple`
> be tuple-like and `pack` be a parameter pack)
>
> Option 1:
>
> auto x = std::get<N>(tuple); // can 'N' be any constexpr?
> foo(std::unpack(tuple));
> foo(std::pp::<2,3>(std::unpack(tuple))); // unpack may be unneeded?
> bar(pack...[N]);
> bar(std::pp::mid<2,3>(<pack>)...);
>
> Option 2:
>
> auto x = [N]tuple; // definitely takes constexpr for 'N'
> foo([:]tuple...);
> foo([2:3]tuple...);
> bar([N]pack);
> bar([2:3]pack);
>
> Note the consistency, and the simplicity of performing common
> operations. (Also note that the second invocation of `foo` in Option 1
> is less efficient compared to Option 2.)

I'd like:

auto x = tpl[N];
using... Is = tpl::indices;
foo(tpl[Is]...);
using... Js = std::slice<2, 3>::values; // in my above example 3 is End, but it might also be the Size - depends on how we define it - or offer both with different names :)
foo(tpl[Js]...);
bar(pck...[N]);
bar(pck...[Js]...);

Yes, you sometimes need a line more or two, but I don't think it's too bad.

> This is what I mean by "most utility". Creating new syntax is hard and
> results in more things for users to learn (see previous comment re:
> teachability). This is *one* syntax that provides *many* features and
> works the same in many contexts. Accordingly, it is easier to learn and
> easier to specify.

Which is why I'd like to use the existing syntax of postfix indexing and apply it in new contexts.

-- Daniel

[Matthew Woehlke]

On 2016-03-15 12:53, Daniel Frey wrote:

> On 14.03.2016, at 22:50, Matthew Woehlke wrote:
>> On 2016-03-14 16:37, Daniel Frey wrote:
>>> On Mon, Mar 14, 2016, 1:35 PM Matthew Woehlke wrote:
>>>> On 2016-03-13 08:44, Daniel Frey wrote:
>>>>> (1) Pack selection:
>>>>>
>>>>> pattern...[I]
>>>>
>>>> Please explain why `[I]pack` is objectionable.
>>>
>>> To me, [I]pack is also extremely unintuitive.
>>>
>>> A technical problem I can see is, that it also raises problems with
>>>
>>> // Is = 0, 2, 1, 3
>>> // Ps = tuple<int, double, float, unsigned>, tuple<...>, tuple<...>, tuple<...>
>>>
>>> [Is]Ps{}... // ??? now what?
>>>
>>> If your prefix syntax may also index into non-packs, what does the
>>> above do? Expand Is first, then Ps? Ps first, then Is? Both in parallel?

Okay, since you're forcing me to actually *look at* that question :-)...

My short answer to that is 'dealing with packs of packs is hard'. A
"true" solution needs some way to specify which expansions apply to
what. Otherwise, someone will always want to do other than what you wanted.

Possible answer: "greedy"; `[Is]` would bind to indexing the pack `Ps`,
which would leave `Is` as the pack to be expanded by `...`:

([0]Ps){}, ([2]Ps){}, ([1]Ps){}, ([3]Ps){}

More likely answer: short of a solution to the larger problem, packs are
not allowed as indices :-).

I tend to think this would be better addressed with a "library" solution
a la Daveed's pack generators. *Lots* more typing, but you'd be able to
decode the result without consulting the specification.

I think my idea handles fewer cases than yours¹, but in a simpler
manner, while Daveed's pack generators are more complicated than your
idea but also *MUCH* more expressive. Your proposal is somewhere in
between; not quite as simple as mine, but not nearly as expressive as
Daveed's.

(¹ For this paragraph, I am referring to your parts (2) and (3) and my
slicing, i.e. not pack indexing, which is comparable in both of our ideas.)

>> I'm not convinced that your syntax solves this (or, alternatively, that
>> mine can't solve it equally well). (I'm *also* not convinced that using
>
> Please explain why my proposal would not solve this.

http://permalink.gmane.org/gmane.comp.lang.c++.isocpp.proposals/25182

Hmm... okay, I guess both can define a "binding order". The not-solved
problem I was thinking about is what to do when you have packs of packs
and need to *control* the binding order. Per the above referenced
message, I don't believe either of us are attempting to address that.

Apologies for the confusion.

> Let me provide more rationale for the syntax I've chosen and why I
> don't like the syntax you proposed. My main concern is that I don't want
> any new syntax, because we already have it: postfix indexing.

Yes, we already have `a[4]` (and it's abominable cousin `4[a]` for bare
pointers). We *don't* have `pack...[N]`. That's *a new syntax*.

> This is exactly what you should use to access elements from a tuple
> t: t[I]. The problem there is not the syntax, it's the semantics.
> Currently, the [I] can not be used to call std::get<I>(t). This is
> what should be made possible.

Do you have a proposal for that? Because I don't. The meaning of that is
already well defined: it invokes operator[]. Which... will never work
for tuple (except perhaps to return a std::any), because what you *want*
is to perform the operation at *compile-time* rather than run-time.

Which... is a great reason for a new and distinctive syntax! It's also a
reason that *you gave* for wanting a new and distinctive syntax.

I submit that prefix index is more than suitable for this purpose; it
clearly indicates that something different from runtime indexing is
occurring, and it can be applied to both tuple-like and parameter pack
indexing (very similar compile-time operations for which consistency
makes sense). The `...[N]` syntax would look very strange applied to a
tuple-like, because `...` implies parameter packs.

> Also, you may think about improving indexing to support slicing, etc.
> - but I don't see that prefix indexing is required or even
> desirable.

Prefix indexing:

- Is visually distinct from postfix indexing (*your* reason!)
- Supports indexing, unpacking, and slicing
- Works consistently with both parameter packs and tuple-likes

Yours *definitely* fails the third point, and... okay, you do later show
the second, but it's unacceptably ugly. (More on both those points below.)

That's not to say that I'm wedded to prefix indexing as such, just that
it is quantitatively the best syntax of which I am presently aware. I'm
open to considering something else that can be shown to be equally
versatile or better.

> Another problem with prefix indexing: The square brackets at the
> begin of an expression nowadays starts a lambda, with your proposal I need to
> first read the rest of the expression to figure out if I look at a
> lambda or prefix indexing.

The first part is true, but hardly the end of the world. The second
part... depends on what you mean by "the rest of the expression". In the
typical case, you will know in a few characters, because either an
integer literal or a `:` will be present.

> And have you considered all cases to be unambiguous?

The worst pathological case I can come up with is:

constexpr auto i = 0;
auto x = [i]{ return 0; };
auto y = [i](tuple);

This still isn't a valid lambda, and I'm unaware of any cases that are
intractably ambiguous.

However, if it seems like a real problem, I'm happy to require that
anything that looks like a lambda up to the closing `]` must actually
*be* a lambda, since there is a trivial solution:

auto y = [(i)]tuple;

This is (AFAIK) unambiguously a compile-time index into `tuple` as of
the `(`.

Most cases are already not ambiguous because the expression is not a
valid capture even without such tricks (especially the common case of an
index that is an integer literal).

> Think about a lambda having [], then some parameters with
> () and a body with {}. But the compile might see {} as just a block
> scope following a prefix indexed expression, as () may be part of an
> expression.

Even if we don't implement the above restriction (in which case you know
by the `]` which you're allowed to have), I still don't see how you
would create an ambiguous case. Either the `]` is directly followed by
an expression of tuple-like type (i.e. no `(` or `{`, so definitely not
a lambda), or the part in `()`s is TTBOMK unambiguously either a
parameter list or a value-expression. (Also... the `{...}` can't be a
block scope; you'd be missing a `;`.)

It's possible I'm mistaken on both points, but I *have* thought about
this, and it will take a counter-example to change my mind.

>> I've heard a lot of people complaining about teachability... IMHO,
>> consistency is important.
>
> I think that my above rationale will also help to teach
> pattern...[I], I fail to see why it should not be at least as
> teachable as your proposed syntax.

Clarification: I don't disagree that either syntax is equally teachable
*for parameter packs*. The trouble is that your syntax either doesn't
work (see below) or is atrocious (i.e. if we used `tuple...[N]` instead)
for tuple-likes. So we either:

1. Have atrocious syntax for tuple-likes
2. Have different syntax for tuple-likes
3. Use something else for both

(2) is what I was referencing w.r.t. consistency and teachability.

>> We *probably* want generalized unpacking. We *probably* want indexed
>> access to parameter packs (your proposal notwithstanding, Daveed's
>> proposal and past proposals make me think that we do). We *may* want (as
>> a language feature) slicing of tuple-likes and/or parameter packs (we
>> almost certainly *do* want it, but once could argue if a library
>> solution is sufficient).
>
> Some amount of slicing can be build on top of my proposal without further language changes. Example:

> [snip example]

Now, *that* is a more compelling example than in your latest draft. Not
more compelling than Daveed's idea, but I'd still encourage you to add
it to your paper.

>> Which do you prefer? (Let `tuple` be tuple-like and `pack` be a

>> parameter pack)
>>
>> Option 1:
>>
>> auto x = std::get<N>(tuple); // can 'N' be any constexpr?
>> foo(std::unpack(tuple));
>> foo(std::pp::<2,3>(std::unpack(tuple))); // unpack may be unneeded?
>> bar(pack...[N]);
>> bar(std::pp::mid<2,3>(<pack>)...);
>>
>> Option 2:
>>
>> auto x = [N]tuple; // definitely takes constexpr for 'N'
>> foo([:]tuple...);
>> foo([2:3]tuple...);
>> bar([N]pack);
>> bar([2:3]pack);
>

> I'd like:
>
> auto x = tpl[N];

Impossible. This already has meaning: run-time indexing. You *might* be
able to twist the language sufficiently to make this mean compile time
indexing *for std::tuple*, but not for types that already have a
run-time `operator[]`, e.g. std::array (which is also tuple-like).

> using... Is = tpl::indices;
> foo(tpl[Is]...);

*Might* be possible, but the syntax is very, very scary (per previous
comment). It's also unacceptably much typing. My main use case is for
code like:

auto p = get_point();
use([:]p...);
// replaces:
// use(p[0], p[1], p[2])
// -or-
// use(p.x(), p.y(), p.z())

If I have to type that mess that you show, it defeats the purpose by
being significantly more complicated than what I was trying to avoid in
the first place. (Oh, and you didn't show how it would work on a
tuple-*like*. I expect you could use some form of library function, but
it would end up even worse than what you showed.)

Assuming I could inline the definition of `Is`, I *might* be able to use
your form... in a macro. Ugh. No, thanks.

--
Matthew

[inkwizyt...@gmail.com]

Today I think about this and I come to conclusion that you will not avoid this problems even when banning templated packs.
Consider template:

template<typename... T>
struct X
{
    using... Ts = T;
};

template<typename... Z>
auto Y()
{
    return F(X<Z>::Ts{}...); //exactly same situation like `Tp<Z>` where `Tp` is templated pack.
};

We have two solution:
a) ban usage of packs that depends on unexpanded packs e.g. `X<Z...>::Ts...` is valid but `X<Z>::Ts...` is invalid.
b) allow multi stage expansion of packs. `g(f(X<Z>::Ts...)...)` where first `...` is expanding `Z` and second set of dot expand rest. Partial step will be `g(f(X<Z1>::Ts, X<Z2>::Ts, X<Z3>::Ts)...)`.

I think you will not able to run from "packs of packs". I you allow that pack can be defined in structure than you can have something like that:

template<typename... T>
void f()
{
    g(T::pack...); //each of `T` have member pack definition `typename... pack = int, long;`
}

this will probably need `typename... T::pack` similar to `typename T::type` too.

[Arthur O'Dwyer]

On Wed, Mar 16, 2016 at 12:27 PM,  <inkwizyt...@gmail.com> wrote:
> On Monday, March 14, 2016 at 9:53:20 PM UTC+1, Daniel Frey wrote:
>> On 14.03.2016, at 01:03, inkwizyt...@gmail.com wrote:
>> >
>> > template<typename Tuple>
>> > using Expand = typename... Tuple::pack; //of if tuple don't have we can
>> > crate helper template that will extract parameter pack from tuple
>>
>> Note that this is not part of what I'm proposing. The using syntax in my
>> proposal for packs does explicitly not allow to be templated, otherwise the
>> same problem with nested packs and the expansion will come up again.
>

> Today I think about this and I come to conclusion that you will not avoid
> this problems even when banning templated packs.
> Consider template:
> template<typename... T>
> struct X
> {
>     using... Ts = T;
> };
>
> template<typename... Z>
> auto Y()
> {
>     return F(X<Z>::Ts{}...); //exactly same situation like `Tp<Z>` where
> `Tp` is templated pack.
> };

My understanding (from Daniel's reply to my most recent post) is that Daniel would ban expressions like the above.

He'd allow "naked packs" to be used only in using... declarations.

template<typename... T> struct X {

    using... Ts = T;  // OK

};

template<typename Y, typename... Z> auto Y() {

    using... MyTs1 = X<Y>::Ts;  // OK

    using... MyTs2 = X<Z>::Ts;  // Error: contains an unexpanded parameter pack

    using... MyTs3 = X<Z>::Ts...;  // Error: can't apply "..." to a pattern containing a naked parameter pack

    F(X<Y>::Ts{}...);  // Error: can't do anything with a naked parameter pack except make a pack alias for it

    F(MyTs1{}...);  // OK

}

I don't think there are any exploitable holes in it, if it's locked down to that extent. But I do think that Daniel needs to get some actual Standardese wording for what you can and can't do with naked packs, before it will be obvious whether there are no holes. His current phrase ("the using syntax... does not allow to be templated") is clearly not clear enough. :)

–Arthur

[Daniel Frey]

> On 16.03.2016, at 20:27, inkwizyt...@gmail.com wrote:
>
>> On Monday, March 14, 2016 at 9:53:20 PM UTC+1, Daniel Frey wrote:
>> > On 14.03.2016, at 01:03, inkwizyt...@gmail.com wrote:
>> >
>> >>
>> > With templated packs, expanding tuples would be easy:
>> >
>> > template<typename Tuple>
>> > using Expand = typename... Tuple::pack; //of if tuple don't have we can crate helper template that will extract parameter pack from tuple
>>
>> Note that this is not part of what I'm proposing. The using syntax in my proposal for packs does explicitly not allow to be templated, otherwise the same problem with nested packs and the expansion will come up again.
>>
>> -- Daniel
>>
>
> Today I think about this and I come to conclusion that you will not avoid this problems even when banning templated packs.
> Consider template:
> template<typename... T>
> struct X
> {
> using... Ts = T;
> };
>
> template<typename... Z>
> auto Y()
> {
> return F(X<Z>::Ts{}...); //exactly same situation like `Tp<Z>` where `Tp` is templated pack.
> };

No, this is not part of my proposal. You don't access ::Ts (a pack) without explicitly using syntax 1.3/2.3 and with that, it is not possible to form a pack of packs.

> I think you will not able to run from "packs of packs". I you allow that pack can be defined in structure than you can have something like that:

I allow packs being defined in structures, but I do not allow the "normal" access. This is what allows me to prevent to control pack creation and avoid having to deal with pack of packs, etc.

> template<typename... T>
> void f()
> {
> g(T::pack...); //each of `T` have member pack definition `typename... pack = int, long;`
> }

Again, not allowed in this way with my proposal. Yes, the next version of the document will put some emphasis on this :)

-- Daniel