[RFC] Generalized Unpacking and Parameter Pack Slicing

[Matthew Woehlke]

I've talked about this on numerous occasions... here at last is a formal
paper. Comments appreciated!

I'd also appreciate if anyone has good, concrete examples for why
extracting single elements from a pack is useful.

Short Summary:
--------------

Add a syntax for slicing parameter packs. Extend this syntax to also
work on product types. Gain ability to convert a product type to a
parameter pack in the process.

Longer summary:
---------------

This is a two-part proposal.

The first part proposes to add parameter pack slicing, of the form
`[I]pack` or `[L:U]pack`. The first syntax says to take the `I`th
element of the pack, and produces a single value. The second syntax says
to take the elements of the pack with indices from `L` (inclusive) until
`U` (exclusive), and produces a modified pack. Either bound may be
omitted, and negative indices may be used to indicate 'count from end'
semantics. Accordingly, `[:]pack` is a no-op. (Slicing has higher
precedence than fold expansion. Refer to the paper for more specifics.)

The second part proposes that parameter pack operations, particularly
slicing (as in the first part), but also (optionally) `sizeof...`,
should also work on product types as well as packs. Critically, this
means that the following would now be semantically equivalent:

std::apply(foo, pt);
foo([:]pt...);

Please refer to the paper for motivating examples.

Postscript:
-----------

Some links in this are known to be broken, as they are intended to refer
to my other draft proposal re: allowing types to be defined in return
type specifications. See also
https://groups.google.com/a/isocpp.org/forum/#!topic/std-proposals/y7dyHcD_BKA.

--
Matthew

[T. C.]

Only skimmed very quickly; I may have more comments later.

1. The first line of "Parameter Pack Slicing" doesn't seem to be formatted correctly: note all the backticks.

2. I don't buy the argument in "improving std::apply" about tuple_cat. You can just make a helper to create a tuple of references from a plain tuple and use tuple_cat on the result of that. Also, the code fragment in that section makes plenty of copies, ironically, and its broken syntax is also distracting.

[Matthew Woehlke]

On 2016-12-19 15:34, T. C. wrote:
> 1. The first line of "Parameter Pack Slicing" doesn't seem to be formatted
> correctly: note all the backticks.

It's correct, as can be seen in the HTML. Double ``s are markup for
literal text; the unbalanced `s appear literally in the output. Here's
the output (sans rich text):

... operator, `[` <slicing_expression> `]`, which ...
... one of <index> or [<index>] `:` [<index>], where ...

The intent is that ``-marked text denotes a literal token. (This is
somewhat important when []s are being thrown around both as literal
tokens and this-is-optional indicators... Unfortunately, it is not easy
in reST to have text that is both bold and literal.)

> 2. I don't buy the argument in "improving std::apply" about tuple_cat. You
> can just make a helper to create a tuple of references from a plain tuple
> and use tuple_cat on the result of that.

...but you still have to build that new tuple, and it sounds like your
invocation is still going to be non-trivially more complicated that the
postulated std::apply taking multiple tuples. (Anyway, this is not part
of the proposal, just an example of "stuff you can do with this feature".)

> Also, the code fragment in that section makes plenty of copies,
> ironically, and its broken syntax is also distracting.

Which syntax is broken?

--
Matthew

[T. C.]

On Monday, December 19, 2016 at 4:11:33 PM UTC-5, Matthew Woehlke wrote:

On 2016-12-19 15:34, T. C. wrote:
> 1. The first line of "Parameter Pack Slicing" doesn't seem to be formatted
> correctly: note all the backticks.

It's correct, as can be seen in the HTML. Double ``s are markup for
literal text; the unbalanced `s appear literally in the output. Here's
the output (sans rich text):

  ... operator, `[` <slicing_expression> `]`, which ...
  ... one of <index> or [<index>] `:` [<index>], where ...

The intent is that ``-marked text denotes a literal token. (This is
somewhat important when []s are being thrown around both as literal
tokens and this-is-optional indicators... Unfortunately, it is not easy
in reST to have text that is both bold and literal.)

I see (I thought that the backticks were remnants from the source). You might want to consider the way the standard's grammar indicates optional elements instead (subscript "opt")

 

> 2. I don't buy the argument in "improving std::apply" about tuple_cat. You
> can just make a helper to create a tuple of references from a plain tuple
> and use tuple_cat on the result of that.

...but you still have to build that new tuple, and it sounds like your
invocation is still going to be non-trivially more complicated that the
postulated std::apply taking multiple tuples. (Anyway, this is not part
of the proposal, just an example of "stuff you can do with this feature".)

The point is that extending std::apply to take multiple tuples doesn't require this feature. Such a thing can easily be implemented using the current tools:

template<class Tuple, size_t... Is>

auto t2r(Tuple&& tuple, std::index_sequence<Is...>) { 

    return std::forward_as_tuple(std::get<Is>(std::forward<Tuple>(tuple))...);

}

template<class Tuple>

auto t2r(Tuple&& tuple) { 

    return t2r(std::forward<Tuple>(tuple), std::make_index_sequence<std::tuple_size_v<std::decay_t<Tuple>>>()); 

}

template<class F, class...Tuples>

decltype(auto) multi_apply(F&&f, Tuples&&... tuples) {

    return apply(std::forward<F>(f), std::tuple_cat(t2r(std::forward<Tuples>(tuples))...)); 

}

with no copying involved. I simply don't think this is a good example to showcase your feature.

> Also, the code fragment in that section makes plenty of copies,
> ironically, and its broken syntax is also distracting.

Which syntax is broken?

Lack of return type on apply and apply_helper. Template parameter list after the declarator-id for the latter.

--
Matthew

[Vicente J. Botet Escriba]

Hi,

glad to see such a proposal.

I'm not against redefining sizeof for product types, but it will be worth mentioning the syntax is possible sizeof([:]t...) without the redefinition.

The <double, double> returned in

    <double, double> calculateTargetCoordinates();

is not a parameter pack but a pack type as defined in P0341. This is of course something new.

I like the [start:stop] operator on parameter packs and product types. I prefer [:] to the user defined operator... proposed in P0341.

Pack types as defined in P0341 should accept this slice operator (we shoild consider Pack types as product types). I'm not sure we will need that Pack types provide directly the expansion operator....

Have you considered strides [start:stop:step]? reverted slices [start:stop:-1]? (see https://docs.python.org/2.3/whatsnew/section-slices.html)

The main difference between the proposed slice operator and python slices is that the result is not a type, but a parameter pack, that can not be on the LHS of an assignment. Have you considered this possibility?

Vicente

[Nicol Bolas]

I think the syntax would be `sizeof...([:]t)`. After all, `[:]t` is a pack, and `sizeof...` returns the number of elements in the pack. No need to redefine anything.

 

The main difference between the proposed slice operator and python slices is that the result is not a type, but a parameter pack, that can not be on the LHS of an assignment. Have you considered this possibility?

That can easily be handled by unpacking it into a tuple of references, which can be assigned to.

[Arthur O'Dwyer]

IIUC (having only skimmed Matthew's paper but debated re packs in this forum often enough ;)) the syntax

template<typename... Ts>
void f(Ts&&... ts)
{
    (... , ([1:3]ts = 42));  // pack on LHS of assignment, plus a C++17 fold-expression with operator-comma
}

int main()
{
    static int v0,v1,v2,v3,v4;
    f(v0,v1,v2,v3,v4);
    assert(v0==0 && v1==42 && v2==42 && v3==0 && v4==0);
}

would indeed fall out naturally. But I believe that what Vicente was referring to is Python's syntax for assigning into a slice of a list:

>>> x = [0,1,2,3,4]

>>> x[1:3] = [42]

>>> print x

[0, 42, 3, 4]

>>> print 'The above is actually syntactic sugar for:'

The above is actually syntactic sugar for:

>>> x.__setitem__(slice(1,3), [42])

>>> print x

[0, 42, 4]

That is, in Python, an assignment whose LHS is a slice-expression actually changes the length of the complete list being sliced! I found some details on Python's implementation of slices-on-the-LHS here. AFAIK, it's purely a syntactic-sugar construct — there is no way to "capture" a slice-expression and later use it to mess with the original complete list, any more than there's a way to "capture" or "perfect-forward" a heterogeneous braced-initializer-list in C++. Notice that slice is a built-in type (read: standard library type) in Python, but it's just a tag type for holding a couple of ints; the actual magic here is happening in x's __setitem__ method (read: a special member function).

Notice also that Python does not support slice-on-the-LHS for all sliceable types. Python's tuples, which are heterogeneous and immutable, do not support slice-on-LHS or even index-on-LHS.

>>> x = (0,1,2,3,4)

>>> x[1:3]

(1, 2)

>>> x[0] = 42

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

TypeError: 'tuple' object does not support item assignment

>>> x[1:3] = (42,)

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

TypeError: 'tuple' object does not support item assignment

Notice that slicing a tuple gives me a tuple, whereas slicing a list gave me a list. This is determined entirely by the behavior of x.__getitem__(slice(1,3)) (read: another special member function). Python has a crazy rich set of special member functions: basically any member function whose name begins and ends with double-underscore is available to assign special meaning to, and they've been taking advantage of that.

HTH,

Arthur

[Barry Revzin]

IIUC (having only skimmed Matthew's paper but debated re packs in this forum often enough ;)) the syntax

template<typename... Ts>
void f(Ts&&... ts)
{
    (... , ([1:3]ts = 42));  // pack on LHS of assignment, plus a C++17 fold-expression with operator-comma
}

int main()
{
    static int v0,v1,v2,v3,v4;
    f(v0,v1,v2,v3,v4);
    assert(v0==0 && v1==42 && v2==42 && v3==0 && v4==0);
}

would indeed fall out naturally. But I believe that what Vicente was referring to is Python's syntax for assigning into a slice of a list:

Given what slicing means in Python and what how unpacking typically works in C++, I think the more natural falling out is:

[1:3]ts#0 = 42;
[1:3]ts#1 = 42;
...
[1:3]ts#N-1 = 42;

not

ts#1 = 42;
ts#2 = 42;

Also, the paper has these two examples:

auto manhattan_distance d = std::abs([:]v) + ...;
auto dot = [:]v * ...;

Aside from the extra "d", what are these supposed to do? Both Manhattan distance and dot product are functions of two vectors, so I don't entirely understand this example. 

[Nicol Bolas]

On Tuesday, December 20, 2016 at 9:47:36 AM UTC-5, Barry Revzin wrote:

Also, the paper has these two examples:

auto manhattan_distance d = std::abs([:]v) + ...;
auto dot = [:]v * ...;

Aside from the extra "d", what are these supposed to do? Both Manhattan distance and dot product are functions of two vectors, so I don't entirely understand this example.

Well, here's what these would actually look like:

auto manhattan_distance = std::abs([:]v1 - [:]v2) + ...
auto dot = ([:]v1 * [:]v2) + ...;

[Matthew Woehlke]

On 2016-12-19 19:16, Vicente J. Botet Escriba wrote:
> I'm not against redefining sizeof for product types, but it will be
> worth mentioning the syntax is possible sizeof([:]t...) without the
> redefinition.

Pretty sure you meant `sizeof...([:]t)`, but yeah, it's a fair point. My
main argument for having `sizeof...(t)` for product type `t` is for
symmetry with prefix `[]` working on either packs or product types. If
folks don't find the consistency argument compelling, I have no problem
dropping that part of the proposal.

> The <double, double> returned in
>
> <double, double> calculateTargetCoordinates();
>
> is not a parameter pack but a pack type as defined in P0341. This is of
> course something new.

...but the expression `calculateTargetCoordinates()` results in a
parameter pack, does it not? That, anyway, is how I read P0341...
(Admittedly, I find P0341... vague and sometimes confusing.)

> Pack types as defined in P0341 should accept this slice operator (we
> shoild consider Pack types as product types). I'm not sure we will need
> that Pack types provide directly the expansion operator....

I hope that this will obviate much of P0341 :-). As I've said in various
places, while the features of P0341 are useful, I don't feel that the
approach in P0341 is desirable; it invents way too much IMHO in the way
of new syntax, new language constructs, etc..

> Have you considered strides [start:stop:step]? reverted slices
> [start:stop:-1]?

I hadn't specifically, but I'll refer you to "Future Direction". My
general impression has been that the more complicated the feature
becomes, the more people are inclined to dislike it. I think pack
generators or other such approaches would be a better solution for this
sort of thing.

> The main difference between the proposed slice operator and python
> slices is that the result is not a type, but a parameter pack, that can
> not be on the LHS of an assignment. Have you considered this possibility?

Doesn't std::tie solve that?

--
Matthew

[Matthew Woehlke]

On 2016-12-20 09:47, Barry Revzin wrote:

> On 2016-12-19 23:34, Arthur O'Dwyer wrote:
>> I believe that what Vicente was referring to is Python's syntax for
>> assigning into a slice of a list:
>>

>> >>> x = [0,1,2,3,4]
>> >>> x[1:3] = [42]
>> >>> print x
>> [0, 42, 3, 4]

Eek... yeah, I don't see that happening in C++. Not with parameter
packs, anyway.

> Given what slicing means in Python and what how unpacking typically works
> in C++, I think the more natural falling out is:
>
> [1:3]ts#0 = 42;
> [1:3]ts#1 = 42;
> ...
> [1:3]ts#N-1 = 42;

I'm... not quite sure what you're saying here?

At any rate, I would say that slices should work no different from other
parameter packs. I'm not sure if this is allowed or not (I think no?):

pack = 42;

...but such an operation on a slice, e.g. `[L:U]pt = 42`, should be
treated the same way.

> Also, the paper has these two examples:

> [snip]

Arf... thanks for catching those, and to Nicol for providing corrected
versions!

--
Matthew

[Vicente J. Botet Escriba]

Oh yes. This is what I have in mind.

 

The main difference between the proposed slice operator and python slices is that the result is not a type, but a parameter pack, that can not be on the LHS of an assignment. Have you considered this possibility?

That can easily be handled by unpacking it into a tuple of references, which can be assigned to.

I know this possibility. I'm asking because the compiler should surely be able to do better. Vicente

[Bengt Gustafsson]

I really like this feature for packs. For "product types" such as tuples I'm more reserved. It seems that converting the product type object to a value pack is the principal operation, and that any slicing should take place after the product type object has been converted to a pack. Using the "nop slice" for this purpose feels contrived.

With this in mind I think a postfix [] would be easier to understand. We already have both compile time (array bounds) and runtime (indexing and array-new) uses of [].

If we get constexpr function parameters I would say that a operator[](constexpr size_t) on tuple would be close at hand. With this proposal and that [0]tup and tup[0] would do the same thing. That's why I think the cleanest would be to limit this proposal to packs and use postfix indexing.

With the advent of [:] for slicing a future direction could be to allow this type of indexing to be overloaded for user defined types (with and without constexpr parameters). This would enable slicing of a tuple to a shorter tuple using tup[1:3] type of syntax.

Right now I don't have any idea on what operator to use for converting a product type to a pack, but maybe that was what the operator...() overloading was about?

[Nicol Bolas]

On Tuesday, December 20, 2016 at 6:52:22 PM UTC-5, Bengt Gustafsson wrote:

I really like this feature for packs. For "product types" such as tuples I'm more reserved.

But without the tuple part of it, it basically isn't a feature anymore. So you can subselect from a parameter pack. That's not nearly a common enough operation to be worth adding syntax for.
 

It seems that converting the product type object to a value pack is the principal operation, and that any slicing should take place after the product type object has been converted to a pack. Using the "nop slice" for this purpose feels contrived.

OK, let's look at this logically. You want syntax A which means "convert tuple to pack". And you want syntax B which means "sub-select pack from pack".

Now, let's say I want to sub-select from a tuple. Well, that means I have to apply syntax A then syntax B. But we know that you cannot sub-select a pack from a tuple, so applying syntax B to a tuple would be a compile error. So there would be no harm in making syntax B implicitly perform A when applied to a tuple. After all, you're just doing what the user obviously wants, right?

At which point, you can simply declare that syntax A is merely a degenerate form of syntax B. And thus, you've re-derived this proposal.

If the use wants to subselect a tuple from a tuple, they can do that: `tuple([1:3]tup...)`. This also allows them to choose how to build that tuple. The above would perform a copy of the elements. `forward_as_tuple([1:3]tup...)` would get references from them.

[Vicente J. Botet Escriba]

Le 20/12/2016 à 16:58, Matthew Woehlke a écrit :
> On 2016-12-19 19:16, Vicente J. Botet Escriba wrote:
>> I'm not against redefining sizeof for product types, but it will be
>> worth mentioning the syntax is possible sizeof([:]t...) without the
>> redefinition.
> Pretty sure you meant `sizeof...([:]t)`, but yeah, it's a fair point. My

Right.

> main argument for having `sizeof...(t)` for product type `t` is for
> symmetry with prefix `[]` working on either packs or product types. If
> folks don't find the consistency argument compelling, I have no problem
> dropping that part of the proposal.

If you consider [:] as a way to convert a product type to a parameter
pack, slicing could become an additional operation.

>> The <double, double> returned in
>>
>> <double, double> calculateTargetCoordinates();
>>
>> is not a parameter pack but a pack type as defined in P0341. This is of
>> course something new.
> ...but the expression `calculateTargetCoordinates()` results in a
> parameter pack, does it not? That, anyway, is how I read P0341...
> (Admittedly, I find P0341... vague and sometimes confusing.)

I agree that the paper could be vague, but I know the intention is to be
a type not a pack.

>
>> Pack types as defined in P0341 should accept this slice operator (we
>> shoild consider Pack types as product types). I'm not sure we will need
>> that Pack types provide directly the expansion operator....
> I hope that this will obviate much of P0341 :-). As I've said in various
> places, while the features of P0341 are useful, I don't feel that the
> approach in P0341 is desirable; it invents way too much IMHO in the way
> of new syntax, new language constructs, etc..

One of the major things P0341 proposes IMO is a type for aggregate
initializes. <T, U> is the type for the expression { t, u }. This is
something we are missing now.
The other is the ability to use packs in other locations, to help define
a simple_tuple type.
I don't think there is to much syntax changes in P0341

>
>> Have you considered strides [start:stop:step]? reverted slices
>> [start:stop:-1]?
> I hadn't specifically, but I'll refer you to "Future Direction". My
> general impression has been that the more complicated the feature
> becomes, the more people are inclined to dislike it. I think pack
> generators or other such approaches would be a better solution for this
> sort of thing.

Maybe you are right, but exploring the global problem is worth the effort.
It is not clear to me yet what pack generators are. I'll re-read the
paper but a clarification here is welcome.

>
>> The main difference between the proposed slice operator and python
>> slices is that the result is not a type, but a parameter pack, that can
>> not be on the LHS of an assignment. Have you considered this possibility?
> Doesn't std::tie solve that?
>

Yes, std::tie will work. If what we want is

[1,3]pp = {a,b},

with tie, we need

tie(([1,3]pp)...) = {a,b};

Do we need to improve the syntactic sugar of each construction?

With [:] and std::slice<I,J,K>(ts..) we can replace

[I, J]pp // your proposed slice

with

[:]slice<I,,J>(pp...) // your proposed product type to parameter
pack + a library function returning a product type view

It is more verbose but it is extensible without changing the language
too much, we need only a product type to parameter pack conversion

tie((slice<1,3>(pp...)) = {a,b};

Do we really need [I, J]p when we could have a more verbose library
solution? Possibly if this could improve the compiler performances. I
believe that proposal should provide more examples where slicing helps
the user to define functions that the compiler could improve the compile
time performances.

What I'm saying is that we need to see what are the minimal operations
we need and what are the syntactic sugar ones.

I believe that we need:

* a way to transform a product type into a parameter pack and

[:]pt -> pp // your proposal

* transform a parameter pack into a product type.
<pp...> -> pt // P0341

While I would like to have a direct product type expansion, I don't see
how P0341 UDT expansion works together with parameter pack expansions

pt... // UDT product type expansion P0341

Then see what are the operations we need on parameter packs and product
types.

If we introduce a slice construct in the language for parameter packs,
why it cannot be on the lhs of a expression? This is why I was
considering that slicing should return a type and not a parameter pack

Do we need a language slicing construct for product types? Should
slicing on a product type result on a parameter pack or a product type view?

Selection of the nth element of a parameter pack and a product type
[N]pp
versus
nth<N>(pp...)

[N]pt
versus
product_type::get<N>(pt)

IMO, the proposal should show the language and the library solutions.

Vicente

P.S. pt stands for product type and pp for parameter pack

[Matthew Woehlke]

On 2016-12-21 02:49, Vicente J. Botet Escriba wrote:
> If you consider [:] as a way to convert a product type to a parameter
> pack, slicing could become an additional operation.

...but then, what does it mean when applied to a pack?

I've structured the proposal as it is to try to minimize the new syntax
introduced. Particularly, there are exactly two new prefix operators, of
the form `[I]` and `[L:U]`, where `L` and/or `U` are optional (`I` is
required). Both work on *either* parameter packs or product types, and
both *produce*, respectively, a single value and a parameter pack,
regardless of the input. That `[:]` happens to also convert a product
type to a parameter pack is a "happy" (albeit intentional) side-effect.

> Le 20/12/2016 à 16:58, Matthew Woehlke a écrit :
>> ...but the expression `calculateTargetCoordinates()` results in a
>> parameter pack, does it not? That, anyway, is how I read P0341...
>> (Admittedly, I find P0341... vague and sometimes confusing.)
>
> I agree that the paper could be vague, but I know the intention is to be
> a type not a pack.

If it's a type, how is it different (besides syntax sugar) from a
compound type as we already have them?

> One of the major things P0341 proposes IMO is a type for aggregate
> initializes. <T, U> is the type for the expression { t, u }. This is
> something we are missing now.

Again, though, how is that different from a tuple? Or maybe it should be
spelled std::initializer_list<T, U>? At least, it seems to me it would
make more sense to give it a type name, even if it has a little of
initializer_list's "magic". Giving it *no* name creates a new, very
distinctive syntax that doesn't seem justified to me.

> The other is the ability to use packs in other locations, to help define
> a simple_tuple type.

For the record, *those* parts I'm generally not opposed to, at least not
on principle. (I'm not too sure about the exact syntax being proposed,
especially as said syntax is not clear, but the features seem okay in
principle.)

>> I hadn't specifically, but I'll refer you to "Future Direction". My
>> general impression has been that the more complicated the feature
>> becomes, the more people are inclined to dislike it. I think pack
>> generators or other such approaches would be a better solution for this
>> sort of thing.
>
> Maybe you are right, but exploring the global problem is worth the effort.
> It is not clear to me yet what pack generators are. I'll re-read the
> paper but a clarification here is welcome.

Talk to Daveed; it's his paper. I really like it, but for some reason he
has never published it (I wish he would!).

The basic idea, though, is that it implements compile-time functions a
la Python's generators (https://wiki.python.org/moin/Generators), which
produce parameter packs. Basically, these let you do all sorts of
complicated things like zipping, stepping, reversing, producing
sequences, and so forth.

> With [:] and std::slice<I,J,K>(ts..) we can replace
>
> [I, J]pp // your proposed slice
>
> with
>
> [:]slice<I,,J>(pp...) // your proposed product type to parameter
> pack + a library function returning a product type view

...but this won't work on parameter packs without first converting them
to product types. At *best* it's a lot more verbose. And it doesn't
solve the problem of extracting single elements from parameter packs.

These are all things that the compiler can do *very easily*, but are
much harder if they have to be expressed as library functions. This is
why the proposal is presented as "let's solve parameter pack problems
(that are very hard to solve with library only approaches)" and "hey,
look how much more useful this is if we allow it to work on product
types as well".

> What I'm saying is that we need to see what are the minimal operations
> we need and what are the syntactic sugar ones.

The *bare minimum* is a) element extraction from a parameter pack, and
b) product type to tuple conversion. One might argue whether *parameter
pack* slicing is strictly needed, although it's a whole heck of a lot
messier and more verbose to try to do without it.

The thing is, with the presented syntax, once you have those, you *also*
get all the rest "for free" (without adding additional new syntax).

> While I would like to have a direct product type expansion, I don't see
> how P0341 UDT expansion works together with parameter pack expansions
>
> pt... // UDT product type expansion P0341

So... this just scares me; you're overloading the fold expression
operator to produce something that could be used in a fold expression. I
don't even want to *think* about the parsing headaches this is going to
create, not just for compilers, but for anyone trying to read the code.

> If we introduce a slice construct in the language for parameter packs,
> why it cannot be on the lhs of a expression?

What does it even *mean* to assign to a parameter pack? Parameter packs,
today, are not mutable. The proposed feature would introduce a limited
form of "mutating", although really it is creating a new pack from a
"view" of an existing pack. Think about Python tuples; a parameter pack
is a lot like a tuple. You can slice it, but you can't assign to it.

--
Matthew

[Matthew Woehlke]

On 2016-12-20 18:52, Bengt Gustafsson wrote:
> I really like this feature for packs. For "product types" such as tuples
> I'm more reserved. It seems that converting the product type object to a
> value pack is the principal operation, and that any slicing should take
> place after the product type object has been converted to a pack. Using the
> "nop slice" for this purpose feels contrived.

It's a fair cop, but I've gone that route because it results in maximum
usefulness out of minimum syntax. Having a totally separate operation
for pt->pp conversion requires an additional syntax, vs. overloading the
pack slicing syntax, and then if you *do* want to slice a product type,
you'd have to use both, rather than just the one syntax. (See also
Nicol's reply.)

FWIW, it's not useless:

template <typename T, size_t N>
std::array<T, N-1> normalize(std::array<T, N> a)
{
return {[:-1]a / [-1]a...};

}

> If we get constexpr function parameters I would say that a
> operator[](constexpr size_t) on tuple would be close at hand.

Will it be provided by default (opt-out) for simple aggregates? Would we
*want* it for simple aggregates? It might be confusing to use "array
indexing" on something that isn't an array. Using a different (prefix)
but familiar (operator[]) addresses this.

--
Matthew

[Barry Revzin]

> Given what slicing means in Python and what how unpacking typically works
> in C++, I think the more natural falling out is:
>
> [1:3]ts#0 = 42;
> [1:3]ts#1 = 42;
> ...
> [1:3]ts#N-1 = 42;

I'm... not quite sure what you're saying here?

You're proposing that the [x:y] syntax can apply to both packs and product types. The question is, in Arthur's example of:

(... , ([1:3]ts = 42));

How do you know whether this is

(a) Pick out two elements of the parameter pack ts... first, THEN assign 42 to both of those. That is, ts#1 = 42; ts#2 = 42;

(b) For each element in the parameter pack, that is a product type, do [1:3]elem = 42; That is, [1:3]ts#0 = 42; [1:3]ts#1 = 42; ...; [1:3]ts#N-1 = 42;

Or more generally, I have a function:

template <class F, class... Ts>
void invoke(F f, Ts... ts) { ... }

And I want to implement invoke() such that it:

(a) calls f on each item in the slice [1:3] of the parameter pack ts. That is, given invoke(f, a, b, c, d), I want it to do f(b); f(c);

(b) calls f on the sub-elements [1:3] of each element of the parameter pack ts. That is, given invoke(f, a, b, c, d), I want it do f([1:3]a...); f([1:3]b...); f([1:3]c...); f([1:3]d...);

How do I implement those two function calls?

[Matthew Woehlke]

On 2016-12-21 11:54, Barry Revzin wrote:
> You're proposing that the [x:y] syntax can apply to both packs and product
> types. The question is, in Arthur's example of:
>
> (... , ([1:3]ts = 42));
>
> How do you know whether this is
> (a) Pick out two elements of the parameter pack ts... first, THEN assign 42
> to both of those. That is, ts#1 = 42; ts#2 = 42;
> (b) For each element in the parameter pack, that is a product type, do [1:3]elem
> = 42; That is, [1:3]ts#0 = 42; [1:3]ts#1 = 42; ...; [1:3]ts#N-1 = 42;

The first; `[:]` has precedence over `...` (the proposal clearly states
this), therefore its operand is `ts`.

This delves partly into the problem of choosing how to apply fold
expressions in the context of multiple levels of packs. That's been
discussed before, and it's... hard. Hard enough that I'm not attempting
to solve that problem. Probably you need to use a lambda or other helper
function for such cases.

> Or more generally, I have a function:
>
> template <class F, class... Ts>
> void invoke(F f, Ts... ts) { ... }
>
> And I want to implement invoke() such that it:
> (a) calls f on each item in the slice [1:3] of the parameter pack ts. That
> is, given invoke(f, a, b, c, d), I want it to do f(b); f(c);

(void)(f([1:3]ts), ...);

> (b) calls f on the sub-elements [1:3] of each element of the parameter pack
> ts. That is, given invoke(f, a, b, c, d), I want it do f([1:3]a...);
> f([1:3]b...); f([1:3]c...); f([1:3]d...);

auto l = [](auto f, auto t) { f([1:3]t...); }
(void)(l(ts), ...);

For similar reasons, the proposed feature does not entirely obviate
std::apply.

--
Matthew

[Matthew Woehlke]

On 2016-12-21 02:49, Vicente J. Botet Escriba wrote:
>>> The main difference between the proposed slice operator and python
>>> slices is that the result is not a type, but a parameter pack, that can
>>> not be on the LHS of an assignment. Have you considered this
>>> possibility?
>>
>> Doesn't std::tie solve that?
>
> Yes, std::tie will work. If what we want is
>
> [1,3]pp = {a,b},
>
> with tie, we need
>
> tie(([1,3]pp)...) = {a,b};
>
> Do we need to improve the syntactic sugar of each construction?

Oddly enough, a coworker was asking a similar question recently,
although for the example he gave, what he really wants is `auto const
[foo, bar] = ...`.

I was looking at this again, and I wonder how often you really want to
write something like the above, and why you wouldn't instead write it:

[1]pp = a;
[3]pp = b;

If this is really a useful feature, probably it should work on any
parameter pack, sliced or otherwise. That would make it orthogonal to
this proposal.

--
Matthew

[Matthew Woehlke]

On 2016-12-19 17:26, T. C. wrote:
> On Monday, December 19, 2016 at 4:11:33 PM UTC-5, Matthew Woehlke wrote:
>> The intent is that ``-marked text denotes a literal token. (This is
>> somewhat important when []s are being thrown around both as literal
>> tokens and this-is-optional indicators... Unfortunately, it is not easy
>> in reST to have text that is both bold and literal.)
>
> I see (I thought that the backticks were remnants from the source). You
> might want to consider the way the standard's grammar indicates optional
> elements instead (subscript "opt")

Thanks. I've tweaked this a bunch. The current version does not have
backticks, uses bold for stuff that appears literally, italic for
placeholders, and subscript "opt" for optional stuff. (Yay for custom
roles and custom CSS!)

>> Which syntax is broken?
>
> Lack of return type on apply and apply_helper. Template parameter list
> after the declarator-id for the latter.

Should be fixed in the latest version.

--
Matthew

[Matthew Woehlke]

I'm posting an update, since there have been non-trivial changes to the
examples and some discussion questions, partly thanks to some in depth
discussions with Bengt Gustafsson.

Again, feedback appreciated, especially on the examples and discussion
points. (The original summary, for context, is quoted below.)

Matthew

[Barry Revzin]

On Wednesday, February 1, 2017 at 6:00:54 PM UTC-6, Matthew Woehlke wrote:

I'm posting an update, since there have been non-trivial changes to the
examples and some discussion questions, partly thanks to some in depth
discussions with Bengt Gustafsson.

Again, feedback appreciated, especially on the examples and discussion
points. (The original summary, for context, is quoted below.)

Matthew

Just wondering. Who is the "we"? The paper is all in first-person plural, but you're the only author listed. 

[Matthew Woehlke]

On 2017-02-01 20:43, Barry Revzin wrote:
> Just wondering. Who is the "we"? The paper is all in first-person plural,
> but you're the only author listed.

It's stylistic; authorial "we" is more formal than first-person
singular, and feels more appropriate to proposal writing. (At least IMHO.)

See also https://en.wikipedia.org/wiki/We.

--
Matthew

[mihailn...@gmail.com]

Hello, the syntax has some major problems

std::tuple<int, int, int> t;

[0][1]t; 

Should I read left to right or right to left? 

I know the answer - right to left. However - we are back in the reverse invocation of get<0>(get<1>())!

Even worse:

[0]t[1];

Is it a tuple element, which is an array or an array element which is a tuple?!?

Sure, there is an answer, based on operator precedence, but one must know and evaluate the precedence in his head to be able to read the code.

Ironically even get<> is better in that case!

As I wrote an year ago*, the answer is to use the dot to enter subobjects-access, not unlike it was used for decades now. 

t.[0].[1]; //< we are digging in! The natural way. Orthogonal to s.field.field;

t.[0][1]; //< sure the 0 element of a tuple(-like) is an array

The moment we press the dot we signal that "we open the type".

After we open it we can index it, slice it or expand it without ambiguity 

  

  t.[0];

  t.[1:];

  f(t...); //< note, just ..., no ambiguity. equivalent of t.... 

  As for an advanced example:

    template<template<class...> class... Ts> //< sizeof...(Ts) is N
    void f()
    {
      func(Ts. ...) ...; //< Param is expanded tuple, func called N times. Note, a dot to prevent ambiguity! 

      func(Ts.) ...; //< Expand each tuple element as a single param and call func N times. Note size must match!

      func(Ts) ...; //< Param is the unexpanded tuple, call func N times with it

      func(Ts...); //< Param is N unexpanded tuples, call func once
    }

Note the subscription syntax is only needed when we need a subscript - when we index or slice.

Of course, we can do both

f(t.[1].[1:4]...); //< dig by index, slice, expand!

An empty slice will only mean copy all, it is not used for pack access (we use dot for that)

auto t2 = t.[:]; //< We open the door an pour out everything. Same as t2 = t1;

*https://groups.google.com/a/isocpp.org/forum/#!searchin/std-proposals/sugar/std-proposals/n-44mMGbCv0/nMeD8_YzBwAJ

[mihailn...@gmail.com]

Actually there is more.

What if we, in the future, consider the subscript operator could take multiple arguments? This can definitely be useful!

However with the current syntax this would be very confusing:

 [0][1]t; 

now becomes 

[1, 0]t;

Not good.

And multiple arguments can be useful.

template<auto... I>

auto adv = t.[I...];

adv is a tuple(-like) created not from a single slice or an index, but a mix indices and slices!

The I can be either an size_ or a pair<size_t, size_t> or a pair<optional<size_t>, optional<size_t>> IFF the index is the last or first in the pack.

This way we have complete freedom over what we select  - we can cherry-pick some elements and consume multiple, separate groups of others.

 

[bastie...@gmail.com]

On Friday, December 29, 2017 at 10:45:06 AM UTC+1, mihailn...@gmail.com wrote:

On Thursday, December 28, 2017 at 2:37:21 PM UTC+2, mihailn...@gmail.com wrote:

Hello, the syntax has some major problems

std::tuple<int, int, int> t;

[0][1]t; 

It's actually simple to translate back to english: "the 0th element of the 1st element of t". Nothing difficult about it.  

Should I read left to right or right to left? 

I know the answer - right to left. However - we are back in the reverse invocation of get<0>(get<1>())!

Even worse:

[0]t[1];

No it's actually really clear. It means [0](t[1]). A prefix operator applies to everything afterwards, it not hard feat to understand.

The same thing applies to prefix*.

std::string *ptr;

*ptr.size(); //error did you mean '->'

(*ptr).size(); //solution

Is it a tuple element, which is an array or an array element which is a tuple?!?

Sure, there is an answer, based on operator precedence, but one must know and evaluate the precedence in his head to be able to read the code.

Ironically even get<> is better in that case!

No operator precedence knowledge is needed, just knowing that [] is a prefix operator is enough and it's clearly visible.

',' separated operator[] will never make consensus due to its impact on legacy code, that's no argument.

adv is a tuple(-like) created not from a single slice or an index, but a mix indices and slices!

The I can be either an size_ or a pair<size_t, size_t> or a pair<optional<size_t>, optional<size_t>> IFF the index is the last or first in the pack.

This way we have complete freedom over what we select  - we can cherry-pick some elements and consume multiple, separate groups of others.

Your ".[]" doesn't highlight the difference between runtime and compile-time and would be confusing for ruby users.

Also the interesting side effect of having a prefix index operator is that all the arguments packs contained in the expression receive the operator:

//with your version
std::forward<Args.[I]>(args.[I]);
//with the prefix version
[I]std::forward<Args>(args)

With a more complex example:

template<class... Args>
void print_reverse(Args&&...args)
{
 if constexpr (sizeof...(Args) > 0)
 {
   auto indexs = std::make_index_sequence<sizeof...(Args) - 1>{};
   std::cout << [-1]indexs;
   ((std::cout << [sizeof...(Args) - [:]indexs - 1]std::forward<Args>(args)), ...);
 }
 std::cout << std::endl;
}

With your syntax would become:

template<class... Args>
void print_reverse(Args&&...args)
{
 if constexpr (sizeof...(Args) > 0)
 {
   auto indexs = std::make_index_sequence<sizeof...(Args) - 1>{};
   std::cout << indexs.[-1];
   ((std::cout << std::forward<Args.[sizeof...(Args) - indexs.[:] - 1]>(args.[sizeof...(Args) - indexs.[:] - 1])), ...);
 }
}