[perl #118089] Feature: list slices to the end

[Ed Avis]

# New Ticket Created by "Ed Avis"
# Please include the string: [perl #118089]
# in the subject line of all future correspondence about this issue.
# <URL: https://rt.perl.org:443/rt3/Ticket/Display.html?id=118089 >


This is a bug report for perl from e...@waniasset.com,
generated with the help of perlbug 1.39 running under perl 5.16.3.


-----------------------------------------------------------------
[Please describe your issue here]

Sometimes you want to slice a list from some index to the end.
For example, to get the whole list apart from the first element.
If it is in a named array variable then you can say

my @end_part = @a[ 1 .. $#a ];

But this becomes more difficult when the list is an intermediate
expression. It would be handy to have a syntax for slicing to
the end of a list, like this:

my @end_part = @a[ 1 .. ];

Here only the left part of the '..' is given and the end of the
slice is implicitly the end of the list. For symmetry, it would
also be useful to have

my @start_part = @a[ .. 5 ];

If overloading the .. operator in this way is too yucky, you
could instead consider borrowing the list slice notation
from Python. That would be a bigger addition to the language
but would fix an un-C-like off-by-one feeling with the current
way of specifying an inclusive range. (List slices in Python
are specified as start:end giving the range start..end-1.)

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
category=core
severity=wishlist
---
Site configuration information for perl 5.16.3:

Configured by Red Hat, Inc. at Thu Apr 11 09:48:29 UTC 2013.

Summary of my perl5 (revision 5 version 16 subversion 3) configuration:

Platform:
osname=linux, osvers=2.6.32-358.2.1.el6.x86_64, archname=x86_64-linux-thread-multi
uname='linux buildvm-08.phx2.fedoraproject.org 2.6.32-358.2.1.el6.x86_64 #1 smp wed feb 20 12:17:37 est 2013 x86_64 x86_64 x86_64 gnulinux '
config_args='-des -Doptimize=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Dccdlflags=-Wl,--enable-new-dtags -Dlddlflags=-shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wl,-z,relro -DDEBUGGING=-g -Dversion=5.16.3 -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dprefix=/usr -Dvendorprefix=/usr -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl5 -Dsitearch=/usr/local/lib64/perl5 -Dprivlib=/usr/share/perl5 -Dvendorlib=/usr/share/perl5/vendor_perl -Darchlib=/usr/lib64/perl5 -Dvendorarch=/usr/lib64/perl5/vendor_perl -Darchname=x86_64-linux-thread-multi -Dlibpth=/usr/local/lib64 /lib64 /usr/lib64 -Duseshrplib -Dusethreads -Duseithreads -Dusedtrace=/usr/bin/dtrace -Duselargefiles -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinp
erl=n -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr -Dd_gethostent_r_proto -Ud_endhostent_r_proto -Ud_sethostent_r_proto -Ud_endprotoent_r_proto -Ud_setprotoent_r_proto -Ud_endservent_r_proto -Ud_setservent_r_proto -Dscriptdir=/usr/bin -Dusesitecustomize'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic',
cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
ccversion='', gccversion='4.7.2 20121109 (Red Hat 4.7.2-8)', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='gcc', ldflags =' -fstack-protector'
libpth=/usr/local/lib64 /lib64 /usr/lib64
libs=-lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc -lgdbm_compat
perllibs=-lresolv -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
libc=, so=so, useshrplib=true, libperl=libperl.so
gnulibc_version='2.16'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,--enable-new-dtags -Wl,-rpath,/usr/lib64/perl5/CORE'
cccdlflags='-fPIC', lddlflags='-shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wl,-z,relro '

Locally applied patches:


---
@INC for perl 5.16.3:
/home/eda/lib/perl5/
/usr/local/lib64/perl5
/usr/local/share/perl5
/usr/lib64/perl5/vendor_perl
/usr/share/perl5/vendor_perl
/usr/lib64/perl5
/usr/share/perl5
.

---
Environment for perl 5.16.3:
HOME=/home/eda
LANG=en_GB.UTF-8
LANGUAGE (unset)
LC_COLLATE=C
LC_CTYPE=en_GB.UTF-8
LC_MESSAGES=en_GB.UTF-8
LC_MONETARY=en_GB.UTF-8
LC_NUMERIC=en_GB.UTF-8
LC_TIME=en_GB.UTF-8
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/home/eda/bin:/home/eda/bin:/usr/local/bin:/usr/bin:/sbin:/usr/sbin:/sbin:/usr/sbin
PERL5LIB=/home/eda/lib/perl5/
PERL_BADLANG (unset)
SHELL=/bin/bash

--
Ed Avis <e...@waniasset.com>


______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________

[John Imrie]

As we have removed $# can we use it for the end of the current list now.

my @end_part = @a[ 1 .. $# ];

Thus making $# the number of elements in the current list.

John

[Father Chrysostomos via RT]

On Sun Jun 09 06:55:59 2013, perl...@rjbs.manxome.org wrote:
>
> For the record, I am open to a patch to provide this kind of change,
> and am not
> particular as to which of the suggestions is used, as long as the work
> is
> sound.

Of the two suggestions, I think $# is best. If list and array slices
could record the length in an interpreter variable localised to the
brackets on the right, $# could be a magical variable accessing that.

That would make all three of these work:

# Omit the first element
@a = (gimme_list())[1..$#];
@a = @b[1..$#];
sub end { $# }
@a = (gimme_list())[1..end];

Do we want that?

As for the [1..] suggestion, would that be a postfix .. operator applied
to the 1? Or would ..] be a special way to end the subscript? I.e.,
should [5.., 2] be allowed or not? And if we can access this number,
why restrict it to ranges?

--

Father Chrysostomos


---
via perlbug: queue: perl5 status: open
https://rt.perl.org:443/rt3/Ticket/Display.html?id=118089

[Dr.Ruud]

On 21/05/2013 16:15, Ed Avis wrote:

> Sometimes you want to slice a list from some index to the end.

> [...] It would be handy to have a syntax for slicing to

> the end of a list, like this:
>
> my @end_part = @a[ 1 .. ];
>
> Here only the left part of the '..' is given and the end of the
> slice is implicitly the end of the list. For symmetry, it would
> also be useful to have
>
> my @start_part = @a[ .. 5 ];

Or support negative indexes inside slicers:

my @end_part = @a[ 1 .. _(-1) ];

(the _() to express topical run time action)


Or a topical $# alias, like $_#.

my @end_part = @a[ 1 .. $_# ];

--
Ruud

[David Nicol]

On Sun, Jul 7, 2013 at 1:51 AM, Dr.Ruud <rvtol+...@isolution.nl> wrote:

Or support negative indexes inside slicers:

  my @end_part = @a[ 1 .. _(-1) ];

(the _() to express topical run time action)

since negative numbers already represent counting from the end of an array, in both scalar element accesses and C<splice>, it seems to me that altering the dot-dot operator's meaning inside slice specifiers to allow negative numbers would be the most compatible new syntax, as well as simplifying slicing to almost-the-end.

a magical $# clearly allows the most flexibility -- there's no way to express "give me the right half of this array" without a temporary variable, otherwise. We don't have that currently, and it's beyond the scope of the feature request under discussion.

     my @right_half = @a[ (0.5 + $# / 2) .. $# ];

is that really an improvement over

     my @right_half = @a[ (0.5 + $#a / 2) .. $#a ];

?

having a magic variable that is localized in this new way will be tricky, and will slow everything down a little as it will have to get localized and set often.

Allowing a malformed range operation, like C< 3 .. -2 >, within slicing operations, to resolve the right side with respect to the array being sliced, might be simpler to implement, by making a new "range in slice" operator that is slightly different from the normal range operator in that it handles a negative on the right side and a positive on the left side by referring to a generally inaccessible $^ReferenceToArrayBeingSliced variable ---- or better yet, by promoting a range operator appearing in an array slice to a method call against the array, rather than a operator that produces indices.

if the range operator when appearing inside a slice stops being just an expression that yields indices and instead becomes part of the slicing syntax itself, fancy stuff like lazy arrays that produce lazy slices, or, non-numeric ranges that slice associative arrays without producing millions of garbage empty slots, will be possible.

   my %Ns = %Phonebook{ 'N' .. 'NZZZ' };  # might resolve to one SELECT call

dln

[Eric Brine]

On Sat, Jul 6, 2013 at 8:36 PM, Father Chrysostomos via RT <perlbug-...@perl.org> wrote:

That would make all three of these work:

@a = (gimme_list())[1..$#];

Small note: Right now, the index expression is evaluated before the list expression, so you'd also have to change the (undocumented) operand evaluation order for list slices.

[Dr.Ruud]

On 08/07/2013 07:00, David Nicol wrote:
> On Sun, Jul 7, 2013 at 1:51 AM, Dr.Ruud <rvtol+...@isolution.nl
> <mailto:rvtol+...@isolution.nl>> wrote:

>> Or support negative indexes inside slicers:
>>
>> my @end_part = @a[ 1 .. _(-1) ];
>>
>> (the _() to express topical run time action)
>
>
> since negative numbers already represent counting from the end of an
> array, in both scalar element accesses and C<splice>, it seems to me
> that altering the dot-dot operator's meaning inside slice specifiers to
> allow negative numbers would be the most compatible new syntax, as well
> as simplifying slicing to almost-the-end.

Well sure, but it 'calls' _(), so I meant no actual negative number
there, but a topical interpretation of it.

--
Ruud

[Father Chrysostomos via RT]

Of course, I had forgotten about that. I should know, considering that
I implemented this function:

$ ./perl -Ilib -MO=Concise,CORE::__PACKAGE__ -e0
CORE::__PACKAGE__:
7 <1> leavesublv[1 ref] K/REFC,1 ->(end)
- <@> lineseq KP ->7
1 <$> coreargs(PV "__PACKAGE__") v ->2
6 <2> lslice K/2 ->7
- <1> ex-list lK ->4
2 <0> pushmark s ->3
3 <$> const[IV 0] s ->4
- <1> ex-list lK ->6
4 <0> pushmark s ->5
5 <0> caller[t1] l ->6
-e syntax OK

I was not sure whether I was entirely comfortable with the proposed
feature. Now I think I am against it, since it really can’t work.
Shall I mark this ticket as rejected?

[David Nicol]

On Mon, Jul 8, 2013 at 3:45 PM, Father Chrysostomos via RT <perlbug-...@perl.org> wrote:

 Now I think I am against it, since it really can’t work.

it isn't possible to recognize range-in-slice-indices at parse time and turn it into an internal operation that takes three arguments, the container, the left range point, and the right range point? Would that require more than one token of lookahead? A fix-up pass that replaces slices when the index is a range operation would also work, but that's messy

this is what's in perly.y 

        |       ary '[' expr ']'                     /* array slice */

                        { $$ = op_prepend_elem(OP_ASLICE,

                                newOP(OP_PUSHMARK, 0),

                                    newLISTOP(OP_ASLICE, 0,

                                        list($3),

                                        ref($1, OP_ASLICE)));

                          TOKEN_GETMAD($2,$$,'[');

                          TOKEN_GETMAD($4,$$,']');

                        }

so I'm talking about adding something like this

        |       ary '[' term DOTDOT term  ']'                     /* array range slice */

                        { $$ = op_prepend_elem(OP_ASLICE_RANGE,

                                newOP(OP_PUSHMARK, 0),

                                    newLISTOP(OP_ASLICE_RANGE, 0,

                                        scalar($3), scalar($5), 

                                        ref($1, OP_ASLICE_RANGE)));

                          TOKEN_GETMAD($2,$$,'[');

                          TOKEN_GETMAD($4,$$,DOTDOT);

                          TOKEN_GETMAD($6,$$,']');

                        }

before that, and creating OP_ASLICE_RANGE.

which might mean making a second "expr" that doesn't include DOTDOT in its termbinop, if yacc is greedy about what it puts into the expr slot, or otherwise handling DOTDOT differently.

[John Imrie]

Isn't _() all ready a valid sub call, being as $_, @_ and _ (the default
file handle) exist, why not &_ as well. In which case someone may have
already defined 'sub _'

John

[Father Chrysostomos via RT]

If that is to apply to list slices as well, then it would mean slight
changes in syntax, such as (func1())[1..func2()] vs
(func1())[(1..func2()}] would result in the functions being called in a
different order.

I don’t feel comfortable with that at all.

[David Nicol]

On Mon, Jul 8, 2013 at 8:00 PM, Father Chrysostomos via RT <perlbug-...@perl.org> wrote:

If that is to apply to list slices as well, then it would mean slight
changes in syntax, such as (func1())[1..func2()] vs
(func1())[(1..func2()}] would result in the functions being called in a
different order.

I don’t feel comfortable with that at all.

The change in undocumented evaluation order is required for a variable-based solution, as the magic last-index variable must be bound before the left and right terms are evaluated, which is a change from how it is now:

$ perl -wle 'sub A { print "A"; 0..9 } sub L { print "L"; 2 } sub R { print "R"; 6 } print ((A)[L .. R])'

print (...) interpreted as function at -e line 1.

L

R

A

23456

$ 

With a new operator solution, there is no reason for the evaluation order to change. 

I don't know all the tools available within the blocks emitted by perly.y.

Are you saying that we currently lack a framework for invoking a new three-argument operator, after evaluating the left term, the right term, and the array expression, in that order, and furthermore, that crafting the required infrastructure would be impossible?

Surely you aren't.

[Aristotle Pagaltzis]

* John Imrie <j.im...@virginmedia.com> [2013-07-09 00:45]:

> In which case someone may have already defined 'sub _'

You think so? :-)

--
*AUTOLOAD=*_;sub _{s/::([^:]*)$/print$1,(",$\/"," ")[defined wantarray]/e;chop;$_}
&Just->another->Perl->hack;
#Aristotle Pagaltzis // <http://plasmasturm.org/>

[Dr.Ruud]

Nobody is married to the _() looks. It is about the topicalizing.

For a scalar topicalizer, maybe pick $_().
And/or for last-index, consider $#().

--
Ruud

[Father Chrysostomos via RT]

OK, I misunderstood. Yes, if the .. is part of the slice operator
(instead of a separate range), it can execute its arguments in any
order. In which case the syntax would be:

( a )[ b..c, d.., e..]

but no parentheses allowed around the [d..].

[Father Chrysostomos via RT]

On Tue Jul 09 01:44:34 2013, e...@waniasset.com wrote:
> Adding a variant of DOTDOT that works only inside list slices is
> tricky for the reasons Fr C. suggests.
> If a new operator is to be added then I do suggest looking closely at
> Python's list slices,
> which in my opinion are superior to the Perl way of specifying an
> inclusive range.

I don’t know Python. Could you summarise its slices?

[Father Chrysostomos via RT]

But low-precedence operators like ‘or’ and ‘and’ complicate things
further. If .. is part of the list slice syntax and perly.y treats it
that way, then it will have lower precedence than ‘or’, as in:

( a )[$b or $c..]

Having the lexer and parser treat postfix .. as a special operator that
makes op.c croak at compile time if it is not a direct child of a slice
might work.

But then branch folding will change the meaning in cases like this:

( a )[$0 or $2..] # compile-time error
( b )[ 0 or $2..] # ok

[David Nicol]

TLDR: David Nicol openly and cheerfully struggles with concepts of yacc grammars, eventually concluding that the approach he has been championing will require too much lookahead. Then he suggests a post-parse fixup approach that would be done at parse-time when range operations appear near the top of slice index expressions.

On Tue, Jul 9, 2013 at 10:31 AM, Father Chrysostomos via RT <perlbug-...@perl.org> wrote:

>
> OK, I misunderstood.  Yes, if the .. is part of the slice operator
> (instead of a separate range), it can execute its arguments in any
> order.  In which case the syntax would be:
>
> ( a )[ b..c, d.., e..]
>
> but no parentheses allowed around the [d..].
>

Python's range operator implies the starting 0 or the ending -1. Perl's doesn't. 

 

But low-precedence operators like ‘or’ and ‘and’ complicate things
further.  If .. is part of the list slice syntax and perly.y treats it
that way, then it will have lower precedence than ‘or’, as in:

( a )[$b or $c..]

I'm imagining the rangey slice only happening when the rangey slice syntax is triggered, otherwise, DOTDOT is the same binary infix operator it has always been. Implying the starting 0 and the ending -1 could also happen then too, okay -- would perly.y see '$b or $c' as a term? Also, the precedence shift wouldn't be a problem, just something to document

 

Having the lexer and parser treat postfix .. as a special operator that
makes op.c croak at compile time if it is not a direct child of a slice
might work.

But then branch folding will change the meaning in cases like this:

( a )[$0 or $2..] # compile-time error
( b )[ 0 or $2..] # ok

if a new clause in perly.y will work, then so will three new clauses, one for [ term .. term ] and one for [ term .. ] and one for [ .. term ].  The second would imply a -1 for the end, the third would imply a 0 for the start.

How do we guard against [ a .. b , c .. d ] triggering the new syntax? It should get parsed as an expression yielding the index list like normal, not a range-slice between a and scalar (b, c .. d). Maybe commas could create series of ranges, but that may beyond the expressive power of Bison. Can Bison create lists and map over them? Using an action in a recursively matched rule?

       range_slice_expression:  term DOTDOT term

                                             { push the terms onto the temporary ranges list }

                                             | term DOTDOT

                                             { push the term, and -1, onto the temp.r.l. }

                                             | DOTDOT term

                                             { push 0, and the term, onto the temp.r.l. }

                                             | expr

                                             { push something that indicates that this one

                                               is not a range

                                             }

       ;

 

       range_slice_expression: range_slice_expression ',' range_slice_expression

       ary '[' range_slice_expression ']' 

       { build a list based on the temporaries set up during matching of range_expression;

         clear the list of ranges

       }

    

I don't think it can be made to work without more than one token of lookahead:

     @array[ !($. % 5) .. !($. % 7) ? 0 .. 4 : -5 .. -1 ]

should match '[' expr ']' and not match '[' term DOTDOT term ']'

because the token after the second notted modulo term is a hook, not a right square bracket or a comma

but to realize this, the parser must maintain ambiguity for longer than one token. So that won't work.

And that leaves, examining the emitted expression tree of the slice indices to see if it has any range operators in it at the top level -- or, at the next-to-top if the top are commas.

 

[Aaron Priven]

Hmm.

@a[qs/new slice syntax, whatever it is/]

Then the syntax doesn't get evaluated until the slicing actually happens.

(Personally I like inclusive ranges.)

-- 

Aaron Priven, aa...@priven.com, www.priven.com/aaron

-- 

Aaron Priven, aa...@priven.com, www.priven.com/aaron

[Tom Christiansen]

> TLDR: David Nicol openly and cheerfully struggles with concepts of

What a curious way to spell "ABSTRACT"! :)

> yacc grammars, eventually concluding that the approach he has
> been championing will require too much lookahead. Then he
> suggests a post-parse fixup approach that would be done at parse-
> time when range operations appear near the top of slice index
> expressions.

--tom

[Father Chrysostomos via RT]

We seem to be talking past each other somewhat.... But this
conversation is fascinating nonetheless.

On Tue Jul 09 12:55:04 2013, david...@gmail.com wrote:
> I'm imagining the rangey slice only happening when the rangey slice syntax
> is triggered, otherwise, DOTDOT is the same binary infix operator it has
> always been.
>
> Implying the starting 0 and the ending -1 could also happen
> then too, okay -- would perly.y see '$b or $c' as a term? Also, the
> precedence shift wouldn't be a problem, just something to document

I think it *would* be a problem. It would be too surprising. (I know,
you seem to find nothing surprising. :-)

>
> > Having the lexer and parser treat postfix .. as a special operator that
> > makes op.c croak at compile time if it is not a direct child of a slice
> > might work.
> >
> > But then branch folding will change the meaning in cases like this:
> >
> > ( a )[$0 or $2..] # compile-time error
> > ( b )[ 0 or $2..] # ok
>
>
>
> if a new clause in perly.y will work, then so will three new clauses, one
> for [ term .. term ] and one for [ term .. ] and one for [ .. term ].

I don’t see why we need special handling for [ term .. term ], since it
would work exactly the same way (wrt observable behaviour) as it
currently does. Also, [ .. term ] is not necessary, as the starting
index is always known.

Having a special rule for parsing [ term .. ] rules out things like [
a.., b..c]

> [experimental grammar rules skipped]
...

> I don't think it can be made to work without more than one token of
> lookahead:
>
> @array[ !($. % 5) .. !($. % 7) ? 0 .. 4 : -5 .. -1 ]
>
> should match '[' expr ']' and not match '[' term DOTDOT term ']'
>
> because the token after the second notted modulo term is a hook, not a
> right square bracket or a comma
>
> but to realize this, the parser must maintain ambiguity for longer
than one
> token. So that won't work.
>
> And that leaves, examining the emitted expression tree of the slice
indices
> to see if it has any range operators in it at the top level -- or, at the
> next-to-top if the top are commas.

The only thing we need special handling for is postfix C<..>. If we
make it a rule that it is a syntactic special case occurring in [...]
and only before , => or ] then it is simply a matter of having the lexer
look for the next term and emitting a POSTDOTDOT instead of a DOTDOT.

The op.c can weed out illegal uses of it in finalize_op.

[Father Chrysostomos via RT]

On Tue Jul 09 21:27:33 2013, sprout wrote:
> Also, [ .. term ] is not necessary, as the starting
> index is always known.

Additionally, (foo)[substr $bar,0,1, .. 5] is already valid syntax.

[David Nicol]

On Tue, Jul 9, 2013 at 11:27 PM, Father Chrysostomos via RT <perlbug-...@perl.org> wrote:

We seem to be talking past each other somewhat....  But this
conversation is fascinating nonetheless.

Yes indeed, starting with different visions of the goal.

 

> if a new clause in perly.y will work, then so will three new clauses, one
> for [ term .. term ] and one for [ term .. ] and one for [ .. term ].

I don’t see why we need special handling for [ term .. term ], since it
would work exactly the same way (wrt observable behaviour) as it
currently does.  Also, [ .. term ] is not necessary, as the starting
index is always known.

The ending index is also always known: it's -1. If we're implying -1 to make pythonists feel more at-home, we should imply 0 too. That's why [ .. term ] is necessary, for balance.

The only thing we need special handling for is postfix C<..>.  If we

make it a rule that it is a syntactic special case occurring in [...]
and only before , => or ] then it is simply a matter of having the lexer
look for the next term and emitting a POSTDOTDOT instead of a DOTDOT.

The op.c can weed out illegal uses of it in finalize_op.

so you wouldn't support

          @some = @array[ 0 .. -3 ]

as a clearer way of writing

          splice @some = @array, -2;

Instead of slicing to the end when the RHS of DOTDOT is empty, I envision using arbitrary negative numbers to mean count-from-the-top, the way scalar indexing works.

 What if the DOTDOT operator produces a "range object" that can be treated as an iterator -- for (RANGE) acts like an iterator now -- the range object contains at least the start and end points. The list a range object flattens to would be unusual when all of the following conditions are true:

    start is non-negative

    end is negative

    the list the range object is flattening into will become slice indices

when all three of those are true, the range object accesses the target array's length and adds it to the end number.

And then, the TIEARRAY and TIEHASH interfaces can get more complicated, by supporting arbitrary range-slice semantics.

[David Nicol]

On Wed, Jul 10, 2013 at 12:00 AM, David Nicol <david...@gmail.com> wrote:

And then, the TIEARRAY and TIEHASH interfaces can get more complicated, by supporting arbitrary range-slice semantics.

such as, a sparse array could return existing elements only, when sliced with a range.

[Father Chrysostomos via RT]

On Tue Jul 09 22:01:01 2013, david...@gmail.com wrote:
> On Tue, Jul 9, 2013 at 11:27 PM, Father Chrysostomos via RT <
> perlbug-...@perl.org> wrote:
> > I don’t see why we need special handling for [ term .. term ], since it
> > would work exactly the same way (wrt observable behaviour) as it
> > currently does. Also, [ .. term ] is not necessary, as the starting
> > index is always known.
> >
>
> The ending index is also always known: it's -1. If we're implying -1 to
> make pythonists feel more at-home,

No, not for that reason. Forget Python. :-) But because it is
convenient in some_complex_expression->[...].

> we should imply 0 too. That's why [ ..
> term ] is necessary, for balance.
>
> The only thing we need special handling for is postfix C<..>. If we
> > make it a rule that it is a syntactic special case occurring in [...]
> > and only before , => or ] then it is simply a matter of having the lexer
> > look for the next term and emitting a POSTDOTDOT instead of a DOTDOT.
> >
> > The op.c can weed out illegal uses of it in finalize_op.
>
>
> so you wouldn't support
>
> @some = @array[ 0 .. -3 ]
>
> as a clearer way of writing
>
> splice @some = @array, -2;

I was going to say ‘Definitely not’, but then, as I started to explain
why, I realised my reasoning was flawed.

That is perhaps not such a bad idea, but for backward compatibility it
should only happen under use v5.20.

With that, we don’t even need open-ended ranges, so we don’t have to
abuse perl’s syntax.

If this range syntax only works that way directly inside a slice (not as
an operand to an operator within the slice), then what should 1..-1 do
elsewhere? It should probably croak, to avoid subtle changes to the
code giving wildly different results (empty lists). Of course, 1..0
would have to continue giving an empty list.

>
> Instead of slicing to the end when the RHS of DOTDOT is empty, I envision
> using arbitrary negative numbers to mean count-from-the-top, the way
scalar
> indexing works.
>
>
> What if the DOTDOT operator produces a "range object" that can be treated
> as an iterator -- for (RANGE) acts like an iterator now -- the range
object
> contains at least the start and end points.

If slices can propagate ‘range context’ to their operands the way other
contexts are applied (and if range ops will return special internal
range SVs), then the whole low-precedence branch folding I mentioned
resolves itself. [$a ? 1..-2 : 3..-4 ] will work.

> The list a range object
> flattens to would be unusual when all of the following conditions are
true:
>
> start is non-negative
> end is negative
> the list the range object is flattening into will become slice indices
>
> when all three of those are true, the range object accesses the target
> array's length and adds it to the end number.

Yes, that all works out nicely.

> And then, the TIEARRAY and TIEHASH interfaces can get more complicated, by
> supporting arbitrary range-slice semantics.

Stop it! You’re making my head spin!

[Aaron Priven]

How would new slicing syntax work with object methods and other times you'd want to pass a slice specification to a sub?

I can see how you could redefine stuff inside @a[in_here], but not $obj->slice(in_here). Obviously if it's just an accessor you can do (@{$obj->elements})(slice), but if something else happens in the method that doesn't work.

[David Nicol]

On Wed, Jul 10, 2013 at 7:03 PM, Aaron Priven <aa...@priven.com> wrote:

How would new slicing syntax work with object methods and other times you'd want to pass a slice specification to a sub?

I can see how you could redefine stuff inside @a[in_here], but not $obj->slice(in_here). Obviously if it's just an accessor you can do (@{$obj->elements})(slice), but if something else happens in the method that doesn't work.

It wouldn't. That's another fine argument against introducing situational syntaces,

If ranges were to defer their expansion as long as possible, a range such as 5 .. -2 could exist as a range object and get passed as part of a slice specification, where it would mean something (from the sixth element to the second element from the end) instead of getting immediately evaulated to the empty list. How to wrap that up for passing as one argument however? Maybe within square brackets, as contents of an array-ref, expansion deferred as long as possible, so if you defer the flattening until actually constructing the slice, you have succeeded.

[Aaron Priven]

On Wed, Jul 10, 2013 at 7:03 PM, Aaron Priven <aa...@priven.com> wrote:

How would new slicing syntax work with object methods and other times you'd want to pass a slice specification to a sub?

On Jul 10, 2013, at 5:41 PM, David Nicol wrote:

It wouldn't. That's another fine argument against introducing situational syntaces,

If ranges were to defer their expansion as long as possible, a range such as 5 .. -2 could exist as a range object and get passed as part of a slice specification, where it would mean something (from the sixth element to the second element from the end) instead of getting immediately evaulated to the empty list. How to wrap that up for passing as one argument however? Maybe within square brackets, as contents of an array-ref, expansion deferred as long as possible, so if you defer the flattening until actually constructing the slice, you have succeeded.

Yes, this is what I meant by suggesting a new slice syntax with a quote-like operator. So:

@ary[qs/slice syntax goes here/] 

or 

$obj->slice(qs/slice syntax goes here/)

And it would become a slice object of some sort or other, that would not be processed until the slice actually occurred.

Like regex syntax, it could have basically nothing to do with regular perl syntax. (I had this whole idea of using Set::IntSpan objects before realizing that negative numbers mean something completely different to Set::IntSpan. Still, there's something about using hyphens instead of dots for ranges that appeals to me)

Of course, [qs/3-/] probably wouldn't satisfy people looking to make [3..] work. I'm not sure how to introduce it more succinctly given the paucity of ASCII characters. @a:[3-5] ? With Unicode we could have @a⁅3-5⁆ or something. 

Or perhaps none of this makes any sense at all.

-- 

Aaron Priven, aa...@priven.com, http://www.priven.com/aaron

[Aristotle Pagaltzis]

* Aaron Priven <aa...@priven.com> [2013-07-11 08:20]:

> Yes, this is what I meant by suggesting a new slice syntax with
> a quote-like operator. So:
>
> @ary[qs/slice syntax goes here/]
>
> or
>
> $obj->slice(qs/slice syntax goes here/)
>
> And it would become a slice object of some sort or other, that would
> not be processed until the slice actually occurred.

C.f. List::Maker maybe.

--
*AUTOLOAD=*_;sub _{s/..([^:]*)$/()[print$1,(",$\/"," ")[defined wantarray]]/e;$_}

[Father Chrysostomos via RT]

I started to implement two of the suggestions in this ticket, but then
ran into pad bugs that I have already fixed on the sprout/padconst
branch. So this is stalled until that is merged. I am waiting for
either Chip to concede or Ricardo to give a green light. See
<https://rt.perl.org/rt3/Ticket/Display.html?id=109744#txn-1230929>.

[Father Chrysostomos via RT]

I started to implement [1..] and found myself liking your idea more and
more.

A CPAN module could provide a function that allows slice(@array,
qs(...)) or slice(@array, "1.., 3..5") as well as slice(a_list(), ...).
There is plenty of room for experimentation here, and it doesn’t have
to be in core.

Anyway, my unfinished work is on the sprout/slice branch.

[Father Chrysostomos via RT]

On Mon Aug 19 07:30:09 2013, sprout wrote:
> Anyway, my unfinished work is on the sprout/slice branch.

And also attached, but without the perly.* regenerated. Including the
latter made RT choke.

[Ed Avis]

Thanks for your work on this.

--
Ed Avis <e...@waniasset.com>

[Father Chrysostomos via RT]

On Mon Aug 19 07:30:09 2013, sprout wrote:

> I started to implement [1..] and found myself liking your idea more and
> more.
>
> A CPAN module could provide a function that allows slice(@array,
> qs(...)) or slice(@array, "1.., 3..5") as well as slice(a_list(), ...).
> There is plenty of room for experimentation here, and it doesn’t have
> to be in core.
>
> Anyway, my unfinished work is on the sprout/slice branch.

The reason I posted this is that I’m not sure whether it would be
worthwhile to continue work at this stage. I would prefer to see a CPAN
implementation of a slice function, rather than [1..] in core.

Is there enough support for this that it would be worthwhile to continue
with the patch?

[Zefram]

Father Chrysostomos via RT wrote:
>The reason I posted this is that I'm not sure whether it would be
>worthwhile to continue work at this stage. I would prefer to see a CPAN
>implementation of a slice function, rather than [1..] in core.

I'd also prefer to see it on CPAN in some form than to further complicate
core operators.

-zefram