Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Smooth or Chunky?

4 views
Skip to first unread message

Larry Wall

unread,
Jan 23, 2007, 9:22:37 PM1/23/07
to perl6language
I've been struggling lately with a missing generalization, and I'm not
sure how it's going to play out, so I thought I'd ask for advice, or
at least think out loud a bit.

Perl has always had functions and listops that take a flat list and
do something with each element. Perl has also had various functions
that return flat lists, and these naturally feed into the listops.
For instance, the map function has always produced a flat list. A
split with captures flattens the captures along with the split values.

Recently I started redefining C<map> to return multislices such that

map { $_, $_ * 10 }, 1..3

seems to return 1,10,2,20,3,30 by default, but in a multidimensional
context:

@@multislice := map { $_, $_ * 10 }, 1..3

it would have the value [1,10], [2,20], [3,30].

Likewise the values returned by loop iterations or by C<take> have been
turned into such multislices.

But then these all seem like special casing something more general.
When I look at the functions we've defined so far, I see that

zip(1,2,3; 4,5,6)

produces

[1,4],[2,5],[3,6]

while

each(1,2,3; 4,5,6)

produces

1,4,2,5,3,6

and then I have to ask myself, "Why in this case do we have two separate
functions that essentially do the same thing?" It's a design smell.

Which leads me to think that zip should really return

1,4; 2,5; 3,6

and let the context either flatten or not. Basically, a list return
is a Capture, so a higher order function that calls a list operator
repeatedly is really returning a Capture of Captures, or a List of Captures,
and that's probably what a "multislice" is really.

However, currently the only way to get "chunky" behavior is to bind
the CoC to a multislice array:

@smooth := zip(1,2,3; 4,5,6)
@@chunky := zip(1,2,3; 4,5,6)

If the default is "smooth", then we need better rvalue syntax for explicitly
turning a CoC into LoA, such that you could say

chunky zip(1,2,3; 4,5,6)

and get the list value

[1,4],[2,5],[3,6]

And indeed, it's easy to define the function in current notation, something
like:

sub chunky (*@@chunky) { return @chunky }

Basically, this is the inverse of [;], which turns LoA into a CoC.

[;] chunky mumble()

But "chunky" is clunky, and I'm wondering what syntactic relief we can
give ourselves here. I think people would get tired of writing

.map({...}).chunky
chunky split
chunky for 1..100 { $_, $_*10 }

I think most other languages would probably just default to returning
a structured value and force the user to flatten explicitly. That doesn't
seem much like the Perl Way though...

Distinguish via unary operators maybe?

|(1,2; 3,4) # smooth: 1,2,3,4
=(1,2; 3,4) # chunky: [1,2],[3,4]

Doesn't seem to work well with method forms though...

(1,2; 3,4)."|"
(1,2; 3,4)."="

We could have a special .| form if the default where .=, but .= is
taken already so .| can't be the default there. Maybe something else
is better than = there.

Or maybe we need a naming convention that distinguishes smooth/chunky
variants of all the named functions/methods. Then we have the
non-commital form:

map

the smooth form

Xmap
mapX

and the chunky form

Ymap
mapY

for some value of X and Y. But that approach starts to get a bit
obnoxious when you want to start adding other similar modifiers, like
whether map is allowed to be parallel or must be executed serially.
It also doesn't work well with operator forms like ¥ and such.

(That almost suggests it should be another metaoperator. Let's all
shudder together now...but not rule out the possibility.)

Adverbs would be another approach. Those could conceivably work on
operators, though a bit oddly insofar as applying to all previous
list-associative siblings:

for 1,2 ¥ 3,4 ¥ 5,6 :smooth -> $a, $b, $c {...}
for 1,2 ¥ 3,4 ¥ 5,6 :chunky -> [$a, $b, $c] {...}

And how do you force the return value of the "for" to be smooth or
chunky? Course, a "chunky" listop would probably do for that.

I should also mention I did (briefly) consider the "null" reduce
operator:

[] zip(1,2;3,4)

to mean "slap [] around each element", but it runs into ambiguity with
the existing [] form indicating an empty list.

Or maybe a multislice array is a special type, so it's really a type cast:

@@(zip(1,2;3,4))

But then people will try to write @@zip and wonder why it doesn't work...

The possibilities are endless, and I don't doubt that you can think of
a few more...

Larry

Darren Duncan

unread,
Jan 23, 2007, 10:23:31 PM1/23/07
to perl6language
At 6:22 PM -0800 1/23/07, Larry Wall wrote:
>Recently I started redefining C<map> to return multislices such that
>
> map { $_, $_ * 10 }, 1..3
>
>seems to return 1,10,2,20,3,30 by default, but in a multidimensional
>context:
>
> @@multislice := map { $_, $_ * 10 }, 1..3
>
>it would have the value [1,10], [2,20], [3,30].

Maybe I'm missing something important that would affect various new
Perl 6 features, but if not then we could simply do this like it is
done in Perl 5, which is to use the "smooth" approach all the time,
and if people want chunks, they do something that would cause
explicit chunking.

Eg, smooth:

map { $_, $_ * 10 }, 1..3

Vs chunky:

map { [$_, $_ * 10] }, 1..3

-- Darren Duncan

Larry Wall

unread,
Jan 24, 2007, 12:57:17 AM1/24/07
to perl6language
On Tue, Jan 23, 2007 at 07:23:31PM -0800, Darren Duncan wrote:
: At 6:22 PM -0800 1/23/07, Larry Wall wrote:
: >Recently I started redefining C<map> to return multislices such that
: >
: > map { $_, $_ * 10 }, 1..3
: >
: >seems to return 1,10,2,20,3,30 by default, but in a multidimensional
: >context:
: >
: > @@multislice := map { $_, $_ * 10 }, 1..3
: >
: >it would have the value [1,10], [2,20], [3,30].
:
: Maybe I'm missing something important that would affect various new
: Perl 6 features, but if not then we could simply do this like it is
: done in Perl 5, which is to use the "smooth" approach all the time,
: and if people want chunks, they do something that would cause
: explicit chunking.

Perl 5 has lots of places where it doesn't scale to a multi-programmer
team very well. Arguably the assumption that you have control over
the entire source code is one of those places.

: Eg, smooth:


:
: map { $_, $_ * 10 }, 1..3
:
: Vs chunky:
:
: map { [$_, $_ * 10] }, 1..3

You seem to have picked the one function where that approach is possible. :)

(Well, arguably C<take> could use that approach too if you have control
of the code doing the gathering. But what about things like C<split>
and C<zip>?)

But the point is *not* to force it one way or the other--the point is
that many such functions would probably prefer not to commit one way or
the other, and they can't do that if they automatically throw away the
"dimensional" information. Looking at it from the other end, you may
not have control of the code that is returning the multislice. If the
code predecides for you, then either you have to explicitly strip away
the top level of [] all the time, or you can't even reproduce the []
structure because the run-length information is discarded. That's why
I think the default should be to hide the top-level [] in what we call
a multislice, and give the user the choice of whether to unflatten it
or leave it (seemingly) flat to begin with.

Larry

Ben Morrow

unread,
Jan 24, 2007, 1:48:17 AM1/24/07
to perl6-l...@perl.org

Quoth la...@wall.org (Larry Wall):

> I should also mention I did (briefly) consider the "null" reduce
> operator:
>
> [] zip(1,2;3,4)
>
> to mean "slap [] around each element", but it runs into ambiguity with
> the existing [] form indicating an empty list.

Would using [[]] instead work? This is (at least to me) nicely visually
indicative of 'build a list of lists'. It is a little punctuation-heavy,
of course; though we could always allow "\x{27E6}\x{27E7}" as an
alternative :).

Ben

--
Although few may originate a policy, we are all able to judge it.
Pericles of Athens, c.430 B.C.
b...@morrow.me.uk

Ruud H.G. van Tol

unread,
Jan 24, 2007, 6:37:54 AM1/24/07
to perl6-l...@perl.org
Larry Wall schreef:

> the point is *not* to force it one way or the other--the point is
> that many such functions would probably prefer not to commit one way
> or the other, and they can't do that if they automatically throw away
> the "dimensional" information.

Like with numbers, this looks to me like a "multiple faces" domain
again.

The "flat" representation is to be postponed until it needs to be
delivered (although when you know beforehand that you are to return a
one-time flat list, you can build up that list, like as a sequence of
references, while putting the data up in a multidimensional structure).

The dimensionally structured data (the multislice?) is the living thing.

(There is the meal, the recipe and the shopping list. I prefer the
shopping list to be in the order that I cross the shop.)

--
Groet, Ruud

Message has been deleted

Blair Sutton

unread,
Jan 24, 2007, 10:55:51 AM1/24/07
to perl6language
Just some ideas for multidimensional map.

map { $_, $_ * 10 }, 1..3

1,10, 2,20, 3,30

map { $_, $_ * 10 }, 1..3

[1,10], [2,20], [3,30]

map { $_[0], $_[1] * 10 }, 1..3, 3..4
1,30, 2,40, 3,undef
or better maybe
1,30, 2,40, 3

map { $_[0], $_[1] * 10 }, 1..3, 2..3, 3 op => []
[1,2,3], [2,3,undef], [3,undef,undef]
or maybe better
[1,2,3],[2,3],[3]

Here's a quite obscure usage: -

map { $_, $_ * 10 }, 1..3, 1..2 op => \&f
f(1,10),f(2,20),f(3,30)

map { $_, $_ * 10, $_ * 100 }, 1, 1, 1 op => \&f;
f(1,10,100)

Maybe an operator can be specified to zip. I.e.

zip(1,2,3; 4,5,6)

[1,4,2,5,3,6]


zip(1,2,3; 4,5,6) op=> []


[1,4],[2,5],[3,6]


zip(1,2,3; 4,5,6) op => \&f
f(1,4),f(2,5),f(3,6)

Just some ideas that may be of help once P6-ified.

Blair

Austin Frank

unread,
Jan 24, 2007, 11:37:50 AM1/24/07
to perl6-l...@perl.org
On Tue, Jan 23 2007, Larry Wall wrote:

> ... Basically, this is the inverse of [;], which turns LoA into a


> CoC.
>
> [;] chunky mumble()
>
> But "chunky" is clunky, and I'm wondering what syntactic relief we

> can give ourselves here. ...


>
> (That almost suggests it should be another metaoperator. Let's all

> shudder together now...but not rule out the possibility.) ...


>
> The possibilities are endless, and I don't doubt that you can think of
> a few more...

To the horror of parsers everywhere, we could make the inverse of [;]
the "outverse" of [;]. What about this?

];[ mumble()
];[ map { $_, $_ * 10 }, 1..3
];[ zip(1,2;3,4)

There's already some precedent for bracketing pairs being pointed the
wrong way, as in (1,1,2,3,5) »+« (1,2,3,5,8). Also, ];[ kind of
looks like a butterfly, which I count as a point in its favor.

In true metaoperator form, you could presumably throw other operators
between the backwards brackets for other chunky operators. I haven't
really thought through what I would want multislice addition ( ]+[ )
or multislice equality testing ( ]==[ ) to mean, though.

Thanks,
/au

--
Austin Frank
http://aufrank.net
GPG Public Key (D7398C2F): http://aufrank.net/personal.asc

Brad Bowman

unread,
Jan 26, 2007, 9:23:30 PM1/26/07
to perl6language
Larry Wall wrote:
> But the point is *not* to force it one way or the other--the point is
> that many such functions would probably prefer not to commit one way or
> the other, and they can't do that if they automatically throw away the
> "dimensional" information.

The dimensional information has to be kept, but perhaps it doesn't need
to be kept in the list returned. Maybe it could be out of band like
$1, @+, pos(), etc in Perl 5, the dimensional structure dangles off the
flat list but can be somehow applied to restructure it. Possibly using
a general list to deep-structure operation.

This is similar to the context-sensitive version in simple cases, but
may be more powerful in cases where the external context doesn't
propagate (except by explicitly using want()). Perhaps when the
lists are generated and store before being returned.

[ from an older message ]


> I think most other languages would probably just default to returning
> a structured value and force the user to flatten explicitly. That doesn't
> seem much like the Perl Way though...

That might not be so bad.

Could the chunky/smooth context information come from List/Capture context?
I'm not sure if that distinction exists.

Gotta go see a film,

Brad

--
If you attach bags of cloves to your body, you will not be effected by
inclemency or colds. -- Hagakure

0 new messages