How to write this as a list comprehension?

[Piet van Oostrum]

Hi,

I am looking for an elegant way to write the following code as a list
comprehension:

labels = []
for then, name in mylist:
_, mn, dy, _, _, _, wd, _, _ = localtime(then)
labels.append(somefunc(mn, day, wd, name))

So mylist is a list of tuples, the first member of the tuple is a time
(as epoch offset) and I neeed to apply a function on some fields of the
localtime of it.

I could define a auxiliary function like:

def auxfunc(then, name):
_, mn, dy, _, _, _, wd, _, _ = localtime(then)
return somefunc(mn, day, wd, name)

and then use
[auxfunc(then, name) for then, name in mylist]

or even
[auxfunc(*tup) for tup in mylist]

But defining the auxfunc takes away the elegance of a list comprehension. I would like to integrate the unpacking of localtime() and calling somefunc within the list comprehension, but I don't see a simple way to do that.

somefunc(mn, day, wd, name) for _, mn, dy, _, _, _, wd, _, _ in [localtime(then)]
(i.e. using a list comprehension on a one element list to do the variable shuffling)
works but I don't find that very elegant.

labels = [somefunc(mn, day, wd, name)
for then, name in mylist
for _, mn, dy, _, _, _, wd, _, _ in [localtime(then)]]

Python misses a 'where' or 'let'-like construction as in Haskell.

Anybody has a more elegant solution?
--
Piet van Oostrum <pi...@vanoostrum.org>
WWW: http://pietvanoostrum.com/
PGP key: [8DAE142BE17999C4]

[Dan Stromberg]

On Fri, Jan 17, 2014 at 3:19 PM, Piet van Oostrum <pi...@vanoostrum.org> wrote:
> Hi,
>
> I am looking for an elegant way to write the following code as a list
> comprehension:
>
> labels = []
> for then, name in mylist:
> _, mn, dy, _, _, _, wd, _, _ = localtime(then)
> labels.append(somefunc(mn, day, wd, name))

My recomendation: Don't use a list comprehension. List comprehensions
and generator expressions are great for quick little things, but
become less readable when you have to string them over multiple
physical lines.

[Rustom Mody]

+1
Yes Ive often been bitten by the lack of a 'comprehension-let'

Something like this is possible??


[somefunc(mn,day,wd,name) for (_, mn,dy,_,_,_,wd,_,_), name) in [localtime(then), name for then, name in mylist]]

Some debugging of the structure will be necessary (if at all possible)
I dont have your functions so cant do it

[Peter Otten]

Options I can think of:

You could do it in two steps...

time_name_pairs = ((localtime(then), name) for then, name in mylist)
labels = [somefunc(t.tm_mon, t.tm_mday, t.tm_wday, name)
for t, name in time_name_pairs]

...or you could inline the helper function...

mon_mday_wday = operator.attrgetter("tm_mon", "tm_day", "tm_wday")
labels = [somefunc(*mon_mday_wday(localtime(then)), name=name)
for then, name in mylist]

-- but both seem less readable than the classical for-loop.

What would a list-comp with `let` or `where` look like? Would it win the
beauty contest against the loop?

[Piet van Oostrum]

Rustom Mody <rusto...@gmail.com> writes:

> On Saturday, January 18, 2014 4:49:55 AM UTC+5:30, Piet van Oostrum wrote:

[...]

>
>> Python misses a 'where' or 'let'-like construction as in Haskell.
>
> +1
> Yes Ive often been bitten by the lack of a 'comprehension-let'

If it used only in a comprehension as in my example you can write instead of 'where vars = expr':
for vars in [expr], unnecessarily construction a one element list.
If there would be a syntax like:
for vars = expr
this could be avoided.

>
> Something like this is possible??
>
>
> [somefunc(mn,day,wd,name) for (_, mn,dy,_,_,_,wd,_,_), name) in
> [localtime(then), name for then, name in mylist]]

It works modulo some corrections in the parentheses:

[somefunc(mn,day,wd,name) for (_, mn,dy,_,_,_,wd,_,_), name in
[(localtime(then), name) for then, name in mylist]]
but I find that hardly more elegant.

[Matěj Cepl]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 2014-01-17, 23:19 GMT, you wrote:
> But defining the auxfunc takes away the elegance of a list
> comprehension.

Au contraire! Remember, that brevity is the sister of talent.

I would definitively vote for

labels = [make_label(then, name) for then, name in mylist]

(always use descriptive names of functions and variables;
auxfunc is a short way to the hell)

Beauty of the list comprehensions is that they show nicely what
list is being processed, how it is filtered (if at all), and
what we do with each element of the generated list. Anything you
add to this simplicity is wrong. Whenever you start to feel you
are missing some methods how to stuff more commands into
a comprehension (or for example multiple embedded ones), you
should start new function.

The same rule applies here as with any other lambda function
(because these are in fact lambda functions): the best way how
to write lambda is to write algorithm somewhere on the side,
describe what this function does in one word, then add `def` in
front of that name, and use so created named function instead.

Best,

Matěj

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iD8DBQFS2l4X4J/vJdlkhKwRAjEgAJ4n1OuANYlVFzlgBZ0f1uMhO/t36gCfdFjE
VmYDJ+F7aN0khzvlY50i0iA=
=Trcc
-----END PGP SIGNATURE-----

[Alain Ketterlin]

Piet van Oostrum <pi...@vanoostrum.org> writes:

[...]

> I could define a auxiliary function like:
>
> def auxfunc(then, name):
> _, mn, dy, _, _, _, wd, _, _ = localtime(then)
> return somefunc(mn, day, wd, name)
>
> and then use
> [auxfunc(then, name) for then, name in mylist]

[...]

> labels = [somefunc(mn, day, wd, name)
> for then, name in mylist
> for _, mn, dy, _, _, _, wd, _, _ in [localtime(then)]]
>
> Python misses a 'where' or 'let'-like construction as in Haskell.

"let x = v in e" really is (lambda x:e)(v)

In your case:

[ (lambda na,ti : somefunc(ti[1],ti[2],ti[6],na))(name,localtime(then))
for then,name in mylist ]

Warning: absolutely untested (not even syntax-checked).

You may also use *localtime(...) and keep underscores.

-- Alain.

[Rustom Mody]

On Saturday, January 18, 2014 2:06:29 PM UTC+5:30, Peter Otten wrote:

> Options I can think of:

> You could do it in two steps...

> time_name_pairs = ((localtime(then), name) for then, name in mylist)
> labels = [somefunc(t.tm_mon, t.tm_mday, t.tm_wday, name)
> for t, name in time_name_pairs]

> ...or you could inline the helper function...

> mon_mday_wday = operator.attrgetter("tm_mon", "tm_day", "tm_wday")
> labels = [somefunc(*mon_mday_wday(localtime(then)), name=name)
> for then, name in mylist]

> -- but both seem less readable than the classical for-loop.

> What would a list-comp with `let` or `where` look like? Would it win the
> beauty contest against the loop?

For me this is neat

[somefunc(mn,day,wd,name) for (then, name) in mylist let (_,mn,dy,_,_,_,wd,_,_) = localtime(then)]

Others may not find it so!

See it across > 1 line (as I guess it will come after being posted!) and its not so neat.

[Jussi Piitulainen]

I would write that on three lines anyway, properly indented:

[ somefunc(mn,day,wd,name)

for (then, name) in mylist
let (_,mn,dy,_,_,_,wd,_,_) = localtime(then) ]

It could be made to use existing keywords:

[ somefunc(mn,day,wd,name)

for (then, name) in mylist

with localtime(then) as (_,mn,dy,_,_,_,wd,_,_) ]

[Piet van Oostrum]

Alain Ketterlin <al...@dpt-info.u-strasbg.fr> writes:

> Piet van Oostrum <pi...@vanoostrum.org> writes:
> [...]

>> Python misses a 'where' or 'let'-like construction as in Haskell.
>
> "let x = v in e" really is (lambda x:e)(v)
>

You are right, but it is a lot less readable IMHO.

[John Allsup]

Hi,

I'd agree with the advice that it's not the best idea: readability sucks
here, but consider the following:


import time

def somefunc(a,b,c,d): # dummy function
return "{} - {} - {} : {}".format(a,b,c,d)
l = [(time.time(),"name {}".format(n)) for n in range(100)] # dummy data

# the line in question
labels = [somefunc(*(lambda t,n:
(t.tm_mon,t.tm_mday,t.tm_wday,n))(time.localtime(x[0]),x[1])) for x in l]


print(labels) # just to see the result


If you find that hard to decipher, the consider the maintainability of
code you write that uses such comprehensions. You need to include
comments that explain what this does, and it is easier to write a
longhand version using .append() and variable assignments. I presume
performance won't be an issue determining the right approach, since then
you'd be using C or C++.

John

On 17/01/2014 23:49, Dan Stromberg wrote:
> On Fri, Jan 17, 2014 at 3:19 PM, Piet van Oostrum <pi...@vanoostrum.org> wrote:

>> Hi,
>>
>> I am looking for an elegant way to write the following code as a list
>> comprehension:
>>
>> labels = []
>> for then, name in mylist:
>> _, mn, dy, _, _, _, wd, _, _ = localtime(then)
>> labels.append(somefunc(mn, day, wd, name))
>

> My recomendation: Don't use a list comprehension. List comprehensions
> and generator expressions are great for quick little things, but
> become less readable when you have to string them over multiple
> physical lines.
>

[Piet van Oostrum]

John Allsup <py...@allsup.co> writes:

> Hi,
>
> I'd agree with the advice that it's not the best idea: readability sucks
> here, but consider the following:
>
>
> import time
>
> def somefunc(a,b,c,d): # dummy function
> return "{} - {} - {} : {}".format(a,b,c,d)
> l = [(time.time(),"name {}".format(n)) for n in range(100)] # dummy data
>
> # the line in question
> labels = [somefunc(*(lambda t,n:
> (t.tm_mon,t.tm_mday,t.tm_wday,n))(time.localtime(x[0]),x[1])) for x in
> l]
>
>
> print(labels) # just to see the result
>
>
> If you find that hard to decipher, the consider the maintainability of
> code you write that uses such comprehensions. You need to include
> comments that explain what this does, and it is easier to write a
> longhand version using .append() and variable assignments. I presume
> performance won't be an issue determining the right approach, since then
> you'd be using C or C++.

I'll stay with

labels = [somefunc(mn, day, wd, name)
for then, name in mylist
for _, mn, dy, _, _, _, wd, _, _ in [localtime(then)]]

where (sic!) the last part means as much as
where _, mn, dy, _, _, _, wd, _, _ = localtime(then)

I find the list comprehension preferable because it makes it more clear
that a new list is constructed from an existing list, something that is
not as immediately clear with the append construction.

[Rhodri James]

On Sat, 18 Jan 2014 16:00:45 -0000, Jussi Piitulainen
<jpii...@ling.helsinki.fi> wrote:

> Rustom Mody writes:
>
>> On Saturday, January 18, 2014 2:06:29 PM UTC+5:30, Peter Otten wrote:
>>
>> > What would a list-comp with `let` or `where` look like? Would it
>> > win the beauty contest against the loop?
>>
>> For me this is neat
>>
>> [somefunc(mn,day,wd,name) for (then, name) in mylist let
>> (_,mn,dy,_,_,_,wd,_,_) = localtime(then)]
>>
>> Others may not find it so!

Count me firmly in the "others" camp. It looks ugly, it flows appallingly
badly as English, and its only going to get worse as you pile in more
variables and expressions. -100 from me.

>> See it across > 1 line (as I guess it will come after being posted!)
>> and its not so neat.
>
> I would write that on three lines anyway, properly indented:
>
> [ somefunc(mn,day,wd,name)
> for (then, name) in mylist
> let (_,mn,dy,_,_,_,wd,_,_) = localtime(then) ]
>
> It could be made to use existing keywords:
>
> [ somefunc(mn,day,wd,name)
> for (then, name) in mylist
> with localtime(then) as (_,mn,dy,_,_,_,wd,_,_) ]

Better, in that it's readable. It's still storing up trouble, though.

Seriously, what's inelegant about this?

def meaningful_name(then, name):
_,mn,dy,_,_,_,wd,_,_ = localtime(then)
return somefunc(mn, dy, wd, name)

...

[meaningful_name(then, name) for (then, name) in mylist]

I assume there's some good reason you didn't want somefunc() to do the
localtime() itself?

--
Rhodri James *-* Wildebeest Herder to the Masses

[Piet van Oostrum]

"Rhodri James" <rho...@wildebst.org.uk> writes:

> On Sat, 18 Jan 2014 16:00:45 -0000, Jussi Piitulainen
> <jpii...@ling.helsinki.fi> wrote:

[...]

>> I would write that on three lines anyway, properly indented:
>>
>> [ somefunc(mn,day,wd,name)
>> for (then, name) in mylist
>> let (_,mn,dy,_,_,_,wd,_,_) = localtime(then) ]
>>
>> It could be made to use existing keywords:
>>
>> [ somefunc(mn,day,wd,name)
>> for (then, name) in mylist
>> with localtime(then) as (_,mn,dy,_,_,_,wd,_,_) ]
>
> Better, in that it's readable. It's still storing up trouble, though.
>
> Seriously, what's inelegant about this?
>
> def meaningful_name(then, name):
> _,mn,dy,_,_,_,wd,_,_ = localtime(then)
> return somefunc(mn, dy, wd, name)
>
> ...
>
> [meaningful_name(then, name) for (then, name) in mylist]
>
> I assume there's some good reason you didn't want somefunc() to do the
> localtime() itself?

Actually in my use case somefunc(mn,day,wd,name) was an expression, not
really a function call. I just presented it like that to make it a more
generalised abstract case. So in my case that would introduce an
additional function outside of the comprehension.

Let me summarize what my problem really was: In a comprehension you
generally need local name(s), which can only be introduced with the
'for' construction. But then the name(s) must be bound to an element of
a sequence/iterator. There is no way to bind the name(s) to a single
object other than putting that object in a one element sequence. I was
just looking for a way to avoid that. Functional programming languages
have a way to do this with the 'let' or 'where' construction which is
missing in Python.

Thanks everybody for your thoughts on this.

[Rustom Mody]

On Monday, January 20, 2014 4:32:40 PM UTC+5:30, Piet van Oostrum wrote:
> "Rhodri James" writes:

> > On Sat, 18 Jan 2014 16:00:45 -0000, Jussi Piitulainen wrote:

> [...]

> >> I would write that on three lines anyway, properly indented:
> >> [ somefunc(mn,day,wd,name)
> >> for (then, name) in mylist
> >> let (_,mn,dy,_,_,_,wd,_,_) = localtime(then) ]
> >> It could be made to use existing keywords:
> >> [ somefunc(mn,day,wd,name)
> >> for (then, name) in mylist
> >> with localtime(then) as (_,mn,dy,_,_,_,wd,_,_) ]
> > Better, in that it's readable. It's still storing up trouble, though.
> > Seriously, what's inelegant about this?
> > def meaningful_name(then, name):
> > _,mn,dy,_,_,_,wd,_,_ = localtime(then)
> > return somefunc(mn, dy, wd, name)
> > ...
> > [meaningful_name(then, name) for (then, name) in mylist]
> > I assume there's some good reason you didn't want somefunc() to do the
> > localtime() itself?

> Actually in my use case somefunc(mn,day,wd,name) was an expression, not
> really a function call. I just presented it like that to make it a more
> generalised abstract case. So in my case that would introduce an
> additional function outside of the comprehension.

Nice! Thanks

One of things that people who decry comprehensions (and in general
hi-level constructs) dont understand is this:

One fundamental characteristic of increasing hi-level-ness of a programming
language is increased *freedom-in-naming* structures.

The 'structures' could be expressions, statements, or whatever is
the thing of the programming language.

A couple of examples will illustrate:

1. In most (imperative) programming languages when a computation is described
as a function it must willy-nilly be named. In lambda-calculus thats an option.

2. In any hi-level programming language, if we have an expression like
sqrt(b*b - 4*a*c)

we can choose to 'chunk it out' in a variety of ways:
1.
t = b*b - 4*a*c
sqrt(t)

2.
t1 = b*b
t2 = 4*a*c
t3 = t1 -t2
sqrt(t3)

Or the original: sqrt(b*b - 4*a*c)

The assembly language programmer has no such choice. He has to write
(the equivalent of)
t1 = b*b
t2 = 4*a
t3 = t2*c
t4 = t1 - t3
sqrt(t4)

Compared to a comprehension, a loop is in assembly-language category
since the list being 'comprehended' is compelled to be named.

And BTW thanks for the singleton list trick:
Sure 'v=e' is way neater than 'for v in [e]' but the for is better than nothing