Inline assignments
Fredrik Tolf
I'm relatively new to Python, and one thing I can't seem to get over is
the lack of in-expression assignments, as present in many other
languages. I'd really like to know how Python regulars solve the
problems posed by that.
For example, I recently wanted to do this:
if callable(f = getattr(self, "cmd_" + name)):
# Do something with f
elif callable(f = getattr(self, "cmdv_" + name)):
# Do something else with f
However, since I can't do that in Python, I ended up using an extra
local variable instead, like this:
f = getattr(self, "cmd_" + name)
f2 = getattr(self, "cmdv_" + name)
if callable(f):
# Do something with f
elif callable(f2):
# Do something with f2
Another common problem for me are while loops. I would often like to do
this:
while (pos = somestring.find("/")) != -1:
part = somestring[:pos]
somestring = somestring[pos + 1:]
# Handle part
However, that, too, is impossible, and I end up doing something like
this:
while True:
pos = somestring.find("/")
if pos == -1: break
# ...
Which is quite ugly. This might have been a bad example, since
somestring.split() could be used instead, but it's still valid for other
situations.
How would you go about these situations?
Thanks for your time!
Fredrik Tolf
Peter Hansen
> I'm relatively new to Python, and one thing I can't seem to get over is
> the lack of in-expression assignments, as present in many other
> languages. I'd really like to know how Python regulars solve the
> problems posed by that.
>
> For example, I recently wanted to do this:
>
> if callable(f = getattr(self, "cmd_" + name)):
> # Do something with f
> elif callable(f = getattr(self, "cmdv_" + name)):
> # Do something else with f
for prefix in ('cmd_', 'cmdv_'):
f = getattr(self, prefix + name)
if callable(f):
# do something with f
(and note that the "if callable(f)" test is probably not required
anyway, though that depends on the specific case)
If the "do something" and "do something else" parts are really doing
different things, then you should put that code into other methods and
include references to those methods in the list along with the prefixes,
so you can keep the code free of the extra layer of logic. A more real
example would be useful in continuing the discussion down that path.
(At the moment, the answer to that looks like it would be "since you are
already looking up a callable, just call it".)
> Another common problem for me are while loops. I would often like to do
> this:
> while (pos = somestring.find("/")) != -1:
> part = somestring[:pos]
> somestring = somestring[pos + 1:]
> # Handle part
>
> However, that, too, is impossible, and I end up doing something like
> this:
> while True:
> pos = somestring.find("/")
> if pos == -1: break
> # ...
>
> Which is quite ugly.
Ugly is very much in the eye of the beholder. In this case, with an eye
to readability and maintainability, I find the latter much more direct
than the former, which blends a search, and assignment, a comparison,
and a conditional control structure all into a single line, making it
very hard to just scan the code quickly and know for sure what is happening.
> This might have been a bad example, since
> somestring.split() could be used instead, but it's still valid for other
> situations.
And each of those other situations usually has an equally
straightforward solution. :-)
-Peter
Duncan Booth
> However, since I can't do that in Python, I ended up using an extra
> local variable instead, like this:
>
> f = getattr(self, "cmd_" + name)
> f2 = getattr(self, "cmdv_" + name)
> if callable(f):
> # Do something with f
> elif callable(f2):
> # Do something with f2
If 'do something' is the same thing:
for prefix in ('cmd_', 'cmdv_'):
f = getattr(self, prefix+name)
if callable(f):
# do something with f
break
If 'do something' is different each time, put each block into a method:
def do_cmd(self, f): ...
def do_cmdv(self, f): ...
...
for prefix, action in (('cmd_', self.do_cmd), ('cmdv_', self.do_cmdv)):
f = getattr(self, prefix+name)
if callable(f):
action(f)
break
>
> Another common problem for me are while loops. I would often like to do
> this:
> while (pos = somestring.find("/")) != -1:
> part = somestring[:pos]
> somestring = somestring[pos + 1:]
> # Handle part
for part in somestring.split("/")[:-1]:
# handle part
Are you sure you didn't want to process the last part of the string as
well? I would have thought that to be more common, and rather harder to
write using your original structure.
>
> Which is quite ugly. This might have been a bad example, since
> somestring.split() could be used instead, but it's still valid for other
> situations.
>
Not really. Your original loop refactors quite nicely by calling a method
or function which returns a sequence of results. The same pattern in will
always be refactorable in the same way. If the appropriate 'split' function
doesn't already exist then you can write it.
A very common way to rewrite this sort of loop these days is to write a
generator. It is usually beneficial to factor out the complex part of the
loop logic in this way as then you only have to write it once no matter how
many loops you have with the same structure.
Alex Martelli
...
> I'm relatively new to Python, and one thing I can't seem to get over is
> the lack of in-expression assignments, as present in many other
> languages. I'd really like to know how Python regulars solve the
> problems posed by that.
In general, we've learned to do without and structure our code
differently -- less compactly, more "airy". Python is not about
maximally compact code -- other languages of roughly equivalent power,
such as Perl, are more suitable if that's your goal; in Python, we aim
for less compactness and more readability (and maintainability). E.g.:
> if callable(f = getattr(self, "cmd_" + name)):
> # Do something with f
> elif callable(f = getattr(self, "cmdv_" + name)):
> # Do something else with f
might become:
f = getattr(self, "cmd_" + name)
if callable(f):
# Do something with f
else:
f = getattr(self, "cmdv_" + name)
if callable(f):
# Do something else with f
7 lines instead of 4 (if the comments stand for one line each), but
exactly the same semantics.
> Another common problem for me are while loops. I would often like to do
That's part of why Python often prefers for loops -- with a generator to
handle the "looping logic" part, if needed. For example,
> while (pos = somestring.find("/")) != -1:
> part = somestring[:pos]
> somestring = somestring[pos + 1:]
> # Handle part
might often become a simple:
for part in somestring.split('/'):
# handle part
but that has slightly different semantics (handles the part after the
last occurrence of / too, doesn't explicitly record pos, ...) which
might be adjusted if needed (e.g. with a [:-1] after the call to split
and before the colon to omit the last part). If the adjustment is
"major", e.g. you do need every variable to be set just like so in the
body, then a generator can help -- with the focus being on a separation
between the business logic (the "handle part" processing) and the
plumbing (generate all 'parts' in sequence), and the latter going in the
generator, not distracting from the handling as may happen when you mix
the two things. The business-logic part will always look like:
for pos, part in splitter(somestring, '/'):
# handle part, using pos as well
and the splitter generator is written elsewhere, a la
def splitter(somestring, separ):
while True:
pos = somestring.find(separ)
if pos == -1: break
yield pos, somestring[:pos]
somestring = somestring[pos+1:]
with the big advantage that you may change the implementation to be more
efficient, if and when needed, without touching the business logic part
which remains nestled in the nice, simple, readable for loop.
for example:
def splitter(somestring, separ):
pos = 0
while True:
nextpos = somestring.find(separ, pos)
if nextpos == -1: break
yield nextpos-pos, somestring[pos:nextpos]
pos = nextpos+1
Same semantics, less busywork, so presumably faster (the timeit module
exists to let you check the speed of code fragments, of course).
All this being said, I do sometimes find myself coding a first version
of some algorithm by following as closely as possible some published
reference code or pseudocode, before enhancing it to be clearer, more
efficient, more Pythonic, or whatever. When that happens, some syntax
sugar to "assign and test in the same expression" does come in handy, to
enable coding of a first version that's closer in spirit to the
non-Python reference version.
It was, of course, no trouble at all to make myself an "assign and test
in the same expression" snippet, and I (again, of course) published it
in the Python Cookbook (both online and printed editions) for others'
benefit. For example:
class Holder(object):
def set(self, value):
self.value = value
return value
data = Holder()
Now, with this tiny piece of "plumbing adapter", you can code, e.g.:
while data.set(somestring.find("/")) != -1:
part = somestring[:data.value]
somestring = somestring[data.value + 1:]
# Handle part
Another use case for this Holder class is when you're using Python to
prototype something that you already know will need to be recoded in
another given language once it's ready -- the motivation is similar to
the one for the main use case of "code as close as possible to a
published reference implementation", namely enabling Python code that is
closer to what's idiomatic in some other language than to idiomatic
Python itself.
Therein, of course, lies the danger -- over-relying on this adapter may
mean you never really learn to "think the Python way" and thus to code
idiomatic Python, even when that would be most appropriate (i.e., in
most cases). Still, Python _is_ a "consenting adults" language, so I
think it's appropriate to show how its power can easily let you keep
using idioms coming from different languages... even when that usage is
not the optimal choice that you could and should make, it's up to you.
Alex
Steven D'Aprano
> Hi list!
>
> I'm relatively new to Python, and one thing I can't seem to get over is
> the lack of in-expression assignments, as present in many other
> languages. I'd really like to know how Python regulars solve the
> problems posed by that.
It is true that there is a whole class of bugs which Python code can't
duplicate because of the lack of in-expression assignments. Somehow
Python programmers manage to muddle along.
> For example, I recently wanted to do this:
>
> if callable(f = getattr(self, "cmd_" + name)):
> # Do something with f
> elif callable(f = getattr(self, "cmdv_" + name)):
> # Do something else with f
>
> However, since I can't do that in Python, I ended up using an extra
> local variable instead, like this:
>
> f = getattr(self, "cmd_" + name)
> f2 = getattr(self, "cmdv_" + name)
> if callable(f):
> # Do something with f
> elif callable(f2):
> # Do something with f2
Here is one way:
f = getattr(self, "cmd_" + name)
if callable(f):
# Do something with f
else:
f = getattr(self, "cmdv_" + name)
if callable(f):
# do something with f
Here is another:
def handle_func(*fs):
"""Do something with the first callable argument."""
for f in fs:
if callable(f):
# do something with f
break
L = [getattr(self, s + name) for s in ("cmd_", "cmdv_")]
handle_func(L)
Here is a third:
L = map(lambda s, name=name: s+name, ("cmd_", "cmdv_"))
L = map(lambda s, me=self: getattr(me, s), L)
L = filter(callable, L)
# do something with L[0]
Or if you are really nuts, you can convert that last one to a one-liner:
# do something with:-
filter(callable, map(lambda s, me=self: getattr(me, s), map(lambda s,
name=name: s+name, ("cmd_", "cmdv_"))))[0]
Or possibly think about using a different algorithm, maybe something like
this:
class Dispatcher:
def f1(self): pass
def f2(self): pass
def __init__(self):
self.fmap = {"cmd_name": self.f1, "cmdv_name": self.f2}
def handle_func(self, s="name"):
try:
f = self.fmap["cmd_"+s]
except KeyError:
f = self.fmap["cmdv_"+s]
f() # or do something with f
Remember that functions and methods are first class objects, you may not
need to pass strings around, then convert the string to a function. Just
pass the function directly.
> Another common problem for me are while loops. I would often like to do
> this:
> while (pos = somestring.find("/")) != -1:
> part = somestring[:pos]
> somestring = somestring[pos + 1:]
> # Handle part
If you are splitting pathnames, you may be able to use os.path instead of
reinventing the wheel.
> However, that, too, is impossible, and I end up doing something like
> this:
> while True:
> pos = somestring.find("/")
> if pos == -1: break
> # ...
>
> Which is quite ugly. This might have been a bad example, since
> somestring.split() could be used instead, but it's still valid for other
> situations.
Such as?
Here is another way:
pos = somestring.find("/")
while pos != -1:
part = somestring[:pos]
somestring = somestring[pos + 1:]
# handle part
And another:
while True:
try:
pos = somestring.index("/")
except ValueError:
break
part = somestring[:pos]
somestring = somestring[pos + 1:]
# handle part
And a third:
start = 0
while True:
try:
pos = somestring.index("/", start)
except ValueError:
break
# handle somestring[start:pos]
start = pos + 1
If you are coming from a Java background, you may consider exceptions to
be a lot of work. In Python, setting up a try...except block is very
lightweight, so long as exceptions are rare. Hitting the except clause is
more work, so you should avoid that idiom if you expect the try to raise
an exception most of the time.
--
Steven.
jil...@gmail.com
find the latter two examples (i.e. use try/except) to be horrible
solutions. It's not a matter of light/heavy weight, it's a matter of
using exceptions for normal loop flow control is a really bad idea.
1) I think it's less clear why the loop works
2) The possibility (in more complicated examples) that this could mask
a real exception (if for example, your function
I recommend reading Effective Java (Bloch), specifically the beginning
of the exceptions chapter, for more details why using exceptions to
exit loops is a bad thing (and while the title is effective java, much
of it is directly applicable to a lot of python code)
Duncan Booth
> Coming from a background that exposed me to far too many languages, I
> find the latter two examples (i.e. use try/except) to be horrible
> solutions. It's not a matter of light/heavy weight, it's a matter of
> using exceptions for normal loop flow control is a really bad idea.
> 1) I think it's less clear why the loop works
> 2) The possibility (in more complicated examples) that this could mask
> a real exception (if for example, your function
I'm waiting for the end of that sentence...
>
> I recommend reading Effective Java (Bloch), specifically the beginning
> of the exceptions chapter, for more details why using exceptions to
> exit loops is a bad thing (and while the title is effective java, much
> of it is directly applicable to a lot of python code)
Can you paraphrase some of the arguments for those of us not fortunate
enough to have a copy of the book? I've heard a lot of arguments against
using exceptions, and most of those I have heard don't apply to Python.
Like it or not, Python uses exceptions for normal loop flow control. That's
a fact of life, live with it: every normal termination of a for loop is an
exception. Real exceptions don't get masked: for loops terminate with
StopIteration, and they only catch StopIteration, so any other exception is
propagated.
Alex Martelli
...
> I recommend reading Effective Java (Bloch), specifically the beginning
> of the exceptions chapter, for more details why using exceptions to
> exit loops is a bad thing (and while the title is effective java, much
> of it is directly applicable to a lot of python code)
I recommend instead using Python books, rather than Java ones, to
explore optimal Python style. Josh is a colleague of mine (as is Matt
Austern, if you're looking for a C++ guru), and his book is part of the
"canon" (books handed out to incoming new engineers) just as much as
mine, but we have three SEPARATE style guidelines for the key three
languages we use -- Java, Python, C++.
Many of us routinely code in two of the three languages (a few code in
all three) and we have no problem "switching gears" to use each language
in the way that best suits that SPECIFIC language, rather than futilely
trying to "code Java in Python" or any of the other 5 wrong ways;-).
Alex
Jorge Godoy
> wrote:
>
>> Coming from a background that exposed me to far too many languages, I
>> find the latter two examples (i.e. use try/except) to be horrible
>> solutions. It's not a matter of light/heavy weight, it's a matter of
>> using exceptions for normal loop flow control is a really bad idea.
>> 1) I think it's less clear why the loop works
>> 2) The possibility (in more complicated examples) that this could mask
>> a real exception (if for example, your function
>
> I'm waiting for the end of that sentence...
While in the loop to mount the list, there was an exception thrown. I mean
raised. :-) ;-)
--
Jorge Godoy <go...@ieee.org>
"Quidquid latine dictum sit, altum sonatur."
- Qualquer coisa dita em latim soa profundo.
- Anything said in Latin sounds smart.
Steven D'Aprano
> Like it or not, Python uses exceptions for normal loop flow control. That's
> a fact of life, live with it: every normal termination of a for loop is an
> exception. Real exceptions don't get masked: for loops terminate with
> StopIteration, and they only catch StopIteration, so any other exception is
> propagated.
In fairness, that is under the hood: the programmer doesn't (normally)
have to deal with the StopIteration itself.
The argument against exceptions is just a disguised argument against GOTO.
It fails to distinguish between bad ways to jump to another part of code
and good ways.
Bad ways include GOTO itself and the awful COME FROM construct.
Good ways include IF, loops of all sorts, function calls, break/continue
from inside a loop, and exceptions.
Of course exceptions can be abused, but then so can any piece of code.
Try...except blocks no more allow you to jump to arbitrary places in your
code than do if...else blocks.
As for using exceptions as signals, there is a reason they are called
EXCEPTION rather than ERROR: they don't necessarily represent errors, and
should not be treated that way. They represent exceptional states.
Exceptional states can be errors, they can signal the end of processing,
or they can signal expected states.
--
Steven.