[Python-ideas] "try with" syntactic sugar
Daniel Stutzbach
try:
with something as f:
many lines of code
except some_error:
handle error
The "with" statement is great, but it results in the bulk of the code being indented twice. I'd like to propose a little syntactic sugar, the "try with":
try with something as f:
many lines of code
except some_error:
handle error
It saves one line of vertical space, and gets rid of an indentation level for the bulk of the code that rests within the "with" statement. Thoughts?
Here's a short script to count how many uses of "with" within your code are immediately preceded by "try":
#!/usr/bin/python
import re, sys
re_with = re.compile(r'(try:[ \t]*)?[\r\n]+[ \t]+with ')
try_with = 0
total = 0
for fname in sys.argv[1:]:
data = open(fname).read()
for match in re_with.findall(data):
if match: try_with += 1
total += 1
print 'try-with:', try_with, 'out of:', total, '(', try_with*100.0/total,'%)'
Usage:
Cashew:~$ /tmp/count_try_with.py *.py
try-with: 17 out of: 25 ( 68.0 %)
--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC
Calvin Spealman
But... Do we also add try for:, try if:, try while:, etc.? Not
terrible, but potentially terrible.
> _______________________________________________
> Python-ideas mailing list
> Python...@python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
>
--
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy
_______________________________________________
Python-ideas mailing list
Python...@python.org
http://mail.python.org/mailman/listinfo/python-ideas
Daniel Stutzbach
But... Do we also add try for:, try if:, try while:, etc.? Not
terrible, but potentially terrible.
"with" blocks very frequently define the scope of an object that might generate a particular type of exception (e.g., file object may throw IOErrors, SQL transactions might through SQL exceptions, etc.).
Eric Smith
...
> The "with" statement is great, but it results in the bulk of the code
> being indented twice. I'd like to propose a little syntactic sugar, the
> "try with":
>
> try with something as f:
> many lines of code
> except some_error:
> handle error
Not commenting on how useful the proposal is (I'm mostly stuck in Python
2.4), but couldn't this be:
with something as f:
many lines of code
except some_error:
handle error
That is, you don't really need the "try", since the above syntax is
currently illegal?
Eric.
Alec Thomas
> Daniel Stutzbach wrote:
> ...
>>
>> The "with" statement is great, but it results in the bulk of the code
>> being indented twice. I'd like to propose a little syntactic sugar, the
>> "try with":
>>
>> try with something as f:
>> many lines of code
>> except some_error:
>> handle error
>
> Not commenting on how useful the proposal is (I'm mostly stuck in Python
> 2.4), but couldn't this be:
>
> with something as f:
> many lines of code
> except some_error:
> handle error
+1
Like the OP, I find myself doing this same sequence of code over and
over. The syntax you describe here would clean the code up
considerably.
--
"Life? Don't talk to me about life." - Marvin
Cesare Di Mauro
> 2009/2/27 Eric Smith <er...@trueblade.com>:
>> Daniel Stutzbach wrote:
>> ...
>>>
>>> The "with" statement is great, but it results in the bulk of the code
>>> being indented twice. I'd like to propose a little syntactic sugar, the
>>> "try with":
>>>
>>> try with something as f:
>>> many lines of code
>>> except some_error:
>>> handle error
>>
>> Not commenting on how useful the proposal is (I'm mostly stuck in Python
>> 2.4), but couldn't this be:
>>
>> with something as f:
>> many lines of code
>> except some_error:
>> handle error
>
> +1
>
> Like the OP, I find myself doing this same sequence of code over and
> over. The syntax you describe here would clean the code up
> considerably.
+1 Same for me. It avoids an additional try: except: surrounding the with.
Daniel Stutzbach
re_with = re.compile(r'(try:[ \t]*)?[\r\n]+[ \t]+with ')
re_with = re.compile(r'(try:[ \t]*)?[\r\n]+[ \t]+with .*:')
Bruce Frederiksen
> Here's an updated regular expression that does a better job of
> ignoring comments and strings:
>
> re_with = re.compile(r'(try:[ \t]*)?[\r\n]+[ \t]+with .*:')
#!/usr/bin/python
import re, sys
re_with = re.compile(r'''
^(?P<indent>\s*) # capture the indent
try:
(?:[ \t]*(?:\#.*)?[\r\n]+)? # newlines (ignoring
comments)
(?P<with>
(?P=indent)[ \t]+ # at higher indent level
with\s (?:[^#:]*(?:(?:\#.*)?[\r\n]+)?)*: # with .*: (ignoring comments
# and newlines before :)
)? # end (?P<with>)
''', re.MULTILINE | re.VERBOSE)
try_with = 0
total = 0
for fname in sys.argv[1:]:
data = open(fname).read()
for match in re_with.finditer(data):
if match.group('with'): try_with += 1
total += 1
print 'try-with:', try_with, 'out of:', total, '(',
try_with*100.0/total,'%)'
When I run this on a project of mine, I get:
try-with: 1 out of: 87 ( 1.14942528736 %)
The pattern that I find myself using is with/for, which can be counted
by this program:
#!/usr/bin/python
import re, sys
re_with = re.compile(r'''
^(?P<indent>\s*) # capture the indent
with\s (?:[^#:]*(?:(?:\#.*)?[\r\n]+)?)*: # with .*: (ignoring comments
# and newlines before :)
(?:[ \t]*(?:\#.*)?[\r\n]+)? # newlines (ignoring comments)
(?P<for>
(?P=indent)[ \t]+ # at higher indent level
for\s (?:[^#:]*(?:(?:\#.*)?[\r\n]+)?)*: # for .*: (ignoring comments
# and newlines before :)
)? # end (?P<for>)
''', re.MULTILINE | re.VERBOSE)
with_for = 0
total = 0
for fname in sys.argv[1:]:
data = open(fname).read()
for match in re_with.finditer(data):
if match.group('for'): with_for += 1
total += 1
print 'with-for:', with_for, 'out of:', total, '(',
with_for*100.0/total,'%)'
On my code, I get:
with-for: 38 out of: 47 ( 80.8510638298 %)
A few days ago, I proposed that a __throw__ method be added to context
managers so that context managers could be used to capture common
try/except usage patterns along with it's current ability to capture
common try/finally patterns.
I am curious whether you see common try/except patterns that could be
captured in a context manager so that you could simply write:
with my_except_handling(something) as x:
many lines of code
rather than:
try:
with something as f:
many lines of code
except some_error:
handle error
and be able to replace several occurrences of try/except/handle error
with the same my_except_handling context manager?
-bruce frederiksen
Daniel Stutzbach
I bow to your superior regular expression knowledge. :) However, your version counts the number of "try"s that are followed by "with". Mine counts the number of "with"s that are preceded by "try" (which I think the more relevant metric?). Any chance you could alter your script?
Bruce Frederiksen
> Hi Bruce,
>
> I bow to your superior regular expression knowledge. :) However,
> your version counts the number of "try"s that are followed by "with".
> Mine counts the number of "with"s that are preceded by "try" (which I
> think the more relevant metric?). Any chance you could alter your script?
Sorry, my mistake.
How about this?
#!/usr/bin/python
import re, sys
re_with = re.compile(r'''
^(?P<indent>\s*) # capture the indent
# (might be try, might be with)
(?P<try>
try:
(?:[ \t]*(?:\#.*)?[\r\n]+)? # newlines (ignoring comments)
(?P=indent)[ \t]+ # kick out the indent level
)?
with\s (?:[^#:]*(?:(?:\#.*)?[\r\n]+)?)*: # with .*: (ignoring comments
# and newlines before :)
''', re.MULTILINE | re.VERBOSE)
try_with = 0
total = 0
for fname in sys.argv[1:]:
data = open(fname).read()
for match in re_with.finditer(data):
if match.group('try'): try_with += 1
total += 1
print 'try-with:', try_with, 'out of:', total, '(',
try_with*100.0/total,'%)'
On my code, I now get:
try-with: 1 out of: 47 ( 2.12765957447 %)
On my last response, I mentioned a suggestion to add __throw__ to
context managers. But then I remembered that the __exit__ method is
already given the exception information if an exception is raised. So
you can already do what I was suggesting now.
I'm still curious as to how often you could share try/except cases by
writing your own context managers.
Daniel Stutzbach
How about this?
I tried it, but it matches quite a few comments and strings. Try putting a "print repr(match.group(0))" in the innermost loop to debug it.
On my last response, I mentioned a suggestion to add __throw__ to context managers. But then I remembered that the __exit__ method is already given the exception information if an exception is raised. So you can already do what I was suggesting now.
I'm still curious as to how often you could share try/except cases by writing your own context managers.
Not particularly often. Much of the time, the exception handler has to clean up the mess when a file is unexpectedly unreadable or SQL explodes, and the clean-up bit is tied to the immediately surrounding code. YMMV.
Arnaud Delobelle
> Around two-thirds of the time, whenever I use the wonderful new "with"
> construct, it's enclosed in a "try-except" block, like this:
>
> try:
> with something as f:
> many lines of code
> except some_error:
> handle error
>
> The "with" statement is great, but it results in the bulk of the code being
> indented twice. I'd like to propose a little syntactic sugar, the "try
> with":
>
> try with something as f:
> many lines of code
> except some_error:
> handle error
>
> It saves one line of vertical space, and gets rid of an indentation level
> for the bulk of the code that rests within the "with" statement. Thoughts?
Every compound statement could be made into an implicit try, i.e.
<compound statement>:
<suite1>
except:
<suite2>
would mean:
try:
<compound statement>:
<suite1>
except:
<suite2>
So you could write:
with something as f:
many lines of code
except some_error:
handle error
--
Arnaud
Gabriel Genellina
<daniel-k6jgQluAQ1RL...@public.gmane.org> escribió:
> On Thu, Feb 26, 2009 at 7:28 PM, Bruce Frederiksen
>
>> How about this?
> I tried it, but it matches quite a few comments and strings. Try
I think a r.e. cannot handle this query well enough.
This script uses the tokenize module and should be immune to those false
positives (but it's much slower)
<code>
import sys, os
from tokenize import generate_tokens
from token import NAME
def process(path):
nfiles = nwith = ntrywith = 0
for base, dirs, files in os.walk(path):
print base
print '%d "try+with" out of %d "with" (%.1f%%) in %d files (partial)'
% (
ntrywith, nwith, ntrywith*100.0/nwith if nwith else 0,
nfiles)
print
for fn in files:
if fn[-3:]!='.py': continue
if 'CVS' in dirs: dirs.remove('CVS')
if '.svn' in dirs: dirs.remove('.svn')
fullfn = os.path.join(base, fn)
#print fn
nfiles += 1
with open(fullfn) as f:
try:
was_try = False
for toknum, tokval, _, _, _ in generate_tokens(f.readline):
if toknum==NAME:
is_with = tokval == 'with'
if is_with:
nwith += 1
if was_try: ntrywith += 1
was_try = tokval == 'try'
except Exception, e: print e
print '%d "try+with" out of %d "with" (%.1f%%) in %d files' % (
ntrywith, nwith, ntrywith*100.0/nwith if nwith else 0,
nfiles)
process(sys.argv[1])
</code>
I got 2/25 on my code (1500 files, but many are rather old to use "with")
--
Gabriel Genellina
Christian Heimes
> try with something as f:
> many lines of code
> except some_error:
> handle error
>
> It saves one line of vertical space, and gets rid of an indentation level
> for the bulk of the code that rests within the "with" statement. Thoughts?
Your proposal sounds like a very good idea. +1 from me.
I like to push your proposal even further and add the full set of exept,
else, finally blocks to the with statement. Additionally I'd like to
have multiple arguments, too.
Example
-------
log.info("Starting copy")
with somelock, open(infile, 'r') as fin, open(outfile, 'w') as fout:
fout.write(fin.read())
except Excption:
log.exception("An error has occured")
else:
log.info("Copy successful")
finally:
log.info("All done")
Equivalent
----------
log.info("Starting copy")
try:
with somelock:
with open(infile, 'r') as fin:
with open(outfile, 'w') as fout:
fout.write(fin.read())
except Excption:
log.exception("An error has occured")
else:
log.info("Copy successful")
finally:
log.info("All done")
Christian
Curt Hagenlocher
<arn...@googlemail.com> wrote:
>
> Every compound statement could be made into an implicit try, i.e.
That way lies madness. What distinguishes "with" from other compound
statements is that it's already about resource management in the face
of possible exceptions.
--
Curt Hagenlocher
cu...@hagenlocher.org
Daniel Stutzbach
I think a r.e. cannot handle this query well enough.
This script uses the tokenize module and should be immune to those false
positives (but it's much slower)
#!/usr/bin/python
from __future__ import with_statement
import sys, os
from tokenize import generate_tokens
from token import NAME
nfiles = nwith = ntrywith = 0
for base, dirs, files in os.walk(path):
print '%d "try+with" out of %d "with" (%.1f%%) in %d files (so far)\
'% (ntrywith, nwith, ntrywith*100.0/nwith if nwith else 0, nfiles)
print base
newdirs = []
for d in list(dirs):
if d == 'CVS' or d == '_darcs' or d[0] == '.': continue
newdirs.append(d)
dirs[:] = newdirs
for fn in files:
if fn[-3:]!='.py': continue
#print fn
nfiles += 1
with open(fullfn) as f:
try:
was_try = False
for toknum, tokval, _, _, _ in generate_tokens(f.readline):
if toknum==NAME:
is_with = tokval == 'with'
if is_with:
nwith += 1
if was_try: ntrywith += 1
was_try = tokval == 'try'
except Exception, e: print e
print '%d "try+with" out of %d "with" (%.1f%%) in %d files' % (
ntrywith, nwith, ntrywith*100.0/nwith if nwith else 0,
nfiles)
process(sys.argv[1:])
Georg Brandl
> Daniel Stutzbach wrote
>> try with something as f:
>> many lines of code
>> except some_error:
>> handle error
>>
>> It saves one line of vertical space, and gets rid of an indentation level
>> for the bulk of the code that rests within the "with" statement. Thoughts?
>
> Your proposal sounds like a very good idea. +1 from me.
>
> I like to push your proposal even further and add the full set of exept,
> else, finally blocks to the with statement.
I'd assumed this is already implicit in the proposal.
> Additionally I'd like to have multiple arguments, too.
+1 to that (it should have been there in the beginning).
> Example
> -------
> log.info("Starting copy")
> with somelock, open(infile, 'r') as fin, open(outfile, 'w') as fout:
> fout.write(fin.read())
> except Excption:
> log.exception("An error has occured")
> else:
> log.info("Copy successful")
> finally:
> log.info("All done")
I still like it better with the "try" before the "with".
Georg
--
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.
Scott David Daniels
> <another way to get to possibly relevant code>
It seems to me the question is, among the single-statement try blocks,
what kinds of (non-function call, non-assignment) statements are the
inner statement, and what frequency does this occur.
On Python5, 25, and 30, I get quite similar numbers:
2.5 2.6 3.0
10 15 14 with
15 11 13 yield
25 26 22 class
32 23 *8 print # print is a function in 3.0, called 8 times
37 33 24 while
44 35 *26 exec # exec is a function in 3.0, called 26 times
9 42 41 pass
49 44 64 raise
67 45 29 del
95 61 40 for
142 68 42 if
182 105 63 from
230 163 107 import
588 307 150 return
These numbers are pulled from the output of the attached
imperfect program, and hand-editted together. The numbers
seem to me to say that try-for and try-while substantially
exceed try-with statements.
--Scott David Daniels
@Acm.Org
Guido van Rossum
> On Fri, Feb 27, 2009 at 2:49 AM, Arnaud Delobelle
> <arn...@googlemail.com> wrote:
>>
>> Every compound statement could be made into an implicit try, i.e.
>
> That way lies madness. What distinguishes "with" from other compound
> statements is that it's already about resource management in the face
> of possible exceptions.
Still, a firm -1 from me. Once we have "try try" I'm sure people are
going to clamor for "try if", "try while", "try for", even (oh horror
:-) "try try". I don't think we should complicate the syntax just to
save one level of indentation occasionally.
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
Daniel Stutzbach
On Fri, Feb 27, 2009 at 4:49 AM, Curt Hagenlocher <cu...@hagenlocher.org> wrote:Still, a firm -1 from me. Once we have "try try" I'm sure people are
> That way lies madness. What distinguishes "with" from other compound
> statements is that it's already about resource management in the face
> of possible exceptions.
going to clamor for "try if", "try while", "try for", even (oh horror
:-) "try try". I don't think we should complicate the syntax just to
save one level of indentation occasionally.
Also, "try if", "try while", and "try for" wouldn't work because they already have a meaning for the "else" clause. "with" does not.
Guido van Rossum
Sorry, but my gut keeps telling me that "try with" is not taking the
language into a direction I am comfortable with. Programming language
design is not a rational science. Most reasoning about is is at best
rationalization of gut feelings, and at worst plain wrong. So, sorry,
but I'm going with my gut feelings, so it's still -1. (Or if you wish,
-1000.)
Daniel Stutzbach
Sorry, but my gut keeps telling me that "try with" is not taking the
language into a direction I am comfortable with. Programming language
design is not a rational science. Most reasoning about is is at best
rationalization of gut feelings, and at worst plain wrong. So, sorry,
but I'm going with my gut feelings, so it's still -1. (Or if you wish,
-1000.)
Guido van Rossum
<dan...@stutzbachenterprises.com> wrote:
> On Sun, Mar 1, 2009 at 4:31 PM, Guido van Rossum <gu...@python.org> wrote:
>>
>> Sorry, but my gut keeps telling me that "try with" is not taking the
>> language into a direction I am comfortable with. Programming language
>> design is not a rational science. Most reasoning about is is at best
>> rationalization of gut feelings, and at worst plain wrong. So, sorry,
>> but I'm going with my gut feelings, so it's still -1. (Or if you wish,
>> -1000.)
>
> I thought that might be the case based on your first response, but figured
> I'd give it one more shot. ;-) I respect your opinion. Consider it
> dropped.
Thanks. I've learned the hard way to trust my gut instincts in this
area. If anyone can figure out how to explain my gut's responses to
people wanting rational answers I'd love to hear about it. :-)
spir
Guido van Rossum <gu...@python.org> s'exprima ainsi:
> Programming language
> design is not a rational science. Most reasoning about is is at best
> rationalization of gut feelings, and at worst plain wrong.
!
;-)
Denis
------
la vita e estrany
PS: Time for starting a "Quotes" section at
http://en.wikipedia.org/wiki/Guido_van_Rossum
?
Chris Rebert
> Le Sun, 1 Mar 2009 14:31:58 -0800,
> Guido van Rossum <gu...@python.org> s'exprima ainsi:
>
>> Programming language
>> design is not a rational science. Most reasoning about is is at best
>> rationalization of gut feelings, and at worst plain wrong.
+1 QOTW!
- Chris
--
Shameless self-promotion:
http://blog.rebertia.com
Guido van Rossum
> On Sun, Mar 1, 2009 at 11:16 PM, spir <denis...@free.fr> wrote:
>> Le Sun, 1 Mar 2009 14:31:58 -0800,
>> Guido van Rossum <gu...@python.org> s'exprima ainsi:
>>
>>> Programming language
>>> design is not a rational science. Most reasoning about is is at best
>>> rationalization of gut feelings, and at worst plain wrong.
>
> +1 QOTW!
Let me add that this formulation was at least in part inspired by
reading "How we decide" by Jonah Lehrer.
http://www.amazon.com/How-We-Decide-Jonah-Lehrer/dp/0618620117
--
--Guido van Rossum (home page: http://www.python.org/~guido/)