How to write verbose scripts

1 view
Skip to first unread message

Steven D'Aprano

unread,
Sep 2, 2008, 12:55:53 PM9/2/08
to
I find myself writing command line tools in Python where I wish to
include "verbose" output to stdout.

I start with a helper function:


def print_(obj, level=0):
if _verbosity >= level:
print obj


And then I end up with functions or methods looking like this:


def parrot(x)
print_("precondition", level=2)
do_something()
print_("status is good...", level=1)
print_("parrot is squawking strongly now", level=2)
do_something_else()
print_("squawk squawk squawk", level=3)
do_more()
print_("postcondition", level=1)
return something


That often means that my functions end up with more message printing code
than actual code. The whole thing seems messy and hard to manage for all
but the smallest scripts.

Worst of all, sometimes the messages I wish to print may be expensive to
compute, and I don't want to waste time computing them if they aren't
going to be printed because the verbosity is too low. But nor do I wish
to fill my code with this:

if _verbosity >= 3:
x = calculate_complicated_thing()
print_(x, level=3)

Is there a better way of doing this than the way I am going about it?

--
Steven

Joe Riopel

unread,
Sep 2, 2008, 1:07:33 PM9/2/08
to Steven D'Aprano, pytho...@python.org
On Tue, Sep 2, 2008 at 12:55 PM, Steven D'Aprano
<st...@remove-this-cybersource.com.au> wrote:
> Is there a better way of doing this than the way I am going about it?

Would the logging module help, and just print the output to the stdout
(or a file) instead?

Mensanator

unread,
Sep 2, 2008, 1:52:58 PM9/2/08
to
On Sep 2, 11:55 am, Steven D'Aprano <st...@REMOVE-THIS-

I do something like this, although I don't know if it would
be an improvement.

def collatz(a,p):
""" 3x+1 sequencer, no loop detection

collatz(a,p)
a: starting value
p: print options (binary)
bit 0 print even numbers (turns off power division)
bit 1 print odd numbers
bit 2 print debug (if any)
returns: CollatzSequenceParameters [R1count, R2count]
"""
ONE = gmpy.mpz(1)
TWO = gmpy.mpz(2)
TWE = gmpy.mpz(3)
a = gmpy.mpz(a)
t = 0
u = 0
done = 0

if (p & 1)==1:
print_evens = True
else:
print_evens = False
if (p & 2)==2:
print_odds = True
else:
print_odds = False
if (p & 4)==4:
print_debug = True
else:
print_debug = False

while done==0:
f = gmpy.scan1(a,0) # locate LS 1-bit
if f>0: # it's even
if print_evens:
print a,
a = a >> 1 # no power division
u += 1
else:
a = a >> f # power division
u += f
else:
if print_odds:
print a,
if a==1:
done = 1
seq_end = t + u
else:
a = a*TWE + ONE
t += 1
return [u,t]

>
> --
> Steven

Diez B. Roggisch

unread,
Sep 2, 2008, 3:06:58 PM9/2/08
to
Steven D'Aprano schrieb:

I use the logging-module.

Regarding the expensive computations: maysomething like this help:

class DeferredToString(object):

def __init__(self, func):
self.func = func

def __repr__(self):
return repr(self.func())

def __str__(self):
return str(self.func())

dts = DeferredToString

Because then you can do

logger.debug("Some text for an: %r", dts(lambda: long_computation()))

Because AFAIK the string is only interpolated if the logging level is
actually put out.

Diez

MRAB

unread,
Sep 2, 2008, 4:53:27 PM9/2/08
to
On Sep 2, 5:55 pm, Steven D'Aprano <st...@REMOVE-THIS-
How about:

def print_(obj, level=0):
if _verbosity >= level:

if callable(obj):
obj = obj()
print obj

so that you could then use:

print_("precondition", level=2)

and:

print_(calculate_complicated_thing, level=3)

or:

print_(lambda: calculate_complicated_thing(argument), level=3)

John Machin

unread,
Sep 2, 2008, 5:14:47 PM9/2/08
to
On Sep 3, 3:52 am, Mensanator <mensana...@aol.com> wrote:
> On Sep 2, 11:55 am, Steven D'Aprano <st...@REMOVE-THIS-
>
> if (p & 1)==1:
> print_evens = True
> else:
> print_evens = False
> if (p & 2)==2:
> print_odds = True
> else:
> print_odds = False
> if (p & 4)==4:
> print_debug = True
> else:
> print_debug = False

No, no, no, you've taken "How to write verbose scripts" rather too
literally; try this:

print_evens = p & 1
print_odds = p & 2
print_debug = p & 4

Mensanator

unread,
Sep 2, 2008, 6:07:11 PM9/2/08
to

Thanks for that. Luckily, the code only does that once per call.

Hendrik van Rooyen

unread,
Sep 3, 2008, 4:34:50 PM9/3/08
to pytho...@python.org

Steven D'Aprano <stev...bersource.com.au> wrote:

>Is there a better way of doing this than the way I am going about it?

Not sure if its "better", but I would keep the messages in a table or dict and
have different tables or dicts for different levels of verbosity, and write a
displayer that knows about the verbosity - This approach has the advantage that
you can write in English and have different verbosities in different languages
too, if the displayer knows about the language.

Verbosity level and language can be display class attributes, and then you can
simply write something like:

display.display("squawk")

and get one of:

squawk
squawk, squawk, squawk
squawk - help the bastards are trying to nail me to my perch!

or the equivalent in Icelandic or Norwegian.

It works, but its a PITA to implement because you have to formally construct the
stuff, and keep them all in sync, across languages and levels of verbosity.

This of course will only work for static messages - If you compute what you
display, I guess you need to either keep the computation in the displayer, or
pass it in, or something - gets messy.

Keeps the code clear though, as it splits what you display away from the
decision of saying something.

I try to avoid it if I can.

- Hendrik

Bruno Desthuilliers

unread,
Sep 3, 2008, 7:14:59 AM9/3/08
to
John Machin a écrit :

> On Sep 3, 3:52 am, Mensanator <mensana...@aol.com> wrote:
>> On Sep 2, 11:55 am, Steven D'Aprano <st...@REMOVE-THIS-
>>
>> if (p & 1)==1:
>> print_evens = True
>> else:
>> print_evens = False
>> if (p & 2)==2:
>> print_odds = True
>> else:
>> print_odds = False
>> if (p & 4)==4:
>> print_debug = True
>> else:
>> print_debug = False
>
> No, no, no, you've taken "How to write verbose scripts" rather too
> literally;

KEYBOARD !

Uwe Schmitt

unread,
Sep 3, 2008, 7:31:52 AM9/3/08
to
On 2 Sep., 18:55, Steven D'Aprano <st...@REMOVE-THIS-

You can save some code if you use function decorators for logging
input and output values of
functions.
So write lots of functions containing one statement and your problem
is solved ;-)

Greetings, Uwe

BJörn Lindqvist

unread,
Sep 3, 2008, 8:50:57 AM9/3/08
to Hendrik van Rooyen, pytho...@python.org
2008/9/3 Hendrik van Rooyen <ma...@microcorp.co.za>:

>
> Steven D'Aprano <stev...bersource.com.au> wrote:
>
>>Is there a better way of doing this than the way I am going about it?
>
> Not sure if its "better", but I would keep the messages in a table or dict and
> have different tables or dicts for different levels of verbosity, and write a
> displayer that knows about the verbosity - This approach has the advantage that
> you can write in English and have different verbosities in different languages
> too, if the displayer knows about the language.

One big downside with that approach is that it becomes much harder to
grep for the log message. Usually when I go through logs, I don't care
what exactly the message says, just that it is easily googleable (uses
some kind of identifying text) and that it is unique so I can know
exactly where it was emitted.


--
mvh Björn

Steven D'Aprano

unread,
Sep 4, 2008, 5:24:28 AM9/4/08
to

Thank you to everyone who answered.

As I feared, it seems that there's no really simple way of dealing with
arbitrary messages at arbitrary parts of my code.

I'm currently playing around with a few ideas, one of which is a
decorator to print information about function calls. However, what I'd
like to avoid is having to explicitly call the decorator on every method.
I guess that there's metaclass trickery I can play. Does anyone have any
pointers on how I can apply a decorator to every method in a class
automatically?

--
Steven

Ben Finney

unread,
Sep 4, 2008, 9:10:32 AM9/4/08
to
Steven D'Aprano <st...@REMOVE-THIS-cybersource.com.au> writes:

> On Tue, 02 Sep 2008 13:07:33 -0400, Joe Riopel wrote:
>
> > On Tue, Sep 2, 2008 at 12:55 PM, Steven D'Aprano
> > <st...@remove-this-cybersource.com.au> wrote:
> >> Is there a better way of doing this than the way I am going about it?
> >
> > Would the logging module help, and just print the output to the stdout
> > (or a file) instead?
>
> Thank you to everyone who answered.
>
> As I feared, it seems that there's no really simple way of dealing with
> arbitrary messages at arbitrary parts of my code.

I would think the 'logging' module *is* the simple way to do this. At
least, it's as simple as it could be without leading to massive
re-visiting of the "arbitrary parts of one's code" when later desiring
to change the way the messages are handled.

--
\ “We must become the change we want to see.” —Mahatma Gandhi |
`\ |
_o__) |
Ben Finney

Lie

unread,
Sep 6, 2008, 5:29:35 PM9/6/08
to

That would be perfect if python is a fully functional language (read
that as: purely functional language). Functional language dictates no
side effects, so logging input and output of functions is pretty much
everything needed to do debugging (which is the main reason for having
verbose mode). It would still be feasible, though, if you keep
yourself from using functions with side-effects (or use as few of them
as possible).

Bjorn Lindqvist says:
> One big downside with that approach is that it becomes much harder to
> grep for the log message. Usually when I go through logs, I don't care
> what exactly the message says, just that it is easily googleable (uses
> some kind of identifying text) and that it is unique so I can know
> exactly where it was emitted.

Actually, I'd prefer a log code (perhaps also as an option, to use -e
for showing up log code besides the log message). The log code would
be unique and referenced only once in the whole application (to nail
down who said what to a single point). The code would make a call to
the log printer with the log code (not the error string) and a few
arguments to be interpolated to the error string (taken from a
dictionary indexed by error code).

The downside is loss of inline documentation by the logging codes.

Reply all
Reply to author
Forward
0 new messages