brave?

[Devin Venable]

I did what I suspect was a fairly brave thing at PyCon.  I attended an Open Spaced entitled "Is This Pythonic?"  No one else was willing to put their code in front of the Python core developers (Jack Diederich and Moshe Zadka), so I offered up on of the files from my PyShards project.  Now, keep in mind that I only decided that Python would be my language of Primacy about 6 months before writing the file that they reviewed.  I tend to use the common subset of programming language features so that I can hop back and forth between programming languages without making my brain explode.   Nonetheless, I wanted to see what it would take to make my code acceptable to the most pedantic of Python programmers, so I volunteered. 

Check out the result:

http://jackdied.blogspot.com/2009/03/is-this-pythonic.html

Jack could not pythonicify the code to his satisfaction without relying on newer and trunk Python features.  At the time of his writing, he was still not satisfied with his result.  

It was an interesting, if not a bit nerve racking, exercise.

 

[Joseph A Holsten]

Wonderful! I'm a complete amateur in python, but this reminds me of
how functional python is. It just puts me in the mood for lisp. So I
thought I'd show the lispy way of handing your code. It's probably
impossible to understand unless you've got an unhealthy fondness for
recursion. It's also possible that it will blow up your stack, cause
the garbage collector to freeze and induce code maintainers to kill you.

def apply_one(func, shard):
''' take a shard, connect to the database, create a cursor, and
pass it to func '''
db = shard.establishConnection()
try:
cursor = db.cursor()
yield func(cursor)
finally:
db.close()

# func first to mimic the parameter order of map()
def apply_all(func, shards):
''' for each shard connect to the database, create a cursor, and
pass it to func '''
return map(lambda shard: apply_one(func, shard), shards)

def selectOneFromShards(shards, sql, args):
''' execute sql on each shard, returning the first row (if any) on
each shard'''
def fetch(cursor):
curser.fetchone(sql, args)
return filter(None, apply_all(fetch, shards))

def selectManyFromShards(shards, sql, args=None, size=None):
''' execute sql on each shard, collecting the rows from each shard,
up to size rows total'''
def fetch(cursor, stillToFetch):
''' take a cursor and fetch its rows, returning no more than
stillToFetch '''
cursor.execute(sql, args)
return cursor.fetchmany(stillToFetch)

def selector(state, shard):
''' apply a shard to the current state '''
if state["size"] is None or state["size"] < 1
# do nothing to the current state
return state
else
# apply a fetch to the shard using the size of the current state
results = apply_one(lambda cursor: fetch(cursor,
state["size"]), shard)
# create an updated state
return {"size": state["size"] - len(results), "results":
state["results"].extend(results)}

# create an initial state from the the size with empty results
initial_state = {"size": size, "results": []}
# reduce each shard into the current state
final_state = reduce(selector, shards, initial_state)
return filter(None, final_state["results"])

# helper functions to handle linked lists
def selectOne(self, sql, args):
return selectOneFromShards(valid_shards(self._shard), sql, args)

def selectMany(self, sql, args=None, size=None):
return selectManyFromShards(valid_shards(self._shard), sql, args,
size)

Also, did that blog post introduce a bug into selectMany? I thought
you had to call cursor.execute(sql, args) and cursor.fetchmany(size),
but it seems to just use cursor.fetchmany(sql, args). Wierd.

[Devin Venable]

The ability to understand the code you are reading is the one issue I have with DRY principal when taken to the extreme.  It's like optimization: it's good for your code but often makes it more difficult to read.  Call it "stylistic optimization".  The code on Jack's blog is extremely pythonic and I definitely want to refactor to incorporate most of the changes.  But I think it could be argued that the original code is just more obvious.  But instead of making the point here, I'll make it on his blog later today.

Devin