Using contextlib with Schevo and Durus 3.9

[Etienne Robillard]

Hi Matthew,

I would like to use contextmanager module and the 'with'
statement to open database (file) instances asynchronously.

The reason is largely because I feel lazy to write __enter__ and
__exit__, as required for having a proper contextmanager class.

The code that fails seems not related to opening the database at all.
It just seems that something is wrong when attempting to get the results
from an iterator like Intersection.

Example:

q1 = Q.Match(db.BlogEntry, 'pub_date', '<=', today)
q2 = Q.Match(db.BlogEntry, 'published', '==', True)
results = Q.Intersection(q1, q2)
ctx = dict(entries=sorted(results()))

Traceback (most recent call last):
File
"/usr/local/lib/python2.6/site-packages/PasteScript-1.7.3-py2.6.egg/paste/script/wsgiserver/__init__.py",
line 1174, in communicate
req.respond()
File
"/usr/local/lib/python2.6/site-packages/PasteScript-1.7.3-py2.6.egg/paste/script/wsgiserver/__init__.py",
line 544, in respond
self._respond()
File
"/usr/local/lib/python2.6/site-packages/PasteScript-1.7.3-py2.6.egg/paste/script/wsgiserver/__init__.py",
line 556, in _respond
response = self.wsgi_app(self.environ, self.start_response)
File "/home/steiner/ncvs/Paste/paste/translogger.py", line 68, in __call__
return self.application(environ, replacement_start_response)
File
"/home/steiner/src/notmm/notmm-0.2.13dev/lib/notmm/controllers/base.py",
line 318, in __call__
return self.application(environ, start_response, exc_info)
File
"/home/steiner/src/notmm/notmm-0.2.13dev/lib/notmm/controllers/base.py",
line 287, in application
response = self.getresponse(environ, start_response, exc_info)
File
"/home/steiner/src/notmm/notmm-0.2.13dev/lib/notmm/controllers/base.py",
line 231, in getresponse
response = callback(request, *args, **kwargs)
File
"/home/steiner/src/notmm/notmm-0.2.13dev/lib/notmm/utils/decorators.py",
line 43, in _wrapper
return view_func(ihook, *args, **kwargs)
File "/home/steiner/src/BlogEngine/blogengine/views.py", line 54, in index
ctx = dict(entries=sorted(results()))
File "/home/steiner/ncvs/schevo-3.0/schevo/query.py", line 40, in __call__
return self._results()
File "/home/steiner/ncvs/schevo-3.0/schevo/query.py", line 415, in
_results
s = set(query())
File "/home/steiner/ncvs/schevo-3.0/schevo/query.py", line 330, in
generator
for obj in on:
File "/home/steiner/ncvs/schevo-3.0/schevo/extent.py", line 76, in
__iter__
oids = self._find(self.name, None)
File "/home/steiner/ncvs/schevo-3.0/schevo/database2.py", line 755, in
_find_entity_oids
return list(entity_maps.keys())
File "build/bdist.linux-i686/egg/durus/persistent.py", line 174, in
_p_load_state
self._p_connection.load_state(self)
File "build/bdist.linux-i686/egg/durus/connection.py", line 185, in
load_state
pickle = self.get_stored_pickle(oid)
File "build/bdist.linux-i686/egg/durus/connection.py", line 114, in
get_stored_pickle
record = self.storage.load(oid)
File "build/bdist.linux-i686/egg/durus/file_storage.py", line 61, in load
result = self.shelf.get_value(oid)
File "build/bdist.linux-i686/egg/durus/shelf.py", line 247, in get_value
position = self.get_position(name)
File "build/bdist.linux-i686/egg/durus/shelf.py", line 231, in
get_position
current = self.file.tell()
File "build/bdist.linux-i686/egg/durus/file.py", line 59, in tell
return self.file.tell()
ValueError: I/O operation on closed file

Here's how I refactored the opening of the database instance
using the "closing" contextmanager:

<--cut here
from contextmanager import closing

code..code..code..

def _open_file(self, url, backend_args=None):
"""Opens the Database file for reading."""

try:
#backend = self.scoped_backend_factory()
with closing(database.open(url, backend_args)) as db:
return db
except TypeError:
...
<--cut here

Any ideas or suggestions how to fix the error above ? I'd really like
the solution to not introduce any third-party libraries like cogen, if
possible... ;-)

Kind regards,
Etienne

[Etienne Robillard]

I think the with_statement could work, but somehow
my error was probably in understanding the 'return' statement
semantics. Thus, it's probably illegal to return a reference
to a variable inside a with block, as it will loses it context and
will closes the database in the __exit__ statement.

Will look forward on using another strategy...


Kind regards,
Etienne

--
Etienne Robillard <robillar...@gmail.com>
Green Tea Hackers Club <http://gthc.org/>
Blog: <http://gthc.org/blog/>
PGP Fingerprint: 178A BF04 23F0 2BF5 535D 4A57 FD53 FD31 98DC 4E57

[Matthew Scott]

If your use of Schevo databases includes opening, reading from, then closing databases, a context manager approach like the one you were trying might be useful.  There is overhead in doing so though, so carefully consider the way you intend to access your databases if you follow that pattern.

Any variable returned by a context manager with the 'with' keyword should be used within the enclosed block.

As you discovered, it's not impossible to return a reference to a variable, but if you do return an object managed by the context manager, Python calls the __exit__ method of the context manager which in this case calls .close() upon the object.

On Mon, Nov 9, 2009 at 05:46, Etienne Robillard <robillar...@gmail.com> wrote:

I think the with_statement could work, but somehow
my error was probably in understanding the 'return' statement
semantics. Thus, it's probably illegal to return a reference
to a variable inside a with block, as it will loses it context and
will closes the database in the __exit__ statement.

--
Matthew Scott
ElevenCraft Inc.