Issue 208 in couchdb-python: Problems with http cache in multitheaded environment.

[couchdb...@googlecode.com]

Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 208 by kxepal: Problems with http cache in multitheaded
environment.
http://code.google.com/p/couchdb-python/issues/detail?id=208

What steps will reproduce the problem?
1. Create multitheaded script
2. Share Database instance between threads
3. Make a lot of requests until crush happens (see exception.log attachment)

There are two different exceptions, but I suppose that they have same
nature.

I'll make test script and patch a little bit later, just have created issue
to not forget about this problem.

Attachments:
exception.log 2.4 KB

[couchdb...@googlecode.com]

Comment #1 on issue 208 by kxepal: Problems with http cache in multitheaded
environment.
http://code.google.com/p/couchdb-python/issues/detail?id=208

Attached patch and test case

Attachments:
01-issue-208-test_http.patch 2.9 KB
02-issue-208-http.patch 901 bytes

[couchdb...@googlecode.com]

Comment #2 on issue 208 by kxepal: Problems with http cache in multitheaded
environment.
http://code.google.com/p/couchdb-python/issues/detail?id=208

Forgot to add test case to suite, also named it well.

Attachments:
01-issue-208-test_http.patch 3.1 KB

[couchdb...@googlecode.com]

Comment #3 on issue 208 by matt.goo...@gmail.com: Problems with http cache

in multitheaded environment.
http://code.google.com/p/couchdb-python/issues/detail?id=208

That looks like it should work, thanks!

I extracted the cache code from http.Session recently to make it more
testable but haven't pushed it yet. My fault entirely, but do you mind if I
rework your patch on top on my changes? Should I post the new diff back
here for you to check before I commit with your name against it?

[couchdb...@googlecode.com]

Comment #4 on issue 208 by kxepal: Problems with http cache in multitheaded
environment.
http://code.google.com/p/couchdb-python/issues/detail?id=208

There is no problem if you make patch for my patch - finally, we all works
for the same project and I always like to see what things I've missed(:

[couchdb...@googlecode.com]

Comment #5 on issue 208 by djc.ochtman: Problems with http cache in
multitheaded environment.
http://code.google.com/p/couchdb-python/issues/detail?id=208

Matt, a ping on this?

[couchdb...@googlecode.com]

Comment #6 on issue 208 by matt.goo...@gmail.com: Problems with http cache

in multitheaded environment.
http://code.google.com/p/couchdb-python/issues/detail?id=208

I've just pushed some code including r6ab0ed59cf1d to extract a Cache class
from the middle of the http.Session code. It doesn't actually fix the issue
but I wanted to do this to make testing easier/possible.

[couchdb...@googlecode.com]

Comment #7 on issue 208 by matt.goo...@gmail.com: Problems with http cache

in multitheaded environment.
http://code.google.com/p/couchdb-python/issues/detail?id=208

It looks like I may have a heavily modified version of kxepal's test that
works against the new Cache class and fails consistently. I'll check it out
more - it's a little insane ;) - and see if kxepal's fix works ok.

[couchdb...@googlecode.com]

Comment #8 on issue 208 by kxepal: Problems with http cache in multitheaded
environment.
http://code.google.com/p/couchdb-python/issues/detail?id=208

Nice refactoring! However, I have a little scepsis about tests, since now
they would proof the fact that dict iterators are not threadsafe. It still
requires to change `.iteritems()` to `.items()` or wrap `_cleanup` method
body with lock to let it work in multithreaded environment [1]. In
perspective, if we'd like to add Python3 support, we still need to use lock
or alien for <Python-2.7 `.itemsview()` method since behavior of `.items()`
is changed, so lock-variant might be more better and optimal.

[1]
http://code.google.com/p/couchdb-python/source/browse/couchdb/http.py?spec=svn6ab0ed59cf1d013ed7a2fcaf3eaf32ac8130965a&name=6ab0ed59cf1d&r=6ab0ed59cf1d013ed7a2fcaf3eaf32ac8130965a#431

[couchdb...@googlecode.com]

Comment #9 on issue 208 by djc.ochtman: Problems with http cache in
multitheaded environment.
http://code.google.com/p/couchdb-python/issues/detail?id=208

This issue has been migrated to GitHub. Please continue discussion here:

https://github.com/djc/couchdb-python/issues/208

--
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings