Issue 207 in couchdb-python: python couchdb data collection stops
couchdb...@googlecode.com
Owner: ----
Labels: Type-Defect Priority-Medium
New issue 207 by grittyg...@gmail.com: python couchdb data collection stops
http://code.google.com/p/couchdb-python/issues/detail?id=207
I am using couchdb to store twitter data. I found that couchdb stops
updating its data base though I keep getting the twitter data. I am sure
that the python program does not halt with any exception, the program keeps
running I confirmed this by stopping the storage to couchdb database and
instead storing the output in a CSV format. I found no problem in the
python program. But when I started storing the python dictionary in couchdb
I found that though the program keeps running the storage in couchdb has
stopped.
I basically store the dictionary that contains twitter data by using the
python couchdb save method, db.save(twitter_dic) where db is the database
instance.I find that some times I get 3GB of data and couchdb stops
storing, sometimes it stops storing even when it reaches 0.6GB.
I dont know what is happening in log but by just seeing I can find these
lines. I am storing the data on a database called "worldtweets4". Following
are few lines of log where the problem might be as you can see the storage
had stopped I dont get any more 'PUT" /databasename/ID.
Mon, 21 Nov 2011 14:57:35 GMT] [info] [<0.17870.56>] 127.0.0.1 - - 'PUT'
/worldtweets4/69084bd085994bc7b60ec8f586c69d8f 201
[Mon, 21 Nov 2011 15:50:29 GMT] [info] [<0.27616.57>] 127.0.0.1 - - 'GET'
/_all_dbs 200
[Mon, 21 Nov 2011 15:50:29 GMT] [info] [<0.23293.57>] 127.0.0.1 - - 'GET'
/_session 200
[Mon, 21 Nov 2011 15:50:29 GMT] [info] [<0.27615.57>] 127.0.0.1 - - 'GET' /
200
[Mon, 21 Nov 2011 15:50:29 GMT] [info] [<0.27617.57>] 127.0.0.1 - - 'GET'
/newdb/ 200
[Mon, 21 Nov 2011 15:50:29 GMT] [info] [<0.27617.57>] 127.0.0.1 - - 'GET'
/worldtweets4/ 200
[Mon, 21 Nov 2011 15:50:29 GMT] [info] [<0.23293.57>] 127.0.0.1 - - 'GET'
/worldtweets3/ 200
[Mon, 21 Nov 2011 15:50:29 GMT] [info] [<0.27616.57>] 127.0.0.1 - - 'GET'
/_users/ 200
This same sequence gets repeated and then some where i get the following
[Mon, 21 Nov 2011 15:59:02 GMT] [info] [<0.28139.57>] 127.0.0.1 - - 'GET'
/worldtweets4/_all_docs?startkey=%22_design%2F%22&endkey=%22_design0%22&include_docs=true
200
[Mon, 21 Nov 2011 15:59:02 GMT] [info] [<0.28140.57>] 127.0.0.1 - - 'GET'
/_config/query_servers/ 200
[Mon, 21 Nov 2011 15:59:02 GMT] [info] [<0.27620.57>] 127.0.0.1 - - 'GET'
/worldtweets4/_all_docs?limit=11&descending=true 200
couchdb...@googlecode.com
Status: Invalid
Comment #1 on issue 207 by matt.goo...@gmail.com: python couchdb data
collection stops
http://code.google.com/p/couchdb-python/issues/detail?id=207
Hi, a ticket isn't a great place for a discussion. Please could you join
the mailing list, http://groups.google.com/group/couchdb-python, and
discuss there. Thanks!
For the record, I've not seen anything like this reported before and
there's not much in the description that suggests there's anything wrong in
couchdb-python. However, I've set up a local test that's saving a
tweet-like document at frequent, random intervals. I'll leave that running
for as long as possible and see what happens.
Closing this ticket for now. Let's re-open if it we find an actual problem.
couchdb...@googlecode.com
Comment #2 on issue 207 by matt.goo...@gmail.com: python couchdb data
collection stops
http://code.google.com/p/couchdb-python/issues/detail?id=207
My local test kept running all day and overnight. AFAICT, there were no
problems and no pauses. It certainly didn't stop. I'm not sure what else to
do until there's some actual code we can run that reliably causes the
problem.