max_document_size limits on hosted services?
16 views
Skip to first unread message
Jens Alfke
Aug 27, 2012, 8:24:21 PM8/27/12
to us...@couchdb.apache.org
I've had a few reports from TouchDB users of problems replicating large documents up to hosted services (IrisCouch or Cloudant). The symptom is that TouchDB uploads ~200k of JSON via _bulk_docs and then either gets a 503 error or just a dropped connection. I'm wondering whether this is a symptom of hitting a server-side limit on the maximum size of a document body.
Looking through the CouchDB config I see a couchdb.max_document_size setting, but it defaults to 4GB, which should be more than sufficient :) It's also set to 4GB on my basic IrisCouch account. Is there some other limit on document size, or known bugs that would cause a server error with very large JSON documents?
—Jens
Looking through the CouchDB config I see a couchdb.max_document_size setting, but it defaults to 4GB, which should be more than sufficient :) It's also set to 4GB on my basic IrisCouch account. Is there some other limit on document size, or known bugs that would cause a server error with very large JSON documents?
—Jens
Adam Kocoloski
Aug 27, 2012, 8:49:36 PM8/27/12
to us...@couchdb.apache.org
We have the maximum size set smaller at Cloudant, but certainly well above the 200 KB you mention here. I'm not aware of other more restrictive limits in our setup (and I'm certain multi-MB uploads work in the general case). Cheers,
Adam
Adam
Tim Tisdall
Aug 28, 2012, 4:42:00 PM8/28/12
to us...@couchdb.apache.org
I'm not familiar with TouchDB, but I get dropped connections when
using _bulk_docs when couch runs out of memory or crashes. I've had
to reduce the number of docs to accommodate this problem, but adding
more RAM may also help. The problem is related to the way linux over
allocates RAM and the oom-killer just killing programs when the system
runs out of memory.
-Tim
using _bulk_docs when couch runs out of memory or crashes. I've had
to reduce the number of docs to accommodate this problem, but adding
more RAM may also help. The problem is related to the way linux over
allocates RAM and the oom-killer just killing programs when the system
runs out of memory.
-Tim
Jason Smith
Aug 28, 2012, 8:32:49 PM8/28/12
to us...@couchdb.apache.org
Hi, Jens.
Cloudant and Iris Couch run dissimilar software. It's a big clue that
you have problems on both. Either you found a deep bug common to both
(e.g. MochiWeb); or maybe they are independent failures; or maybe door
number three.
You say there is sometimes a 503 and sometimes a drop. Is that across
both services?
Finally, while not doubting your symptoms at all, I must say in my
experience, the "hosting" has very rarely been a contributing factor
to a bug. It's basically CouchDB (or BigCouch) running on an x86_64
box.
Anyway, if you want, contact me off-list and I can set up some pcap
rules for your couch and we can investigate the captures.
--
Iris Couch
Cloudant and Iris Couch run dissimilar software. It's a big clue that
you have problems on both. Either you found a deep bug common to both
(e.g. MochiWeb); or maybe they are independent failures; or maybe door
number three.
You say there is sometimes a 503 and sometimes a drop. Is that across
both services?
Finally, while not doubting your symptoms at all, I must say in my
experience, the "hosting" has very rarely been a contributing factor
to a bug. It's basically CouchDB (or BigCouch) running on an x86_64
box.
Anyway, if you want, contact me off-list and I can set up some pcap
rules for your couch and we can investigate the captures.
Iris Couch
Reply all
Reply to author
Forward
0 new messages