Couchdb error when posting document - 413 Request entity too large

[Uday Jarajapu]

We have been using Couchdb for close to two years now to store our performance test results (jmeter, ruby) and like it a lot for its ease of use and flexibility. Thanks for a product that is powerful and easy to use.

Recently, we migrated to a new co-location facility and moved our couchDB during that process. Ever since that move, we have started receiving a weird error message from CouchDB that says '413 Request entity too large' when we try to post our test result documents to the new location. What makes it weirder is that it works for smaller load tests. It fails when we try to post documents larger than 3.4 MB.

We noticed that the max_document_size has been set to 4,294,967,296 (defaults to 4.2G) and the max_attachment_chunk_size has been set to the same value (4.2G) as well. I could not identify any other configuration parameters that might be causing the problem.

Have any of you have run into this problem before? what might the source of this problem be? What would the fix for it be?

Uday

[Mark Hahn]

Did you sniff the packets to see what is really being sent?

[Uday Jarajapu]

I did not sniff the packets but I did measure the size of the documents being sent. It was close to 3.4MB. Any recommendations for a good sniffer ?

[Mark Hahn]

I use tcpdump on linux and then analyze the results in windows using
wireshark. Wireshark has a bit of a learning curve but I think it is worth
it. So far every time I've gotten an error from couch at this level it
turned out to be bad on the wire and was ultimately my fault.

[Uday Jarajapu]

Thanks Mark. I will try this out and update you if I find something interesting.

[James Marca]

Hi,

this is a quick note for whomever is in charge of the wiki (my wiki
account no longer has write ability)

Built-In_Reduce_Functions says that you need to emit numbers for the
_sum operation. As of 1.2.x, this can also be a list of numbers.

I just emitted a list of numbers and got what I expected...that is, a
sum over each element of the list (not a summing up of the elements,
summed.

Example:

no reducing, I emit a list of 4 items for each key:

["2007-02-27", "1213122", "2007-02-27 14:46:00 UTC"] [0.000007830687904038072, 0.000001032961291729275,0.0000012756780418961, 2.209310910297831e-7]
["2007-02-27", "1213122", "2007-02-27 14:46:30 UTC"] [0.000008151766266816958, 0.000001042421330553939,0.000001315925015500657, 2.278253064474915e-7]

"Exact" reducing, same as emit with no reduce:

["2007-02-27", "1213122", "2007-02-27 14:46:00 UTC"] [0.000007830687904038072, 0.000001032961291729275,0.0000012756780418961, 2.209310910297831e-7]
["2007-02-27", "1213122", "2007-02-27 14:46:30 UTC"] [0.000008151766266816958, 0.000001042421330553939,0.000001315925015500657, 2.278253064474915e-7]

level=1 reducing, summing up over each day

["2007-02-27"] [0.003547119931415267, 0.005496433891250249, 0.007064010452051014,0.003552803842322634]
["2007-02-28"] [0.008437247339553495, 0.01664606221946668, 0.01779585111017863,0.01036210620457586]

I observed this behavior in 1.2.x. I know this hasn't always been a
feature, but I couldn't find whether it was 1.1 or 1.0, or what.

Regards,
James Marca

[Adam Kocoloski]

I believe it arrived in 1.1.0:

https://github.com/apache/couchdb/blob/1.1.0/CHANGES#L47

As far as the wiki goes, we needed to lock it down to deal with spam edits. Happy to add you to http://wiki.apache.org/couchdb/ContributorsGroup -- are you JamesMarca on MoinMoin?

Adam

[James Marca]

On Tue, Jan 17, 2012 at 09:21:46PM -0500, Adam Kocoloski wrote:
> On Jan 17, 2012, at 8:31 PM, James Marca wrote:

...

>
> I believe it arrived in 1.1.0:
>
> https://github.com/apache/couchdb/blob/1.1.0/CHANGES#L47
>
> As far as the wiki goes, we needed to lock it down to deal with spam edits. Happy to add you to http://wiki.apache.org/couchdb/ContributorsGroup -- are you JamesMarca on MoinMoin?

I think I am MarcaJames.

More than happy to edit directly again, but I also understand the need
to fight wiki spam. Also, I think I will need to submit a contributor
license.

Regards,
James

[Uday Jarajapu]

Hello Mark,

I was able to capture traffic between my localhost and couchDB while
writing a document that resulted in 413. Looking at the traffic with a
simple display filter set (ip.dst == <ip_of_couchdb>) , I could not see any
big difference that jumped out at me.

The only thing missing was a final ACK during this run. Is there a set
pattern that I should watch out for ?

Uday