Very slow bulk writes when write concern is UNacknowledged

[Tim W]

In migrating code from using the insert command to using execBulkWrite (via PHP library) and I am seeing weird behaviour with w=0

In short, unacknowledged writes are much, much slower than acknowledged ones.

For comparison, 14k small-ish documents insert in ~1.5s with w=1 and w=majority, but over two minutes with w=0. The more documents, the greater the disparity.

I see this with both mmapv1 and WiredTiger on both 1.4.4 and 1.5.3 of the driver under MongoDB 3.4

The machine is Mac OS X (High Sierra).

Should I report this as an issue on Github with some code? I thought I'd ask here first in case it's a known issue. 

[Tim W]

This issue seems to be fixed by upgrading to MongoDB 3.6

[Jeremy Mikola]

On Thu, Mar 21, 2019 at 1:38 PM Tim W <fabia...@gmail.com> wrote:

In migrating code from using the insert command to using execBulkWrite (via PHP library) and I am seeing weird behaviour with w=0

In short, unacknowledged writes are much, much slower than acknowledged ones.

For comparison, 14k small-ish documents insert in ~1.5s with w=1 and w=majority, but over two minutes with w=0. The more documents, the greater the disparity.

I see this with both mmapv1 and WiredTiger on both 1.4.4 and 1.5.3 of the driver under MongoDB 3.4

This issue seems to be fixed by upgrading to MongoDB 3.6

I researched this a bit and it looks like you've unconvered a longstanding bug with how PHPC executes bulk writes in libmongoc (thanks!).

On server versions prior to 3.6, the driver is actually sending one message to the server per write (14k roundtrips in your case) instead one (or a few) large batches. You don't see this behavior when connected to MongoDB 3.6 since libmongoc 1.10+ is able to use the OP_MSG protocol to send unacknowledged writes in a large batch.

If you'd like more details, I've written up an analysis in PHPC-1351. I'm not sure when we'll be able to address this given other priorities, but feel free to watch that issue for further updates. Any solution will likely require some code changes to libmongoc, since there presently is no public API that would allow PHPC to address this on its own, so I would expect to see a CDRIVER issue linked to PHPC-1351 soon.

[Tim W]

Thanks for the write-up Jeremy.

From my personal perspective I'll be upgrading my live servers to 3.6 over the weekend, so that'll fix it for me. I'm just glad I'm not crazy.

I've got a test script I used for my benchmarks if you want it though.

[Tim W]

I've got a test script I used for my benchmarks if you want it though.

https://gist.github.com/timwhitlock/4d5b6e33afc7fe9b00f860606bbccd3b

 

[Jeremy Mikola]

On Thu, Mar 21, 2019 at 5:29 PM Tim W <fabia...@gmail.com> wrote:

I've got a test script I used for my benchmarks if you want it though.

https://gist.github.com/timwhitlock/4d5b6e33afc7fe9b00f860606bbccd3b

Thanks! I'll reference this in the JIRA ticket in case it comes in handy down the line.

 

 

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/95c13542-7ca0-4866-96e9-32cf2f39f44a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.