Azure & Compression

[Matt Johnson]

Something just caught my eye in the docs for the compression bundle:  http://ravendb.net/docs/2.0/server/extending/bundles/compression

It makes sense that compression would reduce disk space at the expense of CPU.  But I hadn't considered that this means fewer I/O operations also.

As environments like Azure have lower than desired IOPS ratings, perhaps a best practice would be to enable the compression bundle if running on Azure?

Anyone in a position to easily test this?

This may be a non-starter, since the compression bundle only comes with the Enterprise license of RavenDB.  Unless that could be relaxed?

[Kijana Woodard]

I think I can. Trying now.

--
You received this message because you are subscribed to the Google Groups "ravendb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Kijana Woodard]

Using /init: https://github.com/kijanawoodard/raven-exercise/blob/master/src/RavenExercise/App.cs

Without compression bundle:

Initialized 1,000,000 documents in 20.4400816566667 minutes. 815 per second

With compression bundle:

Initialized 1,000,000 documents in 8.470159295 minutes. 1968 per second

Unscientific, but I can confirm that there was more "other" activity on the box than during the first run.

[Matt Johnson]

That's quite a boost!  Can you run it a few more times just so we have some additional samples?

Thanks so much!

[Oren Eini (Ayende Rahien)]

Guys,

Can someone aggregate all of this information, that is something that I would like to have to show around.

[Felipe Leusin]

Testing on an Azure Medium VM with a separate (single) data disk:

Without compression:

Initialized 1,000,000 documents in 14.39685183 minutes. 1158 per second. CPU went to ~50%.

With compression:
Initialized 1,000,000 documents in 8.388272615 minutes. 1987 per second. CPU went to ~90%.

[Kijana Woodard]

I've only been able to run Without Compression once more:

Initialized 1,000,000 documents in 19.72251771 minutes. 845 per second

I should mention that I'm running a Large VM with raven running simple by unzipping into c:\raven. No data disks attached.

[Kijana Woodard]

Another run Without Compression:

Initialized 1,000,000 documents in 14.46246879 minutes. 1152 per second

With Compression:

Initialized 1,000,000 documents in 8.16710452666667 minutes. 2041 per second

Initialized 1,000,000 documents in 7.95125412666667 minutes. 2096 per second

[Jahmai Lay]

Impressive but I wonder if the gap would be much smaller if you didn't do so much batching and went across collections and maybe even across databases.

[Matt Watson]

Would someone mind doing that same Azure benchmark again but adding more disks and striping a volume across. So try with 2 or 4 azure disks and see if the compression vs non compression is effect. By adding a lot of raw IO the additional CPU of the compression could be bad and end up being the limiting factor.

[Felipe Leusin]

I'll only be able to try this next week but it`s on my to-do list.

[georgiosd]

Any further thoughts on this one?