ravendb slow write performance?

[Mike]

DB promotes it's self as "High performance - RavenDB is a very fast persistence layer for every type of data model."

But the reality seems to be otherwise.  At least so far, RavenDB actually seems to have good READ performance - but persistence is rather slow.

I opened a question a while back over at stackoverflow, hoping to get some feedback on the issue  - http://stackoverflow.com/questions/10006152/ravendb-slow-write-save-performance.  

So, far there has been very little light shed on the problem.  But, I'm wondering if it doesn't have something to do with authorization.

[Oren Eini (Ayende Rahien)]

Mike,

It is likely that at least some part of this is taken by auth, especially windows auth against a remote server.

At any rate, 200 ms is _way_ too much, but you are aware that you are actually testing this with remote machine cost tied in, right?

If you want to test performance, you can try creating a local test, because that neutralize the network noise.

Also, it is quite important to note what IO you have.

[Mike]

Oren - Thanks for the reply !

So, is there a way to _fix_ the 200ms write problem if running a web app on RavenHQ at AppHarbor?

  >> ...you are aware that you are actually testing this with remote machine cost tied in, right?

Yes, I'm aware I'm testing with a remote machine.  Maybe I'm under-informed, is there another way to deploy a web app using RavenHQ at AppHarbor?

>> If you want to test performance, you can try creating a local test, because that neutralize the network noise.

I just want my application to perform well when I deploy it on AppHarbor.  Since it does about 50/50 read/write, the slow write performance is very concerning.  In the stackoverflow question, I mention 2 tests.  The first is on my local network - although it did use separate machines.  The second (see the "Edit") is at AppHarbor and is just the sample that AppHarbor provides for RavenHQ (which you guys may have provided to frissm?) - you can try it yourself and there are links for the source as well.  I've also tried it on my local machine and while it's a bit faster, it's still +10x times slower than SQL Server or MySQL.

>> Also, it is quite important to note what IO you have.

The AppHarbor sample simply persists a single string (a "Thingy") that you enter.  That's it.  So, there is very little I/O.

[Oren Eini (Ayende Rahien)]

You are testing that on the free instance, and not doing a longevity test. Both of which would have severe performance impact.

I gave it a couple of runs in the app harbor test app, and I don't see a slow down.

Do you have actual measurements? Looking at the code base, I don't see anything that would indicate perf times.

On Mon, May 21, 2012 at 11:31 PM, Mike <mglea...@hotmail.com> wrote:

Oren - Thanks for the reply !

So, is there a way to _fix_ the 200ms write problem if running a web app on RavenHQ at AppHarbor?

  >> ...you are aware that you are actually testing this with remote machine cost tied in, right?

Yes, I'm aware I'm testing with a remote machine.  Maybe I'm under-informed, is there another way to deploy a web app using RavenHQ at AppHarbor?

>> If you want to test performance, you can try creating a local test, because that neutralize the network noise.

I just want my application to perform well when I deploy it on AppHarbor.  Since it does about 50/50 read/write, the slow write performance is very concerning.  In the stackoverflow question, I mention 2 tests.  The first is on my local network - although it did use separate machines.  The second (see the "Edit") is at AppHarbor and is just the sample that AppHarbor provides for RavenHQ (which you guys may have provided to frissm?) - you can try it yourself and there are links for the source as well.  I've also tried it on my local machine and while it's a bit faster, it's still +10x times slower than SQL Server or MySQL.

>> Also, it is quite important to note what IO you have.

The AppHarbor sample simply persists a single string (a "Thingy") that you enter.  That's it.  So, there is very little I/O.

[Mike]

For the second test I used the sample app from https://github.com/friism/RavenDBMvcSample and added the MiniProfiler and RavenDb plugin for MiniProfiler - from: https://nuget.org/packages/MiniProfiler and https://nuget.org/packages/MiniProfiler.RavenDb .

>> You are testing that on the free instance

Yes.  But +200ms is very slow.  I've tried the free MySQL and my recollection is that it's 18-30ms.

>> and not doing a longevity test.

True, but I tried saving a bunch of times in a for loop and had similar results.  Also, I don't count the first save because it's usually longer so the figure is for the 2nd save and beyond. 

Anyway, maybe I can give you access to the Project at Appharbor (if you give me your userID)?

To test it you can:

1. Browse to http://ravendbtest44.apphb.com/Thingy/New 
   
2. Enter a string in the "Name" field and click the "Submit" button.
3. In the upper left-hand corner you'll see the information that MiniProfiler adds to the web page.  Click on the larger number and you'll see an entry for "RavenDb: Query - ..." and the amount of time it took to run.

[Oren Eini (Ayende Rahien)]

Where is the source for what _you_ are doing there?

[Chris Marisic]

On Monday, May 21, 2012 5:33:34 PM UTC-4, Mike wrote:

 Since it does about 50/50 read/write, the slow write performance is very concerning. 

I would spend some time analzying your models and indexes. You most likely didn't model transaction boundaries well and have documents that are too granular and behave like SQL server table rows and not actual documents.

[Beyers Cronje]

I would spend some time analzying your models and indexes. You most likely didn't model transaction boundaries well and have documents that are too granular and behave like SQL server table rows and not actual documents.

The example app on Appharbor just has one model with a single field Name. Saving a single doc took 618ms for me now using the test app http://ravendbtest44.apphb.com/Thingy  Where query only took 39ms.

Have a look at http://ravendbtest44.apphb.com/mini-profiler-resources/results?id=29241993-fe87-45db-a23e-2b15b26a0b39

hingy/Create (618.5 ms)

	duration (ms)	with children (ms)	from start (ms)
http://ravendbtest44.apphb.com:15239/Thingy/C...	59.8	618.5	+0.0
Controller: ThingyController.Create	20.4	346.0	+10.8
RavenDb: Query - /databases/AppHarbor_ad53dbf...	5.1	5.1	+17.1
RavenDb: Query - /databases/AppHarbor_ad53dbf...	320.5	320.5	+29.5
RavenDb: Query - /databases/AppHarbor_ad53dbf...	212.7	212.7	+383.4
shareshow trivialhide time with children

 

[Oren Eini (Ayende Rahien)]

Beyers,

The free ravenhq instances are limited in many respects, you can't really judge performance from those numbers.

[Chris Marisic]

On Tuesday, May 22, 2012 9:03:17 AM UTC-4, Beyers Cronje wrote:

The example app on Appharbor just has one model with a single field Name.

That's a terrible model. If that's your actual use case, they make key value stores, and you'd really only need a key store devoid of values.

[Mike]

Chris - chill.  I think you're missing the point.

[Chris Marisic]

I'm not missing the point, you're testing RavenDB with constraints that are totally unrealworld. You want to benchmark using data as realistic as possible, this means rich models, and that your comparison to sql server is compared against ORM usage.

[Mike]

@Ayende -  it's in the AppHarbor repository.  I just added you (ayende@...) as a collaborator so you can download the source from the same source where it's running - https://appharbor.com/applications/ravendbtest44 .

[Oren Eini (Ayende Rahien)]

I am looking into that

[bcr...@gmail.com]

Chris,

Relax dude, as far as I can tell the question is why is writing to raven so slow for the OP. To illustrate the point the test app just writes a single doc with a single value. No need to retort to bad document design etc. Your argument is like saying ALL unit tests should model some realworld complex design, nonsense.

Sent from my BlackBerry® wireless device

---

From: Chris Marisic <ch...@marisic.com>

Sender: rav...@googlegroups.com

Date: Tue, 22 May 2012 09:46:04 -0700 (PDT)

To: <rav...@googlegroups.com>

ReplyTo: rav...@googlegroups.com

Subject: Re: [RavenDB] ravendb slow write performance?

[Mike]

Sheesh Chris - It's not a benchmark or whatever. It's only the simplest possible sample which shows the problem.

While porting a more complex app, I noticed writes were taking a long time. I tried a few things, but kept seeing slow writes and decided I needed to make sure it wasn't my code or setup. So, I looked around and found a super-simple sample on AppHarbor which was good since I'd like to deploy on Appharbor using RavenHQ. The Appharbor sample is what we're talking about now because it shows the same slow writes.

[Oren Eini (Ayende Rahien)]

Okay, I run some additional tests, and it is pretty much as I said.

You are running on the free plan, which means that this is actually sitting on top of a the EC2 IO system. This is pretty basic, as you can imagine, and the reason why you see those numbers.

Note, however, that when you start using concurrent requests, you see more interesting results.

Higher level plans get faster IO (RAID), and faster performance overall.

[Mike]

Thanks for looking into it Oren.  Interesting, I thought for sure you'd come back and say one of the following:

• There was some change/fix that needs to be made - e.g.  possibly modifying the authentication model so that it would cache the authentication so that subsequent requests are faster...
  or
• "Bless" (i.e. change the RavenDB documentation) to say that in managed environments (like AppHarbor) it's safe to use PreAuthenticate = true and UnsafeAuthenticatedConnectionSharing = true.

Part of the reason I thought these might end up being the outcome is because of the huge impact you see with using PreAuthenticate and  UnsafeAuthenticatedConnectionSharing.  When I tried it recently, it appeared to provide about 10x faster response time and a lot fewer requests per save/write.

Ahh well, it must not be a safe option...

Thanks.

[Oren Eini (Ayende Rahien)]

nMike,

Actually, I never even thought about this. The way RavenHQ auth works, we don't use any of that.

The auth model _is_ cache. Also, note that we use the same auth model for reads and writes.

Just to check, I did both changes in the app, and it doesn't have an impact.

[Justin A]

@Mike - out of interest, what are you final thoughts to this question?

You opened up the post by asking if Raven is slow at writes .. baked up with some reproduction code so we can all verify.

Ayende did, stating that the issue lies with Amazon and their pretty average IO on the low end system the **free** RavenHQ db plan, being used.

So does this end up being a show stopped for you?

Have you had a chance to trying seeing what writes are like on RavenDb .. which is not on RavenHQ **free** plan?

Are you thinking about sticking with RavenHQ still but are now worried that going from a **free** plan to a paid one might still have poor IO results?

Please don't think I'm having a dig at you or anything. Far from it. I'm actually really curious to see what your planning on doing now that some answers have come to light, regarding your initial question.

[Oren Eini (Ayende Rahien)]

Another thing to note here is that this actually parallelize quite well