Performance issue

[Tobi]

Hi!

Currently I'm running some performance tests with RavenDB to see, if it
fits my needs.

I'm basically comparing RavenDB/Embedded against Firebird/NH/Embedded.

But I'm getting some strange results. When inserting about 25.000
documents on my 3 Ghz Intel DualCore with plenty of RAM, a fast HD and
Win7, Firebird makes about 598 documents per second, and RavenDB 134
per second.

On a 2Ghz VIA system with just 512MB of RAM, a slow HD and WinXP,
Firebird makes 119 documents/sec and RavenDB 141 documents/sec.

So why is RavenDB slower on the significantly faster machine?
I still have some 600MHz machine standing around here somewhere - must
be the perfect RavenDB host :-)

Any idea where I should look for the bottleneck?

Tobias

[Matt]

Are you running NOD32 or other AV that intercepts port 8080 traffic.

Still say its a bad idea to choose a known proxy server port.

[Ayende Rahien]

How are you doing this?

A single PUT per document? That is going to be slow.

The recommended approach is to use /bulk_docs and send batches of 128 documents at a time.

[Tobi]

Matt wrote:

> Are you running NOD32 or other AV that intercepts port 8080 traffic.

No, nothing like that. And the test I'm running uses RavenDB the embedded
way, so there's non port 8080 traffic.

Tobias

[Tobi]

> How are you doing this?
> A single PUT per document? That is going to be slow.

It is indeed. While a session per document resulted in 598 docs/sec, a
single session for ALL documents was 1269 docs/sec.

> The recommended approach is to use /bulk_docs and send batches of 128
> documents at a time.

What's the recommended way to do this in the client API? Starting a new
session for each batch of 128 documents is little but awkward to
handle, so I tried something like this:

var count = 0;
using (var session = documentStore.OpenSession())
{
session.MaxNumberOfRequestsPerSession = int.MaxValue;
foreach (var doc in docs)
{
session.Store(doc);

count++;
if ((count % 128) == 0)
{
session.SaveChanges();
session.Clear();
}
}
session.SaveChanges();
}

Tobias

[Matt Warren]

Yes, if you do that the Raven Client will send all the update in one
batch when you call SaveChange(). This does a post to /bulk_docs of
all the docs.

[Ayende Rahien]

Tobi,

You code should work fine, I think.

1,300 docs / sec is low, though. 

You can try setting the transaction mode to lazy (Raven/TransactionMode), that can speed things up significantly.

[Tobi]

> You code should work fine, I think.

It seems, that the session.Clear()-approach is slightly slower than
creating a session for each batch of 128 documents.

> 1,300 docs / sec is low, though.
> You can try setting the transaction mode to lazy (Raven/TransactionMode),
> that can speed things up significantly.

Not exactly "significantly", just little bit - and no difference on
slower machines.

100.000 products, each having a text field, some boolean fields, 5 EAN
numbers and 5 prices.

On my fast development machine:
With docStore.Configuration.TransactionMode = TransactionMode.Safe:
Elapsed: 00:01:40.8548915 - 991/sec

With docStore.Configuration.TransactionMode = TransactionMode.Lazy:
Elapsed: 00:01:39.3103662 - 1006/sec

On a 2 Ghz VIA system (an IBM Anyplace POS system):
With docStore.Configuration.TransactionMode = TransactionMode.Safe:
Elapsed: 00:07:36.0523745 - 219/sec

With docStore.Configuration.TransactionMode = TransactionMode.Lazy:
Elapsed: 00:07:39.5242054 - 217/sec

On an old 650 MHz Celeron with 512MB RAM (An older Toshiba POS system)
With docStore.Configuration.TransactionMode = TransactionMode.Safe:
Elapsed: 00:20:41.5141248 - 80/sec

With docStore.Configuration.TransactionMode = TransactionMode.Lazy:
Elapsed: 00:20:41.4157905 - 80/sec

Maybe I'm still doing something wrong?
What I'm doing is basically:

documentStore.Configuration.DataDirectory = "Data";
documentStore.Configuration.TransactionMode =
Raven.Database.TransactionMode.Lazy;
documentStore.Initialise();
documentStore.DatabaseCommands.PutIndex("Product/Number",
new IndexDefinition<Product>()
{
Map = products =>
from product in products
from number in product.Numbers select new { Number = number }
});

...

var sw = Stopwatch.StartNew();
var batch = new List<Product>();
foreach (var product in products)
{
batch.Add(product);
if (batch.Count == 128)
{

using (var session = documentStore.OpenSession())
{

foreach (var batchedProduct in batch)
session.Store(batchedProduct);
session.SaveChanges();
batch.Clear();

}
}
}
using (var session = documentStore.OpenSession())
{

foreach (var batchedProduct in batch)
session.Store(batchedProduct);
session.SaveChanges();
}
sw.Stop();
Console.WriteLine("Elapsed: {0} - {1}/sec", sw.Elapsed,
products.Count * 1000 / sw.ElapsedMilliseconds);

...

public class Product
{
public string Id { get; set; }
public IList<string> Numbers {get; set;}
public virtual string Text { get; set; }
public IList<ScaledPrice> Prices { get; set; }
}

public class ScaledPrice
{
public int Level { get; set; }
public decimal QtyThreshold { get; set; }
public Price Price { get; set; }
public Vat Vat1 { get; set; }
public Vat Vat2 { get; set; }
}

public class Vat
{
public int Number { get; set; }
public decimal Value { get; set; }
}

Tobias

PS: It seems, there's a tiny spelling error in DocumentStore:
Initialise() -> Initialize()

[Ayende Rahien]

You are doing things in a synchronous manner. 

There isn't a lot of optimizations that we can have here, mostly because we are spending most of the time waiting for disk.

Try parallelizing the work, that will gain you some perf boost.

[Tobi]

> Try parallelizing the work, that will gain you some perf boost.

Ok. Running 3 threads, I get:

slow machine:
- 1 thread -> 80 docs/sec, 3 threads -> 102 docs/sec
- about 27% faster with 3 threads

faster machine:
- 1 thread -> 218 docs/sec, 3 threads -> 237 docs/sec
- about 9% faster with 3 threads

fastest dual core machine:
- 1 thread -> 984 docs/sec, 3 threads -> 1396 docs/sec
- about 40% faster

This is probably the maximum I can get. It's much faster than
Nhibernate/Firebird (which would be at least 1.100.000 Inserts and
Firebird doesn't support batching...)

Tobias