Oracle and DB2 on top of TCP-C. What about RavenDB?

[Josh Close]

Not sure if anyone besides Oren can answer this...

What are your thoughts on this article and how does RavenDB fit into it?

http://dbmsmusings.blogspot.com/2012/05/if-all-these-new-dbms-technologies-are.html 

[Chris Marisic]

I posted a response to this one the article.

[Itamar Syn-Hershko]

The entire article is demagogic. Having skimmed over the actual paper, what he is describing is basically just a transaction scheduler that operates in a way very similar to a message bus. I have no idea why they had to pick on NoSQL using that TPC-C benchmark, especially when in their concluding note they say it can actually be used by every type of a DB system...

Distributed transactions are fundamentally broken, and I'm not convinced they overcame all the possible issues. For example, what happens if one of the nodes crash after an agreement was made, and just before the transaction was committed?

He also ignores a lot of other important factors to consider when weighting RDBMS vs any NoSQL. To name the important 2:

* The cost of reads compared to cost of writes

* Development speed - no schema and a good client API make all the difference

[Chris Marisic]

On Wednesday, May 16, 2012 3:08:03 PM UTC-4, Itamar Syn-Hershko wrote:

* Development speed - no schema and a good client API make all the difference

Indeed. I love the features of RavenDB, but the absolute end all reason I choose RavenDB is the fastest time of development to market comes with RavenDB usage.

[Oren Eini (Ayende Rahien)]

Put simply, I could do a lot more with 30 millions than try to win a stupid benchmark.

More to the point, we can do, on commodity hardware, thousands of writes per second without any special configuration.

I am not sure how they calculate what a transaction is, it seems to be a mix of writes and reads, but given a set of 100 servers, I don't feel like this would be too hard (at ~2000$ per server total cost would be 200,000$).

In other words, this isn't a really interesting benchmark, to be truthful. It is also one that plays strongly to relational databases, because that is what it measures.

[Chris Marisic]

I agree, that post and those benchmarks seem to be entirely meaningless.

I bet those benchmarks were also ran against a table that has like 5 columns, 1 primary key, 0 indexes, 0 foreign keys etc. With absolutely nothing relational done.

[nightwatch]

Chris, I think your claim that Raven is ACID is wrong. You can't be partially ACID - for example Raven doesn't protect you against getting incorrect query results when the index is stale. This breaks the consistency rule.

I don't know why the authors of that article discussed NoSQL at all - after all, TPC-C makes no sense without transactions/ACID

Maybe because they wanted to address the general notion of NoSQL databases being faster and more scalable than SQL - in which case i think they were quite fair in positioning SQL vs NoSQL vs their transactional system 

Oren, your answer is not relevant too. If MySQL can do 1000 inserts per second (it can easily) then I could buy 1000 servers and I would be doing million transactions per second for your 200k dollars. And without any NoSQL madness! Shouldn't therefore MySQL sit on top of the TPC-C list?

 

Last month I implemented a database that was able to do 100 000 transactional inserts per second, on a single server. The server cost me 10 dollars paid to Amazon.

It was an Oracle database (Oracle Berkeley DB to be precise ;) 

Does it mean that for 50 dollars I could beat the TPC-C benchmark? No.

Does it mean that BerkeleyDB is 100 times faster than RavenDB or any other NoSQL? Hell, yes! (in that particular use case)

I think the point of the article is that you can have high performance without NoSQL and NoSQL is somehow 'cutting the corners' to achieve high perf numbers. And their rules were not to cut any corners but implement full TPC-C as required.

R

[Chris Marisic]

On Thursday, May 17, 2012 12:57:46 PM UTC-4, nightwatch wrote:

Chris, I think your claim that Raven is ACID is wrong. You can't be partially ACID - for example Raven doesn't protect you against getting incorrect query results when the index is stale. This breaks the consistency rule.

No. You are absolutely wrong.

RavenDB is fully ACID for Store/Load.

Queries are BASE.

You should not be using query to load documents for modification, you should be using Load. If you use query, you accept that you are working with eventually consistent data. Note as long as you correctly use ETags and optimistic concurrency you will always get a concurrency exception if you attempt to store a stale document.

[Itamar Syn-Hershko]

Let's for a second assume everything you wrote is sound

Why would you care for having such a rapid rate of transactional inserts to a database?

For most OLTP systems, what you care about is reads, and that is where (almost) any non-relational database shines, RavenDB in particular

[nightwatch]

Itamar

I don't think I will ever need more than 50 inserts per second in any OLTP system I'll be working on in my life. I'm just trying to discuss the article that started this topic. The authors have also observed that 500k transactions per second is more than whole humanity can generate so this is far beyond any real-world scenario. But I thought we are talking about maximizing TPC score, not trying to justify the numbers 

BTW 99% of RavenDB users will probably never need 1000 inserts per second  - does it mean you don't have to optimize anything?

In the application I was talking about earlier (that one with 100k inserts per second) insert performance was crucial because it had to store hundreds of millions of records per day. But it wasn't OLTP.

Chris

Even if we assume we are loading data only by document Id

Imagine we have some web store and we are placing an order for an item. 

So we create an order record with a reference to the item: {Id:orders/2893, ItemId:items/12345, Quantity: 1, ....}

And suppose meanwhile someone deletes that inventory item

We commit the transaction and end up with an invalid order that refers a nonexistent document  - so now our database is in some inconsistent state - no ACID even without using queries

[Oren Eini (Ayende Rahien)]

Rafal,

You assume that FK is a requirement for ACID, but that isn't the case.

ACID isn't about FK at all. It is simply that your transactions go in all the way or none at all.

[Chris Marisic]

On Thursday, May 17, 2012 1:48:54 PM UTC-4, nightwatch wrote:

Chris

Even if we assume we are loading data only by document Id

Imagine we have some web store and we are placing an order for an item. 

So we create an order record with a reference to the item: {Id:orders/2893, ItemId:items/12345, Quantity: 1, ....}

And suppose meanwhile someone deletes that inventory item

We commit the transaction and end up with an invalid order that refers a nonexistent document  - so now our database is in some inconsistent state - no ACID even without using queries

 Explain how that exact scenario doesn't exactly occur in a relational model? You're going to use cascade delete and alter the users order they already paid for and delete the item? I think you're confusing ACID with understanding that EVERY SINGLE SYSTEM has some levels of eventually consistency.

So what the order gets through with an invalid item? At some point this going to get resolved, it will get to the shipping department and item 12345 won't exist, so it will get kicked back up the chain and resolved.

[nightwatch]

Yes you're right - FK is an optional constraint - I went a little too far here.
BTW guys I really like Raven, the way it works and the way it makes application development better. And I don't think being transactional/ACID is necessary in every case - especially if in exchange you get great performance and ease of use. But at the same time I agree with the TPC article guys that dropping ACID is the easy solution that sometimes cuts the corners too much (distributed/multi-document transactions?). Yes, Raven supports multi-document/distributed transactions and it's great feature, but it has inherent limitations that haven't been handled in any way - using them you have to  accept all the negative performance consequences.

R

[Itamar Syn-Hershko]

Oh, we are optimizing quite a lot, but we prefer putting our efforts on optimizing reads and indexing with the possible trade off of write speed.

[Chris Marisic]

That's certainly what I want.