RavenDB on Ubuntu with Mono

[Beaker]

I know it's been done before, and I can see chunks of code in the git
repo where Mono is taken into consideration, but something is evading
me.

I have the latest stable Mono release, latest stable Mono Develop, and
the latest RavenDB source all on Ubuntu 12.04. I can compile just
fine. But when I try to run the server, it fails when it tries to
invoke the MEF calls for SatisfyImportsOnce. After doing a google
search, I found one other individual, about a year ago, who faced a
similar issue, who solved it, and then published his repo. Sadly, his
repo fails the same way. It's the same exact error
"MissingManifestResourceException".

Since I am new to RavenDB, and knew the MEF, but obviously you guys
have been working on this (or have gotten it to work) can someone
tell me what I am doing wrong?

Thanks!
Beaker

[Oren Eini (Ayende Rahien)]

This indicate that MEF had some issue, tried to throw an error, then threw an error trying to throw an error.

http://lists.ximian.com/pipermail/mono-bugs/2011-June/112020.html 

You need to look at the stack trace of the error to see if you can figure out what is wrong there.

[Beaker]

There's only two ravenDB calls in the stack trace. I'll run them and
display -> I think I'm just going to need to step through the code to
get to the root of the problem. I think at the heart of this is going
to be some small fundamental difference between how mono and .NET
actually implement the MEF structure.

On May 22, 9:12 am, "Oren Eini (Ayende Rahien)" <aye...@ayende.com>
wrote:

> This indicate that MEF had some issue, tried to throw an error, then threw

> an error trying to throw an error.http://lists.ximian.com/pipermail/mono-bugs/2011-June/112020.html

[Oren Eini (Ayende Rahien)]

Hi,

We intend to do major work on that post 1.2

If anyone wants to jump in, the major pieces would be building a transactional storage on top of BDB and then just testing.

On Sat, Jun 23, 2012 at 11:46 PM, clayton collie <gbo...@gmail.com> wrote:

Is there any movement on this. I really want to use Raven, but everything else in my infrastructure(rabbit, redis, etc) are geared toward linux.

[clayton collie]

May make for some interesting weekend hacking !!

[Oren Eini (Ayende Rahien)]

Agree, let us know if you need any help

[Michael Weber]

Is there a reason that transactional storage couldn't be built upon sqlite instead of bdb.  It seems like bdb is so low-level as to make the port much harder.

mike

[Oren Eini (Ayende Rahien)]

SQLite can't handle threading properly

[Michael Weber]

That's originally what I thought as well, but from their faq.

"SQLite is threadsafe. We make this concession since many users choose to ignore the advice given in the previous paragraph. But in order to be thread-safe, SQLite must be compiled with the SQLITE_THREADSAFE preprocessor macro set to 1. Both the Windows and Linux precompiled binaries in the distribution are compiled this way. If you are unsure if the SQLite library you are linking against is compiled to be threadsafe you can call the sqlite3_threadsafe() interface to find out."

[Oren Eini (Ayende Rahien)]

It is thread safe in the sense that it won't corrupt your data.

It isn't using thread properly because it is basically using locks all over the place.

[Michael Weber]

ah, got it

[Michael Weber]

Just a heads up that I have started hacking the storage engine for use with BDB.  I have gotten it to the point of being able to store a document and retrieve it, but that is about it, there's obviously quite a lot left to do, but it's a start.  I am using Berkeley DB for .NET (http://sourceforge.net/projects/libdb-dotnet/) as the .net interface to the native dll.  It's a BSD licence, so I hope that's compatible with RavenDB.

For anyone interested, you can track progress at  https://github.com/mweber26/ravendb/tree/bdb-storage 

mike

[Oren Eini (Ayende Rahien)]

That is just awesome

[Oren Eini (Ayende Rahien)]

Would love to see you write some posts about it

[Itamar Syn-Hershko]

A while back I did some reading on LevelDB and it seemed to be a better fit, and a more stable option. Worth comparing the two, especially in a multi-threaded scenarios.

On Fri, Aug 17, 2012 at 7:07 AM, Michael Weber <mtw...@gmail.com> wrote:

[Michael Weber]

I just poked around the code for LevelDB a bit and it looks like it has the same problem sqlite.  All multithreaded access is protected by global locks.  I will do some more digging through the code, but at first glance at least that's what it seems like.

mike

[Michael Weber]

http://mikecodespot.blogspot.com/

[Matt Warren]

I also spent a bit of time looking at LevelDB

The main issue I found was that it doesn't support transactions (see https://groups.google.com/d/msg/leveldb/O_iNRkAoObg/GsDzBdjZ_McJ and http://stackoverflow.com/a/9037073/4500).

So transactions would have to be implemented as a custom layer on top of LevelDB. This is what IndexedDB does inside Chrome, you can see the code here  http://code.metager.de/source/xref/WebKit/Source/WebCore/platform/leveldb/LevelDBTransaction.cpp

[Oren Eini (Ayende Rahien)]

Awesome.

[Oren Eini (Ayende Rahien)]

I already implemented transactions for Munin, that was... touchy.

It is something that I would like to avoid.

[Oren Eini (Ayende Rahien)]

Regarding your blog post.

Note that actually saving to disk is a bit more complex than that.

We no longer do things like this:

using(var ms = new MemoryStream()) { data.WriteTo(ms); dataBuffer = ms.ToArray(); } 

We have users that uses documents that are several MB in size, and we have to stream it to disk, instead of copying the data several times.

Also, you aren't using codecs here, which are required so we can use things like encryption & compression.

Also, what build are you working off?

[Michael Weber]

I don't know about streaming yet with respect to BDB, all examples seems to be full-buffer, but I will have to do more research to figure out streaming writing.

Yes, I am using the codecs, the code in that post is out-dated.

I am working off of the github master branch : https://github.com/ravendb/ravendb/tree/ 

[Oren Eini (Ayende Rahien)]

Please use the one from my fork, github.com/ayende/ravendb

The 1.2 branch

[Michael Weber]

Ok -- I will switch over to that branch (for the facet changes as well, and issue a new pull request).

It looks like there is a way to do partial writes in BDB.  I will try to figure out how to integrate it with c# streams.

[Michael Weber]

Oren,

After playing with the BDB port of the storage engine for a while now, I've begun to thing that some of the things that the storage engine is responsible for is duplicated in all storage engines.  For example: EnsureDocumentEtagMatchInTransaction or EnsureNotLockedByTransaction.  If the logic of these ever changes then changes to all storage engines have to be made.   It seems like this logic should be pulled out of the storage engine and the storage engine should simply be responsible for getting the data.  I was thinking something along the lines of:

interface IDocumentReader {

   Document FindDocument(string key);

   void AddDocument(Document a);

   void DeleteDocument(string key);

}

class Document {

public int MetadataSize { get }

public int DocumentSize { get }

public Guid Etag { get }

public Guid LockedByTransaction { get }

public string Key { get }

public DateTime LastModifiedTime { get }

public RavenJObject Metadata { get }

public Stream DocumentStream { get }

}

Again this is just an example and we would have to look a little closer at the operations.  Even a simple IDocumentStorageActions.AddDocument has tons of logic associated with verifying etags, checking for active transactions, this is complicated stuff and seems like it should be outside the storage engine.

mike

[Oren Eini (Ayende Rahien)]

There are some utility functions there that are used, but most of the time, those things are _really_ tied to the internal of how the storage engine actually does things.

[Sergey Shumov]

Any updates on the topic? https://github.com/mweber26/ravendb/commits/bdb-1.2 seems to be abandoned.

[Michael Weber]

It's not 100% dead, it was waiting on Oren to finish the final push for 2.0.  But it now looks like he may have other ideas for mono support.

https://groups.google.com/forum/?fromgroups=#!searchin/ravendb/bdb/ravendb/fB2YpeBm5wk/020YBUvyK8cJ

mike

[clayton collie]

BangDB looks nice

http://highscalability.com/blog/2012/11/29/performance-data-for-leveldb-berkley-db-and-bangdb-for-rando.html

[Oren Eini (Ayende Rahien)]

That looks nice, yes.

--
You received this message because you are subscribed to the Google Groups "ravendb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Khalid Abuhakmeh]

Just read the link on BangDB. How would its performance match up to Esent. Would it be faster, and would it still be ACID?

[Oren Eini (Ayende Rahien)]

No idea to all questions.

[Matt Johnson]

BangDB looks pretty awesome, but it's pure C++.  We would have to write our own managed wrapper assembly (C++/CLI).  BerkeleyDB already has a C# API.

Given the choice, I'd still go for BangDB - but there's certainly more work involved.

[clayton collie]

Only question, from a brief look at the API, is whether streaming/chunking is possible for large values. Didn't want to download the whole kit and kaboodle to find out.

[Oren Eini (Ayende Rahien)]

Matt,

From the time that I deal with BDB C# API.

It was like swimming in the sewers.

Just to give you some ideas, The _esent_ managed library had a bunch of bugs in it that we discovered early on.

I shudder to think what is the current status there.

[Oren Eini (Ayende Rahien)]

Not following you.

[Ryan Heath]

From the docu on that site it seams to always return the whole data blob/chunk when one calls the get method. But perhaps the docu is not thorough or not up-to-date with its current state. 

// Ryan

Sent from my iPhone 

[Ryan Heath]

Hmm, if we manage to create our own managed wrapper, we can at least control the odeur ;)

// Ryan

On Thursday, February 14, 2013, Oren Eini (Ayende Rahien) wrote: