OrientDB and C#(.Net)

[Martin Bean]

So...

I like OrientDB. I like it so much, I want to use it no matter what!
Except when I need to give up C# ;) (problem lies in large amount of
code already made and the usage of WPF(Windows Presentation
Foundation) and WWF(Windows Workflow Foundation) which is needed).

Running OrientDB as a server and connecting to it from .Net is not a
big deal. The REST interface can handle this. It becomes a problem
when you want to run OrientDB embedded or the REST interface becomes
to slow.

I thought about implementing a binary remote connection implementation
in .Net. Started it, but I found my self copying classes from OrientDB
source and adjusting them to C#. This irritated me, fearing the time
changes would be made in OrientDB and I had to run through my C# code
to fix all kinds of things.

So I looked a bit further and found IKVM( http://www.ikvm.net ). I
compiled the JAR files to a single .Net DLL... and it
WORKED!!!!!!!!!!!!!!!!!!! almost......... As it turns out MXBeans are
not implemented in ikvm. This means the OMemoryWatchDog contains code
which is not supported. To see if other problems would rise when
running OrientDB on ikvm I killed OMemoryWatchDog(commenting all code,
except basic structure).

The results are very promising!! So far I tested creating a remote
connection to a server running normal(just Java). I was able to
execute a query using the OrientDB classes/methods from .Net. This
means a client running in native mode(OrientDB jar's compiled/
converted to native .Net assembly).

I'm working on starting a server. Almost got it to work, except for
some PATH issues.

So... the question is: Is it possible to implement a OMemoryWatchDog
which is 'other virtual machines' friendly? I'm no Java expert, so
maybey other people here have some ideas to implement the features
needed for memory management while using native Java and not
MXBeans :)

This would enable .Net developers to use OrientDB more intensively.

(I just the rc7 snapshot)

Cheers!

[Luca Garulli]

Hi Martin,

this is AWESOME! For the fact that OrientDB can be used from C# and .NET platform in general. For IKVM... wow! And for your effort.

If you want to release it as open source why don't publish the project on Github or Google Code? I'll be glad to link it from the official web site.

About MXbeans and OMemoryWatchDog I've to explain how works OrientDB that it's quite new and strange: some internal structures like OMVRB-Tree and Cache are able to allocate, by default, as much memory as they can use and the OMemoryWatchDog agent will warn the classes to free resources.

So in your case you could avoid using OMemoryWatchDog just setting some internal properties defining fixed rules to get maximum memory and not "as much as you can".

Lvc@

[Martin Bean]

Luca,

The fun part is: there is nothing to open source!!

Download IKVM from the website( http://www.ikvm.net ) and unzip it to
a folder.

Open a cmd, and CD to the lib folder of OrientDB.

Execute: ikvmc -out:orientdb.dll -target:library orient-commons-1.0rc7-
SNAPSHOT.jar orientdb-client-1.0rc7-SNAPSHOT.jar orientdb-core-1.0rc7-
SNAPSHOT.jar orientdb-enterprise-1.0rc7-SNAPSHOT.jar orientdb-
server-1.0rc7-SNAPSHOT.jar orientdb-tools-1.0rc7-SNAPSHOT.jar
persistence-api-1.0.jar

It will produce orientdb.dll. Reference this DLL in your .Net project,
along with the IKVM.OpenJDK.Core, IKVM.OpenJDK.Management and
IKVM.OpenJDK.XML.Bind assemblies from the ikvm project.

In your .Net file this can be executed:

OGraphDatabase database = new OGraphDatabase("remote:localhost/
TestDB");
database.open("admin", "admin");

java.util.List result = database.query(new
com.orientechnologies.orient.core.sql.query.OSQLSynchQuery("select
from TestClass"));

Still having some problems getting ORIENTDB_HOME enviroment variable
into OrientDB. It doesnt find the enviroment variable. Is it possible
to give it as a parameter somehow, so OrientDB wont look for it in the
system enviroment variables?? This would help a lot.

About OMemoryWatchDog: Could you give me some more guidelines on how
to avoid using it?? Perhaps making it a setting of OrientDB ???
Because: MXBeans are created in ikvm, just not filled with code. So it
compiles, and as long as the code using MXBeans is not run its ok.

If you want, I could write a wiki page explaining how to setup
OrientDB with ikvm so it can be placed on the official wiki site??

[Luca Garulli]

On 23 November 2011 17:46, Martin Bean <mar...@ulep.eu> wrote:

Luca,

The fun part is: there is nothing to open source!!

 

Nice,

but what about the removal of the WatchDog? Maybe some scripts are enough?

Download IKVM from the website( http://www.ikvm.net ) and unzip it to
a folder.

Open a cmd, and CD to the lib folder of OrientDB.

Execute: ikvmc -out:orientdb.dll -target:library orient-commons-1.0rc7-
SNAPSHOT.jar orientdb-client-1.0rc7-SNAPSHOT.jar orientdb-core-1.0rc7-
SNAPSHOT.jar orientdb-enterprise-1.0rc7-SNAPSHOT.jar orientdb-
server-1.0rc7-SNAPSHOT.jar orientdb-tools-1.0rc7-SNAPSHOT.jar
persistence-api-1.0.jar

It will produce orientdb.dll. Reference this DLL in your .Net project,
along with the IKVM.OpenJDK.Core, IKVM.OpenJDK.Management and
IKVM.OpenJDK.XML.Bind assemblies from the ikvm project.

 

This could be described in a WiKi page or just providing a .bat and .sh scripts

Furthermore if Andrey can insert it into the Continuous Integration we could have a always fresh .dll compiled from latest sources.

In your .Net file this can be executed:

OGraphDatabase database = new OGraphDatabase("remote:localhost/
TestDB");
database.open("admin", "admin");

java.util.List result = database.query(new
com.orientechnologies.orient.core.sql.query.OSQLSynchQuery("select
from TestClass"));

Cool!

 

Still having some problems getting ORIENTDB_HOME enviroment variable
into OrientDB. It doesnt find the enviroment variable. Is it possible
to give it as a parameter somehow, so OrientDB wont look for it in the
system enviroment variables?? This would help a lot.

Yes,

just call via API:

System.setProperty("ORIENTDB_HOME", "C:/orientdb");

 

About OMemoryWatchDog: Could you give me some more guidelines on how
to avoid using it?? Perhaps making it a setting of OrientDB ???

These are the parameters that, if setted, make the WatchDog not necessary:

cache.level1.size 

cache.level2.size

mvrbtree.lazyUpdates

For more information look at http://code.google.com/p/orient/wiki/PerformanceTuning 

Because: MXBeans are created in ikvm, just not filled with code. So it
compiles, and as long as the code using MXBeans is not run its ok.

Nice, so no changes to the code. Super.

 

If you want, I could write a wiki page explaining how to setup
OrientDB with ikvm so it can be placed on the official wiki site??

It would be great. Just give me your Google Code account.

Lvc@

[Luca Garulli]

I've created the empty page: http://code.google.com/p/orient/wiki/DotNET

Lvc@

[Martin Bean]

Short on time right now... but setting the ORIENTDB_HOME resulted in
this:

2011-11-23 06:39:56:052 INFO [OLogManager] OrientDB Server v1.0rc7-
SNAPSHOT is s
tarting up...
2011-11-23 06:39:56:305 INFO [OLogManager] -> Loaded memory database
'temp'
2011-11-23 06:39:56:344 INFO [OLogManager] Listening distributed
connections on
0.0.0.0:2424
2011-11-23 06:39:56:362 INFO [OLogManager] Listening http connections
on 127.0.0
.1:2480
2011-11-23 06:39:56:450 INFO [OLogManager] OrientDB Server v1.0rc7-
SNAPSHOT is a
ctive.waiting...

2011-11-23 06:39:58:452 WARN [OLogManager] Cluster 'default': current
node is th
e new Leader Node
2011-11-23 06:39:58:456 INFO [OLogManager] Cluster 'default':
listening for dist
ributed nodes on IP multicast: /235.1.1.1:2424

OrientDB.NET is RUNNING!! And I can connect to it with the Java
version of Console.bat !! :P Can do info, select etc.

Will experiment a bit more later on, to see if its stable etc.

Will work on Wiki too, I'll send you my google username shortly!

gotta run!

On Nov 23, 6:23 pm, Luca Garulli <l.garu...@gmail.com> wrote:
> On 23 November 2011 17:46, Martin Bean <mar...@ulep.eu> wrote:
>
> > Luca,
>
> > The fun part is: there is nothing to open source!!
>
> Nice,
> but what about the removal of the WatchDog? Maybe some scripts are enough?
>

> Download IKVM from the website(http://www.ikvm.net) and unzip it to

> For more information look athttp://code.google.com/p/orient/wiki/PerformanceTuning

[Luca Garulli]

Cool!

Who knows about performance...

Lvc@

[Martin Bean]

Any ideas on how to test performance?? Some example code maybey? (may
be in Java, I'll make C# of it and run the native Java version)

[Luca Garulli]

A first test could be what is executed on:

> ant stress-test

In particular these classes:

com.orientechnologies.orient.test.database.speed.LocalCreateFlatSpeedTest

com.orientechnologies.orient.test.database.speed.LocalCreateDocumentSpeedTest

com.orientechnologies.orient.test.database.speed.LocalCreateObjectSpeedTest

The test requires the demo database that is created after the test cases:

> ant test

Lvc@

[Martin Bean]

> > About OMemoryWatchDog: Could you give me some more guidelines on how
> > to avoid using it?? Perhaps making it a setting of OrientDB ???
>
> These are the parameters that, if setted, make the WatchDog not necessary:
>
> cache.level1.size
> cache.level2.size
> mvrbtree.lazyUpdates
>
> For more information look athttp://code.google.com/p/orient/wiki/PerformanceTuning

I added the properties to the config file:

<entry name="cache.level1.size" value="0"/>
<entry name="cache.level2.size" value="0"/>
<entry name="mvrbtree.lazyUpdates" value="1"/>

AND set them in code(to be sure):

java.lang.System.setProperty("cache.level1.size", "0");
java.lang.System.setProperty("cache.level2.size", "0");
java.lang.System.setProperty("mvrbtree.lazyUpdates", "1");

But it still executes the OMemoryWatchDog methods, crashing my
application. The only way I run OrientDB(client or server) is to
adjust OMemoryWatchDog(commenting out most of the code).

Any suggestions?

[Martin Bean]

Running the stress-test starts with the following(I did run ant test
first which succeeded):

     [java]
com.orientechnologies.orient.core.exception.ODatabaseException: Can't
open database     [java]     at
com.orientechnologies.orient.core.db.raw.ODatabaseRaw.open(ODatabaseRaw.java:
101)     [java]     at
com.orientechnologies.orient.core.db.ODatabaseWrapperAbstract.open(ODatabaseWrapperAbstract.java:
46)     [java]     at
com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.open(ODatabaseRecordAbstract.java:
101)     [java]     at
com.orientechnologies.orient.test.database.speed.LocalCreateFlatSpeedTest.init(LocalCreateFlatSpeedTest.java:
46)     [java]     at
com.orientechnologies.common.test.SpeedTestData.executeInit(SpeedTestData.java:
177)     [java]     at
com.orientechnologies.common.test.SpeedTestData.go(SpeedTestData.java:
42)     [java]     at
com.orientechnologies.orient.test.database.speed.LocalCreateFlatSpeedTest.main(LocalCreateFlatSpeedTest.java:
35)     [java] Caused by: java.lang.NullPointerException     [java]  
  at
com.orientechnologies.orient.core.storage.fs.OFileClassic.writeData(OFileClassic.java:
197)     [java]     at
com.orientechnologies.orient.core.storage.fs.OFileClassic.setSoftlyClosed(OFileClassic.java:
184)     [java]     at
com.orientechnologies.orient.core.storage.fs.OFileClassic.close(OFileClassic.java:
45)     [java]     at
com.orientechnologies.orient.core.storage.impl.local.OSingleFileSegment.close(OSingleFileSegment.java:
78)     [java]     at
com.orientechnologies.orient.core.storage.impl.local.OStorageLocalTxExecuter.close(OStorageLocalTxExecuter.java:
56)     [java]     at
com.orientechnologies.orient.core.storage.impl.local.OStorageLocal.close(OStorageLocal.java:
290)     [java]     at
com.orientechnologies.orient.core.storage.impl.local.OStorageLocal.open(OStorageLocal.java:
197)     [java]     at
com.orientechnologies.orient.core.db.raw.ODatabaseRaw.open(ODatabaseRaw.java:
83)     [java]     ... 6 more

[Martin Bean]

This was btw with the native Java version! Nothing to do with ikvm
or .Net :)

[Luca Garulli]

Try to create an empty database by opening the console under "bin" directory and type:

> create database local:../databases/demo admin admin local

Then run the stress test only.

Lvc@

[Anton Kulaga]

>The fun part is: there is nothing to open source!!

>Download IKVM from the website( http://www.ikvm.net ) and unzip it to
a folder.

As a C# developer I think that seperate ikvm github brunch would be very useful. Simply because there are issues and pecularities when converting orient with ikvm ( I tried few month ago and faced so many bugs and strange behaviour that gave it up) and it is better to have a separate brunch that is ikvm optimised (probably there are some performance differencies and bottlenecks in ikvm ) and ikvm stable, verified by other C# developers and thus suitable for production usage.

[Martin Bean]

To be honest: I think a native C# port is the best solution. But thats
a lot of work. And you need to maintain it(new features/fixes in the
java version need to be implemented).

I'm testing with ikvm 7.0RC0 and it all runs well. First test was with
the latest stable version(0.46.01) and I didnt get any errors too.
Seems ikvm became mature in the last months??

I've done 1 test with the OrientDB ikvm native version. Results are
not that good. The LocalCreateFlatSpeedTest runs twice as long. This
is inserting 1000.000 records, nothing else.

Still this is usable I think. The client code to connect to a java
server is very usefull. We also want to use the OrientDB.Net server
code to run embedded(single user), speed is not that important in that
case.

Maybey I should look seriously at porting OrientDB to .Net. Time is an
issue, so for now I'll go for ikvm. But if other C# developers are
interested in porting OrientDB as a project(with the approval of Luca
ofcourse), I'm willing to participate!

On Nov 24, 10:35 pm, Anton Kulaga <antonkul...@gmail.com> wrote:
> >The fun part is: there is nothing to open source!!

> >Download IKVM from the website(http://www.ikvm.net) and unzip it to

[Luca Garulli]

Hi,

I agree to have a .NET native client but it's too much expensive and I don't know someone can afford this.

I think that IKVM solution gives high result to very lower cost. If it's the double slower than Java I mean it's a great result! It's always faster than any other NoSQL dbms :-)

Lvc@

[Andreas Rytina]

Hello Martin,
I would also like to do some performance tests of the .NET native client with my notebook and my testdata. Could you provide the DLL, maybe a link to it on the wiki?

Cheers Andy

2011/11/25 Luca Garulli <l.ga...@gmail.com>