Current status of adapting OrientDB for web2py

[Nolan Nichols]

I'm researching the nosql and graph database landscape for a web2py
application that will require the schema to evolve over time and
provide network/graph analysis metrics.

I started by looking at the Tinkerpop (http://tinkerpop.com/) stack
and the Bulbflow (http://bulbflow.com/) python library for interacting
with Tinkerpop graph databases like Neo4j and OrientDB.

It looks like there was interest a few months back in adapting
OrientDB's sql interface for web2py, and there is an open issue:

- http://code.google.com/p/web2py/issues/detail?id=407

A few questions:

What is the current status of an OrientDB/web2py adapter?
Is anyone currently using a graph database with web2py?
Any suggestions for using web2py DAI/templates with non-rdbms sources?

Cheers,

Nolan

[TheSweetlink]

Hello,

I have been using OrientDB and web2py quite successfully for some
months now.

While there are many great projects and efforts to combine these
powerful programs, I do not know of the current status of the web2py
<---> OrientDB adapter.

Bulbflow is a great project too but I could not use it due to OrientDB
specific features that I needed for my project.

Neo4j has had great progress as well.

I ultimately chose OrientDB for several reasons which made it the best
fit for my projects. Here are some of those highlights:

1) multi-master (as far as I know you must pay for commercial license
to get multi-master in Neo4j)

2) the Apache 2.0 license is very liberal.

3) Luca and the OrientDB community are as turbo fast and helpful as
the web2py community. Luca can get patches out in hours to days, not
months to years.

4) SQL syntax + Gremlin graph traversal language = untold power to
grow/analyze your graph dbs.

5) Much more...for details see orienttechnologies.com and click "Learn
More". Do read the entire wiki as there is good documentation and
more coming frequently.


As for getting OrientDB to work with web2py, I have had to write
custom modules to do basic CRUD and used OrientDB's built-in console
to create my schema/maintain the DB.

I've experienced great success with using requests and ujson python
libs to move data back and forth from web2py <--> OrientDB

For Auth I still use a db which will work with web2py's amazing DAL.
I'm sure with enough time you could hack the two to work together but
I prefer to use something proven for auth until such time OrientDB is
included in the DAL.

Massimo has expressed interest in including OrientDB within the DAL
but I would assume that this will take quite some time as he has much
higher priorities for web2py to get to first.

I highly recommend the combination of web2py and OrientDB. It will
take some customization, but once running you will be able to do
things that web2py + traditional RDBMS simply cannot do.

Some of my apps have experienced 10-100x performance boost due to no
longer using JOINS to get data but rather traversing my graph.

Best of luck,
David

On Feb 13, 12:54 pm, Nolan Nichols <nolan.nich...@gmail.com> wrote:
> I'm researching the nosql and graph database landscape for a web2py
> application that will require the schema to evolve over time and
> provide network/graph analysis metrics.
>
> I started by looking at the Tinkerpop (http://tinkerpop.com/) stack
> and the Bulbflow (http://bulbflow.com/) python library for interacting
> with Tinkerpop graph databases like Neo4j and OrientDB.
>
> It looks like there was interest a few months back in adapting
> OrientDB's sql interface for web2py, and there is an open issue:
>

> -http://code.google.com/p/web2py/issues/detail?id=407

[Greg Sier]

+1 on this.
It's in our 2 year plan for a major client.

[Massimo Di Pierro]

You say you have been using it. Do you have a patch? I'll be happy to
take a look and include it.

[David Marko]

I would also appreciate some hints on what libraries do you use to communicate with orientDB, or can you create some simple appliance as demo?

Thanks!

David

[TheSweetlink]

Hello all,

I do not have a patch as it is not for the DAL however I can show you
this handy module I hacked up.

It will add/remove edges to an OrientDB document-graph DB as well as
return the RID of a newly inserted record.

CRUD for now is project dependant on what if any schema you have
implemented in Orient and so not included.

It could use some try, except...to ensure that the transaction
completed but I did not include that as the test for successful insert
will likely rely on your schema if you have one.

NOTE: This is older code that was quickly abstracted from a project to
protect certain parts of code. I'm certain there are several
abstractions/optimizations/fixes that can be applied to this code.
I'll leave it to you to please 2x check and tweak it to fit your
specific project.

The reason for lopping off the first character in some parts is
because RID's are actually strings that begin with a '#'. Please see
http://code.google.com/p/orient/wiki/Concepts#Record and the rest of
the wiki for more details. It is a good read.

@David Marko, I cannot build a whole application as that will take
some time to build an OrientDB instance which would exemplify all the
things you can do.

When I have more free time I may build an appliance because I want
more exposure to both projects.

I can certainly point you in the right direction though and once you
read the wiki everything will make much more sense.


Libs used:

re - to get RID from an INSERTed record
requests - http for humans...GREAT library,
ujson as json - fast moving of JSON data between OrientDB <--> web2py,


Behold! A simple module to add/remove in/out edges with OrientDB's
document-graph db as well as retrieving the RID of a newly inserted
record. May it help you and many more.
http://paste.pocoo.org/show/552147/


-David Bloom


On Feb 16, 12:03 am, Massimo Di Pierro <massimo.dipie...@gmail.com>
wrote:

[bussiere adrien]

Mee to i'am considering using orientdb also.

So if you have any example

regards

Bussiere

[Massimo Di Pierro]

[Massimo Di Pierro]

Let me iterate that writing a DAL adapter does not take much work. All I need is somebody to commit to do some testing for me.

massimo

[Vasile Ermicioi]

hi,

I can write one, 

I tried it (some samples), I like its concept and features (key value db, document db, and graph db, sql  queries,  full text index support and so on)

but I am waiting for the 1.0 release which is planned to be released in mid of April according to their roadmap

[bussiere bussiere]

Thanks by the way for the works.

Bussiere

"Les nouvelles technologies offrent pleins de nouvelles possibilités,
pleins de possibilités d'erreurs surtout en fait."
insurance.aes256 : http://goo.gl/gHyAY

[Nolan Nichols]

Hi Vasile,

You've probably heard that the 1.0 release for OrientDB is out: http://code.google.com/p/orient/downloads/detail?name=orientdb-graphed-1.0.zip

I'd be very curious to take a look at the samples you were working on and if you're still interested in working on a DAL adapter.

Cheers,

Nolan

[luckysmack]

I am curious if anyone has made any advancements with OrientDB. I only recently found out about it and Neo4j. I had been planning on using mongo/couch for my db backend, but based on the way the graph databases are made, it could be of great use. 

@TheSweetlink iv'e also noticed that your pasted example is no longer there, since it seems that paste service no longer exists. gist.github.com perhaps?

Thanks.

[lpg]

bumping this to the future.

[Pumplerod]

I'm curious if anyone has done any more to integrate OrientDB with the DAL.  Or if not, if there is a guide of some sort for creating an adapter.

I'm beginning to design a site around OrientDB and rather than keeping track of two systems one for Auth and one for the rest of my data it seems like it may easier to create the adapter.

[Paolo Valleri]

Hi, contributions are more than welcome :)

You can start by having a look at those other no-sql adapters in https://github.com/web2py/pydal/tree/master/pydal/adapters

Paolo

[Pumplerod]

I had a look, thanks.  Wow.  I think I would need a much better understanding of all the parts before I could take that on.  What are the odds I could make a contribution toward web2py in order to fast-track the adapter?

On Monday, February 13, 2012 at 9:54:05 AM UTC-8, Nolan Nichols wrote:

[黄祥]

any news of orientdb adapter? 

best regards,

stifan