DWH with OrientDB?

[Adolfo Rodriguez]

Hi Luca, All,

sorry for being out for the last 3 months. Despite I keep my eyes on
the forum, I have been working in client side and, also, I have
relocated to Lima, Peru last month, to be able to afford to work in my
project, as in Europe is not possible right now. At least for me.

I plan to go back now to server side and I was planning the task to
create a "kind of" datawarehouse with OrientDB. I have not seem too
much about DWH experiences with graph DBs, but some ideas come to
mind, and I would like to hear your opinions:

In DWH there are 2 important technologies of analysis:

* OLAP: In my opinion OLAP is a workaround to overcome the querying
limitations of RDBMS. It is just about moving data to different data
model and repository (call it multidimensional or whichever you like),
closer to the one representing the business side. In this way, it
enables querying capabilities that would be not feasible over
transactional data. If this is right, with graph DBs traversing
concept, OLAP could be obsolete, as you would be able to query the
graph without too much size limitations (different thing is if you
want to decouple analysis and transactionals but, if so, real time
analysis is lost). Can anyone confirm? Projects as Pentaho Mondrian
would not be of usage anymore since they are targeted for RDBMS.

(as a personal comment, technology has evolved in many areas, as patch-
over-patch, to a big "fit for purpose" nonsense).

* Data Mining is about discovering new patterns in data and learning
from it. I think there is a huge potential making projects as R or
Weka conversant with OrientDB. WDYT?

* I was thinking in integrating Gephi in order to enjoy a visual
handling over my database. But I have just realized that Gephi is only
for visualization and analysis, but does not allow 'designing a data
model'. I have not seeing projects around allowing to design a graph
datamodel. The closer could be http://protege.stanford.edu/ but it is
targeted for OWL ontologies.
So I was considering if creating something ad hoc with some graph GWT
library, (e.g. http://code.google.com/p/gwt-links/
or
http://kolos.math.uni.lodz.pl/~balon/gwt-diagrams-demo/pl.balon.gwt.diagramsexample.GwtDiagramsExample/GwtDiagramsExample.html).
On the other hand, I do not think there are solid theoretical
frameworks about how analyzing actual business concepts represented in
graph technology. So, it is quite a new area from a business point of
view.

I would appreciated if you share your thoughts for OrientDB pointing
to this direction. Anything already planned?

Thanks, regards,

Adolfo

[Geoff]

Adolpho,

The two main issue that you need to address to mimic an OLAP system is
how to handle aggregations and how to query the system. Being able to
transverse related objects quickly is great but you also need to be
able to transverse aggregated results of those objects in a structured
manner. Most of these systems calculate and materialize aggregations
to allow for rapid analysis. Additionally any OLAP tool needs to
support the MDX query syntax so that end user reporting tools can
access the data in a standard manner - similar to what Orient has done
with SQL.

Cheers,
Geoff

> datamodel. The closer could behttp://protege.stanford.edu/but it is

> targeted for OWL ontologies.
> So I was considering if creating something ad hoc with some graph GWT
> library, (e.g.http://code.google.com/p/gwt-links/

> orhttp://kolos.math.uni.lodz.pl/~balon/gwt-diagrams-demo/pl.balon.gwt.d...).

[Adolfo Rodriguez]

Hi Geoff,

I really do not want to mimic OLAP systems. I was considering the
possibility of getting rid of them, if traverse mechanism in Graph DBs
could calculate the aggregated results, on the fly, quick enough.

I understand the motivation of decoupling transactional systems from
analytic systems. This is different, as is more related to good
practices (with current state or arts) and not technical limitations.

But I wondered if OLAP, as a fix to the technical limitation,
consequence of querying burden restrictions, could be overcame with
Graph DBs.
But probably aggregations can get too complex to make them all on the
flight. I agree with you on that.

Thanks.

> > datamodel. The closer could behttp://protege.stanford.edu/butit is

[Geoff]

Hi Adolpho,

Here is a good resource for a set of test cases for performance
testing against popular OLAP tools

http://www.olapcouncil.org/research/bmarkly.htm

A benchmark test harness against this data set would be a good way to
determine strengths and weaknesses of Orient for typical OLAP use
cases.

Regards,
Geoff

[Luca Garulli]

Interesting! I always though OrientDB can outperforms OLAP tools because the power of relationships. Anyone would like to start to work on this?

Lvc@

[Luca Garulli]

Hi,

NuvolaBase is working for a company in gaming with billions of records. What they asked was an extremely fast DWH so we decided to built something raw using OrientDB. I don't know if it could be a product because it's quite raw but probably could be available as a framework for developers.

Lvc@

On 10 November 2012 14:47, SandeepGaur <sandee...@smsbsystems.com> wrote:

Hello Adolfo,

Did you try to use OrientDB for DWH. If yes can you please share your findings

regards

--
 
 
 

[Adolfo Rodriguez]

In relation to your question Sandeep, well, have not reach this point yet. I think also OrientDB 2.0 will include some proprietary Business Intelligence functionality. Probably Luca can extend this info further.

Adolfio

[Valentin Popov]

Luca, hi

Do you have any results of DWH work?

Regards
Valentin

[Luca Garulli]

Hi Valentin,

We've users that store data on OrientDB like a DWH to do analysis. But the point is that using OrientDB as primary database allows you to reduce the need of DWH because the reason to denormalize data is the JOIN cost that doesn't exist in OrientDB.

Lvc@

--

---
You received this message because you are subscribed to the Google Groups "OrientDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to orient-databa...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Valentin Popov]

Thank you for you response 

We will use ODB for storing information like user traffic, number of attachments per user, kind if attachments etc. So scheme is under development right now. 

Regards

Valentin 

PS. Thanks for cool DB ;)  

You received this message because you are subscribed to a topic in the Google Groups "OrientDB" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/orient-database/gefPiz4KyF4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to orient-databa...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

 С Уважением,

Валентин Попов

[Luca Garulli]

Hi Valentin,

What I can suggest is to avoid creating a RDBMS like schema but rather thinking in graph. Take a look at:

http://www.slideshare.net/lvca/why-relationships-are-cool-but-join-sucks-28997951

Soon we'll publish a monthly FREE Webinar about this content.

Lvc@

[Valentin Popov]

Thank you very much! 

Will look at it

 С Уважением,

Валентин Попов