Small management webapp and architectural question

[Yakulu]

Good morning,

I'm a newcomer into the JVM ecosystem (I'm mainly a Python developer)
and one thing I really like here are the graph databases. I've worked
a little with object-oriented and document-oriented databases last
years but I was always missing something.

With Neo4j, I can have all I wanted :

- ACID, transactions
- schema-less data
- embeddable solution
- a sort of native indexation, thanks to relationships between nodes

I'll work on a free software project dedicated to some "Youth and
Culture Centres" (not sure there are equivalents outside France : this
is a place where people can take lessons, do activities at low prices,
driven by non-profit organizations). Each centre has its own "way to
do", so I really need a good dose of flexibility. Schemaless and
relationships seems to me very fine here.

At first, this tool will be a web application for helping managing the
centre. After that, we plan to expand it to the centre's subscribers,
providing them a sort of simple private social network (besides
flexibility, there's another reason for me to choose a graph
database).

Nevertheless, I wonder about few things :

1. Have you ever experimented a business management application based
on Neo4j ? Have you some advices ?

2.a I don't need to have a high rate of insertions or lots of complex
queries. The tool won't stock millions of nodes and relationships
before decades. Nevertheless, I can have to install it on old
hardware. Can Neo4j run on... a Pentium III with 128 or 256MB RAM (or
little ARM box.... let's say we'll run a UNIX OS) ?

2.b We plan to offer the service through the Web for people who don't
want a local install. Have you ever used Neo4j with many, dozens,
maybe hundreds of small databases on the same computer (I can estimate
them at maximum 50.000 nodes and maybe 500.000 relationships per
database in ten years) ?

Thanks a lot, and sorry for my english !

[Michael Hunger]

Welcome to the land of graphs, your English is great.

I think your requirements for your system are very reasonable and shouldn't pose a problem. Sounds like a good cause.

You can also use python against Neo4j-server which should help you get going faster (e.g. py2neo) there are also embedded jvm-python bindings.

As resources, have a look at the videos a video.neo4j.org and the manual at docs.neo4j.org

Regarding your memory requirements, that's not that much, we are running the test-plans on heroku with a similar size, they work but are not most performant and for some queries the memory is not enough to hold intermediate results.
There we also have the approach of hosting many databases on a single machine (hosted on AWS) where we even suspend databases that were not in use for a while and restart them on the first http-request, which saves a lot of resources.

I'm not so sure about existing applications in that space, Thomas Baum wrote a CRM like app using Neo4j, Java, GWT

In general grab a whiteboard and draw the elements of your domain, that becomes your graph and then try to work from your individual use-cases.

I think before wondering about deployments you should work on getting a proof of concept done and see how well you can model your domain and focus on the integration of your webapp with the graph database.

After that we can still discuss the deployment, I think cloud deployment of the graphdb instances is also an option (e.g. on heroku).

Cheers

Michael

[James Thornton]

On Wednesday, August 1, 2012 9:23:34 AM UTC-5, Yakulu wrote:

2.b We plan to offer the service through the Web for people who don't
want a local install. Have you ever used Neo4j with many, dozens,
maybe hundreds of small databases on the same computer (I can estimate
them at maximum 50.000 nodes and maybe 500.000 relationships per
database in ten years) ?

Rexster is a REST server like Neo4j Server, and with Rexster (https://github.com/tinkerpop/rexster/wiki/) you can host multiple Neo4j instances on one server, whereas with Neo4j Server it's one database per server. Bulbs (http://bulbflow.com) is a Neo4j Python client library, and it supports both Neo4j Server and Rexster, whichever you choose.

- James 

[Michael Hunger]

Does Rexter take care of the correct mmio and heap configurations of those neo4j instances? Neo4j assumes it runs alone in its JVM so there might be implications.

Michael

[Yakulu]

Le mercredi 1 août 2012 17:18:44 UTC+2, Michael Hunger a écrit :

Welcome to the land of graphs, your English is great.

Thank you.

 

You can also use python against Neo4j-server which should help you get going faster (e.g. py2neo) there are also embedded jvm-python bindings.

Yes I've seen there are several solutions to interact between Python and Neo4j.
But because I really like the embeddable opportunity (I wish centres can install and test the app without strong computer skills), I'm looking for going to the JVM.
 

Regarding your memory requirements, that's not that much, we are running the test-plans on heroku with a similar size, they work but are not most performant and for some queries the memory is not enough to hold intermediate results.

Fine. As we won't need huge performance here, I think it'll be fast enough.
 

There we also have the approach of hosting many databases on a single machine (hosted on AWS) where we even suspend databases that were not in use for a while and restart them on the first http-request, which saves a lot of resources.

Do you use an Rexter for that, as mentioned by James ?
 

In general grab a whiteboard and draw the elements of your domain, that becomes your graph and then try to work from your individual use-cases.
I think before wondering about deployments you should work on getting a proof of concept done and see how well you can model your domain and focus on the integration of your webapp with the graph database.

Thanks for your advices.

[Yakulu]

Le mercredi 1 août 2012 23:01:39 UTC+2, James Thornton a écrit :

Rexster is a REST server like Neo4j Server, and with Rexster (https://github.com/tinkerpop/rexster/wiki/) you can host multiple Neo4j instances on one server, whereas with Neo4j Server it's one database per server. [...]

Thanks James.

I've already seen the Rexter option, but I didn't know the Neo4j Server limit per server. If I don't need to have a client-server architecture (so neither the Neo4j Server, neither Rexter), do you think I can use multiple embedded instances ?

[Peter Neubauer]

Hi there,
you can have any number of embedded instances in your JVM at the same
time. Just treat them as singletons and pass them around. Also, don't
open them against the same store of course. It depends then on your
heap on how well the caches work, but that remains to be tested in
your application then

Good luck, let us know how it goes!

Cheers,

/peter neubauer

G: neubauer.peter
S: peter.neubauer
P: +46 704 106975
L: http://www.linkedin.com/in/neubauer
T: @peterneubauer

Wanna learn something new? Come to @graphconnect.