Have anyone seen this already? (OrientDB published the "comparison", hmm, "report")

[Andrii Stesin]

http://orientdb.com/orientdb-vs-neo4j/

what are they speaking about, I wonder?!

WBR,

Andrii

[unreal...@googlemail.com]

So I would like to see a benchmark that majors on  the 'deep search' performance of the various graph databases, which would draw upon:

1. Multi threading (for single searches mainly) but for other operations also - limited by the capability of the graph DB 

2. In memory enhancements.- limited by the capability of the graph DB

3. The ability to pointer chase (for optimisation)  - limited by the capability of the graph DB

4. The graph data should comprise complex relationships,  not just straight forward hierarchies. 

5. The benchmark should be scalable (including the ability to fully utilise very capable nodes).

Wayne.

[Max De Marzi Jr.]

It's marketing garbage. Some flawed benchmark that is 5 years old (performed in 2012) on version 1.6.1. 

Never trust vendor benchmarks:

See: https://maxdemarzi.com/2015/10/16/benchmarks-and-superchargers/

and bottom of https://maxdemarzi.com/2017/01/23/our-own-multi-model-database-part-6/

[Max De Marzi Jr.]

This POC hits a bunch of your points => https://maxdemarzi.com/2017/01/06/multi-threading-a-traversal/

[unreal...@googlemail.com]

So is there a generic way of parallel threading for searches ?

match (n:Entity)-[*4]-(p) where p.name contains "......."   Takes a long time on my system and uses 1 thread....

If so, could this not be part of apoc or the like ?

[Andrii Stesin]

On Fri, Mar 31, 2017 at 5:54 PM, Max De Marzi Jr. <maxde...@gmail.com> wrote:
>
> It's marketing garbage. Some flawed benchmark that is 5 years old (performed in 2012) on version 1.6.1.
>
> Never trust vendor benchmarks:
>
> See: https://maxdemarzi.com/2015/10/16/benchmarks-and-superchargers/
> and bottom of https://maxdemarzi.com/2017/01/23/our-own-multi-model-database-part-6/

Exactly.

[Michael Hunger]

There is some of that in apoc.cypher.mapParallel, but it's not as performant as it should be, I have to check again what keeps it back from performing.

How many CPUs do you have on your system?

Cheers, Michael

--
You received this message because you are subscribed to the Google Groups "Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andrii Stesin]

Dear Michael,

It seems to me that you somehow mistargeted this last message.
Probably it should go to some different topic :)

WBR,
Andrii

>> email to neo4j+un...@googlegroups.com.

>> For more options, visit https://groups.google.com/d/optout.
>
>

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Neo4j" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/neo4j/sx_2kxuWz8I/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> neo4j+un...@googlegroups.com.

[unreal...@googlemail.com]

I have 128 threads. However access to systems with much more than this is an option. Currently though, I cannot get Neo4j to scale....

Wayne

To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+un...@googlegroups.com.

[Max De Marzi Jr.]

Scale to do what exactly?

[unreal...@googlemail.com]

My use case is that, given a very capable single node system,  how do I make a single invocation of the example above (or the like) fully utilize this node ?

In other words,  it goes back to the theme that I mentioned in a previous post of making single deep searches parallel.

I find it difficult/impossible to keep such a system busy with serial sets of matches.   Subsequent searches depend on the o/p of previous searches.

Wayne

[Michael Hunger]

Btw. your query

match (n:Entity)-[*4]-(p) where p.name contains "......." 

Add a label to p, add an index to :Label(name)

Then it should perform better,

What do you do with the results from that pattern match ?

How many do you return?

Michael

To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+unsubscribe@googlegroups.com.

[unreal...@googlemail.com]

Interesting - I rarely observe a speed up adding the 2nd label but I will bear this in mind.

The number of results returned vary from 0 to tens of thousands or perhaps just counts.

I often parse these and feed into new queries. I use Linux bash for this; it doesn't really matter; its the queries that take the time.

So should the apoc cypher.mapParallel directive improve the real-time performance / parallelisation of my use case above ?

There are few examples of using this directive on the web.

How might I re-write :

MATCH (n:Entity) where n.name IN ['a1', 'a2', 'a3', 'a4', 'a5'] WITH collect(n) as nodes UNWIND nodes as n UNWIND nodes as m MATCH path = allShortestPaths( (n:Entity)-[*..2]-(m:Entity) ) RETURN path limit 1000;

using the mapParallel directive as an example ?

Wayne

[Michael Hunger]

The addition of the 2nd label (+ index) will change your query from a full graph scan with property filter to an index lookup or seek with an expand, which is many times more efficient.

If you use counts you should also know when to use size ((n)-[:FOO]->()) to get a fast degree.

This should work:

WITH 10 as partitions

MATCH (n:Entity) where n.name IN ['a1', 'a2', 'a3', 'a4', 'a5'] 

WITH partitions, collect(n) as nodes 

UNWIND nodes as n 

call apoc.cypher.mapParallel2("

MATCH path = allShortestPaths( (n)-[*..2]-(_) ) 

RETURN path

", {n:n}, nodes, partitions) yield value

RETURN value.path as path limit 1000;

Cheers, Michael

To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+unsubscribe@googlegroups.com.