Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
Message from discussion Logging Performance and Query Times (via console and SDN)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
BtySgtMajor  
View profile  
 More options Oct 1 2012, 12:24 pm
From: BtySgtMajor <btysgtma...@gmail.com>
Date: Mon, 1 Oct 2012 09:24:23 -0700 (PDT)
Local: Mon, Oct 1 2012 12:24 pm
Subject: Re: [Neo4j] Logging Performance and Query Times (via console and SDN)

Just thought I'd try again here.  Concerning;

>> What's slowing things down is this: My conversion of findAll()'s return

type (i.e. a ClosableIterable) into a Collection (via
IteratorUtil.asCollection(...) is what's taking up the majority of the time.

Has anyone come across this before?  All my nodes/relationships are left at
the default of being lazily loaded, so I'm hoping I'm doing something else
somewhat obvious.  And, the size of the iterable/collection is only 550
nodes.

Thanks again!

On Friday, September 28, 2012 5:08:32 PM UTC-4, BtySgtMajor wrote:

> Actually, something I just came across and hadn't anticipated.

> The repository call is actually pretty quick; on the order of several
> seconds (like no more than 5-10 tops).

> While it could be better (given that it's only 550 nodes being returned),
> it's sufficiently fast for now.

> What's slowing things down is this: My conversion of findAll()'s return
> type (i.e. a ClosableIterable) into a Collection (via
> IteratorUtil.asCollection(...) is what's taking up the majority of the time.

> So, this is either something I've forgotten about since my hardcore Java
> days, or I'm going to have to look into why the conversion is so horribly
> inefficient (I do get the differences between the Iterable concept and the
> Collection concept).  I'm using a "Collection" since I'm assigning it to a
> "select" box in my JSP via Spring MVC.

> Thoughts are welcome (I've probably overlooked something obvious).

> So, long story short: The timing code I need can likely wait until I have
> things straightened out a bit more.  I would love to see my query come down
> from ~5s to a lot less.

> BTW, I found that the findAll() method was just as quick (if not quicker)
> than my use of the same Cypher query and the use of @MapResult, etc.

> Thanks again guys!  Very excited going through all this!

> On Friday, September 28, 2012 3:55:29 PM UTC-4, BtySgtMajor wrote:

>> Hi Michael,

>>   What you're proposing in terms of adding time logging for repos/queries
>> is exactly what I'm looking for.  I don't have any eager loading in place
>> at this point in time anyway (it's the last thing I'd want to add).

>>   I'll give the logging proxy bit a look as I have a chance.  As for the
>> filter option, that might take me a bit longer, and will give that a shot
>> if it comes to that.

>>   With respect to my code, there's really not all that much.

>>   Essentially, from within one of my Spring MVC controllers, I have a
>> line like this:  ("Game" is part of my domain model as I'm doing something
>> similar to IMDB.)

>> Collection<Game> allGames =
>> IteratorUtil.asCollection(gameRepository.findAll());

>>   And, the "gameRepository" is declared like this:

>> public interface GameRepository extends GraphRepository<Game>,
>> RelationshipOperationsRepository<Game> {
>> Game findByGameId(String id);
>> }

>>   That's about it.  Just keeping things simple.

>>   The "Game" model does have a few relationships; however, all
>> relationships/collections are left as lazily loaded so as to avoid a big
>> mess.

>>   Currently, running the code above takes, on average, about 35s to
>> complete the round trip.

>>   If I run the same query via Cypher on the Neo4j console, it takes a
>> whopping 42ms.

>>   Also, if I just fetch a single game via the gameId via SDN, it takes
>> like 600ms.

>>   So, like I said, early days, and I'm trying to really dig in and find
>> out exactly where bottlenecks are occurring so I can rip it apart and
>> really get familiar with it.  Like I said before, I have my hypotheses, but
>> I need ways to test them out. :)

>>   Let me know if you want/need additional info/code and, if so, what it
>> is.

>>   Thanks again, Michael!

>> Cheers,
>> Duncan

>> On Friday, September 28, 2012 2:01:30 PM UTC-4, Michael Hunger wrote:

>>> Duncan,

>>> that is a good point actually.

>>> I can add time logging to the repositories / queries in SDN. Due to the
>>> lazy nature that will be only the query parsing and initial execution
>>> though (might be the full query time though depending on if you have a
>>> streaming or non-streaming server interaction).

>>> Can you share some of the code you're talking about?

>>> An more or less easy way to do it for you might be putting a
>>> logging-proxy in front of the neo4j server

>>> Alternatively if you're more adventurous adding a filter (via an
>>> unmanaged extension) to the neo4j server that logs the request times.
>>> (see:
>>> https://github.com/neo4j/authentication-extension/blob/1.8/src/main/j...,
>>> also make sure to have the META-INF/services/.... with your initializer
>>> class name) there.

>>> HTH

>>> Michael

>>> Am 28.09.2012 um 08:26 schrieb BtySgtMajor:

>>> Hi all,

>>>   So I'm getting into some good stuff here as I dig into Neo4j deeper.
>>>  At present, I'm trying to determine where bottlenecks lie as I develop
>>> some SDN code around a Neo4j server.  Obviously, I can time the Java code
>>> that makes the call out to the server (which is remote--on the same
>>> network, but sitting on a VM on another box), and I can see the time it
>>> takes my queries through the Neo4j console.

>>>   My question is this: Is there a way in the server/logging config in
>>> Neo4j to document/log each query's runtime?  This would help me greatly in
>>> determining where my bottlenecks are.  (I'd imagine it's in the part where
>>> my SDN repository is calling out to the remote Neo4j server, but I'd like
>>> to confirm and then start tweaking things).

>>>   Thanks in advance!  Exciting times!

>>> Cheers,
>>> Duncan

>>> --


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.