Received: by 10.224.110.207 with SMTP id o15mr2918269qap.7.1336127332515;
        Fri, 04 May 2012 03:28:52 -0700 (PDT)
X-BeenThere: neo4j@googlegroups.com
Received: by 10.229.174.211 with SMTP id u19ls848865qcz.2.gmail; Fri, 04 May
 2012 03:28:50 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.224.209.67 with SMTP id gf3mr370467qab.14.1336127330813; Fri,
 04 May 2012 03:28:50 -0700 (PDT)
Authentication-Results: ls.google.com; spf=pass (google.com: domain of
 pnku...@gmail.com designates internal as permitted sender)
 smtp.mail=pnku...@gmail.com; dkim=pass
 header...@gmail.com
Received: by v1g2000yqm.googlegroups.com with HTTP; Fri, 4 May 2012 03:28:50
 -0700 (PDT)
Date: Fri, 4 May 2012 03:28:50 -0700 (PDT)
In-Reply-To: <6455A566-0D3F-4A15-94B3-96CA4105CFDC@neotechnology.com>
References: <f10faabd-310c-474a-83b6-37da61485e5e@m13g2000yqc.googlegroups.com>
 <CAF59RW63CrjZRm0S_jjDdzqBPGdc0vAbYKKALJKBKBcKjUCgfg@mail.gmail.com>
 <21d85345-e277-4c69-a6e7-6875cfba6268@2g2000yqp.googlegroups.com> <6455A566-0D3F-4A15-94B3-96CA4105CFDC@neotechnology.com>
User-Agent: G2/1.0
X-HTTP-UserAgent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:10.0.2)
 Gecko/20100101 Firefox/10.0.2,gzip(gfe)
Message-ID: <3d30124c-a5bd-4b35-b352-b99f81e4f3a9@v1g2000yqm.googlegroups.com>
Subject: Re: Performance while migrating data to Neo4j server
From: Samwillie <pnku...@gmail.com>
To: Neo4j <neo4j@googlegroups.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi Michael, Peter

While I wait for your information on updates to the java-rest-bindings
- I tried the following

Use the locally hosted neo4j server using
RestGraphDatabase(LOCALHOST_GRAPHDB_PATH_URL); It takes an hour for
approx. 20000 nodes, I create nodes and index them. It is much more
faster with an EmbeddedGraphDatabase. The process of how I create
nodes is described above in this thread.

Is this the normal behavior?

Greets

On May 3, 4:44=A0pm, Michael Hunger <michael.hun...@neotechnology.com>
wrote:
> Sam,
>
> I'm currently working on upgrading the java-rest-bindings to a better int=
egration of the batch API.
> I hope to be done by tonight. Would love if you could check it out then.
>
> Mutating Cypher is in 1.8 see the blog post (blog.neo4j.org).
>
> right now it is restApi.executeBatch(new BatchCallback() ....)
>
> but that will most probably change to what Daniel Cox suggested.
>
> Cheers
>
> Michael
>
> Am 03.05.2012 um 16:41 schrieb Samwillie:
>
>
>
>
>
>
>
> > Hi Peter,
>
> > This is what I am trying to do:
>
> > GraphDatabaseService graphDbService =3D new
> > RestGraphDatabase(SERVER_PATH_URL);
> > =A0 =A0 =A0 =A0index =3D graphDbService.index();
> > =A0 =A0Iterator<DummyObject> iterator =3D
> > myDao.getAllDummyObjects().iterator();
> > =A0 =A0 =A0 =A0while (iterator.hasNext()) {
> > =A0 =A0 =A0 =A0 =A0 =A0Node dummyNode =3D graphDbService.createNode();
> > =A0 =A0 =A0 =A0 =A0 =A0//get all dummy objects from relational database
> > =A0 =A0 =A0 =A0 =A0 =A0DummyObject relationalDBDummyObject =3D iterator=
.next();
> > =A0 =A0 =A0 =A0 =A0 =A0Long dummyId =3D relationalDBDummyObject.getId()=
;
> > =A0 =A0 =A0 =A0 =A0 =A0.............
> > =A0 =A0 =A0 =A0 =A0 =A0// migrate all properties from relationalDBDummy=
Object to dummyNode
> > here...
> > =A0 =A0 =A0 =A0 =A0 =A0.............
> > Here is what I do, get all dummyObjects from the mySQL table, iterate
> > through them and for each object that is returned, create a node and
> > index one or two of its properties. I tried without indexing and even
> > this takes a long time.
>
> > @Michael, I am not aware of the techniques you mentioned - and how a
> > batch rest API works. Let me check this in parallel...
>
> > Thanks,
>
> > On May 3, 2:34 pm, Peter Neubauer <peter.neuba...@neotechnology.com>
> > wrote:
> >> Wow,
> >> that is too slow. How are you inserting these nodes?
>
> >> Cheers,
>
> >> /peter neubauer
>
> >> G: =A0neubauer.peter
> >> S: =A0peter.neubauer
> >> P: =A0+46 704 106975
> >> L: =A0http://www.linkedin.com/in/neubauer
> >> T: =A0 @peterneubauer
>
> >> If you can write, you can code - @coderdojomalmo
> >> If you can sketch, you can use a graph database - @neo4j
>
> >> On Thu, May 3, 2012 at 2:33 PM, Samwillie <pnku...@gmail.com> wrote:
> >>> Hi,
>
> >>> I am in the process of migrating data from tables in mySql into nodes
> >>> in Neo4j. There are approximately 20-30,000 table values that I need
> >>> to convert to nodes. Thanks to Peter and Michael, I have configured a
> >>> remote Neo4j Server and can access it with my Java web application
> >>> running on my local machine, which does the migration and creates
> >>> nodes and indexes them on the remote server also setting properties.
>
> >>> I notice a huge delay in doing this, (on the web-admin I see that for
> >>> the last half an hour only around 3000 nodes have been created &
> >>> indexed). The data retrieval from MySQL is fast- During the creation
> >>> of node, I also index these nodes. Is it because of indexing that it
> >>> takes such a long time? Or is there anyother method to make this whol=
e
> >>> process faster?
>
> >>> Many thanks for your reply,
>