Re: [Neo4j] Batch creating unique nodes and relationships via REST

247 views
Skip to first unread message

Peter Neubauer

unread,
Jul 8, 2012, 2:54:11 AM7/8/12
to ne...@googlegroups.com

Hi thete,
Do you also have the resulting json that is sent to the server? That way we could reproduce it more easily.

/peter

Send from mobile.

On Jul 8, 2012 8:49 AM, "Zhemin Lin" <lin.z...@gmail.com> wrote:
Hi.
I'm trying to insert unique nodes / relationships in a batch operation.
I'm not sure if there's anything wrong in my code or Neo4j doesn't support the feature yet.

The code to create unique node id 0, 1 worked as expected.
The problem is in creating unique relationship.

Thanks!

Code:
  (let [ops [{:method "POST"
              :to     "/index/node/vertices?unique"
              :body   {:key "name" :value "Tobias" :properties {:name "Tobias" :n 1}}        ;; OK
              :id     0}
             {:method "POST"
              :to     "/index/node/vertices?unique"
              :body   {:key "name" :value "Anders" :properties {:name "Anders" :n 2}}      ;; OK
              :id     1}
             {:method "POST"
              :to     "/index/node/vertices?unique"
              :body   {:key "name" :value "Anders" :properties {:name "Anders" :n 2}}      ;; OK
              :id     2}
             {:method "POST"
              :to     "/index/relationship/edges/?unique"                                                ;; 500 error
              :body   {:key   "name" 
                       :value "tobias-anders" 
                       :start "{0}"
                       :end   "{1}"
                       :type  "FRIENDS"
                       :data  {:since 1991}}
              :id     3}
        res (doall (batch/perform ops))]

Result:
:body "{\n  \"message\" : \"{\\n  \\\"message\\\" : \\\"For input string: \\\\\\\"{0}\\\\\\\"\\\",\\n  \\\"exception\\\" : \\\"BadInputException\\\",\\n  \\\"stacktrace\\\" : [ \\\"org.neo4j.server.rest.web.RestfulGraphDatabase.extractNodeId(RestfulGraphDatabase.java:162)\\\", \\\"org.neo4j.server.rest.web.RestfulGraphDatabase.extractNodeIdOrNull(RestfulGraphDatabase.java:151)\\\"

Michael Klishin

unread,
Jul 8, 2012, 6:40:50 AM7/8/12
to ne...@googlegroups.com
Peter Neubauer:

> Do you also have the resulting json that is sent to the server? That way we could reproduce it more easily.

There is a discussion on the Neocons mailing list that concludes the issue may be that after certain point
node creating operations do not return location headers:

https://groups.google.com/forum/?fromgroups#!topic/clojure-neo4j/riUK7Endskg

MK

mic...@defprotocol.org

signature.asc

Max De Marzi Jr.

unread,
Jul 10, 2012, 5:15:19 PM7/10/12
to ne...@googlegroups.com
creating a unique node returns an index result, not a node, so it's not currently "referable". See issue 697.

https://github.com/neo4j/community/issues/697

Zhemin Lin

unread,
Jul 16, 2012, 9:39:52 AM7/16/12
to ne...@googlegroups.com
Hi.
I have git clone'd the code for issue #690 (hence #697).
I need your input to decide which solution / workaround I should take.

The issue comes from "19.9 unique indexes" API definition.
When I batch insert a node, it calls InternalJettyServletRequest() in BatchOperations.java:174.
If the node doesn't exist, Neo4j returns 201 created and Location: ... as defined in 19.9.1 (manual 1.8M06).
However, if it does exist, Neo4j returns 200 OK without Location.

Possible solutions:
* Workaround in StreamingBatchOperationResults.java:65 and BatchOperationResults.java:69 to copy locations from body.indexed (quick & dirty)
* Return "Location" when 200 OK (root cause, which violates API definition)

Please comment.
Thanks.

Peter Neubauer

unread,
Jul 16, 2012, 9:42:56 AM7/16/12
to ne...@googlegroups.com
Zhemin,
I think this might be related to a recent contribution by Michael, see
https://github.com/neo4j/community/pull/675 which gives more explicit
error codes. We are trying to enhance that to a level where different
response strategies can be specified, see
https://github.com/neo4j/community/pull/675#issuecomment-7002096 .
Would that help you? Should be in in a couple of days, and then you
could work from there?

Cheers,

/peter neubauer

G: neubauer.peter
S: peter.neubauer
P: +46 704 106975
L: http://www.linkedin.com/in/neubauer
T: @peterneubauer

Wanna learn something new? Come to @graphconnect.

Zhemin Lin

unread,
Jul 16, 2012, 10:04:34 AM7/16/12
to ne...@googlegroups.com
Hi Peter,
Issue #690 (batch creating a unique node in REST) and #675 (MIchael's contribution) are different.
It isn't related with unique=create or not, but rather a missing "Location" in the definition of 19.9.2.
I think it's more reasonable to put "location" in the returning string of 200 OK (in case of GetOrCreate).

I will try to work on it.
Thanks!


2012年7月16日月曜日 21時42分56秒 UTC+8 Peter Neubauer:

Peter Neubauer

unread,
Jul 16, 2012, 10:05:07 AM7/16/12
to ne...@googlegroups.com
That would be great. Thanks Zhemin!

Cheers,

/peter neubauer

G: neubauer.peter
S: peter.neubauer
P: +46 704 106975
L: http://www.linkedin.com/in/neubauer
T: @peterneubauer

Wanna learn something new? Come to @graphconnect.


Zhemin Lin

unread,
Jul 17, 2012, 5:05:08 AM7/17/12
to ne...@googlegroups.com
Hi.
I'm trying to build Neo4j (without my patch).  However, there is always some error message about "Address already in use":

2012/7/17 下午 03:43:55 org.neo4j.server.logging.Logger log
嚴重的: Failed to start Neo Server on port [9999], reason [java.net.BindException: Address already in use]
2012/7/17 下午 03:43:55 org.neo4j.server.logging.Logger log
資訊: Successfully shutdown Neo4j Server.

Results :

Failed tests:   shouldLogInfoWhenDefaultingToTuningPropertiesFileInTheSameDirectoryAsTheNeoServerPropertiesFile(org.neo4j.server.configuration.PropertyFileConfiguratorTest): (..)
  shouldLogShutdown(org.neo4j.server.NeoServerShutdownLoggingFunctionalTest): (..)

Tests run: 677, Failures: 2, Errors: 0, Skipped: 4

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3:43.210s
[INFO] Finished at: Tue Jul 17 15:43:56 CST 2012
[INFO] Final Memory: 37M/360M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.11:test (default-test) on project neo4j-server: There are test failures.

My command was:
cd server
mvn clean package -P initial-build

ljm@ljm-Precision-WorkStation-T5400:~/neo4j/community/server$ java -version
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)

I guess I'd better build Neo4j and pass all the tests before applying my patch.
Thanks!

Peter Neubauer

unread,
Jul 17, 2012, 6:53:06 AM7/17/12
to ne...@googlegroups.com
Do you have a neo4j server running outside the build?

Things are building at my end. Got your code somewhere so I can try on
my machine?

Cheers,

/peter neubauer

G: neubauer.peter
S: peter.neubauer
P: +46 704 106975
L: http://www.linkedin.com/in/neubauer
T: @peterneubauer

Wanna learn something new? Come to @graphconnect.


林哲民 (Lin Zhemin)

unread,
Jul 17, 2012, 7:19:49 AM7/17/12
to ne...@googlegroups.com
Hi Peter,

No.  I did a lsof -i -n and didn't see anything relevant, either.
Attached is my patch.  However, I'm not really sure if it works...

Thanks.

2012/7/17 Peter Neubauer <peter.n...@neotechnology.com>
location.diff

Peter Neubauer

unread,
Jul 17, 2012, 7:23:22 AM7/17/12
to ne...@googlegroups.com
Coolio,
maybe you could drop in a Pull Request so I can check it?

Cheers,

/peter neubauer

G: neubauer.peter
S: peter.neubauer
P: +46 704 106975
L: http://www.linkedin.com/in/neubauer
T: @peterneubauer

Wanna learn something new? Come to @graphconnect.


林哲民 (Lin Zhemin)

unread,
Jul 17, 2012, 7:32:44 AM7/17/12
to ne...@googlegroups.com
Sure... But I really hesitate to check-in untested code.
Pardon me if it breaks anything...

Zhemin Lin

unread,
Jul 17, 2012, 10:10:47 AM7/17/12
to ne...@googlegroups.com
Hi.
Jakewins shared with me a really good reasoning in the pull request.
Really appreciate it.
However, I think we still need a band-aid-like workaround to benefit from batch operation if it is not fully abandoned for Cypher in the near future. :)

Thanks.

Peter Neubauer

unread,
Jul 17, 2012, 1:57:47 PM7/17/12
to ne...@googlegroups.com
Zhemin,
great to hear you got feedback even if not the kind we were expecting. This is OSS, and Pull Requests really work IMHO to discuss these issues. This PR actually made my day, with both you and Jacob involved.

/peter


--

Zhemin Lin

unread,
Jul 26, 2012, 1:34:01 AM7/26/12
to ne...@googlegroups.com
Hi Peter,
I have git push my patch and it passed both streaming and non-streaming functional tests.
May Jacob (or someone) kindly spend his/her time and take a look?
Thank you!

Peter Neubauer

unread,
Jul 26, 2012, 4:12:46 AM7/26/12
to ne...@googlegroups.com
Zhemin,
is it still https://github.com/neo4j/community/pull/714 ? Will have a
look today if I get the time. Thanks!

Cheers,

/peter neubauer

G: neubauer.peter
S: peter.neubauer
P: +46 704 106975
L: http://www.linkedin.com/in/neubauer
T: @peterneubauer

Wanna learn something new? Come to @graphconnect.


林哲民 (Lin Zhemin)

unread,
Jul 26, 2012, 4:16:41 AM7/26/12
to ne...@googlegroups.com
Hi Peter,
Yes, still https://github.com/neo4j/community/pull/714
Thanks!

2012/7/26 Peter Neubauer <peter.n...@neotechnology.com>:
Reply all
Reply to author
Forward
0 new messages