Batch creating unique nodes and relationships via REST

Showing 1-18 of 18 messages
Batch creating unique nodes and relationships via REST Zhemin Lin 7/7/12 9:40 PM
Hi.
I'm trying to insert unique nodes / relationships in a batch operation.
I'm not sure if there's anything wrong in my code or Neo4j doesn't support the feature yet.

The code to create unique node id 0, 1 worked as expected.
The problem is in creating unique relationship.

Thanks!

Code:
  (let [ops [{:method "POST"
              :to     "/index/node/vertices?unique"
              :body   {:key "name" :value "Tobias" :properties {:name "Tobias" :n 1}}        ;; OK
              :id     0}
             {:method "POST"
              :to     "/index/node/vertices?unique"
              :body   {:key "name" :value "Anders" :properties {:name "Anders" :n 2}}      ;; OK
              :id     1}
             {:method "POST"
              :to     "/index/node/vertices?unique"
              :body   {:key "name" :value "Anders" :properties {:name "Anders" :n 2}}      ;; OK
              :id     2}
             {:method "POST"
              :to     "/index/relationship/edges/?unique"                                                ;; 500 error
              :body   {:key   "name" 
                       :value "tobias-anders" 
                       :start "{0}"
                       :end   "{1}"
                       :type  "FRIENDS"
                       :data  {:since 1991}}
              :id     3}
        res (doall (batch/perform ops))]

Result:
:body "{\n  \"message\" : \"{\\n  \\\"message\\\" : \\\"For input string: \\\\\\\"{0}\\\\\\\"\\\",\\n  \\\"exception\\\" : \\\"BadInputException\\\",\\n  \\\"stacktrace\\\" : [ \\\"org.neo4j.server.rest.web.RestfulGraphDatabase.extractNodeId(RestfulGraphDatabase.java:162)\\\", \\\"org.neo4j.server.rest.web.RestfulGraphDatabase.extractNodeIdOrNull(RestfulGraphDatabase.java:151)\\\"
Re: [Neo4j] Batch creating unique nodes and relationships via REST Peter Neubauer 7/7/12 11:54 PM

Hi thete,
Do you also have the resulting json that is sent to the server? That way we could reproduce it more easily.

/peter

Send from mobile.

Re: [Neo4j] Batch creating unique nodes and relationships via REST Michael Klishin 7/8/12 3:40 AM
Peter Neubauer:

> Do you also have the resulting json that is sent to the server? That way we could reproduce it more easily.

There is a discussion on the Neocons mailing list that concludes the issue may be that after certain point
node creating operations do not return location headers:

https://groups.google.com/forum/?fromgroups#!topic/clojure-neo4j/riUK7Endskg

MK

mic...@defprotocol.org

Re: [Neo4j] Batch creating unique nodes and relationships via REST Max De Marzi Jr. 7/10/12 2:15 PM
creating a unique node returns an index result, not a node, so it's not currently "referable". See issue 697.

https://github.com/neo4j/community/issues/697
Re: [Neo4j] Batch creating unique nodes and relationships via REST Zhemin Lin 7/16/12 6:39 AM
Hi.
I have git clone'd the code for issue #690 (hence #697).
I need your input to decide which solution / workaround I should take.

The issue comes from "19.9 unique indexes" API definition.
When I batch insert a node, it calls InternalJettyServletRequest() in BatchOperations.java:174.
If the node doesn't exist, Neo4j returns 201 created and Location: ... as defined in 19.9.1 (manual 1.8M06).
However, if it does exist, Neo4j returns 200 OK without Location.

Possible solutions:
* Workaround in StreamingBatchOperationResults.java:65 and BatchOperationResults.java:69 to copy locations from body.indexed (quick & dirty)
* Return "Location" when 200 OK (root cause, which violates API definition)

Please comment.
Thanks.
Re: [Neo4j] Batch creating unique nodes and relationships via REST Peter Neubauer 7/16/12 6:42 AM
Zhemin,
I think this might be related to a recent contribution by Michael, see
https://github.com/neo4j/community/pull/675 which gives more explicit
error codes. We are trying to enhance that to a level where different
response strategies can be specified, see
https://github.com/neo4j/community/pull/675#issuecomment-7002096 .
Would that help you? Should be in in a couple of days, and then you
could work from there?

Cheers,

/peter neubauer

G:  neubauer.peter
S:  peter.neubauer
P:  +46 704 106975
L:   http://www.linkedin.com/in/neubauer
T:   @peterneubauer

Wanna learn something new? Come to @graphconnect.
Re: [Neo4j] Batch creating unique nodes and relationships via REST Zhemin Lin 7/16/12 7:04 AM
Hi Peter,
Issue #690 (batch creating a unique node in REST) and #675 (MIchael's contribution) are different.
It isn't related with unique=create or not, but rather a missing "Location" in the definition of 19.9.2.
I think it's more reasonable to put "location" in the returning string of 200 OK (in case of GetOrCreate).

I will try to work on it.
Thanks!


2012年7月16日月曜日 21時42分56秒 UTC+8 Peter Neubauer:
Zhemin,
I think this might be related to a recent contribution by Michael, see
https://github.com/neo4j/community/pull/675 which gives more explicit
error codes. We are trying to enhance that to a level where different
response strategies can be specified, see
https://github.com/neo4j/community/pull/675#issuecomment-7002096 .
Would that help you? Should be in in a couple of days, and then you
could work from there?

Cheers,

/peter neubauer

G:  neubauer.peter
S:  peter.neubauer
P:  +46 704 106975
L:   http://www.linkedin.com/in/neubauer
T:   @peterneubauer

Wanna learn something new? Come to @graphconnect.


Re: [Neo4j] Batch creating unique nodes and relationships via REST Peter Neubauer 7/16/12 7:05 AM
That would be great. Thanks Zhemin!

Cheers,

/peter neubauer

G:  neubauer.peter
S:  peter.neubauer
P:  +46 704 106975
L:   http://www.linkedin.com/in/neubauer
T:   @peterneubauer

Wanna learn something new? Come to @graphconnect.


Re: [Neo4j] Batch creating unique nodes and relationships via REST Zhemin Lin 7/17/12 2:05 AM
Hi.
I'm trying to build Neo4j (without my patch).  However, there is always some error message about "Address already in use":

2012/7/17 下午 03:43:55 org.neo4j.server.logging.Logger log
嚴重的: Failed to start Neo Server on port [9999], reason [java.net.BindException: Address already in use]
2012/7/17 下午 03:43:55 org.neo4j.server.logging.Logger log
資訊: Successfully shutdown Neo4j Server.

Results :

Failed tests:   shouldLogInfoWhenDefaultingToTuningPropertiesFileInTheSameDirectoryAsTheNeoServerPropertiesFile(org.neo4j.server.configuration.PropertyFileConfiguratorTest): (..)
  shouldLogShutdown(org.neo4j.server.NeoServerShutdownLoggingFunctionalTest): (..)

Tests run: 677, Failures: 2, Errors: 0, Skipped: 4

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3:43.210s
[INFO] Finished at: Tue Jul 17 15:43:56 CST 2012
[INFO] Final Memory: 37M/360M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.11:test (default-test) on project neo4j-server: There are test failures.

My command was:
cd server
mvn clean package -P initial-build

ljm@ljm-Precision-WorkStation-T5400:~/neo4j/community/server$ java -version
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)

I guess I'd better build Neo4j and pass all the tests before applying my patch.
Thanks!

Re: [Neo4j] Batch creating unique nodes and relationships via REST Peter Neubauer 7/17/12 3:53 AM
Do you have a neo4j server running outside the build?

Things are building at my end. Got your code somewhere so I can try on
my machine?

Cheers,

/peter neubauer

G:  neubauer.peter
S:  peter.neubauer
P:  +46 704 106975
L:   http://www.linkedin.com/in/neubauer
T:   @peterneubauer

Wanna learn something new? Come to @graphconnect.


Re: [Neo4j] Batch creating unique nodes and relationships via REST Zhemin Lin 7/17/12 4:19 AM
Hi Peter,

No.  I did a lsof -i -n and didn't see anything relevant, either.
Attached is my patch.  However, I'm not really sure if it works...

Thanks.

2012/7/17 Peter Neubauer <peter.n...@neotechnology.com>

Do you have a neo4j server running outside the build?

Things are building at my end. Got your code somewhere so I can try on
my machine?

Cheers,

/peter neubauer

G:  neubauer.peter
S:  peter.neubauer
P:  +46 704 106975
L:   http://www.linkedin.com/in/neubauer
T:   @peterneubauer

Wanna learn something new? Come to @graphconnect.


Re: [Neo4j] Batch creating unique nodes and relationships via REST Peter Neubauer 7/17/12 4:23 AM
Coolio,
maybe you could drop in a Pull Request so I can check it?

Cheers,

/peter neubauer

G:  neubauer.peter
S:  peter.neubauer
P:  +46 704 106975
L:   http://www.linkedin.com/in/neubauer
T:   @peterneubauer

Wanna learn something new? Come to @graphconnect.


Re: [Neo4j] Batch creating unique nodes and relationships via REST Zhemin Lin 7/17/12 4:32 AM
Sure... But I really hesitate to check-in untested code.
Pardon me if it breaks anything...

https://github.com/neo4j/community/pull/714

Thanks.

2012/7/17 Peter Neubauer <peter.n...@neotechnology.com>
Coolio,

maybe you could drop in a Pull Request so I can check it?

Cheers,

/peter neubauer

G:  neubauer.peter
S:  peter.neubauer
P:  +46 704 106975
L:   http://www.linkedin.com/in/neubauer
T:   @peterneubauer

Wanna learn something new? Come to @graphconnect.


Re: [Neo4j] Batch creating unique nodes and relationships via REST Zhemin Lin 7/17/12 7:10 AM
Hi.
Jakewins shared with me a really good reasoning in the pull request.
Really appreciate it.
However, I think we still need a band-aid-like workaround to benefit from batch operation if it is not fully abandoned for Cypher in the near future. :)

Thanks.
Re: [Neo4j] Batch creating unique nodes and relationships via REST Peter Neubauer 7/17/12 10:57 AM
Zhemin,
great to hear you got feedback even if not the kind we were expecting. This is OSS, and Pull Requests really work IMHO to discuss these issues. This PR actually made my day, with both you and Jacob involved.

/peter


--
Re: [Neo4j] Batch creating unique nodes and relationships via REST Zhemin Lin 7/25/12 10:34 PM
Hi Peter,
I have git push my patch and it passed both streaming and non-streaming functional tests.
May Jacob (or someone) kindly spend his/her time and take a look?
Thank you!

Re: [Neo4j] Batch creating unique nodes and relationships via REST Peter Neubauer 7/26/12 1:12 AM
Zhemin,
is it still https://github.com/neo4j/community/pull/714 ? Will have a
look today if I get the time. Thanks!

Cheers,

/peter neubauer

G:  neubauer.peter
S:  peter.neubauer
P:  +46 704 106975
L:   http://www.linkedin.com/in/neubauer
T:   @peterneubauer

Wanna learn something new? Come to @graphconnect.


Re: [Neo4j] Batch creating unique nodes and relationships via REST Zhemin Lin 7/26/12 1:16 AM
Hi Peter,
Yes, still https://github.com/neo4j/community/pull/714
Thanks!

2012/7/26 Peter Neubauer <peter.n...@neotechnology.com>: