Rexster REST and ExtJS 4 Rest

[Michael Schmid]

Hi,

I'm currently trying to build an GUI for my Rexster data with the ExtJs 4 Framework.
Retrieving the data via REST and JSON works well and I'm able to e.g. load all vertices into my datagrid.

As I tried to move on with an updating functionality for properties from the grid and send the data back to Rexster
the following problem occured in Safari when executing the 'PUT':
XMLHttpRequest cannot load http://localhost:8182/graphs/neo4j/vertices/1?name=kkk&x=-266.8863525390625&y=1117.643310546875. Request header field Content-Type is not allowed by Access-Control-Allow-Headers.

I know that there are security limitations due to Cross-Site-Scripting issues but both Server services,
Rexster and HTTP run on the same machine (same IP) only on different ports.
So, I think it should work...!? ;-)

BTW:
The above shown URL with .../1?name=...&... is already formated through an custom extension as
ExtJS sends JSON data directly within the HTTP PUT / POST  instead of URL Parameters, by default.
See:
http://docs.sencha.com/ext-js/4-1/#!/api/Ext.data.writer.Json
http://docs.sencha.com/ext-js/4-1/#!/api/Ext.data.proxy.Rest

In addition to the data of the URL ExtJS send the same data with the mentioned 'default' behaviour' in parallel.
This is where the "Content-Type" field is from and why it's set.

The best solution for me would be that Rexster could accept the POST / PUT data directly from the JSON object.

I hope someone has an idea on this...
;-)

Thanks in advance,

Michl

[Stephen Mallette]

Right now, Rexster passes back a wildcard for
Access-Control-Allow-Headers so it seems weird that you would have
this problem. Is it possible for you to try in Chrome to see if the
issue is resolved? I'd just like to make sure I'm looking at the
right problem as I consider this.

There was a time when Dog House and the REST API ran on two separate
ports and this settings was enough for Dog House to run properly at
least on Chrome and FF.

I have some plans to expand control of these kinds of
settings...Rexster is a bit rigid in this area at the moment
unfortunately.

Best regards,

Stephen

> --
>
>

[Michael Schmid]

Hi Stephen,

I tried it in FF 14.0.1 before without success and I also tried it actually with Chrome Version 21.0.1180.79 also without success. ;-(
My FF Firebug says nothing why it doesn't work but Chrome's Firebug tells a little bit more about what it does:

PUT http://localhost:8182/graphs/neo4j/vertices/1?name=zz&x=-266.8863525390625&y=1117.643310546875132ms
Params

name zz
x -266.8863525390625
y 1117.643310546875

Headers

ResponseHeaders

<<Empty>>

RequestHeaders

Content-Type application/json

Put

JSON

_id 1
_type "vertex"
id 1
name "zz"
x "-266.8863525390625"
y "1117.643310546875"

Source

{"id":1,"_type":"vertex","name":"zz","x":"-266.8863525390625","y":"1117.643310546875","_id":1}

Response

<<Empty>>

Regards,

MIchl

[Stephen Mallette]

I think I see the problem:

https://github.com/tinkerpop/rexster/issues/233

Unfortunately, I don't have a workaround at the moment. I'll try to
get this fixed quickly..

Thanks,

Stephen

> --
>
>

[Michael Schmid]

OK, thanks!

I'm going to watch that thread. ;-)

Regards,

Michl

[Stephen Mallette]

The good news is that i don't think Rexster has the problem I thought
it did. The bad news is that I still am not quite sure why you are
seeing a problem. A bit of curl shows me what I want to see (and for
some reason didn't see earlier...i think i must have mangled my curl
statement earlier in the day):

curl -v -X OPTIONS http://localhost:8182/graphs/tinkergraph/vertices/1/
* About to connect() to localhost port 8182 (#0)
* Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 8182 (#0)
> OPTIONS /graphs/tinkergraph/vertices/1/ HTTP/1.1
> User-Agent: curl/7.21.6 (x86_64-pc-linux-gnu) libcurl/7.21.6 OpenSSL/1.0.0e zlib/1.2.3.4 libidn/1.22 librtmp/2.3
> Host: localhost:8182
> Accept: */*
>
< HTTP/1.1 204 No Content
< server: grizzly/2.1.2
< Access-Control-Allow-Methods: DELETE,GET,POST,PUT
< Access-Control-Allow-Headers: *
< Access-Control-Max-Age: 1728000
< Access-Control-Allow-Origin: *
< Date: Mon, 20 Aug 2012 17:01:55 GMT
<
* Connection #0 to host localhost left intact
* Closing connection #0

where Access-Control-Allow-Headers is set to the asterisk wildcard. I
don't know ExtJS, but some light searching of CORS and ExtJS brought
up at least a few links that seemed to mention some additional
configurations you might need to make:

Ext.Ajax.useDefaultXhrHeader
Ext.Ajax.cors

Are you familiar with those? Do they possibly have anything to do
with the problem you're facing?

Stephen

> --
>
>

[Michael Schmid]

Hi Stephen,

I "curled and traced" also a little bit in the sent packets and I came up with the following:
* ExtJS' default behaviour is to send always the "OPTIONS" header on any HTTP Request (GET/PUT/POST/DELETE)
* To specifiy the desired server action it uses the additional "Access-Control-Request-Method" header field. Herein it tells what to do (GET/PUT/POST/DELETE).
* Rexster expects NOT an OPTIONS header it likes the "direct" way much more. When I use a "real" GET/PUT/POST/DELETE Request it works fine.
* ExtJS' <Ext.Ajax.useDefaultXhrHeader> property should generate 'real' requests instead of OPTIONS-based ones. But right now it seems to work only for GET.

So, is it possible to make Rexster lookup for the "Access-Control-Request-Method" and use it's value as 'action' when an OPTIONS request is sent to it?

I'm also going to post this REST bug in the ExtJS world, probably we could solve it from both sides...
;-)

Regards,

Michl

[Stephen Mallette]

What you describe sounds like standard CORS behavior.

https://developer.mozilla.org/en-US/docs/HTTP_access_control#Preflighted_requests

Allow me to try to restate what you wrote:

1. ExtJS makes a pre-flight request with OPTIONS verb.
2. This request supplies the Access-Control-Request-Method with the
verb that is expected to be used.
3. If the server accepts that verb for that URI it will send back a
successful response with the avaialbe Access-Control-Allow-Methods
header.
4. On successful pass of the pre-flight, ExtJS should follow up with
the actual operation (a "real" request as you described it)

So if I do a call with OPTIONS and the include
Access-Control-Request-Method I get:

curl -v -X OPTIONS -H "Access-Control-Request-Method:PUT"
http://localhost:8182/graphs/tinkergraph/vertices/1

* About to connect() to localhost port 8182 (#0)
* Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 8182 (#0)

> OPTIONS /graphs/tinkergraph/vertices/1 HTTP/1.1

> User-Agent: curl/7.21.6 (x86_64-pc-linux-gnu) libcurl/7.21.6 OpenSSL/1.0.0e zlib/1.2.3.4 libidn/1.22 librtmp/2.3
> Host: localhost:8182
> Accept: */*

> Access-Control-Request-Method:PUT

>
< HTTP/1.1 204 No Content
< server: grizzly/2.1.2
< Access-Control-Allow-Methods: DELETE,GET,POST,PUT
< Access-Control-Allow-Headers: *
< Access-Control-Max-Age: 1728000
< Access-Control-Allow-Origin: *

< Date: Tue, 21 Aug 2012 10:02:05 GMT

<
* Connection #0 to host localhost left intact
* Closing connection #0

which most importantly includes this line:

< Access-Control-Allow-Methods: DELETE,GET,POST,PUT

I noticed that interestingly, Rexster hasn't included "OPTIONS" in
that list. For some reason, I didn't think I had to because it would
seem that if my server supported OPTIONS then why would something make
a preflight request to OPTIONS to find out if it supported
OPTIONS...it would just get an error?? but maybe ExtJS is doing
something in addition to what the browser is doing and blocking
there??

With Chrome/FireFox you were describing the traffic you were
seeing...you should be able to see what the response is from the CORS
request. Does it look like the above?

You may not want to do this in the long run, but perhaps setting
Ext.Ajax.cors=false

http://docs.sencha.com/ext-js/4-0/#!/api/Ext.Ajax-cfg-cors

would at least allow your application to work and we could confirm
that it is a problem with ExtJS.

stephen

> --
>
>

[Michael Schmid]

Thanks for the tipp, but I looked at the Ext.Ajax.cors variable already.

I compiled my traces for you to make things hopefully a litte clearer...
;-)

###############################################################

My ExtJS - Settings:

Ext.Ajax.useDefaultXhrHeader = false;
Ext.Ajax.cors = false;

###############################################################

Step #1: GET Request to load the ExtJS Grid -> works well!

192.168.174.129.52988-192.168.173.020.08182: GET /graphs/neo4j/vertices?page=1&start=0&limit=25 HTTP/1.1
Host: mbp266:8182
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:14.0) Gecko/20100101 Firefox/14.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Connection: keep-alive
Referer: http://localhost:8888/graphdb/extClient/index.html
Origin: http://localhost:8888


192.168.173.020.08182-192.168.174.129.52988: HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8
server: grizzly/2.1.2
Access-Control-Allow-Origin: *
Date: Tue, 21 Aug 2012 10:31:06 GMT
Transfer-Encoding: chunked

4000
{"version":"2.1.0","results":[{"_id":0,"_type":"vertex"},{"name":"tessssst","_id":1,"_type":"vertex"},
...

###############################################################

Step #2: UPDATE Request (preflighted) from Firefox / ExtJs -> Stucks after Rexsters OPTIONS response

192.168.174.129.52997-192.168.173.020.08182: OPTIONS /graphs/neo4j/vertices/1?name=this%20should%20be%20put%20to%20the%20server!&x=&y= HTTP/1.1
Host: mbp266:8182
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:14.0) Gecko/20100101 Firefox/14.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Connection: keep-alive
Origin: http://localhost:8888
Access-Control-Request-Method: PUT
Access-Control-Request-Headers: content-type


192.168.173.020.08182-192.168.174.129.52997: HTTP/1.1 204 No Content

server: grizzly/2.1.2
Access-Control-Allow-Methods: DELETE,GET,POST,PUT
Access-Control-Allow-Headers: *
Access-Control-Max-Age: 1728000
Access-Control-Allow-Origin: *

Date: Tue, 21 Aug 2012 10:34:04 GMT

###############################################################


I think the problem is in the answer of the OPTIONS Request.

• The status code is "204 No Content" instead of "200 OK" (comapared to the Mozilla example)
   Your CURL trace shows also a 204.
• Probably the answers Access-Control-Allow-Methods content should also be extended to "OPTIONS"
  as you thought about. I think OPTIONS is just a kind of a "alibi" header. Probably you need it in the
  list because you need the posibility for doing an "public stateless" request against the server to see what
  "real" requests (GET, POST, PUT, DELETE) are allowed on it.

###############################################################

Setting Ext.Ajax.useDefaultXhrHeader = true; produces the following wrong response already for the initial get:

192.168.174.129.53029-192.168.173.020.08182: OPTIONS /graphs/neo4j/vertices?page=1&start=0&limit=25 HTTP/1.1
Host: mbp266:8182
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:14.0) Gecko/20100101 Firefox/14.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Connection: keep-alive
Origin: http://localhost:8888
Access-Control-Request-Method: GET
Access-Control-Request-Headers: x-requested-with


192.168.173.020.08182-192.168.174.129.53029: HTTP/1.1 204 No Content
server: grizzly/2.1.2
Access-Control-Allow-Methods: GET,POST

Access-Control-Allow-Headers: *
Access-Control-Max-Age: 1728000
Access-Control-Allow-Origin: *

Date: Tue, 21 Aug 2012 10:43:18 GMT

###############################################################

On the ExtJs side I crawled through the source code for any other header modifications made but without success.
The whole preflighting seems to be done inside the browser.

Regards,

Michl

[Stephen Mallette]

I just pushed a modification that changes the response from 204 to 200
(couldn't find what the "standard" was there). Also changed OPTIONS
requests to actually return OPTIONS in the response.

Please let me know if that solves the problem.

Stephen

> --
>
>

[Michael Schmid]

I pulled in your changes and compiled, without success... ;-(

But: I modded your code and read again in the Mozilla docs and found a solution!

The problem is as follows:

• The browser uses the OPTIONS state as a kind of test state to check if the server supports the desired
  "Access-Control-Request-Method" AND "Access-Control-Request-Headers" methods.
• We answered through your change as follows:
• Access-Control-Allow-Methods: OPTIONS, DELETE, GET, POST, PUT
  
• Access-Control-Allow-Headers: *
• I figured out that the "Access-Control-Allow-Headers: *" asterisk wildcard is not supported.
  To respond correct the server should send back the value of "Access-Control-Request-Headers"
  in the "Access-Control-Allow-Headers" field.
  => Access-Control-Request-Headers => Access-Control-Allow-Headers
  
  E.g.:
  
  Browser says: Access-Control-Request-Headers: X-Requested-With
  Server must respont: Access-Control-Allow-Headers: X-Requested-With
  
  
  That's it! - I added the the correct response statically into the header, compiled and tested it -> works!
  I'd like to provide you with the correct code but I'm not a Java Geek...
  ;-)

Here's a trace of the successful traffic:

###############################################################

192.168.174.129.53711-192.168.173.020.08182: OPTIONS /graphs/neo4j/vertices?page=1&start=0&limit=25 HTTP/1.1
Host: mbp266:8182
Access-Control-Request-Method: GET
Origin: http://localhost:8888
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_5_8) AppleWebKit/534.50.2 (KHTML, like Gecko) Version/5.0.6 Safari/533.22.3
Referer: http://localhost:8888/graphdb/extClient/index.html
Access-Control-Request-Headers: X-Requested-With
Accept: */*
Accept-Language: de-de
Accept-Encoding: gzip, deflate
Content-Length: 0
Connection: keep-alive


192.168.173.020.08182-192.168.174.129.53711: HTTP/1.1 200 OK
Content-Type: text/html; charset=iso-8859-1
server: grizzly/2.1.2
Access-Control-Allow-Methods: OPTIONS,GET,POST
Access-Control-Allow-Headers: x-requested-with
Access-Control-Max-Age: 1728000
Access-Control-Allow-Origin: *
Date: Tue, 21 Aug 2012 14:54:48 GMT
Transfer-Encoding: chunked

0


192.168.174.129.53711-192.168.173.020.08182: GET /graphs/neo4j/vertices?page=1&start=0&limit=25 HTTP/1.1
Host: mbp266:8182
Origin: http://localhost:8888
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_5_8) AppleWebKit/534.50.2 (KHTML, like Gecko) Version/5.0.6 Safari/533.22.3
Referer: http://localhost:8888/graphdb/extClient/index.html
Accept: */*
Accept-Language: de-de

Accept-Encoding: gzip, deflate
Connection: keep-alive

192.168.173.020.08182-192.168.174.129.53711: HTTP/1.1 200 OK

Content-Type: application/json;charset=UTF-8
server: grizzly/2.1.2
Access-Control-Allow-Origin: *

Date: Tue, 21 Aug 2012 14:54:49 GMT
Transfer-Encoding: chunked

4000
{"version":"2.2.0-SNAPSHOT","results":[{"_id":0,"_type":"vertex"}....

###############################################################

Thanks for your help! Regards,

Michl

[Stephen Mallette]

I've modified Rexster and pushed. If you don't supply the
Access-Control-Request-Headers Rexster will return the asterisk. Can
you please give it a try?

curl -v -X OPTIONS -H

"Access-Control-Request-Headers:X-Requested-With"

http://localhost:8182/graphs/tinkergraph/vertices/1
* About to connect() to localhost port 8182 (#0)
* Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 8182 (#0)
> OPTIONS /graphs/tinkergraph/vertices/1 HTTP/1.1
> User-Agent: curl/7.21.6 (x86_64-pc-linux-gnu) libcurl/7.21.6 OpenSSL/1.0.0e zlib/1.2.3.4 libidn/1.22 librtmp/2.3
> Host: localhost:8182
> Accept: */*

> Access-Control-Request-Headers:X-Requested-With

>
< HTTP/1.1 200 OK
< Content-Type: text/html; charset=iso-8859-1
< server: grizzly/2.1.2

< Access-Control-Allow-Methods: OPTIONS,DELETE,GET,POST,PUT
< Access-Control-Allow-Headers: X-Requested-With
< Access-Control-Max-Age: 1728000
< Access-Control-Allow-Origin: *
< Date: Tue, 21 Aug 2012 20:15:01 GMT
< Transfer-Encoding: chunked

<
* Connection #0 to host localhost left intact
* Closing connection #0

curl -v -X OPTIONS http://localhost:8182/graphs/tinkergraph/vertices/1

* About to connect() to localhost port 8182 (#0)
* Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 8182 (#0)
> OPTIONS /graphs/tinkergraph/vertices/1 HTTP/1.1
> User-Agent: curl/7.21.6 (x86_64-pc-linux-gnu) libcurl/7.21.6 OpenSSL/1.0.0e zlib/1.2.3.4 libidn/1.22 librtmp/2.3
> Host: localhost:8182
> Accept: */*
>

< HTTP/1.1 200 OK
< Content-Type: text/html; charset=iso-8859-1
< server: grizzly/2.1.2

< Access-Control-Allow-Methods: OPTIONS,DELETE,GET,POST,PUT
< Access-Control-Allow-Headers: *
< Access-Control-Max-Age: 1728000
< Access-Control-Allow-Origin: *
< Date: Tue, 21 Aug 2012 20:15:14 GMT
< Transfer-Encoding: chunked

<
* Connection #0 to host localhost left intact
* Closing connection #0

Best regards,

Stephen

> --
>
>

[Michael Schmid]

Good Morning Stephen,

your latest commit works perfectly!

Just a little trace of a PUT on a vertex:

###############################################################

192.168.174.129.53771-192.168.173.020.08182: OPTIONS /graphs/neo4j/vertices/1?name=ddd&x=&y= HTTP/1.1
Host: mbp266:8182

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:14.0) Gecko/20100101 Firefox/14.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3

Accept-Encoding: gzip, deflate
Connection: keep-alive

Origin: http://localhost:8888
Access-Control-Request-Method: PUT

Access-Control-Request-Headers: x-requested-with


192.168.173.020.08182-192.168.174.129.53771: HTTP/1.1 200 OK

Content-Type: text/html; charset=iso-8859-1
server: grizzly/2.1.2
Access-Control-Allow-Methods: OPTIONS,DELETE,GET,POST,PUT

Access-Control-Allow-Headers: x-requested-with
Access-Control-Max-Age: 1728000
Access-Control-Allow-Origin: *

Date: Wed, 22 Aug 2012 06:49:13 GMT
Transfer-Encoding: chunked

0


192.168.174.129.53771-192.168.173.020.08182: PUT /graphs/neo4j/vertices/1?name=ddd&x=&y= HTTP/1.1
Host: mbp266:8182

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:14.0) Gecko/20100101 Firefox/14.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3

Accept-Encoding: gzip, deflate
Connection: keep-alive

X-Requested-With: XMLHttpRequest

Referer: http://localhost:8888/graphdb/extClient/index.html
Origin: http://localhost:8888

Content-Length: 0


192.168.173.020.08182-192.168.174.129.53771: HTTP/1.1 200 OK

Content-Type: application/json;charset=UTF-8
server: grizzly/2.1.2
Access-Control-Allow-Origin: *

Vary: Accept
Date: Wed, 22 Aug 2012 06:49:13 GMT
Transfer-Encoding: chunked

70
{"version":"2.2.0-SNAPSHOT","results":{"name":"ddd","y":"","x":"","_id":1,"_type":"vertex"},"queryTime":8.41216}

192.168.173.020.08182-192.168.174.129.53771: 0

###############################################################

Thanks for your help!

Kind regards,

Michl