Errors with 5.ru example

[Harry]

Hi Jon,

hit a couple of errors. On example 5.ru (minus OAuthFilter and
OpenIDFilter) I get the following errors attempting a POST to /notes

Case 1)

RestClient.post "http://localhost:9000/notes", "[1,2,3,4,5]",
{:content_type=>"application/json"}


Exception `ArgumentError' at (eval):3 - Invalid String value
Response: #<CloudKit::Response:0x1a19db0 @status=500,
@content="{\"error\":\"unknown server error\"}", @meta={"Content-
Type"=>"application/json", "Cache-Control"=>"no-cache"}>
Exception `ArgumentError' at /opt/local/lib/ruby/gems/1.8/gems/
rack-0.9.1/lib/rack/session/abstract/id.rb:91 - Response is not an
array.
ArgumentError: Response is not an array.
/opt/local/lib/ruby/gems/1.8/gems/rack-0.9.1/lib/rack/session/
abstract/id.rb:91:in `commit_session'
/opt/local/lib/ruby/gems/1.8/gems/rack-0.9.1/lib/rack/session/
abstract/id.rb:49:in `context'
/opt/local/lib/ruby/gems/1.8/gems/rack-0.9.1/lib/rack/session/
abstract/id.rb:42:in `call'
/opt/local/lib/ruby/gems/1.8/gems/rack-0.9.1/lib/rack/session/
abstract/id.rb:42:in `call'
/opt/local/lib/ruby/gems/1.8/gems/rack-0.9.1/lib/rack/lint.rb:45:in
`_call'
/opt/local/lib/ruby/gems/1.8/gems/rack-0.9.1/lib/rack/lint.rb:33:in
`call'
/opt/local/lib/ruby/gems/1.8/gems/rack-0.9.1/lib/rack/
showexceptions.rb:23:in `call'
/opt/local/lib/ruby/gems/1.8/gems/rack-0.9.1/lib/rack/commonlogger.rb:
20:in `_call'
/opt/local/lib/ruby/gems/1.8/gems/rack-0.9.1/lib/rack/commonlogger.rb:
13:in `call'
/opt/local/lib/ruby/gems/1.8/gems/thin-1.0.0/lib/thin/connection.rb:
63:in `pre_process'
/opt/local/lib/ruby/gems/1.8/gems/thin-1.0.0/lib/thin/connection.rb:
54:in `process'
/opt/local/lib/ruby/gems/1.8/gems/thin-1.0.0/lib/thin/connection.rb:
39:in `receive_data'
/opt/local/lib/ruby/gems/1.8/gems/eventmachine-0.12.6/lib/
eventmachine.rb:240:in `run_machine'
/opt/local/lib/ruby/gems/1.8/gems/eventmachine-0.12.6/lib/
eventmachine.rb:240:in `run'
/opt/local/lib/ruby/gems/1.8/gems/thin-1.0.0/lib/thin/backends/
base.rb:57:in `start'
/opt/local/lib/ruby/gems/1.8/gems/thin-1.0.0/lib/thin/server.rb:
150:in `start'
/opt/local/lib/ruby/gems/1.8/gems/rack-0.9.1/lib/rack/handler/thin.rb:
11:in `run'
/opt/local/lib/ruby/gems/1.8/gems/rack-0.9.1/bin/rackup:172
/opt/local/bin/rackup:19:in `load'
/opt/local/bin/rackup:19
127.0.0.1 - - [11/Mar/2009 23:48:31] "POST /notes HTTP/1.1" 500 54727
0.1848


Case 2)

If I do a RestClient.post "http://localhost:9000/notes", "{\"id\":\"one
\"}", {:content_type=>"application/json"} instead, the exception above
isn't show but the following is output instead:

Exception `ArgumentError' at (eval):3 - NULL pointer given
127.0.0.1 - - [11/Mar/2009 23:50:07] "POST /notes HTTP/1.1" 201 159
0.0078

It does store the data as

RestClient.get "http://localhost:9000/notes/f27707f0-f0ff-012b-
c3ac-000d932a964c"
=> "{\"id\":\"one\"}"

What could be going on here!?

Thanks & cheerio, Harry.

[Harry]

Failed to mention this is with the latest cloudkit release using Tokyo
Cabinet

[Jon Crosby]

On Thu, Mar 12, 2009 at 12:02 AM, Harry <harry....@gmx.net> wrote:

Hi Jon,

hit a couple of errors. On example 5.ru (minus OAuthFilter and
OpenIDFilter) I get the following errors attempting a POST to /notes

Case 1)

RestClient.post "http://localhost:9000/notes", "[1,2,3,4,5]",
{:content_type=>"application/json"}

This exception looks like it is being thrown due to [1,2,3,4,5] being invalid JSON. I'll put a spec around that and make sure it throws the proper 422.

I'll look into how RestClient is encoding this request and what happens with it on CloudKit's side. I haven't seen the issue in curl or the browser. I am in SLC getting ready for MWRC at the moment (speaking at 9AM) but will look into this immediately afterward.

-Jon

Thanks & cheerio, Harry.

[Harry]

Hi Jon,

>
> > RestClient.post "http://localhost:9000/notes", "[1,2,3,4,5]",
> > {:content_type=>"application/json"}
>
> This exception looks like it is being thrown due to [1,2,3,4,5] being
> invalid JSON. I'll put a spec around that and make sure it throws the proper
> 422.
>

it's perfectly valid JSON and not the cause of the issue - I've
investigated further and found out what's going on.

[146, 155] in /opt/local/lib/ruby/gems/1.8/gems/rufus-tokyo-0.1.9/lib/
rufus/tokyo/cabinet/util.rb
146 #
147 # Turns a Ruby hash into a Tokyo Cabinet Map and returns
it
148 # (don't forget to free the map when you're done with
it !)
149 #
150 def self.from_h (h)
=> 151 h.inject(Map.new) { |m, (k, v)| m[k] = v; m }
152 end
153
154 #
155 # Behaves much like Hash#[] but outputs a
Rufus::Tokyo::Map
/opt/local/lib/ruby/gems/1.8/gems/rufus-tokyo-0.1.9/lib/rufus/tokyo/
cabinet/util.rb:151
h.inject(Map.new) { |m, (k, v)| m[k] = v; m }
(rdb:1) pp h
{"collection_reference"=>"/notes",
"etag"=>"11422af0-f1b3-012b-c3b0-000d932a964c",
"uri"=>"/notes/114232e0-f1b3-012b-c3b0-000d932a964c",
0=>"0",
1=>"1",
"archived"=>"false",
"deleted"=>"false",
"resource_reference"=>"/notes/114232e0-f1b3-012b-c3b0-000d932a964c",
"remote_user"=>"null",
"json"=>"[1,2,3,4]",
"last_modified"=>"Fri, 13 Mar 2009 04:12:18 GMT"}

When I continue to

[61, 70] in /opt/local/lib/ruby/gems/1.8/gems/rufus-tokyo-0.1.9/lib/
rufus/tokyo/cabinet/util.rb
61
62 #
63 # Inserts key/value pair
64 #
65 def []= (k, v)
=> 66 clib.tcmapput2(m, k, v)
67 v
68 end
69
70 #
/opt/local/lib/ruby/gems/1.8/gems/rufus-tokyo-0.1.9/lib/rufus/tokyo/
cabinet/util.rb:66
clib.tcmapput2(m, k, v)
(rdb:1) pp k
0
(rdb:1) pp v
"0"
(rdb:1) n

Exception `ArgumentError' at (eval):3 - Invalid String value

the exception is thrown upstream as a JSON error (which it isn't).

k is viewed as a Fixnum and the clib call fails accordingly as it
expects a string, cf. http://tokyocabinet.sourceforge.net/spex-en.html#tcutilapi

(rdb:1) pp k.class
Fixnum

The fix would be to make sure all keys are indeed strings. BTW, what
do the 0 and 1 keys in env represent?

>
> I'll look into how RestClient is encoding this request and what happens with
> it on CloudKit's side. I haven't seen the issue in curl or the browser. I am
> in SLC getting ready for MWRC at the moment (speaking at 9AM) but will look
> into this immediately afterward.
>
> -Jon
>

Good luck with your talk - please post the slides, too!

Cheerio, Harry.

[Jon Crosby]

On Thu, Mar 12, 2009 at 9:36 PM, Harry <harry....@gmx.net> wrote:

Hi Jon,

>
> > RestClient.post "http://localhost:9000/notes", "[1,2,3,4,5]",
> > {:content_type=>"application/json"}
>
> This exception looks like it is being thrown due to [1,2,3,4,5] being
> invalid JSON. I'll put a spec around that and make sure it throws the proper
> 422.
>

it's perfectly valid JSON and not the cause of the issue - I've
investigated further and found out what's going on.

Thanks for digging into this. You are correct. I have an incorrect assumption baked into CloudKit that assumes only the object structure at the top level whereas arrays are also perfectly valid at this level. A fix for this should probably go out as an 0.11.1 release rather than waiting for the next feature release.

Those keys are probably the result of treating the array like a hash. I'll know for sure once I write some new specs around this.

 

>
> I'll look into how RestClient is encoding this request and what happens with
> it on CloudKit's side. I haven't seen the issue in curl or the browser. I am
> in SLC getting ready for MWRC at the moment (speaking at 9AM) but will look
> into this immediately afterward.
>
> -Jon
>

Good luck with your talk - please post the slides, too!

Thank you. MountainWest has been a great experience. I will definitely post slides on my blog. Video will also be available on the conference site shortly.

-Jon

 

Cheerio, Harry.

[Harry]

Hi Jon,

btw, this happens no matter where the array is. Something like "{\"data
\": [1,2,3]}" will fail equally.

Have you had a chance to look into this yet?

Cheerio, Harry.

[Jon Crosby]

On Fri, Mar 20, 2009 at 11:01 PM, Harry <harry....@gmx.net> wrote:

Hi Jon,

btw, this happens no matter where the array is. Something like "{\"data
\": [1,2,3]}" will fail equally.

Have you had a chance to look into this yet?

Yes, I have been working on a solution for the issue this week. There are two layers:

First, is the bug mentioned in your latest example -- {"data":[1,2,3]} should be stored without errors.

Second, is the previous example that you posted -- [1,2,3]. After looking into this a little more, I believe it will be useful to constrain CloudKit's storage API to JSON Objects. These objects can include JSON Arrays, JSON Objects, and JSON Values as their values, but the top-level objects must be JSON Objects. This will pay off in a big way when the JSON Query support is finished. I also checked into two similar setups -- CouchDB and Persevere -- and they both have the same restriction; documents are JSON Objects.

So while the second point above won't change anytime soon, the first one can and should. I have a few hours to hammer on this today and will report back on this list.

-Jon

[Jon Crosby]

On Sat, Mar 21, 2009 at 1:14 PM, Jon Crosby <jon.r....@gmail.com> wrote:

On Fri, Mar 20, 2009 at 11:01 PM, Harry <harry....@gmx.net> wrote:

Hi Jon,

btw, this happens no matter where the array is. Something like "{\"data
\": [1,2,3]}" will fail equally.

Have you had a chance to look into this yet?

Yes, I have been working on a solution for the issue this week. There are two layers:

First, is the bug mentioned in your latest example -- {"data":[1,2,3]} should be stored without errors.

Second, is the previous example that you posted -- [1,2,3]. After looking into this a little more, I believe it will be useful to constrain CloudKit's storage API to JSON Objects. These objects can include JSON Arrays, JSON Objects, and JSON Values as their values, but the top-level objects must be JSON Objects. This will pay off in a big way when the JSON Query support is finished. I also checked into two similar setups -- CouchDB and Persevere -- and they both have the same restriction; documents are JSON Objects.

So while the second point above won't change anytime soon, the first one can and should. I have a few hours to hammer on this today and will report back on this list.

The fix mentioned above has been pushed to GitHub along with a few new specs. If you are interested in testing it before an 0.11.1 patch release, please take a moment to do so. I would like to push out a new gem on Sunday evening or before.

Thanks,

-Jon