Help me understand Cloudkit

[nolan]

OK, I've looked through the website, examples and the resource class,
but I'm still having a hard time wrapping my head around CloudKit's use
in real-world scenarios. Hopefully a specific example will help me
understand.

I have an app with two separate interfaces. The main one is XMPP, and
most interactions will happen via compliant clients. For instances where
someone's client may not be sufficient, there needs to be a lightweight
web interface. I don't need traditional authentication, as such. Since
there is guaranteed to be a secure stream directly to the user, they'll
just ask the app for a temporary access URL, click on it from their IM
client and make whatever changes they need.

The app currently uses DM/SQLite3, but I've yet to use schemaless
databases in a serious project and think they'd be a good candidate
here, particularly as I grow tired of changing the schema. :) I'm
wondering if CloudKit would work here, but there are some points I need
to understand first.

I see that CK can either be standalone or can run alongside/in front of
other apps. I'm still wrapping my head around the middleware concept.
Let's say my app has users, and each user owns multiple widgets. How do
I use CK for storage while still allowing browsers to view resources? If
for instance I write Sinatra code that accepts the route /users, I can
of course insert that code before or after CK, but I don't see how this
solves my problem. In my tests, calling /users from my browser hands
back json as I'd expect. Is there some way of only triggering CK when
the caller specifically wants json? Or should all my browser-facing
routes include .html?

How do I restrict what records a user sees? If user or widget resources
contain confidential information, obviously I don't want non-owners to
read from or write to another's resources. I'm guessing the answer is
more middleware, but what form does it take, and where does it reside
relative to CK/the app, assuming I want the data available from both the
web view and API to be the same.

Schemaless databases are great for developers, but they seem like a
potential nightmare if the API is opened up. What prevents me as a
malicious user from injecting malicious data into your API? If there's
no schema, could I just do an HTTP POST of the entire human genome into
a field that's supposed to hold a user avatar? :) Or, even better, post
it to a field that doesn't yet exist, just in case the avatar field is
being validated?

I'm sure all these problems have solutions or best practices, they're
just non-obvious to me as my head spins, "Wha? JSON web...appliance?" :)

What I keep looking for are sample CloudKit apps. Maybe a really simple
mailbox? Log in with an OpenID and leave notes for other OpenIDs? It
would be a single view and would make it very clear how to limit
returned results to only records you own, integrate CK into something
like Sinatra, etc. Maybe I'll take a crack at it and post a patch here
if I can figure it out.

[Jon Crosby]

On Sat, Mar 28, 2009 at 11:30 AM, nolan <no...@thewordnerd.info> wrote:

OK, I've looked through the website, examples and the resource class,
but I'm still having a hard time wrapping my head around CloudKit's use
in real-world scenarios. Hopefully a specific example will help me
understand.

I am working on a few example applications to address this issue. One application is a minimal CloudKit app with Sinatra filling in the gaps for areas where server-side templating or static content is required. The second app will probably be a Rails app with an embedded instance of CloudKit.

For a quick idea of use cases, consider any web app built using rich client-side technologies. These include JavaScript frameworks such as Sproutcore (see Apple's MobileME service), Cappuccino (see 280slides.com) and Dojo as well as desktop applications using whatever is available to their operating environment.

These apps require a backing store and producing them using typical MVC web frameworks requires quite a bit of boilerplate code generation, followed by a series of repetitive modifications to the generated code. Secondly, the schema updates become error prone when they need to be pushed down to desktop apps or web apps that can run offline such as with Google Gears, Titanium, AIR, etc. (Sadly, I implemented this at one point and had to monkey patch AR to observe migrations, repeat them in a parallel SQLite database, then parse the logs to generate mirrored migrations to push down to the client apps.)

When Rails 3 arrives, we will have an API for ORMs and CloudKit will likely plug into this API making it a drop-in replacement for ActiveRecord. For that particular use case, it will be more like those who are using CouchDB behind their Rails apps today. This can be accomplished right now with CloudKit::Resource but using the upcoming API will make it easier for existing apps.

I have an app with two separate interfaces. The main one is XMPP, and
most interactions will happen via compliant clients. For instances where
someone's client may not be sufficient, there needs to be a lightweight
web interface. I don't need traditional authentication, as such. Since
there is guaranteed to be a secure stream directly to the user, they'll
just ask the app for a temporary access URL, click on it from their IM
client and make whatever changes they need.

The app currently uses DM/SQLite3, but I've yet to use schemaless
databases in a serious project and think they'd be a good candidate
here,  particularly as I grow tired of changing the schema. :) I'm
wondering if CloudKit would work here, but there are some points I need
to understand first.

I see that CK can either be standalone or can run alongside/in front of
other apps. I'm still wrapping my head around the middleware concept.
Let's say my app has users, and each user owns multiple widgets. How do
I use CK for storage while still allowing browsers to view resources? If
for instance I write Sinatra code that accepts the route /users, I can
of course insert that code before or after CK, but I don't see how this
solves my problem. In my tests, calling /users from my browser hands
back json as I'd expect. Is there some way of only triggering CK when
the caller specifically wants json? Or should all my browser-facing
routes include .html?

In that case, I would put Sinatra (as middleware) in front of CloudKit and bypass only in the case where the Accept header is application/json.

  

How do I restrict what records a user sees? If user or widget resources
contain confidential information, obviously I don't want non-owners to
read from or write to another's resources. I'm guessing the answer is
more middleware, but what form does it take, and where does it reside
relative to CK/the app, assuming I want the data available from both the
web view and API to be the same.

CloudKit provides an OpenID and an OAuth filter for this purpose. If you are using a custom auth setup like the link-based system you described above, then you will need to drop the following information in the Rack env upstream from CloudKit. Normally, you won't have to mess with the internals of the env, but we're digging under the hood a bit here. An auth tutorial should cover this material so that users don't have to dig into the source for this:

# Tell CloudKit to mark the new resources' metadata with the remote user.

# This also restricts access to the creator of the resource for future GETs, etc.

env[CLOUDKIT_AUTH_PRESENCE] = 1

# Tell CloudKit which authenticated user is making this request

env[CLOUDKIT_AUTH_KEY] = 'user123'

Schemaless databases are great for developers, but they seem like a
potential nightmare if the API is opened up. What prevents me as a
malicious user from injecting malicious data into your API? If there's
no schema, could I just do an HTTP POST of the entire human genome into
a field that's supposed to hold a user avatar? :) Or, even better, post
it to a field that doesn't yet exist, just in case the avatar field is
being validated?

The plan for 1.0 is to use JSONSchema for validation. This will allow three benefits:

1. It is optional, so no validation will be performed unless specified for a resource.

2. Because it is JSON, the same validator can be used on the server and the client.

3. Clients will be able to discover the validators for the resources they are using.

 

I'm sure all these problems have solutions or best practices, they're
just non-obvious to me as my head spins, "Wha? JSON web...appliance?" :)

What I keep looking for are sample CloudKit apps. Maybe a really simple
mailbox? Log in with an OpenID and leave notes for other OpenIDs? It
would be a single view and would make it very clear how to limit
returned results to only records you own, integrate CK into something
like Sinatra, etc. Maybe I'll take a crack at it and post a patch here
if I can figure it out.

That sounds like an interesting application. Please let me know if you run into snags or need more explanation of CloudKit internals.

-Jon

 

[Nicholas Orr]

Oh so that's why my message didn't appear on the list - damn confusing web interface :)

Anyways...

My understanding of rack middleware is all theory for the time being.

For now I'm just going to keep things the way they are.

I'll keep my eye on CloudKit and see what comes up

Thanks for the reply Jon,

Nick

On Mon, Aug 31, 2009 at 11:40 PM, Jon Crosby <jon.r....@gmail.com> wrote:

The only solid way to augment CloudKit at the moment is to create middleware that matches the desired route and then performs an action before or after the downstream JSON is fetched or updated. If you are new to Rack middleware, I can put together a quick example to demonstrate the technique.

-Jon

On Mon, Aug 31, 2009 at 3:02 AM, Nicholas Orr <nichol...@zxgen.net> wrote:

I've discovered CloudKit while figuring out how I get at a json
encoded request body.

My app is a pdf generator written in sinatra.
Write a "report" in haml/sass and merge that "report" with data (json
encoded post body), the result is a professional looking pdf with
images complex layout's etc. I use this app with 3 other apps at the
moment, instead of creating a lib or coding the same function 3 times
I've simply extracted this functionality into a http web service. 2 of
these apps are merb, other is C++ .Net (which currently uses crystal
reports which takes 5mins to generate a pdf and my tool takes
2-3secs...)

Looking at CloudKit I see useful bits in it like oatuh, openid, rest.
I can see how packaging my "reports" up into resources provided by
CloudKit would work very nicely.

What I don't get is how then I interact with my sinatra app's generate
a pdf method based on an object from the CloudKit "report" resource.

Any pointers?

Thanks,

Nick