problems with the appdata rest spec
David Primmer
I'm really confused about appdata. It seemed straightforward until I
tried to implement.
Primary Keys?
I've been trying to implement the appdata spec and trying to fit it
into collections and entries and it's clear from the spec that the
natural 'key' of an appdata entry is a useid / appid combination.
However in the spec, there is an example with a guid-like thing
associated with the appdata entry
"<id>urn:guid:example.org:34KJDCSKJN2HHF0DW20394</id>", I can't tell
if this is a hybrid key of user/app or some other separate id.
The format of the appid is not explicitly specified, nor the
appdataentryid (the thing above) to be a guid like the user id is.
If this is truly an appdataentryid, a real life absolute entry url
looks like this:
/appdata/{guid}/@self/{appid}/{appdataentryid}
/appdata/GUID0403923LKFL24/@self/03SL403KBAZOFLSE/AEIDDCSKJN2HHF0DW20394
Yuck.
So that's one thing.
If this is not right, and I don't think it is, then
"/appdata/{guid}/@self/{appid}" locates an entry. If so, what is the
collection url pattern for this AppData entry? Seems like the only
choice is user or app. Right now it appears to be leaning toward user.
If the url to POST to is "/appdata/{guid}/@self/" then the {appid}
becomes the entry id for the appdata entry. And that cannot be.
I guess I just don't know how to do an Atom service with a hybrid key
like this. To me, a logical url pattern to POST to in this case would
be "/appdata/" and the server would generate an ID of
"{guid}+{appid}". The collection being posted to is the container that
hosts a flat list of these unique data items. And in the Atom entry id
element, you'd have <id>urn:guid:example.org:{guid}+{appid}</id>"
/appdata/{guid}/@self/{appid}?fields=count -- Just the count field
for user {guid}, app {appid}
In this object, the content element has multiple sub data elements,
or fields, like 'count' 'poke' and 'last poke', and from the
perspective of Atom, it's one entry. If this is the case, you'd assume
that /appdata/{guid}/@self is a collection of all data the user has
for all their apps. However, this collection is not clearly specified
in the spec.
I know I complained about this previously, but I think in this case,
one parent collection of these entries needs to be specified so people
roughly know where to POST to. I don't see how you can do this purely
in a discovery doc. Making the entry url a combo key like
"/appdata/{guid}+{appid}/" would also make a canonical location for
the collection: "/appdata/". In Atom, it's ok to give a special edit
link for an existing entry, but generally, you're supposed to be able
to infer where to POST to without "help".
Section 6.3 of the spec mentions "@self to select the appdata for user
guid only (a collection)" This seems to be what I'm inferring above
but it's not clear. And it doesn't seem like a very useful collection.
This general pattern led wonder if the common case for pulling all the
data for all the apps a user has is displaying profile pages. Seems
like the url format could just as easily be reversed
davep
Louis Ryan
David Primmer
suppose we can implement it the way we suggest in shindig since the
url patterns in the spec are *examples* of what you could do (I'm
usually arguing for more concrete url patterns in the rest spec but
here I'm using the apparent open nature of the spec against it.)
Btw, I'm still confused if the two variables, guid and appid uniquely
identify an entry or if they point to a collection. How do clients use
more than one entry in the appdata store? The spec seems to indicate
that there is one entry object per app/user combo with multiple
key-val pairs in it.
Btww, I still don't have an answer to what the POST'able url for
appdata is. This is the same as the answer for 'what's the collection
url of an appdata entry'.
davep
John Panzer
I think there is a mismatch between the structure of the appdata URL scheme and the expected use cases that may or may not resolve this issue.As David points out the fully-qualified address of an entry is/appdata/{guid}/@selector/{appid}/{appdataentryid}and while I agree that this is heavy it is consistent and would allow for query facilities over the parent collection using standard query params.My issue is the assumption of use-case this URL enforces. Consider the degenerate parts...1. /appdata/{guid}/@self/{appid} -- All app data entries owned by a specific user and app.
2. /appdata/{guid}/@friends/{appid} -- All app data entries owned by the friends of a specifc user and app.3. /appdata/{guid}/@self -- All app data entries owned by a specific user for all apps
4. /appdata/{guid}/@friends -- All app data entries owned by the friends of a specific user for all apps5. /appdata/{guid} -- All app data entries owned by a specific and all their related users for all apps
6. /appdata -- All app data entries in the systemOf these potential modes of access only 1 & 2 have any utility. The ability to access entries across applications (3,4) is not useful to the applications themselves and seem intrinsically insecure to implement.
The rationale for the modes was to cover all possible bases in the protocol (but individual containers may or may not provide all such modes, and if they do, may not provide it for all situations due to security concerns.)
#3 would be needed to do an uninstall or search utility that is not app-specific for example (I hope a user, or site admin, will be able to uninstall apps for a user without the apps' co-operation...) It certainly doesn't need to be supported by all containers though.
#6 is clearly something that would be available only to system administrators, if ever :)
A url scheme of/appdata/{appid}/{guid}/@selector/{appdataentryid}seems more generally useful
The URL template can be anything the container wants, as long as it matches up the parameters and advertises in the XRDS file. For example:
/appdata.cgi?a={appid}&u={guid}&s={selector}
So I'm not sure what the difference would really be.
John Panzer
In celebration of the new spec, I though I'd post some problems I have with it!
I'm really confused about appdata. It seemed straightforward until I
tried to implement.
Primary Keys?
I've been trying to implement the appdata spec and trying to fit it
into collections and entries and it's clear from the spec that the
natural 'key' of an appdata entry is a useid / appid combination.
However in the spec, there is an example with a guid-like thing
associated with the appdata entry
I'm not seeing that, can you point to it? (Possibly it's a paste-o!)
"<id>urn:guid:example.org:34KJDCSKJN2HHF0DW20394</id>", I can't tell
if this is a hybrid key of user/app or some other separate id.
The format of the appid is not explicitly specified, nor the
appdataentryid (the thing above) to be a guid like the user id is.
So the spec has this as the core example:
/appdata/{guid}/@self/{appid} -- All app data for user {guid}, app {appid}
...and so you'd get /appdata/GUID0403923LKFL24/@self/GUID933999387776
Note: This assumes that every app has a globally unique ID. I had thought at last check that this would be the URL of the app's gadget.xml file, which is universally unique but has some issues; but if that's not the case then there has to be some usable ID that the app knows about that it can use with each container. Not sure what the format is, this spec is intended to draft off of the rest of the spec in this regard.
The next question is what kind of data you get out of this. It says a collection, but the example shows a single Entry (using AtomPub terms here), not a Collection. I think this is an error in the spec. For the JSON format it doesn't matter I think, and I am having trouble imagining anyone using the AtomPub format here -- it's mostly for completeness and edge cases.
So: Should the spec example be fixed up to use an AtomPub Collection (like this: <feed>...</feed>), with 1 Entry per appdata key/value pair? It's a lot more verbose but at least it's consistent and given the small amount of usage I expect the Atom format to get for this particular data type I'm not worried overmuch about the size. And then the spec wouldn't be inconsistent at least.
Kevin Brown
On Sun, Jun 1, 2008 at 12:48 AM, David Primmer <david....@gmail.com> wrote:
In celebration of the new spec, I though I'd post some problems I have with it!
I'm really confused about appdata. It seemed straightforward until I
tried to implement.
Primary Keys?
I've been trying to implement the appdata spec and trying to fit it
into collections and entries and it's clear from the spec that the
natural 'key' of an appdata entry is a useid / appid combination.
However in the spec, there is an example with a guid-like thing
associated with the appdata entry
I'm not seeing that, can you point to it? (Possibly it's a paste-o!)
"<id>urn:guid:example.org:34KJDCSKJN2HHF0DW20394</id>", I can't tell
if this is a hybrid key of user/app or some other separate id.
The format of the appid is not explicitly specified, nor the
appdataentryid (the thing above) to be a guid like the user id is.
So the spec has this as the core example:
/appdata/{guid}/@self/{appid} -- All app data for user {guid}, app {appid}
...and so you'd get /appdata/GUID0403923LKFL24/@self/GUID933999387776
Note: This assumes that every app has a globally unique ID. I had thought at last check that this would be the URL of the app's gadget.xml file, which is universally unique but has some issues; but if that's not the case then there has to be some usable ID that the app knows about that it can use with each container. Not sure what the format is, this spec is intended to draft off of the rest of the spec in this regard.
Using url as the globally unique id would be great (which is why I keep proposing it), but the major drawback is url portability and aliasing.
Today, most containers assign numeric identifiers for each app.
David Primmer
Ok, I think you've clarified that I took the example urls for more
than they were. I personally don't think the is very helpful to new
container implementers as not everyone will have an immediate idea as
to how they'd like to implement the spec. Maybe better for existing
containers. In essence, you're saying, "here are our domain objects,
go make a rest service with them and document it in your discovery
doc".
David Primmer
> On Sun, Jun 1, 2008 at 12:48 AM, David Primmer <david....@gmail.com>
> wrote:
>>
>> In celebration of the new spec, I though I'd post some problems I have
>> with it!
>>
>> I'm really confused about appdata. It seemed straightforward until I
>> tried to implement.
>>
>> Primary Keys?
>>
>> I've been trying to implement the appdata spec and trying to fit it
>> into collections and entries and it's clear from the spec that the
>> natural 'key' of an appdata entry is a useid / appid combination.
>> However in the spec, there is an example with a guid-like thing
>> associated with the appdata entry
>
> I'm not seeing that, can you point to it? (Possibly it's a paste-o!)
>
>>
>> "<id>urn:guid:example.org:34KJDCSKJN2HHF0DW20394</id>", I can't tell
>> if this is a hybrid key of user/app or some other separate id.
I quote the spec doc. See section 2.5 AppData in the atom representation.
>>
>> The format of the appid is not explicitly specified, nor the
>> appdataentryid (the thing above) to be a guid like the user id is.
>
> So the spec has this as the core example:
>
> /appdata/{guid}/@self/{appid} -- All app data for user {guid},
> app {appid}
>
> ...and so you'd get /appdata/GUID0403923LKFL24/@self/GUID933999387776
>
> Note: This assumes that every app has a globally unique ID. I had thought
> at last check that this would be the URL of the app's gadget.xml file, which
> is universally unique but has some issues; but if that's not the case then
> there has to be some usable ID that the app knows about that it can use with
> each container. Not sure what the format is, this spec is intended to draft
> off of the rest of the spec in this regard.
>
> The next question is what kind of data you get out of this. It says a
> collection, but the example shows a single Entry (using AtomPub terms here),
> not a Collection. I think this is an error in the spec. For the JSON
> format it doesn't matter I think, and I am having trouble imagining anyone
> using the AtomPub format here -- it's mostly for completeness and edge
> cases.
>
> So: Should the spec example be fixed up to use an AtomPub Collection (like
> this: <feed>...</feed>), with 1 Entry per appdata key/value pair? It's a
> lot more verbose but at least it's consistent and given the small amount of
> usage I expect the Atom format to get for this particular data type I'm not
> worried overmuch about the size. And then the spec wouldn't be inconsistent
> at least.
>
I think my suggestion of packing all the key/val pairs into a single
entry and making that an entry is fine. I don't know how big these
objects are suppose to get, and it could get unwieldy without partial
updates -- the representation can get big for a client to handle.
However, you've sidestepped the issue of what the container for these
entries is, or I can't find an answer in your reply. This issue with
regard to the atom and json format is only whether the addresses refer
to the same chunk of data. Both atom and json need a url to post to.
I'd like to see a response (if you have the time) to what Louis and I
have suggested around guid+appid. The url format examples, while
looking familiar to those using the other services, is clunky in this
generic storage context.
davep
Louis Ryan
David Primmer
> You example for #3 is much more likely to be a container provided UI
> facility rather than something which should be exposed to application
> developers in a RESTful API. The degenerate cases for the alternate URL
> format are more applicable to consumption by an application developer which
> I believe should take precedence. Spec implementors are likely to infer use
> cases from the structure of the example URLs in the spec whether we want
> them to or not.
> I think the issue here is should Shindig implement the canonical example
> provided in the spec or should the spec pull them from Shindig. Given the
> implementation constraints it should probably be the latter.
I'm starting to lean that way as well. See Bentley's post about
creating a canonical XRDS doc. We could define one for shindig and
then in the future, if people are really interested in having the spec
be open, that would be our non-spec blueprint for REST services.
> @Dave - I worry about the turning the appdata into a single entity rather
> than a collection because the JS API presents it as a queryable collection.
> This is not an insurmountable obstacle but will likely cause a RESTful
> backed implementation of the JS API to perform unnatural acts.
Hang on a second. If you look at the spec again for what is called "An
isolated AppData example" (which I'm referring to as an "entry", the
smallest unit of addressable data), you'll see that we're already
there, Sure, this has query semantics that can change it's format, by
specifying fields, but it's not the same as the next section of the
spec which deals with the Collection, "An AppData Collection Example".
Here you have a list of lists in json and a feed of entries in atom.
From this, I assume that we're just having a name confusion here and
when you say "the JS API presents it as a queryable collection",
you're saying that each appdata entry object can be 'filtered' (I
wouldn't call it a query in this case) to only display certain fields.
See what I'm saying? There's no unnatural act that needs to be
performed. The Backend implementation in shindig is actually not setup
right for the REST API design and needs to be extended. This is a
natural result of the interfaces being written well before REST API.
A collection of single-valued entry objects or one entry with many
values is better than what it is now, which is an Atom Feed that has
one and only one entry which contains all the values. I agree that the
query semantics imply collection, however, making every key-val pair a
stand-alone Atom entry with a guid of it's own is ridiculously
verbose, It's a sad state of affairs.
In the end, I guess I think the wire format is alright. The URL
semantics need to be updated to more naturally express what the
collection is, and the backend interfaces need to be updated.
davep
Ian Boston
Inline:
I would strongly agree with this, get the wire format compact and
right for purpose, then make the URL format fit both the natural rest
semantics. At that point any server technology could be used, but if
its shindig java, let the wire protocol and urls drive the
implementation and pull requirements out for a SPI layer that fits
under the rest api impl.
I have just been through a similar transformation in another project,
and focusing back on bytes on the wire (json format) and driving the
iiop and implementation from that point of view has eliminated a
large number of blockers for both front end and back end developers.
Ian
(all imho)