Document API - Issues, clarification and our roadmap

[James Mullaney]

Hi All,

So I apologise in advance for the length of this message...

Dave and I spent a bunch of time over the last few days trying to wrap our heads around the last piece of the puzzle before we can be considered a fully compliant xAPI LRS. This is the Document API and in trying to flesh out how to build it, it has presented a number of questions as to the best implementation going forward. I thought I would post these here so as to get a confirmation on some matters, and perhaps start debate on others - where possible it would be nice to reach a consensus without having to go back to the ADLNET community.

Some background to those less familiar with the Document API part of the specification - documents are an attempt to "provide document storage for learning activity providers and Agents". This is in the form of binary data, which could be JSON or actual binary files (stop me if I'm making stuff up...that is my interpretation at least!). These 'documents' are related to either an activity (Activity Profiles), an agent (Agent Profiles) or a combination of both (States).

So one of main questions comes along the implementation of the IDs used to store documents, which for the State API is the stateID, the activity and agent APIs is the profileID. What isn't clear to me is whether this ID is being generated by the LRS at the point of creating/storing the document (HTTP PUT) or if it is a Activity Provider generated ID.  It isn't explicitly declared in the spec that this is or isn't an LRS generated value (as far as I can see), so some clarification/thoughts on that would be great. We are currently under the assumption that it is the latter and the Activity Provider provides this ID (the fact that it is a required field for the Single Document PUT gives more evidence to this in my eyes).

Another source of confusion comes around the Agent API and how it deals with "Person" objects. This section in general is a little confusing due to mixed language and terms (as highlighted by Andrew in this Github issue) but our confusion lies mostly with the "Person" object itself and how is it obtaining the data it is required to return. Let me explain...

The Combined Information GET section of the Agent API returns a ".. special, Person Object for a specified Agent." This Person object is an amalgamation of all information we have about a requested Agent, so for example if I submitted a query to the API where the agent request contained "mbox:ja...@ht2.co.uk" then I would expect back a Person object with all the names, mboxes, SHAs, openIDs and accounts that the LRS thought related to ja...@ht2.co.uk. My concern is how the LRS has made the decision to link an agent's information into one "Person" object. The specification is clear that in a statement, "an Agent MUST NOT include more than one (1) Inverse Functional Identifier". But of course this means that we can only ever send one piece of information relating to a user at any one time. If an Activity Provider sent out 2 statements, one with "mbox:ja...@ht2.co.uk" and another with openid of "ht2.co.uk/james" then how would it ever be able to imply that those are the same actor? This means that (unless there is another method of an LRS obtaining Agent information) the Person object returned from the Agent API will only ever be the same as the information you passed to the request. I can only assume I've missed something here...?

One of the other issues for discussion is the Document ID - the spec says that this is "set by the AP" and that the ".. id is stored in the IRL". The actual execution of this is slightly unclear though as this Github issue has pointed out - for the moment I think we will allow this value to be passed in as one of the parameters, as I'm unsure how it could be appended succinctly to any of the API URLs.

So I think that outlines most of our problems - we are currently moving forward with the build of this part of the API and look to have something working by end of week. We will initially take what we consider to be the best route forward, but it would be really helpful if you could throw in your two cents (pence?) to some of the areas I've detailed above in case our interpretation is incorrect or could do with some further thought.

Thanks!
James

[Andrew Downes]

Hi James, 

Lets see if I can help.

1. You are right - the state ID and profile IDs are the IDs provided by the activity provider so they can save the document and get it back later (or somebody else can get it) using the same id. 

2. There is no specified way of the LRS associating agents personas, however the spec deliberately allows that an LRS may do this in some way. For example, a nice feature for learning locker might be a page that lets people choose to associate their multiple accounts (have you ever used Klout?). Theres no requirement to have this feature for spec conformance though. 

3. This table is confusing and probably best ignored. What its trying to do is give an overview of the different document APIs. There's no actual Document id property - this is an umbrella for state id and profile id. We should reject any calls with a document id parameter and give a helpful error. 

I hope that helps. It's probably worth noting that I don't see this as my opinion/interpretation of the spec - this is the intended meaning of the people that wrote the spec, even if these bits are unclear in the physical document. 

Andrew

[Andrew Downes]

Also:

This tool may be useful for testing:

https://github.com/garemoko/tincan-getstate

http://garemoko.github.io/tincan-getstate/tingetstate.htm

I'm sure you could customise it to work with the other document APIs too (maybe with a drop-down to select?). Any code contributions welcome!

Andrew

[James Mullaney]

This is all really helpful, thanks Andrew. I think you've just confirmed the decisions we had come to.

Another quick one that has reared its head - if we are to suppose that a Document can be any form of content (binary file or JSON) then where is that content held in the request? I presumed that the parameters being sent (activityId, stateId, actor) were in the form of POST variables, but in that case does the document content need to have its own POST variable as the POST variables would make up the body?

However, if the document content is to be the body of the HTTP request (which is what I believe the spec alludes to), then would the required parameters need to be passed as GET variables instead? That doesn't seem right given the fact that 'actor' is going to be JSON, and while we could send this in using query parameters, it is surely better served using the POST variables.

For the moment we are under the assumption that it would be included in another POST variable called "content", but this isn't specified in the spec so this now becomes Learning Locker specific (which defeats the purpose of the specification of course!).

J

[Brian J. Miller]

On 2/11/14, 7:21 AM, James Mullaney wrote:
> This is all really helpful, thanks Andrew. I think you've just confirmed
> the decisions we had come to.
>
> Another quick one that has reared its head - if we are to suppose that a
> Document can be any form of content (binary file or JSON) then where is
> that content held in the request? I presumed that the parameters being sent
> (activityId, stateId, actor) were in the form of POST variables, but in
> that case does the document content need to have its own POST variable as
> the POST variables would make up the body?
>
> However, if the document content is to be the body of the HTTP request
> (which is what I believe the spec alludes to), then would the required
> parameters need to be passed as GET variables instead? That doesn't seem
> right given the fact that 'actor' is going to be JSON, and while we could
> send this in using query parameters, it is surely better served using the
> POST variables.
>
> For the moment we are under the assumption that it would be included in
> another POST variable called "content", but this isn't specified in the
> spec so this now becomes Learning Locker specific (which defeats the
> purpose of the specification of course!).
>
> J

I think this is sufficiently unclear in the spec at this point because
of the liberal use of POST throughout.

But, it is my understanding that generally these requests are all PUTs
not POSTs and as they are document storage APIs the parameters you list
are query parameters for the query string and the document goes in the
body of the request. The primary libraries in use today do it this way
at least. (They can also often be looked at for answering these types of
questions, since they work AFAIK against all commonly available LRSs.)

Having said that, POST is a special case particularly for providing
support for IE CORS requests which do not support the other methods. See:

https://github.com/adlnet/xAPI-Spec/blob/master/xAPI.md#78-cross-origin-requests

In that case the parameters normally on the query string are replaced by
a single query parameter of "method" and the additional form parameter
of "content" is specified and must be provided. Note that binary data
can't be supported in this case.

POST also has the special meaning for document merging for
"application/json" documents, but in that case the parameters would
still go on the query string since they don't have to be otherwise
replaced with the "method" parameter.

It may be worth opening an issue against the spec to improve the
handling of "POST" in the description and to make it appear exceptional
rather than the norm.

Brian
--
Brian J. Miller
Rustici Software
brian....@scorm.com

[mrdownes]

I'm not so sure on those ones as the java script library I use handles that. 

Can you look at what the example I sent you is sending and work back from that? 

Sent from Samsung Mobile

-------- Original message --------
From: James Mullaney <jmull...@aol.com>
Date: 11/02/2014 13:21 (GMT+00:00)
To: learnin...@googlegroups.com
Subject: Re: Document API - Issues, clarification and our roadmap

This is all really helpful, thanks Andrew. I think you've just confirmed the decisions we had come to.

Another quick one that has reared its head - if we are to suppose that a Document can be any form of content (binary file or JSON) then where is that content held in the request? I presumed that the parameters being sent (activityId, stateId, actor) were in the form of POST variables, but in that case does the document content need to have its own POST variable as the POST variables would make up the body?

However, if the document content is to be the body of the HTTP request (which is what I believe the spec alludes to), then would the required parameters need to be passed as GET variables instead? That doesn't seem right given the fact that 'actor' is going to be JSON, and while we could send this in using query parameters, it is surely better served using the POST variables.

For the moment we are under the assumption that it would be included in another POST variable called "content", but this isn't specified in the spec so this now becomes Learning Locker specific (which defeats the purpose of the specification of course!).

J

On Tuesday, February 11, 2014 12:37:30 PM UTC, Andrew Downes wrote:

Also:

This tool may be useful for testing:

https://github.com/garemoko/tincan-getstate

http://garemoko.github.io/tincan-getstate/tingetstate.htm

I'm sure you could customise it to work with the other document APIs too (maybe with a drop-down to select?). Any code contributions welcome!

Andrew

--
You received this message because you are subscribed to the Google Groups "Learning Locker" group.
To unsubscribe from this group and stop receiving emails from it, send an email to learning-lock...@googlegroups.com.
To post to this group, send an email to learnin...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[mrdownes]

Oh,  also take a look at my Moodle launch plugin. That sends stuff to the doc api without using a library. 

Andrew 

[James Mullaney]

Unfortunately Andrew we're still building out the API so I can't run your tools to check against them (although they do look useful)! It's all a bit Catch 22 really, we want to build it to specification - but we need to have it built in order to test that it is built to specification...

Brian, stop me if I'm wrong - but in the context of a RESTful API, isn't the use of POST for create methods and PUT for updating?

[Brian J. Miller]

On 2/11/14, 8:38 AM, James Mullaney wrote:
> Unfortunately Andrew we're still building out the API so I can't run your
> tools to check against them (although they do look useful)! It's all a bit
> Catch 22 really, we want to build it to specification - but we need to have
> it built in order to test that it is built to specification...
>
> Brian, stop me if I'm wrong - but in the context of a RESTful API, isn't

> the use of PUT for create methods and POST for updating?
>

This is that "ful" part. PUT is for updating as you already have an
identifier, or it can be for putting a new resource when the identifier
is provided via the client. Alternatively many implement PUT as update
and POST to a non-identified resource (such as a collection) for create [1].

But... that's neither here nor there, the spec generally avoids POST
except for the special cases. Where those special cases are IE
compatibility, multiple statements, document merges, etc.

[1] http://en.wikipedia.org/wiki/REST#Applied_to_Web_Services

[James Mullaney]

OK, so the consensus as far as passing in content is....

• parameters should be passed in as query string parameters - regardless of parameter type (string/json/UUID)
  
• e.g. http://example.com/xAPI/activities/state?activityId=http%3A%2F%2Fexample.com&agent=%7B%22name%22%3A%22James%20Mullaney%22%2C%20%22mbox%22%3A%22mbox%3Ajames%40ht2.co.uk%22%7D
• content is then stored and retrieved from the HTTP request body

...UNLESS we detect a 'method' query sting parameter, at which point...

• parameters will be part of the POST request and retrieved as POST variables
• HTTP headers could then also be part of the POST request
  
• content will be stored in the POST variable 'content'

Something feels a bit odd here that we use the query string parameters to pass in variables for the Document API, but for the rest of the API we use POST variables (which to me seems acceptable, especially given limitations on query string lengths across different platforms)

[Andrew Downes]

Hi James,

What I meant was you could look at the code for those examples to see what's being sent and how. 

For example here: https://github.com/garemoko/moodle-mod_tincanlaunch/blob/master/tincanlaunch/locallib.php#L114

You can see clearly what I'm passing in the header and what's in the content. I'm using a PUT here. 

Hope that helps.

Andrew

[jjhere]

I'm a novice here and wants to learn about Agent Profile API. Looks like you have already worked on it, can you provide some sample inputs to different methods in Agent Profile API so that i can understand it better ?

[Ryan Smith]

Hi jjhere, you might find the tin can prototypes and the xAPI documentation useful for examples and documentation of how to use the LL API.