Some REST API questions
urschrei
Having spent a day messing about the Zotero server REST API, and
implementing some of the API methods in Python (https://github.com/
urschrei/pyzotero), I have some questions regarding it (and its
future, if anyone can comment):
1. Are there plans afoot to update the format of the returned Atom
doc? At the moment, it works, but it's slightly awkward to work with;
for example, the primary use case for the API (as I see it) is to
retrieve a set of items, and work with their data (title, author,
publisher &c.), but the format actually makes this more difficult than
it ought to be, by returning some of that information within the
'content' field. It's pretty simple to grab some items and display
their titles, but in order to display their authors (as opposed to the
'metadata' authors, I have to additionally parse an XHTML fragment. I
understand that item data and metadata must be separate, but this
seems unnecessarily convoluted.
2. Are there any plans afoot to offer the data in other formats. I was
thinking specifically of JSON. I haven't yet checked to see whether a
Zotero item can be completely mapped onto a JSON object (it's not easy
to find even a documented Atom representation. Shouldn't it be on the
same page as the REST API docs?), but again, it seems like an ideal
choice for at least a subset of an item's data.
Dan Stillman
> Hello.
> Having spent a day messing about the Zotero server REST API, and
> implementing some of the API methods in Python (https://github.com/
> urschrei/pyzotero), I have some questions regarding it (and its
> future, if anyone can comment):
>
> 1. Are there plans afoot to update the format of the returned Atom
> doc? At the moment, it works, but it's slightly awkward to work with;
> for example, the primary use case for the API (as I see it) is to
> retrieve a set of items, and work with their data (title, author,
> it ought to be, by returning some of that information within the
> 'content' field. It's pretty simple to grab some items and display
> their titles, but in order to display their authors (as opposed to the
> 'metadata' authors, I have to additionally parse an XHTML fragment. I
> understand that item data and metadata must be separate, but this
> seems unnecessarily convoluted.
If you're already parsing the XML, what does it matter where it is in
the response? We're using Atom+XHTML so that any API URL works in a feed
reader as well, but the XHTML should be pretty trivial to parse.
> 2. Are there any plans afoot to offer the data in other formats. I was
> thinking specifically of JSON. I haven't yet checked to see whether a
> Zotero item can be completely mapped onto a JSON object (it's not easy
> to find even a documented Atom representation. Shouldn't it be on the
> same page as the REST API docs?), but again, it seems like an ideal
> choice for at least a subset of an item's data.
The write API�which we're currently putting the finishing touches
on�uses JSON, so item responses, at least, will be returnable as JSON
(i.e., content=json, as an alternative to the default content=html for
an XHTML 'content' block). We'll have to see if this makes sense for
other types of responses as well, but it's pretty easy to support if so.
Eventually the API will probably support returning items in various
bibliography formats�at least BIBO RDF�but that requires server-side
translation support, which is a little ways off.
Bruce D'Arcus
> On 3/1/11 1:37 PM, urschrei wrote:
>>
>> Hello.
>> Having spent a day messing about the Zotero server REST API, and
>> implementing some of the API methods in Python (https://github.com/
>> urschrei/pyzotero), I have some questions regarding it (and its
>> future, if anyone can comment):
>>
>> 1. Are there plans afoot to update the format of the returned Atom
>> doc? At the moment, it works, but it's slightly awkward to work with;
>> for example, the primary use case for the API (as I see it) is to
>> retrieve a set of items, and work with their data (title, author,
>> publisher&c.), but the format actually makes this more difficult than
>> it ought to be, by returning some of that information within the
>> 'content' field. It's pretty simple to grab some items and display
>> their titles, but in order to display their authors (as opposed to the
>> 'metadata' authors, I have to additionally parse an XHTML fragment. I
>> understand that item data and metadata must be separate, but this
>> seems unnecessarily convoluted.
>
> If you're already parsing the XML, what does it matter where it is in the
> response? We're using Atom+XHTML so that any API URL works in a feed reader
> as well, but the XHTML should be pretty trivial to parse.
I think it would be valuable to have JSON returned as an option.
>> 2. Are there any plans afoot to offer the data in other formats. I was
>> thinking specifically of JSON. I haven't yet checked to see whether a
>> Zotero item can be completely mapped onto a JSON object (it's not easy
>> to find even a documented Atom representation. Shouldn't it be on the
>> same page as the REST API docs?), but again, it seems like an ideal
>> choice for at least a subset of an item's data.
>
> The write API—which we're currently putting the finishing touches on—uses
> JSON, so item responses, at least, will be returnable as JSON (i.e.,
> content=json, as an alternative to the default content=html for an XHTML
> 'content' block). We'll have to see if this makes sense for other types of
> responses as well, but it's pretty easy to support if so.
>
> Eventually the API will probably support returning items in various
> bibliography formats—at least BIBO RDF—but that requires server-side
> translation support, which is a little ways off.
So again: what if some developer (like the OP) wants to help add this
support to help out. How do they do that?
Bruce
Ed Summers
> JSON, so item responses, at least, will be returnable as JSON (i.e.,
> content=json, as an alternative to the default content=html for an XHTML
> 'content' block). We'll have to see if this makes sense for other types of
> responses as well, but it's pretty easy to support if so.
If you keep going down the JSON road, and want to support JSON
representations across the board It might make sense to do something
similar to what Google did with gdata. Or perhaps look at JSON-C?
[1] http://code.google.com/apis/gdata/docs/json.html
[2] http://code.google.com/apis/youtube/2.0/developers_guide_jsonc.html
//Ed
Ed Summers
> So again: what if some developer (like the OP) wants to help add this
> support to help out. How do they do that?
Not to fill in the blanks, but this is the source here:
https://www.zotero.org/svn/dataserver/trunk/
Is it wrong to assume patches are "welcome" on this list?
//Ed
Dan Stillman
> On Tue, Mar 1, 2011 at 2:33 PM, Dan Stillman<dsti...@zotero.org> wrote:
>> JSON, so item responses, at least, will be returnable as JSON (i.e.,
>> content=json, as an alternative to the default content=html for an XHTML
>> 'content' block). We'll have to see if this makes sense for other types of
>> responses as well, but it's pretty easy to support if so.
> If you keep going down the JSON road, and want to support JSON
> representations across the board It might make sense to do something
> similar to what Google did with gdata. Or perhaps look at JSON-C?
>
> [1] http://code.google.com/apis/gdata/docs/json.html
> [2] http://code.google.com/apis/youtube/2.0/developers_guide_jsonc.html
I don't see much advantage to pure JSON representations of the Atom
feeds. The JSON we have running internally is pretty simple�basically
just the item field names and values (with arrays for creators and tags).
Something like JSON-C might be worth considering down the line as a
lightweight alternative to Atom+JSON. But I don't think there's any need
to worry about that unless people start finding Atom+JSON to be
prohibitively heavy.
Dan Stillman
It is not.
Ed Summers
> I don't see much advantage to pure JSON representations of the Atom feeds.
> field names and values (with arrays for creators and tags).
The problem you may run into (I have) is that if you slowly accrete
json serializations, you may find they start seeming a bit adhoc and
inconsistent as a whole. If you already have a consistent atom
serialization across the api (like gdata) it probably would pay
dividends to have a similar approach to the json. But hey, opinions
are cheap, particularly mine :-)
//Ed
Peter Keane
+1 on what Ed is expressing here. Having to actually think carefully
about the JSON format seems to run counter to JSON's advantages
(simple serialization of native data structures), but in fact, pays
off handsomely in the medium/long term. And for building a RESTful
system, well-known, well-understood representation semantics
(*especially* regarding links) is critical. Atom-inspired JSON can,
in my experience, be quite a big win, since systems that interact with
your will share a basic (Atom-ish) understanding of your resources.
I like JSON-C, although I am not convinced it is the only/best game in
town. In fact, w/o a proper schema language, establishing
hard-and-fast JSON formats may be difficult. In my opinion, Atom
presents a meta-model (
http://www.dehora.net/journal/2008/10/07/magnificent-seven-the-value-of-atom/
) that is hard to beat. I think a JSON serialization should at least
aspire to that utility. (By "should" I mean "it would benefit all
concerned, now and in the future...").
A couple of examples:
We use this Atom-inspired JSON format in *all* of our widely diverse
digital collections:
Also, I think it might be very useful to give a close look at the
BibJSON format. Not so much Atom-inspired, but seems a v. good fit
for Zotero, nonetheless:
http://www.bibkn.org/bibjson/index.html#mozTocId220879
And of course, with good documentation and Google's backing, JSON-C is
a fine choice as well.
--Peter Keane
> But hey, opinions
> are cheap, particularly mine :-)
>
> //Ed
>
> --
> You received this message because you are subscribed to the Google Groups "zotero-dev" group.
> To post to this group, send email to zoter...@googlegroups.com.
> To unsubscribe from this group, send email to zotero-dev+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/zotero-dev?hl=en.
>
>
Dan Stillman
> On Tue, Mar 1, 2011 at 9:12 PM, Ed Summers<e...@pobox.com> wrote:
>> On Tue, Mar 1, 2011 at 3:22 PM, Dan Stillman<dsti...@zotero.org> wrote:
>>> I don't see much advantage to pure JSON representations of the Atom feeds.
>>> field names and values (with arrays for creators and tags).
>> The problem you may run into (I have) is that if you slowly accrete
>> json serializations, you may find they start seeming a bit adhoc and
>> inconsistent as a whole. If you already have a consistent atom
>> serialization across the api (like gdata) it probably would pay
>> dividends to have a similar approach to the json.
> +1 on what Ed is expressing here. Having to actually think carefully
> about the JSON format seems to run counter to JSON's advantages
> (simple serialization of native data structures), but in fact, pays
> off handsomely in the medium/long term. And for building a RESTful
> system, well-known, well-understood representation semantics
> (*especially* regarding links) is critical. Atom-inspired JSON can,
> in my experience, be quite a big win, since systems that interact with
> your will share a basic (Atom-ish) understanding of your resources.
To be clear, the JSON we'll be offering for the write API is a
substitute for the XHTML currently included in the 'content' block of
Zotero Atom responses (e.g. https://api.zotero.org/groups/1/items).
We're not serializing item data as Atom beyond the standard Atom fields,
which provide limited summary information for the full item data in
'content'. The point is to be a direct, lossless representation of
Zotero's internal data model for use in building tools that modify
Zotero data (e.g., zotero.org).
I think this will be a worthy discussion to have (and yes, I do see your
point about the value of the Atom link semantics) if we end up offering
a format=json response�as opposed to a content=json Atom response�but
there are no current plans to do so. That could change if there was
significant demand.
chris thatcher
Atom feed renderer | http network | zotero controller
If your controller is forced to separate concerns by moving the view layer to another machine, how would you achieve this? You would still aggregate all the data you need in the controller, but you would probably 'pickle' it over the wire. Since json should provide a minimal serialization of your natural basic php data constructs (eg dictionarys, arrays, strings, numbers, booleans) you could use that without rewritting any out your Atom feed template:
Atom view <- JSON.parse() | http network | <- JSON.stringify() API Controller
Atom is a great protocol and totally sufficient but this is a different issue really has to do with design patterns that will allow you to leverage your api controllers in a lot of really interesting, and free, ways.
For example (forgive my bad php skills, just pretend ;)) from the API controller you can currently see:
switch ($this->queryParams['format']) {
case 'atom':
$this->responseXML = Zotero_Atom::createAtomFeed(
$this->getFeedNamePrefix($this->objectLibraryID) . $title,
$this->uri,
$items,
$totalResults,
$this->queryParams,
$this->apiVersion
);
break;
case 'bib':
echo Zotero_Cite::getBibliographyFromCiteServer($items, $this->queryParams['style'], $this->queryParams['css']);
exit;
default:
throw new Exception("Invalid format");
}
could be reorganized just a little to make the request scoped state more explicitly defined and reused across serializations:$data = array(
"title" => $this->getFeedNamePrefix($this->objectLibraryID).$title,
"uri" => uri,
"items" => $items,
"totalResults" => $totalResults,
"queryParams" => $this->queryParams,
"apiVersion" => $this->apiVersion
)
switch($this->queryParams['format']){
case 'atom':
//render atom with $data object
case 'json':
//render json with $data object
case 'html':
//render html with $data object
case 'bib':
//render bib with $data object
}
In this way all views are treated equally and are conceptually non-lossy in that they can expose as much information as the api controller aggregated and of course they can choose to expose less in a given view.
This has been a successful pattern for us with pictures ( http://www.loc.gov/pictures/?fo=json ) where any url will expose the state we need to render any equivalent view, eg html in most cases for us. It doesn't really cost anything to maintain and helps clarify what data the controller is really aggregating for a given request.
Sorry for barf dump, but I wanted to try to encourage this pattern while the discussion was current.
Thatcher
On 3/1/11 11:06 PM, Peter Keane wrote:
On Tue, Mar 1, 2011 at 9:12 PM, Ed Summers<e...@pobox.com> wrote:
On Tue, Mar 1, 2011 at 3:22 PM, Dan Stillman<dsti...@zotero.org> wrote:
I don't see much advantage to pure JSON representations of the Atom feeds.
The JSON we have running internally is pretty simple—basically just the item
field names and values (with arrays for creators and tags).
The problem you may run into (I have) is that if you slowly accrete
json serializations, you may find they start seeming a bit adhoc and
inconsistent as a whole. If you already have a consistent atom
serialization across the api (like gdata) it probably would pay
dividends to have a similar approach to the json.
+1 on what Ed is expressing here. Having to actually think carefully
about the JSON format seems to run counter to JSON's advantages
(simple serialization of native data structures), but in fact, pays
off handsomely in the medium/long term. And for building a RESTful
system, well-known, well-understood representation semantics
(*especially* regarding links) is critical. Atom-inspired JSON can,
in my experience, be quite a big win, since systems that interact with
your will share a basic (Atom-ish) understanding of your resources.
To be clear, the JSON we'll be offering for the write API is a substitute for the XHTML currently included in the 'content' block of Zotero Atom responses (e.g. https://api.zotero.org/groups/1/items). We're not serializing item data as Atom beyond the standard Atom fields, which provide limited summary information for the full item data in 'content'. The point is to be a direct, lossless representation of Zotero's internal data model for use in building tools that modify Zotero data (e.g., zotero.org).
I think this will be a worthy discussion to have (and yes, I do see your point about the value of the Atom link semantics) if we end up offering a format=json response—as opposed to a content=json Atom response—but there are no current plans to do so. That could change if there was significant demand.
--You received this message because you are subscribed to the Google Groups "zotero-dev" group.
To post to this group, send email to zoter...@googlegroups.com.
To unsubscribe from this group, send email to zotero-dev+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/zotero-dev?hl=en.
--
Christopher Thatcher
urschrei
On Mar 2, 4:37 pm, chris thatcher <thatcher.christop...@gmail.com>
wrote:
> discussion was current.
>
> Thatcher
>
>
>
>
>
> On Wed, Mar 2, 2011 at 1:01 AM, Dan Stillman <dstill...@zotero.org> wrote:
> > On 3/1/11 11:06 PM, Peter Keane wrote:
>
> >> On Tue, Mar 1, 2011 at 9:12 PM, Ed Summers<e...@pobox.com> wrote:
>
>
> >>>> I don't see much advantage to pure JSON representations of the Atom
> >>>> feeds.
> >>>> The JSON we have running internally is pretty simple—basically just the
> >>>> item
> >>>> field names and values (with arrays for creators and tags).
>
> >>> The problem you may run into (I have) is that if you slowly accrete
> >>> json serializations, you may find they start seeming a bit adhoc and
> >>> inconsistent as a whole. If you already have a consistent atom
> >>> serialization across the api (like gdata) it probably would pay
> >>> dividends to have a similar approach to the json.
>
> >> +1 on what Ed is expressing here. Having to actually think carefully
> >> about the JSON format seems to run counter to JSON's advantages
> >> (simple serialization of native data structures), but in fact, pays
> >> off handsomely in the medium/long term. And for building a RESTful
> >> system, well-known, well-understood representation semantics
> >> (*especially* regarding links) is critical. Atom-inspired JSON can,
> >> in my experience, be quite a big win, since systems that interact with
> >> your will share a basic (Atom-ish) understanding of your resources.
>
> > To be clear, the JSON we'll be offering for the write API is a substitute
> > for the XHTML currently included in the 'content' block of Zotero Atom
> > limited summary information for the full item data in 'content'. The point
> > is to be a direct, lossless representation of Zotero's internal data model
> > for use in building tools that modify Zotero data (e.g., zotero.org).
>
> > I think this will be a worthy discussion to have (and yes, I do see your
> > point about the value of the Atom link semantics) if we end up offering a
> > format=json response—as opposed to a content=json Atom response—but there
> > are no current plans to do so. That could change if there was significant
> > demand.
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "zotero-dev" group.
> > To post to this group, send email to zoter...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > zotero-dev+...@googlegroups.com.
> > For more options, visit this group at
> >http://groups.google.com/group/zotero-dev?hl=en.
>
> --
> Christopher Thatcher
necessary. Re-railing somewhat to the original subject (questions): Am
I correct in saying that there's no API call to retrieve a set of
groups of which a user is a member? This means that API calls
requiring <groupid> must know the group ID in advance; there's no way
to perform a 'higher level' or other initial call to return all public
groups, or all groups of which a user is a member.
--
steph
Dan Stillman
> I personally think it's a great discussion to have, so no apologies
> necessary. Re-railing somewhat to the original subject (questions): Am
> I correct in saying that there's no API call to retrieve a set of
> groups of which a user is a member? This means that API calls
> requiring<groupid> must know the group ID in advance; there's no way
> to perform a 'higher level' or other initial call to return all public
> groups, or all groups of which a user is a member.
Group membership calls are not public, but there's been some demand for
a call to determine the libraries the current API key has access to,
along with permissions for each library. So we'll likely be adding that.
Dan Stillman
Addendum: Faolan reminds me that /groups with an API key returns the
groups that the key has access to (including public ones the user
belongs to even if the key doesn't have explicit permissions for it).
Dan Stillman
Sorry, make that /users/<userID>/groups. I've added this call to the API
doc.
Peter Keane
Concerning RESTful practice, it's import to honor the HATEOAS
constraint (Hypermedia as the Engine of Application State), meaning
that with any representation, related representations of interest
should be actual hypertext links. So Instead of parsing the
representation to find a group id and then constructing the proper URL
(that you know how to construct based on some documenation somewhere),
you might have, say, within a representation of a "user" the following
link:
<link rel="http://zotero.org/relation/user_groups"
href="http//zotero.org/user/johndoe/1234/groups">
Documentation for an API should only need to document:
-- what the resources of interest are (users, user groups, titles, etc.)
-- what standardized formats are used for representations (Atom.XHTML,
JSON -- in which case the actual structure of the JSON, noting how
user agent recognizes a hyperlink needs to be specified)
-- define the link rels used (preferably using those found in the IANA
link registry: http://www.iana.org/assignments/link-relations/link-relations.xml
if/when applicable).
--peter
urschrei
Thanks for that. The API call is valid: (https://api.zotero.org/users/
436/groups?key=<my key>), but returns 0 results:
<?xml version="1.0"?>
<feed xmlns="http://www.w3.org/2005/Atom" xmlns:zapi="http://
zotero.org/ns/api">
<title>urschrei’s Groups</title>
<id>http://zotero.org/users/436/groups</id>
<link rel="self" type="application/atom+xml" href="https://
api.zotero.org/users/436/groups"/>
<link rel="first" type="application/atom+xml" href="https://
api.zotero.org/users/436/groups"/>
<link rel="last" type="application/atom+xml" href="https://
api.zotero.org/users/436/groups"/>
<link rel="alternate" type="text/html" href="http://zotero.org/users/
436/groups"/>
<zapi:totalResults>0</zapi:totalResults>
<zapi:apiVersion>1</zapi:apiVersion>
<updated>2011-03-03T14:39:21Z</updated>
</feed>
even though I'm a member of several groups. Similarly, the 'alternate'
link (http://zotero.org/users/436/groups) gives me a 'page not found'
error.
Dan Stillman
> Dan,
> Thanks for that. The API call is valid: (https://api.zotero.org/users/
> 436/groups?key=<my key>), but returns 0 results:
>
>
> <?xml version="1.0"?>
> <feed xmlns="http://www.w3.org/2005/Atom" xmlns:zapi="http://
> zotero.org/ns/api">
> <title>urschrei’s Groups</title>
> <id>http://zotero.org/users/436/groups</id>
> <link rel="self" type="application/atom+xml" href="https://
> api.zotero.org/users/436/groups"/>
> <link rel="first" type="application/atom+xml" href="https://
> api.zotero.org/users/436/groups"/>
> <link rel="last" type="application/atom+xml" href="https://
> api.zotero.org/users/436/groups"/>
> <link rel="alternate" type="text/html" href="http://zotero.org/users/
> 436/groups"/>
> <zapi:totalResults>0</zapi:totalResults>
> <zapi:apiVersion>1</zapi:apiVersion>
> <updated>2011-03-03T14:39:21Z</updated>
> </feed>
>
> even though I'm a member of several groups.
1) The key doesn't have access to your private group.
2) Keys weren't properly providing access to public groups they didn't
have explicit access to. (We had decided they should but apparently
hadn't made the change.) Fixed now.
> Similarly, the 'alternate'
> link (http://zotero.org/users/436/groups) gives me a 'page not found'
> error.
The API currently displays rel="alternate" links in a few places where
it shouldn't.
urschrei
urschrei
I'm getting HTTP 500 errors when calling the /groups/<groupID>/item /
groups/<groupID>/items/top methods. The full URLs I've tried using:
https://api.zotero.org/groups/dfw/items/top?key=[my key]
https://api.zotero.org/groups/dfw/items?key=[my key]
https://api.zotero.org/groups/german_cinema/items/top?key=[my key]
https://api.zotero.org/groups/german_cinema/items?key=[my key]
Those groups are both public, and I'm a member of both.
Jeremy Boggs
> Hi Dan,
> I'm getting HTTP 500 errors when calling the /groups/<groupID>/item /
> groups/<groupID>/items/top methods. The full URLs I've tried using:
> https://api.zotero.org/groups/dfw/items/top?key=[my key]
> https://api.zotero.org/groups/dfw/items?key=[my key]
> https://api.zotero.org/groups/german_cinema/items/top?key=[my key]
> https://api.zotero.org/groups/german_cinema/items?key=[my key]
>
> Those groups are both public, and I'm a member of both.
I think you need to use the group's ID number, not its name, in the URL.
Best,
Jeremy
--
Jeremy Boggs · Associate Director of Research · Center for History and
New Media · George Mason University · chnm.gmu.edu · @clioweb
urschrei
I suspect you're correct, but that means the API call is returning
incorrect information. Here's a single entry:
<entry xmlns:zxfer="http://zotero.org/ns/transfer">
<title>DFW</title>
<author>
<name>urschrei</name>
<uri>http://zotero.org/urschrei</uri>
</author>
<id>http://zotero.org/groups/dfw</id>
<published>2010-01-20T12:31:26Z</published>
<updated>2010-07-04T21:56:22Z</updated>
<zapi:numItems>468</zapi:numItems>
<content type="html">
<div xmlns="http://www.w3.org/1999/xhtml"/>
</content>
</entry>
Shouldn't ID be 10248?
Dan Stillman
> Jeremy,
> I suspect you're correct, but that means the API call is returning
> incorrect information. Here's a single entry:
> <entry xmlns:zxfer="http://zotero.org/ns/transfer">
> <title>DFW</title>
> <author>
> <name>urschrei</name>
> <uri>http://zotero.org/urschrei</uri>
> </author>
> <id>http://zotero.org/groups/dfw</id>
> <published>2010-01-20T12:31:26Z</published>
> <updated>2010-07-04T21:56:22Z</updated>
> <link rel="self" type="application/atom+xml" href="https://
> api.zotero.org/groups/10248"/>
> <link rel="alternate" type="text/html" href="http://zotero.org/
> groups/dfw"/>
> <zapi:numItems>468</zapi:numItems>
> <content type="html">
> <div xmlns="http://www.w3.org/1999/xhtml"/>
> </content>
> </entry>
>
> Shouldn't ID be 10248?
No. <id> is just a unique identifier. <link rel="self"/> tells you the
API URI.
But we will return a 400 instead of a 500 for that error.