Availability Request/Format
Ryan Eby
I was wondering if we should try our hands at getting an object
availability format and request structure settled on. I'm thinking
that a simple availability api could be useful even without jangle,
allow adoption and also give something for jangle to consume.
Previous discussions have jangle consuming json through the connectors
(or ILS). This seems like a simple format that would be easy to
consume by websites, if say LibraryThing wanted to show the status of
something.
As for the format itself are there many examples of status information
as would be expected for an item in current standards, library and non-
library? There's NCIP, SRU listed at
https://project.library.upenn.edu/confluence/display/ilsapi/Real+Time+Search
Should it link back to the full bib/item/object info or include some
non-status information about the object beyond the identifier used?
Just getting some things out there for discussion.
Eby
Godmar Back
Thank you for this cross-post, Ryan, which alerted me to jangle. I
find this project very interesting.
I browsed through the documentation on the jangle website and I am
confused, to say the least. I hope I'm not offending anybody if I just
fire away with questions.
Is Jangle an API or Ruby software, or both? If it's both, is there
intent to keep a separation of the two and allow, for instance, an
implementation of Jangle using alternative platforms? Also, do intend
to specify both the API between the discovery platform and jangle and
the API between jangle and its connectors, or only the former? If
both, what's the latter? Do connectors have to be implemented in Ruby?
What does the "Jangle Core" mentioned on
http://groups.google.com/group/jangle-discuss/web/jangle-spec do? The
page doesn't describe it. It would be nice to outline jangle's core
functionality. What's its purpose?
What is the architecture of jangle's core and connectors? Do the
"core" and "connectors" reside within the same process, machine,
domain, or is this undefined? To what extent are implementations of
Jangle distributed architectures?
The documents mention that Jangle uses REST - but not much more. Is
this REST/XML? REST/JSON? REST/JavaScript widgets? Is there an
unAPI-like connector upfront to learn supported formats? Or has this
not been defined?
What's the relationship between the (closed) ILS DI effort and (open
source) jangle?
Finally, are there running jangle services anywhere?
Thanks!
- Godmar
Ross Singer
I'll step in here.
Let me start by saying Jangle spec development is *just* starting (a
lot of the docs on the wiki were from previous brainstorming when we
were trying to find the traction to move forward -- some housecleaning
is probably in order).
The objective of Jangle has always been to be a RESTful interface to
library services. A couple of weeks ago, it was proposed to use the
Atom Publishing Protocol as the outward facing interface. The idea
being that it's RESTful, has a client base to lean on, and is
extensible. Also, it's a fairly simple constraint to work within to
focus on the resources themselves.
The idea is you'd have the Jangle "core", which is effectively an
AtomPub interface that routes and proxies communication from the
clients to the Jangle "connectors" which are application specific
adapters to abstract the data, schema and business logic in, say,
Voyager or ILLiad into something Jangle specific.
As to your Ruby question, Jangle is intended to be language agnostic
(both core and connectors). Our original prototypes were in Ruby
because both Elliot Smith and are comfortable with it and can develop
prototypes rapidly in it. Also, since we were actually using JRuby,
we had the idea that you could use Java (and therefore Jython or PHP
via Quercus, etc.) to consume the output of the connectors natively
instead of HTTP.
Also we were toying with the notion that the strict convention over
configuration philosophy of Rails would eliminate some of the need to
argue over *how* to implement everything and help us get moving on
actually doing something. That prescription method is also some of
the motivation behind using AtomPub.
I'm not sure that last paragraph cleared anything up. Jangle isn't
Ruby-specific, but pretty much everything written for Jangle has, up
until now, been written in Ruby. Equinox has been toying with a
Jangle server in OpenSRF, however, which is very not-Ruby.
So the connectors are also language agnostic. Again, nothing is set
in concrete, but the idea is that core would communicate to the
connectors via REST/JSON.
As to your last two questions:
> What's the relationship between the (closed) ILS DI effort and (open
> source) jangle?
The motivation for Jangle is create a stable, consistent API for all
library services to be able to more easily build something like the
DLF API on top of. If Jangle connectors exist, you'd (theoretically)
only need one OAI-PMH service or one NCIP server that would be uniform
for any library.
One of the barriers to adoption (in my mind) for NCIP and, I fear, the
DLF ILS API are the complexity of interoperability with regard to
economies of scale. If NCIP is a complicated spec to implement and
the library sytem that I want to interoperate with either has no NCIP
or partial NCIP or a strange interpretation of it (I know, such a
world seems hard to imagine, but bear with me) the incentive of
building a native NCIP service is low. Especially since the local
payoff is so weak, it's not like if I implement this service, I'm
going to then be able to benefit from the wealth of NCIP client
libraries in my programming language of choice to interact with my own
data.
However, if the goal is make the data available to an AtomPub server,
the local benefit is there (since there /are/ client libraries in my
programming language of choice, since it's what the Google uses), and
if the NCIP service can be written on top of that, two birds are
killed with one stone (no birds have been harmed in the making of
Jangle).
So Jangle is intended to make the DLF initiative easier to implement.
I have no faith in vendors delivering anything consistent or timely.
Call me a cynic.
> Finally, are there running jangle services anywhere?
Well, there's no spec yet.
That being said, as we flesh out the spec, I plan to "proof of
concept" our API on OpenBiblio (since it's small and pretty simple and
runs on my Macbook) and Talis Alto (although this will probably be
after 30 day spec releases). The current group includes developers
for Evergreen, Koha and VuFind (which has connectors to several ILSes
already), so I expect that some development will happen in those
arenas as well.
Thanks for the questions. It's hard to "launch" the idea of writing a
spec to help other specs be implemented without a) confusion around
conflicts of overlapping scope b) knowing quite what it look like yet
(esp. since that has yet to be defined! Anyone who is interested is
welcome to participate) and c) this is nobody's full time job.
I wrote up a little more of the background of this here:
http://dilettantes.code4lib.org/?p=112
Thanks, again,
-Ross.
Godmar Back
I'm worried that the semantics of the services jangle may provide (say
circulation information etc.) may not fit the Atom syndication model
well. For instance, consider RFC 4287's atom:id, which must provide an
immutable id for an entry or feed. What meaning could be reasonably
assigned here to an entry if the feed represents ILS search results?
How would jangle make up atom:id's, and what meaning would the term
"Document" as in RFC 4287 have?
- Godmar
Ross Singer
Although, to take your circ example, it may be a matter of shifting
the focus of the noun (say, to the item that is circulated). Also
"immutable" doesn't necessarily mean "resolvable". After all,
"expired" circ URIs could return 403s. That would be perfectly valid.
Also, the ILS search results is the same thing:
http://jangle.example.org/koha/search?dc.title=Nature
Why is this any different than the atom feed for my blog?
-Ross.
Ross Singer
may be confusion and concern.
Jangle isn't really an AtomPub interface to library services. It's a
RESTful interface for library services and, in the first 30 days of
spec revision, we're testing the theory that AtomPub might be the
framework we can hang this off of so we don't have to design that
aspect, too (since there's still a lot we need to consider).
It's perfectly reasonable that AtomPub may not be a good fit. I don't
want to dismiss it offhand for its potential limitations in the face
of library problems before we actually try to apply it, though. After
all, Google is getting a hell of a lot use out of it (well, GData, but
that's just branding) for a wide variety of different services.
Services that don't automatically jump to mind when you think "Atom".
We've also talked about the possibility of a hybrid spec, where we use
the simplicity of AtomPub to handle the 80% cases (record harvesting,
for example, seems right up its alley), and mix in a more
sophisticated REST RPC if there are actions we just can't figure out
how to model in AtomPub.
But, really, this is the advantage of the 30 spec release. If it's
not working, we haven't wasted much time.
-Ross.
Emily Lynema
To make sure that anyone willing to attack this functionality does so in a useful manner, we'd like our document to specify at least the *minimum set of data* that should be returned from the function and possibly how it should be encoded. I think we'd also like to specify a RESTful URL template/form that could be used to make this query, but that's of lesser importance.
I think that the response must include a simple enumerated status message that can be used to make availability decisions across ILS's. Then perhaps a free-text message can provide more locally customized information. The NCIP implementation profile [1] enumerates a list of statuses [2]. I was going to propose that we use this for the enumerated status message, but it doesn't exactly give me the warm fuzzies.
On the jangle-discuss site [3], Andrew Nagy suggested a binary value for availability instead of an enumerated message. I'm warming to that idea, as long as there is a way to figure out the difference between available and not available. For instance, available might mean 'available on the shelf' or 'available library use only' or 'it's online so it's always available' or 'waiting to be reshelved'. Not available could mean a lot of other things - checked out, lost, in process, in transit, on hold, etc. This gives each system a little flexibility in how they define available vs. not available, which might not be a bad thing.
In terms of other information that should be included in the object availability response, Andrew suggested the following:
* Input: Array RecordId
* Output: Bool Availability, String Status, String Location, Bool Reserve, String Call Number, String Due Date, Int Item Number, String ItemID
I have a couple of thoughts:
* I don't think whether something is on reserve needs to be a separate response value. Why not just make 'on reserve' part of the status or location message?
* What's the difference between Item Number and ItemID? Wouldn't both generally just be the barcode?
* Would it be useful to have a boolean for whether or not this item circulates?
* What about an int for number of holds? Although this could be problematic, since some systems express this at the title level and some at the item level, depending on configuration.
* Should information like 'library use only' be a part of the status message, or should that be encoded in a separate element that deals with usage restrictions? Both NCIP and ISO Holdings Schema [4] provide an enumerated element for this [5] (although again, these lists don't exactly give me warm fuzzies).
Terry Reese and I thought it might be useful to be able to retrieve availability using either an item identifier or a bibliographic identifier, depending on what you're doing. Is that overkill?
I think both the item identifier (barcode) and bibliographic record identifier should be included with each item in the response. This is necessary to associate items with titles if I'm requesting availability for more than 1 title at a time. If I were to do this in a programming language, I'd say the response should be a hash keyed by bibliographic record identifier where each value is a list of one or more item objects. But we're trying to be more web services oriented here, so we need to think more in terms of json/xml data. Any thoughts? I've envisioned using NCIP / ISO Holdings to embed holdings/item information into requests for bibliographic records [6]; are those schemas appropriate here or would a simpler, completely new schema be best?
At a broader level, is this function most useful for providing information about the status/availability of an item, or for providing a fairly complete view of information about an item? There has been some argument for an availability request that only includes status information (think availability, status, date due, and identifiers; simpler sometimes *is* better). I personally think it would be applicable across a broader set of use cases if it included other basic information about an item. Can we summon good use cases for including call number and location? What about others not in Andrew's list like item format, library, call number scheme, item notes? Should this really be GetItem rather than GetAvailability?
-emily
[1] http://www.niso.org/kst/reports/standards?step=2&gid=None&project_key=88add52c35a51f8acb31225a533599dfc679f913
[2] NCIP list of open enumerated values for CirculationStatus element:
Available For Pickup
Available On Shelf
Circulation Status Undefined
Claimed Returned Or Never Borrowed
In Process
In Transit Between Library Locations
Lost
Missing
Not Available
On Loan On Order Pending Transfer
Recalled
Waiting To Be Reshelved
[3] http://groups.google.com/group/jangle-discuss/web/jangle-spec
[4] XSD: http://tinyurl.com/64metl
More about ISO Holdings Schema (ISO 20775): http://www.loc.gov/standards/sru/june2007meeting-report.html#holdings
[5] NCIP open enumerated values for ItemUseRestriction element:
Available For Supply Without Return
In Library Use Only
Limited Circulation, Long Loan Period
Limited Circulation, Normal Loan Period
Limited Circulation, Short Loan Period
No Reproduction
Not For Loan
Overnight Only
Renewals Not Permitted
Supervision Required
Term Loan
Use Only In Controlled Access
User Signature Required
ISO Holdings enumerated values for <availableFor> element:
unspecified
loan
physical copy
digital copy
online access
reference look-up
other
[6] Find examples for MARCXML + NCIP and MODS + ISO holdings at the tail end of section 6.4.1, under 'Putting it all together'
https://project.library.upenn.edu/confluence/display/ilsapi/Real+Time+Search
Godmar Back
system using pazpar2's Z39.50 gateway):
1. The ability to issue simultaneous requests for multiple bibitems is
a must have. Typically, we'll issue a batch of requests for all
related ISBNs in an xISBN set. We don't want to have to puzzle
together multiple responses; it complicates the client's coding and it
also raises UI considerations.
2. The design must include consideration for when a library holds
multiple copies of the same item. Don't expect the client to provide
different record identifiers for each copy. Instead, design the result
format so that it's clear when information about multiple copies is
provided. [This is different from, btw, simply providing multiple
search results. The client should be able to find out, effortlessly,
which items are copies of each other.]
- Godmar
Ross Singer
groups to work in concert. If the DLF group doesn't have an existing
protocol in mind for request/response and the Jangle group needs a
'deliverable' to work towards, this may be it.
The only downside I can see is that Jangle would have some assumptions
and shortcuts (such as the fact that every item or record would have a
URI) that may not exist in the DLF context.
What do you think?
Emily, why do the NCIP/ISO Holdings messages leave you wanting? Is it
that you'd really need to mix and match between the two to really get
the full story? It may just be RDF in me talking here, but it seems
perfectly doable to pull from both vocabularies.
-Ross.
Emily Lynema
By the way, are you referring to the message for circulation status or for item use restrictions? I didn't actually list the ISO Holdings messages for circulation status, which I think might be a good option to use that cover most of the use cases John cites in his email to ils-di this morning.
unknown
available
not available
possibly available
In case you didn't see it, John suggested that instead of a binary available/not available, we use an enumerated list something like this:
Available (the simple "yes")
NotAvailable (the simple "no")
ComplexAvailability (the "it depends", perhaps due to multiple items,
context dependence, or other values)
AvailabilityUnknown ("i can't tell"; useful when the query can't be
answered; might also be applicable to server errors
and server unavailability)
AvailabilityError ("i won't answer that"; useful for client
errors, identifiers the system doesn't recognize,
an unauthorized client, etc.)
Personally, I don't think ComplexAvailability is a viable use case. In the scenario of GetAvailability, we are reporting on the availability of individual items, not on the availability of groups of items. And if its a second copy, to me it seems like that should be its own individual item with its own individual availability, even if it doesn't have its own unique barcode (although we're crazy if we allow that in cataloging practice). And if you really can't make the determination, then just use an unknown value. But the ISO Holdings messages does have the 'possibly available' value that maps to ComplexAvailability, so I think we're covered. I also think AvailabilityError is at a broader level; that should be the entire message response if the functionality isn't supplied, the client is unauthorized, etc., it shouldn't be the availability if an individual item.
So I could get behind the ISO holdings list from above for a short enumeration of values for the availability element in the response, particularly if we allow for a free-text status message, as well.
-emily
Emily Lynema
I'm not sure I completely understand your statements about multiple copies of the same item. Especially since each copy must have its own identifier (aka, barcode) if it circulates. And it certainly has an identifier of some sort in the underlying database. I tend to call each new copy a new item in my mind, so I may be missing out on something fundamental in terms of how things are actually done.
Can you take a stab at what it might look like in a response where "The client should be able to find out, effortlessly,
which items are copies of each other." In your experience, when there are multiple copies, of the same item, how are identifiers provided? Is there a single barcode that could be combined with copy number to create a unique identifier?
-emily
Godmar Back
it. I suggest to define the model in terms of how a client could
reasonably act after receiving a response (rather than in terms of
what/how you librarians think!)
"high availability" - the server signals that a client should display
the result to a human. The human should have confidence that they can
get and have (mostly) unfettered use of the item: e.g., the book's on
the shelf to borrow, or an article can be downloaded or electronically
viewed. No client will be embarrassed to display this result to the
user.
"low availability" - the server believes that a human can access the
item, though it may require some hoops for the user to go through: the
book may need to be recalled, or it's on 2-hour reserve, or similar.
Generally, clients will provide a way for the user to opt-in or out
of "low availability" notifications: think a "also show items
currently not on the shelf", "also show reserve items".
"no availability" - the server believe that there's no point for the
client to provide any result to a human. In fact (if this is used as
part of a human-initiated process), the client may say "not
available."
My key point is that the "low availability" settings should not be
used if the server is in doubt, but rather if the server is reasonably
confident that reporting the result to a human will be reasonable,
even though the item is encumbered in some way.
I don't feel too strongly about the availability unknown and errors;
they may not matter for practical purposes. It may be good to
distinguish "error" from "no", but not "no" from "I don't know"
because if you think about it, no server can be certain that they know
which applies. So: fold "No" into "AvailabilityUnknown" and stick with
four answers.
- Godmar
On Wed, Apr 9, 2008 at 10:58 AM, John Mark Ockerbloom
<ocke...@pobox.upenn.edu> wrote:
>
> [going to ils-di only; I can't seem to post to jangle-discuss]
>
>
> Godmar Back wrote:
> > 1. The ability to issue simultaneous requests for multiple bibitems is
> > a must have. Typically, we'll issue a batch of requests for all
> > related ISBNs in an xISBN set. We don't want to have to puzzle
> > together multiple responses; it complicates the client's coding and it
> > also raises UI considerations.
>
> Yes, I've heard this from multiple sources. The spec should include
> the ability to request availability for multiple identifiers at once.
>
> Beyond that, I'd recommend for this (and all the other level 1 functions,
> which are meant to be bare essentials supported soon bv ILS's)
> to keep the required profile as simple as possible. This can still be done
> while allowing room for expansion.
>
> For example, one can define a request
> URL with parameters for IDs and ID type, and allow additional parameters
> to be used by some implementations if some ILS's want to support more complex
> queries. The reply can similarly define XML elements for the reply, for
> items within the reply, for a simple enumerated value denoting availability,
> and for an unstructured message that can displayed with more information about
> the availability. Other elements can be defined by ILS's that want to provide
> more structure to the reply, but I'm hesitant to include more required structure
> than this for Level 1.
>
> We might need more than a boolean for the enumerated availability value. I
> don't think it's necessary to introduce the various kinds of availability
> given by NCIP at the basic "is is available?" level, but as you note, there
> are some nuanced answers that may be necessary in some cases. You
> mention below there may be multiple copies or parts of a bibliographic item,
> which may have different availabilities. And it would probably be useful to
> indicate when the ILS simply can't return a value, either because of problems
> at the server or client end.
>
> So, putting this together, perhaps we could make do with 5 values:
>
> Available (the simple "yes")
> NotAvailable (the simple "no")
> ComplexAvailability (the "it depends", perhaps due to multiple items,
> context dependence, or other values)
> AvailabilityUnknown ("i can't tell"; useful when the query can't be
> answered; might also be applicable to server errors
> and server unavailability)
> AvailabilityError ("i won't answer that"; useful for client
> errors, identifiers the system doesn't recognize,
> an unauthorized client, etc.)
>
> if that's not too complicated. Thoughts?
>
> John
Godmar Back
with a question:
What do we assume about a client's ability to identify an item? In
other words, is what we are discussing a "known item type" interface
or a "search type" interface, or should it be flexible enough for
both?
Assuming, for now, that we're talking about a known-item type
interface, what I'm thinking of is simply the ability of a server to
return multiple responses when asked about the status of a known item.
The client will interpret each response as reporting on the status of
a particular, identical (=interchangeable) copy of this item.
In short, a client sends a set { id1, id2, id3 } to the server, the
server responds with a list { r(id1), r(id2), r(id3) ... }. If there
are multiple copies described by "id1", the server's list will contain
multiple elements { r(id1), r(id1) }. Each response contains the id
of the item to which it belongs, allowing the client to map responses
to requests.
- Godmar
Ross Singer
checked out (which is, "not available, but will be after x date") and
an item that's missing or lost?
Or does checked out correspond to "low availability"?
In my mind there are 3 states:
available
unavailable (with availability date -- which might be never)
unknown
what this doesn't address is:
this thing is on reserve (so, available, but maybe not to you)
"available" but not "accessible" (i.e. it's in storage)
Another question I have... how does this translate to services *other*
than the ILS, like an ERMS?
-Ross.
Godmar Back
>
> My question would be, is there a difference between an item that's
> checked out (which is, "not available, but will be after x date") and
> an item that's missing or lost?
>
> Or does checked out correspond to "low availability"?
> In my mind there are 3 states:
> available
> unavailable (with availability date -- which might be never)
> unknown
>
> what this doesn't address is:
> this thing is on reserve (so, available, but maybe not to you)
> "available" but not "accessible" (i.e. it's in storage)
>
So, it's a step back, no? The 4 or 5 enum proposals address this.
The gist of my proposal was that this standard shouldn't attempt to
strictly define how a particular situation (checked out, recalled,
reserve, whatever local arrangements may exist) maps to an answer ---
rather, we ask that the provider applies a standard (i.e. decision
criteria) which we define when deciding which answer to give.
The decision criteria I am proposing is whether the provider would
want human to see it unprompted (like an "ad") [high], or whether only
interested (opted-in) human should see it [low], or whether the
provider deems it best to not communicate any availability to a human
[no avail], and finally whether the answer should be disregarded this
time [error]. Different implementors may apply different judgment here
in what answer to give - when to say "high", "low", or "no", but
that's fine in my view. In summary, I say we define a 3 word
vocabulary in which to express an answer, but we don't tell
implementors when they have to give which answer.
> Another question I have... how does this translate to services *other*
> than the ILS, like an ERMS?
The beauty of my proposal is that we also wouldn't have to define
this, because the same human-centered criteria could apply.
- Godmar
Ross Singer
actionable, part, clients really only need to focus on that. We could
also define an optional enumerated message to be passed with it that
conforms to some vocabulary. But, I note, this is optional, however,
it would be useful in helping the client make smarter decisions about
what to do, say, in the case of "low availability".
What do you think?
-Ross.
Godmar Back
message - a string that is intended for human consumption may suffice.
(I'm saying this having seen how many different expressions are
locally used in ILS - trying to categorize those might be
prohibitive.)
Also, "high" and "low" were just some terms I made up because I felt
that "complex" was ... to vague, and in order to reinforce my point
that even "low" availability should only be signaled if there's reason
to believe a human could be helped; but I don't know if those terms
are good or if better ones should be found - may "strong/weak"?
"definite/likely"? I'm not sure.
- Godmar
David Fiander
- If you walk in looking for it, you will be able to walk out with it
- We have it, but you might have to wait/special request it/use it in library
- What you talkin' 'bout Willis?
Looking at Open WorldCat as an example, the simple list of "libraries
nearby that claim to own it", is good enough for basic exploration.
If a user really cares after the initial filter indicates that U.
Podunk holds it somehow, then she will check the Podunk catalogue
directly.
At worst, if the protocol is deemed to be inadequate in some way after
it has been implemented in the field and people are building apps,
then we'll at least have some experience to guide future development.
- David
Richard Wallis
- No - The developer can use this to tell his user they can not have the item
- Yes - The developer can use this to move on to logic to request the item
- Maybe - The developer can move on to logic that would involve optional negotiation will other API calls to request the item in some way
Technology Evangelist, Talis
Tel: +44 (0)870 400 5422 (Direct)
Tel: +44 (0)870 400 5000 (Switchboard)
Tel: +44 (0)7767 886 005 (Mobile)
Fax: +44 (0)870 400 5001
Skype: richard.wallis1
IM: rjw...@hotmail.com
I-Name: =Richard.Wallis
Godmar Back
knowledge of the intrinsics of the business underlying the
application, but I'm not sure why when defining the decision criteria
you prefer to focus on a developer's understanding rather than the
understanding an (end) user would bring to the table.
Isn't "get availability" ultimately a communication between an ILS and
a human being at the other end, so any middleware in between should be
designed to reflect (and to some degree, preserve) this fact?
- Godmar
> library?...
>
> [Message clipped]
Eby
availability on many ILS's to take into account complex policies. I do
like the idea of maybe. I'm wondering if the initial negotiation
should have something simple like "member" to represent a card holder
to a library or a member library to a consortia. I think this would
cover a good chunk of generic lending policies. A return of maybe
could lead into the black hole of policies that segment their card
holders into arbitrary groups with however the client wants to support
it I suppose. I wonder if maybe should contain the items that can be
read in the library but not checked out.
Earlier I was thinking available/requestable. If it's available but
not requestable you should be able to view it in the library (maybe
call). If it is requestable but not available then you can put
yourself on the waiting list, if it's both then hell yes.
I'd be interested in seeing how talis handles the availability of
consortia items (if they do already) besides the generic "we hold
this" for requests, etc. Does the public display differ from what
someone with a valid member library card sees? Most displays just
presume I'm part of "the system".
So I guess what I'm asking is should the client know more or should it
just pass something through and let the system decide for it? I don't
foresee my consortia system doing anything with info I pass so I'd
likely have to do something on the client.
Eby
Richard Wallis
But the user does not interact directly with the API. The user interacts with an application that interacts with the API on their behalf.
Much of what we are discussing is what will go in to documentation, that any implementation of the API will have to conform to.
It is that documentation that the application developer will have to grapple with, and be able to understand with probably little or no understanding of the [library] system behind it.
Hence the 'Joe' test.
In defining an API we have two constituencies to satisfy - the developers of applications that will consume the API and the users of those applications. In both cases we can assume nothing about their knowledge and understanding of t
E system behind the API.
~Richard
Linkedin: http://www.linkedin.com/in/richardwallis
Skype: richard.wallis1
IM: rjw...@hotmail.com
- Godmar
> In the case of 'maybe', supplementary information would be useful for the
> developer to pass on to her user. So a status message would be valuable -
> although this could introduce issues around the language of the message and
> hence its usefulness to the eventual end user.
>
> I'm not sure how complex enumeration of states that are only understandable
> to a library person would pass the 'Joe' test. A simple set of possible
> 'Maybe' states could be defined in a way that a developer could reliably
> code against - I can only think of two at the moment. 1)You can have the
> item, after a delay currently estimated to end on a specific date. 2)You
> can apply to the library to request the item by some form of special
> processing.
>
> ~ Richard.
>
>
> On 10 Apr 2008, at 20:57, David Fiander wrote:
>
Godmar Back
should - try to cover every possible case. In particular, the API
description for the 'get availability' service likely won't cover how
to request an item or how to negotiate with other services on how to
request an item; or do people feel it should? In my view, such ideas
would be left to a different service. For now, I'd be content if "get
availability" simply reported on the availability of an item.
Could you agree to separate these two issues? If so, could you
formulate what the "Joe" test then would mean for "get availability"?
On a related note, could we also make progress on the front where
we're discussing how to handle multiple items? I had sent an email
with questions about it.
- Godmar
Emily Lynema
I have no idea if this is the email you were referring to in your note to Richard about handling multiple items. But maybe it is, so I guess I'll respond here. We are assuming GetAvailability is a known item lookup for DLF purposes. We have other functions available for searching. In particular, we have assumed the API user would already have a trusted unique identifier already to work with when calling GetAvailability - an internal bibliographic identifier or an item barcode. I think that when we were creating the DLF proposal, we did not envision that the GetAvailability function would return any more complicated logic about availability than the Search function, so it had a very specific lookup purpose to return a simplified, focused set of data about a particular item. Obviously, the scope of GetAvailability is starting to expand via these discussions, and I begin to be concerned that the vendors who pledged support earlier will not accept this expanded scope.
For the response, we certainly envision that each element in the response contains the identifier of the item it describes. It should also contain the identifier for the bibliographic record to which it is attached. It's unclear to me whether the copy number should be included in the response to indicate that there are multiple copies. I'm still talking with our cataloging folks to determine how identifiers work across multiple items.
-emily
Godmar Back
>
> For the response, we certainly envision that each element in the response
> contains the identifier of the item it describes. It should also contain the
> identifier for the bibliographic record to which it is attached. It's
> unclear to me whether the copy number should be included in the response to
> indicate that there are multiple copies. I'm still talking with our
> cataloging folks to determine how identifiers work across multiple items.
>
Thanks, Emily.
The story is getting more mysterious here --- why do we believe we can
start by defining "get availability" without having solved the problem
of how to get an internal identifier? After all, most ILS currently
provide neither a way to search them nor a way to ask for
availability.
To clarify regarding how to handle multiple copies: I'm not suggesting
to include the "copy number" (as in "number of copies") in the
response. I'm suggesting the server send one response per copy.
- Godmar
Richard Wallis
- Yes [you can have one, or more, if you had a mechanism to request it]
- No [there is not one available]
- Maybe [There maybe one available, with text message and projected availability date (both optional) attached]
Technology Evangelist, Talis
Tel: +44 (0)870 400 5422 (Direct)
Tel: +44 (0)870 400 5000 (Switchboard)
Tel: +44 (0)7767 886 005 (Mobile)
Fax: +44 (0)870 400 5001
Skype: richard.wallis1
IM: rjw...@hotmail.com
Ross Singer
Jangle and ILS-DI. I suppose it doesn't help that ILS-DI is slightly
more abstract than Jangle, at present (although I realize the irony of
saying that about a spec that has absolutely zero documentation
written or agreed upon, yet), since Jangle, even in its nascent state,
has a more concrete way of referring to resources it exposes.
So, Godmar, in your question, while I certainly don't have any sort of
"solution", seems perfectly doable in a Jangle environment (the
question is more REST semantics).
I was talking about this in the #code4lib channel to Galen and Emily
yesterday. My vision of Jangle (all visions expressed are my own and
should not be considered "reality") would be that every resource has a
URI. So we may have to contrive the REST semantics a bit, but for a
multiple item request, you could have something like:
POST http://jangle.example.org/availability/
and the post vars would be something like:
http://jangle.example.org/koha/item/{id1}
http://jangle.example.org/koha/item/{id2}
http://jangle.example.org/koha/resourcethatdesfinesabibrecord/{id3}
http://jangle.example.org/dspace/resourcethatdefinesabibrecord/id{4}
Since the response would (tentatively) be Atom, you'd then have the
resource ID (URI) returned with the availability information. Nothing
would need to be "searched" (I figure the "searching" would have
happened prior to this request, where is where the URIs come from).
Does this address your question?
-Ross.
Emily Lynema
My viewpoint on this discussion is from the context of the full DLF proposal. We define a Search function for which you can enter any type of search and get back record identifiers. We also define harvesting methods where you can harvest records from the ILS inclusive of record identifiers. The 'Basic Discovery Interfaces' level of the profile, which includes the GetAvailability function, is really designed primarily to support discovery applications that are harvesting information from the ILS to create external search tools. That is a current area of high activity, and one of the primary reasons this task force was called together, I think.
So, will this GetAvailability function necessarily be useful for other applications that need to do real time search? Probably less. They really need the Search function that is part of the level 2 'Elementary OPAC supplement' profile.
We have an updated draft version that highlights these various levels of implementation of the proposal (partially based on feedback at the vendor meeting) up on the wiki: https://project.library.upenn.edu/confluence/display/ilsapi/Classes+of+functions
-emily
Mike Rylander
> actionable, part, clients really only need to focus on that. We could
> also define an optional enumerated message to be passed with it that
> conforms to some vocabulary. But, I note, this is optional, however,
> it would be useful in helping the client make smarter decisions about
> what to do, say, in the case of "low availability".
>
> What do you think?
> -Ross.
>
>
/I/ think I'm lurking for too long ... :)
As others have already pointed out, ternary logic makes this easy to
implement, and actually maps perfectly (if we want to talk about
possible implementations) to a nullable boolean field -- IsAvailable?
True, False, NULL (in the SQL sense -- unknown). This is a very good
sign of viability in my opinion.
The way I think about this problem -- remember, this is a problem that
every single ILS (and similar system) has addressed already, so they
(we, Ross? ;) ) /do/ know how to map this stuff; don't believe claims
to the contrary -- is there are two facets to "availability" (well,
three if we consider the runtime test of "can a specific user have X
right now" -- but I consider that another issue entirely, and will
address it further on):
* State -- what Godmar is calling "availability level." This is the
machine (and, as a first-order approximation, human) readable decision
about whether, with all else being equal, a hypothetical, qualified
user could "get" the resource in question.
* Status -- the semantic code that the backend system uses to make
specific decisions about workflow and process on an
item/resource/thing basis.
State is /always/ important to both the user and the machine. Status
is important to software with special specific knowledge of the
semantics surrounding the value, and on occasion to users who need
specialized information about the resource.
Now, there's a third thing that has come up in this discussion, and
that's the per-user "availability." What this really is, speaking in
practical terms as someone who has implemented this logic in an ILS,
is not a test of the item as much as it is a test of the patron/user
and the environment in which the item lives. I call this
"permissibility" -- I guess one could call it context-dependent
availability.
I propose that this third facet, permissibility, be treated in the
jangle case certainly, and maybe in the DLF case as well, as a
separate call. Something like getAvailabilityForUser, perhaps, which
does the whole "can X have Y at this time" dance and returns a
separate structure indicating if/when/how a user may acquire a
resource. IOW, reserve (heh) the front-line availability function in
jangle (whether it maps directly to the DFL proposal or not) for just
yes/no/unknown, and have a second call which retrieves user-specific
context-based information.
*UPDATE*
Reading further in the thread while penning this, I see that Emily
Lynema has suggested that the "availability level" (what I think of as
State, not Status) should be part of the level 2 Elementary OPAC
supplement profile. I can understand that, and were these profiles
being worked on in isolation, I would tend to agree. However, I'd
submit that if we split availability from permissibility now it will
ease implementation later. This split is how Evergreen handles the
problem, and to the best of my knowledge, it's how this is handled in
other systems as well. Practicality would demand that we follow
precedent for the general structure of the API in order to increase
adoption and implementation ... IMO, anyway.
--miker
--
Mike Rylander
| VP, Research and Design
| Equinox Software, Inc. / The Evergreen Experts
| phone: 1-877-OPEN-ILS (673-6457)
| email: mi...@esilibrary.com
| web: http://www.esilibrary.com
John Mark Ockerbloom
I'm sorry for not including jangle-discuss in my last reply. I
tried doing a cross-post the previous time, and the jangle-discuss
portion bounced (since I wasn't yet signed up for that list),
so I didn't try including it in my more recent reply. I'm now signed
up to jangle-discuss, and am happy to talk about issues there
as well as here.
Folks who are only signed up for
one list or the other might find it difficult to follow or
participate in extended crossposted discussions. Nonetheless, I'll
try to remember to reply to both places in conversations that
have to do with the interaction of the ILS-DI work and the
Jangle work.
(And if anyone wants to sign up for the ILS-DI Google Group, we're
happy to have you appoint yourself to that discussion group too :-)
I agree with you, Godmar, that implementors (both inside and outside
the commercial vendor community) are important, as are users.
(Indeed, I hope that there will be some simple test implementations
of our recommendations soon, possibly very soon for the "Level 1"
requirements, and am one of the people working on making that happen.)
We are also trying, where possible, to use standards that allow
varying levels of sophistication, and further development where
appropriate. I'm aware that many folks would like to move towards
more sophisticated structured responses in availability queries.
I'm also aware that the "yes/no/neither..." response is the most
important aspect of some important use cases, and it's one that shouldn't
be too hard for folks to understand and implement on just about any
ILS in active development.
That's one reason we're looking at an XML schema for availability
replies. XML is one example of a cleanly extensible standard:
it allows you to define certain elements with definite
structure, and other elements that have open-ended structure.
The OAI-PMH specification, for example, has certain elements whose
structure is fully specified, and other elements, including
record and "about" structures, that can use any schema.
The OAI-PMH community has then gone on and defined particular
recommended schemas (like the "marc21" XML schema)
for some of these elements.
Similarly, it's possible for the GetAvailability spec to define
a small number of standardized elements (such as one that gives
the "yes/no/neither..." response, and one giving an unstructured
human-readable message with more details). For folks who
want to specify further structured details beyond those
specified (whatever they are) a "details" element could be defined
with no pre-declared structure. That suffices for a simple
but expandable definition suitable for the official DLF ILS-DI
1.0 release. And it may well be that other communities (maybe
Jangle, maybe someone else) settle on useful recommended standards
for the details element, and that becomes an expected part of
replies in the future. (Or perhaps the community will develop
and recommend additional functions in the future, such as the
user-sensitive "permissibility" function that Mike Rylander suggests.)
I think we're going to continue to see lots of new ideas and innovation
in library discovery. To support ever-richer discovery applications,
the development of interoperability standards is going to have to
be a living process, one involving folks representing ILS's,
discovery apps, and their users. What we want to do in the ILS-DI
group is to provide enough guidance to help folks get started quickly,
including existing ILS providers, and encourage folks to implement,
refine, build on, and extend the initial recommendations.
We probably won't please everyone fully in what we do-- there are just
too many different stakeholders for that-- but we're very interested
in listening to voices from many places, and in learning from folks
with more expertise and experience, and can point out mistakes or oversights
that we make. I hope you'll continue to participate in conversations
and critiques with us.
Thanks,
John
Ross Singer
(which, I might add, almost made me want to walk away from this entire
process), I'd like to explain why I object to using OpenURL for this.
1) There's no response format, so this would need to be created. This
would need to be created for any solution chosen, but OpenURL carries
this baggage PLUS the complexity of OpenURL, the lack of ubiquitous
programming language support, the lack of understanding inside and
outside the library community.
2) Your "simplified" request actually is glossing this process over a
bit. The actual request would look like:
http://(host):(port)/(start)?ctx_ver=Z39.88-2004&rft_id=(id)
which is fine, but why bother? Especially since... what will (id) be
in this scenario? A bib number? A barcode? How will you know? If
this ContextObject can only be used against this very specific
resolver and expect anything rational, why would is it worth the
effort of requiring every service to support OpenURL? What would you
do if the spec eventually called for changing status on an item?
I'm totally opposed just using a standard "because we've got it". You
mentioned earlier today, off-list, why not use OAI-PMH? I would back
*that* solution WAY more than OpenURL, if only because the DLF
recommendation is calling for that for harvesting records *anyway*.
There's still the editing problem, although maybe OAI-ORE can address
that?
To me, AtomPub (and, yes, I'm going to continue banging this drum)
makes the most sense. There's CRUD "built in", widespread language
support and the means to extend the response to include whatever
metadata is needed.
In the end, I want to see this API happen. I think falling back on
standards to achieve this is smart and responsible. I think navel
gazing at library standards at the expense of simpler and more
universally recognized solutions (for zero gain) is misguided.
-Ross.
On Fri, Apr 11, 2008 at 5:37 PM, Robert Sanderson <azar...@gmail.com> wrote:
>
> On Fri, Apr 11, 2008 at 6:08 PM, John Mark Ockerbloom
> <ocke...@pobox.upenn.edu> wrote:
> >
> > We are also trying, where possible, to use standards that allow
> > varying levels of sophistication, and further development where
> > appropriate. I'm aware that many folks would like to move towards
> > more sophisticated structured responses in availability queries.
> > I'm also aware that the "yes/no/neither..." response is the most
> > important aspect of some important use cases, and it's one that shouldn't
> > be too hard for folks to understand and implement on just about any
> > ILS in active development.
> >
>
>
> But not OpenURL? For known item search, (eg Availability and Linking)
> OpenURL seems to be the perfect solution. Just define a simple response
> format, hopefully re-using elements from existing schemas. Yes to the
> uninitiated, OpenURL is confusing. That's okay, just profile it:
>
> In order to get the availability of an item you dereference a URI with the
> following template:
> http://(host):(port)/(start)?(openurlstuff)&rft_id=(id)
>
> Where openurlstuff is static. Then when people complain that they also want
> some other piece of metadata about or some other service on the item (in
> FRBR speak) you have the extra functionality that OpenURL makes available.
>
>
> Now I'm waiting with baited breath for the discussion about search and how
> SRU is too heavyweight or too complicated ;)
>
> Rob Sanderson
>
> >
>
Godmar Back
[ I have to admit it also wasn't clear to me what the term "discovery
layer" meant - but now I see it's referring to the "lipstick on the
pig" type front-end systems such as Primo, Endeca, etc. In those
cases, you've pulled the data from the OPAC already, indexed it
yourself, know what items exist, have an internal record identifier
and all you really need is the current status of the item. It makes
sense that you're not searching the catalog. ]
- Godmar