So, HarvestExpandedRecords is next. This is doable but I need a little clarity about what is expected from this versus HarvestHoldingsRecords. Is it just that expanded records are holdings, returned in simpleavailability with the bib id (while HarvestHoldings would return in MFHD or something)?
Anyway, this was *super* simple to do in Jangle. It has taken less than three days to get this functionality from Jangle, with a good percentage of that time being used to fix bugs it uncovered in Jangle :)
Yeah, there is some confusion about HarvestExpandedRecords versus HarvestHoldingsRecords. The original intent behind HarvestHoldingsRecords surrounded MFHD. HarvestExpandedRecords, in our vision, might include both MFHD and simpleAvailability / NCIP / ISO holdings. It is *everything* that exists about a record, including bibliographic marc, holdings marc, and related item information. We came close to getting rid of the HarvestHoldingsRecords, but it sounded like there was a use case related to electronic resource management. I think the folks at National Library of Medicine (which uses machine-readable MFHD, I think) export their holdings records. So HarvestHoldingsRecords is really about the concept that an institution might have holdings records (which would generally be stored in MFHD) aside from bibliographic records and items.
But that doesn't mean that HarvestExpandedRecords should return only simpleAvailability! :) I think the idea was to create a way for folks to return the data they had, without having to convert MFHD into another format or item information into something that looks like MFHD.
> I wanted to mention here that Jangle now supports Level 1: Basic > Discovery Interfaces (outside HarvestExpandedRecords ... more on that > in a minute).
> Here are the URLs to the services (modeled after the Online Books Page example):
> Also, Level 1 support of the DLF ILS API can be found here:
> So, HarvestExpandedRecords is next. This is doable but I need a > little clarity about what is expected from this versus > HarvestHoldingsRecords. Is it just that expanded records are > holdings, returned in simpleavailability with the bib id (while > HarvestHoldings would return in MFHD or something)?
> Anyway, this was *super* simple to do in Jangle. It has taken less > than three days to get this functionality from Jangle, with a good > percentage of that time being used to fix bugs it uncovered in Jangle > :)
> -Ross.
-- Emily Lynema Systems Librarian for Digital Projects Information Technology, NCSU Libraries 919-513-8031 emily_lyn...@ncsu.edu
Ok, I think I see. So the base 'record' is still the bib record, right (i.e. the record count from OAI-PMH should be the same as if you were harvesting bib records, as opposed to holdings records)?
Does ExpandedRecords schema account for these different metadata formats? Since it divvies things up into Bib, Holdings and Items within the record, where would something like ISO holdings go (since it covers all these enities)?
From a Jangle perspective the ExpandedRecord (by this definition) gets a little expensive (which is a Jangle problem, not an ILS-DI problem), what is the expected use case for pulling everything like this at once?
So is it possible for HarvestHoldings to return ISO 20775 or is it really just intended for MFHD?
One last question :) How do systems that don't use MFHD and don't have ISO holdings share their holdings via HarvestExpandedRecords? (The answer to me is that ISO Holdings aren't that hard, but I'm just throwing out the question).
On Thu, Jul 3, 2008 at 3:56 PM, Emily Lynema <emily_lyn...@ncsu.edu> wrote:
> Ross,
> Yeah, there is some confusion about HarvestExpandedRecords versus > HarvestHoldingsRecords. The original intent behind > HarvestHoldingsRecords surrounded MFHD. HarvestExpandedRecords, in our > vision, might include both MFHD and simpleAvailability / NCIP / ISO > holdings. It is *everything* that exists about a record, including > bibliographic marc, holdings marc, and related item information. We came > close to getting rid of the HarvestHoldingsRecords, but it sounded like > there was a use case related to electronic resource management. I think > the folks at National Library of Medicine (which uses machine-readable > MFHD, I think) export their holdings records. So HarvestHoldingsRecords > is really about the concept that an institution might have holdings > records (which would generally be stored in MFHD) aside from > bibliographic records and items.
> But that doesn't mean that HarvestExpandedRecords should return only > simpleAvailability! :) I think the idea was to create a way for folks to > return the data they had, without having to convert MFHD into another > format or item information into something that looks like MFHD.
> Is that clear as mud?
> -emily
> Ross Singer wrote: >> Hi everybody.
>> I wanted to mention here that Jangle now supports Level 1: Basic >> Discovery Interfaces (outside HarvestExpandedRecords ... more on that >> in a minute).
>> Here are the URLs to the services (modeled after the Online Books Page example):
>> Also, Level 1 support of the DLF ILS API can be found here:
>> So, HarvestExpandedRecords is next. This is doable but I need a >> little clarity about what is expected from this versus >> HarvestHoldingsRecords. Is it just that expanded records are >> holdings, returned in simpleavailability with the bib id (while >> HarvestHoldings would return in MFHD or something)?
>> Anyway, this was *super* simple to do in Jangle. It has taken less >> than three days to get this functionality from Jangle, with a good >> percentage of that time being used to fix bugs it uncovered in Jangle >> :)
>> -Ross.
> -- > Emily Lynema > Systems Librarian for Digital Projects > Information Technology, NCSU Libraries > 919-513-8031 > emily_lyn...@ncsu.edu
I don't understand why you don't always return an expanded record as defined using the recordType in Peter's definition [1]--- specifically, for HarvestBibliographicRecords, HarvestExpandedRecords, GetRecords, and Search. Since this definition allows for the embedding of any record type (MarcXML, DC, etc.), and since this definition includes a place to put a record identifier, and since it allows the omission of information (holdings, items, availability) that isn't requested in a particular operation, it provides a simple and general solution that should drastically simplify the creation of clients for this API.
In addition, if a server implemented only HarvestExpandedRecords - for instance -- that server would automatically provide compliant implementations of HarvestBibliographicRecords and HarvestHoldingsRecords.
On Thu, Jul 3, 2008 at 5:51 PM, Ross Singer <rossfsin...@gmail.com> wrote:
> Ok, I think I see. So the base 'record' is still the bib record, > right (i.e. the record count from OAI-PMH should be the same as if you > were harvesting bib records, as opposed to holdings records)?
> Does ExpandedRecords schema account for these different metadata > formats? Since it divvies things up into Bib, Holdings and Items > within the record, where would something like ISO holdings go (since > it covers all these enities)?
> From a Jangle perspective the ExpandedRecord (by this definition) gets > a little expensive (which is a Jangle problem, not an ILS-DI problem), > what is the expected use case for pulling everything like this at > once?
> So is it possible for HarvestHoldings to return ISO 20775 or is it > really just intended for MFHD?
> One last question :) How do systems that don't use MFHD and don't > have ISO holdings share their holdings via HarvestExpandedRecords? > (The answer to me is that ISO Holdings aren't that hard, but I'm just > throwing out the question).
> Thanks! > -Ross.
> On Thu, Jul 3, 2008 at 3:56 PM, Emily Lynema <emily_lyn...@ncsu.edu> wrote:
>> Ross,
>> Yeah, there is some confusion about HarvestExpandedRecords versus >> HarvestHoldingsRecords. The original intent behind >> HarvestHoldingsRecords surrounded MFHD. HarvestExpandedRecords, in our >> vision, might include both MFHD and simpleAvailability / NCIP / ISO >> holdings. It is *everything* that exists about a record, including >> bibliographic marc, holdings marc, and related item information. We came >> close to getting rid of the HarvestHoldingsRecords, but it sounded like >> there was a use case related to electronic resource management. I think >> the folks at National Library of Medicine (which uses machine-readable >> MFHD, I think) export their holdings records. So HarvestHoldingsRecords >> is really about the concept that an institution might have holdings >> records (which would generally be stored in MFHD) aside from >> bibliographic records and items.
>> But that doesn't mean that HarvestExpandedRecords should return only >> simpleAvailability! :) I think the idea was to create a way for folks to >> return the data they had, without having to convert MFHD into another >> format or item information into something that looks like MFHD.
>> Is that clear as mud?
>> -emily
>> Ross Singer wrote: >>> Hi everybody.
>>> I wanted to mention here that Jangle now supports Level 1: Basic >>> Discovery Interfaces (outside HarvestExpandedRecords ... more on that >>> in a minute).
>>> Here are the URLs to the services (modeled after the Online Books Page example):
>>> Also, Level 1 support of the DLF ILS API can be found here:
>>> So, HarvestExpandedRecords is next. This is doable but I need a >>> little clarity about what is expected from this versus >>> HarvestHoldingsRecords. Is it just that expanded records are >>> holdings, returned in simpleavailability with the bib id (while >>> HarvestHoldings would return in MFHD or something)?
>>> Anyway, this was *super* simple to do in Jangle. It has taken less >>> than three days to get this functionality from Jangle, with a good >>> percentage of that time being used to fix bugs it uncovered in Jangle >>> :)
>>> -Ross.
>> -- >> Emily Lynema >> Systems Librarian for Digital Projects >> Information Technology, NCSU Libraries >> 919-513-8031 >> emily_lyn...@ncsu.edu
Godmar Back wrote: > In addition, if a server implemented only HarvestExpandedRecords - for > instance -- that server would automatically provide compliant > implementations of HarvestBibliographicRecords and > HarvestHoldingsRecords.
Well, this would comply with the letter of the recommendation, but you need to consider the possible impact (including performance and complexity) on both the ILS and the client if you go this route.
HarvestExpandedRecords can be a considerably more expensive operation for the ILS (and possibly for the client as well) than just returning the bibliographic records. Consider, for instance, incremental harvesting-- there are potentially lots more things that can change in an expanded record, so you might end up returning a lot more records than you would if you only returned updated bibliographic records.
That's one reason we recommended separate functions for bibliographic records alone and for expanded records. While we didn't go so far as to say you *can't* in effect combine the two, it might well be excessively burdensome for server or client.
the paragraph to which you reply was secondary to my argument - let's set it aside for a minute.
Let's assume the normal case whereby HarvestExpandedRecords, HarvestHoldingsRecords, and HarvestBibliographicRecords are operations of different complexity, and whereby the return different sets of information.
What I'm proposing is to use your defined schema to describe the response elements for each of these operations. So, for instance, for HarvestBibliographicRecords you'd be returning a
> Godmar Back wrote: >> In addition, if a server implemented only HarvestExpandedRecords - for >> instance -- that server would automatically provide compliant >> implementations of HarvestBibliographicRecords and >> HarvestHoldingsRecords.
> Well, this would comply with the letter of the recommendation, but > you need to consider the possible impact (including performance and > complexity) on both the ILS and the client if you go this route.
> HarvestExpandedRecords can be a considerably more expensive operation for > the ILS (and possibly for the client as well) than just returning the > bibliographic records. Consider, for instance, incremental harvesting-- > there are potentially lots more things that can change in an expanded > record, so you might end up returning a lot more records than you would > if you only returned updated bibliographic records.
> That's one reason we recommended separate functions for bibliographic > records alone and for expanded records. While we didn't go so far as to > say you *can't* in effect combine the two, it might well be excessively > burdensome for server or client.
Godmar Back wrote: > What I'm proposing is to use your defined schema to describe the > response elements for each of these operations. So, for instance, for > HarvestBibliographicRecords you'd be returning a
(and a similar structure for HarvestHoldingsRecords, GetRecords, etc., except with different namespace declarations for the stuff inside.)
Yes, that structure is designed to be able to hold a lot of the responses we ask for, as we describe in section 6.4.2 "Putting it all together" in our report. To date, I've only implemented it myself in Level 1 functions, but it should be usable in many of the higher level functions as well. (I confess I haven't yet had the time to look over your implementation, but I hope to when I get the chance.)
> Yes, that structure is designed to be able to hold a lot of the responses > we ask for, as we describe in section 6.4.2 "Putting it all together" in our
6.4.2 is referring explicitly only to GetRecords, Search, HarvestExpandedRecords, and GetAvailability. My suggestion is to expand it to all responses, including HarvestBibliographicRecords etc.
> report. To date, I've only implemented it myself in Level 1 functions, > but it should be usable in many of the higher level functions as well. > (I confess I haven't yet had the time to look over your implementation, > but I hope to when I get the chance.)
If find this problematic, and am proposing that URL [1] should also return DLF records with embedded MODS records (at least as far as the ILS-DI specification is concerned - you can of course provide a service that just returns MODS records also.)
In the context of the ILS-DI specification, I find it problematic because this format does not contain the bibliographic identifier (note that your /record/header/identifier in your MODS response differs from the corresponding /dlf:record/dlf:bibliographic@dlf:id, which means that there's an implicit assumption that the "sampleIdentifier" in the OAI identifier is the bibliographic identifier (?) - this requires a client to know that "in a OAI/mods response, I must extract the ILS-DI bibliographic identifier by transforming the /record/header/identifier in some way, rather than simply being able to pass the /record/metadata/1 child to some generic ILS-DI processing code.)
In essence, what I'm advocating is that an interface that carries the label "ILS-DI compliant" return its metadata records *always* inside <dlf:record> containers - no matter which ILS-DI primitive is being used, such that a) bibliographic identifiers can be obtained in a manner that's independent of the container/collection format used (OAI, SRU, Atom) and such that b) client code does not require any up-front knowledge about which format may be used in a particular response.
Finally, doing so would allow clients to use the metadataPrefix= to request a particular record type for the record that's embedded in the dlf:record. So metadataPrefix=mods on an ILS-DI OAI would return MODS records embedded in DLF:records, but metadataPrefix=marc would return MARC records in DLF:records.
