Islandora OAI-PMH (in Drupal 7)

[Rajesh PS]

Dear all

I have a problem with islandora oai module. in my repository over 20000 records are there, but while exposing the repository using oai module its only showing 20 records. I am trying to harvest these records to vufind installed in the same server. When i tried to harvest records from islandora to vufind i get only 20 records batch and duplicate records are coming in the next batches. that is if i harvest 100 records only 20 unique records are harvesting. I think there is a bug in the islandora oai module file (islandora_oai.module) while issuing resumption_token. any help in this is highly appreciated. 

[Jordan Dukart]

Sorry for the slow response, I'll have to have a look at this when I get
a chance to see if I can replicate/rectify this.

Jordan

> --
> You received this message because you are subscribed to the Google
> Groups "islandora" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to islandora+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[Melissa Anez]

There's also a ticket in JIRA for this issue:  https://jira.duraspace.org/browse/ISLANDORA-907

- Melissa

[Jordan Dukart]

This has been fixed as of https://github.com/Islandora/islandora_oai/pull/16 which has been merged, note you will need to clear your cache and run update.php.

Jordan

[Jordan Dukart]

Was a couple of bugs found by Donald Moses that have been fixed and merged into core. I'd suggest updating the islandora_oai module.

Jordan

[Elin]

Hi Jordan,

We just found this problem (we can only harvest 20 records).  We are currently running Islandora-7.x-1.2 and islandora_oai-7.x-1.2 module ( this version seems haven't been updated). Can we update the latest version islandora_oai-7.x, islandora_oai-7.x-1.3 or islandora_oai-7.x-1.4 instead?

Many thanks.

Kind regards,

Elin

[Jordan Dukart]

It's not recommended to mix and match versions of modules within Islandora. I'd suggest either updating all your modules to HEAD, the former 7.x-1.3 release or wait for the proper 7.x-1.4 release.

Jordan

--
For more information about using this group, please read our Listserv Guidelines: http://islandora.ca/content/welcome-islandora-listserv
---

You received this message because you are subscribed to the Google Groups "islandora" group.
To unsubscribe from this group and stop receiving emails from it, send an email to islandora+...@googlegroups.com.

Visit this group at http://groups.google.com/group/islandora.
For more options, visit https://groups.google.com/d/optout.

[HS]

Thanks for your reply. We can not simply update from our current Islandora 7.x-1.2 to 7.x-1.3 because we did a lot changing in the source code. Is there any way we can do to the OAI 7.x-1.2 module to fix the problem of only generating 20 records?

Many thanks.
Elin

You received this message because you are subscribed to a topic in the Google Groups "islandora" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/islandora/3QWG9K7ywus/unsubscribe.
To unsubscribe from this group and all its topics, send an email to islandora+...@googlegroups.com.

[Chad Hutchens]

Elin,

Have you looked at your Islandora OAI module configuration settings?  I think the default max response size is 20 records.  It can be changed.  Otherwise, to get the remaining sets, you can use the resumption token. 

I don't know what increasing the response size will do with any server performance issues...I wouldn't go overboard with it.

Chad Hutchens

[Jordan Dukart]

I believe the behavior being experienced in this thread refers to this JIRA ticket: https://jira.duraspace.org/browse/ISLANDORA-907 which has the pull request/commit in it that fixed it. That being said, I don't know the specifics of the environment and what you may or may have not changed to offer an easy answer on how to upgrade your OAI. The pull that addressed this issue is over a year old now (https://github.com/Islandora/islandora_oai/pull/16/files), and there have been other bug fixes and the like to the OAI module since then.

Jordan

[HS]

Thanks. Yes I can see it is fixed in islandora oai-7.x-1.3, but not oai-7.x-1.2(release) which we are currently using. You said it is not recommended to mix and match version. So do you have any way to do this version 1.2?

Many thanks.
Elin

[Nick Ruest]

Hi Elin-

You will need to upgrade if you want to take advantage of the fixes.

-nruest

On 14-10-17 10:17 PM, HS wrote:
> Thanks. Yes I can see it is fixed in islandora oai-7.x-1.3, but not
> oai-7.x-1.2(release) which we are currently using. You said it is not
> recommended to mix and match version. So do you have any way to do this
> version 1.2?
>
> Many thanks.
> Elin
>
> On Oct 18, 2014 2:40 PM, "Jordan Dukart" <jor...@discoverygarden.ca

>>> https://github.com/Islandora/i__slandora_oai/pull/16

>>> <https://github.com/Islandora/islandora_oai/pull/16>
>>> which has been merged, note you will need to
>>> clear your cache and run update.php.
>>>
>>> Jordan
>>>
>>>
>>> On 2013-10-07 12:27 PM, Melissa Anez wrote:
>>>> There's also a ticket in JIRA for this issue:

>>>> https://jira.duraspace.org/br__owse/ISLANDORA-907 <https://jira.duraspace.org/browse/ISLANDORA-907>

>>>> > an email to islandora+...@googlegroups.com__.
>>>> > For more options, visit
>>>> https://groups.google.com/grou__ps/opt_out
>>>> <https://groups.google.com/groups/opt_out>.

>>>>
>>>> --
>>>> You received this message because you are
>>>> subscribed to the Google Groups "islandora" group.
>>>> To unsubscribe from this group and stop
>>>> receiving emails from it, send an email to

>>>> islandora+...@googlegroups.com__.
>>>> For more options, visit
>>>> https://groups.google.com/grou__ps/opt_out
>>>> <https://groups.google.com/groups/opt_out>.

>>>
>>>
>>> --
>>> For more information about using this group, please read
>>> our Listserv Guidelines:

>>> http://islandora.ca/content/__welcome-islandora-listserv

>>> <http://islandora.ca/content/welcome-islandora-listserv>
>>> ---
>>> You received this message because you are subscribed to
>>> the Google Groups "islandora" group.
>>> To unsubscribe from this group and stop receiving emails

>>> from it, send an email to islandora+...@__googlegroups.com.
>>> Visit this group at
>>> http://groups.google.com/__group/islandora
>>> <http://groups.google.com/group/islandora>.
>>> For more options, visit
>>> https://groups.google.com/d/__optout
>>> <https://groups.google.com/d/optout>.

>>
>> --
>> For more information about using this group, please read
>> our Listserv Guidelines:

>> http://islandora.ca/content/__welcome-islandora-listserv

>> <http://islandora.ca/content/welcome-islandora-listserv>
>> ---
>> You received this message because you are subscribed to a
>> topic in the Google Groups "islandora" group.
>> To unsubscribe from this topic, visit

>> https://groups.google.com/d/__topic/islandora/3QWG9K7ywus/__unsubscribe
>> <https://groups.google.com/d/topic/islandora/3QWG9K7ywus/unsubscribe>.

>> To unsubscribe from this group and all its topics, send an

>> email to islandora+...@__googlegroups.com.
>> Visit this group at
>> http://groups.google.com/__group/islandora
>> <http://groups.google.com/group/islandora>.
>> For more options, visit
>> https://groups.google.com/d/__optout
>> <https://groups.google.com/d/optout>.

>>
>> --
>> For more information about using this group, please read our
>> Listserv Guidelines:
>> http://islandora.ca/content/welcome-islandora-listserv
>> ---
>> You received this message because you are subscribed to the Google
>> Groups "islandora" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to islandora+...@googlegroups.com

>> <mailto:islandora+...@googlegroups.com>.

>> Visit this group at http://groups.google.com/group/islandora.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> For more information about using this group, please read our
> Listserv Guidelines:
> http://islandora.ca/content/welcome-islandora-listserv
> ---
> You received this message because you are subscribed to a topic in
> the Google Groups "islandora" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/islandora/3QWG9K7ywus/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> islandora+...@googlegroups.com

> <mailto:islandora+...@googlegroups.com>.

> Visit this group at http://groups.google.com/group/islandora.
> For more options, visit https://groups.google.com/d/optout.
>
> --
> For more information about using this group, please read our Listserv
> Guidelines: http://islandora.ca/content/welcome-islandora-listserv
> ---
> You received this message because you are subscribed to the Google
> Groups "islandora" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to islandora+...@googlegroups.com

> <mailto:islandora+...@googlegroups.com>.

[HS]

That would be a problem for us because our current system is developed based on islandora7.x1.2 and its modules. We did a lot changes in the source code. Everything works fine but the oai problem. It would be better to have some solution for the current version.

Many thanks.
Elin

T

On Oct 18, 2014 3:41 PM, "Nick Ruest" <rue...@gmail.com> wrote:

Hi Elin-

You will need to upgrade if you want to take advantage of the fixes.

-nruest

On 14-10-17 10:17 PM, HS wrote:

Thanks. Yes I can see it is fixed in islandora oai-7.x-1.3, but not
oai-7.x-1.2(release) which we are currently using. You said it is not
recommended to mix and match version. So do you have any way to do this
version 1.2?

Many thanks.
Elin

On Oct 18, 2014 2:40 PM, "Jordan Dukart" <jor...@discoverygarden.ca

    send an email to islandora+unsubscribe@googlegroups.com
    <mailto:islandora+unsubscribe@googlegroups.com>.

    Visit this group at http://groups.google.com/group/islandora.
    For more options, visit https://groups.google.com/d/optout.

    --
    For more information about using this group, please read our
    Listserv Guidelines:
    http://islandora.ca/content/welcome-islandora-listserv
    ---
    You received this message because you are subscribed to a topic in
    the Google Groups "islandora" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/topic/islandora/3QWG9K7ywus/unsubscribe.
    To unsubscribe from this group and all its topics, send an email to

    islandora+unsubscribe@googlegroups.com
    <mailto:islandora+unsubscribe@googlegroups.com>.

    Visit this group at http://groups.google.com/group/islandora.
    For more options, visit https://groups.google.com/d/optout.

--
For more information about using this group, please read our Listserv
Guidelines: http://islandora.ca/content/welcome-islandora-listserv
---
You received this message because you are subscribed to the Google
Groups "islandora" group.
To unsubscribe from this group and stop receiving emails from it, send

an email to islandora+unsubscribe@googlegroups.com
<mailto:islandora+unsubscribe@googlegroups.com>.

Visit this group at http://groups.google.com/group/islandora.
For more options, visit https://groups.google.com/d/optout.

--
For more information about using this group, please read our Listserv Guidelines: http://islandora.ca/content/welcome-islandora-listserv
--- You received this message because you are subscribed to a topic in the Google Groups "islandora" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/islandora/3QWG9K7ywus/unsubscribe.

To unsubscribe from this group and all its topics, send an email to islandora+unsubscribe@googlegroups.com.

[Nick Ruest]

Hi Elin-

Unfortunately, we do not have the capacity as a community to backport
these changes. If you would like to see these changes in your code base,
you would need to backport them yourself so they work with all the
changes you made to your code base.

-nruest

On 14-10-17 11:29 PM, HS wrote:
> That would be a problem for us because our current system is developed
> based on islandora7.x1.2 and its modules. We did a lot changes in the
> source code. Everything works fine but the oai problem. It would be
> better to have some solution for the current version.
>
> Many thanks.
> Elin
>
>
> T
>
> On Oct 18, 2014 3:41 PM, "Nick Ruest" <rue...@gmail.com

> <mailto:rue...@gmail.com>> wrote:
>
> Hi Elin-
>
> You will need to upgrade if you want to take advantage of the fixes.
>
> -nruest
>
> On 14-10-17 10:17 PM, HS wrote:
>
> Thanks. Yes I can see it is fixed in islandora oai-7.x-1.3, but not
> oai-7.x-1.2(release) which we are currently using. You said it
> is not
> recommended to mix and match version. So do you have any way to
> do this
> version 1.2?
>
> Many thanks.
> Elin
>
> On Oct 18, 2014 2:40 PM, "Jordan Dukart"

> <jor...@discoverygarden.ca <mailto:jor...@discoverygarden.ca>
> <mailto:jordan@__discoverygarden.ca

> <mailto:jor...@discoverygarden.ca>>> wrote:
>
> I believe the behavior being experienced in this thread
> refers to
> this JIRA ticket:

> https://jira.duraspace.org/__browse/ISLANDORA-907

> <https://jira.duraspace.org/browse/ISLANDORA-907>
> which has the pull request/commit in it that fixed it. That
> being
> said, I don't know the specifics of the environment and
> what you may
> or may have not changed to offer an easy answer on how to
> upgrade
> your OAI. The pull that addressed this issue is over a year
> old now

> (https://github.com/Islandora/__islandora_oai/pull/16/files
> <https://github.com/Islandora/islandora_oai/pull/16/files>), and

> <mailto:jor...@discoverygarden.ca>> wrote:
>
> This has been fixed as of

> https://github.com/Islandora/__i__slandora_oai/pull/16
> <https://github.com/Islandora/i__slandora_oai/pull/16>
>
> <https://github.com/Islandora/__islandora_oai/pull/16

> <https://github.com/Islandora/islandora_oai/pull/16>>
> which has been merged, note you
> will need to
> clear your cache and run update.php.
>
> Jordan
>
>
> On 2013-10-07 12:27 PM, Melissa
> Anez wrote:
>
> There's also a ticket in JIRA
> for this issue:

> https://jira.duraspace.org/br____owse/ISLANDORA-907
> <https://jira.duraspace.org/br__owse/ISLANDORA-907>
> <https://jira.duraspace.org/__browse/ISLANDORA-907

> islandora+...@googlegroups.__com__.
> > For more options, visit
> https://groups.google.com/__grou__ps/opt_out
> <https://groups.google.com/grou__ps/opt_out>
>
> <https://groups.google.com/__groups/opt_out

> <https://groups.google.com/groups/opt_out>>.
>
> --
> You received this message
> because you are
> subscribed to the Google Groups
> "islandora" group.
> To unsubscribe from this group
> and stop
> receiving emails from it, send
> an email to

> islandora+...@googlegroups.__com__.
> For more options, visit
> https://groups.google.com/__grou__ps/opt_out
> <https://groups.google.com/grou__ps/opt_out>
>
> <https://groups.google.com/__groups/opt_out

> <https://groups.google.com/groups/opt_out>>.
>
>
>
> --
> For more information about using this
> group, please read
> our Listserv Guidelines:

> http://islandora.ca/content/____welcome-islandora-listserv
> <http://islandora.ca/content/__welcome-islandora-listserv>

>
> <http://islandora.ca/content/__welcome-islandora-listserv <http://islandora.ca/content/welcome-islandora-listserv>>
> ---
> You received this message because you are
> subscribed to
> the Google Groups "islandora" group.
> To unsubscribe from this group and stop
> receiving emails
> from it, send an email to

> islandora+...@__googlegroups.__com
> <http://googlegroups.com>.
> Visit this group at
> http://groups.google.com/____group/islandora
> <http://groups.google.com/__group/islandora>

> <http://groups.google.com/__group/islandora
> <http://groups.google.com/group/islandora>>.
> For more options, visit

> https://groups.google.com/d/____optout
> <https://groups.google.com/d/__optout>

> <https://groups.google.com/d/__optout
> <https://groups.google.com/d/optout>>.
>
>
> --
> For more information about using this group,
> please read
> our Listserv Guidelines:

> http://islandora.ca/content/____welcome-islandora-listserv
> <http://islandora.ca/content/__welcome-islandora-listserv>

>
> <http://islandora.ca/content/__welcome-islandora-listserv
> <http://islandora.ca/content/welcome-islandora-listserv>>
> ---
> You received this message because you are
> subscribed to a
> topic in the Google Groups "islandora" group.
> To unsubscribe from this topic, visit

> https://groups.google.com/d/____topic/islandora/3QWG9K7ywus/____unsubscribe
> <https://groups.google.com/d/__topic/islandora/3QWG9K7ywus/__unsubscribe>

>
> <https://groups.google.com/d/__topic/islandora/3QWG9K7ywus/__unsubscribe
> <https://groups.google.com/d/topic/islandora/3QWG9K7ywus/unsubscribe>>.
> To unsubscribe from this group and all its
> topics, send an

> email to islandora+...@__googlegroups.__com
> <http://googlegroups.com>.
> Visit this group at
> http://groups.google.com/____group/islandora
> <http://groups.google.com/__group/islandora>

> <http://groups.google.com/__group/islandora
> <http://groups.google.com/group/islandora>>.
> For more options, visit

> https://groups.google.com/d/____optout
> <https://groups.google.com/d/__optout>

> <https://groups.google.com/d/__optout
> <https://groups.google.com/d/optout>>.
>
> --
> For more information about using this group, please
> read our
> Listserv Guidelines:
> http://islandora.ca/content/__welcome-islandora-listserv
> <http://islandora.ca/content/welcome-islandora-listserv>
> ---
> You received this message because you are subscribed to

> the Google
> Groups "islandora" group.
> To unsubscribe from this group and stop receiving
> emails from it,
> send an email to

> islandora+unsubscribe@__googlegroups.com
> <mailto:islandora%2Bunsu...@googlegroups.com>
> <mailto:islandora+unsubscribe@__googlegroups.com
> <mailto:islandora%2Bunsu...@googlegroups.com>>.
> Visit this group at

> http://groups.google.com/__group/islandora
> <http://groups.google.com/group/islandora>.
> For more options, visit
> https://groups.google.com/d/__optout
> <https://groups.google.com/d/optout>.
>
>
> --
> For more information about using this group, please read our
> Listserv Guidelines:
> http://islandora.ca/content/__welcome-islandora-listserv
> <http://islandora.ca/content/welcome-islandora-listserv>
> ---
> You received this message because you are subscribed to a
> topic in
> the Google Groups "islandora" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/__topic/islandora/3QWG9K7ywus/__unsubscribe
> <https://groups.google.com/d/topic/islandora/3QWG9K7ywus/unsubscribe>.
> To unsubscribe from this group and all its topics, send an
> email to

> islandora+unsubscribe@__googlegroups.com
> <mailto:islandora%2Bunsu...@googlegroups.com>
> <mailto:islandora+unsubscribe@__googlegroups.com
> <mailto:islandora%2Bunsu...@googlegroups.com>>.
> Visit this group at

> http://groups.google.com/__group/islandora
> <http://groups.google.com/group/islandora>.
> For more options, visit
> https://groups.google.com/d/__optout
> <https://groups.google.com/d/optout>.
>
> --
> For more information about using this group, please read our
> Listserv
> Guidelines:
> http://islandora.ca/content/__welcome-islandora-listserv
> <http://islandora.ca/content/welcome-islandora-listserv>
> ---

> You received this message because you are subscribed to the Google

> Groups "islandora" group.
> To unsubscribe from this group and stop receiving emails from
> it, send

> an email to islandora+unsubscribe@__googlegroups.com
> <mailto:islandora%2Bunsu...@googlegroups.com>
> <mailto:islandora+unsubscribe@__googlegroups.com
> <mailto:islandora%2Bunsu...@googlegroups.com>>.
> Visit this group at http://groups.google.com/__group/islandora

> <http://groups.google.com/group/islandora>.
> For more options, visit https://groups.google.com/d/__optout
> <https://groups.google.com/d/optout>.
>
>
> --
> For more information about using this group, please read our
> Listserv Guidelines:
> http://islandora.ca/content/__welcome-islandora-listserv
> <http://islandora.ca/content/welcome-islandora-listserv>
> --- You received this message because you are subscribed to a topic
> in the Google Groups "islandora" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/__topic/islandora/3QWG9K7ywus/__unsubscribe
> <https://groups.google.com/d/topic/islandora/3QWG9K7ywus/unsubscribe>.
> To unsubscribe from this group and all its topics, send an email to

> islandora+unsubscribe@__googlegroups.com
> <mailto:islandora%2Bunsu...@googlegroups.com>.
> Visit this group at http://groups.google.com/__group/islandora

> <http://groups.google.com/group/islandora>.
> For more options, visit https://groups.google.com/d/__optout
> <https://groups.google.com/d/optout>.
>
> --
> For more information about using this group, please read our Listserv
> Guidelines: http://islandora.ca/content/welcome-islandora-listserv
> ---
> You received this message because you are subscribed to the Google
> Groups "islandora" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to islandora+...@googlegroups.com
> <mailto:islandora+...@googlegroups.com>.

[Jordan Dukart]

As Nick was alluding to, you'll have to pull out the commits yourself
and attempt to apply them against your codebase. I linked the pull
request where the issue outlined in this thread got fixed. That being
said, as 7.x-1.2 is over a year old there have been other bug fixes to
OAI in that time. However, simply updating the module itself and not any
of the dependencies could lead you down a rabbit hole.

Good luck,

Jordan