OAI import error

[Evelin Bányai]

Dear All,

We have upgraded our DSpace from 7.2 to 8.0 near the end of last year. We followed the guide on this site https://wiki.lyrasis.org/display/DSDOC8x/Upgrading+DSpace

The upgrade seemed to be successful and so far we have encountered only 2 issues. One of is them running the dspace oai import script fails with an error.

Our Environment information:

DSpace version: 8.0

Java: 17.0.13+11-Ubuntu-2ubuntu124.04

Node: v20.18.1

Tomcat: None. We are using the runnable JAR

See the output running the following command below: dspace oai import

Standard Commons Logging discovery in action with spring-jcl: please remove commons-logging.jar from classpath in order to avoid potential conflicts
OAI 2.0 manager action started
There are no indexed documents, using full import.
Full import
Total: 288 items
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://10.128.120.16:8983/solr/oai: ERROR: [doc=123456789/339] unknown field 'item.hasbitstream'
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:681)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:266)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:225)
        at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:106)
        at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:71)
        at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:85)
        at org.dspace.xoai.app.XOAI.index(XOAI.java:342)
        at org.dspace.xoai.app.XOAI.indexAll(XOAI.java:265)
        at org.dspace.xoai.app.XOAI.index(XOAI.java:166)
        at org.dspace.xoai.app.XOAI.main(XOAI.java:628)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:283)
        at org.dspace.app.launcher.ScriptLauncher.handleScript(ScriptLauncher.java:134)
        at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:99)

I can see that there is an unknown field called "item.hasbitstream'", but I'm not sure where it's missing from exactly. The item in the error is most probably the first on its list. So probably the problem is not with this item only? 

Thanks in advance to anyone who can help us with this issue.

Best regards,

Evelin

[mw...@iu.edu]

On Mon, Jan 13, 2025 at 12:04:26PM +0000, Evelin Bányai wrote:
> We have upgraded our DSpace from 7.2 to 8.0 near the end of last year. We followed the guide on this site https://wiki.lyrasis.org/display/DSDOC8x/Upgrading+DSpace
>
> The upgrade seemed to be successful and so far we have encountered only 2 issues. One of is them running the dspace oai import script fails with an error.
>
> Our Environment information:
> DSpace version: 8.0
> Java: 17.0.13+11-Ubuntu-2ubuntu124.04
> Node: v20.18.1
> Tomcat: None. We are using the runnable JAR
>
> See the output running the following command below: dspace oai import
> Standard Commons Logging discovery in action with spring-jcl: please remove commons-logging.jar from classpath in order to avoid potential conflicts
> OAI 2.0 manager action started
> There are no indexed documents, using full import.
> Full import
> Total: 288 items
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://10.128.120.16:8983/solr/oai: ERROR: [doc=123456789/339] unknown field 'item.hasbitstream'
>

> I can see that there is an unknown field called "item.hasbitstream'", but I'm not sure where it's missing from exactly. The item in the error is most probably the first on its list. So probably the problem is not with this item only?

It seems that the field is not defined in the Solr schema for the
'oai' core. That field was added on 2023-12-12 to support RIOXX. The
first thing I would check is whether the core's schema was completely
updated to DSpace 8.0 during the upgrade.

If you need to modify the schema, you should then rebuild the OAI
core: 'bin/dspace oai import -c'


--
Mark H. Wood
Lead Technology Analyst

University Library
Indiana University Indianapolis
755 W. Michigan Street
Indianapolis, IN 46202
317-274-0749
library.indianapolis.iu.edu

[Andrew Thompson]

Hi Evelin,

Have you checked the metadata registry for the field item.hasbitstream?

You might be able to search for the item with this query too:

item.hasbitstream:*

You could try running dspace oai import -c since that will clear the index.

-Andrew

--
All messages to this mailing list should adhere to the Code of Conduct: https://www.lyrasis.org/about/Pages/Code-of-Conduct.aspx
---
You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/dspace-tech/821c45d6-cf35-4c26-aa00-b724536e338cn%40googlegroups.com.

[Evelin Bányai]

Dear Mark,

I'm a bit lost at this point. I can see the old and the new schema.xml in the solr/oai/conf/ folder. The new indeed has the field item.hasbitstream. (The newer one is called schema.xml so I guess this is the one being used.)

I tried restarting the solr service, rebuilding the oai core with no luck so far. I still keep getting the same error.

Thanks,

Evelin

[Andrew Thompson]

Apache Solr has an admin interface available. It’s typically available at the solr host on port 8983. You can browse the schema as well as run queries directly.

Does you Solr instance have any errors in the logs?

Or perhaps the old schema is being loaded instead of the new schema?

-Andrew

On 14 Jan 2025, at 9:07 pm, Evelin Bányai <evelin...@gmail.com> wrote:



--

All messages to this mailing list should adhere to the Code of Conduct: https://www.lyrasis.org/about/Pages/Code-of-Conduct.aspx
---
You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/dspace-tech/fe5daee9-f032-4cc9-a1f7-c723c6f660ebn%40googlegroups.com.

[Evelin Bányai]

Dear Andrew,

I can see the same error on the interface, however upon further inspection the loaded schema content indeed seem to be the old one (here: 8983/solr/#/oai/files?file=schema.xml).

I tried to reload the schema with this command 8983/solr/admin/cores?action=RELOAD&core=oai - which was seemingly successful, but  nothing changes. Could it be a cache problem? Or does it really load the old schema somehow? What should I do differently?

Many thanks,

Evelin

[Andrew Thompson]

Try moving the old schema outside of the directory, restarting Solr and DSpace, and then running the import script.

DSpace at some point will rebuild the Solr index from scratch and needs access to the schema file to do so, and it’s potentially picking up the wrong file. Either that or replace the contents of your old file with the new one and see what happens.

-Andrew

On 15 Jan 2025, at 9:14 pm, Evelin Bányai <evelin...@gmail.com> wrote:

Dear Andrew,

To view this discussion visit https://groups.google.com/d/msgid/dspace-tech/c7372c9a-3f98-4470-b788-1e9d41248c76n%40googlegroups.com.

[Evelin Bányai]

Thanks for the help, I was able finally track down the issue. Turned out our actual solr instance and the loaded schema.xml is located somewhere else than I was looking at... I just accidentally noticed it on the solr admin interface today. 

Best regards,

Evelin