unable to edit metada on some files in DSpace 7.6

[Jerevan]

Hi,

in our institution, we have been using Invenio 1, but we have migrated to DSpace 7.6 (we have been preparing the migratio since DSpace 6.something).

Now in DSpace 7.6, metadata of some Items cannot be edited via API nor the Administration Interface. The only way to edit for example URI is direct database access, which is not ideal.

This problem however, occurs only on 50 entries from the total number of 1500+ entries which had underwent the migration. There is no visible difference on the metadata either.

Any attempt to edit metadata from Admin UI gives this response:

HTTP/1.1 500
Date: Tue, 12 Mar 2024 15:47:38 GMT
Server: Apache/2.4.37 (Red Hat Enterprise Linux) OpenSSL/1.1.1k
Vary: Origin,Access-Control-Request-Method,Access-Control-Request-Headers
Content-Language: en
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-Frame-Options: DENY
Content-Type: application/json;charset=UTF-8
Connection: close
Transfer-Encoding: chunked

{"timestamp":"2024-03-12T15:47:38.237+00:00","status":500,"error":"Internal Server Error","message":"An exception has occurred","path":"/server/api/core/items/065c1a21-7410-4ce6-8f61-a92b30b8c61a"}

in the DSpace logs, there is nothing meaningfull either. From what I've been able to gather, these are logs significant for the performed action:

024-03-12 16:47:32,787 INFO  unknown unknown org.eclipse.jetty.util.log @ Logging initialized @4943ms to org.eclipse.jetty.util.log.Slf4jLog
2024-03-12 16:47:32,989 INFO  unknown unknown org.dspace.importer.external.service.ImportService @ Loading 6 import sources.
2024-03-12 16:47:34,251 INFO  unknown unknown org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor @ Autowired annotation should only be used on methods with parameters: public java.lang.String org.dspace.storage.bitstore.S3BitStoreService.getBucketName()
2024-03-12 16:47:34,810 INFO  unknown unknown org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor @ Autowired annotation is not supported on static methods: public static void org.dspace.rdf.storage.DOIURIGenerator.setFallback(org.dspace.rdf.storage.URIGenerator)
2024-03-12 16:47:34,814 INFO  unknown unknown org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor @ Autowired annotation is not supported on static methods: public static void org.dspace.rdf.storage.DOIHandleURIGenerator.setFallback(org.dspace.rdf.storage.URIGenerator)
2024-03-12 16:47:35,288 INFO  unknown unknown org.ehcache.core.EhcacheManager @ Cache 'sherpa.searchByJournalISSN' removed from Eh107InternalCacheManager.
2024-03-12 16:47:35,290 INFO  unknown unknown org.ehcache.core.EhcacheManager @ Cache 'manifests' removed from Eh107InternalCacheManager.
2024-03-12 16:47:35,292 INFO  unknown unknown org.ehcache.core.EhcacheManager @ Cache 'canvasdimensions' removed from Eh107InternalCacheManager.
2024-03-12 16:47:36,938 INFO  unknown unknown org.apache.commons.beanutils.FluentPropertyBeanIntrospector @ Error when creating PropertyDescriptor for public final void org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)! Ignoring this property.
2024-03-12 16:47:37,250 INFO  unknown unknown org.apache.commons.beanutils.FluentPropertyBeanIntrospector @ Error when creating PropertyDescriptor for public final void org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)! Ignoring this property.
2024-03-12 16:47:37,271 INFO  unknown unknown org.apache.commons.beanutils.FluentPropertyBeanIntrospector @ Error when creating PropertyDescriptor for public final void org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)! Ignoring this property.
2024-03-12 16:47:37,276 INFO  unknown unknown org.apache.commons.beanutils.FluentPropertyBeanIntrospector @ Error when creating PropertyDescriptor for public final void org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)! Ignoring this property.
2024-03-12 16:47:37,279 INFO  unknown unknown org.apache.commons.beanutils.FluentPropertyBeanIntrospector @ Error when creating PropertyDescriptor for public final void org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)! Ignoring this property.
2024-03-12 16:47:37,283 INFO  unknown unknown org.apache.commons.beanutils.FluentPropertyBeanIntrospector @ Error when creating PropertyDescriptor for public org.apache.commons.configuration2.builder.FileBasedBuilderParametersImpl org.apache.commons.configuration2.builder.FileBasedBuilderParametersImpl.setURL(java.net.URL,org.apache.commons.configuration2.io.URLConnectionOptions)! Ignoring this property.
2024-03-12 16:47:37,283 INFO  unknown unknown org.apache.commons.beanutils.FluentPropertyBeanIntrospector @ Error when creating PropertyDescriptor for public java.lang.Object org.apache.commons.configuration2.builder.FileBasedBuilderParametersImpl.setURL(java.net.URL,org.apache.commons.configuration2.io.URLConnectionOptions)! Ignoring this property.
2024-03-12 16:47:37,293 INFO  unknown unknown org.apache.commons.beanutils.FluentPropertyBeanIntrospector @ Error when creating PropertyDescriptor for public final void org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)! Ignoring this property.
2024-03-12 16:47:39,806 INFO  unknown unknown org.ehcache.xml.XmlConfiguration @ Loading Ehcache XML configuration from /opt/dspace/config/ehcache.xml.
2024-03-12 16:47:40,513 INFO  unknown unknown org.terracotta.offheapstore.paging.UpfrontAllocatingPageSource @ Allocating 4MB in chunks
2024-03-12 16:47:40,544 INFO  unknown unknown org.ehcache.core.EhcacheManager @ Cache 'sherpa.searchByJournalISSN' created in Eh107InternalCacheManager.
2024-03-12 16:47:40,556 INFO  unknown unknown org.terracotta.offheapstore.paging.UpfrontAllocatingPageSource @ Allocating 10.0MB in chunks
2024-03-12 16:47:40,562 INFO  unknown unknown org.ehcache.core.EhcacheManager @ Cache 'manifests' created in Eh107InternalCacheManager.
2024-03-12 16:47:40,573 INFO  unknown unknown org.terracotta.offheapstore.paging.UpfrontAllocatingPageSource @ Allocating 4MB in chunks
2024-03-12 16:47:40,578 INFO  unknown unknown org.ehcache.core.EhcacheManager @ Cache 'canvasdimensions' created in Eh107InternalCacheManager.
2024-03-12 16:47:40,620 INFO  unknown unknown org.ehcache.jsr107.Eh107CacheManager @ Registering Ehcache MBean javax.cache:type=CacheStatistics,CacheManager=file./opt/dspace/config/ehcache.xml,Cache=sherpa.searchByJournalISSN
2024-03-12 16:47:40,623 INFO  unknown unknown org.ehcache.jsr107.Eh107CacheManager @ Registering Ehcache MBean javax.cache:type=CacheStatistics,CacheManager=file./opt/dspace/config/ehcache.xml,Cache=manifests
2024-03-12 16:47:40,624 INFO  unknown unknown org.ehcache.jsr107.Eh107CacheManager @ Registering Ehcache MBean javax.cache:type=CacheStatistics,CacheManager=file./opt/dspace/config/ehcache.xml,Cache=canvasdimensions
2024-03-12 16:47:40,634 INFO  unknown unknown org.ehcache.jsr107.Eh107CacheManager @ Registering Ehcache MBean javax.cache:type=CacheStatistics,CacheManager=file./opt/dspace/config/ehcache.xml,Cache=sherpa.searchByJournalISSN
2024-03-12 16:47:40,635 INFO  unknown unknown org.ehcache.jsr107.Eh107CacheManager @ Registering Ehcache MBean javax.cache:type=CacheStatistics,CacheManager=file./opt/dspace/config/ehcache.xml,Cache=manifests
2024-03-12 16:47:40,636 INFO  unknown unknown org.ehcache.jsr107.Eh107CacheManager @ Registering Ehcache MBean javax.cache:type=CacheStatistics,CacheManager=file./opt/dspace/config/ehcache.xml,Cache=canvasdimensions
2024-03-12 16:47:40,872 INFO  unknown unknown org.eclipse.jetty.util.log @ Logging initialized @5378ms to org.eclipse.jetty.util.log.Slf4jLog
2024-03-12 16:47:41,082 INFO  unknown unknown org.dspace.importer.external.service.ImportService @ Loading 6 import sources.
2024-03-12 16:47:42,370 INFO  unknown unknown org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor @ Autowired annotation should only be used on methods with parameters: public java.lang.String org.dspace.storage.bitstore.S3BitStoreService.getBucketName()

There is the request which had triggered this response, I tried to add a metadata value (valid according to the schema) for Handle.net. It let me add the value, but not to save the edited item afterwards

PATCH /server/api/core/items/065c1a21-7410-4ce6-8f61-a92b30b8c61a HTTP/1.1
Host: repozitar.techlib.cz
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:123.0) Gecko/20100101 Firefox/123.0
Accept: application/json, text/plain, */*
Accept-Language: cs;q=1,en-US;q=0.1,en;q=0.09
Accept-Encoding: gzip, deflate, br
Content-Type: application/json; charset=utf-8
authorization: Bearer eyJhbGciOiJIUzI1NiJ9.eyJlaWQiOiJkNWEyYjZhNi0yNTEyLTRjZWMtOTRkNS02OTI3MzM2NTBkZDEiLCJzZyI6WyJkNTNlMDdlMi0yODA0LTRiZmItOTNiNS1jMmU1ZDNkNWQwNjciXSwiYXV0aGVudGljYXRpb25NZXRob2QiOiJsZGFwIiwiZXhwIjoxNzEwMjU5NDE5fQ._l_C2SuSkA2m_oMxSr5f8oneFPDaDPC6XnRj8Ct21rY
X-XSRF-TOKEN: d31cb6c1-89fb-434e-93d0-db8797cb9549
X-CORRELATION-ID: 6f1fdc32-509b-4b18-babc-a73042f71301
X-REFERRER: /entities/publication/065c1a21-7410-4ce6-8f61-a92b30b8c61a/edit/metadata
Content-Length: 130
Origin: https://repozitar.techlib.cz
Connection: keep-alive
Referer: https://repozitar.techlib.cz/entities/publication/065c1a21-7410-4ce6-8f61-a92b30b8c61a/edit/metadata
Cookie: MyHalBrowserCsrfToken=ba9626ff-445f-462d-b3af-cd21bc9d1438; MyHalBrowserToken=eyJhbGciOiJIUzI1NiJ9.eyJlaWQiOiJkNWEyYjZhNi0yNTEyLTRjZWMtOTRkNS02OTI3MzM2NTBkZDEiLCJzZyI6WyJkNTNlMDdlMi0yODA0LTRiZmItOTNiNS1jMmU1ZDNkNWQwNjciXSwiYXV0aGVudGljYXRpb25NZXRob2QiOiJsZGFwIiwiZXhwIjoxNzA0MzY4MTE4fQ.LQSh-FbJ8-sJbIAVfV1DHZ4XX-T1P3Bwg2gHwOVlzU8; DSPACE-XSRF-COOKIE=d31cb6c1-89fb-434e-93d0-db8797cb9549; klaro-anonymous=%7B%22authentication%22%3Atrue%2C%22preferences%22%3Atrue%2C%22acknowledgement%22%3Atrue%7D; INVENIOSESSION=aaccab61510027d7497144ec99375a58; klaro-d5a2b6a6-2512-4cec-94d5-692733650dd1={%22authentication%22:true%2C%22preferences%22:true%2C%22acknowledgement%22:true}; CORRELATION-ID=6f1fdc32-509b-4b18-babc-a73042f71301; dsLanguage=cs; XSRF-TOKEN=d31cb6c1-89fb-434e-93d0-db8797cb9549; dsAuthInfo={%22accessToken%22:%22eyJhbGciOiJIUzI1NiJ9.eyJlaWQiOiJkNWEyYjZhNi0yNTEyLTRjZWMtOTRkNS02OTI3MzM2NTBkZDEiLCJzZyI6WyJkNTNlMDdlMi0yODA0LTRiZmItOTNiNS1jMmU1ZDNkNWQwNjciXSwiYXV0aGVudGljYXRpb25NZXRob2QiOiJsZGFwIiwiZXhwIjoxNzEwMjU5NDE5fQ._l_C2SuSkA2m_oMxSr5f8oneFPDaDPC6XnRj8Ct21rY%22%2C%22expires%22:1710259419000}
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin

I'll appreciate any help or clues as to what might lead to such behaviour, as I am utterly confused why only a few Items have such behaviour when almost all of items in the repository have underwent the same procedures.

Thank you all!

Best regards

David Gerner

[DSpace Technical Support]

Hi David,

Apologies for the delayed response.  A 500 error is an "Internal Server Error" which should mean you'd see an "ERROR" log in your dspace.log file which corresponds to that error.  I don't see that in the logs you shared though, which is odd.  You may want to check again to make sure it wasn't overlooked.

If there's still nothing in the dspace.log, you could **temporarily** tell the REST API to return the full Java Stacktrace of the error (this is often the details needed to debug the problem).  You can do this by adding this to your local.cfg:

server.error.include-stacktrace = always

(This is listed as a step to try in our Troubleshooting guide: https://wiki.lyrasis.org/display/DSPACE/Troubleshoot+an+error#Troubleshootanerror-DSpace7.x(orabove) ... see step 3.)

I *believe* you will have to reboot the backend for this setting to take effect.  Once it is added, then you'd reproduce the 500 error again, and you should see the full Java stacktrace included in that 500 response.   It's important to *turn this back off* (remove the configuration) once you are done.... leaving this enabled in Production can be a security issue as it allows others to force errors in order to see the details of these Java stacktraces, which can give them information about your system.

It's unclear to me as well why this error would occur.  If it's only for some files/items, that would imply something perhaps in the data of those objects.  But, the Java stacktrace should provide more clues.

Tim