./bin/dspace oai import process deletes requests directory

[Steve Michaels]

I've discovered that when the ./bin/dspace oai import process runs, it doesn't just clear the requests cache, it deletes the ./var/oai/requests directory itself.

Is there a configuration that I'm missing, a command flag in the crontab command that's wrong ("$DSPACE/bin/dspace oai import > /dev/null"), or is this a weird bug?

[DSpace Technical Support]

Hi Steve,

I believe that is expected behavior because that "/var/oai/requests" directory *is* the cache.  So, the entire directory is deleted in the code here: https://github.com/DSpace/DSpace/blob/main/dspace-oai/src/main/java/org/dspace/xoai/services/impl/cache/DSpaceXOAICacheService.java#L132-L134

So, by deleting that directory the entire cache is cleared.  The directory is later recreated if it no longer exists.

Tim

[667...@gmail.com]

Hi Tim, 

I currently have the same problem, "oai import" deletes the "requests" directory, and further OAI client requests face 500 "Internal error"

Are you sure it is later recreated? Looking at XOAI.java i do not see where. 

Thanks!

[DSpace Technical Support]

Hi,

At least according to my reading of the code, I believe that "requests" directory should be recreated after it is deleted (which occurs during clearing the cache).

In the OAI caching code, you can see the "requests" directory is returned "getBaseDir()" here:

https://github.com/DSpace/DSpace/blob/dspace-9_x/dspace-oai/src/main/java/org/dspace/xoai/services/impl/cache/DSpaceXOAICacheService.java#L44

And, here, that directory is re-created if it doesn't exist:

https://github.com/DSpace/DSpace/blob/dspace-9_x/dspace-oai/src/main/java/org/dspace/xoai/services/impl/cache/DSpaceXOAICacheService.java#L73-L76

I'd recommend looking closer at the 500 "Internal Error" you are receiving. 500 errors are very generic errors that just mean "something went wrong".  To determine what actually went wrong, you'd want to use the Troubleshooting Guide to look for a more detailed error message.  In this case, it's likely the more detailed error behind that 500 error would be in your DSpace logs or Tomcat logs. 

If you are able to verify that the 500 error is being caused by the missing "requests" directory, then I'd recommend creating a bug ticket in our GitHub repository and describe the exact steps you took, and share the detailed error message that you found in your logs (when that 500 error occurs).

Hopefully that helps,

Tim