Progress on Storage Reports with iRODS Backend

[Dan Galewsky]

I have made a bit of progress getting the storage reports to work in our Duracloud setup using the iRODS storage provider.

It looks like there were two problems that caused the graphical storage reports not to display.

First - it appears that there is a bug in the iRODS api that causes both getSpaceContents and getSpaceContentsChunked to not return anything if the prefix argument has a value. 

i.e. this call will not return anything:

getSpaceContents( "x-duracloud-admin", "report/storage-report");

while this call does work - and returns all of the files in the x-duracloud-admin space:

getSpaceContents( "x-duracloud-admin", "");

I modified the IrodsStorageProvider to ignore the value passed in for the prefix and these calls return all of the storage reports (and the other files in the x-duracloud-admin directory).

Another problem I was encountering was that I somehow had a 'bad' storage report - one that caused the XML parser to choke - and once the parser had a problem it seems like the system gave up on building the storage report cache.

Once I got past these two problems - the system was able to display the graphical storage reports.

I don't think that my hack/fix for the prefix problem is going to work long-term -- or do you think it will be ok?

I may need to get in touch with the author of the irods-api library to see if he has any advice.

Do my conclusions seem to make sense? Any other thoughts?

The good news is that our Duracloud/iRODS implementation is looking good enough to demonstrate to our customers.

Thanks

--Dan Galewsky

[Bill Branan]

Hi Dan,

I'm glad to hear that your work has progressed to the point where you can begin doing demonstrations. I'd be interested to hear how those go.

With regard to the issue you're seeing with the prefix argument not working properly, it's good to know that ignoring the prefix allows the calls to return properly, but as you suggest, that's clearly not the ideal solution. I notice in the javadoc for the getSpaceContents() method this mentioned: "Prefix is assumed to be part of the collection name." So I'm assuming that the prefix arguments you've tried are indeed part of the iRODS collection name. 

I unfortunately don't have an iRODS endpoint to test this against at the moment, so I can't be a great deal of help. You may, however, be able to get assistance from Mike Smorul. He wrote the original IrodsStorageProvider class that is currently in DuraCloud, and I believe he also wrote the client library used to make the connection to iRODS. You can find his contact info here: https://wiki.duraspace.org/display/~msmorul.

If you're able to come up with a solution for the prefix issue that works, please let us know (ideally in the form of a patch) so that we can get the DuraCloud code updated. Ideally the two getSpaceContents methods would work as expected with the prefix. If that turns out to not be possible, I would prefer that the calls work, even if the prefix needs to be ignored. So let me know what you find.

Good luck,

Bill

--Dan Galewsky

--
You received this message because you are subscribed to the Google Groups "DuraCloud Dev" group.
To view this discussion on the web visit https://groups.google.com/d/msg/duracloud-dev/-/Wghq-eyigNgJ.
To post to this group, send email to duracl...@googlegroups.com.
To unsubscribe from this group, send email to duracloud-de...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/duracloud-dev?hl=en.