What does a low level management like iRODS exactly for in fedora commons?

[Jeevan Patnaik]

I am not clear about the actual advantage of having iRODS or any other low level storage management. What are it's benefits exactly and when should we use it?

In Fedora-commons with normal file system low level storage: a datastream created on May 8th, 2009 might be located in the 2009/0508/20/48/ directory. What does iRODS do in place of this? And how can it be helpful here?

[Mike Conway]

I wouldn't really think of iRODS as low level storage management.

iRODS is really the only platform for policy managed data preservation.  It does indeed virtualize storage, providing a global, logical namespace over heterogeneous types of storage, it also allows you to enforce preservation policies at each storage location, no matter what client or access method is used.  It also provides a global metadata catalog that is automatically maintained and reflects the application of your preservation policies, allowing audit and verification of your preservation policies.

In addition, iRODS is developing a pretty powerful metadata management capability, allowing pluggable indexing and query capability that allow synchronization with external indexes (e.g. Elastic Search, MAUI, Jena triple store).

With the pluggable rule engine and asynchronous messaging architecture,it becomes rather straight forward to generate audit and provenance metadata that will track every single operation on your data, pre and post, including any plugins you may develop or utilize.

iRODS is middleware, rather than a prepackaged solution.  This middleware supports plug-ins and configurable policies at all points, so you are not limited by a pre-defined set of tools.  iRODS also can be connected to wide range of preservation, computation, and enterprise services, and can manage large amounts (both in number of obects and size of objects) of data, and efficiently move and manage data using high performance protocols, including third party data transfer protocols.

iRODS also is built to support federation, so that your preservation environment may share data with other institutions while remaining under audit and policy control.  I know that's a lot, it's really complicated, but folks are doing this for many many millions of objects, many many thousands of users, and with a range of object sizes.

For your example on generating paths for datastreams, that is expressed as a policy within iRODS, so when your 'put' an object to the data store, iRODS can apply any sort of rule you wish to compute logical path names, and these logical names can then map to any sort of data store, be it a local disk, an object store, a cloud store, or any combination, automatically.

MC

On 08/07/2015 02:50 AM, Jeevan Patnaik wrote:

I am not clear about the actual advantage of having iRODS or any other low level storage management. What are it's benefits exactly and when should we use it?

In Fedora-commons with normal file system low level storage: a datastream created on May 8th, 2009 might be located in the 2009/0508/20/48/ directory. What does iRODS do in place of this? And how can it be helpful here?

--
--
"iRODS: the Integrated Rule-Oriented Data-management System; A community driven, open source, data grid software solution" https://www.irods.org
 
iROD-Chat: http://groups.google.com/group/iROD-Chat

---
You received this message because you are subscribed to the Google Groups "iRODS-Chat" group.
To unsubscribe from this group and stop receiving emails from it, send an email to irod-chat+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.