has anyone done an RDF or OWL schema for iRODS?

[Conway, Mike]

Hi all,

We're working in several projects that store RDF statements about iRODS data as AVUs, and then uses them to maintain a triple store.  One example is HIVE integration, (see http://thejargonblog.blogspot.com/2013/03/some-work-in-progress-integrating-hive.html)

I'm curious as to whether anyone has created an RDF or OWL schema that represents iRODS ICAT data for files and collections?  I'd like to be able to relate SKOS vocabulary terms to iRODS files and collections in a triple store so we can use SPARQL to query for such data…is there any interest out there for doing so?

MC

Mike Conway

DICE Center

Jargon, Java, Interface Developer

michael...@unc.edu

------------------------------------------------

Google voice/video: Michael....@gmail.com

Skype: michael.c.conway

[Ward, Jewel Hope]

The closest project I know of would be the project Antoine de Torcy, Mason Chua, Jon Crabtree and I worked on c. 2008-2010, where we extracted DDI metadata into iRODS.

http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=5380867

OR:

https://www.irods.org/pubs/Meeting_1003/PROCEEDINGS_iRODS_USER_MEETING-2010.pdf

> --
> --
> "iRODS: the Integrated Rule-Oriented Data-management System; A community driven, open source, data grid software solution" https://www.irods.org
>
> iROD-Chat: http://groups.google.com/group/iROD-Chat
>
> ---
> You received this message because you are subscribed to the Google Groups "iROD-Chat" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to irod-chat+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Jeff Muller]

I can't really say too much about it right now, but the project I'm working on potentially has an interest in relating triple-store data to irods collections and files.  Whether it is a pressing need or not depends on how our next round of prototypes plays out. 

I'd be happy to share our plans after that.

Cheers,
Jeff

[Don Pellegrino]

In the "Demo of SPARQL search in HIVE/iRODS Integration" video, I see at 2:23 that the AVU Attribute field is being used to store the URI of vocabulary terms from the ontology. This seems like a reasonable way to integrate ontology URIs in the AVU schema.

I am exploring the use of certain metadata to control data policy. My approach is to designate a namespace in our .owl ontology that has terms for iRODS data policy. I am trying to develop rules that detect the presence of those terms in the Attribute field using rules triggered from "acPreProcForModifyAVUmetadata." I would like to ensure users cannot modify AVU records having vocabulary from that namespace. For example, the following rule should prevent users other than the rods user from modifying AVU records with attributes in the irods names:

acPreProcForModifyAVUmetadata(*Option,*ItemType,*ItemName,*AName,*AValue,*AUnit) {

  ON( (*AName like "http://testzone01/irods#*") &&

      ($userNameClient != "rods") ) {

    cut;

    fail;

  }

}

So two questions:

1. Has anyone seen issues with using the "Attribute = URI; Value = parsed string; Unit = constant" convention for integrating AVUs with an ontology?
2. Are there any design patterns or example rules for making some AVU units as read-only by end-users?

[Mike Conway]

1. Has anyone seen issues with using the "Attribute = URI; Value = parsed string; Unit = constant" convention for integrating AVUs with an ontology?

This is what we have adopted, and I'm using unit to denote special iRODS values, this way when I want to sweep for items marked up with HIVE vocabulary terms my genquery is just looking for iRODSUserTagging:RDF:HIVE in units which makes a nice efficient query, then I put the full uri for the resource in attribute and overload the value with other additional info.  The delimited approach works though it is then hard to query since it's so arbitrary.

      2.Are there any design patterns or example rules for making some AVU units as read-only by end-users?

Check out and comment here: https://github.com/irods/irods/issues/2054

We've discussed this in the context of the iRODS technical working group with the consortium folks as an enhancement needed for DFC.  This allows a special class of AVU for technical or preservation metadata outside of user modification.


Sounds like cool stuff. This focus on metadata for discovery, preservation, etc has all of the sudden become THE hot topic everywhere.  Hopefully we can have some good discussions and demos of what everyone is thinking here at the upcoming user meeting.

Cheers
MC


--

--
"iRODS: the Integrated Rule-Oriented Data-management System; A community driven, open source, data grid software solution" https://www.irods.org
 
iROD-Chat: http://groups.google.com/group/iROD-Chat

---

You received this message because you are subscribed to the Google Groups "iRODS-Chat" group.

To unsubscribe from this group and stop receiving emails from it, send an email to irod-chat+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Tony Edgin]

We are starting to protect certain namespace AVUs from begin modified by the user.  You could use a similar approach until issue #2054 gets resolved.

Our approach was the same as the one you suggested.  However, you need to consider the *Option parameter.  When *Option == 'mod', you need to check that the user is not changing the name of the AVU to something in the protected namesake.  Also, you'll need to decide how to handle *Option == 'cp'. This happens when a user uses imeta cp to copy all of the AVUs from one ICAT item to another.  The easiest way to prevent this is to not allow your normal users to use imeta cp, by having the rule always fail unless the user is an admin.  We chose to have the operation copy just the unprotected attributes. This is a bit of a hack. Inside the acPreProcForModifyAVUmetadata rule, we perform the copy manually and then fail. We cause the pre rule to fail to prevent iRODS itself from copying the metadata.  

Here's our implementation of the copy rule with some of the obvious functions not shown for space considerations.

266 # If an AVU is not protected, it sets the AVU to the given item

267 setAVUIfUnprotected(*ItemType, *ItemName, *A, *V, *U) {

268   if (!avuProtected(*A)) {

269     msiSetAVU(*ItemType, *ItemName, *A, *V, *U);

270   }

271 }

272 

273 # Copies the unprotected AVUs from a given collection to the given item.

274 cpUnprotectedCollAVUs(*Coll, *TargetType, *TargetName) =

275   foreach (*avu in SELECT META_COLL_ATTR_NAME, META_COLL_ATTR_VALUE, META_COLL_ATTR_UNITS

276                      WHERE COLL_NAME == *Coll) {

277     setAVUIfUnprotected(*TargetType, *TargetName, *avu.META_COLL_ATTR_NAME,

278                         *avu.META_COLL_ATTR_VALUE, *avu.META_COLL_ATTR_UNITS);

279   }

280 

281 # Copies the unprotected AVUs from a given data object to the given item.

282 cpUnprotectedDataObjAVUs(*ObjPath, *TargetType, *TargetName) {

283   msiSplitPath(*ObjPath, *parentColl, *objName);

284   foreach (*avu in SELECT META_DATA_ATTR_NAME, META_DATA_ATTR_VALUE, META_DATA_ATTR_UNITS

285                      WHERE COLL_NAME == *parentColl AND DATA_NAME == *objName) {

286     setAVUIfUnprotected(*TargetType, *TargetName, *avu.META_DATA_ATTR_NAME,

287                         *avu.META_DATA_ATTR_VALUE, *avu.META_DATA_ATTR_UNITS);

288   }

289 }

290 

291 # Copies the unprotected AVUs from a given resource to the given item.

292 cpUnprotectedRescAVUs(*Resc, *TargetType, *TargetName) =

293   foreach (*avu in SELECT META_RESC_ATTR_NAME, META_RESC_ATTR_VALUE, META_RESC_ATTR_UNITS

294                      WHERE RESC_NAME == *Resc) {

295     setAVUIfUnprotected(*TargetType, *TargetName, *avu.META_RESC_ATTR_NAME,

296                         *avu.META_RESC_ATTR_VALUE, *avu.META_RESC_ATTR_UNITS);

297   }

298 

299 # Copies the unprotected AVUs from a given resource group to the given item.

300 cpUnprotectedRescGrpAVUs(*Grp, *TargetType, *TargetName) =

301   foreach (*avu in SELECT META_RESC_GROUP_ATTR_NAME, META_RESC_GROUP_ATTR_VALUE,

302                           META_RESC_GROUP_ATTR_UNITS

303                      WHERE RESC_GROUP_NAME == *Grp) {

304     setAVUIfUnprotected(*TargetType, *TargetName, *avu.META_RESC_GROUP_ATTR_NAME,

305                         *avu.META_RESC_GROUP_ATTR_VALUE, *avu.META_RESC_GROUP_ATTR_UNITS);

306   }

307 

308 # Copies the unprotected AVUs from a given user to the given item.

309 cpUnprotectedUserAVUs(*User, *TargetType, *TargetName) =

310   foreach (*avu in SELECT META_USER_ATTR_NAME, META_USER_ATTR_VALUE, META_USER_ATTR_UNITS

311                      WHERE USER_NAME == *User) {

312     setAVUIfUnprotected(*TargetType, *TargetName, *avu.META_RESC_ATTR_NAME,

313                         *avu.META_RESC_ATTR_VALUE, *avu.META_RESC_ATTR_UNITS);

314   }

431 # This rule ensures that only the non-protected AVUs are copied from one item to another.

432 acPreProcForModifyAVUMetadata(*Option, *SourceItemType, *TargetItemType, *SourceItemName,

433                                   *TargetItemName) {

434   if (!isRodsAdmin($userNameClient)) {

435     if (*SourceItemType == '-c') {

436       cpUnprotectedCollAVUs(*SourceItemName, *TargetItemType, *TargetItemName);

437     } else if (*SourceItemType == '-d') {

438       cpUnprotectedDataObjAVUs(*SourceItemName, *TargetItemType, *TargetItemName);

439     } else if (*SourceItemType == '-g') {

440       cpUnprotectedRescGrpAVUs(*SourceItemName, *TargetItemType, *TargetItemName);

441     } else if (*SourceItemType == '-r') {

442       cpUnprotectedRescAVUs(*SourceItemName, *TargetItemType, *TargetItemName);

443     } else if (*SourceItemType == '-u') {

444       cpUnprotectedUserAVUs(*SourceItemName, *TargetItemType, *TargetItemName);

445     }

446 

447     # fail to prevent iRODS from also copying the protected metadata

448     cut;

449     failmsg(0, "IPLANT SUCCESS:  Successfully copied the unprotected metadata.");

450   }

451 }

Currently, there is no microservice that sets an AVU, so we implemented our own, msiSetAVU.  It is easy to implement, so I won't show it.

Cheers,

Tony

[Don Pellegrino]

Thanks for the example code. That is very helpful.

The signature for acPreProcForModifyAVUMetadata on lines 432 and 433 does not match the signatures for this function that I find in core.re:

acPreProcForModifyAVUMetadata(*Option,*ItemType,*ItemName,*AName,*AValue,*AUnit) { }

acPreProcForModifyAVUMetadata(*Option,*ItemType,*ItemName,*AName,*AValue) { }

I am running iRODS 3.2. Did the signature for this function change or could the example in core.re be incomplete?

[Tony Edgin]

I'm using iRODS 3.3.1. The new signatures were added in 3.3, because the old one wasn't sufficient to cover all of the imeta commands.

[Don Pellegrino]

I have upgraded to iRODS 3.3.1. I forked irods-contrib on GitHub (https://github.com/donpellegrino/irods-contrib/tree/master/avu-protect) and created two rule bases:

avu-protect.re - Prevent AVUs having an attribute with a certain prefix from being modified by non-admin users.

archive.re - Prevent deletion of objects having an AVU archive indicator set.

I also wrote a test suite using the Python unittest framework to test out the logic. The Python code just calls iCommands rather than using the iRODS Python API.

avuprotectest.py

The test suite passes and the iCommands behave as expected. The iRODS PHP web client includes a dialog to modify AVUs and somehow that results in protected AVUs being deleted rather than protected. I have no idea how the PHP web client is circumventing the rules.

If you get a chance to look at the fork of irods-contrib let me know what you think. I would love to get this functionality organized into a set of shareable rule bases rather than trying to maintain it privately.

[Hao Xu]

Hi Don,

Maybe you can try writing the user info, avu modified etc. into the server log in your rules when the checks pass, so that you can see under what condition the PHP client was able to modify the avu.

Thanks,

  Hao