Limitations using static and dynamic PEPs for management data ingestion

[Yonny CARDENAS]

I would like to apply a validation process to all data uploaded to specific irods collection (path).

Users can upload files with any irods client/api (e.g. icommands,gocommands, webdav, irodsfs/fuse, metalnx,…), the validation must be executed after a data object is at rest (i.e. when all content have been uploaded and file registered in iRODS).

I have found limitations depending on whether I use statics or dynamic PEPs:

1) using deprecated static pep acPostProcForPut:
 
The irodsfs/fuse client trigger rule two times (see below) which is called by api DATA_OBJ_CLOSE

2) using dynamic peps:

   + pep_api_data_obj_put_post is only trigged by iput
   
   + pep_api_data_obj_close_pre/post does not provide information about object path
     in contrast with *DATAOBJINP.obj_path of pep_api_data_obj_put_post

I'm trying to do this with a rule but it seems that is not possible, the PEP to use
depends of the API that the client uses…

Any ideas?  

Yonny

# server rule
acPostProcForPut {
   ON( $objPath like "/tempZone/project/ingest/*") {
   assign(*thePath, $filePath);
   msiExecCmd("validate.sh","*thePath","null","null","null",*OUT);
   }
}

% cat validate.sh
#!/bin/sh
echo  "$( date +%s.%N)" $1 >> /tmp/mylog
__EOF__

# client with irodsfs
% cp FILE_IRODSFS_003 /path_to_mount_point

# server side  
$ cat /tmp/mylog
1768494907.599556166 FILE_IRODSFS_003
1768494907.643896740 FILE_IRODSFS_003

# server logs  
irodsServer[2666]:{
"log_category":"legacy"
"log_level":"info"
"log_message":"execCmd:/var/lib/irods/msiExecCmd_bin/validate.sh
     argv:/var/lib/irods/Vault/project/ingest/FILE_IRODSFS_003"
"request_api_name":"DATA_OBJ_CLOSE_AN"
"request_api_number":673
"request_api_version":"d"
"request_release_version":"rods4.3.0"
"server_pid":2666
"server_timestamp":"2026-01-15T16:35:07.596Z"
"server_type":"agent"
"server_zone":"tempZone"
 […]
}

irodsServer[2671]:{
"log_category":"legacy"
"log_level":"info"
"log_message":"execCmd:/var/lib/irods/msiExecCmd_bin/validate.sh
     argv:/var/lib/irods/Vault/project/ingest/FILE_IRODSFS_003"
"request_api_name":"DATA_OBJ_CLOSE_AN"
"request_api_number":673
"request_api_version":"d"
"request_release_version":"rods4.3.0"
"server_pid":2671
"server_timestamp":"2026-01-15T16:35:07.639Z"
"server_type":"agent"
"server_zone":"tempZone"
 […]
}

[Alan King]

Hi Yonny,

You might be interested in this section of our Policy Cookbook called "Enforcing policy when data transfers complete": https://docs.irods.org/4.3.3/administrators/policy_cookbook/#enforcing-policy-when-data-transfers-complete

These policy examples were written with a 4.3.3 (or later) server in mind, but the principles should still apply to a 4.3.0 server. You may need to find another solution for working with the JSON strings described therein because the JSON microservices (i.e. msi_json*) were not introduced until a release after 4.3.0. It may be easiest to use the Python Rule Engine Plugin (https://github.com/irods/irods_rule_engine_plugin_python) for working with the JSON objects.

Regardless, I would recommend upgrading to 4.3.4 or 5.0.2 (the latest iRODS 4.3 and 5 releases at time of writing, respectively), if that's possible for your deployment. 4.3.0 has some issues that were resolved in later releases in addition to the new features.

Let us know if you want more help with implementing a solution.

Alan

--
--
The Integrated Rule-Oriented Data System (iRODS) - https://irods.org
 
iROD-Chat: http://groups.google.com/group/iROD-Chat
---
You received this message because you are subscribed to the Google Groups "iRODS-Chat" group.
To unsubscribe from this group and stop receiving emails from it, send an email to irod-chat+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/irod-chat/3d1987f9-88fc-4f50-b576-84379e75760bn%40googlegroups.com.

--

Alan King

Senior Software Developer | iRODS Consortium

[Yonny CARDENAS]

Hi Alan, 

Thanks for you response, it woks fine. 

Regards, 

Yonny