which db pep is triggered by "iput -f"?

261 views
Skip to first unread message

mustafa dikmen

unread,
Jan 3, 2024, 11:49:25 AM1/3/24
to iRODS-Chat
Hello,

I would like to know which database pep is triggered by "iput -f" operation for an existing object (already registered).

For the first registration it is obvious pep_database_reg_data_obj_post is triggered by "iput -f" and by "iput".

More specifically here is an example;
- make 6 bytes file.txt
- "iput (-f) file.txt" triggers pep_database_reg_data_obj_*.
- change the content of the same file - make 10 bytes file.txt
- "iput -f file.txt" doesnt trigger pep_database_reg_data_obj_*.
- "ils -L file.txt" will show the size of the object is updated in db.

Could you let me know which pep is triggered that updates the db by iput -f?

Thanks.
Regards,
M.Dikmen

Kory Draughn

unread,
Jan 3, 2024, 3:05:40 PM1/3/24
to irod...@googlegroups.com
Hi Mustafa,

The pep_api_data_object_finalize_* set of PEPs is likely what you want. See the following page for info on the inputs and outputs.
Hope that helps.

Kory Draughn
Chief Technologist
iRODS Consortium


--
--
The Integrated Rule-Oriented Data System (iRODS) - https://irods.org
 
iROD-Chat: http://groups.google.com/group/iROD-Chat
---
You received this message because you are subscribed to the Google Groups "iRODS-Chat" group.
To unsubscribe from this group and stop receiving emails from it, send an email to irod-chat+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/irod-chat/c0254c7c-e369-4746-8f49-f2ee4ffbfafdn%40googlegroups.com.

mustafa dikmen

unread,
Jan 4, 2024, 5:24:58 AM1/4/24
to iRODS-Chat
Hi Kory,

Thanks for your suggestion. But unfortunately those PEPs are not triggered by iput (-f). As far as I see with the operation "iput -f", "pep_api_obj_stat_*" and "pep_api_data_obj_put_*" are the PEPs from api group as well as pep_database_commit_* from the database group fired by  "iput -f". However these are dont provide information that I want to see.

In this regard, to focus on a possible alternative workaround, I want to verify something else: All PEPs fired by a specific action contain the same pid number regardless of how/by whom fired, is this correct?

Best Regards,
Mustafa

Kory Draughn

unread,
Jan 4, 2024, 8:21:06 AM1/4/24
to irod...@googlegroups.com
Sorry, I meant these ...
  • pep_database_data_object_finalize_*
Kory Draughn
Chief Technologist
iRODS Consortium

Kory Draughn

unread,
Jan 4, 2024, 8:27:12 AM1/4/24
to irod...@googlegroups.com
Tracking the PID could be useful, but it may not cover all cases (i.e. redirection).

Can you explain what you're trying to achieve? That may give us some ideas for reaching a good solution.

Kory Draughn
Chief Technologist
iRODS Consortium

Kory Draughn

unread,
Jan 4, 2024, 8:37:16 AM1/4/24
to irod...@googlegroups.com
Also, the following should be helpful:
With those two references, figuring out which PEPs are used should be easier.

I noticed that the PEP list on docs.irods.org isn't up to date either. We'll get that fixed.

Kory Draughn
Chief Technologist
iRODS Consortium

mustafa dikmen

unread,
Jan 4, 2024, 9:17:28 AM1/4/24
to irod...@googlegroups.com
Hi Kory,

Thanks for these reference docs, they look useful. Indeed as you said https://docs.irods.org/4.3.1/plugins/dynamic_policy_enforcement_points/ doesnt include "pep_database_data_object_finalize_*".

As for what I am trying to achieve, we would like to build a reliable auditing system (information fetched from pep logs comparing with irods catalog) based on a pre-defined format fetched from pep messages. Having said this, since not all peps include "wanted information, for example object_id" in that pre-defined keys/values, I would like to work with the db peps instead of the api peps.

R/
Mustafa



Kory Draughn

unread,
Jan 4, 2024, 9:32:33 AM1/4/24
to irod...@googlegroups.com
Got it.

Please correct me if I'm wrong, but it sounds like you can watch all database PEPs and when something happens, your policy can package/send the event to the auditing system. Probably using the delay queue.

For now, you'll also need to handle the atomic AVU and atomic ACLs PEPs too. Those APIs don't have database PEPs yet.

Kory Draughn
Chief Technologist
iRODS Consortium

mustafa dikmen

unread,
Jan 4, 2024, 10:08:38 AM1/4/24
to irod...@googlegroups.com
Good to know that atomic apis don't have database peps. We will not need to explore them.

Correct, it is similar to how you described. But, not with all database PEPs. Instead, with PEPs that provide information required. Actually we are still investigating what can be done and planning to work on a narrowed scope (not for all entities).

We believe the main issue in auditing is that,
- information could be fetched from the irods database directly but we cannot completely rely on that information because the irods catalog does not contain the information of who did, how done.
- we can count on PEPs only with regular actions of main clients (according to our experience). For example; seems no way to keep track of the permission change on data objects if permissions are changed recursively on a parent collection. Another example is "ibun" does not trigger any pep. Also, it looks like it is a bit complex to keep track of all actions from all clients...

mustafa dikmen

unread,
Jan 5, 2024, 9:44:02 AM1/5/24
to irod...@googlegroups.com
Hi Kory,

Another question:
I would like to fetch data_id of the data object removed. If the remove operation is executed without the force flag, I am able to capture the data_id information in "pep_database_mod_data_obj_meta_post". But if the unlink operation is executed with the force flag, then the only relevant pep which is triggered is "pep_api_data_obj_unlink_post" and this doesn't contain the data_id information. Is there a way to acquire the id information of the data object deleted without querying the irods catalog?

Thanks.

mustafa dikmen

unread,
Jan 5, 2024, 10:27:59 AM1/5/24
to iRODS-Chat
Concerning with your earlier remark "For now, you'll also need to handle the atomic AVU and atomic ACLs PEPs too. Those APIs don't have database PEPs yet." I guess non-atomic acl APIs dont have database PEPs either, there are only pep_api_mod_access_control_*. Is this correct?

Alan King

unread,
Jan 5, 2024, 3:28:11 PM1/5/24
to irod...@googlegroups.com
Hi Mustafa,

The non-atomic AVU/ACL APIs do have database PEPs:
pep_database_<add|mod|del>_avu_metadata_*
pep_database_mod_access_control_*

I think that these are only triggered by the equivalent API calls, though. The atomic AVU/ACL APIs do not trigger these database PEPs as they have their own database operations that do not use the database plugin interface.

As for the other question regarding unlink, it seems that the data object info for the unlinked object isn't available through the input parameters... You may be able to access the data object info through the dataId session variable but those have been deprecated and are not available in all rule contexts (e.g. delayed rules).



--
Alan King
Senior Software Developer | iRODS Consortium

mustafa dikmen

unread,
Jan 16, 2024, 9:52:16 AM1/16/24
to iRODS-Chat
Hi,

Other than irods_client_http_api which clients do have capability to modify permissions atomically? Thanks.

Kory Draughn

unread,
Jan 16, 2024, 1:25:14 PM1/16/24
to irod...@googlegroups.com
The only clients / libraries that support modifying ACLs atomically are:
  • HTTP API
  • PRC
With the Jargon library being a maybe.

Thanks,

Kory Draughn
Chief Technologist
iRODS Consortium

Kory Draughn

unread,
Jan 16, 2024, 1:38:27 PM1/16/24
to irod...@googlegroups.com
Mustafa,

Sorry. It looks like only the HTTP API supports the atomic ACLs API.

Kory Draughn
Chief Technologist
iRODS Consortium

Reply all
Reply to author
Forward
0 new messages