Re: [iROD-Chat:21449] Check total and available/free user storage quota by using PRC (python-irodsclient)

[Terrell Russell]

Hi KR John,

The quota values are all available via GenQuery (iquota just builds and executes a GenQuery itself)...

$ # confirm totals are calculated / up to date

$ iadmin cu

$ # a general query for all quotas in the system

$ iquest "select QUOTA_USER_NAME, QUOTA_USER_ZONE, QUOTA_LIMIT, QUOTA_OVER"
QUOTA_USER_NAME = quota_group_for_alice
QUOTA_USER_ZONE = tempZone
QUOTA_LIMIT = 20401094656
QUOTA_OVER = -20389538199
------------------------------------------------------------

I have just gone to the python-irodsclient to look for the model to build the same query in PRC... but those models do not yet exist (as far as I can tell).

I have created https://github.com/irods/python-irodsclient/issues/474 to track this work.

Terrell

On Thu, Oct 12, 2023 at 8:15 AM John <kayit...@gmail.com> wrote:

Hello,

I (Admin) have set the user quota on the iRODS server as follows:

$ iadmin mkgroup quota_group_for_alice

$ iadmin atg quota_group_for_alice  alice 

$ iadmin sgq quota_group_for_alice total 20401094656

By using iCommands client, the user "alice" can see her total and available quota:

                             $  iquota | sed -n  '/Group:  quota_group_for_alice/, /Over/p'
                                             Group:  quota_group_for_alice
                                              Zone:  RPTUzone
                                              Quota: 20,401,094,656 (20 billion) bytes
                                              Over:  -592,735,600 (-592 million) bytes (Nearing quota)

Is it possible for alice to check the total and available quota by using PRC (python-irodsclient) instead of using iCommands? If yes, any orientation (codes, ...) would be appreciated.

(Able to check only the available/free quota would also be Ok!).

Thanks.

KR John

--
--
The Integrated Rule-Oriented Data System (iRODS) - https://irods.org
 
iROD-Chat: http://groups.google.com/group/iROD-Chat
---
You received this message because you are subscribed to the Google Groups "iRODS-Chat" group.
To unsubscribe from this group and stop receiving emails from it, send an email to irod-chat+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/irod-chat/d2aaee28-610c-4714-8658-e3cee0f15386n%40googlegroups.com.

[John]

Hi Terrell,

Many thanks.

[Bazi Jana]

Hi,

I want to use the above example (John' s example) to determine the space remaining for user alice (quota_group_for_alice) by using PRC.

I see here https://github.com/irods/python-irodsclient (please check attachment) that this is now possible. I think, this might be easy by using the attached info! But I don' t understand very well, how to achieve this.

Please help me to determinate the space remaining on quota_group_for_alice. (I ' m using the default parameter "total" as resource).

Thanks.

Kind Regards

Jana

[Terrell Russell]

Hi Jana,

The quota work has been added *since* v1.1.9 was released - you can see it here...

https://github.com/irods/python-irodsclient/compare/v1.1.9...96d2cd2

This work will be included in the next release (soon).

In the meantime, you could get the latest code directly...

pip install git+https://github.com/irods/irods-pythonclient.git@main

Terrell

To view this discussion on the web visit https://groups.google.com/d/msgid/irod-chat/89b65baf-ed00-4878-91ad-889d03a7bd18n%40googlegroups.com.

[Bazi Jana]

Thanks

[MBANZA Sara]

Hello,

I' m using PRC 2.0.0.

Could you please help me and give me the codes to determine the free space remaining for a user (for example user with username "sara") on a resource "total" ?

Thank you.

[Terrell Russell]

The README has an example that should show you the way...

https://github.com/irods/python-irodsclient?#quotas-v200

> to determine the space remaining for a certain group on a given resource:
> 
>     from irods.models import Quota
>     session.groups.calculate_usage()
>     group, resource = ['my_group', 'my_limited_resource']
>     space_left_in_bytes = list(session.query(Quota.over).filter(Quota.user_id == session.groups.get(group).id,
                                                            Quota.resc_id == session.resources.get(resource).id))[0][Quota.over] * -1

Remember that user quotas have been 'turned off' as of 4.3.0... 

https://github.com/irods/irods/issues/4481

So, for a group named 'sara' and a resource named 'amazing'...

>     group, resource = ['sara', 'amazing']

Terrell

To view this discussion on the web visit https://groups.google.com/d/msgid/irod-chat/1dceb24a-0de8-408d-bcb7-9e53d15f3f13n%40googlegroups.com.

[MBANZA Sara]

Hello Terrell,

Thanks.

I' m trying the given codes. Unfortunately, I' m getting the error below and don't know why.

    my_session.groups.calculate_usage()
  File "/home/mbanza/myvirtuelenv/lib64/python3.6/site-packages/irods/manager/user_manager.py", line 23, in calculate_usage
    return _do_GeneralAdminRequest(self._get_session, "calculate-usage")
  File "/home/mbanza/myvirtuelenv/lib64/python3.6/site-packages/irods/message/__init__.py", line 815, in _do_GeneralAdminRequest
    response = conn.recv()
  File "/home/mbanza/myvirtuelenv/lib64/python3.6/site-packages/irods/connection.py", line 151, in recv
    raise get_exception_by_code(msg.int_info, err_msg)
irods.exception.SYS_NO_API_PRIV: None
Aborted (core dumped)

[Terrell Russell]

Ah, the SYS_NO_API_PRIV is saying that the calculate_usage() function requires admin rights.

You don't need that for just getting the query results.   It is just to make sure the totals were correct in the database first.

Terrell

To view this discussion on the web visit https://groups.google.com/d/msgid/irod-chat/7aff34c8-68c3-4161-8e8a-565eaaa30c5dn%40googlegroups.com.

[MBANZA Sara]

Is this mean that there is no possibility for rodsuser to determinate the free space by using PRC?

[Terrell Russell]

No, you should be able to do what you are attempting.

A rodsadmin is required to run the equivalent of `iadmin cu` to calculate and populate the totals in the catalog.

A rodsuser can query the quota results once they are in the catalog.

Terrell

 

To view this discussion on the web visit https://groups.google.com/d/msgid/irod-chat/fdf08584-2772-4f5a-9bf1-08e2e8c3cf11n%40googlegroups.com.

[MBANZA Sara]

With PRC still does not work

        from irods.models import Quota
        #my_session.groups.calculate_usage()
        group, resource = ['quota_group_for_mbanza', 'total']
        space_left_in_bytes = list(my_session.query(Quota.over).filter(Quota.user_id == my_session.groups.get(group).id, Quota.resc_id == my_session.resources.get(resource).id))[0][Quota.over] * -1
        print(space_left_in_bytes)

Output:
Traceback (most recent call last):
  File "/home/mbanza/myvirtuelenv/lib64/python3.6/site-packages/irods/manager/resource_manager.py", line 29, in get
    result = query.one()
  File "/home/mbanza/myvirtuelenv/lib64/python3.6/site-packages/irods/query.py", line 247, in one
    raise NoResultFound()
irods.exception.NoResultFound

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
    space_left_in_bytes = list(my_session.query(Quota.over).filter(Quota.user_id == my_session.groups.get(group).id, Quota.resc_id ==my_session.resources.get(resource).id))[0][Quota.over] * -1
  File "/home/mbanza/myvirtuelenv/lib64/python3.6/site-packages/irods/manager/resource_manager.py", line 31, in get
    raise ResourceDoesNotExist()
irods.exception.ResourceDoesNotExist
Aborted (core dumped)

With iCommands works as expected

$ iquota | sed -n  '/Group:  quota_group_for_mbanza/, /Over/p'
  Group:  quota_group_for_mbanza
  Zone:  Testzone

  Quota: 20,401,094,656 (20 billion) bytes

  Over:  -17,589,180,588 (-17 billion) bytes (under quota)

[Alan King]

The resource being specified by the variable resource is "total". If there is no resource in your zone named "total", this would explain why irods.exception.ResourceDoesNotExist is being raised. Try providing the name of a resource which does exist. You can see a list of resources with ilsresc or perform a GenQuery to select RESC_NAME.

Based on the iquota usage shown and the attempted use of "total" in the query, it seems we are more interested in the quota overage of the group for the total quota rather than the quota overage for a group on a particular resource. In that case, you can remove the resc_id filter on the query like this:

        from irods.models import Quota
        #my_session.groups.calculate_

usage()
        group = 'quota_group_for_mbanza'
        space_left_in_bytes = list(my_session.query(Quota.over).filter(Quota.user_id == my_session.groups.get(group).id))[0][Quota.over] * -1
        print(space_left_in_bytes)

For completeness, I think you will need to select Quota.limit in your query as well if you want to get the "Quota" piece being shown by iquota.

Let us know if that works any better.

To view this discussion on the web visit https://groups.google.com/d/msgid/irod-chat/71c47310-3f53-4e20-85e3-a0c1834df799n%40googlegroups.com.

--

Alan King

Senior Software Developer | iRODS Consortium

[MBANZA Sara]

Thank you so much. These codes work as expected. The output is 17589180588  (bytes).

KR,

Sara

[Alan King]

Glad to hear it.

To view this discussion on the web visit https://groups.google.com/d/msgid/irod-chat/ca0ccd3b-23ac-4e85-9a9e-b468376eb006n%40googlegroups.com.