trim on a compound resource on 4.2.11

[Jean-Yves Nief]

hello,

            I found an issue while using trimming on compound resources.
I did not see before I hit a new use case.
Let's say we compound resource Comp with a "Cache" resource (type=cache)
and an "Arch" resource (type=archive).
If I have a file "foo" present on both resources like this:
ils -l foo
gives:
.... 0 Comp;Cache ...... & foo
.... 1 Comp;Arch   ....... & foo
A "itrim -N1" does the job and erase the replica 0, hence ending with:
... 1 Comp;Arch   ....... & foo
If "foo" is staged back on disk:
ils -l foo
gives:
.... 1 Comp;Arch   ....... & foo
.... 2 Comp;Cache   ....... & foo
A "itrim -N1" will erase the copy in the archive resource, ending with this:
ils -l foo
.... 2 Comp;Cache   ....... & foo
That is not good at all as itrim does not respect the hierarchy between
the two resources.
The trick would be to use the "-S" option as I used to do on v3 in order
to point to the Cache resource.
But something like:
> itrim -MN1 -S"Comp;Cache" foo
gives:
..... status = -78000 SYS_RESC_DOES_NOT_EXIST
Total size trimmed = 0.000 MB. Number of files trimmed = 0.
Level 0: Specifying a minimum number of replicas to keep is deprecated.
"-S Cache" does not work either.
What is wrong there ?
thanks,
JY

[Alan King]

Hi,

Have you tried using the -n option of itrim to target the replica by number? For your case this would be: itrim -N1 -n2 foo

If this is part of an automated process, you may wish to run a query to determine which replica number is on the cache resource.

The -S option for itrim can only refer to the root resource of a hierarchy, not a leaf or a semicolon-delimited hierarchy. This is to prevent users from bypassing policy configured in the server (although this is a subject of ongoing philosophical importance). I'm guessing that the pre-4.0 iRODS allowed targeting any resource in a "group", and that's why that was allowed? Before my time. :)

Anyway, we have been contemplating reworking itrim for a while and probably will in the 4.3 series. You can find a discussion about this here: https://github.com/irods/irods/issues/4861 Please let us know what you think as we try to chisel out a future for the trim API. Feedback from the community on this matter would be most helpful.

Hope that helps!

--
--
The Integrated Rule-Oriented Data System (iRODS) - https://irods.org

 iROD-Chat:  http://groups.google.com/group/iROD-Chat
---
You received this message because you are subscribed to the Google Groups "iRODS-Chat" group.
To unsubscribe from this group and stop receiving emails from it, send an email to irod-chat+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/irod-chat/74a1daf6-0b7f-8276-8764-1f2cc5c2c01c%40cc.in2p3.fr.

--

Alan King

Senior Software Developer | iRODS Consortium

[Jean-Yves Nief]

hello Alan,

Alan King wrote:
> Hi,
>
> Have you tried using the -n option of itrim to target the replica by
> number? For your case this would be: itrim -N1 -n2 foo

forgot about this option! That will be fine and usually files are being
trimmed through an automated rule, it is just a question of adding
DATA_REPL_NUM in the query I am already using.

> If this is part of an automated process, you may wish to run a query
> to determine which replica number is on the cache resource.
>
> The -S option for itrim can only refer to the root resource of a
> hierarchy, not a leaf or a semicolon-delimited hierarchy. This is to
> prevent users from bypassing policy configured in the server (although
> this is a subject of ongoing philosophical importance). I'm guessing
> that the pre-4.0 iRODS allowed targeting any resource in a "group",
> and that's why that was allowed? Before my time. :)

do you mean I am an old timer ? Yes I am :-).

>
> Anyway, we have been contemplating reworking itrim for a while and
> probably will in the 4.3 series. You can find a discussion about this
> here: https://github.com/irods/irods/issues/4861 Please let us know
> what you think as we try to chisel out a future for the trim API.
> Feedback from the community on this matter would be most helpful.

Good, I will take a look at it.
thanks,
JY

> <mailto:irod-chat%2Bunsu...@googlegroups.com>.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/irod-chat/74a1daf6-0b7f-8276-8764-1f2cc5c2c01c%40cc.in2p3.fr.
>
>
>
> --
> Alan King
> Senior Software Developer | iRODS Consortium

> --
> --
> The Integrated Rule-Oriented Data System (iRODS) - https://irods.org
>
> iROD-Chat: http://groups.google.com/group/iROD-Chat
> ---
> You received this message because you are subscribed to the Google
> Groups "iRODS-Chat" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to irod-chat+...@googlegroups.com

> <mailto:irod-chat+...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/irod-chat/CADnp3x7x7HKwbGaPdgNkvrY_V2aCdmfZA2gu3oNcOz3iQnUqFw%40mail.gmail.com
> <https://groups.google.com/d/msgid/irod-chat/CADnp3x7x7HKwbGaPdgNkvrY_V2aCdmfZA2gu3oNcOz3iQnUqFw%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[Alan King]

Great, I hope that option gets you to where you need to go!