Delete file older than...

[Jean-Baptiste Denis]

Hello everybody,

i'm running OneFS v6.5.5.12.

Is there an "isilon" way to create a job which will delete files older
than X days (or not accessed since) in a specific folder ? I'd prefer
not to rely on a third party script running find -whatever | xargs rm =)

Jean-Baptiste

[Andrew Stack]

Hello,

To the best of my knowledge, Isilon currently does not have a means of producing a file policy that populates a particular directory based on MTime (or any other policy means).  It's something we have asked for in future releases.    

Regards,

Andrew

Jean-Baptiste

--
You received this message because you are subscribed to the Google Groups "Isilon Technical User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isilon-user-gr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Andrew Stack

System Operations

Genentech

Cell - 650.867.5524

[Peter Serocka]

Would you rely on a 1st/2nd/3rd party find-rm script
PLUS Isilon snapshots?

Take a snapshot, run the script which would need to produce
some kind of report; if checked ok,
then delete the snapshot, otherwise restore.

Peter

> --
> You received this message because you are subscribed to the Google Groups "Isilon Technical User Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to isilon-user-gr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

Peter Serocka
CAS-MPG Partner Institute for Computational Biology (PICB)
Shanghai Institutes for Biological Sciences (SIBS)
Chinese Academy of Sciences (CAS)
320 Yue Yang Rd, Shanghai 200031, China
pser...@picb.ac.cn

[Andrew Stack]

Hello,

3rd party software would need to do a crawl or something similar.  Once it has an understanding of your data it could then be used to move subsets of your data based on your query(s) into a TBD (to be deleted) folder.  You could then apply snaps to this directory or not depending on your needs.  The fastest way to delete data is via treedelete.  Use caution however, as this deletes all references of that data from your cluster including what is present in your snaps.  The same logic would apply to  an in house solution.  

I hope this helps.

/Andrew

[LinuxRox]

Andrew,

if directory is protected by a snapshot, treedelete will delete just that directory from the snapshot, even if that snapshot was taken at parent directory ?

[Jean-Baptiste Denis]

Thank you for the answers abd the implications with the snapshots.

You all seem to agree that I have to rely on a 3rd party find script
that will crawl the directory.

In my case, I'm talking about a potential hundred TB of scratch data.
The policy would be : all the files not modified since X days will be
deleted (the scratch directory will NOT be wiped out every 15 days, it
would be too easy).

I would like to use the internal "power" of the Isilon cluster to
perform the crawl and the deletion. If I want my "find-rm" script to
scale (sort of...), I'll have to reinvent the wheel by splitting the
removal of files in chunks etc...

The suggestion to move the data somewhere else to use TreeDelete is fine
with me, but I still have to rely on an external script to perform the
move which is not what I want with that amount of data.

Andrew Stack wrote:

> Isilon currently does not have a means of producing a file policy
> that populates a particular directory based on MTime (or any other
> policy means). It's something we have asked for in future releases.

Oups, I didn't see this answer =) So I don't have a lot of options here !

[Andrew Stack]

I actually tested this and need to retract my statement.  The snaps do keep the data for whatever the snapshot retention policy for that directory is.  TreeDelete is simply a faster way of deleting data.  Sorry for the confusion.