Permissions on an isilon

[len zaifman]

On the isilon file server i want to move files from one directory tree to another

so

i have  /ifs/tree1/pilot

ls -ld pilot pilot/*

drwxr-xr-x    2 1114  1100  26 Dec  6  2012 pilot

-rw-r--r--    1 1114  1100  66 Nov  1  2012 pilot/trio.txt

I want to move pilot from this tree to tree2

mv pilot /ifs/tree2/

mv: rename pilot to /ifs/tree2/pilot: Operation not permitted

I am root on the isilon

there are no acls, we only use nfs3 , and besides, i am on the isilon cluster,so I think only standard unix permissions are involved.

Any idea what is going on?

I see no error in /var/log/messages

[Jerry Uanino]

the mv is going to cause an rm as well on the source location. if there is a quota on that source location, this will fail. double check and see if you have a quota on that source. remove it.

--
You received this message because you are subscribed to the Google Groups "Isilon Technical User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isilon-user-gr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Saker Klippsten]

Do you have a quota on there?

[Chris Pepper]

I hate this stupid "no 'mv' under quotas" bug -- it interferes with using the cluster so frequently... I hoped it was fixed in ONeFS 7 -- is Isilon ever going to fix it?

Chris

[Luc Simard]

This is not a bug, you can't move data between quota domains

Luc Simard - 415-793-0989
simard.j...@gmail.com
Messages may contain confidential information.
Sent from my iPhone

[Peter Serocka]

It makes a good running gag, but it's hardly a bug.

A "mv" within the same filesystem is expected to behave more or less
atomically -- unlink+link on the toplevel, while all entries (inodes/
LINs) on the subtree keep their identities.

Moving between different filesystem is perfomed by copying and
removing *afterwards*, so there is always the chance to roll back.
Thus, temporarily the space is consumed on both source and target
filesystems. One wouldn't probably expect this when moving within an
one instance of OneFS. And "mv" wouldn't be able to detect that
situation, in particular when run on a client. (Counter check: mount
source and target as different shares; client "mv" will fall back to
copy/remove then.)

In principle, Isilon could provide a local tool similar to TreeDelete.
It would need to create a temporary quota domain for the removed
subtree, and those quota values would need to be subtracted/added at
the enclosing domains simultaneously to unlinking/linking the subtree.

Peter

[J. Lasser]

There is also a sysctl that can be set which, when an NFS client
attempts a move across quota domains, will let the client know to
perform the cp-and-rm operation. Presuming your NFS client supports
this, it's transparent to the end-user.

This is documented in KB article emc14000096, titled "Moving directory
trees across quota domains."

Jon

--
Jon Lasser j...@lasser.org 206-326-0614
. . . and the walls became the world all around . . . (Maurice Sendak)

[Trinh Tran]

You properly have quotas on.  You cannot move data from out quota boundary to another.

Trinh

--
You received this message because you are subscribed to the Google Groups "Isilon Technical User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isilon-user-gr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Thanks,
Trinh Tran
Enterprise Storage Services

[Peter Serocka]

Cool, thanks a lot Jon!

Peter

[len zaifman]

[len zaifman]

Not a possibility as far as I can see.

isi quota quotas view /ifs/tree2  directory

                       Path: /ifs/tree2

                       Type: directory

                  Snapshots: Yes

Thresholds Include Overhead: Yes

                      Usage

                With Overhead : 4.4693T

                 W/O Overhead : 2.9683T

                   Enforced: Yes

                  Container: Yes

                     Linked: -

                 Thresholds

               Hard Threshold : 10.0000T

          Advisory threshold. : 9.0000T

               Soft Threshold : 9.5000T

                   Soft Grace : 2W

du -sk pilot

28      pilot

Just to verify i removed quotas on the destination and tried the mv , same result

isi quota quotas delete /ifs/tree2 directory

Are you sure? (yes/no): yes

 mv pilot /ifs/tree2

mv: rename pilot to /ifs/tree2/pilot: Operation not permitted

So i remain puzzled.

[Peter Serocka]

Both source and target quotas affect this...

Peter

[Chris Pepper]

It is a bug, but you can call it a design flaw if you prefer. Qusota should not break the 'mv' operation. Quotas do not define separate filesystems, but acting like they do loses much of the benefit of the "single coherent global filesystem" which Isilon (rightly) makes so much fuss over.

No actual data copying or removal is necessary, but I understand the quota calculations could be substantial. In that case I would much prefer a slow 'mv' (compared to the normal instantaneous behavior) to working around its unavailability with cp/rsync + "rm -Rf".

Chris

On Jun 12, 2013, at 12:32 PM, Peter Serocka <pser...@picb.ac.cn> wrote:

> It makes a good running gag, but it's hardly a bug.
>

> A "mv" within the same filesystem is expected to behave more or less atomically -- unlink+link on the toplevel, while all entries (inodes/LINs) on the subtree keep their identities.

[Peter Serocka]

On Thu 13 Jun '13 md, at 01:16 st, Chris Pepper wrote:

> It is a bug, but you can call it a design flaw if you prefer.
> Qusota should not break the 'mv' operation. Quotas do not define
> separate filesystems, but acting like they do loses much of the
> benefit of the "single coherent global filesystem" which Isilon
> (rightly) makes so much fuss over.
>
> No actual data copying or removal is necessary, but I understand
> the quota calculations could be substantial.

Let's call it a dilemma. Or a flawed expectation on the behavior of a
"mv".

Peter

[J. Lasser]

The 'slow mv' is available with the sysctl I provided in my previous message.

> --
> You received this message because you are subscribed to the Google Groups "Isilon Technical User Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to isilon-user-gr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Chris Pepper]

Jon,

I do large moves on the cluster myself, not over NFS, and wouldn't want our CentOS clients to drop ACLs.

Thanks,

Chris

[Peter Serocka]

A (new?) workaround
which is more efficient than copying and removing
is *linking* and removing:

find /path/to/source | cpio -pl /path/to/parent-of-target

then check everything,
remove any quotas present on source level,
finally treedelete /path/to/source