VFAT and ftruncate()
David Durham
larger under a vfat mounted file system, I get errno: 'Operation not
permitted'. I can use it to make a file smaller, so I'm just guessing
that the operation is not implemented for making a file bigger? Any
chance someone is/has work/ed on this? I know its possible because win
9x/NT/2000 has ftruncate and it seems to work.
Thanks,
Davy
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majo...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Petr Vandrovec
> >larger under a vfat mounted file system, I get errno: 'Operation not
> >permitted'. I can use it to make a file smaller, so I'm just guessing
> >I know its possible because win 9x/NT/2000 has ftruncate and it seems
> >to work.
>
> additional blocks of zeroes to fill the added file space?
Sparse file cannot exist on FAT system. Windows will just assign blocks
to file, even without clearing them out.
> By the way, I note that the errno is "operation not permitted." The Linux
> FAT driver seems to like abusing this errno for things that do not make
> sense on a FAT filesystem. It's supposed to mean you need more privilege
> (e.g. superuser status) to do it.
No. EACCES means what you are saying. EPERM means that you cannot do that
under any circumstances. There was some discussion about that on LK
couple of months ago.
Petr Vandrovec
vand...@vc.cvut.cz
Bryan Henderson
>When I try to 'int ftruncate(int fd,off_t length)' to make a file
>permitted'. I can use it to make a file smaller, so I'm just guessing
>that the operation is not implemented for making a file bigger? ...
>I know its possible because win 9x/NT/2000 has ftruncate and it seems
>to work.
I hope that someone who knows DOS/Windows filesystems better can add
something, but it seems unlikely to me that a filesystem as old and simple
as FAT could have sparse files. Looking at the Linux FAT code, I do in
fact see that ftruncate() longer is explicitly prohibited, with a comment
indicating that it has to be that way to prevent sparse files from
happening.
The Linux VFAT code just calls the FAT code for everything except filename
stuff.
When it works on Windows, does it create a sparse file or just assign
additional blocks of zeroes to fill the added file space?
A sparse file, of course, is one where there don't have to be assigned disk
blocks backing every location in the file, so ftruncating a file longer
doesn't involve allocating any additional disk space -- it just changes the
file's filesize attribute.
By the way, I note that the errno is "operation not permitted." The Linux
FAT driver seems to like abusing this errno for things that do not make
sense on a FAT filesystem. It's supposed to mean you need more privilege
(e.g. superuser status) to do it.
David Durham
>>When I try to 'int ftruncate(int fd,off_t length)' to make a file
>>larger under a vfat mounted file system, I get errno: 'Operation not
>>permitted'. I can use it to make a file smaller, so I'm just guessing
>>that the operation is not implemented for making a file bigger? ...
>>I know its possible because win 9x/NT/2000 has ftruncate and it seems
>>to work.
>>
>
>I hope that someone who knows DOS/Windows filesystems better can add
>something, but it seems unlikely to me that a filesystem as old and simple
>as FAT could have sparse files. Looking at the Linux FAT code, I do in
>fact see that ftruncate() longer is explicitly prohibited, with a comment
>indicating that it has to be that way to prevent sparse files from
>happening.
>
>The Linux VFAT code just calls the FAT code for everything except filename
>stuff.
>
>When it works on Windows, does it create a sparse file or just assign
>additional blocks of zeroes to fill the added file space?
>
I'll see how microcruft's implementation behaves when making the file
larger... This code I'm working with I wrote a long time ago under
windows, and it just worked there. I'll see if under windows it takes a
long time to make the file bigger. But even so, this is probably the
case, I propose that we implement linux's FAT implementation to write
zeros to the file... This gives a more consistant behavior across all
the file systems... It would be quite a bump in the road if applications
had to detect if the file system is fat that they're writing to and
write zero's or call ftruncate to make a file larger. Don't you agree?
>
>A sparse file, of course, is one where there don't have to be assigned disk
>blocks backing every location in the file, so ftruncating a file longer
>doesn't involve allocating any additional disk space -- it just changes the
>file's filesize attribute.
>
yap.
>
>By the way, I note that the errno is "operation not permitted." The Linux
>FAT driver seems to like abusing this errno for things that do not make
>sense on a FAT filesystem. It's supposed to mean you need more privilege
>(e.g. superuser status) to do it.
>
Well, it would be more indicative of the problem if there were a
"operation not implement" errno value. Which would seem plausible under
a development kernel, huh?
-- Davy
Bryan Henderson
>I propose that we implement linux's FAT implementation to write
>zeros to the file...
I don't see why not.
>It would be quite a bump in the road if applications
>had to detect if the file system is fat that they're writing to and
>write zero's or call ftruncate to make a file larger.
Well, it's not quite that bad today. Rather than detect that the
filesystem type is FAT, I would try the ftruncate() and, if it failed, try
writing zeroes. In fact, now that I say that, I seem to recall 'dd' being
implemented to do just that. Does dd seek=BIGGERTHANFILE work on FAT?
>Well, it would be more indicative of the problem if there were a
>"operation not implement" errno value. Which would seem plausible under
>a development kernel, huh?
The reason a new Linux release doesn't alleviate compatibility concerns
much here is that it isn't just being compatible with past Linux behavior
that's at issue -- it's implementing POSIX and being compatible with other
contemporary Unix systems. We've sorely needed additional errnos for a
long time and rarely given in and made them.
In this particular case, ENOSYS ("Function not implemented") is just about
perfect, except that traditionally it has been used _only_ for the case
that a whole system call type (e.g. truncate()) isn't implemented. A waste
of an errno, if you ask me.
Bryan Henderson
>No. EACCES means what you are saying. EPERM means that you cannot do that
>under any circumstances. There was some discussion about that on LK
>couple of months ago.
It must have been a difficult dicussion, because "cannot do that under any
circumstances" is extremely vague, due to the ambiguousness of "that."
Exactly what did I try that cannot be done under any circumstances? To
enlarge a file? To enlarge a FAT file? To make a file that big? To
change the size of a file? To change the size of a file while not having
write permission to it?
I may go so far as to believe you that a consensus was developed that Linux
should define EPERM as something unrelated to privilege, but it's
definitely the case that EPERM in the Unix world in general is about
privilege. Even the symbol EPERM and the conventional name "Operation not
permitted" suggest that. In English, "not permitted" means, "I could
easily let you do that, but choose not to." I have seen it in most cases
where something can be done only by the superuser or only by an owner (just
checked - Linux kill() uses it that way).
EACCES (Permission Denied), on the other hand, is typically limited to a
failure to access a file due to file permissions. Hence the name. I see,
however, that at least some superuser-only Linux system calls unrelated to
files fail with EACCES.
I just checked the Single Unix specification, and it defines EACCES and
EPERM this way.
Petr Vandrovec
> >No. EACCES means what you are saying. EPERM means that you cannot do that
> >under any circumstances. There was some discussion about that on LK
> >couple of months ago.
>
> It must have been a difficult dicussion, because "cannot do that under any
> circumstances" is extremely vague, due to the ambiguousness of "that."
> Exactly what did I try that cannot be done under any circumstances? To
> enlarge a file? To enlarge a FAT file? To make a file that big? To
> change the size of a file? To change the size of a file while not having
> write permission to it?
It means that you can repeat extending file on FAT from its current size
to size you passed to ftruncate() again and again, but it will not help,
file will not grow. Someone will have to use write...
> I may go so far as to believe you that a consensus was developed that Linux
> should define EPERM as something unrelated to privilege, but it's
> definitely the case that EPERM in the Unix world in general is about
> privilege. Even the symbol EPERM and the conventional name "Operation not
> permitted" suggest that. In English, "not permitted" means, "I could
> easily let you do that, but choose not to."
Exactly. We could extended file for you, but we decided that instead of
writting zeroes to disk for next couple of hours during one ftruncate()
call we leave decision on you - if you really want to extend file, use
something else (loop in write and paint some progress bar for impatient
user). You can lookup discussion why Al Viro (if my memory serves
correctly) decided to not support extending files on filesystems
which do not support holes - linux-fsdevel archive should contain it
(I think that ~18 months ago, but I'm not sure at all).
Best regards,
Petr Vandrovec
vand...@vc.cvut.cz
Bryan Henderson
>>In English, "not permitted" means, "I could
>> easily let you do that, but choose not to."
>
>Exactly. We could extended file for you, but we decided that instead of
>call we leave decision on you
>which do not support holes
If you're right that the only thing keeping the truncate(LARGER) from
succeeding is policy, then I agree that "not permitted" is a valid English
description of the failure. I had the idea it was more of implementation
difficulty or a belief that the request is nonsensical (as obviously the
originators of the function, who named it "truncate," believed). It still
doesn't match conventional use of EPERM, though, which is limited to issues
of privilege.
And I don't think one can defend the use of EPERM for other attribute
change failures in the FAT driver. The reason you can't update the uid on
a FAT file is not an issue of someone permitting you to do it -- it has to
do with the fact that a FAT file has no concept of an owner.
David Durham
>>I propose that we implement linux's FAT implementation to write
>>zeros to the file...
>>
>
>I don't see why not.
>
>>It would be quite a bump in the road if applications
>>had to detect if the file system is fat that they're writing to and
>>write zero's or call ftruncate to make a file larger.
>>
>
>Well, it's not quite that bad today. Rather than detect that the
>filesystem type is FAT, I would try the ftruncate() and, if it failed, try
>writing zeroes. In fact, now that I say that, I seem to recall 'dd' being
>implemented to do just that. Does dd seek=BIGGERTHANFILE work on FAT?
>
I know the CPU overhead isn't that much... But I do an ftruncate more
than occationally so I'm trying something, seeing if it fails, then
trying something else even tho it's going to fail every time. Now I
could have a flag that I set so I don't try it again, but then I'm doing
more effort just to work around some inconsistancy which is easliy
fixable in the kernel. And the fixed kernel would benifit everyone, not
just me, which is why I even went to the effort of starting this thread...