Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Can't delete any files on my filled up ZFS pool

3 views
Skip to first unread message

Thomas Vogt

unread,
Jan 21, 2008, 5:02:29 PM1/21/08
to
Hello

I need help. My ZFS sytem is filled up. I can't delete any files.

root@bert:/tank# rm input.wav
rm: input.wav: No space left on device

ZFS:
root@bert:/tank# zfs list tank
NAME USED AVAIL REFER MOUNTPOINT
tank 109G 0 4.11G /tank

mount:
tank 4.1G 4.1G 0B 100% /tank

FreeBSD 7.0-RC1 #11: Wed Jan 16 04:23:37 CET 2008
ro...@bert.mlan.solnet.ch:/usr/obj/usr/src/sys/BERT i386

Any idea?

Best Regards,
Thomas

_______________________________________________
freebsd...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-curre...@freebsd.org"

Julian H. Stacey

unread,
Jan 21, 2008, 5:24:35 PM1/21/08
to
Thomas Vogt wrote:
> Hello
>
> I need help. My ZFS sytem is filled up. I can't delete any files.
>
> root@bert:/tank# rm input.wav
> rm: input.wav: No space left on device

I know nothing about ZFS :-) (Well nearly, just reading the ZFS pain
on @freebsd lists is enough to scare me off for now ;-) ) But if I
was stuck on this, with no ZFS experts to quickly ask, I'd guess & try:

It needs more space for another Inode, or extended directory
entry, cos its maybe going to create another inode in a
backup/ deleted entity first, so either:

A)
Maybe su ; rm input.wav # if the concept of extra space still exists
# per "tunefs -m" for root as per UFS etc.
Or B)
Perhaps more likely:
truncate existing inode to create some space
before deleting it:
cat /dev/null > input.wav ; rm input.wav

Presumably if you filled it as root, B might still empty it.

There will doubtless be better ZFS answers, but could be interesting
to hear if either of above could work.

--
Julian Stacey. Munich Computer Consultant, BSD Unix C Linux. http://berklix.com

Thomas Vogt

unread,
Jan 21, 2008, 5:45:10 PM1/21/08
to
Hello Julian

Julian H. Stacey wrote:
> Thomas Vogt wrote:
>> Hello
>>
>> I need help. My ZFS sytem is filled up. I can't delete any files.
>>
>> root@bert:/tank# rm input.wav
>> rm: input.wav: No space left on device
>
> I know nothing about ZFS :-) (Well nearly, just reading the ZFS pain
> on @freebsd lists is enough to scare me off for now ;-) ) But if I
> was stuck on this, with no ZFS experts to quickly ask, I'd guess & try:
>
> It needs more space for another Inode, or extended directory
> entry, cos its maybe going to create another inode in a
> backup/ deleted entity first, so either:
>
> A)
> Maybe su ; rm input.wav # if the concept of extra space still exists
> # per "tunefs -m" for root as per UFS etc.

I filled it as root. So it does not work

> Or B)
> Perhaps more likely:
> truncate existing inode to create some space
> before deleting it:
> cat /dev/null > input.wav ; rm input.wav

Nice. B) works fine. Thank you.


> Presumably if you filled it as root, B might still empty it.
>
> There will doubtless be better ZFS answers, but could be interesting
> to hear if either of above could work.

I hope there will be a "ZFS" answer :)

Regards,
Thomas

Bill

unread,
Jan 21, 2008, 6:21:59 PM1/21/08
to
Do you have snapshots on the pool? What is the output from 'zfs list'?

It's possible when you have a snapshot on tank that the delete causes a
copy-on-write for the snapshot that then doesn't have enough space.

Bill

Thomas Vogt

unread,
Jan 21, 2008, 6:31:28 PM1/21/08
to
Hello

Bill wrote:
> Do you have snapshots on the pool? What is the output from 'zfs list'?
> It's possible when you have a snapshot on tank that the delete causes a
> copy-on-write for the snapshot that then doesn't have enough space.

I don't use snapshots.

zfs list


NAME USED AVAIL REFER MOUNTPOINT

tank 104G 5.70G 4.11G /tank
tank/wav 99.5G 5.70G 99.5G /wav

Cheers,
Thomas

Bill

unread,
Jan 21, 2008, 6:51:01 PM1/21/08
to
Does the delete work now?

In your original post, tank was:

NAME USED AVAIL REFER MOUNTPOINT

tank 109G 0 4.11G /tank

and now:

NAME USED AVAIL REFER MOUNTPOINT
tank 104G 5.70G 4.11G /tank

Is it possible a process was writing to /wav, filled the disk, you tried
your 'rm /tank/input.wav' and then the original process writing to /wav
unlinked the file that didn't fit, thus freeing 5.7G?

/tank and /wav share the available free space on tank. If that ends up
being the problem, you can set the reservation option on tank to prevent
it from happening again. If it's not that I'm not sure what it could
be, I'm just trying to point out 'quirky' behavior from ZFS that's
different from UFS and the like, do to its pooling nature.

Bill

Thomas Vogt

unread,
Jan 21, 2008, 7:12:06 PM1/21/08
to
Hello Bill

Bill wrote:
> Does the delete work now?

Sorry for not mentioning. Yes delete work with 'cat /dev/null >
input.wav ; rm input.wav'. Simple use of rm didn't work

>
> In your original post, tank was:
>
> NAME USED AVAIL REFER MOUNTPOINT
> tank 109G 0 4.11G /tank

Before Julians hint with cat /dev/null

>
> and now:
>
> NAME USED AVAIL REFER MOUNTPOINT
> tank 104G 5.70G 4.11G /tank

After Julians hint.


> Is it possible a process was writing to /wav, filled the disk, you tried
> your 'rm /tank/input.wav' and then the original process writing to /wav
> unlinked the file that didn't fit, thus freeing 5.7G?

Unlikely. This pool is only used for storing data. No writing from any
the system.

>
> /tank and /wav share the available free space on tank. If that ends up
> being the problem, you can set the reservation option on tank to prevent
> it from happening again. If it's not that I'm not sure what it could
> be, I'm just trying to point out 'quirky' behavior from ZFS that's
> different from UFS and the like, do to its pooling nature.

You're right. Maybe it's better to set reservation for zfs too.

Peter Schuller

unread,
Jan 22, 2008, 11:00:00 AM1/22/08
to

--xHFwDpU9dbj6ez1V
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

>> There will doubtless be better ZFS answers, but could be interesting
>> to hear if either of above could work.

>=20


> I hope there will be a "ZFS" answer :)

It used to be that this situation could arise even without snapshots
or such "mitigating" circumstances. I'm pretty sure this was fixed,
such that you would not get into a catch-22 under such
circumstances. However I don't have the ML post I read this in, and I
do not know whether the fix has been integrated into FreeBSD (though
this was a while ago, so I would have guess it had).

--
/ Peter Schuller

PGP userID: 0xE9758B7D or 'Peter Schuller <peter.s...@infidyne.com>'
Key retrieval: Send an E-Mail to getp...@scode.org
E-Mail: peter.s...@infidyne.com Web: http://www.scode.org


--xHFwDpU9dbj6ez1V
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.4 (FreeBSD)

iD8DBQFHlg7+DNor2+l1i30RAuK3AKCMyZqJWRMA/REig5GDeMH17ngoWQCghmh6
DiPo65pBVrvctQlLRgXD5kU=
=NLfh
-----END PGP SIGNATURE-----

--xHFwDpU9dbj6ez1V--

0 new messages