What's taking up all my disk space?

[Murk Fletcher]

Hi!

Woke up to a nasty surprise this morning:

/: write failed, filesystem is full
# df -h
Filesystem Size Used Avail Capacity Mounted on
/dev/gpt/rootfs 38G 35G -7.4M 100% /
devfs 1.0K 1.0K 0B 100% /dev
fdescfs 1.0K 1.0K 0B 100% /dev/fd
linprocfs 4.0K 4.0K 0B 100% /usr/compat/linux/proc

I have no idea why this is because I'm only using my FreeBSD VPS to run a
small Rails app in `/usr/home/`:

# du -sh /usr/home
8.6G /usr/home
# du -sh /usr/
12G /usr/
# du -sh /
34G /

Maybe there's a way to use `du` to show all files larger than 1GB and then
pass it on to some other command to sort them by size?

Thanks!

Murk
_______________________________________________
freebsd-...@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questi...@freebsd.org"

[Anton Sayetsky]

26 янв. 2016 г. 12:21 пользователь "Murk Fletcher" <murk.f...@gmail.com>
написал:

>
> Hi!
>
> Woke up to a nasty surprise this morning:
>
> /: write failed, filesystem is full
> # df -h
> Filesystem Size Used Avail Capacity Mounted on
> /dev/gpt/rootfs 38G 35G -7.4M 100% /
> devfs 1.0K 1.0K 0B 100% /dev
> fdescfs 1.0K 1.0K 0B 100% /dev/fd
> linprocfs 4.0K 4.0K 0B 100% /usr/compat/linux/proc
>
> I have no idea why this is because I'm only using my FreeBSD VPS to run a
> small Rails app in `/usr/home/`:
>
> # du -sh /usr/home
> 8.6G /usr/home
> # du -sh /usr/
> 12G /usr/
> # du -sh /
> 34G /
>
> Maybe there's a way to use `du` to show all files larger than 1GB and then
> pass it on to some other command to sort them by size?

Try "du -sh /*" first.

[Michael Schuster]

On Tue, Jan 26, 2016 at 11:21 AM, Murk Fletcher <murk.f...@gmail.com>
wrote:

> devfs 1.0K 1.0K 0B 100% /dev
> fdescfs 1.0K 1.0K 0B 100% /dev/fd
> linprocfs 4.0K 4.0K 0B 100% /usr/compat/linux/proc
>

these are pseudo file systems and don't count against your free disk space

HTH

--
Michael Schuster
http://recursiveramblings.wordpress.com/

[Polytropon]

On Tue, 26 Jan 2016 11:21:25 +0100, Murk Fletcher wrote:
> Hi!
>
> Woke up to a nasty surprise this morning:
>
> /: write failed, filesystem is full
> # df -h
> Filesystem Size Used Avail Capacity Mounted on
> /dev/gpt/rootfs 38G 35G -7.4M 100% /
> devfs 1.0K 1.0K 0B 100% /dev
> fdescfs 1.0K 1.0K 0B 100% /dev/fd
> linprocfs 4.0K 4.0K 0B 100% /usr/compat/linux/proc
>
> I have no idea why this is because I'm only using my FreeBSD VPS to run a
> small Rails app in `/usr/home/`:
>
> # du -sh /usr/home
> 8.6G /usr/home
> # du -sh /usr/
> 12G /usr/
> # du -sh /
> 34G /
>
> Maybe there's a way to use `du` to show all files larger than 1GB and then
> pass it on to some other command to sort them by size?

First determine which top-level directory entry is abnormally
huge:

# du -hs /*

Then narrow down the problem. I suspect /var or /tmp...




--
Polytropon
Magdeburg, Germany
Happy FreeBSD user since 4.0
Andra moi ennepe, Mousa, ...

[Matthew Seaman]

Another good trick is to find large files that have been modified recently:

find / -xdev -type f -mtime -1 -size +10M -ls

will show you all the files greater than 10MiB modified within the last
day on the root partition.

Your du(1) and df(1) outputs are in pretty good agreement as to the
amount of space used, so you probably aren't suffering from the effects
of a process holding an open file descriptor on a file that has been
deleted from the filesystem. That still uses disk space, which shows up
in the df(1) results, but because it's a file without a filname
associated with it, du(1) wouldn't be able to count it. That's something
you can frequently get if you use newsyslog to cycle log files but don't
signal the right process to re-open all its logfiles.

Cheers,

Matthew

[Arthur Chance]

On 26/01/2016 10:23, Anton Sayetsky wrote:
> 26 янв. 2016 г. 12:21 пользователь "Murk Fletcher" <murk.f...@gmail.com>
> написал:
>>
>> Hi!
>>
>> Woke up to a nasty surprise this morning:
>>
>> /: write failed, filesystem is full
>> # df -h
>> Filesystem Size Used Avail Capacity Mounted on
>> /dev/gpt/rootfs 38G 35G -7.4M 100% /
>> devfs 1.0K 1.0K 0B 100% /dev
>> fdescfs 1.0K 1.0K 0B 100% /dev/fd
>> linprocfs 4.0K 4.0K 0B 100% /usr/compat/linux/proc
>>
>> I have no idea why this is because I'm only using my FreeBSD VPS to run a
>> small Rails app in `/usr/home/`:
>>
>> # du -sh /usr/home
>> 8.6G /usr/home
>> # du -sh /usr/
>> 12G /usr/
>> # du -sh /
>> 34G /
>>
>> Maybe there's a way to use `du` to show all files larger than 1GB and then
>> pass it on to some other command to sort them by size?
> Try "du -sh /*" first.

There aren't so many sub-directories in / that it's difficult to spot
the largest but

du -sh /* | sort -rh

will order the list from largest to smallest. Very useful (possibly with
head added to the pipeline) if you've got a lot of subdirectories.

--
Moore's Law of Mad Science: Every eighteen months, the minimum IQ
necessary to destroy the world drops by one point.

[Polytropon]

And in case you want a more detailed list:

# ls -laFG -D "%Y-%m-%d %H:%M:%S" / | sort -r -g +4 | less

The largest files will be on top. Use /var or /tmp instead of / when
you want to have a listing for a specific subtree.

--
Polytropon
Magdeburg, Germany
Happy FreeBSD user since 4.0
Andra moi ennepe, Mousa, ...

[Trond Endrestøl]

On Tue, 26 Jan 2016 11:21+0100, Murk Fletcher wrote:

> Maybe there's a way to use `du` to show all files larger than 1GB and then
> pass it on to some other command to sort them by size?

Have a look at sysutils/ncdu for a ncurses-based du utility.

--
+-------------------------------+------------------------------------+
| Vennlig hilsen, | Best regards, |
| Trond Endrestøl, | Trond Endrestøl, |
| IT-ansvarlig, | System administrator, |
| Fagskolen Innlandet, | Gjøvik Technical College, Norway, |
| tlf. mob. 952 62 567, | Cellular...: +47 952 62 567, |
| sentralbord 61 14 54 00. | Switchboard: +47 61 14 54 00. |
+-------------------------------+------------------------------------+

[Murk Fletcher]

> du -sh /* | sort -rh

Now _that_ is a thing of beauty. This one goes straight to my notes.

Cheers!

[Murk Fletcher]

Thanks again everyone :)

[RW via freebsd-questions]

On Tue, 26 Jan 2016 12:47:14 +0100
Murk Fletcher wrote:

> > du -sh /* | sort -rh
>
> Now _that_ is a thing of beauty. This one goes straight to my notes.

Just bear in mind that * wont match a hidden directory, using -hd1
without the wildcard is more general.

[Esa Karkkainen]

On Tue, Jan 26, 2016 at 11:21:25AM +0100, Murk Fletcher wrote:
> Hi!

Hi,

> Maybe there's a way to use `du` to show all files larger than 1GB
> and then pass it on to some other command to sort them by size?

This shows you 20 largest directories and files over 1GB in size.

# du -axk / | awk '$1 > 2^20 {print}' | sort -rn | head -20

> Thanks!

You're welcome.

Regards,

Esa

--
"In the beginning the Universe was created. This has made a lot of
people very angry and been widely regarded as a bad move."
-- Douglas Adams 1952 - 2001

[Polytropon]

On Tue, 26 Jan 2016 20:05:29 +0200, Esa Karkkainen wrote:
> This shows you 20 largest directories and files over 1GB in size.
>
> # du -axk / | awk '$1 > 2^20 {print}' | sort -rn | head -20

Nice use of awk! Note that you can leave out "{print}" because
this is the default action:

# du -axk / | awk '$1 > 2^20' | sort -rn | head -20

THe condition "$1 > 2^20" is sufficient here.

--
Polytropon
Magdeburg, Germany
Happy FreeBSD user since 4.0
Andra moi ennepe, Mousa, ...

[Esa Karkkainen]

On Wed, Jan 27, 2016 at 08:17:51AM +0100, Polytropon wrote:
> On Tue, 26 Jan 2016 20:05:29 +0200, Esa Karkkainen wrote:
> > # du -axk / | awk '$1 > 2^20 {print}' | sort -rn | head -20
>
> Nice use of awk! Note that you can leave out "{print}" because
> this is the default action:

Force of habit, I'm used to writing one liners that work, or should
work, across multiple unix-like operating systems and versions.

For example I still use "find blah -print", even "-print" is the default
action on most unix-like operating systems find.

IIRC that one liner above, should work on just about any unix-like
OS I've come across, excluding some or all versions of Solaris, because
Solaris "du" doesn't have "-x" option.

Esa

--
"In the beginning the Universe was created. This has made a lot of
people very angry and been widely regarded as a bad move."
-- Douglas Adams 1952 - 2001