[Discuss] disk space analysis tools

[Jerry Feldman]

Since my background is a software engineer not a system administrator, I
have generally used DU to manage my own disk space, or possibly to look
at disk space usage on the BLU servers. But, recently I'm seeing
significant increase in my work's backup server (a WD MyBook). While du
works fine (using the appropriate options), it does take some time to
analyze the results. One issue I have is whether the increase in space
is due to someone moving things around (such as in a client space) that
will break the hard links, but eventually the storage will go down as
the older dailies are removed. I only keep a few weeks because out New
York office backs us up also.

So, what I am looking for is either some decent tools I can use that are
less time consuming than looking at du output.

--
Jerry Feldman <g...@blu.org>
Boston Linux and Unix
PGP key id:3BC1EB90
PGP Key fingerprint: 49E2 C52A FC5A A31F 8D66 C0AF 7CEA 30FC 3BC1 EB90

_______________________________________________
Discuss mailing list
Dis...@blu.org
http://lists.blu.org/mailman/listinfo/discuss

[David Miller]

On Fri, Oct 7, 2011 at 11:47 AM, Jerry Feldman <g...@gapps.blu.org> wrote:

> Since my background is a software engineer not a system administrator, I
> have generally used DU to manage my own disk space, or possibly to look
> at disk space usage on the BLU servers. But, recently I'm seeing
> significant increase in my work's backup server (a WD MyBook). While du
> works fine (using the appropriate options), it does take some time to
> analyze the results. One issue I have is whether the increase in space
> is due to someone moving things around (such as in a client space) that
> will break the hard links, but eventually the storage will go down as
> the older dailies are removed. I only keep a few weeks because out New
> York office backs us up also.
>
> So, what I am looking for is either some decent tools I can use that are
> less time consuming than looking at du output.

Have a look at Baobab. Its a graphical disk space analyzer that works on
local and remote disks.
--
David

[Matthew Gillen]

On 10/07/2011 11:47 AM, Jerry Feldman wrote:
> Since my background is a software engineer not a system administrator, I
> have generally used DU to manage my own disk space, or possibly to look
> at disk space usage on the BLU servers. But, recently I'm seeing
> significant increase in my work's backup server (a WD MyBook). While du
> works fine (using the appropriate options), it does take some time to
> analyze the results. One issue I have is whether the increase in space
> is due to someone moving things around (such as in a client space) that
> will break the hard links, but eventually the storage will go down as
> the older dailies are removed. I only keep a few weeks because out New
> York office backs us up also.
>
> So, what I am looking for is either some decent tools I can use that are
> less time consuming than looking at du output.
>

Ever tried filelight? It's a pretty cool visualization of disk usage.
There's a gnome-equivalent, I can't think of the name...

There's also 'agedu'. From the man page:
agedu - correlate disk usage with last-access times to identify large
and disused data

Matt

[Matthew Gillen]

On 10/07/2011 11:51 AM, Matthew Gillen wrote:
> Ever tried filelight? It's a pretty cool visualization of disk usage.
> There's a gnome-equivalent, I can't think of the name...

David Miller knew the gnome version :-)

On 10/07/2011 11:50 AM, David Miller wrote:
> Have a look at Baobab. Its a graphical disk space analyzer that works
> on local and remote disks.

_______________________________________________

[Jerry Feldman]

On 10/07/2011 11:50 AM, David Miller wrote:

> On Fri, Oct 7, 2011 at 11:47 AM, Jerry Feldman <g...@gapps.blu.org
> <mailto:g...@gapps.blu.org>> wrote:
>
> Since my background is a software engineer not a system administrator, I
> have generally used DU to manage my own disk space, or possibly to look
> at disk space usage on the BLU servers. But, recently I'm seeing
> significant increase in my work's backup server (a WD MyBook). While du
> works fine (using the appropriate options), it does take some time to
> analyze the results. One issue I have is whether the increase in space
> is due to someone moving things around (such as in a client space) that
> will break the hard links, but eventually the storage will go down as
> the older dailies are removed. I only keep a few weeks because out New
> York office backs us up also.
>
> So, what I am looking for is either some decent tools I can use that are
> less time consuming than looking at du output.
>
>
> Have a look at Baobab. Its a graphical disk space analyzer that works
> on local and remote disks.

On 10/07/2011 11:51 AM, Matthew Gillen wrote:

> Ever tried filelight? It's a pretty cool visualization of disk usage.
> There's a gnome-equivalent, I can't think of the name...

Thanks guys. I'll take a look at them and see if they work on the
MyBook, but I also could use it on out servers.

--
Jerry Feldman <g...@blu.org>
Boston Linux and Unix
PGP key id:3BC1EB90
PGP Key fingerprint: 49E2 C52A FC5A A31F 8D66 C0AF 7CEA 30FC 3BC1 EB90

[Ben Eisenbraun]

On Fri, Oct 07, 2011 at 11:47:07AM -0400, Jerry Feldman wrote:
> So, what I am looking for is either some decent tools I can use that are
> less time consuming than looking at du output.

This article lists 6 tools in the Baobab, Filelight, etc, family:

http://www.makeuseof.com/tag/how-to-analyze-your-disk-usage-pattern-in-linux/

I use du2rrd for watching disk usage trends on some of my file systems:

http://oss.oetiker.ch/optools/wiki/du2rrd

-ben

--
the older i get, the more life starts to make sense, and the less i care.
<buck 65>

[Matthew Gillen]

On 10/07/2011 11:51 AM, Matthew Gillen wrote:

> There's also 'agedu'. From the man page:
> agedu - correlate disk usage with last-access times to identify large
> and disused data

This is a cool one: ncdu
It's like filelight, but ncurses-based :-)

I love ncurses programs.

[David Rosenstrauch]

On 10/07/2011 11:51 AM, Matthew Gillen wrote:

> Ever tried filelight? It's a pretty cool visualization of disk usage.
> There's a gnome-equivalent, I can't think of the name...

xdiskusage?

DR

[Ian Stokes-Rees]

And a few more:

multi-platform GUI-based:
jDiskReport for a java-based disk analyzer.

I've also written my own tool for doing this (single Python script):
https://github.com/ijstokes/duscan

This had the advantage of being runnable from a cron job and the
results were always available to me (incl clickable HTML and pie
charts). It has the disadvantage of (potentially) creating O(N)
additional small files, one per directory, if you choose to persist the
disk usage summaries. A sqlite (or similar) version would be a nice
improvement! Anyway, it accumulated disk usage by user and group, and
kept a list of big files. These were (and are) things that are
relevant to me with a multi-TB multi-user system.

And while we're on the topic, but for OS X:
Disk Inventory X for OS X

Ian

[Jerry Feldman]

This might be useful for me here. essentially, our NAS system is only at
about 25%, but our backup server is the one that I need to keep an eye
on. However, now that we are now owned by a cloud company (eg. IBM)
maybe we will migrate to the cloud :-)

On 10/11/2011 02:08 PM, Ian Stokes-Rees wrote:
> And a few more:
>
> multi-platform GUI-based:
> jDiskReport for a java-based disk analyzer.
>
> I've also written my own tool for doing this (single Python script):
> https://github.com/ijstokes/duscan
>
> This had the advantage of being runnable from a cron job and the
> results were always available to me (incl clickable HTML and pie
> charts). It has the disadvantage of (potentially) creating O(N)
> additional small files, one per directory, if you choose to persist the
> disk usage summaries. A sqlite (or similar) version would be a nice
> improvement! Anyway, it accumulated disk usage by user and group, and
> kept a list of big files. These were (and are) things that are
> relevant to me with a multi-TB multi-user system.
>
> And while we're on the topic, but for OS X:
> Disk Inventory X for OS X
>
>

--
Jerry Feldman<g...@blu.org>
Boston Linux and Unix
PGP key id:3BC1EB90
PGP Key fingerprint: 49E2 C52A FC5A A31F 8D66 C0AF 7CEA 30FC 3BC1 EB90

_______________________________________________