Partition /var/lib/mysql too full

[Daniel Petty]

Hi Copressers,

I've got a server here with MYSQL stopped that I can't now restart because I believe its too full.

When I run the command to start mysql, this is what I get:

urcollegian@thecolm1:/$ /etc/init.d/mysql start
 * /etc/init.d/mysql: ERROR: The partition with /var/lib/mysql is too full!

After some Google searching, I also figured this out:

urcollegian@thecolm1:/$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1              19G   19G     0 100% /
udev                   10M  128K  9.9M   2% /dev
none                  249M     0  249M   0% /dev/shm
none                  249M   52K  249M   1% /var/run
none                  249M     0  249M   0% /var/lock
none                  249M     0  249M   0% /lib/init/rw

So the top directory is full, and I believe the reason I can't restart mysql is because bin logs have not (read: never) being cleared. I may be incorrect, but if I'm right, I'm not sure how to do that (and I'm terrified of wiping the entire database). I tried to log into mysql, but got this:

urcollegian@thecolm1:/$ mysql -u root -p
Enter password:
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)

Any help any of you could provide would be much appreciated.

Thank you!

Dan

[Daniel Bachhuber]

Have you tried deleting your log files? 

--
You received this message because you are a part of CoPress (http://www.copress.org/).
- To post a message to this group, send email to cop...@googlegroups.com
- To unsubscribe from this group, send an email to copress+u...@googlegroups.com
- For more options, visit this group at http://groups.google.com/group/copress
- Get connected on Twitter http://www.twitter.com/copress or Facebook http://www.facebook.com/copress
 
http://www.copress.org/

[Daniel Bachhuber]

Run "du -a -h /var/log" and let me know what you get

On Apr 6, 2011, at 6:21 AM, Daniel Petty <dan.s...@gmail.com> wrote:

--

[Albert Sun]

Where are you hosted? Simplest fix might be to just get your hosting provider to increase the size of your partition.

--
Albert Sun
alb...@albertsun.info
215.253.8566

[Daniel Petty]

Here's what that command produced. We're hosted on slicehost, and we've used 52.98 GB of 300 GB. I don't have the correct credentials for root access, it seems, so some of these mysql and apache logs are being denied.

urcollegian@thecolm1:~$ du -a -h /var/log
4.0K    /var/log/mail.warn.4.gz
292K    /var/log/auth.log.4.gz
16K    /var/log/lastlog
4.0K    /var/log/btmp.1.gz
0    /var/log/mysql.log
4.0K    /var/log/mysql.log.2.gz
56K    /var/log/debug.2.gz
364K    /var/log/mail.info.3.gz
344K    /var/log/mail.info.4.gz
20K    /var/log/udev
0    /var/log/apt/term.log
4.0K    /var/log/apt/term.log.1.gz
4.0K    /var/log/apt/term.log.2.gz
4.0K    /var/log/apt/term.log.3.gz
12K    /var/log/apt/term.log.4.gz
28K    /var/log/apt
128K    /var/log/mail.log
4.0K    /var/log/dmesg.2.gz
0    /var/log/dpkg.log
4.0K    /var/log/mysql.log.4.gz
876K    /var/log/kern.log.1
256K    /var/log/syslog.1
4.0K    /var/log/mysql.log.7.gz
12K    /var/log/dmesg.0
16K    /var/log/dpkg.log.3.gz
4.0K    /var/log/mail.warn.2.gz
4.0K    /var/log/mysql.log.3.gz
8.0K    /var/log/faillog
588K    /var/log/mail.log.2.gz
4.0K    /var/log/ntpstats
4.0K    /var/log/mysql.log.1.gz
4.0K    /var/log/aptitude.1.gz
0    /var/log/mail.err
4.0K    /var/log/dpkg.log.2.gz
0    /var/log/boot
40K    /var/log/debug.4.gz
0    /var/log/news/news.notice
0    /var/log/news/news.err
0    /var/log/news/news.crit
4.0K    /var/log/news
224K    /var/log/kern.log
2.4M    /var/log/mail.info.1
4.0K    /var/log/messages.1
5.1M    /var/log/auth.log.1
8.0K    /var/log/dmesg.1.gz
4.0K    /var/log/messages.3.gz
56K    /var/log/kern.log.2.gz
4.0K    /var/log/dpkg.log.1
56K    /var/log/debug.3.gz
64K    /var/log/syslog.7.gz
872K    /var/log/debug.1
12K    /var/log/dmesg
0    /var/log/lpr.log
4.0K    /var/log/vsftpd.log.1.gz
364K    /var/log/mail.log.3.gz
0    /var/log/pycentral.log
8.0K    /var/log/nginx/error.log.27.gz
8.0K    /var/log/nginx/error.log.28.gz
8.0K    /var/log/nginx/error.log.51.gz
12K    /var/log/nginx/error.log.8.gz
4.0K    /var/log/nginx/access.log.44.gz
8.0K    /var/log/nginx/error.log.20.gz
12K    /var/log/nginx/error.log.7.gz
4.0K    /var/log/nginx/access.log.34.gz
4.0K    /var/log/nginx/access.log.13.gz
4.0K    /var/log/nginx/access.log.33.gz
4.0K    /var/log/nginx/access.log.21.gz
4.0K    /var/log/nginx/access.log.1
4.0K    /var/log/nginx/access.log.28.gz
16K    /var/log/nginx/error.log.15.gz
8.0K    /var/log/nginx/access.log.24.gz
4.0K    /var/log/nginx/access.log.37.gz
16K    /var/log/nginx/error.log.44.gz
0    /var/log/nginx/error.log
16K    /var/log/nginx/error.log.26.gz
4.0K    /var/log/nginx/access.log.27.gz
8.0K    /var/log/nginx/error.log.46.gz
4.0K    /var/log/nginx/access.log.23.gz
4.0K    /var/log/nginx/access.log.49.gz
4.0K    /var/log/nginx/access.log.2.gz
56K    /var/log/nginx/error.log.19.gz
4.0K    /var/log/nginx/access.log.11.gz
4.0K    /var/log/nginx/access.log.18.gz
4.0K    /var/log/nginx/access.log.50.gz
4.0K    /var/log/nginx/access.log.6.gz
396K    /var/log/nginx/error.log.1
4.0K    /var/log/nginx/access.log.17.gz
16K    /var/log/nginx/error.log.35.gz
4.0K    /var/log/nginx/access.log.20.gz
8.0K    /var/log/nginx/error.log.52.gz
16K    /var/log/nginx/error.log.40.gz
4.0K    /var/log/nginx/access.log.45.gz
4.0K    /var/log/nginx/access.log.39.gz
4.0K    /var/log/nginx/access.log.48.gz
8.0K    /var/log/nginx/error.log.23.gz
4.0K    /var/log/nginx/access.log.35.gz
8.0K    /var/log/nginx/error.log.48.gz
4.0K    /var/log/nginx/access.log.22.gz
8.0K    /var/log/nginx/error.log.32.gz
4.0K    /var/log/nginx/access.log.26.gz
8.0K    /var/log/nginx/error.log.42.gz
8.0K    /var/log/nginx/error.log.11.gz
12K    /var/log/nginx/error.log.50.gz
16K    /var/log/nginx/error.log.9.gz
4.0K    /var/log/nginx/access.log.53.gz
8.0K    /var/log/nginx/error.log.47.gz
4.0K    /var/log/nginx/access.log.31.gz
4.0K    /var/log/nginx/access.log.16.gz
12K    /var/log/nginx/error.log.43.gz
4.0K    /var/log/nginx/access.log.43.gz
20K    /var/log/nginx/error.log.25.gz
4.0K    /var/log/nginx/access.log.3.gz
4.0K    /var/log/nginx/access.log.46.gz
8.0K    /var/log/nginx/error.log.12.gz
4.0K    /var/log/nginx/access.log.25.gz
8.0K    /var/log/nginx/error.log.36.gz
12K    /var/log/nginx/error.log.31.gz
4.0K    /var/log/nginx/access.log.47.gz
12K    /var/log/nginx/error.log.21.gz
12K    /var/log/nginx/error.log.49.gz
12K    /var/log/nginx/error.log.10.gz
56K    /var/log/nginx/error.log.18.gz
4.0K    /var/log/nginx/access.log.32.gz
4.0K    /var/log/nginx/access.log.40.gz
4.0K    /var/log/nginx/access.log.30.gz
4.0K    /var/log/nginx/access.log.41.gz
12K    /var/log/nginx/error.log.34.gz
4.0K    /var/log/nginx/access.log.29.gz
40K    /var/log/nginx/error.log.14.gz
8.0K    /var/log/nginx/error.log.45.gz
4.0K    /var/log/nginx/access.log.42.gz
36K    /var/log/nginx/error.log.24.gz
4.0K    /var/log/nginx/access.log.9.gz
12K    /var/log/nginx/error.log.30.gz
8.0K    /var/log/nginx/error.log.6.gz
12K    /var/log/nginx/error.log.33.gz
4.0K    /var/log/nginx/access.log.38.gz
8.0K    /var/log/nginx/error.log.3.gz
12K    /var/log/nginx/error.log.17.gz
4.0K    /var/log/nginx/access.log.19.gz
4.0K    /var/log/nginx/access.log.52.gz
12K    /var/log/nginx/error.log.37.gz
4.0K    /var/log/nginx/access.log.4.gz
12K    /var/log/nginx/error.log.39.gz
4.0K    /var/log/nginx/access.log.8.gz
4.0K    /var/log/nginx/access.log.51.gz
8.0K    /var/log/nginx/error.log.2.gz
12K    /var/log/nginx/error.log.16.gz
4.0K    /var/log/nginx/access.log.7.gz
4.0K    /var/log/nginx/access.log.10.gz
8.0K    /var/log/nginx/error.log.53.gz
8.0K    /var/log/nginx/error.log.13.gz
12K    /var/log/nginx/error.log.5.gz
4.0K    /var/log/nginx/access.log.36.gz
4.0K    /var/log/nginx/access.log.14.gz
4.0K    /var/log/nginx/access.log.5.gz
12K    /var/log/nginx/error.log.29.gz
8.0K    /var/log/nginx/error.log.41.gz
4.0K    /var/log/nginx/access.log.12.gz
0    /var/log/nginx/access.log
12K    /var/log/nginx/error.log.4.gz
12K    /var/log/nginx/error.log.22.gz
12K    /var/log/nginx/error.log.38.gz
8.0K    /var/log/nginx/access.log.15.gz
1.4M    /var/log/nginx
4.0K    /var/log/daemon.log.3.gz
4.0K    /var/log/dmesg.3.gz
184K    /var/log/debug
40K    /var/log/kern.log.4.gz
2.4M    /var/log/mail.log.1
0    /var/log/memcached.log
4.0K    /var/log/vsftpd.log.4.gz
du: cannot read directory `/var/log/mysql': Permission denied
4.0K    /var/log/mysql
4.0K    /var/log/mail.warn.1
32K    /var/log/syslog
24K    /var/log/wtmp
260K    /var/log/auth.log.2.gz
276K    /var/log/auth.log.3.gz
4.0K    /var/log/aptitude.2.gz
0    /var/log/btmp
0    /var/log/mysql.err
36K    /var/log/syslog.6.gz
4.0K    /var/log/dmesg.4.gz
4.0K    /var/log/daemon.log.4.gz
16K    /var/log/daemon.log.1
4.0K    /var/log/mysql.log.5.gz
4.0K    /var/log/daemon.log.2.gz
4.0K    /var/log/messages.2.gz
24K    /var/log/syslog.3.gz
720K    /var/log/auth.log
du: cannot read directory `/var/log/apache2': Permission denied
4.0K    /var/log/apache2
36K    /var/log/vsftpd.log.3.gz
24K    /var/log/syslog.2.gz
0    /var/log/mail.warn
72K    /var/log/vsftpd.log.2.gz
4.0K    /var/log/aptitude.3.gz
344K    /var/log/mail.log.4.gz
36K    /var/log/syslog.5.gz
0    /var/log/user.log
588K    /var/log/mail.info.2.gz
4.0K    /var/log/fsck/checkroot
4.0K    /var/log/fsck/checkfs
12K    /var/log/fsck
4.0K    /var/log/mysql.log.6.gz
28K    /var/log/syslog.4.gz
4.0K    /var/log/fontconfig.log
128K    /var/log/mail.info
24K    /var/log/daemon.log
40K    /var/log/messages
0    /var/log/aptitude
du: cannot read directory `/var/log/munin': Permission denied
4.0K    /var/log/munin
4.0K    /var/log/mail.warn.3.gz
56K    /var/log/kern.log.3.gz
4.0K    /var/log/wtmp.1.gz
4.0K    /var/log/messages.4.gz
4.0K    /var/log/vsftpd.log
19M    /var/log

[Albert Sun]

You have 300 GB of disk space? Or is that bandwidth? If you're on the 512 slice then you get 20GB of disk space and 300GB of bandwidth which means you've just run out of disk space and should probably upgrade. Or go through the server and find things you can delete. Any large media files that don't need to be there?

[Daniel Petty]

Yes, yes. Bandwidth. Looked at the wrong thing. It's a 512 slice. Maybe it should just be upgraded, instead of flushing logs?

[Daniel Bachhuber]

Yes, you should upgrade it. Your logs are only ~20 MB, so you probably have too much media

--

http://db.ly/info

[Daniel Petty]

Thanks, all, for the help. This invites a discussion internally about whether we should set up a separate server just for media assets (photos, slideshows, etc.).

[Kevin Koehler]

You should consider using third-party services: Flickr will host all your images for ~ $2 a month. Vimeo, 5 GB of uploads for ~$5 a month. Adds an extra wrinkle to your workflow, though. 

[Daniel Petty]

Vimeo is what The Collegian already uses for video, but hadn't thought about using Flickr for image hosting. We'll take a look at it. Thanks!