Low database disk space warning message from Go Server

[Shivani Shinde]

Hi all,

I recieved this email :

The email has been sent out automatically by the Go server at (IP) to Go administrators.

This server has less than 1024Mb of disk space available at /var/lib/go-server/db to store data. When the available space goes below 100Mb, Go will stop scheduling. Please ensure enough space is available.

I have got this email for the first time related to database disk space. I looked up any issues or help in GoCD Docs, but couldn't find any.

If this info helps:

go@gocd:/var/lib/go-server$ for each in $(ls) ; do du -hs "$each" ; done

14G artifacts

100M cruise.war

8.0K data

13M db

156K felix-cache

1.4G pipelines

16M plugins

32M plugins_work

16K run

173M work

go@gocd:/var/lib/go-server/db$ df -h

Filesystem                   Size  Used Avail Use% Mounted on

udev                         2.0G     0  2.0G   0% /dev

tmpfs                        396M   16M  380M   4% /run

/dev/sda1                     19G   18G  175M 100% /

tmpfs                        2.0G     0  2.0G   0% /dev/shm

tmpfs                        5.0M     0  5.0M   0% /run/lock

tmpfs                        2.0G     0  2.0G   0% /sys/fs/cgroup

/dev/mapper/alef--vg-datalv  788G   27G  722G   4% /data

tmpfs                        396M     0  396M   0% /run/user/999

I store artifacts on /data which is of 20GB.

Has anyone faced this error before? If yes, how did you solve it?

Any help will be appreciated!

Thank you.

[Shivani Shinde]

I referred this thread, but looking at screenshots and directories, it seems the GoCD version is very old. 

I am using GoCD version 19.10.

This is the directory structure of /var/lib/go-server/db directory:

go@gocd:/var/lib/go-server/db$ tree -d

.

├── command_repository

│   └── default

│       ├── cloud

│       ├── deploy

│       ├── java

│       ├── objective-c

│       ├── other

│       ├── package

│       │   ├── nuget

│       │   └── rpm

│       ├── ruby

│       ├── test

│       ├── unix

│       ├── virtual

│       └── windows

├── config.git

├── h2db

└── h2deltas

As suggested in above thread, is it safe to delete contents of this folder?

[kritik...@thoughtworks.com]

This maybe helpful to you: https://docs.gocd.org/19.10.0/faq/admin_out_of_disk_space.html

[Shivani Shinde]

Hi kritika,

Yes, I went through this link. But it states everything about 'artifacts low disk space'. I do not face the Artifact Disk Space issue, I faced Database disk space low issue.

Attaching screenshot for reference:

I have a hard disk of 20GB attached at /data location for artifacts and have also followed Auto Delete Artifacts configuration already:

Is there something more that I could have missed?

Thank you.

[Ketan Padegaonkar]

Artifacts and database are different directories. It is not clear what partition holds your DB directory.

- Ketan

CONFIDENTIALITY. This email and any attachments are confidential to Alef Edge Inc., and may also be privileged, except where the email states it can be disclosed. If this email is received in error, please do not disclose the contents to anyone, notify the sender by return email, and delete this email (and any attachments) from your system.

--
You received this message because you are subscribed to the Google Groups "go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/4a17d2fc-ebce-49b4-aacd-62f0e865243d%40googlegroups.com.

[Jason Smyth]

Hi Shivani,

From the information provided in your initial post, it looks like most of the space on the root volume is consumed by /var/lib/go-server/artifacts (14GB).

My best guess is that this server was originally configured to use this directory for artifacts and then later changed to use /data/artifacts as shown in the screen shot. Assuming /var/lib/go-server/artifacts contains artifacts from old Pipeline runs, it should be safe to delete everything there.

I recommend reviewing the contents of /var/lib/go-server/artifacts and comparing them to what is currently in /data/artifacts. This should allow you to identify anything that does not exist in both locations and then make an educated decision about what is and is not safe to delete from /var/lib/go-server/artifacts.

Hope this helps,

Jason

On Friday, 17 April 2020 05:28:29 UTC-4, Ketan Padegaonkar wrote:

Artifacts and database are different directories. It is not clear what partition holds your DB directory.

- Ketan

On Fri, Apr 17, 2020 at 12:51 PM Shivani Shinde <shivan...@alefedge.com> wrote:

Hi kritika,

Yes, I went through this link. But it states everything about 'artifacts low disk space'. I do not face the Artifact Disk Space issue, I faced Database disk space low issue.

Attaching screenshot for reference:

I have a hard disk of 20GB attached at /data location for artifacts and have also followed Auto Delete Artifacts configuration already:

Is there something more that I could have missed?

Thank you.

CONFIDENTIALITY. This email and any attachments are confidential to Alef Edge Inc., and may also be privileged, except where the email states it can be disclosed. If this email is received in error, please do not disclose the contents to anyone, notify the sender by return email, and delete this email (and any attachments) from your system.

--
You received this message because you are subscribed to the Google Groups "go-cd" group.

To unsubscribe from this group and stop receiving emails from it, send an email to go...@googlegroups.com.

[Shivani Shinde]

Hi Jason,

You guessed it accurately! 

I went through my directories of artifacts, compared them and deleted the ones which weren't of any use. 

This solved the problem and now I could run pipelines again. My doubt arised when I received an email saying "Go-Server's database disk space was low" with path to go-agent/db was mentioned. 

Whereas I had already received another email stating "Disk space running out of space". Hence couldn't get the difference.

Anyway, thank you for your suggestion. :-)

Thanks!

To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/5c6dc9f0-ce4d-4091-b674-3758dbcd3778%40googlegroups.com.