Downtime: serious double disk crash

[Jonas Bardino]

Hi list

Shortly before midnight two disks in the server dk.migrid.org decided to jump ship. We are currently investigating if we can get them back online or if we need to physically replace drives and possibly restore data from backup. Data on the external storage resources like apu0X should be untouched by the problems, but they will of course not be available through MiG as long as the server is down for maintenance.
Due to the holiday tomorrow any physical interaction will have to wait until Tuesday, so hopefully we can get it back to a running state without. More information will follow.

Have a nice weekend!

Cheers, Jonas

[Jonas Bardino]

We managed to get the disks back online for now but may have to replace at least one of them soon as they show some potential signs of pending hardware failure. Anyway, it is back online and resources start reappearing now. Further downtime for disk maintenance will be announced first.

Cheers, Jonas

[Jonas Bardino]

Unfortunately last nights backup triggered the problem again and this time one of the disks refused to come back up at all. We will replace it as soon as possible and are leaving the the server limping along with only the three other disks active now. It is still fully functional only somewhat slower for disk operations.

Cheers, Jonas

[Jonas Bardino]

We will replace the failed disk this afternoon so expect some downtime. More information when it is completed.

Cheers, Jonas

[Jonas Bardino]

On Sunday, May 19, 2013 8:11:45 AM UTC+2, Jonas Bardino wrote:

It is back online and resources are slowly returning.

Cheers, Jonas