Euler Filesystem Outage

[Colin Vanden Heuvel]

Hello Euler Users,

We are experiencing an issue with a disk controller in Euler's storage backend that affects about 25% of Euler's total storage. While we can recover the affected data, it is possible that we will need to go to a read-only state until the hardware is replaced. We are already in contact with the vendor and are requesting expedited service to fix the problem as soon as we can.

I will be taking the cluster offline for a brief period today while I recover the affected data, then I will try to bring it back online. If it is not possible to safely continue full operations, we will stay in read-only mode. I will try to follow up later with more details about the timeline as I have them.

Regards,

Colin Vanden Heuvel

[Colin Vanden Heuvel]

The problem has been temporarily mitigated and Euler has returned to service for the time being. We will try to keep it online as long as possible, but if our mitigations don't prevent the data consistency errors we saw last night from resurfacing, we will force the cluster into read-only mode tomorrow in order to prevent any data corruption over the weekend.

Regards,

Colin

---

From: 'Colin Vanden Heuvel' via euler-users <euler...@g-groups.wisc.edu>
Sent: Thursday, March 13, 2025 11:22
To: 'Colin Vanden Heuvel' via euler-users <euler...@g-groups.wisc.edu>
Subject: [euler-users] Euler Filesystem Outage

 

--
You received this message because you are subscribed to the Google Groups "euler-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to euler-users...@g-groups.wisc.edu.
To view this discussion visit https://groups.google.com/a/g-groups.wisc.edu/d/msgid/euler-users/SN7PR06MB7327FFD0F563EAFDA875DEE2F6D32%40SN7PR06MB7327.namprd06.prod.outlook.com.