A200 cluster rebooting

[John Murray]

Hey All,

We have an A200 cluster that we moved to another location a while back.  We are trying to get it up and running at the moment, but it looks like it is doing a rolling reboot constantly. 

Our CLI knowledge isn't huge and we are looking to get some assistance in that regard.  Getting blinking amber lights at the back of all 4 nodes.  Reading that may be power supply related. 

Any help would be appreciated.

Thanks

[Maciej Fabisiak]

Hi,

You need to connect RS cable to this node and see the boot sequence.

Then we can talk about this.

Maciek

--
You received this message because you are subscribed to the Google Groups "Isilon Technical User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isilon-user-gr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isilon-user-group/CAONW7BYyWJGx_PcFAO3GJi45h8FWaPMd6qnF%2BsgktW-_koh5wA%40mail.gmail.com.

[John Murray]

Hi Maciej, 

Thanks for reaching out.  I have this.  I can get to a # prompt. not sure where to go from there.

John 

To view this discussion on the web visit https://groups.google.com/d/msgid/isilon-user-group/CAMauAjbqEEx_XVS1tXuwDj%2BWEfP4tqnP54K%3DGpmbiiL6sFYdAA%40mail.gmail.com.

[Maciej Fabisiak]

I need entire boot sequence from the begining to the #

So connect rs to this node and power it on.

To view this discussion on the web visit https://groups.google.com/d/msgid/isilon-user-group/CAONW7Bb04eWPEY1zwLRdoR%3DE8Ju%2BEk-LQyCyTkkWN9Y9tK3afA%40mail.gmail.com.

[Anurag Chandra]

The power on sequence will be helpful

What you see as # is the node in single user mode .

/var/log/messages will give you some info as well. 

Thanks

Anurag 

To view this discussion on the web visit https://groups.google.com/d/msgid/isilon-user-group/CAMauAjYnhgho9KAyMnHYjFJOf_uM%2BZg%2BW8L2JQ-HhsrbfKh6ew%40mail.gmail.com.

[John Murray]

Hey,

attached as txt. Thanks

To view this discussion on the web visit https://groups.google.com/d/msgid/isilon-user-group/CANSkps7%2BstQrw_AmJSgNZWGbfqi%2BPQrPjqCb4Kn1ETcBFs4LJg%40mail.gmail.com.

[John Beranek - PA]

That does not look like a very happy node. The system disk is coming up with file system errors, and then the node finally fails with:

mount_efs: Reporting missing logical drive 15 with guid 5bd99dc000018e2d 19749e144a0cccd5 as DOWN
mount_efs: driveConfIdentifySSD: Error mapping lnum 15 to bay
mount_efs: driveConfBayDetermineWCE: Error mapping lnum 15 to bay
The journal for this node is not valid. Please contact Isilon support for resolution of this problem.
panic @ time 1665580614.610, thread 0xfffff8017ef4f780: Assertion Failure
cpuid = 0
Panic occurred in module kernel loaded at 0xffffffff80200000:

John

[John Murray]

Thanks John, 

Do I have any options at this stage.  There is no data on it, so a isi_format_nodes is fine if it brings us back to a day 1 situation, if that works. 

John 

To view this discussion on the web visit https://groups.google.com/d/msgid/isilon-user-group/38f2419c-f382-43e0-81ee-ba8ded715555n%40googlegroups.com.

[Saker Klippsten]

What steps were taken to move this node?

 Was it properly shutdown? 

Did you remove drives from the chassis ?

If yes , are you sure you put the drives back into chassis proper order?

Are you sure you put the nodes back into the chassis in the correct ordered pairs? / order?

Checking Battery Health... Batteries good Checking Isilon Journal integrity... DRAM and disk backup are invalid for local journal. Zero out journal superblock. The Peer journal state is internally consistent, but the Peer journal state does not match the node and cluster state. Journal: cluster guid f8f21e2f483c4689d95b1723b32a77a14e89, device id 2. Disk: cluster guid

mount_efs: Reporting missing logical drive 15 with guid 5bd99dc000018e2d 19749e144a0cccd5 as DOWN

mount_efs: driveConfIdentifySSD: Error mapping lnum 15 to bay

mount_efs: driveConfBayDetermineWCE: Error mapping lnum 15 to bay

The journal for this node is not valid. Please contact Isilon support for resolution of this problem.

panic @ time 1665580614.610, thread 0xfffff8017ef4f780: Assertion Failure

cpuid = 0

Panic occurred in module kernel loaded at 0xffffffff80200000:

-S

On Oct 14, 2022, at 6:57 AM, John Beranek - PA <john.b...@pamediagroup.com> wrote:

That does not look like a very happy node. The system disk is coming up with file system errors, and then the node finally fails with:

To view this discussion on the web visit https://groups.google.com/d/msgid/isilon-user-group/38f2419c-f382-43e0-81ee-ba8ded715555n%40googlegroups.com.

[Paul Carrington]

Also how was the node shutdown? I've seen something similar when a colleague simply used a shutdown -h now instead of the correct isi config and shutdown from within. I know you can use shutdown if you force a flush cache command beforehand. Only other time ive seen this was generated nodes where the node had a double battery failure.

To view this discussion on the web visit https://groups.google.com/d/msgid/isilon-user-group/C8D02429-4786-4333-A589-AEAB34EB0DE9%40gmail.com.

[John Murray]

Shutdown was done from the web console.  All drives and nodes were pulled, but marked (numbered) and they all look to be back in the correct locations.  

To view this discussion on the web visit https://groups.google.com/d/msgid/isilon-user-group/CALmzHC-8EKW7opaZ8Hmnk2WDdza-ByYCKV%3D965Z6S4_U%3Dx05DA%40mail.gmail.com.