Isilon Reimage from USB

[Shawn Winters]

Hi

I have a OneFS image we're trying to revive a IQ1920 we have.   I want to just format it since we cannot login to it.  Can anyone tell me how to take the OneFS_v6.5.4.118_reimg.img and write it to a USB thumb drive?  Does it matter the size?  I tried using DD on a mac, but I think I did it wrong.  I did sudo dd if=OneFS.img of=/dev/disk1 bs=1m

It makes a 500mb partition on the 8GB stick, but when I boot from it on the Isilon it just says missing operating system.  

Thanks

Shawn

[Keith Nargi]

Sean

You should use a smaller USB drive.  Anything larger than 2GB really won't work.  I would recommend using Kingston or sandisk.  512MB,1GB or 2GB is recommended 

—
Sent from Mailbox for iPhone

--
You received this message because you are subscribed to the Google Groups "Isilon Technical User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isilon-user-gr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Winters, Shawn]

OK, i'll give that a shot.

[Erik Weiman]

It's been a while since I've messed with this but you *might* need to do your dd command to rdisk rather than disk. 

Like this:

sudo dd if=OneFS.img of=/dev/rdisk1 bs=1m

--

Erik Weiman 

Sent from my iPhone 4

[Keith Nargi]

I've always done this from an Isilon node where I did dd=/ifs/bla.img of=/dev/da-something bs=4k

—
Sent from Mailbox for iPhone

[Jason Davis]

There is an article in the support community that describes this procedure:

https://community.emc.com/docs/DOC-20612

[Jason Davis]

On second thought, that wasn't all that helpful... the links embedded in this all point back to the old SalesForce based support site. :/

[Winters, Shawn]

Yeah, I found that.  The funny part is this part that I need doesn't work:

See How to burn a USB flash drive with a OneFS reimage build for instructions on how to create such a device.

links to this that shows a permission denied for me at least.  We have a valid support contract on our X200's so I should have access to it.

https://isilon.custhelp.com/app/error//error_id/4

[novemberico]

FWIW your dd syntax is fine (assuming the usb drive shows up as disk1 in sudo diskutil list on your osx box). Here are some ideas if you haven't thought of them already:
 - Did OSX have the usb drive mounted when you ran the dd?
 - Does the md5sum match from where you downloaded the .img from?
 - It's been a while since I've used an isilon, but if you can access the bios on whatever machine you're trying to boot this on you should check to see that the server is configured to boot from a usb stick.

 - You could try zero'ing out the drive before using it, dd if=/dev/zero of=/dev/<whatever diskdev> bs=8M but the dd you ran should've placed the .img at the start of the disk, overwriting gpt or mbr or whatever OneFS uses.

I haven't reimaged an isilon before so let us know if you find something quirky (e.g. Keith's idea to use a usb drive smaller than 2GB)

[Winters, Shawn]

I tried it with a 1GB stick.  I'm pretty sure the image is good.  I can boot my mac from the USB.  The isilon never shows it booting off the usb stick.  Just immediately goes to missing operating system.  Maybe this thing is just shot.  I would expect it to boot from the USB though.

[Cory Snavely]

In our experience, it is nearly impossibly to know which flavor of USB
stick an Isilon node will boot from. Note that I am not saying that
sometimes a given stick will work and sometimes it won't - ones that
work will always work. Apparently, though, trial and error is your only
option to find out. I can confirm we have prepared different brands of
USB sticks in the exact same way, and some worked, while others didn't.
I have always suspected that the node type is a factor, but I don't have
that on any authority.

> <https://isilon.custhelp.com/app/answers/detail/a_id/613> for

> instructions on how to create such a device.
>
> links to this that shows a permission denied for me at least.
> We have a valid support contract on our X200's so I should
> have access to it.
> https://isilon.custhelp.com/app/error//error_id/4
>
> On Tue, Mar 5, 2013 at 4:49 PM, Jason Davis <scr...@gmail.com
> <mailto:scr...@gmail.com>> wrote:
>
> There is an article in the support community that describes
> this procedure:
> https://community.emc.com/docs/DOC-20612
>
>
>
>
> On Tue, Mar 5, 2013 at 3:26 PM, Keith Nargi
> <keith...@gmail.com <mailto:keith...@gmail.com>> wrote:
>
> I've always done this from an Isilon node where I did
> dd=/ifs/bla.img of=/dev/da-something bs=4k
>

> �
> Sent from Mailbox <https://bit.ly/SZvoJe> for iPhone

>
>
> On Tue, Mar 5, 2013 at 4:09 PM, Erik Weiman
> <erik.j...@gmail.com

> <mailto:erik.j...@gmail.com>> wrote:
>
> It's been a while since I've messed with this but
> you *might* need to do your dd command to rdisk
> rather than disk.
>
> Like this:
>
> sudo dd if=OneFS.img of=/dev/rdisk1 bs=1m
>
> --
> Erik Weiman
> Sent from my iPhone 4
>
> On Mar 5, 2013, at 2:48 PM, "Winters, Shawn"

> <swin...@mimeo.com <mailto:swin...@mimeo.com>> wrote:
>
> OK, i'll give that a shot.
>
> On Tue, Mar 5, 2013 at 3:39 PM, Keith Nargi

> <keith...@gmail.com <mailto:keith...@gmail.com>>

> wrote:
>
> Sean
>
> You should use a smaller USB drive. Anything
> larger than 2GB really won't work. I would
> recommend using Kingston or sandisk. 512MB,1GB
> or 2GB is recommended

> �
> Sent from Mailbox <https://bit.ly/SZvoJe> for iPhone

>
>
> On Tue, Mar 5, 2013 at 3:36 PM, Shawn Winters

> <swin...@mimeo.com <mailto:swin...@mimeo.com>>

> wrote:
>
> Hi
>
>
> I have a OneFS image we're trying to revive
> a IQ1920 we have. I want to just format it
> since we cannot login to it. Can anyone
> tell me how to take
> the OneFS_v6.5.4.118_reimg.img and write it
> to a USB thumb drive? Does it matter the
> size? I tried using DD on a mac, but I
> think I did it wrong. I did sudo dd
> if=OneFS.img of=/dev/disk1 bs=1m
>
> It makes a 500mb partition on the 8GB stick,
> but when I boot from it on the Isilon it
> just says missing operating system.
>
> Thanks
> Shawn
>
> --
> You received this message because you are
> subscribed to the Google Groups "Isilon
> Technical User Group" group.
> To unsubscribe from this group and stop
> receiving emails from it, send an email to
> isilon-user-gr...@googlegroups.com

> <mailto:isilon-user-group%2Bunsu...@googlegroups.com>.

> For more options, visit
> https://groups.google.com/groups/opt_out.
>
>
>
> --
> You received this message because you are
> subscribed to the Google Groups "Isilon
> Technical User Group" group.
> To unsubscribe from this group and stop
> receiving emails from it, send an email to
> isilon-user-gr...@googlegroups.com

> <mailto:isilon-user-group%2Bunsu...@googlegroups.com>.

> For more options, visit
> https://groups.google.com/groups/opt_out.
>
>
>
> --
> You received this message because you are subscribed
> to the Google Groups "Isilon Technical User Group"
> group.
> To unsubscribe from this group and stop receiving
> emails from it, send an email to
> isilon-user-gr...@googlegroups.com

> <mailto:isilon-user-gr...@googlegroups.com>.

> For more options, visit
> https://groups.google.com/groups/opt_out.
>
>
>
> --
> You received this message because you are subscribed to
> the Google Groups "Isilon Technical User Group" group.
> To unsubscribe from this group and stop receiving emails
> from it, send an email to
> isilon-user-gr...@googlegroups.com

> <mailto:isilon-user-group%2Bunsu...@googlegroups.com>.

> For more options, visit
> https://groups.google.com/groups/opt_out.
>
>
>
> --
> You received this message because you are subscribed to the
> Google Groups "Isilon Technical User Group" group.
> To unsubscribe from this group and stop receiving emails
> from it, send an email to
> isilon-user-gr...@googlegroups.com

> <mailto:isilon-user-group%2Bunsu...@googlegroups.com>.

> For more options, visit
> https://groups.google.com/groups/opt_out.
>
>
>
> --
> You received this message because you are subscribed to the
> Google Groups "Isilon Technical User Group" group.
> To unsubscribe from this group and stop receiving emails from
> it, send an email to
> isilon-user-gr...@googlegroups.com

> <mailto:isilon-user-group%2Bunsu...@googlegroups.com>.

> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Isilon Technical User Group" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to isilon-user-gr...@googlegroups.com

> <mailto:isilon-user-group%2Bunsu...@googlegroups.com>.

[Erik Weiman]

Try playing around with the first 4 hard drive bays in the node. Sometimes we can get a node to boot up if we pull out some of those drives. Typically start by pulling bay 1 and try booting. The various parts of the OS are mirrored across those drives. Perhaps the node isn't willing to boot from the USB and is getting confused at boot. 

Can you boot the node in single user mode 'boot -s' at the OK prompt if you interrupt the HDD boot?

If you can then I would copy the OneFS install.tar.gz into /var/crash using scp or something and use "isi_reimage -b /var/crash/install.tar.gz "

--

Erik Weiman 

Sent from my iPhone 4

[Keith Nargi]

If you have a 1GB stick already DD'ed.. This is what my next steps would be

1. Plug in the USB memory stick to the top of the two USB port on the
back of the node. If the reader is not attached before boot, the
system will not recognize it and you will have to reboot to start this
process.
2. Power on the node. The system should stop at the following prompt:
3. Warning: reimage will destroy all the data on the node.
4. Are you sure you wish to reimage (yes/no [no])?
5. Enter yes and the process will continue. At this point there will
be no output to the screen for about 10 minutes. When it is done
placing the image on the node it will indicate that it is rebooting
the cluster and subsequently power down the node.

This has always worked for me and I have used Memorex, SanDisk and
Kingston usb devices. Now these are pretty old nodes so the size of
the usb stick might need to be smalller than 1GB.

>> —

>> Sent from Mailbox <https://bit.ly/SZvoJe> for iPhone
>>
>>
>> On Tue, Mar 5, 2013 at 4:09 PM, Erik Weiman
>> <erik.j...@gmail.com
>> <mailto:erik.j...@gmail.com>> wrote:
>>
>> It's been a while since I've messed with this but
>> you *might* need to do your dd command to rdisk
>> rather than disk.
>>
>> Like this:
>>
>> sudo dd if=OneFS.img of=/dev/rdisk1 bs=1m
>>
>> --
>> Erik Weiman
>> Sent from my iPhone 4
>>
>> On Mar 5, 2013, at 2:48 PM, "Winters, Shawn"
>> <swin...@mimeo.com <mailto:swin...@mimeo.com>>
>> wrote:
>>
>> OK, i'll give that a shot.
>>
>> On Tue, Mar 5, 2013 at 3:39 PM, Keith Nargi
>> <keith...@gmail.com <mailto:keith...@gmail.com>>
>> wrote:
>>
>> Sean
>>
>> You should use a smaller USB drive. Anything
>> larger than 2GB really won't work. I would
>> recommend using Kingston or sandisk. 512MB,1GB
>> or 2GB is recommended

>> —

--
Keith

[Winters, Shawn]

Thanks all. Ill give it a shot tomorrow. 

>--
Keith

[Winters, Shawn]

Finally got it to boot off a CF 1GB card and a USB Reader.  I'm stuck here though.  Attached a pic.  It's been over an hour.  Any thoughts?

Passing 'disk0s1a', '2147483647, and 'a' to mountroot

Loading /boot/defaults/loader.conf

Loading isilon_defaults/loader.conf as /boot/loader.conf

[Winters, Shawn]

Never mind, I'm an idiot.  I hooked up the console cable and realized I see a whole different picture.  I think It's working.

[Winters, Shawn]

So it still fails.  I get this error now:

Update/reimage failed. To troubleshoot the failure, review the log files found in /var/log/update_*.txt on failed node(s).

Boot interrupted

Enter full pathname of shell or RETURN for /bin/sh:

How do I view those logs?  I tried vi and look empty.

[Winters, Shawn]

Tried this while booted off the card too.  Guess that doesn't work...

# isi_reformat_node --factory

Reformatting node, restoring factory settings, and destroying

filesystem.

*****

The node will be prepared for shipping and will turn itself off.

Do not power the node back on until it has reached its destination,

or you'll need to run 'isi_reformat_node --factory' again.

*****

Are you sure (yes/no) [no]? yes

This could destroy data

Are you really sure (yes/no) [no]? Yes

Quit without reformat

# isi_reformat_node --factory

Reformatting node, restoring factory settings, and destroying

filesystem.

*****

The node will be prepared for shipping and will turn itself off.

Do not power the node back on until it has reached its destination,

or you'll need to run 'isi_reformat_node --factory' again.

*****

Are you sure (yes/no) [no]? yes

This could destroy data

Are you really sure (yes/no) [no]? yes

No handlers could be found for logger "imdd.devices"

Creating lkg from current root

Traceback (most recent call last):

  File "/usr/bin/isi_lkg_create", line 45, in <module>

    lkg_create(bootdisk.PROVIDER_ID_ROOT, bootdisk.PROVIDER_ID_ROOT_LKG)

  File "/usr/bin/isi_lkg_create", line 32, in lkg_create

    lkg = bootdisk.get_provider(lkg_type)

  File "/usr/local/lib/python2.6/site-packages/isi/sys/bootdisk.py", line 258, in get_provider

    dev = get_provider_dev(provider_id)

  File "/usr/local/lib/python2.6/site-packages/isi/sys/bootdisk.py", line 254, in get_provider_dev

    (provider_id, dev))

EnvironmentError: provider root-lkg device /dev/imdd3a does not exist

Failed to create lkg. Quit without reformat

#

[Donald Coleman]

Shawn,

   Is that a 1920i or x?  I ask because if it is an i-series you may want to use OneFS 5.5.8.9.  Also try the standard .img to dd and boot off the USB drive rather than the reimg.img.  

Sent from my iPhone

[Winters, Shawn]

Yeah, it's 1920i.  I'll give that a shot.  Thanks.

[Winters, Shawn]

so I tried 5.5.8.9 the image, not reimage and I get the same error:

Image version: 5.5.8.9 B_5_5_8_9(RELEASE) (0x505080000000900)

Warning: reimage will destroy all the data on the node.

Are you sure you wish to reimage (yes/no [no])?yes

Please wait, reimaging...

Initiating IMDD...

Update/reimage failed. To troubleshoot the failure, review the log files found in /var/log/update_*.txt on failed node(s).

Killed

Unexpected return code from isi_imdd_start

Killed

MEnter full pathname of shell or RETURN for /bin/sh:

Inside the /var/log/update text files is this:

# cat  update_engine_2013-03-07_14:53:57.txt

2013-03-07 14:54:02,980 DEBUG Executing: /usr/bin/tar -C /var --format=ustar --one-file-system -cf - . | /usr/bin/tar -C /usr/update/var -xpUf -

2013-03-07 14:54:26,412 INFO Update engine starts.

2013-03-07 14:54:32,267 INFO Loading from path: /

2013-03-07 14:54:32,268 INFO Loading image finishes: /

2013-03-07 14:54:32,372 DEBUG Getting versions of node and image begins.

2013-03-07 14:54:32,376 INFO Node version: None

2013-03-07 14:54:32,377 INFO Image version: 5.5.8.9 B_5_5_8_9(RELEASE) (0x505080000000900)

2013-03-07 14:54:32,378 DEBUG Getting versions of node and image finishes.

2013-03-07 14:55:43,837 DEBUG Initializing devices starts.

2013-03-07 14:55:43,839 CRITICAL Failed to mount: /usr/update/root

2013-03-07 14:55:49,551 WARNING Abort from previous error. Update failed.

2013-03-07 14:55:49,552 INFO Update engine finishes.

2013-03-07 14:55:49,553 DEBUG Executing: /usr/bin/pkill -KILL -g 63

# cat update_handler_2013-03-07_14:53:57.txt

2013-03-07 14:54:31,415 DEBUG Finished: INITIATE

2013-03-07 14:54:31,420 INFO Loading image...

2013-03-07 14:54:32,265 DEBUG Starting: LOAD_IMAGE

2013-03-07 14:54:37,295 DEBUG Finished: LOAD_IMAGE

2013-03-07 14:54:37,300 INFO Node version : None

Image version: 5.5.8.9 B_5_5_8_9(RELEASE) (0x505080000000900)

2013-03-07 14:55:42,846 INFO Please wait, reimaging...

2013-03-07 14:55:43,356 INFO Initiating IMDD...

2013-03-07 14:55:43,835 DEBUG Starting: INITIATE_LKG

2013-03-07 14:55:48,865 ERROR Finished [FAILED]: INITIATE_LKG

2013-03-07 14:55:48,866 INFO Update/reimage failed. To troubleshoot the failure, review the log files found in /var/log/update_*.txt on failed node(s).

# Image version: 5.5.8.9 B_5_5_8_9(RELEASE) (0x505080000000900)

[Erik Weiman]

I would try using one of these options if you can get the node booted in single user mode (not off of USB stick)

v6554-1# isi_reformat_node --nolkg -f --factory

If that one fails then try this:

v6554-1# isi_reimage --skip-all-checks -fb /var/crash/OneFS_Install.tar.gz

[Brendan Scherer]

While Isilon won't officially support using thumb drives that aren't their "certified" ones, I have done it with an 8gb pen drive that was nowhere near being close to Isilon tested. The makeusb.exe is garbage. If you are using windows, give Win32 Disk Imager a try. I have seen it work with all sorts of strange usb drives.